Thursday, April 15, 2010

Liveplots package

I have recently released a package I had been using privately for a while. It's called Liveplots and is a package which provides the tools for monitoring computationally intensive, long running numerical simulations. I am aware that similar tools for Java, like Livegraph. But I needed something that was light, simple, and in Python.

Liveplots provide a daemonized plot server (i.e. running as a separate process) which accepts plot commands via xmlrpc, and a file system monitor, which can be used to produce visualizations when data files are created or modified.

Liveplots is fast enough for most applications. The simple example script available with the source code is able to generate about 15 plots per second on my laptop, but your mileage may vary, depending on how fast you computer is. For the file monitor, it doesn't make sense talking about speed since it is event driven but the same upper limmit should apply. An example is provided for it as well. The file monitor works only in Linux since it depends on pyinotify, but the plot server should work everywhere Python and Gnuplot run.

A very simple API is offered by the plot server, basically consisting of three types of plots: histogram, scatter, and line plots. All of these commands are able to plot multiple data-sets on the same set of axes or on a multi-plot page layout. I think this should suffice for now, since the purpose of this package is to provide a quick visual inspection of long running computations, not to produce publication quality plots. I any case, If you would like to see its API extended, please send me patches. New plot commands should be really easy to implement if you are familiar with Gnuplot.

The visualization part is done by Gnuplot, which I chose because it is fast and ubiquitous. It requires Gnuplot.py, which unfortunately is not yet available through easy_install, thus requiring an extra installation step. Liveplots itself is easy_installable.

Reblog this post [with Zemanta]

4 comments:

rowanh said...

This is an excellent project -- I think that I will employ it in some of my projects. Out of interest why did you use gnuplot rather than matplotlib?

Cheers

Flavio Coelho said...

@rowanh: I am a big fan and user of matplotlib. However, it is not suited for this purpose, I needed something very nimble. Matplotlib excel at producing high quality plots and, due to its size, it would be overkill for liveplots.

Mike Driscoll said...

This sounds pretty neat! If you want to add Windows file monitoring support, check out Tim Golden's cookbook recipe here:

http://timgolden.me.uk/python/win32_how_do_i/watch_directory_for_changes.html

- Mike

Flavio Coelho said...

@Mike: Thanks! I looked at the recipe and it seem pretty simple. It's not my top priority ATM, but I'll gladly accept patches that add windows support.

ccp

Amazon