Sunday, January 14, 2007

Object Oriented Bayesian Inference

A couple of days ago, while working on a chapter of my book, I came across a manuscript from Kerman and Gelman, from Columbia University talking about a fully Bayesian computing environment. They argue that the basic building block for such a system, would be a "random variable" data type.

I liked their paper. Especially because I was already thinking along those lines for a while. So I decided to give it a shot. Well it was easy, since I already had a nice library for calculating likelihood functions, I didn't have to implement that. I came down to about a hundred lines of Python code (not counting the likelihood library)

I was pretty impressed with it, I had a object that I could specify from a prior distribution, that I could throw data at it and it would return a posterior distribution, a likelihood function, samples from the posterior, etc.

Pretty nice! It also allows the recursive inference typical of Bayesian statistics by which, if available, the posterior becomes the prior when new data is available.

It can still be extended by inhering from basic types such a numpy array. This would allow for this object to be used as a full-fledged type and participate in all sorts of mathematical operations...

Since the code is a bit long and blogger sucks at posting code, I am simply adding a UML representation of the class. If you are interested in discussing about this, getting access to the code or make suggestions, please email me or just leave a comment.

4 comments:

Jort said...

I am very interested in viewing the code and perhaps using the class since i want to do some simple Bayesian calculations from inside python.

Thanks in advance,

Jort

usagi said...

Hi Jort, thanks for the interest. If you email-me personally at fccoelho at gmail dot com, I'll send you the code. If you have any serious application for it I'd like to hear about it.

cheers,

Flavio

jsalvati said...

Wow, that's pretty cool. Did you tell Andrew Gelman about it? I am sure he would blog about it. I am going to e-mail you to get the source for this.

usagi said...

Hi John,

No, I have not contacted Gelman. It would be nice to hear his opinion on the concept.

As for the code, check this other post of mine, where I have a link to where the code is stored:

http://pyinsci.blogspot.com/2008/05/object-oriented-bayesian-inference-ii.html

thanks for the interest.

ccp

Amazon