The Living Thing / Notebooks : DJ where’s the Bayes line?

Bayesian statistics is controversial amongst frequentists, frequently for terminological reasons.

I’m going to ignore that, because some times it is practical. Even for frequentists it is sometimes refreshing to move your effort from deriving frequentist estimators for intractable models, to just using the damn Bayesian ones.

Anyway, you can avoid learning a lot of tedious frequentist machinery by starting with a belief that your model isn’t too pathological and proceeding accordingly. (You might, of course, be wrong) If you are feeling fancy you might then justify your method on frequentist grounds, but then you are wiping out one interesting source of after-dinner argument.

Doing it

Stan

Stan is the inference toolbox for, especially, hierarchical models.

PyMC3

Pymc3 is pure python, which means you don’t need C++ to fix things like you do in stan. It’s presumably generally slower than stan if you actually do real MC simulations, but I haven’t checked.

Edward

Hot new option from Blei’s lab, leverages trendy deep learning machinery.

Church

Level up you esoterism with Church, a general-purpose Turing-complete Monte Carlo lisp-derivative, which is slow as a thousand miles of baby-arse but does some reputedly cute tricks with modeling human problem-solving, according to creators Noah Goodman and Joshua Tenenbaum.

Other formalisms

PAC-Bayesian

What is this? Something by MacKay, Langford, Shawe-Taylor and Seeger, connection to the frequentist PAC-learning paradigm.