How fast is JAGS?

The OpenBUGS development team have created some validation code called WinComp that allows the results of OpenBUGS to be compared with WinBUGS 1.4 for all the examples in the OpenBUGS manual.  I have wrapped up this code into an R package called BUGSExamples and added the facility to run some of these models under JAGS.  The package gives a common interface to all three BUGS engines.

So now we can do a direct comparison of speed between JAGS and OpenBUGS.  The figure below shows a comparison of run times (in seconds) on my Linux desktop, using the native Linux version of OpenBUGS and JAGS (The BUGSExamples package also allows you to run WinBUGS and OpenBUGS under WINE).  Since run times differ between examples by over two orders of magnitude, the axes are on a log scale.

timings1.png

The most surprising finding is how variable the results are.  Timings can differ by a factor of up to 5 either way, as shown by the blue dotteed lines.  There are 2 outliers for which JAGS can be 5 times slower, but there are also a number of examples for which JAGS appears to be substantially faster.

Before going any further we have to correct the comparison.  It is easy to write a fast but poorly-mixing sampler. What matters is the time it takes to generate the same quantity of information about the posterior distribution.  This can can be adjusted for using the effectiveSize() function from the coda package. For each monitored node,  effectiveSize()  returns the “effective sample size” – an estimate of the number of independent samples from the posterior that would hold the same information.  Assuming all monitored nodes to be of equal importance, I divided the raw run times by the mean effective sample size across all monitored nodes, then multiplied by 1000 to get the time per 1000 effective samples.  These times are shown below.

timings2.png

The two outliers are still there, and this time I have labelled them (Dogs and Leukfr).  Clearly there is a lot of room for improvement!  Profiling of these examples will hopefully give an indication of what is going wrong in JAGS.

Incidentally, these figures are for JAGS with the glm module loaded. The glm module is not loaded by default.  If you are not using it for generalized linear mixed models then you might be missing out. Here is a comparison of JAGS with and without the glm module.

 timings3.png

Many of the examples are not affected because they do not contain GLMMs. Efficiency is substantially increased for the Epil, Oxford, and Seeds examples.  The Rats example actually runs slower due to the overhead of setting up the glm samplers.  If you have used the glm module for a very large GLMM then you have probably noticed that it can take a substantial amount of time to set up the samplers.  This is also something I need to work on.

 If you want to try out the BUGSExamples package then you can install it directly from R Forge with install.packages("BUGSExamples", repos="http://R-Forge.R-project.org")

About these ads

8 thoughts on “How fast is JAGS?

  1. Does anyone of you find the way to parallelize JAGS?? The most expensive phase in this software in the MCMC Update(adapt as well). Because i’m using JAGS with a big database, this phase in particularly slow. I’d lito to parallelize this phase over more than one core?
    Do you know if exist a module to parallelize this phase?
    Thank you
    Marco

      • Does the jags.parallel function just parallelize the MCMC computation over different markov chains? Using this funtion i only compute different chains in different cores. However there is a way to split the MCMC computation for a single chain in more than one cores? I’m using a large dataset(it is a problem similar to the matrix factorization where i sample from a multivariate with variance matrix that is (1000*1000)).
        Thanks
        Marco

Comments are closed.