Distributed Affine-Invariant MCMC Sampler

Abstract

Markov Chain Monte Carlo methods provide a tool for tackling high dimensional problems. With many-core systems readily available today, it is no surprise that leveraging parallelism in these samplers has been a subject of recent research. The focus has been on solutions for shared-memory architectures, however these perform poorly in a distributedmemory environment. This paper introduces a fully decentralized version of an affine invariant sampler. By observing that a pseudorandom number generator makes stochastic algorithms deterministic, communication is both minimized and hidden by computation. Two cases at opposite ends of the communication to-computation ratio spectrum are used during evaluation against the currently available master-slave solution, where a more than tenfold reduction in execution time is measured.