On the Tightness of the Laplace Approximation for Statistical Inference

Abstract

Laplace’s method is used to approximate intractable integrals in a wide range of statistical problems, including Bayesian inference and frequentist marginal likelihood models. It is classically known that the relative error rate of the approximation is not worse than Op(1/n) under standard regularity conditions, where n is the sample size. It is unknown whether the error rate can be better than Op(1/n) in common applications. We provide the first statistical lower bounds showing that the 1/n rate is tight. We prove stochastic lower bounds for two simple models: Bayesian inference on fair coin flips, and frequentist marginal likelihood estimation for an over-dispersed Poisson model. We conclude that any set of assumptions under which a faster rate can be derived must be so restrictive as to exclude these simple models, and hence the 1/n rate is, for practical purposes, the best that can be obtained.

Publication
arXiv Preprint