When
Where
Student: Jake Callahan, Program in Applied Mathematics
Title: Efficient, Robust, and Risk-aware Bayesian Optimal Experimental Design
Advisor: Dr. Jason Pacheco, Department of Computer Science
Location: PAS 588
Abstract: Bayesian optimal experimental design (BOED) provides a principled framework for choosing experiments to learn unknown parameters, but expensive computation, imperfect models, and the risk of uninformative or misleading experiments remain challenges that hinder its practical use. This dissertation develops computational methods, robust information criteria, and risk-sensitive design policies that make BOED more efficient and reliable in scientific and engineering applications.
To reduce the computational cost of BOED, we introduce reverse-annealed sequential Monte Carlo (RA-SMC), an efficient estimator of expected information gain (EIG). In BOED, joint simulation of parameters and hypothetical observations provides an exact posterior sample for each dataset, which we use to initialize a single-particle trajectory from the posterior to the prior. Thermodynamic integration along this trajectory estimates the log marginal likelihood required to evaluate EIG. We demonstrate that variation in information gain across datasets can dominate estimation error within each dataset, justifying the use of single-particle trajectories instead of costly particle populations, and experiments with multimodal posteriors demonstrate accuracy comparable to forward sequential Monte Carlo with orders of magnitude fewer likelihood evaluations.
Even with efficient estimation, computational constraints can require surrogate models, reduced-order models, or approximate inference. These shortcuts can distort both posterior beliefs and the designs favored by EIG. To prevent this distortion from occurring, we develop an axiomatic framework for experimental utility under misspecification in the prior, likelihood, or inference procedure. We characterize a misspecification gap that quantifies how these errors alter the assessed value of an experiment and demonstrate that classical Blackwell monotonicity can fail in the misspecified setting. Our framework replaces this requirement with bounded contraction under sufficiency, or Blackwell contraction: under specified conditions, coarsening observations reduces both true information and the misspecification penalty. We prove that expected generalized information gain (EGIG) satisfies the new axioms and construct a broader family of robust f-information measures that also satisfy these axioms. These results provide a theoretical basis for choosing experiments that balance learning against the risk of misleading inference.
Accounting for misspecification in expected utility does not by itself control the risk of an individual experiment, or set of experiments, yielding low information when actually performed. A policy with high expected utility can still produce misleading inference, incur real-world costs, or risk destroying a specimen before useful data are collected, leading to an experiment that, while informative on average, is not informative in execution. To account for this, we introduce DistBOED, a distributional actor-critic method that learns the distribution of information gained over sequential campaigns and optimizes functions of that distribution to express the experimenter's risk preferences. We derive policy updates for conditional value at risk (CVaR), mean–variance, and entropic risk objectives, and establish a local gradient identity for updates that combine learned predictions with observed returns. Our experiments show that accounting for risk preferences can reduce campaign variability and the frequency of low-information or misleading outcomes while preserving much of the average information gain. For example, in a setting where more precise measurements risk destroying the specimen, DistBOED optimizing CVaR produces a policy that reduces specimen destruction from 52.2% of the time to 11.5% of the time while retaining approximately 92% of the average information gain achieved by mean optimization.