A Matter of Structure: Robust Empirical Inference from Theory to Application

Abstract

Machine learning's success in idealized settings often fails to translate to real-world science, where complex data from astrophysics to genomics violates standard assumptions. To bridge this gap, this dissertation develops targeted contributions across three interconnected dimensions: theoretical foundations, methodological innovations, and applied challenges.

At the theoretical level, this work introduces the Interventional Kullback-Leibler (IKL) divergence, a novel framework for quantifying structural and distributional differences between causal models across multiple interventional environments. Unlike traditional metrics that focus on observational distributions, IKL captures both structural misalignments between causal graphs and parametric differences in observational distributions, guaranteeing similar outcomes under intervention across most environments. This enables principled causal model comparison and iterative discovery in dynamic systems where interventions induce distribution shifts.

On the methodological front, Flow Matching Posterior Estimation (FMPE) addresses scalability limitations in simulation-based inference by leveraging continuous normalizing flows. Rather than estimating conditional probability densities directly, FMPE parameterizes posteriors via vector fields. Since these are easier to estimate, computational resources can be focused on interpreting high-dimensional observations. This architectural insight yields substantial training efficiency improvements (30% reduction) while achieving performance comparable to specialized methods incorporating physical symmetries. Its effectiveness is demonstrated on challenging gravitational wave inference tasks and standard benchmarks.

In applied domains, this dissertation develops a probabilistic framework for adapting gravitational wave parameter estimation to time-varying detector noise. By modeling Power Spectral Densities (PSDs) through an interpretable latent space that separates broadband components from spectral lines, the approach maintains distributional support over anticipated variations rather than attempting to predict future conditions precisely. This enables real-time inference across extended observing runs without retraining, sustaining accuracy comparable to models with full future knowledge using only historical data and minimal target information.

Beyond individual contributions, this research reveals fundamental principles for robust empirical inference. Notably, structural understanding—whether causal graphs, architectural considerations, or physical mechanisms—consistently emerged as key to success across all domains. This understanding enables designing broad synthetic training distributions that anticipate future variations rather than attempting precise prediction of specific scenarios, often yielding superior robustness in dynamic environments.

These findings suggest that the future of scientific machine learning lies not in developing increasingly sophisticated individual techniques, but in understanding fundamental principles that make empirical inference robust and reliable in complex, dynamic settings. This dissertation thus contributes both targeted solutions to scientific inference challenges and broader insights for approaching machine learning in scientific discovery.