Publications
Papers
2026
-
Multiple-Prediction-Powered Inference
ICLRAbstract
Statistical estimation often involves tradeoffs between expensive, high-quality measurements and a variety of lower-quality proxies. We introduce Multiple-Prediction-Powered Inference (MultiPPI): a general framework for constructing statistically efficient estimates by optimally allocating resources across these diverse data sources. This work provides theoretical guarantees about the minimax optimality, finite-sample performance, and asymptotic normality of the MultiPPI estimator. Through experiments across three diverse large language model (LLM) evaluation scenarios, we show that MultiPPI consistently achieves lower estimation error than existing baselines. This advantage stems from its budget-adaptive allocation strategy, which strategically combines subsets of models by learning their complex cost and correlation structures.
2025
-
An Annotated Dataset of Errors in Premodern Greek and Baselines for Detecting Them
NAACL findings -
Discovering the rheology of Antarctic Ice Shelves via physics-informed deep learning
Science
2024
-
Towards Probabilistically-Sound Beam Search with Masked Language Models
NeurIPS Workshop on Statistical Foundations of LLMs and Foundation ModelsAbstract
Beam search with masked language models (MLMs) is challenging in part because joint probability distributions over sequences are not readily available, unlike for autoregressive models. However, estimating such distributions has important domain-specific applications such as ancient text restoration and protein engineering. Here we present probabilistically-sound methods for beam search with MLMs. First, we clarify the conditions under which it is theoretically sound to perform text infilling with MLMs using standard beam search. When these conditions fail, we provide a probabilistically-sound inference time modification with no additional computational complexity and demonstrate that it is superior to the aforementioned beam search in the expected conditions. We then present empirical results comparing several infilling approaches with MLMs across several domains. Notably, our method probes the inductive biases of MLMs and explores the surprising contextual sensitivity of mask tokens for text infilling.
-
Euler Operators for Mis-specified Physics-Informed Neural Networks
ICML Workshop on AI for ScienceAbstract
Integration between equations and data is ubiquitously useful in the physical sciences: for example, data assimilation plays an important role in generating seamless data for climate and geophysics applications. However, in many practical scenarios, we have only partial or imprecise knowledge of these equations, leading to mis-specified problems, i.e. problems for which there is no solution to the given equations when coupled with real-world data. Physics-informed neural networks (PINNs) have been increasingly used for data assimilation, yet their efficacy has primarily been demonstrated for well-specified problems where solutions for the given equation and data exist. Moreover, in applications involving real-world noisy data and complex PDEs, it is often difficult to know whether the given problem is mis-specified. We introduce a framework for deriving well-specified problems from mis-specified problems—based on the Euler-Lagrange equations (also called the Euler operator) and other tools from mathematical analysis—closing the gap between problems which may be mis-specified in practice and the already-proven theory for the well-specified case. Based on this framework, we propose a diagnostic tool for determining whether given PDE problems are mis-specified, and prove generalization bounds for mis-specified PINNs. Our contributions enhance PINN reliability in the presence of partial knowledge of governing equations or imperfect data.
2023
-
Logion: Machine-Learning Based Detection and Correction of Textual Errors in Greek Philology
RANLP Workshop on Ancient Language ProcessingAbstract
We present statistical and machine-learning based techniques for detecting and correcting errors in text and apply them to the challenge of textual corruption in Greek philology. Most ancient Greek texts reach us through a long process of copying, in relay, from earlier manuscripts (now lost). In this process of textual transmission, copying errors tend to accrue. After training a BERT model on the largest premodern Greek dataset used for this purpose to date, we identify and correct previously undetected errors made by scribes in the process of textual transmission, in what is, to our knowledge, the first successful identification of such errors via machine learning. The premodern Greek BERT model we train is available for use at \urlhttps://huggingface.co/cabrooks/LOGION-base
-
Machine Learning and the Future of Philology: A Case Study
Transactions of the American Philological AssociationCite
@article{Graziosi2023, title = {Machine Learning and the Future of Philology: A Case Study}, author = {Graziosi, Barbara and Haubold, Johannes and Cowen-Breen, Charlie and Brooks, Creston}, journal = {Transactions of the American Philological Association}, volume = {153}, number = {1}, pages = {253--284}, year = {2023}, publisher = {Project MUSE}, }
Theses and other work
2023
-
Logion: Technical Report
Technical report -
Guidelines for Sample Sizes: Confirmatory Randomised Controlled Trials are nearly Optimal for Overall Patient Health, if the Right Number of Patients is Chosen
Master’s Thesis
2022
-
Navigating Physically-Informed Loss Landscapes with Stochastic Gradient Descent
Undergraduate Senior Thesis
2021
-
Inferring flow law of ice shelves using physics-informed neural networks
APS Division of Fluid Dynamics