publications

2024

1. 

   Towards Probabilistically-Sound Beam Search with Masked Language Models

   Creston Brooks*, Robert Calef*, Charlie Cowen-Breen*, and 1 more author

   NeurIPS Workshop on Statistical Foundations of LLMs and Foundation Models, 2024

   Abs

   Beam search with masked language models (MLMs) is challenging in part because joint probability distributions over sequences are not readily available, unlike for autoregressive models. However, estimating such distributions has important domain-specific applications such as ancient text restoration and protein engineering. Here we present probabilistically-sound methods for beam search with MLMs. First, we clarify the conditions under which it is theoretically sound to perform text infilling with MLMs using standard beam search. When these conditions fail, we provide a probabilistically-sound inference time modification with no additional computational complexity and demonstrate that it is superior to the aforementioned beam search in the expected conditions. We then present empirical results comparing several infilling approaches with MLMs across several domains. Notably, our method probes the inductive biases of MLMs and explores the surprising contextual sensitivity of mask tokens for text infilling.

2. 

   Euler Operators for Mis-specified Physics-Informed Neural Networks

   Charlie Cowen-Breen, Yongji Wang, Stephen Bates, and 1 more author

   ICML Workshop on AI for Science, 2024

   Abs

   Integration between equations and data is ubiquitously useful in the physical sciences: for example, data assimilation plays an important role in generating seamless data for climate and geophysics applications. However, in many practical scenarios, we have only partial or imprecise knowledge of these equations, leading to mis-specified problems, i.e. problems for which there is no solution to the given equations when coupled with real-world data. Physics-informed neural networks (PINNs) have been increasingly used for data assimilation, yet their efficacy has primarily been demonstrated for well-specified problems where solutions for the given equation and data exist. Moreover, in applications involving real-world noisy data and complex PDEs, it is often difficult to know whether the given problem is mis-specified. We introduce a framework for deriving well-specified problems from mis-specified problems—based on the Euler-Lagrange equations (also called the Euler operator) and other tools from mathematical analysis—closing the gap between problems which may be mis-specified in practice and the already-proven theory for the well-specified case. Based on this framework, we propose a diagnostic tool for determining whether given PDE problems are mis-specified, and prove generalization bounds for mis-specified PINNs. Our contributions enhance PINN reliability in the presence of partial knowledge of governing equations or imperfect data.

2023

1. 

   Logion: Machine-Learning Based Detection and Correction of Textual Errors in Greek Philology

   Charlie Cowen-Breen, Creston Brooks, Johannes Haubold, and 1 more author

   RANLP Workshop on Ancient Language Processing, 2023

   Abs PDF

   We present statistical and machine-learning based techniques for detecting and correcting errors in text and apply them to the challenge of textual corruption in Greek philology. Most ancient Greek texts reach us through a long process of copying, in relay, from earlier manuscripts (now lost). In this process of textual transmission, copying errors tend to accrue. After training a BERT model on the largest premodern Greek dataset used for this purpose to date, we identify and correct previously undetected errors made by scribes in the process of textual transmission, in what is, to our knowledge, the first successful identification of such errors via machine learning. The premodern Greek BERT model we train is available for use at \urlhttps://huggingface.co/cabrooks/LOGION-base

2. 

   Machine Learning and the Future of Philology: A Case Study

   Barbara Graziosi, Johannes Haubold, Charlie Cowen-Breen, and 1 more author

   Transactions of the American Philological Association, 2023

   Bib HTML Website

   @article{Graziosi2023,
     title = {Machine Learning and the Future of Philology: A Case Study},
     author = {Graziosi, Barbara and Haubold, Johannes and Cowen-Breen, Charlie and Brooks, Creston},
     journal = {Transactions of the American Philological Association},
     volume = {153},
     number = {1},
     pages = {253--284},
     year = {2023},
     publisher = {Project MUSE},
   }

3. 

   Discovering the rheology of Antarctic Ice Shelves via physics-informed deep learning

   Yongji Wang, Ching-Yao Lai, and Charlie Cowen-Breen

   (Preprint), 2023

   HTML

4. 

   Logion: Technical Report

   Charlie Cowen-Breen, Creston Brooks, Johannes Haubold, and 1 more author

   2023

   arXiv Code
5. Master’s Thesis

   Guidelines for Sample Sizes: Confirmatory Randomised Controlled Trials are nearly Optimal for Overall Patient Health, if the Right Number of Patients is Chosen

   Charlie Cowen-Breen

   2023

2022

1. Undergraduate Senior Thesis

   Navigating Physically-Informed Loss Landscapes with Stochastic Gradient Descent

   Charlie Cowen-Breen

   2022

2021

1. 

   Inferring flow law of ice shelves using physics-informed neural networks

   Yongji Wang, Elijah Pomerantz, Charlie Cowen-Breen, and 2 more authors

   APS Division of Fluid Dynamics, 2021

   HTML