My research develops mathematical and conceptual foundations for fallible AI systems. The result so far has been a unifying picture that explains many standard but seemingly ad-hoc choices made in practice. A key technical ingredient is a class of models called Probabilistic Dependency Graphs (PDGs) which subsume traditional graphical models, yet can model inconsistent beliefs and most scenarios in machine learning. Indeed, many important algorithms in AI turn out to be instances of an intuitive heuristic approach to resolving probabilistic inconsistency.
For an overview, see my
research statement
[last update March 2024] ;
for (a great deal) more, see
my
dissertation.
Papers and Publications
-
Abstract. We present a generic algorithm for learning and approximate inference with an intuitive epistemic interpretation: iteratively focus on a subset of the model and resolve inconsistencies using the parameters under control. This framework, which we call Local Inconsistency Resolution (LIR) is built upon Probabilistic Dependency Graphs (PDGs), which provide a flexible representational foundation capable of capturing inconsistent beliefs. We show how LIR unifies and generalizes a wide variety of important algorithms in the literature, including the Expectation-Maximization (EM) algorithm, belief propagation, adversarial training, GANs, and GFlowNets. In the last case, LIR actually suggests a more natural loss, which we demonstrate improves GFlowNet convergence. Each method can be recovered as a specific instance of LIR by choosing a procedure to direct focus (attention and control). We implement this algorithm for discrete PDGs and study its properties on synthetically generated PDGs, comparing its behavior to the global optimization semantics of the full PDG.
Notes. Earlier versions of the core idea in this work in “Local Resolution Algorithm, presented at ICML workshops in 2023, and in Chapter 7 of my PhD dissertation. My co-authors helped develop a general implementation and synthetic experiments, test the hypotheses posed by the paper, and generally bring it into the form the paper is in now.
-
![]()
Abstract. Model-based and model-free reinforcement learning are traditionally viewed as separate paradigms: instead of learning a model of the transition kernel P, model-free agents typically estimate value functions tied to a specific policy and reward. In this paper, we challenge this dichotomy by proving that value-based agents trained on a sufficiently rich set of reward functions, e.g. using goal-conditioned RL, implicitly encode a unique and accurate world model. To extract this model in practice, we introduce P-learning, an inverse analogue to Q-learning that samples from an agent’s Q-values, policies and rewards to decode its internal model of the environment. We then provide sufficient conditions on the type and number of goals for which agents encode the true kernel P, covering both stochastic and deterministic MDPs over finite or continuous state spaces. Even when our assumptions are violated, we empirically demonstrate that agents trained on a handful of reward functions encode accurate dynamics in Reacher, MountainCar and stochastic variants of FourRooms. Surprisingly, we find that policies trained exclusively on a Reacher agent’s implicit world model are quasi-optimal on out-of-distribution, velocity-based goals despite position-only training – suggesting that agents contain hidden generalisation capabilities and providing a new lens into the connection between model-based, model-free, and goal-conditioned RL
-
Abstract. When a probabilistic predictor answers many conditional-probability queries, are its answers approximately self-consistent—-and, if so, can this be verified in polynomial-time? This problem is of interest for AI safety, when safety is derived from honesty about probabilistic predictions of unwanted outcomes potentially caused by an AI action. To address it, we construct an interactive PCP as follows. Let a predictive model be specified by a probability circuit P and a circuit Q which outputs confidence in predictions. P and Q together implicitly specify exponentially many probabilistic claims. We show a protocol in which a polynomial time verifier can verify (P, Q)’s approximate consistency. The verifier is given the pair of circuits (P, Q), which it evaluates at only a few points; alongside them it is given a proof oracle, an encoding of a witnessing probability distribution allegedly consistent with the predictions of (P, Q), which it reads at a few locations while interacting with a single un-trusted prover. En-route to the above result, we need to ensure the existence of a sparse witnessing prob- ability distribution consistent with the model predictions. To do so, we first consider witness distributions for the consistency of explicit (rather than specified by a predictor) probabilistic claims: say m claims, each of the form “Pr[Y = 1 | X = x] = p”, over n Boolean variables. Building on a body of literature initiated by Nilsson (Artif. Intel. 1986), we place ℓ2 approximate probabilistic consistency of explicit claims in NP with certificates of length O(mn + log B) in the input bit-precision B; and further show how a small additive completeness–soundness gap removes dependence on the model’s precision B. This will be important for our interactive PCP constructions.
Together these results provide a complexity-theoretic foundation for certifying the self- consistency of probabilistic predictors. We view the explicit Interactive PCP we present as the first step toward the eventual training of predictive models to prove their own consistency.
![]()
Left: certifying an ℓ2 upper bound on consistency is NP-complete for explicitly given claims. Right: verifying an entire implicitly encoded predictive model is in NEXPTIME and can therefore be done in polynomial time by interacting with a powerful prover. We go further and construct an explicit protocol for this purpose.
-
*main contribution, †supervision
![]()
Abstract.
We provide evidence that language models can detect, localize and, to a certain degree, verbalize the
difference between perturbations applied to their activations. More precisely, we either (a) mask activations,
simulating dropout, or (b) add Gaussian noise to them, at a target sentence. We then ask a multiple-choice
question such as “Which of the previous sentences was perturbed?” or “Which of the two perturbations was
applied?”. We test models from the Llama, Olmo, and Qwen families, with sizes between 8B and 32B, all of
which can easily detect and localize the perturbations, often with perfect accuracy. These models can also
learn, when taught in context, to distinguish between dropout and Gaussian noise. Notably, Qwen3-32B’s
zero-shot accuracy in identifying which perturbation was applied improves as a function of the perturbation
strength and, moreover, decreases if the in-context labels are flipped, suggesting a prior for the correct
ones—even modulo controls. Because dropout has been used as a training-regularization technique, while
Gaussian noise is sometimes added during inference, we discuss the possibility of a data-agnostic “training
awareness” signal and the implications for AI safety.
-
![]()
Abstract.
Chain-of-Thought (CoT) reasoning has advanced the capabilities and transparency of language models (LMs); however, reasoning chains can contain inaccurate statements that reduce performance and trustworthiness. To address this, we propose to augment each reasoning step in a CoT with a latent veracity (or correctness) variable. To efficiently explore this expanded space, we introduce Veracity Search (VS), a discrete search algorithm over veracity assignments. It performs otherwise intractable inference in the posterior distribution over latent veracity values by leveraging the LM's joint likelihood over veracity and the final answer as a proxy reward. This efficient inference-time verification method facilitates supervised fine-tuning of an Amortized Veracity Inference (AVI) machine by providing pseudo-labels for veracity. AVI generalizes VS, enabling accurate zero-shot veracity inference in novel contexts. Empirical results demonstrate that VS reliably identifies errors in logical (ProntoQA), mathematical (GSM8K), and commonsense (CommonsenseQA) reasoning benchmarks, with AVI achieving comparable zero-shot accuracy. Finally, we demonstrate the utility of latent veracity inference for providing feedback during self-correction and self-improvement.
-
Abstract. We characterize a notion of confidence that arises when learning or updating beliefs. This notion of trust, or learner’s confidence, can be used alongside (and is easily be mistaken for) probability or likelihood, but it is fundamentally a different concept. Although perhaps not as useful as probability itself, our notion of confidence captures and unifies many concepts in the literature, from Shafer’s weight of evidence, to Kalman gain, as well as number of training epochs and learning rate. We provide a mathematical definition of what it means to learn with confidence, and give two canonical ways of measuring confidence on a continuum. Under additional assumptions, we derive more compact representations of confidence-based learning in terms of vector fields and loss functions. These representations induce an extended language of compound “parallel” observations. We illustrate our framework by analyzing standard ways of updating beliefs.
-
Abstract. We define what it means for a joint probability distribution to be compatible with a set of independent causal mechanisms, at a qualitative level—or, more precisely, with a directed hypergraph A, which is the qualitative structure of a probabilistic dependency graph (PDG). When A represents a qualitative Bayesian network, QIM-compatibility with A reduces to satisfying the appropriate conditional independencies. But giving semantics to hypergraphs using QIM-compatibility lets us do much more. For one thing, we can capture functional dependencies. For another, we can capture important aspects of causality using compatibility: we can use compatibility to understand cyclic causal graphs, and to demonstrate structural compatibility, we must essentially produce a causal model. Finally, QIM-compatibility has deep connections to information theory. Applying compatibility to cyclic structures helps to clarify a longstanding conceptual issue in information theory.
-
Abstract. What should you do with conflicting information? To be rational in the traditional sense, you must immediately resolve the inconsistency, so as to maintain a consistent (probabilistic) picture of the world. But how? And is it really critical to do so immediately? Inconsistency is clearly undesirable, but, as we will soon show, we stand to gain a lot by representing it.
This thesis develops a broad theory of how to approach probabilistic modeling with possibly-inconsistent information, unifying and reframing much of the literature in graphical models and machine learning in the process. The key ingredient is a novel kind of graphical model, called a Probabilistic Dependency Graph (PDG), which allows for arbitrary (even conflicting) pieces of probabilistic information.
- In Part I, we establish PDGs as a generalization of other models of mental state, including traditional graphical models such as Bayesian Networks and Factor Graphs, as well as causal models, and even generalizations of probability distributions, such as Dempster-Shafer Belief functions.
- In Part II, we show that PDGs also capture modern neural representations. Surprisingly, standard loss functions can be viewed as the inconsistency of a PDG that models the situation appropriately. Furthermore, many important algorithms in AI are instances of a simple approach to resolving inconsistencies.
- In Part III, we provide algorithms for PDG inference, and uncover a deep algorithmic equivalence between the problems of inference and calculating a PDG’s numerical degree of inconsistency. We also develop powerful yet inutuitive principles for reasoning with (and about) PDGs.
-
Mixture LanguagesOliver Richardson and Jialu Bao
POPL, 2024 Languages for Inference (LAFI) Workshop
-
The Local Inconsistency Resolution AlgorithmOliver Richardson
ICML, 2023 Workshop on Structured Probabilistic Inference & Generative Modeling (SPIGM); Workshop on Localized Learning (LLW)
-
Inference for Probabilistic Dependency GraphsOliver Richardson, Joseph Halpern, and Christopher De Sa
UAI, 2023
-
![]()
Abstract.
In a world blessed with a great diversity of loss functions, we argue that that choice between them is not a matter of taste or pragmatics, but of model. Probabilistic depencency graphs (PDGs) are probabilistic models that come equipped with a measure of "inconsistency". We prove that many standard loss functions arise as the inconsistency of a natural PDG describing the appropriate scenario, and use the same approach to justify a well-known connection between regularizers and priors. We also show that the PDG inconsistency captures a large class of statistical divergences, and detail benefits of thinking of them in this way, including an intuitive visual language for deriving inequalities between them. In variational inference, we find that the ELBO, a somewhat opaque objective for latent variable models, and variants of it arise for free out of uncontroversial modeling assumptions---as do simple graphical proofs of their corresponding bounds. Finally, we observe that inconsistency becomes the log partition function (free energy) in the setting where PDGs are factor graphs.
-
Probabilistic Dependency GraphsOliver Richardson and Joseph Halpern
AAAI, 2021
-
Complexity and Scale: Understanding the CreativeOliver Richardson
International Association of Computing and Philosophy (IACAP), 2014
-
Capitalization in the St Petersburg Game: Why Statistical Distributions MatterMariam Thalos and Oliver Richardson
Politics, Philosophy & Economics, 2014
Abstract.
We provide evidence that language models can detect, localize and, to a certain degree, verbalize the
difference between perturbations applied to their activations. More precisely, we either (a) mask activations,
simulating dropout, or (b) add Gaussian noise to them, at a target sentence. We then ask a multiple-choice
question such as “Which of the previous sentences was perturbed?” or “Which of the two perturbations was
applied?”. We test models from the Llama, Olmo, and Qwen families, with sizes between 8B and 32B, all of
which can easily detect and localize the perturbations, often with perfect accuracy. These models can also
learn, when taught in context, to distinguish between dropout and Gaussian noise. Notably, Qwen3-32B’s
zero-shot accuracy in identifying which perturbation was applied improves as a function of the perturbation
strength and, moreover, decreases if the in-context labels are flipped, suggesting a prior for the correct
ones—even modulo controls. Because dropout has been used as a training-regularization technique, while
Gaussian noise is sometimes added during inference, we discuss the possibility of a data-agnostic “training
awareness” signal and the implications for AI safety.
Abstract.
Chain-of-Thought (CoT) reasoning has advanced the capabilities and transparency of language models (LMs); however, reasoning chains can contain inaccurate statements that reduce performance and trustworthiness. To address this, we propose to augment each reasoning step in a CoT with a latent veracity (or correctness) variable. To efficiently explore this expanded space, we introduce Veracity Search (VS), a discrete search algorithm over veracity assignments. It performs otherwise intractable inference in the posterior distribution over latent veracity values by leveraging the LM's joint likelihood over veracity and the final answer as a proxy reward. This efficient inference-time verification method facilitates supervised fine-tuning of an Amortized Veracity Inference (AVI) machine by providing pseudo-labels for veracity. AVI generalizes VS, enabling accurate zero-shot veracity inference in novel contexts. Empirical results demonstrate that VS reliably identifies errors in logical (ProntoQA), mathematical (GSM8K), and commonsense (CommonsenseQA) reasoning benchmarks, with AVI achieving comparable zero-shot accuracy. Finally, we demonstrate the utility of latent veracity inference for providing feedback during self-correction and self-improvement.
Abstract.
In a world blessed with a great diversity of loss functions, we argue that that choice between them is not a matter of taste or pragmatics, but of model. Probabilistic depencency graphs (PDGs) are probabilistic models that come equipped with a measure of "inconsistency". We prove that many standard loss functions arise as the inconsistency of a natural PDG describing the appropriate scenario, and use the same approach to justify a well-known connection between regularizers and priors. We also show that the PDG inconsistency captures a large class of statistical divergences, and detail benefits of thinking of them in this way, including an intuitive visual language for deriving inequalities between them. In variational inference, we find that the ELBO, a somewhat opaque objective for latent variable models, and variants of it arise for free out of uncontroversial modeling assumptions---as do simple graphical proofs of their corresponding bounds. Finally, we observe that inconsistency becomes the log partition function (free energy) in the setting where PDGs are factor graphs.
Academic Talks
-
In search of expressive, modular, and (mostly) consistent models
Invited Talk @ CHAI 2026: Tenth Annual Workshop of the Center for Human-Compatible AI (5 June 2026). -
Handling internal conflict in probabilistic models
Invited Talk @ Center for Human-Compatible AI (CHAI) visiting speaker series (30 April 2026). -
The Inconsistency Quantification Problem
@ Brown University Theory Seminar (2 April 2026). -
Structural deficiency with respect to a directed hypergraph: information-theoretic constraints for generalized causal models
@ MIT CSAIL Theory Lunch (27 March 2026). -
The Pursuit of Epistemic Consistency as a "Universal" Objective
@ ILIAD Conference (29 July 2024). -
A Probabilistic Model of Belief, Dependence, and Inconsistency
PhD Defense / B-Exam @ Cornell CS Department (16 July 2024). -
How to Compute with PDGs: Inference, Inconsistency Measurement, and the Close Relationship Between the Two
Invited Talk @ Cornell CS Theory Seminar (19 February 2024). -
Learning, Inference, and the Pursuit of Consistency
Invited Talk @ Cornell CS Student Colloquium (7 December 2023). -
Probabilistic (In)consistency as a Basis for Learning and Inference
Invited Talk @ University of Tenessee CS Dept (29 November 2023). -
A Tutorial on Bialgebra
@ Cornell Seminar on Programming Languages (17 September 2022). -
Loss as the Inconsistency of a Probabilistic Dependency Graph: Choose Your Model, Not Your Loss Function
Invited Talk @ Cornell Seminar on Artificial Intelligence (2 September 2022). -
Probabilistic Dependency Graphs and Inconsistency: How to Model, Measure, and Mitigate Internal Conflict
PhD Proposal / A-Exam @ Cornell CS Department (17 September 2021). -
Probabilistic Dependency Graphs
@ Cornell CS Theory Tea (17 March 2021). -
Cat Thoughts: An Intro to Categorical Thinking
@ Cornell Graduate Student Seminar (20 February 2019).