Benjamin Eysenbach

Room 416

35 Olden St

Princeton NJ 08544

eysenbach@princeton.edu

I run the Princeton Reinforcement Learning Lab, where we study the science of intelligent decision making. How can AI systems learn to explore the world and uncover previously-unknown skills or knowledge? Can agents learn about the structure of the world, about the texture of time? How much of this learning can be done in an unsupervised fashion?

We answer these and similar questions by designing AI algorithms. While we might refer to these as ``reinforcement learning’’ methods, many do not use rewards at all; exploration and learning are instead driven by (often much richer) intrinsic feedback. Our group has developed some of the foremost algorithms and analysis for such self-supervised control methods. Here are a few example papers; here and here are some tutorials to learn more about our research.

My work has been recognized by a NeurIPS Best Paper Award, an NSF CAREER Award, a Hertz Fellowship, an NSF GRFP Fellowship, the Alfred Rheinstein Faculty Award, as well as several teaching awards. Before joining Princeton, I did by PhD in machine learning at CMU under Ruslan Salakhutdinov and Sergey Levine. I spent a number of years at Google Brain/Research before and during my PhD. My undergraduate studies were in math at MIT.

My lab’s research is supported by the National Science Foundation, the Sloan Foundation, NVIDIA, Google, Toyota Research Institute, and Princeton University.

Come do science with us! Check out this page for details on how to get involved in the research, whether as an undergrad, PhD student, postdoc, visiting student, or external collaborator.

news

Jun 7, 2026	Princeton RL @ ICLR 2026! Demystifying the Mechanisms Behind Emergent Exploration in Goal-conditioned RL, led by Mahsa Bastankhah and Grace Liu. Interactive demo! Value Flows, led by Perry Dong and Chongyi Zheng. Intention-Conditioned Flow Occupancy Models, led by Chongyi Zheng. SLAP: Shortcut Learning for Abstract Planning, led by Isabel Liu. Temporal Representations for Exploration: Learning Complex Exploratory Behavior without Extrinsic Rewards, led by Faisal Mohamed. Scaling Goal-conditioned Reinforcement Learning with Multistep Quasimetric Distances, led by Bill Zheng. Occupancy Reward Shaping: Improving Credit Assignment for Offline Goal-Conditioned Reinforcement Learning, led by Aravind Venugopal. Skill Learning via Policy Diversity Yields Identifiable Representations for Reinforcement Learning, led by Patrik Reizinger, Bálint Mucsányi and Siyuan Guo.
Mar 15, 2026	Looking forward to sharing research from our group at upcoming workshops! Frontiers in Online Reinforcement Learning @ IMSI. (Recording coming soon) Physical Foundations of Intelligent Systems @ Princeton AI Lab.
Feb 15, 2026	Honored to receive the Sloan Fellowship!
Jan 2, 2026	Honored and excited to receive a grant from Google to study Accelerating Reinforcement Learning in Hard-To-Verify Domains via Empowerment and Skills.
Dec 1, 2025	Princeton RL @ NeurIPS 2025! I’m excited to share progress we’ve made in RL algorithms (and the many problems still unsolved): 1000 Layer Networks for Self-Supervised RL: Scaling Depth Can Enable New Goal-Reaching Capabilities, led by Kevin Wang and Ishaan Javali and Michał Bortkiewicz. (oral) Normalizing Flows are Capable Models for RL. Led by Raj Ghugare. Contrastive Representations for Temporal Reasoning Led by Alicja Ziarko, with Michal Bortkiewicz, Michal Zawalski, and Piotr Milos. Horizon Reduction Makes Offline RL Scalable. Let by Seohong Park, with Kevin Frans, Deepinder Mann, Aviral Kumar, Sergey Levine. (spotlight) We’re also presenting preliminary work at several of the workshops: Low-Rank Successor Representations Capture Human-Like Generalization. Led by Eva Yi Xie, with Nathaniel D. Daw. (Unifying Representations in Neural Models) Structured Response Diversity with Mutual Information. Led by Devan Shah, Owen Yang, and Daniel Yang, with Chongyi Zheng. (Scaling Environments for Agents) Demystifying emergent exploration in goal-conditioned RL. Led by Mahsa Bastankhah and Grace Liu, with Dilip Arumugam and Thomas L. Griffiths. (Aligning Reinforcement Learning Experimentalists and Theorists; Interpreting Cognition in Deep Learning Models) Training LLM Agents to Empower Humans. Evan Ellis, Vivek Myers, Jens Tuyls, Sergey Levine, Anca Dragan. (Deep Learning for Code) Unsupervised Contrastive Goal Reaching. Led by Ahmed Turkman, with Raj Ghugare. (Aligning Reinforcement Learning Experimentalists and Theorists) Combinatorial Representations for Temporal Reasoning. Led by Alicja Ziarko, with Michał Bortkiewicz, Michał Zawalski, Piotr Miłoś. (Differentiable Learning of Combinatorial Algorithms; Unifying Representations in Neural Models; WiML) Horizon Reduction Makes Offline RL Scalable. Led by Seohong Park, with Kevin Frans, Deepinder Mann, Aviral Kumar, Sergey Levine. (Aligning Reinforcement Learning Experimentalists and Theorists) Finally, I’m organizing a NeurIPS 2025 workshop, Data on the Brain & Mind, together with Eva Yi Xie, Catherine Ji, Vivek Myers, Archer Wang, Mahsa Bastankhah, Jenelle Feather, Erin Grant, and Richard Gao. On December 7th.
Oct 2, 2025	Honored and excited to receive a GPU grant from NVIDIA for research into Interactive Generative Models that Learn via Discovery, not Data.
Jul 15, 2025	Awarded the NSF CAREER award for Unsupervised and Autonomous Reinforcement Learning of Skills
Jul 14, 2025	I gave an ICML tutorial on generative AI and reinforcement learning intrinsic motivation and self-supervised RL, together with Amy Zhang. Recording and slides are available on the tutorial website.
Jun 11, 2025	I gave a tutorial on intrinsic motivation and self-supervised RL at RLDM! Recording and slides are available on the tutorial website.
Apr 24, 2025	Princeton RL @ ICLR 2025! Some say hi in Singapore! A Single Goal is All You Need: Skills and Exploration Emerge from Contrastive RL without Rewards, Demonstrations, or Subgoals. Led by Grace Liu and Michael Tang. Accelerating Goal-Conditioned Reinforcement Learning Algorithms and Research. Led by Michał Bortkiewicz and lots of amazing co-authors. Invariance to Planning in Goal-Conditioned RL. Led by Cathy Ji and Vivek Myers. The “Law’’ of the Unconscious Contrastive Learner: Probabilistic Alignment of Unpaired Modalities. Led by Yongwei Che. Can a MISL Fly? Analysis and Ingredients for Mutual Information Skill Learning. Led by Chongyi Zheng and Jens Tuyls. OGBench: Benchmarking Offline Goal-Conditioned RL. Led by Seohong Park and Kevin Frans.

selected publications

The aim is to highlight a small subset of the work done in the group, and to give a sense for the sorts of problems that we're working on. Please see Google Scholar for a complete and up-to-date list of publications.

2025

BuilderBench–A benchmark for generalist agents

Raj Ghugare, Catherine Ji, Kathryn Wantlin, Jin Schofield, and Benjamin Eysenbach

arXiv preprint arXiv:2510.06288, 2025

PDF Code Website
1000 Layer Networks for Self-Supervised RL: Scaling depth can enable new goal-reaching capabilities

Kevin Wang, Ishaan Javali, MichaĹ Bortkiewicz, Benjamin Eysenbach, and others

In Advances in Neural Information Processing Systems, 2025

PDF Code Website
Intention-Conditioned Flow Occupancy Models

Chongyi Zheng, Seohong Park, Sergey Levine, and Benjamin Eysenbach

arXiv preprint arXiv:2506.08902, 2025

PDF Code Website
Identifying nonequilibrium degrees of freedom in high-dimensional stochastic systems

Catherine Ji, Ravin Raj, Benjamin Eysenbach, and Gautam Reddy

arXiv preprint arXiv:2508.08247, 2025

PDF Code
Contrastive Representations for Temporal Reasoning

Alicja Ziarko, Michal Bortkiewicz, Michal Zawalski, Benjamin Eysenbach, and Piotr Milos

In Advances in Neural Information Processing Systems, 2025

PDF Code Website
Invariance to Planning in Goal-Conditioned RL

Catherine Ji, Vivek Myers, and Benjamin Eysenbach

In International Conference on Learning Representations, 2025

PDF Code Website
Accelerating Goal-Conditioned Reinforcement Learning Algorithms and Research. Led by Michał Bortkiewicz

Michał Bortkiewicz, Władek Pałucki, Vivek Myers, Tadeusz Dziarmaga, Tomasz Arczewski, Łukasz Kuciński, and Benjamin Eysenbach

In International Conference on Learning Representations, 2025

PDF Code
A Single Goal is All You Need: Skills and Exploration Emerge from Contrastive RL without Rewards, Demonstrations, or Subgoals

Grace Liu, Michael Tang, and Benjamin Eysenbach

In International Conference on Learning Representations, 2025

PDF Code Website

2024

Learning to Assist Humans without Inferring Rewards

Vivek Myers, Evan Ellis, Sergey Levine, Benjamin Eysenbach, and Anca Dragan

In Neural Information Processing Systems, 2024

PDF Code Website
Inference via Interpolation: Contrastive Representations Provably Enable Planning and Inference

Benjamin Eysenbach, Vivek Myers, Russ Salakhutdinov, and Sergey Levine

In Neural Information Processing Systems, 2024

PDF Code
Learning Temporal Distances: Contrastive Successor Features Can Provide a Metric Structure for Decision-Making

Vivek Myers, Chongyi Zheng, Anca Dragan, Sergey Levine, and Benjamin Eysenbach

In Forty-first International Conference on Machine Learning, 2024

PDF Code
Closing the Gap between TD Learning and Supervised Learning - A Generalisation Point of View

Raj Ghugare, Geist Matthieu, Glen Berseth, and Benjamin Eysenbach

In International Conference on Learning Representations, 2024

PDF Code
Contrastive Difference Predictive Coding

Chongyi Zheng, Ruslan Salakhutdinov, and Benjamin Eysenbach

In International Conference on Learning Representations, 2024

PDF Code Website
Stabilizing Contrastive RL: Techniques for Robotic Goal Reaching from Offline Data

Chongyi Zheng, Benjamin Eysenbach, Homer Walke, Patrick Yin, Kuan Fang, Ruslan Salakhutdinov, and Sergey Levine

In International Conference on Learning Representations, 2024

PDF Code Website

2023

Contrastive value learning: Implicit Models for Simple Offline RL

Bogdan Mazoure, Benjamin Eysenbach, Ofir Nachum, and Jonathan Tompson

In Conference on Robot Learning, 2023

PDF Code Video
Simplifying Model-based RL: Learning Representations, Latent-space Models, and Policies with One Objective

Raj Ghugare, Homanga Bharadhwaj, Benjamin Eysenbach, Sergey Levine, and Russ Salakhutdinov

In International Conference on Learning Representations , 2023

PDF Code Website
A Connection between One-Step RL and Critic Regularization in Reinforcement Learning

Benjamin Eysenbach, Matthieu Geist, Sergey Levine, and Ruslan Salakhutdinov

In International Conference on Machine Learning, 2023

PDF
Contrastive Example-Based Control

Kyle Beltran Hatch, Benjamin Eysenbach, Rafael Rafailov, Tianhe Yu, Ruslan Salakhutdinov, Sergey Levine, and Chelsea Finn

In Learning for Dynamics and Control Conference, 2023

PDF
Probabilistic Reinforcement Learning: Using Data to Define Desired Outcomes, and Inferring How to Get There

Benjamin Eysenbach

PhD Thesis, Carnegie Mellon University, 2023

PDF

2022

Mismatched No More: Joint Model-Policy Optimization for Model-Based RL

Benjamin Eysenbach, Alexander Khazatsky, Sergey Levine, and Ruslan Salakhutdinov

In Advances in Neural Information Processing Systems, 2022

PDF Code Video
Contrastive Learning as Goal-Conditioned Reinforcement Learning

Benjamin Eysenbach, Tianjun Zhang, Ruslan Salakhutdinov, and Sergey Levine

In Advances in Neural Information Processing Systems, 2022

PDF Code Video Website
Imitating Past Successes can be Very Suboptimal

Benjamin Eysenbach, Soumith Udatha, Russ R Salakhutdinov, and Sergey Levine

In Advances in Neural Information Processing Systems, 2022

PDF Code Video
The Information Geometry of Unsupervised Reinforcement Learning

Benjamin Eysenbach, Ruslan Salakhutdinov, and Sergey Levine

In International Conference on Learning Representations, 2022

PDF Code Video
Maximum Entropy RL (Provably) Solves Some Robust RL Problems

Benjamin Eysenbach, and Sergey Levine

In International Conference on Learning Representations, 2022

PDF Blog

2021

Replacing Rewards with Examples: Example-Based Policy Search via Recursive Classification

Benjamin Eysenbach, Sergey Levine, and Ruslan Salakhutdinov

Advances in Neural Information Processing Systems, 2021

PDF Blog Code Website
C-Learning: Learning to Achieve Goals via Recursive Classification

Benjamin Eysenbach, Ruslan Salakhutdinov, and Sergey Levine

In International Conference on Learning Representations, 2021

PDF Code Website
Robust Predictable Control

Benjamin Eysenbach, Ruslan Salakhutdinov, and Sergey Levine

In Advances in Neural Information Processing Systems, 2021

PDF Code Video Website

2019

Search on the replay buffer: Bridging planning and reinforcement learning

Benjamin Eysenbach, Ruslan Salakhutdinov, and Sergey Levine

In Advances in Neural Information Processing Systems, 2019

PDF Blog Code
Diversity is All You Need: Learning Skills without a Reward Function

Benjamin Eysenbach, Abhishek Gupta, Julian Ibarz, and Sergey Levine

In International Conference on Learning Representations, 2019

PDF Code Website