Prediction, Explanation, or Over-interpretation? (June, 2026)
Can language models verbalize their own computation? We compare three verbalization methods — self-report, activation oracle, and natural language autoencoder — and find that none is consistent across tasks, with itself, or with the others. 🔍
Hi! I'm Xiaoyan(Elena) Bai
柏 晓琰
I am currently a PhD student in CS at University of Chicago , advised by Chenhao Tan . I am also a member of Chicago Human+AI lab. I was a scholar in the MATS 10.0 cohort mentored by Alan Cooney.
Before that, I earned B.E in Computer Science at CSE at Univeristy of Michigan, Ann Arbor. During my undergrad study, I was honored to be a member of Language and Information Technologies (LIT) lab group to work with Dr. Andrew Lee and Professor Rada Mihalcea. I also worked on NLP inference time efficiency with Dr. Haizhong Zheng in Professor Atul Prakash's group.
When I am not doing research, I like traveling, photography and games.
Research Interest: As machine learning continues to advance, so does the potential for the digital divide and unsafety to widen. These issues motivate my research in building transparent, efficient, and more accessible tools for social good. I believe this can be done by building interpretable models and understanding models. In general, I am interested in AI safety, interpretability, and agentic workflow.
News
- 2026I will be organizing the Interp4Discovery workshop at NeurIPS 2026.
- 2026The Last Human-Written Paper: Agent-Native Research Artifacts was accepted at NeurIPS 2026.
- 2026The Story is Not the Science received the ICML AI4Science workshop Best Poster Award.
Selected Publications
The Story is Not the Science:
Execution-Grounded Evaluation of Mechanistic Interpretability Research (ICML2026 AI4Science workshop Best Poster Award)
Xiaoyan Bai, Alexander Baumgartner*, Haojia Sun*, Ari Holtzman, Chenhao Tan
Code |
Blog
Know Thyself? On the Incapability and Implications of AI Self-Recognition (MSLD 2026)
Xiaoyan Bai, Aryan Shrivastava, Ari Holtzman, Chenhao Tan
Code |
MIT Technology Review China
Why Can't Transformers Learn Multiplication? Reverse-Engineering Reveals Long-Range Dependency Pitfalls (preprint)
Xiaoyan Bai*, Itamar Pres*, Yuntian Deng, Chenhao Tan, Stuart Shieber, Fernanda Viégas, Martin Wattenberg, Andrew Lee
Code | FOX 32 Chicago News |
UChicago DSI spotlight |
Harvard D^3 Institute News | UChicago News
Concept Incongruence: An Exploration of Time and Death in Role Playing (NeurIPS 2025) (MMLS 2025, Honorable Mention)
Xiaoyan Bai, Ike Peng*, Aditya Singh*, Chenhao Tan
Code |
Blog |
Website
A Mechanistic Understanding of Alignment Algorithms: A Case Study on DPO and Toxicity (ICML 2024, Oral)
Andrew Lee, Xiaoyan Bai, Itamar Pres, Martin Wattenberg, Jonathan K. Kummerfeld, Rada Mihalcea
Code
Blogs
MechEvalAgent: Grounded Evaluation of Research Agents in Mechanistic Interpretability (November, 2025)
Research agents are getting smarter. They can write convincing PhD-level reports. But has anyone checked if the way they find their results makes any sense? Our framework, MechEvalAgents, verifies the science, not just the story 🤖
Concept Incongruence is Key to AI Safety and Creativity (July, 2025)
⚡️Ever asked an LLM-as-Marilyn Monroe about the 2020 election? Our paper calls this concept incongruence, common in AI and human creativity. Read my blog to learn what we found, why it matters for AI safety and creativity, and what's next.
Professional Experience
- Microsoft
- AI Research Intern (2026)
- Emogent
- Data Science Intern (2021)
Service
- Organizer
- NeurIPS 2026 Interpretability for Discovery workshop
- Reviewer
- TMLR; COML 2026; ICML 2026 (Gold Reviewer); ICLR 2026 NeurIPS 2025; ICLR SCOPE and DeLTa workshops 2025; ACL ARR 2024
Teaching
- University of Chicago
- Teaching Assistant, CMSC 25300: Mathematical Foundations of Machine Learning (Fall 2024)
- University of Michigan
- Grader, EECS 376: Foundations of Computer Science (Fall 2023)
- MIT Beaver Works Summer Institute
- Teaching Assistant, Serious Game and AI (Summer 2023)