Visiting Scientist (AI Institute Postdoctoral Fellowship), Schmidt Sciences AI Institute
Postdoctoral Associate, Massachusetts Institute of Technology (w/ Prof. Marzyeh Ghassemi)
Postdoctoral Scholar, Eric and Wendy Schmidt Center at the Broad Institute of MIT and Harvard (w/ Prof. Caroline Uhler)
Embedded Files
Education
Education
Ph.D. in Computer Science, 2024. University of Illinois at Urbana-Champaign (w/ Prof. Sanmi Koyejo)
Visiting PhD Student in Computer Science, 2022-2024. Stanford University (w/ Prof. Sanmi Koyejo)
B.S. with honors in Mechanical Engineering with minors in Mathematics and Computer Science, 2019. Texas A&M University
On the 2025–26 academic job market, seeking tenure-track positions beginning Fall 2026.
olawale[at]mit[dot]edu
Please see CV for more info.
I am broadly interested in reliable AI and am generally working on the science of measuring AI capabilities and intervening on AI behavior — bridging theory, algorithms, and real-world impact. I develop the principles and practices of reliable AI evaluation. This includes studying the external validity of key benchmarks (ImageNet) in deep learning, the internal validity of benchmarks for out-of-distribution generalization, and frameworks for valid evaluation of latent AI capabilities and traits. I also develop methods to understand and intervene in mechanisms, such as causal versus spurious, that determine AI behavior. My work enables AI systems to generalize and adapt to new environments that differ from their training data, ensuring that AI systems are reliable and safe in dynamic, real-world settings. Application areas of my work include health and medicine, algorithmic fairness, and AI policy.
My research has been supported by a Sloan Scholarship, Beckman Graduate Research Fellowship, GEM Associate Fellowship, NSF Miniature Brain Machinery Traineeship. Additionally, I have interned at Sandia National Laboratories (w/ Dr. Eric Goodman), Google Brain (now Google DeepMind) (w/ Dr. Alex D’Amour), Cruise LLC, and the Max Planck Institute for Intelligent Systems (w/ Dr. Moritz Hardt).
Research Themes
Research Themes
See Publications for more. * denotes equal contribution. α-β denotes alphabetical order.
Valid Measurement of AI Capabilities: AI evaluation is the science of linking measurements, such as benchmark scores, to the real-world capabilities they are meant to represent. My work develops validity-centered analyses of current practices, laying the foundation for more rigorous, scientifically grounded evaluation methods.
ImageNot: A contrast with ImageNet preserves model rankings
Olawale Salaudeen, Moritz Hardt
In review, 2025
[arXiv] [code] [webpage]Measurement to Meaning: A Validity-Centered Framework for AI Evaluation
Olawale Salaudeen*, Anka Reuel*, Ahmed Ahmed, Suhana Bedi, Zachary Robertson, Sudharsan Sundar, Ben Domingue, Angelina Wang, Sanmi Koyejo
Working Paper
[arXiv] [webpage]Toward an Evaluation Science for Generative AI Systems
Laura Weidinger, Inioluwa Deborah Raji, Hanna Wallach, Margaret Mitchell, Angelina Wang, Olawale Salaudeen, Rishi Bommasani, Deep Ganguli, Sanmi Koyejo, William Isaac
In The Bridge 2025, National Academy of Engineering
[arXiv]
Intervening on AI Behavior Mechanisms. AI models often rely on spurious correlations, latching onto easy but misleading signals for decision-making. I design methods that help models focus on the stable, causal patterns instead, so they behave more reliably when conditions change.
Are Domain Generalization Benchmarks with Accuracy on the Line Misspecified?
A version (On Domain Generalization Datasets as Proxy Benchmarks for Causal Representation Learning) was presented as an Oral Presentation at the NeurIPS 2024 Causal Representation Learning Workshop
Olawale Salaudeen, Nicole Chiou, Shiny Weng, Oluwasanmi Koyejo
In TMLR 2025
[arXiv] [code] [webpage] [news]Aggregation Hides OOD Generalization Failures from Spurious Correlations
Olawale Salaudeen, Haoran Zhang, Kumail Alhamoud, Sara Beery, Marzyeh Ghassemi
In review, 2025 (Preprint Incoming)Causally Inspired Regularization Enables Domain General Representations
Olawale Salaudeen, Oluwasanmi Koyejo
In AISTATS 2024
[arXiv] [code] [webpage]
Steering AI Behavior at Inference Time. AI behaviors often become unreliable when they encounter new environments, but they can adapt if provided with the right cues. My work develops methods that utilize context available at inference time to adjust model behavior on the fly, ensuring systems remain reliable and safe when conditions change.
Adapting to Latent Subgroup Shifts via Concepts and Proxies
α–β. Ibrahim Alabdulmohsin*, Nicole Chiou*, Alexander D’Amour*, Arthur Gretton*, Sanmi Koyejo*, Matt J. Kusner*, Stephen R. Pfohl*, Olawale Salaudeen*, Jessica Schrouff*, Katherine Tsai*.
In AISTATS 2023
[arXiv] [code] [webpage]Proxy Methods for Domain Generalization
Katherine Tsai, Stephen R. Pfohl, O. Salaudeen, Nicole Chiou, Matt J. Kusner, Alexander D’Amour, Sanmi Koyejo, Arthur Gretton.
In AISTATS 2024
[arXiv] [code]
Selected Recent News
Selected Recent News
Fall 2025. I am co-organizing the [workshop] on The Science of Benchmarking and Evaluating AI at EurIPS 25 in Copenhagen, Denmark with Moritz Hardt and Joaquiin Vanschoren!
Summer 2025. Our [paper] on the limitations of domain generalization benchmarks and solutions – Are Domain Generalization Benchmarks with Accuracy on the Line Misspecified? – is accepted at TMLR!
Summer 2025. Our [preprint] on the limitations of evaluating AI systems with tests carefully designed for human populations – Stop Evaluating AI with Human Tests, Develop Principled, AI-specific Tests instead – is now available on arXiv!
Summer 2025. Our [preprint] on interpreting disaggregated evaluations of algorithm fairness – Understanding challenges to the interpretation of disaggregated evaluations of algorithmic fairness – is now available on arXiv!
Summer 2025. [service]. I am serving as a program chair for the Machine Learning for Health (ML4H) conference in San Diego, CA, in December. Please reach out if you are interested in sponsoring this great conference!
Summer 2025. [honors/appointment]. I will spend the next year at Schmidt Sciences in NYC as a Visiting Scientist (previously titled AI Institute Fellow) starting this summer! Please reach out if you are in NYC!
Spring 2025. [honors/appointment]. I joined the Eric and Wendy Schmidt Center, led by Prof. Caroline Uhler at the Broad Institute of MIT and Harvard, as a postdoctoral scholar.
Spring 2025. Our [paper] Toward an Evaluation Science for Generative AI Systems appeared in The Bridge's (National Academy of Engineering) latest edition on "AI Promises & Risks."
Spring 2025. I gave a [talk] on addressing distribution shifts with varying levels of deployment distribution information at the MIT LIDS Postdoc NEXUS meeting!
Winter 2025. [service]. I am co-organizing the new AI for Society seminar at MIT.
Winter 2025. Our [paper] titled What’s in a Query: Examining Distribution-based Amortized Fair Ranking will appear at the International World Wide Web Conference (WWW), 2025.
Winter 2025. I was selected as an NYU Tandon Faculty First-Look Fellow; I look forward to visiting and giving a [honors/talk] on our work on distribution shifts at NYU in February; news!
Winter 2025. [service]. I am co-organizing the 30th Annual Sanjoy K. Mitter LIDS Student Conference at MIT.
Winter 2025. I was selected as a Georgia Tech FOCUS Fellow; I look forward to visiting and giving a [honors/talk] on our work on distribution shifts at Georgia Tech in January!
Fall 2024. Our [paper] titled On Domain Generalization Datasets as Proxy Benchmarks for Causal Representation Learning will appear at the Neurips 2024 workshop on causal representation learning as an Oral Presentation.
Fall 2024. [appointment]. I joined the Healthy ML Lab, led by Prof. Marzyeh Ghassemi, at MIT as a postdoctoral associate!
Older News
Older News
Spring 2025. Our [preprint] on domain generalization benchmarks – Are Domain Generalization Benchmarks with Accuracy on the Line Misspecified? – is now available on arXiv!
Summer 2024. I gave a talk on our work on distribution shift at Texas State's Computer Science seminar.
Summer 2024. I gave a [talk] on our work on distribution shift at UT Austin's Institute for Foundations of Machine Learning (IFML).
Summer 2024. I successfully defended my PhD dissertation titled “Towards External Valid Machine Learning: A Spurious Correlations Perspective”!
Spring 2024. I gave a [talk] on AI for critical systems at the MobiliT.AI forum (May 28-29)!
Spring 2024. I gave a [talk] at UIUC Machine Learning Seminar on our work on the external validity of ImageNet; artifacts here!
Spring 2024. Our [preprint] demonstrating the external validity of ImageNet model/architecture rankings – ImageNot: A contrast with ImageNet preserves model ranking – is now available on arXiv!
Winter 2024. Two [papers] on machine learning under distribution shift will appear at AISTATS 2024 (see Publications)!
Winter 2024. I have returned to Stanford from MPI!
Fall 2023. I will join the Social Foundations of Computation department at the Max Planck Institute for Intelligent Systems in Tübingen, Germany this fall as a Research Intern working with Dr. Moritz Hardt!
Spring 2023. I passed my PhD Preliminary Exam!
Spring 2023. I will join Cruise LLC's Autonomous Vehicles Behaviors team in San Francisco, CA this summer as a Machine Learning Intern!
Fall 2022. I have moved to Stanford University as a "student of new faculty (SNF)" with Professor Sanmi Koyejo!
Summer 2022. I am honored to be selected as a top reviewer (10%) of ICML 2022!
Summer 2022. I will join Google Brain (now Google Deepmind) in Cambridge, MA this summer as a Research Intern!
Spring 2021. I gave a [talk] on leveraging causal discovery for fMRI denoising at the Beckman Institute Graduate Student Seminar!
Fall 2021. Our [paper] titled Exploiting Causal Chains for Domain Generalization was accepted at the 2021 NeurIPS Workshop on Distribution Shift!
Fall 2021. I was selected as a Miniature Brain Machinery (MBM) NSF Research Trainee!
Summer 2021. I was selected to receive an Illinois GEM Associate Fellowship!
Spring 2021. I passed my Ph.D. qualifying exam!
Spring 2020. I was selected to receive a 2020 Beckman Institute Graduate Fellowship!
Mentorship
Mentorship
I am happy to mentor students with overlapping research interests. Particularly for undergrads at MIT, programs like UROP are a great mechanism for mentorship.
More generally, I am very happy and available to give advice and feedback on applying to and navigating both undergraduate and graduate programs in computer science and related disciplines – especially for those to whom this type of feedback and guidance would be otherwise unavailable.
Page updated
Google Sites
Report abuse