Papers

Evaluating AI Systems: Validity and Reliability

• Measurement to Meaning: A Validity-Centered Framework for AI Evaluation
  Olawale Salaudeen, Anka Reuel, Ahmed Ahmed, Suhana Bedi, Zachary Robertson, Sudharsan Sundar, Ben Domingue, Angelina Wang, Sanmi Koyejo
  Working Paper
  [arXiv] [webpage]
  
  

• Toward an Evaluation Science for Generative AI Systems
  Laura Weidinger, Inioluwa Deborah Raji, Hanna Wallach, Margaret Mitchell, Angelina Wang, Olawale Salaudeen, Rishi Bommasani,  Deep Ganguli,  Sanmi Koyejo,  William Isaac
  In The Bridge 2025, National Academy of Engineering
  [arXiv]
  
  

• ImageNot: A Contrast with ImageNet Preserves Model Rankings
  Olawale Salaudeen, Moritz hardt
  In Review
  [arXiv] [code] [webpage]

Robustness-Distribution Shifts: Evaluation, Generalization, and Adaptation

• Are Domain Generalization Benchmarks with Accuracy on the Line Misspecified?
  Olawale Salaudeen, Nicole Chiou, Shiny Weng, Sanmi Koyejo
  In Review
  [arXiv] [code] [webpage] [news]
  
  

• On Domain Generalization Datasets as Proxy Benchmarks for Causal Representation Learning
  Olawale Salaudeen, Nicole Chiou, Sanmi Koyejo
  In NeurIPS 2024 Causal Representation Learning Workshop (Oral)
  [paper]
  
  

• Causally Inspired Regularization Enables Domain General Representations
  Olawale Salaudeen, Oluwasanmi Koyejo
  In AISTATS 2024
  [arXiv] [code] [webpage]
  
  

• Exploiting Causal Chains for Domain Generalization
  Olawale Salaudeen, Oluwasanmi Koyejo
  In NeurIPS 2021 DistShift Workshop
  [paper]
  
  

• Proxy Methods for Domain Generalization
  Katherine Tsai, Stephen R. Pfohl, O. Salaudeen, Nicole Chiou, Matt J. Kusner, Alexander D’Amour, Sanmi Koyejo, Arthur Gretton.
  In AISTATS 2024
  [arXiv] [code]
  
  

• Adapting to Latent Subgroup Shifts via Concepts and Proxies
  α–β. Ibrahim Alabdulmohsin*, Nicole Chiou*, Alexander D’Amour*, Arthur Gretton*, Sanmi Koyejo*, Matt J. Kusner*, Stephen R. Pfohl*, Olawale Salaudeen*, Jessica Schrouff*, Katherine Tsai*.
  In AISTATS 2023
  [arXiv] [code] [webpage]
  
  

• Adapting to Shifts in Latent Confounders using Observed Concepts and Proxies
  Matt J. Kusner, Ibrahim Alabdulmohsin, Stephen Pfohl, Olawale Salaudeen, Arthur Gretton, Sanmi Koyejo, Jessica Schrouff, Alexander D’Amour
  In ICML 2022 PODS Workshop
  [paper]
  
  

Algorithmic Fairness

• What’s in a Query: Polarity-Aware Distribution-Based Fair Ranking
  Aparna Balagopalan, Kai Wang, Olawale Salaudeen, Asia Biega, Marzyeh Ghassemi
  In WWW 2025
  [arXiv] [code]
  
  

• Understanding Subgroup Performance Differences of Fair Predictors using Causal Models
  Stephen Robert Pfohl, Natalie Harris, Chirag Nagpal, David Madras, Vishwali Mhasawade, Olawale Salaudeen, Katherine A Heller, Sanmi Koyejo, Alexander Nicholas D'Amour
  In NeurIPS 2023 Workshop on Distribution Shifts: New Frontiers with Foundation Models
  [paper]
  
  

• Addressing Observational Biases in Algorithmic Fairness Assessments
  Chirag Nagpal, Olawale Salaudeen, Sanmi Koyejo, Stephen Pfohl
  NeurIPS 2022 AFCP Workshop (extended abstract)
  [poster]

Neuroscience and Neuroimaging

• Enhancing fMRI Motion Denoising with ICA-AROMA and Causal Discovery
  Olawale Salaudeen, Paul Camacho, Aron Barbey, Brad Sutton, Sanmi Koyejo
  In Review
  [code]
  
  

• Ultra-fast 3D fMRI to Explore Cardiac-Induced Fluctuations in BOLD-Based Functional Imaging
  Brad Sutton, Aaron Anderson, Benjamin Zimmerman, Paul Camacho, Riwei Jin, Charles Marchini, Olawale Salaudeen, Natalie Ramsy, Davide Boido, Serge Charpak, Andrew Webb, Luisa Ciobanu
  International Society for Magnetic Resonance in Medicine (ISMRM), 2022. (abstract).
  [link]