Skip to main content

Algorithmic Bias Auditing — Image Generation Benchmarks

Authors: Mousumi, Thomas
Affiliation: University of Redlands
Duration: Oct 2025 – Present

Overview​

State-of-the-art image generation models can unintentionally amplify demographic representation bias. This project builds a reproducible benchmark to measure those gaps and evaluates practical mitigation strategies that can be deployed without changing downstream product requirements.

What We Built​

  • Bias benchmark suite: A standardized evaluation set and scoring protocol to quantify demographic representation across prompts, styles, and contexts.
  • Reporting pipeline: Aggregates metrics into consistent, comparable summaries to track drift across model versions and prompt templates.

Visual Insight​

Qualitative comparison: 'scientist' depictions across AI image generators

This qualitative grid highlights how different generators can systematically shift who gets depicted as a “scientist” under similar prompts, motivating the need for standardized auditing across models and prompt templates.

Gender representation of scientists across AI image generators

This aggregate view summarizes gender representation patterns across generators, providing a concrete example of how demographic skews can be measured, compared, and tracked over time.

Mitigation Strategies​

  • Prompt engineering: Prompt templates designed to reduce skew while preserving semantic intent.
  • Fine-tuning experiments: Controlled adaptation runs to improve representation balance without collapsing diversity.

Why It Matters​

  • Helps teams detect and quantify bias early, before deployment and scale.
  • Provides actionable knobs (prompts and fine-tuning) that can be integrated into existing workflows.
  • Enables transparent comparisons across models and releases.
Back to Research Portfolio