Skip to main content

Generalization of AI-Generated Image Detectors — A Face-Centric Study

Authors: Thomas, Elizabeth, Fengchen
Focus: Open-set generalization of AI-generated image detection

Overview​

Headline accuracy numbers for AI-image detectors are often illusory: a model that scores near-perfect on its own test set can degrade sharply under cross-data and corruption shifts. This study measures that gap and shows that narrowing the domain to human faces restores strong, transferable detection.

Aligned Real / Synthetic Pipeline​

We start from real COCO photos, caption them with a VLM, neutralize the prompts, then regenerate semantically matched synthetic images across multiple generators (GPT-Image-1, SD 3.5, FLUX.1). Real and synthetic share the same content and differ only in generative origin.

Pipeline: real photo to aligned prompt to multi-generator synthesis and fused detector Semantically aligned real and synthetic image pairs

The Generalization Gap​

A generic detector reaches 0.986 AUROC in-distribution but falls to a worst-group 0.526 under cross-data and corruption shifts (JPEG, noise, blur) — close to random.

In-distribution vs shifted-condition AUROC for a generic detector

Faces Generalize​

Restricting the task to faces, the detector holds up out-of-distribution: 0.997 in-distribution and 0.937–0.979 across held-out generators (GPT-Image, Midjourney) and heavy JPEG laundering.

Face detector AUROC remains high across out-of-distribution generators

Key Findings​

  • Single-dataset accuracy overstates real-world robustness; open-set evaluation is essential.
  • Generic detectors collapse under unseen generators and common corruptions.
  • A face-focused detector retains strong, transferable performance, pointing to domain specialization as a practical path to reliable AI-image forensics.

Publication​

This work is presented at SIGGRAPH 2026 (Poster).

Back to Research Portfolio