Real2Gen — A Frequency-Space Benchmark for Detecting AI-Generated Images

Authors: Thomas, Elizabeth
Focus: Open-set AI-generated image detection under cross-architecture and cross-training regimes

Overview​

Recent breakthroughs in text-to-image generation (diffusion, autoregressive, and hybrid architectures) have dramatically improved photorealism, challenging detectors that rely on spatial artifacts or model-specific signatures. Existing detection datasets often suffer from inconsistent sampling and a lack of verifiable correspondences between real photographs and matched synthetic outputs, limiting systematic study of open-set generalization as generators evolve.

Real2Gen addresses these gaps by introducing a frequency-space benchmark and a continuously expanding dataset designed to evaluate detector robustness under open-set, cross-architecture, and cross-training settings.

Real → Caption → Generate Workflow​

Real2Gen follows a controlled pairing pipeline:

• Real: Start from real photographs.
• Caption: Produce semantically grounded textual descriptions via captioning.
• Generate: Reuse the same captions to synthesize aligned images across a diverse pool of generators (diffusion, autoregressive, and proprietary pipelines).

This enforces semantic equivalence between each real image and its synthetic counterpart, enabling controlled comparisons that better isolate generative artifacts from content variation.

Dataset Preview (Aligned Pairs)​

This preview shows semantically aligned real photographs alongside synthetic outputs created from the same caption. The pairing reduces content confounds and makes it easier to attribute differences to generation artifacts rather than prompt mismatch.

Frequency-Space Evaluation Protocol​

In addition to paired sampling, Real2Gen proposes a frequency-space evaluation protocol that probes detector robustness through spectral characteristics that can persist even when spatial textures become difficult to distinguish.

Frequency-Space Signals (What Persists Across Generators)​

Radially averaged FFT energy reveals systematic shifts between real and synthetic images across frequency bins, suggesting detector cues that remain informative even when spatial textures look similar.

Comparing average radial frequency profiles across multiple generators highlights stable spectral discrepancies and motivates cross-generator evaluation rather than single-model benchmarking.

Key Findings (High Level)​

• Widely used detectors can degrade substantially when confronted with unseen generators.
• Frequency-domain discrepancies between real and synthetic imagery can be more stable and transferable across generator families than purely spatial cues.

Model Behavior (Separation and Confidence)​

PCA of fused feature embeddings shows that real samples and synthetic samples from different generators occupy distinct regions, supporting the use of frequency-aware representations for open-set generalization analysis.

An alternative projection (PC1 vs PC3) provides another view of how generator families cluster in feature space, revealing structure that can be exploited for robustness evaluation.

The pooled posterior distribution illustrates how confidently a detector separates real and synthetic samples, and can be used to study calibration shifts under unseen generators.

Training and validation accuracy curves provide a quick sanity check for optimization stability and help diagnose potential overfitting when transferring to new generators.

Scale and Roadmap​

The current release comprises 100K aligned real–generated pairs with curated train/validation/test splits. The benchmark is growing toward one million samples to support large-scale training, longitudinal analysis, and reproducible evaluation as generative technology continues to evolve.