A Ukraine-led research paper reports that automated deepfake detectors have lost most of their effectiveness against current generative video. On the new DF26 benchmark, temporal detectors that reached an AUROC of 94.3 on the older CelebDF++ set fell to 48.2 — a drop of 49% in roughly a year. For companies that rely on video verification, that is close to a coin toss.
What the DF26 benchmark measured
The work, titled DF26: We Cannot Tell Fake From Real Anymore, was built as a hold-out set for evaluating unseen material. It contains 2,691 videos across three public-speaking scenarios: direct-to-camera or casual addresses, official statements, and studio interviews. The synthetic portion was generated from 271 real clips curated from OpenVid, TalkingCelebs and MAVOS-DD. Matched scene prompts were derived from the frames of those real videos and fed, together with the first frame where the model supported it, into seven video generators, producing 2,420 synthetic clips.
Seven generators were tested. The closed-source commercial models were Grok 1.0, Kling 3.0, Veo 3.1 and Wan 2.6; only text-to-video was produced with them, because image-to-video attempts often triggered deepfake-generation filters or would have violated platform terms. Three open-source models were also trialed: Wan 2.2 A14B, HunyuanVideo 1.5 and LTX 2.3, the last of which can generate speech. Generation ran on the commercial Higgsfield AI platform, with an internal cluster using an NVIDIA H200 with 141GB of VRAM; producing 1,626 videos took about 440 GPU hours.
What this means for business
Detection quality now depends heavily on which generator produced the clip. Performance varied dramatically across models, which suggests detectors were often learning generator-specific traces rather than a general signature of synthetic video. Image-to-video proved harder to detect than text-to-video, for both people and automated systems: PwTF-DVD performed best on I2V material, while GenD-PE led on T2V. The practical consequence is that a detector validated on one vendor's output may not transfer to another's, so a single accuracy figure in a procurement document says little on its own.
The benchmark deliberately excludes shortcuts. No real-world frames with text overlays were allowed into the generation workflows, and candidate clips were checked by Gemini 2.5; segments with more than one face were rejected to enforce a single-speaker setting. AI watermarking was also obfuscated or avoided, since it would give an evaluator an easy signal. The primary metric was Area Under Receiver Operating Characteristic Curve (AUROC), with Equal Error Rate as a complementary measure. Frame-based detectors assessed 32 evenly spaced frames per video and averaged the scores; temporal detectors analyzed the video sequence itself. All detectors in the comparison were trained on the FaceForensics++ dataset.
The gap between old and new benchmarks is the core finding. Most detectors still perform well on Celeb-DF++ (CDFv3), where temporal methods reach AUROC of 94.3 and 92.3, but on DF26 those same methods fall to 48.2 and 61.6. Most approaches degrade substantially and remain near chance; the highest AUROC, 69.7, comes from GenD-PE. The authors state plainly that DF26 is a more challenging benchmark for state-of-the-art detectors. The earlier 2024 finding that human detection sits at 57%, barely above chance, now applies to automated tools as well.
For a small company, the immediate implication is that off-the-shelf deepfake detection should not be treated as a control. A vendor claiming high accuracy on a public benchmark is answering a question about 2022-era manipulation, not about the video a finance team might receive today. Larger organizations with verification workflows — KYC, remote hiring, insurance claims, executive communications — face a harder problem, because the failure is not uniform: it depends on the generator, the scenario and whether the clip is image-to-video or text-to-video. Questions worth putting to any vendor include which generators and which benchmark the claimed accuracy was measured against, and whether the model was tested on material produced after its training cut-off.
The marker to watch is how the DF26 dataset is used. The authors intend it as a hold-out set for evaluating unseen material, so independent results on it — rather than on CelebDF++ — will show whether detection can be rebuilt for diffusion-based video. If AUROC on DF26 stays near chance through the next round of published tests, video-based identity checks should be treated as one signal among several, not as proof.
