The Video Call That Cost $25 Million
In early 2024, a finance employee at the engineering firm Arup joined a video conference with his company's CFO and several colleagues. He recognized their faces. He recognized their voices. Following their instructions, he authorized 15 transfers totaling $25 million to accounts in Hong Kong. There was just one problem: every single person on that call was a deepfake, as CNN reported when the victim was identified.
If you want to understand why learning to spot AI-generated content has become an essential modern skill, that story is the place to start. Deepfakes — synthetic photos, videos, and audio created or manipulated by AI — have graduated from internet curiosity to a tool used in real fraud, real political manipulation, and real harassment. The good news? You're not helpless. With a little knowledge of how this technology works and where it still stumbles, you can dramatically improve your odds of catching a fake.
What Exactly Is a Deepfake, and How Are They Made?
The term "deepfake" is a mashup of "deep learning" and "fake," and it covers a family of techniques rather than a single technology. The earliest deepfakes, circa 2017, used generative adversarial networks — two neural networks locked in a contest where one generates fake images and the other tries to detect them, each improving the other through millions of rounds. Today's tools have largely moved to diffusion models, the same architecture behind popular image generators, which build images from noise guided by training data.
In practice, there are three main flavors you'll encounter:
- Face swaps and face reenactment: replacing one person's face with another's in video, or puppeteering a real face to say and do new things
- Voice cloning: synthesizing someone's voice from a sample — modern systems need as little as 20 to 30 seconds of audio, which is less than most people post in a single Instagram story
- Fully synthetic media: people, scenes, and events generated from scratch by text-to-image and text-to-video models
What changed recently isn't the underlying science — it's accessibility. Creating a convincing deepfake once required serious technical skill and GPU time. Now consumer apps and open-source tools do in minutes what used to take days. The same generative models powering the legitimate creative tools we compared in our AI art generator showdown can be pointed at deception just as easily as at art.
Deepfakes in the Wild: Real Incidents, Real Stakes
The Arup case is the most famous corporate example, but it's far from isolated. The World Economic Forum's analysis of the attack treats it as a preview of a much broader trend: deepfake incidents have increased by more than 1,500% since 2023, and industry analysts project that AI-enabled fraud could drive $40 billion in losses by 2027.
The pattern repeats at every scale. Scammers clone a grandchild's voice from social media clips and call grandparents with fake emergencies. Fraudsters impersonate executives to rush wire transfers past skeptical employees. Fake celebrity endorsements push investment schemes across social platforms. During election seasons around the world, fabricated audio of candidates has surfaced days before voters went to the polls — timed so debunking arrives too late.
It's worth keeping perspective, though. The overwhelming majority of video you see is still authentic, and most deepfakes in circulation are low-effort fakes that fall apart under modest scrutiny. The goal isn't to distrust everything; it's to apply skepticism where the stakes and the incentives warrant it. Awareness, not paranoia.
How to Spot AI-Generated Content: The Visual Tells
Despite rapid progress, AI-generated images and video still make characteristic mistakes. Researchers at MIT's Media Lab have catalogued where deepfakes tend to fail, and their advice boils down to looking closely at the things generative models find hardest to get right.
Here's a practical checklist:
- Skin and texture: Skin that looks airbrushed-smooth or oddly waxy, especially on cheeks and foreheads, or skin whose apparent age doesn't match the hair and eyes
- Eyes and blinking: Blinking that's too frequent, too rare, or absent entirely; reflections in the eyes that don't match the scene's lighting
- Lighting and shadows: Shadows falling in directions that don't make physical sense, or glasses with glare that doesn't shift naturally as the head moves
- Edges and accessories: Flickering or warping where the face meets hair, ears, jewelry, or eyeglasses — boundary zones are hard for face-swap models
- Lip sync: Mouth movements that drift slightly out of step with the audio, particularly on sounds like "b," "m," and "p"
- Hands and backgrounds: Extra or contorted fingers, garbled text on signs, and a telltale "shimmering" in complex backgrounds like water, foliage, or crowds
For still images, slow down and zoom in. For video, watch a suspicious clip more than once — first for content, then specifically for these artifacts. One honest caveat: the best fakes now evade casual inspection, and even automated detectors have struggled as generation quality improves. Visual tells are your first filter, not your final verdict.
Audio Deepfakes: Trust Your Ears, Then Verify
Voice clones may be the most dangerous deepfakes of all, because audio gives you fewer signals to check and phone calls create urgency by design. Still, synthetic voices have tells: unnaturally consistent pacing, flat or slightly "off" emotional inflection, missing breath sounds, and an absence of the normal background texture of a real environment.
The stronger defense is procedural rather than perceptual. If you get an urgent call or voice note from a family member asking for money, hang up and call them back on the number you already have. Many families now use a shared code word for genuine emergencies — a delightfully low-tech answer to a high-tech problem. In the workplace, the lesson of the Arup case is that no payment or credential request should ever be approved on the strength of a call or video meeting alone; verify through a second, independent channel. Boring? Absolutely. Effective? Also absolutely.
Why Spotting AI-Generated Content Is Getting Harder
A fair question at this point: if there's a checklist, why is this still a problem? Because the checklist has a shelf life. Every artifact researchers publicize becomes a target for the next model version to fix. Hands were a famous giveaway in 2023; today's leading generators usually render them correctly. Blinking was once a reliable tell; modern face models blink convincingly. Detection and generation are locked in the same adversarial dance that created deepfakes in the first place.
The numbers bear this out. Automated detection systems that caught the overwhelming majority of fakes a few years ago have seen their real-world accuracy slide significantly as generators improved and bad actors learned to launder content through compression, cropping, and re-recording — tricks that destroy the subtle statistical fingerprints detectors rely on. Social platforms now deploy detection at upload time, and AI labs embed invisible watermarks in their outputs, but neither is a complete answer.
This is why the smart long-term strategy shifts from "detect the fake" to "verify the real" — which brings us to the most promising development in this whole space.
Verification Tools: Content Credentials and C2PA
Spotting artifacts is a stopgap. The longer-term answer is provenance — cryptographically verifiable records of where a piece of media came from and how it was edited. The leading standard here is C2PA (the Coalition for Content Provenance and Authenticity), backed by Adobe, Microsoft, Google, OpenAI, camera makers, and major news organizations. Its consumer-facing label is called Content Credentials, and you can inspect any file that carries them at the official Content Credentials verify site.
Adoption crossed an important threshold over the past year. Adobe's Photoshop, Lightroom, and Firefly attach credentials to exported work; OpenAI's image generators embed them automatically; Samsung's Galaxy S25 brought C2PA tagging to smartphone photos; and camera makers like Nikon and Sony have shipped in-camera signing for professional gear. Even the U.S. National Security Agency published guidance in January 2025 recommending Content Credentials for multimedia integrity.
Two important caveats. First, provenance tells you about origin, not truth — a C2PA manifest can verify that an image came from an AI generator, but a real photo can still be framed misleadingly. Second, absence of credentials proves nothing, since most media today still lacks them and metadata can be stripped. Meanwhile, classic verification still works: reverse image search, checking whether reputable outlets are reporting the same event, and tools like InVID for analyzing video frames.
How to Protect Yourself From Becoming the Raw Material
Detection is half the battle; the other half is limiting what attackers can build from your digital footprint. A voice clone needs source audio. A face swap needs photos. You almost certainly provide both, publicly, every week.
A few sensible precautions:
- Audit your public audio and video. You don't need to go dark — just be aware that public clips are training data for anyone, and consider tightening privacy settings on accounts you don't use professionally.
- Establish verification rituals. Code words with family, callback policies at work, and a personal rule that urgency plus money plus a voice or video request always equals a second check.
- Harden your accounts. Deepfakes are frequently a component of broader identity attacks, so strong authentication matters more than ever — we dug into the strengths and limits of face and voice authentication in Biometrics: Keep Your Identity Secure.
- Know your recourse. If you or someone you know is targeted by intimate-image deepfakes, platforms in the U.S. are now legally required to remove them quickly — more on that below.
The Law Is Catching Up — Slowly
Regulation has finally entered the chat. In May 2025, the U.S. enacted the TAKE IT DOWN Act, the first federal law directly targeting harmful deepfakes; it criminalizes non-consensual intimate imagery, including AI-generated imagery, and requires platforms to remove it within 48 hours of a valid request. In Europe, the EU AI Act's Article 50 transparency rules require deepfakes to be clearly labeled as artificially generated or manipulated, with serious violations exposing companies to fines of up to 6% of global turnover.
None of this makes deepfakes disappear — laws deter the scrupulous and punish the caught, while the most damaging actors operate across borders and outside platforms' reach. But labeling requirements, takedown mechanisms, and provenance standards together shift the ecosystem toward a default where authentic content can prove itself. For a deeper look at how governments are wrestling with these trade-offs, see AI Ethics and Regulation: Taming the Algorithms.
The Bottom Line
Deepfakes are a genuine problem that is genuinely manageable. The technology will keep improving, and the era when you could reliably eyeball a fake is ending — but the era of verifiable media is beginning at the same time, with provenance standards, platform labeling, and new legal recourse all maturing fast.
Your best defense costs nothing: slow down before you share, verify before you act, and treat urgency as a signal to check rather than comply. The Arup employee wasn't foolish — he was rushed, isolated, and confronted with faces he trusted. Build the habit of one extra verification step, and you've defeated the exact mechanism that makes deepfakes dangerous. In a world where seeing is no longer believing, a healthy pause is the new superpower.
