← All projects

Amazon · 2022–2023 · design & build

One-click take-down: cascading removal across every copy, cache and collection.

A state-driven deletion engine that removes a compromised, copyrighted or legally sensitive image and everything derived from it, from database records to edge caches, then blocks it from being uploaded again.

per take-down, down from 10–15 min of manual tracing
< 1 s
take-down requests per week, North America alone
~8,000
orphaned originals purged after 2+ years
~20M
business goal unblocked
$3.6B

01 / architecture

How it works

Take-down workflow: six steps, a volume-aware branch for removing links, and seven layers torn down STATE-DRIVEN WORKFLOW · 6 STEPS Validateid + collection Block checksumno re-upload Discover linksall tenants Remove linksvolume-aware Destroy + cascadevariants, originals Invalidate CDNedge caches no links → skip ahead BRANCH BY VOLUME No linksskip ahead A fewinline, resumable Manyparallel batch jobs Job tracker≤ 20 polls, report Dead-letter bucketfailed deletes, audited DB recordmetadata row Stored objectarchived first Link mapsall tenants Variantsdisplay, resized Raw originalsuploaded files Localizedtranslations CDN cacheedge locations REMOVED FROM EVERY LAYER 1 2 3 4 5 6
  1. 1Block before deleteThe file's checksum is blocklisted first, so banned media can't be uploaded again while the teardown runs, or after it finishes.
  2. 2Cross-tenant discoveryEvery reference to the image is found across customer collections and snapshotted to storage before anything is deleted, for audit and resumability.
  3. 3Volume-aware branchingNo links: skip ahead. A few: remove them inline with a resumable index. Many: fan out to parallel batch deletion jobs.
  4. 4Bounded tracking, dead-letter logA tracker polls batch jobs up to 20 times and reports what failed. Every failed deletion lands in a dead-letter bucket for audit and replay.
  5. 5Cascade to every derivativeRelated variants are found and destroyed, the workflow confirms they're all gone, and only then removes the stored originals. Translated copies are purged automatically too.
  6. 6Edge to storage, rolled out safelyThe run ends with CDN invalidation. Because it handles live traffic, the new workflow was dialled up gradually, with a metric and an alarm for every failure scenario.

02 / trade-offs

Trade-offs, measured

Time to complete one take-down

Seconds, log scale

0.1 s 1 s 10 s 100 s 1,000 s Manualtrace + purge Automatedworkflow 10–15 min < 1 s · 600×+ faster

The manual process also depended on an operator finding every related file. Missing one left a legally sensitive file live.

Backfill: ~20M orphaned originals

Originals left behind by take-downs from before the cascade existed, some over 2 years old

  1. Partitioned SQLfind candidates in the data warehouse
  2. Verify liveconfirm through the live API that no active outputs remain
  3. Archivecopy before delete, for recovery
  4. Purgemultithreaded, throttled to protect live traffic

The files couldn't be fetched through any existing API, so they had to be found in the data warehouse. Each was then checked against the live system before deletion. The purge ran as a tracked change with monitoring throughout.

03 / impact

Impact

take-downs go from a 10–15 minute manual trace to under a second, with no missed files
600×+ faster
take-down requests handled automatically in North America alone
~8,000 / wk
orphaned originals purged, closing a compliance risk that had been open for 2+ years
~20M

What it unlocked