All Scenarios
mediumSAADVAArchitecture

Decoupled Image Processing Pipeline

Users upload photos to your site. Each photo needs: thumbnail generation, content moderation, EXIF extraction, and ML tagging. Upload API must respond in under 1 second.

Key Constraints

Upload API response < 1s
4 independent processing steps per photo
Handle 10× spikes (marketing campaigns)
Failed steps must retry independently

Reference Architecture (interactive 3D)

🖱️ Drag to rotate · 📜 Scroll to zoom

Loading diagram...

SNS Fan-out → 4 SQS Queues → 4 Lambdas

  1. 1API Gateway returns a pre-signed S3 URL (response time < 1s — no payload through API).
  2. 2Client uploads directly to S3 — server never sees the bytes.
  3. 3S3 ObjectCreated event → SNS topic.
  4. 4SNS fans out to 4 SQS queues (one per processing step).
  5. 5Each Lambda consumer scales independently and retries on failure (DLQ after 3 attempts).
  6. 6Spike handling: queues absorb the burst; Lambdas scale 0 → 1000 concurrency automatically.
  7. 7All four write results to DynamoDB keyed by photoId — caller polls or gets a webhook when complete.
Heuristics Applied
✓ Fan-out = SNS + multiple SQS✓ SQS between producers and consumers

Common Traps (Wrong Answers)

  • S3 → Lambda direct (no retry semantics, no DLQ — failures vanish)
  • Synchronous processing in the upload API (breaks the <1s SLA)
  • One queue for all 4 steps (can't scale or fail independently)
  • Skipping the SNS layer between S3 and queues (S3 → Lambda has only 2 retries)

Try the simulator

Build this architecture yourself in the drag-and-drop simulator.

Open Simulator →More Scenarios