“One million viewers” is four workloads

This continues the control plane / data plane split. The next question is what actually happens when a very large audience watches the same live event. My first instinct was "ten million times normal load". That instinct is wrong, and working out why was the most useful part of this exercise.

Load is not one number

The four lanes have different bottlenecks, different scaling tools and different performance requirements. Lumping them into "1M users" hides all the interesting engineering.

1. Control traffic: the boring one

Small, cheap, mostly stateless requests behind a load balancer. Add API servers and the load spreads across them. A cache in front of the database serves repeated reads such as the home page, catalog metadata and profiles. This is a well-understood scaling problem, and it is not where the risk is.

2. Playback traffic: where the numbers get large

Back-of-envelope, assuming a player requests a new segment roughly every 4 seconds:

No origin server or database survives that without help, so almost none of these requests should reach the origin.

For a popular title or a live match, everyone requests the same few segments at the same moment, so cache locality is extremely high. That is exactly what makes edge caching so effective. Popular segments sit at edges close to viewers, which improves latency and throughput and protects the origin: three benefits from one decision.

The live-streaming wrinkle

On-demand segments are immutable once generated, so they can be cached for a very long time. Live segments are produced continuously. Cache too aggressively and viewers see stale data. Cache too little and the origin is hit by everyone asking for "the latest segment" at once. For live content, freshness and cache effectiveness pull in opposite directions, and there is no universal answer, only a chosen trade-off.

3. Event traffic: never write it synchronously

Every pause, seek, skip and finished episode is a signal for analytics and recommendations. At scale none of it should be written synchronously to the primary database.

Decoupling ingestion (fast, append-only) from processing (batch or near real time) means a slow analytics job can never become the reason playback requests start failing.

4. Content processing: independent of audience size

Transcoding, segmentation and manifest generation run asynchronously through queues and worker pools. They do not depend on how many people are watching, so they get their own capacity plan and their own failure handling: retries, idempotent jobs, job status.

Same pattern, different layers

A cache in front of the database for hot application data and a CDN in front of object storage for video are the same pattern applied at two layers: put a cheap, fast, replaceable layer in front of the expensive, authoritative one.

Takeaway

Still open

The hardest case is the one this post only names: everyone pressing play in the same few minutes, the thundering-herd problem for live events. I have not worked it through yet. Cache pre-warming, request coalescing and origin capacity planning are the topics I would start with.