Scaling the UI: Real-Time on the Client

This is Part 2 of Scaling: The UI / Frontend Track, the third series in a pillar on scaling systems.

• Part 1: Getting Bytes to the Client Cheaply: the read path, CDNs, caching, payload shape, and pagination.
• Part 2: Real-Time on the Client: polling, SSE, and websockets, and the cost of holding connections.
• Part 3: The UX of Eventual Consistency: representing an asynchronous backend honestly on screen.
• Part 4: Client Resilience and the Cross-Cut: degradation, retries, offline, auth, and observability.

Part 1 handled data that mostly sits still. Part 2 handles data that is alive. In the food app, once an order is placed the customer wants to watch it happen, order confirmed, restaurant preparing, courier assigned, courier two minutes away. That live status is being produced by the asynchronous backend from the second series, and now it has to reach a screen in near real time. There are several ways to deliver it, they differ sharply in freshness and in cost, and the cost that surprises teams is not the data but the connections. This part is about choosing the right transport and understanding the scaling bill each one brings.

Table of Contents

• The Real-Time Problem
• Polling: Simple and Often Right
• Long-Polling: A Middle Step
• Server-Sent Events: One-Way Streams
• Websockets: Full Duplex
• The Cost Nobody Budgets: Holding Connections
• Fanning Out From the Backend
• Native Callout: Platform Push
• Choosing a Transport
• Conclusion

The Real-Time Problem

The web's default is pull: the client asks, the server answers, and between requests the server has no way to reach the client. Real-time features fight that default, because the new information, the courier just moved, originates on the server, and the server cannot simply speak to a client that has not asked. Every approach below is a different way of working around that one limitation, and each makes a different trade between how fresh the update is, how much load it generates, and how much complexity it adds. Freshness is rarely the hard part. Cost at scale is.

Polling: Simple and Often Right

Polling is the client asking again on a timer: every few seconds, "any update on order 123?" It is the crudest approach and, precisely because it needs nothing beyond ordinary request-response, it is often the correct one.

Its virtue is simplicity. There is no special protocol, no held connection, no new server type; it reuses the entire stateless, cacheable, load-balanced infrastructure from the backend series exactly as it stands. Its cost is waste and lag. Most polls return "no change," so you spend requests to usually learn nothing, and an update can be as stale as your interval, poll every five seconds and news can be five seconds old. But look closely at the food app and much of it does not need better. A five-to-ten-second refresh on order status is completely acceptable to a customer, and polling delivers that with zero new infrastructure. The trap is polling too aggressively across a large user base: a two-second interval times a million waiting orders is half a million requests per second of mostly-nothing, which is real load. Poll at the slowest interval the experience tolerates, and for a great many "real-time" features that interval is generous enough that polling wins on simplicity outright.

Long-Polling: A Middle Step

Long-polling sharpens polling: the client asks, and instead of answering immediately with "no change," the server holds the request open until there actually is an update or a timeout passes, then answers, and the client immediately asks again. The effect is near-instant delivery, the answer comes the moment news exists, without a firehose of empty responses.

It is a clever bridge that needs no special protocol, but it is genuinely a transitional technique. Because the server holds requests open, it inherits the connection-holding cost we are about to examine, without the efficiency of the purpose-built streaming transports that follow. In modern systems long-polling is mostly a fallback for environments where server-sent events or websockets are unavailable. Worth understanding, rarely the first choice today.

Server-Sent Events: One-Way Streams

Server-sent events (SSE) are a purpose-built answer to exactly our problem: the server needs to stream updates to the client, one way. The client opens a single long-lived HTTP connection and the server pushes events down it as they happen, for as long as it stays open.

For the food app's order tracking, SSE fits almost perfectly, because the data flow is one-directional, the server has updates, the client only needs to receive them, and SSE is built for precisely that. It runs over ordinary HTTP, so it works with existing infrastructure more smoothly than websockets, it reconnects automatically when dropped, and it is simpler to operate than a full duplex protocol. Its limitation is the flip side of its design: it is server-to-client only. When the client also needs to send a steady stream upward, SSE alone is not enough. For the common shape of "server streams live updates to a watching client," which is a huge share of real-time features, SSE is often the sweet spot, more efficient than polling, simpler than websockets.

Websockets: Full Duplex

Websockets open a persistent, two-way connection over which either side can send messages at any time, with low overhead once established. They are the most capable real-time transport and the right tool when interaction is genuinely bidirectional and continuous, live chat, collaborative editing, multiplayer, a trading screen taking and streaming orders at once.

That power is not free, and using websockets where a one-way stream would do is a common over-reach. A full-duplex protocol is more complex to operate than SSE or polling: it does not ride ordinary HTTP request-response as naturally, it complicates load balancing because the connection is long-lived and stateful, and it demands its own reconnection and heartbeat handling. For the food app's order tracking, which is one-directional, websockets would be more machinery than the problem needs. Reach for them when you truly have continuous two-way communication; for streaming updates down to a watcher, a lighter transport usually serves better. And whichever you choose, both websockets and SSE run headlong into the cost the next section is about.

The Cost Nobody Budgets: Holding Connections

Here is the point of this whole part, the thing teams consistently miss until it hurts: with polling, the server holds a connection only for the brief moment of each request, but with SSE and websockets, the server holds a connection open for every client, continuously, for as long as they are connected. That changes the scaling problem completely.

Serving a million polling users is a throughput problem, and the stateless, horizontally-scaled backend from the second series handles throughput well. Serving a million users over persistent connections is a concurrency problem, you must hold a million connections open simultaneously, and that is a different and harder thing:

• Memory and file descriptors. Each open connection consumes memory and an operating-system handle on the server. A single machine can hold only so many before it exhausts one or the other, so a million concurrent connections forces a sizable fleet of connection-holding servers whose job is largely just to keep connections alive.
• Load balancing gets harder. Long-lived connections do not spread as cleanly as short requests. A newly added server does not relieve existing connections, because those stay pinned to the servers they were established on until they drop, so scaling out helps new connections but not the current load, and connection draining during deploys becomes its own careful dance.
• Statefulness creeps back in. A held connection is inherently stateful, the association between this user and this specific server, which cuts against the statelessness that made everything else scale. Real-time connection tiers are often a distinct, specialised layer precisely because they violate the assumptions the rest of the system enjoys.

So the decision to use persistent connections is a real architectural commitment with a real bill, not a mere feature toggle. Before choosing SSE or websockets over polling, ask whether the freshness is worth standing up and operating a connection-holding tier at your user count. Often it is; sometimes cheap polling is the wiser scale decision precisely because it sidesteps this entire class of cost.

Fanning Out From the Backend

Holding the connections is only half of it. The other half is getting each update to the right connection, and this is where the frontend rejoins the backend series directly. When the "courier moved" event is produced, the customer watching that order is connected to one particular server among hundreds in the connection tier, and the event has to find its way to exactly that server and no other.

This is a fan-out problem, and it is solved with the same broker thinking from the second series. Updates are published to a messaging or pub/sub layer, and each connection-holding server subscribes to the events for the clients it currently holds, forwarding matching updates down the right connections. The event backbone that decoupled the backend now also bridges from the backend to the exact edge connection that needs each message. It is a clean illustration of the pillar's throughline: the async event infrastructure built for the backend is the same infrastructure that makes client real-time work at scale.

Native Callout: Platform Push

Native mobile apps have an option the web does not, and it changes the scaling math entirely: platform push notifications, through the operating system's push service (APNs on Apple, FCM on Android).

Instead of your servers holding a live connection to every app, you hand the update to the platform's push service and it delivers to the device, even when your app is backgrounded or closed. The scaling consequence is significant: you no longer hold a hundred thousand connections yourself, you fan out to the platform push providers and they carry the last mile and the burden of reaching each device. The whole connection-holding cost from above is largely offloaded to the platform. The tradeoffs are that push is best for discrete, important notifications, your order is on its way, rather than a high-frequency continuous stream like a smoothly moving map dot, and that delivery is best-effort and less immediate than a held connection. So native apps commonly blend the two: platform push for key status changes and to re-engage a backgrounded app, and a held connection like a websocket for high-frequency live updates while the app is open and in the foreground. The native lesson is that you often do not have to hold the connection yourself, and at scale, not holding it is a large saving.

Choosing a Transport

The choice follows directly from the shape of the data flow and the scale, not from which technology is newest.

Start from the questions that actually decide it: Is the flow one-way or two-way? How fresh does it truly need to be? How many concurrent users, and can you afford to hold that many connections? For much of the food app, polling or SSE answers the need without the weight of websockets, and on native, platform push carries the notifications without your servers holding anything. The most common mistake is reaching for the most powerful transport by default and paying its scaling cost for freshness the feature never required.

Conclusion

Real-time on the client is a set of workarounds for the web's pull default, and they differ less in whether they can deliver updates than in what they cost to deliver them at scale. Polling is simple and reuses everything, at the price of waste and lag. SSE streams one-way efficiently; websockets add full duplex and more operational weight. And the cost that dominates the decision is connection-holding: serving many users over persistent connections is a concurrency problem that pulls statefulness back into a system built to avoid it, solved by a specialised connection tier fed by the same broker fan-out as the backend. On native, platform push lets you sidestep much of that cost entirely.

Choosing a transport gets the live data onto the screen. But that data is coming from the eventually-consistent backend of the second series, which means what the screen shows is sometimes ahead of, behind, or briefly at odds with the truth. Part 3 is about representing that honestly, the user experience of eventual consistency, where the whole pillar finally pays off on the client.