What Is a Load Balancer? Roles and Risks

A load balancer accepts client connections on a listener, chooses a healthy backend according to routing rules, and forwards the traffic. It can spread demand and remove failed targets from rotation, but it does not repair application bugs, replicate data, or make an undersized system automatically resilient.

Follow one request through the load balancer

The public address of a service usually points clients toward the load-balancing tier. A listener waits on a defined protocol and port. It may accept a connection directly, terminate TLS, or pass encrypted traffic onward, depending on the design.

A listener sends matching traffic to a backend, sometimes called a pool or target group. The backend is a set of eligible servers, containers, or endpoints. A rule and selection algorithm decide which member receives a new request or connection.

The practical path is:

• Accept: the listener receives traffic for a published address, protocol, and port.
• Classify: a Layer 7 proxy may inspect the host, path, method, or another HTTP property.
• Select: the algorithm chooses from targets currently considered eligible.
• Forward: the balancer creates or reuses an upstream connection, or passes transport traffic to the selected target.
• Observe: health checks and live errors can change which targets remain eligible.

Frontend and backend are useful configuration terms, not universal product names. A frontend describes where traffic enters and which rules apply. A backend describes where accepted traffic can go. Managed services often use listener, rule, target group, pool, or origin for the same boundaries.

L4 and L7 answer different routing questions

Layer 4 balancing works with transport connections such as TCP or UDP. It can select a target using addresses, ports, connection state, and supported hashing inputs without understanding an HTTP URL. This keeps the routing decision independent of application content.

Layer 7 balancing understands an application protocol, most commonly HTTP. It can route one hostname to one backend and an API path to another, add forwarding headers, manage cookies, terminate TLS, or make request-level retry decisions.

Neither layer is automatically better. Choose the lowest layer that can express the routing and observability you need. A transparent TCP service gains little from HTTP parsing, while a shared web entry point often needs Layer 7 rules.

Algorithms distribute work; they do not create capacity

Round robin rotates selections through eligible targets. It is predictable when targets and requests are broadly similar. Weighted round robin gives larger targets a larger share but still depends on sensible weights.

Least connections favours a target with fewer active connections. It can fit long-lived or uneven sessions better than a simple rotation, although connection count is not the same as CPU, memory pressure, or request cost.

Hash-based selection uses a stable input, such as a source address or request property, to improve repeatable placement. The result changes when the target set changes unless the implementation uses a strategy designed to limit remapping.

Algorithms work only with the signals they receive. If every target is saturated, a clever selector merely chooses among saturated targets. Capacity planning, concurrency limits, queues, and backpressure remain application and platform concerns.

A health check proves one narrow condition

An active health check opens a connection or sends a synthetic request on a schedule. A passive check watches errors in real traffic. Thresholds normally avoid ejecting a target after one transient result and avoid returning it after one lucky response.

The check should test the condition required to serve real traffic without turning every dependency into one brittle chain. A TCP connection proves that a port accepted a connection. It does not prove that a login, database query, or write path works.

An HTTP check can verify a deliberate readiness endpoint, expected status, and sometimes response content. Yet a shallow endpoint may stay green while the useful application is broken. A deep endpoint can remove every target when a shared dependency fails.

Health is therefore a routing decision, not a diagnosis. Keep the check representative, watch its false positives and false negatives, and alert on remaining healthy capacity rather than admiring one reassuring green light.

Failover must respect existing connections

Removing a failed target from selection usually affects new traffic first. Existing TCP connections, streams, uploads, and in-flight requests may still be attached to it. Abruptly closing them can turn a tidy deployment into visible errors.

Connection draining stops assigning new work to a target while allowing existing work a limited time to finish. The drain period must reflect real connection duration. A short web request and a long-lived stream need different expectations.

Draining is also useful during deployments and scale-in. The target first becomes unavailable for new selections, then completes or times out existing work, and only then shuts down. Application shutdown hooks and orchestrator readiness must agree with that sequence.

Persistence solves state placement at a cost

Session persistence, or stickiness, tries to return a client to the same target. Cookie-based persistence can identify a browser session at Layer 7. Source-address persistence can work at Layer 4, but many users behind one network address may be grouped together.

Stickiness can help an application that still keeps session state locally. It also makes traffic less even and turns target loss into session loss unless the state lives elsewhere. Treat persistence as a compatibility measure, not a substitute for shared or durable state.

When a target drains, define whether its sticky sessions stay attached, move immediately, or expire. An undocumented choice here often appears as intermittent logouts rather than an obvious load-balancer fault.

TLS termination changes trust boundaries

A Layer 7 balancer often terminates client TLS so it can inspect HTTP and apply routing rules. The balancer then opens a separate connection to the backend. That upstream connection may use plaintext on a trusted private network or TLS with its own authentication policy.

Termination centralizes certificates and policy, but it also makes the balancer a security boundary. Certificate renewal, supported protocols, key access, upstream encryption, and the treatment of the original scheme must be monitored rather than assumed.

TLS passthrough keeps encryption between the client and backend. It preserves end-to-end termination at the target, but the balancer cannot inspect encrypted HTTP properties unless it has another protocol signal available.

Preserve client identity only from trusted proxies

A proxy connection can make the backend see the balancer’s address instead of the client’s. HTTP deployments commonly forward client and scheme information in standardized or conventional headers. Transport proxies may use PROXY protocol where both ends support it.

Never trust client-supplied forwarding headers from the public Internet. The receiving application or proxy must accept identity information only from known proxy addresses and must define how an existing header chain is replaced or extended.

This affects more than logs. Rate limits, access controls, geolocation, audit trails, and abuse investigation can all be wrong when the trust boundary is misconfigured.

Retries can duplicate real work

A balancer may retry after a connection failure or selected upstream error. Retrying a safe read is different from repeating a request that charges a card, creates a record, or starts a job.

HTTP defines idempotent methods as requests whose intended effect is the same when repeated. Automatic retries should follow method semantics and application knowledge. A nominally idempotent request can still need an idempotency key or deduplication when the client cannot tell whether the first attempt completed.

Set retry conditions, budgets, and timeouts together. Unbounded retries amplify an overloaded dependency. Retrying across several layers can multiply work while each layer believes it is being helpful.

Regional balancing and global steering are separate layers

A regional load balancer distributes traffic among targets reachable within its service scope. Global balancing first chooses a region or pool, often using DNS, an anycast edge, or a provider’s global proxy, and then local balancing selects an endpoint.

DNS-based steering returns an address rather than proxying each request. Clients and recursive resolvers can keep that answer until its cache lifetime ends, so a health change does not move every existing client immediately.

A global proxy can make faster request-level decisions, but it introduces a larger managed edge dependency. In either model, failover needs spare capacity in the destination region and data that the application can safely use there.

A load balancer is not automatic high availability

A self-managed balancer on one small VM is a new single point of failure. Rent boring infrastructure if it gives you multiple instances, a working address takeover, independent failure domains, and tested configuration distribution—but verify that those pieces really exist.

Even a redundant balancer only moves traffic. It does not replicate databases, repair corrupt state, make shared storage available, or ensure that a downstream identity or payment service works. Failover without compatible application state can redirect users to a healthy server that cannot complete their request.

Observe the whole path:

• listener connection and TLS errors;
• request rate, latency, and response outcomes;
• healthy, unhealthy, warming, and draining targets;
• per-target connections and saturation;
• retry counts and reasons;
• pool or region failover events;
• configuration changes and certificate expiry;
• application success indicators beyond the health endpoint.

A load balancer is valuable because it creates a controlled decision point between clients and service instances. Its reliability comes from the quality of that decision, the capacity behind it, and the failure modes around it. The label on the box does none of that work.