Cloud Explainers

How a Load Balancer Works with HAProxy

Load balancing & Nginx9 min video

How a load balancer works, shown with HAProxy on real Ubuntu servers: one address shared by three web servers (about three times the pages a second of one), round robin, what runners get when a server stops and there are no health checks (a 503 after three seconds), health checks marking a stopped server DOWN in seconds, an HTTP check catching a server whose disk is full, a basket that empties until a sticky cookie, and X-Forwarded-For.

One address, three servers

Your site runs on one server, and on a busy morning, one server isn't enough. Ana runs the sign-up site for a city marathon. Ben, a runner, points a load-testing tool, wrk, at its page: it keeps thirty-two connections busy for ten seconds. With one server behind the address: about 9,800 pages a second. Sign-ups open at nine. Everyone arrives at nine. So Ana puts two more servers behind the load balancer at that address: about 30,400.

Ben's laptop
wrk -t2 -c32 -d10s http://race.lan/Running 10s test @ http://race.lan/
  2 threads and 32 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency     3.30ms    1.08ms  29.32ms   93.48%
    Req/Sec     4.94k   364.69     9.49k    97.51%
  98883 requests in 10.10s, 137.12MB read
Requests/sec:   9790.59
Transfer/sec:     13.58MB
Ben's laptop
wrk -t2 -c32 -d10s http://race.lan/Running 10s test @ http://race.lan/
  2 threads and 32 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency     1.11ms  691.62us  16.79ms   89.51%
    Req/Sec    15.26k   688.14    16.14k    89.11%
  306812 requests in 10.10s, 425.44MB read
Requests/sec:  30376.90
Transfer/sec:     42.12MB

The load balancer here is HAProxy, free and open source, and it just tripled the pages. We'll watch it share the work and notice a dead server. But how can it spot a dead one, and still miss a broken one?

Pages a second, same address: chart of 1 server: 9800 pages/s, 3 servers: 30400 pages/s

What a load balancer does

So what does a load balancer actually do? It stands at the one address runners use, race dot lan. Every request arrives there first. HAProxy picks a server from its list, passes the request on, and passes the answer back. Ben never talks to a server himself, so Ana can add one, remove one, or fix one, and the address never changes.

Every request goes through one front door: diagram of Ben's laptop, HAProxy, web1, web2, web3, Ana's web servers

Think of airport check-in. One queue, many desks, and an agent at the front who points each traveller to the next desk. Travellers never pick a desk. And a desk with its light off gets nobody: hold on to that one.

Airport check-in, and a load balancer: comparison table of Check-in hall vs HAProxy, One way in, Does the work, Who chooses, Not open

Ana's HAProxy config

Here is Ana's whole setup, a few lines she added to HAProxy's config file. A port is a numbered door on a computer. The frontend is the front door: it listens on port eighty, the normal web port. It hands requests to a backend called web: the list of servers. Each server's app listens on its own door, eighty-eighty, which only HAProxy uses. Balance round robin means: take turns. Before HAProxy loads new settings, Ana runs haproxy dash c, which checks the file for mistakes.

Ana on the load balancer (lb)
sed -n '/^frontend/,/web3/p' /etc/haproxy/haproxy.cfgfrontend signup
    bind :80
    default_backend web

backend web
    balance roundrobin
    server web1 web1:8080
    server web2 web2:8080
    server web3 web3:8080
Ana on the load balancer (lb)
sudo haproxy -c -f /etc/haproxy/haproxy.cfgConfiguration file is valid

Next, Ben uses curl to fetch a tiny test page, slash hello, where each server answers with its own name. Six times in a row: web1, web2, web3, and around again. That's round robin. Each server gets every third request, whoever sends it.

Ben's laptop
for i in 1 2 3 4 5 6; do curl -s race.lan/hello; doneHello from web1
Hello from web2
Hello from web3
Hello from web1
Hello from web2
Hello from web3

Round robin is the simplest way to share, and HAProxy has others. Least conn sends each request to the server with the fewest open connections, good when some requests take much longer than others. Source looks at each visitor's address and always sends that visitor to the same server. For short web pages, round robin is a fine start.

balance: how HAProxy picks a server: cards of roundrobin, leastconn, source, start here

When a server goes down

Now web2 stops, the way a crash or an update would. HAProxy doesn't know. It just keeps following its list. Guess first: what does every third runner get now? An error. Five oh three, Service Unavailable, every time it's web2's turn.

Ana on web2, then Ben's laptop
sudo systemctl stop signup
Ana on web2, then Ben's laptop
for i in 1 2 3 4 5 6; do curl -s race.lan/hello | head -1; done<html><body><h1>503 Service Unavailable</h1>
Hello from web3
Hello from web1
<html><body><h1>503 Service Unavailable</h1>
Hello from web3
Hello from web1

And it isn't quick. Ben times three requests. Two come back fine, status two hundred, in a couple of thousandths of a second. The one that lands on web2 takes three seconds, then fails. HAProxy's log names web2, and the last number on that line, three, is how many times it retried.

Ben's laptop, then Ana on lb
for i in 1 2 3; do curl -s -o /dev/null -w '%{http_code} after %{time_total} s\n' race.lan/hello; done503 after 3.013191 s
200 after 0.002095 s
200 after 0.001558 s
Ben's laptop, then Ana on lb
grep ' 503 ' /var/log/haproxy.log | tail -12026-11-20T18:52:18.835801+00:00 lb haproxy[222]: 10.13.0.50:54202 [20/Nov/2026:18:52:15.823] signup web/web2 0/0/-1/-1/3011 503 216 - - SC-- 1/1/0/0/3 0/0 "GET /hello HTTP/1.1"

Why three seconds? On web2's turn, HAProxy's connection is refused. By default it tries three more times, waiting up to a second between tries, then answers five oh three itself. Every runner whose turn lands on web2 pays that wait.

No checks: every turn on web2 costs three seconds: diagram of Runner, HAProxy, web2, GET /hello, connect, retry x3, 503

Health checks

The fix is one word on each server line: check. Now HAProxy knocks on each server by itself, every two seconds, instead of waiting for a runner to find out.

Turn on health checks: code change of /etc/haproxy/haproxy.cfg, server web1 web1:8080 check (health check every 2 s), server web2 web2:8080 check (health check every 2 s), server web3 web3:8080 check (health check every 2 s)

Ana starts web2, reloads HAProxy so it reads the new file, then stops web2 again and notes the time. Within five seconds, the log says: web2 is DOWN, connection refused, two active servers left. That's three missed knocks in a row: the first, then two more, two seconds apart. And Ben's six requests now go only to web1 and web3. All of them work.

Ana on lb and web2, then Ben's laptop
sudo systemctl reload haproxy
Ana on lb and web2, then Ben's laptop
sudo systemctl stop signup; date +%T18:52:25
Ana on lb and web2, then Ben's laptop
grep 'web2 is DOWN' /var/log/haproxy.log2026-11-20T18:52:29.742177+00:00 lb haproxy[249]: [WARNING]  (249) : Server web/web2 is DOWN, reason: Layer4 connection problem, info: "Connection refused", check duration: 0ms. 2 active and 0 backup servers left. 0 sessions active, 0 requeued, 0 remaining in queue.
2026-11-20T18:52:29.742502+00:00 lb haproxy[249]: Server web/web2 is DOWN, reason: Layer4 connection problem, info: "Connection refused", check duration: 0ms. 2 active and 0 backup servers left. 0 sessions active, 0 requeued, 0 remaining in queue.
Ana on lb and web2, then Ben's laptop
for i in 1 2 3 4 5 6; do curl -s race.lan/hello; doneHello from web1
Hello from web3
Hello from web1
Hello from web3
Hello from web1
Hello from web3

HAProxy also has its own stats page, which Ana turned on with four more lines in the same file. It shows the same thing: web1 and web3 up, web2 down, in red. When Ana starts web2 again, two good checks bring it back, about three seconds later.

Health checks: browser screenshot from the video

Those numbers are defaults. Inter is the two seconds between checks, fall is the three misses, rise is the two good checks, and you can change each one on the server line. Checking faster notices a dead server sooner, but every check is one more request each server has to answer.

check: the defaults, and the words to change them: cards of inter 2s, fall 3, rise 2, trade-off

Up, but broken

Now the break HAProxy can't see. Overnight, an export of the sign-ups filled web3's data disk: one hundred percent. Ben signs up three times, and the one on web3 fails: five hundred, database or disk is full. Yet HAProxy keeps sending runners to web3. Its knock only asks: is the port open? And it is.

Ana on web3, then Ben's laptop
df -h /var/lib/signupFilesystem      Size  Used Avail Use% Mounted on
tmpfs           4.0M  4.0M     0 100% /var/lib/signup
Ana on web3, then Ben's laptop
for i in 1 2 3; do curl -s -d name=Ben race.lan/signup; doneweb1: thanks Ben, you're signed up
web3: 500 sign-up failed: database or disk is full
web2: thanks Ben, you're signed up

So Ana asks a better question. One new line, option H T T P check, tells HAProxy to ask each server for a page called slash health. To answer, the app really tries to save a sign-up, then undoes it. After a reload: web3 is DOWN, wrong status, five hundred. Layer seven means the web answer itself, not just the open port. Ben's sign-ups all work again.

Ana on lb, then Ben's laptop
grep -n 'httpchk' /etc/haproxy/haproxy.cfg43:    option httpchk GET /health
Ana on lb, then Ben's laptop
sudo systemctl reload haproxy
Ana on lb, then Ben's laptop
grep 'web3 is DOWN' /var/log/haproxy.log | tail -12026-11-20T18:52:35.551185+00:00 lb haproxy[422]: Server web/web3 is DOWN, reason: Layer7 wrong status, code: 500, info: "INTERNAL SERVER ERROR", check duration: 2ms. 2 active and 0 backup servers left. 0 sessions active, 0 requeued, 0 remaining in queue.
Ana on lb, then Ben's laptop
for i in 1 2 3; do curl -s -d name=Ben race.lan/signup; doneweb1: thanks Ben, you're signed up
web2: thanks Ben, you're signed up
web1: thanks Ben, you're signed up

So HAProxy keeps a list of servers, and a check on each. The plain check knocks: is the port, the door, open? That catches web2, which wasn't running at all. The HTTP check asks the app itself, and catches web3, answering five hundred. A runner's request skips both, like a desk with its light off.

One address, a list of servers, a check on each: diagram of Runners, HAProxy, web1, web2, web3, Ana's web servers

The basket that empties

With all three servers healthy again: what breaks when requests take turns? Ben adds the ten K to his basket. His curl keeps cookies, small tags a site asks to get back, in a file, like a browser would. web2 takes it. Guess first: he opens his basket three times. What does he see? Empty, twice. Only one of the three servers has his basket.

Ben's laptop
curl -s -c jar -b jar 'race.lan/add?race=10K'web2: added 10K
Ben's laptop
for i in 1 2 3; do curl -s -b jar race.lan/basket; doneweb1: basket empty
web3: basket empty
web2: 10K

In a browser it's the same. A half marathon in the basket, reload, and the next server has never heard of it.

The basket that empties: browser screenshot from the video

Each server keeps baskets in its own memory. Round robin sends Ben's next request somewhere else, to a server that has never seen him. Nothing is broken; the servers just don't share.

The basket lives on one server: diagram of Ben's laptop, HAProxy, web1, web2, web3, Ana's web servers

Two fixes. The lasting one: keep baskets somewhere every server can read, like a database. The quick one is in HAProxy: a cookie setting, and a name for each server. HAProxy tells Ben's browser which server he got, and sends him back there. Now all three answers come from web1, with his ten K.

Ana on lb, then Ben's laptop
grep -n 'cookie' /etc/haproxy/haproxy.cfg44:    cookie SERVERID insert indirect nocache
45:    server web1 web1:8080 check cookie web1
46:    server web2 web2:8080 check cookie web2
47:    server web3 web3:8080 check cookie web3
Ana on lb, then Ben's laptop
curl -s -c jar -b jar 'race.lan/add?race=10K'web1: added 10K
Ana on lb, then Ben's laptop
for i in 1 2 3; do curl -s -b jar race.lan/basket; doneweb1: 10K
web1: 10K
web1: 10K
Ana on lb, then Ben's laptop
grep -o 'SERVERID.*' jarSERVERID	web1

Who sent this request?

One more surprise, and it matters the day Ana must block someone hammering the sign-up page: whose address does a server see? Ben's laptop is ten dot thirteen dot zero dot fifty. But web2's app log, read with journalctl, says the request came from ten dot thirteen dot zero dot ten: HAProxy. From a server's side, the load balancer is the visitor. One more line, option forward for, adds a note to every request, a header called X Forwarded For, with the runner's real address. After a reload, web2's log shows the forwarded-for address, and it's Ben's.

Ben's laptop, then Ana on web2
hostname -I10.13.0.50
Ben's laptop, then Ana on web2
journalctl -u signup -o cat | grep hello | tail -110.13.0.10 xff=- "GET /hello HTTP/1.1" 200
Ben's laptop, then Ana on web2
grep -n 'forwardfor' /etc/haproxy/haproxy.cfg45:    option forwardfor
Ben's laptop, then Ana on web2
sudo systemctl reload haproxy
Ben's laptop, then Ana on web2
for i in 1 2 3; do curl -s race.lan/hello; doneHello from web1
Hello from web2
Hello from web3
Ben's laptop, then Ana on web2
journalctl -u signup -o cat | grep hello | tail -110.13.0.10 xff=10.13.0.50 "GET /hello HTTP/1.1" 200

What if HAProxy stops?

And if every request goes through HAProxy, what if HAProxy itself stops? Then the whole site stops with it. So bigger sites run two load balancers sharing one address, the second one watches the first, and takes over the address if the first stops. A cloud provider's load balancer does that for you.

Two load balancers, one address: diagram of Runners, HAProxy A, HAProxy B, web1, web2, web3, Ana's web servers

Recap

So: one address, a list of servers, taking turns. Health checks, so a dead server gets nobody. An HTTP check, so a broken one doesn't either. And don't keep things like baskets in one server's memory.

A load balancer, in four lines: diagram of frontend + backend, check, option httpchk, cookie / shared store

If Ben's sign-up went through, subscribing brings more of how the internet's plumbing works, tested on real machines.

Everything here ran on real Ubuntu servers with Ubuntu's own HAProxy 2.8, and every command and the lab files are linked in the description. Thanks for watching Cloud Explainers.

Sources

Demos: five containers on our own Mac standing in for a load balancer, three web servers and a runner's laptop; the marathon and its runners are made up. Every number, page and log line was real. Watch the full video on YouTube.

More videos

Python8 min

How to Fix Python ModuleNotFoundError: No Module Named

Go (Golang)5 min

Golang Pointers in 5 Minutes

Kubernetes8 min

How to Fix a Kubernetes Pod Stuck in CrashLoopBackOff

Homelab & self-hosting9 min

How to Fix a Homelab ESXi Host Not Responding

PostgreSQL24 min

PostgreSQL: Full Tutorial for Beginners

Python9 min

How Python Virtual Environments Work

Homelab & self-hosting9 min

How to Fix Cloudflare Tunnel Errors in a Homelab

APIs9 min

How to Fix API CORS Errors: No 'Access-Control-Allow-Origin'

Auth (OAuth, JWT)9 min

How Authenticator App Login Codes Work (TOTP)

Git22 min

Git and GitHub: Full Tutorial for Beginners

Homelab & self-hosting10 min

How a Homelab Hypervisor Works (VMware ESXi)

Kubernetes9 min

How to Fix Kubernetes ImagePullBackOff Errors

DNS8 min

How to Fix DNS 'Temporary Failure in Name Resolution' on Linux

AWS8 min

How to Fix AWS CLI 'Unable to Locate Credentials'

Homelab & self-hosting10 min

Proxmox vs ESXi for Your Homelab: When Each One Wins

PostgreSQL10 min

PostgreSQL UUID vs Auto Increment: Which Primary Key Should You Use?

Kubernetes21 min

Kubernetes: Full Tutorial for Beginners

Networking9 min

How TCP Sockets Work

PostgreSQL7 min

How to Fix PostgreSQL 'Could Not Connect to Server'

Claude & the Claude Developer Platform9 min

Claude Code Hooks Will Finally Make Sense After This

DevOps & CI/CD9 min

DevOps with Ansible Will Finally Make Sense After This

Networking9 min

How SNMP Network Monitoring Works

Git8 min

How to Fix GitHub 'Permission Denied (publickey)' Errors

Docker21 min

Docker: Full Tutorial for Beginners

PostgreSQL7 min

How to Fix PostgreSQL 'Password Authentication Failed'

Linux8 min

Why One Linux Server Handles More Users Than You Think

Networking7 min

How to Fix Network 'Connection Refused' Errors

DevOps & CI/CD9 min

DevOps Pipelines: Jenkins vs GitHub Actions

Linux11 min

Linux Nginx Reverse Proxy: Full Tutorial for Beginners

Networking11 min

Every Networking Term a Beginner Needs in 12 Minutes

Docker6 min

How to Fix Docker 'Permission Denied' on docker.sock

Git7 min

How to Fix Git 'Updates Were Rejected' When You Push

Linux7 min

How to Fix nginx 502 Bad Gateway Errors on Linux

Linux8 min

How to Fix Linux apt 'Could Not Get Lock' Errors (Ubuntu)

Networking9 min

How Email Works on the Internet (SMTP) in 10 Minutes

Python8 min

The Python Roadmap for Beginners

Go (Golang)21 min

Golang REST API with Gin: Full Tutorial for Beginners

Networking10 min

How a WireGuard VPN Network Works

Linux9 min

Linux Makefiles Will Finally Make Sense After This

Auth (OAuth, JWT)9 min

SAML vs OAuth vs OpenID Connect: What's the Difference?

Python7 min

How to Fix Python cron Jobs That Never Run

Linux22 min

Linux Bash Scripting: Full Tutorial for Beginners

Networking10 min

How PXE Network Boot Works

Active Directory9 min

How Active Directory Works

Linux10 min

How to Fix a Linux systemd Service That Won't Start

Linux9 min

Linux Logs: syslog vs journald

Auth (OAuth, JWT)9 min

JWT vs Session Cookies: Which One Should You Use?

Docker10 min

Docker vs Docker Compose: Which One Should You Use?

Docker9 min

How to Fix Docker Compose Containers That Can't Talk to Each Other

Django (Python web framework)22 min

Django Full Tutorial for Beginners

Linux9 min

How Linux systemd Works

Networking8 min

How to Fix 'Windows Cannot Access' Network Share Errors (SMB)

APIs10 min

REST vs gRPC: Which One Should You Use?

Certificates & PKI (ADCS)9 min

How to Fix SSL Certificate Chain Errors (Missing Intermediate)

Python9 min

Every Python Concept for Beginners in 10 Minutes

Databases & SQL9 min

Every SQL Join Type in 10 Minutes

OpenRouter (one API for many AI models)9 min

Why OpenRouter Free Models Hit Their Limit So Fast

Gemini API & Google AI Studio9 min

Why Your Gemini API Free Tier Runs Out So Fast

OpenRouter (one API for many AI models)10 min

OpenRouter Will Finally Make Sense After This

Caching & Redis10 min

How Redis Caching Works

Gemini API & Google AI Studio10 min

Gemini API vs Vertex AI: Which One Should You Use?

Gemini API & Google AI Studio8 min

How to Fix a Leaked Gemini API Key

Go (Golang)9 min

Why Golang Uses Goroutines Instead of Threads

Gemini API & Google AI Studio9 min

How to Fix Gemini API Error 429: Resource Exhausted

Claude & the Claude Developer Platform10 min

Claude Token Pricing Will Finally Make Sense After This

Python11 min

Why Python Web Scraping Returns Nothing (and the Fix)

Python10 min

Python uv vs pip: Which One Should You Use?

Python22 min

Python Automation: Full Tutorial for Beginners

Auth (OAuth, JWT)9 min

How OAuth Login Works

Gemini API & Google AI Studio10 min

Google AI Studio Will Finally Make Sense After This

Docker10 min

What Happens When You Run a Docker Container

Claude & the Claude Developer Platform8 min

How to Fix Claude Code Billing Your API Key Instead of Your Plan

Auth (OAuth, JWT)9 min

Why Passkeys Can't Be Phished Like Passwords

PowerShell10 min

PowerShell vs Bash: Which One Should You Learn?

Linux10 min

The Linux Roadmap for Beginners

Django (Python web framework)9 min

Django vs FastAPI: Which One Should You Use?

Databases & SQL10 min

Why SQLite Is the Most Used Database in the World

Databases & SQL11 min

How a NoSQL Database Like MongoDB Works

Python11 min

How to Fix Python pip error: externally-managed-environment

Databases & SQL9 min

How to Fix MySQL Too Many Connections Database Errors

Go (Golang)9 min

Golang Concurrency Will Finally Make Sense After This

AWS10 min

How Amazon S3 Works

Kubernetes10 min

How Kubernetes Works

Git10 min

How Git Works

Docker10 min

How to Fix Docker Port Already Allocated Errors

PostgreSQL10 min

How PostgreSQL Works

Git8 min

How to Fix Git Merge Conflicts

Linux9 min

How to Fix No Space Left on Device in Linux

Networking10 min

How the Internet Works

Databases & SQL9 min

SQL vs NoSQL: When Each One Wins

Auth (OAuth, JWT)8 min

How to Fix JWT Token Expired Errors (401 Unauthorized)

Claude & the Claude Developer Platform9 min

Claude API vs Claude Pro: Why Your Subscription Doesn't Include It

AI agent skills9 min

Why AI Agent Skills Beat Giant Prompts

Linux10 min

Linux Cron Jobs Will Finally Make Sense After This

Cybersecurity10 min

Security Logs Will Finally Make Sense After This

Cybersecurity8 min

How to Fix a Security Leak: An API Key Pushed to GitHub

DNS9 min

DNS Will Finally Make Sense After This

Cybersecurity9 min

Never Ignore a Security Update

Auth (OAuth, JWT)9 min

Two-Factor Authentication Will Finally Make Sense

Networking8 min

Never Expose a Database to the Internet

Cybersecurity10 min

SSH Keys vs Passwords: Which Is Safer for Security?

Linux10 min

The 5 Levels of Linux Boot: From Power On to Login

Networking12 min

The 5 Levels of Network Firewalls

Cybersecurity10 min

Password Security: Why Hackers Crack Weak Passwords

Linux9 min

Why Your Linux Server Runs Out of Memory

Networking10 min

HTTP vs HTTPS: What Actually Changes on Your Network

Linux12 min

Every Linux File Permission You Need to Know

Linux10 min

Linux Processes vs Threads: When Each One Wins

Networking11 min

The OSI Model Will Finally Make Sense After This

Git11 min

The 7 Levels of Git

Networking10 min

Every Network Troubleshooting Command You Need

Cybersecurity10 min

Hashing vs Encryption in Cybersecurity: When Each One Wins

Networking11 min

Why the Internet Ran Out of IP Addresses

Linux11 min

Linux Package Managers: apt vs Snap vs Flatpak

Python9 min

Why Python Is Slow (and When It Doesn't Matter)

Networking11 min

TCP vs UDP: When Each One Wins

Linux8 min

Stop Running Linux as Root (Do This Instead)

Linux11 min

Linux Users and Groups Will Finally Make Sense

Cybersecurity9 min

Why Phishing Beats Good Security (and How to Spot It)

Docker12 min

The 7 Levels of Docker

AI agents & MCP12 min

AI Agents Will Finally Make Sense After This

Git8 min

Stop Using git push --force (Use This Instead)

AI agent skills10 min

Agent Skills vs MCP: When Each One Wins

Networking12 min

Networking Will Finally Make Sense After This

Linux12 min

Every Linux Command You Actually Need

Cybersecurity12 min

The 7 Levels of Cybersecurity

System design12 min

Every System Design Concept You Need, in One Diagram

PostgreSQL10 min

Why PostgreSQL Ignores Your Index