Why Golang Uses Goroutines Instead of Threads

You've read that a Go server starts a goroutine for every customer, while other languages warn you never to start thousands of threads. So how can Go get away with it? Goroutines run on threads in the end. What's actually different?

People have asked this for years. On the golang-nuts mailing list, back in 2013, someone asked how Go can run a hundred thousand goroutines on a single CPU, switching between all of them without large overhead, when threads each need their own fixed stack. Today we measure all of it.

Ana runs a tea shop, and its order server waits on lots of customers at once. On one machine, we'll start ten thousand real OS threads, then ten thousand goroutines, and compare memory, start time and switching. And one question to keep in mind: when does Go quietly start a real thread for a goroutine anyway? We'll catch it doing that.

So what is a thread? It's the operating system's unit of work. A program asks the kernel, the core of Linux, for one, and the kernel decides when each thread gets a CPU core. Each thread also needs a stack, memory for its function calls, reserved in one piece up front, plus a small stack inside the kernel.

Here's a small program in C, using plain OS threads: one thread per waiting customer. Ten thousand of them, plus the program's own main thread, started in 105 milliseconds. 82 megabytes of memory, 156 more inside the kernel, and 78 gigabytes of address space reserved for their stacks: a promise of room, not memory in use.

c/threads 10000threads started:   10000
OS threads:        10001
time to start:     0.105 s
memory used:       82 MB
kernel stacks:     156 MB
stack reserved:    78 GB

Reserved isn't used: that's eight megabytes of address space per thread, set aside in case it's needed. What each thread really costs is its memory plus its kernel stack: about 24 kilobytes, for a thread that does nothing but wait.

Now ten times as many: a hundred thousand threads. Guess first: what happens? Linux refuses. Thread number 63,462 failed: resource temporarily unavailable, which means the system has no room for one more. This machine allows 63,759 threads in total, a limit Linux sets from its 8 gigabytes of memory, and every thread already running counts too. And remember, these threads did nothing at all but wait.

c/threads 100000thread 63462 failed: Resource temporarily unavailable
threads started:   63461
OS threads:        63462
time to start:     0.645 s
memory used:       515 MB
kernel stacks:     991 MB
stack reserved:    499 GB

cat /proc/sys/kernel/threads-max63759

A goroutine is different: the kernel never sees it. The Go runtime, code that comes inside every Go program, keeps its own list of goroutines, and runs them on a few OS threads, about one per CPU core. When a goroutine has to wait, the runtime puts another one on that thread.

In code, the difference is small. In C, each customer gets a call that asks the kernel for a thread, and the thread waits on a condition: a signal that the shop is open. In Go, you write the word go in front of a function call, and that function runs as a goroutine, waiting on a channel.

The same job in Go: ten thousand goroutines, one per customer. Started in 5 milliseconds, about 21 times faster. 32 megabytes, nothing in the kernel, and the whole program uses just 11 OS threads, for ten thousand goroutines.

go run ./goroutines 10000goroutines started: 10000
OS threads:         11
time to start:      0.005 s
memory used:        32 MB
kernel stacks:      0 MB

A hundred thousand: fine. A million goroutines: started in 697 milliseconds, still on 11 threads. A hundred times more goroutines, and not one extra thread. That's far past where threads ran out on this same machine.

go run ./goroutines 100000goroutines started: 100000
OS threads:         11
time to start:      0.052 s
memory used:        271 MB
kernel stacks:      0 MB

go run ./goroutines 1000000goroutines started: 1000000
OS threads:         11
time to start:      0.697 s
memory used:        2620 MB
kernel stacks:      0 MB

Per task, that million used about 2.7 kilobytes each. A waiting thread cost about 24. So a goroutine is roughly 9 times smaller here, and it needs nothing from the kernel's thread limit.

That's why real Go servers lean on them. Go's own web server, net/http, starts a new goroutine for each connection. And at Uber, engineers wrote in 2022 that, across hundreds of thousands of running services, their Go services showed about eight times more concurrency than their Java ones.

Why is a goroutine so small? Mostly its stack. A thread's stack can't grow, so Linux reserves room for the worst case: eight megabytes here. A goroutine starts with a few kilobytes. When a function needs more, the runtime copies the stack to a bigger block and carries on, up to a gigabyte.

Let's push it. A function that calls itself a million times, each call taking a little room, at least sixty-four bytes. That's sixty-four megabytes of stack or more. ulimit dash s shows a thread's stack size, in kilobytes: eight megabytes. On a C thread, with that fixed eight megabytes: segmentation fault, the program ran off the end of its stack and crashed. Now the same calls in a goroutine, which started with a few kilobytes. Will it crash too? It finishes: a million calls deep. The runtime kept moving its stack to bigger blocks, and the program never noticed.

ulimit -s8192

c/growSegmentation fault

go run ./growcalls deep: 1000000

Memory is half the story; switching is the other half. When one thread has to wait, the CPU leaves your program and enters the kernel. The kernel saves that thread's state, picks the next thread, and switches the CPU over to it. Every switch is a trip into the kernel and back.

A goroutine switch never leaves your program. When a goroutine waits on a channel, the Go runtime saves a few values, takes the next goroutine from its list, and keeps the same thread running. And if a goroutine computes for too long, the runtime interrupts it, so the others still get a turn.

Here's the cost. Two threads passing a token back and forth, two hundred thousand times each way: 6,620 nanoseconds per hand-off. Two goroutines doing the same over a channel: 66. About 100 times faster.

c/switchthreads, one hand-off:    6620 ns

go run ./switchgoroutines, one hand-off: 66 ns

That's what makes a goroutine per customer practical: when thousands of them wake up and wait all day, each switch costs a few dozen nanoseconds, not thousands.

Now the question from the start. A system call is your program asking the kernel to do something, like reading a file. A hundred goroutines that sleep: still 5 threads. A hundred more stuck in a blocking system call, a read the kernel holds: 103 threads. Each stuck goroutine took a real thread with it, so the runtime started new ones to keep the others running.

go run ./blocking100 sleeping goroutines, OS threads: 5
100 more stuck in a system call, OS threads: 103

So that's when Go starts a real thread for a goroutine: when the kernel holds it in a system call. Waiting on the network is different. Those goroutines hand their connection to the network poller, a part of the runtime that watches every network connection at once, so ten thousand customers don't need ten thousand threads.

Side by side: the kernel runs threads, the Go runtime runs goroutines. Fixed stack against a growing one. 24 kilobytes against 2.7. 6,620 nanoseconds per switch against 66. And a ceiling of 63,461 threads, against a million goroutines and counting. On the same machine, goroutines won every measurement we ran.

Goroutines still have limits. Real work runs on threads, so heavy computing is capped by your CPU cores; GOMAXPROCS, a Go setting that defaults to your number of cores, sets how many threads run Go code at once. A million goroutines still took 2.6 gigabytes. Blocking system calls, and C code called from Go, take real threads. And a goroutine that never finishes is a leak.

The idea has spread. Java 21 added virtual threads: the Java runtime schedules them on a few OS threads, much like Go does with goroutines. Go simply had it from the start, so every Go library is written for it.

Every command and the lab files, the C threads and the Go versions, are linked in the description, so you can measure your own machine.

So: threads belong to the kernel and reserve a big stack each. Goroutines live inside your program, start tiny, grow when needed, and share a few threads, until one gets stuck in the kernel.

If this made goroutines click, a like helps YouTube show it to more beginners, and subscribing brings you the next Go video.

Every number here came from one real machine. Thanks for watching Cloud Explainers.

Sources

• golang-nuts, Goroutines vs OS threads (2013)
• Go net/http, Serve ("creating a new service goroutine for each")
• Go runtime, GOMAXPROCS (threads blocked in system calls don't count against it)
• Uber Engineering, Data Race Patterns in Go (Milind Chabbi and Murali Krishna Ramanathan, 2022; ~2,100 Go services, ~8x more concurrency than Java services)
• Linux kernel, /proc/sys/kernel/threads-max (the system-wide thread limit)
• Go FAQ, Why goroutines instead of threads?
• JEP 444: Virtual Threads (Java 21)