From 0db1fbbb5e1e24c35d228aa5b9c0bb40f63c4358 Mon Sep 17 00:00:00 2001 From: mpettibone <94031664+mpettibone@users.noreply.github.com> Date: Tue, 3 Nov 2026 12:56:27 -0600 Subject: [PATCH] Hibernate connection processes after 15 seconds idle An idle connection costs more memory than it needs to. When a keepalive HTTP/1 connection finishes a request, its process keeps whatever heap that request grew: the parsed headers, the conn, the response, all of it garbage now, but it stays until the next collection, and Bandit only forces one every five requests. So a connection that served one request and then went quiet, which is most browser connections most of the time, holds that request's leftovers for as long as it stays open. HTTP/2 connections do the same, and a WebSocket keeps the remains of its upgrade. How much depends on what the plug did: 9 KB for a plug that just sends a response, 21 KB once it does as much as parsing a query string, and 588 KB after serving a templated 60 KB page, in each case for a connection that is doing nothing. The VM has a fix for exactly this: `hibernate_after`. A GenServer started with it hibernates once it has had no message for that long, which runs a full collection, compacts the heap and parks the process; the next message wakes it as usual. This PR sets `genserver_options: [hibernate_after: 15_000]` in the options Bandit passes to Thousand Island, the same way it already defaults the port, transport and handler. Anything the user puts in `genserver_options` is kept, so `hibernate_after: :infinity` turns it off and any other value changes the delay. Nothing else changes: a connection in use never goes 15 seconds without a message, so the only connections that hibernate are the ones sitting idle. This belongs in Bandit rather than Thousand Island, I think. Thousand Island hands `genserver_options` to the handler untouched and stays out of protocol decisions, and what "idle" means and how much a request leaves behind are properties of the protocol. Bandit already owns this kind of per-connection housekeeping (`gc_every_n_keepalive_requests`, `clear_process_dict`, the collection it runs at WebSocket upgrade); this is the piece that covers the time between requests. ## Numbers Measured on `main` at `4812459` in a 2-core VM at 2.1 GHz, OTP 25.3, Elixir 1.17.3. Memory is `Process.info/2` of the connection process, read while the connection is idle; the CPU figures come from `:erlang.statistics/1` on a single scheduler with the client in the same VM, so they are upper bounds. Memory per idle connection, before and after hibernating: | Idle connection | Today | Hibernated | |---|---|---| | HTTP/1 keepalive, plug sends a short body | 8.6 KB | 2.1 KB | | HTTP/1 keepalive, plug also parses the query string | 21.1 KB | 2.1 KB | | HTTP/1 keepalive, plug renders a 60 KB page | 587.7 KB | 2.1 KB | | HTTP/2, one request served | 21.1 KB | 3.3 KB | | WebSocket, after the handshake | 8.6 KB | 2.7 KB | For scale: 3,000 idle connections, a thousand each of the first HTTP/1 case, HTTP/2 and WebSocket, hold 49.6 MB today and 7.9 MB hibernated. With the templated page it is 587 MB per thousand idle connections against 2 MB. What hibernating costs, measured over 2,000 HTTP/1 connections: | Cost | Measured | |---|---| | Hibernating a connection | 8 to 10 us CPU, once per idle period | | First request after waking, CPU | about 8 us more (55 to 64 us) | | First request after waking, latency | p50 80 to 100 us either way, p99 same | | Throughput under keepalive load | 22,089 vs 22,279 req/s, within noise | The last row is the important one for anyone worried about the hot path: a connection under load never hibernates, because it never sits idle for 15 seconds. The cost lands only on connections that were idle, and it is one collection the connection would have had to pay eventually anyway, plus a few microseconds to regrow the heap. For a WebSocket under Phoenix's 30 second heartbeat, which wakes and re-hibernates every cycle, that comes to about 20 us per connection per 30 seconds: roughly 7% of one core for 100,000 connections, against the memory in the table. One side effect that turned out to be a benefit: a connection that was woken serves its next request from a compact heap, so after that request it sits at 8.6 KB instead of growing to 26 KB as it did on `main`. Hibernation resets the growth that keepalive connections otherwise accumulate between forced collections. ## Why 15 seconds Long enough that no connection in use ever hibernates between requests; short enough that the common idle cases do: a browser holding its keepalive connections open for minutes after a page load, a WebSocket between heartbeats, an HTTP/2 connection kept around by a client that has gone quiet. It's also the figure Phoenix uses for the same purpose on channel processes, so a LiveView deployment ends up with both ends of a socket behaving the same way. ## Tests `test/bandit/hibernation_test.exs` is new. Four quick tests use a 100 ms `hibernate_after` passed through `thousand_island_options` to check that an idle connection process actually hibernates and then carries on serving (the next keepalive request, the next HTTP/2 stream, the next WebSocket frame), and that `hibernate_after: :infinity` is respected over Bandit's default, which is the `put_new` behaviour. A fifth test checks the 15 second default itself by waiting for it; it's tagged `:slow` like the other long tests in the suite. Suite, format, credo and dialyzer are clean. --- lib/bandit.ex | 22 +++++- test/bandit/hibernation_test.exs | 118 +++++++++++++++++++++++++++++++ 2 files changed, 139 insertions(+), 1 deletion(-) create mode 100644 test/bandit/hibernation_test.exs diff --git a/lib/bandit.ex b/lib/bandit.ex index cf068fdd..0667ee13 100644 --- a/lib/bandit.ex +++ b/lib/bandit.ex @@ -66,7 +66,12 @@ defmodule Bandit do * `thousand_island_options`: A list of options to pass to Thousand Island. Bandit sets some default values in this list based on your top-level configuration; these values will be overridden by values appearing here. A complete list can be found at - `t:ThousandIsland.options/0` + `t:ThousandIsland.options/0`. One default worth knowing about: Bandit sets + `genserver_options: [hibernate_after: 15_000]`, so that a connection process which has been + idle for 15 seconds is hibernated, which collects and compacts its heap and so releases the + memory left over from its last request until the connection is next used. Pass your own + `hibernate_after` within `genserver_options` to change the delay, or `:infinity` to never + hibernate * `http_options`: A list of options to configure the shared aspects of Bandit's HTTP stack. A complete list can be found at `t:http_options/0` * `http_1_options`: A list of options to configure Bandit's HTTP/1 stack. A complete list can @@ -289,6 +294,14 @@ defmodule Bandit do |> Map.keys() |> Enum.sort() + # How long a connection process may sit idle before it is hibernated. An idle keepalive or + # HTTP/2 connection otherwise holds whatever garbage its last request left (tens of KB for a + # small request, hundreds for a templated page) until the next request or collection; + # hibernating collects and compacts the heap down to a couple of KB. Fifteen seconds is long + # enough that a connection in active use never hibernates between requests, and short enough + # that a connection parked by a browser or between WebSocket heartbeats does + @hibernate_after 15_000 + @doc """ Starts a Bandit server using the provided arguments. See `t:options/0` for specific options to pass to this function. @@ -376,6 +389,13 @@ defmodule Bandit do |> Keyword.put(:transport_options, transport_options) |> Keyword.put_new(:handler_module, Bandit.DelegatingHandler) |> Keyword.put_new(:handler_options, handler_options) + |> Keyword.update( + :genserver_options, + [hibernate_after: @hibernate_after], + fn genserver_options -> + Keyword.put_new(genserver_options, :hibernate_after, @hibernate_after) + end + ) |> ThousandIsland.start_link() |> case do {:ok, pid} -> diff --git a/test/bandit/hibernation_test.exs b/test/bandit/hibernation_test.exs new file mode 100644 index 00000000..5850d6cc --- /dev/null +++ b/test/bandit/hibernation_test.exs @@ -0,0 +1,118 @@ +defmodule HibernationTest do + use ExUnit.Case, async: true + use ServerHelpers + + @hibernating {:current_function, {:erlang, :hibernate, 3}} + + def call(conn, _opts) do + conn = fetch_query_params(conn) + + if conn.query_params["websock"], + do: upgrade_adapter(conn, :websocket, {HibernationTest.Sock, [], []}), + else: send_resp(conn, 200, "OK") + end + + defmodule Sock do + use NoopWebSock + def handle_in({data, opcode: :text}, state), do: {:push, {:text, data}, state} + end + + defp connection_pids(context) do + {:ok, pids} = ThousandIsland.connection_pids(context.server_pid) + pids + end + + # A short hibernate_after keeps these tests quick; the point is that nothing in the connection + # process's loop prevents it from hibernating once idle, and that it carries on afterwards + describe "idle connection processes" do + test "hibernate on HTTP/1 between keepalive requests, then serve the next request", context do + context = + context + |> http_server(thousand_island_options: [genserver_options: [hibernate_after: 100]]) + |> Enum.into(context) + + client = SimpleHTTP1Client.tcp_client(context) + SimpleHTTP1Client.send(client, "GET", "/", ["host: localhost"]) + assert {:ok, "200 OK", _headers, "OK"} = SimpleHTTP1Client.recv_reply(client) + + Process.sleep(300) + assert [pid] = connection_pids(context) + assert Process.info(pid, :current_function) == @hibernating + + SimpleHTTP1Client.send(client, "GET", "/", ["host: localhost"]) + assert {:ok, "200 OK", _headers, "OK"} = SimpleHTTP1Client.recv_reply(client) + end + + test "hibernate on HTTP/2 between requests, then serve the next request", context do + context = + context + |> https_server(thousand_island_options: [genserver_options: [hibernate_after: 100]]) + |> Enum.into(context) + + socket = SimpleH2Client.setup_connection(context) + {:ok, send_ctx} = SimpleH2Client.send_simple_headers(socket, 1, :get, "/", context.port) + assert {:ok, 1, false, _headers, recv_ctx} = SimpleH2Client.recv_headers(socket) + assert {:ok, 1, true, "OK"} = SimpleH2Client.recv_body(socket) + + Process.sleep(300) + assert [pid] = connection_pids(context) + assert Process.info(pid, :current_function) == @hibernating + + SimpleH2Client.send_simple_headers(socket, 3, :get, "/", context.port, send_ctx) + assert {:ok, 3, false, _headers, _ctx} = SimpleH2Client.recv_headers(socket, recv_ctx) + assert {:ok, 3, true, "OK"} = SimpleH2Client.recv_body(socket) + end + + test "hibernate on an idle WebSocket, then handle the next frame", context do + context = + context + |> http_server(thousand_island_options: [genserver_options: [hibernate_after: 100]]) + |> Enum.into(context) + + client = SimpleWebSocketClient.tcp_client(context) + SimpleWebSocketClient.http1_handshake(client, HibernationTest.Sock) + + Process.sleep(300) + assert [pid] = connection_pids(context) + assert Process.info(pid, :current_function) == @hibernating + + SimpleWebSocketClient.send_text_frame(client, "still here") + assert {:ok, "still here"} = SimpleWebSocketClient.recv_text_frame(client) + end + + test "do not hibernate when hibernate_after is set to :infinity", context do + context = + context + |> http_server(thousand_island_options: [genserver_options: [hibernate_after: :infinity]]) + |> Enum.into(context) + + client = SimpleHTTP1Client.tcp_client(context) + SimpleHTTP1Client.send(client, "GET", "/", ["host: localhost"]) + assert {:ok, "200 OK", _headers, "OK"} = SimpleHTTP1Client.recv_reply(client) + + Process.sleep(300) + assert [pid] = connection_pids(context) + refute Process.info(pid, :current_function) == @hibernating + end + + @tag :slow + test "hibernate after 15 seconds by default", context do + # The test helpers' 100ms read_timeout would close the connection first + context = + context + |> http_server(thousand_island_options: [read_timeout: 60_000]) + |> Enum.into(context) + + client = SimpleHTTP1Client.tcp_client(context) + SimpleHTTP1Client.send(client, "GET", "/", ["host: localhost"]) + assert {:ok, "200 OK", _headers, "OK"} = SimpleHTTP1Client.recv_reply(client) + + Process.sleep(14_000) + assert [pid] = connection_pids(context) + refute Process.info(pid, :current_function) == @hibernating + + Process.sleep(4_000) + assert Process.info(pid, :current_function) == @hibernating + end + end +end