Repository navigation
Skip response compression for small bodies and incompressible content… - #740
Merged
Merged
Conversation
… types
Bandit compresses every `send_resp` body when the client accepts it, however short the body is
and whatever it contains. Two cases where that is a loss, and this PR stops doing both.
The first is small bodies. A `{"status":"ok"}` of 15 bytes goes out as 35 bytes of gzip, and a
2 byte `OK` as 22, because the gzip container alone is 18 bytes and text only starts to shrink
somewhere in the low hundreds. Beyond the bytes, every one of those responses sets up and tears
down a zlib stream. I measured that at about 7 µs per response on my machine, and in an eprof run
of a keepalive client hitting an 80 byte JSON endpoint the three zlib NIFs were 8% of the server's
time. Over a loopback connection, the same tiny response costs 14 to 27 µs more when the client
sends `accept-encoding` than when it doesn't; with a 300 byte threshold in place that gap drops to
5 to 9 µs. Since Bandit compresses by default and every browser sends `accept-encoding`, every
small API response from a Phoenix app pays this today.
The second is bodies that are already compressed. A JPEG, an MP4, a zip or a woff2 sent through
`send_resp` (image proxies, generated zips and PDFs, `send_download` with binary data) gets gzipped
too. zlib is slow on bytes it cannot compress: 28 ms per MB here, against 3.5 ms per MB for HTML,
and the output is the same size as the input. A 20 MB zip export spends over half a second of CPU
growing slightly.
Two new `http_options`, both with the obvious defaults and both overridable:
`compress_threshold`, the smallest body that gets compressed, defaulting to 300 bytes. That's the
same name and default as Cowboy's. It only applies to `send_resp`, where the length is known;
chunked responses are compressed exactly as before. An empty body is never compressed, whatever
the threshold.
`incompressible_content_types`, a list of media types that are never compressed. The default
covers the usual suspects: the common image, video and audio types, woff and woff2, zip, gzip, zstd,
bz2, xz, 7z and rar, and `application/octet-stream`, on the grounds that an unknown binary is far
more often a file of one of those kinds than text. `image/svg+xml` is deliberately not on it. The
response's `content-type` is compared without its parameters and case-insensitively (RFC9110§8.3.1),
so `image/png; charset=utf-8` and `IMAGE/PNG` both match. Set the option to `[]` to get today's
behaviour back. The octet-stream entry is the one I'd expect a view on; it's easy to drop.
Two smaller things in the same change. `Compression.new/5` now takes the body length rather than
an empty flag, and `send_resp` reuses that length as the content-length when the body goes out
uncompressed, so an iolist body is walked once instead of twice. And the `vary: accept-encoding`
header is still added below the threshold and for excluded types, since the same resource may be
compressed at another size.
Tests are in a new `test/bandit/compression_test.exs`: unit tests of `Compression.new/5` for the
threshold (below, at, configured, zero, streamed) and the content type list (defaults, parameter
and case handling, svg and untyped responses compressed, configured list), plus four requests
through a real HTTP/1 server. Two existing telemetry tests compressed a 2 byte body to check the
compression metrics; they now send a 10 KB body for the same check. Suite, format, credo and
dialyzer are clean.
Owner
|
Nice! I've been wanting to do something like this for ages |
mpettibone
added a commit
to mpettibone/bandit
that referenced
this pull request
Nov 3, 2026
Every request that carries an `accept-encoding` header goes through `Bandit.Compression.negotiate_content_encodings/2` in `Bandit.Adapter.init/5`, before the plug runs and whatever the response turns out to be. The parser there is general on purpose: it handles q-values, the wildcard, odd spacing and odd casing. That generality has a per-request price. `Plug.Conn.Utils.list/1` splits the value, then every coding gets a `String.split`, a `String.trim` and a `String.downcase`, then there is a `List.keyfind` per configured encoding. For the value Chrome sends that adds up to 5 µs per request in my test VM. Now that mtrudel#740 has stopped small responses paying for zlib, that parse is the largest cost left tied to the header: about 8% of a small JSON API request there, spent reaching the same answer every time. Because it is the same answer every time. Nearly all traffic carries one of about five values: browsers send `gzip, deflate, br, zstd` (Chrome, Edge, Firefox) or `gzip, deflate, br` (Safari) over https and `gzip, deflate` over plain http, curl sends `deflate, gzip, br, zstd`, and Go's and the JVM's HTTP clients send a bare `gzip`. None of them vary in spacing or case. So this PR resolves those five values with a pattern match on the exact string and leaves everything else to the parser. ## How it works `@common_accept_encodings` pairs each value with the codings the client is accepting. A `for` over that list generates one `common_negotiation/2` clause per value. The clause also matches the default encodings list as its second argument, and its body is the negotiation result, computed at compile time by filtering the default encodings down to the accepted ones. A hit therefore costs the two option lookups that happened anyway plus one literal match, and returns a literal. Everything else falls through to the parser, which is unchanged apart from moving into `parse_and_negotiate/2`: - a value that isn't in the table, including one of the five with different spacing or case; - `response_encodings` set to anything but the default, since the second argument then doesn't match and the parser runs with the configured list exactly as before; - `compress: false`, where the early return is untouched. ## Why this shape The parser stays the one definition of the negotiation rules. The table is a cache of its answers for the inputs that matter, not a second implementation of them, and the tests hold it to that: for each of the five values, the fast path result must equal what the parser returns for the same value upcased and for the same value with a trailing space, two spellings the table never matches and the parser normalizes to the same thing. If the parser or the default encodings change and the table is left behind, those tests fail. Resolving the table at compile time also means the results follow the platform. On OTP 28 and later, where `:zstd` is available and in the default encodings, `gzip, deflate, br, zstd` resolves to `["zstd", "gzip", "deflate"]`; on earlier releases it resolves to `["gzip", "deflate"]`, the same as the parser produces in each case. I considered a runtime memo of recent values instead. Per process it would be wiped by Bandit's own process dictionary clearing between keepalive requests, and anything longer lived would need state threaded through the handler and a story for invalidation. A fixed table covers the values that account for nearly all requests with no state at all, and adding a value is one line. I kept it to five rather than chasing every client library, because each row is something to keep right, and a value that isn't in the table still works, it just pays the parse. ## Performance Measured against `main` at `4812459` (which includes mtrudel#740) in a 2-core VM at 2.1 GHz with the BEAM pinned to one core running one scheduler, OTP 25.3, Elixir 1.17.3. The absolute numbers are slow and the VM is noisy; the ratios are the point. Cost of `negotiate_content_encodings/2` itself, per call. Each cell is the best of three sessions of three runs of a million calls, which is the usual way to get a stable number for a CPU-bound function on a shared host: | `accept-encoding` value | Sent by | `main` | This PR | | --- | --- | ---: | ---: | | `gzip, deflate, br, zstd` | Chrome, Edge, Firefox over https | 5,053 ns | 70 ns | | `gzip, deflate, br` | Safari over https | 3,887 ns | 69 ns | | `gzip, deflate` | browsers over plain http, Python requests | 2,636 ns | 66 ns | | `deflate, gzip, br, zstd` | curl `--compressed` | 5,305 ns | 63 ns | | `gzip` | Go `net/http`, OkHttp | 1,935 ns | 62 ns | | `gzip;q=0.8, deflate;q=0.5, br` | anything else, through the parser | 6,148 ns | 5,879 ns | | no header | | 10 ns | 10 ns | End to end, HTTP/1 keepalive over loopback with 32 connections, wrk pinned to the other core, 8 second runs, the two builds alternated and compared in pairs. Medians of seven pairs; the range of the paired differences is in the last column because this VM moves a lot between runs: | Request | `main` req/s | This PR req/s | CPU per request, `main` | CPU per request, this PR | Paired change in req/s | | --- | ---: | ---: | ---: | ---: | --- | | 15 byte JSON, `accept-encoding: gzip, deflate, br, zstd` only | 15,936 | 17,035 | 62.0 µs | 56.9 µs | +6.9% (range +4% to +22%) | | 13 byte text, full Chrome header set | 11,870 | 14,375 | 83.3 µs | 68.8 µs | +18% (range -2% to +28%) | | 15 byte JSON, no `accept-encoding` (identical code on both) | 18,641 | 19,846 | 52.9 µs | 49.8 µs | +6.5% (range -8% to +16%) | The last row runs identical code on both builds, so it is the noise floor: paired runs disagreed by up to 16% with nothing changed. Read the other two rows against that. The per-request CPU saving in paired medians was 5 µs for the JSON shape, which is what the micro-benchmark predicts (5 µs parse replaced by 70 ns), and 12 µs for the browser shape, which is more than the parse alone can account for and is best read as noise-inflated. The sound estimate is the arithmetic: 5 µs off a 62 µs request, so around 8% more requests per core for API traffic from browsers, a little less for page loads with their longer header lists, and nothing for clients that don't send the header. The saving applies to HTTP/2 as well, since the negotiation runs in the shared adapter. ## Tests `test/bandit/compression_negotiation_test.exs` is new: the five equivalence tests described above, the expected result for each table value with the platform dependent `zstd` handled, a configured `response_encodings` and `compress: false` taking the general path, a few values outside the table still parsing, and two requests through a real HTTP/1 server with a browser's value and with `gzip`. The tests pass on `main` as well, by design; the change is meant to be invisible, and they are there to keep it that way. Suite, format, credo and dialyzer are clean.
mtrudel
pushed a commit
that referenced
this pull request
Nov 4, 2026
Every request that carries an `accept-encoding` header goes through `Bandit.Compression.negotiate_content_encodings/2` in `Bandit.Adapter.init/5`, before the plug runs and whatever the response turns out to be. The parser there is general on purpose: it handles q-values, the wildcard, odd spacing and odd casing. That generality has a per-request price. `Plug.Conn.Utils.list/1` splits the value, then every coding gets a `String.split`, a `String.trim` and a `String.downcase`, then there is a `List.keyfind` per configured encoding. For the value Chrome sends that adds up to 5 µs per request in my test VM. Now that #740 has stopped small responses paying for zlib, that parse is the largest cost left tied to the header: about 8% of a small JSON API request there, spent reaching the same answer every time. Because it is the same answer every time. Nearly all traffic carries one of about five values: browsers send `gzip, deflate, br, zstd` (Chrome, Edge, Firefox) or `gzip, deflate, br` (Safari) over https and `gzip, deflate` over plain http, curl sends `deflate, gzip, br, zstd`, and Go's and the JVM's HTTP clients send a bare `gzip`. None of them vary in spacing or case. So this PR resolves those five values with a pattern match on the exact string and leaves everything else to the parser. ## How it works `@common_accept_encodings` pairs each value with the codings the client is accepting. A `for` over that list generates one `common_negotiation/2` clause per value. The clause also matches the default encodings list as its second argument, and its body is the negotiation result, computed at compile time by filtering the default encodings down to the accepted ones. A hit therefore costs the two option lookups that happened anyway plus one literal match, and returns a literal. Everything else falls through to the parser, which is unchanged apart from moving into `parse_and_negotiate/2`: - a value that isn't in the table, including one of the five with different spacing or case; - `response_encodings` set to anything but the default, since the second argument then doesn't match and the parser runs with the configured list exactly as before; - `compress: false`, where the early return is untouched. ## Why this shape The parser stays the one definition of the negotiation rules. The table is a cache of its answers for the inputs that matter, not a second implementation of them, and the tests hold it to that: for each of the five values, the fast path result must equal what the parser returns for the same value upcased and for the same value with a trailing space, two spellings the table never matches and the parser normalizes to the same thing. If the parser or the default encodings change and the table is left behind, those tests fail. Resolving the table at compile time also means the results follow the platform. On OTP 28 and later, where `:zstd` is available and in the default encodings, `gzip, deflate, br, zstd` resolves to `["zstd", "gzip", "deflate"]`; on earlier releases it resolves to `["gzip", "deflate"]`, the same as the parser produces in each case. I considered a runtime memo of recent values instead. Per process it would be wiped by Bandit's own process dictionary clearing between keepalive requests, and anything longer lived would need state threaded through the handler and a story for invalidation. A fixed table covers the values that account for nearly all requests with no state at all, and adding a value is one line. I kept it to five rather than chasing every client library, because each row is something to keep right, and a value that isn't in the table still works, it just pays the parse. ## Performance Measured against `main` at `4812459` (which includes #740) in a 2-core VM at 2.1 GHz with the BEAM pinned to one core running one scheduler, OTP 25.3, Elixir 1.17.3. The absolute numbers are slow and the VM is noisy; the ratios are the point. Cost of `negotiate_content_encodings/2` itself, per call. Each cell is the best of three sessions of three runs of a million calls, which is the usual way to get a stable number for a CPU-bound function on a shared host: | `accept-encoding` value | Sent by | `main` | This PR | | --- | --- | ---: | ---: | | `gzip, deflate, br, zstd` | Chrome, Edge, Firefox over https | 5,053 ns | 70 ns | | `gzip, deflate, br` | Safari over https | 3,887 ns | 69 ns | | `gzip, deflate` | browsers over plain http, Python requests | 2,636 ns | 66 ns | | `deflate, gzip, br, zstd` | curl `--compressed` | 5,305 ns | 63 ns | | `gzip` | Go `net/http`, OkHttp | 1,935 ns | 62 ns | | `gzip;q=0.8, deflate;q=0.5, br` | anything else, through the parser | 6,148 ns | 5,879 ns | | no header | | 10 ns | 10 ns | End to end, HTTP/1 keepalive over loopback with 32 connections, wrk pinned to the other core, 8 second runs, the two builds alternated and compared in pairs. Medians of seven pairs; the range of the paired differences is in the last column because this VM moves a lot between runs: | Request | `main` req/s | This PR req/s | CPU per request, `main` | CPU per request, this PR | Paired change in req/s | | --- | ---: | ---: | ---: | ---: | --- | | 15 byte JSON, `accept-encoding: gzip, deflate, br, zstd` only | 15,936 | 17,035 | 62.0 µs | 56.9 µs | +6.9% (range +4% to +22%) | | 13 byte text, full Chrome header set | 11,870 | 14,375 | 83.3 µs | 68.8 µs | +18% (range -2% to +28%) | | 15 byte JSON, no `accept-encoding` (identical code on both) | 18,641 | 19,846 | 52.9 µs | 49.8 µs | +6.5% (range -8% to +16%) | The last row runs identical code on both builds, so it is the noise floor: paired runs disagreed by up to 16% with nothing changed. Read the other two rows against that. The per-request CPU saving in paired medians was 5 µs for the JSON shape, which is what the micro-benchmark predicts (5 µs parse replaced by 70 ns), and 12 µs for the browser shape, which is more than the parse alone can account for and is best read as noise-inflated. The sound estimate is the arithmetic: 5 µs off a 62 µs request, so around 8% more requests per core for API traffic from browsers, a little less for page loads with their longer header lists, and nothing for clients that don't send the header. The saving applies to HTTP/2 as well, since the negotiation runs in the shared adapter. ## Tests `test/bandit/compression_negotiation_test.exs` is new: the five equivalence tests described above, the expected result for each table value with the platform dependent `zstd` handled, a configured `response_encodings` and `compress: false` taking the general path, a few values outside the table still parsing, and two requests through a real HTTP/1 server with a browser's value and with `gzip`. The tests pass on `main` as well, by design; the change is meant to be invisible, and they are there to keep it that way. Suite, format, credo and dialyzer are clean.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Bandit compresses every
send_respbody when the client accepts it, however short the body is and whatever it contains. Two cases where that is a loss, and this PR stops doing both.The first is small bodies. A
{"status":"ok"}of 15 bytes goes out as 35 bytes of gzip, and a 2 byteOKas 22, because the gzip container alone is 18 bytes and text only starts to shrink somewhere in the low hundreds. Beyond the bytes, every one of those responses sets up and tears down a zlib stream. I measured that at about 7 µs per response on my machine, and in an eprof run of a keepalive client hitting an 80 byte JSON endpoint the three zlib NIFs were 8% of the server's time. Over a loopback connection, the same tiny response costs 14 to 27 µs more when the client sendsaccept-encodingthan when it doesn't; with a 300 byte threshold in place that gap drops to 5 to 9 µs. Since Bandit compresses by default and every browser sendsaccept-encoding, every small API response from a Phoenix app pays this today.The second is bodies that are already compressed. A JPEG, an MP4, a zip or a woff2 sent through
send_resp(image proxies, generated zips and PDFs,send_downloadwith binary data) gets gzipped too. zlib is slow on bytes it cannot compress: 28 ms per MB here, against 3.5 ms per MB for HTML, and the output is the same size as the input. A 20 MB zip export spends over half a second of CPU growing slightly.Two new
http_options, both with the obvious defaults and both overridable:compress_threshold, the smallest body that gets compressed, defaulting to 300 bytes. That's the same name and default as Cowboy's. It only applies tosend_resp, where the length is known; chunked responses are compressed exactly as before. An empty body is never compressed, whatever the threshold.incompressible_content_types, a list of media types that are never compressed. The default covers the usual suspects: the common image, video and audio types, woff and woff2, zip, gzip, zstd, bz2, xz, 7z and rar, andapplication/octet-stream, on the grounds that an unknown binary is far more often a file of one of those kinds than text.image/svg+xmlis deliberately not on it. The response'scontent-typeis compared without its parameters and case-insensitively (RFC9110§8.3.1), soimage/png; charset=utf-8andIMAGE/PNGboth match. Set the option to[]to get today's behaviour back. The octet-stream entry is the one I'd expect a view on; it's easy to drop.Two smaller things in the same change.
Compression.new/5now takes the body length rather than an empty flag, andsend_respreuses that length as the content-length when the body goes out uncompressed, so an iolist body is walked once instead of twice. And thevary: accept-encodingheader is still added below the threshold and for excluded types, since the same resource may be compressed at another size.Tests are in a new
test/bandit/compression_test.exs: unit tests ofCompression.new/5for the threshold (below, at, configured, zero, streamed) and the content type list (defaults, parameter and case handling, svg and untyped responses compressed, configured list), plus four requests through a real HTTP/1 server. Two existing telemetry tests compressed a 2 byte body to check the compression metrics; they now send a 10 KB body for the same check. Suite, format, credo and dialyzer are clean.