Native JA4-TLS from the ClientHello + Go 1.24 - #22
Merged
Conversation
…1.24 Unblocks the half of PRD workstream D that was not waiting on the enrichment migration. Go 1.24 is a hard prerequisite: crypto/tls only began exposing the ClientHello extension list in that release (ClientHelloInfo.Extensions, golang/go#32936, verified against GOROOT/api/go1.24.txt), and without it the extension hash cannot be computed and JA4 is not derivable from the stdlib. Five version pins moved together, not the three the PRD listed — server-go/go.mod, server-go/Dockerfile, docker/Dockerfile (the CI publish image, missing it broke the v1.15.0 publish), plus fcaptcha-cloud's deploy/Dockerfile and its vendored server/go.mod. ## Scope, and a licensing line JA4 (TLS) only. It is BSD-3-Clause with an explicit patent disclaimer, which is what makes it safe in an MIT project people deploy commercially. JA4H, JA4T, JA4L, JA4S, JA4X and JA4SSH are FoxIO License 1.1 — non-commercial, patent- pending, GPL-incompatible — and a test now fails if one appears, pointing at the reason rather than leaving the next person to rediscover it. ## Where it works Only where this process terminates TLS, since the ClientHello is consumed by whoever completes the handshake. Set FCAPTCHA_TLS_CERT and FCAPTCHA_TLS_KEY to serve HTTPS directly; behind Railway, Cloudflare or nginx the TRUSTED_JA4_HEADERS path is unchanged and remains the only source. The startup log now says which of the two is in effect, or that neither is. When both are available the native fingerprint wins: it was derived here and cannot be asserted by a client or a misconfigured proxy, whereas a header is only as good as whoever set it. ## A mislabelled placeholder that would have flagged every Chrome user knownBotJA4Hashes carried a commented example: "t13d1516h2_8daaf6152771_02713d6af862": "Go default stdlib TLS" That is the canonical example from the JA4 spec, and it is Chrome. Measured with this implementation against a real TLS listener: Chromium 141 t13i1515h2_8daaf6152771_806a8c22fdea Go stdlib t13i131000_f57a46bbacb6_e5728521abd4 curl t13i4906h2_0d8feac7bc37_7395dae3b2f3 node https t13i521000_b262b3658495_8e6e362c5eac python urllib t13i171000_ab0a1bf427ad_8e6e362c5eac The middle section hashes the cipher list, i.e. the TLS stack. 8daaf6152771 is Chrome's; Go's is f57a46bbacb6. Uncommenting that line would have scored every Chrome visitor as automation at 0.8/0.9. Replaced with the measured table and the reason the map ships empty. ## What this does and does not detect today Nothing, yet. The map CheckJA4Fingerprint consults is empty, deliberately: a static hash list is defeated by rotating a fingerprint, and the readings above already drift from the spec's own example through version churn alone. Populating it with values nobody here has observed would buy false-positive risk and no detection. A test records that it ships empty so filling it is a deliberate act. The aggregate JA4 Signals model the PRD describes is not built either — it wants population-scale ratios that a single self-hosted instance does not have. What ships is the correct, tested computation and the plumbing, so a fingerprint is available to correlate (workstream F) and to populate a list from real abuse. Tests: 12 covering GREASE exclusion, the count-includes-but-hash-excludes asymmetry for SNI/ALPN, sig-algs deliberately unsorted, order insensitivity, native-over-header precedence, a real TLS handshake end to end, and the licensing boundary. Go E2E unchanged at 89/93.
The Go server was fixed in the previous commit; the same wrong example was copied verbatim into server-node/detection.js and server-python/detection.py. All three claimed t13d1516h2_8daaf6152771_02713d6af862 was "Go default stdlib TLS". It is the canonical example from the JA4 specification, and it is Chrome — 8daaf6152771 hashes Chrome's cipher list, while Go's stdlib client hashes to f57a46bbacb6. Inert while commented out, but one uncomment away from scoring every Chrome visitor as automation at 0.8/0.9, in whichever server it happened to be uncommented in. Each map now ships explicitly empty with the measured table alongside it, and the reason a static hash list is the wrong shape for this: it is defeated by rotating a fingerprint, and the values drift with every browser release.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Unblocks the half of PRD workstream D that wasn't waiting on the enrichment migration.
Go 1.24 is a hard prerequisite, and I verified it rather than taking the PRD's word:
crypto/tlsonly began exposing the ClientHello extension list in 1.24 (ClientHelloInfo.Extensions, golang/go#32936 — confirmed inGOROOT/api/go1.24.txt). Without it the extension hash can't be computed and JA4 isn't derivable from the stdlib at all.Five version pins moved, not the three the PRD listed —
server-go/go.mod,server-go/Dockerfile,docker/Dockerfile(the CI publish image; missing it broke the v1.15.0 publish), plusfcaptcha-cloud'sdeploy/Dockerfileand its vendoredserver/go.mod.Licensing
JA4 (TLS) only. It's BSD-3-Clause with an explicit patent disclaimer. JA4H/JA4T/JA4L/JA4S/JA4X/JA4SSH are FoxIO License 1.1 — non-commercial, patent-pending, GPL-incompatible — and unusable in an MIT project deployed commercially. A test now fails if one appears, with the reason in the failure message.
Where it works
Only where this process terminates TLS, since the ClientHello is consumed by whoever completes the handshake.
FCAPTCHA_TLS_CERT/FCAPTCHA_TLS_KEYserve HTTPS directly; behind Railway/Cloudflare/nginx theTRUSTED_JA4_HEADERSpath is unchanged and remains the only source. The startup log says which is in effect.When both exist the native fingerprint wins — it was derived here and can't be asserted by a client or a misconfigured proxy.
A mislabelled placeholder that would have flagged every Chrome user
knownBotJA4Hashescarried a commented example:That's the canonical example from the JA4 spec, and it's Chrome. Measured with this implementation against a real TLS listener:
t13i1515h2_8daaf6152771_806a8c22fdeat13i131000_f57a46bbacb6_e5728521abd4t13i4906h2_0d8feac7bc37_7395dae3b2f3t13i521000_b262b3658495_8e6e362c5eact13i171000_ab0a1bf427ad_8e6e362c5eacThe middle section hashes the cipher list — the TLS stack itself.
8daaf6152771is Chrome's; Go's isf57a46bbacb6. Uncommenting that line would have scored every Chrome visitor as automation at 0.8/0.9.What this detects today: nothing, deliberately
The map
CheckJA4Fingerprintconsults ships empty. A static hash list is defeated by rotating a fingerprint, and the readings above already drift from the spec's own example through version churn alone — populating it with values nobody here has observed buys false-positive risk and no detection. A test records the empty state so filling it is a deliberate act.The aggregate JA4 Signals model the PRD describes isn't built either: it wants population-scale ratios a single self-hosted instance doesn't have. What ships is the correct computation and the plumbing, so a fingerprint is available to correlate (workstream F) and to populate a list from real observed abuse.
Tests
12 new, covering GREASE exclusion, the count-includes-but-hash-excludes asymmetry for SNI/ALPN, signature algorithms deliberately unsorted, order insensitivity, native-over-header precedence, a real TLS handshake end to end, and the licensing boundary. Go E2E unchanged at 89/93.