Add SSL_CTX_add_client_custom_ext (OpenSSL-compat client custom extensions)#10625
Draft
julek-wolfssl wants to merge 3 commits into
Draft
Add SSL_CTX_add_client_custom_ext (OpenSSL-compat client custom extensions)#10625julek-wolfssl wants to merge 3 commits into
julek-wolfssl wants to merge 3 commits into
Conversation
…sions)
Implements the OpenSSL-compatible legacy client custom extension API for
TLS 1.2 and below. Application-defined extensions carry arbitrary IANA
types, which cannot live in the TLSX list (it keys every extension on a
fixed 72-bit semaphore index that an arbitrary type would overrun), so
they are kept in a separate list on the WOLFSSL_CTX and processed
alongside the unknown-extension handling, mirroring OpenSSL's custext.
- wolfSSL_CTX_add_client_custom_ext() registers a {add,free,parse} method
set with validation (ext_type <= 0xffff, no free_cb without add_cb, not
an internally handled type, no duplicates).
- ClientHello: TLSX_GetRequestSize runs each add_cb, serializes the wire
bytes into a per-connection cache and calls free_cb; TLSX_WriteRequest
copies them out. Honors add_cb returns 1/0/-1 (+alert).
- ServerHello: the unknown-type case in TLSX_Parse dispatches to parse_cb
(<=0 aborts with the requested alert). Gated to non-TLS-1.3 so TLS 1.3
keeps RFC 8446 unsupported_extension behavior.
- Lists/buffers freed in SSL_CtxResourceFree and SSL_ResourceFree.
- openssl/ssl.h maps SSL_CTX_add_client_custom_ext and the custom_ext_*_cb
typedefs.
Adds unit tests covering registration validation, a TLS 1.2 memio
handshake (add/free callbacks balance) and ServerHello parse dispatch.
Builds with and without OPENSSL_EXTRA; full API suite passes.
Address three OpenSSL-compatibility gaps in the client custom extension handling, matching ssl/statem/extensions_cust.c and extensions.c: 1. Unsolicited responses: track the custom extension types actually emitted in the ClientHello (ssl->customExtSent, built alongside customExtData). On ServerHello, a custom extension whose type was not sent is rejected with unsupported_extension, mirroring OpenSSL's SSL_EXT_FLAG_SENT check. add_cb declining to send a type for a given handshake now correctly makes a server echo unsolicited. 2. Resumption: OpenSSL registers the legacy API with SSL_EXT_IGNORE_ON_RESUMPTION and skips parsing on a resumed handshake. Suppress custom-extension parsing when ssl->options.resuming is set so a server echo is silently ignored. (Evaluated at ServerHello-parse time, i.e. the optimistic resumption-attempt state, since wolfSSL finalizes resumption later in CompleteServerHello. The send path is left intact, matching OpenSSL where s->hit is 0 at ClientHello.) 3. Duplicates: the semaphore-based duplicate detection cannot cover arbitrary custom types. Scan the already-parsed portion of the message for an earlier extension of the same registered custom type and abort with DUPLICATE_TLS_EXT_E, matching OpenSSL giving each custom extension its own slot. Adds tests for unsolicited rejection, duplicate rejection, and resumption ignore; updates the parse test to emit first. Builds with and without OPENSSL_EXTRA; full API suite passes.
…hods Follow-up addressing two issues from review of the previous fix: 1. Resumption (was: parse skipped on optimistic resuming flag, then on a cached-ticket heuristic). At ServerHello-parse time ssl->options.resuming only means the client *attempted* resumption, and a cached session ticket does not mean the server agreed to resume. Gate the ignore on the actual server-confirmed signal: resumption was attempted AND the server echoed our session ID back (RFC 5246, and RFC 5077 for tickets with a non-empty session ID). A server that falls back to a full handshake -- whether plain or after a ticket attempt -- does not echo our session ID, so its extensions are still parsed/validated and an unsolicited one is rejected. 2. Flexible client methods (was: API disabled when max version is TLS 1.3). The send path gated on ssl->version < TLS 1.3, but before the handshake that field holds the method's max version, so wolfSSLv23_client_method() / wolfTLS_client_method() (initialized to TLS 1.3) never offered the extension even when negotiating TLS 1.2. The ClientHello now always offers the extension regardless of max version, matching OpenSSL (is_tls13 is false while building the ClientHello). The negotiated-version restriction remains enforced on the parse side, where ssl->version is the negotiated version (TLS 1.3 ServerHellos are routed to DoTls13ServerHello before this code). Adds tests: flexible-method (v23 client -> TLS 1.2) and TLS 1.3 handshakes both offer the extension without breaking; resumption-fallback and ticket-fallback reject an unsolicited extension; resumption-ignored sets up a matching session ID. Builds with and without OPENSSL_EXTRA; full API suite passes.
Contributor
There was a problem hiding this comment.
Pull request overview
Note
Copilot was unable to run its full agentic suite in this review.
Implements OpenSSL-compatible legacy client custom extensions (SSL_CTX_add_client_custom_ext) for TLS 1.2 and below, including registration, ClientHello serialization, ServerHello parse dispatch, and related validation behaviors.
Changes:
- Added public API + OpenSSL-compat typedef/macro mappings for client custom extensions.
- Introduced internal storage and processing for app-defined custom extensions alongside unknown-extension handling.
- Added unit tests for registration validation, handshake behavior across TLS 1.2/1.3, parse dispatch, and unsolicited/duplicate/resumption handling.
Reviewed changes
Copilot reviewed 8 out of 8 changed files in this pull request and generated 5 comments.
Show a summary per file
| File | Description |
|---|---|
| wolfssl/ssl.h | Adds public callback typedefs and wolfSSL_CTX_add_client_custom_ext API declaration. |
| wolfssl/openssl/ssl.h | Maps OpenSSL names (custom_ext_*_cb, SSL_CTX_add_client_custom_ext) onto wolfSSL API. |
| wolfssl/internal.h | Introduces WOLFSSL_CustomExt and adds per-CTX/per-SSL fields for custom ext wire bytes and sent-type tracking. |
| src/tls.c | Implements custom ext registration, ClientHello build/write integration, parse dispatch, and duplicate detection. |
| src/internal.c | Frees custom ext lists/buffers during ctx/ssl teardown. |
| tests/api/test_tls_ext.h | Exposes new custom-ext test declarations. |
| tests/api/test_tls_ext.c | Adds comprehensive unit tests for custom-ext behaviors. |
| tests/api.c | Registers new test cases in the test runner. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Comment on lines
+5019
to
+5045
| #ifdef OPENSSL_EXTRA | ||
| /* OpenSSL-compatible application-defined ("custom") TLS extension callbacks. | ||
| * | ||
| * add_cb is invoked while building an outgoing message. On return: | ||
| * 1 - include the extension; out and outlen describe the data to send. | ||
| * 0 - omit the extension. | ||
| * <0 - fatal error; *al holds the TLS alert to send. | ||
| * If add_cb is NULL a zero-length extension is added to the ClientHello. | ||
| * | ||
| * free_cb (if set) is called after the data returned by add_cb has been | ||
| * copied into the message, to release any allocation it made. | ||
| * | ||
| * parse_cb is invoked for a received extension of the registered type. On | ||
| * return 1 the handshake continues; on return <=0 it is aborted with the | ||
| * alert placed in *al. */ | ||
| typedef int (*wolfSSL_custom_ext_add_cb)(WOLFSSL* s, unsigned int ext_type, | ||
| const unsigned char** out, size_t* outlen, int* al, void* add_arg); | ||
| typedef void (*wolfSSL_custom_ext_free_cb)(WOLFSSL* s, unsigned int ext_type, | ||
| const unsigned char* out, void* add_arg); | ||
| typedef int (*wolfSSL_custom_ext_parse_cb)(WOLFSSL* s, unsigned int ext_type, | ||
| const unsigned char* in, size_t inlen, int* al, void* parse_arg); | ||
|
|
||
| WOLFSSL_API int wolfSSL_CTX_add_client_custom_ext(WOLFSSL_CTX* ctx, | ||
| unsigned int ext_type, wolfSSL_custom_ext_add_cb add_cb, | ||
| wolfSSL_custom_ext_free_cb free_cb, void* add_arg, | ||
| wolfSSL_custom_ext_parse_cb parse_cb, void* parse_arg); | ||
| #endif /* OPENSSL_EXTRA */ |
Comment on lines
+16785
to
+16795
| /* Frees a list of registered custom extension methods. */ | ||
| void TLSX_CustomExt_FreeAll(WOLFSSL_CustomExt* list, void* heap) | ||
| { | ||
| WOLFSSL_CustomExt* meth; | ||
|
|
||
| while ((meth = list) != NULL) { | ||
| list = meth->next; | ||
| XFREE(meth, heap, DYNAMIC_TYPE_TLSX); | ||
| } | ||
| (void)heap; | ||
| } |
Comment on lines
+18019
to
+18030
| else if (type > 62 && TLSX_CustomExt_IsRegistered(ssl, type)) { | ||
| word16 scan = 0; | ||
| word16 upto = (word16)(offset - HELLO_EXT_TYPE_SZ - OPAQUE16_LEN); | ||
| while (scan + HELLO_EXT_TYPE_SZ + OPAQUE16_LEN <= upto) { | ||
| word16 sT, sS; | ||
| ato16(input + scan, &sT); | ||
| ato16(input + scan + HELLO_EXT_TYPE_SZ, &sS); | ||
| if (sT == type) | ||
| return DUPLICATE_TLS_EXT_E; | ||
| scan = (word16)(scan + HELLO_EXT_TYPE_SZ + OPAQUE16_LEN + sS); | ||
| } | ||
| } |
Comment on lines
+1016
to
+1018
| #define TEST_CUSTOM_EXT_TYPE 0x4242 | ||
| #define TEST_CUSTOM_ADD_ARG ((void*)0x1234) | ||
| #define TEST_CUSTOM_PARSE_ARG ((void*)0x5678) |
| int test_wolfSSL_custom_ext_resumption_fallback(void); | ||
| int test_wolfSSL_custom_ext_ticket_fallback(void); | ||
|
|
||
| #endif /* TESTS_API_TEST_TLS_EMS_H */ |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Implements the OpenSSL-compatible legacy client custom extension API
(
SSL_CTX_add_client_custom_ext) for TLS 1.2 and below.Application-defined extensions carry arbitrary IANA types, which cannot live
in the TLSX list (it keys every extension on a fixed 72-bit semaphore index
that an arbitrary type would overrun), so they are kept in a separate list on
the
WOLFSSL_CTXand processed alongside the unknown-extension handling,mirroring OpenSSL's custext.
Commits
Add
SSL_CTX_add_client_custom_ext— registers a{add,free,parse}method set with validation. ClientHello runs each
add_cband serializesthe wire bytes; ServerHello dispatches the unknown-type case to
parse_cb.Gated to non-TLS-1.3 so TLS 1.3 keeps RFC 8446
unsupported_extensionbehavior. Lists/buffers freed in
SSL_CtxResourceFree/SSL_ResourceFree;openssl/ssl.hmaps the API andcustom_ext_*_cbtypedefs.Reject unsolicited/duplicate exts and ignore on resumption — tracks the
custom types emitted in the ClientHello and rejects a server echo of an
unsent type (
unsupported_extension), suppresses parsing on a resumedhandshake (
SSL_EXT_IGNORE_ON_RESUMPTION), and scans for duplicate customtypes (
DUPLICATE_TLS_EXT_E), matchingextensions_cust.c/extensions.c.Confirmed-resumption gate and flexible client methods — gates the
resumption ignore on the server-confirmed signal (resumption attempted AND
the server echoed our session ID), and always offers the extension in the
ClientHello regardless of the method's max version, so
wolfSSLv23_client_method()/wolfTLS_client_method()offer it whennegotiating TLS 1.2. The negotiated-version restriction remains enforced on
the parse side.
Adds unit tests covering registration validation, TLS 1.2/1.3 and
flexible-method handshakes (add/free callbacks balance), ServerHello parse
dispatch, and unsolicited/duplicate/resumption handling. Builds with and
without
OPENSSL_EXTRA; full API suite passes.