From e5cd38ef004049bc12b5d4896b1e68a1187e59eb Mon Sep 17 00:00:00 2001 From: Hugh Nimmo-Smith Date: Mon, 16 Sep 2024 11:57:19 +0100 Subject: [PATCH 001/122] Placeholder --- proposals/xxxx-matrixrtc-livekit.md | 110 ++++++++++++++++++++++++++++ 1 file changed, 110 insertions(+) create mode 100644 proposals/xxxx-matrixrtc-livekit.md diff --git a/proposals/xxxx-matrixrtc-livekit.md b/proposals/xxxx-matrixrtc-livekit.md new file mode 100644 index 00000000000..30d909e434d --- /dev/null +++ b/proposals/xxxx-matrixrtc-livekit.md @@ -0,0 +1,110 @@ +# MSC0000: Template for new MSCs + +*Note: Text written in italics represents notes about the section or proposal process. This document +serves as an example of what a proposal could look like (in this case, a proposal to have a template) +and should be used where possible.* + +*In this first section, be sure to cover your problem and a broad overview of the solution. Covering +related details, such as the expected impact, can also be a good idea. The example in this document +says that we're missing a template and that things are confusing and goes on to say the solution is +a template. There's no major expected impact in this proposal, so it doesn't list one. If your proposal +was more invasive (such as proposing a change to how servers discover each other) then that would be +a good thing to list here.* + +*If you're having troubles coming up with a description, a good question to ask is "how +does this proposal improve Matrix?" - the answer could reveal a small impact, and that is okay.* + +There can never be enough templates in the world, and MSCs shouldn't be any different. The level +of detail expected of proposals can be unclear - this is what this example proposal (which doubles +as a template itself) aims to resolve. + + +## Proposal + +*Here is where you'll reinforce your position from the introduction in more detail, as well as cover +the technical points of your proposal. Including rationale for your proposed solution and detailing +why parts are important helps reviewers understand the problem at hand. Not including enough detail +can result in people guessing, leading to confusing arguments in the comments section. The example +here covers why templates are important again, giving a stronger argument as to why we should have +a template. Afterwards, it goes on to cover the specifics of what the template could look like.* + +Having a default template that everyone can use is important. Without a template, proposals would be +all over the place and the minimum amount of detail may be left out. Introducing a template to the +proposal process helps ensure that some amount of consistency is present across multiple proposals, +even if each author decides to abandon the template. + +The default template should be a markdown document because the MSC process requires authors to write +a proposal in markdown. Using other formats wouldn't make much sense because that would prevent authors +from copy/pasting the template. + +The template should have the following sections: + +* **Introduction** - This should cover the primary problem and broad description of the solution. +* **Proposal** - The gory details of the proposal. +* **Potential issues** - This is where problems with the proposal would be listed, such as changes + that are not backwards compatible. +* **Alternatives** - This section lists alternative solutions to the same + problem which have been considered and dismsissed. +* **Security considerations** - Discussion of what steps were taken to avoid security issues in the + future and any potential risks in the proposal. + +Furthermore, the template should not be required to be followed. However it is strongly recommended to +maintain some sense of consistency between proposals. + + +## Potential issues + +*Not all proposals are perfect. Sometimes there's a known disadvantage to implementing the proposal, +and they should be documented here. There should be some explanation for why the disadvantage is +acceptable, however - just like in this example.* + +Someone is going to have to spend the time to figure out what the template should actually have in it. +It could be a document with just a few headers or a supplementary document to the process explanation, +however more detail should be included. A template that actually proposes something should be considered +because it not only gives an opportunity to show what a basic proposal looks like, it also means that +explanations for each section can be described. Spending the time to work out the content of the template +is beneficial and not considered a significant problem because it will lead to a document that everyone +can follow. + + +## Alternatives + +*This is where alternative solutions could be listed. There's almost always another way to do things +and this section gives you the opportunity to highlight why those ways are not as desirable. The +argument made in this example is that all of the text provided by the template could be integrated +into the proposals introduction, although with some risk of losing clarity.* + +Instead of adding a template to the repository, the assistance it provides could be integrated into +the proposal process itself. There is an argument to be had that the proposal process should be as +descriptive as possible, although having even more detail in the proposals introduction could lead to +some confusion or lack of understanding. Not to mention if the document is too large then potential +authors could be scared off as the process suddenly looks a lot more complicated than it is. For those +reasons, this proposal does not consider integrating the template in the proposals introduction a good +idea. + + +## Security considerations + +*Some proposals may have some security aspect to them that was addressed in the proposed solution. This +section is a great place to outline some of the security-sensitive components of your proposal, such as +why a particular approach was (or wasn't) taken. The example here is a bit of a stretch and unlikely to +actually be worthwhile of including in a proposal, but it is generally a good idea to list these kinds +of concerns where possible.* + +By having a template available, people would know what the desired detail for a proposal is. This is not +considered a risk because it is important that people understand the proposal process from start to end. + +## Unstable prefix + +*If a proposal is implemented before it is included in the spec, then implementers must ensure that the +implementation is compatible with the final version that lands in the spec. This generally means that +experimental implementations should use `/unstable` endpoints, and use vendor prefixes where necessary. +For more information, see [MSC2324](https://github.com/matrix-org/matrix-doc/pull/2324). This section +should be used to document things such as what endpoints and names are being used while the feature is +in development, the name of the unstable feature flag to use to detect support for the feature, or what +migration steps are needed to switch to newer versions of the proposal.* + +## Dependencies + +This MSC builds on MSCxxxx, MSCyyyy and MSCzzzz (which at the time of writing have not yet been accepted +into the spec). From 9fcfe3e112645e422c403676724f172eb526f99d Mon Sep 17 00:00:00 2001 From: Hugh Nimmo-Smith Date: Mon, 16 Sep 2024 11:58:47 +0100 Subject: [PATCH 002/122] Allocate MSC number --- .../{xxxx-matrixrtc-livekit.md => 4195-matrixrtc-livekit.md} | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) rename proposals/{xxxx-matrixrtc-livekit.md => 4195-matrixrtc-livekit.md} (99%) diff --git a/proposals/xxxx-matrixrtc-livekit.md b/proposals/4195-matrixrtc-livekit.md similarity index 99% rename from proposals/xxxx-matrixrtc-livekit.md rename to proposals/4195-matrixrtc-livekit.md index 30d909e434d..79640453a04 100644 --- a/proposals/xxxx-matrixrtc-livekit.md +++ b/proposals/4195-matrixrtc-livekit.md @@ -1,4 +1,4 @@ -# MSC0000: Template for new MSCs +# MSC4195: MatrixRTC using LiveKit *Note: Text written in italics represents notes about the section or proposal process. This document serves as an example of what a proposal could look like (in this case, a proposal to have a template) From 60727441e60fc2057ec4f19c8644f47484428124 Mon Sep 17 00:00:00 2001 From: Hugh Nimmo-Smith Date: Tue, 17 Dec 2024 07:05:53 +0000 Subject: [PATCH 003/122] Latest --- proposals/4195-matrixrtc-livekit.md | 260 +++++++++++++++++++--------- 1 file changed, 181 insertions(+), 79 deletions(-) diff --git a/proposals/4195-matrixrtc-livekit.md b/proposals/4195-matrixrtc-livekit.md index 79640453a04..6352e91a3e0 100644 --- a/proposals/4195-matrixrtc-livekit.md +++ b/proposals/4195-matrixrtc-livekit.md @@ -1,110 +1,212 @@ -# MSC4195: MatrixRTC using LiveKit +# MSC4195: MatrixRTC using LiveKit backend -*Note: Text written in italics represents notes about the section or proposal process. This document -serves as an example of what a proposal could look like (in this case, a proposal to have a template) -and should be used where possible.* +This proposal defines a new [MSC4143](https://github.com/matrix-org/matrix-spec-proposals/pull/4143) compliant +RTC backend Focus type utilising [LiveKit](https://github.com/livekit/livekit) SFUs. -*In this first section, be sure to cover your problem and a broad overview of the solution. Covering -related details, such as the expected impact, can also be a good idea. The example in this document -says that we're missing a template and that things are confusing and goes on to say the solution is -a template. There's no major expected impact in this proposal, so it doesn't list one. If your proposal -was more invasive (such as proposing a change to how servers discover each other) then that would be -a good thing to list here.* +## Proposal -*If you're having troubles coming up with a description, a good question to ask is "how -does this proposal improve Matrix?" - the answer could reveal a small impact, and that is okay.* +The `type` of the Focus is `livekit`. -There can never be enough templates in the world, and MSCs shouldn't be any different. The level -of detail expected of proposals can be unclear - this is what this example proposal (which doubles -as a template itself) aims to resolve. +We introduce a two sub-types of LiveKit Focus: +- LiveKit SFU Focus - a LiveKit SFU backend infrastructure. +- LiveKit oldest membership Focus selection algorithm - an algorithm to select a communal LiveKit SFU based on the age of the session participants. -## Proposal +### LiveKit SFU Focus -*Here is where you'll reinforce your position from the introduction in more detail, as well as cover -the technical points of your proposal. Including rationale for your proposed solution and detailing -why parts are important helps reviewers understand the problem at hand. Not including enough detail -can result in people guessing, leading to confusing arguments in the comments section. The example -here covers why templates are important again, giving a stronger argument as to why we should have -a template. Afterwards, it goes on to cover the specifics of what the template could look like.* +A backend LiveKit SFU Focus is represented in these places: -Having a default template that everyone can use is important. Without a template, proposals would be -all over the place and the minimum amount of detail may be left out. Introducing a template to the -proposal process helps ensure that some amount of consistency is present across multiple proposals, -even if each author decides to abandon the template. +- the `m.rtc_foci` entry of the `.well-known/matrix/client` +- the `foci_preferred` of the `m.rtc.member` state event +- the `focus_active` of the `m.rtc.member` state event. TODO: is this correct? -The default template should be a markdown document because the MSC process requires authors to write -a proposal in markdown. Using other formats wouldn't make much sense because that would prevent authors -from copy/pasting the template. +It is represented as a JSON object with the following fields: -The template should have the following sections: +- `type` required string - `livekit` +- `livekit_service_url` required string - The URL of the LiveKit MatrixRTC backend to use for the session. -* **Introduction** - This should cover the primary problem and broad description of the solution. -* **Proposal** - The gory details of the proposal. -* **Potential issues** - This is where problems with the proposal would be listed, such as changes - that are not backwards compatible. -* **Alternatives** - This section lists alternative solutions to the same - problem which have been considered and dismsissed. -* **Security considerations** - Discussion of what steps were taken to avoid security issues in the - future and any potential risks in the proposal. +An example within a `.well-known/matrix/client`: -Furthermore, the template should not be required to be followed. However it is strongly recommended to -maintain some sense of consistency between proposals. +```json5 +{ + // rest of the .well-known/matrix/client content + "m.rtc_foci": [ + { + "type": "livekit", + "livekit_service_url": "https://livekit-jwt.call.element.dev" + } + ] +} +``` +An example within the `foci_preferred` of the `m.rtc.member` state event: -## Potential issues +```json5 +{ + // rest of the m.rtc.member event + "foci_preferred": [ + { + "type": "livekit", + "livekit_service_url": "https://livekit-jwt.call.element.dev", + } + ] +} +``` -*Not all proposals are perfect. Sometimes there's a known disadvantage to implementing the proposal, -and they should be documented here. There should be some explanation for why the disadvantage is -acceptable, however - just like in this example.* +### LiveKit oldest membership Focus selection algorithm -Someone is going to have to spend the time to figure out what the template should actually have in it. -It could be a document with just a few headers or a supplementary document to the process explanation, -however more detail should be included. A template that actually proposes something should be considered -because it not only gives an opportunity to show what a basic proposal looks like, it also means that -explanations for each section can be described. Spending the time to work out the content of the template -is beneficial and not considered a significant problem because it will lead to a document that everyone -can follow. +We define a new Focus that uses the age of the session participant `m.rtc.member` to determine the Focus to use. +This can be used in the `focus_active` of the `m.rtc.member` state event. -## Alternatives +The Focus is represented as follows: + +- `type` required string - `livekit` +- `focus_selection` required string - `oldest_membership` + +### SFU authentication with LiveKit + +LiveKit SFUs requires a JWT `access_token` to be provided when [connecting to the WebSocket](https://docs.livekit.io/reference/internals/client-protocol/#WebSocket-Parameters). +We standardise the method by which the LiveKit JWT token is obtained by a MatrixRTC application. + +Prerequisites: + +- the `livekit_service_url` for the MatrixRTC backend has been discovered from one of the methods above +- the Matrix client has obtained an OpenID Token from the [Client-Server API](https://spec.matrix.org/v1.11/client-server-api/#openid). + +The JWT token is obtained by making a `POST` request to the `/sfu/get` endpoint of the LiveKit service. + +The `Content-Type` of the request is `application/json` and the body JSON body contains the following fields: + +- `room_id` required string: the room ID of the Matrix room where the `m.rtc.member` event state key is present. +- `session` required object: the contents of the `session` from the `m.rtc.member` event. +- `openid_token` required object: The verbatim OpenID token response obtained from the + [Client-Server API](https://spec.matrix.org/v1.11/client-server-api/#post_matrixclientv3useruseridopenidrequest_token). +- `member` required object: The contents of the `member` from the `m.rtc.member` event. + +If the request is successful an HTTP `200` response is returned with `Content-Type` `application/json` and the body contains: + +- `jwt` string: The JWT token to use for authentication with the SFU. +- `url` string: The URL of the LiveKit SFU to use for the session. + +The LiveKit JWT should have permissions as defined below. + +An example request where `livekit_service_url` is `https://livekit-jwt.call.element.dev/xyz`: + +```http +POST /xyz/sfu/get HTTP/1.1 +Host: livekit-jwt.call.element.dev +Content-Type: application/json + +{ + "room_id": "!tDLCaLXijNtYcJZEey:element.io", + "session": { + "application": "m.call", + "call_id": "" + }, + "openid_token": { + "access_token": "FPkexLLvKbAHKclQhpvgfWxx", + "expires_in": 3600, + "matrix_server_name": "call.ems.host", + "token_type": "Bearer" + }, + "member": { + "id": "xyzABCDEF10123", + "device_id": "DEVICEID", + "user_id": "@user:matrix.domain" + } +} +``` + +An example response: + +```http +HTTP/1.1 200 OK +Content-Type: application/json + +{ + "jwt": "thejwt", + "url": "wss://livekit-jwt.call.element.dev/rtc" +} +``` -*This is where alternative solutions could be listed. There's almost always another way to do things -and this section gives you the opportunity to highlight why those ways are not as desirable. The -argument made in this example is that all of the text provided by the template could be integrated -into the proposals introduction, although with some risk of losing clarity.* +### Pseudonymous LiveKit participant identity -Instead of adding a template to the repository, the assistance it provides could be integrated into -the proposal process itself. There is an argument to be had that the proposal process should be as -descriptive as possible, although having even more detail in the proposals introduction could lead to -some confusion or lack of understanding. Not to mention if the document is too large then potential -authors could be scared off as the process suddenly looks a lot more complicated than it is. For those -reasons, this proposal does not consider integrating the template in the proposals introduction a good -idea. +We use a pseudonymous LiveKit participant identity for privacy so that the Matrix user ID is not exposed to the LiveKit SFU backend. +The pseudonymous LiveKit participant identity is calculated as the SHA-256 hash of the concatenation of: + +- the Matrix user ID +- `|` +- `member`.`device_id` +- `|` +- `member`.`id`. + +e.g. `SHA256(@user:matrix.domain|DEVICEID|xyzABCDEF10123)` + +### LiveKit JWT permission grants + +As well as being a valid [LiveKit JWT](https://docs.livekit.io/home/get-started/authentication/) the following constraints +are applied: + +- `sub`: This is the pseudonymous LiveKit participant identity as described above. +- `video`.`room`: The name of the room should be the unique for the given `room_id` and `session` inputs. It is opaque to the MatrixRTC application. + +The permissions should be sufficient to allow the MatrixRTC application to join the room. + +For example: + +```json +{ + "exp": 1726764439, + "iss": "API2bYPYMoVqjcE", + "nbf": 1726760839, + "sub": "SHA256(@user:matrix.domain|DEVICEID|xyzABCDEF10123)", + "video": { + "canPublish": true, + "canSubscribe": true, + "room": "!gIpOlaUSrXBmgtveWK:call.ems.host_m.call_", + "roomCreate": true, + "roomJoin": true + } +} +``` + +### End-to-end encryption + +End-to-end encryption is mapped into the LiveKit frame level encryption mechanism described [here](https://github.com/livekit/livekit/issues/1035). + +Where a shared password is used by the application it is used as the `string` input to the LiveKit key derivation function (which uses PBKDF2) and all participants use the same derived key for encryption and decryption. + +Where a per-participant key is used it is imported as the byte array input to the LiveKit key derivation function (which uses HKDF). The `index` field of the `m.rtc.encryption_keys` event is used as the key index for the key provider. + +On receipt of the `m.rtc.encryption_keys` event the application can associate the received key with the LiveKit participant identity by calculating the +pseudonymous LiveKit participant identity as described above. + +## Potential issues + +## Alternatives ## Security considerations -*Some proposals may have some security aspect to them that was addressed in the proposed solution. This -section is a great place to outline some of the security-sensitive components of your proposal, such as -why a particular approach was (or wasn't) taken. The example here is a bit of a stretch and unlikely to -actually be worthwhile of including in a proposal, but it is generally a good idea to list these kinds -of concerns where possible.* +To prevent abuse of SFU resources, during the backend service should validate the OpenID token as part of requests to `/sfu/get`. + +The Server-Server API endpoint [/_matrix/federation/v1/openid/userinfo](https://spec.matrix.org/v1.11/server-server-api/#get_matrixfederationv1openiduserinfo) +can be used for this purpose. + +An access control policy should be applied based on the result of the OpenID token validation. For example, +access might be restricted to users of a particular homeserver or to users with a specific role. + +The homeserver restriction could be applied by checking the `matrix_server_name` field of the OpenID token before validating the token. -By having a template available, people would know what the desired detail for a proposal is. This is not -considered a risk because it is important that people understand the proposal process from start to end. +The `room_id` could be validated too, and checking that the Matrix user from the OpenID token is a member of the room. ## Unstable prefix -*If a proposal is implemented before it is included in the spec, then implementers must ensure that the -implementation is compatible with the final version that lands in the spec. This generally means that -experimental implementations should use `/unstable` endpoints, and use vendor prefixes where necessary. -For more information, see [MSC2324](https://github.com/matrix-org/matrix-doc/pull/2324). This section -should be used to document things such as what endpoints and names are being used while the feature is -in development, the name of the unstable feature flag to use to detect support for the feature, or what -migration steps are needed to switch to newer versions of the proposal.* +Assuming that this is accepted at the same time as [MSC4143](https://github.com/matrix-org/matrix-spec-proposals/pull/4143) +no unstable prefix is required as these fields will only be accessed via some other unstable prefix. ## Dependencies -This MSC builds on MSCxxxx, MSCyyyy and MSCzzzz (which at the time of writing have not yet been accepted -into the spec). +This MSC builds on [MSC4143](https://github.com/matrix-org/matrix-spec-proposals/pull/4143) +(which at the time of writing has not yet been accepted into the spec). From d93480ca1cf63439c6a06f52b00b6b5d041781b9 Mon Sep 17 00:00:00 2001 From: fkwp Date: Tue, 14 Oct 2025 15:01:32 +0200 Subject: [PATCH 004/122] rewrite addressing a lot of feedback from offline discussions. --- proposals/4195-matrixrtc-livekit.md | 290 +++++++++++++++++++--------- 1 file changed, 198 insertions(+), 92 deletions(-) diff --git a/proposals/4195-matrixrtc-livekit.md b/proposals/4195-matrixrtc-livekit.md index 6352e91a3e0..395429d3b99 100644 --- a/proposals/4195-matrixrtc-livekit.md +++ b/proposals/4195-matrixrtc-livekit.md @@ -1,111 +1,150 @@ -# MSC4195: MatrixRTC using LiveKit backend +# MSC4195: MatrixRTC Transport using LiveKit Backend -This proposal defines a new [MSC4143](https://github.com/matrix-org/matrix-spec-proposals/pull/4143) compliant -RTC backend Focus type utilising [LiveKit](https://github.com/livekit/livekit) SFUs. +This proposal defines a new [MSC4143](https://github.com/matrix-org/matrix-spec-proposals/pull/4143) +compliant MatrixRTC Transport using [LiveKit](https://github.com/livekit/livekit) Selective +Forwarding Units (SFUs). + +In real-time communication environments, managing media streams among multiple participants can be +complex. This transport proposal uses a **Multi-SFU approach** where each participant publishes +their media directly to a LiveKit SFU, while others subscribe to streams they need. This removes the +need for an SFU election and preserves clear ownership of media. + +Example for two participants from different homeservers A and B + +``` + +------------------+ + | Participant A | + | (Matrix Client) | + +------------------+ + | ^ + | | + | publishes | subscribes + v | + +-------+ +-------+ + | SFU A | | SFU B | + +-------+ +-------+ + | ^ + | | + | subscribes | publishes + v | + +------------------+ + | Participant B | + | (Matrix Client) | + +------------------+ +``` ## Proposal -The `type` of the Focus is `livekit`. +This MSC defines the **LiveKit RTC Transport**, which can appear as one of the **RTC Transports** +offered by a homeserver and being used as transport by clients. -We introduce a two sub-types of LiveKit Focus: +### LiveKit room alias -- LiveKit SFU Focus - a LiveKit SFU backend infrastructure. -- LiveKit oldest membership Focus selection algorithm - an algorithm to select a communal LiveKit SFU based on the age of the session participants. +The name of a LiveKit room is referred to as the **LiveKit alias** (`livekit_alias`). The alias +**MUST** be unique within a given MatrixRTC slot in a Matrix room. It is derived by the +concatenation of `room_id` and `|` and `slot_id`. The value is opaque to the MatrixRTC application. +Within the LiveKit namespace, the `livekit_alias` represents a MatrixRTC slot. -### LiveKit SFU Focus +Participants from the same Matrix deployment (using the same SFU to publish their media) are +considered to use the same `livekit_alias` in order to limit the number of actual LiveKit SFU +connections. -A backend LiveKit SFU Focus is represented in these places: +### Focus type: `livekit_multi_sfu` -- the `m.rtc_foci` entry of the `.well-known/matrix/client` -- the `foci_preferred` of the `m.rtc.member` state event -- the `focus_active` of the `m.rtc.member` state event. TODO: is this correct? +This section defines the JSON format for the LiveKit SFU Transport, covering both homeserver-side +advertisement and client-side consumption. -It is represented as a JSON object with the following fields: +#### Transport Advertisement (homeserver) -- `type` required string - `livekit` -- `livekit_service_url` required string - The URL of the LiveKit MatrixRTC backend to use for the session. +The homeserver announces available LiveKit Transport as a JSON object with the following fields: -An example within a `.well-known/matrix/client`: +- `type` (string, required) \- this **MUST** be `"livekit_multi_sfu` +- `livekit_service_url` (string, required) \- The URL of the service that issues JWT tokens for + connecting this LiveKit SFU. +An example for `GET /_matrix/client/v1/rtc/transports` ```json5 { - // rest of the .well-known/matrix/client content - "m.rtc_foci": [ + "rtc_transports": [ { - "type": "livekit", - "livekit_service_url": "https://livekit-jwt.call.element.dev" + "type": "livekit_multi_sfu", + "livekit_service_url": "https://matrix-rtc.example.com/livekit/jwt" } ] } ``` -An example within the `foci_preferred` of the `m.rtc.member` state event: +#### Transport Usage (client) -```json5 +The mechanism for discovering available RTC transports by clients is already defined in +[MSC4143](https://github.com/matrix-org/matrix-spec-proposals/pull/4143). + +Clients declare the RTC Transport(s) they use to publish RTC data in their `m.rtc.member` state +event by adding a JSON object to the `rtc_transports` array. + +#### Field Descriptions + +* `type` (string, required) \- this **MUST** be `"livekit_multi_sfu"` +* `livekit_service_url` (string, required) \- The URL of the service that issues JWT tokens for + connecting this LiveKit SFU. + +``` { // rest of the m.rtc.member event - "foci_preferred": [ + "rtc_transports": [ { - "type": "livekit", - "livekit_service_url": "https://livekit-jwt.call.element.dev", + "type": "livekit_multi_sfu", + "livekit_service_url": "https://matrix-rtc.example.com/livekit/jwt", } ] } ``` -### LiveKit oldest membership Focus selection algorithm -We define a new Focus that uses the age of the session participant `m.rtc.member` to determine the Focus to use. +### LiveKit SFU Authorisation -This can be used in the `focus_active` of the `m.rtc.member` state event. - -The Focus is represented as follows: - -- `type` required string - `livekit` -- `focus_selection` required string - `oldest_membership` - -### SFU authentication with LiveKit - -LiveKit SFUs requires a JWT `access_token` to be provided when [connecting to the WebSocket](https://docs.livekit.io/reference/internals/client-protocol/#WebSocket-Parameters). -We standardise the method by which the LiveKit JWT token is obtained by a MatrixRTC application. +LiveKit SFUs requires a JWT `access_token` to be provided when [connecting to the +WebSocket](https://docs.livekit.io/reference/internals/client-protocol/#WebSocket-Parameters). We +standardise the method by which the LiveKit JWT token is obtained by a MatrixRTC application. Prerequisites: -- the `livekit_service_url` for the MatrixRTC backend has been discovered from one of the methods above -- the Matrix client has obtained an OpenID Token from the [Client-Server API](https://spec.matrix.org/v1.11/client-server-api/#openid). +* The `livekit_service_url` for the MatrixRTC backend has been discovered from one of the methods above +* The Matrix client has obtained an OpenID Token from the [Client-Server + API](https://spec.matrix.org/v1.11/client-server-api/#openid). The JWT token is obtained by making a `POST` request to the `/sfu/get` endpoint of the LiveKit service. The `Content-Type` of the request is `application/json` and the body JSON body contains the following fields: -- `room_id` required string: the room ID of the Matrix room where the `m.rtc.member` event state key is present. -- `session` required object: the contents of the `session` from the `m.rtc.member` event. -- `openid_token` required object: The verbatim OpenID token response obtained from the - [Client-Server API](https://spec.matrix.org/v1.11/client-server-api/#post_matrixclientv3useruseridopenidrequest_token). -- `member` required object: The contents of the `member` from the `m.rtc.member` event. +* `room_id` required string: the room ID of the Matrix room where the `m.rtc.member` event state key + is present. +* `slot_id` required string: the slot ID from the `m.rtc.member` event. +* `openid_token` required object: The verbatim OpenID token response obtained from the + [Client-Server + API](https://spec.matrix.org/v1.11/client-server-api/#post_matrixclientv3useruseridopenidrequest_token). +* `member` required object: The contents of the `member` from the `m.rtc.member` event. -If the request is successful an HTTP `200` response is returned with `Content-Type` `application/json` and the body contains: +If the request is successful an HTTP `200` response is returned with `Content-Type` +`application/json` and the body contains: -- `jwt` string: The JWT token to use for authentication with the SFU. -- `url` string: The URL of the LiveKit SFU to use for the session. +* `jwt` string: The JWT token to use for authentication with the SFU. +* `url` string: The URL of the LiveKit SFU to use for the session. The LiveKit JWT should have permissions as defined below. -An example request where `livekit_service_url` is `https://livekit-jwt.call.element.dev/xyz`: +An example request where `livekit_service_url` is `https://matrix-rtc.example.com/livekit/jwt`: -```http -POST /xyz/sfu/get HTTP/1.1 -Host: livekit-jwt.call.element.dev +``` +POST /livekit/jwt/sfu/get HTTP/1.1 +Host: matrix-rtc.example.com Content-Type: application/json { "room_id": "!tDLCaLXijNtYcJZEey:element.io", - "session": { - "application": "m.call", - "call_id": "" - }, + "slot_id": "the_id", "openid_token": { - "access_token": "FPkexLLvKbAHKclQhpvgfWxx", + "access_token": "FPkexLLvKbAHKclQhpvgfWxx", "expires_in": 3600, "matrix_server_name": "call.ems.host", "token_type": "Bearer" @@ -120,41 +159,40 @@ Content-Type: application/json An example response: -```http +``` HTTP/1.1 200 OK Content-Type: application/json { "jwt": "thejwt", - "url": "wss://livekit-jwt.call.element.dev/rtc" + "url": "wss://matrix-rtc.example.com/livekit/sfu" } ``` -### Pseudonymous LiveKit participant identity - -We use a pseudonymous LiveKit participant identity for privacy so that the Matrix user ID is not exposed to the LiveKit SFU backend. - -The pseudonymous LiveKit participant identity is calculated as the SHA-256 hash of the concatenation of: +### Pseudonymous LiveKit Participant Identity -- the Matrix user ID -- `|` -- `member`.`device_id` -- `|` -- `member`.`id`. +To protect user privacy, a pseudonymous LiveKit participant identity is used, so the Matrix user ID +is not exposed to the LiveKit SFU backend. This pseudonymous identity is represented by `member.id`. -e.g. `SHA256(@user:matrix.domain|DEVICEID|xyzABCDEF10123)` +### LiveKit JWT Permission Grants -### LiveKit JWT permission grants +As well as being a valid [LiveKit JWT](https://docs.livekit.io/home/get-started/authentication/) the +following constraints are applied: -As well as being a valid [LiveKit JWT](https://docs.livekit.io/home/get-started/authentication/) the following constraints -are applied: +- `sub`: This is the pseudonymous LiveKit participant identity as described above. +- `video`.`room`: `livekit_alias` as defined above -- `sub`: This is the pseudonymous LiveKit participant identity as described above. -- `video`.`room`: The name of the room should be the unique for the given `room_id` and `session` inputs. It is opaque to the MatrixRTC application. +In a Multi-SFU setup, where participants may publish to one SFU and consume from others, the JWT +SHOULD encode access permissions according to the user’s homeserver and their relationship to +the MatrixRTC backend. -The permissions should be sufficient to allow the MatrixRTC application to join the room. +The permissions SHOULD be just sufficient for the MatrixRTC application to operate in a LiveKit +room. Permissions SHOULD be scoped according to the user’s role (publishing or subscribing) and +their relationship to the MatrixRTC backend. All users MUST be able to join the LiveKit room for +which they are authorized. The `roomCreate` permission SHOULD only be granted to users who are +related to the MatrixRTC backend and are allowed to publish media. -For example: +Example for publishing RTC data using a full-access grant ```json { @@ -172,41 +210,109 @@ For example: } ``` +Example for subscribing RTC data with restricted-access grant + +```json +{ + "exp": 1726764439, + "iss": "API2bYPYMoVqjcE", + "nbf": 1726760839, + "sub": "SHA256(@user:matrix.domain|DEVICEID|xyzABCDEF10123)", + "video": { + "canPublish": false, + "canSubscribe": true, + "room": "!gIpOlaUSrXBmgtveWK:call.ems.host_m.call_", + "roomCreate": false, + "roomJoin": true + } +} +``` + ### End-to-end encryption -End-to-end encryption is mapped into the LiveKit frame level encryption mechanism described [here](https://github.com/livekit/livekit/issues/1035). +End-to-end encryption is mapped into the LiveKit frame level encryption mechanism described +[here](https://github.com/livekit/livekit/issues/1035). -Where a shared password is used by the application it is used as the `string` input to the LiveKit key derivation function (which uses PBKDF2) and all participants use the same derived key for encryption and decryption. +Where a shared password is used by the application it is used as the `string` input to the LiveKit +key derivation function (which uses PBKDF2) and all participants use the same derived key for +encryption and decryption. -Where a per-participant key is used it is imported as the byte array input to the LiveKit key derivation function (which uses HKDF). The `index` field of the `m.rtc.encryption_keys` event is used as the key index for the key provider. +Where a per-participant key is used it is imported as the byte array input to the LiveKit key +derivation function (which uses HKDF). The `index` field of the `m.rtc.encryption_keys` event is +used as the key index for the key provider. -On receipt of the `m.rtc.encryption_keys` event the application can associate the received key with the LiveKit participant identity by calculating the -pseudonymous LiveKit participant identity as described above. +On receipt of the `m.rtc.encryption_keys` event the application can associate the received key with +the LiveKit participant identity by calculating the pseudonymous LiveKit participant identity as +described above. ## Potential issues +Pseudonymous `livekit_alias` + +Assuming that LiveKit SFU authorization is handled separately from the actual LiveKit SFU, metadata +leakage can be further limited by using a pseudonymous `livekit_alias`. For example, this could be +derived as: `SHA256(room_id|slot_id|truly random bits)` + +Clients that use the same SFU to publish their media are considered to share the same +`livekit_alias`, which helps limit the number of active LiveKit SFU connections. Consequently, the +“truly random bits” used for pseudonymity need to be shared among clients using the same +`livekit_alias`. + +As described in the MatrixRTC slots section of +[MSC4143](https://github.com/matrix-org/matrix-spec-proposals/pull/4143), slots are currently the +only mechanism for sharing state between clients. Slots are **unencrypted** and subject to potential +state resolution issues, including flip-flop and settling effects, and generally require **higher +power levels** to be managed. While tie-breaking “truly random bits” derived from `m.rtc.member` +(e.g., as part of the `rtc_transports` field) events satisfies shared state encryption, it does +**not** improve the reliability of state propagation. Given that pseudonymous LiveKit participant +IDs already exist, the design prioritizes **reliability over additional pseudonymity**, ensuring +consistent state propagation across clients. + ## Alternatives +Pseudonymous `livekit_alias` + +Assuming that LiveKit SFU authorization is handled separately from the actual LiveKit SFU and can be +trusted, metadata leakage can be further limited by using a pseudonymous `livekit_alias`. For +example, this could be derived as: `SHA256(room_id|slot_id|truly random bits)` where the `truly +random bits` are maintained by the LiveKit SFU authorization service. This requires the service to +be stateful. + ## Security considerations -To prevent abuse of SFU resources, during the backend service should validate the OpenID token as part of requests to `/sfu/get`. +### Resource usage + +To prevent abuse of SFU resources, the LiveKit Authorisation service should validate the OpenID +token as part of requests to `/sfu/get`. -The Server-Server API endpoint [/_matrix/federation/v1/openid/userinfo](https://spec.matrix.org/v1.11/server-server-api/#get_matrixfederationv1openiduserinfo) +The Server-Server API endpoint +[/\_matrix/federation/v1/openid/userinfo](https://spec.matrix.org/v1.11/server-server-api/#get_matrixfederationv1openiduserinfo) can be used for this purpose. -An access control policy should be applied based on the result of the OpenID token validation. For example, -access might be restricted to users of a particular homeserver or to users with a specific role. +An access control policy should be applied based on the result of the OpenID token validation. For +example, access might be restricted to users of a particular homeserver or to users with a specific +role. + +The homeserver restriction could be applied by checking the `matrix_server_name` field of the OpenID +token before validating the token. + +The `room_id` could be validated too, and checking that the Matrix user from the OpenID token is a +member of the room. -The homeserver restriction could be applied by checking the `matrix_server_name` field of the OpenID token before validating the token. +### Pseudonymity -The `room_id` could be validated too, and checking that the Matrix user from the OpenID token is a member of the room. +The LiveKit participant identity is a function of one's Matrix user ID, device ID, and session +membership ID; if all of these values are known or otherwise predictable to the SFU then there is +effectively no guarantee of pseudonymity. Therefore clients must be careful to use randomly +generated session membership IDs with sufficient entropy. ## Unstable prefix -Assuming that this is accepted at the same time as [MSC4143](https://github.com/matrix-org/matrix-spec-proposals/pull/4143) -no unstable prefix is required as these fields will only be accessed via some other unstable prefix. +Assuming that this is accepted at the same time as +[MSC4143](https://github.com/matrix-org/matrix-spec-proposals/pull/4143) no unstable prefix is +required as these fields will only be accessed via some other unstable prefix. ## Dependencies -This MSC builds on [MSC4143](https://github.com/matrix-org/matrix-spec-proposals/pull/4143) -(which at the time of writing has not yet been accepted into the spec). +This MSC builds on [MSC4143](https://github.com/matrix-org/matrix-spec-proposals/pull/4143) (which +at the time of writing has not yet been accepted into the spec). \ No newline at end of file From 394d0447d4d5125e70ca789d02b0bc4a3f47447c Mon Sep 17 00:00:00 2001 From: fkwp Date: Tue, 21 Oct 2025 10:41:58 +0200 Subject: [PATCH 005/122] Address concern wrt. reliance of this MSC on the LiveKit protocol --- proposals/4195-matrixrtc-livekit.md | 31 +++++++++++++++++++++++++++++ 1 file changed, 31 insertions(+) diff --git a/proposals/4195-matrixrtc-livekit.md b/proposals/4195-matrixrtc-livekit.md index 395429d3b99..911bf2952b2 100644 --- a/proposals/4195-matrixrtc-livekit.md +++ b/proposals/4195-matrixrtc-livekit.md @@ -268,6 +268,37 @@ power levels** to be managed. While tie-breaking “truly random bits” derived IDs already exist, the design prioritizes **reliability over additional pseudonymity**, ensuring consistent state propagation across clients. +### Reliance on the LiveKit Protocol and Implementation + +A concern has been raised regarding the reliance of this MSC on the LiveKit protocol, which is +developed and maintained by a commercial entity rather than a formal standards body. This creates a +theoretical risk that future development or licensing changes by LiveKit, Inc. could diverge from +Matrix’s goals or limit interoperability. + +This consideration was already discussed during the design of the MatrixRTC backend, and several +factors help to mitigate the concern: +* **Protocol openness**: The LiveKit protocol and reference implementation are released under the + [Apache 2.0 License](https://github.com/livekit/livekit/blob/master/LICENSE), which allows for + forking and independent evolution. If LiveKit’s direction or license were to change, Matrix could + adopt the current protocol version and evolve it independently under an open governance model. +* **No lock-in at the Matrix level**: MatrixRTC defines a generic transport abstraction (see + [MSC4143](https://github.com/matrix-org/matrix-spec-proposals/pull/4143)), allowing for the + definition of additional or alternative transport types in the future without breaking + compatibility. +* **Extensibility**: Because the LiveKit protocol is open source, nothing prevents the Matrix + community from implementing additional functionality — such as Cascading SFUs or other + federation-oriented features — on top of the existing protocol if required. While this has been + discussed with the LiveKit team and they did not object in principle, such extensions are not + expected to depend on their involvement. +* **Implementation pragmatism**: The choice of LiveKit was primarily pragmatic—to accelerate + development and deployment of a functioning multi-SFU solution—rather than to establish a + permanent dependency. The current multi-SFU model also reduces the importance of features such as + Cascading SFUs that might otherwise require protocol changes. + +In summary, this MSC’s reliance on LiveKit represents a practical implementation path rather than a +long-term commitment to a specific third-party protocol. The current design remains open to future +evolution toward a Matrix-native or jointly standardized MatrixRTC transport. + ## Alternatives Pseudonymous `livekit_alias` From 02e0783382d5a7929170baf27f21caf319c4b641 Mon Sep 17 00:00:00 2001 From: fkwp Date: Tue, 21 Oct 2025 17:00:13 +0200 Subject: [PATCH 006/122] add section about error responses --- proposals/4195-matrixrtc-livekit.md | 35 +++++++++++++++++++++++++++++ 1 file changed, 35 insertions(+) diff --git a/proposals/4195-matrixrtc-livekit.md b/proposals/4195-matrixrtc-livekit.md index 911bf2952b2..1ec26fd5c53 100644 --- a/proposals/4195-matrixrtc-livekit.md +++ b/proposals/4195-matrixrtc-livekit.md @@ -169,6 +169,41 @@ Content-Type: application/json } ``` +#### Error responses + +The LiveKit authorisation service MUST respond with appropriate HTTP status codes and structured +JSON bodies when an error occurs. All error responses MUST include a top-level `"errcode"` string +and a human-readable `"error"` description, following the conventions used in the +[Matrix Client-Server API](https://spec.matrix.org/v1.11/client-server-api/#error-codes). + +Common error responses: + +| HTTP Status | `errcode` | Meaning / Recommended handling | +|--------------|------------|--------------------------------| +| `400 Bad Request` | `M_INVALID_PARAM` | The request body was malformed, missing required fields, or contained invalid values (e.g. missing `room_id`, `slot_id`, or `openid_token`). | +| `401 Unauthorized` | `M_UNAUTHORIZED` | The request could not be authorised. This response is used for all cases where the OpenID token is invalid, expired, could not be verified, or where the requested room or slot is unknown or inaccessible. Clients may attempt to refresh their OpenID token and retry. | +| `429 Too Many Requests` | `M_LIMIT_EXCEEDED` | The client or homeserver has exceeded rate limits for LiveKit token requests. A `retry_after_ms` field SHOULD be included to indicate when retry is allowed. | +| `500 Internal Server Error` | `M_UNKNOWN` | An unexpected internal error occurred while generating the token. The client may retry after a short delay. | + +Example Error Response: +``` +HTTP/1.1 401 Unauthorized +Content-Type: application/json + +{ + "errcode": "M_UNAUTHORIZED", + "error": "The request could not be authorised." +} +``` +For privacy reasons, the authorisation service does not distinguish between invalid credentials, +unknown resources, or insufficient permissions. All such conditions result in a 401 Unauthorized +response with `M_UNAUTHORIZED`. This prevents clients from inferring the existence of specific +rooms, users, or slots based on error responses. + +The LiveKit authorisation service MAY include additional fields (such as `retry_after_ms` or +`reason`) for diagnostic purposes, but clients MUST be prepared to ignore unknown fields. +Implementations SHOULD NOT disclose sensitive information in the `"error"` field. + ### Pseudonymous LiveKit Participant Identity To protect user privacy, a pseudonymous LiveKit participant identity is used, so the Matrix user ID From d93eb77e4697b4c90f48d49b7ddd2f14a92a557f Mon Sep 17 00:00:00 2001 From: fkwp Date: Tue, 21 Oct 2025 17:01:31 +0200 Subject: [PATCH 007/122] address: error handling and information disclosure --- proposals/4195-matrixrtc-livekit.md | 21 +++++++++++++++++++++ 1 file changed, 21 insertions(+) diff --git a/proposals/4195-matrixrtc-livekit.md b/proposals/4195-matrixrtc-livekit.md index 1ec26fd5c53..f146d27a838 100644 --- a/proposals/4195-matrixrtc-livekit.md +++ b/proposals/4195-matrixrtc-livekit.md @@ -372,6 +372,27 @@ membership ID; if all of these values are known or otherwise predictable to the effectively no guarantee of pseudonymity. Therefore clients must be careful to use randomly generated session membership IDs with sufficient entropy. +### Error handling and information disclosure + +Implementations of the `/sfu/get` endpoint SHOULD take care not to disclose sensitive internal +details through error messages. + +Error responses should use generic `"errcode"` values and short, human-readable `"error"` +descriptions that are suitable for client display or logging. Specifically: +* Validation or authorisation failures MUST NOT reveal information about whether a particular Matrix + user, device, or room exists. +* Server-side or federation validation errors (for example, OpenID token verification failures) + SHOULD be reported as `M_UNAUTHORIZED` or `M_FORBIDDEN` without including internal validation + results or upstream responses. +* Detailed diagnostic information (e.g., reasons for policy rejection, internal stack traces, or + upstream HTTP responses) MUST NOT be exposed to clients, but MAY be logged on the server side for + audit and debugging purposes. +* If rate limiting is applied, the inclusion of a numeric `retry_after_ms` value is acceptable, but + other details of rate limiting policy SHOULD NOT be exposed. + +This ensures that error responses remain useful for clients while preventing potential metadata +leakage about users, rooms, or federation trust relationships. + ## Unstable prefix Assuming that this is accepted at the same time as From 1567fd4a057618ef43ff0fa340a09e741e9e7ae8 Mon Sep 17 00:00:00 2001 From: fkwp Date: Tue, 21 Oct 2025 17:02:54 +0200 Subject: [PATCH 008/122] cleanup structure --- proposals/4195-matrixrtc-livekit.md | 96 ++++++++++++++--------------- 1 file changed, 48 insertions(+), 48 deletions(-) diff --git a/proposals/4195-matrixrtc-livekit.md b/proposals/4195-matrixrtc-livekit.md index f146d27a838..7c7254e61a7 100644 --- a/proposals/4195-matrixrtc-livekit.md +++ b/proposals/4195-matrixrtc-livekit.md @@ -41,9 +41,9 @@ offered by a homeserver and being used as transport by clients. ### LiveKit room alias The name of a LiveKit room is referred to as the **LiveKit alias** (`livekit_alias`). The alias -**MUST** be unique within a given MatrixRTC slot in a Matrix room. It is derived by the -concatenation of `room_id` and `|` and `slot_id`. The value is opaque to the MatrixRTC application. -Within the LiveKit namespace, the `livekit_alias` represents a MatrixRTC slot. +MUST be unique within a given MatrixRTC slot in a Matrix room. It is derived by the concatenation +of `room_id` and `|` and `slot_id`. The value is opaque to the MatrixRTC application. Within the +LiveKit namespace, the `livekit_alias` represents a MatrixRTC slot. Participants from the same Matrix deployment (using the same SFU to publish their media) are considered to use the same `livekit_alias` in order to limit the number of actual LiveKit SFU @@ -54,12 +54,14 @@ connections. This section defines the JSON format for the LiveKit SFU Transport, covering both homeserver-side advertisement and client-side consumption. -#### Transport Advertisement (homeserver) +### Transport Advertisement (homeserver) -The homeserver announces available LiveKit Transport as a JSON object with the following fields: +The mechanism for advertising available RTC transports by homeservers is already defined in +[MSC4143](https://github.com/matrix-org/matrix-spec-proposals/pull/4143). -- `type` (string, required) \- this **MUST** be `"livekit_multi_sfu` -- `livekit_service_url` (string, required) \- The URL of the service that issues JWT tokens for +The homeserver announces available LiveKit Transport as a JSON object with the following fields: +* `type` — required `string`: this MUST be `"livekit_multi_sfu` +* `livekit_service_url` — required `string`: The URL of the service that issues JWT tokens for connecting this LiveKit SFU. An example for `GET /_matrix/client/v1/rtc/transports` @@ -74,7 +76,7 @@ An example for `GET /_matrix/client/v1/rtc/transports` } ``` -#### Transport Usage (client) +### Transport Usage (client) The mechanism for discovering available RTC transports by clients is already defined in [MSC4143](https://github.com/matrix-org/matrix-spec-proposals/pull/4143). @@ -82,10 +84,13 @@ The mechanism for discovering available RTC transports by clients is already def Clients declare the RTC Transport(s) they use to publish RTC data in their `m.rtc.member` state event by adding a JSON object to the `rtc_transports` array. -#### Field Descriptions +Other clients in the same MatrixRTC slot discover and subscribe to each other’s media by inspecting +`m.rtc.member` events. Clients use this information to connect to the appropriate SFU and subscribe +to the published media. -* `type` (string, required) \- this **MUST** be `"livekit_multi_sfu"` -* `livekit_service_url` (string, required) \- The URL of the service that issues JWT tokens for +Field Descriptions: +* `type` — required `string`: this MUST be `"livekit_multi_sfu"` +* `livekit_service_url` — required `string`: The URL of the service that issues JWT tokens for connecting this LiveKit SFU. ``` @@ -100,65 +105,61 @@ event by adding a JSON object to the `rtc_transports` array. } ``` - ### LiveKit SFU Authorisation -LiveKit SFUs requires a JWT `access_token` to be provided when [connecting to the -WebSocket](https://docs.livekit.io/reference/internals/client-protocol/#WebSocket-Parameters). We -standardise the method by which the LiveKit JWT token is obtained by a MatrixRTC application. - -Prerequisites: - -* The `livekit_service_url` for the MatrixRTC backend has been discovered from one of the methods above -* The Matrix client has obtained an OpenID Token from the [Client-Server - API](https://spec.matrix.org/v1.11/client-server-api/#openid). - -The JWT token is obtained by making a `POST` request to the `/sfu/get` endpoint of the LiveKit service. +LiveKit SFUs require a JWT `access_token` to be provided when +[connecting to the WebSocket](https://docs.livekit.io/reference/internals/client-protocol/#WebSocket-Parameters). +This section standardises the method by which a MatrixRTC application obtains the LiveKit JWT +token. -The `Content-Type` of the request is `application/json` and the body JSON body contains the following fields: +#### Prerequisites -* `room_id` required string: the room ID of the Matrix room where the `m.rtc.member` event state key - is present. -* `slot_id` required string: the slot ID from the `m.rtc.member` event. -* `openid_token` required object: The verbatim OpenID token response obtained from the - [Client-Server - API](https://spec.matrix.org/v1.11/client-server-api/#post_matrixclientv3useruseridopenidrequest_token). -* `member` required object: The contents of the `member` from the `m.rtc.member` event. +* The `livekit_service_url` for the MatrixRTC backend has been discovered from one of the methods above. +* The Matrix client has obtained an OpenID token from the [Client-Server API](https://spec.matrix.org/v1.11/client-server-api/#openid). -If the request is successful an HTTP `200` response is returned with `Content-Type` -`application/json` and the body contains: +#### Request -* `jwt` string: The JWT token to use for authentication with the SFU. -* `url` string: The URL of the LiveKit SFU to use for the session. - -The LiveKit JWT should have permissions as defined below. +The JWT token is obtained by making a `POST` request to the `/sfu/get` endpoint of the LiveKit service. -An example request where `livekit_service_url` is `https://matrix-rtc.example.com/livekit/jwt`: +The `Content-Type` of the request is `application/json` and the JSON body contains the following +fields: + * `room_id` — required `string`: the Matrix room ID where the `m.rtc.member` event is present. + * `slot_id` — required `string`: the slot ID from the `m.rtc.member` event. + * `openid_token` — required `object`: the verbatim OpenID token response obtained from the + [Client-Server API](https://spec.matrix.org/v1.11/client-server-api/#post_matrixclientv3useruseridopenidrequest_token). + * `member` — required `object`: the contents of the `member` field from the `m.rtc.member` event. +Example request where `livekit_service_url` is `https://matrix-rtc.example.com/livekit/jwt`: ``` POST /livekit/jwt/sfu/get HTTP/1.1 Host: matrix-rtc.example.com Content-Type: application/json { - "room_id": "!tDLCaLXijNtYcJZEey:element.io", + "room_id": "!tDLCaLXijNtYcJZEey:example.com", "slot_id": "the_id", "openid_token": { "access_token": "FPkexLLvKbAHKclQhpvgfWxx", "expires_in": 3600, - "matrix_server_name": "call.ems.host", + "matrix_server_name": "matrix.example.com", "token_type": "Bearer" }, "member": { "id": "xyzABCDEF10123", "device_id": "DEVICEID", - "user_id": "@user:matrix.domain" + "user_id": "@user:matrix.example.com" } } ``` -An example response: +#### Successful response +If the request is successful, an HTTP `200 OK` response is returned with +`Content-Type: application/json`. The response body contains: +* `jwt` — `string`: the JWT token to use for authentication with the SFU. +* `url` — `string`: the URL of the LiveKit SFU to use for the given slot. + +Example response: ``` HTTP/1.1 200 OK Content-Type: application/json @@ -227,9 +228,8 @@ their relationship to the MatrixRTC backend. All users MUST be able to join the which they are authorized. The `roomCreate` permission SHOULD only be granted to users who are related to the MatrixRTC backend and are allowed to publish media. -Example for publishing RTC data using a full-access grant - -```json +Example for publishing RTC data using a full-access grant: +```json5 { "exp": 1726764439, "iss": "API2bYPYMoVqjcE", @@ -247,7 +247,7 @@ Example for publishing RTC data using a full-access grant Example for subscribing RTC data with restricted-access grant -```json +```json5 { "exp": 1726764439, "iss": "API2bYPYMoVqjcE", @@ -282,7 +282,7 @@ described above. ## Potential issues -Pseudonymous `livekit_alias` +### Pseudonymous `livekit_alias` Assuming that LiveKit SFU authorization is handled separately from the actual LiveKit SFU, metadata leakage can be further limited by using a pseudonymous `livekit_alias`. For example, this could be @@ -402,4 +402,4 @@ required as these fields will only be accessed via some other unstable prefix. ## Dependencies This MSC builds on [MSC4143](https://github.com/matrix-org/matrix-spec-proposals/pull/4143) (which -at the time of writing has not yet been accepted into the spec). \ No newline at end of file +at the time of writing has not yet been accepted into the spec). From 59af1637511f1f5cfd9aaf6513d1649aa508ac95 Mon Sep 17 00:00:00 2001 From: fkwp Date: Fri, 7 Nov 2025 10:40:38 +0100 Subject: [PATCH 009/122] Add short intro --- proposals/4195-matrixrtc-livekit.md | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/proposals/4195-matrixrtc-livekit.md b/proposals/4195-matrixrtc-livekit.md index 7c7254e61a7..8492702847f 100644 --- a/proposals/4195-matrixrtc-livekit.md +++ b/proposals/4195-matrixrtc-livekit.md @@ -1,4 +1,8 @@ -# MSC4195: MatrixRTC Transport using LiveKit Backend +# MSC4195: MatrixRTC Transport Using LiveKit Backend + +This MSC defines a LiveKit-based transport for MatrixRTC, allowing clients to publish and subscribe +to real-time media via LiveKit SFUs while maintaining Matrix-native session and membership +semantics. This proposal defines a new [MSC4143](https://github.com/matrix-org/matrix-spec-proposals/pull/4143) compliant MatrixRTC Transport using [LiveKit](https://github.com/livekit/livekit) Selective From 1509c49d705bbdfc27b2c8de5db5a0929b436e45 Mon Sep 17 00:00:00 2001 From: fkwp Date: Fri, 7 Nov 2025 10:41:15 +0100 Subject: [PATCH 010/122] simplify livekit_alias section --- proposals/4195-matrixrtc-livekit.md | 26 ++++++++++++++++++-------- 1 file changed, 18 insertions(+), 8 deletions(-) diff --git a/proposals/4195-matrixrtc-livekit.md b/proposals/4195-matrixrtc-livekit.md index 8492702847f..84ca31098e4 100644 --- a/proposals/4195-matrixrtc-livekit.md +++ b/proposals/4195-matrixrtc-livekit.md @@ -44,14 +44,24 @@ offered by a homeserver and being used as transport by clients. ### LiveKit room alias -The name of a LiveKit room is referred to as the **LiveKit alias** (`livekit_alias`). The alias -MUST be unique within a given MatrixRTC slot in a Matrix room. It is derived by the concatenation -of `room_id` and `|` and `slot_id`. The value is opaque to the MatrixRTC application. Within the -LiveKit namespace, the `livekit_alias` represents a MatrixRTC slot. - -Participants from the same Matrix deployment (using the same SFU to publish their media) are -considered to use the same `livekit_alias` in order to limit the number of actual LiveKit SFU -connections. +The name of a LiveKit room is referred to as the **LiveKit alias** (`livekit_alias`). The alias MUST +be globally unique and dependent on a given MatrixRTC slot in a Matrix room. A minimal +implementation that ensures a baseline of pseudonymity is given by the SHA-256 hash of the +concatenation of the Matrix `room_id`, a pipe character (`|`), and the `slot_id`, +e.g.,`SHA256(room_id|slot_id)`. + +For improved metadata protection, the `livekit_alias` SHOULD be derived as +`SHA256(room_id|slot_id|truly random bits)`, where the `truly random bits` are maintained by the +LiveKit SFU authorisation service. This approach enhances pseudonymity but requires the service to +be **stateful** in order to manage and persist the random bits. + +The resulting value is opaque to the MatrixRTC application. Within the LiveKit namespace, the +`livekit_alias` uniquely represents a MatrixRTC slot. Participants from the same Matrix deployment +(using the same SFU to publish their media) are considered to use the same `livekit_alias` in order +to limit the number of active LiveKit SFU connections. + +The `livekit_alias` is shared with clients as part of their JWT token issued by the authorisation +service. ### Focus type: `livekit_multi_sfu` From 04fa36c8948422b2928b291a1c9b0a8289b3a60f Mon Sep 17 00:00:00 2001 From: fkwp Date: Fri, 7 Nov 2025 10:41:28 +0100 Subject: [PATCH 011/122] typo --- proposals/4195-matrixrtc-livekit.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/proposals/4195-matrixrtc-livekit.md b/proposals/4195-matrixrtc-livekit.md index 84ca31098e4..09712d23462 100644 --- a/proposals/4195-matrixrtc-livekit.md +++ b/proposals/4195-matrixrtc-livekit.md @@ -74,7 +74,7 @@ The mechanism for advertising available RTC transports by homeservers is already [MSC4143](https://github.com/matrix-org/matrix-spec-proposals/pull/4143). The homeserver announces available LiveKit Transport as a JSON object with the following fields: -* `type` — required `string`: this MUST be `"livekit_multi_sfu` +* `type` — required `string`: this MUST be `livekit_multi_sfu` * `livekit_service_url` — required `string`: The URL of the service that issues JWT tokens for connecting this LiveKit SFU. From 087515c33f2e5f49212328a8b3236930db77b141 Mon Sep 17 00:00:00 2001 From: fkwp Date: Fri, 7 Nov 2025 10:42:03 +0100 Subject: [PATCH 012/122] add high level diagram to show the auth process --- proposals/4195-matrixrtc-livekit.md | 22 ++++++++++++++++++++-- 1 file changed, 20 insertions(+), 2 deletions(-) diff --git a/proposals/4195-matrixrtc-livekit.md b/proposals/4195-matrixrtc-livekit.md index 09712d23462..dffbe63aaf7 100644 --- a/proposals/4195-matrixrtc-livekit.md +++ b/proposals/4195-matrixrtc-livekit.md @@ -119,12 +119,30 @@ Field Descriptions: } ``` -### LiveKit SFU Authorisation +### LiveKit SFU Authorisation Service LiveKit SFUs require a JWT `access_token` to be provided when [connecting to the WebSocket](https://docs.livekit.io/reference/internals/client-protocol/#WebSocket-Parameters). This section standardises the method by which a MatrixRTC application obtains the LiveKit JWT -token. +token. A high level overview is depicted int he following diagram + +```mermaid +sequenceDiagram + participant U as 🧑 User + participant M as 🏢 Matrix Homeserver + participant A as 🔐 MatrixRTC Authorisation Service + participant L as 📡 LiveKit SFU + + U->>M: Requests OpenID token + M-->>U: Returns OpenID token + U->>A: Sends OpenID token & room request + A->>M: Validates token via OpenID API + M-->>A: Confirms user identity + A->>A: Generates LiveKit JWT + A->>L: (If full-access user) Create room if missing + A-->>U: Returns LiveKit JWT + U->>L: Connects to room using JWT +``` #### Prerequisites From 9ed3677f14c2961fd3ea1bf7dbb9627b2aeeb941 Mon Sep 17 00:00:00 2001 From: fkwp Date: Fri, 7 Nov 2025 15:55:31 +0100 Subject: [PATCH 013/122] rename /sfu/get to more descriptive /get_token --- proposals/4195-matrixrtc-livekit.md | 14 +++++++------- 1 file changed, 7 insertions(+), 7 deletions(-) diff --git a/proposals/4195-matrixrtc-livekit.md b/proposals/4195-matrixrtc-livekit.md index dffbe63aaf7..d143c5c704e 100644 --- a/proposals/4195-matrixrtc-livekit.md +++ b/proposals/4195-matrixrtc-livekit.md @@ -151,7 +151,7 @@ sequenceDiagram #### Request -The JWT token is obtained by making a `POST` request to the `/sfu/get` endpoint of the LiveKit service. +The JWT token is obtained by making a `POST` request to the `/get_token` endpoint of the LiveKit service. The `Content-Type` of the request is `application/json` and the JSON body contains the following fields: @@ -162,8 +162,8 @@ fields: * `member` — required `object`: the contents of the `member` field from the `m.rtc.member` event. Example request where `livekit_service_url` is `https://matrix-rtc.example.com/livekit/jwt`: -``` -POST /livekit/jwt/sfu/get HTTP/1.1 +```http +POST /livekit/jwt/get_token HTTP/1.1 Host: matrix-rtc.example.com Content-Type: application/json @@ -192,7 +192,7 @@ If the request is successful, an HTTP `200 OK` response is returned with * `url` — `string`: the URL of the LiveKit SFU to use for the given slot. Example response: -``` +```http HTTP/1.1 200 OK Content-Type: application/json @@ -219,7 +219,7 @@ Common error responses: | `500 Internal Server Error` | `M_UNKNOWN` | An unexpected internal error occurred while generating the token. The client may retry after a short delay. | Example Error Response: -``` +```http HTTP/1.1 401 Unauthorized Content-Type: application/json @@ -381,7 +381,7 @@ be stateful. ### Resource usage To prevent abuse of SFU resources, the LiveKit Authorisation service should validate the OpenID -token as part of requests to `/sfu/get`. +token as part of requests to `/get_token`. The Server-Server API endpoint [/\_matrix/federation/v1/openid/userinfo](https://spec.matrix.org/v1.11/server-server-api/#get_matrixfederationv1openiduserinfo) @@ -406,7 +406,7 @@ generated session membership IDs with sufficient entropy. ### Error handling and information disclosure -Implementations of the `/sfu/get` endpoint SHOULD take care not to disclose sensitive internal +Implementations of the `/get_token` endpoint SHOULD take care not to disclose sensitive internal details through error messages. Error responses should use generic `"errcode"` values and short, human-readable `"error"` From 3aca20edc4d98bdf1dac84335d6b1423d507f28b Mon Sep 17 00:00:00 2001 From: fkwp Date: Fri, 7 Nov 2025 16:02:36 +0100 Subject: [PATCH 014/122] Add section about the delegation of delayed events --- proposals/4195-matrixrtc-livekit.md | 68 +++++++++++++++++++++++++++-- 1 file changed, 65 insertions(+), 3 deletions(-) diff --git a/proposals/4195-matrixrtc-livekit.md b/proposals/4195-matrixrtc-livekit.md index d143c5c704e..96a76759836 100644 --- a/proposals/4195-matrixrtc-livekit.md +++ b/proposals/4195-matrixrtc-livekit.md @@ -160,6 +160,9 @@ fields: * `openid_token` — required `object`: the verbatim OpenID token response obtained from the [Client-Server API](https://spec.matrix.org/v1.11/client-server-api/#post_matrixclientv3useruseridopenidrequest_token). * `member` — required `object`: the contents of the `member` field from the `m.rtc.member` event. + * `delay_id` — optional `string`: the delayed event id of the MatrixRTC member leave event. + * `delay_timeout` — optional `string`: number of positive non-zero milliseconds the homeserver + should wait before sending the MatrixRTC member leave event Example request where `livekit_service_url` is `https://matrix-rtc.example.com/livekit/jwt`: ```http @@ -178,9 +181,11 @@ Content-Type: application/json }, "member": { "id": "xyzABCDEF10123", - "device_id": "DEVICEID", - "user_id": "@user:matrix.example.com" - } + "claimed_device_id": "DEVICEID", + "claimed_user_id": "@user:matrix.example.com" + }, + "delay_id": "1234567890", // optional + "delay_timeout": "7200000" // optional (2 hours) } ``` @@ -237,6 +242,63 @@ The LiveKit authorisation service MAY include additional fields (such as `retry_ `reason`) for diagnostic purposes, but clients MUST be prepared to ignore unknown fields. Implementations SHOULD NOT disclose sensitive information in the `"error"` field. +#### Optional Delegated MatrixRTC Membership Lifecycle Tracking using Cancellable Delayed Events + +As described in [MSC4143](https://github.com/matrix-org/matrix-spec-proposals/pull/4143), clients +SHOULD use cancellable delayed events to implement a "deadman switch" for precise MatrixRTC +membership tracking. This involves sending a disconnect event ahead of the connect event as a +delayed event with a reasonable timeout (e.g., 15--30 seconds), and periodically resetting the +delayed event's timer. If the timer expires due to a missing reset, the disconnect event is +automatically emitted, marking the participant as disconnected and ensuring accurate session state +even in cases of sudden disconnection, crashes, or network failures. However, relying on clients to +reset the delayed event timer can be error-prone in adverse network conditions, particularly due to +TCP connection instability. + +Since the LiveKit SFU already maintains authoritative knowledge of each participant's connection +state, the management of cancellable delayed events MAY be delegated to the LiveKit SFU +Authorisation Service. This delegation allows the RTC transport layer to accurately manage and +maintain MatrixRTC membership lifecycles across transient disconnects, ensuring a consistent and +reliable view of session state. A prerequisite for this delegation is that the client includes both +`delay_id` and `delay_timeout` fields as part of the authorisation request. + +Upon successful issuance of a JWT token and once the LiveKit SFU Authorisation Service observes the +client's SFU connection, identified by the LiveKit room `livekit_alias` and the LiveKit identity +specified in `member.id`, it SHOULD issue a `reset` of the delayed event by sending the following +POST request to the homeserver of that client: +```http +POST /_matrix/client/v1/delayed_events/{delay_id}/restart HTTP/1.1 +Host: matrix-rtc.example.com +Content-Type: application/json + +{} +``` + +It then starts a timer corresponding to the specified `delay_timeout`. The timer is periodically +reset while the client remains connected with sufficient headroom (e.g., 80% of `delay_timeout`) to + ensure the reset occurs well before `delay_timeout` expires. If the SFU detects that the client has +disconnected before the timer is reset, the Authorisation Service MUST trigger the `disconnect` +event by sending the following request to the homeserver: +```http +POST /_matrix/client/v1/delayed_events/{delay_id}/send HTTP/1.1 +Host: matrix-rtc.example.com +Content-Type: application/json + +{} +``` + +This ensures that the MatrixRTC membership state remains accurate and consistent, even in the +presence of network interruptions or client crashes. + +Implementations SHOULD verify support for delayed events by querying the client’s homeserver +`_matrix/client/versions` endpoint. If the homeserver does not advertise support for delayed events, +the SFU authorisation request MUST be rejected with the error code `M_UNSUPPORTED` and error message +`MatrixRTC membership lifecycle delegation failed: homeserver does not support delayed events.`. + +Implementations MAY retry failed delayed event POST requests using an exponential backoff strategy +in the event of transient network failures. However, retry attempts MUST cease once either the +configured `delay_timeout` has elapsed or the maximum sticky duration of one hour for the delayed +event has been reached, whichever occurs first. + ### Pseudonymous LiveKit Participant Identity To protect user privacy, a pseudonymous LiveKit participant identity is used, so the Matrix user ID From 68f8868027e59fbb8773394e18d525f2a0c32f19 Mon Sep 17 00:00:00 2001 From: fkwp Date: Fri, 7 Nov 2025 16:03:08 +0100 Subject: [PATCH 015/122] M_INVALID_PARAM -> M_BAD_JSON --- proposals/4195-matrixrtc-livekit.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/proposals/4195-matrixrtc-livekit.md b/proposals/4195-matrixrtc-livekit.md index 96a76759836..b2e268873da 100644 --- a/proposals/4195-matrixrtc-livekit.md +++ b/proposals/4195-matrixrtc-livekit.md @@ -218,7 +218,7 @@ Common error responses: | HTTP Status | `errcode` | Meaning / Recommended handling | |--------------|------------|--------------------------------| -| `400 Bad Request` | `M_INVALID_PARAM` | The request body was malformed, missing required fields, or contained invalid values (e.g. missing `room_id`, `slot_id`, or `openid_token`). | +| `400 Bad Request` | `M_BAD_JSON` | The request body was malformed, missing required fields, or contained invalid values (e.g. missing `room_id`, `slot_id`, or `openid_token`). | | `401 Unauthorized` | `M_UNAUTHORIZED` | The request could not be authorised. This response is used for all cases where the OpenID token is invalid, expired, could not be verified, or where the requested room or slot is unknown or inaccessible. Clients may attempt to refresh their OpenID token and retry. | | `429 Too Many Requests` | `M_LIMIT_EXCEEDED` | The client or homeserver has exceeded rate limits for LiveKit token requests. A `retry_after_ms` field SHOULD be included to indicate when retry is allowed. | | `500 Internal Server Error` | `M_UNKNOWN` | An unexpected internal error occurred while generating the token. The client may retry after a short delay. | From fb0c402e39d317410ba3cecc0c6682947c89df7c Mon Sep 17 00:00:00 2001 From: fkwp Date: Fri, 7 Nov 2025 16:03:45 +0100 Subject: [PATCH 016/122] fix example JSON snippets --- proposals/4195-matrixrtc-livekit.md | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/proposals/4195-matrixrtc-livekit.md b/proposals/4195-matrixrtc-livekit.md index b2e268873da..51a9eb3b261 100644 --- a/proposals/4195-matrixrtc-livekit.md +++ b/proposals/4195-matrixrtc-livekit.md @@ -319,7 +319,7 @@ the MatrixRTC backend. The permissions SHOULD be just sufficient for the MatrixRTC application to operate in a LiveKit room. Permissions SHOULD be scoped according to the user’s role (publishing or subscribing) and their relationship to the MatrixRTC backend. All users MUST be able to join the LiveKit room for -which they are authorized. The `roomCreate` permission SHOULD only be granted to users who are +which they are authorised. The `roomCreate` permission SHOULD only be granted to users who are related to the MatrixRTC backend and are allowed to publish media. Example for publishing RTC data using a full-access grant: @@ -328,11 +328,11 @@ Example for publishing RTC data using a full-access grant: "exp": 1726764439, "iss": "API2bYPYMoVqjcE", "nbf": 1726760839, - "sub": "SHA256(@user:matrix.domain|DEVICEID|xyzABCDEF10123)", + "sub": "xyzABCDEF0123", // member.id "video": { "canPublish": true, "canSubscribe": true, - "room": "!gIpOlaUSrXBmgtveWK:call.ems.host_m.call_", + "room": "SHA256(!gIpOlaUSrXBmgtveWK:call.ems.host|m.call#ROOM)", "roomCreate": true, "roomJoin": true } @@ -346,11 +346,11 @@ Example for subscribing RTC data with restricted-access grant "exp": 1726764439, "iss": "API2bYPYMoVqjcE", "nbf": 1726760839, - "sub": "SHA256(@user:matrix.domain|DEVICEID|xyzABCDEF10123)", + "sub": "xyzABCDEF0123", // member.id "video": { "canPublish": false, "canSubscribe": true, - "room": "!gIpOlaUSrXBmgtveWK:call.ems.host_m.call_", + "room": "SHA256(!gIpOlaUSrXBmgtveWK:call.ems.host|m.call#ROOM)", "roomCreate": false, "roomJoin": true } From b3b1688dba5c407550a5ba0ea69cd08caae936ee Mon Sep 17 00:00:00 2001 From: fkwp Date: Fri, 7 Nov 2025 17:47:52 +0100 Subject: [PATCH 017/122] "Pseudonymous livekit_alias" --> "Source of `truly_random_bits` for Pseudonymous `livekit_alias` Derivation" --- proposals/4195-matrixrtc-livekit.md | 64 ++++++++++++++++++++--------- 1 file changed, 44 insertions(+), 20 deletions(-) diff --git a/proposals/4195-matrixrtc-livekit.md b/proposals/4195-matrixrtc-livekit.md index 51a9eb3b261..54788bd1bc8 100644 --- a/proposals/4195-matrixrtc-livekit.md +++ b/proposals/4195-matrixrtc-livekit.md @@ -376,26 +376,50 @@ described above. ## Potential issues -### Pseudonymous `livekit_alias` - -Assuming that LiveKit SFU authorization is handled separately from the actual LiveKit SFU, metadata -leakage can be further limited by using a pseudonymous `livekit_alias`. For example, this could be -derived as: `SHA256(room_id|slot_id|truly random bits)` - -Clients that use the same SFU to publish their media are considered to share the same -`livekit_alias`, which helps limit the number of active LiveKit SFU connections. Consequently, the -“truly random bits” used for pseudonymity need to be shared among clients using the same -`livekit_alias`. - -As described in the MatrixRTC slots section of -[MSC4143](https://github.com/matrix-org/matrix-spec-proposals/pull/4143), slots are currently the -only mechanism for sharing state between clients. Slots are **unencrypted** and subject to potential -state resolution issues, including flip-flop and settling effects, and generally require **higher -power levels** to be managed. While tie-breaking “truly random bits” derived from `m.rtc.member` -(e.g., as part of the `rtc_transports` field) events satisfies shared state encryption, it does -**not** improve the reliability of state propagation. Given that pseudonymous LiveKit participant -IDs already exist, the design prioritizes **reliability over additional pseudonymity**, ensuring -consistent state propagation across clients. +### Source of `truly_random_bits` for Pseudonymous `livekit_alias` Derivation + +Clients that publish their media through the same SFU and use the same `slot_id` within a given +Matrix room are considered to share the same LiveKit room (`livekit_alias`), which minimizes the +number of active LiveKit SFU connections. + +The derivation of the LiveKit room alias is defined as: `livekit_alias = SHA256(room_id | slot_id | +truly_random_bits)`. + +This construction is part of the proposal and ensures that aliases remain pseudonymous while still +being deterministically derived for a given Matrix room and MatrixRTC slot. The open consideration +is the source of the `truly_random_bits` used in the derivation. + +Two approaches are possible: + +1. **Client-provided `truly_random_bits`** + * Requires coordination between clients sharing the same `slot_id` within a Matrix room to ensure + they use identical random bits; otherwise, different `livekit_alias` values maybe derived and + fragment the session. + * As described in the MatrixRTC slots section of + [MSC4143](https://github.com/matrix-org/matrix-spec-proposals/pull/4143), slots are the intended + mechanism for sharing state between clients. However, slots are **unencrypted** and subject to + state resolution. Therefore, they are not suitable for holding truly random bits`. + * While tie-breaking `truly random bits` derived from `m.rtc.member` events (e.g., within the + `rtc_transports` field) ensures that the data is encrypted shared state, it is subject to + client-side consensus and may flip over time. Overall, it does **not** improve the reliability + of propagating and converging those random bits. + * This approach keeps the LiveKit Authorisation Service stateless + * Requires the removal of the `room_id` field from the access request, which prevents additional + access checks, such as verifying that the user is actually part of the claimed Matrix room. +2. **Authorisation-service-provided random bits** + * The Authorisation Service generates and persists the `truly_random_bits` for each `(room_id, + slot_id)` tuple + * Guarantees consistent alias derivation across clients without requiring client-side + coordination. + * The service becomes stateful, as it must retain the `truly_random_bits` + * The benefit of improved pseudonymity only applies if the LiveKit SFU authorisation service is + operated separately from the actual LiveKit SFU. + * Preserves the `room_id` in the access request, allowing additional access checks, such as + verifying that the user is actually part of the claimed Matrix room. + +Given that pseudonymous LiveKit participant IDs already exist, the design prioritizes **reliability +over additional pseudonymity** by using Authorisation-service-provided random bits, ensuring +consistent `livekit_alias` across clients while enabling additional access checks. ### Reliance on the LiveKit Protocol and Implementation From 5ee153a0c6e7d444bed5e236453ce3882f1c7a0d Mon Sep 17 00:00:00 2001 From: fkwp Date: Fri, 7 Nov 2025 17:48:37 +0100 Subject: [PATCH 018/122] make clear its the matrix room --- proposals/4195-matrixrtc-livekit.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/proposals/4195-matrixrtc-livekit.md b/proposals/4195-matrixrtc-livekit.md index 54788bd1bc8..94a8d7ed4d9 100644 --- a/proposals/4195-matrixrtc-livekit.md +++ b/proposals/4195-matrixrtc-livekit.md @@ -480,8 +480,8 @@ role. The homeserver restriction could be applied by checking the `matrix_server_name` field of the OpenID token before validating the token. -The `room_id` could be validated too, and checking that the Matrix user from the OpenID token is a -member of the room. +The Matrix `room_id` could be validated too, and checking that the Matrix user from the OpenID token +is a member of the room. ### Pseudonymity From 870337bd0a9a65f39a405976a0ff5f012aa6d7c4 Mon Sep 17 00:00:00 2001 From: fkwp Date: Fri, 7 Nov 2025 17:49:39 +0100 Subject: [PATCH 019/122] remove alternative section as the topic is already handled in the security considerations --- proposals/4195-matrixrtc-livekit.md | 8 -------- 1 file changed, 8 deletions(-) diff --git a/proposals/4195-matrixrtc-livekit.md b/proposals/4195-matrixrtc-livekit.md index 94a8d7ed4d9..f10fce40953 100644 --- a/proposals/4195-matrixrtc-livekit.md +++ b/proposals/4195-matrixrtc-livekit.md @@ -454,14 +454,6 @@ evolution toward a Matrix-native or jointly standardized MatrixRTC transport. ## Alternatives -Pseudonymous `livekit_alias` - -Assuming that LiveKit SFU authorization is handled separately from the actual LiveKit SFU and can be -trusted, metadata leakage can be further limited by using a pseudonymous `livekit_alias`. For -example, this could be derived as: `SHA256(room_id|slot_id|truly random bits)` where the `truly -random bits` are maintained by the LiveKit SFU authorization service. This requires the service to -be stateful. - ## Security considerations ### Resource usage From a584a9e26907f150b3a5cf57f7545f8a006f2db6 Mon Sep 17 00:00:00 2001 From: fkwp Date: Mon, 8 Dec 2025 11:32:44 +0100 Subject: [PATCH 020/122] add information about delegation of delayed events --- proposals/4195-matrixrtc-livekit.md | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/proposals/4195-matrixrtc-livekit.md b/proposals/4195-matrixrtc-livekit.md index f10fce40953..d698f36f306 100644 --- a/proposals/4195-matrixrtc-livekit.md +++ b/proposals/4195-matrixrtc-livekit.md @@ -163,6 +163,7 @@ fields: * `delay_id` — optional `string`: the delayed event id of the MatrixRTC member leave event. * `delay_timeout` — optional `string`: number of positive non-zero milliseconds the homeserver should wait before sending the MatrixRTC member leave event + * `delay_cs_api_url` — optional `string`: The Matrix client-server API as used by the client Example request where `livekit_service_url` is `https://matrix-rtc.example.com/livekit/jwt`: ```http @@ -258,8 +259,9 @@ Since the LiveKit SFU already maintains authoritative knowledge of each particip state, the management of cancellable delayed events MAY be delegated to the LiveKit SFU Authorisation Service. This delegation allows the RTC transport layer to accurately manage and maintain MatrixRTC membership lifecycles across transient disconnects, ensuring a consistent and -reliable view of session state. A prerequisite for this delegation is that the client includes both -`delay_id` and `delay_timeout` fields as part of the authorisation request. +reliable view of session state. A prerequisite for this delegation is that the client includes the +following fields: `delay_id`, `delay_timeout` and `delay_cs_api_url` as part of the authorisation +request. Upon successful issuance of a JWT token and once the LiveKit SFU Authorisation Service observes the client's SFU connection, identified by the LiveKit room `livekit_alias` and the LiveKit identity From ad88a5aaf3b5f6c59d850c1349d40b25b73bbf56 Mon Sep 17 00:00:00 2001 From: fkwp Date: Thu, 11 Dec 2025 20:56:40 +0100 Subject: [PATCH 021/122] namespace the pseudonymous LiveKit participant identity to prevent collisions. --- proposals/4195-matrixrtc-livekit.md | 8 +++++--- 1 file changed, 5 insertions(+), 3 deletions(-) diff --git a/proposals/4195-matrixrtc-livekit.md b/proposals/4195-matrixrtc-livekit.md index d698f36f306..3e795f84877 100644 --- a/proposals/4195-matrixrtc-livekit.md +++ b/proposals/4195-matrixrtc-livekit.md @@ -265,8 +265,8 @@ request. Upon successful issuance of a JWT token and once the LiveKit SFU Authorisation Service observes the client's SFU connection, identified by the LiveKit room `livekit_alias` and the LiveKit identity -specified in `member.id`, it SHOULD issue a `reset` of the delayed event by sending the following -POST request to the homeserver of that client: +as specified in the next section (`SHA256(user_id|claimed_device_id|member.id)`), it SHOULD issue +a `reset` of the delayed event by sending the following POST request to the homeserver of that client: ```http POST /_matrix/client/v1/delayed_events/{delay_id}/restart HTTP/1.1 Host: matrix-rtc.example.com @@ -304,7 +304,9 @@ event has been reached, whichever occurs first. ### Pseudonymous LiveKit Participant Identity To protect user privacy, a pseudonymous LiveKit participant identity is used, so the Matrix user ID -is not exposed to the LiveKit SFU backend. This pseudonymous identity is represented by `member.id`. +is not exposed to the LiveKit SFU backend. This pseudonymous identity is given by the SHA-256 hash +of the concatenation of the Matrix `user_id`, a pipe character (`|`) , the `claimed_device_id`, +a pipe character (`|`) and the `member.id` field, e.g, `SHA256(user_id|claimed_device_id|member.id)`. ### LiveKit JWT Permission Grants From a9fa0092c76e654709ee0d4d54d0129d1676ff25 Mon Sep 17 00:00:00 2001 From: Robin Date: Thu, 8 Jan 2026 14:23:40 +0100 Subject: [PATCH 022/122] Specify the encoding of participant identities --- proposals/4195-matrixrtc-livekit.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/proposals/4195-matrixrtc-livekit.md b/proposals/4195-matrixrtc-livekit.md index 3e795f84877..52c7ded52cf 100644 --- a/proposals/4195-matrixrtc-livekit.md +++ b/proposals/4195-matrixrtc-livekit.md @@ -265,7 +265,7 @@ request. Upon successful issuance of a JWT token and once the LiveKit SFU Authorisation Service observes the client's SFU connection, identified by the LiveKit room `livekit_alias` and the LiveKit identity -as specified in the next section (`SHA256(user_id|claimed_device_id|member.id)`), it SHOULD issue +as specified in the next section (`base64(SHA256(user_id|claimed_device_id|member.id))`), it SHOULD issue a `reset` of the delayed event by sending the following POST request to the homeserver of that client: ```http POST /_matrix/client/v1/delayed_events/{delay_id}/restart HTTP/1.1 @@ -304,9 +304,9 @@ event has been reached, whichever occurs first. ### Pseudonymous LiveKit Participant Identity To protect user privacy, a pseudonymous LiveKit participant identity is used, so the Matrix user ID -is not exposed to the LiveKit SFU backend. This pseudonymous identity is given by the SHA-256 hash +is not exposed to the LiveKit SFU backend. This pseudonymous identity is equal to the unpadded base64 encoding of the SHA-256 hash of the concatenation of the Matrix `user_id`, a pipe character (`|`) , the `claimed_device_id`, -a pipe character (`|`) and the `member.id` field, e.g, `SHA256(user_id|claimed_device_id|member.id)`. +a pipe character (`|`) and the `member.id` field, i.e. `base64(SHA256(user_id|claimed_device_id|member.id))`. ### LiveKit JWT Permission Grants From 156da9d6e6ba97ed0c52840e557423e24242b2e8 Mon Sep 17 00:00:00 2001 From: Robin Date: Fri, 9 Jan 2026 13:21:55 +0100 Subject: [PATCH 023/122] Encode all hashes as unpadded base64 --- proposals/4195-matrixrtc-livekit.md | 28 +++++++++++++++------------- 1 file changed, 15 insertions(+), 13 deletions(-) diff --git a/proposals/4195-matrixrtc-livekit.md b/proposals/4195-matrixrtc-livekit.md index 52c7ded52cf..2694abb00f7 100644 --- a/proposals/4195-matrixrtc-livekit.md +++ b/proposals/4195-matrixrtc-livekit.md @@ -46,14 +46,15 @@ offered by a homeserver and being used as transport by clients. The name of a LiveKit room is referred to as the **LiveKit alias** (`livekit_alias`). The alias MUST be globally unique and dependent on a given MatrixRTC slot in a Matrix room. A minimal -implementation that ensures a baseline of pseudonymity is given by the SHA-256 hash of the -concatenation of the Matrix `room_id`, a pipe character (`|`), and the `slot_id`, -e.g.,`SHA256(room_id|slot_id)`. +implementation that ensures a baseline of pseudonymity is given by the +[unpadded base64 encoding](https://spec.matrix.org/v1.17/appendices/#unpadded-base64) of the SHA-256 +hash of the concatenation of the Matrix `room_id`, a pipe character (`|`), and the `slot_id`, i.e. +`base64(SHA256(room_id|slot_id))`. For improved metadata protection, the `livekit_alias` SHOULD be derived as -`SHA256(room_id|slot_id|truly random bits)`, where the `truly random bits` are maintained by the -LiveKit SFU authorisation service. This approach enhances pseudonymity but requires the service to -be **stateful** in order to manage and persist the random bits. +`base64(SHA256(room_id|slot_id|truly random bits))`, where the `truly random bits` are maintained by +the LiveKit SFU authorisation service. This approach enhances pseudonymity but requires the service +to be **stateful** in order to manage and persist the random bits. The resulting value is opaque to the MatrixRTC application. Within the LiveKit namespace, the `livekit_alias` uniquely represents a MatrixRTC slot. Participants from the same Matrix deployment @@ -304,9 +305,10 @@ event has been reached, whichever occurs first. ### Pseudonymous LiveKit Participant Identity To protect user privacy, a pseudonymous LiveKit participant identity is used, so the Matrix user ID -is not exposed to the LiveKit SFU backend. This pseudonymous identity is equal to the unpadded base64 encoding of the SHA-256 hash -of the concatenation of the Matrix `user_id`, a pipe character (`|`) , the `claimed_device_id`, -a pipe character (`|`) and the `member.id` field, i.e. `base64(SHA256(user_id|claimed_device_id|member.id))`. +is not exposed to the LiveKit SFU backend. This pseudonymous identity is equal to the unpadded +base64 encoding of the SHA-256 hash of the concatenation of the Matrix `user_id`, a pipe character +(`|`), the `claimed_device_id`, a pipe character (`|`) and the `member.id` field, i.e. +`base64(SHA256(user_id|claimed_device_id|member.id))`. ### LiveKit JWT Permission Grants @@ -336,7 +338,7 @@ Example for publishing RTC data using a full-access grant: "video": { "canPublish": true, "canSubscribe": true, - "room": "SHA256(!gIpOlaUSrXBmgtveWK:call.ems.host|m.call#ROOM)", + "room": "base64(SHA256(!gIpOlaUSrXBmgtveWK:call.ems.host|m.call#ROOM))", "roomCreate": true, "roomJoin": true } @@ -354,7 +356,7 @@ Example for subscribing RTC data with restricted-access grant "video": { "canPublish": false, "canSubscribe": true, - "room": "SHA256(!gIpOlaUSrXBmgtveWK:call.ems.host|m.call#ROOM)", + "room": "base64(SHA256(!gIpOlaUSrXBmgtveWK:call.ems.host|m.call#ROOM))", "roomCreate": false, "roomJoin": true } @@ -386,8 +388,8 @@ Clients that publish their media through the same SFU and use the same `slot_id` Matrix room are considered to share the same LiveKit room (`livekit_alias`), which minimizes the number of active LiveKit SFU connections. -The derivation of the LiveKit room alias is defined as: `livekit_alias = SHA256(room_id | slot_id | -truly_random_bits)`. +The derivation of the LiveKit room alias is defined as: +`livekit_alias = base64(SHA256(room_id | slot_id | truly_random_bits))`. This construction is part of the proposal and ensures that aliases remain pseudonymous while still being deterministically derived for a given Matrix room and MatrixRTC slot. The open consideration From d5ef7e19690f8a17b9ab6b235ec5d8a531991df0 Mon Sep 17 00:00:00 2001 From: Robin Date: Wed, 11 Mar 2026 20:23:13 +0100 Subject: [PATCH 024/122] Update transport type to match current implementations --- proposals/4195-matrixrtc-livekit.md | 14 +++++++------- 1 file changed, 7 insertions(+), 7 deletions(-) diff --git a/proposals/4195-matrixrtc-livekit.md b/proposals/4195-matrixrtc-livekit.md index 2694abb00f7..d52c4c00e89 100644 --- a/proposals/4195-matrixrtc-livekit.md +++ b/proposals/4195-matrixrtc-livekit.md @@ -64,18 +64,18 @@ to limit the number of active LiveKit SFU connections. The `livekit_alias` is shared with clients as part of their JWT token issued by the authorisation service. -### Focus type: `livekit_multi_sfu` +### Transport type: `livekit` This section defines the JSON format for the LiveKit SFU Transport, covering both homeserver-side advertisement and client-side consumption. -### Transport Advertisement (homeserver) +#### Transport Advertisement (homeserver) The mechanism for advertising available RTC transports by homeservers is already defined in [MSC4143](https://github.com/matrix-org/matrix-spec-proposals/pull/4143). The homeserver announces available LiveKit Transport as a JSON object with the following fields: -* `type` — required `string`: this MUST be `livekit_multi_sfu` +* `type` — required `string`: this MUST be `livekit` * `livekit_service_url` — required `string`: The URL of the service that issues JWT tokens for connecting this LiveKit SFU. @@ -84,14 +84,14 @@ An example for `GET /_matrix/client/v1/rtc/transports` { "rtc_transports": [ { - "type": "livekit_multi_sfu", + "type": "livekit", "livekit_service_url": "https://matrix-rtc.example.com/livekit/jwt" } ] } ``` -### Transport Usage (client) +#### Transport Usage (client) The mechanism for discovering available RTC transports by clients is already defined in [MSC4143](https://github.com/matrix-org/matrix-spec-proposals/pull/4143). @@ -104,7 +104,7 @@ Other clients in the same MatrixRTC slot discover and subscribe to each other’ to the published media. Field Descriptions: -* `type` — required `string`: this MUST be `"livekit_multi_sfu"` +* `type` — required `string`: this MUST be `"livekit"` * `livekit_service_url` — required `string`: The URL of the service that issues JWT tokens for connecting this LiveKit SFU. @@ -113,7 +113,7 @@ Field Descriptions: // rest of the m.rtc.member event "rtc_transports": [ { - "type": "livekit_multi_sfu", + "type": "livekit", "livekit_service_url": "https://matrix-rtc.example.com/livekit/jwt", } ] From 673a33e22fd111d1880b97830ee223be0b83892a Mon Sep 17 00:00:00 2001 From: fkwp Date: Tue, 14 Apr 2026 14:14:18 +0200 Subject: [PATCH 025/122] Stop using '|' delimiters in hashes; use JSON arrays + canonical JSON instead --- proposals/4195-matrixrtc-livekit.md | 74 +++++++++++++++++++++++------ 1 file changed, 60 insertions(+), 14 deletions(-) diff --git a/proposals/4195-matrixrtc-livekit.md b/proposals/4195-matrixrtc-livekit.md index d52c4c00e89..d5384efd96f 100644 --- a/proposals/4195-matrixrtc-livekit.md +++ b/proposals/4195-matrixrtc-livekit.md @@ -42,19 +42,38 @@ Example for two participants from different homeservers A and B This MSC defines the **LiveKit RTC Transport**, which can appear as one of the **RTC Transports** offered by a homeserver and being used as transport by clients. +### Canonical JSON Serialization + +All uses of `JSON.serialize(...)` in this specification MUST use the canonical JSON encoding as +defined by the [Matrix specification](https://spec.matrix.org/v1.18/appendices/#canonical-json). + +Since this MSC exclusively serializes arrays for hashing inputs, implementations MUST ensure: +* The array elements appear in the exact specified order +* Each element is encoded as a JSON string +* The resulting byte sequence used for hashing is the UTF-8 encoding of the canonical JSON output + +For example: +```json5 +["@user:matrix.example.com","DEVICEID","abcd12345"] +``` + +Any deviation (e.g. additional whitespace or different encoding) will result in a different hash +and is therefore non-compliant. + ### LiveKit room alias The name of a LiveKit room is referred to as the **LiveKit alias** (`livekit_alias`). The alias MUST be globally unique and dependent on a given MatrixRTC slot in a Matrix room. A minimal implementation that ensures a baseline of pseudonymity is given by the [unpadded base64 encoding](https://spec.matrix.org/v1.17/appendices/#unpadded-base64) of the SHA-256 -hash of the concatenation of the Matrix `room_id`, a pipe character (`|`), and the `slot_id`, i.e. -`base64(SHA256(room_id|slot_id))`. +hash of the JSON serialization of an array containing the Matrix `room_id` and the `slot_id`, i.e. +`base64(SHA256(JSON.serialize([room_id, slot_id])))`, where `JSON.serialize` is the canonical JSON +serialization defined [above](#canonical-json-serialization). For improved metadata protection, the `livekit_alias` SHOULD be derived as -`base64(SHA256(room_id|slot_id|truly random bits))`, where the `truly random bits` are maintained by -the LiveKit SFU authorisation service. This approach enhances pseudonymity but requires the service -to be **stateful** in order to manage and persist the random bits. +`base64(SHA256(JSON.serialize([room_id, slot_id, truly_random_bits])))`, where the `truly random bits` +are maintained by the LiveKit SFU authorisation service. This approach enhances pseudonymity but +requires the service to be **stateful** in order to manage and persist the random bits. The resulting value is opaque to the MatrixRTC application. Within the LiveKit namespace, the `livekit_alias` uniquely represents a MatrixRTC slot. Participants from the same Matrix deployment @@ -266,8 +285,9 @@ request. Upon successful issuance of a JWT token and once the LiveKit SFU Authorisation Service observes the client's SFU connection, identified by the LiveKit room `livekit_alias` and the LiveKit identity -as specified in the next section (`base64(SHA256(user_id|claimed_device_id|member.id))`), it SHOULD issue -a `reset` of the delayed event by sending the following POST request to the homeserver of that client: +as specified in the next section (`base64(SHA256(JSON.serialize([user_id, claimed_device_id, member.id])))`), +it SHOULD issue a `reset` of the delayed event by sending the following POST request to the +homeserver of that client: ```http POST /_matrix/client/v1/delayed_events/{delay_id}/restart HTTP/1.1 Host: matrix-rtc.example.com @@ -305,10 +325,12 @@ event has been reached, whichever occurs first. ### Pseudonymous LiveKit Participant Identity To protect user privacy, a pseudonymous LiveKit participant identity is used, so the Matrix user ID -is not exposed to the LiveKit SFU backend. This pseudonymous identity is equal to the unpadded -base64 encoding of the SHA-256 hash of the concatenation of the Matrix `user_id`, a pipe character -(`|`), the `claimed_device_id`, a pipe character (`|`) and the `member.id` field, i.e. -`base64(SHA256(user_id|claimed_device_id|member.id))`. +is not exposed to the LiveKit SFU backend. + +This pseudonymous identity is equal to the unpadded base64 encoding of the SHA-256 hash of the JSON +serialization of an array containing the Matrix `user_id`, the `claimed_device_id`, and the +`member.id` field, i.e. `base64(SHA256(JSON.serialize([user_id, claimed_device_id, member.id])))`, +using canonical JSON serialization as defined [above](#canonical-json-serialization). ### LiveKit JWT Permission Grants @@ -338,7 +360,7 @@ Example for publishing RTC data using a full-access grant: "video": { "canPublish": true, "canSubscribe": true, - "room": "base64(SHA256(!gIpOlaUSrXBmgtveWK:call.ems.host|m.call#ROOM))", + "room": "base64(SHA256(JSON.serialize([\"!gIpOlaUSrXBmgtveWK:call.ems.host\", \"m.call#ROOM\"])))", "roomCreate": true, "roomJoin": true } @@ -356,7 +378,7 @@ Example for subscribing RTC data with restricted-access grant "video": { "canPublish": false, "canSubscribe": true, - "room": "base64(SHA256(!gIpOlaUSrXBmgtveWK:call.ems.host|m.call#ROOM))", + "room": "base64(SHA256(JSON.serialize([\"!gIpOlaUSrXBmgtveWK:call.ems.host\", \"m.call#ROOM\"])))", "roomCreate": false, "roomJoin": true } @@ -389,7 +411,7 @@ Matrix room are considered to share the same LiveKit room (`livekit_alias`), whi number of active LiveKit SFU connections. The derivation of the LiveKit room alias is defined as: -`livekit_alias = base64(SHA256(room_id | slot_id | truly_random_bits))`. +`livekit_alias = base64(SHA256(JSON.serialize([room_id, slot_id, truly_random_bits])))`. This construction is part of the proposal and ensures that aliases remain pseudonymous while still being deterministically derived for a given Matrix room and MatrixRTC slot. The open consideration @@ -519,3 +541,27 @@ required as these fields will only be accessed via some other unstable prefix. This MSC builds on [MSC4143](https://github.com/matrix-org/matrix-spec-proposals/pull/4143) (which at the time of writing has not yet been accepted into the spec). + +## Appendix: Hash Derivation Test Vectors + +This appendix provides **verified test vectors** for: + +* `livekit_alias` +* pseudonymous LiveKit participant identity + +All hashes are computed as: + +`base64(SHA256(JSON.serialize([...]))` + +Where `JSON.serialize` uses **Matrix canonical JSON** as defined in: +https://spec.matrix.org/v1.18/appendices/#canonical-json + +--- + +### Test Vectors + +| Case | Input (logical) | Canonical JSON | SHA-256 (hex) | Base64 (unpadded) | +|------|------------------|----------------|---------------|-------------------| +| LiveKit alias (no random bits) | `["!roomid:example.com", "slot123"]` | `["!roomid:example.com","slot123"]` | `0140e634342254799661113eaca06f89e597f00512cdea5e9ee8fabbe77f9fd7` | `AUDmNDQiVHmWYRE+rKBvieWX8AUSzepenuj6u+d/n9c` | +| LiveKit alias (with random bits) | `["!roomid:example.com", "slot123", "random123"]` | `["!roomid:example.com","slot123","random123"]` | `20c78377e2b7308a894c8db4117048adea4a92184e46f7f7abc7f1deb96b8539` | `IMeDd+K3MIqJTI20EXBIrepKkhhORvf3q8fx3rlrhTk` | +| Participant identity | `["@alice:example.com", "DEVICE123", "memberABC"]` | `["@alice:example.com","DEVICE123","memberABC"]` | `27e4f8e6d1abbb173e1eb50ea89265c90495df79bbdbc0a67b8fafb7cfd25ab5` | `J+T45tGruxc+HrUOqJJlyQSV33m728Cme4+vt8/SWrU` | \ No newline at end of file From cd797a9e5923a2eb52c8789783054a3975b168c1 Mon Sep 17 00:00:00 2001 From: fkwp Date: Tue, 14 Apr 2026 16:14:20 +0200 Subject: [PATCH 026/122] Apply suggestions from code review Co-authored-by: Hugh Nimmo-Smith --- proposals/4195-matrixrtc-livekit.md | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/proposals/4195-matrixrtc-livekit.md b/proposals/4195-matrixrtc-livekit.md index d5384efd96f..a353c3087bd 100644 --- a/proposals/4195-matrixrtc-livekit.md +++ b/proposals/4195-matrixrtc-livekit.md @@ -241,7 +241,8 @@ Common error responses: |--------------|------------|--------------------------------| | `400 Bad Request` | `M_BAD_JSON` | The request body was malformed, missing required fields, or contained invalid values (e.g. missing `room_id`, `slot_id`, or `openid_token`). | | `401 Unauthorized` | `M_UNAUTHORIZED` | The request could not be authorised. This response is used for all cases where the OpenID token is invalid, expired, could not be verified, or where the requested room or slot is unknown or inaccessible. Clients may attempt to refresh their OpenID token and retry. | -| `429 Too Many Requests` | `M_LIMIT_EXCEEDED` | The client or homeserver has exceeded rate limits for LiveKit token requests. A `retry_after_ms` field SHOULD be included to indicate when retry is allowed. | +| `429 Too Many Requests` | `M_LIMIT_EXCEEDED` | The client or homeserver has exceeded rate limits for LiveKit token requests. Please refer to the existing [spec](https://spec.matrix.org/v1.18/client-server-api/#common-error-codes) for further details.| +| `403 Forbidden` | `M_USER_LIMIT_EXCEEDED` | The user has exceeded a configured quota or usage limit. Please refer to the existing [spec](https://spec.matrix.org/v1.18/client-server-api/#common-error-codes) for further details.| | `500 Internal Server Error` | `M_UNKNOWN` | An unexpected internal error occurred while generating the token. The client may retry after a short delay. | Example Error Response: @@ -259,7 +260,7 @@ unknown resources, or insufficient permissions. All such conditions result in a response with `M_UNAUTHORIZED`. This prevents clients from inferring the existence of specific rooms, users, or slots based on error responses. -The LiveKit authorisation service MAY include additional fields (such as `retry_after_ms` or +The LiveKit authorisation service MAY include additional fields (such as `reason`) for diagnostic purposes, but clients MUST be prepared to ignore unknown fields. Implementations SHOULD NOT disclose sensitive information in the `"error"` field. From 91ea153f3847ebe1a2ee8559ac03e3a14accad60 Mon Sep 17 00:00:00 2001 From: Johannes Marbach Date: Mon, 8 Jun 2026 10:37:52 +0200 Subject: [PATCH 027/122] Fix typo --- proposals/4195-matrixrtc-livekit.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/proposals/4195-matrixrtc-livekit.md b/proposals/4195-matrixrtc-livekit.md index a353c3087bd..f57323632ba 100644 --- a/proposals/4195-matrixrtc-livekit.md +++ b/proposals/4195-matrixrtc-livekit.md @@ -144,7 +144,7 @@ Field Descriptions: LiveKit SFUs require a JWT `access_token` to be provided when [connecting to the WebSocket](https://docs.livekit.io/reference/internals/client-protocol/#WebSocket-Parameters). This section standardises the method by which a MatrixRTC application obtains the LiveKit JWT -token. A high level overview is depicted int he following diagram +token. A high level overview is depicted in the following diagram ```mermaid sequenceDiagram From d2bf3944c44b036e1a3f7b920b1fdde86b59c3ac Mon Sep 17 00:00:00 2001 From: Johannes Marbach Date: Mon, 8 Jun 2026 15:33:48 +0200 Subject: [PATCH 028/122] Add delay_cs_api_url parameter in example --- proposals/4195-matrixrtc-livekit.md | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/proposals/4195-matrixrtc-livekit.md b/proposals/4195-matrixrtc-livekit.md index f57323632ba..5f6a81fca01 100644 --- a/proposals/4195-matrixrtc-livekit.md +++ b/proposals/4195-matrixrtc-livekit.md @@ -206,7 +206,8 @@ Content-Type: application/json "claimed_user_id": "@user:matrix.example.com" }, "delay_id": "1234567890", // optional - "delay_timeout": "7200000" // optional (2 hours) + "delay_timeout": "7200000", // optional (2 hours) + "delay_cs_api_url": "https://matrix-client.matrix.org/" // optional } ``` From 923bfebbbc162657b4e7bcaa549bc24917f8ddf7 Mon Sep 17 00:00:00 2001 From: Johannes Marbach Date: Mon, 8 Jun 2026 15:37:35 +0200 Subject: [PATCH 029/122] User "LiveKit" to label the service in the chart --- proposals/4195-matrixrtc-livekit.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/proposals/4195-matrixrtc-livekit.md b/proposals/4195-matrixrtc-livekit.md index 5f6a81fca01..fd08c213ca4 100644 --- a/proposals/4195-matrixrtc-livekit.md +++ b/proposals/4195-matrixrtc-livekit.md @@ -150,7 +150,7 @@ token. A high level overview is depicted in the following diagram sequenceDiagram participant U as 🧑 User participant M as 🏢 Matrix Homeserver - participant A as 🔐 MatrixRTC Authorisation Service + participant A as 🔐 LiveKit Authorisation Service participant L as 📡 LiveKit SFU U->>M: Requests OpenID token From a99151a49209c012bc79e0d741895fd9063ec299 Mon Sep 17 00:00:00 2001 From: Johannes Marbach Date: Tue, 9 Jun 2026 11:17:32 +0200 Subject: [PATCH 030/122] Clarify choice of canonical JSON Signed-off-by: Johannes Marbach --- proposals/4195-matrixrtc-livekit.md | 27 ++++++++++++++++++++++----- 1 file changed, 22 insertions(+), 5 deletions(-) diff --git a/proposals/4195-matrixrtc-livekit.md b/proposals/4195-matrixrtc-livekit.md index fd08c213ca4..5512894db75 100644 --- a/proposals/4195-matrixrtc-livekit.md +++ b/proposals/4195-matrixrtc-livekit.md @@ -44,13 +44,15 @@ offered by a homeserver and being used as transport by clients. ### Canonical JSON Serialization -All uses of `JSON.serialize(...)` in this specification MUST use the canonical JSON encoding as +This proposal uses JSON arrays and Canonical JSON encoding to ensure stable hashing inputs. +All uses of `JSON.serialize(...)` in the following text MUST use the Canonical JSON encoding as defined by the [Matrix specification](https://spec.matrix.org/v1.18/appendices/#canonical-json). -Since this MSC exclusively serializes arrays for hashing inputs, implementations MUST ensure: -* The array elements appear in the exact specified order -* Each element is encoded as a JSON string -* The resulting byte sequence used for hashing is the UTF-8 encoding of the canonical JSON output +Additionally, implementations MUST ensure that: + +* The array elements appear in the exact specified order. +* Each element is encoded as a JSON string. +* The resulting byte sequence used for hashing is the UTF-8 encoding of the canonical JSON output. For example: ```json5 @@ -484,6 +486,21 @@ evolution toward a Matrix-native or jointly standardized MatrixRTC transport. ## Alternatives +### String concatenation of hashing inputs + +Instead of using canonical JSON, the hashing inputs could be concatenated with a suitable delimiter +such as `|`. This is prone to delimiter injection, however. As an example, the inputs `("a|b", "c")` +and `("a", "b|c")` both produce the concatenation `"a|b|c"` and, hence, the same hash. Using JSON +arrays and Canonical JSON serialisation avoids this problem. Since the Canonical JSON serialisation +of string arrays is trivial, this doesn't meaningfully increase implementation complexity. + +### JSON objects as hashing inputs + +Instead of JSON arrays, JSON objects could be used for the hashing inputs. This would reduce the +chances of accidentally using the wrong order of array elements. On the downside, however, the +Canonical JSON serialisation of objects is significantly more complex than for arrays. Overall, +this would likely result in a higher chance of implementation errors. + ## Security considerations ### Resource usage From 18aee18387ddb15db712f805f6fa0110985f0c49 Mon Sep 17 00:00:00 2001 From: Johannes Marbach Date: Tue, 9 Jun 2026 16:59:16 +0200 Subject: [PATCH 031/122] Split endpoint and clarify why we need to pass the CS API location Signed-off-by: Johannes Marbach --- proposals/4195-matrixrtc-livekit.md | 196 +++++++++++++++++++--------- 1 file changed, 136 insertions(+), 60 deletions(-) diff --git a/proposals/4195-matrixrtc-livekit.md b/proposals/4195-matrixrtc-livekit.md index 5512894db75..b8d80f07435 100644 --- a/proposals/4195-matrixrtc-livekit.md +++ b/proposals/4195-matrixrtc-livekit.md @@ -143,6 +143,54 @@ Field Descriptions: ### LiveKit SFU Authorisation Service +This section describes endpoints on the SFU Authorisation Service. + +#### General requirements for all endpoints + +##### Prerequisites + +* The `livekit_service_url` for the MatrixRTC backend has been discovered from one of the methods above. +* The Matrix client has obtained an OpenID token from the [Client-Server API](https://spec.matrix.org/v1.11/client-server-api/#openid). + +##### Error responses + +The LiveKit authorisation service MUST respond with appropriate HTTP status codes and structured +JSON bodies when an error occurs. All error responses MUST include a top-level `"errcode"` string +and a human-readable `"error"` description, following the conventions used in the +[Matrix Client-Server API](https://spec.matrix.org/v1.11/client-server-api/#error-codes). + +Common error responses: + +| HTTP Status | `errcode` | Meaning / Recommended handling | +|--------------|------------|--------------------------------| +| `400 Bad Request` | `M_BAD_JSON` | The request body was malformed, missing required fields, or contained invalid values (e.g. missing `room_id`, `slot_id`, or `openid_token`). | +| `401 Unauthorized` | `M_UNAUTHORIZED` | The request could not be authorised. This response is used for all cases where the OpenID token is invalid, expired, could not be verified, or where the requested room or slot is unknown or inaccessible. Clients may attempt to refresh their OpenID token and retry. | +| `429 Too Many Requests` | `M_LIMIT_EXCEEDED` | The client or homeserver has exceeded rate limits. Please refer to the existing [spec](https://spec.matrix.org/v1.18/client-server-api/#common-error-codes) for further details.| +| `403 Forbidden` | `M_USER_LIMIT_EXCEEDED` | The user has exceeded a configured quota or usage limit. Please refer to the existing [spec](https://spec.matrix.org/v1.18/client-server-api/#common-error-codes) for further details.| +| `500 Internal Server Error` | `M_UNKNOWN` | An unexpected internal error occurred while generating the token. The client may retry after a short delay. | + +Example Error Response: + +```http +HTTP/1.1 401 Unauthorized +Content-Type: application/json + +{ + "errcode": "M_UNAUTHORIZED", + "error": "The request could not be authorised." +} +``` +For privacy reasons, the authorisation service does not distinguish between invalid credentials, +unknown resources, or insufficient permissions. All such conditions result in a 401 Unauthorized +response with `M_UNAUTHORIZED`. This prevents clients from inferring the existence of specific +rooms, users, or slots based on error responses. + +The LiveKit authorisation service MAY include additional fields (such as +`reason`) for diagnostic purposes, but clients MUST be prepared to ignore unknown fields. +Implementations SHOULD NOT disclose sensitive information in the `"error"` field. + +#### Acquiring a token for the SFU + LiveKit SFUs require a JWT `access_token` to be provided when [connecting to the WebSocket](https://docs.livekit.io/reference/internals/client-protocol/#WebSocket-Parameters). This section standardises the method by which a MatrixRTC application obtains the LiveKit JWT @@ -166,12 +214,7 @@ sequenceDiagram U->>L: Connects to room using JWT ``` -#### Prerequisites - -* The `livekit_service_url` for the MatrixRTC backend has been discovered from one of the methods above. -* The Matrix client has obtained an OpenID token from the [Client-Server API](https://spec.matrix.org/v1.11/client-server-api/#openid). - -#### Request +##### Request The JWT token is obtained by making a `POST` request to the `/get_token` endpoint of the LiveKit service. @@ -182,10 +225,6 @@ fields: * `openid_token` — required `object`: the verbatim OpenID token response obtained from the [Client-Server API](https://spec.matrix.org/v1.11/client-server-api/#post_matrixclientv3useruseridopenidrequest_token). * `member` — required `object`: the contents of the `member` field from the `m.rtc.member` event. - * `delay_id` — optional `string`: the delayed event id of the MatrixRTC member leave event. - * `delay_timeout` — optional `string`: number of positive non-zero milliseconds the homeserver - should wait before sending the MatrixRTC member leave event - * `delay_cs_api_url` — optional `string`: The Matrix client-server API as used by the client Example request where `livekit_service_url` is `https://matrix-rtc.example.com/livekit/jwt`: ```http @@ -206,14 +245,11 @@ Content-Type: application/json "id": "xyzABCDEF10123", "claimed_device_id": "DEVICEID", "claimed_user_id": "@user:matrix.example.com" - }, - "delay_id": "1234567890", // optional - "delay_timeout": "7200000", // optional (2 hours) - "delay_cs_api_url": "https://matrix-client.matrix.org/" // optional + } } ``` -#### Successful response +##### Successful response If the request is successful, an HTTP `200 OK` response is returned with `Content-Type: application/json`. The response body contains: @@ -231,42 +267,6 @@ Content-Type: application/json } ``` -#### Error responses - -The LiveKit authorisation service MUST respond with appropriate HTTP status codes and structured -JSON bodies when an error occurs. All error responses MUST include a top-level `"errcode"` string -and a human-readable `"error"` description, following the conventions used in the -[Matrix Client-Server API](https://spec.matrix.org/v1.11/client-server-api/#error-codes). - -Common error responses: - -| HTTP Status | `errcode` | Meaning / Recommended handling | -|--------------|------------|--------------------------------| -| `400 Bad Request` | `M_BAD_JSON` | The request body was malformed, missing required fields, or contained invalid values (e.g. missing `room_id`, `slot_id`, or `openid_token`). | -| `401 Unauthorized` | `M_UNAUTHORIZED` | The request could not be authorised. This response is used for all cases where the OpenID token is invalid, expired, could not be verified, or where the requested room or slot is unknown or inaccessible. Clients may attempt to refresh their OpenID token and retry. | -| `429 Too Many Requests` | `M_LIMIT_EXCEEDED` | The client or homeserver has exceeded rate limits for LiveKit token requests. Please refer to the existing [spec](https://spec.matrix.org/v1.18/client-server-api/#common-error-codes) for further details.| -| `403 Forbidden` | `M_USER_LIMIT_EXCEEDED` | The user has exceeded a configured quota or usage limit. Please refer to the existing [spec](https://spec.matrix.org/v1.18/client-server-api/#common-error-codes) for further details.| -| `500 Internal Server Error` | `M_UNKNOWN` | An unexpected internal error occurred while generating the token. The client may retry after a short delay. | - -Example Error Response: -```http -HTTP/1.1 401 Unauthorized -Content-Type: application/json - -{ - "errcode": "M_UNAUTHORIZED", - "error": "The request could not be authorised." -} -``` -For privacy reasons, the authorisation service does not distinguish between invalid credentials, -unknown resources, or insufficient permissions. All such conditions result in a 401 Unauthorized -response with `M_UNAUTHORIZED`. This prevents clients from inferring the existence of specific -rooms, users, or slots based on error responses. - -The LiveKit authorisation service MAY include additional fields (such as -`reason`) for diagnostic purposes, but clients MUST be prepared to ignore unknown fields. -Implementations SHOULD NOT disclose sensitive information in the `"error"` field. - #### Optional Delegated MatrixRTC Membership Lifecycle Tracking using Cancellable Delayed Events As described in [MSC4143](https://github.com/matrix-org/matrix-spec-proposals/pull/4143), clients @@ -283,15 +283,90 @@ Since the LiveKit SFU already maintains authoritative knowledge of each particip state, the management of cancellable delayed events MAY be delegated to the LiveKit SFU Authorisation Service. This delegation allows the RTC transport layer to accurately manage and maintain MatrixRTC membership lifecycles across transient disconnects, ensuring a consistent and -reliable view of session state. A prerequisite for this delegation is that the client includes the -following fields: `delay_id`, `delay_timeout` and `delay_cs_api_url` as part of the authorisation -request. - -Upon successful issuance of a JWT token and once the LiveKit SFU Authorisation Service observes the -client's SFU connection, identified by the LiveKit room `livekit_alias` and the LiveKit identity -as specified in the next section (`base64(SHA256(JSON.serialize([user_id, claimed_device_id, member.id])))`), -it SHOULD issue a `reset` of the delayed event by sending the following POST request to the -homeserver of that client: +reliable view of session state. + +```mermaid +sequenceDiagram + participant U as 🧑 User + participant M as 🏢 Matrix Homeserver + participant A as 🔐 LiveKit Authorisation Service + participant L as 📡 LiveKit SFU + + U->>M: Schedules delayed disconnect event + M-->>U: Returns delay ID + U->>A: Delegates management of delayed disconnect event + A->>M: Renews delayed disconnect event + U->>U: Looses connection + L->>A: Notifies about participant disconnect + A->>M: Triggers sending of disconnect event +``` + +##### Request + +The delegation is carried out by making a `POST` request to the `/delegate_delayed_leave` endpoint +of the LiveKit service. + +The `Content-Type` of the request is `application/json` and the JSON body contains the following +fields: + * `room_id` — required `string`: the Matrix room ID where the `m.rtc.member` event is present. + * `slot_id` — required `string`: the slot ID from the `m.rtc.member` event. + * `openid_token` — required `object`: the verbatim OpenID token response obtained from the + [Client-Server API](https://spec.matrix.org/v1.11/client-server-api/#post_matrixclientv3useruseridopenidrequest_token). + * `member` — required `object`: the contents of the `member` field from the `m.rtc.member` event. + * `delay_id` — required `string`: the delayed event id of the MatrixRTC member leave event. + * `delay_timeout` — required `string`: number of positive non-zero milliseconds the homeserver + should wait before sending the MatrixRTC member leave event + * `delay_cs_api_url` — required `string`: The Matrix client-server API as used by the client. This + is required because publishing the [.well-known document](https://spec.matrix.org/v1.18/client-server-api/#well-known-uris) + for auto-discovery is not mandatory. Hence, there is, in general, no way to know how to access a + server's CS-API given its server name. + +Example request where `livekit_service_url` is `https://matrix-rtc.example.com/livekit/jwt`: +```http +POST /livekit/jwt/delegate_delayed_leave HTTP/1.1 +Host: matrix-rtc.example.com +Content-Type: application/json + +{ + "room_id": "!tDLCaLXijNtYcJZEey:example.com", + "slot_id": "the_id", + "openid_token": { + "access_token": "FPkexLLvKbAHKclQhpvgfWxx", + "expires_in": 3600, + "matrix_server_name": "matrix.example.com", + "token_type": "Bearer" + }, + "member": { + "id": "xyzABCDEF10123", + "claimed_device_id": "DEVICEID", + "claimed_user_id": "@user:matrix.example.com" + }, + "delay_id": "1234567890", + "delay_timeout": "7200000", + "delay_cs_api_url": "https://matrix-client.matrix.org/" +} +``` + +##### Successful response + +If the request is successful, an HTTP `200 OK` response is returned with +`Content-Type: application/json`. The response body contains an empty +JSON object for future extension. + +Example response: +```http +HTTP/1.1 200 OK +Content-Type: application/json + +{} +``` + +Once the LiveKit SFU Authorisation Service observes the client's SFU connection, identified by +the LiveKit room `livekit_alias` and the LiveKit identity as specified in the next section +(`base64(SHA256(JSON.serialize([user_id, claimed_device_id, member.id])))`), it SHOULD issue +a `reset` of the delayed event by sending the following POST request to the homeserver of +that client: + ```http POST /_matrix/client/v1/delayed_events/{delay_id}/restart HTTP/1.1 Host: matrix-rtc.example.com @@ -305,6 +380,7 @@ reset while the client remains connected with sufficient headroom (e.g., 80% of ensure the reset occurs well before `delay_timeout` expires. If the SFU detects that the client has disconnected before the timer is reset, the Authorisation Service MUST trigger the `disconnect` event by sending the following request to the homeserver: + ```http POST /_matrix/client/v1/delayed_events/{delay_id}/send HTTP/1.1 Host: matrix-rtc.example.com From 160b144037cd9e089964a79f1be4fc18d93baabd Mon Sep 17 00:00:00 2001 From: Johannes Marbach Date: Tue, 9 Jun 2026 17:04:19 +0200 Subject: [PATCH 032/122] Add OpenID steps to delegation chart Signed-off-by: Johannes Marbach --- proposals/4195-matrixrtc-livekit.md | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-) diff --git a/proposals/4195-matrixrtc-livekit.md b/proposals/4195-matrixrtc-livekit.md index b8d80f07435..de9ad3ff4aa 100644 --- a/proposals/4195-matrixrtc-livekit.md +++ b/proposals/4195-matrixrtc-livekit.md @@ -292,9 +292,14 @@ sequenceDiagram participant A as 🔐 LiveKit Authorisation Service participant L as 📡 LiveKit SFU + U->>M: Requests OpenID token + M-->>U: Returns OpenID token U->>M: Schedules delayed disconnect event M-->>U: Returns delay ID - U->>A: Delegates management of delayed disconnect event + U->>A: Sends OpenID token & delegation parameters + A->>M: Validates token via OpenID API + M-->>A: Confirms user identity + A-->>U: Confirms delegation A->>M: Renews delayed disconnect event U->>U: Looses connection L->>A: Notifies about participant disconnect From a670fb068b3ddf0a5e2b6ce35b83ffa82a6db4f8 Mon Sep 17 00:00:00 2001 From: Johannes Marbach Date: Tue, 9 Jun 2026 17:12:55 +0200 Subject: [PATCH 033/122] Add alternative of joining both endpoints Signed-off-by: Johannes Marbach --- proposals/4195-matrixrtc-livekit.md | 11 +++++++++++ 1 file changed, 11 insertions(+) diff --git a/proposals/4195-matrixrtc-livekit.md b/proposals/4195-matrixrtc-livekit.md index de9ad3ff4aa..60adb43a60c 100644 --- a/proposals/4195-matrixrtc-livekit.md +++ b/proposals/4195-matrixrtc-livekit.md @@ -582,6 +582,17 @@ chances of accidentally using the wrong order of array elements. On the downside Canonical JSON serialisation of objects is significantly more complex than for arrays. Overall, this would likely result in a higher chance of implementation errors. +### Combination of token request and delegation + +Instead of using separate endpoints, the token request and the delegation of the delayed disconnect +event could be combined in a single endpoint. This creates a race condition, however. As per +[MSC4143](https://github.com/matrix-org/matrix-spec-proposals/pull/4143), the disconnect event +carries a relation to the associated join event. This means a client would have to send its join +event before requesting an SFU token. The associated Livekit room will only be created when the +token is requested though. As a result, a client on another homeserver could attempt to connect to +the SFU in the meantime. Since the Livekit room doesn't yet exist, this would result in an error. +Separating the endpoints avoids this issue. + ## Security considerations ### Resource usage From 1f97f4b706282eafb1edd945fdb58e02f4b4d820 Mon Sep 17 00:00:00 2001 From: Johannes Marbach Date: Tue, 9 Jun 2026 17:23:41 +0200 Subject: [PATCH 034/122] Apply suggestion from @Johennes --- proposals/4195-matrixrtc-livekit.md | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/proposals/4195-matrixrtc-livekit.md b/proposals/4195-matrixrtc-livekit.md index 60adb43a60c..e1b573f03b4 100644 --- a/proposals/4195-matrixrtc-livekit.md +++ b/proposals/4195-matrixrtc-livekit.md @@ -612,7 +612,8 @@ The homeserver restriction could be applied by checking the `matrix_server_name` token before validating the token. The Matrix `room_id` could be validated too, and checking that the Matrix user from the OpenID token -is a member of the room. +is a member of the room. This would require a dedicated way for the LiveKit Authorisation Service to +perform these checks via the homeserver though. ### Pseudonymity From 3327a8962f974c3e31d4657813dc866724806aff Mon Sep 17 00:00:00 2001 From: Johannes Marbach Date: Tue, 9 Jun 2026 17:29:45 +0200 Subject: [PATCH 035/122] Add lack of HKDF in SDKs as potential issue Signed-off-by: Johannes Marbach --- proposals/4195-matrixrtc-livekit.md | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/proposals/4195-matrixrtc-livekit.md b/proposals/4195-matrixrtc-livekit.md index e1b573f03b4..6e372695d87 100644 --- a/proposals/4195-matrixrtc-livekit.md +++ b/proposals/4195-matrixrtc-livekit.md @@ -565,6 +565,13 @@ In summary, this MSC’s reliance on LiveKit represents a practical implementati long-term commitment to a specific third-party protocol. The current design remains open to future evolution toward a Matrix-native or jointly standardized MatrixRTC transport. +### Lack of HKDF support in some LiveKit client SDKs + +Some LiveKit SDKs currently only support PBKDF2 but don't allow using HKDF. One example of this is +the Flutter SDK (see [livekit/client-sdk-flutter#974](https://github.com/livekit/client-sdk-flutter/issues/974)). +Upstream implementation efforts such as [livekit/rust-sdks#796](https://github.com/livekit/rust-sdks/issues/796) +will be required to close these gaps. + ## Alternatives ### String concatenation of hashing inputs From 050858e02e63f8eb64f2b3c0dd0f5e5e6ff4f9c8 Mon Sep 17 00:00:00 2001 From: Johannes Marbach Date: Wed, 10 Jun 2026 09:01:30 +0200 Subject: [PATCH 036/122] Add highlander principle for delegations Signed-off-by: Johannes Marbach --- proposals/4195-matrixrtc-livekit.md | 13 ++++++++++--- 1 file changed, 10 insertions(+), 3 deletions(-) diff --git a/proposals/4195-matrixrtc-livekit.md b/proposals/4195-matrixrtc-livekit.md index 6e372695d87..57c73b8cb8e 100644 --- a/proposals/4195-matrixrtc-livekit.md +++ b/proposals/4195-matrixrtc-livekit.md @@ -327,6 +327,7 @@ fields: server's CS-API given its server name. Example request where `livekit_service_url` is `https://matrix-rtc.example.com/livekit/jwt`: + ```http POST /livekit/jwt/delegate_delayed_leave HTTP/1.1 Host: matrix-rtc.example.com @@ -352,11 +353,17 @@ Content-Type: application/json } ``` + + ##### Successful response -If the request is successful, an HTTP `200 OK` response is returned with -`Content-Type: application/json`. The response body contains an empty -JSON object for future extension. +The service MUST only maintain a single delegated event per `room_id`, `slot_id`, +`member` and MXID (as determined by verifyng the OpenID token). Requests to delegate +a different `delay_id` MUST invalidate earlier delegations for the same parameters. + +If the delegation request is successful, an HTTP `200 OK` response is returned with +`Content-Type: application/json`. The response body contains an empty JSON object +for future extension. Example response: ```http From e5d19093efe2a707e692fe1758fcb93b78d717ce Mon Sep 17 00:00:00 2001 From: Johannes Marbach Date: Wed, 10 Jun 2026 09:11:57 +0200 Subject: [PATCH 037/122] Change example to silence spell checker Signed-off-by: Johannes Marbach --- proposals/4195-matrixrtc-livekit.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/proposals/4195-matrixrtc-livekit.md b/proposals/4195-matrixrtc-livekit.md index 57c73b8cb8e..1d8f1d04017 100644 --- a/proposals/4195-matrixrtc-livekit.md +++ b/proposals/4195-matrixrtc-livekit.md @@ -688,6 +688,6 @@ https://spec.matrix.org/v1.18/appendices/#canonical-json | Case | Input (logical) | Canonical JSON | SHA-256 (hex) | Base64 (unpadded) | |------|------------------|----------------|---------------|-------------------| -| LiveKit alias (no random bits) | `["!roomid:example.com", "slot123"]` | `["!roomid:example.com","slot123"]` | `0140e634342254799661113eaca06f89e597f00512cdea5e9ee8fabbe77f9fd7` | `AUDmNDQiVHmWYRE+rKBvieWX8AUSzepenuj6u+d/n9c` | +| LiveKit alias (no random bits) | `["!roomid:example.com", "slot1234"]` | `["!roomid:example.com","slot1234"]` | `3bce37ed6dfe8e6ccc563a083f7b4dc1b9be5f11d093688aa4e03b6aac37a927` | `O8437W3+jmzMVjoIP3tNwbm+XxHQk2iKpOA7aqw3qSc` | | LiveKit alias (with random bits) | `["!roomid:example.com", "slot123", "random123"]` | `["!roomid:example.com","slot123","random123"]` | `20c78377e2b7308a894c8db4117048adea4a92184e46f7f7abc7f1deb96b8539` | `IMeDd+K3MIqJTI20EXBIrepKkhhORvf3q8fx3rlrhTk` | | Participant identity | `["@alice:example.com", "DEVICE123", "memberABC"]` | `["@alice:example.com","DEVICE123","memberABC"]` | `27e4f8e6d1abbb173e1eb50ea89265c90495df79bbdbc0a67b8fafb7cfd25ab5` | `J+T45tGruxc+HrUOqJJlyQSV33m728Cme4+vt8/SWrU` | \ No newline at end of file From 935b0f1d821252c8d601abd600c425497c571e33 Mon Sep 17 00:00:00 2001 From: Johannes Marbach Date: Wed, 10 Jun 2026 09:34:20 +0200 Subject: [PATCH 038/122] Kill blank lines Signed-off-by: Johannes Marbach --- proposals/4195-matrixrtc-livekit.md | 2 -- 1 file changed, 2 deletions(-) diff --git a/proposals/4195-matrixrtc-livekit.md b/proposals/4195-matrixrtc-livekit.md index 1d8f1d04017..c2f8074e3f2 100644 --- a/proposals/4195-matrixrtc-livekit.md +++ b/proposals/4195-matrixrtc-livekit.md @@ -353,8 +353,6 @@ Content-Type: application/json } ``` - - ##### Successful response The service MUST only maintain a single delegated event per `room_id`, `slot_id`, From a7252d5f3834cc40504da2f46af418868046ed6e Mon Sep 17 00:00:00 2001 From: Johannes Marbach Date: Thu, 11 Jun 2026 16:31:30 +0200 Subject: [PATCH 039/122] Clarify retry behaviour --- proposals/4195-matrixrtc-livekit.md | 11 ++++++++--- 1 file changed, 8 insertions(+), 3 deletions(-) diff --git a/proposals/4195-matrixrtc-livekit.md b/proposals/4195-matrixrtc-livekit.md index c2f8074e3f2..f5528fb0537 100644 --- a/proposals/4195-matrixrtc-livekit.md +++ b/proposals/4195-matrixrtc-livekit.md @@ -408,9 +408,14 @@ the SFU authorisation request MUST be rejected with the error code `M_UNSUPPORTE `MatrixRTC membership lifecycle delegation failed: homeserver does not support delayed events.`. Implementations MAY retry failed delayed event POST requests using an exponential backoff strategy -in the event of transient network failures. However, retry attempts MUST cease once either the -configured `delay_timeout` has elapsed or the maximum sticky duration of one hour for the delayed -event has been reached, whichever occurs first. +in the event of transient network failures. However, retry attempts MUST cease once the configured +`delay_timeout` has elapsed. + +As per [MSC4143](https://github.com/matrix-org/matrix-spec-proposals/pull/4143), participants are +considered disconnected once their member event becomes unsticky. Therefore, as a heuristic, +implementations SHOULD also stop retries once the maximum sticky duration of one hour has elapsed. +The underlying thought here is that if the authorisation service cannot reach the homeserver, +the participant likely cannot reach it either and, thus, cannot update their sticky member event. ### Pseudonymous LiveKit Participant Identity From 7ccda7d7cbbd80ba5819fd63511d6b076654ef47 Mon Sep 17 00:00:00 2001 From: Johannes Marbach Date: Thu, 11 Jun 2026 16:49:26 +0200 Subject: [PATCH 040/122] Recommend delay timeout --- proposals/4195-matrixrtc-livekit.md | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/proposals/4195-matrixrtc-livekit.md b/proposals/4195-matrixrtc-livekit.md index f5528fb0537..402caffab2b 100644 --- a/proposals/4195-matrixrtc-livekit.md +++ b/proposals/4195-matrixrtc-livekit.md @@ -320,7 +320,9 @@ fields: * `member` — required `object`: the contents of the `member` field from the `m.rtc.member` event. * `delay_id` — required `string`: the delayed event id of the MatrixRTC member leave event. * `delay_timeout` — required `string`: number of positive non-zero milliseconds the homeserver - should wait before sending the MatrixRTC member leave event + should wait before sending the MatrixRTC member leave event. Clients SHOULD not use values smaller + than 1 hour to avoid unnecessarily frequent `/reset`s of the delayed event. Service implementations + MAY reject requests with a timeout below 1 hour with `M_BAD_JSON`. * `delay_cs_api_url` — required `string`: The Matrix client-server API as used by the client. This is required because publishing the [.well-known document](https://spec.matrix.org/v1.18/client-server-api/#well-known-uris) for auto-discovery is not mandatory. Hence, there is, in general, no way to know how to access a From ca058dd02155645f5a471f0db73f1af359f8da28 Mon Sep 17 00:00:00 2001 From: Johannes Marbach Date: Fri, 12 Jun 2026 09:23:35 +0200 Subject: [PATCH 041/122] A joint endpoint complicates error handling Signed-off-by: Johannes Marbach --- proposals/4195-matrixrtc-livekit.md | 3 +++ 1 file changed, 3 insertions(+) diff --git a/proposals/4195-matrixrtc-livekit.md b/proposals/4195-matrixrtc-livekit.md index 402caffab2b..daa41130719 100644 --- a/proposals/4195-matrixrtc-livekit.md +++ b/proposals/4195-matrixrtc-livekit.md @@ -612,6 +612,9 @@ token is requested though. As a result, a client on another homeserver could att the SFU in the meantime. Since the Livekit room doesn't yet exist, this would result in an error. Separating the endpoints avoids this issue. +Additionally, a joint endpoint introduces the problem of having to handle the case where one of +the two operations succeeds but the other fails. + ## Security considerations ### Resource usage From a9c98da061f77bc02139ea7191478aa625a848ef Mon Sep 17 00:00:00 2001 From: Johannes Marbach Date: Fri, 12 Jun 2026 09:29:18 +0200 Subject: [PATCH 042/122] Detail OpenID verification Signed-off-by: Johannes Marbach --- proposals/4195-matrixrtc-livekit.md | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/proposals/4195-matrixrtc-livekit.md b/proposals/4195-matrixrtc-livekit.md index daa41130719..a04721c7661 100644 --- a/proposals/4195-matrixrtc-livekit.md +++ b/proposals/4195-matrixrtc-livekit.md @@ -152,6 +152,13 @@ This section describes endpoints on the SFU Authorisation Service. * The `livekit_service_url` for the MatrixRTC backend has been discovered from one of the methods above. * The Matrix client has obtained an OpenID token from the [Client-Server API](https://spec.matrix.org/v1.11/client-server-api/#openid). +##### OpenID token verification + +An all endpoints listed below, the service MUST validate the supplied OpenID token with the +homeserver using [`/_matrix/federation/v1/openid/userinfo`](https://spec.matrix.org/v1.18/server-server-api/#get_matrixfederationv1openiduserinfo). +Additionally, it MUST verify that the returned user ID matches `claimed_user_id`. If either +check fails, the service MUST reject the request with `M_UNAUTHORIZED`. + ##### Error responses The LiveKit authorisation service MUST respond with appropriate HTTP status codes and structured From 4f1dca4f5258d59c68704d3349fc9929f933e67b Mon Sep 17 00:00:00 2001 From: Johannes Marbach Date: Fri, 19 Jun 2026 13:24:30 +0200 Subject: [PATCH 043/122] Use restart rather than reset Signed-off-by: Johannes Marbach --- proposals/4195-matrixrtc-livekit.md | 16 ++++++++-------- 1 file changed, 8 insertions(+), 8 deletions(-) diff --git a/proposals/4195-matrixrtc-livekit.md b/proposals/4195-matrixrtc-livekit.md index a04721c7661..c7a7fefb3a0 100644 --- a/proposals/4195-matrixrtc-livekit.md +++ b/proposals/4195-matrixrtc-livekit.md @@ -279,11 +279,11 @@ Content-Type: application/json As described in [MSC4143](https://github.com/matrix-org/matrix-spec-proposals/pull/4143), clients SHOULD use cancellable delayed events to implement a "deadman switch" for precise MatrixRTC membership tracking. This involves sending a disconnect event ahead of the connect event as a -delayed event with a reasonable timeout (e.g., 15--30 seconds), and periodically resetting the -delayed event's timer. If the timer expires due to a missing reset, the disconnect event is +delayed event with a reasonable timeout (e.g., 15--30 seconds), and periodically restart the +delayed event's timer. If the timer expires due to a missing restart, the disconnect event is automatically emitted, marking the participant as disconnected and ensuring accurate session state even in cases of sudden disconnection, crashes, or network failures. However, relying on clients to -reset the delayed event timer can be error-prone in adverse network conditions, particularly due to +restart the delayed event timer can be error-prone in adverse network conditions, particularly due to TCP connection instability. Since the LiveKit SFU already maintains authoritative knowledge of each participant's connection @@ -328,7 +328,7 @@ fields: * `delay_id` — required `string`: the delayed event id of the MatrixRTC member leave event. * `delay_timeout` — required `string`: number of positive non-zero milliseconds the homeserver should wait before sending the MatrixRTC member leave event. Clients SHOULD not use values smaller - than 1 hour to avoid unnecessarily frequent `/reset`s of the delayed event. Service implementations + than 1 hour to avoid unnecessarily frequent `/restart`s of the delayed event. Service implementations MAY reject requests with a timeout below 1 hour with `M_BAD_JSON`. * `delay_cs_api_url` — required `string`: The Matrix client-server API as used by the client. This is required because publishing the [.well-known document](https://spec.matrix.org/v1.18/client-server-api/#well-known-uris) @@ -383,7 +383,7 @@ Content-Type: application/json Once the LiveKit SFU Authorisation Service observes the client's SFU connection, identified by the LiveKit room `livekit_alias` and the LiveKit identity as specified in the next section (`base64(SHA256(JSON.serialize([user_id, claimed_device_id, member.id])))`), it SHOULD issue -a `reset` of the delayed event by sending the following POST request to the homeserver of +a `/restart` of the delayed event by sending the following POST request to the homeserver of that client: ```http @@ -395,9 +395,9 @@ Content-Type: application/json ``` It then starts a timer corresponding to the specified `delay_timeout`. The timer is periodically -reset while the client remains connected with sufficient headroom (e.g., 80% of `delay_timeout`) to - ensure the reset occurs well before `delay_timeout` expires. If the SFU detects that the client has -disconnected before the timer is reset, the Authorisation Service MUST trigger the `disconnect` +restarted while the client remains connected with sufficient headroom (e.g., 80% of `delay_timeout`) to + ensure the restart occurs well before `delay_timeout` expires. If the SFU detects that the client has +disconnected before the timer is restarted, the Authorisation Service MUST trigger the `disconnect` event by sending the following request to the homeserver: ```http From 30ea61cc072e905c4da2ff83077f3acf432b8b53 Mon Sep 17 00:00:00 2001 From: Johannes Marbach Date: Wed, 24 Jun 2026 13:55:27 +0200 Subject: [PATCH 044/122] Remove delay_cs_api_url in favour of server discovery Signed-off-by: Johannes Marbach --- proposals/4195-matrixrtc-livekit.md | 19 ++++++++++++------- 1 file changed, 12 insertions(+), 7 deletions(-) diff --git a/proposals/4195-matrixrtc-livekit.md b/proposals/4195-matrixrtc-livekit.md index c7a7fefb3a0..6ab552209cb 100644 --- a/proposals/4195-matrixrtc-livekit.md +++ b/proposals/4195-matrixrtc-livekit.md @@ -330,10 +330,6 @@ fields: should wait before sending the MatrixRTC member leave event. Clients SHOULD not use values smaller than 1 hour to avoid unnecessarily frequent `/restart`s of the delayed event. Service implementations MAY reject requests with a timeout below 1 hour with `M_BAD_JSON`. - * `delay_cs_api_url` — required `string`: The Matrix client-server API as used by the client. This - is required because publishing the [.well-known document](https://spec.matrix.org/v1.18/client-server-api/#well-known-uris) - for auto-discovery is not mandatory. Hence, there is, in general, no way to know how to access a - server's CS-API given its server name. Example request where `livekit_service_url` is `https://matrix-rtc.example.com/livekit/jwt`: @@ -357,8 +353,7 @@ Content-Type: application/json "claimed_user_id": "@user:matrix.example.com" }, "delay_id": "1234567890", - "delay_timeout": "7200000", - "delay_cs_api_url": "https://matrix-client.matrix.org/" + "delay_timeout": "7200000" } ``` @@ -411,7 +406,10 @@ Content-Type: application/json This ensures that the MatrixRTC membership state remains accurate and consistent, even in the presence of network interruptions or client crashes. -Implementations SHOULD verify support for delayed events by querying the client’s homeserver +Implementations SHOULD resolve the location of the client-server API by using [.well-known discovery] +for the `matrix_server_name` supplied in the OpenID token. + +Additionally, implementations SHOULD verify support for delayed events by querying the homeserver's `_matrix/client/versions` endpoint. If the homeserver does not advertise support for delayed events, the SFU authorisation request MUST be rejected with the error code `M_UNSUPPORTED` and error message `MatrixRTC membership lifecycle delegation failed: homeserver does not support delayed events.`. @@ -591,6 +589,13 @@ the Flutter SDK (see [livekit/client-sdk-flutter#974](https://github.com/livekit Upstream implementation efforts such as [livekit/rust-sdks#796](https://github.com/livekit/rust-sdks/issues/796) will be required to close these gaps. +### Missing .well-known documents + +As per the current spec, publishing the location of the client-server API in a .well-known document is +not mandatory. Consequently, resolving the URL using .well-known discovery can fail. This should usually +only occur in corporate setups and private federations though. Implementations MAY allow hardcoding the +mapping from server name to client-server API URL to address these cases. + ## Alternatives ### String concatenation of hashing inputs From 5e21431f751aae1b73ff1b439ff75177da4070aa Mon Sep 17 00:00:00 2001 From: Johannes Marbach Date: Thu, 25 Jun 2026 09:46:39 +0200 Subject: [PATCH 045/122] Add error behaviour for server resolution Signed-off-by: Johannes Marbach --- proposals/4195-matrixrtc-livekit.md | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/proposals/4195-matrixrtc-livekit.md b/proposals/4195-matrixrtc-livekit.md index 6ab552209cb..b2e787cede7 100644 --- a/proposals/4195-matrixrtc-livekit.md +++ b/proposals/4195-matrixrtc-livekit.md @@ -407,7 +407,9 @@ This ensures that the MatrixRTC membership state remains accurate and consistent presence of network interruptions or client crashes. Implementations SHOULD resolve the location of the client-server API by using [.well-known discovery] -for the `matrix_server_name` supplied in the OpenID token. +for the `matrix_server_name` supplied in the OpenID token. If the resolution is performed when processing +the `/delegate_delayed_leave` request, resolution failures MUST result in the request being rejected +with 400 / `M_BAD_JSON`. Additionally, implementations SHOULD verify support for delayed events by querying the homeserver's `_matrix/client/versions` endpoint. If the homeserver does not advertise support for delayed events, From a8464ae82ca85eae2ca1882cd8a38305a14b8bd2 Mon Sep 17 00:00:00 2001 From: Johannes Marbach Date: Tue, 14 Jul 2026 13:51:06 +0200 Subject: [PATCH 046/122] Add missing well-known link --- proposals/4195-matrixrtc-livekit.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/proposals/4195-matrixrtc-livekit.md b/proposals/4195-matrixrtc-livekit.md index b2e787cede7..002939e93bf 100644 --- a/proposals/4195-matrixrtc-livekit.md +++ b/proposals/4195-matrixrtc-livekit.md @@ -411,6 +411,8 @@ for the `matrix_server_name` supplied in the OpenID token. If the resolution is the `/delegate_delayed_leave` request, resolution failures MUST result in the request being rejected with 400 / `M_BAD_JSON`. +[.well-known discovery]: https://spec.matrix.org/v1.18/client-server-api/#well-known-uris + Additionally, implementations SHOULD verify support for delayed events by querying the homeserver's `_matrix/client/versions` endpoint. If the homeserver does not advertise support for delayed events, the SFU authorisation request MUST be rejected with the error code `M_UNSUPPORTED` and error message From afdcf273e507152699ffb0cbfa3f364550f2b112 Mon Sep 17 00:00:00 2001 From: Johannes Marbach Date: Tue, 21 Jul 2026 09:13:20 +0200 Subject: [PATCH 047/122] First stab at eliminating the service in favor of CS/SS endpoints Signed-off-by: Johannes Marbach --- proposals/4195-matrixrtc-livekit.md | 389 ++++++++++++---------------- 1 file changed, 169 insertions(+), 220 deletions(-) diff --git a/proposals/4195-matrixrtc-livekit.md b/proposals/4195-matrixrtc-livekit.md index 002939e93bf..503d2da376a 100644 --- a/proposals/4195-matrixrtc-livekit.md +++ b/proposals/4195-matrixrtc-livekit.md @@ -74,16 +74,14 @@ serialization defined [above](#canonical-json-serialization). For improved metadata protection, the `livekit_alias` SHOULD be derived as `base64(SHA256(JSON.serialize([room_id, slot_id, truly_random_bits])))`, where the `truly random bits` -are maintained by the LiveKit SFU authorisation service. This approach enhances pseudonymity but -requires the service to be **stateful** in order to manage and persist the random bits. +are maintained by the server. The resulting value is opaque to the MatrixRTC application. Within the LiveKit namespace, the `livekit_alias` uniquely represents a MatrixRTC slot. Participants from the same Matrix deployment (using the same SFU to publish their media) are considered to use the same `livekit_alias` in order to limit the number of active LiveKit SFU connections. -The `livekit_alias` is shared with clients as part of their JWT token issued by the authorisation -service. +The `livekit_alias` is shared with clients as part of their JWT token issued by the server. ### Transport type: `livekit` @@ -97,16 +95,13 @@ The mechanism for advertising available RTC transports by homeservers is already The homeserver announces available LiveKit Transport as a JSON object with the following fields: * `type` — required `string`: this MUST be `livekit` -* `livekit_service_url` — required `string`: The URL of the service that issues JWT tokens for - connecting this LiveKit SFU. An example for `GET /_matrix/client/v1/rtc/transports` ```json5 { "rtc_transports": [ { - "type": "livekit", - "livekit_service_url": "https://matrix-rtc.example.com/livekit/jwt" + "type": "livekit" } ] } @@ -126,147 +121,155 @@ to the published media. Field Descriptions: * `type` — required `string`: this MUST be `"livekit"` -* `livekit_service_url` — required `string`: The URL of the service that issues JWT tokens for - connecting this LiveKit SFU. ``` { // rest of the m.rtc.member event "rtc_transports": [ { - "type": "livekit", - "livekit_service_url": "https://matrix-rtc.example.com/livekit/jwt", + "type": "livekit" } ] } ``` -### LiveKit SFU Authorisation Service +### Additions to the Client-Server and Server-Server API -This section describes endpoints on the SFU Authorisation Service. - -#### General requirements for all endpoints - -##### Prerequisites - -* The `livekit_service_url` for the MatrixRTC backend has been discovered from one of the methods above. -* The Matrix client has obtained an OpenID token from the [Client-Server API](https://spec.matrix.org/v1.11/client-server-api/#openid). - -##### OpenID token verification - -An all endpoints listed below, the service MUST validate the supplied OpenID token with the -homeserver using [`/_matrix/federation/v1/openid/userinfo`](https://spec.matrix.org/v1.18/server-server-api/#get_matrixfederationv1openiduserinfo). -Additionally, it MUST verify that the returned user ID matches `claimed_user_id`. If either -check fails, the service MUST reject the request with `M_UNAUTHORIZED`. +#### Acquiring a token for the SFU -##### Error responses +LiveKit SFUs require a JWT `access_token` to be provided when +[connecting to the WebSocket](https://docs.livekit.io/reference/internals/client-protocol/#WebSocket-Parameters). +This section standardises the method by which a MatrixRTC application obtains the LiveKit JWT +token. A high level overview is depicted in the following diagram -The LiveKit authorisation service MUST respond with appropriate HTTP status codes and structured -JSON bodies when an error occurs. All error responses MUST include a top-level `"errcode"` string -and a human-readable `"error"` description, following the conventions used in the -[Matrix Client-Server API](https://spec.matrix.org/v1.11/client-server-api/#error-codes). +```mermaid +sequenceDiagram + autonumber + + participant U as 🧑 Alice + + box floralwhite alice.com + participant H as 🏢 Homeserver + participant L as 📡 LiveKit SFU + end + + box floralwhite bob.com + participant H2 as 🏢 Homeserver + end + + + participant O as 👨‍🦰 Bob + + U->>H: /get_token + activate H + H->>H: Verify user's
room membership + H->>L: Request token + activate L + L-->>H: Return token & URL + deactivate L + H-->>U: Return token & URL + deactivate H + + U->>L: Publish media stream + + O->>H2: /get_token + activate H2 + H2->>H2: Verify user's
room membership + H2->>H: /get_token + activate H + H->>H: Verify servers's
room membership + H->>L: Request token + activate L + L-->>H: Return token & URL + deactivate L + H-->>H2: Return token & URL + deactivate H + H2-->>O: Return token & URL + deactivate H2 + + O->>L: Subscribe to media streams +``` -Common error responses: +The JWT token is obtained by making an authenticated `POST` request to a new Client-Server endpoint +`/_matrix/client/v1/rtc/livekit/get_token`. -| HTTP Status | `errcode` | Meaning / Recommended handling | -|--------------|------------|--------------------------------| -| `400 Bad Request` | `M_BAD_JSON` | The request body was malformed, missing required fields, or contained invalid values (e.g. missing `room_id`, `slot_id`, or `openid_token`). | -| `401 Unauthorized` | `M_UNAUTHORIZED` | The request could not be authorised. This response is used for all cases where the OpenID token is invalid, expired, could not be verified, or where the requested room or slot is unknown or inaccessible. Clients may attempt to refresh their OpenID token and retry. | -| `429 Too Many Requests` | `M_LIMIT_EXCEEDED` | The client or homeserver has exceeded rate limits. Please refer to the existing [spec](https://spec.matrix.org/v1.18/client-server-api/#common-error-codes) for further details.| -| `403 Forbidden` | `M_USER_LIMIT_EXCEEDED` | The user has exceeded a configured quota or usage limit. Please refer to the existing [spec](https://spec.matrix.org/v1.18/client-server-api/#common-error-codes) for further details.| -| `500 Internal Server Error` | `M_UNKNOWN` | An unexpected internal error occurred while generating the token. The client may retry after a short delay. | +The `Content-Type` of the request is `application/json` and the JSON body contains the following +fields: -Example Error Response: + * `server_name` — `string`: the [server name](https://spec.matrix.org/v1.19/appendices/#server-name) + of the `m.rtc.member` event's `sender`. Defaults to the server's own server name if omitted. + * `room_id` — required `string`: the Matrix room ID where the `m.rtc.member` event is present. + * `slot_id` — required `string`: the slot ID from the `m.rtc.member` event. + * `member` — required `object`: the contents of the `member` field from the `m.rtc.member` event. ```http -HTTP/1.1 401 Unauthorized -Content-Type: application/json +POST /_matrix/client/v1/rtc/livekit/get_token HTTP/1.1 { - "errcode": "M_UNAUTHORIZED", - "error": "The request could not be authorised." + "server_name": "example.com", + "room_id": "!tDLCaLXijNtYcJZEey:example.com", + "slot_id": "the_id", + "member": { + "id": "xyzABCDEF10123", + "claimed_device_id": "DEVICEID" + } } ``` -For privacy reasons, the authorisation service does not distinguish between invalid credentials, -unknown resources, or insufficient permissions. All such conditions result in a 401 Unauthorized -response with `M_UNAUTHORIZED`. This prevents clients from inferring the existence of specific -rooms, users, or slots based on error responses. -The LiveKit authorisation service MAY include additional fields (such as -`reason`) for diagnostic purposes, but clients MUST be prepared to ignore unknown fields. -Implementations SHOULD NOT disclose sensitive information in the `"error"` field. +Upon receiving the request, the server verifies that the requesting user is joined to the room +identified by `room_id`. If the user is not joined, the request MUST be rejected with HTTP 401 / +`M_UNAUTHORIZED`. -#### Acquiring a token for the SFU +If `server_name` is the server's own name, it obtains a token from its own SFU and if successful +returns an HTTP `200 OK` response is returned with `Content-Type: application/json`. The response body +contains: -LiveKit SFUs require a JWT `access_token` to be provided when -[connecting to the WebSocket](https://docs.livekit.io/reference/internals/client-protocol/#WebSocket-Parameters). -This section standardises the method by which a MatrixRTC application obtains the LiveKit JWT -token. A high level overview is depicted in the following diagram +* `jwt` — `string`: the JWT token to use for authentication with the SFU. +* `url` — `string`: the URL of the LiveKit SFU to use for the given slot. -```mermaid -sequenceDiagram - participant U as 🧑 User - participant M as 🏢 Matrix Homeserver - participant A as 🔐 LiveKit Authorisation Service - participant L as 📡 LiveKit SFU - - U->>M: Requests OpenID token - M-->>U: Returns OpenID token - U->>A: Sends OpenID token & room request - A->>M: Validates token via OpenID API - M-->>A: Confirms user identity - A->>A: Generates LiveKit JWT - A->>L: (If full-access user) Create room if missing - A-->>U: Returns LiveKit JWT - U->>L: Connects to room using JWT -``` +```http +HTTP/1.1 200 OK -##### Request +{ + "jwt": "thejwt", + "url": "wss://matrix-rtc.example.com/livekit/sfu" +} +``` -The JWT token is obtained by making a `POST` request to the `/get_token` endpoint of the LiveKit service. +If `server_name` points to a remote server, the server triggers a `POST` request to a new authenticated +Server-Server endpoint `/_matrix/federation/v1/rtc/livekit/get_token`. The `Content-Type` of the +request is `application/json` and the JSON body contains the following fields: -The `Content-Type` of the request is `application/json` and the JSON body contains the following -fields: * `room_id` — required `string`: the Matrix room ID where the `m.rtc.member` event is present. * `slot_id` — required `string`: the slot ID from the `m.rtc.member` event. - * `openid_token` — required `object`: the verbatim OpenID token response obtained from the - [Client-Server API](https://spec.matrix.org/v1.11/client-server-api/#post_matrixclientv3useruseridopenidrequest_token). * `member` — required `object`: the contents of the `member` field from the `m.rtc.member` event. -Example request where `livekit_service_url` is `https://matrix-rtc.example.com/livekit/jwt`: ```http -POST /livekit/jwt/get_token HTTP/1.1 -Host: matrix-rtc.example.com -Content-Type: application/json +POST /_matrix/federation/v1/rtc/livekit/get_token HTTP/1.1 { "room_id": "!tDLCaLXijNtYcJZEey:example.com", "slot_id": "the_id", - "openid_token": { - "access_token": "FPkexLLvKbAHKclQhpvgfWxx", - "expires_in": 3600, - "matrix_server_name": "matrix.example.com", - "token_type": "Bearer" - }, "member": { "id": "xyzABCDEF10123", - "claimed_device_id": "DEVICEID", - "claimed_user_id": "@user:matrix.example.com" + "claimed_device_id": "DEVICEID" } } ``` -##### Successful response +The receiving server verifies that the requesting server is joined to the room identified by `room_id`. +If either the receiving server or the requesting server are not joined, the request MUST be rejected with +HTTP 401 / `M_UNAUTHORIZED`. + +Otherwise, the receiving server obtains a token from its own SFU and if successful +returns an HTTP `200 OK` response is returned with `Content-Type: application/json`. The response body +contains: -If the request is successful, an HTTP `200 OK` response is returned with -`Content-Type: application/json`. The response body contains: * `jwt` — `string`: the JWT token to use for authentication with the SFU. * `url` — `string`: the URL of the LiveKit SFU to use for the given slot. -Example response: ```http HTTP/1.1 200 OK -Content-Type: application/json { "jwt": "thejwt", @@ -274,6 +277,8 @@ Content-Type: application/json } ``` +The requesting server then forwards the response to its client as above. + #### Optional Delegated MatrixRTC Membership Lifecycle Tracking using Cancellable Delayed Events As described in [MSC4143](https://github.com/matrix-org/matrix-spec-proposals/pull/4143), clients @@ -286,147 +291,113 @@ even in cases of sudden disconnection, crashes, or network failures. However, re restart the delayed event timer can be error-prone in adverse network conditions, particularly due to TCP connection instability. -Since the LiveKit SFU already maintains authoritative knowledge of each participant's connection -state, the management of cancellable delayed events MAY be delegated to the LiveKit SFU -Authorisation Service. This delegation allows the RTC transport layer to accurately manage and +Since the LiveKit SFU, which is tied to the homeserver, already maintains authoritative knowledge of +each participant's connection state, the management of cancellable delayed events MAY be delegated +to the homeserver. This delegation allows the RTC transport layer to accurately manage and maintain MatrixRTC membership lifecycles across transient disconnects, ensuring a consistent and reliable view of session state. ```mermaid sequenceDiagram - participant U as 🧑 User - participant M as 🏢 Matrix Homeserver - participant A as 🔐 LiveKit Authorisation Service - participant L as 📡 LiveKit SFU - - U->>M: Requests OpenID token - M-->>U: Returns OpenID token - U->>M: Schedules delayed disconnect event - M-->>U: Returns delay ID - U->>A: Sends OpenID token & delegation parameters - A->>M: Validates token via OpenID API - M-->>A: Confirms user identity - A-->>U: Confirms delegation - A->>M: Renews delayed disconnect event - U->>U: Looses connection - L->>A: Notifies about participant disconnect - A->>M: Triggers sending of disconnect event + autonumber + + participant U as 🧑 Alice + + box floralwhite alice.com + participant H as 🏢 Homeserver + participant L as 📡 LiveKit SFU + end + + U->>H: Send m.rtc.member event
to join session + activate H + H-->>U: ​ + deactivate H + + U->>H: Schedule delayed m.rtc.member event
to leave session + activate H + H-->>U: ​ + deactivate H + + Note over U,L: Obtain SFU token and URL as shown in the chart above + + U->>L: Publish media stream + + U->>H: /delegate_delayed_leave + activate H + H-->>U: Confirm delegation + H->>H: Reschedule delayed
leave event + H->>H: Reschedule delayed
leave event + H->>H: Reschedule delayed
leave event + + U->>U: Loses connectivity + + L->>H: Trigger disconnect webhook + H->>H: Trigger sending
leave event + deactivate H ``` ##### Request -The delegation is carried out by making a `POST` request to the `/delegate_delayed_leave` endpoint -of the LiveKit service. +The delegation is carried out by making a `POST` request to a new authenticated endpoint +`/_matrix/client/v1/rtc/livekit/delegate_delayed_leave`. The `Content-Type` of the request is `application/json` and the JSON body contains the following fields: + * `room_id` — required `string`: the Matrix room ID where the `m.rtc.member` event is present. * `slot_id` — required `string`: the slot ID from the `m.rtc.member` event. - * `openid_token` — required `object`: the verbatim OpenID token response obtained from the - [Client-Server API](https://spec.matrix.org/v1.11/client-server-api/#post_matrixclientv3useruseridopenidrequest_token). * `member` — required `object`: the contents of the `member` field from the `m.rtc.member` event. * `delay_id` — required `string`: the delayed event id of the MatrixRTC member leave event. - * `delay_timeout` — required `string`: number of positive non-zero milliseconds the homeserver - should wait before sending the MatrixRTC member leave event. Clients SHOULD not use values smaller - than 1 hour to avoid unnecessarily frequent `/restart`s of the delayed event. Service implementations - MAY reject requests with a timeout below 1 hour with `M_BAD_JSON`. - -Example request where `livekit_service_url` is `https://matrix-rtc.example.com/livekit/jwt`: ```http -POST /livekit/jwt/delegate_delayed_leave HTTP/1.1 -Host: matrix-rtc.example.com -Content-Type: application/json +POST /_matrix/client/v1/rtc/livekit/delegate_delayed_leave HTTP/1.1 { "room_id": "!tDLCaLXijNtYcJZEey:example.com", "slot_id": "the_id", - "openid_token": { - "access_token": "FPkexLLvKbAHKclQhpvgfWxx", - "expires_in": 3600, - "matrix_server_name": "matrix.example.com", - "token_type": "Bearer" - }, "member": { "id": "xyzABCDEF10123", - "claimed_device_id": "DEVICEID", - "claimed_user_id": "@user:matrix.example.com" + "claimed_device_id": "DEVICEID" }, - "delay_id": "1234567890", - "delay_timeout": "7200000" + "delay_id": "1234567890" } ``` +When delegating delayed events, Clients SHOULD NOT use values smaller than 1 hour for the `delay_timeout` +to avoid unnecessarily frequent restarts of the delayed event. Servers MAY reject requests when the delegated +event has a timeout below 1 hour with `M_BAD_JSON`. + ##### Successful response -The service MUST only maintain a single delegated event per `room_id`, `slot_id`, -`member` and MXID (as determined by verifyng the OpenID token). Requests to delegate -a different `delay_id` MUST invalidate earlier delegations for the same parameters. +The server MUST only maintain a single delegated event per `room_id`, `slot_id`, +`member` and MXID. Requests to delegate a different `delay_id` MUST invalidate earlier +delegations for the same parameters. If the delegation request is successful, an HTTP `200 OK` response is returned with `Content-Type: application/json`. The response body contains an empty JSON object for future extension. -Example response: ```http HTTP/1.1 200 OK -Content-Type: application/json {} ``` -Once the LiveKit SFU Authorisation Service observes the client's SFU connection, identified by +Once the homeserver observes the client's SFU connection (either by receiving a +[webhook](https://docs.livekit.io/intro/basics/rooms-participants-tracks/webhooks-events/) +from the SFU or by polling the connection status from the SFU), identified by the LiveKit room `livekit_alias` and the LiveKit identity as specified in the next section (`base64(SHA256(JSON.serialize([user_id, claimed_device_id, member.id])))`), it SHOULD issue -a `/restart` of the delayed event by sending the following POST request to the homeserver of -that client: - -```http -POST /_matrix/client/v1/delayed_events/{delay_id}/restart HTTP/1.1 -Host: matrix-rtc.example.com -Content-Type: application/json - -{} -``` +a restart of the delayed event. -It then starts a timer corresponding to the specified `delay_timeout`. The timer is periodically +It then starts a timer corresponding to the delayed event's `delay_timeout`. The timer is periodically restarted while the client remains connected with sufficient headroom (e.g., 80% of `delay_timeout`) to - ensure the restart occurs well before `delay_timeout` expires. If the SFU detects that the client has -disconnected before the timer is restarted, the Authorisation Service MUST trigger the `disconnect` -event by sending the following request to the homeserver: - -```http -POST /_matrix/client/v1/delayed_events/{delay_id}/send HTTP/1.1 -Host: matrix-rtc.example.com -Content-Type: application/json - -{} -``` - -This ensures that the MatrixRTC membership state remains accurate and consistent, even in the -presence of network interruptions or client crashes. - -Implementations SHOULD resolve the location of the client-server API by using [.well-known discovery] -for the `matrix_server_name` supplied in the OpenID token. If the resolution is performed when processing -the `/delegate_delayed_leave` request, resolution failures MUST result in the request being rejected -with 400 / `M_BAD_JSON`. - -[.well-known discovery]: https://spec.matrix.org/v1.18/client-server-api/#well-known-uris - -Additionally, implementations SHOULD verify support for delayed events by querying the homeserver's -`_matrix/client/versions` endpoint. If the homeserver does not advertise support for delayed events, -the SFU authorisation request MUST be rejected with the error code `M_UNSUPPORTED` and error message -`MatrixRTC membership lifecycle delegation failed: homeserver does not support delayed events.`. - -Implementations MAY retry failed delayed event POST requests using an exponential backoff strategy -in the event of transient network failures. However, retry attempts MUST cease once the configured -`delay_timeout` has elapsed. - -As per [MSC4143](https://github.com/matrix-org/matrix-spec-proposals/pull/4143), participants are -considered disconnected once their member event becomes unsticky. Therefore, as a heuristic, -implementations SHOULD also stop retries once the maximum sticky duration of one hour has elapsed. -The underlying thought here is that if the authorisation service cannot reach the homeserver, -the participant likely cannot reach it either and, thus, cannot update their sticky member event. +ensure the restart occurs well before `delay_timeout` expires. If the homeserver detects that the client +has disconnected before the timer is restarted (either by receiving a +[webhook](https://docs.livekit.io/intro/basics/rooms-participants-tracks/webhooks-events/) +from the SFU or by polling the connection status from the SFU), the server MUST trigger sending of the +delegated delayed leave event. This ensures that the MatrixRTC membership state remains accurate and +consistent, even in the presence of network interruptions or client crashes. ### Pseudonymous LiveKit Participant Identity @@ -537,22 +508,20 @@ Two approaches are possible: `rtc_transports` field) ensures that the data is encrypted shared state, it is subject to client-side consensus and may flip over time. Overall, it does **not** improve the reliability of propagating and converging those random bits. - * This approach keeps the LiveKit Authorisation Service stateless * Requires the removal of the `room_id` field from the access request, which prevents additional access checks, such as verifying that the user is actually part of the claimed Matrix room. -2. **Authorisation-service-provided random bits** - * The Authorisation Service generates and persists the `truly_random_bits` for each `(room_id, +2. **Server-provided random bits** + * The server generates and persists the `truly_random_bits` for each `(room_id, slot_id)` tuple * Guarantees consistent alias derivation across clients without requiring client-side coordination. - * The service becomes stateful, as it must retain the `truly_random_bits` - * The benefit of improved pseudonymity only applies if the LiveKit SFU authorisation service is + * The benefit of improved pseudonymity only applies if the server is operated separately from the actual LiveKit SFU. * Preserves the `room_id` in the access request, allowing additional access checks, such as verifying that the user is actually part of the claimed Matrix room. Given that pseudonymous LiveKit participant IDs already exist, the design prioritizes **reliability -over additional pseudonymity** by using Authorisation-service-provided random bits, ensuring +over additional pseudonymity** by using server-provided random bits, ensuring consistent `livekit_alias` across clients while enabling additional access checks. ### Reliance on the LiveKit Protocol and Implementation @@ -633,26 +602,6 @@ the two operations succeeds but the other fails. ## Security considerations -### Resource usage - -To prevent abuse of SFU resources, the LiveKit Authorisation service should validate the OpenID -token as part of requests to `/get_token`. - -The Server-Server API endpoint -[/\_matrix/federation/v1/openid/userinfo](https://spec.matrix.org/v1.11/server-server-api/#get_matrixfederationv1openiduserinfo) -can be used for this purpose. - -An access control policy should be applied based on the result of the OpenID token validation. For -example, access might be restricted to users of a particular homeserver or to users with a specific -role. - -The homeserver restriction could be applied by checking the `matrix_server_name` field of the OpenID -token before validating the token. - -The Matrix `room_id` could be validated too, and checking that the Matrix user from the OpenID token -is a member of the room. This would require a dedicated way for the LiveKit Authorisation Service to -perform these checks via the homeserver though. - ### Pseudonymity The LiveKit participant identity is a function of one's Matrix user ID, device ID, and session From bc196acba63f39bb7be5234bfdee951ee9f1b104 Mon Sep 17 00:00:00 2001 From: Johannes Marbach Date: Wed, 19 Aug 2026 16:32:00 +0200 Subject: [PATCH 048/122] Refer to MSC4519 Co-authored-by: Travis Ralston --- proposals/4195-matrixrtc-livekit.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/proposals/4195-matrixrtc-livekit.md b/proposals/4195-matrixrtc-livekit.md index 503d2da376a..cf2673f0f06 100644 --- a/proposals/4195-matrixrtc-livekit.md +++ b/proposals/4195-matrixrtc-livekit.md @@ -40,7 +40,7 @@ Example for two participants from different homeservers A and B ## Proposal This MSC defines the **LiveKit RTC Transport**, which can appear as one of the **RTC Transports** -offered by a homeserver and being used as transport by clients. +offered by a homeserver and being used as transport by clients, as per [MSC4519](https://github.com/matrix-org/matrix-spec-proposals/pull/4519). ### Canonical JSON Serialization From d4253954f66705181854c2f4aa3e7c1470607022 Mon Sep 17 00:00:00 2001 From: Johannes Marbach Date: Wed, 19 Aug 2026 17:46:59 +0200 Subject: [PATCH 049/122] List MSC4519 as a dependency Co-authored-by: Travis Ralston --- proposals/4195-matrixrtc-livekit.md | 3 +++ 1 file changed, 3 insertions(+) diff --git a/proposals/4195-matrixrtc-livekit.md b/proposals/4195-matrixrtc-livekit.md index cf2673f0f06..a8bd1f54c16 100644 --- a/proposals/4195-matrixrtc-livekit.md +++ b/proposals/4195-matrixrtc-livekit.md @@ -641,6 +641,9 @@ required as these fields will only be accessed via some other unstable prefix. This MSC builds on [MSC4143](https://github.com/matrix-org/matrix-spec-proposals/pull/4143) (which at the time of writing has not yet been accepted into the spec). +This MSC additionally requires [MSC4519](https://github.com/matrix-org/matrix-spec-proposals/pull/4519) +to be accepted. + ## Appendix: Hash Derivation Test Vectors This appendix provides **verified test vectors** for: From 87c20ba53393d45f8d0d1ee5bc838ebd540bb406 Mon Sep 17 00:00:00 2001 From: Johannes Marbach Date: Mon, 24 Aug 2026 11:16:07 +0200 Subject: [PATCH 050/122] Add url parameter to be able to distinguish multiple SFUs per homeserver --- proposals/4195-matrixrtc-livekit.md | 27 +++++++++++++++++++-------- 1 file changed, 19 insertions(+), 8 deletions(-) diff --git a/proposals/4195-matrixrtc-livekit.md b/proposals/4195-matrixrtc-livekit.md index a8bd1f54c16..03c0f8af177 100644 --- a/proposals/4195-matrixrtc-livekit.md +++ b/proposals/4195-matrixrtc-livekit.md @@ -94,14 +94,19 @@ The mechanism for advertising available RTC transports by homeservers is already [MSC4143](https://github.com/matrix-org/matrix-spec-proposals/pull/4143). The homeserver announces available LiveKit Transport as a JSON object with the following fields: + * `type` — required `string`: this MUST be `livekit` +* `url` - required `string`: WebSocket URL of the LiveKit SFU. This enables running more than one SFU + per homeserver. An example for `GET /_matrix/client/v1/rtc/transports` + ```json5 { "rtc_transports": [ { - "type": "livekit" + "type": "livekit", + "url": "ws://livekit.example.com } ] } @@ -120,14 +125,17 @@ Other clients in the same MatrixRTC slot discover and subscribe to each other’ to the published media. Field Descriptions: + * `type` — required `string`: this MUST be `"livekit"` +* `url` - required `string`: WebSocket URL of the LiveKit SFU. -``` +```json5 { // rest of the m.rtc.member event "rtc_transports": [ { - "type": "livekit" + "type": "livekit", + "url": "ws://livekit.example.com } ] } @@ -198,6 +206,7 @@ fields: * `server_name` — `string`: the [server name](https://spec.matrix.org/v1.19/appendices/#server-name) of the `m.rtc.member` event's `sender`. Defaults to the server's own server name if omitted. + * `url` - required `string`: WebSocket URL of the LiveKit SFU. * `room_id` — required `string`: the Matrix room ID where the `m.rtc.member` event is present. * `slot_id` — required `string`: the slot ID from the `m.rtc.member` event. * `member` — required `object`: the contents of the `member` field from the `m.rtc.member` event. @@ -207,6 +216,7 @@ POST /_matrix/client/v1/rtc/livekit/get_token HTTP/1.1 { "server_name": "example.com", + "url": "ws://livekit.example.com, "room_id": "!tDLCaLXijNtYcJZEey:example.com", "slot_id": "the_id", "member": { @@ -220,9 +230,8 @@ Upon receiving the request, the server verifies that the requesting user is join identified by `room_id`. If the user is not joined, the request MUST be rejected with HTTP 401 / `M_UNAUTHORIZED`. -If `server_name` is the server's own name, it obtains a token from its own SFU and if successful -returns an HTTP `200 OK` response is returned with `Content-Type: application/json`. The response body -contains: +If `server_name` is the server's own name and `url` matches one of the server's own SFUs, it obtains a +token from that SFU. If successful, the server returns an HTTP `200 OK` response with `Content-Type: application/json`. The response body contains: * `jwt` — `string`: the JWT token to use for authentication with the SFU. * `url` — `string`: the URL of the LiveKit SFU to use for the given slot. @@ -240,6 +249,7 @@ If `server_name` points to a remote server, the server triggers a `POST` request Server-Server endpoint `/_matrix/federation/v1/rtc/livekit/get_token`. The `Content-Type` of the request is `application/json` and the JSON body contains the following fields: + * `url` - required `string`: WebSocket URL of the LiveKit SFU. * `room_id` — required `string`: the Matrix room ID where the `m.rtc.member` event is present. * `slot_id` — required `string`: the slot ID from the `m.rtc.member` event. * `member` — required `object`: the contents of the `member` field from the `m.rtc.member` event. @@ -248,6 +258,7 @@ request is `application/json` and the JSON body contains the following fields: POST /_matrix/federation/v1/rtc/livekit/get_token HTTP/1.1 { + "url": "ws://livekit.example.com, "room_id": "!tDLCaLXijNtYcJZEey:example.com", "slot_id": "the_id", "member": { @@ -261,8 +272,8 @@ The receiving server verifies that the requesting server is joined to the room i If either the receiving server or the requesting server are not joined, the request MUST be rejected with HTTP 401 / `M_UNAUTHORIZED`. -Otherwise, the receiving server obtains a token from its own SFU and if successful -returns an HTTP `200 OK` response is returned with `Content-Type: application/json`. The response body +Otherwise, if `url` matches one of the receiving server's SFUs, it obtains a token from that SFU. If successful, +an HTTP `200 OK` response is returned with `Content-Type: application/json`. The response body contains: * `jwt` — `string`: the JWT token to use for authentication with the SFU. From 72ded0d173a0c72ef19a4add52097eb66a29768e Mon Sep 17 00:00:00 2001 From: Johannes Marbach Date: Mon, 24 Aug 2026 11:18:46 +0200 Subject: [PATCH 051/122] Switch to 403 / M_FORBIDDEN --- proposals/4195-matrixrtc-livekit.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/proposals/4195-matrixrtc-livekit.md b/proposals/4195-matrixrtc-livekit.md index 03c0f8af177..71983465f43 100644 --- a/proposals/4195-matrixrtc-livekit.md +++ b/proposals/4195-matrixrtc-livekit.md @@ -227,8 +227,8 @@ POST /_matrix/client/v1/rtc/livekit/get_token HTTP/1.1 ``` Upon receiving the request, the server verifies that the requesting user is joined to the room -identified by `room_id`. If the user is not joined, the request MUST be rejected with HTTP 401 / -`M_UNAUTHORIZED`. +identified by `room_id`. If the user is not joined, the request MUST be rejected with HTTP 403 / +`M_FORBIDDEN`. If `server_name` is the server's own name and `url` matches one of the server's own SFUs, it obtains a token from that SFU. If successful, the server returns an HTTP `200 OK` response with `Content-Type: application/json`. The response body contains: @@ -270,7 +270,7 @@ POST /_matrix/federation/v1/rtc/livekit/get_token HTTP/1.1 The receiving server verifies that the requesting server is joined to the room identified by `room_id`. If either the receiving server or the requesting server are not joined, the request MUST be rejected with -HTTP 401 / `M_UNAUTHORIZED`. +HTTP 403 / `M_FORBIDDEN`. Otherwise, if `url` matches one of the receiving server's SFUs, it obtains a token from that SFU. If successful, an HTTP `200 OK` response is returned with `Content-Type: application/json`. The response body From 7be5ed2c417abf1bc4286543f053c03f639ac12d Mon Sep 17 00:00:00 2001 From: Johannes Marbach Date: Mon, 24 Aug 2026 11:22:38 +0200 Subject: [PATCH 052/122] Add unstable prefixes for new endpoints --- proposals/4195-matrixrtc-livekit.md | 8 +++++++- 1 file changed, 7 insertions(+), 1 deletion(-) diff --git a/proposals/4195-matrixrtc-livekit.md b/proposals/4195-matrixrtc-livekit.md index 71983465f43..68044b8a75f 100644 --- a/proposals/4195-matrixrtc-livekit.md +++ b/proposals/4195-matrixrtc-livekit.md @@ -645,7 +645,13 @@ leakage about users, rooms, or federation trust relationships. Assuming that this is accepted at the same time as [MSC4143](https://github.com/matrix-org/matrix-spec-proposals/pull/4143) no unstable prefix is -required as these fields will only be accessed via some other unstable prefix. +required for the `livekit` type indentifier as it will only be accessed via some other unstable prefix. + +Apart from this, the endpoints introduced should be referred to as follows: + +- `/_matrix/client/v1/rtc/livekit/get_token` -> `/_matrix/client/unstable/io.element.msc4195/rtc/livekit/get_token` +- `/_matrix/federation/v1/rtc/livekit/get_token` -> `/_matrix/federation/unstable/io.element.msc4195/rtc/livekit/get_token` +- `/_matrix/client/v1/rtc/livekit/delegate_delayed_leave` -> `/_matrix/client/unstable/io.element.msc4195/rtc/livekit/delegate_delayed_leave` ## Dependencies From 2d5aac19ae5b710e598c493ba265cd6700f2c9c6 Mon Sep 17 00:00:00 2001 From: Johannes Marbach Date: Mon, 24 Aug 2026 11:42:46 +0200 Subject: [PATCH 053/122] Clarify what happens when url doesn't match an SFU --- proposals/4195-matrixrtc-livekit.md | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/proposals/4195-matrixrtc-livekit.md b/proposals/4195-matrixrtc-livekit.md index 68044b8a75f..adaecca62e2 100644 --- a/proposals/4195-matrixrtc-livekit.md +++ b/proposals/4195-matrixrtc-livekit.md @@ -230,6 +230,9 @@ Upon receiving the request, the server verifies that the requesting user is join identified by `room_id`. If the user is not joined, the request MUST be rejected with HTTP 403 / `M_FORBIDDEN`. +If `server_name` is the server's own name and `url` does not match one of the server's own SFUs, +the request is rejected with HTTP 400 / `M_INVALID_PARAM`. + If `server_name` is the server's own name and `url` matches one of the server's own SFUs, it obtains a token from that SFU. If successful, the server returns an HTTP `200 OK` response with `Content-Type: application/json`. The response body contains: @@ -272,6 +275,9 @@ The receiving server verifies that the requesting server is joined to the room i If either the receiving server or the requesting server are not joined, the request MUST be rejected with HTTP 403 / `M_FORBIDDEN`. +If `url` does not match one of the receiving server's own SFUs, the request is rejected with +HTTP 400 / `M_INVALID_PARAM`. + Otherwise, if `url` matches one of the receiving server's SFUs, it obtains a token from that SFU. If successful, an HTTP `200 OK` response is returned with `Content-Type: application/json`. The response body contains: From 8c760dc463f3724570ef8becab55f892f0f9cc58 Mon Sep 17 00:00:00 2001 From: Johannes Marbach Date: Mon, 24 Aug 2026 14:22:53 +0200 Subject: [PATCH 054/122] Remove url from response --- proposals/4195-matrixrtc-livekit.md | 8 ++------ 1 file changed, 2 insertions(+), 6 deletions(-) diff --git a/proposals/4195-matrixrtc-livekit.md b/proposals/4195-matrixrtc-livekit.md index adaecca62e2..e3deb16715c 100644 --- a/proposals/4195-matrixrtc-livekit.md +++ b/proposals/4195-matrixrtc-livekit.md @@ -237,14 +237,12 @@ If `server_name` is the server's own name and `url` matches one of the server's token from that SFU. If successful, the server returns an HTTP `200 OK` response with `Content-Type: application/json`. The response body contains: * `jwt` — `string`: the JWT token to use for authentication with the SFU. -* `url` — `string`: the URL of the LiveKit SFU to use for the given slot. ```http HTTP/1.1 200 OK { - "jwt": "thejwt", - "url": "wss://matrix-rtc.example.com/livekit/sfu" + "jwt": "thejwt" } ``` @@ -283,14 +281,12 @@ an HTTP `200 OK` response is returned with `Content-Type: application/json`. The contains: * `jwt` — `string`: the JWT token to use for authentication with the SFU. -* `url` — `string`: the URL of the LiveKit SFU to use for the given slot. ```http HTTP/1.1 200 OK { - "jwt": "thejwt", - "url": "wss://matrix-rtc.example.com/livekit/sfu" + "jwt": "thejwt" } ``` From 170a9c9a4917a5b90c173f12cb7a07d0d49b82ec Mon Sep 17 00:00:00 2001 From: Johannes Marbach Date: Thu, 27 Aug 2026 13:04:48 +0200 Subject: [PATCH 055/122] Improve introduction --- proposals/4195-matrixrtc-livekit.md | 68 +++++++++++++++-------------- 1 file changed, 36 insertions(+), 32 deletions(-) diff --git a/proposals/4195-matrixrtc-livekit.md b/proposals/4195-matrixrtc-livekit.md index e3deb16715c..d582c907ef5 100644 --- a/proposals/4195-matrixrtc-livekit.md +++ b/proposals/4195-matrixrtc-livekit.md @@ -1,42 +1,46 @@ -# MSC4195: MatrixRTC Transport Using LiveKit Backend +# MSC4195: LiveKit transport for MatrixRTC -This MSC defines a LiveKit-based transport for MatrixRTC, allowing clients to publish and subscribe -to real-time media via LiveKit SFUs while maintaining Matrix-native session and membership -semantics. +[MSC4143] introduces MatrixRTC as an extensible framework for real-time communication in Matrix. +MatrixRTC uses so called transports to transfer the actual RTC data between RTC members. This +proposal introduces a transport based on the [LiveKit] Selective Forwarding Unit (SFU). The SFU +intelligently relays RTC data between members without them having to connect to each other directly. -This proposal defines a new [MSC4143](https://github.com/matrix-org/matrix-spec-proposals/pull/4143) -compliant MatrixRTC Transport using [LiveKit](https://github.com/livekit/livekit) Selective -Forwarding Units (SFUs). +The LiveKit SFU is integrated into Matrix in a multi-SFU configuration. In this setup, a homeserver +may operate one or more SFUs. RTC members always publish their RTC data to a local SFU and announce +their SFU choice via their `m.rtc.member` event. Other members then subscribe to the RTC data on the +publishing member's SFU – which might be different from the SFU they're publishing on themselves. +The homeserver provides mechanisms for discovering local SFUs and for acquiring access tokens for +both local and remote SFUs. This approach removes the need for an SFU election process and allows +servers to guard access to their SFUs. -In real-time communication environments, managing media streams among multiple participants can be -complex. This transport proposal uses a **Multi-SFU approach** where each participant publishes -their media directly to a LiveKit SFU, while others subscribe to streams they need. This removes the -need for an SFU election and preserves clear ownership of media. - -Example for two participants from different homeservers A and B +The example below illustrates how two members from different homeservers A and B publish and +subscribe to each other's RTC streams. ``` - +------------------+ - | Participant A | - | (Matrix Client) | - +------------------+ - | ^ - | | - | publishes | subscribes - v | - +-------+ +-------+ - | SFU A | | SFU B | - +-------+ +-------+ - | ^ - | | - | subscribes | publishes - v | - +------------------+ - | Participant B | - | (Matrix Client) | - +------------------+ + ┌─────────────────────────────────┐ + ┌───────┤ Client A │◀──────┐ + │ └──┬──────────────────────────────┘ │ + │ │ │ + publish │ │ discover SFU │ subscribe + │ │ get SFU authorisation │ + │ │ │ + ▼ ▼ │ + ┌───────┐ ┌──────────┐ federation ┌──────────┐ ┌───┴───┐ + │ SFU A │ │ Server A │◀─────────────▶│ Server B │ │ SFU B │ + └───┬───┘ └──────────┘ └──────────┘ └───────┘ + │ ▲ ▲ + │ │ │ +subscribe │ discover SFU │ │ publish + │ get SFU authorisation │ │ + │ │ │ + │ ┌──────────────────────────────┴──┐ │ + └──────▶│ Client B ├───────┘ + └─────────────────────────────────┘ ``` +[MSC4143]: https://github.com/matrix-org/matrix-spec-proposals/pull/4143 +[LiveKit]: https://github.com/livekit/livekit + ## Proposal This MSC defines the **LiveKit RTC Transport**, which can appear as one of the **RTC Transports** From cabb4b5ef526895dffa74fc41b0f87858d41c3f9 Mon Sep 17 00:00:00 2001 From: Johannes Marbach Date: Thu, 27 Aug 2026 13:06:57 +0200 Subject: [PATCH 056/122] Fix typo --- proposals/4195-matrixrtc-livekit.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/proposals/4195-matrixrtc-livekit.md b/proposals/4195-matrixrtc-livekit.md index d582c907ef5..23e4e988ffa 100644 --- a/proposals/4195-matrixrtc-livekit.md +++ b/proposals/4195-matrixrtc-livekit.md @@ -651,7 +651,7 @@ leakage about users, rooms, or federation trust relationships. Assuming that this is accepted at the same time as [MSC4143](https://github.com/matrix-org/matrix-spec-proposals/pull/4143) no unstable prefix is -required for the `livekit` type indentifier as it will only be accessed via some other unstable prefix. +required for the `livekit` type identifier as it will only be accessed via some other unstable prefix. Apart from this, the endpoints introduced should be referred to as follows: From d5cbf6bc7458a241a6e38606077926af343cff23 Mon Sep 17 00:00:00 2001 From: Johannes Marbach Date: Thu, 27 Aug 2026 13:29:04 +0200 Subject: [PATCH 057/122] Drag up and clean-up transport discovery and announcement --- proposals/4195-matrixrtc-livekit.md | 112 +++++++++++++--------------- 1 file changed, 52 insertions(+), 60 deletions(-) diff --git a/proposals/4195-matrixrtc-livekit.md b/proposals/4195-matrixrtc-livekit.md index 23e4e988ffa..2143dd349f9 100644 --- a/proposals/4195-matrixrtc-livekit.md +++ b/proposals/4195-matrixrtc-livekit.md @@ -43,8 +43,58 @@ subscribe │ discover SFU │ │ publish ## Proposal -This MSC defines the **LiveKit RTC Transport**, which can appear as one of the **RTC Transports** -offered by a homeserver and being used as transport by clients, as per [MSC4519](https://github.com/matrix-org/matrix-spec-proposals/pull/4519). +A new transport type `m.livekit` is introduced. Homeservers that support this transport announce it +to clients by including a dedicated object in the response of the`/_matrix/client/v1/rtc/transports` +endpoint from [MSC4519]. The object has the following schema: + +- `type` (required, string): The transport's type identifier. MUST be `m.livekit`. +- `url` (required, string): The SFU's WebSocket URL. This allows differentiating SFUs when the + server operates more than one SFU. + +Below is an example of a response from `/_matrix/client/v1/rtc/transports`: + +```json5 +{ + "transports": [{ + "type": "m.livekit", + "url": "ws://livekit.example.com" + }] +} +``` + +Once a client decides to publish media under a discovered transport, it includes the same object +in the `transports` array of its respective `m.rtc.member` event. This gives other clients in the +same RTC slot, the information required to subscribe to the published media. + +Below is an example of an appropriate membership event: + +```json5 +{ + "type": "m.rtc.member", + "content": { + "slot_id": "...", + "member": { + "id": "{member_id}", + "membership": "join" + }, + "application": { + ... + }, + "transports": { + "published": [{ + "type": "m.livekit", + "url": "ws://livekit.example.com" + }], + "can_subscribe": [ "m.livekit" ] + }, + "sticky_key": "{member_id}" + }, + ... +} +``` + +[MSC4519]: https://github.com/matrix-org/matrix-spec-proposals/pull/4519 + ### Canonical JSON Serialization @@ -87,64 +137,6 @@ to limit the number of active LiveKit SFU connections. The `livekit_alias` is shared with clients as part of their JWT token issued by the server. -### Transport type: `livekit` - -This section defines the JSON format for the LiveKit SFU Transport, covering both homeserver-side -advertisement and client-side consumption. - -#### Transport Advertisement (homeserver) - -The mechanism for advertising available RTC transports by homeservers is already defined in -[MSC4143](https://github.com/matrix-org/matrix-spec-proposals/pull/4143). - -The homeserver announces available LiveKit Transport as a JSON object with the following fields: - -* `type` — required `string`: this MUST be `livekit` -* `url` - required `string`: WebSocket URL of the LiveKit SFU. This enables running more than one SFU - per homeserver. - -An example for `GET /_matrix/client/v1/rtc/transports` - -```json5 -{ - "rtc_transports": [ - { - "type": "livekit", - "url": "ws://livekit.example.com - } - ] -} -``` - -#### Transport Usage (client) - -The mechanism for discovering available RTC transports by clients is already defined in -[MSC4143](https://github.com/matrix-org/matrix-spec-proposals/pull/4143). - -Clients declare the RTC Transport(s) they use to publish RTC data in their `m.rtc.member` state -event by adding a JSON object to the `rtc_transports` array. - -Other clients in the same MatrixRTC slot discover and subscribe to each other’s media by inspecting -`m.rtc.member` events. Clients use this information to connect to the appropriate SFU and subscribe -to the published media. - -Field Descriptions: - -* `type` — required `string`: this MUST be `"livekit"` -* `url` - required `string`: WebSocket URL of the LiveKit SFU. - -```json5 -{ - // rest of the m.rtc.member event - "rtc_transports": [ - { - "type": "livekit", - "url": "ws://livekit.example.com - } - ] -} -``` - ### Additions to the Client-Server and Server-Server API #### Acquiring a token for the SFU From 4cbea034c98df45a7e3a5aa7ff0ca8da5e114014 Mon Sep 17 00:00:00 2001 From: Johannes Marbach Date: Fri, 28 Aug 2026 13:29:11 +0200 Subject: [PATCH 058/122] Clean-up LiveKit primitive mapping and /get_token endpoint descriptions --- proposals/4195-matrixrtc-livekit.md | 238 +++++++++++++--------------- 1 file changed, 107 insertions(+), 131 deletions(-) diff --git a/proposals/4195-matrixrtc-livekit.md b/proposals/4195-matrixrtc-livekit.md index 2143dd349f9..9d18071fb6e 100644 --- a/proposals/4195-matrixrtc-livekit.md +++ b/proposals/4195-matrixrtc-livekit.md @@ -43,6 +43,8 @@ subscribe │ discover SFU │ │ publish ## Proposal +### Discovering and announcing transports + A new transport type `m.livekit` is introduced. Homeservers that support this transport announce it to clients by including a dedicated object in the response of the`/_matrix/client/v1/rtc/transports` endpoint from [MSC4519]. The object has the following schema: @@ -72,9 +74,9 @@ Below is an example of an appropriate membership event: { "type": "m.rtc.member", "content": { - "slot_id": "...", + "slot_id": "the_id", "member": { - "id": "{member_id}", + "id": "xyzABCDEF10123", "membership": "join" }, "application": { @@ -87,7 +89,7 @@ Below is an example of an appropriate membership event: }], "can_subscribe": [ "m.livekit" ] }, - "sticky_key": "{member_id}" + "sticky_key": "xyzABCDEF10123" }, ... } @@ -95,117 +97,113 @@ Below is an example of an appropriate membership event: [MSC4519]: https://github.com/matrix-org/matrix-spec-proposals/pull/4519 +### Mapping MatrixRTC members to LiveKit -### Canonical JSON Serialization +LiveKit encapsulates RTC sessions in so called [LiveKit rooms]. Within a LiveKit room, +[LiveKit participants] can publish and subscribe to RTC data streams through a WebSocket +connection that is guarded with an access token. A LiveKit room is identified by a unique +room "name" string while a LiveKit participant is identified by a unique "identity" string. +These LiveKit primitives need to be mapped to the `m.rtc.member` events for MatrixRTC members +from [MSC4143]. -This proposal uses JSON arrays and Canonical JSON encoding to ensure stable hashing inputs. -All uses of `JSON.serialize(...)` in the following text MUST use the Canonical JSON encoding as -defined by the [Matrix specification](https://spec.matrix.org/v1.18/appendices/#canonical-json). +#### LiveKit room names -Additionally, implementations MUST ensure that: +LiveKit room names are derived by homeservers and shared with clients as part of the +LiveKit access token issued by the homeserver (see [below]). To ensures a baseline of +pseudonymity and avoid exposing unnecessary metadata to the SFU, the derivation is +performed using the following steps: -* The array elements appear in the exact specified order. -* Each element is encoded as a JSON string. -* The resulting byte sequence used for hashing is the UTF-8 encoding of the canonical JSON output. +1. Construct a JSON array containing the `room_id` and `slot_id` of the `m.rtc.member` + event (in that precise order). +1. Perform a [Canonical JSON] serialization of the array. +1. Take the UTF-8 encoding of the canonicalization output and hash it with SHA-256. +1. Encode the result using [unpadded base64]. -For example: -```json5 -["@user:matrix.example.com","DEVICEID","abcd12345"] +``` +livekit_room_name = Base64( SHA256( Canonicalize( [ room_id, slot_id ] ) ) ) ``` -Any deviation (e.g. additional whitespace or different encoding) will result in a different hash -and is therefore non-compliant. +This procedure ensures that each MatrixRTC slot unambigously maps to one LiveKit room on +each involved SFU. As a result, the number of WebSocket connections required to participate +in an RTC session scales with the number of participating SFUs which should commonly mean +the number of participating homeservers. This is much more efficient for clients compared to +using separate LiveKit rooms per MatrixRTC member where the number of required WebSocket +connections would scale with the number of session members. -### LiveKit room alias +For improved metadata protection, servers MAY add a `salt` generated from a cryptographically +secure random number generator to the input JSON array when deriving LiveKit room names. -The name of a LiveKit room is referred to as the **LiveKit alias** (`livekit_alias`). The alias MUST -be globally unique and dependent on a given MatrixRTC slot in a Matrix room. A minimal -implementation that ensures a baseline of pseudonymity is given by the -[unpadded base64 encoding](https://spec.matrix.org/v1.17/appendices/#unpadded-base64) of the SHA-256 -hash of the JSON serialization of an array containing the Matrix `room_id` and the `slot_id`, i.e. -`base64(SHA256(JSON.serialize([room_id, slot_id])))`, where `JSON.serialize` is the canonical JSON -serialization defined [above](#canonical-json-serialization). +``` +livekit_room_name = Base64( SHA256( Canonicalize( [ room_id, slot_id, salt ] ) ) ) +``` -For improved metadata protection, the `livekit_alias` SHOULD be derived as -`base64(SHA256(JSON.serialize([room_id, slot_id, truly_random_bits])))`, where the `truly random bits` -are maintained by the server. +The value of `salt` MUST be persisted on the server and SHOULD be rotated once all +LiveKit participants have left the LiveKit room. This ensures that a different LiveKit +room is used for the next MatrixRTC session in the same slot and further reduces the +amount of metadata exposed to the SFU. -The resulting value is opaque to the MatrixRTC application. Within the LiveKit namespace, the -`livekit_alias` uniquely represents a MatrixRTC slot. Participants from the same Matrix deployment -(using the same SFU to publish their media) are considered to use the same `livekit_alias` in order -to limit the number of active LiveKit SFU connections. +### LiveKit participant identities -The `livekit_alias` is shared with clients as part of their JWT token issued by the server. +LiveKit participant identities are derived by both homeservers and clients. Homeservers require +the identity to generate LiveKit access tokens (see [below]). Clients use the identity to map +MXIDs to LiveKit participants, for instance, to display a user name and avatar on a video stream. +To avoid exposing unnecessary metadata to the SFU, the derivation process uses the following steps: -### Additions to the Client-Server and Server-Server API +1. Construct a JSON array containing the `sender` and `member.id` properties from the `m.rtc.member` + event (in that precise order). +1. Perform a [Canonical JSON] serialization of the array. +1. Take the UTF-8 encoding of the canonicalization output and hash it with SHA-256. +1. Encode the result using [unpadded base64]. -#### Acquiring a token for the SFU +``` +livekit_participant_identity = Base64( SHA256( Canonicalize( [ sender, member.id ] ) ) ) +``` -LiveKit SFUs require a JWT `access_token` to be provided when -[connecting to the WebSocket](https://docs.livekit.io/reference/internals/client-protocol/#WebSocket-Parameters). -This section standardises the method by which a MatrixRTC application obtains the LiveKit JWT -token. A high level overview is depicted in the following diagram +Note that `sender` is included here because according to [MSC4143], member IDs are unique per +member and session for a single user only. Due to these uniqueness properties, additional salting +is not required here. -```mermaid -sequenceDiagram - autonumber +[LiveKit rooms]: https://docs.livekit.io/intro/basics/rooms-participants-tracks/rooms/ +[LiveKit participants]: https://docs.livekit.io/intro/basics/rooms-participants-tracks/participants/ +[below]: #acquiring-livekit-access-tokens +[Canonical JSON]: (https://spec.matrix.org/v1.18/appendices/#canonical-json) +[unpadded base64]: https://spec.matrix.org/v1.17/appendices/#unpadded-base64 - participant U as 🧑 Alice +### Acquiring LiveKit access tokens - box floralwhite alice.com - participant H as 🏢 Homeserver - participant L as 📡 LiveKit SFU - end +As mentioned above, [WebSocket] connections to LiveKit rooms are needed for publishing and subscribing +to RTC streams. The LiveKit SFU requires an access token in the form of a JWT for these connections. +In order to enable additional access control checks, responsibility for issuing these tokens is +assigned to home servers. - box floralwhite bob.com - participant H2 as 🏢 Homeserver - end +Servers can [generate] the tokens by using one of the LiveKit SDKs and inputing a set of parameters +including the LiveKit room name and the LiveKit participant identifier. The procedure also requires +secrets agreed upon between the homeserver and the respective SFU. This means homeservers can +only generate tokens for their own SFUs. To allow clients to request tokens for both local and +remote SFUs, a new pair of authenticated Client-Server and Server-Server endpoints is introduced: +- `POST /_matrix/client/v1/rtc/livekit/get_token` +- `POST /_matrix/federation/v1/rtc/livekit/get_token` - participant O as 👨‍🦰 Bob +To request a token, a client `POST`s to `/get_token` including in the body a JSON object with the +following schema: - U->>H: /get_token - activate H - H->>H: Verify user's
room membership - H->>L: Request token - activate L - L-->>H: Return token & URL - deactivate L - H-->>U: Return token & URL - deactivate H +- `server_name` (string): The [server name](https://spec.matrix.org/v1.19/appendices/#server-name) + for which a token is requested. Defaults to the server's own server name if omitted. +- `url` (required, string): The WebSocket URL of the LiveKit SFU for which a token is requested. +- `room_id` (required, string): The room ID where the associated `m.rtc.member` event (see below) was sent. +- `slot_id` (required, string): The contents of the `slot_id` property of the associated `m.rtc.member` event. +- `member_id` (required, string): The `member.id` property of the associated `m.rtc.member` event. - U->>L: Publish media stream +When requesting a token for publishing, the associated `m.rtc.member` event is the member's own event. +The client uses its own server name for `server_name` and the WebSocket URL discovered from +`/_matrix/client/v1/rtc/transports` for `url`. - O->>H2: /get_token - activate H2 - H2->>H2: Verify user's
room membership - H2->>H: /get_token - activate H - H->>H: Verify servers's
room membership - H->>L: Request token - activate L - L-->>H: Return token & URL - deactivate L - H-->>H2: Return token & URL - deactivate H - H2-->>O: Return token & URL - deactivate H2 - - O->>L: Subscribe to media streams -``` +If, on the other hand, the token is requested for subscribing, the associated `m.rtc.member` event is +another member's event. In this case, the client derives the value for `server_name` from the `sender` +of that event and takes `url` from the respective `transports` array element in the event. -The JWT token is obtained by making an authenticated `POST` request to a new Client-Server endpoint -`/_matrix/client/v1/rtc/livekit/get_token`. - -The `Content-Type` of the request is `application/json` and the JSON body contains the following -fields: - - * `server_name` — `string`: the [server name](https://spec.matrix.org/v1.19/appendices/#server-name) - of the `m.rtc.member` event's `sender`. Defaults to the server's own server name if omitted. - * `url` - required `string`: WebSocket URL of the LiveKit SFU. - * `room_id` — required `string`: the Matrix room ID where the `m.rtc.member` event is present. - * `slot_id` — required `string`: the slot ID from the `m.rtc.member` event. - * `member` — required `object`: the contents of the `member` field from the `m.rtc.member` event. +Below is an example of a token request for the `m.rtc.membership` example given further up. ```http POST /_matrix/client/v1/rtc/livekit/get_token HTTP/1.1 @@ -215,10 +213,7 @@ POST /_matrix/client/v1/rtc/livekit/get_token HTTP/1.1 "url": "ws://livekit.example.com, "room_id": "!tDLCaLXijNtYcJZEey:example.com", "slot_id": "the_id", - "member": { - "id": "xyzABCDEF10123", - "claimed_device_id": "DEVICEID" - } + "member_id": "xyzABCDEF10123" } ``` @@ -229,10 +224,9 @@ identified by `room_id`. If the user is not joined, the request MUST be rejected If `server_name` is the server's own name and `url` does not match one of the server's own SFUs, the request is rejected with HTTP 400 / `M_INVALID_PARAM`. -If `server_name` is the server's own name and `url` matches one of the server's own SFUs, it obtains a -token from that SFU. If successful, the server returns an HTTP `200 OK` response with `Content-Type: application/json`. The response body contains: - -* `jwt` — `string`: the JWT token to use for authentication with the SFU. +If `server_name` is the server's own name and `url` matches one of the server's own SFUs, the server +generates a token for the SFU and responds with HTTP 200 and a JSON object with a single required +property `jwt` holding the token. ```http HTTP/1.1 200 OK @@ -242,14 +236,9 @@ HTTP/1.1 200 OK } ``` -If `server_name` points to a remote server, the server triggers a `POST` request to a new authenticated -Server-Server endpoint `/_matrix/federation/v1/rtc/livekit/get_token`. The `Content-Type` of the -request is `application/json` and the JSON body contains the following fields: - - * `url` - required `string`: WebSocket URL of the LiveKit SFU. - * `room_id` — required `string`: the Matrix room ID where the `m.rtc.member` event is present. - * `slot_id` — required `string`: the slot ID from the `m.rtc.member` event. - * `member` — required `object`: the contents of the `member` field from the `m.rtc.member` event. +If, converserly, `server_name` points to a remote server, the server triggers a `POST` request to +the `/get_token` federation endpoint on that server. The body of the request contains the same JSON +object received in the client request but with `server_name` omitted. An example is given below: ```http POST /_matrix/federation/v1/rtc/livekit/get_token HTTP/1.1 @@ -258,25 +247,19 @@ POST /_matrix/federation/v1/rtc/livekit/get_token HTTP/1.1 "url": "ws://livekit.example.com, "room_id": "!tDLCaLXijNtYcJZEey:example.com", "slot_id": "the_id", - "member": { - "id": "xyzABCDEF10123", - "claimed_device_id": "DEVICEID" - } + "member_id": "xyzABCDEF10123" } ``` -The receiving server verifies that the requesting server is joined to the room identified by `room_id`. -If either the receiving server or the requesting server are not joined, the request MUST be rejected with -HTTP 403 / `M_FORBIDDEN`. +Upon receiving the request, the remote server verifies that the origin server is joined to the room +identified by `room_id`. If the origin server is not joined or the remote server doesn't know the room, +the request MUST be rejected with HTTP 403 / `M_FORBIDDEN`. If `url` does not match one of the receiving server's own SFUs, the request is rejected with HTTP 400 / `M_INVALID_PARAM`. -Otherwise, if `url` matches one of the receiving server's SFUs, it obtains a token from that SFU. If successful, -an HTTP `200 OK` response is returned with `Content-Type: application/json`. The response body -contains: - -* `jwt` — `string`: the JWT token to use for authentication with the SFU. +Otherwise, the remote server generates a token for its SFU and returns it in the same response format +used for the Client-Server endpoint. ```http HTTP/1.1 200 OK @@ -286,9 +269,12 @@ HTTP/1.1 200 OK } ``` -The requesting server then forwards the response to its client as above. +The origin server then forwards the token to its client as above. -#### Optional Delegated MatrixRTC Membership Lifecycle Tracking using Cancellable Delayed Events +[generate]: https://docs.livekit.io/frontends/build/authentication/custom/ + + +### Optional Delegated MatrixRTC Membership Lifecycle Tracking using Cancellable Delayed Events As described in [MSC4143](https://github.com/matrix-org/matrix-spec-proposals/pull/4143), clients SHOULD use cancellable delayed events to implement a "deadman switch" for precise MatrixRTC @@ -345,7 +331,7 @@ sequenceDiagram deactivate H ``` -##### Request +#### Request The delegation is carried out by making a `POST` request to a new authenticated endpoint `/_matrix/client/v1/rtc/livekit/delegate_delayed_leave`. @@ -376,7 +362,7 @@ When delegating delayed events, Clients SHOULD NOT use values smaller than 1 hou to avoid unnecessarily frequent restarts of the delayed event. Servers MAY reject requests when the delegated event has a timeout below 1 hour with `M_BAD_JSON`. -##### Successful response +#### Successful response The server MUST only maintain a single delegated event per `room_id`, `slot_id`, `member` and MXID. Requests to delegate a different `delay_id` MUST invalidate earlier @@ -408,16 +394,6 @@ from the SFU or by polling the connection status from the SFU), the server MUST delegated delayed leave event. This ensures that the MatrixRTC membership state remains accurate and consistent, even in the presence of network interruptions or client crashes. -### Pseudonymous LiveKit Participant Identity - -To protect user privacy, a pseudonymous LiveKit participant identity is used, so the Matrix user ID -is not exposed to the LiveKit SFU backend. - -This pseudonymous identity is equal to the unpadded base64 encoding of the SHA-256 hash of the JSON -serialization of an array containing the Matrix `user_id`, the `claimed_device_id`, and the -`member.id` field, i.e. `base64(SHA256(JSON.serialize([user_id, claimed_device_id, member.id])))`, -using canonical JSON serialization as defined [above](#canonical-json-serialization). - ### LiveKit JWT Permission Grants As well as being a valid [LiveKit JWT](https://docs.livekit.io/home/get-started/authentication/) the From b61f655620fc796513525c86619f8679da046eb3 Mon Sep 17 00:00:00 2001 From: Johannes Marbach Date: Fri, 28 Aug 2026 14:47:43 +0200 Subject: [PATCH 059/122] Clean up delegation section --- proposals/4195-matrixrtc-livekit.md | 159 +++++++++++++++------------- 1 file changed, 83 insertions(+), 76 deletions(-) diff --git a/proposals/4195-matrixrtc-livekit.md b/proposals/4195-matrixrtc-livekit.md index 9d18071fb6e..28fbbcabde2 100644 --- a/proposals/4195-matrixrtc-livekit.md +++ b/proposals/4195-matrixrtc-livekit.md @@ -206,7 +206,7 @@ of that event and takes `url` from the respective `transports` array element in Below is an example of a token request for the `m.rtc.membership` example given further up. ```http -POST /_matrix/client/v1/rtc/livekit/get_token HTTP/1.1 +POST /_matrix/client/v1/rtc/livekit/get_token { "server_name": "example.com", @@ -229,7 +229,7 @@ generates a token for the SFU and responds with HTTP 200 and a JSON object with property `jwt` holding the token. ```http -HTTP/1.1 200 OK +200 OK { "jwt": "thejwt" @@ -241,7 +241,7 @@ the `/get_token` federation endpoint on that server. The body of the request con object received in the client request but with `server_name` omitted. An example is given below: ```http -POST /_matrix/federation/v1/rtc/livekit/get_token HTTP/1.1 +POST /_matrix/federation/v1/rtc/livekit/get_token { "url": "ws://livekit.example.com, @@ -262,7 +262,7 @@ Otherwise, the remote server generates a token for its SFU and returns it in the used for the Client-Server endpoint. ```http -HTTP/1.1 200 OK +200 OK { "jwt": "thejwt" @@ -273,24 +273,26 @@ The origin server then forwards the token to its client as above. [generate]: https://docs.livekit.io/frontends/build/authentication/custom/ +### Optional delegated delayed leave events -### Optional Delegated MatrixRTC Membership Lifecycle Tracking using Cancellable Delayed Events +As described in [MSC4143], clients SHOULD use delayed events to implement a "deadman switch" +for precise MatrixRTC membership tracking. This involves scheduling a delayed leave event and +periodically restarting it. If the client unexpectedly loses connectivity, the server triggers +the sending of the leave event once the delay expires. However, relying on clients to restart +the delayed event can be error-prone in adverse network conditions, particularly due to TCP +connection instability. -As described in [MSC4143](https://github.com/matrix-org/matrix-spec-proposals/pull/4143), clients -SHOULD use cancellable delayed events to implement a "deadman switch" for precise MatrixRTC -membership tracking. This involves sending a disconnect event ahead of the connect event as a -delayed event with a reasonable timeout (e.g., 15--30 seconds), and periodically restart the -delayed event's timer. If the timer expires due to a missing restart, the disconnect event is -automatically emitted, marking the participant as disconnected and ensuring accurate session state -even in cases of sudden disconnection, crashes, or network failures. However, relying on clients to -restart the delayed event timer can be error-prone in adverse network conditions, particularly due to -TCP connection instability. +The LiveKit SFU, on the other hand, maintains authoritative knowledge of each member's real-time +connection state through its WebSocket connections. Additionally, the SFU is able to trigger +[webhooks] upon connection state changes. These features can be used to create a delegation +mechanism for delayed leave events on the homeserver. A client first schedules its delayed leave +event and then delegates management of the event to its homeserver. The homeserver keeps restarting +the event while the participant is connected to the SFU and triggers sending the event once it finds the +participant disconnected from the SFU. This mechanism allows for higher reliability and accuracy +when compared to client-maintained delayed leave events. -Since the LiveKit SFU, which is tied to the homeserver, already maintains authoritative knowledge of -each participant's connection state, the management of cancellable delayed events MAY be delegated -to the homeserver. This delegation allows the RTC transport layer to accurately manage and -maintain MatrixRTC membership lifecycles across transient disconnects, ensuring a consistent and -reliable view of session state. +The following sequence diagram illustrates the conceptual procedure which is described in more detail +below. ```mermaid sequenceDiagram @@ -303,96 +305,101 @@ sequenceDiagram participant L as 📡 LiveKit SFU end - U->>H: Send m.rtc.member event
to join session - activate H - H-->>U: ​ - deactivate H - U->>H: Schedule delayed m.rtc.member event
to leave session activate H - H-->>U: ​ + H-->>U: ​Confirm scheduling deactivate H - Note over U,L: Obtain SFU token and URL as shown in the chart above - - U->>L: Publish media stream - U->>H: /delegate_delayed_leave activate H H-->>U: Confirm delegation - H->>H: Reschedule delayed
leave event - H->>H: Reschedule delayed
leave event - H->>H: Reschedule delayed
leave event + + H->>L: Wait for connection + L-->>H: Participant connected + + H->>H: Restart delayed
leave event + H->>H: Restart delayed
leave event + + H->>L: Sanity check connection state + L-->>H: Participant still connected + + H->>H: Restart delayed
leave event U->>U: Loses connectivity - L->>H: Trigger disconnect webhook + L->>H: Participant disconnected webhook H->>H: Trigger sending
leave event deactivate H ``` -#### Request +Clients delegate delayed leave events to their homeserver by `POST`ing to a new authenticated endpoint +`/_matrix/client/v1/rtc/livekit/delegate_delayed_leave`. The body of the request contains a JSON +object with the following schema: -The delegation is carried out by making a `POST` request to a new authenticated endpoint -`/_matrix/client/v1/rtc/livekit/delegate_delayed_leave`. +- `room_id` (required, `string`): The room ID in which the delayed `m.rtc.member` event was scheduled. +- `slot_id` (required, `string`): The contents of the `slot_id` property of the `m.rtc.member` event. +- `member_id` (required, `string`): The `member.id` property of the `m.rtc.member` event. +- `delay_id` (required, `string`): The delayed event ID obtained when scheduling the `m.rtc.member` event. -The `Content-Type` of the request is `application/json` and the JSON body contains the following -fields: - - * `room_id` — required `string`: the Matrix room ID where the `m.rtc.member` event is present. - * `slot_id` — required `string`: the slot ID from the `m.rtc.member` event. - * `member` — required `object`: the contents of the `member` field from the `m.rtc.member` event. - * `delay_id` — required `string`: the delayed event id of the MatrixRTC member leave event. +Below is an example of a request: ```http -POST /_matrix/client/v1/rtc/livekit/delegate_delayed_leave HTTP/1.1 +POST /_matrix/client/v1/rtc/livekit/delegate_delayed_leave { "room_id": "!tDLCaLXijNtYcJZEey:example.com", "slot_id": "the_id", - "member": { - "id": "xyzABCDEF10123", - "claimed_device_id": "DEVICEID" - }, + "member_id": "id", "delay_id": "1234567890" } ``` -When delegating delayed events, Clients SHOULD NOT use values smaller than 1 hour for the `delay_timeout` -to avoid unnecessarily frequent restarts of the delayed event. Servers MAY reject requests when the delegated -event has a timeout below 1 hour with `M_BAD_JSON`. - -#### Successful response - -The server MUST only maintain a single delegated event per `room_id`, `slot_id`, -`member` and MXID. Requests to delegate a different `delay_id` MUST invalidate earlier -delegations for the same parameters. +When scheduling delayed events that are meant to be delegated, clients SHOULD use a `delay_timeout` of +at least 1 hour. This avoids unnecessarily frequent restarts of the delayed event. Servers MAY reject +delegation requests with HTTP 400 / `M_INVALID_PARAM` when the delegated event has a lower timeout. -If the delegation request is successful, an HTTP `200 OK` response is returned with -`Content-Type: application/json`. The response body contains an empty JSON object -for future extension. +Otherwise, if the request parameters are valid, the server responds with HTTP 200 and an empty JSON +object to confirm the delegation. ```http -HTTP/1.1 200 OK +200 OK {} ``` -Once the homeserver observes the client's SFU connection (either by receiving a -[webhook](https://docs.livekit.io/intro/basics/rooms-participants-tracks/webhooks-events/) -from the SFU or by polling the connection status from the SFU), identified by -the LiveKit room `livekit_alias` and the LiveKit identity as specified in the next section -(`base64(SHA256(JSON.serialize([user_id, claimed_device_id, member.id])))`), it SHOULD issue -a restart of the delayed event. - -It then starts a timer corresponding to the delayed event's `delay_timeout`. The timer is periodically -restarted while the client remains connected with sufficient headroom (e.g., 80% of `delay_timeout`) to -ensure the restart occurs well before `delay_timeout` expires. If the homeserver detects that the client -has disconnected before the timer is restarted (either by receiving a -[webhook](https://docs.livekit.io/intro/basics/rooms-participants-tracks/webhooks-events/) -from the SFU or by polling the connection status from the SFU), the server MUST trigger sending of the -delegated delayed leave event. This ensures that the MatrixRTC membership state remains accurate and -consistent, even in the presence of network interruptions or client crashes. +The server then derives the LiveKit room alias and LiveKit participant identity from the `room_id`, +`slot_id` and `member_id` parameters as well as the request's authorization as described above. The +server then waits for the participant to connect to the SFU. How long the server waits before giving +up is left as an implementation detail. If it waits longer than the delegated event's `delay_timeout`, +it MUST restart the event periodically and with sufficient headroom to the expiration time. + +Once the server observes the LiveKit particpant's connection on the SFU, it MUST begin (or continue) +restarting the delayed event periodically – again, with sufficient headroom. The server then continues +to monitor the participants connection state. Once the server detects that the participant has +disconnected, it MUST trigger the sending of the delegated leave event. + +For maximum reliability, it is RECOMMENDED to use a combination of polling and listening to SFU [webhooks] +to monitor for SFU (dis)connections. + +The server MUST only maintain a single delegated event per `room_id`, `slot_id`, `member` and MXID. +Requests to delegate a different `delay_id` MUST invalidate earlier delegations for the same parameters. + +[webhooks]: https://docs.livekit.io/intro/basics/rooms-participants-tracks/webhooks-events/ + + + + + + + + + + + + + + + ### LiveKit JWT Permission Grants From 334d65adce191778c9e13addb1378df09950fc5e Mon Sep 17 00:00:00 2001 From: Johannes Marbach Date: Fri, 28 Aug 2026 15:32:18 +0200 Subject: [PATCH 060/122] Clean up token properties section --- proposals/4195-matrixrtc-livekit.md | 104 ++++++++++------------------ 1 file changed, 37 insertions(+), 67 deletions(-) diff --git a/proposals/4195-matrixrtc-livekit.md b/proposals/4195-matrixrtc-livekit.md index 28fbbcabde2..cc6b25b1f94 100644 --- a/proposals/4195-matrixrtc-livekit.md +++ b/proposals/4195-matrixrtc-livekit.md @@ -271,6 +271,43 @@ used for the Client-Server endpoint. The origin server then forwards the token to its client as above. +#### Access token properties + +Different properties and grants can be applied when generating tokens using LiveKit's SDKs. Servers need +to ensure these are set appropriately so that clients can connect correctly and securely. In particular, +servers MUST apply the following settings: + +- `sub`: The LiveKit participant identity, derived as described above. +- `video.room`: The LiveKit room name, derived as described above. +- `video.roomCreate`: Always `true`. This allows clients to create the LiveKit room if it doesn't yet + exist on the SFU. +- `video.roomJoin`: Always `true`. This enables clients to join the LiveKit room if it exists. +- `video.canPublish`: `true` if the token was requested by a local user. `false` otherwise. This enforces + the multi-SFU configuration and ensures clients can only publish RTC data on a local SFU. +- `video.canSubscribe`: Always `true`. This lets clients subscribe to RTC data on both local and + remote SFUs. +- `video.canUpdateOwnMetadata`: Always `true`. This lets clients update their own metadata. The latter is + a single string that can store any data. + +Below is an example of a LiveKit JWT for a local user: + +```json5 +{ + "exp": 1726764439, + "iss": "API2bYPYMoVqjcE", + "nbf": 1726760839, + "sub": "{livekit_participant_identity}", + "video": { + "room": "{livekit_room_name}", + "roomCreate": true, + "roomJoin": true, + "canPublish": true, + "canSubscribe": true, + "canUpdateOwnMetadata": true + } +} +``` + [generate]: https://docs.livekit.io/frontends/build/authentication/custom/ ### Optional delegated delayed leave events @@ -387,73 +424,6 @@ Requests to delegate a different `delay_id` MUST invalidate earlier delegations [webhooks]: https://docs.livekit.io/intro/basics/rooms-participants-tracks/webhooks-events/ - - - - - - - - - - - - - - -### LiveKit JWT Permission Grants - -As well as being a valid [LiveKit JWT](https://docs.livekit.io/home/get-started/authentication/) the -following constraints are applied: - -- `sub`: This is the pseudonymous LiveKit participant identity as described above. -- `video`.`room`: `livekit_alias` as defined above - -In a Multi-SFU setup, where participants may publish to one SFU and consume from others, the JWT -SHOULD encode access permissions according to the user’s homeserver and their relationship to -the MatrixRTC backend. - -The permissions SHOULD be just sufficient for the MatrixRTC application to operate in a LiveKit -room. Permissions SHOULD be scoped according to the user’s role (publishing or subscribing) and -their relationship to the MatrixRTC backend. All users MUST be able to join the LiveKit room for -which they are authorised. The `roomCreate` permission SHOULD only be granted to users who are -related to the MatrixRTC backend and are allowed to publish media. - -Example for publishing RTC data using a full-access grant: -```json5 -{ - "exp": 1726764439, - "iss": "API2bYPYMoVqjcE", - "nbf": 1726760839, - "sub": "xyzABCDEF0123", // member.id - "video": { - "canPublish": true, - "canSubscribe": true, - "room": "base64(SHA256(JSON.serialize([\"!gIpOlaUSrXBmgtveWK:call.ems.host\", \"m.call#ROOM\"])))", - "roomCreate": true, - "roomJoin": true - } -} -``` - -Example for subscribing RTC data with restricted-access grant - -```json5 -{ - "exp": 1726764439, - "iss": "API2bYPYMoVqjcE", - "nbf": 1726760839, - "sub": "xyzABCDEF0123", // member.id - "video": { - "canPublish": false, - "canSubscribe": true, - "room": "base64(SHA256(JSON.serialize([\"!gIpOlaUSrXBmgtveWK:call.ems.host\", \"m.call#ROOM\"])))", - "roomCreate": false, - "roomJoin": true - } -} -``` - ### End-to-end encryption End-to-end encryption is mapped into the LiveKit frame level encryption mechanism described From 99a70de9ff6a2afb4563c754466f0cb154ce0fe8 Mon Sep 17 00:00:00 2001 From: Johannes Marbach Date: Fri, 28 Aug 2026 16:02:26 +0200 Subject: [PATCH 061/122] Fix typos --- proposals/4195-matrixrtc-livekit.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/proposals/4195-matrixrtc-livekit.md b/proposals/4195-matrixrtc-livekit.md index cc6b25b1f94..f8c1bcacb0f 100644 --- a/proposals/4195-matrixrtc-livekit.md +++ b/proposals/4195-matrixrtc-livekit.md @@ -123,7 +123,7 @@ performed using the following steps: livekit_room_name = Base64( SHA256( Canonicalize( [ room_id, slot_id ] ) ) ) ``` -This procedure ensures that each MatrixRTC slot unambigously maps to one LiveKit room on +This procedure ensures that each MatrixRTC slot unambiguously maps to one LiveKit room on each involved SFU. As a result, the number of WebSocket connections required to participate in an RTC session scales with the number of participating SFUs which should commonly mean the number of participating homeservers. This is much more efficient for clients compared to @@ -176,7 +176,7 @@ to RTC streams. The LiveKit SFU requires an access token in the form of a JWT fo In order to enable additional access control checks, responsibility for issuing these tokens is assigned to home servers. -Servers can [generate] the tokens by using one of the LiveKit SDKs and inputing a set of parameters +Servers can [generate] the tokens by using one of the LiveKit SDKs and inputting a set of parameters including the LiveKit room name and the LiveKit participant identifier. The procedure also requires secrets agreed upon between the homeserver and the respective SFU. This means homeservers can only generate tokens for their own SFUs. To allow clients to request tokens for both local and @@ -410,7 +410,7 @@ server then waits for the participant to connect to the SFU. How long the server up is left as an implementation detail. If it waits longer than the delegated event's `delay_timeout`, it MUST restart the event periodically and with sufficient headroom to the expiration time. -Once the server observes the LiveKit particpant's connection on the SFU, it MUST begin (or continue) +Once the server observes the LiveKit participant's connection on the SFU, it MUST begin (or continue) restarting the delayed event periodically – again, with sufficient headroom. The server then continues to monitor the participants connection state. Once the server detects that the participant has disconnected, it MUST trigger the sending of the delegated leave event. From 8173d48689df41c2e7b5ea8ec3f0c74173db4c38 Mon Sep 17 00:00:00 2001 From: Johannes Marbach Date: Mon, 31 Aug 2026 08:26:51 +0200 Subject: [PATCH 062/122] Clean-up encryption section --- proposals/4195-matrixrtc-livekit.md | 35 ++++++++++++++++++++--------- 1 file changed, 24 insertions(+), 11 deletions(-) diff --git a/proposals/4195-matrixrtc-livekit.md b/proposals/4195-matrixrtc-livekit.md index f8c1bcacb0f..e4ce6dcd86c 100644 --- a/proposals/4195-matrixrtc-livekit.md +++ b/proposals/4195-matrixrtc-livekit.md @@ -426,20 +426,33 @@ Requests to delegate a different `delay_id` MUST invalidate earlier delegations ### End-to-end encryption -End-to-end encryption is mapped into the LiveKit frame level encryption mechanism described -[here](https://github.com/livekit/livekit/issues/1035). +[MSC4143] introduced the `m.per_member` mechanism for letting clients generate a generic per-member secret +that is distributed to other clients via `m.rtc.encryption_key` to-deivce messages. -Where a shared password is used by the application it is used as the `string` input to the LiveKit -key derivation function (which uses PBKDF2) and all participants use the same derived key for -encryption and decryption. +```json5 +{ + "room_id": "{room_id}", + "member_id": "{member_id}", + "media_key": { + "index": , + "key": "{encoded_key}", + "format": "m.base64" + } +} +``` + +To map this secret into LiveKit's frame-level [encryption] mechanism, clients use LiveKit's SDKs to implement +a [custom key provider]. The secret in `media_key.key` is then used as the raw byte input to LiveKit's HKDF-based +key derivation function, keyed by `media_key.index` and associated with the respective LiveKit participant identity +derived as described [above]. + +Clients MUST use a keyring size of 256 when initialising the custom key provider to align with the [0, 255] range +of `media_key.index` as per [MSC4143]. -Where a per-participant key is used it is imported as the byte array input to the LiveKit key -derivation function (which uses HKDF). The `index` field of the `m.rtc.encryption_keys` event is -used as the key index for the key provider. +[encryption]: https://docs.livekit.io/transport/encryption/ +[custom key provider]: https://docs.livekit.io/transport/encryption/start/#custom-key-provider +[above]: #liveKit-participant-identities -On receipt of the `m.rtc.encryption_keys` event the application can associate the received key with -the LiveKit participant identity by calculating the pseudonymous LiveKit participant identity as -described above. ## Potential issues From 0380a015329417f9548292bcaf195ffe31b2b72b Mon Sep 17 00:00:00 2001 From: Johannes Marbach Date: Mon, 31 Aug 2026 09:22:57 +0200 Subject: [PATCH 063/122] Add token dance chart and apply minor fixes --- proposals/4195-matrixrtc-livekit.md | 66 ++++++++++++++++++++++++++--- 1 file changed, 61 insertions(+), 5 deletions(-) diff --git a/proposals/4195-matrixrtc-livekit.md b/proposals/4195-matrixrtc-livekit.md index e4ce6dcd86c..85faa880f21 100644 --- a/proposals/4195-matrixrtc-livekit.md +++ b/proposals/4195-matrixrtc-livekit.md @@ -50,8 +50,8 @@ to clients by including a dedicated object in the response of the`/_matrix/clien endpoint from [MSC4519]. The object has the following schema: - `type` (required, string): The transport's type identifier. MUST be `m.livekit`. -- `url` (required, string): The SFU's WebSocket URL. This allows differentiating SFUs when the - server operates more than one SFU. +- `url` (required, string): The SFU's WebSocket URL. Clients use this URL to connect to the SFU + via one of LiveKit's [client SDKs]. Below is an example of a response from `/_matrix/client/v1/rtc/transports`: @@ -96,6 +96,7 @@ Below is an example of an appropriate membership event: ``` [MSC4519]: https://github.com/matrix-org/matrix-spec-proposals/pull/4519 +[client SDKs]: https://docs.livekit.io/transport/sdk-platforms/ ### Mapping MatrixRTC members to LiveKit @@ -146,8 +147,8 @@ amount of metadata exposed to the SFU. LiveKit participant identities are derived by both homeservers and clients. Homeservers require the identity to generate LiveKit access tokens (see [below]). Clients use the identity to map -MXIDs to LiveKit participants, for instance, to display a user name and avatar on a video stream. -To avoid exposing unnecessary metadata to the SFU, the derivation process uses the following steps: +MXIDs to LiveKit participants. To avoid exposing unnecessary metadata to the SFU, the derivation +process uses the following steps: 1. Construct a JSON array containing the `sender` and `member.id` properties from the `m.rtc.member` event (in that precise order). @@ -271,6 +272,61 @@ used for the Client-Server endpoint. The origin server then forwards the token to its client as above. +The sequence chart below illustrates how two users from different homeservers discover SFUs and obtain +tokens for both publishing and subscribing to RTC streams. + +```mermaid +sequenceDiagram + autonumber + + participant U as 🧑 Alice + + box floralwhite alice.com + participant H as 🏢 Homeserver + participant L as 📡 LiveKit SFU + end + + box floralwhite bob.com + participant L1 as 📡 LiveKit SFU + participant H1 as 🏢 Homeserver + end + + participant U1 as 👨 Bob + + U->>H: Discover LiveKit SFU + activate H + H-->>U: Return SFU WebSocket URL + deactivate H + + U->>H: Request SFU access token + activate H + H-->>U: If authorised, return access token + deactivate H + + U->>L: Connect to SFU and start publishing + activate L + + U->>H: Publish SFU URL in m.rtc.member event + + Note over U1,L1: Publishing analogous to Alice (steps 1-6) + + U1->>H1: Discover Alice's SFU from
her m.rtc.member event + + U1->>H1: Request access token for Alice's SFU + activate H1 + H1->>H: Request SFU access token + activate H + H-->>H1: If authorised, return access token + deactivate H + H1-->>U1: Return access token + deactivate H1 + + U1->>L: Connect to Alice's SFU and start subscribing + deactivate L + + Note over U,L: Subscribing analogous to Bob (steps 7-13) +``` + #### Access token properties Different properties and grants can be applied when generating tokens using LiveKit's SDKs. Servers need @@ -427,7 +483,7 @@ Requests to delegate a different `delay_id` MUST invalidate earlier delegations ### End-to-end encryption [MSC4143] introduced the `m.per_member` mechanism for letting clients generate a generic per-member secret -that is distributed to other clients via `m.rtc.encryption_key` to-deivce messages. +that is distributed to other clients via `m.rtc.encryption_key` to-device messages. ```json5 { From 51d0977932837f5351b0196365617633765a6a67 Mon Sep 17 00:00:00 2001 From: Johannes Marbach Date: Mon, 31 Aug 2026 10:32:53 +0200 Subject: [PATCH 064/122] Clean up potential issues --- proposals/4195-matrixrtc-livekit.md | 110 ++++++++-------------------- 1 file changed, 32 insertions(+), 78 deletions(-) diff --git a/proposals/4195-matrixrtc-livekit.md b/proposals/4195-matrixrtc-livekit.md index 85faa880f21..e67e0973b0d 100644 --- a/proposals/4195-matrixrtc-livekit.md +++ b/proposals/4195-matrixrtc-livekit.md @@ -512,79 +512,15 @@ of `media_key.index` as per [MSC4143]. ## Potential issues -### Source of `truly_random_bits` for Pseudonymous `livekit_alias` Derivation - -Clients that publish their media through the same SFU and use the same `slot_id` within a given -Matrix room are considered to share the same LiveKit room (`livekit_alias`), which minimizes the -number of active LiveKit SFU connections. - -The derivation of the LiveKit room alias is defined as: -`livekit_alias = base64(SHA256(JSON.serialize([room_id, slot_id, truly_random_bits])))`. - -This construction is part of the proposal and ensures that aliases remain pseudonymous while still -being deterministically derived for a given Matrix room and MatrixRTC slot. The open consideration -is the source of the `truly_random_bits` used in the derivation. - -Two approaches are possible: - -1. **Client-provided `truly_random_bits`** - * Requires coordination between clients sharing the same `slot_id` within a Matrix room to ensure - they use identical random bits; otherwise, different `livekit_alias` values maybe derived and - fragment the session. - * As described in the MatrixRTC slots section of - [MSC4143](https://github.com/matrix-org/matrix-spec-proposals/pull/4143), slots are the intended - mechanism for sharing state between clients. However, slots are **unencrypted** and subject to - state resolution. Therefore, they are not suitable for holding truly random bits`. - * While tie-breaking `truly random bits` derived from `m.rtc.member` events (e.g., within the - `rtc_transports` field) ensures that the data is encrypted shared state, it is subject to - client-side consensus and may flip over time. Overall, it does **not** improve the reliability - of propagating and converging those random bits. - * Requires the removal of the `room_id` field from the access request, which prevents additional - access checks, such as verifying that the user is actually part of the claimed Matrix room. -2. **Server-provided random bits** - * The server generates and persists the `truly_random_bits` for each `(room_id, - slot_id)` tuple - * Guarantees consistent alias derivation across clients without requiring client-side - coordination. - * The benefit of improved pseudonymity only applies if the server is - operated separately from the actual LiveKit SFU. - * Preserves the `room_id` in the access request, allowing additional access checks, such as - verifying that the user is actually part of the claimed Matrix room. - -Given that pseudonymous LiveKit participant IDs already exist, the design prioritizes **reliability -over additional pseudonymity** by using server-provided random bits, ensuring -consistent `livekit_alias` across clients while enabling additional access checks. - -### Reliance on the LiveKit Protocol and Implementation - -A concern has been raised regarding the reliance of this MSC on the LiveKit protocol, which is -developed and maintained by a commercial entity rather than a formal standards body. This creates a -theoretical risk that future development or licensing changes by LiveKit, Inc. could diverge from -Matrix’s goals or limit interoperability. - -This consideration was already discussed during the design of the MatrixRTC backend, and several -factors help to mitigate the concern: -* **Protocol openness**: The LiveKit protocol and reference implementation are released under the - [Apache 2.0 License](https://github.com/livekit/livekit/blob/master/LICENSE), which allows for - forking and independent evolution. If LiveKit’s direction or license were to change, Matrix could - adopt the current protocol version and evolve it independently under an open governance model. -* **No lock-in at the Matrix level**: MatrixRTC defines a generic transport abstraction (see - [MSC4143](https://github.com/matrix-org/matrix-spec-proposals/pull/4143)), allowing for the - definition of additional or alternative transport types in the future without breaking - compatibility. -* **Extensibility**: Because the LiveKit protocol is open source, nothing prevents the Matrix - community from implementing additional functionality — such as Cascading SFUs or other - federation-oriented features — on top of the existing protocol if required. While this has been - discussed with the LiveKit team and they did not object in principle, such extensions are not - expected to depend on their involvement. -* **Implementation pragmatism**: The choice of LiveKit was primarily pragmatic—to accelerate - development and deployment of a functioning multi-SFU solution—rather than to establish a - permanent dependency. The current multi-SFU model also reduces the importance of features such as - Cascading SFUs that might otherwise require protocol changes. - -In summary, this MSC’s reliance on LiveKit represents a practical implementation path rather than a -long-term commitment to a specific third-party protocol. The current design remains open to future -evolution toward a Matrix-native or jointly standardized MatrixRTC transport. +### Client-provided salts for LiveKit room names + +The method for mapping MatrixRTC sessions to [LiveKit room names] includes an optional server-side +salt. Instead of doing this on the server, clients could generate this salt to reduce metadata +shared with the server. This is complicated, however, because it would require clients to coordinate +in order to agree on the same salt. A natural place to maintain the salt with little to no client +coordination is the `m.rtc.slot` state event. While state events are not encryptable, this still +shares the salt with the homeserver, however. Maintaining the salt on the homeserver is a compromise +that leaks some metadata to the homeserver but still hides it from the SFU. ### Lack of HKDF support in some LiveKit client SDKs @@ -593,12 +529,30 @@ the Flutter SDK (see [livekit/client-sdk-flutter#974](https://github.com/livekit Upstream implementation efforts such as [livekit/rust-sdks#796](https://github.com/livekit/rust-sdks/issues/796) will be required to close these gaps. -### Missing .well-known documents +### Reliance on the LiveKit protocol implementations -As per the current spec, publishing the location of the client-server API in a .well-known document is -not mandatory. Consequently, resolving the URL using .well-known discovery can fail. This should usually -only occur in corporate setups and private federations though. Implementations MAY allow hardcoding the -mapping from server name to client-server API URL to address these cases. +While being open source, LiveKit is developed and maintained by a commercial entity and is not an +open standard. As a result, future development or licensing changes by LiveKit, Inc could diverge +from Matrix’s goals or limit interoperability. This is mitigated by the following factors: + +- Protocol openness: The LiveKit protocol and reference implementation are released under the + [Apache 2.0 License] which allows for forking and independent evolution. If LiveKit’s direction + or license were to change, Matrix could adopt the current protocol version and evolve it + independently under an open governance model. +* No lock-in at the Matrix level: As per [MSC4143], transports in MatrixRTC are a generic abstraction + that allows definiting additional or alternative transport types in the future without breaking + compatibility. +* Extensibility: Because the LiveKit protocol is open source, nothing prevents the Matrix community + from implementing additional functionality (such as cascading SFUs or other federation-oriented + features) on top of the existing protocol if required. While this has been discussed with the + LiveKit team and they did not object in principle, such extensions are not expected to depend on + their involvement. +* Implementation pragmatism: The choice of LiveKit is pragmatic and helps accelerate development + and deployment of a functioning multi-SFU solution without necessarily establishing a permanent + dependency. The current multi-SFU model also reduces the importance of features such as cascading + SFUs that might otherwise require protocol changes. + +[Apache 2.0 License]: https://github.com/livekit/livekit/blob/master/LICENSE ## Alternatives From 269eb45d8d5cca1c4b1537905fe657155dd77d5f Mon Sep 17 00:00:00 2001 From: Johannes Marbach Date: Mon, 31 Aug 2026 10:37:27 +0200 Subject: [PATCH 065/122] Clean up alternatives section --- proposals/4195-matrixrtc-livekit.md | 41 +++++++++-------------------- 1 file changed, 13 insertions(+), 28 deletions(-) diff --git a/proposals/4195-matrixrtc-livekit.md b/proposals/4195-matrixrtc-livekit.md index e67e0973b0d..bc7949f0771 100644 --- a/proposals/4195-matrixrtc-livekit.md +++ b/proposals/4195-matrixrtc-livekit.md @@ -556,34 +556,19 @@ from Matrix’s goals or limit interoperability. This is mitigated by the follow ## Alternatives -### String concatenation of hashing inputs - -Instead of using canonical JSON, the hashing inputs could be concatenated with a suitable delimiter -such as `|`. This is prone to delimiter injection, however. As an example, the inputs `("a|b", "c")` -and `("a", "b|c")` both produce the concatenation `"a|b|c"` and, hence, the same hash. Using JSON -arrays and Canonical JSON serialisation avoids this problem. Since the Canonical JSON serialisation -of string arrays is trivial, this doesn't meaningfully increase implementation complexity. - -### JSON objects as hashing inputs - -Instead of JSON arrays, JSON objects could be used for the hashing inputs. This would reduce the -chances of accidentally using the wrong order of array elements. On the downside, however, the -Canonical JSON serialisation of objects is significantly more complex than for arrays. Overall, -this would likely result in a higher chance of implementation errors. - -### Combination of token request and delegation - -Instead of using separate endpoints, the token request and the delegation of the delayed disconnect -event could be combined in a single endpoint. This creates a race condition, however. As per -[MSC4143](https://github.com/matrix-org/matrix-spec-proposals/pull/4143), the disconnect event -carries a relation to the associated join event. This means a client would have to send its join -event before requesting an SFU token. The associated Livekit room will only be created when the -token is requested though. As a result, a client on another homeserver could attempt to connect to -the SFU in the meantime. Since the Livekit room doesn't yet exist, this would result in an error. -Separating the endpoints avoids this issue. - -Additionally, a joint endpoint introduces the problem of having to handle the case where one of -the two operations succeeds but the other fails. +### Canonical JSON variations + +The procedures for deriving LiveKit room names and LiveKit participant identifiers involve [Canonical JSON]. +As an alternative, the hashing inputs could be concatenated with a suitable delimiter such as `|`. This +is prone to delimiter injection, however. As an example, the inputs `("a|b", "c")` and `("a", "b|c")` +both produce the concatenation `"a|b|c"` and, hence, the same hash. Using JSON arrays and Canonical JSON +avoids this problem. Since the Canonical JSON serialisation of string arrays is trivial, this also doesn't +meaningfully increase implementation complexity. + +Furthermore, instead of JSON arrays, JSON objects could be used for the hashing inputs. This would reduce +the chances of accidentally using the wrong order of array elements. On the downside, however, the +Canonical JSON serialisation for objects is significantly more complex than for arrays. Overall, this +would likely result in a higher chance of implementation errors. ## Security considerations From b8331e9bc1c4cc54d9f93a646ce2d1b797c26ee0 Mon Sep 17 00:00:00 2001 From: Johannes Marbach Date: Mon, 31 Aug 2026 10:50:04 +0200 Subject: [PATCH 066/122] Remaining clean-up --- proposals/4195-matrixrtc-livekit.md | 58 ++++++++++++----------------- 1 file changed, 23 insertions(+), 35 deletions(-) diff --git a/proposals/4195-matrixrtc-livekit.md b/proposals/4195-matrixrtc-livekit.md index bc7949f0771..784491397cd 100644 --- a/proposals/4195-matrixrtc-livekit.md +++ b/proposals/4195-matrixrtc-livekit.md @@ -584,17 +584,17 @@ generated session membership IDs with sufficient entropy. Implementations of the `/get_token` endpoint SHOULD take care not to disclose sensitive internal details through error messages. -Error responses should use generic `"errcode"` values and short, human-readable `"error"` +Error responses should use generic `errcode` values and short, human-readable `error` descriptions that are suitable for client display or logging. Specifically: -* Validation or authorisation failures MUST NOT reveal information about whether a particular Matrix + +- Validation or authorisation failures MUST NOT reveal information about whether a particular Matrix user, device, or room exists. -* Server-side or federation validation errors (for example, OpenID token verification failures) - SHOULD be reported as `M_UNAUTHORIZED` or `M_FORBIDDEN` without including internal validation - results or upstream responses. -* Detailed diagnostic information (e.g., reasons for policy rejection, internal stack traces, or +- Server-side or federation validation errors SHOULD be reported as `M_UNAUTHORIZED` or `M_FORBIDDEN` + without including internal validation results or upstream responses. +- Detailed diagnostic information (e.g., reasons for policy rejection, internal stack traces, or upstream HTTP responses) MUST NOT be exposed to clients, but MAY be logged on the server side for audit and debugging purposes. -* If rate limiting is applied, the inclusion of a numeric `retry_after_ms` value is acceptable, but +- If rate limiting is applied, the inclusion of a numeric `retry_after_ms` value is acceptable, but other details of rate limiting policy SHOULD NOT be exposed. This ensures that error responses remain useful for clients while preventing potential metadata @@ -602,11 +602,11 @@ leakage about users, rooms, or federation trust relationships. ## Unstable prefix -Assuming that this is accepted at the same time as -[MSC4143](https://github.com/matrix-org/matrix-spec-proposals/pull/4143) no unstable prefix is -required for the `livekit` type identifier as it will only be accessed via some other unstable prefix. +Assuming that this proposal is accepted at the same time as [MSC4143] no unstable prefix is +required for the `livekit` type identifier as it will only be accessed via some other unstable +prefix. -Apart from this, the endpoints introduced should be referred to as follows: +Apart from this, the endpoints introduced above should be referred to as follows: - `/_matrix/client/v1/rtc/livekit/get_token` -> `/_matrix/client/unstable/io.element.msc4195/rtc/livekit/get_token` - `/_matrix/federation/v1/rtc/livekit/get_token` -> `/_matrix/federation/unstable/io.element.msc4195/rtc/livekit/get_token` @@ -614,32 +614,20 @@ Apart from this, the endpoints introduced should be referred to as follows: ## Dependencies -This MSC builds on [MSC4143](https://github.com/matrix-org/matrix-spec-proposals/pull/4143) (which -at the time of writing has not yet been accepted into the spec). - -This MSC additionally requires [MSC4519](https://github.com/matrix-org/matrix-spec-proposals/pull/4519) -to be accepted. - -## Appendix: Hash Derivation Test Vectors - -This appendix provides **verified test vectors** for: +This proposal depends on [MSC4143] and [MSC4519]. -* `livekit_alias` -* pseudonymous LiveKit participant identity +## Appendix: hash derivation test vectors -All hashes are computed as: +Below are provided verified test vectors for the LiveKit room name and LiveKit participant identity, derived as +described above. Further test vectors can be obtained with the following shell commands. -`base64(SHA256(JSON.serialize([...]))` - -Where `JSON.serialize` uses **Matrix canonical JSON** as defined in: -https://spec.matrix.org/v1.18/appendices/#canonical-json - ---- - -### Test Vectors +```sh +printf '%s' "${CANONICAL_JSON}" | openssl dgst -sha256 # SHA-256 (hex) +printf '%s' "${CANONICAL_JSON}" | openssl dgst -sha256 -binary | openssl base64 -A | tr -d '=' # Base64 (unpadded) +``` | Case | Input (logical) | Canonical JSON | SHA-256 (hex) | Base64 (unpadded) | -|------|------------------|----------------|---------------|-------------------| -| LiveKit alias (no random bits) | `["!roomid:example.com", "slot1234"]` | `["!roomid:example.com","slot1234"]` | `3bce37ed6dfe8e6ccc563a083f7b4dc1b9be5f11d093688aa4e03b6aac37a927` | `O8437W3+jmzMVjoIP3tNwbm+XxHQk2iKpOA7aqw3qSc` | -| LiveKit alias (with random bits) | `["!roomid:example.com", "slot123", "random123"]` | `["!roomid:example.com","slot123","random123"]` | `20c78377e2b7308a894c8db4117048adea4a92184e46f7f7abc7f1deb96b8539` | `IMeDd+K3MIqJTI20EXBIrepKkhhORvf3q8fx3rlrhTk` | -| Participant identity | `["@alice:example.com", "DEVICE123", "memberABC"]` | `["@alice:example.com","DEVICE123","memberABC"]` | `27e4f8e6d1abbb173e1eb50ea89265c90495df79bbdbc0a67b8fafb7cfd25ab5` | `J+T45tGruxc+HrUOqJJlyQSV33m728Cme4+vt8/SWrU` | \ No newline at end of file +|------|-----------------|----------------|---------------|-------------------| +| LiveKit room alias (no random bits) | `["!roomid:example.com", "slot1234"]` | `["!roomid:example.com","slot1234"]` | `3bce37ed6dfe8e6ccc563a083f7b4dc1b9be5f11d093688aa4e03b6aac37a927` | `O8437W3+jmzMVjoIP3tNwbm+XxHQk2iKpOA7aqw3qSc` | +| LiveKit room alias (with random bits) | `["!roomid:example.com", "slot123", "random123"]` | `["!roomid:example.com","slot123","random123"]` | `20c78377e2b7308a894c8db4117048adea4a92184e46f7f7abc7f1deb96b8539` | `IMeDd+K3MIqJTI20EXBIrepKkhhORvf3q8fx3rlrhTk` | +| LiveKit participant identity | `["@alice:example.com", "DEVICE123", "memberABC"]` | `["@alice:example.com","DEVICE123","memberABC"]` | `337567b0b5eb91bc480c83573bae2ef0f6731720fd6581624142d1d9db21598b` | `M3VnsLXrkbxIDINXO64u8PZzFyD9ZYFiQULR2dshWYs` | From 48dc692b30a53154788f6ce75082d44d4faaf875 Mon Sep 17 00:00:00 2001 From: Johannes Marbach Date: Mon, 31 Aug 2026 10:50:54 +0200 Subject: [PATCH 067/122] Fix typo --- proposals/4195-matrixrtc-livekit.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/proposals/4195-matrixrtc-livekit.md b/proposals/4195-matrixrtc-livekit.md index 784491397cd..a9adb1407f8 100644 --- a/proposals/4195-matrixrtc-livekit.md +++ b/proposals/4195-matrixrtc-livekit.md @@ -540,7 +540,7 @@ from Matrix’s goals or limit interoperability. This is mitigated by the follow or license were to change, Matrix could adopt the current protocol version and evolve it independently under an open governance model. * No lock-in at the Matrix level: As per [MSC4143], transports in MatrixRTC are a generic abstraction - that allows definiting additional or alternative transport types in the future without breaking + that allows defining additional or alternative transport types in the future without breaking compatibility. * Extensibility: Because the LiveKit protocol is open source, nothing prevents the Matrix community from implementing additional functionality (such as cascading SFUs or other federation-oriented From 29643a5fc187a825fd5ba72af670c94dce72b318 Mon Sep 17 00:00:00 2001 From: Johannes Marbach Date: Mon, 31 Aug 2026 10:55:48 +0200 Subject: [PATCH 068/122] Add rate limiting --- proposals/4195-matrixrtc-livekit.md | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/proposals/4195-matrixrtc-livekit.md b/proposals/4195-matrixrtc-livekit.md index a9adb1407f8..3c97d112d99 100644 --- a/proposals/4195-matrixrtc-livekit.md +++ b/proposals/4195-matrixrtc-livekit.md @@ -186,6 +186,8 @@ remote SFUs, a new pair of authenticated Client-Server and Server-Server endpoin - `POST /_matrix/client/v1/rtc/livekit/get_token` - `POST /_matrix/federation/v1/rtc/livekit/get_token` +The server SHOULD apply rate limiting to both of these endpoints. + To request a token, a client `POST`s to `/get_token` including in the body a JSON object with the following schema: @@ -477,6 +479,8 @@ to monitor for SFU (dis)connections. The server MUST only maintain a single delegated event per `room_id`, `slot_id`, `member` and MXID. Requests to delegate a different `delay_id` MUST invalidate earlier delegations for the same parameters. +It is RECOMMENDED that servers apply rate limiting to the delegation endpoint. + [webhooks]: https://docs.livekit.io/intro/basics/rooms-participants-tracks/webhooks-events/ From b14f6ee8fcad82e82faaf1894e51617e29250f91 Mon Sep 17 00:00:00 2001 From: Johannes Marbach Date: Mon, 31 Aug 2026 12:49:01 +0200 Subject: [PATCH 069/122] Flesh out security considerations --- proposals/4195-matrixrtc-livekit.md | 48 ++++++++++++++++++++++++----- 1 file changed, 41 insertions(+), 7 deletions(-) diff --git a/proposals/4195-matrixrtc-livekit.md b/proposals/4195-matrixrtc-livekit.md index 3c97d112d99..159079acf3a 100644 --- a/proposals/4195-matrixrtc-livekit.md +++ b/proposals/4195-matrixrtc-livekit.md @@ -562,7 +562,7 @@ from Matrix’s goals or limit interoperability. This is mitigated by the follow ### Canonical JSON variations -The procedures for deriving LiveKit room names and LiveKit participant identifiers involve [Canonical JSON]. +The procedures for deriving LiveKit room names and LiveKit participant identities involve [Canonical JSON]. As an alternative, the hashing inputs could be concatenated with a suitable delimiter such as `|`. This is prone to delimiter injection, however. As an example, the inputs `("a|b", "c")` and `("a", "b|c")` both produce the concatenation `"a|b|c"` and, hence, the same hash. Using JSON arrays and Canonical JSON @@ -576,12 +576,46 @@ would likely result in a higher chance of implementation errors. ## Security considerations -### Pseudonymity - -The LiveKit participant identity is a function of one's Matrix user ID, device ID, and session -membership ID; if all of these values are known or otherwise predictable to the SFU then there is -effectively no guarantee of pseudonymity. Therefore clients must be careful to use randomly -generated session membership IDs with sufficient entropy. +### Kicking users from the SFU on room leave + +Since MatrixRTC sessions are tied to Matrix rooms, servers should to take care that client connections +to the SFU don't exceed past the point where a user leaves the associated room. This is important +because otherwise a malicious user being kicked from a room might continue to be connected to an +ongoing RTC session related to the room. To prevent this, servers SHOULD remove any associated LiveKit +participant identities from the related LiveKit rooms when a user leaves a Matrix room. + +It should be noted, that removing a participant from a LiveKit room also revokes their access token +in the cloud version of LiveKit. This is _not_ the case in the self-hosted version, however. Homeservers +that rely on a self-hosted LiveKit instance should issue access tokens with a sufficiently short TTL +to mitigate this. + +### Reducing metadata leakage to the SFU + +With SFUs always being tied to homeservers under this proposal, two principal deployment models exist +on the server side. On the one hand, the SFU can be self-hosted. This means the homeserver operator is +also the SFU operator and hiding metadata known to the homeserver from the SFU has limited value. On +the other hand, the LiveKit deployment can also be outsourced, for instance, by using [LiveKit Cloud]. +This introduces another entity with access to only the SFU into the threat model. In order to handle +the latter case, this proposal takes steps to hide metadata known to the homeserver from the SFU where +possible. + +For one thing, [LiveKit room names] are pseudonymised which prevents the SFU from learning about room +or slot IDs. If the same slot is used repeatedly for a meeting, the SFU could still apply heuristics to +establish a connection between RTC sessions and the room. The addition of the server-side salt described +above, eliminates this leak, too. + +For another, [LiveKit participant identities] are pseudonymised as well which prevents the SFU from +correllating SFU participants with Matrix users. The identity derivation process involves the value of +`member.id` which clients change every time they join a slot. As a result, the SFU is unable to track +Matrix users across different calls and no further salting is required. + +The LiveKit SFU and the homeserver necessarily form a high trust relationship. In order for the homeserver +to extend SFU access tokens, secrets need to be agreed upon between the homeserver and the SFU. This +is a one-time configuration step, however. No networking is required between the homeserver and the +SFU to generate access tokens. + +[LiveKit Cloud]: https://cloud.livekit.io +[LiveKit participant identities]: #liveKit-participant-identities ### Error handling and information disclosure From 35301ec00ca216a170df8153f2cac0d479303409 Mon Sep 17 00:00:00 2001 From: Johannes Marbach Date: Mon, 31 Aug 2026 12:49:38 +0200 Subject: [PATCH 070/122] Add link --- proposals/4195-matrixrtc-livekit.md | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/proposals/4195-matrixrtc-livekit.md b/proposals/4195-matrixrtc-livekit.md index 159079acf3a..14b99a115b5 100644 --- a/proposals/4195-matrixrtc-livekit.md +++ b/proposals/4195-matrixrtc-livekit.md @@ -584,11 +584,13 @@ because otherwise a malicious user being kicked from a room might continue to be ongoing RTC session related to the room. To prevent this, servers SHOULD remove any associated LiveKit participant identities from the related LiveKit rooms when a user leaves a Matrix room. -It should be noted, that removing a participant from a LiveKit room also revokes their access token +It should be noted, that removing a participant from a LiveKit room also [revokes] their access token in the cloud version of LiveKit. This is _not_ the case in the self-hosted version, however. Homeservers that rely on a self-hosted LiveKit instance should issue access tokens with a sufficiently short TTL to mitigate this. +[revokes]: https://docs.livekit.io/frontends/reference/tokens-grants/#token-revocation + ### Reducing metadata leakage to the SFU With SFUs always being tied to homeservers under this proposal, two principal deployment models exist From faba6510b60f18c67194ebe7211ac31082bf5e5e Mon Sep 17 00:00:00 2001 From: Johannes Marbach Date: Mon, 31 Aug 2026 12:54:54 +0200 Subject: [PATCH 071/122] Further security details --- proposals/4195-matrixrtc-livekit.md | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/proposals/4195-matrixrtc-livekit.md b/proposals/4195-matrixrtc-livekit.md index 14b99a115b5..512eeb880ca 100644 --- a/proposals/4195-matrixrtc-livekit.md +++ b/proposals/4195-matrixrtc-livekit.md @@ -616,6 +616,11 @@ to extend SFU access tokens, secrets need to be agreed upon between the homeserv is a one-time configuration step, however. No networking is required between the homeserver and the SFU to generate access tokens. +Apart from this the homeserver relies on the SFU to truthfully respond to connection status checks +and to carry out on-demand room removals. A malicious SFU operator could fake these. Due to the +pseudonymization described above, they would not be able to relate RTC data to Matrix rooms or users, +however. In unencrypted RTC sessions, the SFU operator of course has full access to the RTC streams. + [LiveKit Cloud]: https://cloud.livekit.io [LiveKit participant identities]: #liveKit-participant-identities From 5ffa2d30528a6a5a3954d0baacd66b96399cb45e Mon Sep 17 00:00:00 2001 From: Johannes Marbach Date: Mon, 31 Aug 2026 12:56:30 +0200 Subject: [PATCH 072/122] Fix typo --- proposals/4195-matrixrtc-livekit.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/proposals/4195-matrixrtc-livekit.md b/proposals/4195-matrixrtc-livekit.md index 512eeb880ca..f4393c2136f 100644 --- a/proposals/4195-matrixrtc-livekit.md +++ b/proposals/4195-matrixrtc-livekit.md @@ -607,7 +607,7 @@ establish a connection between RTC sessions and the room. The addition of the se above, eliminates this leak, too. For another, [LiveKit participant identities] are pseudonymised as well which prevents the SFU from -correllating SFU participants with Matrix users. The identity derivation process involves the value of +correlating SFU participants with Matrix users. The identity derivation process involves the value of `member.id` which clients change every time they join a slot. As a result, the SFU is unable to track Matrix users across different calls and no further salting is required. From 259a3d771c484721c76f905425ed689f2a64ebd7 Mon Sep 17 00:00:00 2001 From: Johannes Marbach Date: Mon, 31 Aug 2026 13:03:18 +0200 Subject: [PATCH 073/122] Add missing quotes --- proposals/4195-matrixrtc-livekit.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/proposals/4195-matrixrtc-livekit.md b/proposals/4195-matrixrtc-livekit.md index f4393c2136f..859c0cf8af5 100644 --- a/proposals/4195-matrixrtc-livekit.md +++ b/proposals/4195-matrixrtc-livekit.md @@ -213,7 +213,7 @@ POST /_matrix/client/v1/rtc/livekit/get_token { "server_name": "example.com", - "url": "ws://livekit.example.com, + "url": "ws://livekit.example.com", "room_id": "!tDLCaLXijNtYcJZEey:example.com", "slot_id": "the_id", "member_id": "xyzABCDEF10123" @@ -247,7 +247,7 @@ object received in the client request but with `server_name` omitted. An example POST /_matrix/federation/v1/rtc/livekit/get_token { - "url": "ws://livekit.example.com, + "url": "ws://livekit.example.com", "room_id": "!tDLCaLXijNtYcJZEey:example.com", "slot_id": "the_id", "member_id": "xyzABCDEF10123" From 246ef3be100ab0883dcd2e151dae88ef182448f1 Mon Sep 17 00:00:00 2001 From: Johannes Marbach Date: Mon, 31 Aug 2026 13:18:53 +0200 Subject: [PATCH 074/122] Further fixes from code review --- proposals/4195-matrixrtc-livekit.md | 35 +++++++++++++++-------------- 1 file changed, 18 insertions(+), 17 deletions(-) diff --git a/proposals/4195-matrixrtc-livekit.md b/proposals/4195-matrixrtc-livekit.md index 859c0cf8af5..7fd79ca13de 100644 --- a/proposals/4195-matrixrtc-livekit.md +++ b/proposals/4195-matrixrtc-livekit.md @@ -46,7 +46,7 @@ subscribe │ discover SFU │ │ publish ### Discovering and announcing transports A new transport type `m.livekit` is introduced. Homeservers that support this transport announce it -to clients by including a dedicated object in the response of the`/_matrix/client/v1/rtc/transports` +to clients by including a dedicated object in the response of the `/_matrix/client/v1/rtc/transports` endpoint from [MSC4519]. The object has the following schema: - `type` (required, string): The transport's type identifier. MUST be `m.livekit`. @@ -110,7 +110,7 @@ from [MSC4143]. #### LiveKit room names LiveKit room names are derived by homeservers and shared with clients as part of the -LiveKit access token issued by the homeserver (see [below]). To ensures a baseline of +LiveKit access token issued by the homeserver (see [below]). To ensure a baseline of pseudonymity and avoid exposing unnecessary metadata to the SFU, the derivation is performed using the following steps: @@ -143,7 +143,7 @@ LiveKit participants have left the LiveKit room. This ensures that a different L room is used for the next MatrixRTC session in the same slot and further reduces the amount of metadata exposed to the SFU. -### LiveKit participant identities +#### LiveKit participant identities LiveKit participant identities are derived by both homeservers and clients. Homeservers require the identity to generate LiveKit access tokens (see [below]). Clients use the identity to map @@ -167,8 +167,8 @@ is not required here. [LiveKit rooms]: https://docs.livekit.io/intro/basics/rooms-participants-tracks/rooms/ [LiveKit participants]: https://docs.livekit.io/intro/basics/rooms-participants-tracks/participants/ [below]: #acquiring-livekit-access-tokens -[Canonical JSON]: (https://spec.matrix.org/v1.18/appendices/#canonical-json) -[unpadded base64]: https://spec.matrix.org/v1.17/appendices/#unpadded-base64 +[Canonical JSON]: https://spec.matrix.org/v1.19/appendices/#canonical-json +[unpadded base64]: https://spec.matrix.org/v1.19/appendices/#unpadded-base64 ### Acquiring LiveKit access tokens @@ -239,7 +239,7 @@ property `jwt` holding the token. } ``` -If, converserly, `server_name` points to a remote server, the server triggers a `POST` request to +If, conversely, `server_name` points to a remote server, the server triggers a `POST` request to the `/get_token` federation endpoint on that server. The body of the request contains the same JSON object received in the client request but with `server_name` omitted. An example is given below: @@ -326,7 +326,7 @@ sequenceDiagram U1->>L: Connect to Alice's SFU and start subscribing deactivate L - Note over U,L: Subscribing analogous to Bob (steps 7-13) + Note over U,L: Subscribing analogous to Bob (steps 7-12) ``` #### Access token properties @@ -402,7 +402,7 @@ sequenceDiagram U->>H: Schedule delayed m.rtc.member event
to leave session activate H - H-->>U: ​Confirm scheduling + H-->>U: Confirm scheduling deactivate H U->>H: /delegate_delayed_leave @@ -449,9 +449,9 @@ POST /_matrix/client/v1/rtc/livekit/delegate_delayed_leave } ``` -When scheduling delayed events that are meant to be delegated, clients SHOULD use a `delay_timeout` of -at least 1 hour. This avoids unnecessarily frequent restarts of the delayed event. Servers MAY reject -delegation requests with HTTP 400 / `M_INVALID_PARAM` when the delegated event has a lower timeout. +When scheduling delayed events that are meant to be delegated, clients SHOULD use a delay of at least +1 hour. This avoids unnecessarily frequent restarts of the delayed event. Servers MAY reject delegation +requests with HTTP 400 / `M_INVALID_PARAM` when the delegated event has a lower timeout. Otherwise, if the request parameters are valid, the server responds with HTTP 200 and an empty JSON object to confirm the delegation. @@ -465,8 +465,8 @@ object to confirm the delegation. The server then derives the LiveKit room alias and LiveKit participant identity from the `room_id`, `slot_id` and `member_id` parameters as well as the request's authorization as described above. The server then waits for the participant to connect to the SFU. How long the server waits before giving -up is left as an implementation detail. If it waits longer than the delegated event's `delay_timeout`, -it MUST restart the event periodically and with sufficient headroom to the expiration time. +up is left as an implementation detail. If it waits longer than the delegated event's delay, it MUST +restart the event periodically and with sufficient headroom to the expiration time. Once the server observes the LiveKit participant's connection on the SFU, it MUST begin (or continue) restarting the delayed event periodically – again, with sufficient headroom. The server then continues @@ -511,8 +511,7 @@ of `media_key.index` as per [MSC4143]. [encryption]: https://docs.livekit.io/transport/encryption/ [custom key provider]: https://docs.livekit.io/transport/encryption/start/#custom-key-provider -[above]: #liveKit-participant-identities - +[above]: #livekit-participant-identities ## Potential issues @@ -526,6 +525,8 @@ coordination is the `m.rtc.slot` state event. While state events are not encrypt shares the salt with the homeserver, however. Maintaining the salt on the homeserver is a compromise that leaks some metadata to the homeserver but still hides it from the SFU. +[LiveKit room names]: #livekit-room-names + ### Lack of HKDF support in some LiveKit client SDKs Some LiveKit SDKs currently only support PBKDF2 but don't allow using HKDF. One example of this is @@ -648,7 +649,7 @@ leakage about users, rooms, or federation trust relationships. ## Unstable prefix Assuming that this proposal is accepted at the same time as [MSC4143] no unstable prefix is -required for the `livekit` type identifier as it will only be accessed via some other unstable +required for the `m.livekit` type identifier as it will only be accessed via some other unstable prefix. Apart from this, the endpoints introduced above should be referred to as follows: @@ -675,4 +676,4 @@ printf '%s' "${CANONICAL_JSON}" | openssl dgst -sha256 -binary | openssl base64 |------|-----------------|----------------|---------------|-------------------| | LiveKit room alias (no random bits) | `["!roomid:example.com", "slot1234"]` | `["!roomid:example.com","slot1234"]` | `3bce37ed6dfe8e6ccc563a083f7b4dc1b9be5f11d093688aa4e03b6aac37a927` | `O8437W3+jmzMVjoIP3tNwbm+XxHQk2iKpOA7aqw3qSc` | | LiveKit room alias (with random bits) | `["!roomid:example.com", "slot123", "random123"]` | `["!roomid:example.com","slot123","random123"]` | `20c78377e2b7308a894c8db4117048adea4a92184e46f7f7abc7f1deb96b8539` | `IMeDd+K3MIqJTI20EXBIrepKkhhORvf3q8fx3rlrhTk` | -| LiveKit participant identity | `["@alice:example.com", "DEVICE123", "memberABC"]` | `["@alice:example.com","DEVICE123","memberABC"]` | `337567b0b5eb91bc480c83573bae2ef0f6731720fd6581624142d1d9db21598b` | `M3VnsLXrkbxIDINXO64u8PZzFyD9ZYFiQULR2dshWYs` | +| LiveKit participant identity | `["@alice:example.com", "memberABC"]` | `["@alice:example.com","DEVICE123","memberABC"]` | `337567b0b5eb91bc480c83573bae2ef0f6731720fd6581624142d1d9db21598b` | `M3VnsLXrkbxIDINXO64u8PZzFyD9ZYFiQULR2dshWYs` | From e2955da598a49f6fffab9bdf6e81e6bea72ba9af Mon Sep 17 00:00:00 2001 From: Johannes Marbach Date: Mon, 31 Aug 2026 14:07:22 +0200 Subject: [PATCH 075/122] Highlight lack of authentication --- proposals/4195-matrixrtc-livekit.md | 14 ++++++++++++++ 1 file changed, 14 insertions(+) diff --git a/proposals/4195-matrixrtc-livekit.md b/proposals/4195-matrixrtc-livekit.md index 7fd79ca13de..ca3afe1d138 100644 --- a/proposals/4195-matrixrtc-livekit.md +++ b/proposals/4195-matrixrtc-livekit.md @@ -534,6 +534,20 @@ the Flutter SDK (see [livekit/client-sdk-flutter#974](https://github.com/livekit Upstream implementation efforts such as [livekit/rust-sdks#796](https://github.com/livekit/rust-sdks/issues/796) will be required to close these gaps. +### Lack of per-sender authentication + +LiveKit's frame-level encryption is based on [SFrame], which by design does not authenticate the +sender of individual frames. A participant who has obtained another member's current `media_key` — +including a colluding SFU operator — could therefore forge frames that appear to originate from that +member. This is an inherent limitation of SFrame rather than something introduced by this proposal. +Its practical impact is reduced by [MSC4143]'s requirement that clients rotate the key whenever the +set of members joined to a slot changes, which follows the SFrame specification's [own recommendation] +for achieving forward secrecy. + +[SFrame]: https://www.ietf.org/archive/id/draft-ietf-sframe-enc-04.html +[not authenticate]: https://www.ietf.org/archive/id/draft-ietf-sframe-enc-04.html#name-no-per-sender-authentication +[own recommendation]: https://www.ietf.org/archive/id/draft-ietf-sframe-enc-04.html#name-key-management-2 + ### Reliance on the LiveKit protocol implementations While being open source, LiveKit is developed and maintained by a commercial entity and is not an From 7d0913c2afaac39dd4a367479a49c2cc06017fe0 Mon Sep 17 00:00:00 2001 From: Johannes Marbach Date: Mon, 31 Aug 2026 15:26:55 +0200 Subject: [PATCH 076/122] ws -> wss --- proposals/4195-matrixrtc-livekit.md | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/proposals/4195-matrixrtc-livekit.md b/proposals/4195-matrixrtc-livekit.md index ca3afe1d138..954bca9eb7f 100644 --- a/proposals/4195-matrixrtc-livekit.md +++ b/proposals/4195-matrixrtc-livekit.md @@ -59,7 +59,7 @@ Below is an example of a response from `/_matrix/client/v1/rtc/transports`: { "transports": [{ "type": "m.livekit", - "url": "ws://livekit.example.com" + "url": "wss://livekit.example.com" }] } ``` @@ -85,7 +85,7 @@ Below is an example of an appropriate membership event: "transports": { "published": [{ "type": "m.livekit", - "url": "ws://livekit.example.com" + "url": "wss://livekit.example.com" }], "can_subscribe": [ "m.livekit" ] }, @@ -213,7 +213,7 @@ POST /_matrix/client/v1/rtc/livekit/get_token { "server_name": "example.com", - "url": "ws://livekit.example.com", + "url": "wss://livekit.example.com", "room_id": "!tDLCaLXijNtYcJZEey:example.com", "slot_id": "the_id", "member_id": "xyzABCDEF10123" @@ -247,7 +247,7 @@ object received in the client request but with `server_name` omitted. An example POST /_matrix/federation/v1/rtc/livekit/get_token { - "url": "ws://livekit.example.com", + "url": "wss://livekit.example.com", "room_id": "!tDLCaLXijNtYcJZEey:example.com", "slot_id": "the_id", "member_id": "xyzABCDEF10123" From e9d8fd50eb0a4c9dfc19027c6d50c6d257cf183e Mon Sep 17 00:00:00 2001 From: Johannes Marbach Date: Mon, 31 Aug 2026 15:34:32 +0200 Subject: [PATCH 077/122] transports -> transports.published --- proposals/4195-matrixrtc-livekit.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/proposals/4195-matrixrtc-livekit.md b/proposals/4195-matrixrtc-livekit.md index 954bca9eb7f..25dd288cd22 100644 --- a/proposals/4195-matrixrtc-livekit.md +++ b/proposals/4195-matrixrtc-livekit.md @@ -65,8 +65,8 @@ Below is an example of a response from `/_matrix/client/v1/rtc/transports`: ``` Once a client decides to publish media under a discovered transport, it includes the same object -in the `transports` array of its respective `m.rtc.member` event. This gives other clients in the -same RTC slot, the information required to subscribe to the published media. +in the `transports.published` array of its respective `m.rtc.member` event. This gives other clients +in the same RTC slot, the information required to subscribe to the published media. Below is an example of an appropriate membership event: From 2b2396a5e9d643ce709869fb4818a2554978dcbf Mon Sep 17 00:00:00 2001 From: Johannes Marbach Date: Mon, 31 Aug 2026 15:57:36 +0200 Subject: [PATCH 078/122] Mention WebRTC connection --- proposals/4195-matrixrtc-livekit.md | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/proposals/4195-matrixrtc-livekit.md b/proposals/4195-matrixrtc-livekit.md index 25dd288cd22..9e2afaeab01 100644 --- a/proposals/4195-matrixrtc-livekit.md +++ b/proposals/4195-matrixrtc-livekit.md @@ -100,12 +100,12 @@ Below is an example of an appropriate membership event: ### Mapping MatrixRTC members to LiveKit -LiveKit encapsulates RTC sessions in so called [LiveKit rooms]. Within a LiveKit room, -[LiveKit participants] can publish and subscribe to RTC data streams through a WebSocket -connection that is guarded with an access token. A LiveKit room is identified by a unique -room "name" string while a LiveKit participant is identified by a unique "identity" string. -These LiveKit primitives need to be mapped to the `m.rtc.member` events for MatrixRTC members -from [MSC4143]. +LiveKit encapsulates RTC sessions in so-called [LiveKit rooms]. Within a LiveKit room, +[LiveKit participants] use a WebSocket signaling connection that is guarded with an +access token. Publishing and subscribing to RTC streams then happens over WebRTC. +A LiveKit room is identified by a unique room "name" string while a LiveKit participant +is identified by a unique "identity" string. These LiveKit primitives need to be mapped +to the `m.rtc.member` events for MatrixRTC members from [MSC4143]. #### LiveKit room names From e0b97404e02734fa9408d239115bfbf48cb61a01 Mon Sep 17 00:00:00 2001 From: Johannes Marbach Date: Mon, 31 Aug 2026 15:59:45 +0200 Subject: [PATCH 079/122] Emphasize that it's two connections --- proposals/4195-matrixrtc-livekit.md | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/proposals/4195-matrixrtc-livekit.md b/proposals/4195-matrixrtc-livekit.md index 9e2afaeab01..d4d9c776c25 100644 --- a/proposals/4195-matrixrtc-livekit.md +++ b/proposals/4195-matrixrtc-livekit.md @@ -125,11 +125,11 @@ livekit_room_name = Base64( SHA256( Canonicalize( [ room_id, slot_id ] ) ) ) ``` This procedure ensures that each MatrixRTC slot unambiguously maps to one LiveKit room on -each involved SFU. As a result, the number of WebSocket connections required to participate -in an RTC session scales with the number of participating SFUs which should commonly mean -the number of participating homeservers. This is much more efficient for clients compared to -using separate LiveKit rooms per MatrixRTC member where the number of required WebSocket -connections would scale with the number of session members. +each involved SFU. As a result, the number of connections (WebSocket + WebRTC) required to +participate in an RTC session scales with the number of participating SFUs which should +commonly mean the number of participating homeservers. This is much more efficient for +clients compared to using separate LiveKit rooms per MatrixRTC member where the number +of required WebSocket connections would scale with the number of session members. For improved metadata protection, servers MAY add a `salt` generated from a cryptographically secure random number generator to the input JSON array when deriving LiveKit room names. From ee6b17120f0f6bfc7be2421acb96ccdec8465f10 Mon Sep 17 00:00:00 2001 From: Johannes Marbach Date: Mon, 31 Aug 2026 16:00:50 +0200 Subject: [PATCH 080/122] Remove WebSocket --- proposals/4195-matrixrtc-livekit.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/proposals/4195-matrixrtc-livekit.md b/proposals/4195-matrixrtc-livekit.md index d4d9c776c25..0de1cd8a8bb 100644 --- a/proposals/4195-matrixrtc-livekit.md +++ b/proposals/4195-matrixrtc-livekit.md @@ -129,7 +129,7 @@ each involved SFU. As a result, the number of connections (WebSocket + WebRTC) r participate in an RTC session scales with the number of participating SFUs which should commonly mean the number of participating homeservers. This is much more efficient for clients compared to using separate LiveKit rooms per MatrixRTC member where the number -of required WebSocket connections would scale with the number of session members. +of required connections would scale with the number of session members. For improved metadata protection, servers MAY add a `salt` generated from a cryptographically secure random number generator to the input JSON array when deriving LiveKit room names. From fac2da6cb125324a5af4d08bf916a552f227be6d Mon Sep 17 00:00:00 2001 From: Johannes Marbach Date: Mon, 31 Aug 2026 16:03:05 +0200 Subject: [PATCH 081/122] Link to MSC4140 --- proposals/4195-matrixrtc-livekit.md | 13 +++++++------ 1 file changed, 7 insertions(+), 6 deletions(-) diff --git a/proposals/4195-matrixrtc-livekit.md b/proposals/4195-matrixrtc-livekit.md index 0de1cd8a8bb..f3945815cf7 100644 --- a/proposals/4195-matrixrtc-livekit.md +++ b/proposals/4195-matrixrtc-livekit.md @@ -370,12 +370,12 @@ Below is an example of a LiveKit JWT for a local user: ### Optional delegated delayed leave events -As described in [MSC4143], clients SHOULD use delayed events to implement a "deadman switch" -for precise MatrixRTC membership tracking. This involves scheduling a delayed leave event and -periodically restarting it. If the client unexpectedly loses connectivity, the server triggers -the sending of the leave event once the delay expires. However, relying on clients to restart -the delayed event can be error-prone in adverse network conditions, particularly due to TCP -connection instability. +As described in [MSC4143], clients SHOULD use delayed events as per [MSC4140] to implement a +"deadman switch" for precise MatrixRTC membership tracking. This involves scheduling a delayed +leave event and periodically restarting it. If the client unexpectedly loses connectivity, the +server triggers the sending of the leave event once the delay expires. However, relying on clients +to restart the delayed event can be error-prone in adverse network conditions, particularly due +to TCP connection instability. The LiveKit SFU, on the other hand, maintains authoritative knowledge of each member's real-time connection state through its WebSocket connections. Additionally, the SFU is able to trigger @@ -481,6 +481,7 @@ Requests to delegate a different `delay_id` MUST invalidate earlier delegations It is RECOMMENDED that servers apply rate limiting to the delegation endpoint. +[MSC4140]: https://github.com/matrix-org/matrix-spec-proposals/pull/4140 [webhooks]: https://docs.livekit.io/intro/basics/rooms-participants-tracks/webhooks-events/ From 20c9d552018e06ba0bfd831e85bc4051b43a4b03 Mon Sep 17 00:00:00 2001 From: Johannes Marbach Date: Mon, 31 Aug 2026 16:05:32 +0200 Subject: [PATCH 082/122] New paragraph --- proposals/4195-matrixrtc-livekit.md | 12 +++++++----- 1 file changed, 7 insertions(+), 5 deletions(-) diff --git a/proposals/4195-matrixrtc-livekit.md b/proposals/4195-matrixrtc-livekit.md index f3945815cf7..1a798e28974 100644 --- a/proposals/4195-matrixrtc-livekit.md +++ b/proposals/4195-matrixrtc-livekit.md @@ -462,11 +462,13 @@ object to confirm the delegation. {} ``` -The server then derives the LiveKit room alias and LiveKit participant identity from the `room_id`, -`slot_id` and `member_id` parameters as well as the request's authorization as described above. The -server then waits for the participant to connect to the SFU. How long the server waits before giving -up is left as an implementation detail. If it waits longer than the delegated event's delay, it MUST -restart the event periodically and with sufficient headroom to the expiration time. +The server derives the LiveKit room alias and LiveKit participant identity from the `room_id`, +`slot_id` and `member_id` parameters as well as the request's authorization as described above. + +The server then waits for the participant to connect to the SFU. How long the server waits before +giving up is left as an implementation detail. If the waiting duration exceeds the delegated event's +delay, the server MUST restart the event periodically and with sufficient headroom to the expiration +time. Once the server observes the LiveKit participant's connection on the SFU, it MUST begin (or continue) restarting the delayed event periodically – again, with sufficient headroom. The server then continues From a8361843837c4d655048e0ddc7a5134ec95f691d Mon Sep 17 00:00:00 2001 From: Johannes Marbach Date: Mon, 31 Aug 2026 16:06:22 +0200 Subject: [PATCH 083/122] Fix typo --- proposals/4195-matrixrtc-livekit.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/proposals/4195-matrixrtc-livekit.md b/proposals/4195-matrixrtc-livekit.md index 1a798e28974..84a47b6d435 100644 --- a/proposals/4195-matrixrtc-livekit.md +++ b/proposals/4195-matrixrtc-livekit.md @@ -596,7 +596,7 @@ would likely result in a higher chance of implementation errors. ### Kicking users from the SFU on room leave -Since MatrixRTC sessions are tied to Matrix rooms, servers should to take care that client connections +Since MatrixRTC sessions are tied to Matrix rooms, servers should take care that client connections to the SFU don't exceed past the point where a user leaves the associated room. This is important because otherwise a malicious user being kicked from a room might continue to be connected to an ongoing RTC session related to the room. To prevent this, servers SHOULD remove any associated LiveKit From ee2ba76fd52b3f84a684a0b2a926a39a0bf33cf2 Mon Sep 17 00:00:00 2001 From: Johannes Marbach Date: Mon, 31 Aug 2026 16:07:00 +0200 Subject: [PATCH 084/122] Add Matrix room --- proposals/4195-matrixrtc-livekit.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/proposals/4195-matrixrtc-livekit.md b/proposals/4195-matrixrtc-livekit.md index 84a47b6d435..51963957569 100644 --- a/proposals/4195-matrixrtc-livekit.md +++ b/proposals/4195-matrixrtc-livekit.md @@ -597,7 +597,7 @@ would likely result in a higher chance of implementation errors. ### Kicking users from the SFU on room leave Since MatrixRTC sessions are tied to Matrix rooms, servers should take care that client connections -to the SFU don't exceed past the point where a user leaves the associated room. This is important +to the SFU don't exceed past the point where a user leaves the associated Matrix room. This is important because otherwise a malicious user being kicked from a room might continue to be connected to an ongoing RTC session related to the room. To prevent this, servers SHOULD remove any associated LiveKit participant identities from the related LiveKit rooms when a user leaves a Matrix room. From d080c8bb0b031a6acb6be97b3e0f57ac2db2e909 Mon Sep 17 00:00:00 2001 From: Johannes Marbach Date: Tue, 1 Sep 2026 09:51:34 +0200 Subject: [PATCH 085/122] Include bans --- proposals/4195-matrixrtc-livekit.md | 11 ++++++----- 1 file changed, 6 insertions(+), 5 deletions(-) diff --git a/proposals/4195-matrixrtc-livekit.md b/proposals/4195-matrixrtc-livekit.md index 51963957569..be410c837f5 100644 --- a/proposals/4195-matrixrtc-livekit.md +++ b/proposals/4195-matrixrtc-livekit.md @@ -594,13 +594,14 @@ would likely result in a higher chance of implementation errors. ## Security considerations -### Kicking users from the SFU on room leave +### Kicking users from the SFU on room leave/ban Since MatrixRTC sessions are tied to Matrix rooms, servers should take care that client connections -to the SFU don't exceed past the point where a user leaves the associated Matrix room. This is important -because otherwise a malicious user being kicked from a room might continue to be connected to an -ongoing RTC session related to the room. To prevent this, servers SHOULD remove any associated LiveKit -participant identities from the related LiveKit rooms when a user leaves a Matrix room. +to the SFU don't exceed past the point where a user is no longer joined to the associated Matrix room. +This is important because otherwise a malicious user being kicked from a room might continue to be +connected to an ongoing RTC session related to the room. To prevent this, servers SHOULD remove any +associated LiveKit participant identities from the related LiveKit rooms when a user leaves or is +banned from a Matrix room. It should be noted, that removing a participant from a LiveKit room also [revokes] their access token in the cloud version of LiveKit. This is _not_ the case in the self-hosted version, however. Homeservers From 9bc663dcebfa5c5c43018dbd81a22ce8e75cb3ad Mon Sep 17 00:00:00 2001 From: Johannes Marbach Date: Wed, 2 Sep 2026 07:46:56 +0200 Subject: [PATCH 086/122] Eliminate dependency on MSC4519 and make the transport optional --- proposals/4195-matrixrtc-livekit.md | 23 ++++++++++++++--------- 1 file changed, 14 insertions(+), 9 deletions(-) diff --git a/proposals/4195-matrixrtc-livekit.md b/proposals/4195-matrixrtc-livekit.md index be410c837f5..d6d48067b94 100644 --- a/proposals/4195-matrixrtc-livekit.md +++ b/proposals/4195-matrixrtc-livekit.md @@ -2,8 +2,9 @@ [MSC4143] introduces MatrixRTC as an extensible framework for real-time communication in Matrix. MatrixRTC uses so called transports to transfer the actual RTC data between RTC members. This -proposal introduces a transport based on the [LiveKit] Selective Forwarding Unit (SFU). The SFU -intelligently relays RTC data between members without them having to connect to each other directly. +proposal introduces an OPTIONAL transport based on the [LiveKit] Selective Forwarding Unit (SFU). +The SFU intelligently relays RTC data between members without them having to connect to each other +directly. The LiveKit SFU is integrated into Matrix in a multi-SFU configuration. In this setup, a homeserver may operate one or more SFUs. RTC members always publish their RTC data to a local SFU and announce @@ -47,7 +48,7 @@ subscribe │ discover SFU │ │ publish A new transport type `m.livekit` is introduced. Homeservers that support this transport announce it to clients by including a dedicated object in the response of the `/_matrix/client/v1/rtc/transports` -endpoint from [MSC4519]. The object has the following schema: +endpoint from [MSC4143]. The object has the following schema: - `type` (required, string): The transport's type identifier. MUST be `m.livekit`. - `url` (required, string): The SFU's WebSocket URL. Clients use this URL to connect to the SFU @@ -95,7 +96,6 @@ Below is an example of an appropriate membership event: } ``` -[MSC4519]: https://github.com/matrix-org/matrix-spec-proposals/pull/4519 [client SDKs]: https://docs.livekit.io/transport/sdk-platforms/ ### Mapping MatrixRTC members to LiveKit @@ -554,26 +554,31 @@ for achieving forward secrecy. ### Reliance on the LiveKit protocol implementations While being open source, LiveKit is developed and maintained by a commercial entity and is not an -open standard. As a result, future development or licensing changes by LiveKit, Inc could diverge +open standard. As a result, future development or licensing changes by LiveKit, Inc. could diverge from Matrix’s goals or limit interoperability. This is mitigated by the following factors: - Protocol openness: The LiveKit protocol and reference implementation are released under the [Apache 2.0 License] which allows for forking and independent evolution. If LiveKit’s direction or license were to change, Matrix could adopt the current protocol version and evolve it independently under an open governance model. -* No lock-in at the Matrix level: As per [MSC4143], transports in MatrixRTC are a generic abstraction +- No lock-in at the Matrix level: As per [MSC4143], transports in MatrixRTC are a generic abstraction that allows defining additional or alternative transport types in the future without breaking compatibility. -* Extensibility: Because the LiveKit protocol is open source, nothing prevents the Matrix community +- Extensibility: Because the LiveKit protocol is open source, nothing prevents the Matrix community from implementing additional functionality (such as cascading SFUs or other federation-oriented features) on top of the existing protocol if required. While this has been discussed with the LiveKit team and they did not object in principle, such extensions are not expected to depend on their involvement. -* Implementation pragmatism: The choice of LiveKit is pragmatic and helps accelerate development +- Implementation pragmatism: The choice of LiveKit is pragmatic and helps accelerate development and deployment of a functioning multi-SFU solution without necessarily establishing a permanent dependency. The current multi-SFU model also reduces the importance of features such as cascading SFUs that might otherwise require protocol changes. +To further reduce the risk and as already mentioned at the beginning of this proposal, the `m.livekit` +transport is made OPTIONAL for implementations. Given that as of writing this is the only available +transport for MatrixRTC, users of implementations that don't implement `m.livekit` will not be able +to use MatrixRTC for now. + [Apache 2.0 License]: https://github.com/livekit/livekit/blob/master/LICENSE ## Alternatives @@ -678,7 +683,7 @@ Apart from this, the endpoints introduced above should be referred to as follows ## Dependencies -This proposal depends on [MSC4143] and [MSC4519]. +This proposal depends on [MSC4143]. ## Appendix: hash derivation test vectors From e54229f8f80b73db5ee481e659c849d589a874cb Mon Sep 17 00:00:00 2001 From: Johannes Marbach Date: Wed, 2 Sep 2026 07:59:19 +0200 Subject: [PATCH 087/122] Fix typo. Co-authored-by: Hubert Chathi --- proposals/4195-matrixrtc-livekit.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/proposals/4195-matrixrtc-livekit.md b/proposals/4195-matrixrtc-livekit.md index d6d48067b94..743c5a34e67 100644 --- a/proposals/4195-matrixrtc-livekit.md +++ b/proposals/4195-matrixrtc-livekit.md @@ -472,7 +472,7 @@ time. Once the server observes the LiveKit participant's connection on the SFU, it MUST begin (or continue) restarting the delayed event periodically – again, with sufficient headroom. The server then continues -to monitor the participants connection state. Once the server detects that the participant has +to monitor the participant's connection state. Once the server detects that the participant has disconnected, it MUST trigger the sending of the delegated leave event. For maximum reliability, it is RECOMMENDED to use a combination of polling and listening to SFU [webhooks] From 99c54ad79054e8bff2b49f3960c8206361f8a816 Mon Sep 17 00:00:00 2001 From: Johannes Marbach Date: Wed, 2 Sep 2026 08:00:49 +0200 Subject: [PATCH 088/122] Remove leftover device ID in test vectors --- proposals/4195-matrixrtc-livekit.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/proposals/4195-matrixrtc-livekit.md b/proposals/4195-matrixrtc-livekit.md index 743c5a34e67..1584570322c 100644 --- a/proposals/4195-matrixrtc-livekit.md +++ b/proposals/4195-matrixrtc-livekit.md @@ -699,4 +699,4 @@ printf '%s' "${CANONICAL_JSON}" | openssl dgst -sha256 -binary | openssl base64 |------|-----------------|----------------|---------------|-------------------| | LiveKit room alias (no random bits) | `["!roomid:example.com", "slot1234"]` | `["!roomid:example.com","slot1234"]` | `3bce37ed6dfe8e6ccc563a083f7b4dc1b9be5f11d093688aa4e03b6aac37a927` | `O8437W3+jmzMVjoIP3tNwbm+XxHQk2iKpOA7aqw3qSc` | | LiveKit room alias (with random bits) | `["!roomid:example.com", "slot123", "random123"]` | `["!roomid:example.com","slot123","random123"]` | `20c78377e2b7308a894c8db4117048adea4a92184e46f7f7abc7f1deb96b8539` | `IMeDd+K3MIqJTI20EXBIrepKkhhORvf3q8fx3rlrhTk` | -| LiveKit participant identity | `["@alice:example.com", "memberABC"]` | `["@alice:example.com","DEVICE123","memberABC"]` | `337567b0b5eb91bc480c83573bae2ef0f6731720fd6581624142d1d9db21598b` | `M3VnsLXrkbxIDINXO64u8PZzFyD9ZYFiQULR2dshWYs` | +| LiveKit participant identity | `["@alice:example.com", "memberABC"]` | `["@alice:example.com","memberABC"]` | `337567b0b5eb91bc480c83573bae2ef0f6731720fd6581624142d1d9db21598b` | `M3VnsLXrkbxIDINXO64u8PZzFyD9ZYFiQULR2dshWYs` | From aed47513960d2a822e732e327bdc92633e2dd049 Mon Sep 17 00:00:00 2001 From: Johannes Marbach Date: Wed, 2 Sep 2026 08:27:10 +0200 Subject: [PATCH 089/122] Correct token request input parameters --- proposals/4195-matrixrtc-livekit.md | 33 +++++++++++++++++++---------- 1 file changed, 22 insertions(+), 11 deletions(-) diff --git a/proposals/4195-matrixrtc-livekit.md b/proposals/4195-matrixrtc-livekit.md index 1584570322c..844be5f3010 100644 --- a/proposals/4195-matrixrtc-livekit.md +++ b/proposals/4195-matrixrtc-livekit.md @@ -194,17 +194,27 @@ following schema: - `server_name` (string): The [server name](https://spec.matrix.org/v1.19/appendices/#server-name) for which a token is requested. Defaults to the server's own server name if omitted. - `url` (required, string): The WebSocket URL of the LiveKit SFU for which a token is requested. -- `room_id` (required, string): The room ID where the associated `m.rtc.member` event (see below) was sent. -- `slot_id` (required, string): The contents of the `slot_id` property of the associated `m.rtc.member` event. -- `member_id` (required, string): The `member.id` property of the associated `m.rtc.member` event. - -When requesting a token for publishing, the associated `m.rtc.member` event is the member's own event. -The client uses its own server name for `server_name` and the WebSocket URL discovered from -`/_matrix/client/v1/rtc/transports` for `url`. - -If, on the other hand, the token is requested for subscribing, the associated `m.rtc.member` event is -another member's event. In this case, the client derives the value for `server_name` from the `sender` -of that event and takes `url` from the respective `transports` array element in the event. +- `room_id` (required, string): The ID of the room in which the associated MatrixRTC session is + taking place. +- `slot_id` (required, string): The ID of the slot in which the associated MatrixRTC session is + taking place. +- `member_id` (required, string): The `member.id` property of the requesting user's own `m.rtc.member` + event. + +As mentioned before, clients always publish RTC media on a local SFU. Consequently, when requesting +a token for publishing, the client uses its own server name for `server_name` and the WebSocket URL +discovered from `/_matrix/client/v1/rtc/transports` for `url`. + +For subscribing to RTC media, clients need to connect to the publisher's chosen local SFU – which +might be on a different server. In this case, the subscribing client derives the `server_name` and +`url` parameters needed in the token request from the publisher's `m.rtc.member` event. In particular, +`server_name` is obtained by parsing the event's `sender` and `url` is taken from the respective +`transports` array element in the event. + +Note that as explained [later], tokens always include the grant required for subscribing. This means +that a token obtained for publishing on a local SFU also allows the bearer to subscribe to RTC streams +in the same LiveKit room on that SFU. As a result, clients only need to issue one token request per +SFU involved in the session. Below is an example of a token request for the `m.rtc.membership` example given further up. @@ -367,6 +377,7 @@ Below is an example of a LiveKit JWT for a local user: ``` [generate]: https://docs.livekit.io/frontends/build/authentication/custom/ +[later]: #access-token-properties ### Optional delegated delayed leave events From 2b9c2b69c2d34026de9ee519e5434894b8d33478 Mon Sep 17 00:00:00 2001 From: Johannes Marbach Date: Wed, 2 Sep 2026 08:31:51 +0200 Subject: [PATCH 090/122] Clarify that sub holds the requesting user's identity --- proposals/4195-matrixrtc-livekit.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/proposals/4195-matrixrtc-livekit.md b/proposals/4195-matrixrtc-livekit.md index 844be5f3010..1f26010e455 100644 --- a/proposals/4195-matrixrtc-livekit.md +++ b/proposals/4195-matrixrtc-livekit.md @@ -345,7 +345,7 @@ Different properties and grants can be applied when generating tokens using Live to ensure these are set appropriately so that clients can connect correctly and securely. In particular, servers MUST apply the following settings: -- `sub`: The LiveKit participant identity, derived as described above. +- `sub`: The LiveKit participant identity of the user that requested the token, derived as described above. - `video.room`: The LiveKit room name, derived as described above. - `video.roomCreate`: Always `true`. This allows clients to create the LiveKit room if it doesn't yet exist on the SFU. From 3a8004650de73a74e8d92f75ddc041c134cf2f47 Mon Sep 17 00:00:00 2001 From: Johannes Marbach Date: Wed, 2 Sep 2026 08:45:19 +0200 Subject: [PATCH 091/122] Add missing user_id parameter on SS request --- proposals/4195-matrixrtc-livekit.md | 8 +++++++- 1 file changed, 7 insertions(+), 1 deletion(-) diff --git a/proposals/4195-matrixrtc-livekit.md b/proposals/4195-matrixrtc-livekit.md index 1f26010e455..952c194d1e5 100644 --- a/proposals/4195-matrixrtc-livekit.md +++ b/proposals/4195-matrixrtc-livekit.md @@ -251,12 +251,18 @@ property `jwt` holding the token. If, conversely, `server_name` points to a remote server, the server triggers a `POST` request to the `/get_token` federation endpoint on that server. The body of the request contains the same JSON -object received in the client request but with `server_name` omitted. An example is given below: +object received in the client request with the following differences: + +- `server_name` is omitted since it is equal to the receiving server's own server name. +- An additional REQUIRED property `user_id` is included and holds the requesting user's Matrix ID. + +An example of the request is given below: ```http POST /_matrix/federation/v1/rtc/livekit/get_token { + "user_id": "@alice:alice.com", "url": "wss://livekit.example.com", "room_id": "!tDLCaLXijNtYcJZEey:example.com", "slot_id": "the_id", From 03a0ee96e57cde07ad14876cc43828ebcacae3eb Mon Sep 17 00:00:00 2001 From: Johannes Marbach Date: Wed, 2 Sep 2026 09:28:25 +0200 Subject: [PATCH 092/122] Fix typo Co-authored-by: gewitternacht <60887951+gewitternacht@users.noreply.github.com> --- proposals/4195-matrixrtc-livekit.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/proposals/4195-matrixrtc-livekit.md b/proposals/4195-matrixrtc-livekit.md index 952c194d1e5..70fba192d28 100644 --- a/proposals/4195-matrixrtc-livekit.md +++ b/proposals/4195-matrixrtc-livekit.md @@ -216,7 +216,7 @@ that a token obtained for publishing on a local SFU also allows the bearer to su in the same LiveKit room on that SFU. As a result, clients only need to issue one token request per SFU involved in the session. -Below is an example of a token request for the `m.rtc.membership` example given further up. +Below is an example of a token request for the `m.rtc.member` example given further up. ```http POST /_matrix/client/v1/rtc/livekit/get_token From 92bb5796af0eb90a24aa1dd760bf53a6081e1c68 Mon Sep 17 00:00:00 2001 From: Johannes Marbach Date: Wed, 2 Sep 2026 09:29:22 +0200 Subject: [PATCH 093/122] Be slightly more verbose to improve legibility Co-authored-by: gewitternacht <60887951+gewitternacht@users.noreply.github.com> --- proposals/4195-matrixrtc-livekit.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/proposals/4195-matrixrtc-livekit.md b/proposals/4195-matrixrtc-livekit.md index 70fba192d28..176ee8217f7 100644 --- a/proposals/4195-matrixrtc-livekit.md +++ b/proposals/4195-matrixrtc-livekit.md @@ -188,7 +188,7 @@ remote SFUs, a new pair of authenticated Client-Server and Server-Server endpoin The server SHOULD apply rate limiting to both of these endpoints. -To request a token, a client `POST`s to `/get_token` including in the body a JSON object with the +To request a token, a client `POST`s to the `/get_token` client endpoint including in the body a JSON object with the following schema: - `server_name` (string): The [server name](https://spec.matrix.org/v1.19/appendices/#server-name) From daec5418ef270c1ef594063e4baf38ce8cc716b2 Mon Sep 17 00:00:00 2001 From: Johannes Marbach Date: Wed, 2 Sep 2026 09:59:01 +0200 Subject: [PATCH 094/122] Also reject requests when the server doesn't know the room Co-authored-by: gewitternacht <60887951+gewitternacht@users.noreply.github.com> --- proposals/4195-matrixrtc-livekit.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/proposals/4195-matrixrtc-livekit.md b/proposals/4195-matrixrtc-livekit.md index 176ee8217f7..911419fcc9f 100644 --- a/proposals/4195-matrixrtc-livekit.md +++ b/proposals/4195-matrixrtc-livekit.md @@ -231,7 +231,7 @@ POST /_matrix/client/v1/rtc/livekit/get_token ``` Upon receiving the request, the server verifies that the requesting user is joined to the room -identified by `room_id`. If the user is not joined, the request MUST be rejected with HTTP 403 / +identified by `room_id`. If the user is not joined, or the server doesn't know the room, the request MUST be rejected with HTTP 403 / `M_FORBIDDEN`. If `server_name` is the server's own name and `url` does not match one of the server's own SFUs, From 508e8ca80a70d3e492efeeeeee52f00ef7df4044 Mon Sep 17 00:00:00 2001 From: Johannes Marbach Date: Wed, 2 Sep 2026 10:01:05 +0200 Subject: [PATCH 095/122] member -> member_id Co-authored-by: gewitternacht <60887951+gewitternacht@users.noreply.github.com> --- proposals/4195-matrixrtc-livekit.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/proposals/4195-matrixrtc-livekit.md b/proposals/4195-matrixrtc-livekit.md index 911419fcc9f..8d662c1f44c 100644 --- a/proposals/4195-matrixrtc-livekit.md +++ b/proposals/4195-matrixrtc-livekit.md @@ -495,7 +495,7 @@ disconnected, it MUST trigger the sending of the delegated leave event. For maximum reliability, it is RECOMMENDED to use a combination of polling and listening to SFU [webhooks] to monitor for SFU (dis)connections. -The server MUST only maintain a single delegated event per `room_id`, `slot_id`, `member` and MXID. +The server MUST only maintain a single delegated event per `room_id`, `slot_id`, `member_id` and MXID. Requests to delegate a different `delay_id` MUST invalidate earlier delegations for the same parameters. It is RECOMMENDED that servers apply rate limiting to the delegation endpoint. From 2b215b53ca5e3258e815c18c07ecdd2168f4af4d Mon Sep 17 00:00:00 2001 From: Johannes Marbach Date: Wed, 2 Sep 2026 10:02:48 +0200 Subject: [PATCH 096/122] Extend note across other home server Co-authored-by: gewitternacht <60887951+gewitternacht@users.noreply.github.com> --- proposals/4195-matrixrtc-livekit.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/proposals/4195-matrixrtc-livekit.md b/proposals/4195-matrixrtc-livekit.md index 8d662c1f44c..81eb31a4e68 100644 --- a/proposals/4195-matrixrtc-livekit.md +++ b/proposals/4195-matrixrtc-livekit.md @@ -342,7 +342,7 @@ sequenceDiagram U1->>L: Connect to Alice's SFU and start subscribing deactivate L - Note over U,L: Subscribing analogous to Bob (steps 7-12) + Note over U,H1: Subscribing analogous to Bob (steps 7-12) ``` #### Access token properties From bf19f8910e782e46bdaa89577bba002e4f0b4be6 Mon Sep 17 00:00:00 2001 From: Johannes Marbach Date: Wed, 2 Sep 2026 10:05:49 +0200 Subject: [PATCH 097/122] Add sender --- proposals/4195-matrixrtc-livekit.md | 1 + 1 file changed, 1 insertion(+) diff --git a/proposals/4195-matrixrtc-livekit.md b/proposals/4195-matrixrtc-livekit.md index 81eb31a4e68..6124e17d421 100644 --- a/proposals/4195-matrixrtc-livekit.md +++ b/proposals/4195-matrixrtc-livekit.md @@ -74,6 +74,7 @@ Below is an example of an appropriate membership event: ```json5 { "type": "m.rtc.member", + "sender": "@alice:alice.com", "content": { "slot_id": "the_id", "member": { From a71afab33ff561f12c991f9925727d2776daee9a Mon Sep 17 00:00:00 2001 From: Johannes Marbach Date: Wed, 2 Sep 2026 10:09:29 +0200 Subject: [PATCH 098/122] Fix broken WebSocket link --- proposals/4195-matrixrtc-livekit.md | 11 ++++++----- 1 file changed, 6 insertions(+), 5 deletions(-) diff --git a/proposals/4195-matrixrtc-livekit.md b/proposals/4195-matrixrtc-livekit.md index 6124e17d421..45796c8b44e 100644 --- a/proposals/4195-matrixrtc-livekit.md +++ b/proposals/4195-matrixrtc-livekit.md @@ -103,10 +103,11 @@ Below is an example of an appropriate membership event: LiveKit encapsulates RTC sessions in so-called [LiveKit rooms]. Within a LiveKit room, [LiveKit participants] use a WebSocket signaling connection that is guarded with an -access token. Publishing and subscribing to RTC streams then happens over WebRTC. -A LiveKit room is identified by a unique room "name" string while a LiveKit participant -is identified by a unique "identity" string. These LiveKit primitives need to be mapped -to the `m.rtc.member` events for MatrixRTC members from [MSC4143]. +access token. Publishing and subscribing to RTC streams then happens over WebRTC (see +[here] for further details). A LiveKit room is identified by a unique room "name" string +while a LiveKit participant is identified by a unique "identity" string. These LiveKit +primitives need to be mapped to the `m.rtc.member` events for MatrixRTC members from +[MSC4143]. #### LiveKit room names @@ -173,7 +174,7 @@ is not required here. ### Acquiring LiveKit access tokens -As mentioned above, [WebSocket] connections to LiveKit rooms are needed for publishing and subscribing +As mentioned above, WebSocket connections to LiveKit rooms are needed for publishing and subscribing to RTC streams. The LiveKit SFU requires an access token in the form of a JWT for these connections. In order to enable additional access control checks, responsibility for issuing these tokens is assigned to home servers. From 4dbe315ea490e25ce91b395e0232cb281efe8e97 Mon Sep 17 00:00:00 2001 From: Johannes Marbach Date: Wed, 2 Sep 2026 10:11:33 +0200 Subject: [PATCH 099/122] Clarify local --- proposals/4195-matrixrtc-livekit.md | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/proposals/4195-matrixrtc-livekit.md b/proposals/4195-matrixrtc-livekit.md index 45796c8b44e..a130eb736fb 100644 --- a/proposals/4195-matrixrtc-livekit.md +++ b/proposals/4195-matrixrtc-livekit.md @@ -7,12 +7,12 @@ The SFU intelligently relays RTC data between members without them having to con directly. The LiveKit SFU is integrated into Matrix in a multi-SFU configuration. In this setup, a homeserver -may operate one or more SFUs. RTC members always publish their RTC data to a local SFU and announce -their SFU choice via their `m.rtc.member` event. Other members then subscribe to the RTC data on the -publishing member's SFU – which might be different from the SFU they're publishing on themselves. -The homeserver provides mechanisms for discovering local SFUs and for acquiring access tokens for -both local and remote SFUs. This approach removes the need for an SFU election process and allows -servers to guard access to their SFUs. +may operate one or more SFUs. RTC members always publish their RTC data to a local SFU managed by +their homeserver and announce their SFU choice via their `m.rtc.member` event. Other members then +subscribe to the RTC data on the publishing member's SFU – which might be different from the SFU +they're publishing on themselves. The homeserver provides mechanisms for discovering local SFUs +and for acquiring access tokens for both local and remote SFUs. This approach removes the need for +an SFU election process and allows servers to guard access to their SFUs. The example below illustrates how two members from different homeservers A and B publish and subscribe to each other's RTC streams. From a5c8db23c2ef3f1dada29b24ac835fc9a3c3377b Mon Sep 17 00:00:00 2001 From: Johannes Marbach Date: Wed, 2 Sep 2026 10:37:05 +0200 Subject: [PATCH 100/122] Member IDs are non-deterministic --- proposals/4195-matrixrtc-livekit.md | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/proposals/4195-matrixrtc-livekit.md b/proposals/4195-matrixrtc-livekit.md index a130eb736fb..1ccd35d30c8 100644 --- a/proposals/4195-matrixrtc-livekit.md +++ b/proposals/4195-matrixrtc-livekit.md @@ -651,8 +651,9 @@ above, eliminates this leak, too. For another, [LiveKit participant identities] are pseudonymised as well which prevents the SFU from correlating SFU participants with Matrix users. The identity derivation process involves the value of -`member.id` which clients change every time they join a slot. As a result, the SFU is unable to track -Matrix users across different calls and no further salting is required. +`member.id` which, as per [MSC4143], is non-deterministic and changed every time a client joins a slot. +As a result, the SFU is unable to track Matrix users across different calls and no further salting is +required. The LiveKit SFU and the homeserver necessarily form a high trust relationship. In order for the homeserver to extend SFU access tokens, secrets need to be agreed upon between the homeserver and the SFU. This From d2ab562ae0a56d721cdf08f4940e6177b9151dff Mon Sep 17 00:00:00 2001 From: Johannes Marbach Date: Wed, 2 Sep 2026 13:39:09 +0200 Subject: [PATCH 101/122] Clarify forwarding of federation errors --- proposals/4195-matrixrtc-livekit.md | 8 ++++++-- 1 file changed, 6 insertions(+), 2 deletions(-) diff --git a/proposals/4195-matrixrtc-livekit.md b/proposals/4195-matrixrtc-livekit.md index 1ccd35d30c8..231887e6c1a 100644 --- a/proposals/4195-matrixrtc-livekit.md +++ b/proposals/4195-matrixrtc-livekit.md @@ -276,10 +276,14 @@ Upon receiving the request, the remote server verifies that the origin server is identified by `room_id`. If the origin server is not joined or the remote server doesn't know the room, the request MUST be rejected with HTTP 403 / `M_FORBIDDEN`. -If `url` does not match one of the receiving server's own SFUs, the request is rejected with +If `url` does not match one of the remote server's own SFUs, the request is rejected with HTTP 400 / `M_INVALID_PARAM`. -Otherwise, the remote server generates a token for its SFU and returns it in the same response format +HTTP 403 / `M_FORBIDDEN` and HTTP 400 / `M_INVALID_PARAM` errors from the remote server MUST be relayed +back to the client by the origin server. Any other error MUST result in HTTP 502 / `M_UNKNOWN` in +the client response. + +If no errors occured, the remote server generates a token for its SFU and returns it in the same response format used for the Client-Server endpoint. ```http From c6e6e05d5e911d9a0d0edfa1b3bfd8c04c0c0773 Mon Sep 17 00:00:00 2001 From: Johannes Marbach Date: Wed, 2 Sep 2026 14:44:24 +0200 Subject: [PATCH 102/122] Deny room creation by clients --- proposals/4195-matrixrtc-livekit.md | 18 ++++++++++++------ 1 file changed, 12 insertions(+), 6 deletions(-) diff --git a/proposals/4195-matrixrtc-livekit.md b/proposals/4195-matrixrtc-livekit.md index 231887e6c1a..5fa47902ea7 100644 --- a/proposals/4195-matrixrtc-livekit.md +++ b/proposals/4195-matrixrtc-livekit.md @@ -233,13 +233,15 @@ POST /_matrix/client/v1/rtc/livekit/get_token ``` Upon receiving the request, the server verifies that the requesting user is joined to the room -identified by `room_id`. If the user is not joined, or the server doesn't know the room, the request MUST be rejected with HTTP 403 / -`M_FORBIDDEN`. +identified by `room_id`. If the user is not joined, or the server doesn't know the room, the request +MUST be rejected with HTTP 403 / `M_FORBIDDEN`. If `server_name` is the server's own name and `url` does not match one of the server's own SFUs, the request is rejected with HTTP 400 / `M_INVALID_PARAM`. If `server_name` is the server's own name and `url` matches one of the server's own SFUs, the server +derives the associated LiveKit room name and ensures that the room exists, [creating] it if needed. +This is REQUIRED because otherwise clients won't be able to connect to the room. The server then generates a token for the SFU and responds with HTTP 200 and a JSON object with a single required property `jwt` holding the token. @@ -283,8 +285,8 @@ HTTP 403 / `M_FORBIDDEN` and HTTP 400 / `M_INVALID_PARAM` errors from the remote back to the client by the origin server. Any other error MUST result in HTTP 502 / `M_UNKNOWN` in the client response. -If no errors occured, the remote server generates a token for its SFU and returns it in the same response format -used for the Client-Server endpoint. +If no errors occurred, the remote server ensures the room exists and generates a token for its SFU, +returning it in the same response format used for the Client-Server endpoint. ```http 200 OK @@ -359,8 +361,10 @@ servers MUST apply the following settings: - `sub`: The LiveKit participant identity of the user that requested the token, derived as described above. - `video.room`: The LiveKit room name, derived as described above. -- `video.roomCreate`: Always `true`. This allows clients to create the LiveKit room if it doesn't yet - exist on the SFU. +- `video.roomCreate`: Always `false`. This grant, somewhat [counterintuitively], also allows the token + holder to delete the LiveKit room which includes kicking all joined participants. Since this is a possible + denial-of-service vector, room creation is exclusively and preemptively performed by the homeserver as + described above. - `video.roomJoin`: Always `true`. This enables clients to join the LiveKit room if it exists. - `video.canPublish`: `true` if the token was requested by a local user. `false` otherwise. This enforces the multi-SFU configuration and ensures clients can only publish RTC data on a local SFU. @@ -390,6 +394,8 @@ Below is an example of a LiveKit JWT for a local user: [generate]: https://docs.livekit.io/frontends/build/authentication/custom/ [later]: #access-token-properties +[creating]: https://docs.livekit.io/reference/other/roomservice-api/#createroom +[counterintuitively]: https://docs.livekit.io/frontends/reference/tokens-grants/#video-grant ### Optional delegated delayed leave events From e6857a22e6a49653dd363083a1735f9fcba8465b Mon Sep 17 00:00:00 2001 From: Johannes Marbach Date: Wed, 2 Sep 2026 15:32:15 +0200 Subject: [PATCH 103/122] Specify requirements for exp and nbf --- proposals/4195-matrixrtc-livekit.md | 15 ++++++++------- 1 file changed, 8 insertions(+), 7 deletions(-) diff --git a/proposals/4195-matrixrtc-livekit.md b/proposals/4195-matrixrtc-livekit.md index 5fa47902ea7..720dbd1fe65 100644 --- a/proposals/4195-matrixrtc-livekit.md +++ b/proposals/4195-matrixrtc-livekit.md @@ -360,6 +360,12 @@ to ensure these are set appropriately so that clients can connect correctly and servers MUST apply the following settings: - `sub`: The LiveKit participant identity of the user that requested the token, derived as described above. +- `exp`: When using a self-hosted LiveKit SFU, servers SHOULD use a sufficiently short expiration time (`exp`) + because [token revocation] is a LiveKit Cloud feature only. Otherwise, the expiration time is less + significant because the SFU [proactively refreshes tokens] via a client's WebSocket signalling connection. + Servers SHOULD rely on the default expiration time of 6 hours used in LiveKit's SDKs. +- `nbf`: The current time. This is required because LiveKit Cloud uses the token's not-before (`nbf`) + timestamp in [token revocation]. - `video.room`: The LiveKit room name, derived as described above. - `video.roomCreate`: Always `false`. This grant, somewhat [counterintuitively], also allows the token holder to delete the LiveKit room which includes kicking all joined participants. Since this is a possible @@ -395,6 +401,8 @@ Below is an example of a LiveKit JWT for a local user: [generate]: https://docs.livekit.io/frontends/build/authentication/custom/ [later]: #access-token-properties [creating]: https://docs.livekit.io/reference/other/roomservice-api/#createroom +[token revocation]: https://docs.livekit.io/frontends/reference/tokens-grants/#token-revocation +[proactively refreshes tokens]: https://docs.livekit.io/frontends/reference/tokens-grants/#token-refresh [counterintuitively]: https://docs.livekit.io/frontends/reference/tokens-grants/#video-grant ### Optional delegated delayed leave events @@ -637,13 +645,6 @@ connected to an ongoing RTC session related to the room. To prevent this, server associated LiveKit participant identities from the related LiveKit rooms when a user leaves or is banned from a Matrix room. -It should be noted, that removing a participant from a LiveKit room also [revokes] their access token -in the cloud version of LiveKit. This is _not_ the case in the self-hosted version, however. Homeservers -that rely on a self-hosted LiveKit instance should issue access tokens with a sufficiently short TTL -to mitigate this. - -[revokes]: https://docs.livekit.io/frontends/reference/tokens-grants/#token-revocation - ### Reducing metadata leakage to the SFU With SFUs always being tied to homeservers under this proposal, two principal deployment models exist From 566a9eb80a6ba93adf8b4b8c02585f4fc84a04bb Mon Sep 17 00:00:00 2001 From: Johannes Marbach Date: Wed, 2 Sep 2026 15:41:13 +0200 Subject: [PATCH 104/122] Clarify how to obtain the MXID --- proposals/4195-matrixrtc-livekit.md | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/proposals/4195-matrixrtc-livekit.md b/proposals/4195-matrixrtc-livekit.md index 720dbd1fe65..b7ea917347d 100644 --- a/proposals/4195-matrixrtc-livekit.md +++ b/proposals/4195-matrixrtc-livekit.md @@ -499,8 +499,9 @@ object to confirm the delegation. {} ``` -The server derives the LiveKit room alias and LiveKit participant identity from the `room_id`, -`slot_id` and `member_id` parameters as well as the request's authorization as described above. +The server derives the LiveKit room alias and LiveKit participant identity as described above from +the `room_id`, `slot_id` and `member_id` parameters as well as the user's Matrix ID as derived from +the request's access token. The server then waits for the participant to connect to the SFU. How long the server waits before giving up is left as an implementation detail. If the waiting duration exceeds the delegated event's From 53ab213af52d28b683c110495f7560f38274b74a Mon Sep 17 00:00:00 2001 From: Johannes Marbach Date: Wed, 2 Sep 2026 15:47:36 +0200 Subject: [PATCH 105/122] Clarify delegation request parameter validation --- proposals/4195-matrixrtc-livekit.md | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/proposals/4195-matrixrtc-livekit.md b/proposals/4195-matrixrtc-livekit.md index b7ea917347d..11e3a837002 100644 --- a/proposals/4195-matrixrtc-livekit.md +++ b/proposals/4195-matrixrtc-livekit.md @@ -490,8 +490,10 @@ When scheduling delayed events that are meant to be delegated, clients SHOULD us 1 hour. This avoids unnecessarily frequent restarts of the delayed event. Servers MAY reject delegation requests with HTTP 400 / `M_INVALID_PARAM` when the delegated event has a lower timeout. -Otherwise, if the request parameters are valid, the server responds with HTTP 200 and an empty JSON -object to confirm the delegation. +If the server cannot find the delayed event based on the `delay_id` or if it can find the delayed event +but it belongs to another room or user, the delegation request MUST be rejected with HTTP 400 / `M_INVALID_PARAM`. + +Otherwise, the server responds with HTTP 200 and an empty JSON object to confirm the delegation. ```http 200 OK From 051f12bb4227c71bd4e1d5190b79eb04c3744e5f Mon Sep 17 00:00:00 2001 From: Johannes Marbach Date: Wed, 2 Sep 2026 16:06:51 +0200 Subject: [PATCH 106/122] Remove expiration time recommendation for LiveKit Cloud --- proposals/4195-matrixrtc-livekit.md | 1 - 1 file changed, 1 deletion(-) diff --git a/proposals/4195-matrixrtc-livekit.md b/proposals/4195-matrixrtc-livekit.md index 11e3a837002..978281bcccb 100644 --- a/proposals/4195-matrixrtc-livekit.md +++ b/proposals/4195-matrixrtc-livekit.md @@ -363,7 +363,6 @@ servers MUST apply the following settings: - `exp`: When using a self-hosted LiveKit SFU, servers SHOULD use a sufficiently short expiration time (`exp`) because [token revocation] is a LiveKit Cloud feature only. Otherwise, the expiration time is less significant because the SFU [proactively refreshes tokens] via a client's WebSocket signalling connection. - Servers SHOULD rely on the default expiration time of 6 hours used in LiveKit's SDKs. - `nbf`: The current time. This is required because LiveKit Cloud uses the token's not-before (`nbf`) timestamp in [token revocation]. - `video.room`: The LiveKit room name, derived as described above. From 2954f3eaccb935c9115fd4cecc2c68b1e614642e Mon Sep 17 00:00:00 2001 From: Johannes Marbach Date: Wed, 2 Sep 2026 16:26:38 +0200 Subject: [PATCH 107/122] Clarify derivation process --- proposals/4195-matrixrtc-livekit.md | 15 ++++++++------- 1 file changed, 8 insertions(+), 7 deletions(-) diff --git a/proposals/4195-matrixrtc-livekit.md b/proposals/4195-matrixrtc-livekit.md index 978281bcccb..31c92a5b6b6 100644 --- a/proposals/4195-matrixrtc-livekit.md +++ b/proposals/4195-matrixrtc-livekit.md @@ -500,14 +500,14 @@ Otherwise, the server responds with HTTP 200 and an empty JSON object to confirm {} ``` -The server derives the LiveKit room alias and LiveKit participant identity as described above from -the `room_id`, `slot_id` and `member_id` parameters as well as the user's Matrix ID as derived from -the request's access token. +The server then takes the user's Matrix ID (as resolved from the request's access token) as well as the +`room_id`, `slot_id` and `member_id` parameters from the request body to derive the LiveKit room alias +and LiveKit participant identity. The procedure for this was given [earlier]. -The server then waits for the participant to connect to the SFU. How long the server waits before -giving up is left as an implementation detail. If the waiting duration exceeds the delegated event's -delay, the server MUST restart the event periodically and with sufficient headroom to the expiration -time. +Following that, the server waits for the participant to connect to the room on the SFU. How long the +server waits before giving up is left as an implementation detail. If the waiting duration exceeds the +delegated event's delay, the server MUST restart the event periodically and with sufficient headroom to +the expiration time. Once the server observes the LiveKit participant's connection on the SFU, it MUST begin (or continue) restarting the delayed event periodically – again, with sufficient headroom. The server then continues @@ -523,6 +523,7 @@ Requests to delegate a different `delay_id` MUST invalidate earlier delegations It is RECOMMENDED that servers apply rate limiting to the delegation endpoint. [MSC4140]: https://github.com/matrix-org/matrix-spec-proposals/pull/4140 +[earlier]: #mapping-matrixrtc-members-to-livekit [webhooks]: https://docs.livekit.io/intro/basics/rooms-participants-tracks/webhooks-events/ From f241dd76cc5637f39bb54dc82d139fa8ca55b880 Mon Sep 17 00:00:00 2001 From: Johannes Marbach Date: Wed, 2 Sep 2026 17:03:44 +0200 Subject: [PATCH 108/122] Add resource abuse section --- proposals/4195-matrixrtc-livekit.md | 13 +++++++++++++ 1 file changed, 13 insertions(+) diff --git a/proposals/4195-matrixrtc-livekit.md b/proposals/4195-matrixrtc-livekit.md index 31c92a5b6b6..a590bb427f6 100644 --- a/proposals/4195-matrixrtc-livekit.md +++ b/proposals/4195-matrixrtc-livekit.md @@ -639,6 +639,19 @@ would likely result in a higher chance of implementation errors. ## Security considerations +### Resource abuse + +Homeservers preemptively create LiveKit rooms when SFU tokens are requested by both local and remote +users. Thus, any Matrix room member is able to trigger the creation of an associated LiveKit room. +LiveKit rooms themselves are lightweight, however, and applying rate limitting on the `/get_token` +endpoints further mitigates this problem. + +Users publishing and subscribing to RTC data within LiveKit rooms has a larger resource impact +though. Any Matrix room member is able to connect to an associated LiveKit room and publish and/or +subscribe to media streams. Again, rate limitting the `/get_token` endpoints mitigates this concern. +Servers MAY apply additional countermeasures such as limitting the maximum allowed lifetime of LiveKit +rooms or restricting SFU access to trusted users and/or servers. + ### Kicking users from the SFU on room leave/ban Since MatrixRTC sessions are tied to Matrix rooms, servers should take care that client connections From 7e005c2dee8f0ee2b99ffe9660cc3c11e5d4f00f Mon Sep 17 00:00:00 2001 From: Johannes Marbach Date: Wed, 2 Sep 2026 17:54:21 +0200 Subject: [PATCH 109/122] Rearrange kicking section --- proposals/4195-matrixrtc-livekit.md | 20 +++++++++++--------- 1 file changed, 11 insertions(+), 9 deletions(-) diff --git a/proposals/4195-matrixrtc-livekit.md b/proposals/4195-matrixrtc-livekit.md index a590bb427f6..6557b96ec7e 100644 --- a/proposals/4195-matrixrtc-livekit.md +++ b/proposals/4195-matrixrtc-livekit.md @@ -397,6 +397,17 @@ Below is an example of a LiveKit JWT for a local user: } ``` +#### Kicking users from the SFU on room leave/ban + +A MatrixRTC session is tied to exactly one Matrix room and exactly one LiveKit room per SFU involved +in the session. Access to LiveKit rooms is guarded by LiveKit access tokens. This means that it would +be desirable to couple the lifetime of LiveKit tokens to the period of room membership. This is important +because a malicious user being kicked from a Matrix room could otherwise continue to be connected to an +ongoing RTC session related to the room. To prevent this, servers SHOULD remove any associated LiveKit +participant identities from the related LiveKit rooms when a user leaves or is banned from a Matrix room. +Note that this doesn't obsolete the recommendation to use sufficiently short-lived access tokens in +self-hosted LiveKit deployments given in the previous section. + [generate]: https://docs.livekit.io/frontends/build/authentication/custom/ [later]: #access-token-properties [creating]: https://docs.livekit.io/reference/other/roomservice-api/#createroom @@ -652,15 +663,6 @@ subscribe to media streams. Again, rate limitting the `/get_token` endpoints mit Servers MAY apply additional countermeasures such as limitting the maximum allowed lifetime of LiveKit rooms or restricting SFU access to trusted users and/or servers. -### Kicking users from the SFU on room leave/ban - -Since MatrixRTC sessions are tied to Matrix rooms, servers should take care that client connections -to the SFU don't exceed past the point where a user is no longer joined to the associated Matrix room. -This is important because otherwise a malicious user being kicked from a room might continue to be -connected to an ongoing RTC session related to the room. To prevent this, servers SHOULD remove any -associated LiveKit participant identities from the related LiveKit rooms when a user leaves or is -banned from a Matrix room. - ### Reducing metadata leakage to the SFU With SFUs always being tied to homeservers under this proposal, two principal deployment models exist From fcba45417c2bfbb164b735230fc2d0a91d4d37c8 Mon Sep 17 00:00:00 2001 From: Johannes Marbach Date: Wed, 2 Sep 2026 17:55:34 +0200 Subject: [PATCH 110/122] Fix typos --- proposals/4195-matrixrtc-livekit.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/proposals/4195-matrixrtc-livekit.md b/proposals/4195-matrixrtc-livekit.md index 6557b96ec7e..01812623c8b 100644 --- a/proposals/4195-matrixrtc-livekit.md +++ b/proposals/4195-matrixrtc-livekit.md @@ -654,13 +654,13 @@ would likely result in a higher chance of implementation errors. Homeservers preemptively create LiveKit rooms when SFU tokens are requested by both local and remote users. Thus, any Matrix room member is able to trigger the creation of an associated LiveKit room. -LiveKit rooms themselves are lightweight, however, and applying rate limitting on the `/get_token` +LiveKit rooms themselves are lightweight, however, and applying rate limiting on the `/get_token` endpoints further mitigates this problem. Users publishing and subscribing to RTC data within LiveKit rooms has a larger resource impact though. Any Matrix room member is able to connect to an associated LiveKit room and publish and/or -subscribe to media streams. Again, rate limitting the `/get_token` endpoints mitigates this concern. -Servers MAY apply additional countermeasures such as limitting the maximum allowed lifetime of LiveKit +subscribe to media streams. Again, rate limiting the `/get_token` endpoints mitigates this concern. +Servers MAY apply additional countermeasures such as limiting the maximum allowed lifetime of LiveKit rooms or restricting SFU access to trusted users and/or servers. ### Reducing metadata leakage to the SFU From 80370c2907d3a55089c65118204d273d53ac67e6 Mon Sep 17 00:00:00 2001 From: Johannes Marbach Date: Thu, 3 Sep 2026 07:36:03 +0200 Subject: [PATCH 111/122] Be more precise about who can publish --- proposals/4195-matrixrtc-livekit.md | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/proposals/4195-matrixrtc-livekit.md b/proposals/4195-matrixrtc-livekit.md index 01812623c8b..3b86f1a07d4 100644 --- a/proposals/4195-matrixrtc-livekit.md +++ b/proposals/4195-matrixrtc-livekit.md @@ -658,10 +658,10 @@ LiveKit rooms themselves are lightweight, however, and applying rate limiting on endpoints further mitigates this problem. Users publishing and subscribing to RTC data within LiveKit rooms has a larger resource impact -though. Any Matrix room member is able to connect to an associated LiveKit room and publish and/or -subscribe to media streams. Again, rate limiting the `/get_token` endpoints mitigates this concern. -Servers MAY apply additional countermeasures such as limiting the maximum allowed lifetime of LiveKit -rooms or restricting SFU access to trusted users and/or servers. +though. Any Matrix room member is able to connect to an associated LiveKit room and subscribe to media +streams. Local room members can also publish media. Again, rate limiting the `/get_token` endpoints +mitigates this concern. Servers MAY apply additional countermeasures such as limiting the maximum +allowed lifetime of LiveKit rooms or restricting SFU access to trusted users and/or servers. ### Reducing metadata leakage to the SFU From 1cf693b379d7d096c1ff150b658c75535f6d28f1 Mon Sep 17 00:00:00 2001 From: Johannes Marbach Date: Thu, 3 Sep 2026 09:12:10 +0200 Subject: [PATCH 112/122] Add section about side stepping RTC membership --- proposals/4195-matrixrtc-livekit.md | 20 ++++++++++++++++++++ 1 file changed, 20 insertions(+) diff --git a/proposals/4195-matrixrtc-livekit.md b/proposals/4195-matrixrtc-livekit.md index 3b86f1a07d4..e298d69d352 100644 --- a/proposals/4195-matrixrtc-livekit.md +++ b/proposals/4195-matrixrtc-livekit.md @@ -697,6 +697,26 @@ however. In unencrypted RTC sessions, the SFU operator of course has full access [LiveKit Cloud]: https://cloud.livekit.io [LiveKit participant identities]: #liveKit-participant-identities +### Side stepping MatrixRTC membership + +Given that `m.rtc.member` events are encrypted, the homeserver has no way to verify whether a user +requesting an SFU token has actually joined the slot with the claimed `member_id`. As a result, +room members could subscribe to RTC streams even without sending an RTC member event. + +In encrypted rooms, they wouldn't receive the keys needed to decrypt the streams but could still +access metadata. This includes whether participants are publishing audio, video or a screenshare +and whether they are currently speaking (both exposed unencrypted over LiveKit's WebSocket signalling +connection). + +In unencrypted rooms, in turn, the published media is accessible directly given that it isn't encrypted. + +As a mitigation, [MSC4143] already [includes] a RECOMMENDATION for clients to notify their users +about RTC streams or identities that cannot be mapped to RTC members. For the `m.livekit` transport +this means that clients SHOULD notify users when they observe LiveKit participants for which no +corresponding `m.rtc.member` event exists. + +[includes]: https://github.com/matrix-org/matrix-spec-proposals/blob/toger5/matrixRTC/proposals/4143-matrix-rtc.md#unmappable-rtc-streams + ### Error handling and information disclosure Implementations of the `/get_token` endpoint SHOULD take care not to disclose sensitive internal From 4e7863cc64f1f9810888e4673d12767dcca26186 Mon Sep 17 00:00:00 2001 From: Johannes Marbach Date: Thu, 3 Sep 2026 09:15:37 +0200 Subject: [PATCH 113/122] The SS endpoint MUST also check if the user is joined --- proposals/4195-matrixrtc-livekit.md | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-) diff --git a/proposals/4195-matrixrtc-livekit.md b/proposals/4195-matrixrtc-livekit.md index e298d69d352..e171aa245f2 100644 --- a/proposals/4195-matrixrtc-livekit.md +++ b/proposals/4195-matrixrtc-livekit.md @@ -274,9 +274,10 @@ POST /_matrix/federation/v1/rtc/livekit/get_token } ``` -Upon receiving the request, the remote server verifies that the origin server is joined to the room -identified by `room_id`. If the origin server is not joined or the remote server doesn't know the room, -the request MUST be rejected with HTTP 403 / `M_FORBIDDEN`. +Upon receiving the request, the remote server verifies that `user_id` belongs to the origin server and +that the user is joined to the room identified by `room_id`. If the user is from another server or not +not joined to the room or if the remote server doesn't know the room, the request MUST be rejected with +HTTP 403 / `M_FORBIDDEN`. If `url` does not match one of the remote server's own SFUs, the request is rejected with HTTP 400 / `M_INVALID_PARAM`. From a60ca882d9672ff87697e7201fabaac6256b85b6 Mon Sep 17 00:00:00 2001 From: Johannes Marbach Date: Thu, 3 Sep 2026 10:11:03 +0200 Subject: [PATCH 114/122] Add missing url parameter on delegation endpoint --- proposals/4195-matrixrtc-livekit.md | 13 +++++++++---- 1 file changed, 9 insertions(+), 4 deletions(-) diff --git a/proposals/4195-matrixrtc-livekit.md b/proposals/4195-matrixrtc-livekit.md index e171aa245f2..c300c84a3aa 100644 --- a/proposals/4195-matrixrtc-livekit.md +++ b/proposals/4195-matrixrtc-livekit.md @@ -479,10 +479,11 @@ Clients delegate delayed leave events to their homeserver by `POST`ing to a new `/_matrix/client/v1/rtc/livekit/delegate_delayed_leave`. The body of the request contains a JSON object with the following schema: -- `room_id` (required, `string`): The room ID in which the delayed `m.rtc.member` event was scheduled. -- `slot_id` (required, `string`): The contents of the `slot_id` property of the `m.rtc.member` event. -- `member_id` (required, `string`): The `member.id` property of the `m.rtc.member` event. -- `delay_id` (required, `string`): The delayed event ID obtained when scheduling the `m.rtc.member` event. +- `url` (required, string): The WebSocket URL of the LiveKit SFU that the user has connected to. +- `room_id` (required, string): The room ID in which the delayed `m.rtc.member` event was scheduled. +- `slot_id` (required, string): The contents of the `slot_id` property of the `m.rtc.member` event. +- `member_id` (required, string): The `member.id` property of the `m.rtc.member` event. +- `delay_id` (required, string): The delayed event ID obtained when scheduling the `m.rtc.member` event. Below is an example of a request: @@ -490,6 +491,7 @@ Below is an example of a request: POST /_matrix/client/v1/rtc/livekit/delegate_delayed_leave { + "url": "wss://livekit.example.com", "room_id": "!tDLCaLXijNtYcJZEey:example.com", "slot_id": "the_id", "member_id": "id", @@ -504,6 +506,9 @@ requests with HTTP 400 / `M_INVALID_PARAM` when the delegated event has a lower If the server cannot find the delayed event based on the `delay_id` or if it can find the delayed event but it belongs to another room or user, the delegation request MUST be rejected with HTTP 400 / `M_INVALID_PARAM`. +If `url` does not match any of the remote server's own SFUs, the request is rejected with +HTTP 400 / `M_INVALID_PARAM`. + Otherwise, the server responds with HTTP 200 and an empty JSON object to confirm the delegation. ```http From 183c568853db79167c837c706a17a2be49e2fc12 Mon Sep 17 00:00:00 2001 From: Johannes Marbach Date: Thu, 3 Sep 2026 11:05:48 +0200 Subject: [PATCH 115/122] Mention power levels check --- proposals/4195-matrixrtc-livekit.md | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/proposals/4195-matrixrtc-livekit.md b/proposals/4195-matrixrtc-livekit.md index c300c84a3aa..fa5c6706cb8 100644 --- a/proposals/4195-matrixrtc-livekit.md +++ b/proposals/4195-matrixrtc-livekit.md @@ -721,6 +721,11 @@ about RTC streams or identities that cannot be mapped to RTC members. For the `m this means that clients SHOULD notify users when they observe LiveKit participants for which no corresponding `m.rtc.member` event exists. +Note that while the server could verify that the user at least has the necessary power level to +send `m.rtc.member` events, a malicious server could fake the `user_id` when requesting an SFU +token over federation. Given this and the limitations around denying side stepping `m.rtc.member` +events, a power levels check wouldn't lead to a significant improvement. + [includes]: https://github.com/matrix-org/matrix-spec-proposals/blob/toger5/matrixRTC/proposals/4143-matrix-rtc.md#unmappable-rtc-streams ### Error handling and information disclosure From b7401db3cd52ef61cd5330425c2ca1b5c11cd17b Mon Sep 17 00:00:00 2001 From: Johannes Marbach Date: Tue, 8 Sep 2026 08:40:01 +0200 Subject: [PATCH 116/122] Reiterate optionality Co-authored-by: Travis Ralston --- proposals/4195-matrixrtc-livekit.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/proposals/4195-matrixrtc-livekit.md b/proposals/4195-matrixrtc-livekit.md index fa5c6706cb8..1d972d2061c 100644 --- a/proposals/4195-matrixrtc-livekit.md +++ b/proposals/4195-matrixrtc-livekit.md @@ -46,7 +46,7 @@ subscribe │ discover SFU │ │ publish ### Discovering and announcing transports -A new transport type `m.livekit` is introduced. Homeservers that support this transport announce it +A new transport type `m.livekit` is introduced, which homeservers MAY support. Homeservers that support this transport announce it to clients by including a dedicated object in the response of the `/_matrix/client/v1/rtc/transports` endpoint from [MSC4143]. The object has the following schema: From a314bc2b7014d42700d1124d5ea89832894a182f Mon Sep 17 00:00:00 2001 From: Johannes Marbach Date: Tue, 8 Sep 2026 08:43:30 +0200 Subject: [PATCH 117/122] Clarify that the endpoints are mandatory when supporting m.livekit Co-authored-by: Travis Ralston --- proposals/4195-matrixrtc-livekit.md | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/proposals/4195-matrixrtc-livekit.md b/proposals/4195-matrixrtc-livekit.md index 1d972d2061c..94df1b0a896 100644 --- a/proposals/4195-matrixrtc-livekit.md +++ b/proposals/4195-matrixrtc-livekit.md @@ -183,7 +183,8 @@ Servers can [generate] the tokens by using one of the LiveKit SDKs and inputting including the LiveKit room name and the LiveKit participant identifier. The procedure also requires secrets agreed upon between the homeserver and the respective SFU. This means homeservers can only generate tokens for their own SFUs. To allow clients to request tokens for both local and -remote SFUs, a new pair of authenticated Client-Server and Server-Server endpoints is introduced: +remote SFUs, a new pair of authenticated Client-Server and Server-Server endpoints is introduced. +These endpoints MUST be implemented by servers supporting the `m.livekit` transport. They are: - `POST /_matrix/client/v1/rtc/livekit/get_token` - `POST /_matrix/federation/v1/rtc/livekit/get_token` From 17a8d4970ed7911ad5b149ff1f8823a6d111d3c1 Mon Sep 17 00:00:00 2001 From: Johannes Marbach Date: Tue, 8 Sep 2026 08:44:27 +0200 Subject: [PATCH 118/122] The delegation endpoint is required when a server supports the transport Co-authored-by: Travis Ralston --- proposals/4195-matrixrtc-livekit.md | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/proposals/4195-matrixrtc-livekit.md b/proposals/4195-matrixrtc-livekit.md index 94df1b0a896..a21635241a8 100644 --- a/proposals/4195-matrixrtc-livekit.md +++ b/proposals/4195-matrixrtc-livekit.md @@ -477,7 +477,8 @@ sequenceDiagram ``` Clients delegate delayed leave events to their homeserver by `POST`ing to a new authenticated endpoint -`/_matrix/client/v1/rtc/livekit/delegate_delayed_leave`. The body of the request contains a JSON +`/_matrix/client/v1/rtc/livekit/delegate_delayed_leave`. Servers supporting `m.livekit` +MUST support this endpoint too. The body of the request contains a JSON object with the following schema: - `url` (required, string): The WebSocket URL of the LiveKit SFU that the user has connected to. From 4b472c7e7bf1399551f68be11e12cc5cafb06fea Mon Sep 17 00:00:00 2001 From: Johannes Marbach Date: Tue, 8 Sep 2026 09:12:25 +0200 Subject: [PATCH 119/122] /get_token is available to guests --- proposals/4195-matrixrtc-livekit.md | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/proposals/4195-matrixrtc-livekit.md b/proposals/4195-matrixrtc-livekit.md index a21635241a8..dab261f8302 100644 --- a/proposals/4195-matrixrtc-livekit.md +++ b/proposals/4195-matrixrtc-livekit.md @@ -189,7 +189,8 @@ These endpoints MUST be implemented by servers supporting the `m.livekit` transp - `POST /_matrix/client/v1/rtc/livekit/get_token` - `POST /_matrix/federation/v1/rtc/livekit/get_token` -The server SHOULD apply rate limiting to both of these endpoints. +The server SHOULD apply rate limiting to both of these endpoints. Guest access to the endpoints is +allowed so that guest users can participate in MatrixRTC sessions using the LiveKit transport. To request a token, a client `POST`s to the `/get_token` client endpoint including in the body a JSON object with the following schema: From 6572c14eeb3004bc6802ea6ba7ee838774fc3090 Mon Sep 17 00:00:00 2001 From: Johannes Marbach Date: Tue, 8 Sep 2026 09:14:03 +0200 Subject: [PATCH 120/122] /delegate_delayed_leave is accessible to guests --- proposals/4195-matrixrtc-livekit.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/proposals/4195-matrixrtc-livekit.md b/proposals/4195-matrixrtc-livekit.md index dab261f8302..debf98c6050 100644 --- a/proposals/4195-matrixrtc-livekit.md +++ b/proposals/4195-matrixrtc-livekit.md @@ -477,7 +477,7 @@ sequenceDiagram deactivate H ``` -Clients delegate delayed leave events to their homeserver by `POST`ing to a new authenticated endpoint +Clients delegate delayed leave events to their homeserver by `POST`ing to a new authenticated, rate limited, and guest-accessible endpoint `/_matrix/client/v1/rtc/livekit/delegate_delayed_leave`. Servers supporting `m.livekit` MUST support this endpoint too. The body of the request contains a JSON object with the following schema: From 94ba07effc230de4d157be7f41a9c930e3034c2a Mon Sep 17 00:00:00 2001 From: Johannes Marbach Date: Tue, 8 Sep 2026 09:18:41 +0200 Subject: [PATCH 121/122] Use stable link --- proposals/4195-matrixrtc-livekit.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/proposals/4195-matrixrtc-livekit.md b/proposals/4195-matrixrtc-livekit.md index debf98c6050..3a469923246 100644 --- a/proposals/4195-matrixrtc-livekit.md +++ b/proposals/4195-matrixrtc-livekit.md @@ -729,7 +729,7 @@ send `m.rtc.member` events, a malicious server could fake the `user_id` when req token over federation. Given this and the limitations around denying side stepping `m.rtc.member` events, a power levels check wouldn't lead to a significant improvement. -[includes]: https://github.com/matrix-org/matrix-spec-proposals/blob/toger5/matrixRTC/proposals/4143-matrix-rtc.md#unmappable-rtc-streams +[includes]: https://github.com/matrix-org/matrix-spec-proposals/pull/4143/changes#diff-dc6525cff19aa3db827a41713f35cab1894558074be60455a55e1126a21a2b36R703 ### Error handling and information disclosure From 3cb83b1322be4fb699b144e560755d0b8e57fafe Mon Sep 17 00:00:00 2001 From: Johannes Marbach Date: Wed, 9 Sep 2026 14:19:08 +0200 Subject: [PATCH 122/122] Specify key length requirement --- proposals/4195-matrixrtc-livekit.md | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-) diff --git a/proposals/4195-matrixrtc-livekit.md b/proposals/4195-matrixrtc-livekit.md index 3a469923246..8eaf77f88bf 100644 --- a/proposals/4195-matrixrtc-livekit.md +++ b/proposals/4195-matrixrtc-livekit.md @@ -564,9 +564,10 @@ that is distributed to other clients via `m.rtc.encryption_key` to-device messag } ``` -To map this secret into LiveKit's frame-level [encryption] mechanism, clients use LiveKit's SDKs to implement -a [custom key provider]. The secret in `media_key.key` is then used as the raw byte input to LiveKit's HKDF-based -key derivation function, keyed by `media_key.index` and associated with the respective LiveKit participant identity +For the `m.livekit` transport a generic secret of at least 32 bytes is REQUIRED. To map this secret into +LiveKit's frame-level [encryption] mechanism, clients use LiveKit's SDKs to implement a [custom key provider]. +The full secret in `media_key.key` is then used as the raw byte input to LiveKit's HKDF-based key derivation +function, keyed by `media_key.index` and associated with the respective LiveKit participant identity derived as described [above]. Clients MUST use a keyring size of 256 when initialising the custom key provider to align with the [0, 255] range