MSC4195: LiveKit transport for MatrixRTC

[hughns]

Rendered

---

SCT Stuff:

FCP tickyboxes

MSC checklist

---

Server

• m.livekit in /transports: Add support for specifying the SFU WebSocket URL together with configured LiveKit transports element-hq/synapse#20146

Note

For backwards compatibility, the endpoint also returns livekit_service_url. The stems from a previous iteration of the proposal and has been fully replaced by url in the current text.

• Client-Server & Server-Server endpoints: https://github.com/element-hq/lk-jwt-service implements the endpoints introduced by this proposal. People interested in reviewing the implementation should start in https://github.com/element-hq/lk-jwt-service/blob/main/src/handler.rs#L1245. The service is integrated into https://github.com/element-hq/synapse/ as an application service by means of:
  • Add experimental support for letting application services proxy namespaces in the C-S and S-S API as per MSC4512 element-hq/synapse#19972
  • Add experimental support for sending federation requests from app services as per MSC4512 element-hq/synapse#19977
  • Add experimental support for MSC4502: Targeted and unrestricted room member queries element-hq/synapse#19974.
• The three pull requests above depend on MSC4512 and MSC4502. This is an implementation detail of the Synapse implementation, however. There is no dependency between the current proposal and these two MSCs.

Note

For backwards compatibility with implementations of previous versions of this MSC, https://github.com/element-hq/lk-jwt-service exhibits the following differences with respect to the proposal text:

• The legacy endpoints /sfu/get, /get_token and /delegate_delayed_leave are still included. These have been replaced by proper Client-Server and Server-Server endpoints in the proposal which are also implemented in https://github.com/element-hq/lk-jwt-service.
• The device ID is still included in the LiveKit participant identity derivation. On requests, the ID is supplied as claimed_device_id wrapped together with the member ID in a member object. The proposal has eliminated claimed_device_id and replaced the member object with member_id.
• The delay_timeout request parameter is still included in the /delegate_delayed_leave endpoint. This parameter has been removed from the proposal because it can be inferred from the delayed event ID.
• Extended tokens do not contain the grant to create LiveKit rooms. Instead the service creates rooms itself (using its own token). This is because client implementations are not yet updated to perform the room creation themselves.

Furthermore, https://github.com/element-hq/lk-jwt-service doesn't currently implement kicking participants from the SFU on room leaves.

Client

• LiveKit participant identity derivation: Adapt LiveKit Identity hash calculation to latest MSC4195 update matrix-js-sdk#5268
• Initial multi-SFU implementation: A new world: "Multi-SFU" media transport element-hq/element-call#3512
• Taking the LiveKit room name from the JWT access token: The advertised livekit_alias in membership is deprecated element-hq/element-call#3706
• Switch to Client-Server endpoints for token requests: Add livekit rtc cs api endpoints to client (and embedded client) matrix-js-sdk#5496
• Delegation of delayed leave events: Delegate delayed events using dedicated endpoint element-hq/element-call#4209

Note

Similar to what's mentioned above, the client implementations also still support the legacy livekit_service_url property in transports objects in membership events.

[turt2live]

Overall this looks good to me at a high level. I haven't reviewed the LiveKit components in exhaustive detail - I'm more focusing on checklist/process requirements.

[turt2live]

MSCs proposed for Final Comment Period (FCP) should meet the requirements outlined in the checklist prior to being accepted into the spec. This checklist is a bit long, but aims to reduce the number of follow-on MSCs after a feature lands.

SCT members: please check off things you check for, and raise a concern against FCP if the checklist is incomplete. If an item doesn't apply, prefer to check it rather than remove it. Unchecking items is encouraged where applicable.

MSC authors: feel free to ask in a thread on your MSC or in the#matrix-spec:matrix.org room for clarification of any of these points.

• Are appropriate implementation(s) specified in the MSC’s PR description?
• Are all MSCs that this MSC depends on already accepted?
• For each new endpoint that is introduced:
  • Have authentication requirements been specified?
  • Have rate-limiting requirements been specified?
  • Have guest access requirements been specified?
  • Are error responses specified?
    • Does each error case have a specified errcode (e.g. M_FORBIDDEN) and HTTP status code?
      • If a new errcode is introduced, is it clear that it is new?
• Will the MSC require a new room version, and if so, has that been made clear?
  • Is the reason for a new room version clearly stated? For example, modifying the set of redacted fields changes how event IDs are calculated, thus requiring a new room version.
• Are backwards-compatibility concerns appropriately addressed?
• Are the endpoint conventions honoured?
  • Do HTTP endpoints use_underscores_like_this?
  • Will the endpoint return unbounded data? If so, has pagination been considered?
  • If the endpoint utilises pagination, is it consistent with the appendices?
• An introduction exists and clearly outlines the problem being solved. Ideally, the first paragraph should be understandable by a non-technical audience.
• All outstanding threads are resolved
  • All feedback is incorporated into the proposal text itself, either as a fix or noted as an alternative
• While the exact sections do not need to be present, the details implied by the proposal template are covered. Namely:
  • Introduction
  • Proposal text
  • Potential issues
  • Alternatives
  • Dependencies
• Stable identifiers are used throughout the proposal, except for the unstable prefix section
  • Unstable prefixes consider the awkward accepted-but-not-merged state
  • Chosen unstable prefixes do not pollute any global namespace (use “org.matrix.mscXXXX”, not “org.matrix”).
• Changes have applicable Sign Off from all authors/editors/contributors
• There is a dedicated "Security Considerations" section which detail any possible attacks/vulnerabilities this proposal may introduce, even if this is "None.". See RFC3552 for things to think about, but in particular pay attention to the OWASP Top Ten.

[turt2live]

@mscbot fcp merge
@mscbot concern Dependency - MatrixRTC (concurrent FCP permitted)
@mscbot concern Editorial - Links to branch names are prone to breaking
@mscbot concern Checklist - Authentication, rate-limiting, and guest accessibility requirements need clarifying on endpoints
@mscbot concern Transport explicitly needs to be optional (add MAY), but endpoints MUST be implemented with transport support

[mscbot]

Team member @mscbot has proposed to merge this. The next step is review by the rest of the tagged people:

• @clokep
• @dbkr
• @uhoreg
• @turt2live
• @ara4n
• @anoadragon453
• @richvdh
• @tulir
• @erikjohnston
• @KitsuneRal

Concerns:

• Dependency - MatrixRTC (concurrent FCP permitted)
• Editorial - Links to branch names are prone to breaking
• Checklist - Authentication, rate-limiting, and guest accessibility requirements need clarifying on endpoints
• Transport explicitly needs to be optional (add MAY), but endpoints MUST be implemented with transport support

Once at least 75% of reviewers approve (and there are no outstanding concerns), this will enter its final comment period. If you spot a major issue that hasn't been raised at any point in this process, please speak up!

See this document for information about what commands tagged team members can give me.