Encrypt m.new_content separately for message edits to aid clients which do not support edits

[bwindels]

Server-side aggregation of m.replace events should replace the original event's content with the m.new_content in the content of the replacing event. As it stands, riot on all platforms encrypts m.new_content as part of encrypting the whole content, so the server can't access it.

As a consequence, clients that don't support edits (and ignore m.new_content) would show the fallback content of the replacement when receiving the original message after server-side aggregation has happened. The original event would also look like a relation, although that can be safely ignored by unsupporting clients.

We need to encrypt m.new_content separately, so the server can pick it out to replace the original content with it, while preserving the rest of the original message.

The new format of an edit in an encrypted room (after encryption) would look like this:

"content": {
  	"algorithm": "m.megolm.v1.aes-sha2",
  	"ciphertext": "AwgDErABDBWSkLKJ5IQRePEP/kjpmVETrDzJp5jpWRJ0IIB/XAI0i6J8Cgpe/UEzhB9IpwJCMaWh7QKqN1h4kJn+nA55S/VgJEE01/g5Trihg5YGsaC5KGhEuH6gGG4tbtgX9hhwdN4PyTvmdSWxMtS8egKKBwS7Wux/461BX4UsSz25XqECIu6P8u7X/uBh5c6tTae6wTLVCvUTDtdYUnRk956GAbDnYgc27m8A16W8ex2BcgIE2MkzGJphZUm6PMa7vC9QOdTNACpylxtCrSEJhUWH8y4KseVXavVLi28lt4QhoRw8dLcId6LDJfexf+I8wMFr4HQ6DSd6iAg",
  	"device_id": "JHYBCAOWGF",
  	"m.relates_to": {
  		"event_id": "$5xJzoD0Cbz7vxIFcyPOCIJ2ORy4_MpHQUW4nIUdlmvg",
  		"rel_type": "m.replace"
  	},
  	"sender_key": "Ern8B3Dj56iXRvFeubfe5PDHp8bYWaHZbLiUcK1oAAc",
  	"session_id": "6/cfdpp8mMCEgbDJsmxAY66sBHlDeTfarVLeLu5VRzw",
  	"m.new_content": {
  		"algorithm": "m.megolm.v1.aes-sha2",
  		"ciphertext": "AwgDErABDBWSkLKJ5IQRePEP/kjpmVETrDzJp5jpWRJ0IIB/XAI0i6J8Cgpe/UEzhB9IpwJCMaWh7QKqN1h4kJn+nA55S/VgJEE01/g5Trihg5YGsaC5KGhEuH6gGG4tbtgX9hhwdN4PyTvmdSWxMtS8egKKBwS7Wux/461BX4UsSz25XqECIu6P8u7X/uBh5c6tTae6wTLVCvUTDtdYUnRk956GAbDnYgc27m8A16W8ex2BcgIE2MkzGJphZUm6PMa7vC9QOdTNACpylxtCrSEJhUWH8y4KseVXavVLi28lt4QhoRw8dLcId6LDJfexf+I8wMFr4HQ6DSd6iAg",
  		"device_id": "JHYBCAOWGF",
  		"sender_key": "Ern8B3Dj56iXRvFeubfe5PDHp8bYWaHZbLiUcK1oAAc",
  		"session_id": "6/cfdpp8mMCEgbDJsmxAY66sBHlDeTfarVLeLu5VRzw"
  	}

Also see discussion on matrix.

[bwindels]

Unsure about priority, the only downside atm is that non-supporting clients will show the fallback content, even after server-side aggregation ... Which doesn't seem like a disaster to me, but would be good not to change the format after taking the feature out of labs.

[bwindels]

Unsure if this needs any changes in synapse, e.g. whether or not it's blocked by backend.

[richvdh]

I spent a while being confused by this, but I think I understand it now. For the record:

It's referring to this part of the MSC:

Whenever an m.replace is to be bundled with an event as above, the server should also modify the content of the original event according to the m.new_content of the most recent edit (determined as above).

and this:

Clients can often ignore message edit events, since any events the server returns via the C-S API will be updated by the server to account for subsequent edits.

However, clients should apply the replacement themselves when the server is unable to do so. This happens in the following situations:
...
2. The original event (and hence its replacement) are encrypted.

So, a client that supports encryption but not message edits will show the original message in the original place in the timeline, for encrypted events (but not for unencrypted events).

This was also discussed in this thread on the MSC, but by the time we came to spec it, it was already in production, so we had to go with what we had.

Obviously, changing this now would require an MSC, and given the lack of complaints about it, it's hard to see it as a priority. I guess there aren't many clients in the world that support encryption but not edits.

If such clients exist, and want to see this changed, I'd expect to track it as a matrix-spec issue in the first instance. Accordingly, I'm going to close it.