BIP40: Stratum wire protocol

[ben221199]

The Stratum wire protocol has a long history since @slush0 introduced it. However, the protocol never got standardized in a formal way, so many implementations have been based on incomplete documents or on other implementations. With this document I finally want to give Stratum its place between the other BIPs, so that developers can just read this document and don't have to search through years of code or dead pages that need to be revived with Wayback Machine.

[ben221199]

Yesterday, I sent two registration requests to IANA to get some information officially egistered. The registries are:

• Media Types (https://www.iana.org/assignments/media-types/media-types.xhtml)
• WebSocket Subprotocol Name Registry (https://www.iana.org/assignments/websocket/websocket.xml#subprotocol-name)

Today, I got a response that for both registrations some things have to be done first, before IANA can add these records to their registry. In case of the Media Types, this is process is a little bit more difficult. For the WebSocket Subprotocol Name Registry it is just letting them know when this pull request is merged and I have a permalink to the specification.

Also, it is possible that I will send more registration requests for other IANA registries too. For example, the Service Name and Transport Protocol Port Number Registry (https://www.iana.org/assignments/service-names-port-numbers/service-names-port-numbers.xhtml) is a possible candidate. In that case, I will post new comments about that too.

[jonatack]

Some editorial feedback.

[jonatack]

This BIP refers to BIP41, The Stratum mining protocol, for which a BIP number was assigned and an entry exists in the README, but there doesn't seem to be pull request to add the BIP draft -- any update on that?

[ben221199]

Hi @jonatack, thank you for feedback. I will take a look at it. I didn't have time to work on BIP 40 lately. Yes, both BIP 40 and BIP 41 are already assigned by the README and BIP 41 is also mentioned in this BIP 40. The goal is to finish BIP 40 (Stratum wire protocol) first and then start working on BIP 41 (Stratum mining protocol), referring back to BIP 40, because the mining protocol is based on the wire protocol.

[ben221199]

The media type application/stratum has been registered by IANA at Media Types (https://www.iana.org/assignments/media-types/media-types.xhtml).

[ben221199]

They will be in seperate BIPs and I will make it clear they are not the same. As far as I know, the mining protocol only implements the mining service.

[ben221199]

Hi @murchandamus,

Any updates on the merge progress?

[murchandamus]

Hi Ben, thank you for your patience and for your grit. I read the BIP text today, for someone that has little familiarity with the Stratum Wire Protocol it reads fine, but I cannot provide a lot of detailed feedback on the specification.

I did quickly read over the descriptions of the Services. Thanks for all that work! I did notice a couple potential inconsistencies, but the submission would surely benefit from someone more familiar with the subject matter reviewing it.

It has all the sections we expect in BIPs. The Rationale is a bit short for such a long specification, but that’s easily understandable given the circumstances.

This appears to be close to mergeable as Draft, I’d just like to see the couple editorial feedback items addressed. Please let us know if you expect another review or further changes when you address the review, as to clarify whether it is ready to be merged from your end.

[ben221199]

I addressed all feedback. There remains one that cannot be resolved yet, but you will talk with the other BIP Editors about it. If it were up to me, it could be merged.

[luke-jr]

IIRC, BIP 40 was meant to be the "wire protocol", meaning the JSON/TCP frameworks, which BIP 41 (Stratum v1 mining) was based on. But what you've written here deals largely with the Stratum wallet(?) protocol that would also be using BIP 40's wire protocol, nowadays known as the Electrum protocol.

[ben221199]

I don't think that is a big problem, because Stratum was developed for Electrum. Also, I still wrote the "wire protocol", because everything is about the wire protocol in this document, except for Specification: Services. Well, in that case I guess you should see the service files just as completion of the whole project. Alternatively, we can move these service files to Sathoshi Labs SLIP-40, but I think you make things much more difficult then.

[murchandamus]

@luke-jr is your comment a blocking issue? If so, do you have a recommendation what @ben221199 should do to address it?

@jonatack: Did you want to give this another pass?

It seems fine to me from an editorial standpoint, but I’m no expert on Stratum and was not involved in the discussion back then.

[luke-jr]

I think my suggestion is to assign a new BIP number for the wallet interface. Ideally, breaking it up to populate BIP 40 with the actual wire protocol at the same time.

[ben221199]

I think my suggestion is to assign a new BIP number for the wallet interface. Ideally, breaking it up to populate BIP 40 with the actual wire protocol at the same time.

I understand your suggestion and the reason behind it, but I don't fully agree with it. Partly because I think it would be more confusing to have even more BIPs, but also because the history of Stratum: both the wire protocol and the commands were developed in some symbiosis. I think it should just stay in BIP 40, but if it MUST be seperated, I guess SLIP-40 would make more sense than a new BIP.

[ben221199]

🙃

[ben221199]

Can we still get this merged this year? 🙂

[jonatack]

@jonatack: Did you want to give this another pass?

Yes indeed, sorry for not seeing your ping.

[Sjors]

Partly because I think it would be more confusing to have even more BIPs, but also because the history of Stratum: both the wire protocol and the commands were developed in some symbiosis.

Let's say I wanted to vibe code an electrum wallet or an electrum server from spec(s). Which documents do I need?

Same question for if I wanted to build a Stratum v1 ASIC, or a new pool software.

The answers to those four questions could guide how to organize the BIPs. Ideally each implementer doesn't have to waste time on parts of the protocol(s) they don't need. History can be explained in footnotes.

I would also be inclined to rename the wallet version to "electrum wire protocol" just to reduce confusion - even if it's not historically accurate, but maybe that's pushing too far :-)

(and then keep BIP 40 for the mining bits)

Create a BIP for Stratum V2.

At this point I don't think that's necessary. The spec can be found at https://github.com/stratum-mining/sv2-spec (rendered on https://stratumprotocol.org/specification) and it's still undergoing changes. It's more of living document than a typical BIP.

I noticed this document uses the term "Stratum Reference Implementation" in a few places, which is a term used in Stratum v2 land too (SRI): https://github.com/stratum-mining/sv2-apps

Would be good to clarify the distinction.

[ben221199]

Can I ask what is keeping this PR from being merged? @murchandamus @jonatack

[jonatack]

Looks like the CI is seeing a minor issue and it would be good to address the review feedback in #1557 (comment). I started on reviewing this some time ago and didn't finish, will do that this week.

[murchandamus]

Hi @ben221199, as indicated in September, I’m fine with merging this from an editorial standpoint, but several reviewers indicated that the scope of your proposal didn’t match their expectation for the BIP40 placeholder. What do you think about this proposal being assigned a different number, given that people balk at it being referred to as BIP40? You could then perhaps add a “Replaces: 40” header and expand the explanation regarding its relationship to BIP40 at the top of the document.

Sorry for not chiming in again since then. I’d be happy to publish it with a different number around the end of the month, if there are no further developments here otherwise.

[ben221199]

@murchandamus

To address the issue about the BIP number... The BIPs 40 and 41 were historically assigned Stratum wire protocol and Stratum mining protocol respectively. The first one (BIP 40) to define the whole protocol between SPV clients and servers, and the second one (BIP 41) to take this same protocol and implement it with some different mining-specific commands. I would rather start a discussion on dropping the wire word from the BIP 40 name than talking about a new BIP number, because I don't see how that solves anything.

[ben221199]

But I have to say, BIP 40 contains the wire protocol after all, so I don't see a problem with keeping it.

[Sjors]

Concept NACK, unless someone can explain how this BIP provides practical benefit for anyone. I suggested above in #1557 (comment) how that could be done. There's no point in just filling up historical BIP numbers.

[ben221199]

I don't know what you mean. The BIP number 40 was assigned for the document that would describe the whole Stratum protocol, so that is what I did: I wrote that document.

[Sjors]

This number was assigned almost 13 years ago in e12d37e. At the time it was considered useful, but since then the wallet and mining protocol have drifted apart. If no mining or wallet implementer can benefit from a BIP, then it just becomes an exercise in archeology.

Maybe we should unallocate 40, or re-allocate it to the Electrum protocol. For what overlap remains between these two protocols, one can always refer to specific paragraphs in the other.

[ben221199]

At the time it was considered useful, but since then the wallet and mining protocol have drifted apart.

I don't know what you mean by this, because it has never been the same thing. The mining variant (and I don't mean V2 with this) reused many things, yes, but that was all.

If no mining or wallet implementer can benefit from a BIP, then it just becomes an exercise in archeology.

There are still many tools around that would benefit from a BIP. I have seen what can happen (e.g. LBRY Hub implementation) when you don't have a BIP and just program something that looks like it.

Maybe we should unallocate 40, or re-allocate it to the Electrum protocol. For what overlap remains between these two protocols, one can always refer to specific paragraphs in the other.

My plan was to write BIP 40, in which the wire protocol and some services are described (as assigned). I have written that document in the past 2 years. That work is actually done. As soon as this merge is through, I can and will start working on BIP 41, which will only describe the mining variant (as assigned), and will make many references to BIP 40.

As far as I can read, you only seem to disagree on the history section and maybe the name of this BIP, and you seem to have some confusion about BIP 40, BIP 41 and mining variants. To keep the discussion clear, please keep Stratum V2 out of the discussion. I know it exists, but I am not (yet) interested in it. To clarify about the BIPS:

• BIP 40 = Stratum wire protocol (without mining methods; already written)
• BIP 41 = Stratum mining protocol (references to BIP40 when it reuses things; yet to be written)

[Sjors]

Maybe it's useful if you open a draft BIP41, and then demonstrate that the text is necessary and sufficient for someone to implement e.g. a pool or mining client from scratch.

[murchandamus]

Firstly, thanks to @ben221199, for the documentation effort here. I would like to find a way to publication for this work. After rereading the comments on this PR and rereading the proposal itself, I must admit that I still feel a bit out of my element.

It sounds to me that the main concern of reviewers’ is that Electrum Stratum and Mining Stratum have diverged. My understanding is that it were preferred if the proposed BIP40 document were narrowed to the application-agnostic wire protocol (transport layer) part, and the specification of the services were split off to one or more new companion BIPs. (Please correct me, if I misconstrued that!) Hopefully pruning copies of this document down to the relevant separated parts would not be too much work, and @ben221199 could get additional BIPs out of such an approach. Alternatively, it still seems plausible to me to assign a new number and that the preamble of this document declare that this document replaces BIP40.

I would not want this work to be discarded or abandoned. Especially, progress toward a common specification of the Electrum wallet/server protocol sounds useful to me. I’d be happy to see any alternative constructive suggestions that would bring us closer to publication.