-
Notifications
You must be signed in to change notification settings - Fork 319
blog: Yjs durable streams on Electric Cloud #4059
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Changes from 24 commits
0f8100f
2ffd824
d143c47
3bf15c6
b09a1ce
32b18af
83b09bf
1f5afa6
cc1cae2
fee16a0
397d463
153a28e
de80976
c84835f
db7ee45
cebc5b9
ca5ff5d
dbb210b
6650613
9269f37
8dec0a3
75ff484
05e9fa9
7f3fd68
2dde8a8
9c3049f
eab02b9
952b75f
d524a14
d042259
a000e20
d3fd29a
048f1e7
10d0fc3
498b548
b523b79
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,100 @@ | ||
| --- | ||
| title: 'Yjs CRDTs over HTTP on Durable Streams' | ||
| description: >- | ||
| y-durable-streams is a new open-protocol Yjs provider with built-in compaction, presence, and CDN caching built on Durable Streams. | ||
| excerpt: >- | ||
| Sync Yjs documents over plain HTTP. y-durable-streams brings built-in compaction, real-time presence, and fan-out via CDN to collaborative apps and agentic systems. | ||
| authors: [balegas] | ||
| image: /img/blog/yjs-durable-streams-on-electric-cloud/header.png | ||
|
||
| tags: [durable-streams, cloud, release, sync, collaboration] | ||
| outline: [2, 3] | ||
| post: true | ||
| published: true | ||
| --- | ||
|
|
||
| [Yjs](https://yjs.dev) is the de facto library for collaborative editing on the web. It's battle-proven, CRDT-based, and powers tools like [TipTap](https://tiptap.dev), [CodeMirror](https://codemirror.net), [BlockNote](https://www.blocknotejs.org/) and more. And it's not just collaboration between humans anymore — agents are increasingly editing documents, generating code, and filling in forms alongside users. Whether it's humans or agents collaborating, they need reliable, conflict-free sync. | ||
|
||
|
|
||
| >[!info] 🚀 Try it now | ||
| >Sign up to [Electric Cloud](https://dashboard.electric-sql.com), create a Yjs service, and connect your app. | ||
|
||
| >See the [`y-durable-streams` source](https://github.com/durable-streams/durable-streams/tree/main/packages/y-durable-streams) and [demo app](https://github.com/durable-streams/durable-streams/tree/main/examples/yjs-demo) on GitHub. | ||
|
|
||
| ## The problem with WebSockets | ||
|
||
|
|
||
| Most Yjs setups are built on WebSockets to relay updates to clients. WebSockets are point-to-point connections with no fan-out distribution. They require sticky connections and content can't be cached at a CDN, which means there is a latency penalty for every user or agent that needs to retrieve the initial state of a document. | ||
|
|
||
| There is no standardized reference implementation if you want to implement this yourself. There are hosted services you can buy, but that means vendor lock-in and a new piece of infrastructure to add to your stack. | ||
|
||
|
|
||
| ## Yjs on bare HTTP | ||
|
||
|
|
||
| We've built [`y-durable-streams`](https://github.com/durable-streams/durable-streams/tree/main/packages/y-durable-streams) — a new Yjs provider on [Durable Streams](/primitives/durable-streams), an open HTTP protocol for persistent, resumable, real-time streams. | ||
|
||
|
|
||
| Instead of WebSocket relay servers, document updates flow through plain HTTP. Clients POST edits and subscribe for real-time updates via SSE or long-polling — no persistent connections, no sticky sessions. Because it's standard HTTP, it works with the infrastructure you already have: load balancers, reverse proxies, CDNs. Snapshots are cacheable at the edge, so fan-out scales without extra effort. | ||
|
||
|
|
||
| The provider handles document sync, server-side compaction, and awareness out of the box. It's open source, backed by an [open protocol specification](https://github.com/durable-streams/durable-streams/blob/main/packages/y-durable-streams/YJS-PROTOCOL.md), and now live on [Electric Cloud](/cloud). | ||
|
|
||
| ## How it works | ||
|
|
||
| Each document is backed by a durable stream — a persistent, append-only log. Every edit is appended to the stream, and all connected clients receive it in real time. When a new client joins, it doesn't replay the full history. Instead, it loads a **snapshot** — a compacted version of the document at a point in time — and subscribes for live updates from there. As documents grow, the server automatically compacts accumulated updates into new snapshots, so initial sync stays fast. | ||
|
|
||
| Presence — cursors, selections, user info — is handled through **awareness streams**. These are ephemeral: clients only receive the latest changes, and each stream has a built-in TTL that garbage-collects stale state from disconnected clients. A default awareness stream is created with every document, and you can create additional named ones for different purposes. | ||
|
|
||
| For the full details, see the [Yjs Durable Streams Protocol](https://github.com/durable-streams/durable-streams/blob/main/packages/y-durable-streams/YJS-PROTOCOL.md) specification. | ||
|
|
||
| ## Demo | ||
|
||
|
|
||
| A demo of `y-durable-streams` live on Electric Cloud. Game state is stored in a Yjs Y.Map — each grid cell is a CRDT entry with last-writer-wins conflict resolution. Player presence uses awareness streams. Open it in multiple tabs, share the room link, or try it on your phone. | ||
|
|
||
| <iframe src="/demos/territory-wars/index.html" style="width:100%;height:500px;border:none" sandbox="allow-same-origin allow-scripts allow-popups" allow="clipboard-write"></iframe> | ||
|
|
||
| ## Get started | ||
|
|
||
| Here's how to set up a collaborative text editor. Create a Yjs document with awareness and point it at your endpoint: | ||
|
|
||
| ```typescript | ||
| import { YjsProvider } from '@durable-streams/y-durable-streams' | ||
| import { Awareness } from 'y-protocols/awareness' | ||
| import * as Y from 'yjs' | ||
|
|
||
| const doc = new Y.Doc() | ||
| const awareness = new Awareness(doc) | ||
|
|
||
| const provider = new YjsProvider({ | ||
| doc, | ||
| awareness, | ||
| baseUrl: 'https://api.electric-sql.cloud/v1/stream/svc-your-service', | ||
| docId: 'my-document', | ||
| }) | ||
| ``` | ||
|
|
||
| Then wire it into your editor. Here's an example with TipTap: | ||
|
|
||
| ```typescript | ||
| import { useEditor, EditorContent } from '@tiptap/react' | ||
| import StarterKit from '@tiptap/starter-kit' | ||
| import Collaboration from '@tiptap/extension-collaboration' | ||
| import CollaborationCursor from '@tiptap/extension-collaboration-cursor' | ||
|
|
||
| const editor = useEditor({ | ||
| extensions: [ | ||
| StarterKit.configure({ history: false }), | ||
| Collaboration.configure({ document: doc }), | ||
| CollaborationCursor.configure({ provider }), | ||
| ], | ||
| }) | ||
| ``` | ||
|
|
||
| The provider handles sync, compaction, and awareness. Cursors, selections, and user presence work out of the box — every client that connects to the same `docId` sees the same document, in real time. | ||
|
|
||
| Clone the [demo app](https://github.com/durable-streams/durable-streams/tree/main/examples/yjs-demo) to see a working example, or drop the provider into your existing Yjs project. | ||
|
|
||
| ## No lock-in | ||
|
|
||
| The Yjs Durable Streams protocol is [fully documented](https://github.com/durable-streams/durable-streams/blob/main/packages/y-durable-streams/YJS-PROTOCOL.md) with a conformance test suite — you can self-host, switch providers, or build your own compatible server at any time. | ||
|
|
||
| Electric Cloud implements the documented protocol strictly — no proprietary extensions, no vendor-specific APIs. It's just a faster way to get started. | ||
|
|
||
| ## Next steps | ||
|
|
||
| - sign up to [Electric Cloud](https://dashboard.electric-sql.com) | ||
|
||
| - check the [example app](https://github.com/durable-streams/durable-streams/tree/main/examples/yjs-demo) | ||
| - any questions, let us know in [Discord](https://discord.electric-sql.com) | ||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,162 @@ | ||
| --- | ||
| title: '...' | ||
| description: >- | ||
| ... | ||
| excerpt: >- | ||
| ... | ||
| authors: [balegas] | ||
| image: /img/blog/yjs-durable-streams-on-electric-cloud/header.jpg | ||
| tags: [durable-streams, cloud, release, sync, collaboration] | ||
| outline: [2, 3] | ||
| post: true | ||
| published: false | ||
| --- | ||
|
|
||
| <!-- STRUCTURAL NOTE: This is a release post. Lead with what shipped and why | ||
| it matters. Factual, brisk, no preamble. Let the content do the work. --> | ||
|
|
||
| Yjs durable streams are now available on Electric Cloud. Real-time | ||
| collaborative editing as a managed service — sync Yjs documents over HTTP | ||
| instead of WebSockets. | ||
|
|
||
| Open protocol. Sub-50ms latency. Runs on the edge. Built-in compaction. No | ||
| infrastructure to manage. Create a service, point your Yjs app at it. | ||
|
|
||
| <!-- TONE: Compressed, confident. This paragraph IS the pitch. | ||
| Expand each line slightly into prose but keep it tight. --> | ||
|
|
||
| :::info | ||
| - [Create a Yjs service on Electric Cloud](https://github.com/durable-streams/durable-streams/tree/main/examples/yjs-demo) | ||
| - [`y-durable-streams` on GitHub](https://github.com/durable-streams/durable-streams/tree/main/packages/y-durable-streams) | ||
| - [Demo app](https://github.com/durable-streams/durable-streams/tree/main/examples/yjs-demo) | ||
| ::: | ||
|
|
||
| ## Context | ||
|
|
||
| <!-- STRUCTURAL NOTE: Brief orientation only. 2-3 bullets. Not a backstory — | ||
| just enough for readers who haven't been following durable streams. --> | ||
|
|
||
| - Durable Streams is an open HTTP protocol for persistent, resumable, | ||
| real-time streams. Append-only logs with their own URL that clients can | ||
| read from any position | ||
| - `y-durable-streams` is a Yjs provider that replaces WebSocket-based sync | ||
| with plain HTTP — works with standard load balancers and CDNs, no sticky | ||
| sessions | ||
| - Electric Cloud has been running durable streams services since January. | ||
| Yjs is the latest service type | ||
|
|
||
| ## What's shipping | ||
|
|
||
| <!-- STRUCTURAL NOTE: Concrete capabilities. Lead with benefits, | ||
| follow with specifics. Each bullet = something the reader can now do. --> | ||
|
|
||
| - Managed Yjs sync on Electric Cloud — one click to create a service, get | ||
| an endpoint, connect your Yjs app | ||
| - Sub-50ms latency, edge-deployed, scales without you thinking about it | ||
| - Built-in server-side compaction — accumulated updates get merged into | ||
| snapshots automatically, initial loads stay fast as documents grow | ||
| - Awareness and presence out of the box — cursors, selections, user status | ||
| over the same HTTP transport | ||
| - Open protocol, no vendor lock-in — `y-durable-streams` works against any | ||
| durable streams server, self-hosted or cloud | ||
| - Just HTTP — no WebSocket servers, no sticky sessions, CDN-friendly | ||
|
|
||
| <!-- ASSET: demo video or GIF of collaborative editor with multiple cursors | ||
| and presence indicators --> | ||
|
|
||
| ## Get started | ||
|
|
||
| <!-- STRUCTURAL NOTE: Show don't tell. The reader should be able to try it | ||
| from this section. One-click setup — keep it minimal. --> | ||
|
|
||
| <!-- AUTHOR NOTE: Include the direct link to create a Yjs service on Cloud. | ||
| This is a single click — don't over-explain the process. --> | ||
|
|
||
| - Link to create a Yjs durable streams service on Electric Cloud | ||
| - Clone the demo app and point it at your cloud endpoint: | ||
|
|
||
| ```typescript | ||
| import { YjsProvider } from '@durable-streams/y-durable-streams' | ||
| import * as Y from 'yjs' | ||
|
|
||
| const doc = new Y.Doc() | ||
| const provider = new YjsProvider({ | ||
| doc, | ||
| baseUrl: 'https://your-service.electric-sql.cloud/v1/yjs', | ||
| docId: 'my-document', | ||
| }) | ||
| ``` | ||
|
|
||
| <!-- AUTHOR NOTE: Confirm the exact baseUrl pattern for cloud endpoints. | ||
| Mention it works with any Yjs editor binding — CodeMirror, ProseMirror, | ||
| TipTap, BlockNote, etc. --> | ||
|
|
||
| *** | ||
|
|
||
| Next steps: | ||
|
|
||
| - [Sign up for Electric Cloud](https://dashboard.electric-sql.cloud/) | ||
| - [Try the demo](https://github.com/durable-streams/durable-streams/tree/main/examples/yjs-demo) | ||
| - [Join Discord](https://discord.electric-sql.com) | ||
|
|
||
| *** | ||
|
|
||
| <!-- DELETE EVERYTHING BELOW THIS LINE BEFORE PUBLISHING --> | ||
|
|
||
| <!-- ## Meta | ||
|
|
||
| ### Intent | ||
|
|
||
| - **What:** Yjs durable streams now available as a managed service on Electric Cloud | ||
| - **Hook:** Open protocol, sub-50ms latency, runs on the edge, built-in compaction, just HTTP — no WebSocket infrastructure | ||
| - **Takeaway:** Works with your existing stack, no vendor lock-in, cheap to run and scale | ||
| - **CTAs:** Sign up for Cloud, try the demo | ||
| - **Authority:** Electric team, experts in sync, background in CRDTs | ||
|
|
||
| ### Title brief | ||
|
|
||
| Direction: Something direct and specific. "Yjs durable streams on Electric Cloud" | ||
| or "Real-time collaboration with Yjs on Electric Cloud". Sentence case, not Title | ||
| Case. Keep it short. | ||
|
|
||
| ### Description brief | ||
|
|
||
| For SEO. Convey: managed Yjs sync service on Electric Cloud, HTTP-based (not | ||
| WebSocket), sub-50ms latency, open protocol, no vendor lock-in. Target developers | ||
| searching for Yjs hosting, Yjs sync, collaborative editing infrastructure. | ||
|
|
||
| ### Excerpt brief | ||
|
|
||
| For the blog listing card. Max 3 short sentences. What shipped (Yjs durable | ||
| streams on Cloud), why it's interesting (HTTP, fast, open protocol), try it now. | ||
| Match word length of existing post excerpts. | ||
|
|
||
| ### Image prompt | ||
|
|
||
| Abstract visualization of collaborative editing — multiple cursors or document | ||
| nodes connected over a network. Dark theme background. Electric brand colors: | ||
| purple (#D0BCFF), green (#00d2a0), cyan (#75fbfd). Center-center composition. | ||
| 16:9 aspect ratio, ~1536x950px. High-quality JPG. Use /blog-image-brief for | ||
| a detailed prompt with reference analysis. | ||
|
|
||
| ### Asset checklist | ||
|
|
||
| - [ ] Demo video or GIF: collaborative editor with multiple cursors and presence | ||
| - [ ] Code snippet: confirm exact cloud baseUrl pattern for YjsProvider | ||
| - [ ] Info box links: cloud dashboard URL, create-service link, demo repo link | ||
|
|
||
| ### Typesetting checklist | ||
|
|
||
| - [ ] Use non-breaking spaces where appropriate to avoid widows and orphans | ||
| - [ ] Title uses sentence case, not Title Case | ||
| - [ ] Check title, image, and general post at different screen widths | ||
| - [ ] No LLM tells: "it's worth noting", "importantly", "in conclusion", | ||
| "let's dive in", "at its core", "in today's landscape" | ||
|
|
||
| ### Open questions | ||
|
|
||
| - Exact URL pattern for cloud Yjs service endpoints | ||
| - Performance numbers beyond sub-50ms to include? | ||
| - Specific "coming next" items if any are decided later | ||
|
|
||
| --> |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Tweak the filename date to 31st March. N.b.: fixing any links referring to it.