Skip to content

Add outbound sync API#137

Open
marcus-pousette wants to merge 24 commits into
cybersemics:mainfrom
marcus-pousette:sync/app-facing-controller
Open

Add outbound sync API#137
marcus-pousette wants to merge 24 commits into
cybersemics:mainfrom
marcus-pousette:sync/app-facing-controller

Conversation

@marcus-pousette

@marcus-pousette marcus-pousette commented May 5, 2026

Copy link
Copy Markdown
Collaborator

Summary

Adds a small outbound sync helper for app code that needs to publish committed local ops through one SyncPeer that owns one or more outbound transports.

Scope of this PR: replace app-owned outbound local-op upload queues. It does not replace transport discovery, websocket status UI, auth setup, manual reconciliation, or inbound/live subscription filtering. Those pieces still live in the app or are handled by #138.

queueOps owns the committed-local-op path: it wakes live subscriptions on the localPeer, queues exact remote upload for outbound targets, dedupes pending TreeCRDT ops by meta.id, and handles offline/no-peer retry. attachTarget is the app-facing remote setup helper: it attaches the remote transport to the peer and registers it for outbound upload in one call. Custom op shapes may provide opKey to override the dedupe key. The helper does not expose peer filtering callbacks or fallback reconciliation hooks.

EM Integration Before

Source reference: the current Antonov548/em fork reports committed local TreeCRDT ops from pushQueue.ts, then handles pending local-op upload in treecrdtWebSocketSync.ts and pushTreecrdtLocalOpsToRemote.

Shortened, the current shape is: Redux/database code reports local TreeCRDT ops once.

const maybeOps = await db.updateThoughts(batchPayload);
if (batch.local && Array.isArray(maybeOps) && maybeOps.length > 0) {
  void pushTreecrdtLocalOpsToRemote(maybeOps as readonly Operation[]);
}

The awkward part is inside the sync module. It has to keep its own pending queue because local writes can happen before the websocket handle is ready, and upload failures need to preserve ops for retry:

let syncHandle: TreecrdtWebSocketSync | null = null;
let pendingLocalOps: Operation[] = [];

async function flushPendingLocalOps(): Promise<void> {
  if (!syncHandle || pendingLocalOps.length === 0) return;

  const ops = pendingLocalOps;
  pendingLocalOps = [];
  try {
    await syncHandle.pushLocalOps(ops);
  } catch (err) {
    pendingLocalOps = [...ops, ...pendingLocalOps];
    console.warn('TreeCRDT pushLocalOps failed', err);
  }
}

export async function pushTreecrdtLocalOpsToRemote(ops: readonly Operation[]): Promise<void> {
  if (ops.length === 0) return;
  if (!syncHandle) {
    if (getTreecrdtSyncBaseUrl()) pendingLocalOps.push(...ops);
    return;
  }
  try {
    await syncHandle.pushLocalOps(ops);
  } catch (err) {
    console.warn('TreeCRDT pushLocalOps failed', err);
  }
}

That makes the app sync module responsible for pending op storage, retry ordering, empty-batch policy, and the distinction between committed local ops and full reconciliation.

EM Integration After

With #137, an em-style sync module can create the outbound helper as soon as it has the local SyncPeer, before the websocket transport is connected. Local writes can then be reported immediately; if no outbound target is registered yet, the helper keeps the ops queued until attachTarget runs.

let outbound: OutboundSync<Operation> | null = null;

function startTreecrdtSync(backend: SyncBackend<Operation>) {
  const peer = new SyncPeer<Operation>(backend);

  outbound = createOutboundSync({
    localPeer: peer,
    isOnline: () => navigator.onLine,
    pushOptions: () => ({ maxOpsPerBatch: TREECRDT_SYNC_MAX_OPS_PER_BATCH }),
    pushTimeoutMs: () => TREECRDT_REMOTE_SYNC_TIMEOUT_MS,
    onError: ({ error }) => {
      console.warn('TreeCRDT pushLocalOps failed', error);
    },
  });

  const transport = connectWebSocketTransport();
  outbound.attachTarget('remote:sync-server', transport);

  registerBeforeTreecrdtClose(() => {
    outbound?.close();
  });
}

export function pushTreecrdtLocalOpsToRemote(ops: readonly Operation[]): void {
  outbound?.queueOps(ops);
}

The Redux/database call site stays simple. The sync module no longer needs a separate pendingLocalOps queue, a manual flushPendingLocalOps retry path, or app-provided op keys/op-ref derivation for standard TreeCRDT operations.

@marcus-pousette marcus-pousette changed the title Add app-facing sync controller Add outbound sync API May 5, 2026
@marcus-pousette marcus-pousette force-pushed the sync/app-facing-controller branch from d7c5bb6 to edcaec2 Compare May 5, 2026 12:07
@marcus-pousette marcus-pousette changed the base branch from runtime-storage-api-cross-tab to main May 5, 2026 12:07
@marcus-pousette marcus-pousette marked this pull request as ready for review May 5, 2026 12:07
@marcus-pousette marcus-pousette marked this pull request as draft June 1, 2026 20:46
@marcus-pousette marcus-pousette marked this pull request as ready for review June 2, 2026 19:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant