Sync / foldkit-sync

Advanced: the replica

Sync.mount is the normal path. Underneath, a Replica works on the shared slice alone, and an application that is not a Foldkit runtime (a worker, a server-side actor, a test) uses it directly.

ts
const program = Effect.gen(function* () {
  const storage = yield* Sync.indexedDb('todos/tab-1')
  const replica = yield* TodoSync.openReplica(ReplicaId.make('tab-1'), storage)

  yield* replica.submit(Message.CreatedTodo({ id: 't1', title: 'Milk' }))
  const optimistic = yield* replica.shared // includes the pending edit

  yield* replica.synchronize // one exchange
  return optimistic
}).pipe(Effect.scoped, Effect.provide(Sync.transport.socket({ url: 'wss://example.com/sync' })))

submit(message) checks that the Message is durable, replays it against the current optimistic state, gives it a stable operation id, saves it to the outbox, publishes the new optimistic value, and wakes the loop. It sends nothing. A Message replay refuses never enters the outbox.

synchronize takes one snapshot of the cursor and outbox, performs one exchange, validates the answer, and reconciles it with the replica's current state, so an edit submitted while the request was in flight survives.

The rest of the surface: committed (the base alone), pending, cursor, status, statusChanges, changes (status and shared from one snapshot), snapshot (the same plus nextLocalSequence), start, and close.

The exchange

One request, one answer. A custom server implements this; a custom transport carries it.

request    cursor
           pending operations
           epoch, once the replica has heard one

answer     operations     committed after the cursor, contiguous
           acknowledged   ids the server accepted without repeating them
           rejected       ids the server refused
           reasons?       [{ opId, reason }]: why, for some of rejected; at most 500 characters each
           checkpoint?    { cursor, model } when the history is gone
           more?          true when another page follows
           epoch?         the server's history

An acknowledged operation leaves the outbox once its committed copy has arrived or the last page has been read. A rejected one leaves at once. The replica validates every field; see when things go wrong.

Transports

Transport is an Effect service with one operation. Sync ships:

  • Sync.transport.socket({ url, … }): a reconnecting WebSocket client. It queues exchanges while connecting, retries on a jittered backoff capped at maxRetryDelay (5 s), keeps request ids stable across resends, and after maxRetries (5) consecutive failed connections fails queued work and fails new exchanges fast until a socket is healthy again. maxQueue (64) bounds the queue. Instead of a url, pass makeSocket: () => SocketLike to connect over something else; it is called again on each reconnect.
  • Sync.transport.serve(socket, { exchange, changes? }): the server side of one accepted socket. With changes, a subscription to the document's commits, it sends a notice after each; the client's loop exchanges on every notice, so a tab that is only reading still sees others' edits without polling.
  • Sync.transport.portSocket(port, { signal? }): a socket over a MessagePort, for a server in a worker or in the page. A port tells neither end when the other goes, so its onClose fires on close() or when signal aborts: a host aborts it when the pane it serves is gone, and serve stops. A client opens one per connection: Sync.transport.socket({ makeSocket: () => Sync.transport.portSocket(open()) }).
  • Sync.transport.loopback(handler), fromPromise(client), toPromise(shape), and nativeSocket for the platform WebSocket.

A refusal is data in the answer; only a wire failure is a TransportError.

Presence

"Who is viewing this" and "where is their caret" should vanish when the peer does, not replay next week. Sync.presence.make({ id, ttl, decodeValue, channel?, throttle? }) is a TTL'd peer registry outside the log: values are decoded before they enter it, a peer that stops refreshing is pruned, and throttle sends at most one value per interval, the latest. Sync.presence.socketChannel(transport.socket) shares the sync socket across reconnects; Sync.presence.hub and .serve are the server side. .serve stamps each frame with the connection's peerId, so a peer cannot speak as another, and with opens (the exchange's own) joins only a connection that may open the document, relaying nothing to or from one refused. It asks again with each frame the connection sends, and one no longer let open the document leaves. examples/pages shows remote carets.

Last-writer-wins fields (experimental)

Server order decides concurrent edits. When one field should instead keep the newest logical write even from a device that reconnects late, Sync.lww.register(schema) gives it a stamped schema and a pure merge; Sync.lww.openClock allocates stamps from a persisted counter, before dispatch and never in update. It is a per-field rule, not a CRDT mode: sets, counters, and text stay outside.

ts
const Title = Sync.lww.register(Schema.String)
const Shared = Schema.Struct({ title: Title.schema })
// in update: title: Title.merge(model.title, message.title)

Without a Foldkit application

Sync.define({ documentId, message, shared, empty, durable, replay }) is the protocol primitive forApplication compiles to, for a client with no application to derive from. Its replay is unguarded; the author owns its agreement with whatever applies the Message elsewhere.

Joining an assembly

TodoSync.wiring() adds the contract to a foldkit-bundle assembly so Module.validate can report two owners of one field. It routes nothing and runs nothing; Sync owns its runtime through mount.

What Sync owns, and does not

Owns the replication contract (shared slice, durable set, replay, codecs, journalContract), the replica (outbox, base, cursor, optimistic view, status), reconciliation, the transport seam, and the optional presence and LWW helpers.

Does not own the rest of the Model, server-derived caches (foldkit-remote), the authoritative order and storage (foldkit-durable or your server), peer-to-peer merging, or migration of your own Messages and snapshots.

Limits

  • IndexedDB is the only built-in storage; Storage is three members to implement.
  • One server order per document; no CRDT and no collaborative text algorithm.
  • Binding the socket transport to a platform WebSocket server is glue the examples show with ws; on workers, foldkit-sync/do defines the object and Sync.transport.workerSocket wraps its accepted pair.

Older spellings

Sync groups the exports and wraps nothing; every name is also exported on its own with the same signature: defineSync is Sync.define, syncMetrics is Sync.metrics, layerSocket/layerLoopback/layerFromPromise/toPromise/ serveSocket/nativeSocket/workerSocket are Sync.transport.*, createPresence/ createPresenceHub/servePresence/socketPresenceChannel/ loopbackPresenceChannel are Sync.presence.*, and lwwRegister/openLwwClock are Sync.lww.*. documentId('todos') and DocumentId.make('todos') make the same branded value, as do the other id constructors.

See also