Sync / Replicated state
3. The server: an exchange over the journal
The journal takes its codecs, empty snapshot, reducer, replica binding, and the
authorize rules from the contract. The exchange is one request from a
replica and one answer:
request cursor, pending operations, the epoch the replica last saw
answer operations committed after the cursor, in order
acknowledged ids accepted (also when their payload is not repeated)
rejected ids refused
reasons? [{ opId, reason }]: why, in words for a person, for some of rejected
checkpoint? { cursor, model } when the tail was compacted away
more? another page follows
epoch this server's historyfoldkit-sync/journal answers it over the journal. What is yours is who the
principal is:
ts
import { Effect } from 'effect'
import { ActorId, Journal, OpId } from 'foldkit-durable'
import type { SocketLike } from 'foldkit-sync'
import { serveJournal } from 'foldkit-sync/journal'
const server = Effect.gen(function* () {
const journal = yield* Journal.make({
...TodoSync.journalContract(),
file: 'todos.sqlite',
opId: operation => OpId.make(operation.opId),
actorId: (principal: Principal) => ActorId.make(principal.actorId),
validate: ({ operation, cursor }) => {
if (operation.baseCursor > cursor) throw new Error('Operation is ahead of the server')
},
})
// One accepted socket; `principal` is what authenticating the connection established.
return (socket: SocketLike, principal: Principal) =>
serveJournal(socket, {
sync: TodoSync,
journal,
principal,
// A principal that may only read has every operation rejected.
admit: () => principal.canWrite,
})
}).pipe(Effect.scoped)journalExchange is the same exchange as a TransportClient, for an
in-process replica; settle applies what committed to another store
(journal.recover) after each exchange. It keeps the rules a server has to
keep:
- Check the cursor before appending anything. If the replica's cursor is ahead of the journal, the exchange must fail before any commit, or a commit's acknowledgement would be lost with the failed read and the replica would resend it.
- Reject what will fail again; fail what might not. A policy refusal, a
payload that cannot decode, and an id reused with different content are
deterministic, so they are returned as
rejectedand the replica drops them. AJournalError(storage) is left to fail the exchange, so the replica keeps the edit and retries. A handler that throws on a bad payload would be retried with the same outbox forever. - Acknowledge a repeat.
appendanswers a retransmitted id from history (Committed, orAlreadyCommittedonce its payload was compacted) without applying it twice. Both are acknowledged. - Page, and checkpoint below the floor.
readwith alimitandmore: truelet a replica that was offline for weeks catch up in bounded steps. A read below the compaction floor fails withCompactedCursorError; the answer is then a checkpoint fromload, which the replica adopts as its base before replaying its own pending edits. - Name the history. The replica stores the
epochand sends it back; a replica from another history is answered from0, and it rebuilds. This is how a server reset is survived; see when things go wrong. - The principal comes from the connection. It is established when the socket is authenticated and never read from an operation, so a client cannot claim to be someone else. Durable also binds each replica id to the first actor that commits from it, and refuses another's.
examples/sync/src/journal.ts writes the
exchange by hand, the rules above as code, with server-side effects settled
before the acknowledgement, and
server.ts beside it binds serve to a ws
server with a per-connection token.