Build outward

Searching text: the kernel's primitive operator#

Some questions the kernel cannot say. A UPC is typed from its start, so a search should match from its start, not anywhere. Text.startsWith is the kernel's one primitive operator:

import { Text } from 'foldkit-entity'

const search = Expr.input('search', Schema.String)
const ByCode = Query.from(Product).pipe(Query.where(Text.startsWith(fields.upc, search)))

// `label` declares no collation, so a call over it names one.
const ByLabel = Query.from(Product).pipe(
  Query.where(Text.startsWith(fields.label, search, { collation: Collation.asciiFold })),
)

It compares under the text's collation: binary exactly, asciiFold folding ASCII letters on both sides. Text with no collation is refused where the call is written, because what such text compares as is each database's own choice, and the client could not agree with all of them. A null on either side is unknown, and the empty prefix holds for every row. % and _ are characters, never wildcards. The registry demo's product search uses it.

An operator of your own#

A primitive operator states what it means and leaves the computing to each interpreter. Declare it, then implement it for each one that runs it:

import { sql } from 'drizzle-orm'
import { Operators, Order } from 'foldkit-entity'
import { DrizzleOperators } from 'foldkit-remote-drizzle'

/** How far a price lies from a target, in cents. */
const distance = Expr.operator('acme.distance', {
  input: [Schema.Int, Schema.Int],
  output: Schema.Int,
  description: 'How far the first number lies from the second.',
})

const memory = Operators.merge(
  Operators.kernel,
  Operators.memory(Operators.implement(distance, (cents, target) => Math.abs(cents - target))),
)
const sqlOperators = DrizzleOperators.merge(
  DrizzleOperators.kernel,
  DrizzleOperators.make(
    DrizzleOperators.implement(distance, ({ args: [cents, target] }) => sql`abs(${cents} - ${target})`),
  ),
)

• The server takes both tables where the domain is bound: bind(entities, storage, { operators: { memory, sql: sqlOperators } }). Every query, invariant and write over the domain runs the operator by them, and one that calls an operator either table lacks is refused where it is registered, not on the first request that uses it. A guard's Read rules take the SQL table where they are compiled (GuardDrizzle.visible(Policy, entities, { operators })).
• The client takes the memory table as Remote.make({ …, operators: memory }), so it judges and orders the rows it holds by the same meaning.
• Null is strict by default: a null argument gives null, and the implementation never sees one. nulls: 'handled' operators, implemented with Operators.implementHandlingNulls, answer for null themselves.
• Text results keep a collation when the operator says collation: 'operands': a join of two binary texts is binary text, and compares and orders so.

Ordering by a computed value#

A call orders when its result does: a number, or text with a collation. "Nearest price to what I can spend" is one term:

const budget = Expr.input('budget', Schema.Int)
const Nearest = Query.from(Product).pipe(Query.orderBy(Order.asc(distance(fields.cents, budget))))

The server compiles the call as an expression and pages by its value, keyset cursors included, the same in SQLite and Postgres; ties break by id. The client orders by the memory implementation, so an edited row is placed where the server would put it rather than refetched. Index the expression if the list must be fast.

Writing a computed value#

A declared write can set a field from the row before the write and the input, through an operator. The computation travels as data, by operator name, so the server compiles it into the same update that checks the write, and the client predicts it:

import { Write } from 'foldkit-entity'

const joined = Expr.operator('acme.joined', {
  input: [Schema.String, Schema.String],
  output: Schema.String,
  collation: 'operands',
})

const Retitle = Entity.input(
  Product,
  Schema.Struct({ id: Schema.String, suffix: Schema.String }),
  { suffix: Entity.unmapped },
)
const retitle = Write.update(Retitle, {
  id: 'id',
  compute: { label: ({ row, input }) => joined(row.label, input.suffix) },
})

• Write.apply(row, bound, { operators }) gives the row the write leaves (row, beside the fields it could not work out, unknown): the one function the writer explains a miss by, a guard judges and Remote shows while the write is pending. A live update to the row meanwhile is computed from, not hidden.
• A retry must not apply a computation twice. Without expect (a revision the row must still hold), a computed assignment lands only through a writer with a ledger, and then only where the writer pins the row it read (a guarded writer with a revision) or as op(field, amount) whose operator was tested commutative and associative, over every value, for each implementation, so two concurrent writes give the same row in either order.

The client agrees with the server#

Because the client runs the same body by the same meaning, it can answer questions about the rows it already holds without asking:

matching(store, Query, input, { operators }) ──▶ { matched, skipped }

matching (from foldkit-remote) runs the body by the table it is given, Remote.make's for a domain's own reads. matched is in the body's order; skipped is what it could not judge: a row missing a field the body reads, or holding text no interpreter answers for (a NUL). Each row is judged alone, so one such row never blanks the rest. Data.filtered(model, list, Query, input) filters a loaded list with it and says whether the answer is complete. An operator the client's table lacks cannot be judged there: a filter answers nothing and says it is incomplete, and a live row it cannot judge is the server's to place.

Holding implementations to each other#

A primitive operator has as many meanings as implementations unless something checks them. An operator may declare the equations it keeps, and Conformance tests each implementation:

import { Law } from 'foldkit-entity'
import { Conformance } from 'foldkit-entity/conformance'

const concat = Expr.operator('acme.concat', {
  input: [Schema.String, Schema.String],
  output: Schema.String,
  collation: 'operands',
  laws: [Law.associative, Law.identity(''), Law.preservesCollation],
})
const concatMemory = Operators.memory(
  Operators.implement(concat, (left, right) => `${left}${right}`),
)

const verdict = Conformance.laws(concat, concatMemory)
const recorded =
  verdict._tag === 'Holds' ? Operators.withEvidence(concatMemory, verdict.evidence) : concatMemory

const report = await Conformance.operator(concat, {
  implementations: { memory: Conformance.memory(concatMemory) },
})

• A law describes; nothing rewrites a body by one. It is checked where it is declared (a witness the argument cannot store is refused), and tested by seeded sampling: the empty string, mixed case, non-ASCII and astral text, zero, negatives, fractions. Declaring concat commutative fails with "a" and "A".
• Evidence belongs to an implementation. A memory table takes only what Conformance.laws tested on its own implementation; an SQL table only what Conformance.lawsBy made, never a record built by hand or tested in memory, and which runner made it is your word, so run it over the table's SQL. A table that overrides an implementation drops its evidence, and that of a law calling it as a witness. The commuting writes above require it in both the memory and the SQL table, for laws tested over every value.
• Conformance.operator runs one set of generated calls through every implementation given (null where admitted, NUL text, each collation) and reports where they disagree, a refusal included. Give it an SQL runner beside the memory one to hold the database to the same answers.

Publishing what a server runs#

Operators.describe(op) is an operator's signature as JSON: its name and version, its description, each argument and its result as a JSON Schema of the stored side as JSON carries it, its null and collation rules, its laws, and whether it is defined. It never includes a defined operator's body. OperatorDescription is the Schema that checks one, so a description another server publishes can be decoded, and Operators.describeTable(table) lists a table's operators with their evidence.

The version is part of the name. A table holding acme.slug does not answer a body calling acme.slug@2, and the refusal names both (acme.slug@2 (this table has acme.slug)), so a client and a server on different deploys see which side is behind.