Entity / foldkit-entity

Naming a question: operators

What operators make possible, end to end (a rule named once, search and order the kernel lacks, writes that compute, one meaning checked across interpreters), is the operators guide; this section and the next three are the reference.

A predicate an application asks in many places (in a public query, a visitor's Read rule, an invariant) can be given a name. Expr.operator declares one by what a call of it means:

import { Schema } from 'effect'
import { Entity, Expr, Query } from 'foldkit-entity'

const Story = Entity.define(
  'Story',
  Schema.Struct({ id: Schema.String, publishedAt: Schema.NullOr(Schema.String) }),
)

const published = Expr.operator('acme.published', {
  input: [Schema.Unknown],
  output: Schema.Boolean,
  definedAs: publishedAt => Expr.isNotNull(publishedAt),
})

const PublicStories = Query.from(Story).pipe(Query.where(published(Story.fields.publishedAt)))
Query.show(PublicStories)
// FROM Story
// WHERE acme.published(Story.publishedAt)

It computes nothing of its own. definedAs builds an expression from a call's arguments, once per call, and every interpreter runs that expression (Expr.expanded writes a predicate's calls out), so the operator needs no SQL, no evaluator, and no support list. The SQL a body compiles to is the SQL of the body written out by hand.
What it adds is a name. Query.show prints the call, and Query.dependencies(body).operators lists each operator a body uses.
It reads only its arguments. A definedAs that closes over a field or an input, instead of taking it, is refused when called: nothing asking what the call reads (Guard's checks, Remote's matching) would see it.
Arguments are checked by their stored form, at compile time and when called: a field, scalar or plain value of the input's stored type, or a predicate where it takes a boolean. A boolean output makes a call a predicate, usable in and, or, not, a guard or an invariant with no cast; any other makes it a scalar of that type.
Ids are namespaced (acme.published), and foldkit. is reserved. version (default 1) changes when what it means does; a call prints it after the first (acme.published@2(…)).

foldkit-cms declares one over its published role: Cms.published(entity).

Operators each interpreter implements

Without definedAs, an operator is primitive: it means what each interpreter's implementation does, and every implementation is held to the same statement of it. The kernel ships one, Text.startsWith; an application declares its own and implements it, here for the in-memory interpreter:

import { Schema } from 'effect'
import { Collation, Entity, Expr, Operators, Query, Text, evaluate } from 'foldkit-entity'

const Product = Entity.define(
  'Product',
  Schema.Struct({ id: Schema.String, upc: Schema.String, label: Schema.String }),
).pipe(Entity.annotateMembers({ upc: Collation.of(Collation.binary) }))
const search = Expr.input('search', Schema.String)

// The UPC declares `binary`, so it compares exactly; the label declares nothing,
// so this call is given a collation.
const ByCode = Query.from(Product).pipe(Query.where(Text.startsWith(Product.fields.upc, search)))
Text.startsWith(Product.fields.label, search, { collation: Collation.asciiFold })

const joined = Expr.operator('acme.joined', {
  input: [Schema.String, Schema.String],
  output: Schema.String,
  collation: 'operands',
})
const operators = Operators.merge(
  Operators.kernel,
  Operators.memory(Operators.implement(joined, (left, right) => `${left}${right}`)),
)
evaluate(ByCode, { search: '012' }, [{ id: 'p1', upc: '0123', label: 'Bolt' }], { operators })
// [{ id: 'p1', upc: '0123', label: 'Bolt' }]

startsWith compares under its text's collation (binary exactly, asciiFold folding ASCII letters on both sides), is unknown when either side is null, and holds for the empty prefix; text with no collation, or a backend's locale, is refused where it is written.

A table is a value. Operators.memory refuses one operator implemented twice, or two under one name; merge refuses an overlap, and override replaces on purpose. Operators.checked(table) checks each answer against the operator's output schema, for tests. Nothing registers globally: evaluate, evaluateEach, decided, truthOver, Entity.judge, Entity.judgeInput and Entity.check(entity, { operators }) each take the table, Operators.kernel by default.
Null is strict by default. A null argument gives null (unknown for a predicate) and the implementation never sees one. nulls: 'handled' operators are implemented with Operators.implementHandlingNulls and answer for null themselves; a predicate answers null for unknown.
Text has a collation or no comparison. collation: 'operands' makes a text result carry the collation its text arguments share, so Expr.lt(joined(a, b), x) compares under it; 'required' (as startsWith) refuses text with none. A call may be given one: op(a, b, { collation }).
A call orders when its result does. Order.asc(joined(Product.fields.upc, '!')) orders by the call's text under the collation it carries, and a call giving a number orders as one; anything else, text with no collation included, is refused where the term or Order.chosen alternative is written. evaluate runs it per row by its table, and compareRows(terms, a, b, query, { input, operators }) compares two rows the same way.
What a table lacks is refused by name. Query.unsupported(body, supported, operators) and Expr.unsupported(nodes, { operations, operators }) name each operation and operator missing; evaluate refuses such a body. An invariant calling an operator its judge's table lacks is Undecided with that operator in operators, which a form passes and a server refuses to start without.
One refusing row refuses the run, unless each row is judged alone. evaluate refuses a run when any row holds what no interpreter answers for (text with a NUL). evaluateEach(body, input, rows, { operators }) sets such a row aside in refused and still answers the others, in order; what no one row causes (an operator the table lacks, an input the body reads holding a NUL, order keys of different kinds) still refuses it. A client judging the rows it holds wants the second.
NUL is refused in any text argument, and an implementation that throws is reported as a QueryEvaluateError naming its operator.

Laws, and holding an implementation to them

An operator may declare the equations every implementation of it keeps. A law describes: nothing rewrites a body by one. Conformance.laws tests an implementation against them, and what it finds is evidence its table records.

import { Schema } from 'effect'
import { Expr, Law, Operators } from 'foldkit-entity'
import { Conformance } from 'foldkit-entity/conformance'

const concat = Expr.operator('acme.concat', {
  input: [Schema.String, Schema.String],
  output: Schema.String,
  collation: 'operands',
  laws: [Law.associative, Law.identity(''), Law.preservesCollation],
})
const table = Operators.memory(Operators.implement(concat, (left, right) => `${left}${right}`))

const verdict = Conformance.laws(concat, table) // { _tag: 'Holds', evidence: [...] }
const recorded =
  verdict._tag === 'Holds' ? Operators.withEvidence(table, verdict.evidence) : table
Operators.describeTable(recorded) // [{ name: 'acme.concat', …, laws, evidence }]

A law is checked where it is declared. Law.commutative, associative, idempotent, identity(value, side?), absorbing(value, side?), inverse(other, identity) and distributes(over) are laws of an operator of two arguments; monotonic(direction, position) names an argument it has, and preservesCollation needs collation: 'operands'. A witness that is not a value the argument's schema stores is refused.
Its domain is every non-null value of the input schemas, narrowed with Law.within(law, schema) where a law holds of less (a square is increasing over the non-negative numbers only). Strict null makes both sides null, so no law needs a null case.
Testing is sampling, seeded. Cases come from a fixed sample of each kind (the empty string, mixed case, non-ASCII and astral text, %, _ and \; zero, negatives, a fraction), kept where the schemas decode them, up to cases (200) a law chosen by seed (0), under each of binary and asciiFold where the operator or a witness its law calls compares text (so a record of such a law counts twice the cases). The verdict is Holds with one Tested record a law, or the first Counterexample with both sides and its collation: declaring concat commutative finds "a" and "A", which give "aA" and "Aa". Two sides that both refuse agree, but are no case: a record counts only cases whose sides both answered, and a law with none is Unanswered, with the first refusal's reason, and records nothing.
Evidence is an implementation's. Operators.withEvidence refuses a record of an operator the table lacks or a law it does not declare, and a Tested record Conformance.laws did not make of this table's own implementation (Operators.testedOn(record) names the one it did), or tested beside another implementation of a witness its law calls; override drops a replaced implementation's records, and those of a law that called it as a witness (inverse, distributes); merge keeps both tables'. An SQL table (DrizzleOperators.withEvidence) takes only records Conformance.lawsBy made (Operators.testedByRunner): not one built by hand, copied, or tested in memory. Which runner made it, the database's or another, is the caller's word, so run it over the table's own SQL (compileCall). Evidence.Proven records a proof made elsewhere, never checked here. A law's domain is described as a JSON Schema with the definitions it refers to.
A defined operator's laws describe it; nothing records them. It has no implementation to hold evidence. A write computing through it is expanded first, so it commutes by the primitive operator it calls, by that one's evidence.

Conformance.operator(op, { implementations }) runs one set of calls through every implementation given and reports where they disagree, a refusal included: generated from each argument's samples, null where its schema admits one, and text holding NUL, under binary and asciiFold where the operator compares text. Conformance.memory(table) is the reference interpreter's runner; a defined operator's expansion runs beside the others as expansion, a name an implementation of it may not take. foldkit-remote-drizzle runs the kernel's operators and an application's through it on SQLite and Postgres.

One operator, from declaration to description

The concat above, as an application ships it: declared once, implemented for each interpreter that runs it, held to itself, and described for whoever asks what a server runs.

import { sql } from 'drizzle-orm'
import { Operators } from 'foldkit-entity'
import { Conformance } from 'foldkit-entity/conformance'
import { DrizzleOperators } from 'foldkit-remote-drizzle'

// In memory, as above; in SQL, for a binding (`bind(…, { operators: { memory, sql } })`).
const memory = Operators.memory(Operators.implement(concat, (left, right) => `${left}${right}`))
const sqlTable = DrizzleOperators.make(
  DrizzleOperators.implement(concat, ({ args: [left, right] }) => sql`(${left} || ${right})`),
)

// Its laws, against the memory implementation.
const held = Conformance.laws(concat, memory)

// The two implementations against each other: an SQL runner selects
// `compileCall(call, binding, name)` from a database (foldkit-remote-drizzle's
// test/operatorConformance.test.ts has one); here the memory one alone.
const report = await Conformance.operator(concat, {
  implementations: { memory: Conformance.memory(memory) },
})

// Its signature as JSON: name, version, each argument and the result as a JSON
// Schema of what is stored, its null and collation rules, its laws.
Operators.describe(concat)

Operators.describe gives an OperatorDescription, a Schema whose decoder checks a description another server publishes; it says what an operator takes and keeps, never a defined operator's body. Operators.describeTable(table) lists each operator a table implements so, with its evidence, and Query.operators(body) names each operator a body calls (a defined one and the primitive ones it calls). A version is part of an operator's name: a table holding acme.slug does not answer a body calling acme.slug@2, and the refusal names both (acme.slug@2 (this table has acme.slug)), so a client and a server on different deploys see which side is behind.

Operators each interpreter implements#

Laws, and holding an implementation to them#

One operator, from declaration to description#

Operators each interpreter implements

Laws, and holding an implementation to them

One operator, from declaration to description