Media / foldkit-media
Running the resizing server
foldkit-media/server serves the /_image/ route that Provider.hosted
writes, for originals no CDN can fetch. MediaApi defines the Effect HttpApi
endpoint; the handler decodes each path with the space's Schema. URLs outside
the grammar (leading zeros, malformed hashes) return 400; well-formed URLs
outside the configured space return 404. Variants cache for a year; error
responses cache for one minute at the edge and are marked no-store for
browsers so rolling deploys recover quickly.
Variant URLs carry unguessable SHA-256 hashes, though every page rendering an image publishes its hash; keep sensitive files out of public variant routes.
The server needs three Effect services, each a Layer you choose per host:
| Service | Job | Kind |
|---|---|---|
Originals | Original bytes by content hash | Authoritative; read-only |
Transformer | Resize and encode without enlarging | Stateless compute |
Variants | Cached variants by encoded request | Rebuildable cache |
A request is served from Variants when cached; otherwise the original is
read, transformed, stored, and served. Failures return structured error JSON
({ "error": "NotFound" }) without server internals.
Every response gets its headers from headersFor. Unhandled defects (such as
failing stores or missing Worker layers) produce a framework 500 response.
| Answer | Status | Cache-Control |
|---|---|---|
Variant (ETag: "<hash>/<spelling>") | 200 | public, max-age=31536000, immutable |
Conditional hit (If-None-Match) | 304 | Same as 200; skips reading bytes |
| Malformed URL | 400 | no-store (CDN-Cache-Control: public, max-age=60) |
| Width/format outside space | 404 | Same as 400; next release may add it |
| Retired revision not in store | 302 | Redirect to current revision; 60s CDN cache |
| Missing original | 404 | Same as 400; upload may arrive later |
Original over maxBytes (20 MB) | 413 | Same as 400 |
| Transformer failure | 502 | no-store |
| Rate limited | 429 | no-store, Retry-After |
| Quota spent | 503 | no-store, Retry-After: 3600 |
Every image response also sends X-Content-Type-Options: nosniff,
Access-Control-Allow-Origin: *, Cross-Origin-Resource-Policy: cross-origin,
Timing-Allow-Origin: *, and Server-Timing (edge, store, or made).
Because the format is encoded in the URL path, Vary is left unset.
serveFetch({ images?, uploads?, otherwise? })builds a Worker fetch handler owning/_image/and/_media/. Unmatched subpaths andOPTIONSpreflights under those prefixes return 404; other paths route tootherwise(404 by default).layer({ provider?, maxBytes?, coalesce? })mounts the route as an EffectHttpRouterlayer for Node servers.variantsNoneskips variant caching for deployments fronted by an external cache.
Stores
Originals and Variants read from any Effect KeyValueStore
(effect/persistence):
import { NodeServices } from '@effect/platform-node'
import { Layer } from 'effect'
import { KeyValueStore } from 'effect/persistence'
import { originalsKeyValueStore, variantsKeyValueStore } from 'foldkit-media/server'
const stores = Layer.mergeAll(originalsKeyValueStore(), variantsKeyValueStore()).pipe(
Layer.provide(KeyValueStore.layerFileSystem('./media')),
Layer.provide(NodeServices.layer),
)KeyValueStore.layerMemoryfor tests,layerFileSystemon disk (requiringFileSystemandPath),layerSqlover SQLite or D1, orkeyValueStoreR2over R2.variantsKeyValueStore({ prefix })stores variants undervariants/<hash>/<spelling>, deriving media type from the key extension. Read or write errors emit aStoreFaileddiagnostic and fall back to transforming from the original.originalsKeyValueStore({ prefix })reads originals by content hash. Store failures return 500 rather than 404 so missing storage fails loudly.
Rate limits
Only cache misses trigger transforms, so Admission runs after a cache miss and
before reading the original. Cached variants serve all clients without counting
against rate limits:
admissionRateLimiter({ limit, window, algorithm? })wraps Effect'sRateLimiter(effect/persistence):limitmisses per client perwindow, backed by memory or Redis.admissionRateLimit(binding)(foldkit-media/cloudflare) wraps a Cloudflare Workers rate-limit binding ([[ratelimits]]).- Exceeding the limit returns
429withRetry-After. Limiter failures emitAdmissionFailedand admit the request so a down limiter keeps images online. - Clients are identified by
CF-Connecting-IP, then connection address. PassclientOfto read another proxy header (X-Forwarded-Foris ignored by default since clients can spoof it).
On a Node server
foldkit-media/sharp provides the Transformer (with sharp as an optional
peer):
import { NodeServices } from '@effect/platform-node'
import { Layer } from 'effect'
import { KeyValueStore } from 'effect/persistence'
import { layer, originalsKeyValueStore, variantsKeyValueStore } from 'foldkit-media/server'
import { transformerSharp } from 'foldkit-media/sharp'
const media = layer().pipe(
Layer.provide(
Layer.mergeAll(
transformerSharp({ concurrency: 4, maxPixels: 50_000_000 }),
originalsKeyValueStore(),
variantsKeyValueStore(),
),
),
Layer.provide(KeyValueStore.layerFileSystem('./media')),
Layer.provide(NodeServices.layer),
)transformerSharpandinspectorSharpnormalize EXIF orientation so portrait phone photos are measured, resized, and placeheld upright.transformerSharpchecks header dimensions before decoding and rejects images abovemaxPixels(50 megapixels by default) with413before allocating pixel buffers. It caps output width at the original's width and encodes at the URL's quality.limitTransforms(n)caps concurrent transforms on anyTransformer(defaults to CPU core count).layercoalesces concurrent misses for the same variant into one transform and store write (coalescedefaults totrueon Node andfalseinserveFetch, where Worker requests run in separate request contexts).
Uploads
Uploaded images enter through POST /_media/upload as raw image/* request
bodies. The route streams up to maxBytes (20 MB by default), extracts facts
with Inspector, stores the bytes under their SHA-256 hash alongside
StoredFacts, and returns a 201 JSON HashedAsset. The variant server can
serve that hash immediately.
Authorization is required. UploadPolicy.admit(request, route) runs before
reading any body bytes, receiving 'Upload' or 'DescribeUrl'. It fails with
UploadForbidden (403) or UploadUnauthenticated (401) when identity is
missing so clients know to refresh a session.
import { Layer } from 'effect'
import { GuardMedia } from 'foldkit-guard/media'
import {
uploadLayer,
uploadPolicyAny,
uploadPolicyBearer,
uploadsKeyValueStore,
} from 'foldkit-media/server'
import { inspectorSharp } from 'foldkit-media/sharp'
const uploads = uploadLayer({ maxBytes: 10 * 1024 * 1024 }).pipe(
Layer.provide(
Layer.mergeAll(
uploadsKeyValueStore(),
inspectorSharp(),
uploadPolicyAny(
uploadPolicyBearer(process.env['UPLOAD_TOKEN']),
GuardMedia.uploadPolicy(Policy, { resolve, Upload: [G.isEditor] }),
),
),
),
)uploadPolicyBearer(secret)acceptsAuthorization: Bearer <secret>using constant-time digest comparison and rejects requests while the secret is unset or empty.GuardMedia.uploadPolicy(foldkit-guard/media) authorizes each route using your guard set's principal guards (G,Policy) and sessionresolve(such asAuthServer.resolverfromfoldkit-auth/server). Routes omitted from the map reject all requests; here editors can upload while URL fact lookups stay closed.uploadPolicyAny(a, b, …)tries policies in order, admitting on the first match or returning the last refusal.- Custom policies use
Layer.succeed(UploadPolicy, { admit }).
Registering an upload. Mutations receive only the submitted hash and look
up server-recorded facts via StoredFacts.get(hash) (backed by
storedFactsKeyValueStore() or storedFactsR2(bucket)), returning none if
no upload recorded that hash. Client-submitted editor metadata such as a focal
point travels alongside the hash.
uploadsKeyValueStore() and uploadsR2(bucket) share backing stores with
originals, and serveFetch's uploads mounts the routes in a Worker. Refusals
return minimal status codes with no-store (401/403 from policy, 415 for
unrecognized image bodies, 413 above byte/pixel limits, 400 for truncated
streams). Requiring image/* without CORS OPTIONS prevents cross-origin form
or cookie uploads from other sites.
In the browser, upload(file, { url?, headers? }) (foldkit-media/browser)
returns a Stream emitting Progressed { loaded, total } and then
Uploaded { asset }. Failures emit UploadFailed { reason, status } (Forbidden,
Unauthenticated, TooLarge, Unsupported, Malformed, Network, or
Unexpected). It uses XMLHttpRequest for upload progress and cancels when the
Effect stream is interrupted:
import { Effect, Match, Stream } from 'effect'
import { upload } from 'foldkit-media/browser'
const send = (file: File) =>
upload(file).pipe(
Stream.runForEach(event =>
Effect.sync(() =>
Match.valueTags(event, {
Progressed: ({ loaded, total }) => console.log(`${loaded} of ${total} bytes`),
Uploaded: ({ asset }) => console.log('kept as', asset.key),
}),
),
),
)Same-origin browser uploads send the page's session cookie automatically; reserve bearer tokens for deploy scripts so secrets stay off client pages.
Remote image URLs get their facts from GET /_media/facts?url=, protected by
the same UploadPolicy with 'DescribeUrl'. Configure allowed hosts in
uploadLayer({ facts: { allow } }) (empty by default). Unreachable or invalid
images return 422.
Files
Forms often need arbitrary attachments (PDFs, spreadsheets). fileLayer adds a
separate route set at POST /_media/files and GET /_media/files/:key, storing
bytes in FileStore under random 32-hex keys distinct from the 64-hex hashes
used by /_image/.
Ownership. The upload route owns the bytes and recorded metadata (key,
name, type, size). Your application owns which database row holds a
FileAsset and who may download it:
- The browser runs
uploadFile(file)toPOST /_media/files, receiving a mintedFileAsset. - Your form submits the
FileAsset; your server mutation authorizes the write and callskeepOrClaimto bind the file to the row. GET /_media/files/:keychecksFileReadPolicyfor the requester and streams the file as a download attachment.
Each upload receives a fresh 128-bit random key and stores its own copy so duplicate uploads reveal nothing about existing files.
import { Effect, Layer, Option } from 'effect'
import { SecureRandom } from 'effect/crypto'
import { KeyValueStore } from 'effect/persistence'
import type { FileAsset } from 'foldkit-media'
import {
FileUploadPolicy,
UploadForbidden,
fileLayer,
fileReadPolicy,
filesKeyValueStore,
keepOrClaim,
} from 'foldkit-media/server'
// Your own: who a request is, and which rows hold which file.
declare const userOf: (headers: Headers) => Option.Option<string>
declare const rowHolding: (key: string) => Effect.Effect<Option.Option<{ readonly owner: string }>>
const files = fileLayer({ accept: ['application/pdf', 'image/*'] }).pipe(
Layer.provide(
Layer.mergeAll(
filesKeyValueStore(),
Layer.succeed(FileUploadPolicy, {
admit: request =>
Option.match(userOf(new Headers(request.headers)), {
onNone: () => Effect.fail(new UploadForbidden({})),
onSome: Effect.succeed,
}),
}),
fileReadPolicy({
resolve: request => Effect.succeed(userOf(new Headers(request.headers))),
read: (user, key) =>
rowHolding(key).pipe(
Effect.map(row =>
Option.exists(row, ({ owner }) => Option.contains(user, owner)),
),
),
}),
),
),
Layer.provide(Layer.merge(KeyValueStore.layerMemory, SecureRandom.layerWebCrypto)),
)
// In the mutation that saves a row, once the write is authorized:
const attached = (held: Option.Option<FileAsset>, submitted: FileAsset, user: string) =>
keepOrClaim(held, submitted, user)FileUploadPolicy.admit(request)resolves the uploader's stable owner ID before reading bytes. Blank IDs fail with403and emitOwnerMissing.fileReadPolicy({ resolve, read })authenticates the reader first (returning401FileUnauthenticatedwhen unknown), then checksread(user, key). Refused reads return404identically to missing keys so key existence stays hidden.keepOrClaim(held, submitted, owner)keeps an unchanged key in place when re-saving a row, or claims a newly uploaded key fromFileStorewithinclaimWithin(24 hours by default, failingFileUnclaimedfor expired, claimed, or foreign keys). Call it after authorizing the mutation so refused writes leave keys unclaimed.
Downloads execute safely. Responses use application/octet-stream,
X-Content-Type-Options: nosniff, Content-Security-Policy: sandbox; default-src 'none',
Cross-Origin-Resource-Policy: same-origin, Cache-Control: private, no-store,
and an attachment Content-Disposition filename from the stored record.
Upload validation. Bodies arrive as application/octet-stream with headers
x-foldkit-file-name (percent-encoded UTF-8, up to 255 bytes, free of control
chars or slashes) and x-foldkit-file-type (parsed media type matched against
required accept, or '*'). Because clients declare the type, inspect file
contents directly with inspect: (bytes, declared) => Effect<void, FileRefused>
when you need magic-number checks.
| Answer | Why |
|---|---|
201 | Kept; body is the FileAsset |
400 | Invalid/missing name or truncated body |
401 / 403 | Unauthenticated or policy rejected |
408 | Slower than minBytesPerSecond (32 KB/s over 10s) |
413 | Exceeded maxBytes (20 MB) |
415 | Wrong content type, outside accept, or inspect failed |
429 | Admission rate limit reached (Retry-After) |
503 | Concurrency slots full (maxConcurrent: 2) |
Memory. Uploads hold roughly twice their size in memory while streaming (80 MB peak at defaults, fitting inside a 128 MB Worker). Downloads also buffer in memory, so serve high-traffic large files via signed R2 URLs instead.
In the browser, uploadFile(file, { url?, headers? }) (foldkit-media/browser)
emits Progressed and FileUploaded { asset }, adding TooSlow, RateLimited,
and Busy failure reasons. Mount in a Worker via serveFetch's files option.
Written out at build
Static sites without an image server can emit every referenced variant during
build using materialize inside foldkit-ssr's staticSite emit:
import { NodeServices } from '@effect/platform-node'
import { readFileSync } from 'node:fs'
import { Layer } from 'effect'
import { KeyValueStore } from 'effect/persistence'
import {
materialize,
originalsKeyValueStore,
serveFetch,
variantsKeyValueStore,
} from 'foldkit-media/server'
import { transformerSharp } from 'foldkit-media/sharp'
import type { BuiltPage } from 'foldkit-ssr/vite'
const staticMedia = serveFetch<object>({
images: {
layer: () =>
Layer.mergeAll(transformerSharp(), originalsKeyValueStore(), variantsKeyValueStore()).pipe(
Layer.provide(KeyValueStore.layerFileSystem('.media')),
// The disk itself: Node's file system and paths.
Layer.provide(NodeServices.layer),
),
},
})
const env = {}
export const site = {
// …
emit: (pages: ReadonlyArray<BuiltPage>) =>
materialize({
pages,
fetch: request => staticMedia(request, env),
headersFile: readFileSync('public/_headers', 'utf8'),
}),
}- Missing or broken variants fail the build by name.
- Exceeding
maxFiles(20,000 by default) fails the build. - Include
headersRule()inpublic/_headersso static hosts serve variants with immutable caching;materializeverifiesheadersFilecontains it. - Keeping
Variantson disk lets repeat builds skip re-encoding.
On Cloudflare
foldkit-media/cloudflare wires R2, Cloudflare Images, and Workers rate
limiting into serveFetch:
import type { ImagesBinding, R2Bucket, RateLimit } from '@cloudflare/workers-types'
import { Layer } from 'effect'
import {
admissionRateLimit,
inspectorImages,
originalsR2,
transformerImages,
uploadsR2,
variantsR2,
} from 'foldkit-media/cloudflare'
import { serveFetch, uploadPolicyBearer } from 'foldkit-media/server'
interface WorkerEnv {
readonly MEDIA: R2Bucket
readonly IMAGES: ImagesBinding
readonly LIMITER: RateLimit
readonly UPLOAD_TOKEN: string
}
export default {
fetch: serveFetch<WorkerEnv>({
images: {
layer: env =>
Layer.mergeAll(
originalsR2(env.MEDIA),
transformerImages(env.IMAGES),
variantsR2(env.MEDIA, { cache: true }),
admissionRateLimit(env.LIMITER),
),
},
uploads: {
layer: env =>
Layer.mergeAll(
uploadsR2(env.MEDIA),
inspectorImages(env.IMAGES),
uploadPolicyBearer(env.UPLOAD_TOKEN),
),
},
// The application's own routes.
otherwise: async () => new Response('Not found', { status: 404 }),
}),
}transformerImages(images)scales down without enlarging and returns503on quota exhaustion (code9422) or502with anImageFaileddiagnostic.keyValueStoreR2(bucket)exposes an R2 bucket as an EffectKeyValueStore.variantsR2(bucket, { prefix?, cache? })stores variants in R2 so each size is generated once, optionally checking the Worker Cache API (cache: trueor a customCacheinstance) first.originalsR2(bucket)streams originals directly to avoid buffering full files in Worker memory.
Cloudflare bills each unique width per original once per month, so a smaller
width ladder caps monthly transformation cost. See
examples/cloudflare for a complete Worker setup.
Diagnostics
When the server recovers from a recoverable failure, it emits a typed
MediaDiagnostic through foldkit-surface's Diagnostics.Reporter (which
defaults to Effect logging and wrangler tail):
_tag | level | When |
|---|---|---|
StoreFailed | error | A variant, edge, or facts cache read/write failed and fell back |
AdmissionFailed | error | Rate limiter failed; request admitted without limiting |
ImageFailed | error / warning | Transform failed (error), or upload inspection rejected a body (warning) |
FactsUnanswered | warning | Facts route returned Unavailable for url with reason |
RequestDefect | error | Unhandled defect answered 500 by serveFetch |
Provide a custom reporter as an Effect Layer alongside your media layers:
import { Effect, Layer } from 'effect'
import { Diagnostics } from 'foldkit-surface'
serveFetch<WorkerEnv>({
images: {
layer: env =>
Layer.mergeAll(
originalsR2(env.MEDIA),
transformerImages(env.IMAGES),
variantsR2(env.MEDIA, { cache: true }),
Layer.succeed(Diagnostics.Reporter, diagnostic =>
Effect.logError(`media: ${diagnostic._tag}: ${diagnostic.message}`),
),
),
},
})In tests, Diagnostics.collect() gives a reporter and the array it records.