This is the specification of the library: what each piece of a program does, how the runtime runs it, and what
holds at each point where the process may die. The words it uses are defined in glossary.md, and
concepts.md shows the pieces and the life of an entity on one page.
update is a pure function from a message and the state to the next state. It rejects an inbound message
that does not apply, and answers an internal message that cannot arrive with a defect.work(state) derives the operations and the timers under way from the state, by key.
A key that leaves the state ends its work, and nothing has to cancel it.work derives from
it. A new version of a schema reads the version before through a migration.The sections up to Where it runs are the contract: what a program, its boundary, its tests and an operator rely on. The last section, How the runtime keeps it, is for whoever changes the runtime.
A server holds two kinds of state:
An entity is addressed by its id, handles one message at a time, and its state survives crashes, restarts and
deploys. Its program is written as in effect-tea: a pure update, states that say which operation is under
way, reject for what may arrive and does not apply, a defect for what cannot arrive.
The same intuition is behind Erlang's gen_server, Akka Persistence and Restate's virtual objects. What this
library adds is effect-tea's discipline: everything that matters is in the state, and every behaviour is
tested by calling pure functions, crashes included.
| Piece | What it does |
|---|---|
schemas |
state, flags, inbound, internal, operation, rejection, sender, projection |
name |
what the runtime stores the entity under, and what other entities see it as |
keys |
every key the work under way is held under: a name, or key with its parameter's schema |
init |
the first state, from the flags and the time of the creation, or a rejection of the flags |
update |
(message, meta, children) => (state) => the next state, a rejection, or a defect |
work |
the work under way in each state, by key: operations, as data, and timers |
operations |
how each kind of operation runs, what it answers, and what a crash does to it |
project |
what callers read of the state, with the actions it allows, from the children's too |
topics |
the parts of the projection the log publishes, by name, each selected from it as JSON |
children |
the parts the state holds, each placed with child or each |
onEvent |
what the program does with each event a child raises |
indexes |
what a query across the program's entities filters and orders by, read off the projection |
migrations |
how the versions before this one are read, in order |
An entity's loop runs every piece of the table, with its children and the bound of callers' messages:
flowchart LR
caller -->|create, with flags| init
init -->|Created or Rejected| caller
init --> state([state])
caller -->|inbound message, with a request id| bound{within the bound of callers' messages}
bound -->|yes| update
bound -->|no: Full| caller
update -->|Accepted, Rejected or Suspended| caller
update --> state
update -->|wrapped message| child[a child's update]
child -->|raised event| onEvent
onEvent --> state
child --> state
state --> work[work, by key, the children's included]
work -->|operations| runner[runners]
work -->|timers| timers
runner -->|completion, same incarnation| update
timers -->|firing, same incarnation| update
runner -->|send, create| other[another entity]
state --> project
project -->|projection| read[read, query, count, sum, countBy]
project -->|a row in each topic whose selection changes| outbox
A program is declared in one place, program(schemas)(…): the schemas first, which give every type the program
is written in, and then what it does. Everything else is inferred from the pieces — the keys work may use,
the messages each kind of operation answers, the names a query may order by, and the services the operations run
in (decision 0035).
A program leaves out what it does not have, and each piece left out is empty
(decision 0180). The schemas of the internal
messages, the operations and the reasons left out are Schema.Never, and the sender's is Schema.Null. keys
left out is [], and operations and indexes left out are {}. work may be left out only where no key is declared, and
operations only where there is no kind of operation, so a program that holds work says what work it holds.
update may be left out only where there is no own message, inbound or internal, and leaves the state as it is
then (decision 0202). A parent whose only
messages are its children's still gives inbound: Schema.Never.
There are no commands. Every piece of work that must survive a crash is derived from the state by work
(decision 0003). An operation is data, as effect-tea's
commands are, and its runner sees it, never the state
(decision 0018). Everything that leaves the process has a schema: what
is stored, and the projection callers read
(decision 0025), and each of them encodes to what
JSON holds (decision 0030). A program gives the schema of its
senders too: a queued message's origin is stored through it, so a message handled after a restart keeps who sent
it (decision 0027).
A state is an object, as in the Guide (decision 0031). A tag is no rule
of the types, and the Guide's one state machine per concern is how the examples are written: it is also what
gives work and project a case to match on.
Nothing that crosses the library is mutated: the program and its schemas, its states, flags, messages, operations, timers and projections, and what the library gives back. A new value is a new object, as the Guide says of a state. The library relies on it — a step whose next state is the same object stores nothing callers read, and an operation changed in place is never seen to change — and, as for purity, checks nothing of it (decision 0052).
A program is checked where it enters — deploy, which a fix is too, and rehearse — before anything is stored
or run: one whose name is empty, or one of whose indexes is not a word, is refused. The check is pure and made
once, since a program is never mutated
(decision 0054,
decision 0136).
A program's types are what Handle, Rehearsal and Published take. TypesOf<typeof p> gives them, of the program
or of a reference to it, so a boundary names the handle it launched as Handle<TypesOf<typeof order>>
(decision 0188).
examples/order/order.ts is a whole program.
A program may hold children inside its entity, as effect-tea's parent does, stored in its one state
(decision 0133). The library routes each child's
messages, holds its work under its keys, lifts its runners and wraps its reasons, so the composed program is a
program the machine takes as any other. examples/cart holds a line of a cart for each article.
A child is a part, part(schemas)(…): a program's pieces without a name, indexes or migrations, which are the
parent's. Its schemas are a program's, and the sender it reads when it reads one. It leaves out what a program does. A part may also give two schemas
that only type it, since nothing stores what they describe:
event, what it raises for its parent. Its update returns raise(event)(state), and the parent's
onEvent handles the raised event in the same step.external, what only its parent sends it. An external message is no inbound one: a caller cannot send it,
so a fact the parent hands down as one truth, a checkout, reaches the child from the parent alone.A part may hold children of its own, so a tree is as deep as a program needs.
children holds each child: child(part, { at }) where an optic of Effect focuses, and
each(part, { id, at }) for children by id, as the entries an optic focuses, such as
Optic.id<State>().key('lines').compose(Optic.entries()). An id has a schema, and two ids are one when they
encode alike, as the parameters of a key are.update takes the parent's own messages, meta, and children: children.Line.update(id, message) sends a
child an external message, and returns the parent's step. A child that rejects it, or is not there, is a defect.
andThen and forEach add up the steps of a parent that is itself a part, keeping their raised events.
forEach over messages to children of one name by id puts the children back once, and not once per child, with
the same step as one message after the other gives. A child that raises an event puts them back before its parent
handles it (decision 0155).
One andThen after the other puts the children back once per message, since andThen gives its state at once.
A step that tells many children is therefore written with forEach, over the ids, or over the steps already made
with (step) => step (decision 0240).onEvent has a handler for each child that raises events, and is required exactly when one does. A handler of
children by id takes the id first.project takes children, with each child's projection: children.Line(state, id).init makes the whole state, and starts a child with the child's own init. A part's init never rejects:
the parent calls it, and decides in its own init or update whether the child starts
(decision 0182).{ _tag: <child>, message }, with id for a child by id. meta.from reaches the child as it reached the parent.
An inbound message to a child that is not there is rejected.<child>.<key>. A key of a child by id takes the id as parameter,
or the id and the key's own parameter as a pair.<child>.<tag>, as
{ _tag: 'Line.Reserve', id, operation }, run by the child's runner with its delivery.{ _tag: <child>, id, reason }.A name with a . is composition's: a parent whose own key, operation or child has one is refused where it is
declared. A child's name is composition's too. A parent's own message or reason whose _tag may be a child's name
does not compile, nor an own operation whose _tag may be <child>.<tag>. A tag typed string, a template that
reaches a child's name, and a schema with no _tag that takes any, as a record, are refused alike. The composed
schemas try each child's wrapper before the parent's own schema, and the own schema refuses a value tagged as a
child's, so a cast past the compiler fails where the value is encoded or decoded, and no field is dropped
(decision 0270). A child that leaves the state with an operation under way is a defect, as any key dropped while its
operation runs. A child that asks its parent to drop it leaves its work first, from a state that holds none: work a
state still holds after its completion runs again, so a parent that kept the child would run it twice. A child by id removed and added again comes back under new incarnations, so an answer of its earlier
run is dropped.
An item is inside the entity when a rule holds across it and the rest within one step. It is an entity of its own when others address it, or when its parent would hold more than one step should write. Inside the entity, it is a child when it is a concern of its own, which the parent hears of through its raised events and never reads to decide. Otherwise it is the parent's own state. A parent's step lists the work of every child that is there, so a step costs more with every child (see what a step costs).
A message is one of two kinds, and the rule for a message that does not apply follows from its kind.
Accepted or Rejected, with the reason update gave, if any
(decision 0010). A caller that retries with the same id
gets the same answer, and the message is applied once. The boundary gives send a keep, the time its
protocol retries for, and keep is required, since only the boundary knows that time. The reply is then kept for
that time from the step that answers, and a retry after it is a new request, applied again. A retry sent before
the answer is answered by it and is not applied, however short the keep
(decision 0282). A keep of
'forever' keeps the reply for ever. A step that stores a reply deletes its entity's replies past their time, and
Handle.expire those of the whole program, which the sweep runs at an interval
(decisions 0125,
0245). A program whose sender is
null takes none, and its callers' requests leave the sender out. Request ids are their
sender's: another entity's are
stored under its address, apart from everyone else's
(decision 0015), and a caller's under its sender, as the
program's schema writes it (decision 0048). A
request is a value — a caller's sender and request id, or the run of another entity's operation and the name it
gave — stored as its schema writes it, and never parsed out of a string
(decision 0059).
A caller's request carries its message too. A second request under a caller's sender and request id is a retry
only where its message, as the inbound schema encodes it with the fields of each object in order, is the same
text. One that carries another message is answered Reused: nothing of it is applied, queued or stored, and the
first request's reply is kept. So a value whose encoding keeps the order it was built in, as a set, is the same
request only built alike, and a retry decoded from the same request is. A reply another version stored is
compared once read through the migrations of inbound. A boundary answers Reused as a refusal of the caller's
request, as 422 over HTTP. A program whose sender is null keeps one space of request ids for every caller, so
its boundary gives request ids unique across its callers, as a UUID each. A request after a reply's keep is a
new request, whatever it carries
(decision 0269).
meta.from says who sent it, read off the request, so that
update decides who may do what. An inbound message may arrive in any state: a slow client, a retry, a second
device, a malicious caller. One that does not apply is rejected.
The runtime trusts the sender it is given: whoever calls create and send is the boundary, which
authenticates the caller and writes the sender as the identity it authenticated. A caller the boundary does not
authenticate never reaches the runtime, and nothing is stored for it. What a sender may do is domain, so a sender
that may not is rejected, never a defect
(decision 0124).
A caller may be another entity's operation: meta.from is then Entity, with the sender's name and id,
filled in by the runtime (see entities that talk).work(state), in the same incarnation that sent it
(decision 0004). So an internal message in a state
where its key is absent cannot arrive. update answers it with a defect.meta.at is the time the runtime handles the message, read once, before update. A program that needs time
reads it there, and update stays pure. A timer's message is the fact that its time has passed: update
trusts it, and never compares meta.at with the deadline. Another node's clock may be a few milliseconds
behind.
An entity is created by a request of its own, with its flags, answered Created once and AlreadyExists
after, or Rejected where its program's init rejects the flags, as the paragraph after this one says. The flags
are stored with the entity, and a caller's retry of the creation that carries other flags is answered Reused, as
a message is (decision 0269). A
message to an entity that does not exist is answered NotFound, and creates nothing
(decision 0012,
decision 0017). A caller gives the flags in the request, as
create(id, { flags, requestId, sender }), and a program whose flags are null leaves them out
(decision 0245). A creation and a message may share a request id,
and each is answered apart (decision 0050). init
sees no sender, so the program does not decide who may create an entity. The boundary decides it, or an entity
does: the entity's update rejects the request, or keeps work whose operation calls create. A defect at creation
is given to its creator, and nothing is stored, so a retry after a fix creates the entity
(decision 0124).
A program's init may reject the flags, with reject or rejectWith and the program's reasons, as update
rejects a message. A rule of the domain at creation that the flags' schema cannot hold, such as one that reads the
time of the creation, lives there. The creator is answered Rejected, with the reason. The runtime stores neither
the entity nor the reply: a retry under the same request id runs init again, as a message answered NotFound is
applied once its entity exists. On Cluster, Cluster saves the door's answer, and the creator's client resets a saved
Rejected before it sends the creation, so that a retry is delivered again
(decision 0182). Another entity's operation is the exception: its
rejection is kept while its work holds, so that a run after a crash is given it again, as
entities that talk says.
work(state) returns the work of the state, under the keys the program declares and no others: the names of
the keys are finite, and a name a program computes is not one of them
(decision 0033). A key may take a parameter, which is data,
as the subsections below say. The types catch a key never declared; a
declared key beside a misspelt one they accept, so the machine checks what work returns against keys
wherever it reads it, and a key the program does not declare is a defect
(decision 0037). So is an operation the program gives
no runner for, which a cast lets through
(decision 0054). The runtime compares the work with
the previous state's, as effect-tea compares subscriptions:
Work ends when its message is handled. In the step that handles it, its key counts as gone, so a next state that returns it starts new work: that is how a timer repeats (decision 0008).
Two kinds of work exist.
Schema.TaggedUnion, which says what to do.
The program's operations give a runner for each tag, which turns it into an effect that ends with one of the
internal messages its kind of operation answers
(decision 0034). Expected failures become
messages, as in effect-tea's commands. A transient failure is retried inside the runner, with Effect.retry and
a schedule that ends, under the same idempotency key. Once the schedule ends, the runner delivers a message
that says so, and the state decides what follows: a new attempt behind a timer, with a count of attempts the
state holds, or an end. The runtime counts no retries, and never makes a defect of them
(decision 0130). The runner says what a
crash does to it
(decision 0011): an idempotent
one receives an idempotency key (the entity's program and id, the operation's key and its incarnation:
decision 0015), and passes it on; one that may
act twice runs again; one that must act at most once is not run again after a crash, and delivers its
unknown message.
A restarted operation gets the same idempotency key. A new attempt in a later incarnation gets another. A
service that caps a key's length gets its digest: a runner declared with operation.digest is given the SHA-256
of the key, in 43 characters of base64url. The key's form is part of the delivery stored with the operation, so a
deploy cannot change it under an operation under way
(decision 0123). An
operation leaves the state only through its completion: a step whose next state drops the key of an operation
still running is a defect (decision 0005).
A droppable operation, declared with operation.droppable, is the exception. It acts on nothing, a computation
or a read, so a step may drop it or change it while it runs: the runtime interrupts it without waiting for its
finalizers, and a change starts it again as new work. A crash runs it again. It sends to no entity and creates
none, since a request acts on its receiver: one that does dies
(decision 0128). A droppable runner whose
services hold Entities does not compile, and the runtime's check is for one that escapes the type
(decision 0129).timer(at, message) takes no type parameter: the message keeps its literal tag, and work
checks it against the program's internal messages
(decision 0181). A timer is due at a whole millisecond, as
the storage holds it, and a timer due between two is a defect
(decision 0266).A stream from a socket or a feed is no work of a program, since it dies with the process. The boundary reads it,
and sends each item as an inbound message, under a request id it derives from the item's offset or id, so that an
item sent again after a crash gets the stored reply. update rejects an item the state does not want
(decision 0127).
The boundary acknowledges an item to its source once send answers Accepted or Rejected, since every other
answer stores nothing of the item. After Suspended or Full it sends the same item again, under the same request
id, after a wait. After NotFound it creates the entity and sends again, or drops the item on purpose and logs it.
After Reused it stops without acknowledging, and an operator decides. A source gives an item again after a crash
between the answer and the acknowledgement, which may come after the keep. So the entity keeps the offset of the
last item applied and rejects an item at or before it, or the boundary gives a keep of 'forever'. A discard does
not get a stream past an item the entity dies on, since the item sent again is a new request: the boundary skips it,
or a fix applies or rejects it. An entity that stays suspended holds the items after its own
(decision 0272).
examples/stream is such a boundary.
A kept key keeps its work only if the work is the same as it is stored: an operation that encodes to the JSON of
the one under way, a timer at the same time whose message encodes alike. A property a schema does not declare is
not stored, and makes no difference: what a step compares is what a crash compares
(decision 0058). A timer that changes is armed again, as new work. An operation
that changes while it runs is a defect, as one dropped is: otherwise an operation restarted after a crash,
built from the stored state, would act with other parameters under the same idempotency key. The rule holds
until update is given the operation's message, even while its completion waits in the mailbox and a crash
could no longer rebuild it: the program sees the step that handles the message, never the moment the completion
is recorded. A key whose kind changes, from a timer to an operation, is new work
(decision 0014,
decision 0018).
What runs is always what the state says. After a crash the runtime does not look for what was running: it
computes work(state) again. It compares that with the work stored with the state, so that a new program,
after a deploy, keeps the incarnations of the same work and starts anything else as new work. A new program that
changes or drops an operation under way, delivers it otherwise, or bears another name is a defect
(decision 0020,
decision 0022).
A program lists its keys. A key without a parameter is its name, and a key with one is key(name, schema), as in
keys: ['fee', key('charge', Schema.String)]. A name listed twice is a defect where the program is declared
(decision 0197). Each name is a
literal type: a list written outside the call without as const is typed string[], and does not compile
(decision 0199). Under a key with a
parameter, work returns a list of pairs, each a parameter and the piece of work under it. An order holds a charge
for each of its lines so, in the step that changes the order
(decision 0132).
A key is a name and a parameter, as data: the declared name, and the parameter as its schema encodes it, with the
fields of each object in order. A key without a parameter has none. So two parameters that encode alike are one
key, as two pieces of work that encode alike are one piece of work. A parameter its schema refuses is a defect, and
so are two pieces of work whose parameters encode alike. Every output, message, suspension and row that names a key
holds the name and the parameter apart, and nothing reads a key's text to tell them apart
(decision 0136). The idempotency key holds the name,
or the name and the parameter's JSON, in place: ["Order","o-1",["charge","line-7"],4].
Each run of a key's work has an incarnation, drawn from one counter of the entity, stored with its row. Work that starts anew under any key draws the next one: work under a key that comes back after leaving, that comes back after its work delivered, that changes kind, or a timer that changed. An internal message carries the incarnation of the work that sent it, and the runtime drops it when the key's incarnation has moved on. The counter never goes back, so a key whose work is gone has no row: when it comes back, it draws an incarnation no run of it had (decision 0132).
Without it, the second attempt of a payment could receive the late answer of the first. The runtime makes this impossible, so that no program has to remember attempt numbers.
An entity sends a message to another from an operation, with send(receiver, id, message, name), and creates one
with create(receiver, id, name, flags), whose flags come last and are left out where the receiver's flags are
null (decision 0245). The runtime gives
each run of an operation its own Entities, so the request is that run's — its entity, key and incarnation — under
the name, whatever the operation's delivery, and no operation can send under another's
(decision 0060): an operation that sends twice under one name to
the same entity dies (decision 0047). A restarted operation sends
the same requests, and gets the receiver's stored replies, kept while the work that sent them is stored
(decision 0021). An answer no entity
stores is kept for the operation the same way: a creation init rejected, and a message's NotFound. A run after a
crash is given the kept answer, though another creator has made the entity since, and its message is not applied
(decision 0253). What an operation's
request carries is not compared, since the runtime makes the request and no two intents share it. A run after a crash
that builds another message, or other flags, under a name is given the reply of the first one, which alone the
receiver holds. So an operation whose completion reports what it sent builds the message from its stored value and
the replies it was given, never from what it reads again
(decision 0269). While the receiver is
suspended, send waits for its resumption, and an operation that reaches a program not deployed yet waits for
its deploy (decision 0045): each is woken by
what it waits for, and nothing looks again on a clock
(decision 0063). On Effect Cluster the
receiver may be resumed on another runner, which tells the sender's entity Resumed
(decision 0167). A receiver that crashes has no reply
yet either, and the operation sends again after a while; only a defect of the sender's own suspends it
(decision 0046). A full receiver stored nothing
of the message, and the operation sends again after the same while: it is never given Full
(decision 0147). An operation creates an
entity the same way, with create and a name. Both name the receiver by its program, or by a reference:
reference<typeof inventory>('Inventory') is the program's name with its types, made where an import type brings
none of its code. The compiler checks the name against the program's, and the runtime reads only the name. A
program whose name is typed string, as one typed Program<T> by hand, has no reference, since no name could be
checked against it, and a sender passes the program itself. Two
programs that talk both ways import only each other's types, and a program compares a sender's name with a
reference's (decision 0183). An operation that talks only to
entities may run at least once:
run again, it sends the same requests, and acts once. update never sends: a step never waits on another entity
(decision 0009).
examples/booking holds seats in an event's inventory this way, and releases them when
the booking is cancelled or expires.
After each message the runtime stores, in one transaction:
It stores the state, not the messages that led to it: a fixed update changes what happens next, never what
already happened (decision 0002). The messages may be
kept as an audit, but nothing replays them.
A step writes the state whole. A step that changes the projection writes it in the read model. A step that changes
what at least one topic selects writes it once more, in the step's publication of the outbox, however many topics
the step is written in. Each row of a topic names the publication, and the selector runs where the row is read
(decision 0140,
decision 0154). A program that declares no
topics writes its projection twice.
The spike spikes/large-states.test.ts saw a step written in five topics take as long as one written in one, and
a step take about 5 ms with a state of 48 KB, 21 ms with 0.5 MB and 240 ms with 5 MB, when the projection holds
the state. It took about a third of that from 0.5 MB up when the projection holds a count. The spike
spikes/many-children.test.ts saw a parent's step take 3.5 ms with 1 000
children by id and 10 ms with 10 000, about half as much again as a flat state of that size takes. Such a step is
synchronous, and holds every other entity of the process while it runs.
So the runtime refuses no state for its size, and warns of a large step. A step that writes more than
warn.characters of the state and of the projection, in the read model and in its publication, 262 144 by default,
logs a warning through Effect's logger, once per activation of the entity. A large state is split by the program:
an item goes into an entity of its own
once its parent would hold more than one step should write
(decision 0135).
The runtime writes all of it through the program's schemas, as JSON, in tables of its own: the entity, the
keys with their incarnations and work, the mailbox in order, the replies by request, the read model, the
outbox with the publication of each step, the number of its partitions and the head of each, its consumers and
their cursors, and on Cluster the versions and the requests it keeps (src/storage.ts). Every
table, index, trigger and function the runtime makes is named with effecttea_, as effecttea_entities, so that
the tables of an application in the same database are never taken for its own. The prefix is fixed, and an
application names no table of its own with it. A launch on a database that already holds an object under one of the
runtime's names dies, and leaves none of the runtime's objects behind
(decision 0218). On
PostgreSQL the runtime makes every object in the schema that the client's search_path names first. A launch dies
where a schema further on the search_path holds the tables, and makes nothing, since the database cannot tell a
schema put in front of them on purpose from one made later by chance
(decision 0263). Only effecttea_migrations is named with that schema, which
current_schema() gives, since Migrator would otherwise find it in any schema of the search_path
(decision 0217). A launcher that sets the search_path of the runtime's role, as Cluster asks for
its own tables, puts them in a schema of their own. A database is still held by one runtime, whatever its schemas,
since the advisory lock that holds it is the database's
(decision 0215). Each row carries the version of the program that wrote it, and is read
by migrating its JSON to the program's version (see schemas that change). The row around
the JSON is read through its table's schema, which says what the storage writes: nothing is filled in or cast, and
a row it refuses fails the call that meets it (decision 0062). A schema whose
encoded side is not JSON does not compile: a time is held as Schema.DateTimeUtcFromMillis, not as
Schema.DateTimeUtc (decision 0030). Json is Effect's
Schema.Json, under the name the library had for it (decision 0098). A value its schema refuses,
or that encodes to a number JSON does not hold (NaN, Infinity) or to an array with a hole, cannot be stored.
The machine encodes what a step would store, and a step that would store such a value is a defect; an inbound
message that does not encode is its caller's defect,
and reaches no entity (decision 0040). So are flags
that do not encode: init sees only flags its schema takes
(decision 0049). A -0 is stored as 0, which it
equals (decision 0210). A
rejection's reason is given as a caller reads it from its JSON, so a first reply and a retry give the same reason,
-0 as 0 (decision 0214).
How the machine and the storage write these rows, and how a reply is kept and deleted, is under what the storage writes.
A program's schemas are what is stored, so a version of the program that changes one finds the version before it
in the database. migrations is how it reads it, in order: the first is the step from version 1 to version 2,
and the program's version is how many migrations it carries, plus one — a migration is the bump, and there is
none to declare (decision 0036). A rejection's reason
is kept as long as the reply that carries it: a caller's for the time its boundary gave, or for ever
(decision 0125), and an entity's until the work
that asked ends. The migrations of rejection are kept as long as a reply of a version they read is stored
(decision 0100).
A migration is a record of pure functions from the JSON a version wrote to the JSON the next one reads, by
kind of stored value: state, flags, operation, inbound, internal, rejection, sender and projection,
and a kind left out is unchanged. flags reads the flags stored with the request that created an entity, which a
caller's retry of the creation is compared with, and inbound reads the message of a stored reply too. Both are
read as the migrations write them, and not decoded
(decision 0269). It works on the JSON, which every schema encodes to
(decision 0030), so the types and schemas of the version before do
not live on in the code.
Every row carries the version that wrote it, and is read by migrating its JSON from that version and decoding it
with the program's schema: the machine sees values of the program's own version only. A row is written at the
program's version whenever an event writes it, and an event writes only what it changed: the state, a key, a
message in the mailbox, a reply and the row a query reads are migrated at every read until they are written again
(decision 0057), and
reindex writes the read model again, which a deploy that changed the projection's schema runs, as one that
changed an index does.
This is what lets a deploy change the shape of an operation under way: the stored operation, migrated, must be
the one the new work computes from the migrated state, and a migration that gives anything else suspends the
entity at its activation, as a program that changes an operation does
(decision 0020). A row written by a version
the program does not have is not read at all: the call that meets it fails, and the process serves the entities
it can read (decision 0043). reindex is
the exception, since it meets every row: it passes over such an entity, and names it
(decision 0122).
A program keeps every migration it ever had, since a row is migrated only when it is written again
(decision 0126). A migration
once deployed is never changed, since it is how a version that exists in storage is read. A test records cases of
each migration: a JSON value a version wrote of one kind, and the JSON value the next version reads of it.
migrationViolations checks the program against them.
check(program, { limit }) finds, before a deploy, the stored entities a new program would not run. It reads each
entity as an activation reads it, through the new program's migrations and from a snapshot of the entity's own, and
steps its activation with the machine, in memory. Inside a transaction the caller left open, each entity is read at
the caller's level, as Databases says. So it runs work, project, the indexes and the topics on the stored
state, compares the operations under way with the new work, and calls the unknown of an at-most-once operation
that may have run. A suspended entity is stepped as a resume by the new program would activate it, and the
message it died on is not handled. The check launches nothing, holds nothing and writes nothing, so it runs on the
database the old version holds (decision 0247).
On SQLite the check's client is opened readonly. A writable client of @effect/sql-sqlite-bun begins each
transaction with BEGIN IMMEDIATE, which waits for the old version's steps even where it only reads
(decision 0267).
It gives the entities not suspended and the suspended ones apart: how many it checked, how many fail, and the first
limit failures in the order it read them. A failure is Load, a migration or a schema that dies on a stored row,
so that every activation of the entity dies, or Activation, an activation that suspends the entity. Each gives
its cause, as a suspension renders it. What the runtime throws fails the whole call, as it fails reindex. The
entities are read by pages of their ids, so the check is a reading of a moment: an operation under way at the check
may have ended by the deploy. renames are the launch's, and the check reads the entities of a name renamed to the
program's as the program's. The check dies on tables that a migration of another release made, since it migrates
nothing.
A program is renamed by its launch: launch takes renames, a record from each old name to its new name
(decision 0156). A rename is a move of the
database, as the number of partitions is, so it is the launch's, and the program only carries its new name.
The launch runs every rename in one transaction, after it holds the database and before any deploy, so no entity is loaded while the rows move. Every row stored under the old name moves to the new one, in every table. Every reply whose sender is the old name names the new one, and so does every request the old name's entities sent: in the mailboxes, in the replies and in the rows of the entities they created. A sender that sends again after the rename keys its request with the new name, and finds the reply stored for it. A caller's retry finds its reply too.
An entity keeps the idempotency keys of its work. From its first rename it stores the name its keys are drawn from, and its keys are drawn from that name from then on, so an operation under way runs again under the key it was given. Its state, versions, incarnations, partition and suspension do not change, and the consumers keep their names and cursors. A state or a message that holds an address of the old name is the program's data, which its own migration moves.
A rename whose old name holds nothing moves nothing, so it may stay in the launch for as long as a database of the
old name may be launched. A rename onto a name that holds an entity, a head of the log or a consumer dies, and
moves nothing. An empty name, a name renamed to itself, and a name both renamed and the new name of another are
refused before the database is held. Two names may be renamed to one, as a program renamed twice declares both of
its former names: { a: 'c', b: 'c' } (decision 0176). A deploy of a program whose name the launch
renames dies, since it is the old program.
On Effect Cluster other runners share the database, and Cluster stores a door's requests under the program's name. So the launch marks each old name renamed, in the transaction that moves the rows, and the newest version moves to the new name. The check of every step dies for a renamed name, a runner of it is outranked as an older version is, and a runner of the old code that launches again dies. The new program's door is registered under each old name too, and steps the entity under the new name, so a request Cluster saved under the old name is delivered. A door's step names an operation's sender by its new name, so that a retry sent under the old name is answered from the reply the rename wrote again, and not applied twice (decision 0176).
work(state),
compares it with the stored work, starts its operations and arms its timers.meta.at, calls update, checks the step (defects included, and work,
project and the indexes on the next state), stores it, answers the caller, and starts or ends the work
whose keys changed.
An internal message update rejects is a defect, since it cannot be late
(decision 0014). A message that finds the mailbox empty is
received and handled in one event, and one transaction; the runtime takes a caller's message only once the
mailbox is empty, so a burst waits in memory, unstored
(decision 0074). An
entity holds at most callers.messages callers' messages, an option of the runtime, 4096 by default: the messages
of send offered to it and not yet applied. A send past the bound is answered Full at once, and nothing of
its message is offered or stored, so a retry under the same request id is a new try. The first refusal of an
activation logs a warning that names the entity
(decision 0147). On Effect Cluster a
caller is never answered Full: its message waits in Cluster's table until the entity takes it
(decision 0165).Restart activates the entity again, and arms the same timers in the same
incarnations. The timers stored are the stored work's, which says each one's time: the earliest wakes the
entity, and nothing else needs storing. So a process starts with the entities whose state holds an operation,
which only an activation runs again, and with those whose mailbox holds a message a crash left before its
step; the stored times wake the rest — except a suspended entity's, which nothing of fires
until an operator resumes it (decision 0029,
decision 0071). A process
over its capacity unloads the least recently used of its idle entities as soon as it holds more, and never a busy
one, nor one whose timer is due (decision 0076,
decision 0078).
The capacity is 10 000 entities by default, Effect Cluster's own number, and 'unbounded' leaves time alone to
unload. A loaded entity holds about 8 KB beside its decoded state, so 10 000 small ones hold about 100 MB
(decision 0151).These rules are a pure state machine, src/machine.ts: its events are a message arriving,
an operation completing or dying, a timer firing, a step, the process dying, and an operator resuming or
discarding. A runtime interprets its outputs (decision 0013).
The program is not part of the machine: each event is stepped by the program that runs the entity, and a deploy
is a Restart stepped by another one, so the machine and the runtime never see two programs
(decision 0064).
| The process dies… | After the restart |
|---|---|
| before a step's transaction commits | nothing of the step was seen: the message is handled again, a caller's on retry |
| after the commit, before the reply reaches the caller | the caller retries with the same request id, and gets the stored reply |
| after the commit, before the work it started | activation computes work(state) and starts it |
| while an operation runs, its effect done in the world but not recorded | it runs again, with the same idempotency key, or delivers unknown if at most once |
| after an operation's completion is recorded, before it is handled | it is handled as any message in the mailbox |
| after a timer fired, before its message is handled | its message is still in the mailbox, or the timer fires again from the state |
The fourth line is the one the library cannot close. An effect on the world is at least once, or at most once and then perhaps not at all. A service with idempotency keys, such as a payment provider, makes it exactly once. One without them, such as an email, may act twice, or is declared at most once, and the state says when it is not known (decision 0011). Another entity makes it exactly once, with its stored replies (decision 0009).
On a phone, a defect ends the app. On a server it must not end the process, and an entity whose message died
would receive it again and die again, forever. So a defect suspends the entity: its state stays as it was
before the message, the message stays unhandled, its timers are disarmed, and nothing is handled until someone
resumes it or discards the message (decision 0006). Its
operations under way run to their end: each completion is recorded behind the message the entity died on, and is
handled once the entity is resumed, so a charge under way when an unrelated message dies is neither run again nor
left unknown. An operation still running at the resumption is kept. One that dies meanwhile leaves the suspension
as it is, and runs again at the resumption (decision 0113). A defect in an
operation, in work or project on a step's next state, or in a new program activating the stored state,
suspends it too. A defect is what the program does: what one of its functions or schemas throws, each caught on
its own, or a rule of the design it breaks. Anything else thrown while an event is applied — a bug of the machine,
a row the runtime corrupted — is the runtime's, and crashes the entity instead
(decision 0061). Resuming activates it as after a
crash, by the program deployed: a fixed program is a deploy, which the entity stays suspended through, and the
resumption follows it (decision 0064). A caller's
discarded message stores no reply, and the caller's retry is a new message
(decision 0014). A discarded message an operation sent stores a
rejection with no reason, kept while the work that sent it holds. The operation's send sends the request again at
the resumption, and is given the rejection instead of the message being applied again
(decision 0260). An internal message is not discarded: the state
before it still holds its work, so an operation's completion would run the operation again under another
idempotency key, and a timer's firing would fire again at once. Only a program that handles it gets the entity
past it (decision 0041,
decision 0044). A caller meanwhile is answered Suspended, which
is not stored: every caller whose message waits in the mailbox when the entity suspends, and every message that
arrives while it is, which is not queued. Suspended says only that the request has no reply yet. The caller
retries under the same request id, and gets the stored reply once the entity is resumed: the message is applied
once, whichever retry finds it
(decision 0010,
decision 0023).
An operator finds the suspended entities with suspended, the longest suspended first: each with what it died on,
the cause, since when, and the operations of other entities that wait for its resumption. The step that answers an
operation Suspended stores its wait, and the step that takes the entity out of its suspension deletes the waits.
So the list is read from storage, on any runner and after a restart, and holds a wait while the run that sent it
holds (decision 0265).
The cause is stored as text, as what the storage writes says. Nobody is told on a clock: a sender waits as long as it takes, and a deployment that wants an alarm reads the list. resume and discard say
what they did: Resumed or Discarded; NotDiscarded, for an entity that died on an internal message, in an
operation or on its activation; StillSuspended, with the new cause, when the program deployed dies on the stored
state, and a discard then keeps its message; NotSuspended; or NotFound. Each operator is told what the
operator's own event did, even when two operate on one entity at once (decision 0114).
An operator decides between resume and discard with inspect(id). It gives what is stored of one entity: the
state, the mailbox with the next message first, the work under way with the oldest first, and the suspension as
suspended lists it. It reads what is stored and activates nothing, so an entity this process does not hold is
inspected too. Its rows are read through the migrations from one snapshot, so a step that commits meanwhile is read
whole or not at all (decision 0185). Inside a
transaction the caller left open, inspect reads at the caller's level, as Databases says.
project(state) is what callers read of an entity: its status, and the actions it allows. It is the
counterpart of effect-tea's model, and follows the same rule: it shows no action that update would reject.
A caller learns how an operation ended by reading the projection. The reply says only whether the message was
accepted and, for a rejection, perhaps why: an acceptance never carries data. The projection is JSON, and
project converts what the state holds, a time to its milliseconds: read, a query, an index and a topic read
one shape, and the compiler refuses a projection schema that decodes to anything else
(decision 0139). A projection shows what a caller
acts on, not the whole state: it is written twice in each step that changes it, and the outbox keeps it
(decision 0135).
The projection is the one contract of the entity: what a consumer of the log needs of it is in the projection
too, and a boundary that answers a caller leaves out what that caller is not shown
(decision 0138).
Queries across entities — "the orders nobody paid" — read the same thing. The runtime keeps one row per
entity, written in the step's transaction, holding the encoded projection; indexes gives, for each name, a
pure function from the projection to a scalar, which is what a query filters and orders by. So a query answers
with the values a caller reads of a single entity, and nothing is queried that a caller cannot read
(decision 0032). A time is its milliseconds, as a stored time is.
An index that gives anything but null, a string, a boolean or a finite number is the program's defect, and
suspends the entity: (projection) => projection.tags[0] is typed a string, and gives undefined for a projection
with no tag (decision 0206).
read gives the same row for one entity, and never calls project. It takes no sender, so the boundary alone
decides who may read a projection
(decision 0124). A new project that dies on a stored state
is then no reader's defect. The entity suspends at its activation or its next step, as on any defect of the
program, and a reader meanwhile gets the projection its last event wrote. After a deploy that changes project,
read and query both give the rows as they are, until a step or reindex writes them again. A row is read
through the projection's migrations and schema, and one of them that dies on it gives the defect to whoever reads
the row, read or query (decision 0121).
query takes the criteria that hold, an index to order by and a limit, and gives the ids found with their
projections, at most limit of them. The limit is required: a reading that holds a program's whole read model in
one array is never what its caller meant, and a call without one is the caller's defect
(decision 0141). A criterion's value is
null, a boolean, a finite number or a string that is text a database holds, and so is the id and the value of the
row passed as after. Anything else is the caller's defect, for query, count, sum and countBy. SQLite finds
no row under NaN, where PostgreSQL holds it a number past every other, and PostgreSQL takes no U+0000 in a
parameter of text, so each dialect would answer the same call otherwise
(decision 0207).
A query pages after the last row it gave. Each row found carries indexes, the value of each index as its row
stores it, which is what the query compared. A caller passes the last row of a page as after, and is given the
rows past it in the order of the query: past the value of that row for the index ordered by, and, among the rows that tie,
past the id of that row, since the id breaks ties in either order. Nothing is held between two pages, so a row no step changes
is given once, on one page, and a row whose index a step changes between two pages is given where it stands when
the next is read (decision 0143).
A number over the entities that match is read in one statement, which reads no projection. count takes the
criteria a query takes, and gives how many entities match them. sum takes an index and the same criteria, and
gives the total of that index over the entities that match, a double, 0 over none. A row whose index holds null,
or no value at all, adds nothing, as in SQL. A row whose index holds a string or a boolean is the caller's defect:
SQLite would read the string as 0 and true as 1, and the total would look right. The call dies, naming the
index and the least id of such a row. Both read the rows as they stand while reindex runs, as a query does
(decision 0148).
A sum is also the caller's defect when the numbers it adds, taken without their sign, may add up past the largest double. Each number without its sign is counted in units of 2^980, cut to an integer, and the sum is refused once those units and the count of numbers reach 2^44. Both dialects compute that count exactly, so both refuse the same sums, and below the bound no partial total overflows, in whatever order the rows are added (decision 0277). PostgreSQL adds the numbers exactly and rounds the total once. SQLite adds doubles with a compensated sum. So the two dialects may give totals that differ in the last digits, by a few units of the last place of the total without signs (decision 0209).
A count per value of an index is read in one statement too. countBy takes an index, the criteria a query takes
and a limit, and gives each value the index holds among the entities that match, with how many hold it, so every
count is read on one snapshot. The values come in the order a query orders the index by, null first, at most
limit of them, and a value no entity holds has no entry. true and 1 are two values, though a query orders a
boolean as its number, and a boolean comes after that number. An index a row lacks is null, as a query reads it.
A grouping by an index of many values, such as a customer, is a report's question past its limit. A sum is not
grouped (decision 0150).
The read model answers what the program's application reads to act, from the rows a step writes in its own
transaction, so its answers are consistent with the states. Its questions are declared: an index for each value a
question filters, orders or adds by, and a question added is a deploy and a reindex. It answers in four forms,
the entities whose indexes match, their count, the sum of one index over them, and their count per value of one
index. Any other question over the entities is a report: how many per day, the total per customer, the
entities whose status is one of several, a word in a text, an entity with what another program holds of it. A report is built from the log: a consumer of a
topic writes a store made for the question, a table of a relational database, a search index or a warehouse, and
the question is asked of that store. What such a consumer needs is in the projection, which is the entity's one
contract. A form the read model gains is one statement over the indexes, under the criteria a query takes, which
reads no projection (decision 0149,
decision 0150).
A descending order is the ascending order reversed, the ids of the rows that tie included, so the rows that hold no value come last. A page reads only the rows it gives, however late in the order its cursor is, as what the storage writes says.
reindex writes every row again from the stored states, which is what an index added or changed
by a new version of the program needs. It runs while entities step, and writes a row only if the state it was
computed from is still the one stored, in one statement: a step that committed since keeps the row it wrote
(decision 0070). An entity whose row the program
cannot compute keeps the row it had: its state is one no version reads, or a migration, a schema, project or an
index dies on it. reindex writes the rest, and gives how many it wrote and the entities it passed over, each with
the cause. What the runtime throws, a corrupted row's included, still fails the whole call
(decision 0122).
examples/order/runtime.test.ts asks for the orders still awaiting their
payment.
A queue, an event store or another service learns what an entity did from an outbox: a step that changes what callers read writes one row in its own transaction, in each topic of the log whose selection changed, holding the entity's address, the message the step handled and the value the topic selects of the projection it left. An entity's creation writes a row in every topic, with no message. Nothing more is declared: the message names the fact, and the projection is already the public contract (decision 0038).
A program may declare topics: by name, a pure function that selects, of the projection, what the topic
watches, as JSON (decision 0138). Different readers act on
different parts of what an entity shows: a billing service reads what an order was charged, a dashboard its
status. A topic per reader writes a row only when what that reader acts on changes, compared as canonical JSON by
the runtime, and a consumer of one topic reads none of the others' rows. A row gives the value its topic selects,
typed by the selector, so a consumer has a contract of its own: the row's publication stores the projection, which is read
through the projection's migrations, and the selector of the program deployed is applied to it where the row is
read (decision 0140). A step stores its message and its
projection once, in a publication keyed by its partition and its seq, and each of its rows names it. A prune
drops a publication with the last row that names it, so a publication stays while one topic has not read past it
(decision 0154). A selector that dies on a row gives
the defect to whoever reads it, as the schema does. A program that declares no topics publishes its projection as
the one topic projection, whose value is the projection whole. A topic selects and does not decide, as indexes
do: a part of the projection, the whole of it, all of it but a field, a value nested in it, or a comparison of it
with a threshold, which changes when the threshold is crossed. What it selects is JSON, as the projection is
(decision 0139). A topic has no schema and no migration of its own:
what a consumer needs is in the projection, and a boundary that answers a caller leaves out what the caller is not
shown. A topic that dies on a projection, or selects what JSON does not hold, suspends
the entity, as project does. examples/order publishes its status and its
charge.
The log is ordered within a partition, and promises no order across partitions
(decision 0137). Each entity belongs to one partition,
drawn from its id when it is created and stored with it. The partition is Math.abs(Hash.string(id) % partitions) + 1,
Cluster's formula for a shard. launch takes the number of partitions as log.partitions, 300 by default. The
first launch stores the number in the database, and a launch given another number fails, since a change would
move entities between partitions, and every consumer's cursors with them.
Each row carries its partition and a seq, which rises with every step that publishes in the partition and is
never reused. A step draws its seq once, by moving the head of its partition, in its own transaction, which holds
the head's row until it commits. Every row the step writes, one per topic, takes that seq, so a step takes one
lock however many topics it is written in, and a topic's seqs have gaps where a step was written in other topics
only. So the steps of one partition commit in the order of their seq, and the steps of different partitions
do not wait for each other. The rows of one entity therefore come in the order its steps committed. A row of one
entity written before a row of another may come after it, as on Kafka. A consumer that needs an effect after its
cause in another entity waits for the cause, or reads the cause's projection, which a row holds whole.
A consumer is a name within one topic, and it keeps a cursor per partition, stored: one name in two topics is two
consumers, each declared at deploy. fetch(topic, consumer, limit) gives at most limit rows of the topic past its
cursors, the oldest first. ack(topic, consumer, through) takes a list of partitions and seqs, which the rows fetch gave are, and
moves each of those cursors once the rows are acted on. A cursor never moves back, and
never past the head of its partition, so a crash before the ack delivers the rows again: at least once,
deduplicated by the topic, the partition and the seq. A consumer that must act on a row once keeps cursors of
its own, written in the transaction of its work, and drops a row delivered again behind them: the cursor it acks
then serves the prune alone, and may lag. A consumer whose store is a table of the runtime's database writes it
and acks in one transaction: ack joins a transaction the caller left open on the client the runtime was
launched with, so the store and the cursors commit together. Within that transaction the caller waits for nothing
a step does, since the transaction may hold the one connection of the database, as on SQLite and PGlite, and a
step's commit waits for it
(decision 0150). So a call that
runs a step or waits for one, within a transaction of the client, dies as the caller's defect: deploy, create,
send, resume, discard and a fetch that waits. The handler of consume runs within its page's transaction, and
a handler that must create or send stores the request for a fiber outside the transaction
(decision 0256). A partition a
consumer has no cursor for is read from 0.
prune drops, in each topic and partition, what every consumer of the topic has read there, through the lowest
cursor. A topic no consumer is stored for is dropped whole, since no reader that keeps cursors needs its rows
(decision 0152).
consumers says the topic of each consumer, whether the deploy declares it, how many rows are past its cursors,
and in which partitions. It says too, as since, the time of the step that wrote the oldest of the rows the consumer
reads next, or null where it has read every row. In each partition the row read next is the one of the lowest
seq past the cursor, since a step's time may run against the order its partition committed in. The time is the
clock of the process that wrote the step, so an age read against another clock may come out a little below zero.
The runtime records no metric of the age and runs nothing at an interval for it: an application that wants a gauge
reads consumers at an interval of its own
(decision 0224).
published(topic, after, limit) reads past cursors the caller holds, and registers nothing; [] is the whole
topic
(decision 0039).
fetch(topic, consumer, limit, { wait: { millis } }) waits where it finds no row: until a step commits a row in the topic,
and then reads again, for at most millis from the call. It gives the first page with a row, or none once the
time is over or the runtime stops. A step tells each topic it wrote in once its transaction has committed, and a
consumer of another topic is not woken. The reading takes its topic's signal before it reads, so that a commit
between its reading and its wait is not missed. A consumer that waits polls nothing, and a consumer that stops
calling costs the runtime nothing
(decision 0142). On Effect Cluster a reading that
waits is woken by another runner's commit too, as
Effect Cluster inside the runtime says. A partition a handle is
given is an integer from 1 to the number of partitions, a topic is one the program publishes, and a seq, a limit
and a wait are non-negative integers. The limit is required, of fetch and published as of query, so that no
reading holds a whole topic in one array; a caller that wants more reads again, past the last row it was given
(decision 0141). Anything else is its
caller's defect, which stores nothing
(decision 0053). So is a request or a query left
out: a call of the handle returns its effect, and the effect dies with the defect. The log is kept for its
consumers and is not an archive: a program whose past must stay readable keeps it where callers read it, in the
state or in entities of its own.
consume(topic, consumer, handler, { limit }) runs the loop of a consumer: it reads a page with fetch, runs
handler on it and acks it in one transaction of the client the runtime was launched with, and reads the next
page only once that one is acked. Without wait it ends at the first page that is empty, or once the runtime
stops, and the rows it did not read stay for the next reader. Given wait: true, a reading that finds no row waits for a commit in the topic with no time limit, and the reader ends only when the
runtime stops. A failure of handler or of the transaction rolls the page back and fails the reader, and the
caller retries it. A reader within a transaction of the client is the caller's defect, since its wait would hold
the connection the commit that ends it waits for
(decision 0184).
The runtime writes the log and runs a reader, and does not deliver the log elsewhere: the transport and the
retries are the consumer's.
A consumer whose work on a row is slow is read by several readers, each of which names the partitions it reads:
fetch(topic, consumer, limit, { partitions }) reads those partitions only, from the consumer's cursors there, and
so does a reader of consume given partitions
(decision 0153). The cursors stay the
consumer's, whichever reader acked them, so a prune reads them as one, and a reader added or removed moves no
cursor. A slow row holds only its reader's partitions. The handle gives the number of partitions as partitions.
The division is the caller's, and the runtime holds no reader. A fixed pool divides with a line, the reader i of
n reading the partitions p where p % n === i. A pool whose readers come and go divides with what tells it its
members, an orchestrator's ordinal or a lease of its own. Two readers that name one partition are given the same
rows, and their work on them interleaves. So a division that changes keeps the order within each partition only
where the readers of the old division stop, after their acks, before the readers of the new one start. A partition
no reader names is read by nobody, and holds its topic, which consumers shows. A wait ends at a commit in the
topic, so a reader woken by a commit in another reader's partition reads an empty page and waits again. A list of
partitions that is empty, or holds what is no partition of the log, is the caller's defect.
The consumers are declared where the program is deployed, by topic:
deploy(program, { consumers: { status: ['dashboard'] } }). Who reads the log is the deployment's, so the same
program is deployed with other consumers, or none, in a test. The handle is typed by the names declared, so a name
mistyped does not compile, and a name not declared is the caller's defect. The deploy stores each consumer it
declares that is not stored yet, so a consumer holds its topic from the deploy that first declares it, and a
reading writes nothing. A consumer stored and no longer declared is kept, and holds its topic: the deploy logs a
warning for it, consumers shows it as not declared, and forget drops it. A consumer the deploy declares is not
forgotten, since a deploy without it is how it goes
(decision 0152).
The runtime drops nothing of the log by time. A prune, by hand or by the sweep, drops only what every
stored consumer has read, so a consumer that stalls holds its topic under any sweep. A database that is full fails
the commit of every step, and each caller is given the defect, while the readings, the acks and forget still answer. An operator
finds in consumers the consumer that holds the log, deploys without it, forgets it and prunes, and the steps
commit again (decision 0152).
On a disk that is really full, a file of SQLite in WAL mode still commits a forget, and not a prune of many rows, which writes a page of the WAL for each page it changes. A prune in batches needs about as much room as one prune, since its first batch fails on the full volume too. The operator stops the callers first, since the steps they retry take the room freed within a second. The operator then frees room on the volume, about a twentieth of what the stalled consumer holds, and room for the steps the runtime takes of its own: the timers due, the operations stored and the steps a crash took out. Then the operator deploys without the consumer, forgets it, prunes, and lets the callers in again. The runtime keeps no room of its own: it does not hold the volume, and a file of reserved room is the operator's to keep and drop (decision 0223, decision 0245, decision 0278).
The runtime sweeps each program it deploys at an interval: it runs expire, then prune. launch takes sweep,
as { millis }, every 300 000 ms by default, or { millis, prune: false } to leave the log to an operator's
prune, or 'none' to leave both to an operator. The first sweep runs one interval after the deploy. A sweep that
fails logs a warning with its cause, and the next one runs at the next interval, while the deployment goes on. A
sweep that is neither 'none' nor a positive integer of millis, with prune left out or false, is the
launcher's defect. On Effect Cluster one runner sweeps a program, as a singleton of Cluster named
@fgaudo/effect-tea-persistent/<program>/sweep, so that runners do not contend for the same rows.
A sweep changes no answer. It deletes only the replies past their keep, the replies and the answers kept for work
of another entity that has ended, and the rows of the log that every stored consumer has read. The step of the work
that ends deletes those replies and answers itself, and a sweep takes those it leaves where it runs beside the step
that stores them, on a pool of PostgreSQL
(decision 0289). A topic no consumer is stored for keeps no rows under a sweep, so a deployment that wants a
topic's history declares its consumer before the topic's first step, or sweeps with prune: false. forget stays
an operator's call. SQLite gives no room back to the volume once rows are deleted, so a sweep bounds the growth of
the file and shrinks nothing.
prune and expire delete in batches of 256 rows, read in the order of an index, and repeat until a batch deletes
nothing. A batch skips the rows a step or another batch holds
(decisions 0285,
0288). Each batch is a transaction of its own. A short transaction holds SQLite's one writer for less time, so a
sweep does not stall the steps. The count each gives is the sum of its batches. A batch of prune reads the cursors
in its own transaction, and a batch of expire on Cluster deletes Cluster's rows under each id it takes. The runtime
calls neither Cluster's deleteProcessed nor its MessageRetention, since neither spares an operation's request
ids kept under its work, nor a caller's rows its client has not read
(decision 0245).
A step that changes the state without changing what callers read publishes nothing, and neither does a
rejected message, an internal message dropped, or a step that suspends the entity. The booking's inventory
shows what that costs: confirming seats moves them from held to sold and leaves available where it was, so the
sale is not published. A program whose fact must reach other systems shows it to callers, where a topic selects
it.
A deploy publishes nothing of itself, and reindex writes the read model again without publishing. A step is
written in a topic when what the topic selects changed in that step, as the program deployed computes the
projection before and after: a new project reaches the log at an entity's next step that changes what a topic
selects, and a
consumer that needs every entity as the new program shows it reads them through query, once reindex has run.
A projection whose schema changed is read through its migrations in the log too
(decision 0072,
decision 0138).
Most of a program is pure, so most tests need no runtime:
update's rules, one test per message, for generated states;update: every action it shows is accepted, and every accepted action is
shown. projectionViolations checks it, as effect-tea's dispatchViolations checks a view, for a state and
the messages its readers could send, once the test says which field of the projection stands for which message,
and for which sender
(decision 0026,
decision 0213). The test is given the projection as
a caller reads it, decoded from its JSON, so that -0 is 0
(decision 0276).
examples/order/order.test.ts checks it for generated states;migrationViolations checks it, and names a migration no case records
(decision 0126);rehearse runs a scenario on the program without storage, cluster or clock. It is the runtime's machine, which
handles the mailbox to its end after each act of the test. Time stands at the start's, and moves to a timer's
time when it fires, and forward by a duration with advance. The test decides when each operation answers, when
time passes, and when the process dies:
send delivers an inbound message, and returns the reply; sendFrom delivers one from another entity. The
request comes last, as send(message, { sender, requestId }). Given a request id, it is a request the test
names: a second send under the same sender and request id is a retry, given the stored reply and not applied
again, and the reply is kept through a restart and a deploy. Without one, each send is a new request
(decision 0200). A second send under the same
sender and request id that carries another message is answered Reused, and is not applied
(decision 0269). A program whose sender is
null may leave the sender out, and the request with it, as send(message)
(decision 0245);pending(key) finds the work under way under one of the program's keys, with its idempotency key and its
value, which a test asserts as it asserts a command in effect-tea. A key with a parameter takes it too:
pending('charge', 'line-7'). It fails the test when there is none, and underWay(key) gives null instead;answer(pending, message) completes an operation, and says whether the message was delivered or dropped. It
fails the test when the step that handles the message returns the same operation under its key, since that
operation would run again at once, turn after turn, with no pause
(decision 0130);fire(pending) fires a timer;projection gives the projection as a caller of the runtime reads it: encoded by its schema, written as JSON and
decoded again, so that -0 is 0
(decision 0211). state gives the
state as the entity holds it;advance(duration) moves time forward, and fires each timer due within it at its time, in the order of the
times. A message after it is handled at the time reached. A world advances the one time of all its entities
(decision 0187). Time moves by whole milliseconds, as
the runtime's clock reads it: a duration between two, as Duration.micros(1500n), fails the test, and so does a
start time between two (decision 0281). A repeating timer armed from its own
last due time catches up after a downtime: it fires once for each period missed, at once, as the runtime fires it.
At one time, a key fires again only with a due time later than the one it last fired with. A key armed again at a
due time no later would fire at once, turn after turn, and fails the test
(decision 0266);answer, fire and a member's run take a key in place of the pending, with its parameter for a key that takes
one: answer('charge', message) acts on the work under way under the key, in its current incarnation. A test
keeps a pending to act on an incarnation that has moved on, and sees its message dropped
(decision 0186);restart drops everything that is not stored, and activates the entity again: an at-most-once operation
under way delivers its unknown message. It rehearses a passivation too, which is the same thing — on an
entity with only timers under way, where nothing delivers an unknown
(decision 0029);deploy(program) restarts it with a new version of the program, which keeps the work under way it returns
the same and starts the rest as new work. One that changes, drops or delivers otherwise an operation under way
fails the test, as a defect.examples/order/scenarios.test.ts shows them, and
src/rehearse.test.ts rehearses deploys of the order.
A migration is a pure function on JSON, and rehearse encodes what a step stores only to check it
(decision 0040). It reads back only the
projection, through no migration, so a
rehearsed deploy does not exercise one: the runtime's tests deploy a version whose state, operation and
projection schemas changed.
rehearse runs one entity, and the test answers for the entities it talks to. So two programs that disagree on
what a state accepts pass both rehearsals. rehearse.together(at, scenario) runs a world of entities on the
machine, with one time and one process for all of them
(decision 0131):
world.create(program, id, flags) creates an entity, and gives a member of the world: the surface of rehearse
without restart and deploy, and with run. A world runs one program under each name, as a process does, so a
creation by another program under a name the world runs fails the test. The flags come last, and a program whose
flags are null leaves them out, as the start of rehearse does
(decision 0245);run(pending) runs the operation's runner, with Entities provided by the world and the other services by the
scenario, and completes the operation with its message, as answer does. A run asks for every service of its
program's runners, since a key may hold any of the program's operations. A test provides a service it does not
run with Layer.mock(Service, {}), whose members die when used, so a runner the test meant to answer fails the
test if it runs (decision 0189);send
returns. The world keeps the reply under the run's request while the work that asked is stored, so a run after a
restart is given the stored reply. A receiver that does not exist answers NotFound, and create creates an
entity in the world. A NotFound and a creation init rejects are kept for the run's work too, as the runtime
keeps them (decision 0253).
A create through a reference fails the test where the world runs no program of its name, since a reference
brings no code to run (decision 0183);run(pending, { crash: true }), or run(key, { crash: true }), runs the runner to its end, drops its message,
and restarts the world: the process died between the replies and the completion;world.restart restarts every entity, and world.deploy(program) restarts them with a new version of one
program. world.advance(duration) moves the one time of the world, and a member has no advance of its own
(decision 0187).What the runtime makes a defect fails the test: a run that sends twice under one name to one entity, a droppable
operation that sends, and a defect of the sender or of the receiver. A suspended receiver is never reached, since
its defect has failed the test first. A world does not choose the order of steps of the entities: each act of the
test runs to its end before the next, which is one order the runtime can take.
examples/booking/scenarios.test.ts rehearses the booking and the inventory
together.
How the library tests its own machine and runtime is under how the machine and the runtime are tested.
| effect-tea | here | why |
|---|---|---|
| commands, started by a step | none: operations are derived from the state | a command started before a crash would be lost |
| subscriptions, by key | operations and timers, by key, with incarnations | work outlives the process, and its answers are delivered at least once |
| a subscription whose value changes starts anew | an operation whose value changes while it runs is a defect | an operation cannot be interrupted, and a crash would restart it changed |
| work that should stop is a subscription | a droppable operation, which acts on nothing | only work with no act in the world can be cut halfway, and run again |
| an ended subscription waits for its key to leave | a key kept after its work delivered is new work | kept, it would not run again until a crash, which starts it anew |
| a subscription listens to a service's connection | the boundary reads the stream, and sends each item as a message | a listening dies with the process, and a sent item is stored |
| a model, with the messages the view sends | a projection, JSON, with the actions it allows | a caller is remote: it receives data, never functions |
| a defect ends the run | a defect suspends the entity | one entity's bug must not stop the others |
init always starts the program |
a program's init may reject the flags, and a part's may not |
a caller's flags may break a rule of the domain |
| one program, a tree of children | many entities, by id, each a tree of parts in one state | an entity is the unit of storage and of concurrency |
| a child is any program | a child is a part: no name, indexes or migrations | the entity, its row and its versions are the parent's |
| external messages are among a child's messages | external messages are no inbound ones | a caller may send an inbound message, and only a parent an external one |
each takes an optic by id, and subscribed |
each takes one optic over the entries |
every child there holds work, so a step lists them all |
| a parent minds ids that come back | an id that comes back has new incarnations | the machine drops an answer of an earlier run |
the ids of children, compared with Equal |
the parameters of a key, compared by their canonical JSON | the machine compares work by what is stored, and Equal is not stored |
| no persistence | the state after each message | the state is the data |
On one process, with SQLite or PostgreSQL: the mailbox, the states and the timers sit in one database, so each step
is one transaction (decisions 0007, 0157). A runtime holds its
database until it ends: a second runtime on it dies as it launches
(decision 0080). The hold refuses no reader, on SQLite as on a
server of PostgreSQL. A file of SQLite is held by a file of its own beside it, named by the database's path and
-effecttea-hold, which the runtime's connection attaches in SQLite's exclusive locking mode. An exclusive
transaction that writes no row takes the hold file's lock, so that a launch on a full volume takes it. The database
itself stays in the normal locking mode, so another connection reads it while the runtime runs, and the end of the
process gives the hold back however it ends
(decision 0267). A database of
PostgreSQL is held by a session advisory lock, on a connection the runtime reserves until it ends. A database of
PGlite is held in the process by its client. Its data directory is held too, while the client a runtime launched on
is open, so a launch on another client over the directory dies
(decision 0255). A data directory of
PGlite opened by another process is the launcher's defect, which the runtime does not see. Once it holds the
database, a launch runs the migrations of the tables not yet recorded in effecttea_migrations, with Effect's
Migrator (decision 0216). The first migration is the schema of
the first release, and a later change of a table is a migration after it. Migrator runs the pending migrations in
one transaction, so that two launches on one database run each migration once. A launch given tables: 'existing'
runs no migration, for a role of PostgreSQL that may only read and write rows, and an operator runs createTables
under the tables' owner before each new release. On
Cluster the runners share the database, and sharding creates the tables before any runner registers
(decision 0191). launch gives a runtime, deploy(program)
a handle to one program's entities — create, send, read, and suspended, inspect, resume and discard for an
operator — and
the operations of a program are run with the services it was deployed with
(src/runtime.ts, decision 0028).
The storage speaks SQLite and PostgreSQL, in the same tables, with the same columns, and a statement both read alike is written once. A launch on a client of any other dialect is the launcher's defect (decision 0157).
READ COMMITTED, where a step that draws its seq behind another of
its partition waits for it, and a higher level would fail it. A reading of one snapshot begins at REPEATABLE READ
(decision 0222). The migrations run in Migrator's
transaction, at the level the role sets. Any level holds, since Migrator locks its table before it reads the
history (decision 0225).
createTables dies within a transaction of its client, where the migrations' transaction would be a savepoint
begun after the caller's snapshot
(decision 0279).REPEATABLE READ on PostgreSQL where the
transaction is the reading's own: a page of a query, inspect, suspended, and the reading of each entity by
check. So a page gives a row once though a step moves it between its parts, and inspect reads a step whole or
not at all.READ COMMITTED each statement reads a snapshot of its own, so a step committed between two parts gives a row of a
page twice, and inspect may read half of a step. A caller that runs one of the four readings in its own
transaction, and needs one snapshot, begins that transaction at REPEATABLE READ. A test of a page on a server
holds this, a page read in a caller’s transaction gives a row twice at READ COMMITTED, and once at REPEATABLE READ of src/postgres.test.ts, and the four readings begin their transaction alike
(decisions 0158,
0185,
0222).@effect/sql-pg a bigint. Either is read as a number up to 2^53
(decision 0222).reindex page the entities from the empty id up, so an entity under it
would never be woken nor indexed. A handle, an operation and a rehearsal refuse it as the defect of whoever gave it
(decision 0179).create and send, and a rehearsal's send,
refuse one that is no text a database holds as the caller's defect, since the storage writes it as it is in
request_id (decision 0204).search_path is set by ALTER ROLE, and two runs of createTables at once
(decision 0222). A runtime whose session the server ends loses
its hold without knowing it, which is an open question.An entity with nothing to do is unloaded, and the earliest of its stored timers wakes it
(decision 0029); how long it stays loaded is an option of the
runtime. A launch checks each of its options, and one that is no number of the kind it names is the launcher's
defect. Only an option left out, or a field of it left out, takes the default, and a null is refused.
retry, passivate and warn are integers, since a time becomes the text of a duration, and
Effect reads no Infinity millis and no NaN millis
(decision 0208). retry and passivate are a millisecond at
least, since the waker, a sender, Cluster's round and an entity's wait each try again when their sleep ends, and
under 0 each would try again at once, for ever. warn may be 0
(decision 0251). A timer fires once its time has come on the
wall clock, and never before. A host suspended while a timer sleeps delays it by an hour at most once the host runs
again, as fibers and locks says. An event the runtime cannot finish — a write that fails — crashes its entity alone: it leaves the
process, its callers are given the defect, and the next one loads it from what is stored
(decision 0042); nothing else waits
for its operations to stop (decision 0056).
The entity is not loaded again in that process while the stop of its work runs. A caller and a door's step are given
a defect that says so, to try again, and a stop that outlives retry is logged once. The waker leaves the entity out
of its readings until the stop ends, and the stop's end pokes it. The stop is work of the process, so settled does not
return while it runs. On Cluster a shard lost is let go only once every stop of its entities has ended. So the runner
that gains the shard runs no entity beside a fiber of its former run. A run whose entity crashed sends nothing: its
operations' send and create die, and one already waiting ends with the defect, so that no stop waits for a load
or a shard its own stop holds back. A droppable operation dropped before the crash is the exception. Its stop is not
the crash's, and its finalizer may still run beside the next run, as it may beside the next step without a crash
(decision 0274).
An activation the waker cannot finish crashes its entity alone too, and the waker tries it again after retry
(decision 0077). The waker's own reading
of the timers goes on alike: a stored time it refuses leaves its entity out until retry, and a reading that fails is
read again after retry, each logged once
(decision 0259). Every crash does the
same, whatever the entity's timers: the waker activates the entity again after retry, so that an operation or a
message it stored runs with no caller coming. A crash that hands a shard to another runner is the one exception. On
Cluster an entity is not unloaded while a door's step is not known to be committed, and a door's step that loaded
the entity and rolled back leaves it to be crashed and loaded again by the next step
(decision 0252).
Everything a deployment runs runs in a scope of its own: a deploy that cannot finish before it registers its deployment, interrupted included, stops what it started, is never seen by other programs, and gives its name back (decisions 0051, 0102); one interrupted after keeps its name, with its deployment and its waker (decision 0078). A process deploys a name once: a new version, a fix included, is deployed by a new process (decision 0075).
A process launched with drain: { millis } drains its at-most-once operations when the scope it was launched in
closes, so that a deploy does not deliver unknown for each one under way. During the drain, the runtime takes no
caller's message, creation, resume or discard: a caller that arrives waits, is interrupted when the drain ends, and
its retry is answered by the next process. The timers, the waker, and the operations that run again at the next
activation stop at once, and a step of the drain starts no such operation and arms no timer. The drain waits, up to
millis, for the at-most-once operations under way and the commits of their completions, for an at-most-once
operation such a step starts, and for a creation committed whose work has not started. Whatever is under way after
millis stops as in a crash. A process launched without the option ends with a crash, as it always did
(decision 0119). The drain reaches a
deploy still activating as it reaches a deployment registered: the deploy activates no entity once the drain began,
and registers with no waker running, and the next process activates the entities left
(decision 0261). The runtime closes a sequential
scope of its own, so that the drain runs first and the database is let go last whatever the strategy of the scope it
was launched in (decision 0120). The drain
holds whether that scope is closed by Scope.close or by the interruption of the fiber that launched the runtime, as
runMain stops a process on a signal. On Cluster it ends before the runner lets its shards go, when the runtime is
launched in a layer built over the runner's Sharding
(decision 0219).
A signal often reaches a process while its main fiber still deploys. The interruption then reaches the deploy before
the drain begins, and the deploy undoes itself. With a drain, the undo of a deploy drains its own deployment first: it
stops the deployment's timers, waker and operations that run again, and waits up to millis for its at-most-once
operations, then closes the deployment's scope. The same holds of a deploy that fails while the process goes on, and
the process's other callers are not held meanwhile. So a stop during a deploy may wait millis twice, once in the
undo and once in the drain
(decision 0280).
The runtime records spans and metrics through Effect's Tracer and Metric, which an application exports with
OtlpTracer, OtlpMetrics or PrometheusMetrics of effect/observability
(decision 0220). Each event the
machine steps is a span tea.step, with the entity's tea.entity.name and tea.entity.id, the event's tag as
tea.event, and the tags of what the step set going as tea.outputs. A completion, a firing and a defect of work
name its tea.key and its tea.incarnation. A step that suspends its entity has tea.suspended, and says in
tea.suspension.on what the entity died on. The run of an operation is a span tea.operation, with its tag and its
delivery. Every span of the runtime is a root span. The step of a caller's request links the caller's span, and an
operation links the step that started it, so a trace follows a request into the work it set going by links. Three
metrics count by program: tea.mailbox.length, a histogram of the mailbox each step that writes leaves,
tea.suspensions, the steps that suspend an entity, and tea.operations.running, the operations running in the
process. No attribute holds a message, a state or a projection.
On Effect Cluster, each runner runs the runtime beside Cluster's tables, in one SQLite file that every runner
shares, on one machine. sharding(programs) builds a runner's Sharding, and launch of the entry on Cluster gives its runtime,
whose deploy serves each program through the program's door
(decisions 0081,
0159). A step commits
only where its commit checks the shard. On Cluster that is SQLite, until Cluster has a fence, so the runtime does
not run on Cluster across machines yet
(decision 0084).
examples/order/cluster.test.ts shows a runner inside a test's process.
examples/server runs the orders as a deployment does: a process that serves them over HTTP,
killed during a charge and started again on the same file, and two runners on one file, of which the one that survives
finishes the orders of the one killed, or of the one stopped by SIGTERM
(decision 0178). What Cluster gives and what
it leaves to the runtime is in cluster.md.
sharding records the version of each program the runner names, before Cluster registers the runner. A launch
older than the newest version recorded dies, so a rollback is refused.RunnerHealth the runtime gives Cluster, only while it runs the newest version of
every program it names, and while the launcher's own RunnerHealth says so. A rolling deploy then moves every
shard to the newer runners.cluster_locks, names its
runner and has not expired, which it reads in its transaction too. The condition is the exact complement of the
one under which Cluster lets another runner take the lease
(decision 0175).retry, until a runner of the newest version that
holds the shard steps it. The checks are a guard beside the count of the steps that wrote the entity's row.sharding named its program at the version deployed. A process with no
runnerAddress is a client of the cluster alone: it holds no shard, and deploys any program
(decision 0174).How a door steps an entity, how a runner holds its shards, and how the runtime deletes Cluster's rows is under Effect Cluster inside the runtime.
create and send, an operator's resume and discard, and an operation's requests go through
Cluster's client of the door, from whatever runner they are on. A message answered Suspended or NotFound is
reset once and sent again, and a caller is never answered Full
(decisions 0165,
0166).Suspended registers for the receiver's resumption before it sends. The step that resumes or
discards the receiver tells Resumed to the entities whose operations it answered Suspended
(decisions 0167,
0172).The library exports what a program, a test and an operator use, and nothing the runtime alone uses: the machine, the storage and their helpers may change without a note, and a runtime on another store imports them from their modules, as the tests do (decision 0055).
The surface is four entries of the package @fgaudo/effect-tea-persistent
(decisions 0193,
0195):
., src/index.ts, gives what a program, its boundary and an operator use on one process;./program, src/index.program.ts, gives what declares a program and nothing that
runs it, so that a browser imports a program's module for its schemas, and . gives it too;./testing, src/index.testing.ts, gives rehearse with the replies
accepted and rejected, projectionViolations, migrationViolations and digestOf, for a user's tests;./cluster, src/index.cluster.ts, gives sharding and launch, for a launcher on
Effect Cluster, with Recorded as a type alone.A process on one machine imports neither Cluster nor the checks of a test, and a browser reaches no module of Node
through ./program. A test of the form of src/ checks both.
The library gives no map of an answer to a status of a protocol, since the status is the boundary's choice, and a
boundary may not speak HTTP. A boundary matches on the _tag of Creation, Answer and Operated, with one
entry for each tag that the compiler checks, so that a tag the library adds fails to compile in the boundary.
examples/server/server.ts is the template over HTTP
(decision 0190).
This section says how the runtime keeps the contract of the sections before it. A program, its boundary and its tests rely on none of it. It is for whoever changes the runtime, the storage or the tests of the library.
What an event writes is the machine's: besides the next machine, a transition gives the rows it writes, already encoded — the state when it changed, the keys that changed, the messages that leave the mailbox and the ones queued, the replies added, the work that ended, the read model's row and the outbox's rows — and the storage writes them and decides nothing. Each value is encoded once, and the projection of a state computed once (decision 0057).
A reply to another entity is deleted in the transaction that ends the work that asked for it, which the machine names, or in the one that stores it, if that work has already ended: only the storage sees another entity's keys (decision 0021).
An answer no entity stores, given to another entity's operation, lies in a table of its own, effecttea_answers, and
is deleted as a reply to another entity is. A row is keyed by the receiver, the request's text, and whether the
request was a creation or a message, since an operation may give both one name
(decision 0253).
The machine holds no reply. A message that arrives is given the reply stored under its request, which the runtime reads by the request just before the step, where no other event of the entity is applied; an activation reads none, and what a loaded entity holds grows with its mailbox and its keys only. A deletion changes no machine, so only a step changes one (decision 0065).
A reply's row is keyed by its request's text and by the version that wrote it: a migration of the sender may give one caller's request the text another's had, and its reply then replaces nothing that other caller's retry reads (decision 0069).
The cause of a suspension is stored as text, and holds the program's data wherever defect or an error of a schema
cites it. So each U+0000 and each lone surrogate in it is written as the escape JSON gives it, such as \u0000,
which every database holds
(decision 0205).
An index that is an integer beyond 2^53 is written with the exact digits of its double, so that the database reads
the number the query compares: the digits JSON.stringify writes spell another integer to SQLite and to PostgreSQL
(decision 0146).
The storage indexes each index of the program by its value and then the id, and reads a page as at most three ranges of that index, in one transaction: the rows that tie with the cursor past its id, the rows past its value, and, in a descending order, the rows that hold no value (decision 0145).
The two dialects hold the same rows in the same columns:
json_extract
does: its kind, its number, and its string, by its bytes. Text of PostgreSQL is compared by its bytes, so ids are
ordered alike too.request_id
(decision 0158).A timer sleeps by Effect.sleep for an hour at most at a time, and reads the wall clock after each sleep. A loaded
entity's timer sleeps again for the time left until the time it stores has come, and the waker reads the stored
timers again. So a wall clock set back fires no timer early, and a host suspended delays one by an hour at most
(decision 0221). An hour is shorter
than the host's limit of 2^31 − 1 milliseconds, past which Effect sets a time as timers of the host in turn
(decision 0194).
Each entity is found, loaded, unloaded and crashed under a lock of its own, so the callers of one entity never
wait on another's activation (decision 0067). An
activation and a creation run detached from their caller, which only waits for them: one that gives up leaves
them to finish, and only a crash of the process stops them halfway
(decision 0079). What runs detached is
counted until it ends, however it ends, by a fiber forked with it
(decision 0110); a fiber forked from
the runtime says it is interruptible, wherever it is forked and whatever Effect's default, so that its scope stops it
(decision 0271).
Bookkeeping another fiber reads changes in one synchronous step, never across a yield: a fiber is held only while
it has not ended, and an entity leaves the process in one step
(decision 0068). That bookkeeping stays in
plain values, changed in one Effect.sync, and in none of Effect's Tx modules: each wait of the runtime depends on
I/O or on the time, which no transaction waits for, and an entity's lock stays a Semaphore, which serves its waiters
in order (decision 0177). An idle entity's wait
ends with an item a sleep puts in the entity's queue, and nothing races its take, so an event offered as the wait
runs out is not lost (decision 0116).
Every fork of the runtime runs without the transaction of the fiber that forks it, through forkApart or
forkChildApart, which also make it interruptible, and a count another fiber reads
is raised and lowered by one Effect.acquireUseRelease, unless a comment says why two fibers raise and lower it. A
test of the suite reads the code and fails on any other form
(decision 0164).
An entity's loop yields after a step that leaves it more to do, so that another entity waits for one step of a busy
one at most. Effect yields a fiber after a count of operations, and a step whose encoding and write take long is few
operations. A loop with nothing to do starts its wait at once
(decision 0134).
Create, Message, Resume,
Discard and Resumed. Each is persisted, and its handler steps the runtime's entity in Cluster's transaction, so
a step commits with its reply (decision 0081). A
rename (decision 0156) moves none of
Cluster's rows, which are stored under the entity's type. The program's door is registered under each old name
too, and steps what Cluster saved there (decision 0176).requestKey. Cluster answers a retry within a version, and
the runtime's replies answer one across versions
(decision 0085).Died with its cause. The creator's client resets
a Died Cluster saved before it sends, and gives it as the creator's defect, as one process does: a caller is
given the defect, an operation's sender is suspended, and a retry runs init again
(decision 0291).maxIdleTime is the runtime's passivation,
and Cluster's maxResidentEntities is left unbounded
(decision 0092).A runner learns of another runner's commit from the file: while a reading of the log waits, it reads SQLite's
data_version at Cluster's interval of replies, and wakes every reading that waits where the version moved
(decision 0163).
keep
is past, by a step of the entity or by Handle.expire
(decision 0171).
A message sent with keep: 'forever' keeps its rows for ever. A creation's client keeps its id as a keep of 0
does, and a retry after its rows go is answered Created from the entity's row
(decision 0245).keep, a creation's included. Another client of the same request, on another runner, reads the reply
from Cluster's table only at its poll, and would wait for ever once the rows are gone
(decision 0248).The machine is tested with properties, under random orders of messages, completions, timers, crashes and
defects (src/machine.test.ts). The runtime is tested against it
(src/runtime/loop.test.ts): a scenario, and then random sequences of events, are driven
on the real runtime with the process dying at each point of the table of
what survives a crash, and what the database holds
afterwards is compared with where the machine ends on the events that committed. They run on Effect's
TestClock, and move time instead of waiting for it
(decision 0066). A moved clock gives a fiber it did not wake
one turn, so a test that moves the points where fibers yield asserts what holds once the clock has passed, not
when (decision 0101); a test may run
the wall clock ahead of the sleeps, to stand for a clock that moved between two steps
(decision 0103). The runtime's tests choose
where fibers yield: a scheduler of the tests' answers Effect's Scheduler from a schedule it records, so a run
that fails is replayed exactly, and explores by random priorities over many seeds, or by delay bounding around one
point; none runs under budgets of steps
(decision 0104). The scheduler preempts a
fiber from the second step of a run, where Effect yields under a budget of 2, and lets it through at the task of the
yield it made
(decision 0244). A fiber that began
its run on another fiber's stack, started or resumed inline, is asked whether it yields even with no task ready, so
that a run reaches the orders in which Effect's budgets give a parent back before its child's run goes on
(decision 0117). PCT finds a bug of depth
d with a probability of at least 1 / (n k^(d - 1)), for n fibers and k steps; the explored tests have 3 to 48
fibers and 79 to 3,584 steps in their window, so no number of seeds guarantees a bug of depth 2, but the bound is the
worst case, and a bug most schedules reach is found in a few runs. A run of the tests runs one seed in twenty of each
explored test, and a release every one, as AGENTS.md says; a schedule is drawn from its seed alone,
so a failure in either names a seed that replays it in both. What the runtime did is told
to a probe, a reference in the context with a default that does nothing, which a test provides and the library
does not export (decision 0073). A test waits for the
process to settle, with nothing under way, an entity unloading included
(decision 0109). The machine's properties check
that what an event writes, applied to the rows before it, gives the rows after it, and
src/storage.test.ts that the storage writes those rows as given
(decision 0057).
The tests of src/runtime/callers.test.ts under two entities that talk run the booking and
the inventory of src/fixtures/ on it, on the test's clock, with a probe and values frozen through.
examples/booking/runtime.test.ts runs the example's booking on real time,
through the public surface alone.