diff --git a/README.md b/README.md index 0ff6a28..efcf40b 100644 --- a/README.md +++ b/README.md @@ -21,15 +21,15 @@ read-decide-append flow safely. Backends store events. -Materialized views are not stored by Factos itself. A `factos.View` is an -in-memory fold over events. If you want a durable read model, your application -stores the result wherever it wants: PostgreSQL tables, Redis, SQLite, files, or -something else. +Materialized views are not stored by Factos itself. A projection is ordinary +application code, often a pure fold over events. If you want a durable read +model, your application stores the result wherever it wants: PostgreSQL tables, +Redis, SQLite, files, or something else. -Reactors are also not stored or executed by Factos. A `factos.Reactor` maps -committed event records to application-owned effect values. Your application -chooses whether to run those effects immediately, persist them in an outbox, -retry them, or ignore them during replay. +Effect derivation is also ordinary application code. A pure function can map +recorded event envelopes to application-owned effect values. Your application +chooses whether to persist those values atomically, run best-effort work after +commit, retry delivery, or ignore effects during replay. So the durable state provided by the backend is: @@ -40,16 +40,26 @@ Everything else is application state built from that log. ## What does `factos` provide? -The core package is store-independent. It provides the types and pure functions -used by backends and applications: +The core package is store-independent. It provides the shared types and +functions used by backends and applications: - `Decider`: your command decision logic as pure data. -- `Query`: the event types and tags needed for a decision. +- `DecisionContext`: the recorded facts a command must consider. - `Context`: previously stored events folded into decision state. - `AppendCondition`: the condition a backend must protect before appending. -- `Recorded`: a stored event plus backend metadata. -- `View`: an in-memory projection fold. -- `Reactor`: a pure mapping from committed records to effect values. +- `factos.EventCodec(event, backend_payload)`: application encoding and decoding + for a backend payload. +- `factos.Event(payload)`: an event proposed for persistence with its descriptor. +- `factos.Recorded(payload)`: a stored event plus backend metadata. +- `factos.DispatchBuilder`: store-independent dispatch configuration. +- `factos.Subscription`: a dispatch-bound application callback and consistency + mode. + +Applications create these values with shared functions such as `factos.codec`, +`factos.new_event`, `factos.new_dispatch`, `factos.new_subscription`, and +`factos.with_subscriptions`. The consistency modes and dispatch error +constructors are shared as well. A concrete backend executes the dispatch +builder against storage and formats its store-specific errors. A decider has this shape: @@ -65,9 +75,9 @@ factos.decider( ```gleam fn evolve(state: State, event: Event) -> State { - let TicketWindow(capacity, sold) = state + let TicketWindow(capacity:, sold:) = state case event { - TicketSold(_) -> TicketWindow(capacity: capacity, sold: sold + 1) + TicketSold(buyer: _) -> TicketWindow(capacity:, sold: sold + 1) } } ``` @@ -77,25 +87,27 @@ returns new events: ```gleam fn decide(state: State, command: Command) -> Result(List(Event), DomainError) { - let TicketWindow(capacity, sold) = state + let TicketWindow(capacity:, sold:) = state case command { - BuyTicket(buyer) -> + BuyTicket(buyer:) -> case sold < capacity { - True -> Ok([TicketSold(buyer)]) - False -> Error(SoldOut(capacity)) + True -> Ok([TicketSold(buyer:)]) + False -> Error(SoldOut(capacity:)) } } } ``` -You can test this without any database: +You can test this without any database by destructuring the decider and folding +the relevant history directly: ```gleam -factos.compute_events( - decider: ticket_decider(), - events: [TicketSold("renata")], - command: BuyTicket("lucy"), -) +let factos.Decider(initial:, decide:, evolve:) = ticket_decider() +let state = + [TicketSold(buyer: "renata")] + |> list.fold(initial, evolve) + +decide(state, BuyTicket(buyer: "lucy")) ``` ## Events, commands, and command sourcing @@ -103,30 +115,41 @@ factos.compute_events( Factos stores events: facts that were accepted by the application. A backend row is an event record, not a command record. -The core package also provides command-handling helpers (`Decider`, `Context`, and -`dispatch` functions in the backends). Those helpers are an opinionated way to -build command processing on top of an event log: +The core package also provides command-handling helpers (`Decider`, `Context`, +and the builder created by `factos.new_dispatch`). A concrete backend's +`dispatch` function executes that shared builder. Together they provide an +opinionated way to build command processing on top of an event log: ```text command + relevant previous events -> accepted new events or domain error ``` If you want lower-level event sourcing, you can use the same stored event log, -codecs, views, and reads without treating Factos as a complete command framework. -The command-dispatch path is a convenience for applications that want that -standard shape. +shared codecs, projection folds, and reads without treating Factos as a complete +command framework. The command-dispatch path is a convenience for applications +that want that standard shape. + +## Decision contexts and tags -## Queries and tags +Every dispatch builder requires a `DecisionContext`: the facts that can change +the command's answer and must therefore remain stable until append. -Backends do not understand your event payload bytes. If a command needs to find -facts by a payload value, write that value as a tag. +Use the explicit variant that matches the rule: + +- `factos.NoContext` reads no history and permits an unconditional append; +- `factos.AllEvents` reads and protects the complete event log; +- `factos.Matching(items:)` selects facts by event type and tags. + +Backends do not inspect your event payload when selecting a context. If a +command must find facts by a payload value, expose that value as a tag when +recording the event. For a ticket-sale capacity rule: ```gleam -fn sale_query() -> factos.Query { - factos.query([ - factos.query_item( +fn sale_context() -> factos.DecisionContext { + factos.Matching(items: [ + factos.item( types: [factos.event_type("TicketSold")], tags: [factos.tag("event:gleamconf-2026")], ), @@ -137,18 +160,22 @@ fn sale_query() -> factos.Query { This tells the backend: "read the accepted ticket-sale facts for this event and protect that same context before appending more ticket sales". -Query semantics are small: +Matching semantics are deliberately small: -- query items are OR-combined; +- items are OR-combined; - event types inside one item are OR-combined; - tags inside one item are AND-combined; - empty event types match any event type; -- empty tags add no tag constraint. +- empty tags add no tag constraint; +- `Matching(items: [])` matches no events. + +Prefer `NoContext` over an empty `Matching` value when a command intentionally +does not depend on history. The explicit variant preserves that design decision. ## Simulate domain scenarios -`factos/simulate` runs stateful domain scenarios against the core query, context, -append-condition, and decider semantics without a database: +`factos/simulate` runs stateful domain scenarios against the same decision +context, append-condition, and decider semantics without a database: ```gleam import factos/simulate @@ -167,16 +194,12 @@ fn describe_event(event: Event) -> factos.EventDescriptor { let store = simulate.new(describe_event) - |> simulate.given( - stream: "ticket-sales", - events: [TicketSold(buyer: "renata")], - ) + |> simulate.given(events: [TicketSold(buyer: "renata")]) let assert Ok(simulate.Commit(store:, events: [recorded])) = simulate.dispatch( store, - stream: "ticket-sales", - query: sale_query(), + decision_context: sale_context(), decider: ticket_decider(), command: BuyTicket(buyer: "lucy"), ) @@ -187,69 +210,72 @@ it. The descriptor should adapt the same application-owned type, version, tags, and metadata mapping used by production codecs. The simulator is an executable reference for core semantics, not a backend -interface. Keep codec fidelity, SQL selection, stream-revision races, -transactions, retries, outboxes, and real concurrency in backend integration -tests. `factos.compute_events` remains the smaller helper when history is already -filtered and no recorded store transition is needed. +interface. Keep codec fidelity, storage predicate selection, transaction +rollback, retries, outboxes, and real concurrency in backend integration tests. +Directly destructuring and folding a `Decider` remains the smaller approach when +history is already filtered and no recorded store transition is needed. ## How are views computed? -A view is an in-memory fold over events: +A projection is an ordinary pure application function. Fold the events with the +state and evolution logic that the read model needs: ```gleam -let sold_count = - factos.view(initial: 0, evolve: fn(count, event) { +fn count_sold_tickets(events: List(Event)) -> Int { + list.fold(events, 0, fn(count, event) { case event { - TicketSold(_) -> count + 1 + TicketSold(buyer: _) -> count + 1 } }) +} ``` -You can run it over events you already have: +Run the function over events you already have: ```gleam -factos.project(view: sold_count, events: events) +let sold_count = count_sold_tickets(events) ``` Or your application can read events from a backend and store the projected value -itself. Factos does not maintain a projection table automatically. - -Views can always be recomputed if the original events are still decodable. That +itself. Factos does not maintain a projection table automatically. Projection +folds can always be recomputed if the original events are still decodable. That is why event codec compatibility matters. -The pure `factos` package keeps this storage-independent contract. The -PostgreSQL backend `factos_pog` can attach a strong subscription callback to a -dispatch. Projection rows written through its transaction connection commit or -roll back with the originating event and are query-visible when dispatch -returns successfully. +The shared `factos.new_subscription` and `factos.with_subscriptions` functions +can attach an application callback to a dispatch builder. A +`factos.StrongConsistency` callback runs through the backend's transaction +connection, so projection rows it writes commit or roll back with the +originating event and are query-visible when dispatch returns successfully. +The current PostgreSQL and SQLite backends implement `factos.FireAndForget` by +starting independent asynchronous work only after commit. This is best-effort +work, not a supervised or durable delivery mechanism. ## How are effects handled? -Reactors turn committed event records into effect values: +Effects are derived by an ordinary pure application function over a recorded +event: ```gleam pub type Effect { AnnounceTicketSale(buyer: String, position: factos.SequencePosition) } -fn ticket_reactor() -> factos.Reactor(Event, Effect) { - factos.reactor(fn(recorded) { - case recorded.event { - TicketSold(buyer) -> [ - AnnounceTicketSale(buyer: buyer, position: recorded.position), - ] - } - }) +fn ticket_effects(recorded: factos.Recorded(Event)) -> List(Effect) { + case recorded.event { + TicketSold(buyer:) -> [ + AnnounceTicketSale(buyer:, position: recorded.position), + ] + } } ``` -After dispatch: +After dispatch, apply that function to the committed records: ```gleam -let effects = factos.react_all(ticket_reactor(), dispatch.events) +let effects = list.flat_map(dispatch.events, ticket_effects) ``` Factos does not send the email, publish the webhook, or mark the effect as done. -It keeps that work explicit so your application can choose the durability and -retry strategy. +It keeps derivation separate from execution so your application can choose the +durability and retry strategy. diff --git a/backends/factos_cf/gleam.toml b/backends/factos_cf/gleam.toml index b1308f6..a823dbf 100644 --- a/backends/factos_cf/gleam.toml +++ b/backends/factos_cf/gleam.toml @@ -21,8 +21,6 @@ factos = { path = "../.." } cf = { git = "https://tangled.org/renatillas.dev/cf", ref = "main" } gleam_javascript = ">= 1.0.0 and < 2.0.0" gleam_stdlib = ">= 1.0.0 and < 2.0.0" -gleam_time = ">= 1.8.0 and < 2.0.0" -gleam_json = ">= 3.1.0 and < 4.0.0" [dev_dependencies] cf_miniflare = { git = "https://tangled.org/renatillas.dev/cf", ref = "main", path = "cf_miniflare" } diff --git a/backends/factos_cf/manifest.toml b/backends/factos_cf/manifest.toml index 6fcf20f..662e4d1 100644 --- a/backends/factos_cf/manifest.toml +++ b/backends/factos_cf/manifest.toml @@ -11,9 +11,7 @@ packages = [ { name = "cf_miniflare", version = "1.0.0", build_tools = ["gleam"], requirements = ["cf", "gleam_javascript", "gleam_stdlib"], source = "git", repo = "https://tangled.org/renatillas.dev/cf", commit = "ec3a10e176f50c3066d82e22d6cac972ac95a45e", path = "cf_miniflare" }, { name = "factos", version = "1.0.0", build_tools = ["gleam"], requirements = ["gleam_stdlib"], source = "local", path = "../.." }, { name = "gleam_javascript", version = "1.0.0", build_tools = ["gleam"], requirements = ["gleam_stdlib"], otp_app = "gleam_javascript", source = "hex", outer_checksum = "EF6C77A506F026C6FB37941889477CD5E4234FCD4337FF0E9384E297CB8F97EB" }, - { name = "gleam_json", version = "3.1.0", build_tools = ["gleam"], requirements = ["gleam_stdlib"], otp_app = "gleam_json", source = "hex", outer_checksum = "44FDAA8847BE8FC48CA7A1C089706BD54BADCC4C45B237A992EDDF9F2CDB2836" }, { name = "gleam_stdlib", version = "1.0.3", build_tools = ["gleam"], requirements = [], otp_app = "gleam_stdlib", source = "hex", outer_checksum = "1F543AFBA5D33DA493E6087F4E4C4F20D899411343512686C98A8ABB2963CF22" }, - { name = "gleam_time", version = "1.8.0", build_tools = ["gleam"], requirements = ["gleam_stdlib"], otp_app = "gleam_time", source = "hex", outer_checksum = "533D8723774D61AD4998324F5DD1DABDCDBFABAFB9E87CB5D03C6955448FC97D" }, { name = "gleeunit", version = "1.11.0", build_tools = ["gleam"], requirements = ["gleam_stdlib"], otp_app = "gleeunit", source = "hex", outer_checksum = "EC31ABA74256AEA531EDF8169931D775BBB384FED0A8A1BDC4DD9354E3E21826" }, ] @@ -22,7 +20,5 @@ cf = { git = "https://tangled.org/renatillas.dev/cf", ref = "main" } cf_miniflare = { git = "https://tangled.org/renatillas.dev/cf", ref = "main", path = "cf_miniflare" } factos = { path = "../.." } gleam_javascript = { version = ">= 1.0.0 and < 2.0.0" } -gleam_json = { version = ">= 3.1.0 and < 4.0.0" } gleam_stdlib = { version = ">= 1.0.0 and < 2.0.0" } -gleam_time = { version = ">= 1.8.0 and < 2.0.0" } gleeunit = { version = ">= 1.0.0 and < 2.0.0" } diff --git a/backends/factos_cf/src/factos/factos_cf.gleam b/backends/factos_cf/src/factos/factos_cf.gleam index aac7e19..c40db7f 100644 --- a/backends/factos_cf/src/factos/factos_cf.gleam +++ b/backends/factos_cf/src/factos/factos_cf.gleam @@ -1,14 +1,18 @@ //// Cloudflare Workers D1 backend for Factos. //// -//// This backend stores accepted facts in an append-only D1 table. Event payload -//// serialization remains application-owned; the backend stores a string payload -//// plus Factos query metadata (`EventType` and `Tag`). +//// Accepted facts are stored in one append-only, globally ordered D1 table. +//// Event serialization remains application-owned through +//// `factos.EventCodec(event, String)`. //// -//// D1 operations are asynchronous, so public functions return -//// `Promise(Result(_, _))`. The stream and context dispatch functions use one -//// conditional `insert ... select ... returning` statement for appends. That keeps -//// the append condition and writes in the same SQLite statement, which is the -//// atomic boundary D1 exposes through prepared statements. +//// D1 operations are asynchronous, so public operations return +//// `Promise(Result(_, _))`. Each append uses one conditional +//// `insert ... select ... returning` statement. The decision-context check and +//// every accepted event therefore share the atomic statement boundary exposed by +//// D1 prepared statements. +//// +//// D1 has no interactive transaction in which arbitrary callbacks can run, and +//// a Worker execution context is not part of this backend client. Dispatch +//// rejects builders containing subscriptions before performing database IO. import cf/d1 import factos @@ -17,177 +21,57 @@ import gleam/dynamic/decode import gleam/int import gleam/javascript/array import gleam/javascript/promise.{type Promise} -import gleam/json import gleam/list import gleam/pair import gleam/result import gleam/string -pub type Proposed(event) { - /// A domain event prepared for D1 persistence. - /// - /// The application codec creates this value. `id` should identify the event for - /// the application. `type_` and `tags` are store-visible query metadata. `data` - /// is an opaque string owned by the application codec, typically JSON. - Proposed( - id: String, - event: event, - type_: factos.EventType, - version: Int, - tags: List(factos.Tag), - metadata: factos.Metadata, - data: String, - ) -} - -/// A raw event row read from D1 before domain decoding. -/// -/// Applications normally do not construct this directly. The backend reads -/// `StoredEvent` values from the `factos_events` table and passes them to the -/// application's `EventCodec.decode` function. -/// -/// The fields map directly to stored columns: -/// -/// - `position` is the global append-only sequence position. -/// - `id` is the application-provided event identifier. -/// - `stream` is the logical stream name. -/// - `revision` is the zero-based stream revision. -/// - `type_`, `version`, `tags`, and `metadata` are query and compatibility -/// metadata used by Factos. -/// - `data` is the application-owned serialized payload. -pub type StoredEvent { - StoredEvent( - position: Int, - id: String, - stream: String, - revision: Int, - type_: factos.EventType, - version: Int, - tags: List(factos.Tag), - metadata: factos.Metadata, - data: String, - ) -} - -/// Backend client bundling a D1 database binding. -/// -/// Create this once from the Worker environment's D1 binding and pass it to the -/// backend functions. Keeping the database inside `Client` makes APIs stable if -/// more Cloudflare bindings or runtime configuration are needed later. -pub opaque type Client { - Client(database: d1.Database) -} - -pub fn new(database: d1.Database) -> Client { - Client(database) -} - -/// Application-owned codec for this D1 backend. -/// -pub opaque type EventCodec(event, state) { - EventCodec( - encode: fn(event) -> Proposed(event), - decode: fn(StoredEvent) -> Result(factos.Decoded(event), EventDecodeError), - ) -} - -/// Create a new codec -/// -/// `encode` turns a domain event into a `Proposed` event ready for persistence. -/// This is where the application chooses event IDs, event types, schema -/// versions, tags, metadata, and payload serialization. -/// -/// `decode` turns a stored row back into a domain event. Decode failures are -/// returned as `DecodeError` and stop load/read flows rather than panicking. -pub fn codec( - encode encode: fn(event) -> Proposed(event), - decode decode: fn(StoredEvent) -> - Result(factos.Decoded(event), EventDecodeError), -) -> EventCodec(event, state) { - EventCodec(encode:, decode:) -} - -/// Result of a successful append. -/// -/// `current_revision` is the new stream revision after the append. For an empty -/// append it is the current revision that was observed. -/// -/// `position` is the global event position assigned to the last appended event. -/// Empty appends use `factos.NoPosition` because no new event row was written. -pub type Append { - Append(current_revision: Int, position: factos.SequencePosition) -} - -pub type Dispatch(event) { - /// Result of a successful dispatch. - /// - /// `append` has the stream revision and final global position. `events` are the - /// committed events recorded by this dispatch, suitable for pure Factos - /// reactors or backend-specific durable effect adapters. - Dispatch(append: Append, events: List(factos.Recorded(event))) -} - -pub type Error(domain_error) { - /// The decider rejected the command with a domain error. - DomainError(domain_error) - - EventDecodeError(EventDecodeError) - - /// D1 returned an error while running a query. - StoreError(String) - - /// A D1 row did not match the backend's expected shape. - RowDecodeError(List(decode.DecodeError)) - - /// A stream revision or context append condition failed. - AppendConditionFailed(factos.AppendCondition) -} - -pub type EventDecodeError { - UnknownEventType(String) - /// The application codec could not decode a stored event. - InvalidPayload(json.DecodeError) -} - -type AppendMode { - CurrentStream - ExpectedStream(factos.Revision) - ContextCondition(factos.Query, factos.SequencePosition) +/// A failure produced by the D1 storage adapter. +pub type StoreError { + /// D1 rejected a query or returned an unsuccessful run result. + D1Error(message: String) + /// A D1 row did not have the shape required by the event store. + RowDecodeError(errors: List(decode.DecodeError)) + /// Dispatch-bound subscriptions require guarantees D1 cannot provide here. + SubscriptionsNotSupported } type QuerySql { QuerySql(sql: String, values: List(String)) } -/// Create or update the D1 schema required by this backend. +type PendingEvent(event) { + PendingEvent(id: String, event: event, encoded: factos.Event(String)) +} + +/// Create the fresh D1 schema required by this backend. /// -/// This function is idempotent and safe to run during application startup or in -/// tests. It creates `factos_events`, the append-only event store, and indexes -/// used by stream reads, context queries, and projection cursors. -pub fn migrate(client: Client) -> Promise(Result(Nil, Error(_))) { - let Client(database:) = client +/// `factos_cf` is not published, so this schema is the clean streamless +/// bootstrap contract. Applications must run it before dispatching commands. +pub fn migrate( + database: d1.Database, +) -> Promise( + Result(Nil, factos.Error(domain_error, subscription_error, StoreError)), +) { use _ <- promise.try_await(execute_migration( database, " create table if not exists factos_events ( position integer primary key autoincrement, - id text not null, - stream text not null, - revision integer not null, + id text not null unique, type text not null, version integer not null, tags text not null, metadata text not null, - data text not null, - unique(stream, revision) + data text not null ) ", )) use _ <- promise.try_await(execute_migration( database, " - create index if not exists factos_events_stream_revision - on factos_events(stream, revision) + create index if not exists factos_events_type_position + on factos_events(type, position) ", )) execute_migration( @@ -199,280 +83,282 @@ pub fn migrate(client: Client) -> Promise(Result(Nil, Error(_))) { ) } -fn execute_migration( - database: d1.Database, - sql: String, -) -> Promise(Result(Nil, Error(domain_error))) { - d1.prepare(database, sql) - |> d1.run - |> promise.map(fn(result) { - result - |> result.map(fn(_) { Nil }) - |> result.map_error(StoreError) - }) -} - -/// Read and fold the facts selected by a command-context query. +/// Execute a shared dispatch builder against D1. /// -/// This is the read half of a context-first command flow. It selects all events -/// matching `query`, decodes them with `codec`, and folds them through the -/// decider's `evolve` function starting from `decider.initial`. +/// A failed decision-context condition is retried by rerunning the complete +/// read-decide-append attempt, up to the builder's configured attempt count. +/// Decider, codec, and event-id functions must therefore be deterministic and +/// side-effect free. /// -/// The returned `factos.Context` includes an append condition that protects the -/// caller from stale decisions. Pass the same query to `dispatch_with_context` -/// when you want the backend to perform the full read-decide-append flow. -pub fn read_context( - client: Client, - query query: factos.Query, - decider decider: factos.Decider(command, state, event, domain_error), - codec codec: EventCodec(event, state), -) -> Promise(Result(factos.Context(event, state), Error(domain_error))) { - use events <- promise.map_try(read_matching_events( - client.database, - query, - codec, - )) - let position = highest_recorded_position(events) - Ok(factos.Context( - query:, - state: factos.evolve_recorded( - initial: decider.initial, - events: events, - evolve: decider.evolve, - ), - events: events, - position: position, - append_condition: factos.FailIfEventsMatch(query, position), - )) -} - -/// Run a full context-first read-decide-append command flow. -/// -/// This function: -/// -/// - appends those events only if no matching events were added meanwhile. -/// -/// Use this for commands whose validity depends on facts outside a single -/// stream. If the context changed between the read and the append, the function -/// returns `AppendConditionFailed`. -pub fn dispatch_with_query( - client: Client, - stream stream_name: String, - query query: factos.Query, - decider decider: factos.Decider(command, state, event, domain_error), - codec codec: EventCodec(event, state), - command command: command, -) -> Promise(Result(Dispatch(event), Error(domain_error))) { - use context <- promise.try_await(read_context( - client, - query:, +/// Builders containing subscriptions return +/// `factos.StoreError(SubscriptionsNotSupported)` without database IO. D1 cannot +/// run strong callbacks inside the conditional append statement, and this +/// backend does not own the Worker execution context required for durable +/// post-response work. +pub fn dispatch( + builder: factos.DispatchBuilder( + command, + state, + event, + String, + domain_error, + subscription_error, + d1.Database, + ), + command: command, + event_id event_id: fn() -> String, +) -> Promise( + Result( + factos.Dispatch(event), + factos.Error(domain_error, subscription_error, StoreError), + ), +) { + let factos.DispatchBuilder( + connection: database, + decision_context:, decider:, codec:, - )) + retry_attempts:, + subscriptions:, + ) = builder - use #(context, events) <- promise.try_await( - factos.decide_context(context, command, decider) - |> result.map_error(DomainError) - |> promise.resolve(), - ) - - append_with_condition( - client.database, - stream_name, - events, - codec, - context.append_condition, - ) + case subscriptions { + [] -> + dispatch_with_retries( + database, + decision_context, + decider, + codec, + command, + event_id, + retry_attempts, + ) + [_, ..] -> + promise.resolve(Error(factos.StoreError(SubscriptionsNotSupported))) + } } -/// Load and fold one stream. -/// -/// folds them through `decider.evolve`. This does not append events. -pub fn load_stream( - client: Client, - stream stream_name: String, +@internal +pub fn read( + database: d1.Database, + decision_context decision_context: factos.DecisionContext, decider decider: factos.Decider(command, state, event, domain_error), - codec codec: EventCodec(event, state), -) -> Promise(Result(factos.LoadedStream(event, state), Error(domain_error))) { - use events <- promise.map_try(read_stream_events( - client.database, - stream_name, + codec codec: factos.EventCodec(event, String), +) -> Promise( + Result( + factos.Context(event, state), + factos.Error(domain_error, subscription_error, StoreError), + ), +) { + use events <- promise.map_try(read_matching_events( + database, + decision_context, codec, )) - Ok(factos.LoadedStream( - stream: stream_name, - state: factos.evolve_recorded( - initial: decider.initial, - events:, - evolve: decider.evolve, + let factos.Decider(initial:, evolve:, ..) = decider + let position = factos.highest_recorded_position(events) + + Ok(factos.Context( + decision_context:, + state: factos.evolve_recorded(initial:, events:, evolve:), + events:, + position:, + append_condition: factos.FailIfEventsMatch( + decision_context:, + after: position, ), - events: events, - revision: stream_revision(events), )) } -/// Run a stream-based read-decide-append command flow. -/// -/// This function loads one stream, asks the decider to handle `command`, and -/// appends the produced events with an expected-revision condition. If another -/// write has advanced the stream, the append fails with `AppendConditionFailed`. -/// -/// The returned dispatch includes the committed recorded events so callers can -/// run pure Factos reactors or persist backend-specific durable effects. -pub fn dispatch( - client: Client, - stream stream_name: String, - decider decider: factos.Decider(command, state, event, domain_error), - codec codec: EventCodec(event, state), - command command: command, -) -> Promise(Result(Dispatch(event), Error(domain_error))) { - let Client(database:) = client - use loaded <- promise.try_await(load_stream( - client, - stream: stream_name, - decider: decider, - codec: codec, - )) - let factos.Decider(_, decide, _) = decider - case decide(loaded.state, command) { - Error(error) -> promise.resolve(Error(DomainError(error))) - Ok(events) -> - append_stream_events( - database, - stream_name, - events, - codec, - loaded.revision, - ) +/// Render the backend-specific error carried by `factos.StoreError`. +pub fn store_error_to_string(error: StoreError) -> String { + case error { + D1Error(message:) -> "D1 error: " <> message + RowDecodeError(errors:) -> + "D1 row decode error: " + <> { errors |> list.map(decode_error_to_string) |> string.join(", ") } + SubscriptionsNotSupported -> + "dispatch subscriptions are not supported by the D1 backend" } } -fn append_with_condition( +fn execute_migration( database: d1.Database, - stream_name: String, - events: List(event), - codec: EventCodec(event, state), - condition: factos.AppendCondition, -) -> Promise(Result(Dispatch(event), Error(domain_error))) { - case condition { - factos.NoAppendCondition -> - append_events(database, stream_name, events, codec, CurrentStream) - factos.FailIfEventsMatch(query, after) -> - append_events( - database, - stream_name, - events, - codec, - ContextCondition(query, after), - ) + sql: String, +) -> Promise( + Result(Nil, factos.Error(domain_error, subscription_error, StoreError)), +) { + d1.prepare(database, sql) + |> d1.run + |> promise.map(fn(query_result) { + query_result + |> result.map(fn(_) { Nil }) + |> result.map_error(d1_error) + }) +} + +fn dispatch_with_retries( + database: d1.Database, + decision_context: factos.DecisionContext, + decider: factos.Decider(command, state, event, domain_error), + codec: factos.EventCodec(event, String), + command: command, + event_id: fn() -> String, + attempts_remaining: Int, +) -> Promise( + Result( + factos.Dispatch(event), + factos.Error(domain_error, subscription_error, StoreError), + ), +) { + use dispatch_result <- promise.await(dispatch_once( + database, + decision_context, + decider, + codec, + command, + event_id, + )) + + case dispatch_result { + Error(factos.AppendConditionFailed(condition)) -> + case attempts_remaining > 1 { + True -> + dispatch_with_retries( + database, + decision_context, + decider, + codec, + command, + event_id, + attempts_remaining - 1, + ) + False -> promise.resolve(Error(factos.AppendConditionFailed(condition))) + } + Ok(dispatch) -> promise.resolve(Ok(dispatch)) + Error(factos.DomainError(error)) -> + promise.resolve(Error(factos.DomainError(error))) + Error(factos.SubscriptionError(error: error)) -> + promise.resolve(Error(factos.SubscriptionError(error: error))) + Error(factos.StoreError(error)) -> + promise.resolve(Error(factos.StoreError(error))) + Error(factos.DecodeError(error)) -> + promise.resolve(Error(factos.DecodeError(error))) } } -fn append_stream_events( +fn dispatch_once( database: d1.Database, - stream_name: String, - events: List(event), - codec: EventCodec(event, state), - expected: factos.Revision, -) -> Promise(Result(Dispatch(event), Error(domain_error))) { - append_events(database, stream_name, events, codec, ExpectedStream(expected)) + decision_context: factos.DecisionContext, + decider: factos.Decider(command, state, event, domain_error), + codec: factos.EventCodec(event, String), + command: command, + event_id: fn() -> String, +) -> Promise( + Result( + factos.Dispatch(event), + factos.Error(domain_error, subscription_error, StoreError), + ), +) { + use context <- promise.try_await(read( + database, + decision_context:, + decider:, + codec:, + )) + + case factos.decide_context(context, command, decider) { + Error(error) -> promise.resolve(Error(factos.DomainError(error))) + Ok(#(context, events)) -> + append_events(database, events, codec, event_id, context.append_condition) + } } fn append_events( database: d1.Database, - stream_name: String, events: List(event), - codec: EventCodec(event, state), - mode: AppendMode, -) -> Promise(Result(Dispatch(event), Error(domain_error))) { + codec: factos.EventCodec(event, String), + event_id: fn() -> String, + condition: factos.AppendCondition, +) -> Promise( + Result( + factos.Dispatch(event), + factos.Error(domain_error, subscription_error, StoreError), + ), +) { case events { [] -> - current_revision(database, stream_name) - |> promise.map(fn(result) { - result - |> result.map(fn(revision) { - let append = - Append(current_revision: revision, position: factos.NoPosition) - Dispatch(append:, events: []) - }) - }) + promise.resolve( + Ok(factos.Dispatch(position: factos.NoPosition, events: [])), + ) [_, ..] -> { - let EventCodec(encode, _) = codec - let proposed_events = list.map(events, encode) - let #(sql, values) = append_sql(stream_name, proposed_events, mode) + let factos.EventCodec(encode:, ..) = codec + let pending_events = + list.map(events, fn(event) { + PendingEvent(id: event_id(), event:, encoded: encode(event)) + }) + let #(sql, values) = append_sql(pending_events, condition) let event_statement = d1.prepare(database, sql) |> d1.bind(values) d1.batch(database, [event_statement]) - |> decode_batch_result(stream_name, proposed_events, mode) + |> decode_batch_result(pending_events, condition) } } } fn decode_batch_result( batch_result: Promise(Result(array.Array(d1.RunResult), String)), - stream_name: String, - events: List(Proposed(event)), - mode: AppendMode, -) -> Promise(Result(Dispatch(event), Error(domain_error))) { - use result <- promise.map(batch_result) - use run_results <- result.try(result |> result.map_error(StoreError)) - let run_result_list = array.to_list(run_results) + events: List(PendingEvent(event)), + condition: factos.AppendCondition, +) -> Promise( + Result( + factos.Dispatch(event), + factos.Error(domain_error, subscription_error, StoreError), + ), +) { + use query_result <- promise.map(batch_result) + use run_results <- result.try(query_result |> result.map_error(d1_error)) use first_result <- result.try( - list.first(run_result_list) - |> result.replace_error(StoreError("no event insert result in batch")), + list.first(array.to_list(run_results)) + |> result.replace_error(d1_error("no event insert result in batch")), ) - let d1.RunResult(success: success, results: event_rows, ..) = first_result + let d1.RunResult(success:, results: event_rows, ..) = first_result use _ <- result.try(case success { True -> Ok(Nil) - False -> Error(StoreError("event insert failed")) + False -> Error(d1_error("event insert failed")) }) - use appended <- result.try(decode_append_rows(event_rows)) - case list.length(appended) == list.length(events) { + use positions <- result.try(decode_append_positions(event_rows)) + + case list.length(positions) == list.length(events) { + False -> Error(factos.AppendConditionFailed(condition)) True -> { - let #(position, revision) = last_append_row(appended) - let append = - Append( - current_revision: revision, - position: factos.SequencePosition(position), - ) - Ok(Dispatch( - append: append, - events: recorded_append_rows(stream_name, events, appended), + let recorded_events = recorded_append_rows(events, positions) + Ok(factos.Dispatch( + position: factos.highest_recorded_position(recorded_events), + events: recorded_events, )) } - False -> Error(AppendConditionFailed(append_condition_for(mode))) } } fn recorded_append_rows( - stream_name: String, - events: List(Proposed(event)), - rows: List(#(Int, Int)), + events: List(PendingEvent(event)), + positions: List(Int), ) -> List(factos.Recorded(event)) { - case events, rows { + case events, positions { [], _ -> [] _, [] -> [] - [event, ..events], [row, ..rows] -> { - let Proposed( - id:, - event: domain_event, - type_:, - version:, - tags:, - metadata:, - .., - ) = event - let #(position, revision) = row + [pending, ..remaining_events], [position, ..remaining_positions] -> { + let PendingEvent(id:, event:, encoded:) = pending + let factos.Event(descriptor:, ..) = encoded [ factos.Recorded( id:, - stream: stream_name, - revision:, position: factos.SequencePosition(position), - event: domain_event, - descriptor: factos.EventDescriptor(type_:, version:, tags:, metadata:), + event:, + descriptor:, ), - ..recorded_append_rows(stream_name, events, rows) + ..recorded_append_rows(remaining_events, remaining_positions) ] } } @@ -480,140 +366,33 @@ fn recorded_append_rows( fn read_matching_events( database: d1.Database, - query: factos.Query, - codec: EventCodec(event, state), -) -> Promise(Result(List(factos.Recorded(event)), Error(domain_error))) { - let #(where_sql, params) = query_to_sql(query) - - d1.prepare( - database, - "select position, id, stream, revision, type, version, tags, metadata, data - from factos_events " <> where_sql <> " order by position", - ) - |> d1.bind(params) + decision_context: factos.DecisionContext, + codec: factos.EventCodec(event, String), +) -> Promise( + Result( + List(factos.Recorded(event)), + factos.Error(domain_error, subscription_error, StoreError), + ), +) { + let #(where_sql, parameters) = query_to_sql(decision_context) + + d1.prepare(database, "select position, id, type, version, tags, metadata, data + from factos_events " <> where_sql <> " order by position") + |> d1.bind(parameters) |> d1.raw - |> promise.map(fn(result) { - use rows <- result.try(result |> result.map_error(StoreError)) + |> promise.map(fn(query_result) { + use rows <- result.try(query_result |> result.map_error(d1_error)) decode_rows(rows, codec) }) } -fn query_to_sql(query: factos.Query) { - case query { - factos.AllEvents -> #("", []) - - factos.Query(items) -> { - case items { - [] -> #("where 1 = 0", []) - [_, ..] -> { - let built_items = - items - |> list.map(query_item_to_sql) - - let where_sql = - built_items - |> list.map(pair.first) - |> string.join(" or ") - - let params = - built_items - |> list.flat_map(pair.second) - - #("where " <> where_sql, params) - } - } - } - } -} - -fn query_item_to_sql(item: factos.QueryItem) { - let #(types_sql, types_params) = types_to_sql(item.types) - let #(tag_sql, tag_params) = tags_to_sql(item.tags) - - #( - "(" <> types_sql <> " and " <> tag_sql <> ")", - list.append(types_params, tag_params), - ) -} - -fn types_to_sql(types: List(factos.EventType)) { - case types { - [] -> #("1 = 1", []) - - [_, ..] -> { - let placeholders = - types - |> list.map(fn(_) { "?" }) - |> string.join(", ") - - let params = - types - |> list.map(factos.event_type_name) - - #("type in (" <> placeholders <> ")", params) - } - } -} - -fn tags_to_sql(tags: List(factos.Tag)) { - case tags { - [] -> #("1 = 1", []) - - [_, ..] -> { - let clauses = - tags - |> list.map(fn(_) { "instr(tags, char(10) || ? || char(10)) > 0" }) - |> string.join(" and ") - - let params = - tags - |> list.map(factos.tag_value) - - #("(" <> clauses <> ")", params) - } - } -} - -fn read_stream_events( - database: d1.Database, - stream_name: String, - codec: EventCodec(event, state), -) -> Promise(Result(List(factos.Recorded(event)), Error(domain_error))) { - d1.prepare( - database, - "select position, id, stream, revision, type, version, tags, metadata, data from factos_events where stream = ? order by revision", - ) - |> d1.bind([stream_name]) - |> d1.raw - |> promise.map(fn(result) { - use rows <- result.try(result |> result.map_error(StoreError)) - decode_rows(rows, codec) - }) -} - -fn current_revision( - database: d1.Database, - stream_name: String, -) -> Promise(Result(Int, Error(domain_error))) { - d1.prepare( - database, - "select coalesce(max(revision), -1) from factos_events where stream = ?", - ) - |> d1.bind([stream_name]) - |> d1.raw - |> promise.map(fn(result) { - use rows <- result.try(result |> result.map_error(StoreError)) - case array.to_list(rows) { - [row, ..] -> decode_int_field(row, 0) - [] -> Ok(-1) - } - }) -} - fn decode_rows( rows: array.Array(array.Array(Dynamic)), - codec: EventCodec(event, state), -) -> Result(List(factos.Recorded(event)), Error(domain_error)) { + codec: factos.EventCodec(event, String), +) -> Result( + List(factos.Recorded(event)), + factos.Error(domain_error, subscription_error, StoreError), +) { rows |> array.to_list |> list.try_map(fn(row) { decode_row(row, codec) }) @@ -621,162 +400,199 @@ fn decode_rows( fn decode_row( row: array.Array(Dynamic), - codec: EventCodec(event, state), -) -> Result(factos.Recorded(event), Error(domain_error)) { + codec: factos.EventCodec(event, String), +) -> Result( + factos.Recorded(event), + factos.Error(domain_error, subscription_error, StoreError), +) { use stored <- result.try(decode_stored_event(row)) - let EventCodec(decode: decode_event, ..) = codec - use decoded <- result.try( - decode_event(stored) |> result.map_error(EventDecodeError), + let factos.EventCodec(decode: decode_event, ..) = codec + use event <- result.try( + decode_event(stored) |> result.map_error(factos.DecodeError), ) - let factos.Decoded(event:, descriptor:) = decoded - let StoredEvent(position:, id:, stream:, revision:, ..) = stored + let factos.Recorded(id:, position:, descriptor:, ..) = stored - Ok(factos.Recorded( - id:, - stream:, - revision:, - position: factos.SequencePosition(position), - event:, - descriptor:, - )) + Ok(factos.Recorded(id:, position:, event:, descriptor:)) } fn decode_stored_event( row: array.Array(Dynamic), -) -> Result(StoredEvent, Error(domain_error)) { +) -> Result( + factos.Recorded(String), + factos.Error(domain_error, subscription_error, StoreError), +) { use position <- result.try(decode_int_field(row, 0)) use id <- result.try(decode_string_field(row, 1)) - use stream <- result.try(decode_string_field(row, 2)) - use revision <- result.try(decode_int_field(row, 3)) - use type_name <- result.try(decode_string_field(row, 4)) - use version <- result.try(decode_int_field(row, 5)) - use tags <- result.try(decode_string_field(row, 6)) - use metadata <- result.try(decode_string_field(row, 7)) - use data <- result.try(decode_string_field(row, 8)) - - Ok(StoredEvent( - position: position, - id: id, - stream: stream, - revision: revision, - type_: factos.event_type(type_name), - version: version, - tags: tags_from_text(tags), - metadata: metadata_from_text(metadata), - data: data, + use type_name <- result.try(decode_string_field(row, 2)) + use version <- result.try(decode_int_field(row, 3)) + use tags <- result.try(decode_string_field(row, 4)) + use metadata <- result.try(decode_string_field(row, 5)) + use data <- result.try(decode_string_field(row, 6)) + + Ok(factos.Recorded( + id:, + position: factos.SequencePosition(position), + event: data, + descriptor: factos.EventDescriptor( + type_: factos.event_type(type_name), + version:, + tags: tags_from_text(tags), + metadata: metadata_from_text(metadata), + ), )) } -fn decode_append_rows( +fn decode_append_positions( rows: array.Array(Dynamic), -) -> Result(List(#(Int, Int)), Error(domain_error)) { +) -> Result( + List(Int), + factos.Error(domain_error, subscription_error, StoreError), +) { rows |> array.to_list |> list.try_map(fn(row) { - use position <- result.try( - decode.run(row, append_row_decoder()) - |> result.map_error(RowDecodeError), - ) - Ok(position) + decode.run(row, append_position_decoder()) + |> result.map_error(row_decode_error) }) } -fn append_row_decoder() -> decode.Decoder(#(Int, Int)) { +fn append_position_decoder() -> decode.Decoder(Int) { use position <- decode.field("position", decode.int) - use revision <- decode.field("revision", decode.int) - decode.success(#(position, revision)) + decode.success(position) } fn decode_int_field( row: array.Array(Dynamic), index: Int, -) -> Result(Int, Error(domain_error)) { +) -> Result(Int, factos.Error(domain_error, subscription_error, StoreError)) { use value <- result.try( array.get(row, index) - |> result.replace_error(RowDecodeError([])), + |> result.replace_error(row_decode_error([])), ) decode.run(value, decode.int) - |> result.map_error(RowDecodeError) + |> result.map_error(row_decode_error) } fn decode_string_field( row: array.Array(Dynamic), index: Int, -) -> Result(String, Error(domain_error)) { +) -> Result(String, factos.Error(domain_error, subscription_error, StoreError)) { use value <- result.try( array.get(row, index) - |> result.replace_error(RowDecodeError([])), + |> result.replace_error(row_decode_error([])), ) decode.run(value, decode.string) - |> result.map_error(RowDecodeError) + |> result.map_error(row_decode_error) } -fn append_sql( - stream_name: String, - events: List(Proposed(event)), - mode: AppendMode, +fn query_to_sql( + decision_context: factos.DecisionContext, ) -> #(String, List(String)) { - let rows = - events - |> list.index_map(fn(event, index) { - append_select_sql(stream_name, event, mode, index) - }) + case decision_context { + factos.AllEvents -> #("", []) + factos.Matching(items: []) | factos.NoContext -> #("where 1 = 0", []) + factos.Matching(items: [_, ..] as items) -> { + let built_items = list.map(items, query_item_to_sql) + let where_sql = + built_items + |> list.map(pair.first) + |> string.join(" or ") + let parameters = built_items |> list.flat_map(pair.second) + #("where " <> where_sql, parameters) + } + } +} + +fn query_item_to_sql(item: factos.Item) -> #(String, List(String)) { + let factos.Item(types:, tags:) = item + let #(types_sql, type_parameters) = types_to_sql(types) + let #(tags_sql, tag_parameters) = tags_to_sql(tags) + + #( + "(" <> types_sql <> " and " <> tags_sql <> ")", + list.append(type_parameters, tag_parameters), + ) +} + +fn types_to_sql(types: List(factos.EventType)) -> #(String, List(String)) { + case types { + [] -> #("1 = 1", []) + [_, ..] -> #( + "type in (" <> placeholders(list.length(types)) <> ")", + list.map(types, factos.event_type_name), + ) + } +} + +fn tags_to_sql(tags: List(factos.Tag)) -> #(String, List(String)) { + case tags { + [] -> #("1 = 1", []) + [_, ..] -> #( + "(" + <> string.join( + list.repeat( + "instr(tags, char(10) || ? || char(10)) > 0", + list.length(tags), + ), + with: " and ", + ) + <> ")", + list.map(tags, factos.tag_value), + ) + } +} +fn append_sql( + events: List(PendingEvent(event)), + condition: factos.AppendCondition, +) -> #(String, List(String)) { + let rows = list.map(events, append_select_sql(_, condition)) let sql = - "insert into factos_events (id, stream, revision, type, version, tags, metadata, data) " + "insert into factos_events (id, type, version, tags, metadata, data) " <> string.join(list.map(rows, fn(row) { row.0 }), with: " union all ") - <> " returning position, revision" - + <> " returning position" let values = rows |> list.flat_map(fn(row) { row.1 }) #(sql, values) } fn append_select_sql( - stream_name: String, - event: Proposed(event), - mode: AppendMode, - index: Int, + pending: PendingEvent(event), + condition: factos.AppendCondition, ) -> #(String, List(String)) { - let Proposed(id, _, type_, version, tags, metadata, data) = event - let base_values = [ - id, - stream_name, - stream_name, - int.to_string(index + 1), - factos.event_type_name(type_), - int.to_string(version), - tags_to_text(tags), - metadata_to_text(metadata), - data, - ] - - let #(condition, condition_values) = append_condition_sql(stream_name, mode) + let PendingEvent(id:, encoded:, ..) = pending + let factos.Event( + payload: data, + descriptor: factos.EventDescriptor(type_:, version:, tags:, metadata:), + ) = encoded + let #(condition_sql, condition_values) = append_condition_sql(condition) + #( - "select ?, ?, (select coalesce(max(revision), -1) from factos_events where stream = ?) + cast(? as integer), ?, cast(? as integer), ?, ?, ? where " - <> condition, - list.append(base_values, condition_values), + "select ?, ?, cast(? as integer), ?, ?, ? where " <> condition_sql, + list.append( + [ + id, + factos.event_type_name(type_), + int.to_string(version), + tags_to_text(tags), + metadata_to_text(metadata), + data, + ], + condition_values, + ), ) } fn append_condition_sql( - stream_name: String, - mode: AppendMode, + condition: factos.AppendCondition, ) -> #(String, List(String)) { - case mode { - CurrentStream -> #("1 = 1", []) - ExpectedStream(expected) -> #( - "(select coalesce(max(revision), -1) from factos_events where stream = ?) = cast(? as integer)", - [stream_name, int.to_string(revision_to_int(expected))], - ) - ContextCondition(query, after) -> { - let QuerySql(sql, values) = matching_events_after_sql(query, after) - #("not exists (" <> sql <> ")", values) - } - } + let factos.FailIfEventsMatch(decision_context:, after:) = condition + let QuerySql(sql:, values:) = + matching_events_after_sql(decision_context, after) + #("not exists (" <> sql <> ")", values) } fn matching_events_after_sql( - query: factos.Query, + decision_context: factos.DecisionContext, after: factos.SequencePosition, ) -> QuerySql { let after_position = case after { @@ -784,37 +600,34 @@ fn matching_events_after_sql( factos.SequencePosition(position) -> position } - case query { + case decision_context { factos.AllEvents -> QuerySql( sql: "select 1 from factos_events where position > cast(? as integer) limit 1", values: [int.to_string(after_position)], ) - factos.Query(items:) -> - case items { - [] -> - QuerySql(sql: "select 1 from factos_events where 1 = 0", values: []) - [_, ..] -> { - let item_sql = list.map(items, query_item_sql) - QuerySql( - sql: "select 1 from factos_events where position > cast(? as integer) and (" - <> string.join( - list.map(item_sql, fn(item) { item.sql }), - with: " or ", - ) - <> ") limit 1", - values: [ - int.to_string(after_position), - ..list.flat_map(item_sql, fn(item) { item.values }) - ], - ) - } - } + factos.Matching(items: []) | factos.NoContext -> + QuerySql(sql: "select 1 from factos_events where 1 = 0", values: []) + factos.Matching(items: [_, ..] as items) -> { + let item_sql = list.map(items, query_item_sql) + QuerySql( + sql: "select 1 from factos_events where position > cast(? as integer) and (" + <> string.join( + list.map(item_sql, fn(item) { item.sql }), + with: " or ", + ) + <> ") limit 1", + values: [ + int.to_string(after_position), + ..list.flat_map(item_sql, fn(item) { item.values }) + ], + ) + } } } -fn query_item_sql(item: factos.QueryItem) -> QuerySql { - let factos.QueryItem(types, tags) = item +fn query_item_sql(item: factos.Item) -> QuerySql { + let factos.Item(types:, tags:) = item let type_sql = case types { [] -> QuerySql(sql: "1 = 1", values: []) [_, ..] -> @@ -845,44 +658,6 @@ fn placeholders(count: Int) -> String { list.repeat("?", count) |> string.join(with: ", ") } -fn append_condition_for(mode: AppendMode) -> factos.AppendCondition { - case mode { - CurrentStream -> factos.NoAppendCondition - ExpectedStream(_) -> factos.NoAppendCondition - ContextCondition(query, after) -> factos.FailIfEventsMatch(query, after) - } -} - -fn stream_revision(events: List(factos.Recorded(event))) -> factos.Revision { - case list.reverse(events) { - [] -> factos.NoEvents - [event, ..] -> factos.CurrentRevision(event.revision) - } -} - -fn highest_recorded_position( - events: List(factos.Recorded(event)), -) -> factos.SequencePosition { - case list.reverse(events) { - [] -> factos.NoPosition - [event, ..] -> event.position - } -} - -fn revision_to_int(revision: factos.Revision) -> Int { - case revision { - factos.NoEvents -> -1 - factos.CurrentRevision(revision) -> revision - } -} - -fn last_append_row(rows: List(#(Int, Int))) -> #(Int, Int) { - case list.reverse(rows) { - [row, ..] -> row - [] -> #(-1, -1) - } -} - fn tags_to_text(tags: List(factos.Tag)) -> String { case tags { [] -> "" @@ -927,43 +702,25 @@ fn metadata_from_text(metadata: String) -> factos.Metadata { } } -pub fn error_to_string( - error: Error(domain_error), - domain_error_to_string: fn(domain_error) -> String, -) -> String { - case error { - DomainError(error) -> domain_error_to_string(error) - StoreError(error) -> "store error: " <> error - RowDecodeError(decode_error) -> - "database row decode error: " - <> list.map(decode_error, decode_error_to_string) |> string.join(", ") - AppendConditionFailed(factos.NoAppendCondition) -> - "append to event failed: No append condition" - AppendConditionFailed(factos.FailIfEventsMatch(query: _, after: _)) -> - "append to event failed: Events matched" - EventDecodeError(UnknownEventType(type_)) -> "unknown event type: " <> type_ - EventDecodeError(InvalidPayload(json.UnexpectedEndOfInput)) -> - "invalid json: unexpected end of input" - EventDecodeError(InvalidPayload(json.UnexpectedByte(string))) -> - "invalid json: unexpected byte" <> string - EventDecodeError(InvalidPayload(json.UnexpectedSequence(string))) -> - "invalid json: unexpected sequence" <> string - EventDecodeError(InvalidPayload(json.UnableToDecode(decode_errors))) -> - "invalid json: " - <> list.map(decode_errors, decode_error_to_string) |> string.join(",") - } +fn d1_error( + message: String, +) -> factos.Error(domain_error, subscription_error, StoreError) { + factos.StoreError(D1Error(message:)) } -fn decode_error_to_string(decode_error: decode.DecodeError) -> String { - case decode_error { - decode.DecodeError(expected:, found:, path:) -> - "expected: " - <> expected - <> ", found: " - <> found - <> ", " - <> "path: [" - <> path |> string.join(",") - <> "]" - } +fn row_decode_error( + errors: List(decode.DecodeError), +) -> factos.Error(domain_error, subscription_error, StoreError) { + factos.StoreError(RowDecodeError(errors:)) +} + +fn decode_error_to_string(error: decode.DecodeError) -> String { + let decode.DecodeError(expected:, found:, path:) = error + "expected: " + <> expected + <> ", found: " + <> found + <> ", path: [" + <> string.join(path, ",") + <> "]" } diff --git a/backends/factos_cf/test/factos_cf_test.gleam b/backends/factos_cf/test/factos_cf_test.gleam index afaa059..5f5a694 100644 --- a/backends/factos_cf/test/factos_cf_test.gleam +++ b/backends/factos_cf/test/factos_cf_test.gleam @@ -9,7 +9,7 @@ import gleam/option import gleam/result import gleeunit -pub fn main() { +pub fn main() -> Nil { gleeunit.main() } @@ -33,332 +33,245 @@ type TestDatabase { TestDatabase(miniflare: miniflare.Miniflare, database: d1.Database) } -fn test_client(database: d1.Database) -> factos_cf.Client { - factos_cf.new(database) -} - -pub fn dispatch_stream_appends_and_loads_events_test() -> Promise(Nil) { - use test_database <- promise.await(new_test_database()) - let client = test_client(test_database.database) - use migrate_result <- promise.await(factos_cf.migrate(client)) - case migrate_result { - Ok(Nil) -> Nil - Error(error) -> { - let message = "migration failed: " <> error_to_string(error) - panic as message - } - } - use clear_result <- promise.await(clear_events(test_database.database)) - case clear_result { - Ok(Nil) -> Nil - Error(error) -> { - let message = "clear events failed: " <> error - panic as message - } - } - - use append_result <- promise.await(factos_cf.dispatch( - client, - stream: "reservation-renata", - decider: reservation_decider(), - codec: codec(), - command: Reserve("renata"), +pub fn shared_dispatch_persists_and_reads_context_test() -> Promise(Nil) { + use database <- with_test_database + use dispatch_result <- promise.await(dispatch_reservation( + database, + factos.NoContext, + "renata", )) - case append_result { - Ok(dispatch) -> { - let assert factos_cf.Append( - current_revision: 0, - position: factos.SequencePosition(_), - ) = dispatch.append - let assert [recorded] = dispatch.events - assert recorded.event == Reserved("renata") - Nil - } - Error(error) -> { - let message = "stream append failed: " <> error_to_string(error) - panic as message - } - } + let assert Ok(dispatch) = dispatch_result + let assert factos.SequencePosition(_) = dispatch.position + let assert [recorded] = dispatch.events + assert_reserved_recorded( + recorded, + position: dispatch.position, + id: "event-renata", + name: "renata", + ) - use loaded_result <- promise.await(factos_cf.load_stream( - client, - stream: "reservation-renata", - decider: reservation_decider(), - codec: codec(), + use context_result <- promise.await(read_reservations( + database, + factos.AllEvents, )) - let loaded = case loaded_result { - Ok(loaded) -> loaded - Error(_) -> panic as "stream load failed" - } - case loaded.state { - ["renata"] -> Nil - _ -> panic as "unexpected loaded state" - } - case loaded.revision { - factos.CurrentRevision(0) -> Nil - _ -> panic as "unexpected stream revision" - } - case list.length(loaded.events) { - 1 -> Nil - _ -> panic as "unexpected loaded event count" - } + let assert Ok(context) = context_result + assert context.state == ["renata"] + assert context.events == [recorded] + assert context.position == dispatch.position + assert context.append_condition + == factos.FailIfEventsMatch( + decision_context: factos.AllEvents, + after: dispatch.position, + ) - use Nil <- promise.await(dispose(test_database)) promise.resolve(Nil) } -pub fn dispatch_context_rejects_changed_context_test() -> Promise(Nil) { - use test_database <- promise.await(new_test_database()) - let client = test_client(test_database.database) - use migrate_result <- promise.await(factos_cf.migrate(client)) - case migrate_result { - Ok(Nil) -> Nil - Error(error) -> { - let message = "migration failed: " <> error_to_string(error) - panic as message - } - } - use clear_result <- promise.await(clear_events(test_database.database)) - case clear_result { - Ok(Nil) -> Nil - Error(error) -> { - let message = "clear events failed: " <> error - panic as message - } - } - - let query = - factos.query([ - factos.query_item(types: [factos.event_type("reserved")], tags: [ - factos.tag("name:context-renata"), - ]), - ]) - - use first_result <- promise.await(factos_cf.dispatch_with_query( - client, - stream: "reservation-renata", - query: query, - decider: reservation_decider(), - codec: codec(), - command: Reserve("context-renata"), - )) - case first_result { - Ok(dispatch) -> { - let assert factos_cf.Append( - current_revision: 0, - position: factos.SequencePosition(_), - ) = dispatch.append - let assert [recorded] = dispatch.events - assert recorded.event == Reserved("context-renata") - Nil - } - Error(error) -> { - let message = "context append failed: " <> error_to_string(error) - panic as message - } - } +pub fn shared_dispatch_rejects_duplicate_from_matching_context_test() -> Promise( + Nil, +) { + use database <- with_test_database + let decision_context = reservation_context("renata") - use second_result <- promise.await(factos_cf.dispatch_with_query( - client, - stream: "reservation-renata-duplicate", - query: query, - decider: reservation_decider(), - codec: codec(), - command: Reserve("context-renata"), + use first_result <- promise.await(dispatch_reservation( + database, + decision_context, + "renata", )) - case second_result { - Error(factos_cf.DomainError(AlreadyReserved("context-renata"))) -> Nil - _ -> panic as "duplicate context command should fail" - } + let assert Ok(_) = first_result - use context_result <- promise.await(factos_cf.read_context( - client, - query: query, - decider: reservation_decider(), - codec: codec(), + use duplicate_result <- promise.await(dispatch_reservation( + database, + decision_context, + "renata", )) - let context = case context_result { - Ok(context) -> context - Error(_) -> panic as "context read failed" - } - case context.state { - ["context-renata"] -> Nil - _ -> panic as "unexpected context state" - } - case context.position { - factos.SequencePosition(_) -> Nil - _ -> panic as "unexpected context position" - } + let assert Error(factos.DomainError(AlreadyReserved(name: "renata"))) = + duplicate_result - use Nil <- promise.await(dispose(test_database)) promise.resolve(Nil) } -pub fn context_semantics_conformance_test() -> Promise(Nil) { - use test_database <- promise.await(new_test_database()) - let client = test_client(test_database.database) - use migrate_result <- promise.await(factos_cf.migrate(client)) - let assert Ok(Nil) = migrate_result - use clear_result <- promise.await(clear_events(test_database.database)) - let assert Ok(Nil) = clear_result +pub fn decision_context_conformance_test() -> Promise(Nil) { + use database <- with_test_database - let no_matches = empty_query() - use renata_result <- promise.await(factos_cf.dispatch_with_query( - client, - stream: "conformance-renata", - query: no_matches, - decider: reservation_decider(), - codec: codec(), - command: Reserve(name: "renata"), + use renata_result <- promise.await(dispatch_reservation( + database, + factos.NoContext, + "renata", )) let assert Ok(renata_dispatch) = renata_result - use lucy_result <- promise.await(factos_cf.dispatch_with_query( - client, - stream: "conformance-lucy", - query: no_matches, - decider: reservation_decider(), - codec: codec(), - command: Reserve(name: "lucy"), + use lucy_result <- promise.await(dispatch_reservation( + database, + factos.NoContext, + "lucy", )) let assert Ok(lucy_dispatch) = lucy_result - use marc_result <- promise.await(factos_cf.dispatch_with_query( - client, - stream: "conformance-marc", - query: factos.AllEvents, - decider: reservation_decider(), - codec: codec(), - command: Reserve(name: "marc"), + use marc_result <- promise.await(dispatch_reservation( + database, + factos.AllEvents, + "marc", )) let assert Ok(marc_dispatch) = marc_result - let renata_position = renata_dispatch.append.position - let lucy_position = lucy_dispatch.append.position - let marc_position = marc_dispatch.append.position - use empty_context_result <- promise.await(factos_cf.read_context( - client, - query: no_matches, - decider: reservation_decider(), - codec: codec(), + use empty_result <- promise.await(read_reservations( + database, + factos.Matching(items: []), )) - let assert Ok(empty_context) = empty_context_result + let assert Ok(empty_context) = empty_result assert empty_context.state == [] assert empty_context.events == [] assert empty_context.position == factos.NoPosition - assert empty_context.append_condition - == factos.FailIfEventsMatch(query: no_matches, after: factos.NoPosition) - let compound_query = reservation_conformance_query() - use compound_context_result <- promise.await(factos_cf.read_context( - client, - query: compound_query, - decider: reservation_decider(), - codec: codec(), + let compound_context = reservation_conformance_context() + use compound_result <- promise.await(read_reservations( + database, + compound_context, )) - let assert Ok(compound_context) = compound_context_result - let assert [lucy] = compound_context.events + let assert Ok(context) = compound_result + let assert [lucy] = context.events assert_reserved_recorded( lucy, - stream: "conformance-lucy", - revision: 0, - position: lucy_position, + position: lucy_dispatch.position, + id: "event-lucy", name: "lucy", ) - assert compound_context.state == ["lucy"] - assert compound_context.position == lucy_position - assert compound_context.append_condition - == factos.FailIfEventsMatch(query: compound_query, after: lucy_position) - - use all_context_result <- promise.await(factos_cf.read_context( - client, - query: factos.AllEvents, - decider: reservation_decider(), - codec: codec(), - )) - let assert Ok(all_context) = all_context_result - let assert [renata, lucy, marc] = all_context.events + assert context.state == ["lucy"] + assert context.position == lucy_dispatch.position + assert context.append_condition + == factos.FailIfEventsMatch( + decision_context: compound_context, + after: lucy_dispatch.position, + ) + + use all_result <- promise.await(read_reservations(database, factos.AllEvents)) + let assert Ok(all_context) = all_result + let assert [renata, all_lucy, marc] = all_context.events assert_reserved_recorded( renata, - stream: "conformance-renata", - revision: 0, - position: renata_position, + position: renata_dispatch.position, + id: "event-renata", name: "renata", ) assert_reserved_recorded( - lucy, - stream: "conformance-lucy", - revision: 0, - position: lucy_position, + all_lucy, + position: lucy_dispatch.position, + id: "event-lucy", name: "lucy", ) assert_reserved_recorded( marc, - stream: "conformance-marc", - revision: 0, - position: marc_position, + position: marc_dispatch.position, + id: "event-marc", name: "marc", ) assert all_context.state == ["marc", "lucy", "renata"] - assert all_context.position == marc_position - assert all_context.append_condition - == factos.FailIfEventsMatch(query: factos.AllEvents, after: marc_position) + assert all_context.position == marc_dispatch.position - use Nil <- promise.await(dispose(test_database)) promise.resolve(Nil) } -pub fn empty_context_dispatch_is_a_no_op_test() -> Promise(Nil) { - use test_database <- promise.await(new_test_database()) - let client = test_client(test_database.database) - use migrate_result <- promise.await(factos_cf.migrate(client)) - let assert Ok(Nil) = migrate_result - use clear_result <- promise.await(clear_events(test_database.database)) - let assert Ok(Nil) = clear_result - - use seeded_result <- promise.await(factos_cf.dispatch( - client, - stream: "conformance-no-op", - decider: reservation_decider(), - codec: codec(), - command: Reserve(name: "no-op"), +pub fn empty_dispatch_has_no_position_or_events_test() -> Promise(Nil) { + use database <- with_test_database + use seeded_result <- promise.await(dispatch_reservation( + database, + factos.NoContext, + "seeded", )) let assert Ok(seeded_dispatch) = seeded_result - let assert [seeded] = seeded_dispatch.events - use no_op_result <- promise.await(factos_cf.dispatch_with_query( - client, - stream: "conformance-no-op", - query: factos.AllEvents, - decider: empty_reservation_decider(), - codec: codec(), - command: Reserve(name: "ignored"), - )) - let assert Ok(no_op_dispatch) = no_op_result - assert no_op_dispatch.events == [] - assert no_op_dispatch.append.current_revision == 0 - assert no_op_dispatch.append.position == factos.NoPosition - - use loaded_result <- promise.await(factos_cf.load_stream( - client, - stream: "conformance-no-op", - decider: reservation_decider(), - codec: codec(), + let builder = + factos.new_dispatch( + connection: database, + decision_context: factos.AllEvents, + decider: empty_reservation_decider(), + codec: codec(), + ) + use empty_result <- promise.await( + factos_cf.dispatch(builder, Reserve(name: "ignored"), event_id: fn() { + "unused" + }), + ) + let assert Ok(empty_dispatch) = empty_result + assert empty_dispatch + == factos.Dispatch(position: factos.NoPosition, events: []) + + use context_result <- promise.await(read_reservations( + database, + factos.AllEvents, )) - let assert Ok(loaded) = loaded_result - assert loaded.events == [seeded] - assert loaded.state == ["no-op"] - assert loaded.revision == factos.CurrentRevision(0) - - use context_result <- promise.await(factos_cf.read_context( - client, - query: factos.AllEvents, - decider: reservation_decider(), - codec: codec(), + let assert Ok(context) = context_result + assert context.events == seeded_dispatch.events + assert context.state == ["seeded"] + + promise.resolve(Nil) +} + +pub fn subscriptions_are_rejected_before_database_io_test() -> Promise(Nil) { + use database <- with_test_database + let subscription = + factos.new_subscription( + decision_context: factos.AllEvents, + consistency: factos.StrongConsistency, + handle: fn(_database, _recorded) { Ok(Nil) }, + ) + let builder = + factos.new_dispatch( + connection: database, + decision_context: factos.NoContext, + decider: reservation_decider(), + codec: codec(), + ) + |> factos.with_subscriptions(subscriptions: [subscription]) + + use dispatch_result <- promise.await( + factos_cf.dispatch(builder, Reserve(name: "not-written"), event_id: fn() { + "event-not-written" + }), + ) + let assert Error(factos.StoreError(factos_cf.SubscriptionsNotSupported)) = + dispatch_result + + use context_result <- promise.await(read_reservations( + database, + factos.AllEvents, )) let assert Ok(context) = context_result - assert context.events == [seeded] - assert context.state == ["no-op"] - assert context.position == seeded.position + assert context.events == [] + + promise.resolve(Nil) +} + +pub fn unknown_stored_event_uses_shared_decode_error_test() -> Promise(Nil) { + use database <- with_test_database + use insert_result <- promise.await(insert_raw_event( + database, + id: "unknown-event", + type_: "unknown", + data: "payload", + )) + let assert Ok(Nil) = insert_result + + use context_result <- promise.await(read_reservations( + database, + factos.AllEvents, + )) + let assert Error(factos.DecodeError(factos.UnknownEvent)) = context_result + promise.resolve(Nil) +} + +fn with_test_database(run: fn(d1.Database) -> Promise(Nil)) -> Promise(Nil) { + use test_database <- promise.await(new_test_database()) + use migration_result <- promise.await(migrate_test_database( + test_database.database, + )) + case migration_result { + Ok(Nil) -> Nil + Error(error) -> panic as error_to_string(error) + } + use clear_result <- promise.await(clear_events(test_database.database)) + let assert Ok(Nil) = clear_result + use Nil <- promise.await(run(test_database.database)) use Nil <- promise.await(dispose(test_database)) promise.resolve(Nil) } @@ -371,7 +284,6 @@ fn new_test_database() -> Promise(TestDatabase) { "2026-06-26", ) |> miniflare.with_d1_database(database_binding) - let miniflare = miniflare.new([worker]) use Nil <- promise.await(miniflare.ready(miniflare)) @@ -380,7 +292,6 @@ fn new_test_database() -> Promise(TestDatabase) { database_binding, option.Some(worker_name), )) - promise.resolve(TestDatabase(miniflare:, database:)) } @@ -388,55 +299,85 @@ fn dispose(test_database: TestDatabase) -> Promise(Nil) { miniflare.dispose(test_database.miniflare) } +fn migrate_test_database( + database: d1.Database, +) -> Promise(Result(Nil, factos.Error(DomainError, Nil, factos_cf.StoreError))) { + factos_cf.migrate(database) +} + fn clear_events(database: d1.Database) -> Promise(Result(Nil, String)) { d1.prepare(database, "delete from factos_events") |> d1.run |> promise.map(fn(run_result) { run_result |> result.map(fn(_) { Nil }) }) } -fn error_to_string(error: factos_cf.Error(DomainError)) -> String { - case error { - factos_cf.DomainError(_) -> "domain error" - factos_cf.EventDecodeError(_) -> "decode error" - factos_cf.StoreError(error) -> error - factos_cf.RowDecodeError(_) -> "row decode error" - factos_cf.AppendConditionFailed(_) -> "append condition failed" - } +fn insert_raw_event( + database: d1.Database, + id id: String, + type_ type_: String, + data data: String, +) -> Promise(Result(Nil, String)) { + d1.prepare( + database, + "insert into factos_events (id, type, version, tags, metadata, data) + values (?, ?, 1, '', '', ?)", + ) + |> d1.bind([id, type_, data]) + |> d1.run + |> promise.map(fn(run_result) { run_result |> result.map(fn(_) { Nil }) }) } -fn reservation_decider() -> factos.Decider( - Command, - List(String), - Event, - DomainError, +fn dispatch_reservation( + database: d1.Database, + decision_context: factos.DecisionContext, + name: String, +) -> Promise( + Result( + factos.Dispatch(Event), + factos.Error(DomainError, Nil, factos_cf.StoreError), + ), ) { - factos.decider(initial: [], decide: decide, evolve: evolve) + factos.new_dispatch( + connection: database, + decision_context:, + decider: reservation_decider(), + codec: codec(), + ) + |> factos_cf.dispatch(Reserve(name:), event_id: fn() { "event-" <> name }) } -fn empty_reservation_decider() -> factos.Decider( - Command, - List(String), - Event, - DomainError, +fn read_reservations( + database: d1.Database, + decision_context: factos.DecisionContext, +) -> Promise( + Result( + factos.Context(Event, List(String)), + factos.Error(DomainError, Nil, factos_cf.StoreError), + ), ) { - factos.decider( - initial: [], - decide: fn(_state, _command) { Ok([]) }, - evolve: evolve, + factos_cf.read( + database, + decision_context:, + decider: reservation_decider(), + codec: codec(), ) } -fn empty_query() -> factos.Query { - factos.Query(items: []) +fn reservation_context(name: String) -> factos.DecisionContext { + factos.Matching(items: [ + factos.item(types: [factos.event_type("reserved")], tags: [ + factos.tag("name:" <> name), + ]), + ]) } -fn reservation_conformance_query() -> factos.Query { - factos.query([ - factos.query_item(types: [factos.event_type("reserved")], tags: [ +fn reservation_conformance_context() -> factos.DecisionContext { + factos.Matching(items: [ + factos.item(types: [factos.event_type("reserved")], tags: [ factos.tag("name:renata"), factos.tag("name:lucy"), ]), - factos.query_item( + factos.item( types: [ factos.event_type("unknown"), factos.event_type("reserved"), @@ -446,76 +387,87 @@ fn reservation_conformance_query() -> factos.Query { ]) } -fn assert_reserved_recorded( - recorded: factos.Recorded(Event), - stream stream_name: String, - revision revision: Int, - position position: factos.SequencePosition, - name name: String, -) -> Nil { - assert recorded.id == "event-" <> name - assert recorded.stream == stream_name - assert recorded.revision == revision - assert recorded.position == position - assert recorded.descriptor.type_ == factos.event_type("reserved") - assert recorded.descriptor.version == 1 - assert recorded.descriptor.tags == [factos.tag("name:" <> name)] - assert recorded.descriptor.metadata == factos.empty_metadata() - assert recorded.event == Reserved(name: name) +fn reservation_decider() -> factos.Decider( + Command, + List(String), + Event, + DomainError, +) { + factos.decider(initial: [], decide:, evolve:) +} + +fn empty_reservation_decider() -> factos.Decider( + Command, + List(String), + Event, + DomainError, +) { + factos.decider(initial: [], decide: fn(_state, _command) { Ok([]) }, evolve:) } fn decide( state: List(String), command: Command, ) -> Result(List(Event), DomainError) { - case command { - Reserve(name) -> - case list.contains(state, name) { - True -> Error(AlreadyReserved(name)) - False -> Ok([Reserved(name)]) - } + let Reserve(name:) = command + case list.contains(state, name) { + True -> Error(AlreadyReserved(name:)) + False -> Ok([Reserved(name:)]) } } fn evolve(state: List(String), event: Event) -> List(String) { - case event { - Reserved(name) -> [name, ..state] - } + let Reserved(name:) = event + [name, ..state] } -fn codec() -> factos_cf.EventCodec(Event, List(String)) { - factos_cf.codec(encode:, decode:) +fn codec() -> factos.EventCodec(Event, String) { + factos.codec(encode:, decode:) } -fn encode(event: Event) -> factos_cf.Proposed(Event) { - case event { - Reserved(name) -> - factos_cf.Proposed( - id: "event-" <> name, - event: event, - type_: factos.event_type("reserved"), - version: 1, - tags: [factos.tag("name:" <> name)], - metadata: factos.empty_metadata(), - data: name, - ) - } +fn encode(event: Event) -> factos.Event(String) { + let Reserved(name:) = event + factos.new_event(type_: factos.event_type("reserved"), version: 1, data: name) + |> factos.with_tags(tags: [factos.tag("name:" <> name)]) } fn decode( - stored: factos_cf.StoredEvent, -) -> Result(factos.Decoded(Event), factos_cf.EventDecodeError) { - case factos.event_type_name(stored.type_) { - "reserved" -> - Ok(factos.Decoded( - event: Reserved(stored.data), - descriptor: factos.EventDescriptor( - type_: stored.type_, - version: stored.version, - tags: stored.tags, - metadata: stored.metadata, - ), - )) - other -> Error(factos_cf.UnknownEventType(other)) + stored: factos.Recorded(String), +) -> Result(Event, factos.DecodeError) { + case + factos.event_type_name(stored.descriptor.type_), + stored.descriptor.version + { + "reserved", 1 -> Ok(Reserved(name: stored.event)) + _, _ -> Error(factos.UnknownEvent) } } + +fn assert_reserved_recorded( + recorded: factos.Recorded(Event), + position position: factos.SequencePosition, + id id: String, + name name: String, +) -> Nil { + assert recorded.id == id + assert recorded.position == position + assert recorded.descriptor.type_ == factos.event_type("reserved") + assert recorded.descriptor.version == 1 + assert recorded.descriptor.tags == [factos.tag("name:" <> name)] + assert recorded.descriptor.metadata == factos.empty_metadata() + assert recorded.event == Reserved(name:) +} + +fn error_to_string( + error: factos.Error(DomainError, Nil, factos_cf.StoreError), +) -> String { + factos.error_to_string( + error, + fn(error) { + let AlreadyReserved(name:) = error + "already reserved: " <> name + }, + fn(_) { "subscription error" }, + factos_cf.store_error_to_string, + ) +} diff --git a/backends/factos_pog/README.md b/backends/factos_pog/README.md index ade2350..30b5bfd 100644 --- a/backends/factos_pog/README.md +++ b/backends/factos_pog/README.md @@ -3,20 +3,21 @@ `factos_pog` is the PostgreSQL backend for Factos, implemented with [`pog`](https://hex.pm/packages/pog). -It stores accepted facts in an append-only PostgreSQL event log, reads the facts -relevant to a command, runs your pure `factos.Decider`, and appends new facts -only if the relevant context is still stable. +It stores accepted facts in one append-only, globally ordered event log. For +each command it reads the required `factos.DecisionContext`, runs the +application's pure `factos.Decider`, and appends new facts only while that +context remains current. -Use this package when PostgreSQL is your event store and your consistency rules -are expressed with Factos event types and tags. +There are no streams or per-stream revisions. Selective consistency boundaries +are expressed with event types and tags. ## Guides -- [How it works](how-it-works) — transaction and storage overview. -- [Subscriptions](subscriptions) — dispatch-bound transactional and +- [How it works](how-it-works.html) — transactions, storage, and consistency. +- [Subscriptions](subscriptions.html) — dispatch-bound strong and fire-and-forget callbacks. -- [Durable effects](durable-effects) — application-owned transactional outbox - delivery. +- [Durable effects](durable-effects.html) — application-owned transactional + outbox delivery. ## Install @@ -28,14 +29,13 @@ factos_pog = ">= 2.0.0 and < 3.0.0" ## Set up the schema -The backend ships reusable dbmate-compatible migrations in `priv/dbmate/`. -Application databases should vendor those files into their own migration -repository, commit them, and run them with their normal migration tool before -dispatching commands. The application migration repository owns ordering and -execution history; `factos_pog` owns only the reusable schema artifacts. +The package ships dbmate-compatible migrations in `priv/dbmate/`. Vendor those +files into the application's migration repository, commit them, and run them +with the normal migration tool before dispatching commands. The application +owns migration ordering and execution history; `factos_pog` owns the reusable +schema artifacts. -In an Erlang-target migration tool, locate the package `priv` directory and copy -the package migrations into your application migration directory: +On the Erlang target, locate the package migration directory with: ```gleam import gleam/erlang/application @@ -44,233 +44,256 @@ let assert Ok(priv_directory) = application.priv_directory("factos_pog") let migrations_directory = priv_directory <> "/dbmate" ``` -`priv/migrations.sql` is a fresh-bootstrap schema, not an application's +`priv/migrations.sql` is a fresh-bootstrap schema. It is not an application's append-only migration history. The current schema contains: -- `factos_events`: append-only event rows with JSONB metadata and data; -- `factos_event_tags`: indexed tag rows used for context reads; -- a transaction-scoped append lock that keeps global event positions ordered by - commit. +- `factos_events`: event id, global position, type, version, tags, JSONB + metadata, and JSONB data; +- `factos_event_tags`: indexed tag rows used for selective context reads; +- a transaction-scoped append lock that makes global positions follow commit + order. -The dbmate history retains the `1.0.0` event-store baseline and one `2.0.0` -upgrade. `20260816000100_factos_pog_v2.sql` converts stored JSON to JSONB, -enforces UUIDv4 event identity, and orders event positions by commit. The -backend stores no subscription cursors or checkpoints. +Event ids are globally unique UUIDv4 strings. The schema has no stream, +revision, subscription, checkpoint, projection, or outbox columns or tables. + +The dbmate history retains the `1.0.0` event-store baseline and the `2.0.0` +cutover. `20260816000100_factos_pog_v2.sql` converts metadata and event data to +JSONB, enforces UUIDv4 identity, removes the legacy stream and revision model, +and adds commit-ordered global positions. ## Define a codec -Your domain event type remains yours. PostgreSQL stores JSONB event data plus -queryable descriptors, so the application provides an event codec. +The domain event type remains application-owned. PostgreSQL stores JSONB event +data and a store-visible descriptor. The shared codec uses a JSON string as the +backend representation: ```gleam -fn ticket_codec() -> factos_pog.EventCodec(Event) { - factos_pog.codec(encode:, decode:) +fn ticket_codec() -> factos.EventCodec(Event, String) { + factos.codec(encode:, decode:) } ``` -The encoder prepares JSON for persistence: +The encoder prepares an event for persistence: ```gleam -fn encode(event: Event) -> factos_pog.Proposed { +fn encode(event: Event) -> factos.Event(String) { case event { TicketSold(ticket_id:, buyer:) -> - factos_pog.new_proposed( + factos.new_event( type_: factos.event_type("TicketSold"), version: 1, data: json.object([ #("ticket_id", json.string(ticket_id)), #("buyer", json.string(buyer)), - ]), + ]) + |> json.to_string, ) - |> factos_pog.with_tags(tags: [ + |> factos.with_tags(tags: [ factos.tag("ticket:" <> ticket_id), ]) } } ``` -`new_proposed` requires the durable type, schema version, and JSON data. It -starts with no tags and empty metadata. Add either optional value through -`with_tags` or `with_metadata`. +`new_event` starts with no tags and empty metadata. Replace either optional +value through `with_tags` or `with_metadata`. -The decoder selects a JSON decoder from the stored event descriptor. The -backend parses the JSON and reconstructs the recorded event envelope: +The decoder receives the stored descriptor, id, position, and JSON string: ```gleam -fn decode( - descriptor: factos.EventDescriptor, -) -> Result(decode.Decoder(Event), factos_pog.DecodeError) { - case factos.event_type_name(descriptor.type_), descriptor.version { - "TicketSold", 1 -> Ok(ticket_sold_decoder()) - _, _ -> Error(factos_pog.UnknownEvent) +fn decode(stored: factos.Recorded(String)) -> Result(Event, factos.DecodeError) { + case + factos.event_type_name(stored.descriptor.type_), + stored.descriptor.version + { + "TicketSold", 1 -> + json.parse(stored.event, using: ticket_sold_decoder()) + |> result.map_error(fn(_) { factos.InvalidData }) + _, _ -> Error(factos.UnknownEvent) } } ``` -Tags are the query contract. If future commands need to find an event by a data -value, expose that value as a tag when writing the event. +The backend reconstructs a `factos.Recorded(event)` containing the event id, +global position, decoded domain event, and descriptor. + +Tags are part of the context-selection contract. If a future command must find +an event by a payload value, expose that value as a tag when encoding the event. +Metadata does not participate in matching. ## Dispatch commands -Pass the codec directly to each dispatch builder: +Every dispatch requires an explicit decision context: ```gleam +fn ticket_context(ticket_id: String) -> factos.DecisionContext { + factos.Matching(items: [ + factos.item( + types: [factos.event_type("TicketSold")], + tags: [factos.tag("ticket:" <> ticket_id)], + ), + ]) +} + let assert Ok(dispatch) = - factos_pog.new_dispatch( + factos.new_dispatch( connection:, - stream: buyer_stream(attempt), + decision_context: ticket_context(ticket_id), decider: ticket_decider(), codec: ticket_codec(), ) - |> factos_pog.with_query(query: sale_query()) |> factos_pog.dispatch( - BuyTicket(buyer: buyer_name(attempt)), + BuyTicket(ticket_id:, buyer: "renata"), event_id: uuid.v4_string, ) ``` -The backend: +Use `factos.NoContext` for a command that intentionally ignores history, +`factos.AllEvents` for a global boundary, or `factos.Matching(items:)` for +selective type-and-tag matching. `Matching(items: [])` selects nothing; prefer +`NoContext` when that is the command's intent. + +A dispatch: 1. opens a PostgreSQL `SERIALIZABLE` transaction; -2. reads rows matching the configured query, or the target stream; -3. decodes and folds them into decision state; +2. reads and decodes records selected by the decision context; +3. folds them into temporary state; 4. runs the decider; -5. appends only if the observed context is still current; -6. inserts event and tag-index rows; +5. appends only if no selected record appeared after the observed position; +6. inserts the event and tag-index rows; 7. runs matching strong subscription callbacks with the transaction connection; -8. retries serialization/deadlock failures up to the builder's retry attempts; -9. commits, then starts matching fire-and-forget subscription work. +8. commits; +9. starts matching fire-and-forget work in independent processes. -`dispatch.events` contains the committed records inserted by this dispatch. +Serialization and deadlock failures retry the complete transaction up to the +builder's configured attempt count. Override the default five attempts with +`factos.with_retry_attempts`. -Leave the query unset when one stream revision is intentionally the consistency -boundary: - -```gleam -let assert Ok(dispatch) = - factos_pog.new_dispatch( - connection:, - stream: "ticket-sale-renata", - decider: ticket_decider(), - codec: ticket_codec(), - ) - |> factos_pog.dispatch(BuyTicket(buyer: "renata"), event_id: uuid.v4_string) -``` +Decider and codec functions can run more than once and must be deterministic +and side-effect free. The event-id function can also be called again after an +aborted attempt; generate ids locally rather than reserving them through an +external side effect. -`with_query` controls the facts that must remain stable while the command -transaction commits. It does not wait for a separately maintained read model. +`dispatch.events` is the append-ordered list inserted by this dispatch. +`dispatch.position` is the last inserted global position, or +`factos.NoPosition` when the decider produced no events. Empty dispatches invoke +no subscription callbacks. ## Subscribe to a dispatch -A subscription filters the decoded events produced by one dispatch and handles -each matching `factos.Recorded` envelope. Attach subscriptions to the dispatch -builder: +A subscription filters only the decoded records accepted by the dispatch to +which it is attached: ```gleam -let subscription_supervisor = factos_pog.new_subscription_supervisor() +let registrations = + factos.Matching(items: [ + factos.item( + types: [factos.event_type("UserRegistered")], + tags: [], + ), + ]) -let subscriptions = [ - factos_pog.new_subscription( - query: factos.AllEvents, - consistency: factos_pog.StrongConsistency, +let projection = + factos.new_subscription( + decision_context: registrations, + consistency: factos.StrongConsistency, handle: fn(transaction_connection, recorded) { - ticket_projection.apply(transaction_connection, recorded) + user_projection.apply(transaction_connection, recorded) }, - ), - factos_pog.new_subscription( - query: factos.query([ - factos.query_item( - types: [factos.event_type("TicketSold")], - tags: [], - ), - ]), - consistency: factos_pog.FireAndForget( - supervisor: subscription_supervisor, - ), - handle: fn(connection, recorded) { - ticket_observer.handle(connection, recorded) - }, - ), -] - -let assert Ok(dispatch) = - factos_pog.new_dispatch( - connection:, - stream: "ticket-sale-renata", - decider: ticket_decider(), - codec: ticket_codec(), - ) - |> factos_pog.with_subscriptions(subscriptions:) - |> factos_pog.dispatch( - BuyTicket(buyer: "renata"), - event_id: uuid.v4_string, ) ``` -All subscriptions in one list share a callback error type. +Attach the complete subscription list with `with_subscriptions`. All entries in +one list share one callback error type: + +```gleam +factos.new_dispatch( + connection:, + decision_context: registration_context(username), + decider: user_decider(), + codec: event_codec(), +) +|> factos.with_subscriptions(subscriptions: [projection]) +|> factos_pog.dispatch( + RegisterUser(username:), + event_id: uuid.v4_string, +) +``` ### Strong consistency `StrongConsistency` callbacks run in subscription-list order and event append -order inside the dispatch's serializable transaction. The supplied connection -is the transaction-scoped connection. This makes a PostgreSQL projection row -and its originating event one atomic commit. +order inside the dispatch transaction. Every PostgreSQL operation must use the +supplied transaction connection. The first callback `Error` becomes -`SubscriptionError(error: callback_error)` and rolls back the accepted events, -tag rows, that callback's writes, and every earlier strong callback write. -Callback failures are not retried. +`SubscriptionError(error: callback_error)` and rolls back all accepted events, +tag rows, the failing callback's writes, and every earlier strong callback +write. Callback failures are not retried. -A PostgreSQL serialization or deadlock failure still retries the complete -dispatch transaction, including any strong callbacks already run by the -aborted attempt. Keep their side effects on the supplied transaction connection -so rollback and retry remain safe. +A PostgreSQL serialization or deadlock failure does retry the complete +transaction, including strong callbacks already run by the aborted attempt. +Keep all strong callback side effects on the supplied connection so rollback +and retry remain safe. ### Fire and forget -Allocate one subscription supervisor at application startup and install it in -the application's supervision tree: +Use the shared fire-and-forget consistency mode: ```gleam -let subscription_supervisor = factos_pog.new_subscription_supervisor() - -static_supervisor.new(strategy: static_supervisor.OneForOne) -|> static_supervisor.add( - factos_pog.supervised_subscription_supervisor(subscription_supervisor), -) -|> static_supervisor.start +let observer = + factos.new_subscription( + decision_context: registrations, + consistency: factos.FireAndForget, + handle: fn(connection, recorded) { + registration_observer.handle(connection, recorded) + }, + ) ``` -Retain the same opaque `subscription_supervisor` value when constructing -`FireAndForget(supervisor:)` subscriptions. The backend starts one supervised -temporary child for each matching fire-and-forget subscription only after the -dispatch commits. That child handles matching records in append order with the -builder's ordinary Pog connection. - -Fire-and-forget callbacks do not delay or alter the committed dispatch result. -A returned `Error` is ignored and processing continues with the next record. A -panic terminates that temporary child and drops its remaining records. Separate +After the final successful commit, `factos_pog` starts one process per matching +fire-and-forget subscription. The process receives the builder's ordinary Pog +connection and handles matching records in append order. Different subscriptions and dispatches may run concurrently. -This mode has no cursor, catch-up, checkpoint, replay, or retry. If the -supervisor is unavailable, work is dropped. Use it only for best-effort work. -Durable external delivery requires an application-owned transactional outbox; -see [Durable effects](durable-effects). +Dispatch does not wait for these callbacks. Returned `Ok` and `Error` values +are ignored and processing continues. A panic terminates that process and drops +its remaining records. + +This mode has no historical catch-up, cursor, checkpoint, replay, retry, or +dead-letter policy. Use it only for best-effort work. Durable external delivery +requires an application-owned transactional outbox; see +[Durable effects](durable-effects.html). + +## Handle errors -### Custom consumer runtimes +Dispatch returns +`factos.Error(domain_error, subscription_error, pog.QueryError)`: -`read_after` remains available for applications that need their own scheduler, -batching, checkpoint, and recovery protocol. Poll ordered reads from the event -log, persist an application-owned checkpoint, and treat those records as the -durable source of truth. +- `factos.DomainError(error)` — the decider rejected the command; +- `factos.SubscriptionError(error:)` — a strong callback failed; +- `factos.StoreError(error)` — PostgreSQL or Pog failed; +- `factos.AppendConditionFailed(condition)` — the selected context changed; +- `factos.DecodeError(error)` — the stored event type or data could not be + decoded. + +`factos.error_to_string` accepts one formatter for each generic error type: + +```gleam +factos.error_to_string( + error, + domain_error_to_string, + subscription_error_to_string, + factos_pog.query_error_to_string, +) +``` ## Run tests The integration suite shares one externally managed PostgreSQL service and -creates an isolated database for each test. Start and stop the checked-in -Compose service around the suite: +creates an isolated database for each test: ```sh docker compose up --wait -d @@ -284,9 +307,8 @@ The defaults match `compose.yml`. Override a remote or CI service with ## Tradeoff: serializable contention -PostgreSQL `SERIALIZABLE` isolation protects arbitrary event-type/tag predicates -without a global application lock. Conflicting transactions can abort and retry, -so deciders, event codecs, and event-id generators must remain pure. Strong -subscription callbacks may rerun after an aborted attempt and must keep side -effects inside the supplied transaction. Fire-and-forget work starts only -after a successful final commit. +PostgreSQL `SERIALIZABLE` isolation protects arbitrary event-type-and-tag +predicates without a global application lock. Conflicting transactions can +abort and retry. Choose the narrowest decision context that fully protects the +business rule; `AllEvents` deliberately makes every append part of one global +consistency boundary. diff --git a/backends/factos_pog/dev/factos_pog_dev.gleam b/backends/factos_pog/dev/factos_pog_dev.gleam index 40fd4ce..f82af18 100644 --- a/backends/factos_pog/dev/factos_pog_dev.gleam +++ b/backends/factos_pog/dev/factos_pog_dev.gleam @@ -2,7 +2,6 @@ import factos import factos/factos_pog import gleam/dynamic/decode import gleam/erlang/application -import gleam/erlang/atom import gleam/erlang/process import gleam/int import gleam/io @@ -10,7 +9,6 @@ import gleam/json import gleam/list import gleam/option.{Some} import gleam/otp/actor -import gleam/otp/static_supervisor import gleam/result import gleam/string import gleamy/bench @@ -80,7 +78,10 @@ type State { } type WorkerMessage { - WorkerDone(worker: Int, result: Result(Nil, factos_pog.Error(Nil, Nil))) + WorkerDone( + worker: Int, + result: Result(Nil, factos.Error(Nil, Nil, pog.QueryError)), + ) } fn setup_sequential(input: BenchmarkInput) -> fn(BenchmarkInput) -> Nil { @@ -203,7 +204,7 @@ fn run_sequential(connection: pog.Connection, remaining: Int) -> Nil { case remaining <= 0 { True -> Nil False -> { - let assert Ok(_) = dispatch_once(connection, "sequential") + let assert Ok(_) = dispatch_once(connection) run_sequential(connection, remaining - 1) } } @@ -230,7 +231,7 @@ fn spawn_workers( False -> { let worker = worker_count process.spawn(fn() { - let result = run_worker(connection, worker, operations_count) + let result = run_worker(connection, operations_count) process.send(subject, WorkerDone(worker: worker, result: result)) }) spawn_workers(connection, subject, worker_count - 1, operations_count) @@ -254,9 +255,12 @@ fn wait_for_workers( "benchmark worker " <> int.to_string(worker) <> " failed: " - <> factos_pog.error_to_string(error, fn(_) { "nil" }, fn(_) { - "nil" - }), + <> factos.error_to_string( + error, + fn(_) { "nil" }, + fn(_) { "nil" }, + factos_pog.query_error_to_string, + ), ) panic as "benchmark worker failed" } @@ -267,32 +271,27 @@ fn wait_for_workers( fn run_worker( connection: pog.Connection, - worker: Int, remaining: Int, -) -> Result(Nil, factos_pog.Error(Nil, Nil)) { +) -> Result(Nil, factos.Error(Nil, Nil, pog.QueryError)) { case remaining <= 0 { True -> Ok(Nil) False -> { - use _ <- result.try(dispatch_once( - connection, - "concurrent-" <> int.to_string(worker), - )) - run_worker(connection, worker, remaining - 1) + use _ <- result.try(dispatch_once(connection)) + run_worker(connection, remaining - 1) } } } fn dispatch_once( connection: pog.Connection, - stream_name: String, -) -> Result(factos_pog.Dispatch(Event), factos_pog.Error(Nil, Nil)) { - factos_pog.new_dispatch( - connection: connection, - stream: stream_name, +) -> Result(factos.Dispatch(Event), factos.Error(Nil, Nil, pog.QueryError)) { + factos.new_dispatch( + connection:, + decision_context: factos.NoContext, decider: decider(), codec: codec(), ) - |> factos_pog.with_retry_attempts(100) + |> factos.with_retry_attempts(100) |> factos_pog.dispatch(Increment, event_id: uuid.v4_string) } @@ -307,16 +306,11 @@ fn smoke_subscription(connection: pog.Connection) -> Nil { ) |> pog.execute(on: connection) - let name = process.new_name("test") - let assert Ok(actor.Started(pid: supervisor_pid, ..)) = - static_supervisor.new(strategy: static_supervisor.OneForOne) - |> static_supervisor.add(factos_pog.supervised(name)) - |> static_supervisor.start let observed_events = process.new_subject() let strong_projection = - factos_pog.new_subscription( - query: factos.AllEvents, - consistency: factos_pog.StrongConsistency, + factos.new_subscription( + decision_context: factos.AllEvents, + consistency: factos.StrongConsistency, handle: fn(transaction_connection, recorded) { let factos.Recorded(event: Incremented(value:), ..) = recorded pog.query( @@ -331,9 +325,9 @@ fn smoke_subscription(connection: pog.Connection) -> Nil { }, ) let fire_observer = - factos_pog.new_subscription( - query: factos.AllEvents, - consistency: factos_pog.FireAndForget(name:), + factos.new_subscription( + decision_context: factos.AllEvents, + consistency: factos.FireAndForget, handle: fn(_connection, recorded) { process.send(observed_events, recorded) Ok(Nil) @@ -341,13 +335,13 @@ fn smoke_subscription(connection: pog.Connection) -> Nil { ) let assert Ok(dispatch) = - factos_pog.new_dispatch( + factos.new_dispatch( connection:, - stream: "subscription-smoke", + decision_context: factos.AllEvents, decider: decider(), codec: codec(), ) - |> factos_pog.with_subscriptions(subscriptions: [ + |> factos.with_subscriptions(subscriptions: [ strong_projection, fire_observer, ]) @@ -366,8 +360,6 @@ fn smoke_subscription(connection: pog.Connection) -> Nil { let assert Ok(observed) = process.receive(observed_events, within: 5000) assert observed == recorded - process.unlink(supervisor_pid) - process.send_abnormal_exit(supervisor_pid, atom.create("shutdown")) Nil } @@ -394,25 +386,30 @@ fn evolve(state: State, event: Event) -> State { } } -fn codec() -> factos_pog.EventCodec(Event) { - factos_pog.codec(encode:, decode:) +fn codec() -> factos.EventCodec(Event, String) { + factos.codec(encode:, decode:) } -fn encode(event: Event) -> factos_pog.Proposed { +fn encode(event: Event) -> factos.Event(String) { let Incremented(value:) = event - factos_pog.new_proposed( + factos.new_event( type_: factos.event_type("Incremented"), version: 1, - data: json.int(value), + data: json.int(value) |> json.to_string, ) - |> factos_pog.with_tags(tags: [factos.tag("benchmark")]) + |> factos.with_tags(tags: [factos.tag("benchmark")]) } fn decode( - descriptor: factos.EventDescriptor, -) -> Result(decode.Decoder(Event), factos_pog.DecodeError) { - case factos.event_type_name(descriptor.type_), descriptor.version { - "Incremented", 1 -> Ok(decode.int |> decode.map(Incremented)) - _, _ -> Error(factos_pog.UnknownEvent) + stored: factos.Recorded(String), +) -> Result(Event, factos.DecodeError) { + case + factos.event_type_name(stored.descriptor.type_), + stored.descriptor.version + { + "Incremented", 1 -> + json.parse(stored.event, using: decode.int |> decode.map(Incremented)) + |> result.map_error(fn(_) { factos.InvalidData }) + _, _ -> Error(factos.UnknownEvent) } } diff --git a/backends/factos_pog/docs/durable-effects.md b/backends/factos_pog/docs/durable-effects.md index 394fecb..6d78fde 100644 --- a/backends/factos_pog/docs/durable-effects.md +++ b/backends/factos_pog/docs/durable-effects.md @@ -1,8 +1,8 @@ # Durable Effects -`FireAndForget(supervisor:)` is not durable delivery. It starts only after a -successful dispatch commit, but missing supervision, a callback panic, or a node -failure can drop the work. It has no catch-up or retry. +`factos.FireAndForget` is not durable delivery. It starts only after a +successful dispatch commit, but a callback panic or node failure can drop the +work. It has no catch-up or retry. For durable external work, use an application-owned transactional outbox. Read [How `factos_pog` works](how-it-works.html) for the transaction model and @@ -17,15 +17,13 @@ type Effect { CreditLedger(account_id: String, amount: Int) } -fn ledger_reactor() -> factos.Reactor(Event, Effect) { - factos.reactor(fn(recorded) { - case recorded.event { - PaymentCaptured(account_id:, amount:) -> [ - CreditLedger(account_id:, amount:), - ] - PaymentDeclined(account_id: _, reason: _) -> [] - } - }) +fn ledger_effects(recorded: factos.Recorded(Event)) -> List(Effect) { + case recorded.event { + PaymentCaptured(account_id:, amount:) -> [ + CreditLedger(account_id:, amount:), + ] + PaymentDeclined(account_id: _, reason: _) -> [] + } } ``` @@ -39,11 +37,11 @@ supplied transaction connection: ```gleam let durable_ledger_effects = - factos_pog.new_subscription( - query: factos.AllEvents, - consistency: factos_pog.StrongConsistency, + factos.new_subscription( + decision_context: factos.AllEvents, + consistency: factos.StrongConsistency, handle: fn(transaction_connection, recorded) { - let effects = factos.react(ledger_reactor(), recorded) + let effects = ledger_effects(recorded) ledger_outbox.insert_all( transaction_connection, source_event_id: recorded.id, @@ -53,27 +51,27 @@ let durable_ledger_effects = ) let assert Ok(dispatch) = - factos_pog.new_dispatch( + factos.new_dispatch( connection:, - stream: payment_stream, + decision_context: payment_context(command), decider: payment_decider(), codec: event_codec(), ) - |> factos_pog.with_subscriptions(subscriptions: [ + |> factos.with_subscriptions(subscriptions: [ durable_ledger_effects, ]) |> factos_pog.dispatch(command, event_id: uuid.v4_string) ``` The accepted event and its outbox rows commit or roll back together. A strong -callback error returns `SubscriptionError(error:)` and rolls back the dispatch. +callback error returns `factos.SubscriptionError(error:)` and rolls back the dispatch. PostgreSQL serialization and deadlock failures can rerun the callback in a new transaction. Writes from the aborted attempt do not survive. Do not call the external destination from this callback; that side effect cannot be rolled back. Give each outbox operation a deterministic unique identity. When one operation -maps to one source event, combine `Recorded.id` with the effect kind. When one +maps to one source event, combine `recorded.id` with the effect kind. When one domain operation spans several events, use the stable domain-operation identity instead. @@ -117,9 +115,9 @@ directly from a strong subscription: ```gleam let user_projection = - factos_pog.new_subscription( - query: factos.AllEvents, - consistency: factos_pog.StrongConsistency, + factos.new_subscription( + decision_context: factos.AllEvents, + consistency: factos.StrongConsistency, handle: fn(transaction_connection, recorded) { user_projection.apply(transaction_connection, recorded) }, @@ -133,8 +131,10 @@ operation identity. ## Replay is an application decision -Projection rebuilds can read historical records with `read_after` and replace -derived tables under an application-controlled checkpoint. +Projection rebuilds require an application-owned recovery reader over the +globally ordered event log and an application-controlled checkpoint. +`factos_pog` dispatch subscriptions intentionally process only records accepted +by their attached dispatch; they do not provide historical replay. External-effect replay is more dangerous: replaying events may resend email, charge accounts, or republish messages. Require an operator to select the range diff --git a/backends/factos_pog/docs/how-it-works.md b/backends/factos_pog/docs/how-it-works.md index 798fa42..077bd82 100644 --- a/backends/factos_pog/docs/how-it-works.md +++ b/backends/factos_pog/docs/how-it-works.md @@ -1,101 +1,118 @@ # How `factos_pog` Works -`factos_pog` is the PostgreSQL event-store backend for Factos. It owns event -persistence, command-context consistency, dispatch-bound subscription -execution, and ordered recovery reads. Applications own codecs, projections, -external effects, durable outboxes, checkpoint policy, and application-level -supervision. +`factos_pog` owns PostgreSQL event persistence, command-context consistency, +transaction retries, and dispatch-bound subscription execution. Applications own +domain events, codecs, projections, durable outboxes, external delivery, and +application supervision. ## Dispatch flow -`new_dispatch` receives an event codec and creates command-specific consistency -configuration. Without `with_query`, one stream revision is the consistency -boundary. With `with_query`, the boundary is an arbitrary Factos event-type/tag -query. +`factos.new_dispatch` requires the command's `factos.DecisionContext` together +with its decider and event codec. The context is not inferred: + +- `NoContext` reads no history; +- `AllEvents` reads the complete event log; +- `Matching(items:)` selects history by event type and tags. A dispatch: 1. opens a PostgreSQL `SERIALIZABLE` transaction; -2. reads and decodes the relevant committed events; +2. reads and decodes the records selected by the decision context; 3. folds them into temporary state with the decider's `evolve` function; 4. calls the pure `decide` function; -5. verifies that no relevant event appeared after the observed position; +5. verifies that no selected record appeared after the observed position; 6. inserts accepted event and tag-index rows; -7. filters the accepted records for each strong subscription and runs its - callbacks with the transaction connection; -8. commits the transaction; -9. starts matching fire-and-forget subscriptions as supervised temporary work. +7. runs matching strong subscription callbacks with the transaction connection; +8. commits; +9. starts matching fire-and-forget subscriptions in independent processes. + +`Dispatch.events` is the append-ordered list accepted by that dispatch. +`Dispatch.position` is the final global position, or `NoPosition` when +the command produced no events. An empty dispatch invokes no subscription +callback. + +## Retry boundary -`Dispatch.events` is the append-ordered list of records accepted by that -dispatch. No-event commands invoke no subscription callbacks. +Only PostgreSQL serialization failures (`40001`) and deadlocks (`40P01`) retry +automatically. The builder's attempt count includes the first attempt and +defaults to five. -Serialization and deadlock conflicts retry up to the builder's configured -attempt count. Deciders, codecs, event-id generators, and strong subscription -callbacks can therefore run more than once. Deciders, codecs, and event-id -generators must be pure. Strong callbacks must keep side effects on the supplied -transaction connection so an aborted attempt rolls them back. +The complete transaction reruns, so a decider, codec, event-id function, or +strong callback may execute more than once for one logical command. Deciders +and codecs must be deterministic and side-effect free. Generate event ids +locally; do not reserve them through an external service. Strong callbacks must +keep all effects on the supplied transaction connection so an aborted attempt +rolls them back. + +Domain rejection, callback errors, decoding errors, other store errors, and an +explicit `factos.AppendConditionFailed` result do not retry. ## Storage model -`factos_events` is append-only. It stores global position, application event id, -stream revision, type, version, tags, JSONB metadata, and JSONB event data. -`factos_event_tags(position, tag)` mirrors tags into indexed rows used by context -queries. +`factos_events` is append-only. Each row stores: + +- a commit-ordered global position; +- a globally unique UUIDv4 application event id; +- event type and schema version; +- tags; +- JSONB application metadata; +- JSONB event data. + +`factos_event_tags(position, tag)` mirrors tags into indexed rows for selective +context reads. -The unique `(stream, revision)` constraint protects stream order. Event ids must -be UUIDv4 strings and are globally unique. A transaction-scoped append lock is -acquired before identity values are allocated, so global event positions follow -commit order rather than the order of aborted insert attempts. +There is no stream or per-stream revision. A transaction-scoped advisory append +lock is acquired before PostgreSQL allocates identity values, making global +positions follow commit order rather than the order of aborted insert attempts. The schema contains no subscription table, cursor, checkpoint, projection, or effect outbox. ## Command-context consistency -A Factos query is a disjunction of query items. Within one item, type matching -and every requested tag are conjunctive. Empty type or tag lists mean no -restriction for that dimension; an empty query matches nothing. +For `Matching`, items are OR-combined. Within one item, event types are +OR-combined and tags are AND-combined. Empty types match any type; empty tags +add no tag restriction. An empty item list matches no records. -For context dispatch, `factos_pog` records the highest selected global position -and appends with `FailIfEventsMatch(query, position)`. PostgreSQL serializable -isolation protects both the read predicate and append. A conflicting concurrent -transaction aborts instead of allowing both commands to accept stale context. +The read records the highest global position among selected events. Append uses: -For stream dispatch, the current stream revision is the append boundary. +```gleam +factos.FailIfEventsMatch( + decision_context: decision_context, + after: observed_position, +) +``` -Command-context consistency ends when the event transaction commits. A strong -subscription can place a PostgreSQL projection inside that same boundary. -Separately maintained projections have their own consistency model. +PostgreSQL serializable isolation protects both the selection predicate and the +append. A conflicting transaction aborts rather than allowing both commands to +accept decisions made from stale facts. + +`NoContext` matches nothing and is therefore an unconditional append. +`AllEvents` creates a global consistency boundary. Choose the narrowest context +that contains every fact capable of changing the command's answer. ## Dispatch-bound subscriptions -`new_subscription` combines a Factos query, one consistency mode, and a callback. -The dispatch codec has already decoded all accepted records. Filtering uses -their event descriptors, and matching callbacks receive the complete -`factos.Recorded` envelope. +`factos.new_subscription` combines a decision context, one consistency mode, and a +callback. It filters only the accepted records from the dispatch carrying that +subscription. The dispatch codec has already decoded those records. Strong subscriptions run in list order. Within one subscription, matching -records run in event append order. The first callback `Error` becomes -`SubscriptionError(error:)`; the surrounding transaction rolls back all -accepted events and every strong callback write. Callback errors are not -retryable, while backend serialization and deadlock errors are. - -Fire-and-forget subscriptions are enqueued only after the final commit. Each -matching subscription gets one temporary child under the application-installed -`SubscriptionSupervisor`; that child processes its matching records in append -order. Separate subscriptions and dispatches may run concurrently. - -A fire callback's return value cannot alter the committed result. Returned -errors are ignored and processing continues. A panic terminates the temporary -child and drops its remaining records. Missing or restarting supervision also -drops the work. There is no catch-up or retry. - -## Ordered recovery reads - -`read_after` loads matching records from the durable event log in global -position order. A custom durable consumer polls this API, persists an -application-owned checkpoint, and defines its own ownership, batching, -concurrency, retry, and poison-event policy. +records run in append order. The first callback `Error` becomes +`factos.SubscriptionError(error:)`; PostgreSQL rolls back the accepted events, tag +rows, and every strong callback write in that transaction. A callback error is +not retryable, although an enclosing serialization or deadlock failure can rerun +the callback in a fresh attempt. + +After the final commit, each matching `factos.FireAndForget` subscription gets +one independent process. That process handles matching records in append order +with the builder's ordinary Pog connection. Separate subscriptions and +dispatches may run concurrently. + +A fire-and-forget callback's return value cannot alter the committed dispatch. +Returned errors are ignored and processing continues. A panic terminates the +process and drops its remaining records. There is no historical catch-up or +retry. ## Effects and delivery guarantees @@ -103,8 +120,7 @@ No transaction can atomically commit PostgreSQL rows and an external HTTP request, message publish, or email send. Fire-and-forget work is therefore best-effort. -For durable external delivery, a strong callback can insert an -application-owned outbox row using the dispatch transaction connection. The -event, projection changes, and outbox row then commit or roll back together. An -application worker owns delivery retries, backoff, idempotency, dead letters, -and operational replay. +For durable delivery, a strong callback inserts an application-owned outbox row +through the dispatch transaction connection. The event, projection changes, and +outbox row then commit or roll back together. An application worker owns +delivery retries, backoff, idempotency, dead letters, and operational replay. diff --git a/backends/factos_pog/docs/subscriptions.md b/backends/factos_pog/docs/subscriptions.md index ab09bea..19fce1c 100644 --- a/backends/factos_pog/docs/subscriptions.md +++ b/backends/factos_pog/docs/subscriptions.md @@ -1,79 +1,83 @@ # Subscriptions -`factos_pog` subscriptions bind callbacks to one command dispatch. They filter -the decoded records in `Dispatch.events`; they do not own a cursor or consume -historical events. +`factos.Subscription` binds a callback to one command dispatch. Subscriptions +filter the decoded records in `Dispatch.events`; they do not consume historical +or observe records committed by another dispatch. Choose the consistency mode from the callback's required commit boundary: -- `StrongConsistency` runs inside the dispatch transaction. -- `FireAndForget(supervisor:)` starts supervised best-effort work after commit. +- `factos.StrongConsistency` runs inside the event transaction; +- `factos.FireAndForget` starts best-effort work after commit. ## Configure and attach subscriptions -Queries use event types and tags. Values needed for selective routing must be -stored as tags because subscription filtering does not inspect event JSON: +Subscriptions use the same `factos.DecisionContext` matching rules as command +dispatch. Values needed for selective routing must be stored as tags because +matching does not inspect event JSON: ```gleam let registrations = - factos.query([ - factos.query_item( + factos.Matching(items: [ + factos.item( types: [factos.event_type("UserRegistered")], tags: [], ), ]) let user_projection_subscription = - factos_pog.new_subscription( - query: registrations, - consistency: factos_pog.StrongConsistency, + factos.new_subscription( + decision_context: registrations, + consistency: factos.StrongConsistency, handle: fn(transaction_connection, recorded) { user_projection.apply(transaction_connection, recorded) }, ) ``` -`new_subscription` is infallible. The codec attached to the dispatch has -already decoded each `factos.Recorded` envelope before subscription filtering. -The callback receives the complete event id, stream, revision, global position, -descriptor, metadata, tags, and domain event. +`new_subscription` is infallible. `NoContext` matches no accepted record, +`AllEvents` matches every accepted record, and `Matching(items:)` applies +type-and-tag selection. -Attach the subscriptions to a dispatch builder: +The dispatch codec has already decoded each matching +`factos.Recorded(event)`. A callback receives the event id, global position, +descriptor, metadata, tags, and domain event. There is no stream or per-stream +revision in the envelope. + +Attach the complete list to a dispatch builder: ```gleam let assert Ok(dispatch) = - factos_pog.new_dispatch( + factos.new_dispatch( connection:, - stream: "user-renata", + decision_context: registration_context(username), decider: user_decider(), codec: event_codec(), ) - |> factos_pog.with_subscriptions(subscriptions: [ + |> factos.with_subscriptions(subscriptions: [ user_projection_subscription, ]) |> factos_pog.dispatch( - RegisterUser(user_id: "renata"), + RegisterUser(username:), event_id: uuid.v4_string, ) ``` -`with_subscriptions` replaces the builder's complete list. Every subscription -in that list must use the same callback error type. +`with_subscriptions` replaces the complete list, allowing the builder to adopt +the callbacks' shared error type without an error-conversion wrapper. -Subscriptions only receive records accepted by this dispatch. They do not -replay older rows or observe events committed by other dispatches. +An empty dispatch and a non-matching subscription invoke no callback. ## Strong consistency -Strong callbacks run after the event and tag rows have been inserted but before -the transaction commits. Use the supplied transaction connection for all -PostgreSQL work: +Strong callbacks run after accepted event and tag rows have been inserted but +before commit. Use the supplied transaction connection for every PostgreSQL +operation: ```gleam let projection = - factos_pog.new_subscription( - query: factos.AllEvents, - consistency: factos_pog.StrongConsistency, + factos.new_subscription( + decision_context: factos.AllEvents, + consistency: factos.StrongConsistency, handle: fn(transaction_connection, recorded) { user_projection.apply(transaction_connection, recorded) }, @@ -81,73 +85,49 @@ let projection = ``` The backend traverses subscriptions in list order and matching records in event -append order. A callback can therefore observe writes made by an earlier strong -callback in the same dispatch. +append order. A callback can observe database writes made by an earlier strong +callback in the same dispatch transaction. The first returned `Error` becomes -`factos_pog.SubscriptionError(error: callback_error)`. PostgreSQL rolls back: +`factos.SubscriptionError(error: callback_error)`. PostgreSQL rolls back: -- all event and tag rows accepted by the command; +- every event and tag row accepted by the command; - the failing callback's database writes; -- every earlier strong callback write in that transaction. +- every earlier strong callback write in the transaction. Callback errors are not retryable. PostgreSQL serialization failures and -deadlocks remain retryable, so a strong callback may run again after an aborted -attempt. Its external effects cannot be rolled back. Keep all side effects on -the supplied transaction connection. - -An empty dispatch and a non-matching query invoke no callback. +deadlocks are retryable, so a strong callback can run again after an aborted +attempt. External effects cannot be rolled back. Keep all strong callback work +on the supplied transaction connection. ## Fire and forget -Allocate one opaque subscription supervisor when the application starts: - -```gleam -let subscription_supervisor = factos_pog.new_subscription_supervisor() -``` - -Install its child specification in the application supervision tree. There is -deliberately no unsupervised convenience starter: - -```gleam -static_supervisor.new(strategy: static_supervisor.OneForOne) -|> static_supervisor.add( - factos_pog.supervised_subscription_supervisor(subscription_supervisor), -) -|> static_supervisor.start -``` - -Pass that same value to each fire-and-forget subscription: +Use the shared fire-and-forget consistency mode: ```gleam let observer = - factos_pog.new_subscription( - query: registrations, - consistency: factos_pog.FireAndForget( - supervisor: subscription_supervisor, - ), + factos.new_subscription( + decision_context: registrations, + consistency: factos.FireAndForget, handle: fn(connection, recorded) { registration_observer.handle(connection, recorded) }, ) ``` -After a successful final commit, the backend starts one supervised temporary -child per matching fire-and-forget subscription. Each child invokes its -callback for matching records in append order. Different subscriptions and -different dispatches can run concurrently, so there is no global ordering -guarantee. - -The dispatch does not wait for these callbacks. Their `Ok` or `Error` result -cannot change the committed dispatch; a returned `Error` is ignored and the -child continues with later records. A panic terminates that temporary child and -drops its remaining records. Temporary children are never restarted. +After the final successful commit, the backend starts one process per matching +fire-and-forget subscription. Each process receives the builder's ordinary Pog +connection and invokes its callback for matching records in append order. +Different subscriptions and dispatches can run concurrently; there is no +global callback ordering guarantee. -If the supervisor is missing or restarting, the already committed dispatch -still succeeds and the work is dropped. +Dispatch does not wait for these callbacks. Their `Ok` or `Error` result cannot +change the committed dispatch. A returned `Error` is ignored and the process +continues with later records. A panic terminates that process and drops its +remaining records. -Fire-and-forget has no cursor, catch-up, checkpoint, reset, replay, retry -schedule, or dead-letter policy. It is suitable only for best-effort work. +Fire-and-forget has no historical catch-up, cursor, checkpoint, replay, retry, +or dead-letter policy. It is suitable only for best-effort work. ## Durable external work @@ -156,16 +136,6 @@ PostgreSQL event transaction. Fire-and-forget therefore cannot provide durable external delivery. For durable effects, have a strong callback insert an application-owned outbox -row with the event transaction. A separate application worker can deliver that -row with its own retry and idempotency policy. See -[Durable effects](durable-effects). - -## Custom consumer runtimes - -`read_after` remains public for applications that need historical catch-up, -batching, or an independent checkpoint protocol. A custom durable runtime polls -ordered records, persists its own checkpoint, and resumes from that checkpoint -after startup or failure. - -This API does not impose a scheduler, ownership model, retry policy, or -checkpoint schema. +row with the event transaction. A separate application worker delivers that row +with its own retry and idempotency policy. See +[Durable effects](durable-effects.html). diff --git a/backends/factos_pog/priv/dbmate/20260816000100_factos_pog_v2.sql b/backends/factos_pog/priv/dbmate/20260816000100_factos_pog_v2.sql index e6545be..37b2e98 100644 --- a/backends/factos_pog/priv/dbmate/20260816000100_factos_pog_v2.sql +++ b/backends/factos_pog/priv/dbmate/20260816000100_factos_pog_v2.sql @@ -7,6 +7,15 @@ alter table factos_events alter column metadata type jsonb using metadata::jsonb, alter column data type jsonb using convert_from(data, 'UTF8')::jsonb; +-- Stream names and revisions are not part of the Factos event model. This +-- cutover intentionally discards them. Event identity and global position +-- preserve the authoritative event history. +drop index if exists factos_events_stream_revision; +alter table factos_events + drop constraint factos_events_stream_revision_key, + drop column stream, + drop column revision; + -- Sequence values are allocated before commit. Taking this transaction-scoped -- lock before defaults are evaluated makes event positions commit ordered. @@ -29,6 +38,21 @@ execute function factos_pog_lock_event_append(); drop trigger if exists factos_pog_event_append_lock on factos_events; drop function if exists factos_pog_lock_event_append(); +-- The removed stream metadata cannot be reconstructed. Give each event a +-- synthetic one-event stream so the v1 uniqueness contract can be restored. +alter table factos_events + add column stream text, + add column revision integer; +update factos_events +set stream = 'factos-v2-rollback-' || position::text, + revision = 0; +alter table factos_events + alter column stream set not null, + alter column revision set not null, + add constraint factos_events_stream_revision_key unique (stream, revision); +create index factos_events_stream_revision + on factos_events(stream, revision); + alter table factos_events alter column metadata type text using metadata::text, alter column data type bytea using convert_to(data::text, 'UTF8'), diff --git a/backends/factos_pog/priv/migrations.sql b/backends/factos_pog/priv/migrations.sql index 9a11e70..85aa2ce 100644 --- a/backends/factos_pog/priv/migrations.sql +++ b/backends/factos_pog/priv/migrations.sql @@ -3,19 +3,13 @@ create table if not exists factos_events ( id text not null unique check ( id ~* '^[0-9a-f]{8}-[0-9a-f]{4}-4[0-9a-f]{3}-[89ab][0-9a-f]{3}-[0-9a-f]{12}$' ), - stream text not null, - revision integer not null, type text not null, version integer not null, tags text not null, metadata jsonb not null, - data jsonb not null, - unique(stream, revision) + data jsonb not null ); -create index if not exists factos_events_stream_revision - on factos_events(stream, revision); - create index if not exists factos_events_position on factos_events(position); diff --git a/backends/factos_pog/src/factos/factos_pog.gleam b/backends/factos_pog/src/factos/factos_pog.gleam index 54ccef2..7b9ba3b 100644 --- a/backends/factos_pog/src/factos/factos_pog.gleam +++ b/backends/factos_pog/src/factos/factos_pog.gleam @@ -1,19 +1,19 @@ //// PostgreSQL backend for Factos using the `pog` package. //// //// This backend stores accepted facts in an append-only `factos_events` table. -//// The event history is the source of truth; projections and stream-shaped reads -//// are derived views over that history. +//// The globally ordered event history is the source of truth; projections and +//// other read models are application-owned derived views. //// -//// The context dispatch flow follows the Command Context Consistency idea from -//// "Simply Event Sourcing": a command selects the facts required for its decision, -//// folds them into temporary state, decides new facts, and appends those facts -//// only when no relevant facts appeared after the observed context position. +//// The dispatch flow follows Command Context Consistency: a command selects the +//// facts required for its decision, folds them into temporary state, decides new +//// facts, and appends only when no selected fact appeared after the observed +//// context position. //// -//// The query contract is intentionally tag-based. PostgreSQL stores JSON event -//// data, an event type, and tags. This keeps domain serialization outside the -//// backend, but it means any payload value needed for a selective consistency -//// query must be written as a tag. +//// Selective contexts use event types and tags. PostgreSQL stores JSON event +//// data without understanding the domain payload, so any payload value needed +//// for context selection must also be written as a tag. +import exception import factos import gleam/dict import gleam/dynamic/decode @@ -21,261 +21,56 @@ import gleam/erlang/process import gleam/int import gleam/json import gleam/list -import gleam/otp/actor -import gleam/otp/factory_supervisor -import gleam/otp/supervision import gleam/result import gleam/string import pog -/// A domain event prepared for PostgreSQL persistence. -/// -/// Start with `new_proposed`, then optionally replace its empty tags or metadata -/// through the builder functions. -pub opaque type Proposed { - Proposed(data: json.Json, descriptor: factos.EventDescriptor) -} - -/// Prepare a domain event with empty tags and metadata for persistence. -pub fn new_proposed( - type_ type_: factos.EventType, - version version: Int, - data data: json.Json, -) -> Proposed { - Proposed( - descriptor: factos.EventDescriptor( - type_:, - version:, - tags: [], - metadata: factos.empty_metadata(), - ), - data:, - ) -} - -/// Replace the tags on a proposed event. -pub fn with_tags(proposed: Proposed, tags tags: List(factos.Tag)) -> Proposed { - Proposed( - ..proposed, - descriptor: factos.EventDescriptor(..proposed.descriptor, tags:), - ) -} - -/// Replace the metadata on a proposed event. -pub fn with_metadata( - proposed: Proposed, - metadata metadata: factos.Metadata, -) -> Proposed { - Proposed( - ..proposed, - descriptor: factos.EventDescriptor(..proposed.descriptor, metadata:), - ) -} - -/// Application-owned PostgreSQL event codec. -/// -/// `encode` turns a domain event into stored JSON and its descriptor. `decode` -/// selects a JSON decoder from the stored descriptor, so tagged union event -/// types can validate the event type and version before decoding their payload. -/// -/// WARNING: codecs used by dispatch must be pure. Serializable dispatch may call -/// either function more than once for the same logical operation. -pub type EventCodec(event) { - EventCodec( - encode: fn(event) -> Proposed, - decode: fn(factos.EventDescriptor) -> - Result(decode.Decoder(event), DecodeError), - ) -} - -type StoredRow { - StoredRow( - position: Int, - id: String, - stream: String, - revision: Int, - descriptor: factos.EventDescriptor, - data: String, - ) -} - -pub type Append { - /// Result of a successful append. - /// - /// `current_revision` is the latest revision of the target stream after the - /// append. `position` is the global position of the last inserted event, or - /// `NoPosition` when no events were produced. - Append(current_revision: Int, position: factos.SequencePosition) -} - -pub type Dispatch(event) { - /// Result of a successful dispatch. - /// - /// `append` has the stream revision and final global position. `events` are the - /// committed events recorded by this dispatch, suitable for pure Factos - /// reactors and application-owned durable subscribers. - Dispatch(append: Append, events: List(factos.Recorded(event))) -} - -pub opaque type DispatchBuilder( - command, - state, - event, - domain_error, - subscription_error, -) { - DispatchBuilder( - connection: pog.Connection, - stream: String, - query: DispatchQuery, - decider: factos.Decider(command, state, event, domain_error), - codec: EventCodec(event), - retry_attempts: Int, - subscriptions: List(Subscription(event, subscription_error)), - ) -} - -/// When a subscription callback executes relative to the event commit. -pub type SubscriptionConsistency { - FireAndForget( - name: process.Name(factory_supervisor.Message(fn() -> Nil, Nil)), - ) - StrongConsistency -} - -/// A dispatch-bound event subscription. -/// -/// The query filters the already-decoded events produced by one dispatch. -pub opaque type Subscription(event, subscription_error) { - Subscription( - query: factos.Query, - consistency: SubscriptionConsistency, - handle: fn(pog.Connection, factos.Recorded(event)) -> - Result(Nil, subscription_error), - ) -} - -type DispatchQuery { - StreamQuery - ContextQuery(factos.Query) -} - -/// Start building an event dispatch. -/// -/// By default the builder uses one-stream consistency, 5 attempts for retryable -/// serializable transaction conflicts, and no subscriptions. -/// -/// WARNING: decider and codec functions must be pure. Strong subscription -/// callbacks may run again after a serialization or deadlock retry. -pub fn new_dispatch( - connection connection: pog.Connection, - stream stream_name: String, - decider decider: factos.Decider(command, state, event, domain_error), - codec codec: EventCodec(event), -) -> DispatchBuilder(command, state, event, domain_error, Nil) { - DispatchBuilder( - connection:, - stream: stream_name, - query: StreamQuery, - decider:, - codec:, - retry_attempts: 5, - subscriptions: [], - ) -} - -/// Use a context query instead of one-stream consistency. -pub fn with_query( - builder: DispatchBuilder( - command, - state, - event, - domain_error, - subscription_error, - ), - query query: factos.Query, -) -> DispatchBuilder(command, state, event, domain_error, subscription_error) { - DispatchBuilder(..builder, query: ContextQuery(query)) -} - -/// Replace the subscriptions attached to this dispatch. -/// -/// Replacing the complete list lets the builder adopt the callbacks' shared -/// error type without an error-conversion wrapper. -pub fn with_subscriptions( - builder: DispatchBuilder( - command, - state, - event, - domain_error, - previous_subscription_error, - ), - subscriptions subscriptions: List(Subscription(event, subscription_error)), -) -> DispatchBuilder(command, state, event, domain_error, subscription_error) { - DispatchBuilder(..builder, subscriptions:) +type QuerySql { + QuerySql(sql: String, parameters: List(QueryParameter)) } -/// Override retry attempts for PostgreSQL serializable/deadlock conflicts. -pub fn with_retry_attempts( - builder: DispatchBuilder( - command, - state, - event, - domain_error, - subscription_error, - ), - attempts attempts: Int, -) -> DispatchBuilder(command, state, event, domain_error, subscription_error) { - DispatchBuilder(..builder, retry_attempts: int.max(attempts, 1)) +type QueryParameter { + IntParameter(Int) + TextParameter(String) } /// Dispatch a command and atomically append its accepted events. pub fn dispatch( - builder: DispatchBuilder( + builder: factos.DispatchBuilder( command, state, event, + String, domain_error, subscription_error, + pog.Connection, ), command: command, event_id event_id: fn() -> String, -) -> Result(Dispatch(event), Error(domain_error, subscription_error)) { - let DispatchBuilder( +) -> Result( + factos.Dispatch(event), + factos.Error(domain_error, subscription_error, pog.QueryError), +) { + let factos.DispatchBuilder( connection:, - stream:, - query:, + decision_context:, decider:, codec:, retry_attempts:, subscriptions:, ) = builder - let result = case query { - StreamQuery -> - dispatch_stream( - connection, - stream:, - decider:, - codec:, - command:, - event_id:, - retry_attempts:, - subscriptions:, - ) - ContextQuery(query) -> - dispatch_query( - connection, - stream:, - query:, - decider:, - codec:, - command:, - event_id:, - retry_attempts:, - subscriptions:, - ) - } + let result = + dispatch_context( + connection, + decision_context:, + decider:, + codec:, + command:, + event_id:, + retry_attempts:, + subscriptions:, + ) case result { Error(error) -> Error(error) @@ -290,177 +85,110 @@ pub fn dispatch( } } -pub type Error(domain_error, subscription_error) { - /// The decider rejected the command with a domain error. - DomainError(domain_error) - - /// A strong subscription callback rejected an event. - SubscriptionError(error: subscription_error) - - /// PostgreSQL or `pog` returned an error while running a query. - StoreError(pog.QueryError) - - /// A stream revision or context append condition failed. - AppendConditionFailed(factos.AppendCondition) - DecodeError(DecodeError) -} - -pub type DecodeError { - UnknownEvent - InvalidData -} - -type QuerySql { - QuerySql(sql: String, parameters: List(QueryParameter)) -} - -type QueryParameter { - IntParameter(Int) - TextParameter(String) -} - -/// Configure a dispatch-bound subscription. -/// -/// Strong callbacks share the dispatch transaction. Fire-and-forget callbacks -/// are scheduled as supervised temporary work only after a successful commit. -pub fn new_subscription( - query query: factos.Query, - consistency consistency: SubscriptionConsistency, - handle handle: fn(pog.Connection, factos.Recorded(event)) -> - Result(Nil, subscription_error), -) -> Subscription(event, subscription_error) { - Subscription(query:, consistency:, handle:) -} - -/// Install a subscription supervisor in an application supervision tree. -pub fn supervised( - name: process.Name(_), -) -> supervision.ChildSpecification(Nil) { - factory_supervisor.worker_child(start_fire_and_forget_child) - |> factory_supervisor.named(name) - |> factory_supervisor.restart_strategy(supervision.Temporary) - |> factory_supervisor.supervised - |> supervision.map_data(fn(_) { Nil }) -} - -fn start_fire_and_forget_child(work: fn() -> Nil) -> actor.StartResult(Nil) { - let pid = process.spawn(work) - Ok(actor.Started(pid:, data: Nil)) -} - -/// Create a new codec. -/// -/// `encode` prepares a domain event for storage. `decode` selects the JSON -/// decoder for a stored descriptor. Return `UnknownEvent` for unsupported -/// type/version combinations; malformed payloads become `InvalidData`. -/// -/// WARNING: codecs used by dispatch must be pure. Serializable dispatch may -/// retry and call either function more than once for the same logical operation. -pub fn codec( - encode encode: fn(event) -> Proposed, - decode decode: fn(factos.EventDescriptor) -> - Result(decode.Decoder(event), DecodeError), -) -> EventCodec(event) { - EventCodec(encode:, decode:) -} - @internal pub fn read( connection: pog.Connection, - query query: factos.Query, + decision_context decision_context: factos.DecisionContext, decider decider: factos.Decider(command, state, event, domain_error), - codec codec: EventCodec(event), + codec codec: factos.EventCodec(event, String), ) -> Result( factos.Context(event, state), - Error(domain_error, subscription_error), + factos.Error(domain_error, subscription_error, pog.QueryError), ) { - let factos.Decider(initial, _, evolve) = decider + let factos.Decider(initial:, evolve:, ..) = decider - use events <- result.try(read_matching_events(connection, query, codec)) - let position = highest_recorded_position(events) + use events <- result.try(read_matching_events( + connection, + decision_context, + codec, + )) + let position = factos.highest_recorded_position(events) Ok(factos.Context( - query:, - state: factos.evolve_recorded( - initial: initial, - events: events, - evolve: evolve, + decision_context:, + state: factos.evolve_recorded(initial:, events:, evolve:), + events:, + position:, + append_condition: factos.FailIfEventsMatch( + decision_context:, + after: position, ), - events: events, - position: position, - append_condition: factos.FailIfEventsMatch(query, position), )) } @internal pub fn read_after( connection: pog.Connection, - query query: factos.Query, + decision_context decision_context: factos.DecisionContext, after after: factos.SequencePosition, limit limit: Int, - codec codec: EventCodec(event), + codec codec: factos.EventCodec(event, String), ) -> Result( List(factos.Recorded(event)), - Error(domain_error, subscription_error), + factos.Error(domain_error, subscription_error, pog.QueryError), ) { case limit <= 0 { True -> Ok([]) False -> { let QuerySql(where_sql, parameters) = - matching_events_after_sql_from(query, after, parameter_index: 1) + matching_events_after_sql_from( + decision_context, + after, + parameter_index: 1, + ) let limit_parameter = "$" <> int.to_string(list.length(parameters) + 1) use rows <- result.try( pog.query( - "select position, id, stream, revision, type, version, tags, metadata, data::text + "select position, id, type, version, tags, metadata, data::text from factos_events - " - <> where_sql - <> " + " <> where_sql <> " order by position - limit " - <> limit_parameter, + limit " <> limit_parameter, ) |> with_parameters(parameters) |> pog.parameter(pog.int(limit)) |> pog.returning(stored_row_decoder()) |> pog.execute(on: connection) |> result.map(fn(returned) { returned.rows }) - |> result.map_error(StoreError), + |> result.map_error(factos.StoreError), ) decode_stored_rows(rows, codec) } } } -fn dispatch_query( +fn dispatch_context( connection: pog.Connection, - stream stream_name: String, - query query: factos.Query, + decision_context decision_context: factos.DecisionContext, decider decider: factos.Decider(command, state, event, domain_error), - codec codec: EventCodec(event), + codec codec: factos.EventCodec(event, String), command command: command, event_id event_id: fn() -> String, retry_attempts retry_attempts: Int, - subscriptions subscriptions: List(Subscription(event, subscription_error)), -) -> Result(Dispatch(event), Error(domain_error, subscription_error)) { + subscriptions subscriptions: List( + factos.Subscription(event, subscription_error, pog.Connection), + ), +) -> Result( + factos.Dispatch(event), + factos.Error(domain_error, subscription_error, pog.QueryError), +) { use transaction_connection <- run_serializable_transaction( connection, retry_attempts, ) use context <- result.try(read( transaction_connection, - query:, - decider:, - codec:, + decision_context, + decider, + codec, )) use pair <- result.try( factos.decide_context(context, command, decider) - |> result.map_error(DomainError), + |> result.map_error(factos.DomainError), ) let #(context, events) = pair - use dispatch <- result.try(append_context_events( + use dispatch <- result.try(append_events( transaction_connection, - stream_name, events, codec, event_id, @@ -474,83 +202,18 @@ fn dispatch_query( Ok(dispatch) } -/// Load and fold one stream. -/// -/// This supports classic stream-revision consistency. The returned -/// `factos.LoadedStream` contains the folded state, decoded recorded events, and -/// current stream revision. -pub fn load_stream( - connection: pog.Connection, - stream stream_name: String, - decider decider: factos.Decider(command, state, event, domain_error), - codec codec: EventCodec(event), -) -> Result( - factos.LoadedStream(event, state), - Error(domain_error, subscription_error), -) { - let factos.Decider(initial, _, evolve) = decider - use events <- result.try(read_stream_events(connection, stream_name, codec)) - - Ok(factos.LoadedStream( - stream: stream_name, - state: factos.evolve_recorded( - initial: initial, - events: events, - evolve: evolve, - ), - events: events, - revision: stream_revision(events), - )) -} - -fn dispatch_stream( - connection: pog.Connection, - stream stream_name: String, - decider decider: factos.Decider(command, state, event, domain_error), - codec codec: EventCodec(event), - command command: command, - event_id event_id: fn() -> String, - retry_attempts retry_attempts: Int, - subscriptions subscriptions: List(Subscription(event, subscription_error)), -) -> Result(Dispatch(event), Error(domain_error, subscription_error)) { - use transaction_connection <- run_serializable_transaction( - connection, - retry_attempts, - ) - use loaded <- result.try(load_stream( - transaction_connection, - stream: stream_name, - decider:, - codec:, - )) - let factos.Decider(_, decide, _) = decider - use events <- result.try( - decide(loaded.state, command) - |> result.map_error(DomainError), - ) - use dispatch <- result.try(append_stream_events( - transaction_connection, - stream_name, - events, - codec, - loaded.revision, - event_id, - factos.NoAppendCondition, - )) - use _ <- result.try(run_strong_subscriptions( - transaction_connection, - subscriptions, - dispatch.events, - )) - Ok(dispatch) -} - fn run_serializable_transaction( connection: pog.Connection, retry_attempts: Int, work: fn(pog.Connection) -> - Result(Dispatch(event), Error(domain_error, subscription_error)), -) -> Result(Dispatch(event), Error(domain_error, subscription_error)) { + Result( + factos.Dispatch(event), + factos.Error(domain_error, subscription_error, pog.QueryError), + ), +) -> Result( + factos.Dispatch(event), + factos.Error(domain_error, subscription_error, pog.QueryError), +) { run_serializable_transaction_attempt( connection, work, @@ -561,13 +224,19 @@ fn run_serializable_transaction( fn run_serializable_transaction_attempt( connection: pog.Connection, work: fn(pog.Connection) -> - Result(Dispatch(event), Error(domain_error, subscription_error)), + Result( + factos.Dispatch(event), + factos.Error(domain_error, subscription_error, pog.QueryError), + ), attempts_remaining attempts_remaining: Int, -) -> Result(Dispatch(event), Error(domain_error, subscription_error)) { - let result = run_serializable_transaction_once(connection, work) - case result { +) -> Result( + factos.Dispatch(event), + factos.Error(domain_error, subscription_error, pog.QueryError), +) { + let transaction_result = run_serializable_transaction_once(connection, work) + case transaction_result { Ok(dispatch) -> Ok(dispatch) - Error(error) -> { + Error(error) -> case attempts_remaining > 1 && retryable_transaction_error(error) { True -> run_serializable_transaction_attempt( @@ -577,15 +246,20 @@ fn run_serializable_transaction_attempt( ) False -> Error(error) } - } } } fn run_serializable_transaction_once( connection: pog.Connection, work: fn(pog.Connection) -> - Result(Dispatch(event), Error(domain_error, subscription_error)), -) -> Result(Dispatch(event), Error(domain_error, subscription_error)) { + Result( + factos.Dispatch(event), + factos.Error(domain_error, subscription_error, pog.QueryError), + ), +) -> Result( + factos.Dispatch(event), + factos.Error(domain_error, subscription_error, pog.QueryError), +) { case { use transaction_connection <- pog.transaction(connection) @@ -594,49 +268,56 @@ fn run_serializable_transaction_once( } { Ok(dispatch) -> Ok(dispatch) - Error(pog.TransactionQueryError(error)) -> Error(StoreError(error)) + Error(pog.TransactionQueryError(error)) -> Error(factos.StoreError(error)) Error(pog.TransactionRolledBack(error)) -> Error(error) } } fn set_serializable_isolation( connection: pog.Connection, -) -> Result(Nil, Error(domain_error, subscription_error)) { +) -> Result(Nil, factos.Error(domain_error, subscription_error, pog.QueryError)) { pog.query("set transaction isolation level serializable") |> pog.execute(on: connection) |> result.map(nil_constant) - |> result.map_error(StoreError) + |> result.map_error(factos.StoreError) } fn retryable_transaction_error( - error: Error(domain_error, subscription_error), + error: factos.Error(domain_error, subscription_error, pog.QueryError), ) -> Bool { case error { - StoreError(pog.PostgresqlError(code: "40001", ..)) -> True - StoreError(pog.PostgresqlError(code: "40P01", ..)) -> True - DomainError(_) -> False - SubscriptionError(_) -> False - StoreError(_) -> False - AppendConditionFailed(_) -> False - DecodeError(_) -> False + factos.StoreError(pog.PostgresqlError(code: "40001", ..)) -> True + factos.StoreError(pog.PostgresqlError(code: "40P01", ..)) -> True + factos.DomainError(_) -> False + factos.SubscriptionError(error: _) -> False + factos.StoreError(_) -> False + factos.AppendConditionFailed(_) -> False + factos.DecodeError(_) -> False } } fn run_strong_subscriptions( connection: pog.Connection, - subscriptions: List(Subscription(event, subscription_error)), + subscriptions: List( + factos.Subscription(event, subscription_error, pog.Connection), + ), events: List(factos.Recorded(event)), -) -> Result(Nil, Error(domain_error, subscription_error)) { +) -> Result(Nil, factos.Error(domain_error, subscription_error, pog.QueryError)) { case subscriptions { [] -> Ok(Nil) - [Subscription(query:, consistency:, handle:), ..remaining] -> + [factos.Subscription(decision_context:, consistency:, handle:), ..remaining] -> case consistency { - FireAndForget(name: _) -> + factos.FireAndForget -> run_strong_subscriptions(connection, remaining, events) - StrongConsistency -> { + factos.StrongConsistency -> { use _ <- result.try( - run_strong_subscription_events(connection, query, handle, events) - |> result.map_error(SubscriptionError), + run_strong_subscription_events( + connection, + decision_context, + handle, + events, + ) + |> result.map_error(factos.SubscriptionError), ) run_strong_subscriptions(connection, remaining, events) } @@ -646,7 +327,7 @@ fn run_strong_subscriptions( fn run_strong_subscription_events( connection: pog.Connection, - query: factos.Query, + decision_context: factos.DecisionContext, handle: fn(pog.Connection, factos.Recorded(event)) -> Result(Nil, subscription_error), events: List(factos.Recorded(event)), @@ -654,39 +335,50 @@ fn run_strong_subscription_events( case events { [] -> Ok(Nil) [recorded, ..remaining] -> - case factos.matches_descriptor(recorded.descriptor, query) { + case factos.matches_decision_context(recorded, decision_context) { True -> { use _ <- result.try(handle(connection, recorded)) - run_strong_subscription_events(connection, query, handle, remaining) + run_strong_subscription_events( + connection, + decision_context, + handle, + remaining, + ) } False -> - run_strong_subscription_events(connection, query, handle, remaining) + run_strong_subscription_events( + connection, + decision_context, + handle, + remaining, + ) } } } fn enqueue_fire_and_forget_subscriptions( connection: pog.Connection, - subscriptions: List(Subscription(event, subscription_error)), + subscriptions: List( + factos.Subscription(event, subscription_error, pog.Connection), + ), events: List(factos.Recorded(event)), ) -> Nil { use subscription <- list.each(subscriptions) case subscription.consistency { - StrongConsistency -> Nil - - FireAndForget(name:) -> { + factos.StrongConsistency -> Nil + factos.FireAndForget -> { let events = - list.filter(events, fn(recorded) { - factos.matches_descriptor(recorded.descriptor, subscription.query) - }) + list.filter(events, factos.matches_decision_context( + _, + subscription.decision_context, + )) case events { [] -> Nil - events -> { - let factory = factory_supervisor.get_by_name(name) let _ = { - use <- factory_supervisor.start_child(factory) + use <- exception.rescue + use <- process.spawn() run_fire_and_forget_events(connection, events, subscription.handle) } Nil @@ -711,117 +403,65 @@ fn run_fire_and_forget_events( } } -fn append_context_events( +fn append_events( connection: pog.Connection, - stream_name: String, events: List(event), - codec: EventCodec(event), + codec: factos.EventCodec(event, String), event_id: fn() -> String, condition: factos.AppendCondition, -) -> Result(Dispatch(event), Error(domain_error, subscription_error)) { - use revision <- result.try( - current_revision(connection, stream_name) - |> result.map_error(StoreError), - ) - append_stream_events( - connection, - stream_name, - events, - codec, - factos.CurrentRevision(revision), - event_id, - condition, - ) -} - -fn append_stream_events( - connection: pog.Connection, - stream_name: String, - events: List(event), - codec: EventCodec(event), - expected: factos.Revision, - event_id: fn() -> String, - condition: factos.AppendCondition, -) -> Result(Dispatch(event), Error(domain_error, subscription_error)) { +) -> Result( + factos.Dispatch(event), + factos.Error(domain_error, subscription_error, pog.QueryError), +) { case events { - [] -> { - let append = - Append( - current_revision: revision_to_int(expected), - position: factos.NoPosition, - ) - Ok(Dispatch(append:, events: [])) - } - [_, ..] -> { + [] -> Ok(factos.Dispatch(position: factos.NoPosition, events: [])) + [_, ..] -> insert_events( connection, - stream_name, events, codec, event_id, - revision_to_int(expected) + 1, - expected, condition, factos.NoPosition, [], ) - } - } -} - -fn map_event_insert_error( - error: pog.QueryError, -) -> Error(domain_error, subscription_error) { - case error { - pog.ConstraintViolated(constraint: "factos_events_stream_revision_key", ..) -> - AppendConditionFailed(factos.NoAppendCondition) - _ -> StoreError(error) } } fn insert_events( connection: pog.Connection, - stream: String, events: List(event), - codec: EventCodec(event), + codec: factos.EventCodec(event, String), event_id: fn() -> String, - revision: Int, - expected: factos.Revision, condition: factos.AppendCondition, position: factos.SequencePosition, recorded_events: List(factos.Recorded(event)), -) -> Result(Dispatch(event), Error(domain_error, subscription_error)) { +) -> Result( + factos.Dispatch(event), + factos.Error(domain_error, subscription_error, pog.QueryError), +) { case events { - [] -> { - let append = Append(current_revision: revision - 1, position:) - Ok(Dispatch(append:, events: list.reverse(recorded_events))) - } - [event, ..rest] -> { - let EventCodec(encode:, ..) = codec - let Proposed( - data:, + [] -> Ok(factos.Dispatch(position:, events: list.reverse(recorded_events))) + [event, ..remaining] -> { + let factos.Event( + payload:, descriptor: factos.EventDescriptor(type_:, version:, tags:, metadata:), - ) = encode(event) + ) = codec.encode(event) let id = event_id() - use returned_position <- result.try(insert_event_if_revision_matches( + use returned_position <- result.try(insert_event_if_condition_matches( connection, - stream, - revision:, - expected:, condition:, id:, type_:, version:, tags:, metadata:, - data:, + payload:, )) let position = factos.SequencePosition(returned_position) let recorded = factos.Recorded( - id: id, - stream:, - revision:, + id:, position:, event:, descriptor: factos.EventDescriptor(type_:, version:, tags:, metadata:), @@ -829,13 +469,13 @@ fn insert_events( use _ <- result.try(insert_event_tags(connection, position, tags)) insert_events( connection, - stream, - rest, + remaining, codec, event_id, - revision + 1, - factos.CurrentRevision(revision), - factos.NoAppendCondition, + factos.FailIfEventsMatch( + decision_context: factos.NoContext, + after: factos.NoPosition, + ), position, [recorded, ..recorded_events], ) @@ -843,188 +483,140 @@ fn insert_events( } } -fn insert_event_if_revision_matches( +fn insert_event_if_condition_matches( connection: pog.Connection, - stream_name: String, - revision revision: Int, - expected expected: factos.Revision, condition condition: factos.AppendCondition, id id: String, type_ type_: factos.EventType, version version: Int, tags tags: List(factos.Tag), metadata metadata: factos.Metadata, - data data: json.Json, -) -> Result(Int, Error(domain_error, subscription_error)) { + payload payload: String, +) -> Result(Int, factos.Error(domain_error, subscription_error, pog.QueryError)) { let QuerySql(condition_sql, condition_parameters) = - append_condition_to_having_sql(condition) + append_condition_to_sql(condition) use returned <- result.try( pog.query(" - insert into factos_events (id, stream, revision, type, version, tags, metadata, data) - select $1, $2, $3, $4, $5, $6, $7, $8 - from factos_events - where stream = $2 - having coalesce(max(revision), -1) = $9 + insert into factos_events (id, type, version, tags, metadata, data) + select $1, $2, $3, $4, $5, $6 + where true " <> condition_sql <> " returning position ") |> pog.parameter(pog.text(id)) - |> pog.parameter(pog.text(stream_name)) - |> pog.parameter(pog.int(revision)) |> pog.parameter(pog.text(factos.event_type_name(type_))) |> pog.parameter(pog.int(version)) |> pog.parameter(pog.text(tags_to_json(tags))) |> pog.parameter(pog.text(metadata_to_json(metadata))) - |> pog.parameter(pog.text(json.to_string(data))) - |> pog.parameter(pog.int(revision_to_int(expected))) + |> pog.parameter(pog.text(payload)) |> with_parameters(condition_parameters) |> pog.returning(int_field_decoder()) |> pog.execute(on: connection) - |> result.map_error(map_event_insert_error), + |> result.map_error(factos.StoreError), ) case returned.rows { [position, ..] -> Ok(position) - [] -> Error(AppendConditionFailed(condition)) + [] -> Error(factos.AppendConditionFailed(condition)) } } -fn append_condition_to_having_sql( - condition: factos.AppendCondition, -) -> QuerySql { +fn append_condition_to_sql(condition: factos.AppendCondition) -> QuerySql { case condition { - factos.NoAppendCondition -> QuerySql(sql: "", parameters: []) - factos.FailIfEventsMatch(query, after) -> { + factos.FailIfEventsMatch(decision_context:, after:) -> { let QuerySql(where_sql, parameters) = - matching_events_after_sql_from(query, after, parameter_index: 10) + matching_events_after_sql_from( + decision_context, + after, + parameter_index: 7, + ) QuerySql(sql: " and not exists ( select 1 from factos_events " <> where_sql <> " - )", parameters: parameters) + )", parameters:) } } } fn read_matching_events( connection: pog.Connection, - query: factos.Query, - codec: EventCodec(event), + decision_context: factos.DecisionContext, + codec: factos.EventCodec(event, String), ) -> Result( List(factos.Recorded(event)), - Error(domain_error, subscription_error), + factos.Error(domain_error, subscription_error, pog.QueryError), ) { - let QuerySql(where_sql, parameters) = query_to_sql(query, 1) + let QuerySql(where_sql, parameters) = query_to_sql(decision_context, 1) use rows <- result.try( - pog.query( - "select position, id, stream, revision, type, version, tags, metadata, data::text + pog.query("select position, id, type, version, tags, metadata, data::text from factos_events - " - <> where_sql - <> " - order by position", - ) + " <> where_sql <> " + order by position") |> with_parameters(parameters) |> pog.returning(stored_row_decoder()) |> pog.execute(on: connection) |> result.map(fn(returned) { returned.rows }) - |> result.map_error(StoreError), + |> result.map_error(factos.StoreError), ) decode_stored_rows(rows, codec) } -fn read_stream_events( - connection: pog.Connection, - stream_name: String, - codec: EventCodec(event), -) -> Result( - List(factos.Recorded(event)), - Error(domain_error, subscription_error), -) { - use rows <- result.try( - pog.query( - "select position, id, stream, revision, type, version, tags, metadata, data::text - from factos_events - where stream = $1 - order by revision", - ) - |> pog.parameter(pog.text(stream_name)) - |> pog.returning(stored_row_decoder()) - |> pog.execute(on: connection) - |> result.map(fn(returned) { returned.rows }) - |> result.map_error(StoreError), - ) - decode_stored_rows(rows, codec) -} - -fn stored_row_decoder() -> decode.Decoder(StoredRow) { +fn stored_row_decoder() -> decode.Decoder(factos.Recorded(String)) { use position <- decode.field(0, decode.int) use id <- decode.field(1, decode.string) - use stream <- decode.field(2, decode.string) - use revision <- decode.field(3, decode.int) - use type_ <- decode.field(4, decode.string |> decode.map(factos.event_type)) - use version <- decode.field(5, decode.int) - use tags <- decode.field(6, tags_column_decoder()) - use metadata <- decode.field(7, metadata_column_decoder()) - use data <- decode.field(8, decode.string) - decode.success(StoredRow( - position:, + use type_ <- decode.field(2, decode.string |> decode.map(factos.event_type)) + use version <- decode.field(3, decode.int) + use tags <- decode.field(4, tags_column_decoder()) + use metadata <- decode.field(5, metadata_column_decoder()) + use event <- decode.field(6, decode.string) + decode.success(factos.Recorded( + position: factos.SequencePosition(position), id:, - stream:, - revision:, descriptor: factos.EventDescriptor(type_:, version:, tags:, metadata:), - data:, + event:, )) } fn decode_stored_rows( - rows: List(StoredRow), - codec: EventCodec(event), + rows: List(factos.Recorded(String)), + codec: factos.EventCodec(event, String), ) -> Result( List(factos.Recorded(event)), - Error(domain_error, subscription_error), + factos.Error(domain_error, subscription_error, pog.QueryError), ) { decode_stored_rows_loop(rows, codec, []) } fn decode_stored_rows_loop( - rows: List(StoredRow), - codec: EventCodec(event), + rows: List(factos.Recorded(String)), + codec: factos.EventCodec(event, String), decoded: List(factos.Recorded(event)), ) -> Result( List(factos.Recorded(event)), - Error(domain_error, subscription_error), + factos.Error(domain_error, subscription_error, pog.QueryError), ) { case rows { [] -> Ok(list.reverse(decoded)) - [row, ..rest] -> { - use event <- result.try(decode_stored_row(row, codec)) - decode_stored_rows_loop(rest, codec, [event, ..decoded]) + [row, ..remaining] -> { + use recorded <- result.try(decode_stored_row(row, codec)) + decode_stored_rows_loop(remaining, codec, [recorded, ..decoded]) } } } fn decode_stored_row( - row: StoredRow, - codec: EventCodec(event), -) -> Result(factos.Recorded(event), Error(domain_error, subscription_error)) { - let StoredRow(position:, id:, stream:, revision:, descriptor:, data:) = row - let EventCodec(decode: select_decoder, ..) = codec - use event_decoder <- result.try( - select_decoder(descriptor) |> result.map_error(DecodeError), - ) + stored: factos.Recorded(String), + codec: factos.EventCodec(event, String), +) -> Result( + factos.Recorded(event), + factos.Error(domain_error, subscription_error, pog.QueryError), +) { + let factos.Recorded(position:, id:, descriptor:, ..) = stored use event <- result.try( - json.parse(data, using: event_decoder) - |> result.map_error(fn(_) { DecodeError(InvalidData) }), + codec.decode(stored) |> result.map_error(factos.DecodeError), ) - Ok(factos.Recorded( - position: factos.SequencePosition(position), - id:, - stream:, - revision:, - event:, - descriptor:, - )) + Ok(factos.Recorded(id:, position:, event:, descriptor:)) } fn metadata_column_decoder() -> decode.Decoder(factos.Metadata) { @@ -1043,58 +635,16 @@ fn metadata_column_decoder() -> decode.Decoder(factos.Metadata) { } } -fn current_revision( - connection: pog.Connection, - stream_name: String, -) -> Result(Int, pog.QueryError) { - use returned <- result.try( - pog.query( - "select coalesce(max(revision), -1) from factos_events where stream = $1", - ) - |> pog.parameter(pog.text(stream_name)) - |> pog.returning(int_field_decoder()) - |> pog.execute(on: connection), - ) - - case returned.rows { - [revision, ..] -> Ok(revision) - [] -> Ok(-1) - } -} - fn int_field_decoder() -> decode.Decoder(Int) { use value <- decode.field(0, decode.int) decode.success(value) } -fn stream_revision(events: List(factos.Recorded(event))) -> factos.Revision { - case list.reverse(events) { - [] -> factos.NoEvents - [event, ..] -> factos.CurrentRevision(event.revision) - } -} - -fn highest_recorded_position( - events: List(factos.Recorded(event)), -) -> factos.SequencePosition { - case list.reverse(events) { - [] -> factos.NoPosition - [event, ..] -> event.position - } -} - -fn revision_to_int(revision: factos.Revision) -> Int { - case revision { - factos.NoEvents -> -1 - factos.CurrentRevision(revision) -> revision - } -} - fn insert_event_tags( connection: pog.Connection, position: factos.SequencePosition, tags: List(factos.Tag), -) -> Result(Nil, Error(domain_error, subscription_error)) { +) -> Result(Nil, factos.Error(domain_error, subscription_error, pog.QueryError)) { case position, tags { factos.NoPosition, _ -> Ok(Nil) _, [] -> Ok(Nil) @@ -1113,34 +663,36 @@ fn insert_event_tags( tags, )) |> pog.execute(on: connection) - |> result.map_error(StoreError), + |> result.map_error(factos.StoreError), ) Ok(Nil) } } } -fn query_to_sql(query: factos.Query, parameter_index: Int) -> QuerySql { - case query { +fn query_to_sql( + decision_context: factos.DecisionContext, + parameter_index: Int, +) -> QuerySql { + case decision_context { factos.AllEvents -> QuerySql(sql: "", parameters: []) - factos.Query(items) -> { - case items { - [] -> QuerySql(sql: "where 1 = 0", parameters: []) - [_, ..] -> { - let #(sql, parameters, _) = - build_query_items_sql(items, parameter_index, [], []) - QuerySql( - sql: "where " <> string.join(list.reverse(sql), with: " or "), - parameters: list.reverse(parameters), - ) - } - } + + factos.Matching(items: []) | factos.NoContext -> + QuerySql(sql: "where 1 = 0", parameters: []) + + factos.Matching(items: [_, ..] as items) -> { + let #(sql, parameters, _) = + build_query_items_sql(items, parameter_index, [], []) + QuerySql( + sql: "where " <> string.join(list.reverse(sql), with: " or "), + parameters: list.reverse(parameters), + ) } } } fn matching_events_after_sql_from( - query: factos.Query, + query: factos.DecisionContext, after: factos.SequencePosition, parameter_index parameter_index: Int, ) -> QuerySql { @@ -1155,30 +707,29 @@ fn matching_events_after_sql_from( QuerySql(sql: "where position > " <> after_placeholder, parameters: [ IntParameter(after_position), ]) - factos.Query(items) -> { - case items { - [] -> QuerySql(sql: "where 1 = 0", parameters: []) - [_, ..] -> { - let #(sql, parameters, _) = - build_query_items_sql(items, parameter_index + 1, [], [ - IntParameter(after_position), - ]) - QuerySql( - sql: "where position > " - <> after_placeholder - <> " and (" - <> string.join(list.reverse(sql), with: " or ") - <> ")", - parameters: list.reverse(parameters), - ) - } - } + + factos.Matching(items: [_, ..] as items) -> { + let #(sql, parameters, _) = + build_query_items_sql(items, parameter_index + 1, [], [ + IntParameter(after_position), + ]) + QuerySql( + sql: "where position > " + <> after_placeholder + <> " and (" + <> string.join(list.reverse(sql), with: " or ") + <> ")", + parameters: list.reverse(parameters), + ) } + + factos.Matching(items: []) | factos.NoContext -> + QuerySql(sql: "where 1 = 0", parameters: []) } } fn build_query_items_sql( - items: List(factos.QueryItem), + items: List(factos.Item), parameter_index: Int, sql: List(String), parameters: List(QueryParameter), @@ -1198,8 +749,8 @@ fn build_query_items_sql( } } -fn query_item_to_sql(item: factos.QueryItem, parameter_index: Int) -> QuerySql { - let factos.QueryItem(types, tags) = item +fn query_item_to_sql(item: factos.Item, parameter_index: Int) -> QuerySql { + let factos.Item(types:, tags:) = item let QuerySql(type_sql, type_parameters) = types_to_sql(types, parameter_index) let QuerySql(tag_sql, tag_parameters) = tags_to_sql(tags, parameter_index + list.length(type_parameters)) @@ -1323,26 +874,8 @@ fn metadata_to_json(metadata: factos.Metadata) -> String { |> json.to_string } -pub fn error_to_string( - error: Error(domain_error, subscription_error), - domain_error_to_string: fn(domain_error) -> String, - subscription_error_to_string: fn(subscription_error) -> String, -) -> String { - case error { - DomainError(error) -> domain_error_to_string(error) - SubscriptionError(error:) -> - "subscription error: " <> subscription_error_to_string(error) - StoreError(error) -> "store error: " <> query_error_to_string(error) - AppendConditionFailed(factos.NoAppendCondition) -> - "append to event failed: No append condition" - AppendConditionFailed(factos.FailIfEventsMatch(query: _, after: _)) -> - "append to event failed: Events matched" - DecodeError(UnknownEvent) -> "unknown event decoded" - DecodeError(InvalidData) -> "invalid data stored in database" - } -} - -fn query_error_to_string(error: pog.QueryError) -> String { +/// Render a PostgreSQL query error. +pub fn query_error_to_string(error: pog.QueryError) -> String { case error { pog.ConstraintViolated(message:, constraint:, detail:) -> "constraint violated: " <> constraint <> ": " <> message <> " " <> detail diff --git a/backends/factos_pog/test/factos_pog_test.gleam b/backends/factos_pog/test/factos_pog_test.gleam index 3be795d..cca45d4 100644 --- a/backends/factos_pog/test/factos_pog_test.gleam +++ b/backends/factos_pog/test/factos_pog_test.gleam @@ -3,14 +3,12 @@ import factos import factos/factos_pog import gleam/dynamic/decode import gleam/erlang/application -import gleam/erlang/atom import gleam/erlang/process import gleam/int import gleam/json import gleam/list import gleam/option import gleam/otp/actor -import gleam/otp/static_supervisor import gleam/result import gleam/string import gleeunit @@ -83,8 +81,8 @@ type FireSubscriptionMessage { type SubscriptionDispatchMessage { SubscriptionDispatchFinished( result: Result( - factos_pog.Dispatch(Event), - factos_pog.Error(DomainError, String), + factos.Dispatch(Event), + factos.Error(DomainError, String, pog.QueryError), ), ) } @@ -137,7 +135,7 @@ pub fn dbmate_upgrade_and_v2_rollback_preserve_contract_test() { let assert Ok([factos.Recorded(event:, descriptor:, ..)]) = factos_pog.read_after( connection, - query: factos.AllEvents, + decision_context: factos.AllEvents, after: factos.NoPosition, limit: 10, codec: codec(), @@ -149,9 +147,9 @@ pub fn dbmate_upgrade_and_v2_rollback_preserve_contract_test() { assert_event_store_objects(connection) let assert Ok(_v2_dispatch) = - factos_pog.new_dispatch( + factos.new_dispatch( connection:, - stream: "migrated-v2", + decision_context: factos.NoContext, decider: decider(), codec: codec(), ) @@ -175,6 +173,18 @@ pub fn dbmate_upgrade_and_v2_rollback_preserve_contract_test() { |> pog.execute(on: connection) assert legacy_event.rows == [#("migration-correlation", "renata")] + let assert Ok(rollback_identity) = + pog.query( + " + select stream, revision::text + from factos_events + where id = 'b3b12f1d-6d85-4f1f-9c2a-94766a34f006' + ", + ) + |> pog.returning(string_pair_decoder()) + |> pog.execute(on: connection) + assert rollback_identity.rows == [#("factos-v2-rollback-1", "0")] + let assert Ok(removed_objects) = pog.query( " @@ -200,21 +210,18 @@ pub fn fire_and_forget_subscription_runs_after_commit_without_blocking_test() { use connection <- with_test_connection() reset_schema(connection) reset_subscription_test_state(connection) - let name = process.new_name("test") - let supervisor_pid = start_test_subscription_supervisor(name) let strong_barriers = process.new_subject() let fire_deliveries = process.new_subject() let dispatch_results = process.new_subject() let subscriptions = [ blocking_strong_subscription(strong_barriers, name: "strong"), - blocking_fire_subscription(name, fire_deliveries), + blocking_fire_subscription(fire_deliveries), ] let _first_dispatch_pid = start_subscription_dispatch_worker( connection, - stream: "fire-after-commit-first", username: "renata", subscriptions:, results: dispatch_results, @@ -254,7 +261,6 @@ pub fn fire_and_forget_subscription_runs_after_commit_without_blocking_test() { let _second_dispatch_pid = start_subscription_dispatch_worker( connection, - stream: "fire-after-commit-second", username: "maria", subscriptions:, results: dispatch_results, @@ -270,12 +276,12 @@ pub fn fire_and_forget_subscription_runs_after_commit_without_blocking_test() { let FireSubscriptionStarted( pid: second_fire_pid, event: second_fire_event, - release: _second_fire_release, + release: second_fire_release, ) = receive_fire_subscription(fire_deliveries) assert second_fire_event == second_recorded assert process.is_alive(second_fire_pid) let second_fire_monitor = process.monitor(second_fire_pid) - stop_test_supervisor(supervisor_pid) + process.send(second_fire_release, Nil) wait_for_monitor(second_fire_monitor) assert all_recorded_events(connection) @@ -294,7 +300,6 @@ pub fn strong_subscription_commits_with_dispatch_test() { let _dispatch_pid = start_subscription_dispatch_worker( connection, - stream: "strong-commit", username: "renata", subscriptions: [subscription], results: dispatch_results, @@ -319,19 +324,17 @@ pub fn strong_subscription_failure_rolls_back_dispatch_test() { reset_schema(connection) reset_subscription_test_state(connection) - let name = process.new_name("test") - let supervisor_pid = start_test_subscription_supervisor(name) let fire_deliveries = process.new_subject() let insert_projection = - factos_pog.new_subscription( - query: factos.AllEvents, - consistency: factos_pog.StrongConsistency, + factos.new_subscription( + decision_context: factos.AllEvents, + consistency: factos.StrongConsistency, handle: insert_test_projection, ) let fail_after_observing_projection = - factos_pog.new_subscription( - query: factos.AllEvents, - consistency: factos_pog.StrongConsistency, + factos.new_subscription( + decision_context: factos.AllEvents, + consistency: factos.StrongConsistency, handle: fn(transaction_connection, recorded) { let factos.Recorded(id:, ..) = recorded case @@ -344,22 +347,22 @@ pub fn strong_subscription_failure_rolls_back_dispatch_test() { }, ) let fire_subscription = - factos_pog.new_subscription( - query: factos.AllEvents, - consistency: factos_pog.FireAndForget(name:), + factos.new_subscription( + decision_context: factos.AllEvents, + consistency: factos.FireAndForget, handle: fn(_connection, recorded) { report_fire_event(fire_deliveries, recorded) }, ) let result = - factos_pog.new_dispatch( + factos.new_dispatch( connection:, - stream: "strong-failure", + decision_context: username_decision_context("renata"), decider: decider(), codec: codec(), ) - |> factos_pog.with_subscriptions(subscriptions: [ + |> factos.with_subscriptions(subscriptions: [ insert_projection, fail_after_observing_projection, fire_subscription, @@ -370,42 +373,40 @@ pub fn strong_subscription_failure_rolls_back_dispatch_test() { ) let assert Error(dispatch_error) = result - let assert factos_pog.SubscriptionError(error: "expected strong failure") = + let assert factos.SubscriptionError(error: "expected strong failure") = dispatch_error - assert factos_pog.error_to_string( + assert factos.error_to_string( dispatch_error, fn(_) { "domain" }, fn(error) { error }, + factos_pog.query_error_to_string, ) == "subscription error: expected strong failure" assert all_recorded_events(connection) == [] assert projected_users(connection) == [] - let assert Ok(loaded) = - factos_pog.load_stream( + let assert Ok(context) = + factos_pog.read( connection, - stream: "strong-failure", + decision_context: username_decision_context("renata"), decider: decider(), codec: codec(), ) - assert loaded.state == Available - assert loaded.revision == factos.NoEvents - assert loaded.events == [] + assert context.state == Available + assert context.events == [] let assert Error(Nil) = process.receive(fire_deliveries, within: 200) - stop_test_supervisor(supervisor_pid) + Nil } pub fn subscriptions_filter_and_skip_empty_dispatch_test() { use connection <- with_test_connection() reset_schema(connection) - let name = process.new_name("test") - let supervisor_pid = start_test_subscription_supervisor(name) let invocations = process.new_subject() - let nonmatching_query = username_query("maria") + let nonmatching_decision_context = username_decision_context("maria") let nonmatching_subscriptions = [ - factos_pog.new_subscription( - query: nonmatching_query, - consistency: factos_pog.StrongConsistency, + factos.new_subscription( + decision_context: nonmatching_decision_context, + consistency: factos.StrongConsistency, handle: fn(connection, recorded) { report_subscription_invocation( invocations, @@ -415,9 +416,9 @@ pub fn subscriptions_filter_and_skip_empty_dispatch_test() { ) }, ), - factos_pog.new_subscription( - query: nonmatching_query, - consistency: factos_pog.FireAndForget(name:), + factos.new_subscription( + decision_context: nonmatching_decision_context, + consistency: factos.FireAndForget, handle: fn(connection, recorded) { report_subscription_invocation( invocations, @@ -430,13 +431,13 @@ pub fn subscriptions_filter_and_skip_empty_dispatch_test() { ] let assert Ok(matching_dispatch) = - factos_pog.new_dispatch( + factos.new_dispatch( connection:, - stream: "subscription-filter", + decision_context: username_decision_context("renata"), decider: decider(), codec: codec(), ) - |> factos_pog.with_subscriptions(subscriptions: nonmatching_subscriptions) + |> factos.with_subscriptions(subscriptions: nonmatching_subscriptions) |> factos_pog.dispatch( RegisterUser(username: "renata"), event_id: uuid.v4_string, @@ -445,9 +446,9 @@ pub fn subscriptions_filter_and_skip_empty_dispatch_test() { let assert Error(Nil) = process.receive(invocations, within: 200) let all_event_subscriptions = [ - factos_pog.new_subscription( - query: factos.AllEvents, - consistency: factos_pog.StrongConsistency, + factos.new_subscription( + decision_context: factos.AllEvents, + consistency: factos.StrongConsistency, handle: fn(connection, recorded) { report_subscription_invocation( invocations, @@ -457,9 +458,9 @@ pub fn subscriptions_filter_and_skip_empty_dispatch_test() { ) }, ), - factos_pog.new_subscription( - query: factos.AllEvents, - consistency: factos_pog.FireAndForget(name:), + factos.new_subscription( + decision_context: factos.AllEvents, + consistency: factos.FireAndForget, handle: fn(connection, recorded) { report_subscription_invocation( invocations, @@ -471,30 +472,30 @@ pub fn subscriptions_filter_and_skip_empty_dispatch_test() { ), ] let assert Ok(empty_dispatch) = - factos_pog.new_dispatch( + factos.new_dispatch( connection:, - stream: "subscription-empty", + decision_context: factos.NoContext, decider: empty_decider(), codec: codec(), ) - |> factos_pog.with_subscriptions(subscriptions: all_event_subscriptions) + |> factos.with_subscriptions(subscriptions: all_event_subscriptions) |> factos_pog.dispatch( RegisterUser(username: "ignored"), event_id: uuid.v4_string, ) assert empty_dispatch.events == [] - assert empty_dispatch.append.position == factos.NoPosition + assert empty_dispatch.position == factos.NoPosition let assert Error(Nil) = process.receive(invocations, within: 200) - stop_test_supervisor(supervisor_pid) + Nil } fn blocking_strong_subscription( barriers: process.Subject(ProjectionBarrierMessage), name name: String, -) -> factos_pog.Subscription(Event, String) { - factos_pog.new_subscription( - query: factos.AllEvents, - consistency: factos_pog.StrongConsistency, +) -> factos.Subscription(Event, String, pog.Connection) { + factos.new_subscription( + decision_context: factos.AllEvents, + consistency: factos.StrongConsistency, handle: fn(connection, recorded) { use _ <- result.try(insert_test_projection(connection, recorded)) block_projection(name, barriers, recorded) @@ -503,12 +504,11 @@ fn blocking_strong_subscription( } fn blocking_fire_subscription( - name: process.Name(_), deliveries: process.Subject(FireSubscriptionMessage), -) -> factos_pog.Subscription(Event, String) { - factos_pog.new_subscription( - query: factos.AllEvents, - consistency: factos_pog.FireAndForget(name:), +) -> factos.Subscription(Event, String, pog.Connection) { + factos.new_subscription( + decision_context: factos.AllEvents, + consistency: factos.FireAndForget, handle: fn(_connection, recorded) { let release = process.new_subject() process.send( @@ -541,30 +541,23 @@ fn report_subscription_invocation( Ok(Nil) } -fn start_test_subscription_supervisor(name: process.Name(_)) -> process.Pid { - let assert Ok(actor.Started(pid:, ..)) = - static_supervisor.new(strategy: static_supervisor.OneForOne) - |> static_supervisor.add(factos_pog.supervised(name)) - |> static_supervisor.start - pid -} - fn start_subscription_dispatch_worker( connection: pog.Connection, - stream stream_name: String, username username: String, - subscriptions subscriptions: List(factos_pog.Subscription(Event, String)), + subscriptions subscriptions: List( + factos.Subscription(Event, String, pog.Connection), + ), results results: process.Subject(SubscriptionDispatchMessage), ) -> process.Pid { process.spawn(fn() { let result = - factos_pog.new_dispatch( + factos.new_dispatch( connection:, - stream: stream_name, + decision_context: username_decision_context(username), decider: decider(), codec: codec(), ) - |> factos_pog.with_subscriptions(subscriptions:) + |> factos.with_subscriptions(subscriptions:) |> factos_pog.dispatch(RegisterUser(username:), event_id: uuid.v4_string) process.send(results, SubscriptionDispatchFinished(result:)) }) @@ -572,7 +565,10 @@ fn start_subscription_dispatch_worker( fn receive_subscription_dispatch( results: process.Subject(SubscriptionDispatchMessage), -) -> Result(factos_pog.Dispatch(Event), factos_pog.Error(DomainError, String)) { +) -> Result( + factos.Dispatch(Event), + factos.Error(DomainError, String, pog.QueryError), +) { let assert Ok(SubscriptionDispatchFinished(result:)) = process.receive(results, within: 10_000) result @@ -601,7 +597,7 @@ fn all_recorded_events( let assert Ok(events) = factos_pog.read_after( connection, - query: factos.AllEvents, + decision_context: factos.AllEvents, after: factos.NoPosition, limit: 100, codec: codec(), @@ -623,9 +619,9 @@ pub fn read_after_orders_filters_and_bounds_pages_test() { ["renata", "maria", "lucy"] |> list.each(fn(username) { let assert Ok(_) = - factos_pog.new_dispatch( + factos.new_dispatch( connection:, - stream: "user-" <> username, + decision_context: username_decision_context(username), decider: decider(), codec: codec(), ) @@ -636,7 +632,7 @@ pub fn read_after_orders_filters_and_bounds_pages_test() { let assert Ok([renata, maria]) = factos_pog.read_after( connection, - query: factos.AllEvents, + decision_context: factos.AllEvents, after: factos.NoPosition, limit: 2, codec: codec(), @@ -647,7 +643,7 @@ pub fn read_after_orders_filters_and_bounds_pages_test() { let assert Ok([lucy]) = factos_pog.read_after( connection, - query: factos.AllEvents, + decision_context: factos.AllEvents, after: maria.position, limit: 2, codec: codec(), @@ -657,7 +653,7 @@ pub fn read_after_orders_filters_and_bounds_pages_test() { let assert Ok([filtered]) = factos_pog.read_after( connection, - query: username_query("maria"), + decision_context: username_decision_context("maria"), after: factos.NoPosition, limit: 10, codec: codec(), @@ -667,7 +663,7 @@ pub fn read_after_orders_filters_and_bounds_pages_test() { let assert Ok([]) = factos_pog.read_after( connection, - query: factos.AllEvents, + decision_context: factos.AllEvents, after: factos.NoPosition, limit: 0, codec: codec(), @@ -680,47 +676,37 @@ pub fn dispatch_builder_with_one_retry_attempt_persists_events_test() { reset_schema(connection) let assert Ok(dispatch) = - factos_pog.new_dispatch( + factos.new_dispatch( connection: connection, - stream: "user-renata", + decision_context: username_decision_context("renata"), decider: decider(), codec: codec(), ) - |> factos_pog.with_retry_attempts(attempts: 1) + |> factos.with_retry_attempts(attempts: 1) |> factos_pog.dispatch(RegisterUser("renata"), event_id: uuid.v4_string) - let assert factos_pog.Append( - current_revision: 0, - position: factos.SequencePosition(_), - ) = dispatch.append + let assert factos.SequencePosition(_) = dispatch.position let assert [recorded] = dispatch.events assert_user_recorded( recorded, - stream: "user-renata", - revision: 0, - position: dispatch.append.position, + position: dispatch.position, username: "renata", ) - let reactor = factos.reactor(react: fn(recorded) { [recorded.event] }) - assert factos.react_all(reactor: reactor, events: dispatch.events) - == [ - UserRegistered("renata"), - ] - let assert Ok(loaded) = - factos_pog.load_stream( + let assert Ok(context) = + factos_pog.read( connection, - stream: "user-renata", + decision_context: username_decision_context("renata"), decider: decider(), codec: codec(), ) - assert loaded.state == Taken - assert loaded.revision == factos.CurrentRevision(0) + assert context.state == Taken + assert context.events == [recorded] Nil } -pub fn concurrent_dispatch_same_stream_allows_one_empty_state_append_test() { +pub fn concurrent_dispatch_same_context_allows_one_empty_state_append_test() { use connection <- with_test_connection() reset_schema(connection) @@ -729,14 +715,12 @@ pub fn concurrent_dispatch_same_stream_allows_one_empty_state_append_test() { connection, messages: messages, worker: "first", - stream: "concurrent-user-renata", command: RegisterUser(username: "renata"), ) start_blocked_dispatch_worker( connection, messages: messages, worker: "second", - stream: "concurrent-user-renata", command: RegisterUser(username: "renata"), ) @@ -748,53 +732,48 @@ pub fn concurrent_dispatch_same_stream_allows_one_empty_state_append_test() { let second_result = receive_dispatch_finished(messages) let results = [first_result, second_result] assert list.count(results, where: is_successful_dispatch) == 1 - assert list.count(results, where: is_stale_stream_dispatch) == 1 + assert list.count(results, where: is_stale_context_dispatch) == 1 - let assert Ok(loaded) = - factos_pog.load_stream( + let assert Ok(context) = + factos_pog.read( connection, - stream: "concurrent-user-renata", + decision_context: username_decision_context("renata"), decider: decider(), codec: codec(), ) - assert loaded.state == Taken - assert list.length(loaded.events) == 1 - let assert [event] = loaded.events + assert context.state == Taken + let assert [event] = context.events assert event.event == UserRegistered("renata") Nil } -pub fn concurrent_dispatch_with_query_retries_duplicate_username_test() { +pub fn concurrent_dispatch_with_decision_context_retries_duplicate_username_test() { use connection <- with_test_connection() reset_schema(connection) reset_subscription_test_state(connection) install_one_commit_serialization_failure(connection) - let query = username_query("renata") + let decision_context = username_decision_context("renata") let messages = process.new_subject() let projection_barriers = process.new_subject() let fire_deliveries = process.new_subject() - let name = process.new_name("test") - let supervisor_pid = start_test_subscription_supervisor(name) let subscriptions = [ concurrent_strong_subscription(projection_barriers), - concurrent_fire_subscription(name, fire_deliveries), + concurrent_fire_subscription(fire_deliveries), ] - start_blocked_query_dispatch_worker( + start_blocked_decision_context_dispatch_worker( connection, messages:, worker: "first", - stream: "concurrent-query-renata-1", - query:, + decision_context:, command: RegisterUser(username: "renata"), subscriptions:, ) - start_blocked_query_dispatch_worker( + start_blocked_decision_context_dispatch_worker( connection, messages:, worker: "second", - stream: "concurrent-query-renata-2", - query:, + decision_context:, command: RegisterUser(username: "renata"), subscriptions:, ) @@ -845,10 +824,14 @@ pub fn concurrent_dispatch_with_query_retries_duplicate_username_test() { let assert Error(Nil) = process.receive(projection_barriers, within: 200) let assert Ok(context) = - factos_pog.read(connection, query:, decider: decider(), codec: codec()) + factos_pog.read( + connection, + decision_context:, + decider: decider(), + codec: codec(), + ) assert context.state == Taken assert context.events == [committed_attempt] - stop_test_supervisor(supervisor_pid) } fn install_one_commit_serialization_failure(connection: pog.Connection) -> Nil { @@ -888,25 +871,24 @@ fn install_one_commit_serialization_failure(connection: pog.Connection) -> Nil { fn concurrent_strong_subscription( barriers: process.Subject(ProjectionBarrierMessage), -) -> factos_pog.Subscription(Event, Nil) { - factos_pog.new_subscription( - query: factos.AllEvents, - consistency: factos_pog.StrongConsistency, +) -> factos.Subscription(Event, Nil, pog.Connection) { + factos.new_subscription( + decision_context: factos.AllEvents, + consistency: factos.StrongConsistency, handle: fn(connection, recorded) { let assert Ok(Nil) = insert_test_projection(connection, recorded) - let assert Ok(Nil) = block_projection(recorded.stream, barriers, recorded) + let assert Ok(Nil) = block_projection(recorded.id, barriers, recorded) Ok(Nil) }, ) } fn concurrent_fire_subscription( - name: process.Name(_), deliveries: process.Subject(factos.Recorded(Event)), -) -> factos_pog.Subscription(Event, Nil) { - factos_pog.new_subscription( - query: factos.AllEvents, - consistency: factos_pog.FireAndForget(name:), +) -> factos.Subscription(Event, Nil, pog.Connection) { + factos.new_subscription( + decision_context: factos.AllEvents, + consistency: factos.FireAndForget, handle: fn(_connection, recorded) { process.send(deliveries, recorded) Ok(Nil) @@ -919,40 +901,29 @@ pub fn dispatch_builder_with_query_filters_before_decoding_unknown_events_test() reset_schema(connection) insert_unknown_event(connection) - let query = username_query("renata") + let decision_context = username_decision_context("renata") let assert Ok(dispatch) = - factos_pog.new_dispatch( + factos.new_dispatch( connection: connection, - stream: "user-renata", + decision_context:, decider: decider(), codec: codec(), ) - |> factos_pog.with_query(query: query) |> factos_pog.dispatch(RegisterUser("renata"), event_id: uuid.v4_string) - let assert factos_pog.Append( - current_revision: 0, - position: factos.SequencePosition(_), - ) = dispatch.append + let assert factos.SequencePosition(_) = dispatch.position let assert [recorded] = dispatch.events assert_user_recorded( recorded, - stream: "user-renata", - revision: 0, - position: dispatch.append.position, + position: dispatch.position, username: "renata", ) - let reactor = factos.reactor(react: fn(recorded) { [recorded.event] }) - assert factos.react_all(reactor: reactor, events: dispatch.events) - == [ - UserRegistered("renata"), - ] let assert Ok(context) = factos_pog.read( connection, - query: query, + decision_context:, decider: decider(), codec: codec(), ) @@ -964,41 +935,31 @@ pub fn dispatch_builder_with_query_filters_before_decoding_unknown_events_test() Nil } -pub fn dispatch_builder_with_query_handles_many_streams_test() { +pub fn dispatch_builder_with_query_handles_many_events_test() { use connection <- with_test_connection() reset_schema(connection) let query = - factos.query([ - factos.query_item(types: [factos.event_type("Incremented")], tags: [ + factos.Matching([ + factos.item(types: [factos.event_type("Incremented")], tags: [ factos.tag("counter:load"), ]), ]) let assert Ok(dispatch) = dispatch_counter_context_many(connection, query, 25) - let assert factos_pog.Append( - current_revision: 0, - position: factos.SequencePosition(_), - ) = dispatch.append + let assert factos.SequencePosition(_) = dispatch.position let assert [recorded] = dispatch.events assert_counter_recorded( recorded, - stream: "counter-context-1", - revision: 0, - position: dispatch.append.position, + position: dispatch.position, value: 25, type_: factos.event_type("Incremented"), ) - let reactor = factos.reactor(react: fn(recorded) { [recorded.event] }) - assert factos.react_all(reactor: reactor, events: dispatch.events) - == [ - Incremented(25), - ] let assert Ok(context) = factos_pog.read( connection, - query: query, + decision_context: query, decider: counter_decider(), codec: counter_codec(), ) @@ -1012,51 +973,48 @@ pub fn context_semantics_conformance_test() { use connection <- with_test_connection() reset_schema(connection) - let no_matches = empty_query() + let decision_context = empty_query() let assert Ok(renata_dispatch) = - factos_pog.new_dispatch( + factos.new_dispatch( connection:, - stream: "conformance-renata", + decision_context:, decider: accepting_decider(), codec: codec(), ) - |> factos_pog.with_query(query: no_matches) |> factos_pog.dispatch( RegisterUser(username: "renata"), event_id: uuid.v4_string, ) let assert Ok(lucy_dispatch) = - factos_pog.new_dispatch( + factos.new_dispatch( connection:, - stream: "conformance-lucy", + decision_context:, decider: accepting_decider(), codec: codec(), ) - |> factos_pog.with_query(query: no_matches) |> factos_pog.dispatch( RegisterUser(username: "lucy"), event_id: uuid.v4_string, ) let assert Ok(marc_dispatch) = - factos_pog.new_dispatch( + factos.new_dispatch( connection:, - stream: "conformance-marc", + decision_context: factos.AllEvents, decider: accepting_decider(), codec: codec(), ) - |> factos_pog.with_query(query: factos.AllEvents) |> factos_pog.dispatch( RegisterUser(username: "marc"), event_id: uuid.v4_string, ) - let renata_position = renata_dispatch.append.position - let lucy_position = lucy_dispatch.append.position - let marc_position = marc_dispatch.append.position + let renata_position = renata_dispatch.position + let lucy_position = lucy_dispatch.position + let marc_position = marc_dispatch.position let assert Ok(empty_context) = factos_pog.read( connection, - query: no_matches, + decision_context:, decider: accepting_decider(), codec: codec(), ) @@ -1064,73 +1022,55 @@ pub fn context_semantics_conformance_test() { assert empty_context.events == [] assert empty_context.position == factos.NoPosition assert empty_context.append_condition - == factos.FailIfEventsMatch(query: no_matches, after: factos.NoPosition) + == factos.FailIfEventsMatch(decision_context:, after: factos.NoPosition) let compound_query = username_conformance_query() let assert Ok(compound_context) = factos_pog.read( connection, - query: compound_query, + decision_context: compound_query, decider: accepting_decider(), codec: codec(), ) let assert [lucy] = compound_context.events - assert_user_recorded( - lucy, - stream: "conformance-lucy", - revision: 0, - position: lucy_position, - username: "lucy", - ) + assert_user_recorded(lucy, position: lucy_position, username: "lucy") assert compound_context.state == Taken assert compound_context.position == lucy_position assert compound_context.append_condition - == factos.FailIfEventsMatch(query: compound_query, after: lucy_position) + == factos.FailIfEventsMatch( + decision_context: compound_query, + after: lucy_position, + ) let assert Ok(all_context) = factos_pog.read( connection, - query: factos.AllEvents, + decision_context: factos.AllEvents, decider: accepting_decider(), codec: codec(), ) let assert [renata, lucy, marc] = all_context.events - assert_user_recorded( - renata, - stream: "conformance-renata", - revision: 0, - position: renata_position, - username: "renata", - ) - assert_user_recorded( - lucy, - stream: "conformance-lucy", - revision: 0, - position: lucy_position, - username: "lucy", - ) - assert_user_recorded( - marc, - stream: "conformance-marc", - revision: 0, - position: marc_position, - username: "marc", - ) + assert_user_recorded(renata, position: renata_position, username: "renata") + assert_user_recorded(lucy, position: lucy_position, username: "lucy") + assert_user_recorded(marc, position: marc_position, username: "marc") assert all_context.state == Taken assert all_context.position == marc_position assert all_context.append_condition - == factos.FailIfEventsMatch(query: factos.AllEvents, after: marc_position) + == factos.FailIfEventsMatch( + decision_context: factos.AllEvents, + after: marc_position, + ) Nil } -pub fn empty_context_dispatch_is_a_no_op_test() { +pub fn empty_matching_context_dispatch_is_a_no_op_test() { use connection <- with_test_connection() reset_schema(connection) let assert Ok(seeded_dispatch) = - factos_pog.new_dispatch( + factos.new_dispatch( connection:, - stream: "conformance-no-op", + decision_context: username_decision_context("no-op"), decider: decider(), codec: codec(), ) @@ -1141,36 +1081,23 @@ pub fn empty_context_dispatch_is_a_no_op_test() { let assert [seeded] = seeded_dispatch.events let assert Ok(no_op_dispatch) = - factos_pog.new_dispatch( + factos.new_dispatch( connection:, - stream: "conformance-no-op", + decision_context: factos.AllEvents, decider: empty_decider(), codec: codec(), ) - |> factos_pog.with_query(query: factos.AllEvents) |> factos_pog.dispatch( RegisterUser(username: "ignored"), event_id: uuid.v4_string, ) assert no_op_dispatch.events == [] - assert no_op_dispatch.append.current_revision == 0 - assert no_op_dispatch.append.position == factos.NoPosition - - let assert Ok(loaded) = - factos_pog.load_stream( - connection, - stream: "conformance-no-op", - decider: decider(), - codec: codec(), - ) - assert loaded.events == [seeded] - assert loaded.state == Taken - assert loaded.revision == factos.CurrentRevision(0) + assert no_op_dispatch.position == factos.NoPosition let assert Ok(context) = factos_pog.read( connection, - query: factos.AllEvents, + decision_context: factos.AllEvents, decider: decider(), codec: codec(), ) @@ -1180,13 +1107,48 @@ pub fn empty_context_dispatch_is_a_no_op_test() { Nil } +pub fn no_context_dispatch_skips_existing_events_test() { + use connection <- with_test_connection() + reset_schema(connection) + + let assert Ok(first_dispatch) = + factos.new_dispatch( + connection:, + decision_context: username_decision_context("renata"), + decider: decider(), + codec: codec(), + ) + |> factos_pog.dispatch( + RegisterUser(username: "renata"), + event_id: uuid.v4_string, + ) + let assert Ok(second_dispatch) = + factos.new_dispatch( + connection:, + decision_context: factos.NoContext, + decider: decider(), + codec: codec(), + ) + |> factos_pog.dispatch( + RegisterUser(username: "renata"), + event_id: uuid.v4_string, + ) + + let assert [first] = first_dispatch.events + let assert [second] = second_dispatch.events + assert first.event == UserRegistered(username: "renata") + assert second.event == UserRegistered(username: "renata") + assert first.position != second.position + assert all_recorded_events(connection) == [first, second] +} + type DispatchMessage { DispatchReady(worker: String, release: process.Subject(Nil)) DispatchFinished( worker: String, result: Result( - factos_pog.Dispatch(Event), - factos_pog.Error(DomainError, Nil), + factos.Dispatch(Event), + factos.Error(DomainError, Nil, pog.QueryError), ), ) } @@ -1195,41 +1157,43 @@ fn start_blocked_dispatch_worker( connection: pog.Connection, messages messages: process.Subject(DispatchMessage), worker worker: String, - stream stream_name: String, command command: Command, ) -> process.Pid { + let decision_context = case command { + RegisterUser(username:) -> username_decision_context(username) + } process.spawn(fn() { let result = - factos_pog.new_dispatch( - connection: connection, - stream: stream_name, - decider: blocking_decider(messages, worker: worker), + factos.new_dispatch( + connection:, + decision_context:, + decider: blocking_decider(messages, worker:), codec: codec(), ) |> factos_pog.dispatch(command, event_id: uuid.v4_string) - process.send(messages, DispatchFinished(worker: worker, result: result)) + process.send(messages, DispatchFinished(worker:, result:)) }) } -fn start_blocked_query_dispatch_worker( +fn start_blocked_decision_context_dispatch_worker( connection: pog.Connection, messages messages: process.Subject(DispatchMessage), worker worker: String, - stream stream_name: String, - query query: factos.Query, + decision_context decision_context: factos.DecisionContext, command command: Command, - subscriptions subscriptions: List(factos_pog.Subscription(Event, Nil)), + subscriptions subscriptions: List( + factos.Subscription(Event, Nil, pog.Connection), + ), ) -> process.Pid { process.spawn(fn() { let result = - factos_pog.new_dispatch( + factos.new_dispatch( connection:, - stream: stream_name, + decision_context:, decider: blocking_decider(messages, worker:), codec: codec(), ) - |> factos_pog.with_query(query:) - |> factos_pog.with_subscriptions(subscriptions:) + |> factos.with_subscriptions(subscriptions:) |> factos_pog.dispatch(command, event_id: uuid.v4_string) process.send(messages, DispatchFinished(worker:, result:)) }) @@ -1269,7 +1233,10 @@ fn receive_dispatch_ready( fn receive_dispatch_finished( messages: process.Subject(DispatchMessage), -) -> Result(factos_pog.Dispatch(Event), factos_pog.Error(DomainError, Nil)) { +) -> Result( + factos.Dispatch(Event), + factos.Error(DomainError, Nil, pog.QueryError), +) { let assert Ok(message) = process.receive(messages, within: 10_000) case message { DispatchFinished(worker: _, result:) -> result @@ -1281,7 +1248,10 @@ fn receive_dispatch_finished( } fn is_successful_dispatch( - result: Result(factos_pog.Dispatch(Event), factos_pog.Error(DomainError, Nil)), + result: Result( + factos.Dispatch(Event), + factos.Error(DomainError, Nil, pog.QueryError), + ), ) -> Bool { case result { Ok(_) -> True @@ -1289,21 +1259,27 @@ fn is_successful_dispatch( } } -fn is_stale_stream_dispatch( - result: Result(factos_pog.Dispatch(Event), factos_pog.Error(DomainError, Nil)), +fn is_stale_context_dispatch( + result: Result( + factos.Dispatch(Event), + factos.Error(DomainError, Nil, pog.QueryError), + ), ) -> Bool { case result { - Error(factos_pog.AppendConditionFailed(factos.NoAppendCondition)) -> True - Error(factos_pog.DomainError(AlreadyTaken)) -> True + Error(factos.AppendConditionFailed(_)) -> True + Error(factos.DomainError(AlreadyTaken)) -> True _ -> False } } fn is_already_taken_dispatch( - result: Result(factos_pog.Dispatch(Event), factos_pog.Error(DomainError, Nil)), + result: Result( + factos.Dispatch(Event), + factos.Error(DomainError, Nil, pog.QueryError), + ), ) -> Bool { case result { - Error(factos_pog.DomainError(AlreadyTaken)) -> True + Error(factos.DomainError(AlreadyTaken)) -> True _ -> False } } @@ -1523,18 +1499,6 @@ fn string_pair_decoder() -> decode.Decoder(#(String, String)) { decode.success(#(first, second)) } -fn stop_test_supervisor(supervisor_pid: process.Pid) -> Nil { - process.unlink(supervisor_pid) - let monitor = process.monitor(supervisor_pid) - process.send_abnormal_exit(supervisor_pid, atom.create("shutdown")) - let assert Ok(Nil) = - process.new_selector() - |> process.select_specific_monitor(monitor, fn(_) { Nil }) - |> process.selector_receive(5000) - process.demonitor_process(monitor) - Nil -} - fn execute_migration_file(connection: pog.Connection) -> Nil { let assert Ok(priv_directory) = application.priv_directory("factos_pog") let assert Ok(sql) = simplifile.read(priv_directory <> "/migrations.sql") @@ -1628,6 +1592,24 @@ fn assert_event_store_objects(connection: pog.Connection) -> Nil { select 'factos_subscriptions_absent' where to_regclass('factos_subscriptions') is null union all + select 'revision_column_absent' + where not exists ( + select 1 + from information_schema.columns + where table_schema = current_schema() + and table_name = 'factos_events' + and column_name = 'revision' + ) + union all + select 'stream_column_absent' + where not exists ( + select 1 + from information_schema.columns + where table_schema = current_schema() + and table_name = 'factos_events' + and column_name = 'stream' + ) + union all select 'factos_outbox' where to_regclass('factos_outbox') is not null order by 1 @@ -1640,6 +1622,8 @@ fn assert_event_store_objects(connection: pog.Connection) -> Nil { "factos_pog_event_append_lock", "factos_pog_lock_event_append", "factos_subscriptions_absent", + "revision_column_absent", + "stream_column_absent", ] } @@ -1650,36 +1634,30 @@ fn string_column_decoder() -> decode.Decoder(String) { fn assert_uuidv4_identity_contract(connection: pog.Connection) -> Nil { let valid_id = "b3b12f1d-6d85-4f1f-9c2a-94766a34f011" - assert insert_event_succeeds(connection, valid_id, "valid-stream") + assert insert_event_succeeds(connection, valid_id) [ - #("customer-registered", "semantic-stream"), - #("event-b3b12f1d-6d85-4f1f-9c2a-94766a34f012", "prefixed-stream"), - #("b3b12f1d-6d85-5f1f-9c2a-94766a34f013", "wrong-version-stream"), - #("not-a-uuid", "malformed-stream"), + "customer-registered", + "event-b3b12f1d-6d85-4f1f-9c2a-94766a34f012", + "b3b12f1d-6d85-5f1f-9c2a-94766a34f013", + "not-a-uuid", ] - |> list.each(fn(case_) { - let #(invalid_id, stream_name) = case_ - assert !insert_event_succeeds(connection, invalid_id, stream_name) + |> list.each(fn(invalid_id) { + assert !insert_event_succeeds(connection, invalid_id) }) - assert !insert_event_succeeds(connection, valid_id, "different-stream") + assert !insert_event_succeeds(connection, valid_id) } -fn insert_event_succeeds( - connection: pog.Connection, - id: String, - stream_name: String, -) -> Bool { +fn insert_event_succeeds(connection: pog.Connection, id: String) -> Bool { case pog.query( " - insert into factos_events (id, stream, revision, type, version, tags, metadata, data) - values ($1, $2, 0, 'ContractChecked', 1, '', '{}'::jsonb, 'null'::jsonb) + insert into factos_events (id, type, version, tags, metadata, data) + values ($1, 'ContractChecked', 1, '', '{}'::jsonb, 'null'::jsonb) ", ) |> pog.parameter(pog.text(id)) - |> pog.parameter(pog.text(stream_name)) |> pog.execute(on: connection) { Ok(_) -> True @@ -1693,17 +1671,15 @@ fn insert_unknown_event(connection: pog.Connection) -> Nil { pog.query( " with inserted as ( - insert into factos_events (id, stream, revision, type, version, tags, metadata, data) - values ($1, $2, $3, $4, $5, $6, $7::jsonb, $8::jsonb) + insert into factos_events (id, type, version, tags, metadata, data) + values ($1, $2, $3, $4, $5::jsonb, $6::jsonb) returning position ) insert into factos_event_tags(position, tag) - select position, $9 from inserted + select position, $7 from inserted ", ) |> pog.parameter(pog.text("b3b12f1d-6d85-4f1f-9c2a-94766a34f004")) - |> pog.parameter(pog.text("unknown-stream")) - |> pog.parameter(pog.int(0)) |> pog.parameter(pog.text("UnknownEventType")) |> pog.parameter(pog.int(1)) |> pog.parameter(pog.text("\n" <> tag <> "\n")) @@ -1714,9 +1690,9 @@ fn insert_unknown_event(connection: pog.Connection) -> Nil { Nil } -fn username_query(username: String) -> factos.Query { - factos.query([ - factos.query_item(types: [factos.event_type("UserRegistered")], tags: [ +fn username_decision_context(username: String) -> factos.DecisionContext { + factos.Matching([ + factos.item(types: [factos.event_type("UserRegistered")], tags: [ factos.tag("username:" <> username), ]), ]) @@ -1724,15 +1700,11 @@ fn username_query(username: String) -> factos.Query { fn assert_user_recorded( recorded: factos.Recorded(Event), - stream stream_name: String, - revision revision: Int, position position: factos.SequencePosition, username username: String, ) -> Nil { let assert Ok(event_id) = uuid.from_string(recorded.id) assert uuid.version(event_id) == uuid.V4 - assert recorded.stream == stream_name - assert recorded.revision == revision assert recorded.position == position assert recorded.descriptor.type_ == factos.event_type("UserRegistered") assert recorded.descriptor.version == 1 @@ -1780,17 +1752,17 @@ fn empty_decider() -> factos.Decider(Command, State, Event, DomainError) { ) } -fn empty_query() -> factos.Query { - factos.Query(items: []) +fn empty_query() -> factos.DecisionContext { + factos.Matching(items: []) } -fn username_conformance_query() -> factos.Query { - factos.query([ - factos.query_item(types: [factos.event_type("UserRegistered")], tags: [ +fn username_conformance_query() -> factos.DecisionContext { + factos.Matching([ + factos.item(types: [factos.event_type("UserRegistered")], tags: [ factos.tag("username:renata"), factos.tag("username:lucy"), ]), - factos.query_item( + factos.item( types: [ factos.event_type("UnknownEventType"), factos.event_type("UserRegistered"), @@ -1800,24 +1772,24 @@ fn username_conformance_query() -> factos.Query { ]) } -fn codec() -> factos_pog.EventCodec(Event) { - factos_pog.codec(encode:, decode:) +fn codec() -> factos.EventCodec(Event, String) { + factos.codec(encode:, decode:) } -fn encode(event: Event) -> factos_pog.Proposed { +fn encode(event: Event) -> factos.Event(String) { proposed_event(event) } -fn proposed_event(event: Event) -> factos_pog.Proposed { - factos_pog.new_proposed( +fn proposed_event(event: Event) -> factos.Event(String) { + factos.new_event( type_: factos.event_type("UserRegistered"), version: 1, - data: json.string(event.username), + data: json.string(event.username) |> json.to_string, ) - |> factos_pog.with_tags(tags: [ + |> factos.with_tags(tags: [ factos.tag("username:" <> event.username), ]) - |> factos_pog.with_metadata( + |> factos.with_metadata( metadata: factos.metadata([ #(factos.correlation_id, "event-" <> event.username), ]), @@ -1825,47 +1797,43 @@ fn proposed_event(event: Event) -> factos_pog.Proposed { } fn decode( - descriptor: factos.EventDescriptor, -) -> Result(decode.Decoder(Event), factos_pog.DecodeError) { - case factos.event_type_name(descriptor.type_), descriptor.version { - "UserRegistered", 1 -> Ok(decode.string |> decode.map(UserRegistered)) - _, _ -> Error(factos_pog.UnknownEvent) + stored: factos.Recorded(String), +) -> Result(Event, factos.DecodeError) { + case + factos.event_type_name(stored.descriptor.type_), + stored.descriptor.version + { + "UserRegistered", 1 -> + json.parse( + stored.event, + using: decode.string |> decode.map(UserRegistered), + ) + |> result.map_error(fn(_) { factos.InvalidData }) + _, _ -> Error(factos.UnknownEvent) } } fn dispatch_counter_context_many( connection: pog.Connection, - query: factos.Query, + decision_context: factos.DecisionContext, remaining: Int, -) -> Result(factos_pog.Dispatch(CounterEvent), factos_pog.Error(Nil, Nil)) { - case remaining { - 0 -> - factos_pog.new_dispatch( - connection: connection, - stream: "counter-context-0", - decider: counter_decider(), - codec: counter_codec(), - ) - |> factos_pog.with_query(query: query) - |> factos_pog.dispatch(Increment, event_id: uuid.v4_string) - _ -> { - let stream_name = "counter-context-" <> int.to_string(remaining) - let result = - factos_pog.new_dispatch( - connection: connection, - stream: stream_name, - decider: counter_decider(), - codec: counter_codec(), - ) - |> factos_pog.with_query(query: query) - |> factos_pog.dispatch(Increment, event_id: uuid.v4_string) - case remaining, result { - 1, _ -> result - _, Ok(_) -> - dispatch_counter_context_many(connection, query, remaining - 1) - _, Error(error) -> Error(error) - } - } +) -> Result( + factos.Dispatch(CounterEvent), + factos.Error(Nil, Nil, pog.QueryError), +) { + let result = + factos.new_dispatch( + connection:, + decision_context:, + decider: counter_decider(), + codec: counter_codec(), + ) + |> factos_pog.dispatch(Increment, event_id: uuid.v4_string) + case remaining, result { + 1, _ -> result + _, Ok(_) -> + dispatch_counter_context_many(connection, decision_context, remaining - 1) + _, Error(error) -> Error(error) } } @@ -1900,43 +1868,44 @@ fn counter_evolve(state: CounterState, event: CounterEvent) -> CounterState { } } -fn counter_codec() -> factos_pog.EventCodec(CounterEvent) { - factos_pog.codec(encode: encode_counter_event, decode: decode_counter_event) +fn counter_codec() -> factos.EventCodec(CounterEvent, String) { + factos.codec(encode: encode_counter_event, decode: decode_counter_event) } -fn encode_counter_event(event: CounterEvent) -> factos_pog.Proposed { +fn encode_counter_event(event: CounterEvent) -> factos.Event(String) { case event { Incremented(value) -> - factos_pog.new_proposed( + factos.new_event( type_: factos.event_type("Incremented"), version: 1, - data: json.int(value), + data: json.int(value) |> json.to_string, ) - |> factos_pog.with_tags(tags: [factos.tag("counter:load")]) + |> factos.with_tags(tags: [factos.tag("counter:load")]) } } fn decode_counter_event( - descriptor: factos.EventDescriptor, -) -> Result(decode.Decoder(CounterEvent), factos_pog.DecodeError) { - case factos.event_type_name(descriptor.type_), descriptor.version { - "Incremented", 1 -> Ok(decode.int |> decode.map(Incremented)) - _, _ -> Error(factos_pog.UnknownEvent) + stored: factos.Recorded(String), +) -> Result(CounterEvent, factos.DecodeError) { + case + factos.event_type_name(stored.descriptor.type_), + stored.descriptor.version + { + "Incremented", 1 -> + json.parse(stored.event, using: decode.int |> decode.map(Incremented)) + |> result.map_error(fn(_) { factos.InvalidData }) + _, _ -> Error(factos.UnknownEvent) } } fn assert_counter_recorded( recorded: factos.Recorded(CounterEvent), - stream stream_name: String, - revision revision: Int, position position: factos.SequencePosition, value value: Int, type_ type_: factos.EventType, ) -> Nil { let assert Ok(event_id) = uuid.from_string(recorded.id) assert uuid.version(event_id) == uuid.V4 - assert recorded.stream == stream_name - assert recorded.revision == revision assert recorded.position == position assert recorded.descriptor.type_ == type_ assert recorded.descriptor.version == 1 diff --git a/backends/factos_sqlight/README.md b/backends/factos_sqlight/README.md index 4b5fcdb..bdb278c 100644 --- a/backends/factos_sqlight/README.md +++ b/backends/factos_sqlight/README.md @@ -3,17 +3,13 @@ `factos_sqlight` is the SQLite backend for Factos, implemented with [`sqlight`](https://hex.pm/packages/sqlight). -It stores accepted facts in an append-only SQLite event log, reads the facts -relevant to a command, runs your pure `factos.Decider`, and appends new facts -only if the relevant context is still stable. +It stores accepted facts in one append-only, globally ordered event log. For +each command it reads the required `factos.DecisionContext`, folds those records +with a pure `factos.Decider`, and appends accepted events in the same SQLite +transaction. -Use this package when SQLite is your local, embedded, or single-node event store -and your consistency rules are expressed with Factos event types and tags. - -It persists event records and durable outbox effects only. It does not maintain -materialized views and it does not execute side effects. Applications build read -models and effect delivery on top of the committed records and leased outbox -messages. +Use this package for embedded or single-node applications whose consistency +boundaries can be expressed with stable event types and tags. ## Install @@ -21,228 +17,229 @@ messages. [dependencies] factos = ">= 1.0.0 and < 2.0.0" factos_sqlight = ">= 1.0.0 and < 2.0.0" -gleam_erlang = ">= 1.0.0 and < 2.0.0" sqlight = ">= 1.1.0 and < 2.0.0" ``` -For local unreleased development in this repository, use path dependencies: - -```toml -[dependencies] -factos = { path = "../factos" } -factos_sqlight = { path = ".." } -``` +For local development in this repository, use path dependencies. ## Set up the schema -The backend ships reusable dbmate-compatible migrations in `priv/dbmate/`. -Application databases should vendor those files into their own migration -repository, commit them, and run them with their normal migration tool before -dispatching commands. The application migration repository owns ordering and -execution history; `factos_sqlight` owns only the reusable schema artifacts. - -In an Erlang-target migration tool, locate the package `priv` directory and copy -the package migrations into your application migration directory: +The package ships a dbmate-compatible migration in `priv/dbmate/`. Vendor that +file into the application's immutable migration history and run it before +commands are dispatched. -```gleam -import gleam/erlang/application - -let assert Ok(priv_directory) = application.priv_directory("factos_sqlight") -let migrations_directory = priv_directory <> "/dbmate" -``` +`priv/migrations.sql` and `factos_sqlight.migrate` are fresh-bootstrap +conveniences. They are not a replacement for an application-owned migration +history. -`priv/migrations.sql` and `factos_sqlight.migrate` are retained as conveniences -for fresh bootstrap examples. Do not treat either as the append-only migration -history for an application database. +The schema contains one backend-owned table: -The migrations create: +- `factos_events`: append-only events ordered by `position`. -- `factos_events`: append-only event rows; -- `factos_outbox`: durable integration effects produced atomically with events. +There are no stream, per-stream revision, projection, subscription, checkpoint, +or outbox tables. Applications own read models and durable delivery state. ## Define a codec -Your domain event type remains yours. SQLite stores opaque bytes plus queryable -metadata, so the application provides an event codec. +SQLite stores opaque bytes. The application codec maps domain events to the +shared `factos.Event(BitArray)` storage contract and decodes raw rows back into +domain events. ```gleam -fn ticket_codec() -> factos_sqlight.EventCodec(Event) { - factos_sqlight.codec(encode: encode_event, decode: decode_event) +import factos +import factos/factos_sqlight +import gleam/bit_array +import gleam/result + +fn ticket_codec() -> factos.EventCodec(Event, BitArray) { + factos.codec(encode: encode_event, decode: decode_event) } -``` -The encoder prepares an event for persistence: +fn encode_event(event: Event) -> factos.Event(BitArray) { + let TicketSold(buyer:) = event -```gleam -fn encode_event(event: Event) -> factos_sqlight.Proposed(Event) { - case event { - TicketSold(buyer) -> - factos_sqlight.Proposed( - id: "ticket-sold-" <> buyer, - event: event, - type_: factos.event_type("TicketSold"), - version: 1, - tags: [factos.tag("event:gleamconf-2026")], - metadata: factos.empty_metadata(), - data: bit_array.from_string(buyer), - ) - } + factos.new_event( + type_: factos.event_type("TicketSold"), + version: 1, + data: bit_array.from_string(buyer), + ) + |> factos.with_tags(tags: [factos.tag("event:gleamconf-2026")]) } -``` - -The decoder turns stored rows back into domain events: -```gleam fn decode_event( - stored: factos_sqlight.StoredEvent, -) -> Result(factos.Decoded(Event), factos_sqlight.DecodeError) { - case factos.event_type_name(stored.type_) { - "TicketSold" -> { + stored: factos.Recorded(BitArray), +) -> Result(Event, factos.DecodeError) { + let descriptor = stored.descriptor + + case factos.event_type_name(descriptor.type_), descriptor.version { + "TicketSold", 1 -> { use buyer <- result.try( - bit_array.to_string(stored.data) - |> result.replace_error(factos_sqlight.InvalidData), + bit_array.to_string(stored.event) + |> result.replace_error(factos.InvalidData), ) - Ok(factos.Decoded( - event: TicketSold(buyer), - descriptor: factos.EventDescriptor( - type_: stored.type_, - version: stored.version, - tags: stored.tags, - metadata: stored.metadata, - ), - )) + Ok(TicketSold(buyer:)) } - _ -> Error(factos_sqlight.UnknownEvent) + _, _ -> Error(factos.UnknownEvent) } } ``` -Tags are the query contract. If future commands need to find an event by payload -value, expose that value as a tag when writing the event. +The SQLite row decoder reads the `data` column with +`gleam/dynamic/decode.bit_array`, so the application decoder receives the raw +payload as `stored.event` on a `factos.Recorded(BitArray)`. + +The descriptor read from SQLite remains authoritative on the resulting +`factos.Recorded` value. Tags are the selective-read contract: if a future +command must find an event by a payload value, expose that value as a stable tag +when encoding the event. ## Dispatch commands -`dispatch` is the single write API. Build a dispatch with the required stream, -decider, and codec, then add a context query, reactor, or retry configuration -only when that command needs them. Pass the command as the second argument to -`dispatch`. +Every builder requires an explicit decision context. Use: + +- `factos.NoContext` for commands that intentionally ignore prior facts; +- `factos.AllEvents` for global rules; +- `factos.Matching(items:)` for selective type-and-tag rules. ```gleam -let builder = - factos_sqlight.new_dispatch( +fn sale_context(event_id: String) -> factos.DecisionContext { + factos.Matching(items: [ + factos.item( + types: [factos.event_type("TicketSold")], + tags: [factos.tag("event:" <> event_id)], + ), + ]) +} + +let assert Ok(dispatch) = + factos.new_dispatch( connection: connection, - stream: buyer_stream(attempt), + decision_context: sale_context("gleamconf-2026"), decider: ticket_decider(), codec: ticket_codec(), ) - |> factos_sqlight.with_query(query: sale_query()) - -let assert Ok(dispatch) = - factos_sqlight.dispatch(builder, BuyTicket(buyer_name(attempt))) + |> factos_sqlight.dispatch( + BuyTicket(buyer: "renata"), + event_id: new_event_id, + ) ``` -The backend: +The application supplies `event_id: fn() -> String`. `factos_events.id` is +unique, so a duplicate id fails the transaction rather than recording two facts +with one identity. -1. opens a SQLite `BEGIN IMMEDIATE` transaction; -2. reads rows matching the builder's query, or the builder's stream when no - query is configured; -3. decodes and folds them into decision state; -4. runs the decider; -5. appends only if the stream revision check and any - `FailIfEventsMatch(query, after)` check still hold; -6. inserts the new events; -7. inserts outbox effects if the builder has a reactor; -8. retries transient SQLite busy/locked transaction starts up to the builder's - retry attempts; -9. returns append metadata and committed records. +A dispatch: -The return type is: +1. starts `BEGIN IMMEDIATE`, acquiring SQLite's writer lock; +2. reads and decodes records selected by the decision context; +3. folds them into temporary decision state; +4. runs the pure decider; +5. verifies the context append condition; +6. inserts accepted events in append order; +7. runs matching strong subscriptions; +8. commits; +9. starts matching fire-and-forget work; +10. returns `factos.Dispatch(position:, events:)`. -```gleam -pub type Dispatch(event) { - Dispatch(append: Append, events: List(factos.Recorded(event))) -} -``` +An empty decision produces `factos.NoPosition`, records no events, does not call +the id generator, and invokes no subscriptions. -`dispatch.events` contains the committed records inserted by this dispatch. Use -those records for reactors or durable effect adapters. +`factos.with_retry_attempts` controls retries for SQLite busy and lock errors. +Deciders, codecs, id generation, and strong callbacks may therefore run more +than once for a logical command. Keep retryable work deterministic and keep +strong callback side effects on the supplied transaction connection. -## Stream-only dispatch +## Dispatch-bound subscriptions -Leave the query unset when one stream revision is intentionally the consistency -boundary: +A subscription filters only events accepted by the dispatch carrying it. It is +not a historical consumer and has no cursor, replay, checkpoint, or catch-up +state. ```gleam +let projection = + factos.new_subscription( + decision_context: factos.AllEvents, + consistency: factos.StrongConsistency, + handle: fn(transaction_connection, recorded) { + projection.insert(transaction_connection, recorded) + }, + ) + let assert Ok(dispatch) = - factos_sqlight.new_dispatch( + factos.new_dispatch( connection: connection, - stream: "ticket-sale-renata", + decision_context: sale_context("gleamconf-2026"), decider: ticket_decider(), codec: ticket_codec(), ) - |> factos_sqlight.dispatch(BuyTicket("renata")) + |> factos.with_subscriptions(subscriptions: [projection]) + |> factos_sqlight.dispatch( + BuyTicket(buyer: "renata"), + event_id: new_event_id, + ) ``` -Stream-only dispatch remains useful for stream-shaped rules, but a builder with -`with_query` is the better fit when a command depends on facts selected by event -type and tag. +### Strong consistency -## React durably after commit +`factos.StrongConsistency` runs each matching callback once per accepted record, +in subscription order and then append order, before commit. All callbacks receive +the dispatch transaction connection. -`factos_sqlight` does not run side effects. Attach a pure reactor to persist -effect envelopes into `factos_outbox` in the same transaction as the events: +The first returned callback error becomes +`factos.SubscriptionError(error:)`. SQLite rolls back the callback writes, every +accepted event, and writes made by earlier strong callbacks. Subscription +failures are not retried on their own. -```gleam -let assert Ok(dispatch) = - factos_sqlight.new_dispatch( - connection: connection, - stream: "ticket-sale-renata", - decider: ticket_decider(), - codec: ticket_codec(), - ) - |> factos_sqlight.with_reactor( - reactor: ticket_reactor(), - codec: ticket_effect_codec(), - ) - |> factos_sqlight.dispatch(BuyTicket("renata")) -``` +Use this mode to update SQLite projections or insert application-owned outbox +rows atomically with accepted events. Do not perform external network or file IO +inside the transaction. -Application workers lease and acknowledge durable effects: +### Fire and forget -```gleam -let assert Ok(messages) = - factos_sqlight.lease_outbox( - connection, - consumer: "email-worker", - target: "smtp", - limit: 10, - lease_for_milliseconds: 30_000, - ) -``` +`factos.FireAndForget` starts one independent asynchronous process for each +matching subscription after the final commit succeeds. There is no supervisor +to install or name to configure. -Use `ack_outbox` after successful delivery and `nack_outbox` to release a leased -message for retry after a delay. +Each process handles that subscription's matching records in append order. +Returned callback errors are ignored and processing continues. A panic +terminates the process and drops its remaining records; the process is not +supervised or restarted. Separate subscriptions and dispatches may run +concurrently, with no ordering guarantee between their processes. -## Tradeoff: single-writer SQLite transactions +Failure to start or finish this best-effort work cannot change the already +committed dispatch result. This mode has no retry, replay, idempotency, +checkpoint, or dead-letter guarantee. For durable external delivery, insert an +application-owned outbox row from a strong callback and run a separate worker. -SQLite serializes writers. `factos_sqlight` uses `BEGIN IMMEDIATE` so context -checks and event inserts happen while the connection owns the write transaction. -This preserves Factos append-condition correctness for one SQLite database, but -unrelated writers queue behind the active writer. +## Errors -For embedded or local-first applications this is often the desired tradeoff. For -high-throughput multi-node workloads, use a server backend such as `factos_pog`. +`factos.Error(domain_error, subscription_error, sqlight.Error)` distinguishes +the shared constructors: -## Example +- `factos.DomainError` from the decider; +- `factos.SubscriptionError(error:)` from a strong callback; +- `factos.StoreError` carrying a `sqlight.Error`; +- `factos.AppendConditionFailed` when relevant context changed; +- `factos.DecodeError` for unsupported or invalid stored events. -Run the concurrent restaurant order example: +Use the shared formatter with application formatters and the backend's SQLite +store-error formatter: -```sh -cd examples/orders -gleam run +```gleam +factos.error_to_string( + error, + domain_error_to_string, + subscription_error_to_string, + factos_sqlight.sqlight_error_to_string, +) ``` -The example uses path dependencies back to the local `factos` and -`factos_sqlight` packages. It dispatches many orders concurrently into one SQLite -database, using stream dispatch for each order and loading the final stream state -for verification. +## SQLite concurrency tradeoff + +`BEGIN IMMEDIATE` serializes writers before reading command context. This makes +the decision read, append check, event inserts, and strong callback writes one +stable transaction, but unrelated writers queue behind the active writer. + +That is usually the right tradeoff for embedded and local-first applications. +Use a server backend such as `factos_pog` for high-throughput, multi-node write +workloads. diff --git a/backends/factos_sqlight/docs/how-it-works.md b/backends/factos_sqlight/docs/how-it-works.md index 8392cce..4d0861a 100644 --- a/backends/factos_sqlight/docs/how-it-works.md +++ b/backends/factos_sqlight/docs/how-it-works.md @@ -1,102 +1,151 @@ # How `factos_sqlight` Works -`factos_sqlight` is a SQLite event-store backend for the Factos core model. +`factos_sqlight` owns SQLite event persistence, transaction retries, dispatch +execution, and SQLite store-error formatting. Shared codecs, event records, +dispatch builders, subscriptions, consistency modes, and error constructors +live in `factos`. Applications own their domain events, projection folds, effect +derivation, durable delivery, and operational policies. -It exposes one write path: build a `DispatchBuilder` with -`factos_sqlight.new_dispatch` and pass it to `factos_sqlight.dispatch`. The -builder can protect either: +## Storage model -- one stream revision, when no query is configured; -- an arbitrary Factos event-type/tag context, when `with_query` is used. - -The builder can also attach a reactor with `with_reactor`, causing produced -effects to be inserted into `factos_outbox` atomically with the events. - -`dispatch` returns `Dispatch(event)`, which includes append metadata and the -committed `factos.Recorded(event)` values inserted by the dispatch. - -## Event rows - -The append-only table is `factos_events`: +The backend owns one append-only table, `factos_events`: | Column | Meaning | | --- | --- | -| `position` | Global append order. | -| `id` | Application event id. | -| `stream` | Stream name. | -| `revision` | Per-stream revision. | -| `type` | Store-visible event type name. | -| `version` | Event version from the application codec. | -| `tags` | Newline-encoded store-visible tags. | -| `metadata` | Newline-encoded application metadata. | +| `position` | Monotonic global append order. | +| `id` | Unique application-supplied event identity. | +| `type` | Stable store-visible event type. | +| `version` | Codec version. | +| `tags` | Store-visible values used by selective decision contexts. | +| `metadata` | Application metadata. | | `data` | Opaque application bytes. | -Tags are stored in the event row. Query predicates filter by event type and exact -newline-bounded tag text before payload decoding, so unrelated unknown event -types do not break typed/tagged context reads. - -## Dispatch flow - -A builder without `with_query` is for commands whose consistency boundary is one -stream. A builder with `with_query` is for commands whose consistency boundary is -a `factos.Query`. - -The SQLite transaction does this: - -1. starts `BEGIN IMMEDIATE`; -2. selects candidate rows from either the stream or the configured query; -3. decodes candidate rows using the application codec; -4. folds events with the decider's `evolve` function; -5. runs the decider with the command; -6. checks the stream revision or `FailIfEventsMatch(query, after)`; -7. inserts produced events; -8. inserts outbox effects when the builder has a reactor; -9. commits and returns `Dispatch(event)`. +There is no stream or per-stream revision. Facts are ordered once, globally. +`factos.Recorded` therefore contains an id, global position, decoded domain +event, and descriptor; it has no stream identity. -SQLite serializes writers for a database file. `BEGIN IMMEDIATE` means the -connection takes the write transaction before reading the decision context, so no -other writer can change the checked stream or context until this dispatch commits -or rolls back. - -Transient busy/locked transaction starts can be retried with `with_retry_attempts`. -Deciders, codecs, and reactors must still be pure/idempotent because retry can -call them more than once for the same logical command. - -## Stream dispatch - -Stream-only dispatch remains useful when the business rule really is protected -by one stream. If the rule needs event types and tags across streams, add -`with_query` to the builder. +There are also no backend-owned projection, subscription, checkpoint, or outbox +tables. Those are application concerns with application-specific schemas and +operational policies. ## Codec boundary -The backend never interprets event payload bytes. The application codec decides: - -- how to encode event payloads; -- which event type name to store; -- which tags to expose for future queries; -- how to decode old stored rows; -- how to handle unknown or invalid data. - -This keeps the backend generic and makes the query contract visible at write -time. - -## Reactors and outbox effects - -`dispatch.events` contains committed records, not merely domain events. Records -include id, stream, revision, global position, type, version, tags, metadata, and -payload. +A `factos.EventCodec(event, BitArray)` has two application functions: -That makes them suitable for pure `factos.Reactor` values. When a builder uses -`with_reactor`, `factos_sqlight` stores the reactor's encoded effects in -`factos_outbox` in the same SQLite transaction as the source events. +- `encode` maps a domain event to `factos.Event(BitArray)`; +- `decode` validates a `factos.Recorded(BitArray)` and reconstructs the domain + event. -The outbox table stores delivery envelopes keyed by `(consumer, effect_key)`. -Workers use: +The backend row decoder reads the `data` column with +`gleam/dynamic/decode.bit_array` and places that raw `BitArray` in +`recorded.event`. The descriptor supplies the event type, version, tags, and +metadata. SQLite persists it beside the payload. When the application decoder +returns a domain event, the stored descriptor stays authoritative on the +resulting `factos.Recorded`; the decoder cannot silently replace the tags or +type used by consistency checks. -- `lease_outbox` to claim pending messages for a consumer and target; -- `ack_outbox` to mark delivered messages; -- `nack_outbox` to release a message for retry after a delay. +The backend filters by event type and exact newline-bounded tag values before it +calls the payload decoder. An unrelated unknown event type therefore does not +break a selective context that excludes it. -`factos_sqlight` still does not execute effects. It only makes effect delivery -durable and retryable for application workers. +## Decision contexts + +Every `factos.DispatchBuilder` requires one `factos.DecisionContext`: + +- `factos.NoContext` reads and protects no previous facts; +- `factos.AllEvents` reads and protects the complete log; +- `factos.Matching(items:)` OR-combines items; event types are OR-combined and + tags are AND-combined inside each item. + +An empty matching-item list selects no events. Contexts are explicit on the +builder; there is no implicit stream fallback. + +A context read returns decoded records, folded decision state, the highest +observed global position, and +`factos.FailIfEventsMatch(decision_context:, after:)`. That append condition +states exactly which newly appended facts would invalidate the command's +decision. + +## Dispatch transaction + +`factos_sqlight.dispatch` performs one `BEGIN IMMEDIATE` transaction: + +1. acquire SQLite's writer lock; +2. select and decode the builder's decision context; +3. fold records with the decider's pure `evolve` function; +4. run the pure `decide` function; +5. verify that no matching event appeared after the observed position; +6. call the supplied event-id function and insert each accepted event in order; +7. run matching `factos.StrongConsistency` callbacks in subscription and append + order; +8. commit; +9. start an independent process for each matching `factos.FireAndForget` + subscription; +10. return `factos.Dispatch(position:, events:)`. + +`BEGIN IMMEDIATE` acquires the writer lock before the read. Other writers cannot +change the checked context before this transaction commits or rolls back. The +explicit append condition remains the storage contract even though the SQLite +lock makes a concurrent match impossible during this transaction. + +A command producing no events returns `factos.NoPosition`. It performs no inserts, +does not call the event-id function, and invokes no subscription callbacks. + +## Transaction retries + +SQLite may reject transaction work with busy or lock errors. +`factos.with_retry_attempts` bounds automatic retries; its minimum is one +attempt. Domain errors, subscription errors, decode errors, and +append-condition failures are not retryable. + +A retry reruns the complete read-decide-append transaction. Deciders and codecs +must be deterministic and side-effect free. Event-id generation may be called +again. Strong callbacks must keep changes on the supplied transaction connection +because an aborted attempt rolls those writes back before another attempt. + +## Strong subscriptions + +A `factos.StrongConsistency` subscription filters only records accepted by its +carrying dispatch. Matching callbacks receive the transaction connection and +execute before commit. + +Callbacks run one subscription at a time and one record at a time. The first +returned error becomes `factos.SubscriptionError(error:)`; no later callback +runs. SQLite then rolls back: + +- every event inserted by the dispatch; +- writes from the failing callback; +- writes from all earlier strong callbacks. + +This mode is appropriate for transactionally maintained SQLite projections and +application-owned outbox rows. External IO does not belong inside the +transaction. + +## Fire-and-forget subscriptions + +A `factos.FireAndForget` subscription starts only after the final commit. Each +matching subscription gets one independent asynchronous process directly; there +is no supervisor name, supervision tree integration, or restart policy. + +The process handles that subscription's matching records in append order. +Returned callback errors are ignored and processing continues. A panic +terminates the process and drops its remaining records. Separate subscriptions +and dispatches may run concurrently, with no ordering guarantee between their +processes. + +Starting and running these processes is best effort and cannot change the +already committed dispatch result. This mode has no historical catch-up, +checkpoint, replay, retry, idempotency, or dead-letter mechanism. Durable +delivery requires an application-owned outbox inserted from a strong callback +and consumed by a separate worker with the application's chosen supervision and +recovery policy. + +## Concurrency tradeoff + +SQLite serializes writers for one database file. This makes the complete Factos +transaction straightforward and strong, but unrelated commands wait behind a +long-running strong callback. + +Keep strong callbacks short and database-local. Use fire-and-forget only when +loss is acceptable. For multi-node or highly concurrent write workloads, use a +server backend such as `factos_pog`. diff --git a/backends/factos_sqlight/examples/orders/.gitignore b/backends/factos_sqlight/examples/orders/.gitignore deleted file mode 100644 index 982745b..0000000 --- a/backends/factos_sqlight/examples/orders/.gitignore +++ /dev/null @@ -1,7 +0,0 @@ -*.beam -*.ez -/build -**/build -node_modules -**/node_modules -erl_crash.dump diff --git a/backends/factos_sqlight/examples/orders/gleam.toml b/backends/factos_sqlight/examples/orders/gleam.toml deleted file mode 100644 index 534517e..0000000 --- a/backends/factos_sqlight/examples/orders/gleam.toml +++ /dev/null @@ -1,13 +0,0 @@ -name = "orders_sqlight" -version = "1.0.0" - -[dependencies] -factos = { path = "../../../.." } -factos_sqlight = { path = "../.." } -gleam_erlang = ">= 1.0.0 and < 2.0.0" -gleam_stdlib = ">= 1.0.0 and < 2.0.0" -simplifile = ">= 2.0.0 and < 3.0.0" -sqlight = ">= 1.1.0 and < 2.0.0" - -[dev_dependencies] -gleeunit = ">= 1.0.0 and < 2.0.0" diff --git a/backends/factos_sqlight/examples/orders/manifest.toml b/backends/factos_sqlight/examples/orders/manifest.toml deleted file mode 100644 index f9459aa..0000000 --- a/backends/factos_sqlight/examples/orders/manifest.toml +++ /dev/null @@ -1,28 +0,0 @@ -# Do not manually edit this file, it is managed by Gleam. -# -# This file locks the dependency versions used, to make your build -# deterministic and to prevent unexpected versions from being included -# in your application. -# -# You should check this file into your source control repository. - -packages = [ - { name = "esqlite", version = "0.9.0", build_tools = ["rebar3"], requirements = [], otp_app = "esqlite", source = "hex", outer_checksum = "CCF72258A4EE152EC7AD92AA9A03552EB6CA1B06B65C93AD5B6E55C302E05855" }, - { name = "factos", version = "1.0.0", build_tools = ["gleam"], requirements = ["gleam_stdlib"], source = "local", path = "../../../.." }, - { name = "factos_sqlight", version = "1.0.0", build_tools = ["gleam"], requirements = ["factos", "gleam_stdlib", "sqlight"], source = "local", path = "../.." }, - { name = "filepath", version = "1.1.2", build_tools = ["gleam"], requirements = ["gleam_stdlib"], otp_app = "filepath", source = "hex", outer_checksum = "B06A9AF0BF10E51401D64B98E4B627F1D2E48C154967DA7AF4D0914780A6D40A" }, - { name = "gleam_erlang", version = "1.3.0", build_tools = ["gleam"], requirements = ["gleam_stdlib"], otp_app = "gleam_erlang", source = "hex", outer_checksum = "1124AD3AA21143E5AF0FC5CF3D9529F6DB8CA03E43A55711B60B6B7B3874375C" }, - { name = "gleam_stdlib", version = "1.0.5", build_tools = ["gleam"], requirements = [], otp_app = "gleam_stdlib", source = "hex", outer_checksum = "CEE5B6C076A85B45F60C585F4316C63EC8B7127C119D5738C3958A9C4D50404E" }, - { name = "gleeunit", version = "1.11.0", build_tools = ["gleam"], requirements = ["gleam_stdlib"], otp_app = "gleeunit", source = "hex", outer_checksum = "EC31ABA74256AEA531EDF8169931D775BBB384FED0A8A1BDC4DD9354E3E21826" }, - { name = "simplifile", version = "2.7.0", build_tools = ["gleam"], requirements = ["filepath", "gleam_stdlib"], otp_app = "simplifile", source = "hex", outer_checksum = "A2727627B063E87351934C7F7F008F2D1FDB16F6DE0B8C79F9E46459CFC9C164" }, - { name = "sqlight", version = "1.2.0", build_tools = ["gleam"], requirements = ["esqlite", "gleam_stdlib"], otp_app = "sqlight", source = "hex", outer_checksum = "841768D0A81107EE2DB46B949354F284A68B2E8098315C28A1FE3C07D206C17A" }, -] - -[requirements] -factos = { path = "../../../.." } -factos_sqlight = { path = "../.." } -gleam_erlang = { version = ">= 1.0.0 and < 2.0.0" } -gleam_stdlib = { version = ">= 1.0.0 and < 2.0.0" } -gleeunit = { version = ">= 1.0.0 and < 2.0.0" } -simplifile = { version = ">= 2.0.0 and < 3.0.0" } -sqlight = { version = ">= 1.1.0 and < 2.0.0" } diff --git a/backends/factos_sqlight/examples/orders/src/order_workflow.gleam b/backends/factos_sqlight/examples/orders/src/order_workflow.gleam deleted file mode 100644 index d33acb9..0000000 --- a/backends/factos_sqlight/examples/orders/src/order_workflow.gleam +++ /dev/null @@ -1,894 +0,0 @@ -import factos -import factos/factos_sqlight -import gleam/bit_array -import gleam/erlang/application -import gleam/erlang/process -import gleam/int -import gleam/io -import gleam/list -import gleam/result -import gleam/string -import simplifile -import sqlight - -const order_count = 40 - -const concurrency = 8 - -const receive_timeout = 30_000 - -const write_retries = 200 - -pub type Command { - OpenOrder(table: Int) - AddItem(sku: String, name: String, price: Int) - RemoveItem(sku: String) - SubmitOrder - StartPreparing - MarkReady - Serve - Pay(amount: Int) - CancelOrder(reason: String) -} - -pub type Event { - OrderOpened(table: Int) - ItemAdded(sku: String, name: String, price: Int) - ItemRemoved(sku: String) - OrderSubmitted - PreparationStarted - OrderMarkedReady - OrderServed - PaymentReceived(amount: Int) - OrderCancelled(reason: String) -} - -pub type Item { - Item(sku: String, name: String, price: Int) -} - -pub type State { - NoOrder - Draft(table: Int, items: List(Item)) - Submitted(table: Int, items: List(Item)) - Preparing(table: Int, items: List(Item)) - Ready(table: Int, items: List(Item)) - Served(table: Int, items: List(Item)) - Paid(table: Int, items: List(Item), amount: Int) - Cancelled(reason: String) -} - -pub type DomainError { - OrderAlreadyOpen - OrderNotOpen - DuplicateItem(sku: String) - ItemNotFound(sku: String) - EmptyOrder - PaymentTooLow(required: Int, paid: Int) - WorkerTimedOut(remaining: Int) - InvalidTransition(action: String, state: String) -} - -pub type KitchenSummary { - KitchenSummary( - opened: Int, - submitted: Int, - preparing: Int, - ready: Int, - served: Int, - paid: Int, - cancelled: Int, - revenue: Int, - ) -} - -pub type ExampleResult { - ExampleResult( - order_id: String, - final_state: State, - kitchen_summary: KitchenSummary, - recorded_events: Int, - ) -} - -pub type StressResult { - StressResult( - orders: Int, - paid_orders: Int, - cancelled_orders: Int, - recorded_events: Int, - revenue: Int, - ) -} - -type WorkerResult { - WorkerResult(final_state: State, recorded_events: Int, revenue: Int) -} - -type WorkerMessage { - WorkerFinished( - order_number: Int, - result: Result(WorkerResult, factos_sqlight.Error(DomainError)), - ) -} - -pub fn main() -> Nil { - case run() { - Ok(result) -> - io.println( - "restaurant stress workflow completed: " - <> int.to_string(result.orders) - <> " orders, " - <> int.to_string(result.recorded_events) - <> " events", - ) - Error(_) -> io.println("restaurant stress workflow failed") - } -} - -pub fn run() -> Result(StressResult, factos_sqlight.Error(DomainError)) { - let database_path = "/tmp/factos_examples_stress.sqlite3" - log("reset database " <> database_path) - let _ = simplifile.delete_file(database_path) - - log("prepare database") - use _ <- result.try(prepare_database(database_path)) - - let workers = process.new_subject() - - let _ = - int.range(from: 1, to: concurrency + 1, with: Nil, run: fn(_, order_number) { - spawn_order(workers, database_path, order_number) - Nil - }) - - collect_workers( - workers, - database_path: database_path, - remaining: order_count, - next_order: concurrency + 1, - summary: StressResult(0, 0, 0, 0, 0), - ) -} - -fn spawn_order( - workers: process.Subject(WorkerMessage), - database_path: String, - order_number: Int, -) -> Nil { - log("spawn order " <> int.to_string(order_number)) - let _ = - process.spawn(fn() { - log("order " <> int.to_string(order_number) <> " started") - let result = run_order(database_path, order_number) - log( - "order " - <> int.to_string(order_number) - <> " finished with " - <> result_to_string(result), - ) - process.send(workers, WorkerFinished(order_number, result)) - }) - Nil -} - -fn prepare_database( - database_path: String, -) -> Result(Nil, factos_sqlight.Error(DomainError)) { - use connection <- sqlight.with_connection(database_path) - use _ <- result.try(configure_connection(connection)) - execute_migration_file(connection) -} - -fn execute_migration_file( - connection: sqlight.Connection, -) -> Result(Nil, factos_sqlight.Error(DomainError)) { - use priv_directory <- result.try( - application.priv_directory("factos_sqlight") - |> result.map_error(fn(_error) { - factos_sqlight.StoreError(sqlight.SqlightError( - code: sqlight.Cantopen, - message: "could not locate factos_sqlight priv directory", - offset: -1, - )) - }), - ) - use sql <- result.try( - simplifile.read(priv_directory <> "/migrations.sql") - |> result.map_error(fn(_error) { - factos_sqlight.StoreError(sqlight.SqlightError( - code: sqlight.Cantopen, - message: "could not read factos_sqlight migrations.sql", - offset: -1, - )) - }), - ) - sqlight.exec(sql, on: connection) - |> result.map_error(factos_sqlight.StoreError) -} - -fn configure_connection( - connection: sqlight.Connection, -) -> Result(Nil, factos_sqlight.Error(DomainError)) { - sqlight.exec( - "pragma journal_mode = wal; pragma busy_timeout = 50", - on: connection, - ) - |> result.map_error(factos_sqlight.StoreError) -} - -fn run_order( - database_path: String, - order_number: Int, -) -> Result(WorkerResult, factos_sqlight.Error(DomainError)) { - use connection <- sqlight.with_connection(database_path) - use _ <- result.try(configure_connection(connection)) - - let order_id = "stress-" <> int.to_string(order_number) - use _ <- result.try(dispatch_commands( - connection, - order_number, - order_id, - workflow(order_number), - )) - - log("order " <> int.to_string(order_number) <> " loading stream") - use loaded <- result.try(factos_sqlight.load_stream( - connection, - stream: order_stream(order_id), - decider: order_decider(), - codec: order_codec(), - )) - - Ok(WorkerResult( - final_state: loaded.state, - recorded_events: list.length(loaded.events), - revenue: revenue(loaded.state), - )) -} - -fn collect_workers( - workers: process.Subject(WorkerMessage), - database_path database_path: String, - remaining remaining: Int, - next_order next_order: Int, - summary summary: StressResult, -) -> Result(StressResult, factos_sqlight.Error(DomainError)) { - case remaining { - 0 -> Ok(summary) - _ -> - case process.receive(workers, within: receive_timeout) { - Ok(WorkerFinished(order_number, Ok(result))) -> { - log("collector received order " <> int.to_string(order_number)) - case next_order <= order_count { - True -> spawn_order(workers, database_path, next_order) - False -> Nil - } - collect_workers( - workers, - database_path: database_path, - remaining: remaining - 1, - next_order: next_order + 1, - summary: add_worker_result(summary, result), - ) - } - Ok(WorkerFinished(order_number, Error(error))) -> { - log( - "collector received error from order " - <> int.to_string(order_number) - <> ": " - <> store_error_to_string(error), - ) - Error(error) - } - Error(Nil) -> { - log( - "collector timed out with " - <> int.to_string(remaining) - <> " remaining", - ) - Error(factos_sqlight.DomainError(WorkerTimedOut(remaining))) - } - } - } -} - -fn add_worker_result( - summary: StressResult, - result: WorkerResult, -) -> StressResult { - let #(paid_orders, cancelled_orders) = case result.final_state { - Paid(_, _, _) -> #(summary.paid_orders + 1, summary.cancelled_orders) - Cancelled(_) -> #(summary.paid_orders, summary.cancelled_orders + 1) - NoOrder - | Draft(_, _) - | Submitted(_, _) - | Preparing(_, _) - | Ready(_, _) - | Served(_, _) -> #(summary.paid_orders, summary.cancelled_orders) - } - - StressResult( - orders: summary.orders + 1, - paid_orders: paid_orders, - cancelled_orders: cancelled_orders, - recorded_events: summary.recorded_events + result.recorded_events, - revenue: summary.revenue + result.revenue, - ) -} - -fn workflow(order_number: Int) -> List(Command) { - let draft_commands = [ - OpenOrder(table: order_number), - AddItem(sku: "burger", name: "House Burger", price: 16), - AddItem(sku: "fries", name: "Fries", price: 6), - AddItem(sku: "shake", name: "Vanilla Shake", price: 8), - RemoveItem(sku: "shake"), - SubmitOrder, - ] - - case should_cancel(order_number) { - True -> - list.append(draft_commands, [ - CancelOrder(reason: "guest left before kitchen started"), - ]) - False -> - list.append(draft_commands, [ - StartPreparing, - MarkReady, - Serve, - Pay(amount: 25), - ]) - } -} - -fn should_cancel(order_number: Int) -> Bool { - int.modulo(order_number, by: 5) == Ok(0) -} - -fn dispatch_commands( - connection: sqlight.Connection, - order_number: Int, - order_id: String, - commands: List(Command), -) -> Result(Nil, factos_sqlight.Error(DomainError)) { - case commands { - [] -> Ok(Nil) - [command, ..rest] -> { - log( - "order " - <> int.to_string(order_number) - <> " dispatch " - <> command_to_string(command), - ) - use _ <- result.try(dispatch_with_retry( - connection, - order_number, - order_id, - command, - attempts: write_retries, - )) - dispatch_commands(connection, order_number, order_id, rest) - } - } -} - -fn dispatch_with_retry( - connection: sqlight.Connection, - order_number: Int, - order_id: String, - command: Command, - attempts attempts: Int, -) -> Result(factos_sqlight.Dispatch(Event), factos_sqlight.Error(DomainError)) { - let result = dispatch(connection, order_id, command) - - case attempts > 0, result { - _, Ok(append) -> Ok(append) - True, Error(factos_sqlight.StoreError(_)) -> { - log( - "order " - <> int.to_string(order_number) - <> " retry " - <> command_to_string(command) - <> " after SQLite store error; attempts left " - <> int.to_string(attempts - 1), - ) - process.sleep(retry_delay(order_number, attempts)) - dispatch_with_retry( - connection, - order_number, - order_id, - command, - attempts: attempts - 1, - ) - } - _, Error(error) -> { - log( - "order " - <> int.to_string(order_number) - <> " failed " - <> command_to_string(command) - <> " with " - <> store_error_to_string(error), - ) - Error(error) - } - } -} - -fn retry_delay(order_number: Int, attempts: Int) -> Int { - case int.modulo(order_number + attempts, by: 10) { - Ok(offset) -> 5 + offset - Error(Nil) -> 5 - } -} - -fn dispatch( - connection: sqlight.Connection, - order_id: String, - command: Command, -) -> Result(factos_sqlight.Dispatch(Event), factos_sqlight.Error(DomainError)) { - factos_sqlight.new_dispatch( - connection: connection, - stream: order_stream(order_id), - decider: order_decider(), - codec: order_codec(), - ) - |> factos_sqlight.with_retry_attempts(attempts: write_retries) - |> factos_sqlight.dispatch(command) -} - -fn order_stream(order_id: String) -> String { - "restaurant-order-" <> order_id -} - -fn revenue(state: State) -> Int { - case state { - Paid(_, _, amount) -> amount - NoOrder - | Draft(_, _) - | Submitted(_, _) - | Preparing(_, _) - | Ready(_, _) - | Served(_, _) - | Cancelled(_) -> 0 - } -} - -pub fn order_decider() -> factos.Decider(Command, State, Event, DomainError) { - factos.decider(initial: NoOrder, decide:, evolve:) -} - -fn decide(state: State, command: Command) -> Result(List(Event), DomainError) { - case state, command { - NoOrder, OpenOrder(table) -> Ok([OrderOpened(table)]) - NoOrder, _ -> Error(OrderNotOpen) - - Draft(_, _), OpenOrder(_) -> Error(OrderAlreadyOpen) - Draft(_, items), AddItem(sku, name, price) -> - case has_item(items, sku) { - True -> Error(DuplicateItem(sku)) - False -> Ok([ItemAdded(sku, name, price)]) - } - Draft(_, items), RemoveItem(sku) -> - case has_item(items, sku) { - True -> Ok([ItemRemoved(sku)]) - False -> Error(ItemNotFound(sku)) - } - Draft(_, items), SubmitOrder -> - case list.is_empty(items) { - True -> Error(EmptyOrder) - False -> Ok([OrderSubmitted]) - } - Draft(_, _), CancelOrder(reason) -> Ok([OrderCancelled(reason)]) - Draft(_, _), StartPreparing - | Draft(_, _), MarkReady - | Draft(_, _), Serve - | Draft(_, _), Pay(_) - -> invalid(command, state) - - Submitted(_, _), StartPreparing -> Ok([PreparationStarted]) - Submitted(_, _), CancelOrder(reason) -> Ok([OrderCancelled(reason)]) - Submitted(_, _), OpenOrder(_) - | Submitted(_, _), AddItem(_, _, _) - | Submitted(_, _), RemoveItem(_) - | Submitted(_, _), SubmitOrder - | Submitted(_, _), MarkReady - | Submitted(_, _), Serve - | Submitted(_, _), Pay(_) - -> invalid(command, state) - - Preparing(_, _), MarkReady -> Ok([OrderMarkedReady]) - Preparing(_, _), OpenOrder(_) - | Preparing(_, _), AddItem(_, _, _) - | Preparing(_, _), RemoveItem(_) - | Preparing(_, _), SubmitOrder - | Preparing(_, _), StartPreparing - | Preparing(_, _), Serve - | Preparing(_, _), Pay(_) - | Preparing(_, _), CancelOrder(_) - -> invalid(command, state) - - Ready(_, _), Serve -> Ok([OrderServed]) - Ready(_, _), OpenOrder(_) - | Ready(_, _), AddItem(_, _, _) - | Ready(_, _), RemoveItem(_) - | Ready(_, _), SubmitOrder - | Ready(_, _), StartPreparing - | Ready(_, _), MarkReady - | Ready(_, _), Pay(_) - | Ready(_, _), CancelOrder(_) - -> invalid(command, state) - - Served(_, items), Pay(amount) -> { - let required = total(items) - case amount >= required { - True -> Ok([PaymentReceived(amount)]) - False -> Error(PaymentTooLow(required: required, paid: amount)) - } - } - Served(_, _), OpenOrder(_) - | Served(_, _), AddItem(_, _, _) - | Served(_, _), RemoveItem(_) - | Served(_, _), SubmitOrder - | Served(_, _), StartPreparing - | Served(_, _), MarkReady - | Served(_, _), Serve - | Served(_, _), CancelOrder(_) - -> invalid(command, state) - - Paid(_, _, _), _ -> invalid(command, state) - Cancelled(_), _ -> invalid(command, state) - } -} - -fn evolve(state: State, event: Event) -> State { - case event { - OrderOpened(table) -> Draft(table: table, items: []) - ItemAdded(sku, name, price) -> - add_item_to_state(state, Item(sku, name, price)) - ItemRemoved(sku) -> remove_item_from_state(state, sku) - OrderSubmitted -> move_to_submitted(state) - PreparationStarted -> move_to_preparing(state) - OrderMarkedReady -> move_to_ready(state) - OrderServed -> move_to_served(state) - PaymentReceived(amount) -> move_to_paid(state, amount) - OrderCancelled(reason) -> Cancelled(reason) - } -} - -fn add_item_to_state(state: State, item: Item) -> State { - case state { - Draft(table, items) -> Draft(table: table, items: [item, ..items]) - NoOrder - | Submitted(_, _) - | Preparing(_, _) - | Ready(_, _) - | Served(_, _) - | Paid(_, _, _) - | Cancelled(_) -> state - } -} - -fn remove_item_from_state(state: State, sku: String) -> State { - case state { - Draft(table, items) -> - Draft( - table: table, - items: list.filter(items, fn(item) { item.sku != sku }), - ) - NoOrder - | Submitted(_, _) - | Preparing(_, _) - | Ready(_, _) - | Served(_, _) - | Paid(_, _, _) - | Cancelled(_) -> state - } -} - -fn move_to_submitted(state: State) -> State { - case state { - Draft(table, items) -> Submitted(table: table, items: items) - NoOrder - | Submitted(_, _) - | Preparing(_, _) - | Ready(_, _) - | Served(_, _) - | Paid(_, _, _) - | Cancelled(_) -> state - } -} - -fn move_to_preparing(state: State) -> State { - case state { - Submitted(table, items) -> Preparing(table: table, items: items) - NoOrder - | Draft(_, _) - | Preparing(_, _) - | Ready(_, _) - | Served(_, _) - | Paid(_, _, _) - | Cancelled(_) -> state - } -} - -fn move_to_ready(state: State) -> State { - case state { - Preparing(table, items) -> Ready(table: table, items: items) - NoOrder - | Draft(_, _) - | Submitted(_, _) - | Ready(_, _) - | Served(_, _) - | Paid(_, _, _) - | Cancelled(_) -> state - } -} - -fn move_to_served(state: State) -> State { - case state { - Ready(table, items) -> Served(table: table, items: items) - NoOrder - | Draft(_, _) - | Submitted(_, _) - | Preparing(_, _) - | Served(_, _) - | Paid(_, _, _) - | Cancelled(_) -> state - } -} - -fn move_to_paid(state: State, amount: Int) -> State { - case state { - Served(table, items) -> Paid(table: table, items: items, amount: amount) - NoOrder - | Draft(_, _) - | Submitted(_, _) - | Preparing(_, _) - | Ready(_, _) - | Paid(_, _, _) - | Cancelled(_) -> state - } -} - -pub fn kitchen_summary_view() -> factos.View(KitchenSummary, Event) { - factos.view( - initial: KitchenSummary(0, 0, 0, 0, 0, 0, 0, 0), - evolve: evolve_summary, - ) -} - -fn evolve_summary(summary: KitchenSummary, event: Event) -> KitchenSummary { - case event { - OrderOpened(_) -> KitchenSummary(..summary, opened: summary.opened + 1) - ItemAdded(_, _, _) | ItemRemoved(_) -> summary - OrderSubmitted -> - KitchenSummary(..summary, submitted: summary.submitted + 1) - PreparationStarted -> - KitchenSummary(..summary, preparing: summary.preparing + 1) - OrderMarkedReady -> KitchenSummary(..summary, ready: summary.ready + 1) - OrderServed -> KitchenSummary(..summary, served: summary.served + 1) - PaymentReceived(amount) -> - KitchenSummary( - ..summary, - paid: summary.paid + 1, - revenue: summary.revenue + amount, - ) - OrderCancelled(_) -> - KitchenSummary(..summary, cancelled: summary.cancelled + 1) - } -} - -pub fn order_codec() -> factos_sqlight.EventCodec(Event) { - factos_sqlight.codec(encode: encode_event, decode: decode_event) -} - -fn encode_event(event: Event) -> factos_sqlight.Proposed(Event) { - case event { - OrderOpened(table) -> - proposed(event, "OrderOpened", [int.to_string(table)], []) - ItemAdded(sku, name, price) -> - proposed(event, "ItemAdded", [sku, name, int.to_string(price)], [ - factos.tag("sku:" <> sku), - ]) - ItemRemoved(sku) -> - proposed(event, "ItemRemoved", [sku], [ - factos.tag("sku:" <> sku), - ]) - OrderSubmitted -> proposed(event, "OrderSubmitted", [], []) - PreparationStarted -> proposed(event, "PreparationStarted", [], []) - OrderMarkedReady -> proposed(event, "OrderMarkedReady", [], []) - OrderServed -> proposed(event, "OrderServed", [], []) - PaymentReceived(amount) -> - proposed(event, "PaymentReceived", [int.to_string(amount)], []) - OrderCancelled(reason) -> proposed(event, "OrderCancelled", [reason], []) - } -} - -fn proposed( - event: Event, - type_name: String, - fields: List(String), - tags: List(factos.Tag), -) -> factos_sqlight.Proposed(Event) { - factos_sqlight.Proposed( - id: "example-" <> type_name <> "-" <> fields_to_payload(fields), - event: event, - type_: factos.event_type(type_name), - version: 1, - tags: [factos.tag("restaurant"), ..tags], - metadata: factos.empty_metadata(), - data: bit_array.from_string(fields_to_payload(fields)), - ) -} - -fn decode_event( - stored: factos_sqlight.StoredEvent, -) -> Result(factos.Decoded(Event), factos_sqlight.DecodeError) { - let type_name = factos.event_type_name(stored.type_) - let fields = - stored.data - |> bit_array.to_string - |> result.replace_error(factos_sqlight.InvalidData) - |> result.map(payload_to_fields) - - use event <- result.try(decode_fields(type_name, fields)) - Ok(factos.Decoded( - event:, - descriptor: factos.EventDescriptor( - type_: stored.type_, - version: stored.version, - tags: stored.tags, - metadata: stored.metadata, - ), - )) -} - -fn decode_fields( - type_name: String, - fields_result: Result(List(String), factos_sqlight.DecodeError), -) -> Result(Event, factos_sqlight.DecodeError) { - use fields <- result.try(fields_result) - case type_name, fields { - "OrderOpened", [table] -> { - use table <- result.try(parse_int(table, type_name)) - Ok(OrderOpened(table)) - } - "ItemAdded", [sku, name, price] -> { - use price <- result.try(parse_int(price, type_name)) - Ok(ItemAdded(sku, name, price)) - } - "ItemRemoved", [sku] -> Ok(ItemRemoved(sku)) - "OrderSubmitted", [] -> Ok(OrderSubmitted) - "PreparationStarted", [] -> Ok(PreparationStarted) - "OrderMarkedReady", [] -> Ok(OrderMarkedReady) - "OrderServed", [] -> Ok(OrderServed) - "PaymentReceived", [amount] -> { - use amount <- result.try(parse_int(amount, type_name)) - Ok(PaymentReceived(amount)) - } - "OrderCancelled", [reason] -> Ok(OrderCancelled(reason)) - _, _ -> Error(factos_sqlight.UnknownEvent) - } -} - -fn parse_int( - value: String, - _type_name: String, -) -> Result(Int, factos_sqlight.DecodeError) { - int.parse(value) - |> result.replace_error(factos_sqlight.InvalidData) -} - -fn fields_to_payload(fields: List(String)) -> String { - string.join(fields, with: "|") -} - -fn payload_to_fields(payload: String) -> List(String) { - case string.is_empty(payload) { - True -> [] - False -> string.split(payload, on: "|") - } -} - -fn has_item(items: List(Item), sku: String) -> Bool { - list.any(items, fn(item) { item.sku == sku }) -} - -fn total(items: List(Item)) -> Int { - list.fold(items, 0, fn(total, item) { total + item.price }) -} - -fn invalid(command: Command, state: State) -> Result(List(Event), DomainError) { - Error(InvalidTransition(command_to_string(command), state_to_string(state))) -} - -fn command_to_string(command: Command) -> String { - case command { - OpenOrder(_) -> "OpenOrder" - AddItem(_, _, _) -> "AddItem" - RemoveItem(_) -> "RemoveItem" - SubmitOrder -> "SubmitOrder" - StartPreparing -> "StartPreparing" - MarkReady -> "MarkReady" - Serve -> "Serve" - Pay(_) -> "Pay" - CancelOrder(_) -> "CancelOrder" - } -} - -fn result_to_string( - result: Result(WorkerResult, factos_sqlight.Error(DomainError)), -) -> String { - case result { - Ok(worker_result) -> - "ok " - <> state_to_string(worker_result.final_state) - <> " events=" - <> int.to_string(worker_result.recorded_events) - Error(error) -> "error " <> store_error_to_string(error) - } -} - -fn store_error_to_string(error: factos_sqlight.Error(DomainError)) -> String { - case error { - factos_sqlight.DomainError(error) -> - "domain:" <> domain_error_to_string(error) - factos_sqlight.DecodeError(error) -> - "decode:" <> factos_sqlight_decode_error_to_string(error) - factos_sqlight.StoreError(sqlight.SqlightError(code, message, _)) -> - "sqlite(code=" - <> int.to_string(sqlight.error_code_to_int(code)) - <> ", message=" - <> message - <> ")" - factos_sqlight.AppendConditionFailed(_) -> "append-condition-failed" - } -} - -fn domain_error_to_string(error: DomainError) -> String { - case error { - OrderAlreadyOpen -> "OrderAlreadyOpen" - OrderNotOpen -> "OrderNotOpen" - DuplicateItem(sku) -> "DuplicateItem(" <> sku <> ")" - ItemNotFound(sku) -> "ItemNotFound(" <> sku <> ")" - EmptyOrder -> "EmptyOrder" - PaymentTooLow(required, paid) -> - "PaymentTooLow(required=" - <> int.to_string(required) - <> ", paid=" - <> int.to_string(paid) - <> ")" - WorkerTimedOut(remaining) -> - "WorkerTimedOut(remaining=" <> int.to_string(remaining) <> ")" - InvalidTransition(action, state) -> - "InvalidTransition(" <> action <> ", " <> state <> ")" - } -} - -fn factos_sqlight_decode_error_to_string( - error: factos_sqlight.DecodeError, -) -> String { - case error { - factos_sqlight.UnknownEvent -> "UnknownEvent" - factos_sqlight.InvalidData -> "InvalidData" - } -} - -fn log(message: String) -> Nil { - io.println("[factos-example] " <> message) -} - -fn state_to_string(state: State) -> String { - case state { - NoOrder -> "NoOrder" - Draft(_, _) -> "Draft" - Submitted(_, _) -> "Submitted" - Preparing(_, _) -> "Preparing" - Ready(_, _) -> "Ready" - Served(_, _) -> "Served" - Paid(_, _, _) -> "Paid" - Cancelled(_) -> "Cancelled" - } -} diff --git a/backends/factos_sqlight/examples/orders/src/orders_sqlight.gleam b/backends/factos_sqlight/examples/orders/src/orders_sqlight.gleam deleted file mode 100644 index 09c68b8..0000000 --- a/backends/factos_sqlight/examples/orders/src/orders_sqlight.gleam +++ /dev/null @@ -1,5 +0,0 @@ -import order_workflow - -pub fn main() -> Nil { - order_workflow.main() -} diff --git a/backends/factos_sqlight/examples/orders/test/orders_sqlight_test.gleam b/backends/factos_sqlight/examples/orders/test/orders_sqlight_test.gleam deleted file mode 100644 index be92879..0000000 --- a/backends/factos_sqlight/examples/orders/test/orders_sqlight_test.gleam +++ /dev/null @@ -1,16 +0,0 @@ -import gleeunit -import order_workflow - -pub fn main() -> Nil { - gleeunit.main() -} - -pub fn restaurant_order_example_runs_concurrently_under_stress_test() { - let assert Ok(order_workflow.StressResult( - orders: 40, - paid_orders: 32, - cancelled_orders: 8, - recorded_events: 376, - revenue: 800, - )) = order_workflow.run() -} diff --git a/backends/factos_sqlight/gleam.toml b/backends/factos_sqlight/gleam.toml index 2ffbd7e..61e7c79 100644 --- a/backends/factos_sqlight/gleam.toml +++ b/backends/factos_sqlight/gleam.toml @@ -24,8 +24,9 @@ source = "./docs/how-it-works.md" factos = { path = "../.." } gleam_stdlib = ">= 1.0.0 and < 2.0.0" sqlight = ">= 1.1.0 and < 2.0.0" +gleam_erlang = ">= 1.3.0 and < 2.0.0" +exception = ">= 2.1.1 and < 3.0.0" [dev_dependencies] gleeunit = ">= 1.0.0 and < 2.0.0" -gleam_erlang = ">= 1.3.0 and < 2.0.0" simplifile = ">= 2.5.0 and < 3.0.0" diff --git a/backends/factos_sqlight/manifest.toml b/backends/factos_sqlight/manifest.toml index 254d8d9..3fc1c02 100644 --- a/backends/factos_sqlight/manifest.toml +++ b/backends/factos_sqlight/manifest.toml @@ -8,6 +8,7 @@ packages = [ { name = "esqlite", version = "0.9.0", build_tools = ["rebar3"], requirements = [], otp_app = "esqlite", source = "hex", outer_checksum = "CCF72258A4EE152EC7AD92AA9A03552EB6CA1B06B65C93AD5B6E55C302E05855" }, + { name = "exception", version = "2.1.1", build_tools = ["gleam"], requirements = ["gleam_stdlib"], otp_app = "exception", source = "hex", outer_checksum = "6BDEA95248093599391C3B5DF1835C5C6A86C353C2F99CE539B450E3432FE117" }, { name = "factos", version = "1.0.0", build_tools = ["gleam"], requirements = ["gleam_stdlib"], source = "local", path = "../.." }, { name = "filepath", version = "1.1.2", build_tools = ["gleam"], requirements = ["gleam_stdlib"], otp_app = "filepath", source = "hex", outer_checksum = "B06A9AF0BF10E51401D64B98E4B627F1D2E48C154967DA7AF4D0914780A6D40A" }, { name = "gleam_erlang", version = "1.3.0", build_tools = ["gleam"], requirements = ["gleam_stdlib"], otp_app = "gleam_erlang", source = "hex", outer_checksum = "1124AD3AA21143E5AF0FC5CF3D9529F6DB8CA03E43A55711B60B6B7B3874375C" }, @@ -18,6 +19,7 @@ packages = [ ] [requirements] +exception = { version = ">= 2.1.1 and < 3.0.0" } factos = { path = "../.." } gleam_erlang = { version = ">= 1.3.0 and < 2.0.0" } gleam_stdlib = { version = ">= 1.0.0 and < 2.0.0" } diff --git a/backends/factos_sqlight/priv/dbmate/20260703000100_factos_sqlight_event_store.sql b/backends/factos_sqlight/priv/dbmate/20260703000100_factos_sqlight_event_store.sql index ae6a065..92812fe 100644 --- a/backends/factos_sqlight/priv/dbmate/20260703000100_factos_sqlight_event_store.sql +++ b/backends/factos_sqlight/priv/dbmate/20260703000100_factos_sqlight_event_store.sql @@ -1,22 +1,16 @@ -- migrate:up create table if not exists factos_events ( position integer primary key autoincrement, - id text not null, - stream text not null, - revision integer not null, + id text not null unique, type text not null, version integer not null, tags text not null, metadata text not null default '', - data blob not null, - unique(stream, revision) + data blob not null ); -create index if not exists factos_events_stream_revision - on factos_events(stream, revision); - -create index if not exists factos_events_position - on factos_events(position); +create index if not exists factos_events_type_position + on factos_events(type, position); -- migrate:down drop table if exists factos_events; diff --git a/backends/factos_sqlight/priv/dbmate/20260704000100_factos_sqlight_outbox.sql b/backends/factos_sqlight/priv/dbmate/20260704000100_factos_sqlight_outbox.sql deleted file mode 100644 index a21a042..0000000 --- a/backends/factos_sqlight/priv/dbmate/20260704000100_factos_sqlight_outbox.sql +++ /dev/null @@ -1,26 +0,0 @@ --- migrate:up -create table if not exists factos_outbox ( - id integer primary key autoincrement, - source_position integer not null, - source_event_id text not null, - source_context text not null, - consumer text not null, - effect_key text not null, - target text not null, - type text not null, - metadata text not null default '', - payload blob not null, - status text not null default 'pending', - attempts integer not null default 0, - available_at_ms integer not null default 0, - locked_until_ms integer, - delivered_at_ms integer, - last_error text, - unique(consumer, effect_key) -); - -create index if not exists factos_outbox_pending - on factos_outbox(consumer, target, status, available_at_ms, locked_until_ms, id); - --- migrate:down -drop table if exists factos_outbox; diff --git a/backends/factos_sqlight/priv/migrations.sql b/backends/factos_sqlight/priv/migrations.sql index 7ebd95d..65a9c6e 100644 --- a/backends/factos_sqlight/priv/migrations.sql +++ b/backends/factos_sqlight/priv/migrations.sql @@ -1,41 +1,12 @@ create table if not exists factos_events ( position integer primary key autoincrement, - id text not null, - stream text not null, - revision integer not null, + id text not null unique, type text not null, version integer not null, tags text not null, metadata text not null default '', - data blob not null, - unique(stream, revision) + data blob not null ); -create index if not exists factos_events_stream_revision - on factos_events(stream, revision); - -create index if not exists factos_events_position - on factos_events(position); - -create table if not exists factos_outbox ( - id integer primary key autoincrement, - source_position integer not null, - source_event_id text not null, - source_context text not null, - consumer text not null, - effect_key text not null, - target text not null, - type text not null, - metadata text not null default '', - payload blob not null, - status text not null default 'pending', - attempts integer not null default 0, - available_at_ms integer not null default 0, - locked_until_ms integer, - delivered_at_ms integer, - last_error text, - unique(consumer, effect_key) -); - -create index if not exists factos_outbox_pending - on factos_outbox(consumer, target, status, available_at_ms, locked_until_ms, id); +create index if not exists factos_events_type_position + on factos_events(type, position); diff --git a/backends/factos_sqlight/src/factos/factos_sqlight.gleam b/backends/factos_sqlight/src/factos/factos_sqlight.gleam index eb110a0..42a8775 100644 --- a/backends/factos_sqlight/src/factos/factos_sqlight.gleam +++ b/backends/factos_sqlight/src/factos/factos_sqlight.gleam @@ -1,649 +1,211 @@ //// SQLite backend for Factos using the `sqlight` package. //// //// This backend stores accepted facts in an append-only `factos_events` table. -//// The event history is the source of truth; projections and stream-shaped reads -//// are derived views over that history. +//// The globally ordered event history is the source of truth. Projection folds +//// and effect derivation remain ordinary application functions. //// -//// The context dispatch flow follows the Command Context Consistency idea from -//// "Simply Event Sourcing": a command selects the facts required for its -//// decision, folds them into temporary state, decides new facts, and appends -//// those facts only when no relevant facts appeared after the observed context -//// position. -//// -//// SQLite uses `BEGIN IMMEDIATE` around dispatch. That serializes writers for the -//// local database file, making stream revision and context append checks -//// transactionally stable without PostgreSQL's serializable retry machinery. +//// Dispatch uses `BEGIN IMMEDIATE`. SQLite acquires the writer lock before the +//// command context is read, so every accepted command observes and protects one +//// stable event-log state through append, strong callbacks, and commit. +import exception import factos import gleam/dynamic/decode -import gleam/int +import gleam/erlang/process import gleam/list import gleam/result import gleam/string import sqlight -pub type Proposed(event) { - /// A domain event prepared for SQLite persistence. - /// - /// The application codec creates this value. `id` should identify the event for - /// the application. `type_` and `tags` are store-visible query metadata. `data` - /// is opaque bytes owned by the application codec. - Proposed( - id: String, - event: event, - type_: factos.EventType, - version: Int, - tags: List(factos.Tag), - metadata: factos.Metadata, - data: BitArray, - ) -} - -pub type StoredEvent { - /// A raw event row read from SQLite before domain decoding. - /// - /// Decoders receive this value so they can inspect stored metadata and bytes. - /// `position` is the global append order. `revision` is the per-stream revision. - StoredEvent( - position: Int, - id: String, - stream: String, - revision: Int, - type_: factos.EventType, - version: Int, - tags: List(factos.Tag), - metadata: factos.Metadata, - data: BitArray, - ) -} - -pub type EventCodec(event) { - /// Application-owned SQLite event codec. - /// - /// `encode` converts a domain event into bytes and metadata. `decode` converts a - /// stored row back into a `factos.Decoded` domain event. Decode failures are - /// returned as `DecodeError` and stop load/read flows rather than panicking. - /// - /// WARNING: codecs used by dispatch must be pure. - /// - /// Dispatch may retry after a transient SQLite busy/locked response. That can - /// call `encode` and `decode` more than once for the same logical operation, so - /// codec functions must not perform IO, mutate external state, allocate ids - /// from an external system, publish messages, or otherwise create host-system - /// side effects. - EventCodec( - encode: fn(event) -> Proposed(event), - decode: fn(StoredEvent) -> Result(factos.Decoded(event), DecodeError), - ) -} - -pub type Append { - /// Result of a successful append. - /// - /// `current_revision` is the latest revision of the target stream after the - /// append. `position` is the global position of the last inserted event, or - /// `NoPosition` when no events were produced. - Append(current_revision: Int, position: factos.SequencePosition) -} - -pub type Dispatch(event) { - /// Result of a successful dispatch. - /// - /// `append` has the stream revision and final global position. `events` are the - /// committed events recorded by this dispatch, suitable for pure Factos - /// reactors or backend-specific durable effect adapters. - Dispatch(append: Append, events: List(factos.Recorded(event))) -} - -pub type ProposedEffect(effect) { - /// A durable integration effect prepared for outbox persistence. - /// - /// Applications own the effect payload and type names. Factos stores the - /// delivery envelope so workers can lease, retry, and acknowledge effects - /// without knowing the domain payload. - ProposedEffect( - effect: effect, - consumer: String, - key: String, - target: String, - type_: String, - metadata: factos.Metadata, - payload: BitArray, - ) -} - -pub type EffectCodec(effect) { - /// Application-owned outbox effect codec. - /// - /// WARNING: effect codecs used by dispatch must be pure. This function must - /// only build a deterministic durable outbox envelope; it must not execute the - /// effect or perform any other host-system side effect. - EffectCodec(encode: fn(effect) -> ProposedEffect(effect)) -} - -pub opaque type DispatchBuilder(command, state, event, domain_error, effect) { - DispatchBuilder( - connection: sqlight.Connection, - stream: String, - query: DispatchQuery, - decider: factos.Decider(command, state, event, domain_error), - event_codec: EventCodec(event), - effects: DispatchEffects(event, effect), - retry_attempts: Int, - ) -} - -type DispatchQuery { - StreamQuery - ContextQuery(factos.Query) -} - -type DispatchEffects(event, effect) { - NoEffects - ReactorEffects( - reactor: factos.Reactor(event, effect), - effect_codec: EffectCodec(effect), - ) -} - -pub type OutboxMessage { - /// A leased outbox message ready for delivery by an application worker. - OutboxMessage( - id: Int, - source_position: factos.SequencePosition, - source_event_id: String, - source_context: String, - consumer: String, - key: String, - target: String, - type_: String, - metadata: factos.Metadata, - payload: BitArray, - ) -} - -pub type Error(domain_error) { - /// The decider rejected the command with a domain error. - DomainError(domain_error) - - /// SQLite returned an error. - StoreError(sqlight.Error) - - /// A stream revision or context append condition failed. - AppendConditionFailed(factos.AppendCondition) - - /// The application codec could not decode a stored event. - DecodeError(DecodeError) -} - -pub type DecodeError { - UnknownEvent - InvalidData -} - -type QuerySql { - QuerySql(sql: String, arguments: List(sqlight.Value)) -} - -/// Start building an event dispatch. +/// Execute a shared dispatch builder against SQLite. /// -/// By default the builder uses one-stream consistency, no reactor effects, and 5 -/// attempts for transient SQLite busy/locked transaction starts. +/// Dispatch uses `BEGIN IMMEDIATE` to read the decision context, decide, append, +/// and run strong subscriptions in one transaction. The shared builder starts +/// with five attempts for retryable SQLite lock conflicts. /// -/// WARNING: every function passed into dispatch must be pure. -pub fn new_dispatch( - connection connection: sqlight.Connection, - stream stream_name: String, - decider decider: factos.Decider(command, state, event, domain_error), - codec codec: EventCodec(event), -) -> DispatchBuilder(command, state, event, domain_error, effect) { - DispatchBuilder( - connection: connection, - stream: stream_name, - query: StreamQuery, - decider: decider, - event_codec: codec, - effects: NoEffects, - retry_attempts: 5, - ) -} - -/// Use a context query instead of one-stream consistency. -pub fn with_query( - builder: DispatchBuilder(command, state, event, domain_error, effect), - query query: factos.Query, -) -> DispatchBuilder(command, state, event, domain_error, effect) { - let DispatchBuilder( - connection:, - stream:, - decider:, - event_codec:, - effects:, - retry_attempts:, - .., - ) = builder - DispatchBuilder( - connection:, - stream:, - query: ContextQuery(query), - decider:, - event_codec:, - effects:, - retry_attempts:, - ) -} - -/// Persist reactor effects into the outbox in the same transaction as events. -pub fn with_reactor( - builder: DispatchBuilder(command, state, event, domain_error, old_effect), - reactor reactor: factos.Reactor(event, effect), - codec effect_codec: EffectCodec(effect), -) -> DispatchBuilder(command, state, event, domain_error, effect) { - let DispatchBuilder( - connection:, - stream:, - query:, - decider:, - event_codec:, - retry_attempts:, - .., - ) = builder - DispatchBuilder( - connection:, - stream:, - query:, - decider:, - event_codec:, - effects: ReactorEffects(reactor:, effect_codec:), - retry_attempts:, - ) -} - -/// Override retry attempts for transient SQLite busy/locked transaction starts. -pub fn with_retry_attempts( - builder: DispatchBuilder(command, state, event, domain_error, effect), - attempts attempts: Int, -) -> DispatchBuilder(command, state, event, domain_error, effect) { - let DispatchBuilder( - connection:, - stream:, - query:, - decider:, - event_codec:, - effects:, - .., - ) = builder - DispatchBuilder( - connection:, - stream:, - query:, - decider:, - event_codec:, - effects:, - retry_attempts: int.max(attempts, 1), - ) -} - -/// Dispatch a command with a configured builder. +/// Decider and codec functions must be pure. Strong subscription callbacks can +/// run again if the transaction is retried after they return. +/// +/// After the final commit, every matching `factos.FireAndForget` subscription +/// starts an independent asynchronous process. Those processes are not +/// supervised and cannot change the committed dispatch result. pub fn dispatch( - builder: DispatchBuilder(command, state, event, domain_error, effect), + builder: factos.DispatchBuilder( + command, + state, + event, + BitArray, + domain_error, + subscription_error, + sqlight.Connection, + ), command: command, -) -> Result(Dispatch(event), Error(domain_error)) { - let DispatchBuilder( + event_id event_id: fn() -> String, +) -> Result( + factos.Dispatch(event), + factos.Error(domain_error, subscription_error, sqlight.Error), +) { + let factos.DispatchBuilder( connection:, - stream:, - query:, + decision_context:, decider:, - event_codec:, - effects:, + codec:, retry_attempts:, + subscriptions:, ) = builder - case query, effects { - StreamQuery, NoEffects -> - dispatch_stream( - connection, - stream:, - decider:, - codec: event_codec, - command:, - retry_attempts:, - ) - ContextQuery(query), NoEffects -> - dispatch_query( - connection, - stream:, - query:, - decider:, - codec: event_codec, - command:, - retry_attempts:, - ) - StreamQuery, ReactorEffects(reactor:, effect_codec:) -> - dispatch_stream_with_reactor( - connection, - stream:, - decider:, - event_codec:, - command:, - reactor:, - effect_codec:, - retry_attempts:, - ) - ContextQuery(query), ReactorEffects(reactor:, effect_codec:) -> - dispatch_query_with_reactor( + let result = + dispatch_context( + connection, + decision_context:, + decider:, + codec:, + command:, + event_id:, + retry_attempts:, + subscriptions:, + ) + + case result { + Error(error) -> Error(error) + Ok(dispatch) -> { + enqueue_fire_and_forget_subscriptions( connection, - stream:, - query:, - decider:, - event_codec:, - command:, - reactor:, - effect_codec:, - retry_attempts:, + subscriptions, + dispatch.events, ) + Ok(dispatch) + } } } -/// Create a new codec. -pub fn codec( - encode encode: fn(event) -> Proposed(event), - decode decode: fn(StoredEvent) -> Result(factos.Decoded(event), DecodeError), -) -> EventCodec(event) { - EventCodec(encode:, decode:) -} - -/// Create an effect codec. -pub fn effect_codec( - encode encode: fn(effect) -> ProposedEffect(effect), -) -> EffectCodec(effect) { - EffectCodec(encode:) +type QuerySql { + QuerySql(sql: String, arguments: List(sqlight.Value)) } -/// Create or update the SQLite schema required by this backend. -pub fn migrate(connection: sqlight.Connection) -> Result(Nil, Error(_)) { +/// Create the fresh SQLite schema required by this backend. +/// +/// Applications with existing databases should copy the statements into their +/// own immutable migration history rather than using this convenience at startup. +pub fn migrate( + connection: sqlight.Connection, +) -> Result(Nil, factos.Error(domain_error, subscription_error, sqlight.Error)) { sqlight.exec(migration_sql, on: connection) - |> result.map_error(StoreError) + |> result.map_error(factos.StoreError) } -const migration_sql = " -create table if not exists factos_events ( - position integer primary key autoincrement, - id text not null, - stream text not null, - revision integer not null, - type text not null, - version integer not null, - tags text not null, - metadata text not null default '', - data blob not null, - unique(stream, revision) -); -create index if not exists factos_events_stream_revision - on factos_events(stream, revision); -create index if not exists factos_events_position - on factos_events(position); -create table if not exists factos_outbox ( - id integer primary key autoincrement, - source_position integer not null, - source_event_id text not null, - source_context text not null, - consumer text not null, - effect_key text not null, - target text not null, - type text not null, - metadata text not null default '', - payload blob not null, - status text not null default 'pending', - attempts integer not null default 0, - available_at_ms integer not null default 0, - locked_until_ms integer, - delivered_at_ms integer, - last_error text, - unique(consumer, effect_key) -); -create index if not exists factos_outbox_pending - on factos_outbox(consumer, target, status, available_at_ms, locked_until_ms, id); -" +/// Render the SQLite store error carried by `factos.StoreError`. +pub fn sqlight_error_to_string(error: sqlight.Error) -> String { + let sqlight.SqlightError(message:, ..) = error + message +} -/// Read and fold the facts selected by a command-context query. -pub fn read_context( +@internal +pub fn read( connection: sqlight.Connection, - query query: factos.Query, + decision_context decision_context: factos.DecisionContext, decider decider: factos.Decider(command, state, event, domain_error), - codec codec: EventCodec(event), -) -> Result(factos.Context(event, state), Error(domain_error)) { - let factos.Decider(initial, _, evolve) = decider - - use events <- result.try(read_matching_events(connection, query, codec)) - let position = highest_recorded_position(events) + codec codec: factos.EventCodec(event, BitArray), +) -> Result( + factos.Context(event, state), + factos.Error(domain_error, subscription_error, sqlight.Error), +) { + let factos.Decider(initial:, evolve:, ..) = decider + + use events <- result.try(read_matching_events( + connection, + decision_context, + codec, + )) + let position = factos.highest_recorded_position(events) Ok(factos.Context( - query:, - state: factos.evolve_recorded( - initial: initial, - events: events, - evolve: evolve, + decision_context:, + state: factos.evolve_recorded(initial:, events:, evolve:), + events:, + position:, + append_condition: factos.FailIfEventsMatch( + decision_context:, + after: position, ), - events: events, - position: position, - append_condition: factos.FailIfEventsMatch(query, position), )) } -/// Read committed events after a global sequence position. +/// Read an ordered recovery window after a global position. /// -/// This is the bounded polling primitive used by durable subscriptions and -/// process managers. Events are returned in global append order. -pub fn read_events_after( +/// This is intentionally a low-level building block. A durable consumer owns its +/// checkpoint, retries, idempotency, and supervision outside this package. +@internal +pub fn read_after( connection: sqlight.Connection, - query query: factos.Query, + decision_context decision_context: factos.DecisionContext, after after: factos.SequencePosition, limit limit: Int, - codec codec: EventCodec(event), -) -> Result(List(factos.Recorded(event)), Error(domain_error)) { + codec codec: factos.EventCodec(event, BitArray), +) -> Result( + List(factos.Recorded(event)), + factos.Error(domain_error, subscription_error, sqlight.Error), +) { case limit <= 0 { True -> Ok([]) False -> { let QuerySql(where_sql, arguments) = - matching_events_after_sql(query, after) - sqlight.query( - "select position, id, stream, revision, type, version, tags, metadata, data - from factos_events - " - <> where_sql - <> " + matching_events_after_sql(decision_context, after) + + sqlight.query("select position, id, type, version, tags, metadata, data + from factos_events " <> where_sql <> " order by position - limit ?", - on: connection, - with: list.append(arguments, [sqlight.int(limit)]), - expecting: stored_event_decoder(), - ) - |> result.map_error(StoreError) - |> result.try(decode_rows(_, codec)) + limit ?", on: connection, with: list.append(arguments, [ + sqlight.int(limit), + ]), expecting: stored_event_decoder()) + |> result.map_error(factos.StoreError) + |> result.try(decode_stored_events(_, codec)) } } } -fn dispatch_query( +fn dispatch_context( connection: sqlight.Connection, - stream stream_name: String, - query query: factos.Query, + decision_context decision_context: factos.DecisionContext, decider decider: factos.Decider(command, state, event, domain_error), - codec codec: EventCodec(event), + codec codec: factos.EventCodec(event, BitArray), command command: command, + event_id event_id: fn() -> String, retry_attempts retry_attempts: Int, -) -> Result(Dispatch(event), Error(domain_error)) { + subscriptions subscriptions: List( + factos.Subscription(event, subscription_error, sqlight.Connection), + ), +) -> Result( + factos.Dispatch(event), + factos.Error(domain_error, subscription_error, sqlight.Error), +) { run_immediate_transaction( connection, retry_attempts, fn(transaction_connection) { - use context <- result.try(read_context( + use context <- result.try(read( transaction_connection, - query: query, - decider: decider, - codec: codec, - )) - use pair <- result.try( - factos.decide_context(context, command, decider) - |> result.map_error(DomainError), - ) - let #(context, events) = pair - - append_context_events( - transaction_connection, - stream_name, - events, + decision_context, + decider, codec, - context.append_condition, - ) - }, - ) -} - -fn dispatch_query_with_reactor( - connection: sqlight.Connection, - stream stream_name: String, - query query: factos.Query, - decider decider: factos.Decider(command, state, event, domain_error), - event_codec event_codec: EventCodec(event), - command command: command, - reactor reactor: factos.Reactor(event, effect), - effect_codec effect_codec: EffectCodec(effect), - retry_attempts retry_attempts: Int, -) -> Result(Dispatch(event), Error(domain_error)) { - run_immediate_transaction( - connection, - retry_attempts, - fn(transaction_connection) { - use context <- result.try(read_context( - transaction_connection, - query: query, - decider: decider, - codec: event_codec, )) use pair <- result.try( factos.decide_context(context, command, decider) - |> result.map_error(DomainError), + |> result.map_error(factos.DomainError), ) let #(context, events) = pair - use dispatch <- result.try(append_context_events( - transaction_connection, - stream_name, - events, - event_codec, - context.append_condition, - )) - use _ <- result.try(insert_outbox_effects( - transaction_connection, - dispatch.events, - reactor, - effect_codec, - )) - - Ok(dispatch) - }, - ) -} - -/// Load and fold one stream. -pub fn load_stream( - connection: sqlight.Connection, - stream stream_name: String, - decider decider: factos.Decider(command, state, event, domain_error), - codec codec: EventCodec(event), -) -> Result(factos.LoadedStream(event, state), Error(domain_error)) { - let factos.Decider(initial, _, evolve) = decider - use events <- result.try(read_stream_events(connection, stream_name, codec)) - - Ok(factos.LoadedStream( - stream: stream_name, - state: factos.evolve_recorded( - initial: initial, - events: events, - evolve: evolve, - ), - events: events, - revision: stream_revision(events), - )) -} - -fn dispatch_stream( - connection: sqlight.Connection, - stream stream_name: String, - decider decider: factos.Decider(command, state, event, domain_error), - codec codec: EventCodec(event), - command command: command, - retry_attempts retry_attempts: Int, -) -> Result(Dispatch(event), Error(domain_error)) { - run_immediate_transaction( - connection, - retry_attempts, - fn(transaction_connection) { - use loaded <- result.try(load_stream( + use dispatch <- result.try(append_events( transaction_connection, - stream: stream_name, - decider: decider, - codec: codec, - )) - let factos.Decider(_, decide, _) = decider - use events <- result.try( - decide(loaded.state, command) - |> result.map_error(DomainError), - ) - - append_stream_events( - transaction_connection, - stream_name, events, codec, - loaded.revision, - factos.NoAppendCondition, - ) - }, - ) -} - -fn dispatch_stream_with_reactor( - connection: sqlight.Connection, - stream stream_name: String, - decider decider: factos.Decider(command, state, event, domain_error), - event_codec event_codec: EventCodec(event), - command command: command, - reactor reactor: factos.Reactor(event, effect), - effect_codec effect_codec: EffectCodec(effect), - retry_attempts retry_attempts: Int, -) -> Result(Dispatch(event), Error(domain_error)) { - run_immediate_transaction( - connection, - retry_attempts, - fn(transaction_connection) { - use loaded <- result.try(load_stream( - transaction_connection, - stream: stream_name, - decider: decider, - codec: event_codec, - )) - let factos.Decider(_, decide, _) = decider - use events <- result.try( - decide(loaded.state, command) - |> result.map_error(DomainError), - ) - use dispatch <- result.try(append_stream_events( - transaction_connection, - stream_name, - events, - event_codec, - loaded.revision, - factos.NoAppendCondition, + event_id, + context.append_condition, )) - use _ <- result.try(insert_outbox_effects( + use _ <- result.try(run_strong_subscriptions( transaction_connection, + subscriptions, dispatch.events, - reactor, - effect_codec, )) - Ok(dispatch) }, ) @@ -652,8 +214,15 @@ fn dispatch_stream_with_reactor( fn run_immediate_transaction( connection: sqlight.Connection, retry_attempts: Int, - work: fn(sqlight.Connection) -> Result(Dispatch(event), Error(domain_error)), -) -> Result(Dispatch(event), Error(domain_error)) { + work: fn(sqlight.Connection) -> + Result( + factos.Dispatch(event), + factos.Error(domain_error, subscription_error, sqlight.Error), + ), +) -> Result( + factos.Dispatch(event), + factos.Error(domain_error, subscription_error, sqlight.Error), +) { run_immediate_transaction_attempt( connection, work, @@ -663,11 +232,19 @@ fn run_immediate_transaction( fn run_immediate_transaction_attempt( connection: sqlight.Connection, - work: fn(sqlight.Connection) -> Result(Dispatch(event), Error(domain_error)), + work: fn(sqlight.Connection) -> + Result( + factos.Dispatch(event), + factos.Error(domain_error, subscription_error, sqlight.Error), + ), attempts_remaining attempts_remaining: Int, -) -> Result(Dispatch(event), Error(domain_error)) { - let result = run_immediate_transaction_once(connection, work) - case result { +) -> Result( + factos.Dispatch(event), + factos.Error(domain_error, subscription_error, sqlight.Error), +) { + let transaction_result = run_immediate_transaction_once(connection, work) + + case transaction_result { Ok(dispatch) -> Ok(dispatch) Error(error) -> case attempts_remaining > 1 && retryable_transaction_error(error) { @@ -684,26 +261,41 @@ fn run_immediate_transaction_attempt( fn run_immediate_transaction_once( connection: sqlight.Connection, - work: fn(sqlight.Connection) -> Result(Dispatch(event), Error(domain_error)), -) -> Result(Dispatch(event), Error(domain_error)) { + work: fn(sqlight.Connection) -> + Result( + factos.Dispatch(event), + factos.Error(domain_error, subscription_error, sqlight.Error), + ), +) -> Result( + factos.Dispatch(event), + factos.Error(domain_error, subscription_error, sqlight.Error), +) { use _ <- result.try( sqlight.exec("begin immediate", on: connection) - |> result.map_error(StoreError), + |> result.map_error(factos.StoreError), ) - - let result = work(connection) - finish_transaction(connection, result) + work(connection) + |> finish_transaction(connection) } fn finish_transaction( + transaction_result: Result( + factos.Dispatch(event), + factos.Error(domain_error, subscription_error, sqlight.Error), + ), connection: sqlight.Connection, - result: Result(Dispatch(event), Error(domain_error)), -) -> Result(Dispatch(event), Error(domain_error)) { - case result { +) -> Result( + factos.Dispatch(event), + factos.Error(domain_error, subscription_error, sqlight.Error), +) { + case transaction_result { Ok(dispatch) -> case sqlight.exec("commit", on: connection) { Ok(Nil) -> Ok(dispatch) - Error(error) -> Error(StoreError(error)) + Error(error) -> { + let _ = sqlight.exec("rollback", on: connection) + Error(factos.StoreError(error)) + } } Error(error) -> { let _ = sqlight.exec("rollback", on: connection) @@ -712,371 +304,325 @@ fn finish_transaction( } } -fn retryable_transaction_error(error: Error(_)) -> Bool { +fn retryable_transaction_error( + error: factos.Error(domain_error, subscription_error, sqlight.Error), +) -> Bool { case error { - StoreError(sqlight.SqlightError(code: sqlight.Busy, ..)) -> True - StoreError(sqlight.SqlightError(code: sqlight.BusyRecovery, ..)) -> True - StoreError(sqlight.SqlightError(code: sqlight.BusySnapshot, ..)) -> True - StoreError(sqlight.SqlightError(code: sqlight.BusyTimeout, ..)) -> True - StoreError(sqlight.SqlightError(code: sqlight.Locked, ..)) -> True + factos.StoreError(sqlight.SqlightError(code:, ..)) -> + retryable_sqlite_code(code) + factos.DomainError(_) -> False + factos.SubscriptionError(error: _) -> False + factos.AppendConditionFailed(_) -> False + factos.DecodeError(_) -> False + } +} + +fn retryable_sqlite_code(code: sqlight.ErrorCode) -> Bool { + case code { + sqlight.Busy -> True + sqlight.BusyRecovery -> True + sqlight.BusySnapshot -> True + sqlight.BusyTimeout -> True + sqlight.Locked -> True _ -> False } } -fn append_context_events( +fn run_strong_subscriptions( connection: sqlight.Connection, - stream_name: String, - events: List(event), - codec: EventCodec(event), - condition: factos.AppendCondition, -) -> Result(Dispatch(event), Error(domain_error)) { - use revision <- result.try( - current_revision(connection, stream_name) - |> result.map_error(StoreError), - ) - append_stream_events( - connection, - stream_name, - events, - codec, - factos.CurrentRevision(revision), - condition, - ) + subscriptions: List( + factos.Subscription(event, subscription_error, sqlight.Connection), + ), + events: List(factos.Recorded(event)), +) -> Result(Nil, factos.Error(domain_error, subscription_error, sqlight.Error)) { + case subscriptions { + [] -> Ok(Nil) + [factos.Subscription(decision_context:, consistency:, handle:), ..remaining] -> + case consistency { + factos.FireAndForget -> + run_strong_subscriptions(connection, remaining, events) + factos.StrongConsistency -> { + use _ <- result.try( + run_strong_subscription_events( + connection, + decision_context, + handle, + events, + ) + |> result.map_error(factos.SubscriptionError), + ) + run_strong_subscriptions(connection, remaining, events) + } + } + } } -fn append_stream_events( +fn run_strong_subscription_events( connection: sqlight.Connection, - stream_name: String, - events: List(event), - codec: EventCodec(event), - expected: factos.Revision, - condition: factos.AppendCondition, -) -> Result(Dispatch(event), Error(domain_error)) { + decision_context: factos.DecisionContext, + handle: fn(sqlight.Connection, factos.Recorded(event)) -> + Result(Nil, subscription_error), + events: List(factos.Recorded(event)), +) -> Result(Nil, subscription_error) { case events { - [] -> { - let append = - Append( - current_revision: revision_to_int(expected), - position: factos.NoPosition, - ) - Ok(Dispatch(append:, events: [])) - } - [_, ..] -> { - use current <- result.try( - current_revision(connection, stream_name) - |> result.map_error(StoreError), - ) - case expected_matches(expected, current) { - False -> Error(AppendConditionFailed(factos.NoAppendCondition)) - True -> - case has_matching_events_after_condition(connection, condition) { - Error(error) -> Error(StoreError(error)) - Ok(True) -> Error(AppendConditionFailed(condition)) - Ok(False) -> - insert_events( - connection, - stream_name, - events, - codec, - current + 1, - factos.NoPosition, - [], - ) + [] -> Ok(Nil) + [recorded, ..remaining] -> + case factos.matches_decision_context(recorded, decision_context) { + True -> { + use _ <- result.try(handle(connection, recorded)) + run_strong_subscription_events( + connection, + decision_context, + handle, + remaining, + ) + } + False -> + run_strong_subscription_events( + connection, + decision_context, + handle, + remaining, + ) + } + } +} + +fn enqueue_fire_and_forget_subscriptions( + connection: sqlight.Connection, + subscriptions: List( + factos.Subscription(event, subscription_error, sqlight.Connection), + ), + events: List(factos.Recorded(event)), +) -> Nil { + use subscription <- list.each(subscriptions) + case subscription.consistency { + factos.StrongConsistency -> Nil + factos.FireAndForget -> { + let events = + list.filter(events, factos.matches_decision_context( + _, + subscription.decision_context, + )) + + case events { + [] -> Nil + events -> { + let _ = { + use <- exception.rescue + use <- process.spawn() + run_fire_and_forget_events(connection, events, subscription.handle) } + Nil + } } } } } -fn has_matching_events_after_condition( +fn run_fire_and_forget_events( + connection: sqlight.Connection, + events: List(factos.Recorded(event)), + handle: fn(sqlight.Connection, factos.Recorded(event)) -> + Result(Nil, subscription_error), +) -> Nil { + case events { + [] -> Nil + [recorded, ..remaining] -> { + let _ = handle(connection, recorded) + run_fire_and_forget_events(connection, remaining, handle) + } + } +} + +fn append_events( connection: sqlight.Connection, + events: List(event), + codec: factos.EventCodec(event, BitArray), + event_id: fn() -> String, condition: factos.AppendCondition, -) -> Result(Bool, sqlight.Error) { - case condition { - factos.NoAppendCondition -> Ok(False) - factos.FailIfEventsMatch(query, after) -> - has_matching_events_after(connection, query, after) +) -> Result( + factos.Dispatch(event), + factos.Error(domain_error, subscription_error, sqlight.Error), +) { + case events { + [] -> Ok(factos.Dispatch(position: factos.NoPosition, events: [])) + [_, ..] -> + case has_matching_events_after_condition(connection, condition) { + Error(error) -> Error(factos.StoreError(error)) + Ok(True) -> Error(factos.AppendConditionFailed(condition)) + Ok(False) -> + insert_events( + connection, + events, + codec, + event_id, + factos.NoPosition, + [], + ) + } } } fn insert_events( connection: sqlight.Connection, - stream_name: String, events: List(event), - codec: EventCodec(event), - revision: Int, + codec: factos.EventCodec(event, BitArray), + event_id: fn() -> String, position: factos.SequencePosition, recorded_events: List(factos.Recorded(event)), -) -> Result(Dispatch(event), Error(domain_error)) { +) -> Result( + factos.Dispatch(event), + factos.Error(domain_error, subscription_error, sqlight.Error), +) { case events { - [] -> { - let append = Append(current_revision: revision - 1, position: position) - Ok(Dispatch(append:, events: list.reverse(recorded_events))) - } - [event, ..rest] -> { - let EventCodec(encode:, ..) = codec - let Proposed(id:, type_:, version:, tags:, metadata:, data:, ..) = - encode(event) + [] -> Ok(factos.Dispatch(position:, events: list.reverse(recorded_events))) + [event, ..remaining] -> { + let factos.Event( + payload:, + descriptor: factos.EventDescriptor(type_:, version:, tags:, metadata:), + ) = codec.encode(event) + let id = event_id() use positions <- result.try( sqlight.query( - " - insert into factos_events (id, stream, revision, type, version, tags, metadata, data) - values (?, ?, ?, ?, ?, ?, ?, ?) - returning position - ", + "insert into factos_events (id, type, version, tags, metadata, data) + values (?, ?, ?, ?, ?, ?) + returning position", on: connection, with: [ sqlight.text(id), - sqlight.text(stream_name), - sqlight.int(revision), sqlight.text(factos.event_type_name(type_)), sqlight.int(version), sqlight.text(tags_to_text(tags)), sqlight.text(metadata_to_text(metadata)), - sqlight.blob(data), + sqlight.blob(payload), ], expecting: int_field_decoder(), ) - |> result.map_error(map_event_insert_error), + |> result.map_error(factos.StoreError), ) - let position = case positions { - [position, ..] -> factos.SequencePosition(position) - [] -> position - } - let recorded = - factos.Recorded( - id:, - stream: stream_name, - revision:, - position:, - event:, - descriptor: factos.EventDescriptor(type_:, version:, tags:, metadata:), - ) - insert_events( - connection, - stream_name, - rest, - codec, - revision + 1, - position, - [recorded, ..recorded_events], - ) - } - } -} - -fn map_event_insert_error(error: sqlight.Error) -> Error(domain_error) { - case error { - sqlight.SqlightError(code: sqlight.Constraint, ..) -> - AppendConditionFailed(factos.NoAppendCondition) - sqlight.SqlightError(code: sqlight.ConstraintUnique, ..) -> - AppendConditionFailed(factos.NoAppendCondition) - _ -> StoreError(error) - } -} -fn insert_outbox_effects( - connection: sqlight.Connection, - events: List(factos.Recorded(event)), - reactor: factos.Reactor(event, effect), - effect_codec: EffectCodec(effect), -) -> Result(Nil, Error(domain_error)) { - case events { - [] -> Ok(Nil) - [event, ..rest] -> { - use _ <- result.try(insert_outbox_effects_for_event( - connection, - event, - factos.react(reactor, event), - effect_codec, - )) - insert_outbox_effects(connection, rest, reactor, effect_codec) - } - } -} - -fn insert_outbox_effects_for_event( - connection: sqlight.Connection, - event: factos.Recorded(event), - effects: List(effect), - effect_codec: EffectCodec(effect), -) -> Result(Nil, Error(domain_error)) { - case effects { - [] -> Ok(Nil) - [effect, ..rest] -> { - use _ <- result.try(insert_outbox_effect( - connection, - event, - effect, - effect_codec, - )) - insert_outbox_effects_for_event(connection, event, rest, effect_codec) + case positions { + [] -> Error(factos.DecodeError(factos.InvalidData)) + [returned_position, ..] -> { + let position = factos.SequencePosition(returned_position) + let recorded = + factos.Recorded( + id:, + position:, + event:, + descriptor: factos.EventDescriptor( + type_:, + version:, + tags:, + metadata:, + ), + ) + insert_events(connection, remaining, codec, event_id, position, [ + recorded, + ..recorded_events + ]) + } + } } } } -fn insert_outbox_effect( +fn has_matching_events_after_condition( connection: sqlight.Connection, - event: factos.Recorded(event), - effect: effect, - effect_codec: EffectCodec(effect), -) -> Result(Nil, Error(domain_error)) { - let EffectCodec(encode) = effect_codec - let ProposedEffect(_, consumer, key, target, type_, metadata, payload) = - encode(effect) - let source_position = case event.position { - factos.NoPosition -> -1 - factos.SequencePosition(position) -> position - } - - sqlight.query( - " - insert into factos_outbox ( - source_position, - source_event_id, - source_context, - consumer, - effect_key, - target, - type, - metadata, - payload - ) - values (?, ?, ?, ?, ?, ?, ?, ?, ?) - on conflict (consumer, effect_key) do nothing - returning id - ", - on: connection, - with: [ - sqlight.int(source_position), - sqlight.text(event.id), - sqlight.text(event.stream), - sqlight.text(consumer), - sqlight.text(key), - sqlight.text(target), - sqlight.text(type_), - sqlight.text(metadata_to_text(metadata)), - sqlight.blob(payload), - ], - expecting: int_field_decoder(), - ) - |> result.map(fn(_) { Nil }) - |> result.map_error(StoreError) + condition: factos.AppendCondition, +) -> Result(Bool, sqlight.Error) { + let factos.FailIfEventsMatch(decision_context:, after:) = condition + has_matching_events_after(connection, decision_context, after) } fn read_matching_events( connection: sqlight.Connection, - query: factos.Query, - codec: EventCodec(event), -) -> Result(List(factos.Recorded(event)), Error(domain_error)) { - let QuerySql(where_sql, arguments) = query_to_sql(query) - sqlight.query( - "select position, id, stream, revision, type, version, tags, metadata, data - from factos_events - " <> where_sql <> " - order by position", - on: connection, - with: arguments, - expecting: stored_event_decoder(), - ) - |> result.map_error(StoreError) - |> result.try(decode_rows(_, codec)) -} - -fn read_stream_events( - connection: sqlight.Connection, - stream_name: String, - codec: EventCodec(event), -) -> Result(List(factos.Recorded(event)), Error(domain_error)) { - sqlight.query( - "select position, id, stream, revision, type, version, tags, metadata, data from factos_events where stream = ? order by revision", - on: connection, - with: [sqlight.text(stream_name)], - expecting: stored_event_decoder(), - ) - |> result.map_error(StoreError) - |> result.try(decode_rows(_, codec)) -} - -fn decode_rows( - rows: List(StoredEvent), - codec: EventCodec(event), -) -> Result(List(factos.Recorded(event)), Error(domain_error)) { + decision_context: factos.DecisionContext, + codec: factos.EventCodec(event, BitArray), +) -> Result( + List(factos.Recorded(event)), + factos.Error(domain_error, subscription_error, sqlight.Error), +) { + let QuerySql(where_sql, arguments) = query_to_sql(decision_context) + sqlight.query("select position, id, type, version, tags, metadata, data + from factos_events " <> where_sql <> " + order by position", on: connection, with: arguments, expecting: stored_event_decoder()) + |> result.map_error(factos.StoreError) + |> result.try(decode_stored_events(_, codec)) +} + +fn decode_stored_events( + rows: List(factos.Recorded(BitArray)), + codec: factos.EventCodec(event, BitArray), +) -> Result( + List(factos.Recorded(event)), + factos.Error(domain_error, subscription_error, sqlight.Error), +) { + decode_stored_events_loop(rows, codec, []) +} + +fn decode_stored_events_loop( + rows: List(factos.Recorded(BitArray)), + codec: factos.EventCodec(event, BitArray), + decoded: List(factos.Recorded(event)), +) -> Result( + List(factos.Recorded(event)), + factos.Error(domain_error, subscription_error, sqlight.Error), +) { case rows { - [] -> Ok([]) - [row, ..rest] -> { - use recorded <- result.try(decode_row(row, codec)) - use rest <- result.try(decode_rows(rest, codec)) - Ok([recorded, ..rest]) + [] -> Ok(list.reverse(decoded)) + [row, ..remaining] -> { + use recorded <- result.try(decode_stored_event(row, codec)) + decode_stored_events_loop(remaining, codec, [recorded, ..decoded]) } } } -fn decode_row( - row: StoredEvent, - codec: EventCodec(event), -) -> Result(factos.Recorded(event), Error(domain_error)) { - let EventCodec(decode: decode_event, ..) = codec - use decoded <- result.try(decode_event(row) |> result.map_error(DecodeError)) - let factos.Decoded(event:, descriptor:) = decoded - let StoredEvent(position:, id:, stream:, revision:, ..) = row +fn decode_stored_event( + stored: factos.Recorded(BitArray), + codec: factos.EventCodec(event, BitArray), +) -> Result( + factos.Recorded(event), + factos.Error(domain_error, subscription_error, sqlight.Error), +) { + let factos.Recorded(position:, id:, descriptor:, ..) = stored + let factos.EventCodec(decode:, ..) = codec + use event <- result.try( + decode(stored) |> result.map_error(factos.DecodeError), + ) - Ok(factos.Recorded( - id:, - stream:, - revision:, - position: factos.SequencePosition(position), - event:, - descriptor:, - )) + Ok(factos.Recorded(id:, position:, event:, descriptor:)) } -fn stored_event_decoder() -> decode.Decoder(StoredEvent) { +fn stored_event_decoder() -> decode.Decoder(factos.Recorded(BitArray)) { use position <- decode.field(0, decode.int) use id <- decode.field(1, decode.string) - use stream <- decode.field(2, decode.string) - use revision <- decode.field(3, decode.int) - use type_name <- decode.field(4, decode.string) - use version <- decode.field(5, decode.int) - use tags <- decode.field(6, decode.string) - use metadata <- decode.field(7, decode.string) - use data <- decode.field(8, decode.bit_array) - decode.success(StoredEvent( - position: position, - id: id, - stream: stream, - revision: revision, - type_: factos.event_type(type_name), - version: version, - tags: tags_from_text(tags), - metadata: metadata_from_text(metadata), - data: data, - )) -} - -fn current_revision( - connection: sqlight.Connection, - stream_name: String, -) -> Result(Int, sqlight.Error) { - use rows <- result.map(sqlight.query( - "select coalesce(max(revision), -1) from factos_events where stream = ?", - on: connection, - with: [sqlight.text(stream_name)], - expecting: int_field_decoder(), + use type_name <- decode.field(2, decode.string) + use version <- decode.field(3, decode.int) + use tags <- decode.field(4, decode.string) + use metadata <- decode.field(5, decode.string) + use event <- decode.field(6, decode.bit_array) + decode.success(factos.Recorded( + position: factos.SequencePosition(position), + id:, + descriptor: factos.EventDescriptor( + type_: factos.event_type(type_name), + version:, + tags: tags_from_text(tags), + metadata: metadata_from_text(metadata), + ), + event:, )) - - case rows { - [revision, ..] -> revision - [] -> -1 - } } fn has_matching_events_after( connection: sqlight.Connection, - query: factos.Query, + decision_context: factos.DecisionContext, after: factos.SequencePosition, ) -> Result(Bool, sqlight.Error) { - let QuerySql(where_sql, arguments) = matching_events_after_sql(query, after) + let QuerySql(where_sql, arguments) = + matching_events_after_sql(decision_context, after) sqlight.query( "select 1 from factos_events " <> where_sql <> " limit 1", on: connection, @@ -1096,55 +642,23 @@ fn int_field_decoder() -> decode.Decoder(Int) { decode.success(value) } -fn stream_revision(events: List(factos.Recorded(event))) -> factos.Revision { - case list.reverse(events) { - [] -> factos.NoEvents - [event, ..] -> factos.CurrentRevision(event.revision) - } -} - -fn highest_recorded_position( - events: List(factos.Recorded(event)), -) -> factos.SequencePosition { - case list.reverse(events) { - [] -> factos.NoPosition - [event, ..] -> event.position - } -} - -fn expected_matches(expected: factos.Revision, current: Int) -> Bool { - case expected { - factos.NoEvents -> current == -1 - factos.CurrentRevision(revision) -> current == revision - } -} - -fn revision_to_int(revision: factos.Revision) -> Int { - case revision { - factos.NoEvents -> -1 - factos.CurrentRevision(revision) -> revision - } -} - -fn query_to_sql(query: factos.Query) -> QuerySql { - case query { +fn query_to_sql(decision_context: factos.DecisionContext) -> QuerySql { + case decision_context { factos.AllEvents -> QuerySql(sql: "", arguments: []) - factos.Query(items) -> - case items { - [] -> QuerySql(sql: "where 1 = 0", arguments: []) - [_, ..] -> { - let #(sql, arguments) = build_query_items_sql(items, [], []) - QuerySql( - sql: "where " <> string.join(list.reverse(sql), with: " or "), - arguments: list.reverse(arguments), - ) - } - } + factos.Matching(items: []) | factos.NoContext -> + QuerySql(sql: "where 1 = 0", arguments: []) + factos.Matching(items: [_, ..] as items) -> { + let #(sql, arguments) = build_query_items_sql(items, [], []) + QuerySql( + sql: "where " <> string.join(list.reverse(sql), with: " or "), + arguments: list.reverse(arguments), + ) + } } } fn matching_events_after_sql( - query: factos.Query, + decision_context: factos.DecisionContext, after: factos.SequencePosition, ) -> QuerySql { let after_position = case after { @@ -1152,41 +666,37 @@ fn matching_events_after_sql( factos.SequencePosition(position) -> position } - case query { + case decision_context { factos.AllEvents -> QuerySql(sql: "where position > ?", arguments: [ sqlight.int(after_position), ]) - factos.Query(items) -> - case items { - [] -> QuerySql(sql: "where 1 = 0", arguments: []) - [_, ..] -> { - let #(sql, arguments) = - build_query_items_sql(items, [], [ - sqlight.int(after_position), - ]) - QuerySql( - sql: "where position > ? and (" - <> string.join(list.reverse(sql), with: " or ") - <> ")", - arguments: list.reverse(arguments), - ) - } - } + factos.Matching(items: []) | factos.NoContext -> + QuerySql(sql: "where 1 = 0", arguments: []) + factos.Matching(items: [_, ..] as items) -> { + let #(sql, arguments) = + build_query_items_sql(items, [], [sqlight.int(after_position)]) + QuerySql( + sql: "where position > ? and (" + <> string.join(list.reverse(sql), with: " or ") + <> ")", + arguments: list.reverse(arguments), + ) + } } } fn build_query_items_sql( - items: List(factos.QueryItem), + items: List(factos.Item), sql: List(String), arguments: List(sqlight.Value), ) -> #(List(String), List(sqlight.Value)) { case items { [] -> #(sql, arguments) - [item, ..rest] -> { + [item, ..remaining] -> { let QuerySql(item_sql, item_arguments) = query_item_to_sql(item) build_query_items_sql( - rest, + remaining, [item_sql, ..sql], list.append(list.reverse(item_arguments), arguments), ) @@ -1194,8 +704,8 @@ fn build_query_items_sql( } } -fn query_item_to_sql(item: factos.QueryItem) -> QuerySql { - let factos.QueryItem(types, tags) = item +fn query_item_to_sql(item: factos.Item) -> QuerySql { + let factos.Item(types:, tags:) = item let QuerySql(type_sql, type_arguments) = types_to_sql(types) let QuerySql(tag_sql, tag_arguments) = tags_to_sql(tags) @@ -1238,128 +748,16 @@ fn placeholders(count: Int) -> String { |> string.join(with: ", ") } -fn placeholder_list(remaining: Int, acc: List(String)) -> List(String) { +fn placeholder_list( + remaining: Int, + placeholders: List(String), +) -> List(String) { case remaining <= 0 { - True -> acc - False -> placeholder_list(remaining - 1, ["?", ..acc]) + True -> placeholders + False -> placeholder_list(remaining - 1, ["?", ..placeholders]) } } -const sqlite_now_milliseconds = "cast((julianday('now') - 2440587.5) * 86400000 as integer)" - -/// Lease pending outbox messages for a consumer and target. -/// -/// Leased messages stay in `pending` status but are hidden from competing -/// workers until `locked_until_ms` expires. -pub fn lease_outbox( - connection: sqlight.Connection, - consumer consumer: String, - target target: String, - limit limit: Int, - lease_for_milliseconds lease_for_milliseconds: Int, -) -> Result(List(OutboxMessage), Error(_)) { - case limit <= 0 { - True -> Ok([]) - False -> - sqlight.query(" - update factos_outbox - set - locked_until_ms = " <> sqlite_now_milliseconds <> " + ?, - attempts = attempts + 1 - where id in ( - select id - from factos_outbox - where consumer = ? - and target = ? - and status = 'pending' - and available_at_ms <= " <> sqlite_now_milliseconds <> " - and (locked_until_ms is null or locked_until_ms <= " <> sqlite_now_milliseconds <> ") - order by id - limit ? - ) - returning - id, - source_position, - source_event_id, - source_context, - consumer, - effect_key, - target, - type, - metadata, - payload - ", on: connection, with: [ - sqlight.int(lease_for_milliseconds), - sqlight.text(consumer), - sqlight.text(target), - sqlight.int(limit), - ], expecting: outbox_message_decoder()) - |> result.map_error(StoreError) - } -} - -/// Mark an outbox message as delivered. -pub fn ack_outbox( - connection: sqlight.Connection, - id id: Int, -) -> Result(Nil, Error(_)) { - sqlight.exec(" - update factos_outbox - set status = 'delivered', - delivered_at_ms = " <> sqlite_now_milliseconds <> ", - locked_until_ms = null - where id = " <> int.to_string(id), on: connection) - |> result.map_error(StoreError) -} - -/// Release an outbox message for retry after a delay. -pub fn nack_outbox( - connection: sqlight.Connection, - id id: Int, - error error: String, - retry_after_milliseconds retry_after_milliseconds: Int, -) -> Result(Nil, Error(_)) { - sqlight.query(" - update factos_outbox - set locked_until_ms = null, - last_error = ?, - available_at_ms = " <> sqlite_now_milliseconds <> " + ? - where id = ? - returning id - ", on: connection, with: [ - sqlight.text(error), - sqlight.int(retry_after_milliseconds), - sqlight.int(id), - ], expecting: int_field_decoder()) - |> result.map(fn(_) { Nil }) - |> result.map_error(StoreError) -} - -fn outbox_message_decoder() -> decode.Decoder(OutboxMessage) { - use id <- decode.field(0, decode.int) - use source_position <- decode.field(1, decode.int) - use source_event_id <- decode.field(2, decode.string) - use source_context <- decode.field(3, decode.string) - use consumer <- decode.field(4, decode.string) - use key <- decode.field(5, decode.string) - use target <- decode.field(6, decode.string) - use type_ <- decode.field(7, decode.string) - use metadata <- decode.field(8, decode.string) - use payload <- decode.field(9, decode.bit_array) - decode.success(OutboxMessage( - id: id, - source_position: factos.SequencePosition(source_position), - source_event_id: source_event_id, - source_context: source_context, - consumer: consumer, - key: key, - target: target, - type_: type_, - metadata: metadata_from_text(metadata), - payload: payload, - )) -} - fn tags_to_text(tags: List(factos.Tag)) -> String { case tags { [] -> "" @@ -1404,22 +802,18 @@ fn metadata_from_text(metadata: String) -> factos.Metadata { } } -pub fn error_to_string( - error: Error(domain_error), - domain_error_to_string: fn(domain_error) -> String, -) -> String { - case error { - DomainError(error) -> domain_error_to_string(error) - StoreError(sqlight.SqlightError(code:, message:, offset: _)) -> - "sqlite error " - <> int.to_string(sqlight.error_code_to_int(code)) - <> ": " - <> message - AppendConditionFailed(factos.NoAppendCondition) -> - "append to event failed: No append condition" - AppendConditionFailed(factos.FailIfEventsMatch(query: _, after: _)) -> - "append to event failed: Events matched" - DecodeError(UnknownEvent) -> "unknown event decoded" - DecodeError(InvalidData) -> "invalid data stored in database" - } -} +/// Render a backend error with application formatters for its generic errors. +const migration_sql = " +create table if not exists factos_events ( + position integer primary key autoincrement, + id text not null unique, + type text not null, + version integer not null, + tags text not null, + metadata text not null default '', + data blob not null +); + +create index if not exists factos_events_type_position + on factos_events(type, position); +" diff --git a/backends/factos_sqlight/test/factos_sqlight_test.gleam b/backends/factos_sqlight/test/factos_sqlight_test.gleam index aad27c5..031e863 100644 --- a/backends/factos_sqlight/test/factos_sqlight_test.gleam +++ b/backends/factos_sqlight/test/factos_sqlight_test.gleam @@ -1,10 +1,12 @@ import factos import factos/factos_sqlight import gleam/bit_array +import gleam/dynamic/decode import gleam/erlang/application -import gleam/int +import gleam/erlang/process import gleam/list import gleam/result +import gleam/string import gleeunit import simplifile import sqlight @@ -15,6 +17,8 @@ pub fn main() -> Nil { type Command { RegisterUser(username: String) + RegisterPair(first: String, second: String) + DoNothing } type Event { @@ -22,809 +26,656 @@ type Event { } type State { - Available - Taken + State(usernames: List(String)) } type DomainError { - AlreadyTaken + AlreadyTaken(username: String) } -type Effect { - WelcomeEmail(username: String) -} - -type CounterCommand { - Increment +type FireMessage { + FireStarted( + pid: process.Pid, + event: factos.Recorded(Event), + committed_events: Int, + release: process.Subject(Nil), + ) } -type CounterEvent { - Incremented(value: Int) -} +pub fn bootstrap_and_dbmate_migrations_create_streamless_schema_test() -> Nil { + use bootstrap_connection <- sqlight.with_connection(":memory:") + let assert Ok(Nil) = factos_sqlight.migrate(bootstrap_connection) + assert_streamless_schema(bootstrap_connection) -type CounterState { - CounterState(total: Int) + use dbmate_connection <- sqlight.with_connection(":memory:") + execute_dbmate_event_store_migration(dbmate_connection) + assert_streamless_schema(dbmate_connection) } -pub fn dispatch_stream_persists_events_test() { +pub fn dispatch_uses_decision_context_and_application_event_ids_test() -> Nil { use connection <- sqlight.with_connection(":memory:") execute_migration_file(connection) - let assert Ok(dispatch) = - factos_sqlight.new_dispatch( - connection: connection, - stream: "user-renata", - decider: decider(), - codec: codec(), - ) - |> factos_sqlight.dispatch(RegisterUser("renata")) + let assert Ok(renata_dispatch) = + dispatch_user(connection, "renata", event_id: "event-renata") + let assert [renata] = renata_dispatch.events + assert renata_dispatch.position == renata.position + assert renata.id == "event-renata" + assert renata.event == UserRegistered(username: "renata") + assert renata.descriptor.tags == [factos.tag("username:renata")] + + let duplicate = dispatch_user(connection, "renata", event_id: "unused") + assert duplicate + == Error(factos.DomainError(AlreadyTaken(username: "renata"))) + + let assert Ok(maria_dispatch) = + dispatch_user(connection, "maria", event_id: "event-maria") + let assert [maria] = maria_dispatch.events + assert maria.position != renata.position + assert count_events(connection) == 2 - let assert factos_sqlight.Append( - current_revision: 0, - position: factos.SequencePosition(_), - ) = dispatch.append - let assert [recorded] = dispatch.events - assert_user_recorded( - recorded, - stream: "user-renata", - revision: 0, - position: dispatch.append.position, - username: "renata", - ) - let reactor = factos.reactor(react: fn(recorded) { [recorded.event] }) - assert factos.react_all(reactor: reactor, events: dispatch.events) - == [ - UserRegistered("renata"), - ] - - let assert Ok(loaded) = - factos_sqlight.load_stream( + let assert Ok(context) = + factos_sqlight.read( connection, - stream: "user-renata", + decision_context: username_context("renata"), decider: decider(), codec: codec(), ) - - assert loaded.state == Taken - assert loaded.revision == factos.CurrentRevision(0) -} - -pub fn dispatch_stream_handles_many_events_test() { - use connection <- sqlight.with_connection(":memory:") - execute_migration_file(connection) - - let assert Ok(dispatch) = dispatch_counter_stream_many(connection, 250) - let assert factos_sqlight.Append( - current_revision: 249, - position: factos.SequencePosition(_), - ) = dispatch.append - let assert [recorded] = dispatch.events - assert_counter_recorded( - recorded, - stream: "counter-load", - revision: 249, - position: dispatch.append.position, - value: 250, - ) - let reactor = factos.reactor(react: fn(recorded) { [recorded.event] }) - assert factos.react_all(reactor: reactor, events: dispatch.events) - == [ - Incremented(250), - ] - - let assert Ok(loaded) = - factos_sqlight.load_stream( - connection, - stream: "counter-load", - decider: counter_decider(), - codec: counter_codec(), + assert context.state == State(usernames: ["renata"]) + assert context.events == [renata] + assert context.position == renata.position + assert context.append_condition + == factos.FailIfEventsMatch( + decision_context: username_context("renata"), + after: renata.position, ) - - assert loaded.state == CounterState(250) - assert loaded.revision == factos.CurrentRevision(249) - assert list.length(loaded.events) == 250 } -pub fn dispatch_context_handles_many_streams_test() { +pub fn duplicate_event_id_rolls_back_dispatch_test() -> Nil { use connection <- sqlight.with_connection(":memory:") execute_migration_file(connection) - let query = counter_query() - - let assert Ok(dispatch) = - dispatch_counter_context_many(connection, query, 100) - let assert factos_sqlight.Append( - current_revision: 0, - position: factos.SequencePosition(_), - ) = dispatch.append - let assert [recorded] = dispatch.events - assert_counter_recorded( - recorded, - stream: "counter-context-1", - revision: 0, - position: dispatch.append.position, - value: 100, - ) - let reactor = factos.reactor(react: fn(recorded) { [recorded.event] }) - assert factos.react_all(reactor: reactor, events: dispatch.events) - == [ - Incremented(100), - ] - - let assert Ok(context) = - factos_sqlight.read_context( - connection, - query: query, - decider: counter_decider(), - codec: counter_codec(), - ) - - assert context.state == CounterState(100) - assert list.length(context.events) == 100 - assert context.position != factos.NoPosition + let assert Ok(_) = dispatch_user(connection, "renata", event_id: "same-id") + let result = dispatch_user(connection, "maria", event_id: "same-id") + let assert Error(factos.StoreError(sqlight.SqlightError( + code: sqlight.ConstraintUnique, + .., + ))) = result + assert count_events(connection) == 1 } -pub fn read_events_after_filters_unknown_events_before_decoding_test() { +pub fn read_after_filters_orders_and_rejects_unknown_events_test() -> Nil { use connection <- sqlight.with_connection(":memory:") execute_migration_file(connection) - let assert Ok(_) = dispatch_unknown_counter_event(connection) - let assert Ok(visible_dispatch) = - dispatch_counter_stream(connection, "visible") + let assert Ok(first_dispatch) = + dispatch_user(connection, "renata", event_id: "first") + let assert [first] = first_dispatch.events + let assert Ok(second_dispatch) = + dispatch_user(connection, "maria", event_id: "second") + let assert [second] = second_dispatch.events - let assert Error(factos_sqlight.DecodeError(factos_sqlight.UnknownEvent)) = - factos_sqlight.read_events_after( + let assert Ok([renata]) = + factos_sqlight.read_after( connection, - query: factos.AllEvents, + decision_context: username_context("renata"), after: factos.NoPosition, limit: 10, - codec: counter_codec(), + codec: codec(), ) + assert renata == first - let assert Ok([recorded]) = - factos_sqlight.read_events_after( + let assert Ok([maria]) = + factos_sqlight.read_after( connection, - query: counter_query(), - after: factos.NoPosition, - limit: 10, - codec: counter_codec(), + decision_context: factos.AllEvents, + after: first.position, + limit: 1, + codec: codec(), ) - assert_counter_recorded( - recorded, - stream: "visible", - revision: 0, - position: visible_dispatch.append.position, - value: 1, - ) -} - -pub fn read_events_after_returns_bounded_windows_after_position_test() { - use connection <- sqlight.with_connection(":memory:") - execute_migration_file(connection) - - let assert Ok(_) = dispatch_counter_stream_many(connection, 3) + assert maria == second - let assert Ok(first_window) = - factos_sqlight.read_events_after( + let assert Ok([]) = + factos_sqlight.read_after( connection, - query: counter_query(), + decision_context: factos.AllEvents, after: factos.NoPosition, - limit: 2, - codec: counter_codec(), + limit: 0, + codec: codec(), ) - let assert [first, second] = first_window - assert_counter_recorded( - first, - stream: "counter-load", - revision: 0, - position: first.position, - value: 1, - ) - assert_counter_recorded( - second, - stream: "counter-load", - revision: 1, - position: second.position, - value: 2, - ) - let assert Ok(second_window) = - factos_sqlight.read_events_after( + let assert Ok(_) = + factos.new_dispatch( + connection:, + decision_context: factos.NoContext, + decider: accepting_decider(), + codec: hidden_codec(), + ) + |> factos_sqlight.dispatch(RegisterUser(username: "hidden"), event_id: fn() { + "hidden" + }) + + let assert Error(factos.DecodeError(factos.UnknownEvent)) = + factos_sqlight.read_after( connection, - query: counter_query(), + decision_context: factos.AllEvents, after: second.position, - limit: 2, - codec: counter_codec(), + limit: 10, + codec: codec(), ) - let assert [third] = second_window - assert_counter_recorded( - third, - stream: "counter-load", - revision: 2, - position: third.position, - value: 3, - ) + Nil } -pub fn reactor_outbox_leases_nacks_and_acks_effects_test() { +pub fn empty_dispatch_has_no_position_and_skips_subscriptions_test() -> Nil { use connection <- sqlight.with_connection(":memory:") execute_migration_file(connection) + let invocations = process.new_subject() + let event_id_invocations = process.new_subject() + let subscriptions = [ + factos.new_subscription( + decision_context: factos.AllEvents, + consistency: factos.StrongConsistency, + handle: fn(_connection, _recorded) { + process.send(invocations, "strong") + Ok(Nil) + }, + ), + ] let assert Ok(dispatch) = - factos_sqlight.new_dispatch( - connection: connection, - stream: "user-renata", - decider: decider(), + factos.new_dispatch( + connection:, + decision_context: factos.NoContext, + decider: empty_decider(), codec: codec(), ) - |> factos_sqlight.with_reactor( - reactor: welcome_reactor(), - codec: effect_codec(), - ) - |> factos_sqlight.dispatch(RegisterUser("renata")) - let assert [event] = dispatch.events + |> factos.with_subscriptions(subscriptions:) + |> factos_sqlight.dispatch(DoNothing, event_id: fn() { + process.send(event_id_invocations, Nil) + "unused" + }) + + assert dispatch == factos.Dispatch(position: factos.NoPosition, events: []) + let assert Error(Nil) = process.receive(invocations, within: 100) + let assert Error(Nil) = process.receive(event_id_invocations, within: 100) + Nil +} - let assert Ok([message]) = - factos_sqlight.lease_outbox( - connection, - consumer: "mailer", - target: "email", - limit: 1, - lease_for_milliseconds: 60_000, +pub fn strong_subscription_commits_with_dispatch_test() -> Nil { + use connection <- sqlight.with_connection(":memory:") + execute_migration_file(connection) + create_projection_table(connection) + let subscription = + factos.new_subscription( + decision_context: factos.AllEvents, + consistency: factos.StrongConsistency, + handle: insert_projection, ) - assert message.source_position == event.position - assert message.source_event_id == event.id - assert message.source_context == "user-renata" - assert message.consumer == "mailer" - assert message.key == "welcome:renata" - assert message.target == "email" - assert message.type_ == "WelcomeEmail" - assert message.metadata == factos.metadata([#("kind", "welcome")]) - let assert Ok("welcome:renata") = bit_array.to_string(message.payload) - let assert Ok([]) = - factos_sqlight.lease_outbox( + let assert Ok(dispatch) = + dispatch_user_with_subscriptions( connection, - consumer: "mailer", - target: "email", - limit: 1, - lease_for_milliseconds: 60_000, + "renata", + event_id: "strong-event", + subscriptions: [subscription], ) + let assert [recorded] = dispatch.events + assert projection_rows(connection) == [#("strong-event", "renata")] + assert count_events(connection) == 1 + assert recorded.id == "strong-event" +} - let assert Ok(Nil) = - factos_sqlight.nack_outbox( - connection, - id: message.id, - error: "temporary smtp failure", - retry_after_milliseconds: 0, +pub fn strong_subscription_failure_rolls_back_everything_test() -> Nil { + use connection <- sqlight.with_connection(":memory:") + execute_migration_file(connection) + create_projection_table(connection) + let fire_deliveries = process.new_subject() + let insert = + factos.new_subscription( + decision_context: factos.AllEvents, + consistency: factos.StrongConsistency, + handle: insert_projection, ) - - let assert Ok([retried]) = - factos_sqlight.lease_outbox( - connection, - consumer: "mailer", - target: "email", - limit: 1, - lease_for_milliseconds: 60_000, + let fail_after_observing_insert = + factos.new_subscription( + decision_context: factos.AllEvents, + consistency: factos.StrongConsistency, + handle: fn(transaction_connection, recorded) { + case projection_rows(transaction_connection) { + [#(id, _)] if id == recorded.id -> Error("expected strong failure") + _ -> Error("earlier strong callback was not visible") + } + }, + ) + let fire = + factos.new_subscription( + decision_context: factos.AllEvents, + consistency: factos.FireAndForget, + handle: fn(_connection, _recorded) { + process.send(fire_deliveries, Nil) + Ok(Nil) + }, ) - assert retried.id == message.id - assert retried.key == message.key - let assert Ok(Nil) = factos_sqlight.ack_outbox(connection, id: retried.id) - let assert Ok([]) = - factos_sqlight.lease_outbox( + let result = + dispatch_user_with_subscriptions( connection, - consumer: "mailer", - target: "email", - limit: 1, - lease_for_milliseconds: 60_000, + "renata", + event_id: "rolled-back", + subscriptions: [insert, fail_after_observing_insert, fire], + ) + let assert Error(error) = result + assert error == factos.SubscriptionError(error: "expected strong failure") + assert factos.error_to_string( + error, + fn(_) { "domain" }, + fn(subscription_error) { subscription_error }, + fn(_) { "sqlight" }, ) + == "subscription error: expected strong failure" + assert count_events(connection) == 0 + assert projection_rows(connection) == [] + let assert Error(Nil) = process.receive(fire_deliveries, within: 100) + Nil } -pub fn context_semantics_conformance_test() -> Nil { +pub fn subscriptions_filter_nonmatching_dispatch_events_test() -> Nil { use connection <- sqlight.with_connection(":memory:") execute_migration_file(connection) + let invocations = process.new_subject() + let subscriptions = [ + factos.new_subscription( + decision_context: username_context("maria"), + consistency: factos.StrongConsistency, + handle: fn(_connection, _recorded) { + process.send(invocations, "strong") + Ok(Nil) + }, + ), + factos.new_subscription( + decision_context: username_context("maria"), + consistency: factos.FireAndForget, + handle: fn(_connection, _recorded) { + process.send(invocations, "fire") + Ok(Nil) + }, + ), + ] - let no_matches = empty_query() - let assert Ok(renata_dispatch) = - factos_sqlight.new_dispatch( - connection:, - stream: "conformance-renata", - decider: accepting_decider(), - codec: codec(), - ) - |> factos_sqlight.with_query(query: no_matches) - |> factos_sqlight.dispatch(RegisterUser(username: "renata")) - let assert Ok(lucy_dispatch) = - factos_sqlight.new_dispatch( - connection:, - stream: "conformance-lucy", - decider: accepting_decider(), - codec: codec(), - ) - |> factos_sqlight.with_query(query: no_matches) - |> factos_sqlight.dispatch(RegisterUser(username: "lucy")) - let assert Ok(marc_dispatch) = - factos_sqlight.new_dispatch( - connection:, - stream: "conformance-marc", - decider: accepting_decider(), - codec: codec(), - ) - |> factos_sqlight.with_query(query: factos.AllEvents) - |> factos_sqlight.dispatch(RegisterUser(username: "marc")) - let renata_position = renata_dispatch.append.position - let lucy_position = lucy_dispatch.append.position - let marc_position = marc_dispatch.append.position - - let assert Ok(empty_context) = - factos_sqlight.read_context( + let assert Ok(_) = + dispatch_user_with_subscriptions( connection, - query: no_matches, - decider: accepting_decider(), - codec: codec(), + "renata", + event_id: "filtered", + subscriptions:, ) - assert empty_context.state == Available - assert empty_context.events == [] - assert empty_context.position == factos.NoPosition - assert empty_context.append_condition - == factos.FailIfEventsMatch(query: no_matches, after: factos.NoPosition) - - let compound_query = username_conformance_query() - let assert Ok(compound_context) = - factos_sqlight.read_context( - connection, - query: compound_query, - decider: accepting_decider(), - codec: codec(), + let assert Error(Nil) = process.receive(invocations, within: 150) + assert count_events(connection) == 1 +} + +pub fn fire_and_forget_runs_after_commit_without_blocking_test() -> Nil { + use connection <- sqlight.with_connection(":memory:") + execute_migration_file(connection) + let deliveries = process.new_subject() + let subscription = + factos.new_subscription( + decision_context: factos.AllEvents, + consistency: factos.FireAndForget, + handle: fn(callback_connection, recorded) { + let release = process.new_subject() + process.send( + deliveries, + FireStarted( + pid: process.self(), + event: recorded, + committed_events: count_events(callback_connection), + release:, + ), + ) + case process.receive(release, within: 5000) { + Ok(Nil) -> Error("ignored fire error") + Error(Nil) -> Error("release timed out") + } + }, ) - let assert [lucy] = compound_context.events - assert_user_recorded( - lucy, - stream: "conformance-lucy", - revision: 0, - position: lucy_position, - username: "lucy", - ) - assert compound_context.state == Taken - assert compound_context.position == lucy_position - assert compound_context.append_condition - == factos.FailIfEventsMatch(query: compound_query, after: lucy_position) - let assert Ok(all_context) = - factos_sqlight.read_context( + let assert Ok(dispatch) = + dispatch_user_with_subscriptions( connection, - query: factos.AllEvents, - decider: accepting_decider(), - codec: codec(), + "renata", + event_id: "fire-event", + subscriptions: [subscription], ) - let assert [renata, lucy, marc] = all_context.events - assert_user_recorded( - renata, - stream: "conformance-renata", - revision: 0, - position: renata_position, - username: "renata", - ) - assert_user_recorded( - lucy, - stream: "conformance-lucy", - revision: 0, - position: lucy_position, - username: "lucy", - ) - assert_user_recorded( - marc, - stream: "conformance-marc", - revision: 0, - position: marc_position, - username: "marc", - ) - assert all_context.state == Taken - assert all_context.position == marc_position - assert all_context.append_condition - == factos.FailIfEventsMatch(query: factos.AllEvents, after: marc_position) -} - -pub fn empty_context_dispatch_is_a_no_op_test() -> Nil { + let FireStarted(pid:, event:, committed_events:, release:) = + receive_fire_message(deliveries) + assert dispatch.events == [event] + assert committed_events == 1 + assert process.is_alive(pid) + let monitor = process.monitor(pid) + process.send(release, Nil) + wait_for_monitor(monitor) + assert count_events(connection) == 1 +} + +pub fn fire_and_forget_continues_after_callback_error_in_append_order_test() -> Nil { use connection <- sqlight.with_connection(":memory:") execute_migration_file(connection) - - let assert Ok(seeded_dispatch) = - factos_sqlight.new_dispatch( - connection:, - stream: "conformance-no-op", - decider: decider(), - codec: codec(), + let deliveries = process.new_subject() + let ids = process.new_subject() + process.send(ids, "pair-first") + process.send(ids, "pair-second") + let subscription = + factos.new_subscription( + decision_context: factos.AllEvents, + consistency: factos.FireAndForget, + handle: fn(_connection, recorded) { + let UserRegistered(username:) = recorded.event + process.send(deliveries, username) + Error("ignored") + }, ) - |> factos_sqlight.dispatch(RegisterUser(username: "no-op")) - let assert [seeded] = seeded_dispatch.events - let assert Ok(no_op_dispatch) = - factos_sqlight.new_dispatch( + let assert Ok(dispatch) = + factos.new_dispatch( connection:, - stream: "conformance-no-op", - decider: empty_decider(), - codec: codec(), - ) - |> factos_sqlight.with_query(query: factos.AllEvents) - |> factos_sqlight.dispatch(RegisterUser(username: "ignored")) - assert no_op_dispatch.events == [] - assert no_op_dispatch.append.current_revision == 0 - assert no_op_dispatch.append.position == factos.NoPosition - - let assert Ok(loaded) = - factos_sqlight.load_stream( - connection, - stream: "conformance-no-op", - decider: decider(), + decision_context: factos.NoContext, + decider: accepting_decider(), codec: codec(), ) - assert loaded.events == [seeded] - assert loaded.state == Taken - assert loaded.revision == factos.CurrentRevision(0) - - let assert Ok(context) = - factos_sqlight.read_context( - connection, - query: factos.AllEvents, - decider: decider(), - codec: codec(), + |> factos.with_subscriptions(subscriptions: [subscription]) + |> factos_sqlight.dispatch( + RegisterPair(first: "renata", second: "maria"), + event_id: fn() { receive_event_id(ids) }, ) - assert context.events == [seeded] - assert context.state == Taken - assert context.position == seeded.position -} - -fn execute_migration_file(connection: sqlight.Connection) -> Nil { - let assert Ok(priv_directory) = application.priv_directory("factos_sqlight") - let assert Ok(sql) = simplifile.read(priv_directory <> "/migrations.sql") - let assert Ok(Nil) = sqlight.exec(sql, on: connection) + assert list.map(dispatch.events, fn(recorded) { recorded.id }) + == ["pair-first", "pair-second"] + let assert Ok("renata") = process.receive(deliveries, within: 5000) + let assert Ok("maria") = process.receive(deliveries, within: 5000) Nil } fn decider() -> factos.Decider(Command, State, Event, DomainError) { - factos.decider(initial: Available, decide:, evolve:) -} - -fn decide(state: State, command: Command) -> Result(List(Event), DomainError) { - case state, command { - Available, RegisterUser(username) -> Ok([UserRegistered(username)]) - Taken, RegisterUser(_) -> Error(AlreadyTaken) - } -} - -fn evolve(_state: State, _event: Event) -> State { - Taken + factos.decider( + initial: State(usernames: []), + decide: fn(state, command) { + let State(usernames:) = state + case command { + RegisterUser(username:) -> + case list.contains(usernames, username) { + True -> Error(AlreadyTaken(username:)) + False -> Ok([UserRegistered(username:)]) + } + RegisterPair(first:, second:) -> + Ok([ + UserRegistered(username: first), + UserRegistered(username: second), + ]) + DoNothing -> Ok([]) + } + }, + evolve: fn(state, event) { + let State(usernames:) = state + let UserRegistered(username:) = event + State(usernames: list.append(usernames, [username])) + }, + ) } fn accepting_decider() -> factos.Decider(Command, State, Event, DomainError) { factos.decider( - initial: Available, + initial: State(usernames: []), decide: fn(_state, command) { case command { RegisterUser(username:) -> Ok([UserRegistered(username:)]) + RegisterPair(first:, second:) -> + Ok([ + UserRegistered(username: first), + UserRegistered(username: second), + ]) + DoNothing -> Ok([]) } }, - evolve: evolve, + evolve: fn(state, _event) { state }, ) } fn empty_decider() -> factos.Decider(Command, State, Event, DomainError) { factos.decider( - initial: Available, + initial: State(usernames: []), decide: fn(_state, _command) { Ok([]) }, - evolve: evolve, + evolve: fn(state, _event) { state }, ) } -fn empty_query() -> factos.Query { - factos.Query(items: []) -} - -fn username_conformance_query() -> factos.Query { - factos.query([ - factos.query_item(types: [factos.event_type("UserRegistered")], tags: [ - factos.tag("username:renata"), - factos.tag("username:lucy"), - ]), - factos.query_item( - types: [ - factos.event_type("UnknownEventType"), - factos.event_type("UserRegistered"), - ], - tags: [factos.tag("username:lucy")], - ), - ]) +fn codec() -> factos.EventCodec(Event, BitArray) { + factos.codec(encode: encode_event, decode: decode_event) } -fn codec() -> factos_sqlight.EventCodec(Event) { - factos_sqlight.codec(encode:, decode:) +fn hidden_codec() -> factos.EventCodec(Event, BitArray) { + factos.codec( + encode: fn(event) { + let UserRegistered(username:) = event + factos.new_event( + type_: factos.event_type("HiddenEvent"), + version: 1, + data: bit_array.from_string(username), + ) + }, + decode: decode_event, + ) } -fn encode(event: Event) -> factos_sqlight.Proposed(Event) { - factos_sqlight.Proposed( - id: "event-" <> event.username, - event: event, +fn encode_event(event: Event) -> factos.Event(BitArray) { + let UserRegistered(username:) = event + factos.new_event( type_: factos.event_type("UserRegistered"), version: 1, - tags: [factos.tag("username:" <> event.username)], - metadata: factos.empty_metadata(), - data: bit_array.from_string(event.username), + data: bit_array.from_string(username), ) + |> factos.with_tags(tags: [factos.tag("username:" <> username)]) } -fn decode( - stored: factos_sqlight.StoredEvent, -) -> Result(factos.Decoded(Event), factos_sqlight.DecodeError) { - case factos.event_type_name(stored.type_) { - "UserRegistered" -> { +fn decode_event( + stored: factos.Recorded(BitArray), +) -> Result(Event, factos.DecodeError) { + let descriptor = stored.descriptor + case factos.event_type_name(descriptor.type_), descriptor.version { + "UserRegistered", 1 -> { use username <- result.try( - bit_array.to_string(stored.data) - |> result.replace_error(factos_sqlight.InvalidData), + bit_array.to_string(stored.event) + |> result.replace_error(factos.InvalidData), ) - Ok(factos.Decoded( - event: UserRegistered(username), - descriptor: factos.EventDescriptor( - type_: stored.type_, - version: stored.version, - tags: stored.tags, - metadata: stored.metadata, - ), - )) + Ok(UserRegistered(username:)) } - _ -> Error(factos_sqlight.UnknownEvent) + _, _ -> Error(factos.UnknownEvent) } } -fn welcome_reactor() -> factos.Reactor(Event, Effect) { - factos.reactor(react: fn(recorded) { - case recorded.event { - UserRegistered(username) -> [WelcomeEmail(username)] - } - }) -} - -fn effect_codec() -> factos_sqlight.EffectCodec(Effect) { - factos_sqlight.effect_codec(encode: encode_effect) -} - -fn encode_effect(effect: Effect) -> factos_sqlight.ProposedEffect(Effect) { - case effect { - WelcomeEmail(username) -> - factos_sqlight.ProposedEffect( - effect: effect, - consumer: "mailer", - key: "welcome:" <> username, - target: "email", - type_: "WelcomeEmail", - metadata: factos.metadata([#("kind", "welcome")]), - payload: bit_array.from_string("welcome:" <> username), - ) - } +fn username_context(username: String) -> factos.DecisionContext { + factos.Matching(items: [ + factos.item(types: [factos.event_type("UserRegistered")], tags: [ + factos.tag("username:" <> username), + ]), + ]) } -fn assert_user_recorded( - recorded: factos.Recorded(Event), - stream stream_name: String, - revision revision: Int, - position position: factos.SequencePosition, - username username: String, -) -> Nil { - assert recorded.id == "event-" <> username - assert recorded.stream == stream_name - assert recorded.revision == revision - assert recorded.position == position - assert recorded.descriptor.type_ == factos.event_type("UserRegistered") - assert recorded.descriptor.version == 1 - assert recorded.descriptor.tags == [factos.tag("username:" <> username)] - assert recorded.descriptor.metadata == factos.empty_metadata() - assert recorded.event == UserRegistered(username) -} - -fn dispatch_counter_stream( +fn dispatch_user( connection: sqlight.Connection, - stream_name: String, -) -> Result(factos_sqlight.Dispatch(CounterEvent), factos_sqlight.Error(Nil)) { - factos_sqlight.new_dispatch( - connection: connection, - stream: stream_name, - decider: counter_decider(), - codec: counter_codec(), + username: String, + event_id event_id: String, +) -> Result( + factos.Dispatch(Event), + factos.Error(DomainError, Nil, sqlight.Error), +) { + factos.new_dispatch( + connection:, + decision_context: username_context(username), + decider: decider(), + codec: codec(), ) - |> factos_sqlight.dispatch(Increment) + |> factos_sqlight.dispatch(RegisterUser(username:), event_id: fn() { + event_id + }) } -fn dispatch_unknown_counter_event( +fn dispatch_user_with_subscriptions( connection: sqlight.Connection, -) -> Result(factos_sqlight.Dispatch(CounterEvent), factos_sqlight.Error(Nil)) { - factos_sqlight.new_dispatch( - connection: connection, - stream: "hidden", - decider: counter_decider(), - codec: unknown_counter_codec(), + username: String, + event_id event_id: String, + subscriptions subscriptions: List( + factos.Subscription(Event, String, sqlight.Connection), + ), +) -> Result( + factos.Dispatch(Event), + factos.Error(DomainError, String, sqlight.Error), +) { + factos.new_dispatch( + connection:, + decision_context: username_context(username), + decider: decider(), + codec: codec(), ) - |> factos_sqlight.dispatch(Increment) + |> factos.with_subscriptions(subscriptions:) + |> factos_sqlight.dispatch(RegisterUser(username:), event_id: fn() { + event_id + }) } -fn dispatch_counter_stream_many( +fn insert_projection( connection: sqlight.Connection, - remaining: Int, -) -> Result(factos_sqlight.Dispatch(CounterEvent), factos_sqlight.Error(Nil)) { - case remaining { - 0 -> dispatch_counter_stream(connection, "counter-load") - _ -> { - let result = dispatch_counter_stream(connection, "counter-load") - case remaining, result { - 1, _ -> result - _, Ok(_) -> dispatch_counter_stream_many(connection, remaining - 1) - _, Error(error) -> Error(error) - } - } - } + recorded: factos.Recorded(Event), +) -> Result(Nil, String) { + let UserRegistered(username:) = recorded.event + sqlight.query( + "insert into test_projection (event_id, username) + values (?, ?) + returning event_id", + on: connection, + with: [sqlight.text(recorded.id), sqlight.text(username)], + expecting: string_field_decoder(), + ) + |> result.map(fn(_rows) { Nil }) + |> result.map_error(fn(_error) { "projection insert failed" }) } -fn dispatch_counter_context_many( - connection: sqlight.Connection, - query: factos.Query, - remaining: Int, -) -> Result(factos_sqlight.Dispatch(CounterEvent), factos_sqlight.Error(Nil)) { - case remaining { - 0 -> dispatch_counter_context(connection, "counter-context-0", query) - _ -> { - let stream_name = "counter-context-" <> int.to_string(remaining) - let result = dispatch_counter_context(connection, stream_name, query) - case remaining, result { - 1, _ -> result - _, Ok(_) -> - dispatch_counter_context_many(connection, query, remaining - 1) - _, Error(error) -> Error(error) - } - } - } +fn create_projection_table(connection: sqlight.Connection) -> Nil { + let assert Ok(Nil) = + sqlight.exec( + "create table test_projection ( + event_id text primary key, + username text not null + )", + on: connection, + ) + Nil } -fn dispatch_counter_context( - connection: sqlight.Connection, - stream_name: String, - query: factos.Query, -) -> Result(factos_sqlight.Dispatch(CounterEvent), factos_sqlight.Error(Nil)) { - factos_sqlight.new_dispatch( - connection: connection, - stream: stream_name, - decider: counter_decider(), - codec: counter_codec(), - ) - |> factos_sqlight.with_query(query: query) - |> factos_sqlight.dispatch(Increment) +fn projection_rows(connection: sqlight.Connection) -> List(#(String, String)) { + let assert Ok(rows) = + sqlight.query( + "select event_id, username from test_projection order by event_id", + on: connection, + with: [], + expecting: string_pair_decoder(), + ) + rows } -fn counter_decider() -> factos.Decider( - CounterCommand, - CounterState, - CounterEvent, - Nil, -) { - factos.decider( - initial: CounterState(0), - decide: counter_decide, - evolve: counter_evolve, - ) +fn count_events(connection: sqlight.Connection) -> Int { + let assert Ok([count]) = + sqlight.query( + "select count(*) from factos_events", + on: connection, + with: [], + expecting: int_field_decoder(), + ) + count } -fn counter_decide( - state: CounterState, - command: CounterCommand, -) -> Result(List(CounterEvent), Nil) { - let CounterState(total) = state - case command { - Increment -> Ok([Incremented(total + 1)]) - } +fn receive_event_id(ids: process.Subject(String)) -> String { + let assert Ok(id) = process.receive(ids, within: 1000) + id } -fn counter_evolve(state: CounterState, event: CounterEvent) -> CounterState { - let CounterState(total) = state - case event { - Incremented(_) -> CounterState(total + 1) - } +fn receive_fire_message( + deliveries: process.Subject(FireMessage), +) -> FireMessage { + let assert Ok(message) = process.receive(deliveries, within: 5000) + message } -fn counter_codec() -> factos_sqlight.EventCodec(CounterEvent) { - factos_sqlight.codec( - encode: encode_counter_event, - decode: decode_counter_event, - ) -} - -fn unknown_counter_codec() -> factos_sqlight.EventCodec(CounterEvent) { - factos_sqlight.codec(encode: encode_unknown_counter_event, decode: fn(_) { - Error(factos_sqlight.UnknownEvent) - }) +fn wait_for_monitor(monitor: process.Monitor) -> Nil { + let assert Ok(Nil) = + process.new_selector() + |> process.select_specific_monitor(monitor, fn(_) { Nil }) + |> process.selector_receive(5000) + process.demonitor_process(monitor) +} + +fn assert_streamless_schema(connection: sqlight.Connection) -> Nil { + let assert Ok(columns) = + sqlight.query( + "select name from pragma_table_info('factos_events') order by cid", + on: connection, + with: [], + expecting: string_field_decoder(), + ) + assert columns + == ["position", "id", "type", "version", "tags", "metadata", "data"] + + let assert Ok(tables) = + sqlight.query( + "select name from sqlite_master + where type = 'table' and name like 'factos_%' + order by name", + on: connection, + with: [], + expecting: string_field_decoder(), + ) + assert tables == ["factos_events"] + + let assert Ok(indices) = + sqlight.query( + "select name from sqlite_master + where type = 'index' and tbl_name = 'factos_events' + order by name", + on: connection, + with: [], + expecting: string_field_decoder(), + ) + assert list.contains(indices, "factos_events_type_position") } -fn encode_counter_event( - event: CounterEvent, -) -> factos_sqlight.Proposed(CounterEvent) { - case event { - Incremented(value) -> - factos_sqlight.Proposed( - id: "counter-event-" <> int.to_string(value), - event: event, - type_: factos.event_type("Incremented"), - version: 1, - tags: [factos.tag("counter:load")], - metadata: factos.empty_metadata(), - data: bit_array.from_string(int.to_string(value)), - ) - } +fn execute_migration_file(connection: sqlight.Connection) -> Nil { + let assert Ok(priv_directory) = application.priv_directory("factos_sqlight") + let assert Ok(sql) = simplifile.read(priv_directory <> "/migrations.sql") + let assert Ok(Nil) = sqlight.exec(sql, on: connection) + Nil } -fn encode_unknown_counter_event( - event: CounterEvent, -) -> factos_sqlight.Proposed(CounterEvent) { - case event { - Incremented(value) -> - factos_sqlight.Proposed( - id: "unknown-counter-event-" <> int.to_string(value), - event: event, - type_: factos.event_type("HiddenIncremented"), - version: 1, - tags: [factos.tag("counter:hidden")], - metadata: factos.empty_metadata(), - data: bit_array.from_string(int.to_string(value)), - ) - } +fn execute_dbmate_event_store_migration(connection: sqlight.Connection) -> Nil { + let assert Ok(priv_directory) = application.priv_directory("factos_sqlight") + let assert Ok(sql) = + simplifile.read( + priv_directory <> "/dbmate/20260703000100_factos_sqlight_event_store.sql", + ) + let assert [up, ..] = string.split(sql, on: "-- migrate:down") + let assert Ok(Nil) = sqlight.exec(up, on: connection) + Nil } -fn decode_counter_event( - stored: factos_sqlight.StoredEvent, -) -> Result(factos.Decoded(CounterEvent), factos_sqlight.DecodeError) { - case factos.event_type_name(stored.type_) { - "Incremented" -> { - use text <- result.try( - bit_array.to_string(stored.data) - |> result.replace_error(factos_sqlight.InvalidData), - ) - use value <- result.try( - int.parse(text) - |> result.replace_error(factos_sqlight.InvalidData), - ) - Ok(factos.Decoded( - event: Incremented(value), - descriptor: factos.EventDescriptor( - type_: stored.type_, - version: stored.version, - tags: stored.tags, - metadata: stored.metadata, - ), - )) - } - _ -> Error(factos_sqlight.UnknownEvent) - } +fn int_field_decoder() -> decode.Decoder(Int) { + use value <- decode.field(0, decode.int) + decode.success(value) } -fn counter_query() -> factos.Query { - factos.query([ - factos.query_item(types: [factos.event_type("Incremented")], tags: [ - factos.tag("counter:load"), - ]), - ]) +fn string_field_decoder() -> decode.Decoder(String) { + use value <- decode.field(0, decode.string) + decode.success(value) } -fn assert_counter_recorded( - recorded: factos.Recorded(CounterEvent), - stream stream_name: String, - revision revision: Int, - position position: factos.SequencePosition, - value value: Int, -) -> Nil { - assert recorded.id == "counter-event-" <> int.to_string(value) - assert recorded.stream == stream_name - assert recorded.revision == revision - assert recorded.position == position - assert recorded.descriptor.type_ == factos.event_type("Incremented") - assert recorded.descriptor.version == 1 - assert recorded.descriptor.tags == [factos.tag("counter:load")] - assert recorded.descriptor.metadata == factos.empty_metadata() - assert recorded.event == Incremented(value) +fn string_pair_decoder() -> decode.Decoder(#(String, String)) { + use first <- decode.field(0, decode.string) + use second <- decode.field(1, decode.string) + decode.success(#(first, second)) } diff --git a/docs/core-model.md b/docs/core-model.md index 2012a72..f5f0d29 100644 --- a/docs/core-model.md +++ b/docs/core-model.md @@ -46,41 +46,41 @@ It has three parts: 2. `evolve`: how an accepted fact changes decision state; 3. `decide`: how a command is accepted or rejected from that state. -The state is not necessarily a stored read model. It is the temporary state needed -for one decision. +The state is not necessarily a stored read model. It is the temporary state +needed for one decision. ```gleam fn decide(state: State, command: Command) -> Result(List(Event), DomainError) { - let TicketWindow(capacity, sold) = state + let TicketWindow(capacity:, sold:) = state case command { - BuyTicket(buyer) -> + BuyTicket(buyer:) -> case sold < capacity { - True -> Ok([TicketSold(buyer)]) - False -> Error(SoldOut(capacity)) + True -> Ok([TicketSold(buyer:)]) + False -> Error(SoldOut(capacity:)) } } } ``` -A decider can be tested without storage: +A decider's pure functions can be tested directly: ```gleam -factos.compute_events( - decider: ticket_decider(), - events: [TicketSold("renata")], - command: BuyTicket("lucy"), -) +let factos.Decider(initial:, decide:, evolve:) = ticket_decider() +let state = + list.fold([TicketSold(buyer: "renata")], initial, evolve) + +decide(state, BuyTicket(buyer: "lucy")) ``` -## Queries +## Decision contexts -A `Query` describes the facts relevant to a command. The backend uses it to read -history and protect the append. +A `DecisionContext` describes the facts relevant to one command. A backend uses +it both to read history and to protect the resulting append: ```gleam -fn sale_query() -> factos.Query { - factos.query([ - factos.query_item( +fn sale_context() -> factos.DecisionContext { + factos.Matching(items: [ + factos.item( types: [factos.event_type("TicketSold")], tags: [factos.tag("event:gleamconf-2026")], ), @@ -88,16 +88,25 @@ fn sale_query() -> factos.Query { } ``` -Query semantics are deliberately simple: +The three variants make the dependency explicit: + +- `NoContext` reads no history and represents an unconditional append; +- `AllEvents` reads and protects the complete event log; +- `Matching(items:)` selects events through type-and-tag matching. -- query items are OR-combined; +Within `Matching`: + +- items are OR-combined; - event types inside one item are OR-combined; - tags inside one item are AND-combined; - empty types match any event type; -- empty tags add no tag constraint. +- empty tags add no tag constraint; +- an empty item list matches no events. -`EventType` and `Tag` are opaque wrappers so applications are deliberate about -what is visible to stores. +Use `NoContext` rather than `Matching(items: [])` when ignoring history is an +intentional command-design choice. `EventType` and `Tag` are opaque wrappers so +applications are deliberate about which payload information is visible to +stores. ## Contexts @@ -105,7 +114,7 @@ A backend read returns a `Context(event, state)`: ```gleam factos.Context( - query: query, + decision_context: sale_context(), state: folded_state, events: recorded_events, position: observed_position, @@ -113,27 +122,34 @@ factos.Context( ) ``` -The context contains the facts that were used to make the decision and the -condition needed to keep that decision valid until append time. +The context contains the matching facts used to make the decision, their folded +state, the highest observed global position, and the condition needed to keep +that decision valid until append time. ## Append conditions -The key condition is: +The shared condition is: ```gleam -factos.FailIfEventsMatch(query, after: position) +factos.FailIfEventsMatch( + decision_context: sale_context(), + after: observed_position, +) ``` -It means the backend must not append the newly decided facts if another matching -fact was accepted after the observed position. +It means the backend must not append the newly decided facts if another +matching fact was accepted after the observed position. This is the +context-first consistency boundary: the boundary is the facts needed by the +rule, not a fixed aggregate object. -This is the context-first consistency boundary. The boundary is the facts needed -by the rule, not a fixed aggregate object. +`FailIfEventsMatch(decision_context: NoContext, ...)` is unconditional because +no event can match `NoContext`. Backends may still reject an append for their +own storage errors. ## Domain simulations `factos/simulate` is an immutable, in-memory executable reference for the core -query, context, append-condition, and decider semantics: +decision-context, append-condition, and decider semantics: ```gleam import factos/simulate @@ -152,16 +168,12 @@ fn describe_event(event: Event) -> factos.EventDescriptor { let store = simulate.new(describe_event) - |> simulate.given( - stream: "ticket-sales", - events: [TicketSold(buyer: "renata")], - ) + |> simulate.given(events: [TicketSold(buyer: "renata")]) let assert Ok(simulate.Commit(store:, events: committed)) = simulate.dispatch( store, - stream: "ticket-sales", - query: sale_query(), + decision_context: sale_context(), decider: ticket_decider(), command: BuyTicket(buyer: "lucy"), ) @@ -176,20 +188,19 @@ interleaving: ```gleam let context = - simulate.read_context(store, sale_query(), ticket_decider()) + simulate.read_context( + store, + decision_context: sale_context(), + decider: ticket_decider(), + ) // This matching fact arrives after the context was read. let store = - simulate.given( - store, - stream: "ticket-sales", - events: [TicketSold(buyer: "marc")], - ) + simulate.given(store, events: [TicketSold(buyer: "marc")]) let result = simulate.append( store, - stream: "ticket-sales", events: [TicketSold(buyer: "lucy")], condition: context.append_condition, ) @@ -197,15 +208,14 @@ let result = let assert Error(simulate.AppendConditionFailed(condition: _)) = result ``` -An interleaved event outside `sale_query` leaves the same condition valid and the -append succeeds; the matching `TicketSold` above makes the context stale and -rejects the whole batch. +An interleaved event outside `sale_context()` leaves the same condition valid; +the matching `TicketSold` above makes the context stale and rejects the whole +batch. -The simulator is not a backend interface. Keep codec fidelity, SQL selection, -stream-revision races, transaction rollback, retries, outbox behavior, and real -concurrency in backend integration tests. `factos.compute_events` remains the -narrower helper when history is already filtered and no recorded store -transition is needed. +The simulator is not a backend interface. Keep codec fidelity, storage +predicate selection, transaction rollback, retries, outbox behavior, and real +concurrency in backend integration tests. Test `decide` and `evolve` directly +when no recorded store transition is needed. ## Recorded events @@ -215,8 +225,6 @@ Backends decode stored data into `Recorded(event)` values: ```gleam factos.Recorded( id: id, - stream: stream, - revision: revision, position: position, event: event, descriptor: factos.EventDescriptor( @@ -228,51 +236,47 @@ factos.Recorded( ) ``` -`revision` is per stream. `position` is a global log position. Context-first -checks use global positions because the facts relevant to a command may live in -many streams. +`position` is the event's global log position. `NoPosition` means no global +position was observed; `SequencePosition(Int)` carries a backend-specific +ordered position. Factos deliberately has no stream or per-stream revision in +its core record model. -## Views +## Projection folds -A `View(state, event)` is a pure projection fold: +A read model is ordinary pure application code. Fold the events with the state +and evolution function that the projection needs: ```gleam -let sold_count = - factos.view(initial: 0, evolve: fn(count, event) { +fn count_sold_tickets(events: List(Event)) -> Int { + list.fold(events, 0, fn(count, event) { case event { - TicketSold(_) -> count + 1 + TicketSold(buyer: _) -> count + 1 } }) +} ``` -The core package can run the computation: - -```gleam -factos.project(view: sold_count, events: events) -``` - -It does not decide where the projected state is stored. +Factos does not add a `View` wrapper because that record would enforce no +invariant and would not decide where projected state is stored. -## Reactors +## Effect derivation -A `Reactor(event, effect)` is a pure reaction from committed recorded events to -application-owned effect values: +Follow-up work is likewise an application-owned pure function over a recorded +event: ```gleam -fn ticket_reactor() -> factos.Reactor(Event, Effect) { - factos.reactor(fn(recorded) { - case recorded.event { - TicketSold(buyer) -> [ - AnnounceTicketSale(buyer: buyer, position: recorded.position), - ] - } - }) +fn ticket_effects(recorded: factos.Recorded(Event)) -> List(Effect) { + case recorded.event { + TicketSold(buyer:) -> [ + AnnounceTicketSale(buyer:, position: recorded.position), + ] + } } ``` -Reactors do not run IO. They make follow-up work explicit as data. Application or -infrastructure code decides whether that work is executed immediately, persisted -to an outbox, retried, or skipped during replay. +Keep effect derivation separate from execution. Application or infrastructure +code decides whether work is executed after commit, persisted to an outbox, +retried, or skipped during replay. ## What stays outside core @@ -284,7 +288,7 @@ The core package intentionally does not solve: - projection repositories; - durable effect delivery; - retries and dead letters; -- subscriptions and catch-up workers; +- subscription scheduling and historical catch-up workers; - deployment topology. Backends and applications own those decisions. diff --git a/docs/domain-driven-design.md b/docs/domain-driven-design.md index 29aefe7..4610673 100644 --- a/docs/domain-driven-design.md +++ b/docs/domain-driven-design.md @@ -34,9 +34,10 @@ model changes. DDD models often become clearer when intent, accepted facts, and decision state are separated. -- A command is intent: `BuyTicket("renata")`. -- An event is an accepted fact: `TicketSold("renata")`. -- State is what the decision needs to know: `TicketWindow(capacity: 100, sold: 42)`. +- A command is intent: `BuyTicket(buyer: "renata")`. +- An event is an accepted fact: `TicketSold(buyer: "renata")`. +- State is what the decision needs to know: + `TicketWindow(capacity: 100, sold: 42)`. - A domain error explains business rejection: `SoldOut(capacity: 100)`. Factos represents this with a `Decider`: @@ -69,18 +70,22 @@ The key design question is: Factos calls that set of facts the command context. -For a ticket-sale capacity rule, the context can be all ticket-sale facts for one -event: +For a ticket-sale capacity rule, the decision context can select every ticket +sale for one event: ```gleam -factos.query([ - factos.query_item( +factos.Matching(items: [ + factos.item( types: [factos.event_type("TicketSold")], tags: [factos.tag("event:gleamconf-2026")], ), ]) ``` +`NoContext` explicitly marks a command that ignores history. `AllEvents` marks a +command whose answer can change with any accepted fact. `Matching(items:)` +expresses a narrower type-and-tag boundary such as the ticket rule above. + That context is more precise than saying every command must belong to one aggregate root. diff --git a/docs/event-sourcing.md b/docs/event-sourcing.md index 5b740ff..ffba869 100644 --- a/docs/event-sourcing.md +++ b/docs/event-sourcing.md @@ -1,120 +1,140 @@ # Event Logs and Command Dispatch -Factos stores events, and it also provides helpers for command dispatch. +Factos stores events and provides shared types for command dispatch. Those ideas +are related but not identical: -Those two ideas are related but not identical: +- an event is an accepted fact: `TicketSold(buyer: "renata")`; +- a command is intent: `BuyTicket(buyer: "renata")`. -- Event sourcing stores facts that happened: `TicketSold("renata")`. -- Command handling receives orders: `BuyTicket("renata")`. +A backend stores accepted events. A `Decider` and the backend's dispatch API +provide one standard way to process commands on top of that log. -The backend stores the accepted events. The `Decider` and `dispatch` helpers are a -standard way to process commands on top of that event log. +## What a backend persists -## What the backend persists +A backend such as `factos_pog` persists: -A backend such as `factos_pog` persists event records: +- an application event id; +- a global sequence position; +- event type and schema version; +- tags used for selective decision contexts; +- application metadata; +- the encoded event payload. -- event id; -- stream; -- stream revision; -- global position; -- event type; -- event version; -- tags; -- metadata; -- opaque payload bytes. +There is no stream or per-stream revision in the Factos event model. Facts are +ordered in one append-only log. A decoded `factos.Recorded(event)` contains the +id, global position, domain event, and its descriptor. -It does not persist commands. It does not persist materialized views. It does not -execute side effects. +Factos does not persist commands. The core package does not maintain materialized +views or execute external side effects. ## What dispatch does -A dispatch function combines an event log with a command handler: +A dispatch combines an event log with a command-side decision: ```text -command + previous events -> new events or domain error +command + selected previous facts -> new facts or domain error ``` -For `factos_pog.dispatch_with_query`, the previous events are selected by a -`factos.Query`: +Every command names its dependency with a `DecisionContext`: ```gleam -factos.query([ - factos.query_item( - types: [factos.event_type("TicketSold")], - tags: [factos.tag("event:gleamconf-2026")], - ), -]) +fn sale_context() -> factos.DecisionContext { + factos.Matching(items: [ + factos.item( + types: [factos.event_type("TicketSold")], + tags: [factos.tag("event:gleamconf-2026")], + ), + ]) +} ``` -The backend reads those events, folds them into state, calls the decider, and -appends the resulting events only if the query context is still stable. +The backend reads matching records, decodes and folds them into temporary state, +calls the decider, and attempts to append the resulting events. -## What makes the append safe +Use: -A context read observes a global event-log position. The backend then protects the -append with: +- `NoContext` when the decision intentionally ignores history; +- `AllEvents` when every prior fact can affect the decision; +- `Matching(items:)` for a selective type-and-tag boundary. + +For `Matching`, items are OR-combined. Within one item, types are OR-combined and +tags are AND-combined. Empty types or tags remove that dimension's restriction; +an empty item list matches no events. + +## What makes an append safe + +A context read observes the highest global position among its selected events. +The shared append condition is: ```gleam -factos.FailIfEventsMatch(query, after: position) +factos.FailIfEventsMatch( + decision_context: sale_context(), + after: position, +) ``` -That means: +It means: -> do not append these new events if another event matching the same query was -> accepted after the position used for the decision. +> Do not append these new facts if another fact selected by the same decision +> context was accepted after the position used for the decision. -This is how Factos lets the consistency boundary follow the rule. The boundary -can be a tag, a set of event types, one stream, many streams, or all events. +This lets the consistency boundary follow the business rule instead of a fixed +aggregate or stream. `AllEvents` creates a global boundary. `Matching` creates a +selective boundary. `NoContext` matches nothing and therefore represents an +unconditional append. -## What views are +A concrete backend decides how to protect this condition. For example, +`factos_pog` uses PostgreSQL serializable transactions and retries retryable +serialization or deadlock conflicts. -A `View` is not a durable projection table. It is a pure fold: +## Projections are application code + +A read model is an ordinary pure fold, not a special core type or a durable +projection table: ```gleam -factos.view(initial: 0, evolve: fn(count, event) { - case event { - TicketSold(_) -> count + 1 - } -}) +fn count_sold_tickets(events: List(Event)) -> Int { + list.fold(events, 0, fn(count, event) { + case event { + TicketSold(buyer: _) -> count + 1 + } + }) +} ``` -You can run the fold over any list of events. To make a materialized view durable, -your application stores the folded result. +An application that needs a materialized view stores the result itself or +updates its projection from committed records. Projections remain recomputable +while stored history is decodable, so event versioning and codec compatibility +remain application responsibilities. -Views can be recomputed as long as the stored event history can still be decoded. -That makes event versioning and codec compatibility an application responsibility. +## Effects are application code -## What reactors are - -A `Reactor` is also pure. It maps committed recorded events to effect values: +Derive effect values with an ordinary pure function: ```gleam -factos.react_all(ticket_reactor(), dispatch.events) +fn ticket_effects(recorded: factos.Recorded(Event)) -> List(Effect) { + case recorded.event { + TicketSold(buyer:) -> [ + AnnounceTicketSale(buyer:, position: recorded.position), + ] + } +} ``` -Factos does not execute those effects. That is deliberate: replaying old events -should not accidentally resend emails, charge cards, or publish webhooks. +Keeping derivation separate from execution prevents replay from implicitly +sending email, charging a card, or publishing a webhook. ## Where Factos is opinionated -Factos is low-level about storage and high-level enough to standardize command -dispatch. - -It is opinionated that: - -- facts are stored as an append-only event log; -- command decisions should be pure; -- context reads should produce append conditions; -- backends should return committed records after append; -- projections and effects should remain explicit application code. +Factos is opinionated that: -It is not opinionated about: +- accepted facts form an append-only globally ordered log; +- command decisions are pure; +- every dispatch names a decision context; +- context reads produce append conditions; +- backends return committed records; +- projections and effects remain explicit application code. -- your event names; -- your command names; -- payload encoding; -- projection storage; -- subscription infrastructure; -- effect retry policy; -- deployment topology. +It does not prescribe event or command names, payload encoding, projection +storage, subscription infrastructure, effect retry policy, or deployment +topology. diff --git a/examples/course_subscriptions/README.md b/examples/course_subscriptions/README.md index 35a9378..07918b5 100644 --- a/examples/course_subscriptions/README.md +++ b/examples/course_subscriptions/README.md @@ -21,15 +21,16 @@ package sets the same configurable constraint to five. ## DCB approach `StudentSubscribedToCourse` is tagged with both `course:` and -`student:`. A subscription command builds one query from two items: -the course history needed to calculate capacity and occupancy, and the student's -history needed to calculate their subscription count. +`student:`. A subscription command builds one decision context from +two items: the course history needed to calculate capacity and occupancy, and +the student's history needed to calculate their subscription count. -Factos folds those matching events into command-specific state. Factos Pog then -runs the read, decision, and conditional append in a serializable PostgreSQL -transaction. Concurrent attempts therefore enforce both sides of the constraint -without a read model, reservation saga, or aggregate spanning every course and -student. +Factos folds those matching events into command-specific state. A shared +`factos.Dispatch(Event)` is built with `factos.new_dispatch`, then +`factos_pog.dispatch` runs the read, decision, and conditional append in a +serializable PostgreSQL transaction. Concurrent attempts therefore enforce both +sides of the constraint without a read model, reservation saga, or aggregate +spanning every course and student. The package also demonstrates: @@ -37,7 +38,19 @@ The package also demonstrates: - course-capacity changes; - domain errors for missing, full, unchanged, and duplicate cases; - synchronized concurrent subscription attempts against the same course; -- JSON event payloads and course/student tags. +- a shared `factos.EventCodec(Event, String)` whose encoder serializes JSON with + `json.to_string` and whose decoder parses `factos.Recorded(String)`. + +The codec is built with `factos.codec`. Its encoder returns +`factos.Event(String)` values created by `factos.new_event` and enriched with +`factos.with_tags`. Its decoder checks the recorded descriptor before parsing +the stored string, returning `factos.InvalidData` for malformed JSON and +`factos.UnknownEvent` for unsupported event types or versions. + +The public dispatch result uses only shared orchestration types: +`Result(factos.Dispatch(Event), factos.Error(Error, Nil, pog.QueryError))`. +`Nil` is the subscription-error type because this example has no subscriptions; +`pog.QueryError` remains the PostgreSQL backend error. The command-to-state flow is diagrammed in the module documentation in [`src/course_subscription.gleam`](src/course_subscription.gleam). @@ -62,7 +75,8 @@ container. No developer-managed database is required. ## Package layout - [`src/course_subscription.gleam`](src/course_subscription.gleam) contains the - commands, events, decider, DCB queries, PostgreSQL codec, and dispatch API. + commands, events, decider, DCB decision contexts, shared String-backed codec, + and PostgreSQL-backed dispatch API. - [`test/course_subscriptions_test.gleam`](test/course_subscriptions_test.gleam) verifies the source scenarios and concurrent constraint enforcement. - [`dev/course_subscriptions_dev.gleam`](dev/course_subscriptions_dev.gleam) is diff --git a/examples/course_subscriptions/dev/course_subscriptions_dev.gleam b/examples/course_subscriptions/dev/course_subscriptions_dev.gleam index 4f3e4a1..87e3144 100644 --- a/examples/course_subscriptions/dev/course_subscriptions_dev.gleam +++ b/examples/course_subscriptions/dev/course_subscriptions_dev.gleam @@ -32,8 +32,8 @@ type WorkerMessage { WorkerFinished( worker: String, result: Result( - factos_pog.Dispatch(course_subscription.Event), - factos_pog.Error(course_subscription.Error, Nil), + factos.Dispatch(course_subscription.Event), + factos.Error(course_subscription.Error, Nil, pog.QueryError), ), ) } @@ -51,7 +51,7 @@ pub fn run() -> Result(ExampleResult, testcontainer_error.Error) { course_subscription.DefineCourse(course_id: "c1", capacity: 2), uuid.v4_string, ) - let assert Error(factos_pog.DomainError(course_subscription.CourseAlreadyExists( + let assert Error(factos.DomainError(course_subscription.CourseAlreadyExists( course_id: "c1", ))) = course_subscription.dispatch( @@ -59,7 +59,7 @@ pub fn run() -> Result(ExampleResult, testcontainer_error.Error) { course_subscription.DefineCourse(course_id: "c1", capacity: 15), uuid.v4_string, ) - let assert Error(factos_pog.DomainError(course_subscription.CourseDoesNotExist( + let assert Error(factos.DomainError(course_subscription.CourseDoesNotExist( course_id: "c0", ))) = course_subscription.dispatch( @@ -70,7 +70,7 @@ pub fn run() -> Result(ExampleResult, testcontainer_error.Error) { ), uuid.v4_string, ) - let assert Error(factos_pog.DomainError(course_subscription.CapacityUnchanged( + let assert Error(factos.DomainError(course_subscription.CapacityUnchanged( capacity: 2, ))) = course_subscription.dispatch( @@ -85,7 +85,7 @@ pub fn run() -> Result(ExampleResult, testcontainer_error.Error) { uuid.v4_string, ) - let assert Error(factos_pog.DomainError(course_subscription.CourseDoesNotExist( + let assert Error(factos.DomainError(course_subscription.CourseDoesNotExist( course_id: "missing", ))) = course_subscription.dispatch( @@ -103,7 +103,7 @@ pub fn run() -> Result(ExampleResult, testcontainer_error.Error) { course_id: "c1", ), ) - let assert Error(factos_pog.DomainError( + let assert Error(factos.DomainError( course_subscription.StudentAlreadySubscribed, )) = course_subscription.dispatch( @@ -128,7 +128,7 @@ pub fn run() -> Result(ExampleResult, testcontainer_error.Error) { course_id: "c1", ), ) - let assert Error(factos_pog.DomainError(course_subscription.CourseFullyBooked( + let assert Error(factos.DomainError(course_subscription.CourseFullyBooked( course_id: "c1", ))) = course_subscription.dispatch( @@ -157,7 +157,7 @@ pub fn run() -> Result(ExampleResult, testcontainer_error.Error) { ), ) }) - let assert Error(factos_pog.DomainError(course_subscription.StudentCourseLimitReached( + let assert Error(factos.DomainError(course_subscription.StudentCourseLimitReached( limit: 5, ))) = course_subscription.dispatch( @@ -184,7 +184,7 @@ pub fn run() -> Result(ExampleResult, testcontainer_error.Error) { let assert Ok(events) = factos_pog.read_after( connection, - query: factos.AllEvents, + decision_context: factos.AllEvents, after: factos.NoPosition, limit: 100, codec: course_subscription.codec(), @@ -239,8 +239,8 @@ fn run_concurrent_final_seat( connection: pog.Connection, ) -> List( Result( - factos_pog.Dispatch(course_subscription.Event), - factos_pog.Error(course_subscription.Error, Nil), + factos.Dispatch(course_subscription.Event), + factos.Error(course_subscription.Error, Nil, pog.QueryError), ), ) { let messages = process.new_subject() @@ -291,8 +291,8 @@ fn receive_worker_ready( fn receive_worker_finished( messages: process.Subject(WorkerMessage), ) -> Result( - factos_pog.Dispatch(course_subscription.Event), - factos_pog.Error(course_subscription.Error, Nil), + factos.Dispatch(course_subscription.Event), + factos.Error(course_subscription.Error, Nil, pog.QueryError), ) { let assert Ok(message) = process.receive(messages, within: 10_000) let assert WorkerFinished(worker: _, result:) = message @@ -301,8 +301,8 @@ fn receive_worker_finished( fn is_accepted( result: Result( - factos_pog.Dispatch(course_subscription.Event), - factos_pog.Error(course_subscription.Error, Nil), + factos.Dispatch(course_subscription.Event), + factos.Error(course_subscription.Error, Nil, pog.QueryError), ), ) -> Bool { case result { @@ -313,12 +313,12 @@ fn is_accepted( fn is_fully_booked( result: Result( - factos_pog.Dispatch(course_subscription.Event), - factos_pog.Error(course_subscription.Error, Nil), + factos.Dispatch(course_subscription.Event), + factos.Error(course_subscription.Error, Nil, pog.QueryError), ), ) -> Bool { case result { - Error(factos_pog.DomainError(course_subscription.CourseFullyBooked( + Error(factos.DomainError(course_subscription.CourseFullyBooked( course_id: "c8", ))) -> True Ok(_) | Error(_) -> False diff --git a/examples/course_subscriptions/manifest.toml b/examples/course_subscriptions/manifest.toml index c9aaf3f..163c8b0 100644 --- a/examples/course_subscriptions/manifest.toml +++ b/examples/course_subscriptions/manifest.toml @@ -11,8 +11,8 @@ packages = [ { name = "cowl", version = "1.1.0", build_tools = ["gleam"], requirements = ["gleam_stdlib"], otp_app = "cowl", source = "hex", outer_checksum = "7849E7C789D7228243A4253138FC883720A0BB44AEF406102328CADC64C3CA2B" }, { name = "envie", version = "1.2.0", build_tools = ["gleam"], requirements = ["gleam_stdlib"], otp_app = "envie", source = "hex", outer_checksum = "E7EBA39310F32A40BF3EDDD7CD9C7A2BC289909983D357411C22873415BC322A" }, { name = "exception", version = "2.1.1", build_tools = ["gleam"], requirements = ["gleam_stdlib"], otp_app = "exception", source = "hex", outer_checksum = "6BDEA95248093599391C3B5DF1835C5C6A86C353C2F99CE539B450E3432FE117" }, - { name = "factos", version = "1.0.0", build_tools = ["gleam"], requirements = ["gleam_stdlib"], source = "local", path = "../.." }, - { name = "factos_pog", version = "2.0.0", build_tools = ["gleam"], requirements = ["factos", "gleam_erlang", "gleam_json", "gleam_otp", "gleam_stdlib", "pog"], source = "local", path = "../../backends/factos_pog" }, + { name = "factos", version = "2.0.0", build_tools = ["gleam"], requirements = ["gleam_stdlib"], source = "local", path = "../.." }, + { name = "factos_pog", version = "2.0.0", build_tools = ["gleam"], requirements = ["exception", "factos", "gleam_erlang", "gleam_json", "gleam_otp", "gleam_stdlib", "pog"], source = "local", path = "../../backends/factos_pog" }, { name = "filepath", version = "1.1.2", build_tools = ["gleam"], requirements = ["gleam_stdlib"], otp_app = "filepath", source = "hex", outer_checksum = "B06A9AF0BF10E51401D64B98E4B627F1D2E48C154967DA7AF4D0914780A6D40A" }, { name = "gleam_crypto", version = "1.6.0", build_tools = ["gleam"], requirements = ["gleam_stdlib"], otp_app = "gleam_crypto", source = "hex", outer_checksum = "2DE9E4EF53CF6FEE049D4F765731F7178F7A11AEFAE00EEE63BF7536B354AD3F" }, { name = "gleam_erlang", version = "1.3.0", build_tools = ["gleam"], requirements = ["gleam_stdlib"], otp_app = "gleam_erlang", source = "hex", outer_checksum = "1124AD3AA21143E5AF0FC5CF3D9529F6DB8CA03E43A55711B60B6B7B3874375C" }, diff --git a/examples/course_subscriptions/src/course_subscription.gleam b/examples/course_subscriptions/src/course_subscription.gleam index 398869c..50b59e5 100644 --- a/examples/course_subscriptions/src/course_subscription.gleam +++ b/examples/course_subscriptions/src/course_subscription.gleam @@ -1,63 +1,15 @@ -//// Enforce course and student subscription constraints with Dynamic +///// Enforce course and student subscription constraints with Dynamic //// Consistency Boundaries. //// //// This implements the example at //// https://dcb.events/examples/course-subscriptions/ using Factos and //// PostgreSQL. -//// -//// ## One command flow -//// -//// A subscription command builds only the state needed to decide whether that -//// student can join that course: -//// -//// ```text -//// SubscribeStudentToCourse(student_id: "s1", course_id: "c1") -//// | -//// v -//// initial_state(command) -//// SubscribingStudent( -//// course: CourseMissing, -//// course_subscription_count: 0, -//// student_subscription_count: 0, -//// subscription: NotSubscribed, -//// ) -//// | -//// v -//// query(command) -//// course:c1 -> definition, capacity changes, subscriptions -//// student:s1 -> subscriptions to any course -//// | -//// v -//// evolve(state, event) for every matching stored event -//// | -//// v -//// SubscribingStudent( -//// course: CoursePresent(capacity: 2), -//// course_subscription_count: 1, -//// student_subscription_count: 3, -//// subscription: NotSubscribed, -//// ) -//// | -//// v -//// decide(state, command) -//// CourseMissing -> CourseDoesNotExist -//// course count >= capacity -> CourseFullyBooked -//// Subscribed -> StudentAlreadySubscribed -//// student count >= 5 -> StudentCourseLimitReached -//// otherwise -> StudentSubscribedToCourse -//// | -//// v -//// append the decided event -//// ``` -//// -//// The query is the Dynamic Consistency Boundary: it combines course-tagged and -//// student-tagged history, and `evolve` folds that history into the command-specific -//// state before `decide` applies the constraints. import factos import factos/factos_pog import gleam/dynamic/decode import gleam/json +import gleam/result import pog const student_course_limit = 5 @@ -255,60 +207,72 @@ fn evolve(state: State, event: Event) -> State { } } -pub fn codec() -> factos_pog.EventCodec(Event) { - factos_pog.codec(encode: encode_event, decode: decode_event) +pub fn codec() -> factos.EventCodec(Event, String) { + factos.codec(encode:, decode:) } -fn encode_event(event: Event) -> factos_pog.Proposed { +fn encode(event: Event) -> factos.Event(String) { case event { CourseDefined(course_id:, capacity:) -> - factos_pog.new_proposed( + factos.new_event( type_: factos.event_type("CourseDefined"), version: 1, data: json.object([ #("course_id", json.string(course_id)), #("capacity", json.int(capacity)), - ]), + ]) + |> json.to_string, ) - |> factos_pog.with_tags(tags: [ + |> factos.with_tags(tags: [ factos.tag("course:" <> course_id), ]) CourseCapacityChanged(course_id:, new_capacity:) -> - factos_pog.new_proposed( + factos.new_event( type_: factos.event_type("CourseCapacityChanged"), version: 1, data: json.object([ #("course_id", json.string(course_id)), #("new_capacity", json.int(new_capacity)), - ]), + ]) + |> json.to_string, ) - |> factos_pog.with_tags(tags: [ + |> factos.with_tags(tags: [ factos.tag("course:" <> course_id), ]) StudentSubscribedToCourse(student_id:, course_id:) -> - factos_pog.new_proposed( + factos.new_event( type_: factos.event_type("StudentSubscribedToCourse"), version: 1, data: json.object([ #("student_id", json.string(student_id)), #("course_id", json.string(course_id)), - ]), + ]) + |> json.to_string, ) - |> factos_pog.with_tags(tags: [ + |> factos.with_tags(tags: [ factos.tag("student:" <> student_id), factos.tag("course:" <> course_id), ]) } } -fn decode_event( - descriptor: factos.EventDescriptor, -) -> Result(decode.Decoder(Event), factos_pog.DecodeError) { - case factos.event_type_name(descriptor.type_), descriptor.version { - "CourseDefined", 1 -> Ok(course_defined_decoder()) - "CourseCapacityChanged", 1 -> Ok(course_capacity_changed_decoder()) - "StudentSubscribedToCourse", 1 -> Ok(student_subscribed_decoder()) - _, _ -> Error(factos_pog.UnknownEvent) +fn decode( + stored: factos.Recorded(String), +) -> Result(Event, factos.DecodeError) { + case + factos.event_type_name(stored.descriptor.type_), + stored.descriptor.version + { + "CourseDefined", 1 -> + json.parse(stored.event, using: course_defined_decoder()) + |> result.replace_error(factos.InvalidData) + "CourseCapacityChanged", 1 -> + json.parse(stored.event, using: course_capacity_changed_decoder()) + |> result.replace_error(factos.InvalidData) + "StudentSubscribedToCourse", 1 -> + json.parse(stored.event, using: student_subscribed_decoder()) + |> result.replace_error(factos.InvalidData) + _, _ -> Error(factos.UnknownEvent) } } @@ -334,28 +298,27 @@ pub fn dispatch( connection: pog.Connection, command: Command, event_id: fn() -> String, -) -> Result(factos_pog.Dispatch(Event), factos_pog.Error(Error, Nil)) { - factos_pog.new_dispatch( +) -> Result(factos.Dispatch(Event), factos.Error(Error, Nil, pog.QueryError)) { + factos.new_dispatch( connection:, - stream: stream(command), decider: factos.decider(initial: initial(command), decide:, evolve:), + decision_context: decision_context(command), codec: codec(), ) - |> factos_pog.with_query(query(command)) |> factos_pog.dispatch(command, event_id:) } -fn query(command: Command) -> factos.Query { +fn decision_context(command: Command) -> factos.DecisionContext { case command { DefineCourse(course_id:, capacity: _) -> - factos.query([ - factos.query_item(types: [factos.event_type("CourseDefined")], tags: [ + factos.Matching([ + factos.item(types: [factos.event_type("CourseDefined")], tags: [ factos.tag("course:" <> course_id), ]), ]) ChangeCourseCapacity(course_id:, new_capacity: _) -> - factos.query([ - factos.query_item( + factos.Matching([ + factos.item( types: [ factos.event_type("CourseDefined"), factos.event_type("CourseCapacityChanged"), @@ -364,8 +327,8 @@ fn query(command: Command) -> factos.Query { ), ]) SubscribeStudentToCourse(student_id:, course_id:) -> - factos.query([ - factos.query_item( + factos.Matching([ + factos.item( types: [ factos.event_type("CourseDefined"), factos.event_type("CourseCapacityChanged"), @@ -373,19 +336,10 @@ fn query(command: Command) -> factos.Query { ], tags: [factos.tag("course:" <> course_id)], ), - factos.query_item( + factos.item( types: [factos.event_type("StudentSubscribedToCourse")], tags: [factos.tag("student:" <> student_id)], ), ]) } } - -fn stream(command: Command) { - case command { - DefineCourse(course_id:, ..) | ChangeCourseCapacity(course_id:, ..) -> - "course-" <> course_id - SubscribeStudentToCourse(student_id:, course_id:) -> - "subscription-" <> course_id <> ":" <> student_id - } -} diff --git a/examples/dynamic_product_price/README.md b/examples/dynamic_product_price/README.md index b877033..b8806df 100644 --- a/examples/dynamic_product_price/README.md +++ b/examples/dynamic_product_price/README.md @@ -11,16 +11,17 @@ containing several products must be accepted or rejected as one decision. ## DCB approach -Price events are tagged with `product:`. An order query contains one -item for each product in the cart, so the decision state contains only the price -history relevant to that cart. The decider reconstructs the stable price and all -prices still inside the grace period, then validates every displayed price before -emitting one `ProductsOrdered` event. - -Factos Pog performs the read, decision, and conditional append in a serializable -PostgreSQL transaction. A concurrent price change that affects the query causes -the order decision to retry against the new history; a partially validated cart -is never persisted. +Price events are tagged with `product:`. An order decision context +contains one item for each product in the cart, so the temporary decision state +contains only the price history relevant to that cart. The decider reconstructs +the stable price and all prices still inside the grace period, then validates +every displayed price before emitting one `ProductsOrdered` event. + +The shared `factos` API owns the codec, event metadata, and dispatch builder. +`factos_pog.dispatch` executes that dispatch in a serializable PostgreSQL +transaction. A concurrent price change selected by the decision context causes +the order to retry against the new history; a partially validated cart is never +persisted. The package demonstrates: @@ -29,12 +30,13 @@ The package demonstrates: - rejection of prices that were never valid or have expired; - atomic multi-product cart validation; - reporting the first invalid product; -- JSON events with product tags. +- JSON-string event payloads with product tags and recorded-minute metadata. The source example uses relative `minutesAgo` metadata for illustration. This -implementation stores an absolute `recorded_minute` and supplies -`current_minute` in the command, keeping the decider deterministic across -serializable retries. +implementation stores an absolute `recorded_minute` in Factos event metadata. +The codec decodes that metadata from `factos.Recorded(String)` while parsing its +JSON payload, and the command supplies `current_minute`, keeping the decider +deterministic across serializable retries. ## Run it @@ -56,8 +58,8 @@ container. No developer-managed database is required. ## Package layout - [`src/dynamic_product_price.gleam`](src/dynamic_product_price.gleam) contains - the commands, events, price decision model, DCB queries, codec, and dispatch - API. + the commands, events, price decision model, DCB decision contexts, codec, and + dispatch API. - [`test/dynamic_product_price_test.gleam`](test/dynamic_product_price_test.gleam) verifies the price and cart boundaries. - [`dev/dynamic_product_price_dev.gleam`](dev/dynamic_product_price_dev.gleam) is diff --git a/examples/dynamic_product_price/dev/dynamic_product_price_dev.gleam b/examples/dynamic_product_price/dev/dynamic_product_price_dev.gleam index 6f49ef5..8156650 100644 --- a/examples/dynamic_product_price/dev/dynamic_product_price_dev.gleam +++ b/examples/dynamic_product_price/dev/dynamic_product_price_dev.gleam @@ -31,8 +31,8 @@ type WorkerMessage { WorkerFinished( worker: String, result: Result( - factos_pog.Dispatch(dynamic_product_price.Event), - factos_pog.Error(dynamic_product_price.Error, Nil), + factos.Dispatch(dynamic_product_price.Event), + factos.Error(dynamic_product_price.Error, Nil, pog.QueryError), ), ) } @@ -174,7 +174,7 @@ pub fn run() -> Result(ExampleResult, testcontainer_error.Error) { let assert Ok(events) = factos_pog.read_after( connection, - query: factos.AllEvents, + decision_context: factos.AllEvents, after: factos.NoPosition, limit: 100, codec: dynamic_product_price.codec(), @@ -255,12 +255,12 @@ fn require_order( fn require_invalid_order( result: Result( - factos_pog.Dispatch(dynamic_product_price.Event), - factos_pog.Error(dynamic_product_price.Error, Nil), + factos.Dispatch(dynamic_product_price.Event), + factos.Error(dynamic_product_price.Error, Nil, pog.QueryError), ), product_id product_id: String, ) -> Nil { - let assert Error(factos_pog.DomainError(dynamic_product_price.InvalidPrice( + let assert Error(factos.DomainError(dynamic_product_price.InvalidPrice( product_id: invalid_product_id, ))) = result assert invalid_product_id == product_id @@ -272,8 +272,8 @@ fn run_concurrent_orders( items: List(dynamic_product_price.OrderItem), ) -> List( Result( - factos_pog.Dispatch(dynamic_product_price.Event), - factos_pog.Error(dynamic_product_price.Error, Nil), + factos.Dispatch(dynamic_product_price.Event), + factos.Error(dynamic_product_price.Error, Nil, pog.QueryError), ), ) { let messages = process.new_subject() @@ -338,8 +338,8 @@ fn receive_worker_ready( fn receive_worker_finished( messages: process.Subject(WorkerMessage), ) -> Result( - factos_pog.Dispatch(dynamic_product_price.Event), - factos_pog.Error(dynamic_product_price.Error, Nil), + factos.Dispatch(dynamic_product_price.Event), + factos.Error(dynamic_product_price.Error, Nil, pog.QueryError), ) { let assert Ok(message) = process.receive(messages, within: 10_000) let assert WorkerFinished(worker: _, result:) = message @@ -348,8 +348,8 @@ fn receive_worker_finished( fn is_accepted( result: Result( - factos_pog.Dispatch(dynamic_product_price.Event), - factos_pog.Error(dynamic_product_price.Error, Nil), + factos.Dispatch(dynamic_product_price.Event), + factos.Error(dynamic_product_price.Error, Nil, pog.QueryError), ), ) -> Bool { case result { diff --git a/examples/dynamic_product_price/manifest.toml b/examples/dynamic_product_price/manifest.toml index c9aaf3f..163c8b0 100644 --- a/examples/dynamic_product_price/manifest.toml +++ b/examples/dynamic_product_price/manifest.toml @@ -11,8 +11,8 @@ packages = [ { name = "cowl", version = "1.1.0", build_tools = ["gleam"], requirements = ["gleam_stdlib"], otp_app = "cowl", source = "hex", outer_checksum = "7849E7C789D7228243A4253138FC883720A0BB44AEF406102328CADC64C3CA2B" }, { name = "envie", version = "1.2.0", build_tools = ["gleam"], requirements = ["gleam_stdlib"], otp_app = "envie", source = "hex", outer_checksum = "E7EBA39310F32A40BF3EDDD7CD9C7A2BC289909983D357411C22873415BC322A" }, { name = "exception", version = "2.1.1", build_tools = ["gleam"], requirements = ["gleam_stdlib"], otp_app = "exception", source = "hex", outer_checksum = "6BDEA95248093599391C3B5DF1835C5C6A86C353C2F99CE539B450E3432FE117" }, - { name = "factos", version = "1.0.0", build_tools = ["gleam"], requirements = ["gleam_stdlib"], source = "local", path = "../.." }, - { name = "factos_pog", version = "2.0.0", build_tools = ["gleam"], requirements = ["factos", "gleam_erlang", "gleam_json", "gleam_otp", "gleam_stdlib", "pog"], source = "local", path = "../../backends/factos_pog" }, + { name = "factos", version = "2.0.0", build_tools = ["gleam"], requirements = ["gleam_stdlib"], source = "local", path = "../.." }, + { name = "factos_pog", version = "2.0.0", build_tools = ["gleam"], requirements = ["exception", "factos", "gleam_erlang", "gleam_json", "gleam_otp", "gleam_stdlib", "pog"], source = "local", path = "../../backends/factos_pog" }, { name = "filepath", version = "1.1.2", build_tools = ["gleam"], requirements = ["gleam_stdlib"], otp_app = "filepath", source = "hex", outer_checksum = "B06A9AF0BF10E51401D64B98E4B627F1D2E48C154967DA7AF4D0914780A6D40A" }, { name = "gleam_crypto", version = "1.6.0", build_tools = ["gleam"], requirements = ["gleam_stdlib"], otp_app = "gleam_crypto", source = "hex", outer_checksum = "2DE9E4EF53CF6FEE049D4F765731F7178F7A11AEFAE00EEE63BF7536B354AD3F" }, { name = "gleam_erlang", version = "1.3.0", build_tools = ["gleam"], requirements = ["gleam_stdlib"], otp_app = "gleam_erlang", source = "hex", outer_checksum = "1124AD3AA21143E5AF0FC5CF3D9529F6DB8CA03E43A55711B60B6B7B3874375C" }, diff --git a/examples/dynamic_product_price/src/dynamic_product_price.gleam b/examples/dynamic_product_price/src/dynamic_product_price.gleam index 409c720..435d67d 100644 --- a/examples/dynamic_product_price/src/dynamic_product_price.gleam +++ b/examples/dynamic_product_price/src/dynamic_product_price.gleam @@ -233,11 +233,11 @@ fn classify_age(current_minute: Int, recorded_minute: Int) -> PriceAge { } } -pub fn codec() -> factos_pog.EventCodec(Event) { - factos_pog.codec(encode: encode_event, decode: decode_event) +pub fn codec() -> factos.EventCodec(Event, String) { + factos.codec(encode: encode_event, decode: decode_event) } -fn encode_event(event: Event) -> factos_pog.Proposed { +fn encode_event(event: Event) -> factos.Event(String) { case event { ProductDefined(product_id:, price:, recorded_minute:) -> proposed_event( @@ -285,20 +285,20 @@ fn proposed_event( type_ type_name: String, data data: json.Json, tags tags: List(factos.Tag), -) -> factos_pog.Proposed { - factos_pog.new_proposed( +) -> factos.Event(String) { + factos.new_event( type_: factos.event_type(type_name), version: 1, - data:, + data: json.to_string(data), ) - |> factos_pog.with_tags(tags:) + |> factos.with_tags(tags:) } fn with_recorded_minute( - proposed: factos_pog.Proposed, + proposed: factos.Event(String), recorded_minute: Int, -) -> factos_pog.Proposed { - factos_pog.with_metadata( +) -> factos.Event(String) { + factos.with_metadata( proposed, metadata: factos.metadata([ #(recorded_minute_key, int.to_string(recorded_minute)), @@ -307,49 +307,60 @@ fn with_recorded_minute( } fn decode_event( - descriptor: factos.EventDescriptor, -) -> Result(decode.Decoder(Event), factos_pog.DecodeError) { - case factos.event_type_name(descriptor.type_), descriptor.version { + stored: factos.Recorded(String), +) -> Result(Event, factos.DecodeError) { + case + factos.event_type_name(stored.descriptor.type_), + stored.descriptor.version + { "ProductDefined", 1 -> { use recorded_minute <- result.try(decode_recorded_minute( - descriptor.metadata, + stored.descriptor.metadata, )) - Ok( - product_defined_decoder() - |> decode.map(fn(data) { - ProductDefined(product_id: data.0, price: data.1, recorded_minute:) - }), + json.parse( + stored.event, + using: product_defined_decoder() + |> decode.map(fn(data) { + ProductDefined(product_id: data.0, price: data.1, recorded_minute:) + }), ) + |> result.map_error(fn(_) { factos.InvalidData }) } "ProductPriceChanged", 1 -> { use recorded_minute <- result.try(decode_recorded_minute( - descriptor.metadata, + stored.descriptor.metadata, )) - Ok( - product_price_changed_decoder() - |> decode.map(fn(data) { - ProductPriceChanged( - product_id: data.0, - new_price: data.1, - recorded_minute:, - ) - }), + json.parse( + stored.event, + using: product_price_changed_decoder() + |> decode.map(fn(data) { + ProductPriceChanged( + product_id: data.0, + new_price: data.1, + recorded_minute:, + ) + }), ) + |> result.map_error(fn(_) { factos.InvalidData }) } "ProductsOrdered", 1 -> - Ok(ordered_items_decoder() |> decode.map(ProductsOrdered)) - _, _ -> Error(factos_pog.UnknownEvent) + json.parse( + stored.event, + using: ordered_items_decoder() |> decode.map(ProductsOrdered), + ) + |> result.map_error(fn(_) { factos.InvalidData }) + _, _ -> Error(factos.UnknownEvent) } } fn decode_recorded_minute( metadata: factos.Metadata, -) -> Result(Int, factos_pog.DecodeError) { +) -> Result(Int, factos.DecodeError) { use value <- result.try( factos.metadata_get(metadata, recorded_minute_key) - |> result.replace_error(factos_pog.InvalidData), + |> result.replace_error(factos.InvalidData), ) - int.parse(value) |> result.replace_error(factos_pog.InvalidData) + int.parse(value) |> result.replace_error(factos.InvalidData) } fn product_defined_decoder() -> decode.Decoder(#(String, Int)) { @@ -379,23 +390,22 @@ pub fn dispatch( connection: pog.Connection, command: Command, event_id: fn() -> String, -) -> Result(factos_pog.Dispatch(Event), factos_pog.Error(Error, Nil)) { - factos_pog.new_dispatch( +) -> Result(factos.Dispatch(Event), factos.Error(Error, Nil, pog.QueryError)) { + factos.new_dispatch( connection:, - stream: stream(command), decider: factos.decider(initial: initial(command), decide:, evolve:), + decision_context: decision_context(command), codec: codec(), ) - |> factos_pog.with_query(query(command)) |> factos_pog.dispatch(command, event_id:) } -fn query(command: Command) -> factos.Query { +fn decision_context(command: Command) -> factos.DecisionContext { case command { DefineProduct(product_id:, price: _, recorded_minute: _) | ChangeProductPrice(product_id:, new_price: _, recorded_minute: _) -> - factos.query([ - factos.query_item( + factos.Matching([ + factos.item( types: [ factos.event_type("ProductDefined"), factos.event_type("ProductPriceChanged"), @@ -407,7 +417,7 @@ fn query(command: Command) -> factos.Query { items |> list.map(fn(item) { let OrderItem(product_id:, displayed_price: _) = item - factos.query_item( + factos.item( types: [ factos.event_type("ProductDefined"), factos.event_type("ProductPriceChanged"), @@ -415,16 +425,6 @@ fn query(command: Command) -> factos.Query { tags: [factos.tag("product:" <> product_id)], ) }) - |> factos.query - } -} - -fn stream(command: Command) -> String { - case command { - DefineProduct(product_id:, price: _, recorded_minute: _) - | ChangeProductPrice(product_id:, new_price: _, recorded_minute: _) -> - "product-" <> product_id - OrderProducts(order_id:, items: _, current_minute: _) -> - "order-" <> order_id + |> factos.Matching } } diff --git a/examples/invoice_number/README.md b/examples/invoice_number/README.md index 312c173..e4a57c4 100644 --- a/examples/invoice_number/README.md +++ b/examples/invoice_number/README.md @@ -10,23 +10,25 @@ gapless sequence even when several invoices are created concurrently. ## DCB approach -The decision query matches every `InvoiceCreated` event. Folding that history -produces the next number, starting at `1`. Although each command writes to its -own invoice stream, the untagged global query is the dynamic consistency -boundary shared by every invoice-number allocation. +The decision context matches every `InvoiceCreated` fact. Folding that history +produces the next number, starting at `1`. The same global type-based context is +shared by every invoice-number allocation; there is no per-invoice stream or +revision boundary. -Factos Pog runs the read, number allocation, and conditional append in a -serializable PostgreSQL transaction. Competing commands cannot commit the same -number: a serialization conflict retries one command against the newly -committed invoice and assigns the following number. +Factos defines the string codec, dispatch builder, and shared result/error types; +Factos Pog executes that dispatch in a serializable PostgreSQL transaction. +Competing commands cannot commit the same number: a serialization conflict +retries one command against the newly committed invoice and assigns the +following number. The package demonstrates: - allocation beginning at invoice `1`; - sequential gapless numbering; - synchronized concurrent allocations; -- a global consistency boundary independent of write-stream identity; -- JSON event payloads and per-invoice tags. +- a global consistency boundary independent of invoice identity; +- JSON encoded into string event payloads through the shared Factos codec, plus + per-invoice tags. This straightforward version replays all `InvoiceCreated` events for each allocation. The source article discusses snapshots and last-event reads as @@ -52,7 +54,8 @@ container. No developer-managed database is required. ## Package layout - [`src/invoice_number.gleam`](src/invoice_number.gleam) contains the command, - event, sequence decider, global query, codec, and dispatch API. + event, sequence decider, global decision context, shared Factos string codec, + and PostgreSQL-backed dispatch API. - [`test/invoice_number_test.gleam`](test/invoice_number_test.gleam) verifies sequential and concurrent allocation. - [`dev/invoice_number_dev.gleam`](dev/invoice_number_dev.gleam) is the runnable diff --git a/examples/invoice_number/dev/invoice_number_dev.gleam b/examples/invoice_number/dev/invoice_number_dev.gleam index 3926488..4ec38d9 100644 --- a/examples/invoice_number/dev/invoice_number_dev.gleam +++ b/examples/invoice_number/dev/invoice_number_dev.gleam @@ -29,8 +29,8 @@ type WorkerMessage { WorkerFinished( worker: String, result: Result( - factos_pog.Dispatch(invoice_number.Event), - factos_pog.Error(Nil, Nil), + factos.Dispatch(invoice_number.Event), + factos.Error(Nil, Nil, pog.QueryError), ), ) } @@ -78,7 +78,7 @@ pub fn run() -> Result(ExampleResult, testcontainer_error.Error) { let assert Ok(events) = factos_pog.read_after( connection, - query: factos.AllEvents, + decision_context: factos.AllEvents, after: factos.NoPosition, limit: 100, codec: invoice_number.codec(), @@ -115,7 +115,7 @@ pub fn main() -> Nil { } fn dispatch_invoice_number( - dispatch: factos_pog.Dispatch(invoice_number.Event), + dispatch: factos.Dispatch(invoice_number.Event), ) -> Int { let assert [recorded] = dispatch.events let invoice_number.InvoiceCreated(invoice_number:, invoice_data: _) = @@ -126,7 +126,10 @@ fn dispatch_invoice_number( fn run_concurrent_invoices( connection: pog.Connection, ) -> List( - Result(factos_pog.Dispatch(invoice_number.Event), factos_pog.Error(Nil, Nil)), + Result( + factos.Dispatch(invoice_number.Event), + factos.Error(Nil, Nil, pog.QueryError), + ), ) { let messages = process.new_subject() start_worker( @@ -188,7 +191,10 @@ fn receive_worker_ready( fn receive_worker_finished( messages: process.Subject(WorkerMessage), -) -> Result(factos_pog.Dispatch(invoice_number.Event), factos_pog.Error(Nil, Nil)) { +) -> Result( + factos.Dispatch(invoice_number.Event), + factos.Error(Nil, Nil, pog.QueryError), +) { let assert Ok(message) = process.receive(messages, within: 10_000) let assert WorkerFinished(worker: _, result:) = message result diff --git a/examples/invoice_number/manifest.toml b/examples/invoice_number/manifest.toml index c9aaf3f..163c8b0 100644 --- a/examples/invoice_number/manifest.toml +++ b/examples/invoice_number/manifest.toml @@ -11,8 +11,8 @@ packages = [ { name = "cowl", version = "1.1.0", build_tools = ["gleam"], requirements = ["gleam_stdlib"], otp_app = "cowl", source = "hex", outer_checksum = "7849E7C789D7228243A4253138FC883720A0BB44AEF406102328CADC64C3CA2B" }, { name = "envie", version = "1.2.0", build_tools = ["gleam"], requirements = ["gleam_stdlib"], otp_app = "envie", source = "hex", outer_checksum = "E7EBA39310F32A40BF3EDDD7CD9C7A2BC289909983D357411C22873415BC322A" }, { name = "exception", version = "2.1.1", build_tools = ["gleam"], requirements = ["gleam_stdlib"], otp_app = "exception", source = "hex", outer_checksum = "6BDEA95248093599391C3B5DF1835C5C6A86C353C2F99CE539B450E3432FE117" }, - { name = "factos", version = "1.0.0", build_tools = ["gleam"], requirements = ["gleam_stdlib"], source = "local", path = "../.." }, - { name = "factos_pog", version = "2.0.0", build_tools = ["gleam"], requirements = ["factos", "gleam_erlang", "gleam_json", "gleam_otp", "gleam_stdlib", "pog"], source = "local", path = "../../backends/factos_pog" }, + { name = "factos", version = "2.0.0", build_tools = ["gleam"], requirements = ["gleam_stdlib"], source = "local", path = "../.." }, + { name = "factos_pog", version = "2.0.0", build_tools = ["gleam"], requirements = ["exception", "factos", "gleam_erlang", "gleam_json", "gleam_otp", "gleam_stdlib", "pog"], source = "local", path = "../../backends/factos_pog" }, { name = "filepath", version = "1.1.2", build_tools = ["gleam"], requirements = ["gleam_stdlib"], otp_app = "filepath", source = "hex", outer_checksum = "B06A9AF0BF10E51401D64B98E4B627F1D2E48C154967DA7AF4D0914780A6D40A" }, { name = "gleam_crypto", version = "1.6.0", build_tools = ["gleam"], requirements = ["gleam_stdlib"], otp_app = "gleam_crypto", source = "hex", outer_checksum = "2DE9E4EF53CF6FEE049D4F765731F7178F7A11AEFAE00EEE63BF7536B354AD3F" }, { name = "gleam_erlang", version = "1.3.0", build_tools = ["gleam"], requirements = ["gleam_stdlib"], otp_app = "gleam_erlang", source = "hex", outer_checksum = "1124AD3AA21143E5AF0FC5CF3D9529F6DB8CA03E43A55711B60B6B7B3874375C" }, diff --git a/examples/invoice_number/src/invoice_number.gleam b/examples/invoice_number/src/invoice_number.gleam index 6b875a6..9ff8a8d 100644 --- a/examples/invoice_number/src/invoice_number.gleam +++ b/examples/invoice_number/src/invoice_number.gleam @@ -9,6 +9,7 @@ import factos/factos_pog import gleam/dynamic/decode import gleam/int import gleam/json +import gleam/result import pog pub type InvoiceData { @@ -50,11 +51,11 @@ fn evolve(state: State, event: Event) -> State { } } -pub fn codec() -> factos_pog.EventCodec(Event) { - factos_pog.codec(encode: encode_event, decode: decode_event) +pub fn codec() -> factos.EventCodec(Event, String) { + factos.codec(encode: encode_event, decode: decode_event) } -fn encode_event(event: Event) -> factos_pog.Proposed { +fn encode_event(event: Event) -> factos.Event(String) { let InvoiceCreated(invoice_number:, invoice_data:) = event let InvoiceData(reference:) = invoice_data let data = @@ -62,23 +63,29 @@ fn encode_event(event: Event) -> factos_pog.Proposed { #("invoice_number", json.int(invoice_number)), #("invoice_data", json.object([#("reference", json.string(reference))])), ]) + |> json.to_string - factos_pog.new_proposed( + factos.new_event( type_: factos.event_type("InvoiceCreated"), version: 1, data:, ) - |> factos_pog.with_tags(tags: [ + |> factos.with_tags(tags: [ factos.tag("invoice:" <> int.to_string(invoice_number)), ]) } fn decode_event( - descriptor: factos.EventDescriptor, -) -> Result(decode.Decoder(Event), factos_pog.DecodeError) { - case factos.event_type_name(descriptor.type_), descriptor.version { - "InvoiceCreated", 1 -> Ok(event_decoder()) - _, _ -> Error(factos_pog.UnknownEvent) + stored: factos.Recorded(String), +) -> Result(Event, factos.DecodeError) { + case + factos.event_type_name(stored.descriptor.type_), + stored.descriptor.version + { + "InvoiceCreated", 1 -> + json.parse(stored.event, using: event_decoder()) + |> result.map_error(fn(_) { factos.InvalidData }) + _, _ -> Error(factos.UnknownEvent) } } @@ -97,24 +104,18 @@ pub fn dispatch( connection: pog.Connection, command: Command, event_id: fn() -> String, -) -> Result(factos_pog.Dispatch(Event), factos_pog.Error(Nil, Nil)) { - factos_pog.new_dispatch( +) -> Result(factos.Dispatch(Event), factos.Error(Nil, Nil, pog.QueryError)) { + factos.new_dispatch( connection:, - stream: stream(command), decider: factos.decider(initial: initial(command), decide:, evolve:), codec: codec(), + decision_context: decision_context(command), ) - |> factos_pog.with_query(query(command)) |> factos_pog.dispatch(command, event_id:) } -fn query(_command: Command) -> factos.Query { - factos.query([ - factos.query_item(types: [factos.event_type("InvoiceCreated")], tags: []), +fn decision_context(_command: Command) -> factos.DecisionContext { + factos.Matching(items: [ + factos.item(types: [factos.event_type("InvoiceCreated")], tags: []), ]) } - -fn stream(command: Command) -> String { - let CreateInvoice(invoice_id:, invoice_data: _) = command - "invoice-" <> invoice_id -} diff --git a/examples/opt_in_token/README.md b/examples/opt_in_token/README.md index 5fa344c..c2098df 100644 --- a/examples/opt_in_token/README.md +++ b/examples/opt_in_token/README.md @@ -12,15 +12,16 @@ belong to the same pending request. A token can be used once and expires after ## DCB approach `SignUpInitiated` and `SignUpConfirmed` carry both `email:` and -`otp:` tags. Confirmation queries the conjunction of those tags and folds -the matching facts into one of three useful states: no pending sign-up, unused -token, or already-used token. +`otp:` tags. Confirmation uses a matching decision context that requires +both tags, then folds the selected facts into one of three useful states: no +pending sign-up, unused token, or already-used token. The decider rejects an unknown email/token pair, a replayed token, or an expired token. A valid decision copies the name from the initiation event into -`SignUpConfirmed`. Factos Pog executes the read, decision, and conditional -append in a serializable PostgreSQL transaction, so concurrent confirmations -cannot consume the same token twice. +`SignUpConfirmed`. The shared Factos dispatch contract owns the domain event +codec and result types, while Factos Pog executes the read, decision, and +conditional append in a serializable PostgreSQL transaction. Concurrent +confirmations therefore cannot consume the same token twice. The package demonstrates: @@ -28,12 +29,13 @@ The package demonstrates: - matching a token to its email address; - one-time consumption under sequential and concurrent requests; - a 60-minute validity boundary; -- JSON events with email and token tags. +- JSON event payloads with email and token tags; +- initiation-minute metadata decoding, including invalid-metadata rejection. The source example uses relative `minutesAgo` metadata for illustration. This -implementation stores an absolute `initiated_minute` and supplies -`current_minute` in the command, keeping the decider deterministic across -serializable retries. +implementation stores an absolute `initiated_minute` in Factos event metadata +and supplies `current_minute` in the command, keeping the decider deterministic +across serializable retries. ## Run it @@ -55,7 +57,7 @@ container. No developer-managed database is required. ## Package layout - [`src/opt_in_token.gleam`](src/opt_in_token.gleam) contains the commands, - events, token decision model, DCB queries, codec, and dispatch API. + events, token decision model, DCB decision contexts, codec, and dispatch API. - [`test/opt_in_token_test.gleam`](test/opt_in_token_test.gleam) verifies valid, invalid, expired, replayed, and concurrent confirmations. - [`dev/opt_in_token_dev.gleam`](dev/opt_in_token_dev.gleam) is the runnable diff --git a/examples/opt_in_token/dev/opt_in_token_dev.gleam b/examples/opt_in_token/dev/opt_in_token_dev.gleam index dac4867..85b0364 100644 --- a/examples/opt_in_token/dev/opt_in_token_dev.gleam +++ b/examples/opt_in_token/dev/opt_in_token_dev.gleam @@ -31,8 +31,8 @@ type WorkerMessage { WorkerFinished( worker: String, result: Result( - factos_pog.Dispatch(opt_in_token.Event), - factos_pog.Error(opt_in_token.Error, Nil), + factos.Dispatch(opt_in_token.Event), + factos.Error(opt_in_token.Error, Nil, pog.QueryError), ), ) } @@ -186,7 +186,7 @@ pub fn run() -> Result(ExampleResult, testcontainer_error.Error) { let assert Ok(events) = factos_pog.read_after( connection, - query: factos.AllEvents, + decision_context: factos.AllEvents, after: factos.NoPosition, limit: 100, codec: opt_in_token.codec(), @@ -250,18 +250,18 @@ fn require_initiation( fn require_domain_error( result: Result( - factos_pog.Dispatch(opt_in_token.Event), - factos_pog.Error(opt_in_token.Error, Nil), + factos.Dispatch(opt_in_token.Event), + factos.Error(opt_in_token.Error, Nil, pog.QueryError), ), expected expected: opt_in_token.Error, ) -> Nil { - let assert Error(factos_pog.DomainError(actual)) = result + let assert Error(factos.DomainError(actual)) = result assert actual == expected Nil } fn assert_confirmation( - dispatch: factos_pog.Dispatch(opt_in_token.Event), + dispatch: factos.Dispatch(opt_in_token.Event), email_address email_address: String, otp otp: String, name name: String, @@ -276,8 +276,8 @@ fn run_concurrent_confirmation( connection: pog.Connection, ) -> List( Result( - factos_pog.Dispatch(opt_in_token.Event), - factos_pog.Error(opt_in_token.Error, Nil), + factos.Dispatch(opt_in_token.Event), + factos.Error(opt_in_token.Error, Nil, pog.QueryError), ), ) { let messages = process.new_subject() @@ -340,8 +340,8 @@ fn receive_worker_ready( fn receive_worker_finished( messages: process.Subject(WorkerMessage), ) -> Result( - factos_pog.Dispatch(opt_in_token.Event), - factos_pog.Error(opt_in_token.Error, Nil), + factos.Dispatch(opt_in_token.Event), + factos.Error(opt_in_token.Error, Nil, pog.QueryError), ) { let assert Ok(message) = process.receive(messages, within: 10_000) let assert WorkerFinished(worker: _, result:) = message @@ -350,8 +350,8 @@ fn receive_worker_finished( fn is_accepted( result: Result( - factos_pog.Dispatch(opt_in_token.Event), - factos_pog.Error(opt_in_token.Error, Nil), + factos.Dispatch(opt_in_token.Event), + factos.Error(opt_in_token.Error, Nil, pog.QueryError), ), ) -> Bool { case result { @@ -362,12 +362,12 @@ fn is_accepted( fn is_already_used( result: Result( - factos_pog.Dispatch(opt_in_token.Event), - factos_pog.Error(opt_in_token.Error, Nil), + factos.Dispatch(opt_in_token.Event), + factos.Error(opt_in_token.Error, Nil, pog.QueryError), ), ) -> Bool { case result { - Error(factos_pog.DomainError(opt_in_token.OtpAlreadyUsed)) -> True + Error(factos.DomainError(opt_in_token.OtpAlreadyUsed)) -> True Ok(_) | Error(_) -> False } } diff --git a/examples/opt_in_token/manifest.toml b/examples/opt_in_token/manifest.toml index c9aaf3f..163c8b0 100644 --- a/examples/opt_in_token/manifest.toml +++ b/examples/opt_in_token/manifest.toml @@ -11,8 +11,8 @@ packages = [ { name = "cowl", version = "1.1.0", build_tools = ["gleam"], requirements = ["gleam_stdlib"], otp_app = "cowl", source = "hex", outer_checksum = "7849E7C789D7228243A4253138FC883720A0BB44AEF406102328CADC64C3CA2B" }, { name = "envie", version = "1.2.0", build_tools = ["gleam"], requirements = ["gleam_stdlib"], otp_app = "envie", source = "hex", outer_checksum = "E7EBA39310F32A40BF3EDDD7CD9C7A2BC289909983D357411C22873415BC322A" }, { name = "exception", version = "2.1.1", build_tools = ["gleam"], requirements = ["gleam_stdlib"], otp_app = "exception", source = "hex", outer_checksum = "6BDEA95248093599391C3B5DF1835C5C6A86C353C2F99CE539B450E3432FE117" }, - { name = "factos", version = "1.0.0", build_tools = ["gleam"], requirements = ["gleam_stdlib"], source = "local", path = "../.." }, - { name = "factos_pog", version = "2.0.0", build_tools = ["gleam"], requirements = ["factos", "gleam_erlang", "gleam_json", "gleam_otp", "gleam_stdlib", "pog"], source = "local", path = "../../backends/factos_pog" }, + { name = "factos", version = "2.0.0", build_tools = ["gleam"], requirements = ["gleam_stdlib"], source = "local", path = "../.." }, + { name = "factos_pog", version = "2.0.0", build_tools = ["gleam"], requirements = ["exception", "factos", "gleam_erlang", "gleam_json", "gleam_otp", "gleam_stdlib", "pog"], source = "local", path = "../../backends/factos_pog" }, { name = "filepath", version = "1.1.2", build_tools = ["gleam"], requirements = ["gleam_stdlib"], otp_app = "filepath", source = "hex", outer_checksum = "B06A9AF0BF10E51401D64B98E4B627F1D2E48C154967DA7AF4D0914780A6D40A" }, { name = "gleam_crypto", version = "1.6.0", build_tools = ["gleam"], requirements = ["gleam_stdlib"], otp_app = "gleam_crypto", source = "hex", outer_checksum = "2DE9E4EF53CF6FEE049D4F765731F7178F7A11AEFAE00EEE63BF7536B354AD3F" }, { name = "gleam_erlang", version = "1.3.0", build_tools = ["gleam"], requirements = ["gleam_stdlib"], otp_app = "gleam_erlang", source = "hex", outer_checksum = "1124AD3AA21143E5AF0FC5CF3D9529F6DB8CA03E43A55711B60B6B7B3874375C" }, diff --git a/examples/opt_in_token/src/opt_in_token.gleam b/examples/opt_in_token/src/opt_in_token.gleam index faaea00..b961682 100644 --- a/examples/opt_in_token/src/opt_in_token.gleam +++ b/examples/opt_in_token/src/opt_in_token.gleam @@ -170,11 +170,11 @@ fn evolve(state: State, event: Event) -> State { } } -pub fn codec() -> factos_pog.EventCodec(Event) { - factos_pog.codec(encode: encode_event, decode: decode_event) +pub fn codec() -> factos.EventCodec(Event, String) { + factos.codec(encode: encode_event, decode: decode_event) } -fn encode_event(event: Event) -> factos_pog.Proposed { +fn encode_event(event: Event) -> factos.Event(String) { case event { SignUpInitiated(email_address:, otp:, name:, initiated_minute:) -> proposed_event( @@ -182,7 +182,7 @@ fn encode_event(event: Event) -> factos_pog.Proposed { data: sign_up_data(email_address, otp, name), tags: sign_up_tags(email_address, otp), ) - |> factos_pog.with_metadata( + |> factos.with_metadata( metadata: factos.metadata([ #(initiated_minute_key, int.to_string(initiated_minute)), ]), @@ -215,54 +215,61 @@ fn proposed_event( type_ type_name: String, data data: json.Json, tags tags: List(factos.Tag), -) -> factos_pog.Proposed { - factos_pog.new_proposed( +) -> factos.Event(String) { + factos.new_event( type_: factos.event_type(type_name), version: 1, - data:, + data: json.to_string(data), ) - |> factos_pog.with_tags(tags:) + |> factos.with_tags(tags:) } fn decode_event( - descriptor: factos.EventDescriptor, -) -> Result(decode.Decoder(Event), factos_pog.DecodeError) { - case factos.event_type_name(descriptor.type_), descriptor.version { + stored: factos.Recorded(String), +) -> Result(Event, factos.DecodeError) { + case + factos.event_type_name(stored.descriptor.type_), + stored.descriptor.version + { "SignUpInitiated", 1 -> { use initiated_minute <- result.try(decode_initiated_minute( - descriptor.metadata, + stored.descriptor.metadata, )) - Ok( - sign_up_decoder() - |> decode.map(fn(data) { - SignUpInitiated( - email_address: data.0, - otp: data.1, - name: data.2, - initiated_minute:, - ) - }), + json.parse( + stored.event, + using: sign_up_decoder() + |> decode.map(fn(data) { + SignUpInitiated( + email_address: data.0, + otp: data.1, + name: data.2, + initiated_minute:, + ) + }), ) + |> result.map_error(fn(_) { factos.InvalidData }) } "SignUpConfirmed", 1 -> - Ok( - sign_up_decoder() - |> decode.map(fn(data) { - SignUpConfirmed(email_address: data.0, otp: data.1, name: data.2) - }), + json.parse( + stored.event, + using: sign_up_decoder() + |> decode.map(fn(data) { + SignUpConfirmed(email_address: data.0, otp: data.1, name: data.2) + }), ) - _, _ -> Error(factos_pog.UnknownEvent) + |> result.map_error(fn(_) { factos.InvalidData }) + _, _ -> Error(factos.UnknownEvent) } } fn decode_initiated_minute( metadata: factos.Metadata, -) -> Result(Int, factos_pog.DecodeError) { +) -> Result(Int, factos.DecodeError) { use value <- result.try( factos.metadata_get(metadata, initiated_minute_key) - |> result.replace_error(factos_pog.InvalidData), + |> result.replace_error(factos.InvalidData), ) - int.parse(value) |> result.replace_error(factos_pog.InvalidData) + int.parse(value) |> result.replace_error(factos.InvalidData) } fn sign_up_decoder() -> decode.Decoder(#(String, String, String)) { @@ -276,18 +283,17 @@ pub fn dispatch( connection: pog.Connection, command: Command, event_id: fn() -> String, -) -> Result(factos_pog.Dispatch(Event), factos_pog.Error(Error, Nil)) { - factos_pog.new_dispatch( +) -> Result(factos.Dispatch(Event), factos.Error(Error, Nil, pog.QueryError)) { + factos.new_dispatch( connection:, - stream: stream(command), decider: factos.decider(initial: initial(command), decide:, evolve:), + decision_context: decision_context(command), codec: codec(), ) - |> factos_pog.with_query(query(command)) |> factos_pog.dispatch(command, event_id:) } -fn query(command: Command) -> factos.Query { +fn decision_context(command: Command) -> factos.DecisionContext { let #(email_address, otp) = case command { InitiateSignUp( sign_up_id: _, @@ -301,8 +307,8 @@ fn query(command: Command) -> factos.Query { otp, ) } - factos.query([ - factos.query_item( + factos.Matching(items: [ + factos.item( types: [ factos.event_type("SignUpInitiated"), factos.event_type("SignUpConfirmed"), @@ -314,10 +320,3 @@ fn query(command: Command) -> factos.Query { ), ]) } - -fn stream(command: Command) -> String { - case command { - InitiateSignUp(sign_up_id:, ..) -> "sign-up-" <> sign_up_id - ConfirmSignUp(confirmation_id:, ..) -> "confirmation-" <> confirmation_id - } -} diff --git a/examples/performance/README.md b/examples/performance/README.md index d6e14a1..fd0ecbd 100644 --- a/examples/performance/README.md +++ b/examples/performance/README.md @@ -8,26 +8,27 @@ by those examples. ## What it measures The benchmark starts an isolated PostgreSQL container, installs the Factos Pog -event-store migration, and performs a heterogeneous preload of: +event-store migration, and uses the shared Factos event codec and dispatch +builder with Factos Pog as the PostgreSQL execution backend. Codec payloads are +JSON serialized to strings. It performs a heterogeneous preload of: - 8 concurrent workers; - 20,000 dispatches; - 100 events per dispatch; -- 2,000,000 total events across four event types and 20,000 streams. +- 2,000,000 total events across four event types. -It reports elapsed time, dispatches per second, and events per second, verifies -the preload counts, then runs four matrices: +Each command uses `factos.NoContext` because the benchmark decider derives its +event batch entirely from the command and does not inspect prior facts. It +reports elapsed time, dispatches per second, and events per second, verifies the +preload counts, then runs three matrices: -1. **Event count:** fresh-stream dispatches containing 1, 5, 10, or 100 events. +1. **Event count:** no-context dispatches containing 1, 5, 10, or 100 events. 2. **Worker count:** groups of 1, 2, 4, 8, or 16 concurrent 100-event dispatches. -3. **Stream history:** a fresh stream compared with a stream containing 10,000 - events. -4. **Dispatcher write amplification:** no tags, one tag per event, and one tag - plus one durable outbox effect per event. +3. **Codec write amplification:** no tags compared with one tag per event. The matrix tables report iterations per second, minimum latency, mean latency, -and p99 latency. The benchmark also verifies that the tagged and durable cases -actually persist tag and outbox rows. +and p99 latency. The benchmark also verifies that the tagged case persists tag +rows. ## Run it @@ -50,5 +51,6 @@ are controlled. ## Package layout - [`src/factos_pog_performance.gleam`](src/factos_pog_performance.gleam) - contains the benchmark decider, event codec, worker orchestration, preload, - verification queries, and benchmark matrices. + contains the benchmark decider, shared Factos event codec and dispatch setup, + PostgreSQL-backed worker orchestration, preload verification queries, and + benchmark matrices. diff --git a/examples/performance/manifest.toml b/examples/performance/manifest.toml index 3a6106e..0834d50 100644 --- a/examples/performance/manifest.toml +++ b/examples/performance/manifest.toml @@ -11,8 +11,8 @@ packages = [ { name = "cowl", version = "1.1.0", build_tools = ["gleam"], requirements = ["gleam_stdlib"], otp_app = "cowl", source = "hex", outer_checksum = "7849E7C789D7228243A4253138FC883720A0BB44AEF406102328CADC64C3CA2B" }, { name = "envie", version = "1.2.0", build_tools = ["gleam"], requirements = ["gleam_stdlib"], otp_app = "envie", source = "hex", outer_checksum = "E7EBA39310F32A40BF3EDDD7CD9C7A2BC289909983D357411C22873415BC322A" }, { name = "exception", version = "2.1.1", build_tools = ["gleam"], requirements = ["gleam_stdlib"], otp_app = "exception", source = "hex", outer_checksum = "6BDEA95248093599391C3B5DF1835C5C6A86C353C2F99CE539B450E3432FE117" }, - { name = "factos", version = "1.0.0", build_tools = ["gleam"], requirements = ["gleam_stdlib"], source = "local", path = "../.." }, - { name = "factos_pog", version = "2.0.0", build_tools = ["gleam"], requirements = ["factos", "gleam_erlang", "gleam_json", "gleam_otp", "gleam_stdlib", "pog"], source = "local", path = "../../backends/factos_pog" }, + { name = "factos", version = "2.0.0", build_tools = ["gleam"], requirements = ["gleam_stdlib"], source = "local", path = "../.." }, + { name = "factos_pog", version = "2.0.0", build_tools = ["gleam"], requirements = ["exception", "factos", "gleam_erlang", "gleam_json", "gleam_otp", "gleam_stdlib", "pog"], source = "local", path = "../../backends/factos_pog" }, { name = "filepath", version = "1.1.2", build_tools = ["gleam"], requirements = ["gleam_stdlib"], otp_app = "filepath", source = "hex", outer_checksum = "B06A9AF0BF10E51401D64B98E4B627F1D2E48C154967DA7AF4D0914780A6D40A" }, { name = "gleam_crypto", version = "1.6.0", build_tools = ["gleam"], requirements = ["gleam_stdlib"], otp_app = "gleam_crypto", source = "hex", outer_checksum = "2DE9E4EF53CF6FEE049D4F765731F7178F7A11AEFAE00EEE63BF7536B354AD3F" }, { name = "gleam_erlang", version = "1.3.0", build_tools = ["gleam"], requirements = ["gleam_stdlib"], otp_app = "gleam_erlang", source = "hex", outer_checksum = "1124AD3AA21143E5AF0FC5CF3D9529F6DB8CA03E43A55711B60B6B7B3874375C" }, diff --git a/examples/performance/src/factos_pog_performance.gleam b/examples/performance/src/factos_pog_performance.gleam index 909b1d2..b925769 100644 --- a/examples/performance/src/factos_pog_performance.gleam +++ b/examples/performance/src/factos_pog_performance.gleam @@ -42,7 +42,10 @@ pub type Event { } type WorkerMessage { - WorkerDone(worker: Int, result: Result(Nil, factos_pog.Error(Nil, Nil))) + WorkerDone( + worker: Int, + result: Result(Nil, factos.Error(Nil, Nil, pog.QueryError)), + ) } type BenchmarkJob { @@ -53,19 +56,13 @@ type BenchmarkWorkerStarted { BenchmarkWorkerStarted(jobs: process.Subject(BenchmarkJob)) } -type StreamMode { - FreshStream(prefix: String) - HotStream(name: String) -} - type BenchmarkInput { BenchmarkInput( connection: pog.Connection, decider: factos.Decider(Command, State, Event, Nil), - codec: factos_pog.EventCodec(Event), + codec: factos.EventCodec(Event, String), event_count: Int, worker_count: Int, - stream_mode: StreamMode, ) } @@ -134,11 +131,11 @@ fn evolve(state: State, event: Event) -> State { } } -fn codec(tags: List(factos.Tag)) -> factos_pog.EventCodec(Event) { - factos_pog.codec(encode: fn(event) { encode(event, tags) }, decode:) +fn codec(tags: List(factos.Tag)) -> factos.EventCodec(Event, String) { + factos.codec(encode: fn(event) { encode(event, tags) }, decode:) } -fn encode(event: Event, tags: List(factos.Tag)) -> factos_pog.Proposed { +fn encode(event: Event, tags: List(factos.Tag)) -> factos.Event(String) { let #(type_name, data) = case event { UserRegistered(user_id:) -> #( "performance.user_registered", @@ -152,33 +149,49 @@ fn encode(event: Event, tags: List(factos.Tag)) -> factos_pog.Proposed { UserSuspended(reason:) -> #("performance.user_suspended", reason) } - factos_pog.new_proposed( + factos.new_event( type_: factos.event_type(type_name), version: 1, - data: json.string(data), + data: json.string(data) |> json.to_string, ) - |> factos_pog.with_tags(tags:) + |> factos.with_tags(tags:) } fn decode( - descriptor: factos.EventDescriptor, -) -> Result(decode.Decoder(Event), factos_pog.DecodeError) { - case factos.event_type_name(descriptor.type_), descriptor.version { + stored: factos.Recorded(String), +) -> Result(Event, factos.DecodeError) { + case + factos.event_type_name(stored.descriptor.type_), + stored.descriptor.version + { "performance.user_registered", 1 -> - Ok( - integer_string_decoder() - |> decode.map(fn(user_id) { UserRegistered(user_id:) }), + json.parse( + stored.event, + using: integer_string_decoder() + |> decode.map(fn(user_id) { UserRegistered(user_id:) }), ) + |> result.map_error(fn(_) { factos.InvalidData }) "performance.email_changed", 1 -> - Ok(decode.string |> decode.map(fn(email) { EmailChanged(email:) })) + json.parse( + stored.event, + using: decode.string |> decode.map(fn(email) { EmailChanged(email:) }), + ) + |> result.map_error(fn(_) { factos.InvalidData }) "performance.balance_adjusted", 1 -> - Ok( - integer_string_decoder() - |> decode.map(fn(delta) { BalanceAdjusted(delta:) }), + json.parse( + stored.event, + using: integer_string_decoder() + |> decode.map(fn(delta) { BalanceAdjusted(delta:) }), ) + |> result.map_error(fn(_) { factos.InvalidData }) "performance.user_suspended", 1 -> - Ok(decode.string |> decode.map(fn(reason) { UserSuspended(reason:) })) - _, _ -> Error(factos_pog.UnknownEvent) + json.parse( + stored.event, + using: decode.string + |> decode.map(fn(reason) { UserSuspended(reason:) }), + ) + |> result.map_error(fn(_) { factos.InvalidData }) + _, _ -> Error(factos.UnknownEvent) } } @@ -190,11 +203,11 @@ fn integer_string_decoder() -> decode.Decoder(Int) { } } -fn empty_codec() -> factos_pog.EventCodec(Event) { +fn empty_codec() -> factos.EventCodec(Event, String) { codec([]) } -fn tagged_codec() -> factos_pog.EventCodec(Event) { +fn tagged_codec() -> factos.EventCodec(Event, String) { codec([factos.tag("performance")]) } @@ -249,7 +262,6 @@ fn run() -> Result(Nil, testcontainer_error.Error) { run_event_count_benchmarks(connection, benchmark_decider, benchmark_codec) run_worker_count_benchmarks(connection, benchmark_decider, benchmark_codec) - run_stream_benchmarks(connection, benchmark_decider, benchmark_codec) run_codec_benchmarks(connection, benchmark_decider, benchmark_codec) io.println("benchmark complete") @@ -261,7 +273,7 @@ fn run() -> Result(Nil, testcontainer_error.Error) { fn spawn_workers( connection: pog.Connection, benchmark_decider: factos.Decider(Command, State, Event, Nil), - benchmark_codec: factos_pog.EventCodec(Event), + benchmark_codec: factos.EventCodec(Event, String), worker_messages: process.Subject(WorkerMessage), worker worker: Int, ) -> Nil { @@ -293,24 +305,18 @@ fn spawn_workers( fn run_worker( connection: pog.Connection, benchmark_decider: factos.Decider(Command, State, Event, Nil), - benchmark_codec: factos_pog.EventCodec(Event), + benchmark_codec: factos.EventCodec(Event, String), worker: Int, dispatch_index dispatch_index: Int, -) -> Result(Nil, factos_pog.Error(Nil, Nil)) { +) -> Result(Nil, factos.Error(Nil, Nil, pog.QueryError)) { case dispatch_index >= dispatches_per_worker { True -> Ok(Nil) False -> { - let stream_name = - "performance-" - <> int.to_string(worker) - <> "-" - <> int.to_string(dispatch_index) let batch = { worker - 1 } * dispatches_per_worker + dispatch_index use _ <- result.try(dispatch_once( connection, benchmark_decider, benchmark_codec, - stream_name, batch, events_per_dispatch, )) @@ -340,10 +346,11 @@ fn wait_for_workers( "performance worker " <> int.to_string(worker) <> " failed: " - <> factos_pog.error_to_string( + <> factos.error_to_string( error, fn(_) { "nil" }, fn(_) { "nil" }, + factos_pog.query_error_to_string, ) } } @@ -353,18 +360,16 @@ fn wait_for_workers( fn dispatch_once( connection: pog.Connection, benchmark_decider: factos.Decider(Command, State, Event, Nil), - benchmark_codec: factos_pog.EventCodec(Event), - stream_name: String, + benchmark_codec: factos.EventCodec(Event, String), batch: Int, event_count: Int, -) -> Result(factos_pog.Dispatch(Event), factos_pog.Error(Nil, Nil)) { - factos_pog.new_dispatch( +) -> Result(factos.Dispatch(Event), factos.Error(Nil, Nil, pog.QueryError)) { + factos.new_dispatch( connection:, - stream: stream_name, decider: benchmark_decider, + decision_context: factos.NoContext, codec: benchmark_codec, ) - |> factos_pog.with_retry_attempts(attempts: 100) |> factos_pog.dispatch( AppendBatch(batch:, event_count:), event_id: uuid.v4_string, @@ -374,7 +379,7 @@ fn dispatch_once( fn run_event_count_benchmarks( connection: pog.Connection, benchmark_decider: factos.Decider(Command, State, Event, Nil), - benchmark_codec: factos_pog.EventCodec(Event), + benchmark_codec: factos.EventCodec(Event, String), ) -> Nil { io.println("event count matrix") bench.run( @@ -386,7 +391,6 @@ fn run_event_count_benchmarks( benchmark_codec, 1, 1, - FreshStream(prefix: "performance-event-count-"), ), benchmark_input( "5 events", @@ -395,7 +399,6 @@ fn run_event_count_benchmarks( benchmark_codec, 5, 1, - FreshStream(prefix: "performance-event-count-"), ), benchmark_input( "10 events", @@ -404,7 +407,6 @@ fn run_event_count_benchmarks( benchmark_codec, 10, 1, - FreshStream(prefix: "performance-event-count-"), ), benchmark_input( "100 events", @@ -413,14 +415,10 @@ fn run_event_count_benchmarks( benchmark_codec, 100, 1, - FreshStream(prefix: "performance-event-count-"), ), ], [ - bench.Function( - label: "fresh-stream dispatch", - function: benchmark_dispatch, - ), + bench.Function(label: "no-context dispatch", function: benchmark_dispatch), ], benchmark_options(), ) @@ -431,7 +429,7 @@ fn run_event_count_benchmarks( fn run_worker_count_benchmarks( connection: pog.Connection, benchmark_decider: factos.Decider(Command, State, Event, Nil), - benchmark_codec: factos_pog.EventCodec(Event), + benchmark_codec: factos.EventCodec(Event, String), ) -> Nil { io.println("worker count matrix") bench.run( @@ -443,7 +441,6 @@ fn run_worker_count_benchmarks( benchmark_codec, 100, 1, - FreshStream(prefix: "performance-worker-count-"), ), benchmark_input( "2 workers", @@ -452,7 +449,6 @@ fn run_worker_count_benchmarks( benchmark_codec, 100, 2, - FreshStream(prefix: "performance-worker-count-"), ), benchmark_input( "4 workers", @@ -461,7 +457,6 @@ fn run_worker_count_benchmarks( benchmark_codec, 100, 4, - FreshStream(prefix: "performance-worker-count-"), ), benchmark_input( "8 workers", @@ -470,7 +465,6 @@ fn run_worker_count_benchmarks( benchmark_codec, 100, 8, - FreshStream(prefix: "performance-worker-count-"), ), benchmark_input( "16 workers", @@ -479,7 +473,6 @@ fn run_worker_count_benchmarks( benchmark_codec, 100, 16, - FreshStream(prefix: "performance-worker-count-"), ), ], [ @@ -497,57 +490,10 @@ fn run_worker_count_benchmarks( ) } -fn run_stream_benchmarks( - connection: pog.Connection, - benchmark_decider: factos.Decider(Command, State, Event, Nil), - benchmark_codec: factos_pog.EventCodec(Event), -) -> Nil { - let hot_stream = "performance-hot-10k" - let assert Ok(_) = - dispatch_once( - connection, - benchmark_decider, - benchmark_codec, - hot_stream, - 0, - 10_000, - ) - - io.println("stream history matrix") - bench.run( - [ - benchmark_input( - "fresh stream", - connection, - benchmark_decider, - benchmark_codec, - 100, - 1, - FreshStream(prefix: "performance-stream-comparison-"), - ), - benchmark_input( - "hot stream (10k)", - connection, - benchmark_decider, - benchmark_codec, - 100, - 1, - HotStream(name: hot_stream), - ), - ], - [ - bench.Function(label: "100-event dispatch", function: benchmark_dispatch), - ], - benchmark_options(), - ) - |> bench.table([bench.IPS, bench.Min, bench.Mean, bench.P(99)]) - |> io.println -} - fn run_codec_benchmarks( connection: pog.Connection, benchmark_decider: factos.Decider(Command, State, Event, Nil), - empty_codec empty_codec: factos_pog.EventCodec(Event), + empty_codec empty_codec: factos.EventCodec(Event, String), ) -> Nil { let tagged_codec = tagged_codec() @@ -561,7 +507,6 @@ fn run_codec_benchmarks( empty_codec, 100, 1, - FreshStream(prefix: "performance-empty-codec-"), ), benchmark_input( "one tag per event", @@ -570,7 +515,6 @@ fn run_codec_benchmarks( tagged_codec, 100, 1, - FreshStream(prefix: "performance-tagged-codec-"), ), ], [ @@ -599,10 +543,9 @@ fn benchmark_input( label: String, connection: pog.Connection, benchmark_decider: factos.Decider(Command, State, Event, Nil), - benchmark_codec: factos_pog.EventCodec(Event), + benchmark_codec: factos.EventCodec(Event, String), event_count: Int, worker_count: Int, - stream_mode: StreamMode, ) -> bench.Input(BenchmarkInput) { bench.Input( label:, @@ -612,7 +555,6 @@ fn benchmark_input( codec: benchmark_codec, event_count:, worker_count:, - stream_mode:, ), ) } @@ -631,28 +573,12 @@ fn benchmark_dispatch(input: BenchmarkInput) -> Nil { decider:, codec:, event_count:, - stream_mode:, worker_count: _, ) = input - let assert Ok(_) = - dispatch_once( - connection, - decider, - codec, - benchmark_stream_name(stream_mode), - 0, - event_count, - ) + let assert Ok(_) = dispatch_once(connection, decider, codec, 0, event_count) Nil } -fn benchmark_stream_name(stream_mode: StreamMode) -> String { - case stream_mode { - FreshStream(prefix:) -> prefix <> uuid.v4_string() - HotStream(name:) -> name - } -} - fn setup_concurrent_dispatch( input: BenchmarkInput, ) -> fn(BenchmarkInput) -> Nil { @@ -696,18 +622,10 @@ fn run_benchmark_worker( decider:, codec:, event_count:, - stream_mode:, worker_count: _, ) = input let result = - dispatch_once( - connection, - decider, - codec, - benchmark_stream_name(stream_mode), - 0, - event_count, - ) + dispatch_once(connection, decider, codec, 0, event_count) |> result.map(fn(_) { Nil }) process.send(reply_to, WorkerDone(worker:, result:)) run_benchmark_worker(input, jobs, worker) @@ -741,13 +659,7 @@ fn verify_preload(connection: pog.Connection) -> Nil { #("performance.user_suspended", 500_000), ] - let assert Ok(stream_counts) = - pog.query("select count(distinct stream) from factos_events") - |> pog.returning(int_decoder()) - |> pog.execute(on: connection) - assert stream_counts.rows == [20_000] - - io.println("verified preload: 2000000 events, 500000 per type, 20000 streams") + io.println("verified preload: 2000000 events, 500000 per type") } fn type_count_decoder() -> decode.Decoder(#(String, Int)) { diff --git a/examples/prevent_record_duplication/README.md b/examples/prevent_record_duplication/README.md index 69f321e..e8d5117 100644 --- a/examples/prevent_record_duplication/README.md +++ b/examples/prevent_record_duplication/README.md @@ -12,14 +12,16 @@ most once while retaining control of the domain entity identifier. ## DCB approach Every `OrderPlaced` event is tagged with both `order:` and -`idempotency:`. A placement command queries only -`OrderPlaced` events carrying its idempotency tag. The folded state therefore +`idempotency:`. A placement command selects only +`OrderPlaced` facts carrying its idempotency tag. The folded state therefore answers one question: has this token already been used? -Factos Pog runs that query, the pure decision, and the conditional append in a -serializable PostgreSQL transaction. When concurrent requests use the same -token, only one can append `OrderPlaced`; the other retries against the committed -fact and returns `Resubmission`. +Factos supplies the store-independent codec, dispatch, and error contract; the +codec serializes each JSON payload to a string for PostgreSQL storage. Factos Pog +executes that dispatch by reading the decision context, running the pure +decision, and conditionally appending in a serializable PostgreSQL transaction. +When concurrent requests use the same token, only one can append `OrderPlaced`; +the other retries against the committed fact and returns `Resubmission`. The order ID remains independent from the client-provided idempotency token. No token table, pre-issued server token, or read model is required. @@ -50,8 +52,8 @@ is required. ## Package layout - [`src/prevent_record_duplication.gleam`](src/prevent_record_duplication.gleam) - contains the command, event, idempotency decider, DCB query, codec, and - dispatch API. + contains the command, event, idempotency decider, DCB decision context, shared + Factos codec, and PostgreSQL-backed dispatch API. - [`test/prevent_record_duplication_test.gleam`](test/prevent_record_duplication_test.gleam) verifies new, repeated, and concurrent token submissions. - [`dev/prevent_record_duplication_dev.gleam`](dev/prevent_record_duplication_dev.gleam) diff --git a/examples/prevent_record_duplication/dev/prevent_record_duplication_dev.gleam b/examples/prevent_record_duplication/dev/prevent_record_duplication_dev.gleam index e7388a7..24d8c31 100644 --- a/examples/prevent_record_duplication/dev/prevent_record_duplication_dev.gleam +++ b/examples/prevent_record_duplication/dev/prevent_record_duplication_dev.gleam @@ -30,8 +30,8 @@ type WorkerMessage { WorkerFinished( worker: String, result: Result( - factos_pog.Dispatch(prevent_record_duplication.Event), - factos_pog.Error(prevent_record_duplication.Error, Nil), + factos.Dispatch(prevent_record_duplication.Event), + factos.Error(prevent_record_duplication.Error, Nil, pog.QueryError), ), ) } @@ -52,9 +52,7 @@ pub fn run() -> Result(ExampleResult, testcontainer_error.Error) { ), uuid.v4_string, ) - let assert Error(factos_pog.DomainError( - prevent_record_duplication.Resubmission, - )) = + let assert Error(factos.DomainError(prevent_record_duplication.Resubmission)) = prevent_record_duplication.dispatch( connection, prevent_record_duplication.PlaceOrder( @@ -84,7 +82,7 @@ pub fn run() -> Result(ExampleResult, testcontainer_error.Error) { let assert Ok(stored_events) = factos_pog.read_after( connection, - query: factos.AllEvents, + decision_context: factos.AllEvents, after: factos.NoPosition, limit: 10, codec: prevent_record_duplication.codec(), @@ -124,8 +122,8 @@ fn run_concurrent_scenario( connection: pog.Connection, ) -> List( Result( - factos_pog.Dispatch(prevent_record_duplication.Event), - factos_pog.Error(prevent_record_duplication.Error, Nil), + factos.Dispatch(prevent_record_duplication.Event), + factos.Error(prevent_record_duplication.Error, Nil, pog.QueryError), ), ) { let messages = process.new_subject() @@ -176,8 +174,8 @@ fn receive_worker_ready( fn receive_worker_finished( messages: process.Subject(WorkerMessage), ) -> Result( - factos_pog.Dispatch(prevent_record_duplication.Event), - factos_pog.Error(prevent_record_duplication.Error, Nil), + factos.Dispatch(prevent_record_duplication.Event), + factos.Error(prevent_record_duplication.Error, Nil, pog.QueryError), ) { let assert Ok(message) = process.receive(messages, within: 10_000) let assert WorkerFinished(worker: _, result:) = message @@ -186,8 +184,8 @@ fn receive_worker_finished( fn is_accepted( result: Result( - factos_pog.Dispatch(prevent_record_duplication.Event), - factos_pog.Error(prevent_record_duplication.Error, Nil), + factos.Dispatch(prevent_record_duplication.Event), + factos.Error(prevent_record_duplication.Error, Nil, pog.QueryError), ), ) -> Bool { case result { @@ -198,13 +196,12 @@ fn is_accepted( fn is_resubmission( result: Result( - factos_pog.Dispatch(prevent_record_duplication.Event), - factos_pog.Error(prevent_record_duplication.Error, Nil), + factos.Dispatch(prevent_record_duplication.Event), + factos.Error(prevent_record_duplication.Error, Nil, pog.QueryError), ), ) -> Bool { case result { - Error(factos_pog.DomainError(prevent_record_duplication.Resubmission)) -> - True + Error(factos.DomainError(prevent_record_duplication.Resubmission)) -> True Ok(_) | Error(_) -> False } } diff --git a/examples/prevent_record_duplication/manifest.toml b/examples/prevent_record_duplication/manifest.toml index c9aaf3f..163c8b0 100644 --- a/examples/prevent_record_duplication/manifest.toml +++ b/examples/prevent_record_duplication/manifest.toml @@ -11,8 +11,8 @@ packages = [ { name = "cowl", version = "1.1.0", build_tools = ["gleam"], requirements = ["gleam_stdlib"], otp_app = "cowl", source = "hex", outer_checksum = "7849E7C789D7228243A4253138FC883720A0BB44AEF406102328CADC64C3CA2B" }, { name = "envie", version = "1.2.0", build_tools = ["gleam"], requirements = ["gleam_stdlib"], otp_app = "envie", source = "hex", outer_checksum = "E7EBA39310F32A40BF3EDDD7CD9C7A2BC289909983D357411C22873415BC322A" }, { name = "exception", version = "2.1.1", build_tools = ["gleam"], requirements = ["gleam_stdlib"], otp_app = "exception", source = "hex", outer_checksum = "6BDEA95248093599391C3B5DF1835C5C6A86C353C2F99CE539B450E3432FE117" }, - { name = "factos", version = "1.0.0", build_tools = ["gleam"], requirements = ["gleam_stdlib"], source = "local", path = "../.." }, - { name = "factos_pog", version = "2.0.0", build_tools = ["gleam"], requirements = ["factos", "gleam_erlang", "gleam_json", "gleam_otp", "gleam_stdlib", "pog"], source = "local", path = "../../backends/factos_pog" }, + { name = "factos", version = "2.0.0", build_tools = ["gleam"], requirements = ["gleam_stdlib"], source = "local", path = "../.." }, + { name = "factos_pog", version = "2.0.0", build_tools = ["gleam"], requirements = ["exception", "factos", "gleam_erlang", "gleam_json", "gleam_otp", "gleam_stdlib", "pog"], source = "local", path = "../../backends/factos_pog" }, { name = "filepath", version = "1.1.2", build_tools = ["gleam"], requirements = ["gleam_stdlib"], otp_app = "filepath", source = "hex", outer_checksum = "B06A9AF0BF10E51401D64B98E4B627F1D2E48C154967DA7AF4D0914780A6D40A" }, { name = "gleam_crypto", version = "1.6.0", build_tools = ["gleam"], requirements = ["gleam_stdlib"], otp_app = "gleam_crypto", source = "hex", outer_checksum = "2DE9E4EF53CF6FEE049D4F765731F7178F7A11AEFAE00EEE63BF7536B354AD3F" }, { name = "gleam_erlang", version = "1.3.0", build_tools = ["gleam"], requirements = ["gleam_stdlib"], otp_app = "gleam_erlang", source = "hex", outer_checksum = "1124AD3AA21143E5AF0FC5CF3D9529F6DB8CA03E43A55711B60B6B7B3874375C" }, diff --git a/examples/prevent_record_duplication/src/prevent_record_duplication.gleam b/examples/prevent_record_duplication/src/prevent_record_duplication.gleam index 82983d9..981cac8 100644 --- a/examples/prevent_record_duplication/src/prevent_record_duplication.gleam +++ b/examples/prevent_record_duplication/src/prevent_record_duplication.gleam @@ -8,6 +8,7 @@ import factos import factos/factos_pog import gleam/dynamic/decode import gleam/json +import gleam/result import pog pub type Command { @@ -50,35 +51,37 @@ fn evolve(state: State, event: Event) -> State { } } -pub fn codec() -> factos_pog.EventCodec(Event) { - factos_pog.codec(encode: encode_event, decode: decode_event) +pub fn codec() -> factos.EventCodec(Event, String) { + factos.codec(encode: encode_event, decode: decode_event) } -fn encode_event(event: Event) -> factos_pog.Proposed { +fn encode_event(event: Event) -> factos.Event(String) { let OrderPlaced(order_id:, idempotency_token:) = event let data = json.object([ #("order_id", json.string(order_id)), #("idempotency_token", json.string(idempotency_token)), ]) + |> json.to_string - factos_pog.new_proposed( - type_: factos.event_type("OrderPlaced"), - version: 1, - data:, - ) - |> factos_pog.with_tags(tags: [ + factos.new_event(type_: factos.event_type("OrderPlaced"), version: 1, data:) + |> factos.with_tags(tags: [ factos.tag("order:" <> order_id), factos.tag("idempotency:" <> idempotency_token), ]) } fn decode_event( - descriptor: factos.EventDescriptor, -) -> Result(decode.Decoder(Event), factos_pog.DecodeError) { - case factos.event_type_name(descriptor.type_), descriptor.version { - "OrderPlaced", 1 -> Ok(event_decoder()) - _, _ -> Error(factos_pog.UnknownEvent) + stored: factos.Recorded(String), +) -> Result(Event, factos.DecodeError) { + case + factos.event_type_name(stored.descriptor.type_), + stored.descriptor.version + { + "OrderPlaced", 1 -> + json.parse(stored.event, using: event_decoder()) + |> result.map_error(fn(_) { factos.InvalidData }) + _, _ -> Error(factos.UnknownEvent) } } @@ -92,27 +95,21 @@ pub fn dispatch( connection: pog.Connection, command: Command, event_id: fn() -> String, -) -> Result(factos_pog.Dispatch(Event), factos_pog.Error(Error, Nil)) { - factos_pog.new_dispatch( +) -> Result(factos.Dispatch(Event), factos.Error(Error, Nil, pog.QueryError)) { + factos.new_dispatch( connection:, - stream: stream(command), decider: factos.decider(initial: initial(command), decide:, evolve:), + decision_context: decision_context(command), codec: codec(), ) - |> factos_pog.with_query(query(command)) |> factos_pog.dispatch(command, event_id:) } -fn query(command: Command) -> factos.Query { +fn decision_context(command: Command) -> factos.DecisionContext { let PlaceOrder(order_id: _, idempotency_token:) = command - factos.query([ - factos.query_item(types: [factos.event_type("OrderPlaced")], tags: [ + factos.Matching([ + factos.item(types: [factos.event_type("OrderPlaced")], tags: [ factos.tag("idempotency:" <> idempotency_token), ]), ]) } - -fn stream(command: Command) -> String { - let PlaceOrder(order_id:, idempotency_token: _) = command - "order-" <> order_id -} diff --git a/examples/unique_username/README.md b/examples/unique_username/README.md index cb0fc37..52dc587 100644 --- a/examples/unique_username/README.md +++ b/examples/unique_username/README.md @@ -15,11 +15,12 @@ Every fact that affects a claim is tagged with the username: `AccountRegistered` and `AccountClosed` carry one `username:` tag, while `UsernameChanged` carries tags for both the old and new values. -Registration queries the complete history for the requested tag and folds it -into `Available`, `Claimed`, or `RetainedUntil`. Factos Pog runs that read, the -pure decision, and the conditional append in a serializable PostgreSQL -transaction. Concurrent registrations for one username therefore produce one -accepted registration and one `UsernameClaimed` result. +Registration selects the complete history for the requested tag and folds it +into `Available`, `Claimed`, or `RetainedUntil`. The shared Factos dispatch +builder describes that decision; the `factos_pog` backend reads the context and +conditionally appends in a serializable PostgreSQL transaction. Concurrent +registrations for one username therefore produce one accepted registration and +one `UsernameClaimed` result. The package demonstrates: @@ -28,12 +29,18 @@ The package demonstrates: - release after account closure; - moving a claim from an old username to a new username; - three-day retention for closed or changed usernames; -- JSON events with claim tags. +- JSON event payloads with claim tags and recorded-day metadata. + +The codec produces shared `factos.Event(String)` values, serializing JSON before +the PostgreSQL backend stores it as JSONB. Its decoder receives +`factos.Recorded(String)`, validates the event descriptor and payload, and +recovers `recorded_day` from metadata for closures and username changes. The source example uses relative `daysAgo` metadata for illustration. This implementation stores an absolute `recorded_day` and supplies `current_day` in the registration command, keeping the decider deterministic across serializable -retries. +retries. Missing or invalid recorded-day metadata and malformed JSON payloads +decode as `factos.InvalidData`. The example deliberately tags the raw username. A production system should normalize usernames before deciding uniqueness and may hash tag values when the @@ -59,7 +66,7 @@ container. No developer-managed database is required. ## Package layout - [`src/unique_username.gleam`](src/unique_username.gleam) contains the commands, - events, claim decision model, DCB queries, codec, and dispatch API. + events, claim decision model, DCB decision contexts, codec, and dispatch API. - [`test/unique_username_test.gleam`](test/unique_username_test.gleam) verifies registration, release, retention, username changes, and concurrency. - [`dev/unique_username_dev.gleam`](dev/unique_username_dev.gleam) is the diff --git a/examples/unique_username/dev/unique_username_dev.gleam b/examples/unique_username/dev/unique_username_dev.gleam index a134432..0ba0e91 100644 --- a/examples/unique_username/dev/unique_username_dev.gleam +++ b/examples/unique_username/dev/unique_username_dev.gleam @@ -32,8 +32,8 @@ type WorkerMessage { WorkerFinished( worker: String, result: Result( - factos_pog.Dispatch(unique_username.Event), - factos_pog.Error(unique_username.Error, Nil), + factos.Dispatch(unique_username.Event), + factos.Error(unique_username.Error, Nil, pog.QueryError), ), ) } @@ -150,7 +150,7 @@ pub fn run() -> Result(ExampleResult, testcontainer_error.Error) { let assert Ok(events) = factos_pog.read_after( connection, - query: factos.AllEvents, + decision_context: factos.AllEvents, after: factos.NoPosition, limit: 100, codec: unique_username.codec(), @@ -209,12 +209,12 @@ fn require_registration( fn require_claimed( result: Result( - factos_pog.Dispatch(unique_username.Event), - factos_pog.Error(unique_username.Error, Nil), + factos.Dispatch(unique_username.Event), + factos.Error(unique_username.Error, Nil, pog.QueryError), ), username username: String, ) -> Nil { - let assert Error(factos_pog.DomainError(unique_username.UsernameClaimed( + let assert Error(factos.DomainError(unique_username.UsernameClaimed( username: claimed_username, ))) = result assert claimed_username == username @@ -225,8 +225,8 @@ fn run_concurrent_claim( connection: pog.Connection, ) -> List( Result( - factos_pog.Dispatch(unique_username.Event), - factos_pog.Error(unique_username.Error, Nil), + factos.Dispatch(unique_username.Event), + factos.Error(unique_username.Error, Nil, pog.QueryError), ), ) { let messages = process.new_subject() @@ -278,8 +278,8 @@ fn receive_worker_ready( fn receive_worker_finished( messages: process.Subject(WorkerMessage), ) -> Result( - factos_pog.Dispatch(unique_username.Event), - factos_pog.Error(unique_username.Error, Nil), + factos.Dispatch(unique_username.Event), + factos.Error(unique_username.Error, Nil, pog.QueryError), ) { let assert Ok(message) = process.receive(messages, within: 10_000) let assert WorkerFinished(worker: _, result:) = message @@ -288,8 +288,8 @@ fn receive_worker_finished( fn is_accepted( result: Result( - factos_pog.Dispatch(unique_username.Event), - factos_pog.Error(unique_username.Error, Nil), + factos.Dispatch(unique_username.Event), + factos.Error(unique_username.Error, Nil, pog.QueryError), ), ) -> Bool { case result { @@ -300,14 +300,13 @@ fn is_accepted( fn is_claimed( result: Result( - factos_pog.Dispatch(unique_username.Event), - factos_pog.Error(unique_username.Error, Nil), + factos.Dispatch(unique_username.Event), + factos.Error(unique_username.Error, Nil, pog.QueryError), ), ) -> Bool { case result { - Error(factos_pog.DomainError(unique_username.UsernameClaimed( - username: "raced", - ))) -> True + Error(factos.DomainError(unique_username.UsernameClaimed(username: "raced"))) -> + True Ok(_) | Error(_) -> False } } diff --git a/examples/unique_username/manifest.toml b/examples/unique_username/manifest.toml index c9aaf3f..163c8b0 100644 --- a/examples/unique_username/manifest.toml +++ b/examples/unique_username/manifest.toml @@ -11,8 +11,8 @@ packages = [ { name = "cowl", version = "1.1.0", build_tools = ["gleam"], requirements = ["gleam_stdlib"], otp_app = "cowl", source = "hex", outer_checksum = "7849E7C789D7228243A4253138FC883720A0BB44AEF406102328CADC64C3CA2B" }, { name = "envie", version = "1.2.0", build_tools = ["gleam"], requirements = ["gleam_stdlib"], otp_app = "envie", source = "hex", outer_checksum = "E7EBA39310F32A40BF3EDDD7CD9C7A2BC289909983D357411C22873415BC322A" }, { name = "exception", version = "2.1.1", build_tools = ["gleam"], requirements = ["gleam_stdlib"], otp_app = "exception", source = "hex", outer_checksum = "6BDEA95248093599391C3B5DF1835C5C6A86C353C2F99CE539B450E3432FE117" }, - { name = "factos", version = "1.0.0", build_tools = ["gleam"], requirements = ["gleam_stdlib"], source = "local", path = "../.." }, - { name = "factos_pog", version = "2.0.0", build_tools = ["gleam"], requirements = ["factos", "gleam_erlang", "gleam_json", "gleam_otp", "gleam_stdlib", "pog"], source = "local", path = "../../backends/factos_pog" }, + { name = "factos", version = "2.0.0", build_tools = ["gleam"], requirements = ["gleam_stdlib"], source = "local", path = "../.." }, + { name = "factos_pog", version = "2.0.0", build_tools = ["gleam"], requirements = ["exception", "factos", "gleam_erlang", "gleam_json", "gleam_otp", "gleam_stdlib", "pog"], source = "local", path = "../../backends/factos_pog" }, { name = "filepath", version = "1.1.2", build_tools = ["gleam"], requirements = ["gleam_stdlib"], otp_app = "filepath", source = "hex", outer_checksum = "B06A9AF0BF10E51401D64B98E4B627F1D2E48C154967DA7AF4D0914780A6D40A" }, { name = "gleam_crypto", version = "1.6.0", build_tools = ["gleam"], requirements = ["gleam_stdlib"], otp_app = "gleam_crypto", source = "hex", outer_checksum = "2DE9E4EF53CF6FEE049D4F765731F7178F7A11AEFAE00EEE63BF7536B354AD3F" }, { name = "gleam_erlang", version = "1.3.0", build_tools = ["gleam"], requirements = ["gleam_stdlib"], otp_app = "gleam_erlang", source = "hex", outer_checksum = "1124AD3AA21143E5AF0FC5CF3D9529F6DB8CA03E43A55711B60B6B7B3874375C" }, diff --git a/examples/unique_username/src/unique_username.gleam b/examples/unique_username/src/unique_username.gleam index e58e582..9f03bfd 100644 --- a/examples/unique_username/src/unique_username.gleam +++ b/examples/unique_username/src/unique_username.gleam @@ -132,11 +132,11 @@ fn evolve(state: State, event: Event) -> State { } } -pub fn codec() -> factos_pog.EventCodec(Event) { - factos_pog.codec(encode: encode_event, decode: decode_event) +pub fn codec() -> factos.EventCodec(Event, String) { + factos.codec(encode: encode_event, decode: decode_event) } -fn encode_event(event: Event) -> factos_pog.Proposed { +fn encode_event(event: Event) -> factos.Event(String) { case event { AccountRegistered(username:) -> proposed_event( @@ -150,7 +150,7 @@ fn encode_event(event: Event) -> factos_pog.Proposed { data: json.object([#("username", json.string(username))]), tags: [factos.tag("username:" <> username)], ) - |> factos_pog.with_metadata(metadata: recorded_day_metadata(recorded_day)) + |> factos.with_metadata(metadata: recorded_day_metadata(recorded_day)) UsernameChanged(old_username:, new_username:, recorded_day:) -> proposed_event( type_: "UsernameChanged", @@ -163,7 +163,7 @@ fn encode_event(event: Event) -> factos_pog.Proposed { factos.tag("username:" <> new_username), ], ) - |> factos_pog.with_metadata(metadata: recorded_day_metadata(recorded_day)) + |> factos.with_metadata(metadata: recorded_day_metadata(recorded_day)) } } @@ -171,13 +171,13 @@ fn proposed_event( type_ type_name: String, data data: json.Json, tags tags: List(factos.Tag), -) -> factos_pog.Proposed { - factos_pog.new_proposed( +) -> factos.Event(String) { + factos.new_event( type_: factos.event_type(type_name), version: 1, - data:, + data: json.to_string(data), ) - |> factos_pog.with_tags(tags:) + |> factos.with_tags(tags:) } fn recorded_day_metadata(recorded_day: Int) -> factos.Metadata { @@ -185,46 +185,59 @@ fn recorded_day_metadata(recorded_day: Int) -> factos.Metadata { } fn decode_event( - descriptor: factos.EventDescriptor, -) -> Result(decode.Decoder(Event), factos_pog.DecodeError) { - case factos.event_type_name(descriptor.type_), descriptor.version { + stored: factos.Recorded(String), +) -> Result(Event, factos.DecodeError) { + case + factos.event_type_name(stored.descriptor.type_), + stored.descriptor.version + { "AccountRegistered", 1 -> - Ok( - username_decoder() - |> decode.map(fn(username) { AccountRegistered(username:) }), + json.parse( + stored.event, + using: username_decoder() + |> decode.map(fn(username) { AccountRegistered(username:) }), ) + |> result.map_error(fn(_) { factos.InvalidData }) "AccountClosed", 1 -> { - use recorded_day <- result.try(decode_recorded_day(descriptor.metadata)) - Ok( - username_decoder() - |> decode.map(fn(username) { AccountClosed(username:, recorded_day:) }), + use recorded_day <- result.try(decode_recorded_day( + stored.descriptor.metadata, + )) + json.parse( + stored.event, + using: username_decoder() + |> decode.map(fn(username) { AccountClosed(username:, recorded_day:) }), ) + |> result.map_error(fn(_) { factos.InvalidData }) } "UsernameChanged", 1 -> { - use recorded_day <- result.try(decode_recorded_day(descriptor.metadata)) - Ok( - changed_names_decoder() - |> decode.map(fn(names) { - UsernameChanged( - old_username: names.0, - new_username: names.1, - recorded_day:, - ) - }), + use recorded_day <- result.try(decode_recorded_day( + stored.descriptor.metadata, + )) + json.parse( + stored.event, + using: changed_names_decoder() + |> decode.map(fn(names) { + UsernameChanged( + old_username: names.0, + new_username: names.1, + recorded_day:, + ) + }), ) + |> result.map_error(fn(_) { factos.InvalidData }) } - _, _ -> Error(factos_pog.UnknownEvent) + _, _ -> Error(factos.UnknownEvent) } } fn decode_recorded_day( metadata: factos.Metadata, -) -> Result(Int, factos_pog.DecodeError) { +) -> Result(Int, factos.DecodeError) { use value <- result.try( factos.metadata_get(metadata, recorded_day_key) - |> result.replace_error(factos_pog.InvalidData), + |> result.replace_error(factos.InvalidData), ) - int.parse(value) |> result.replace_error(factos_pog.InvalidData) + int.parse(value) |> result.replace_error(factos.InvalidData) } fn username_decoder() -> decode.Decoder(String) { @@ -242,23 +255,22 @@ pub fn dispatch( connection: pog.Connection, command: Command, event_id: fn() -> String, -) -> Result(factos_pog.Dispatch(Event), factos_pog.Error(Error, Nil)) { - factos_pog.new_dispatch( +) -> Result(factos.Dispatch(Event), factos.Error(Error, Nil, pog.QueryError)) { + factos.new_dispatch( connection:, - stream: stream(command), decider: factos.decider(initial: initial(command), decide:, evolve:), + decision_context: query(command), codec: codec(), ) - |> factos_pog.with_query(query(command)) |> factos_pog.dispatch(command, event_id:) } -fn query(command: Command) -> factos.Query { +fn query(command: Command) -> factos.DecisionContext { case command { RegisterAccount(account_id: _, username:, current_day: _) | RecordAccountClosed(account_id: _, username:, recorded_day: _) -> - factos.query([ - factos.query_item( + factos.Matching(items: [ + factos.item( types: [ factos.event_type("AccountRegistered"), factos.event_type("AccountClosed"), @@ -273,8 +285,8 @@ fn query(command: Command) -> factos.Query { new_username:, recorded_day: _, ) -> - factos.query([ - factos.query_item( + factos.Matching(items: [ + factos.item( types: [ factos.event_type("AccountRegistered"), factos.event_type("AccountClosed"), @@ -282,7 +294,7 @@ fn query(command: Command) -> factos.Query { ], tags: [factos.tag("username:" <> old_username)], ), - factos.query_item( + factos.item( types: [ factos.event_type("AccountRegistered"), factos.event_type("AccountClosed"), @@ -293,17 +305,3 @@ fn query(command: Command) -> factos.Query { ]) } } - -fn stream(command: Command) -> String { - let account_id = case command { - RegisterAccount(account_id:, username: _, current_day: _) - | RecordAccountClosed(account_id:, username: _, recorded_day: _) - | RecordUsernameChanged( - account_id:, - old_username: _, - new_username: _, - recorded_day: _, - ) -> account_id - } - "account-" <> account_id -} diff --git a/gleam.toml b/gleam.toml index 0ec3fec..e5860c0 100644 --- a/gleam.toml +++ b/gleam.toml @@ -1,5 +1,5 @@ name = "factos" -version = "1.0.0" +version = "2.0.0" description = "Store-independent context-first Event Sourcing primitives for Gleam." licences = ["MIT"] links = [ diff --git a/manifest.toml b/manifest.toml index 1fc09fd..b93d518 100644 --- a/manifest.toml +++ b/manifest.toml @@ -7,7 +7,7 @@ # You should check this file into your source control repository. packages = [ - { name = "gleam_stdlib", version = "1.0.3", build_tools = ["gleam"], requirements = [], otp_app = "gleam_stdlib", source = "hex", outer_checksum = "1F543AFBA5D33DA493E6087F4E4C4F20D899411343512686C98A8ABB2963CF22" }, + { name = "gleam_stdlib", version = "1.0.5", build_tools = ["gleam"], requirements = [], otp_app = "gleam_stdlib", source = "hex", outer_checksum = "CEE5B6C076A85B45F60C585F4316C63EC8B7127C119D5738C3958A9C4D50404E" }, { name = "gleeunit", version = "1.11.0", build_tools = ["gleam"], requirements = ["gleam_stdlib"], otp_app = "gleeunit", source = "hex", outer_checksum = "EC31ABA74256AEA531EDF8169931D775BBB384FED0A8A1BDC4DD9354E3E21826" }, ] diff --git a/src/factos.gleam b/src/factos.gleam index 7e36498..9170bf5 100644 --- a/src/factos.gleam +++ b/src/factos.gleam @@ -2,18 +2,18 @@ //// //// Factos keeps the domain model in the application. The core module models //// facts, command contexts, pure decision components, and pure views. Concrete -//// storage concerns live in backend packages such as `factos_sqlight` and -//// `factos_kurrentdb_erlang`. +//// storage concerns live in backend packages such as `factos_pog` and +//// `factos_sqlight`. //// //// This package follows a context-first reading of Event Sourcing: accepted //// facts are the authoritative state of the system, and the facts relevant to a -//// command are considered before new facts are accepted. Aggregates, stream-per- -//// object storage, CQRS, projections, and message brokers are implementation -//// choices rather than prerequisites. +//// command are considered before new facts are accepted. Aggregates, CQRS, +//// projections, and message brokers are implementation choices rather than +//// prerequisites. //// //// The central flow is: //// -//// 1. Select a command context with `Query`. +//// 1. Select a `DecisionContext` for the command. //// 2. Fold the matching recorded events into a temporary decision state. //// 3. Run a pure `Decider`. //// 4. Append the produced facts only if the context is still stable. @@ -21,15 +21,168 @@ //// Backends implement the storage-specific parts of that flow. This module keeps //// the shared types and pure computations small and portable. +import gleam/int import gleam/list -import gleam/option.{type Option, None, Some} import gleam/result +pub type EventCodec(event, encoded_event) { + EventCodec( + encode: fn(event) -> Event(encoded_event), + decode: fn(Recorded(encoded_event)) -> Result(event, DecodeError), + ) +} + +/// Create an event codec. +pub fn codec( + encode encode: fn(event) -> Event(encoded_event), + decode decode: fn(Recorded(encoded_event)) -> Result(event, DecodeError), +) -> EventCodec(event, encoded_event) { + EventCodec(encode:, decode:) +} + +pub type DecodeError { + /// The codec does not support the stored event type and version. + UnknownEvent + + /// The stored byte payload is invalid for the selected event. + InvalidData +} + +pub type Subscription(event, subscription_error, connection) { + Subscription( + decision_context: DecisionContext, + consistency: SubscriptionConsistency, + handle: fn(connection, Recorded(event)) -> Result(Nil, subscription_error), + ) +} + +pub type Error(domain_error, subscription_error, store_error) { + /// The decider rejected the command with a domain error. + DomainError(domain_error) + + /// A strong subscription callback rejected an event. + SubscriptionError(error: subscription_error) + + /// The backend store returned an error. + StoreError(store_error) + + /// The command context changed before its events could be appended. + AppendConditionFailed(AppendCondition) + + /// A stored event could not be decoded by the application codec. + DecodeError(DecodeError) +} + +pub type SubscriptionConsistency { + /// Start one asynchronous worker after a successful final commit. + FireAndForget + /// Run matching callbacks inside the dispatch transaction. + StrongConsistency +} + +/// Configure a dispatch-bound subscription. +/// +/// The decision context filters only records accepted by the dispatch carrying +/// this subscription. Strong callbacks share the dispatch transaction. +/// Fire-and-forget callbacks start asynchronously only after a successful +/// commit. +pub fn new_subscription( + decision_context decision_context: DecisionContext, + consistency consistency: SubscriptionConsistency, + handle handle: fn(connection, Recorded(event)) -> + Result(Nil, subscription_error), +) -> Subscription(event, subscription_error, connection) { + Subscription(decision_context:, consistency:, handle:) +} + +pub type DispatchBuilder( + command, + state, + event, + encoded_event, + domain_error, + subscription_error, + connection, +) { + DispatchBuilder( + connection: connection, + decision_context: DecisionContext, + decider: Decider(command, state, event, domain_error), + codec: EventCodec(event, encoded_event), + retry_attempts: Int, + subscriptions: List(Subscription(event, subscription_error, connection)), + ) +} + +pub fn new_dispatch( + connection connection: connection, + decision_context decision_context: DecisionContext, + decider decider: Decider(command, state, event, domain_error), + codec codec: EventCodec(event, encoded_event), +) -> DispatchBuilder( + command, + state, + event, + encoded_event, + domain_error, + subscription_error, + connection, +) { + DispatchBuilder( + connection:, + decision_context:, + decider:, + codec:, + retry_attempts: 5, + subscriptions: [], + ) +} + +pub fn with_subscriptions( + builder: DispatchBuilder( + command, + state, + event, + encoded_event, + domain_error, + subscription_error, + connection, + ), + subscriptions subscriptions: List( + Subscription(event, subscription_error, connection), + ), +) -> DispatchBuilder( + command, + state, + event, + encoded_event, + domain_error, + subscription_error, + connection, +) { + DispatchBuilder(..builder, subscriptions:) +} + +pub fn with_retry_attempts( + builder: DispatchBuilder( + command, + state, + event, + encoded_event, + domain_error, + subscription_error, + connection, + ), + attempts attempts: Int, +) { + DispatchBuilder(..builder, retry_attempts: int.max(attempts, 1)) +} + /// A store-visible event type name. /// -/// Event types are part of the query contract. A backend may use them for -/// efficient context reads, and applications should keep names stable enough -/// for stored history to remain decodable. +/// Event types are part of the decision-context selection contract. A backend +/// may use them for efficient context reads, and applications should keep names +/// stable enough for stored history to remain decodable. pub opaque type EventType { EventType(String) } @@ -37,7 +190,7 @@ pub opaque type EventType { /// A store-visible tag value. /// /// Tags expose selected payload information to the event store so commands can -/// query the facts relevant to a decision. For example, an event payload may +/// select the facts relevant to a decision. For example, an event payload may /// contain `username: "renata"`, while the stored event also carries the tag /// `username:renata`. pub opaque type Tag { @@ -46,8 +199,9 @@ pub opaque type Tag { /// Application metadata attached to a recorded event. /// -/// Metadata is not part of query matching. It is intended for operational and -/// audit context such as correlation ids, causation ids, actors, and timestamps. +/// Metadata does not participate in decision-context matching. It is intended +/// for operational and audit context such as correlation ids, causation ids, +/// actors, and timestamps. pub opaque type Metadata { Metadata(List(#(String, String))) } @@ -65,50 +219,52 @@ pub type EventDescriptor { ) } -pub type Query { +/// The facts a command must consider before deciding. +/// +/// A decision can explicitly ignore history, match every recorded event, or +/// select events using OR-combined `Item` values. See `Item` for the matching +/// rules inside each branch. +pub type DecisionContext { + /// Decide from the decider's initial state without reading prior events. + NoContext + + /// Read and protect every event selected by the matching items. + Matching(items: List(Item)) + /// Match every recorded event. AllEvents - - /// Match events using one or more query items. - /// - /// Query items are OR-combined. See `QueryItem` for the matching rules inside - /// each item. - Query(items: List(QueryItem)) } -pub type QueryItem { - /// One branch of a command-context query. +pub type Item { + /// One branch of a selective command context. /// /// Within an item, event types are OR-combined and tags are AND-combined. Empty /// `types` means any event type matches. Empty `tags` means no tag constraint. /// - /// A query item with `types: [UserRegistered, UsernameReserved]` and + /// An item with `types: [UserRegistered, UsernameReserved]` and /// `tags: [username:renata]` means: events of either type that also have the /// `username:renata` tag. - QueryItem(types: List(EventType), tags: List(Tag)) + Item(types: List(EventType), tags: List(Tag)) } pub type SequencePosition { /// No global position was observed. NoPosition - /// A backend-specific global sequence position. - /// - /// Positions are used by context append conditions to express "after this - /// observed point in history". They are not stream revisions. + /// A backend-specific global sequence position used by context append + /// conditions to express "after this observed point in history". SequencePosition(Int) } pub type AppendCondition { - /// Append without an additional context condition. - NoAppendCondition - - /// Append only if no event matching `query` appeared after `after`. + /// Append only if no event selected by `decision_context` appeared after + /// `after`. /// /// This models Command Context Consistency. The decision was made from the - /// matching facts visible at `after`, so the append must fail if that relevant - /// context changed before the new facts are recorded. - FailIfEventsMatch(query: Query, after: SequencePosition) + /// selected facts visible at `after`, so the append must fail if that relevant + /// context changed before the new facts are recorded. `NoContext` selects no + /// events and therefore represents an unconditional append. + FailIfEventsMatch(decision_context: DecisionContext, after: SequencePosition) } pub type Decider(command, state, event, domain_error) { @@ -127,8 +283,9 @@ pub type Decider(command, state, event, domain_error) { /// command when it retries a transaction conflict. These functions must be /// deterministic and side-effect free: do not perform IO, mutate external /// state, allocate ids from an external system, publish messages, or otherwise - /// affect the host system. Return domain events only; external effects belong - /// in durable outbox records after commit. + /// affect the host system. Return domain events only. Derive effects outside + /// the decider, persist durable intent atomically where supported, and execute + /// external IO only after commit. Decider( initial: state, decide: fn(state, command) -> Result(List(event), domain_error), @@ -136,74 +293,40 @@ pub type Decider(command, state, event, domain_error) { ) } -pub type View(state, event) { - /// A pure projection fold. - /// - /// Views derive read-side state from events. They are intentionally only the - /// computation; persistence, delivery, rebuilds, and subscription management are - /// outside the core library. - View(initial: state, evolve: fn(state, event) -> state) -} - -pub type Reactor(event, effect) { - /// A pure event reaction. - /// - /// Reactors inspect committed recorded events and produce application-owned - /// effect values. They do not run IO. Applications or backend adapters decide - /// whether to execute effects immediately, persist them durably, retry them, or - /// ignore them during replay. - /// - /// WARNING: reactors used by retrying backends must be pure. - /// - /// A backend may run `react` more than once for the same committed event while - /// retrying a transaction conflict. Reactors must only derive effect values; - /// they must not execute IO, publish messages, call external services, mutate - /// state, or allocate externally-visible ids. Persist durable effect values and - /// execute them after commit. - Reactor(react: fn(Recorded(event)) -> List(effect)) -} - -pub type Revision { - /// A stream has no events. - NoEvents - - /// The last known revision of a stream. - CurrentRevision(Int) -} - -pub type Decoded(event) { - /// A domain event decoded from backend storage. - /// - /// Backends use codecs supplied by the application. The decoded value includes - /// the domain event plus the event type and tags that should participate in - /// query matching, along with non-query event metadata. - Decoded(event: event, descriptor: EventDescriptor) +pub type Event(event) { + Event(payload: event, descriptor: EventDescriptor) } pub type Recorded(event) { /// A stored event with backend metadata. /// - /// `revision` is the per-stream revision. `position` is the global sequence - /// position used for context consistency. `type_` and `tags` are store-visible - /// query metadata. + /// `position` is the global sequence position used for ordering and context + /// consistency. The descriptor carries store-visible type, version, tags, and + /// application metadata. Recorded( id: String, - stream: String, - revision: Int, position: SequencePosition, event: event, descriptor: EventDescriptor, ) } +/// Result of a successful command dispatch. +/// +/// `position` is the final global position assigned to this dispatch, or +/// `NoPosition` when no events were produced. `events` preserves append order. +pub type Dispatch(event) { + Dispatch(position: SequencePosition, events: List(Recorded(event))) +} + pub type Context(event, state) { /// A command context read from history. /// - /// The context contains the query that selected the relevant facts, the folded - /// decision state, the matching recorded events, the highest observed position, - /// and the append condition needed to protect the decision. + /// The context contains the requested decision context, folded state, matching + /// recorded events, highest observed position, and append condition needed to + /// protect the decision. Context( - query: Query, + decision_context: DecisionContext, state: state, events: List(Recorded(event)), position: SequencePosition, @@ -211,19 +334,6 @@ pub type Context(event, state) { ) } -pub type LoadedStream(event, state) { - /// A stream read from history. - /// - /// This is the stream-consistency counterpart to `Context`. It contains the - /// folded state for one stream and its current revision. - LoadedStream( - stream: String, - state: state, - events: List(Recorded(event)), - revision: Revision, - ) -} - /// Wrap an event type name. pub fn event_type(name: String) -> EventType { EventType(name) @@ -289,6 +399,42 @@ pub fn metadata_remove(metadata: Metadata, key: String) -> Metadata { Metadata(list.filter(entries, fn(entry) { entry.0 != key })) } +/// Prepare a domain event with empty tags and metadata for persistence. +pub fn new_event( + type_ type_: EventType, + version version: Int, + data payload: payload, +) -> Event(payload) { + Event( + payload:, + descriptor: EventDescriptor( + type_:, + version:, + tags: [], + metadata: empty_metadata(), + ), + ) +} + +/// Replace the tags on a proposed event. +pub fn with_tags( + proposed: Event(payload), + tags tags: List(Tag), +) -> Event(payload) { + Event(..proposed, descriptor: EventDescriptor(..proposed.descriptor, tags:)) +} + +/// Replace the metadata on a proposed event. +pub fn with_metadata( + proposed: Event(payload), + metadata metadata: Metadata, +) -> Event(payload) { + Event( + ..proposed, + descriptor: EventDescriptor(..proposed.descriptor, metadata:), + ) +} + /// Metadata key used to correlate all facts and effects for one operation. pub const correlation_id = "correlation_id" @@ -313,26 +459,12 @@ pub const idempotency_key = "idempotency_key" /// Metadata key used to carry the public operation id. pub const operation_id = "operation_id" -/// Build a query from query items. -/// -/// An empty list becomes `AllEvents`; otherwise the query contains the supplied -/// items. Query items are OR-combined by `matches_query`. -pub fn query(items: List(QueryItem)) -> Query { - case items { - [] -> AllEvents - [_, ..] -> Query(items) - } -} - -/// Build one command-context query branch. +/// Build one selective command-context branch. /// /// Event types are OR-combined. Tags are AND-combined. Empty lists act as wildcards /// for that part of the item. -pub fn query_item( - types types: List(EventType), - tags tags: List(Tag), -) -> QueryItem { - QueryItem(types:, tags:) +pub fn item(types types: List(EventType), tags tags: List(Tag)) -> Item { + Item(types:, tags:) } /// Build a pure command-side decider. @@ -353,64 +485,6 @@ pub fn decider( Decider(initial:, decide:, evolve:) } -/// Build a pure projection view. -/// -/// A view folds events into read-side state. It does not prescribe where that -/// state is stored or how events are delivered. -pub fn view( - initial initial: state, - evolve evolve: fn(state, event) -> state, -) -> View(state, event) { - View(initial:, evolve:) -} - -/// Build a pure event reactor. -/// -/// Reactors are the side-effect planning counterpart to views: they consume -/// committed recorded events and return application-owned effect values without -/// executing IO. -/// -/// WARNING: reactors used by retrying backends must be pure. `react` may be -/// called more than once for the same committed event during a transaction -/// retry. It must only derive effect values; executing IO belongs after commit. -pub fn reactor( - react react: fn(Recorded(event)) -> List(effect), -) -> Reactor(event, effect) { - Reactor(react:) -} - -/// Fold events with a decider and decide which new events a command produces. -/// -/// This is useful for unit tests and for in-memory command handling. It does not -/// perform any append or consistency check. -pub fn compute_events( - decider decider: Decider(command, state, event, domain_error), - events events: List(event), - command command: command, -) -> Result(List(event), domain_error) { - let Decider(initial, decide, evolve) = decider - decide(fold_events(initial, events, evolve), command) -} - -/// Decide from an optional current state and return the state after produced events. -/// -/// If `current` is `None`, the decider's initial state is used. The function first -/// runs the decider, then folds the produced events into the decision state. -pub fn compute_state( - decider decider: Decider(command, state, event, domain_error), - current current: Option(state), - command command: command, -) -> Result(state, domain_error) { - let Decider(initial, decide, evolve) = decider - let state = case current { - Some(state) -> state - None -> initial - } - - use events <- result.try(decide(state, command)) - Ok(fold_events(state, events, evolve)) -} - /// Fold recorded events into state using a domain evolution function. /// /// Backends use this after decoding stored events. Only the domain event payload is @@ -424,71 +498,6 @@ pub fn evolve_recorded( evolve(state, recorded.event) } -/// Project events from a view's initial state. -pub fn project( - view view: View(state, event), - events events: List(event), -) -> state { - let View(initial, evolve) = view - fold_events(initial, events, evolve) -} - -/// Project events starting from an already materialized view state. -pub fn project_from( - view view: View(state, event), - state state: state, - events events: List(event), -) -> state { - let View(_, evolve) = view - fold_events(state, events, evolve) -} - -/// Produce effect values for one committed recorded event. -pub fn react( - reactor reactor: Reactor(event, effect), - event event: Recorded(event), -) -> List(effect) { - let Reactor(react) = reactor - react(event) -} - -/// Produce effect values for committed recorded events, preserving event order. -pub fn react_all( - reactor reactor: Reactor(event, effect), - events events: List(Recorded(event)), -) -> List(effect) { - use event <- list.flat_map(events) - react(reactor, event) -} - -/// Merge two reactors that consume the same event type. -/// -/// The resulting reactor runs both reactors for each event and concatenates their -/// produced effects in argument order. -pub fn merge_reactors( - first first: Reactor(event, effect), - second second: Reactor(event, effect), -) -> Reactor(event, effect) { - use event <- Reactor - list.append(react(first, event), react(second, event)) -} - -/// Merge two views that consume the same event type. -/// -/// The resulting view keeps both states in a tuple and evolves both for every -/// event. This is a convenience for composing small pure projections. -pub fn merge_views( - first first: View(first_state, event), - second second: View(second_state, event), -) -> View(#(first_state, second_state), event) { - let View(first_initial, first_evolve) = first - let View(second_initial, second_evolve) = second - - use state, event <- View(initial: #(first_initial, second_initial)) - let #(first_state, second_state) = state - #(first_evolve(first_state, event), second_evolve(second_state, event)) -} - /// Run a command against a previously read context. /// /// The returned tuple preserves the original context alongside the newly produced @@ -502,18 +511,25 @@ pub fn decide_context( Ok(#(context, events)) } -/// Test whether a recorded event belongs to a query-defined context. -pub fn matches_query(recorded: Recorded(event), query: Query) -> Bool { - matches_descriptor(recorded.descriptor, query) +/// Test whether a recorded event belongs to a decision context. +pub fn matches_decision_context( + recorded: Recorded(event), + decision_context: DecisionContext, +) -> Bool { + matches_descriptor(recorded.descriptor, decision_context) } -/// Test whether an event descriptor belongs to a query-defined context. +/// Test whether an event descriptor belongs to a decision context. /// /// This is useful for routing an event before decoding its domain payload. -pub fn matches_descriptor(descriptor: EventDescriptor, query: Query) -> Bool { - case query { +fn matches_descriptor( + descriptor: EventDescriptor, + decision_context: DecisionContext, +) -> Bool { + case decision_context { AllEvents -> True - Query(items) -> list.any(items, matches_item(descriptor, _)) + Matching(items:) -> list.any(items, matches_item(descriptor, _)) + NoContext -> False } } @@ -536,18 +552,20 @@ pub fn highest_position( } } -fn fold_events( - initial: state, - events: List(event), - evolve: fn(state, event) -> state, -) -> state { - use state, event <- list.fold(events, initial) - evolve(state, event) +/// Return the highest global position carried by recorded events. +/// +/// Empty input returns `NoPosition`. The input need not already be ordered. +pub fn highest_recorded_position( + events: List(Recorded(event)), +) -> SequencePosition { + list.fold(events, NoPosition, fn(position, recorded) { + highest_position(position, recorded.position) + }) } -fn matches_item(descriptor: EventDescriptor, item: QueryItem) -> Bool { +fn matches_item(descriptor: EventDescriptor, item: Item) -> Bool { let EventDescriptor(type_:, tags:, ..) = descriptor - let QueryItem(types:, tags: required_tags) = item + let Item(types:, tags: required_tags) = item matches_types(type_, types) && matches_tags(tags, required_tags) } @@ -571,3 +589,21 @@ fn matches_tags(event_tags: List(Tag), required_tags: List(Tag)) -> Bool { } } } + +pub fn error_to_string( + error: Error(domain_error, subscription_error, store_error), + domain_error_to_string: fn(domain_error) -> String, + subscription_error_to_string: fn(subscription_error) -> String, + store_error_to_string: fn(store_error) -> String, +) -> String { + case error { + DomainError(error) -> domain_error_to_string(error) + SubscriptionError(error:) -> + "subscription error: " <> subscription_error_to_string(error) + StoreError(error) -> store_error_to_string(error) + AppendConditionFailed(FailIfEventsMatch(decision_context: _, after: _)) -> + "append to event failed: events matched" + DecodeError(UnknownEvent) -> "unknown event decoded" + DecodeError(InvalidData) -> "invalid data stored in database" + } +} diff --git a/src/factos/simulate.gleam b/src/factos/simulate.gleam index e35c12b..9728ab3 100644 --- a/src/factos/simulate.gleam +++ b/src/factos/simulate.gleam @@ -1,10 +1,9 @@ //// Deterministic in-memory execution of Factos domain scenarios. //// -//// The simulator composes the same query, context, append-condition, and decider -//// semantics used by storage backends without modelling backend IO concerns. +//// The simulator composes the same decision-context, append-condition, and +//// decider semantics used by storage backends without modelling backend IO. import factos -import gleam/dict import gleam/int import gleam/list @@ -14,7 +13,6 @@ pub opaque type Store(event) { records_reversed: List(factos.Recorded(event)), describe_event: fn(event) -> factos.EventDescriptor, next_position: Int, - next_revisions: dict.Dict(String, Int), ) } @@ -32,62 +30,55 @@ pub type AppendError { pub fn new( describe_event describe_event: fn(event) -> factos.EventDescriptor, ) -> Store(event) { - Store( - records_reversed: [], - describe_event:, - next_position: 1, - next_revisions: dict.new(), - ) + Store(records_reversed: [], describe_event:, next_position: 1) } /// Seed events that the domain has already accepted. -pub fn given( - store: Store(event), - stream stream: String, - events events: List(event), -) -> Store(event) { - let #(store, _) = record_batch(store, stream, events) +pub fn given(store: Store(event), events events: List(event)) -> Store(event) { + let #(store, _) = record_batch(store, events) store } -/// Read recorded events matching a command-context query. +/// Read recorded events selected by a decision context. pub fn read( store: Store(event), - query query: factos.Query, + decision_context decision_context: factos.DecisionContext, ) -> List(factos.Recorded(event)) { let Store(records_reversed:, ..) = store records_reversed - |> list.filter(factos.matches_query(_, query)) + |> list.filter(factos.matches_decision_context(_, decision_context)) |> list.reverse } -/// Read and fold the context selected by a query. +/// Read and fold the state selected by a decision context. pub fn read_context( store: Store(event), - query query: factos.Query, + decision_context decision_context: factos.DecisionContext, decider decider: factos.Decider(command, state, event, domain_error), ) -> factos.Context(event, state) { let factos.Decider(initial:, evolve:, ..) = decider - let events = read(store, query) + let events = read(store, decision_context) let position = list.fold(events, factos.NoPosition, fn(position, recorded) { factos.highest_position(position, recorded.position) }) factos.Context( - query:, + decision_context:, state: factos.evolve_recorded(initial:, events:, evolve:), events:, position:, - append_condition: factos.FailIfEventsMatch(query:, after: position), + append_condition: factos.FailIfEventsMatch( + decision_context:, + after: position, + ), ) } /// Record events when the supplied command context is still current. pub fn append( store: Store(event), - stream stream: String, events events: List(event), condition condition: factos.AppendCondition, ) -> Result(Commit(event), AppendError) { @@ -97,7 +88,7 @@ pub fn append( case append_condition_failed(store, condition) { True -> Error(AppendConditionFailed(condition:)) False -> { - let #(store, events) = record_batch(store, stream, events) + let #(store, events) = record_batch(store, events) Ok(Commit(store:, events:)) } } @@ -107,17 +98,16 @@ pub fn append( /// Read, decide, and record one command as an atomic immutable transition. pub fn dispatch( store: Store(event), - stream stream: String, - query query: factos.Query, + decision_context context: factos.DecisionContext, decider decider: factos.Decider(command, state, event, domain_error), command command: command, ) -> Result(Commit(event), domain_error) { - let context = read_context(store, query, decider) + let context = read_context(store, context, decider) case factos.decide_context(context, command, decider) { Error(error) -> Error(error) Ok(#(_, events)) -> { - let #(store, events) = record_batch(store, stream, events) + let #(store, events) = record_batch(store, events) Ok(Commit(store:, events:)) } } @@ -127,63 +117,43 @@ fn append_condition_failed( store: Store(event), condition: factos.AppendCondition, ) -> Bool { - case condition { - factos.NoAppendCondition -> False - factos.FailIfEventsMatch(query:, after:) -> { - let Store(records_reversed:, ..) = store - - list.any(records_reversed, fn(recorded) { - factos.matches_query(recorded, query) - && case after, recorded.position { - factos.NoPosition, factos.NoPosition -> True - factos.NoPosition, factos.SequencePosition(_) -> True - factos.SequencePosition(_), factos.NoPosition -> False - factos.SequencePosition(after), factos.SequencePosition(position) -> - position > after - } - }) + let factos.FailIfEventsMatch(decision_context:, after:) = condition + + let Store(records_reversed:, ..) = store + + list.any(records_reversed, fn(recorded) { + factos.matches_decision_context(recorded, decision_context) + && case after, recorded.position { + factos.NoPosition, factos.NoPosition -> True + factos.NoPosition, factos.SequencePosition(_) -> True + factos.SequencePosition(_), factos.NoPosition -> False + factos.SequencePosition(after), factos.SequencePosition(position) -> + position > after } - } + }) } fn record_batch( store: Store(event), - stream: String, events: List(event), ) -> #(Store(event), List(factos.Recorded(event))) { case events { [] -> #(store, []) [_, ..] -> { - let Store( - records_reversed:, - describe_event:, - next_position:, - next_revisions:, - ) = store - let next_revision = case dict.get(next_revisions, stream) { - Ok(next_revision) -> next_revision - Error(Nil) -> 0 - } - let #(records_reversed, batch_reversed, next_position, next_revision) = + let Store(records_reversed:, describe_event:, next_position:) = store + let #(records_reversed, batch_reversed, next_position) = list.fold( events, - #(records_reversed, [], next_position, next_revision), + #(records_reversed, [], next_position), fn(accumulator, event) { - let #( - records_reversed, - batch_reversed, - next_position, - next_revision, - ) = accumulator + let #(records_reversed, batch_reversed, next_position) = accumulator let factos.EventDescriptor(type_:, version:, tags:, metadata:) = describe_event(event) let recorded = factos.Recorded( id: "factos-simulate-" <> int.to_string(next_position), - stream:, - revision: next_revision, position: factos.SequencePosition(next_position), - event: event, + event:, descriptor: factos.EventDescriptor( type_:, version:, @@ -196,17 +166,10 @@ fn record_batch( [recorded, ..records_reversed], [recorded, ..batch_reversed], next_position + 1, - next_revision + 1, ) }, ) - let store = - Store( - records_reversed:, - describe_event:, - next_position:, - next_revisions: dict.insert(next_revisions, stream, next_revision), - ) + let store = Store(records_reversed:, describe_event:, next_position:) #(store, list.reverse(batch_reversed)) } diff --git a/test/factos_test.gleam b/test/factos_test.gleam index fc5799c..bbc85b8 100644 --- a/test/factos_test.gleam +++ b/test/factos_test.gleam @@ -3,7 +3,6 @@ import factos/simulate import gleam/int import gleam/list -import gleam/option.{None, Some} import gleeunit pub fn main() -> Nil { @@ -32,11 +31,14 @@ type DomainError { pub fn decide_context_uses_decider_test() { let context = factos.Context( - query: factos.AllEvents, + decision_context: factos.AllEvents, state: UsernameAvailable, events: [], position: factos.NoPosition, - append_condition: factos.NoAppendCondition, + append_condition: factos.FailIfEventsMatch( + decision_context: factos.NoContext, + after: factos.NoPosition, + ), ) assert factos.decide_context( @@ -49,8 +51,8 @@ pub fn decide_context_uses_decider_test() { pub fn query_matches_by_type_and_tags_test() { let username_query = - factos.query([ - factos.query_item( + factos.Matching([ + factos.item( types: [ factos.event_type("UsernameReserved"), factos.event_type("UserRegistered"), @@ -63,48 +65,48 @@ pub fn query_matches_by_type_and_tags_test() { recorded( UsernameReserved("renata"), [factos.tag("username:renata")], - revision: 0, + position: 0, ) let wrong_tag = recorded( UsernameReserved("lucy"), [factos.tag("username:lucy")], - revision: 1, + position: 1, ) let wrong_type = recorded( DisplayNameChanged("Renata"), [factos.tag("username:renata")], - revision: 2, + position: 2, ) - assert factos.matches_query(matching, username_query) - assert !factos.matches_query(wrong_tag, username_query) - assert !factos.matches_query(wrong_type, username_query) + assert factos.matches_decision_context(matching, username_query) + assert !factos.matches_decision_context(wrong_tag, username_query) + assert !factos.matches_decision_context(wrong_type, username_query) } pub fn query_items_are_or_combined_test() { let query = - factos.query([ - factos.query_item(types: [factos.event_type("UserRegistered")], tags: [ + factos.Matching([ + factos.item(types: [factos.event_type("UserRegistered")], tags: [ factos.tag("user:1"), ]), - factos.query_item(types: [factos.event_type("DisplayNameChanged")], tags: [ + factos.item(types: [factos.event_type("DisplayNameChanged")], tags: [ factos.tag("user:2"), ]), ]) let event = - recorded(DisplayNameChanged("R"), [factos.tag("user:2")], revision: 0) + recorded(DisplayNameChanged("R"), [factos.tag("user:2")], position: 0) - assert factos.matches_query(event, query) + assert factos.matches_decision_context(event, query) } -pub fn empty_query_matches_all_events_test() { - let query = factos.query([]) - let event = recorded(DisplayNameChanged("R"), [], revision: 0) +pub fn empty_matching_context_matches_no_events_test() { + let decision_context = factos.Matching(items: []) + let event = recorded(DisplayNameChanged("R"), [], position: 0) - assert factos.matches_query(event, query) + assert !factos.matches_decision_context(event, decision_context) } pub fn metadata_helpers_get_put_and_remove_values_test() { @@ -125,320 +127,161 @@ pub fn metadata_helpers_get_put_and_remove_values_test() { assert factos.metadata_get(metadata, factos.actor) == Ok("user_123") } -pub fn highest_position_keeps_later_position_test() { - let early = factos.SequencePosition(10) - let later = factos.SequencePosition(11) - - assert factos.highest_position(early, later) == later - assert factos.highest_position(factos.NoPosition, early) == early -} - -pub fn revision_models_empty_and_loaded_streams_test() { - assert revision_label(factos.NoEvents) == "empty" - assert revision_label(factos.CurrentRevision(2)) == "revision:2" -} - -pub fn decider_computes_events_from_history_test() { - let decider = username_decider() - - assert factos.compute_events( - decider: decider, - events: [], - command: RegisterUser("renata"), - ) - == Ok([UserRegistered("renata")]) - - assert factos.compute_events( - decider: decider, - events: [UsernameReserved("renata")], - command: RegisterUser("renata"), - ) - == Error(UsernameAlreadyTaken) -} - -pub fn decider_computes_state_from_optional_state_test() { - let decider = username_decider() - - assert factos.compute_state( - decider: decider, - current: None, - command: RegisterUser("renata"), +pub fn proposed_event_builders_replace_tags_and_metadata_test() -> Nil { + let metadata = factos.metadata([#("source", "registration")]) + let proposed = + factos.new_event( + type_: factos.event_type("UserRegistered"), + version: 1, + data: "renata", ) - == Ok(UsernameTaken) + |> factos.with_tags(tags: [factos.tag("username:renata")]) + |> factos.with_metadata(metadata:) - assert factos.compute_state( - decider: decider, - current: Some(UsernameTaken), - command: RegisterUser("renata"), + assert proposed + == factos.Event( + payload: "renata", + descriptor: factos.EventDescriptor( + type_: factos.event_type("UserRegistered"), + version: 1, + tags: [factos.tag("username:renata")], + metadata:, + ), ) - == Error(UsernameAlreadyTaken) } -pub fn view_projects_events_test() { - let view = - factos.view(initial: 0, evolve: fn(count, event) { - case event { - UserRegistered(_) -> count + 1 - UsernameReserved(_) -> count - DisplayNameChanged(_) -> count - } - }) +pub fn highest_position_keeps_later_position_test() { + let early = factos.SequencePosition(10) + let later = factos.SequencePosition(11) - assert factos.project(view: view, events: [ - UsernameReserved("renata"), - UserRegistered("renata"), - UserRegistered("lucy"), - ]) - == 2 -} - -pub fn merge_views_projects_same_events_into_tuple_state_test() { - let registration_count = - factos.view(initial: 0, evolve: fn(count, event) { - case event { - UserRegistered(_) -> count + 1 - UsernameReserved(_) -> count - DisplayNameChanged(_) -> count - } - }) - let display_name_count = - factos.view(initial: 0, evolve: fn(count, event) { - case event { - DisplayNameChanged(_) -> count + 1 - UsernameReserved(_) -> count - UserRegistered(_) -> count - } - }) - - let merged = factos.merge_views(registration_count, display_name_count) - - assert factos.project(view: merged, events: [ - UserRegistered("renata"), - DisplayNameChanged("Renata"), - DisplayNameChanged("Rena"), - ]) - == #(1, 2) + assert factos.highest_position(early, later) == later + assert factos.highest_position(factos.NoPosition, early) == early } -pub fn react_maps_one_recorded_event_into_effect_values_test() -> Nil { - let event = - recorded( - UserRegistered("renata"), - [factos.tag("username:renata")], - revision: 5, - ) - let reactor = - factos.reactor(react: fn(recorded) { - case recorded.event { - UserRegistered(username) -> [ - recorded.id - <> ":" - <> int.to_string(recorded.revision) - <> ":" - <> username, - ] - UsernameReserved(_) -> [] - DisplayNameChanged(_) -> [] - } - }) - - assert factos.react(reactor: reactor, event: event) == ["event-5:5:renata"] -} - -pub fn react_all_flattens_reactions_in_recorded_event_order_test() -> Nil { - let reactor = - factos.reactor(react: fn(recorded) { - case recorded.event { - UserRegistered(username) -> [ - "registered:" <> username, - "welcome:" <> username, - ] - UsernameReserved(_) -> [] - DisplayNameChanged(name) -> ["display:" <> name] - } - }) - - assert factos.react_all(reactor: reactor, events: [ - recorded(UserRegistered("renata"), [], revision: 0), - recorded(UsernameReserved("renata"), [], revision: 1), - recorded(DisplayNameChanged("Rena"), [], revision: 2), - recorded(UserRegistered("lucy"), [], revision: 3), - ]) - == [ - "registered:renata", - "welcome:renata", - "display:Rena", - "registered:lucy", - "welcome:lucy", - ] -} +pub fn dispatch_records_final_position_without_an_append_wrapper_test() -> Nil { + let events = [ + recorded(UserRegistered("renata"), [], position: 10), + recorded(UserRegistered("lucy"), [], position: 12), + recorded(UserRegistered("marc"), [], position: 11), + ] + let position = factos.highest_recorded_position(events) + let dispatch = factos.Dispatch(position:, events:) -pub fn merge_reactors_combines_outputs_for_the_same_recorded_event_test() -> Nil { - let event = - recorded( - UserRegistered("renata"), - [factos.tag("username:renata")], - revision: 7, - ) - let audit = - factos.reactor(react: fn(recorded) { - case recorded.event { - UserRegistered(username) -> [ - "audit:" <> recorded.id <> ":" <> username, - ] - UsernameReserved(_) -> [] - DisplayNameChanged(_) -> [] - } - }) - let notification = - factos.reactor(react: fn(recorded) { - case recorded.event { - UserRegistered(username) -> [ - "notify:" <> int.to_string(recorded.revision) <> ":" <> username, - ] - UsernameReserved(_) -> [] - DisplayNameChanged(_) -> [] - } - }) - - let merged = factos.merge_reactors(audit, notification) - - assert factos.react(reactor: merged, event: event) - == ["audit:event-7:renata", "notify:7:renata"] + assert position == factos.SequencePosition(12) + assert dispatch.events == events + assert factos.highest_recorded_position([]) == factos.NoPosition } pub fn simulate_given_assigns_deterministic_record_envelopes_test() -> Nil { + let store = simulate.new(describe_event) let store = - simulate.new(describe_event) - |> simulate.given(stream: "account-1", events: [ + simulate.given(store, events: [ UsernameReserved(username: "renata"), UserRegistered(username: "renata"), ]) - |> simulate.given(stream: "unused", events: []) - |> simulate.given(stream: "account-2", events: [ - DisplayNameChanged(name: "Renata"), - ]) + let store = simulate.given(store, events: []) + let store = + simulate.given(store, events: [DisplayNameChanged(name: "Renata")]) assert simulate.read(store, factos.AllEvents) == [ - factos.Recorded( - id: "factos-simulate-1", - stream: "account-1", - revision: 0, - position: factos.SequencePosition(1), - descriptor: factos.EventDescriptor( - type_: factos.event_type("UsernameReserved"), - version: 1, - tags: [factos.tag("username:renata")], - metadata: factos.empty_metadata(), - ), - event: UsernameReserved(username: "renata"), - ), - factos.Recorded( - id: "factos-simulate-2", - stream: "account-1", - revision: 1, - position: factos.SequencePosition(2), - descriptor: factos.EventDescriptor( - type_: factos.event_type("UserRegistered"), - version: 1, - tags: [factos.tag("username:renata")], - metadata: factos.empty_metadata(), - ), - event: UserRegistered(username: "renata"), - ), - factos.Recorded( - id: "factos-simulate-3", - stream: "account-2", - revision: 0, - position: factos.SequencePosition(3), - descriptor: factos.EventDescriptor( - type_: factos.event_type("DisplayNameChanged"), - version: 1, - tags: [], - metadata: factos.empty_metadata(), - ), - event: DisplayNameChanged(name: "Renata"), - ), + simulated_record(UsernameReserved(username: "renata"), position: 1), + simulated_record(UserRegistered(username: "renata"), position: 2), + simulated_record(DisplayNameChanged(name: "Renata"), position: 3), ] } pub fn simulate_read_and_context_use_query_order_and_highest_match_test() -> Nil { + let store = simulate.new(describe_event) let store = - simulate.new(describe_event) - |> simulate.given(stream: "account-renata", events: [ + simulate.given(store, events: [ UserRegistered(username: "renata"), - ]) - |> simulate.given(stream: "account-lucy", events: [ UserRegistered(username: "lucy"), - ]) - |> simulate.given(stream: "account-marc", events: [ UserRegistered(username: "marc"), ]) - let renata = - simulated_record( - UserRegistered(username: "renata"), - stream: "account-renata", - revision: 0, - position: 1, - ) - let lucy = - simulated_record( - UserRegistered(username: "lucy"), - stream: "account-lucy", - revision: 0, - position: 2, - ) - let marc = - simulated_record( - UserRegistered(username: "marc"), - stream: "account-marc", - revision: 0, - position: 3, - ) + let renata = simulated_record(UserRegistered(username: "renata"), position: 1) + let lucy = simulated_record(UserRegistered(username: "lucy"), position: 2) + let marc = simulated_record(UserRegistered(username: "marc"), position: 3) - let no_matches = empty_query() - assert simulate.read(store, no_matches) == [] - assert simulate.read_context(store, no_matches, username_decider()) + let decision_context = empty_query() + assert simulate.read(store, decision_context) == [] + assert simulate.read_context(store, decision_context, username_decider()) == factos.Context( - query: no_matches, + decision_context:, state: UsernameAvailable, events: [], position: factos.NoPosition, append_condition: factos.FailIfEventsMatch( - query: no_matches, + decision_context:, after: factos.NoPosition, ), ) - let compound_query = username_conformance_query() - assert simulate.read(store, compound_query) == [lucy] - assert simulate.read_context(store, compound_query, username_decider()) + let decision_context = username_conformance_context() + assert simulate.read(store, decision_context) == [lucy] + assert simulate.read_context(store, decision_context, username_decider()) == factos.Context( - query: compound_query, + decision_context:, state: UsernameTaken, events: [lucy], position: factos.SequencePosition(2), append_condition: factos.FailIfEventsMatch( - query: compound_query, + decision_context:, after: factos.SequencePosition(2), ), ) + let decision_context = factos.AllEvents assert simulate.read(store, factos.AllEvents) == [renata, lucy, marc] - assert simulate.read_context(store, factos.AllEvents, username_decider()) + assert simulate.read_context(store, decision_context, username_decider()) == factos.Context( - query: factos.AllEvents, + decision_context:, state: UsernameTaken, events: [renata, lucy, marc], position: factos.SequencePosition(3), append_condition: factos.FailIfEventsMatch( - query: factos.AllEvents, + decision_context:, after: factos.SequencePosition(3), ), ) } +pub fn simulate_no_context_ignores_recorded_history_test() -> Nil { + let store = + simulate.given(simulate.new(describe_event), events: [ + UsernameReserved(username: "renata"), + ]) + + assert simulate.read_context(store, factos.NoContext, username_decider()) + == factos.Context( + decision_context: factos.NoContext, + state: UsernameAvailable, + events: [], + position: factos.NoPosition, + append_condition: factos.FailIfEventsMatch( + factos.NoContext, + factos.NoPosition, + ), + ) + + let assert Ok(simulate.Commit(store:, events: committed)) = + simulate.dispatch( + store, + decision_context: factos.NoContext, + decider: username_decider(), + command: RegisterUser(username: "renata"), + ) + let registered = + simulated_record(UserRegistered(username: "renata"), position: 2) + + assert committed == [registered] + assert simulate.read(store, factos.AllEvents) + == [ + simulated_record(UsernameReserved(username: "renata"), position: 1), + registered, + ] +} + pub fn simulate_append_records_batch_in_order_without_condition_test() -> Nil { let events = [ UsernameReserved(username: "renata"), @@ -447,54 +290,39 @@ pub fn simulate_append_records_batch_in_order_without_condition_test() -> Nil { let assert Ok(simulate.Commit(store:, events: committed)) = simulate.append( simulate.new(describe_event), - stream: "account-1", - events: events, - condition: factos.NoAppendCondition, + events:, + condition: factos.FailIfEventsMatch( + decision_context: factos.NoContext, + after: factos.NoPosition, + ), ) let expected = [ - simulated_record( - UsernameReserved(username: "renata"), - stream: "account-1", - revision: 0, - position: 1, - ), - simulated_record( - UserRegistered(username: "renata"), - stream: "account-1", - revision: 1, - position: 2, - ), + simulated_record(UsernameReserved(username: "renata"), position: 1), + simulated_record(UserRegistered(username: "renata"), position: 2), ] assert committed == expected assert simulate.read(store, factos.AllEvents) == expected } -pub fn simulate_dispatch_reuses_recorded_facts_across_streams_test() -> Nil { - let query = username_query("renata") +pub fn simulate_dispatch_reuses_matching_recorded_facts_test() -> Nil { + let decision_context = username_context("renata") let assert Ok(simulate.Commit(store:, events: committed)) = simulate.dispatch( simulate.new(describe_event), - stream: "account-1", - query: query, + decision_context:, decider: username_decider(), command: RegisterUser(username: "renata"), ) let expected = [ - simulated_record( - UserRegistered(username: "renata"), - stream: "account-1", - revision: 0, - position: 1, - ), + simulated_record(UserRegistered(username: "renata"), position: 1), ] assert committed == expected let assert Error(error) = simulate.dispatch( store, - stream: "account-2", - query: query, + decision_context:, decider: username_decider(), command: RegisterUser(username: "renata"), ) @@ -503,84 +331,64 @@ pub fn simulate_dispatch_reuses_recorded_facts_across_streams_test() -> Nil { } pub fn simulate_append_without_position_requires_empty_match_set_test() -> Nil { - let query = username_query("renata") + let decision_context = username_context("renata") let context = simulate.read_context( simulate.new(describe_event), - query, + decision_context, username_decider(), ) let condition = context.append_condition assert context.position == factos.NoPosition assert condition - == factos.FailIfEventsMatch(query: query, after: factos.NoPosition) + == factos.FailIfEventsMatch(decision_context:, after: factos.NoPosition) let assert Ok(simulate.Commit(store:, events: first_events)) = simulate.append( simulate.new(describe_event), - stream: "account-1", events: [UserRegistered(username: "renata")], - condition: condition, + condition:, ) assert first_events - == [ - simulated_record( - UserRegistered(username: "renata"), - stream: "account-1", - revision: 0, - position: 1, - ), - ] + == [simulated_record(UserRegistered(username: "renata"), position: 1)] let assert Error(simulate.AppendConditionFailed(condition: rejected)) = simulate.append( store, - stream: "account-2", events: [UserRegistered(username: "renata")], - condition: condition, + condition:, ) assert rejected == condition } pub fn simulate_stale_context_append_rejects_later_match_test() -> Nil { - let query = username_query("renata") + let decision_context = username_context("renata") let initial_store = - simulate.new(describe_event) - |> simulate.given(stream: "account-1", events: [ + simulate.given(simulate.new(describe_event), events: [ UserRegistered(username: "renata"), ]) - let context = simulate.read_context(initial_store, query, username_decider()) + let context = + simulate.read_context(initial_store, decision_context, username_decider()) let condition = context.append_condition let interleaved_store = - simulate.given(initial_store, stream: "account-1", events: [ + simulate.given(initial_store, events: [ UsernameReserved(username: "renata"), ]) let accepted_before_rejection = [ - simulated_record( - UserRegistered(username: "renata"), - stream: "account-1", - revision: 0, - position: 1, - ), - simulated_record( - UsernameReserved(username: "renata"), - stream: "account-1", - revision: 1, - position: 2, - ), + simulated_record(UserRegistered(username: "renata"), position: 1), + simulated_record(UsernameReserved(username: "renata"), position: 2), ] assert context.position == factos.SequencePosition(1) let assert Error(simulate.AppendConditionFailed(condition: rejected)) = simulate.append( interleaved_store, - stream: "account-1", events: [ UsernameReserved(username: "renata"), UserRegistered(username: "renata"), ], - condition: condition, + condition:, ) assert rejected == condition assert simulate.read(interleaved_store, factos.AllEvents) @@ -589,17 +397,13 @@ pub fn simulate_stale_context_append_rejects_later_match_test() -> Nil { let assert Ok(simulate.Commit(store:, events: committed)) = simulate.append( interleaved_store, - stream: "account-1", events: [DisplayNameChanged(name: "Renata")], - condition: factos.NoAppendCondition, - ) - let next = - simulated_record( - DisplayNameChanged(name: "Renata"), - stream: "account-1", - revision: 2, - position: 3, + condition: factos.FailIfEventsMatch( + decision_context: factos.NoContext, + after: factos.NoPosition, + ), ) + let next = simulated_record(DisplayNameChanged(name: "Renata"), position: 3) assert committed == [next] assert simulate.read(store, factos.AllEvents) @@ -607,45 +411,28 @@ pub fn simulate_stale_context_append_rejects_later_match_test() -> Nil { } pub fn simulate_stale_context_append_allows_unrelated_later_event_test() -> Nil { - let query = username_query("renata") + let decision_context = username_context("renata") let initial_store = - simulate.new(describe_event) - |> simulate.given(stream: "account-1", events: [ + simulate.given(simulate.new(describe_event), events: [ UserRegistered(username: "renata"), ]) let condition = - simulate.read_context(initial_store, query, username_decider()).append_condition + simulate.read_context(initial_store, decision_context, username_decider()).append_condition let interleaved_store = - simulate.given(initial_store, stream: "account-2", events: [ + simulate.given(initial_store, events: [ DisplayNameChanged(name: "Renata"), ]) let assert Ok(simulate.Commit(store:, events: committed)) = simulate.append( interleaved_store, - stream: "account-3", events: [UsernameReserved(username: "renata")], - condition: condition, + condition:, ) let appended = - simulated_record( - UsernameReserved(username: "renata"), - stream: "account-3", - revision: 0, - position: 3, - ) + simulated_record(UsernameReserved(username: "renata"), position: 3) let expected = [ - simulated_record( - UserRegistered(username: "renata"), - stream: "account-1", - revision: 0, - position: 1, - ), - simulated_record( - DisplayNameChanged(name: "Renata"), - stream: "account-2", - revision: 0, - position: 2, - ), + simulated_record(UserRegistered(username: "renata"), position: 1), + simulated_record(DisplayNameChanged(name: "Renata"), position: 2), appended, ] @@ -654,79 +441,52 @@ pub fn simulate_stale_context_append_allows_unrelated_later_event_test() -> Nil } pub fn simulate_empty_append_ignores_stale_condition_test() -> Nil { - let query = username_query("renata") + let decision_context = username_context("renata") let initial_store = - simulate.new(describe_event) - |> simulate.given(stream: "account-1", events: [ + simulate.given(simulate.new(describe_event), events: [ UserRegistered(username: "renata"), ]) let condition = - simulate.read_context(initial_store, query, username_decider()).append_condition + simulate.read_context(initial_store, decision_context, username_decider()).append_condition let stale_store = - simulate.given(initial_store, stream: "account-1", events: [ + simulate.given(initial_store, events: [ UsernameReserved(username: "renata"), ]) let assert Ok(simulate.Commit(store:, events: [])) = - simulate.append( - stale_store, - stream: "account-1", - events: [], - condition: condition, - ) + simulate.append(stale_store, events: [], condition:) let assert Ok(simulate.Commit(store:, events: committed)) = simulate.append( store, - stream: "account-1", events: [DisplayNameChanged(name: "Renata")], - condition: factos.NoAppendCondition, - ) - let next = - simulated_record( - DisplayNameChanged(name: "Renata"), - stream: "account-1", - revision: 2, - position: 3, + condition: factos.FailIfEventsMatch( + decision_context: factos.NoContext, + after: factos.NoPosition, + ), ) + let next = simulated_record(DisplayNameChanged(name: "Renata"), position: 3) assert committed == [next] assert simulate.read(store, factos.AllEvents) == [ - simulated_record( - UserRegistered(username: "renata"), - stream: "account-1", - revision: 0, - position: 1, - ), - simulated_record( - UsernameReserved(username: "renata"), - stream: "account-1", - revision: 1, - position: 2, - ), + simulated_record(UserRegistered(username: "renata"), position: 1), + simulated_record(UsernameReserved(username: "renata"), position: 2), next, ] } pub fn simulate_empty_dispatch_is_a_no_op_test() -> Nil { let seeded_store = - simulate.new(describe_event) - |> simulate.given(stream: "account-1", events: [ + simulate.given(simulate.new(describe_event), events: [ UsernameReserved(username: "renata"), ]) let seeded = [ - simulated_record( - UsernameReserved(username: "renata"), - stream: "account-1", - revision: 0, - position: 1, - ), + simulated_record(UsernameReserved(username: "renata"), position: 1), ] let assert Ok(simulate.Commit(store:, events: [])) = simulate.dispatch( seeded_store, - stream: "account-1", - query: factos.AllEvents, + decision_context: factos.AllEvents, decider: empty_decider(), command: RegisterUser(username: "ignored"), ) @@ -735,17 +495,13 @@ pub fn simulate_empty_dispatch_is_a_no_op_test() -> Nil { let assert Ok(simulate.Commit(store:, events: committed)) = simulate.append( store, - stream: "account-1", events: [UserRegistered(username: "renata")], - condition: factos.NoAppendCondition, - ) - let next = - simulated_record( - UserRegistered(username: "renata"), - stream: "account-1", - revision: 1, - position: 2, + condition: factos.FailIfEventsMatch( + decision_context: factos.NoContext, + after: factos.NoPosition, + ), ) + let next = simulated_record(UserRegistered(username: "renata"), position: 2) assert committed == [next] assert simulate.read(store, factos.AllEvents) == list.append(seeded, [next]) @@ -799,9 +555,9 @@ fn describe_event(event: Event) -> factos.EventDescriptor { } } -fn username_query(username: String) -> factos.Query { - factos.query([ - factos.query_item( +fn username_context(username: String) -> factos.DecisionContext { + factos.Matching([ + factos.item( types: [ factos.event_type("UsernameReserved"), factos.event_type("UserRegistered"), @@ -811,17 +567,17 @@ fn username_query(username: String) -> factos.Query { ]) } -fn empty_query() -> factos.Query { - factos.Query(items: []) +fn empty_query() -> factos.DecisionContext { + factos.Matching(items: []) } -fn username_conformance_query() -> factos.Query { - factos.query([ - factos.query_item(types: [factos.event_type("UserRegistered")], tags: [ +fn username_conformance_context() -> factos.DecisionContext { + factos.Matching([ + factos.item(types: [factos.event_type("UserRegistered")], tags: [ factos.tag("username:renata"), factos.tag("username:lucy"), ]), - factos.query_item( + factos.item( types: [ factos.event_type("UnknownEventType"), factos.event_type("UserRegistered"), @@ -841,27 +597,20 @@ fn empty_decider() -> factos.Decider(Command, State, Event, DomainError) { fn simulated_record( event: Event, - stream stream: String, - revision revision: Int, position position: Int, ) -> factos.Recorded(Event) { - let factos.EventDescriptor(type_:, version:, tags:, metadata:) = - describe_event(event) - factos.Recorded( id: "factos-simulate-" <> int.to_string(position), - stream:, - revision:, position: factos.SequencePosition(position), - descriptor: factos.EventDescriptor(type_:, version:, tags:, metadata:), event:, + descriptor: describe_event(event), ) } fn recorded( event: Event, tags tags: List(factos.Tag), - revision revision: Int, + position position: Int, ) -> factos.Recorded(Event) { let type_ = case event { UsernameReserved(_) -> factos.event_type("UsernameReserved") @@ -870,23 +619,14 @@ fn recorded( } factos.Recorded( - id: "event-" <> int.to_string(revision), - stream: "user-1", - revision:, - position: factos.SequencePosition(revision), + id: "event-" <> int.to_string(position), + position: factos.SequencePosition(position), + event:, descriptor: factos.EventDescriptor( - type_: type_, + type_:, version: 1, - tags: tags, + tags:, metadata: factos.empty_metadata(), ), - event:, ) } - -fn revision_label(revision: factos.Revision) -> String { - case revision { - factos.NoEvents -> "empty" - factos.CurrentRevision(revision) -> "revision:" <> int.to_string(revision) - } -}