lib

Core libraries for Radroots
git clone https://radroots.dev/git/lib.git
Log | Files | Refs | README

README.md (24659B)


      1 # radroots_storage
      2 
      3 `radroots_storage` defines the backend-neutral persistence contracts used by
      4 Radroots hosts. It owns canonical event persistence, durable operation journal
      5 state, outbox and delivery evidence, projection coordination, protected-record
      6 metadata, reliability operations, and high-level atomic workflow commits.
      7 
      8 Projection document inventory selects one projection and explicitly selects
      9 one generation or all generations. Pages contain at most 256 records and
     10 16 MiB of opaque values, with independently scoped continuations and explicit
     11 corrupt-record locators. This is a live scan, not a frozen snapshot. Callers
     12 must fence their own mutations before treating a traversal as complete.
     13 Storage neither interprets document payloads nor grants deletion authority.
     14 
     15 The package does not expose SQL, filesystem handles, database pools, raw
     16 transactions, encryption keys, transport clients, schedulers, or application
     17 state. Concrete backends implement these contracts; `radroots_storage_sqlite`
     18 is the native durable backend and the opt-in [`memory`] module is the bounded
     19 deterministic reference implementation.
     20 
     21 The authoritative package charter is the
     22 [`radroots_storage` section of the Release V1 specification](../../contracts/crates/release_v1/radroots_crates_release_v1.toml).
     23 
     24 ## Typical flow
     25 
     26 1. A host selects a concrete backend and owns its lifecycle.
     27 2. The host records operation preparation in [`Journal`] with a caller-owned
     28    idempotency key.
     29 3. Verified events enter [`EventStore`] with explicit source provenance.
     30 4. [`Outbox`] stores transport-neutral delivery intent and evidence.
     31 5. [`ProjectionStore`] records checkpoints and rebuild coordination while
     32    domain reducers remain outside storage.
     33 6. Advanced hosts use [`atomic::AtomicStorage`] to commit related journal,
     34    event, outbox, and projection transitions as one local durable operation.
     35 7. [`BackupSource`] exposes explicit backup, staged restore, integrity, status,
     36    and close operations without leaking backend handles.
     37 
     38 [`Journal`]: crate::Journal
     39 [`EventStore`]: crate::EventStore
     40 [`Outbox`]: crate::Outbox
     41 [`ProjectionStore`]: crate::ProjectionStore
     42 [`BackupSource`]: crate::BackupSource
     43 
     44 ```rust
     45 use futures_executor::block_on;
     46 use radroots_storage::{
     47     BackupSource,
     48     event::SourceGeneration,
     49     memory::MemoryStorage,
     50     status::{ShutdownState, StorageBackend},
     51 };
     52 
     53 let generation = SourceGeneration::new([1; 32])?;
     54 let storage = MemoryStorage::new(generation);
     55 let status = block_on(BackupSource::status(&storage))?;
     56 
     57 assert_eq!(status.backend(), StorageBackend::Memory);
     58 assert_eq!(status.shutdown(), ShutdownState::Open);
     59 # Ok::<(), radroots_storage::Error>(())
     60 ```
     61 
     62 The same program is available as
     63 [`examples/memory_status.rs`](examples/memory_status.rs).
     64 
     65 ## Public capability boundary
     66 
     67 The crate root exposes the ordinary aggregate [`Storage`] capability plus
     68 `EventStore`, `Journal`, `Outbox`, `ProjectionStore`, `BackupSource`,
     69 `StorageStatus`, and `Error`. Advanced contracts remain in their owning
     70 modules so ordinary consumers do not accidentally depend on workflow or
     71 protected-record internals.
     72 
     73 All SPIs are externally implementable and dyn-compatible. Native traits retain
     74 `Send + Sync` and boxed `Future + Send`; browser traits and futures may stay
     75 local. Every SPI uses the same target-aware transport host bounds and future
     76 alias, allowing the host to choose the async executor. SPI declarations and
     77 pure value imports do not
     78 install an executor, spawn hidden
     79 workers, read a clock, generate identities, or perform implicit retries.
     80 
     81 The opt-in `protected_outbox` feature supplies pure local buyer/seller-outbox
     82 enrollment at `workspace::local_outbox` and the validated shared protected-grant
     83 codec at `workspace::grant`. The separate `workspace::seller_stock` policy
     84 selects fresh stock-owner enrollment for a qualified native host; pure policy
     85 values confer no durable authority.
     86 `LocalOutboxConfig::new` selects a fresh buyer outbox; `seller` explicitly
     87 selects a fresh seller public-authoring/communication outbox. Authenticated
     88 bindings retain that exact scope. Neither scope grants firm stock, delegation,
     89 organization access or recovery of old obligations. Scope separates route,
     90 genesis and credential commitments; changing an actor cannot upgrade a grant.
     91 Concrete native/browser hosts share these rules and exact envelope context.
     92 Shared `inventory::StockAbortRequest`, `StockAbortOutcome` and `StockAbortReceipt`
     93 carry explicit original-preparation cleanup selectors and reports.
     94 `InventoryStore::abort_proven_unissued` and `stock_preparation_abort` are typed
     95 transaction/query ports; unsupported hosts return `UnsupportedOperation`, whose
     96 stable code is `capability_unavailable`.
     97 Report reconstruction checks namespace, committed anchor, progress and bounded
     98 canonical after-images, and supplies no credential, absence or commit proof.
     99 Qualified stores must authenticate all original claims and exact release effects
    100 inside their owned atomic transaction. The native implementation uses its same
    101 bounded connection owner; ordinary outboxes acquire no stock authority.
    102 The codec's Serde wire stays private; decoding enforces the 16 KiB limit and
    103 validates current scope. Enrollment takes qualified host identities and proof
    104 without acquiring a backend or establishing durable authority. Concrete hosts
    105 must protect, commit and authenticate that state through their explicit owned
    106 lifecycle. This feature does not enable a database, provider or runtime.
    107 Its `command::protected` module supplies the shared native/browser immutable
    108 Prepared codec and pure binding/commitment checks. The complete codec is bounded
    109 to 64 KiB; a concrete host must split it to meet any smaller private-record
    110 budget, protect each record and atomically retain the full set. Decoding is a
    111 historical observation and supplies no current grant, actual commit or issuance.
    112 New preparations retain scope explicitly in version 3. Historical version 2
    113 preparations remain readable only with their original buyer-outbox authority;
    114 no actor descriptor can infer a seller grant during recovery.
    115 
    116 Common admission retains a typed command family. Fresh public listing T0
    117 requires an authenticated seller outbox and valid bounded event inputs. The
    118 current history and approval prerequisites remain explicit refusals. Chat-only
    119 ports require actual chat data and return `capability_unavailable` for a public
    120 preparation; no synthetic peer or receipt bridges the families.
    121 
    122 `BackupSource::settle_backup_writes` waits for earlier owner writes, including
    123 work whose caller was cancelled. The host excludes new writes before settling
    124 and retains that exclusion through related inventory and capture. Settling does
    125 not create a snapshot or an ongoing reservation.
    126 
    127 `BackupSource::capture_backup`, `verify_backup`, and `finalize_backup` invoke
    128 actual owner operations. Their default implementation returns typed
    129 `BackupCapabilityError::Unsupported`; reliability metadata transitions cannot
    130 substitute for a snapshot. The boundary returns manifests and completion, never
    131 paths or database handles. Hosts coordinate application state and referenced
    132 files separately; per-member snapshots do not imply a global transaction.
    133 
    134 Actual backup capture can retain incomplete staging after interruption. It
    135 does not replace live data or report that staging as finalized. The host must
    136 retain the original plan and manifest and reconcile an ambiguous result before
    137 retrying; cancellation never authorizes deleting retained evidence.
    138 
    139 ## Cancellation and commit points
    140 
    141 Dropping a returned future requests cancellation. Read operations may stop
    142 without side effects. Transactional record operations preserve their local
    143 durable commit point:
    144 
    145 - before the commit point, cancellation or failure leaves no partial state;
    146 - after a successful commit, cancellation cannot claim rollback;
    147 - replaying the same identity and canonical input returns the original result;
    148 - reusing an identity with different input fails as a conflict;
    149 - atomic workflow commits publish every requested mutation or none of them;
    150 - rollback failure never replaces the primary operation failure.
    151 
    152 `JournalState::Committed` is the durable operation-journal boundary.
    153 `AtomicStorage::commit` is the aggregate local workflow boundary. Network
    154 publication is outside this crate and is not implied by either state.
    155 
    156 `AuthoredAtomicCommand::RecordSigned` retains an already-created, cryptographically
    157 verified signature after its original signing lease expires or is superseded.
    158 The backend retrieves and checks its immutable signing claim receipt, including
    159 the full claim, operation, artifact and exact retained plan. This command performs
    160 no signing and supplies no permission to schedule another phase. Active
    161 `ApplySigned` and work claims retain their strict fences.
    162 
    163 The first exact signed bytes are immutable. Repeated identical evidence is
    164 idempotent; different raw bytes conflict. Cancelled or terminally failed signing
    165 may retain valid bytes while preserving its stop and failure state, with no
    166 admission or delivery claim. Receipt replay returns historical state, so hosts
    167 must query current durable status before deciding what work may follow.
    168 
    169 ## Events, journal, outbox, and projections
    170 
    171 Event storage preserves the exact signed event, verification/admission stage,
    172 source generation, monotonically increasing position, and every unique
    173 transport provenance observation. Queries are bounded and generation-aware;
    174 backends fail closed on corrupt rows or source changes.
    175 
    176 Current replacement heads are selected from verified and visible admissions
    177 before interpreting application payloads. A verified-only winner supersedes an
    178 older visible record without itself becoming visible. Raw records cannot select
    179 heads, and only visible contract-valid author-authorized deletion requests
    180 suppress events. Deleted winners never revive predecessors. Consumers use the
    181 shared visibility snapshot digest to detect admission changes even when the raw
    182 event count remains unchanged.
    183 
    184 The journal records a command lifecycle under a validated idempotency key and
    185 optimistic revision. The outbox persists explicit multi-target delivery plans,
    186 leases, attempts, normalized receipts, partial success, and satisfaction
    187 evidence without owning a transport adapter. Projection storage owns only
    188 checkpoints, invalidation/rebuild state, and event-index manifests; reducer
    189 algorithms and projected domain rows stay with their domain owners.
    190 
    191 Idempotency-key construction validates borrowed input before allocating its
    192 bounded owned representation, so rejected oversized input cannot force a
    193 second attacker-sized allocation at this public boundary.
    194 
    195 ## Draft queries and atomic submission
    196 
    197 `AuthoredDraftStore::query_authored_drafts` returns bounded current-head pages
    198 under an independently selected author, payload schema and optional immutable
    199 scope. Continue using the returned scoped cursor. Corrupt rows have individual
    200 repair locators; applications retain their evidence and continue other work.
    201 Pages scan stable draft IDs, so a later sweep must revisit new IDs inserted
    202 behind the cursor. A page holds at most 256 records and 4 MiB of decoded payload.
    203 
    204 `AuthoredAtomicCommand::PrepareFromDraft` joins a captured source revision,
    205 a distinct initial intent and the existing authored preparation in one commit.
    206 The application puts the complete frozen semantic request in the intent payload
    207 and owns strict payload validation. Reserve a stable command ID before effects.
    208 The author and command identify the receipt; context, source, intent and every
    209 preparation field are compared for exact replay. Replay precedes fresh source
    210 CAS and survives later editing. The source remains editable. The stored
    211 submission receipt retains the immutable source-to-intent association, and the
    212 ordinary Prepare receipt allows existing signing orchestration to resume.
    213 
    214 No signing or transport effect occurs in this storage transaction. Public
    215 backends must implement the new query method and submission command explicitly.
    216 
    217 ## Protected metadata and security
    218 
    219 `private_artifact` stores bounded metadata and opaque durable secret references.
    220 It never accepts domain plaintext, ciphertext bytes, or an active secret
    221 capability. Encryption, wrapping, and provider access belong to
    222 `radroots_secrets` and the concrete backend.
    223 
    224 Identifiers, paths, query sizes, revisions, timestamps, and result sets are
    225 bounded and validated before backend work. Secret references and idempotency
    226 keys have redacted diagnostics. Public errors are stable and do not expose SQL,
    227 filesystem, key-provider, or transport implementation messages. The crate
    228 forbids unsafe code.
    229 
    230 ## Backup, restore, integrity, and close
    231 
    232 Backup and restore are explicit multi-stage operations. A backend captures a
    233 versioned member plan, verifies exact member digests, and finalizes only after
    234 all expected members are present. Restore uses isolated staging and cannot
    235 replace live state before complete verification. Relative member paths reject
    236 absolute paths, traversal, duplicates, and unsafe separators.
    237 
    238 `BackupSource::stage_restore` and `finalize_restore` invoke actual owner
    239 operations. Unsupported backends return `RestoreCapabilityError::Unsupported`;
    240 reliability metadata cannot substitute for restored storage. Staging retains
    241 live state and refuses existing staging. Finalization verifies, closes the
    242 owner and installs through its recovery protocol. Hosts must reopen explicitly
    243 and reconcile historical operations before delivery. Identity, related media
    244 and durable application delivery guards remain host responsibilities.
    245 
    246 Canceling SQLite finalization retains writer authority. A subsequent explicit
    247 close drains both pools before releasing that authority, permitting guarded
    248 reopen. Cancellation is not evidence of either successful restore or rollback.
    249 
    250 Status and integrity inspection are passive. `close` is explicit and
    251 idempotent; once closed, an implementation rejects ordinary operations.
    252 Backend-specific durability fields are discriminated by `StorageBackend`:
    253 memory does not pretend to use WAL or a process writer lock, while writable
    254 SQLite status requires its governed lock, WAL, and busy-timeout contract.
    255 
    256 ## Serialization
    257 
    258 `AuthoredDraftQuery::new` selects an exact author, schema and optional scope;
    259 an absent scope selects only unscoped drafts. `AuthoredDraftQuery::for_author`
    260 explicitly traverses that author's schema across all scopes using the same
    261 bounded pages. Its version-2 continuation includes an explicit selection marker
    262 and cannot be used for an exact-scope query or another author/schema. Existing
    263 version-1 continuations keep their original bytes and meaning. Both traversals
    264 are live views: callers must revisit earlier IDs when new work can be inserted.
    265 
    266 The optional `serde` feature serializes passive identities, requests, records,
    267 receipts, status values, manifests, and coordination metadata. Deserialization
    268 revalidates invariants rather than trusting encoded revisions, digests, paths,
    269 cardinality, lifecycle transitions, or derived state.
    270 
    271 Serialization is not a database schema or wire-protocol authority. Backend
    272 schemas are private to their implementation, and cross-language runtime DTOs
    273 remain owned by `radroots_protocol`.
    274 
    275 ## Features
    276 
    277 | Feature | Default | Contract |
    278 | --- | --- | --- |
    279 | `command_journal` | no | Enables typed protected command admission and complete chat-plan bind/query contracts through the portable Protocol profile. It starts no worker and grants no execution authority. |
    280 | `memory` | yes | Enables the deterministic bounded in-memory reference backend. It installs no task, clock, entropy source, filesystem, or global state. |
    281 | `serde` | yes | Adds validated serialization to passive storage values. It does not serialize backend handles, active secret capabilities, or transactions. |
    282 
    283 Features are additive. `--no-default-features` exposes the backend-neutral SPI
    284 without an implementation. `memory` and `serde` are supported independently,
    285 and `--all-features` enables all declared features.
    286 
    287 ## Paired authored revisions
    288 
    289 `AuthoredDraftStore::append_authored_draft_pair` installs exactly two distinct
    290 opaque draft revisions for one author in one atomic commit. Both expected heads
    291 and existing per-draft bounds apply. A partial existing pair or either conflict
    292 leaves both heads unchanged. Unsupported backends return `BackendUnavailable`;
    293 there is no sequential-write fallback.
    294 
    295 Exact replay returns both historical snapshots even after their heads advance.
    296 It proves the requested snapshots exist, not current ownership or permission to
    297 sign or deliver. Callers retain application policy and must recheck current
    298 heads before effects. Losing the result after commit cannot establish rollback.
    299 
    300 ## Capacity failures
    301 
    302 `Error::SpaceInsufficient` reports exhausted storage capacity without exposing
    303 backend details. It does not prove rollback or absence of earlier effects.
    304 Retain pending requests, original operation identities and ambiguous receipts;
    305 reconcile existing state before retrying. Capacity diagnosis grants no eviction
    306 or automatic retry authority. Backend implementations that cannot distinguish
    307 capacity failures may continue returning `BackendUnavailable`.
    308 
    309 ## Intended consumers
    310 
    311 - `radroots_storage_sqlite` implements the contracts for native durable state.
    312 - `radroots_sync` coordinates bounded ingest, projection, enqueue, and delivery.
    313 - `radroots_sdk` composes storage with signing and transport implementations.
    314 - Service and application hosts may implement or inject a backend while
    315   retaining ownership of paths, runtime, cancellation, clocks, and lifecycle.
    316 
    317 Applications that only need ordinary Radroots operations should normally use
    318 `radroots` or `radroots_sdk`. Implement this package directly when providing a
    319 storage backend or advanced host composition.
    320 
    321 ## Copyright
    322 
    323 Except as otherwise noted, all files in the `radroots_storage` distribution are
    324 
    325  Copyright (c) 2025 Tyson Lupul
    326 
    327 For information on usage and redistribution, and for a DISCLAIMER OF ALL
    328 WARRANTIES, see LICENSE included in the `radroots_storage` distribution.
    329 
    330 ## Protected command and signing-plan transactions
    331 
    332 With `command_journal`, `command::OperationJournal` defines admission/query and
    333 complete immutable signing-plan bind/query, first-claim and obtained-evidence
    334 capture/query transactions. Concrete adapters own
    335 current authentication, exact indexed-fact comparisons, fixed first-issue
    336 windows and observed commit/rollback completion. `PlanBindingOutcome::Unknown`
    337 retains the original workspace, operation and command anchor for reconciliation;
    338 a missing observer cannot establish rollback. `BoundSigningPlan` is a structural
    339 store report, separate from signature, business installation and delivery.
    340 
    341 
    342 `ClaimAttemptOutcome::Claimed` reports an actual newly committed claim;
    343 `Retained` and `Unknown` never establish permission to invoke again. Structural
    344 constructors do not qualify an adapter's persistence or dispatch behavior.
    345 `SealEvidence::verify` authenticates the bounded original signature bytes and
    346 every expected field; native capture must additionally compare its own retained
    347 claim. Capture borrows the caller's evidence and credential so failure cannot
    348 consume the sole obtained copy. It preserves the first optional observation and
    349 does not apply a new issuance deadline. Queries still require current access.
    350 
    351 
    352 `OperationJournal::operation_snapshot` queries the complete original T0/T1/S
    353 graph coherently. The concrete adapter authenticates every retained claim and
    354 obtained artifact, then compares both presence and absence together before
    355 returning. Per-recipient reports share one owned plan; bounded boxed collections
    356 discard spare caller capacity. The scoped progress digest excludes query time
    357 and diagnostic correlation. Structural snapshot construction supplies no
    358 proof of backend qualification, dispatch permission or business installation.
    359 
    360 `install_transition_and_outbox` requires original operation/command/plan
    361 selectors. Concrete owners atomically install one logical chat result and retain
    362 separate publication work for actual obtained recipient artifacts. Original
    363 installation and later work transactions remain distinct. A coherent snapshot
    364 may carry the owned bounded message reconstructed from the original protected
    365 command and plan, with validated installation/basis/work reports. Structural
    366 constructors supply no authentication or persistence guarantee. Uncertain T2
    367 completion never releases claims or obtained evidence.
    368 
    369 Public T2 reports use `ListingInstallation` and `PublicPublicationWork`. The
    370 snapshot exposes the original coordinate/revision and exact evidence/work
    371 references without inventing a message or private recipient. Concrete owners
    372 must fence the initial coordinate before issuance and atomically retain the
    373 revision and work once. First-issue expiry does not erase authenticated obtained
    374 evidence or impose a new T2 deadline; current access still gates installation.
    375 
    376 Delivery preparation retains an exact outer representation and complete bounded
    377 target intent against actual original publication work before transport. Only
    378 an actual fresh confirmed claim may proceed once through its qualified owner;
    379 replay and unknown outcomes grant no dispatch right. Retries retain new attempt
    380 identities while reusing original exact bytes. Canonical endpoint budgets cannot
    381 be reset by changing a target scope.
    382 
    383 The first full target identity set and completion policy stay fixed for each
    384 recipient's work. Each attempt independently retains a nonempty exact subset;
    385 only those endpoints spend attempt slots. Shared acceptance evaluation accumulates
    386 original target facts across retries without weakening the full criterion.
    387 An exhausted optional endpoint cannot block a required endpoint. A distinct
    388 current-access attempt may retain a new delivery deadline, while replay keeps
    389 its original deadline and never repeats participant issuance.
    390 
    391 `record_delivery_observation` appends independently scoped accepted, rejected,
    392 timeout, disconnect, unknown and unattempted facts. Original credential and
    393 actual retained attempt linkage are required; late capture grants no new lease
    394 or read rights. The first original optional time in each class remains unchanged.
    395 Snapshots carry bounded summaries and original observations without duplicating
    396 raw outer bytes. Their v3 progress digest includes these facts; structural
    397 construction and an adapter's report establish no backend qualification.
    398 
    399 The shared inventory boundary owns `StockAcceptanceRequest`,
    400 `StockAcceptanceReceipt` and `StockAcceptanceOutcome`. Its
    401 `InventoryStore::prepare_stock_acceptance` port requires the concrete host's
    402 actual protected T0 transaction. It does not sign or install an agreement.
    403 Requests copy bounded captured commands and retain independent inventory,
    404 private-source and public-listing minima. Report construction checks bounded
    405 metadata, not consent, authority, actual holds or a commit. Unsupported hosts
    406 refuse explicitly; no memory or runtime fallback is constructed. The native
    407 adapter uses its selected existing worker. Signing/Nostr remain outside Storage's
    408 dependency graph. Complete ordinary commerce ownership remains separate work.
    409 
    410 `InventoryStore::capture_stock_evidence` borrows a bounded zeroizing
    411 `StockEvidenceRequest`; the caller retains original bytes across errors or an
    412 unknown append. The request contains untrusted original selectors and at most
    413 64 KiB of wire bytes. A concrete host authenticates the actual retained claim
    414 and verifies its operation, command digest, recipient, intent and signature.
    415 `StockEvidenceReceipt` and `StockEvidenceOutcome` report append identity only.
    416 Their constructors provide no protected claim, authority, installation or safe
    417 release proof. Capture neither signs nor renews current issuing permission.
    418 Unsupported hosts refuse. An ordinary owner must separately retain admitted
    419 tasks, failed captures and resource leases until durable settlement.
    420 
    421 `InventoryStore::reconcile_stock_evidence` explicitly resolves original capture
    422 after an error or unknown append. A qualified host authenticates retained capture
    423 ownership before replacing an uncertain connection, keeps its namespace writer
    424 lease through read-only preflight and validates the existing namespace without
    425 initialization or migration. Confirmation permits only the matching original
    426 capture; no first signature, new command or renewed issuing grant is supplied.
    427 The borrowed evidence remains available after failure. Unsupported hosts refuse;
    428 ordinary capture is not an implicit recovery fallback.