README.md (24659B)
1 # radroots_storage 2 3 `radroots_storage` defines the backend-neutral persistence contracts used by 4 Radroots hosts. It owns canonical event persistence, durable operation journal 5 state, outbox and delivery evidence, projection coordination, protected-record 6 metadata, reliability operations, and high-level atomic workflow commits. 7 8 Projection document inventory selects one projection and explicitly selects 9 one generation or all generations. Pages contain at most 256 records and 10 16 MiB of opaque values, with independently scoped continuations and explicit 11 corrupt-record locators. This is a live scan, not a frozen snapshot. Callers 12 must fence their own mutations before treating a traversal as complete. 13 Storage neither interprets document payloads nor grants deletion authority. 14 15 The package does not expose SQL, filesystem handles, database pools, raw 16 transactions, encryption keys, transport clients, schedulers, or application 17 state. Concrete backends implement these contracts; `radroots_storage_sqlite` 18 is the native durable backend and the opt-in [`memory`] module is the bounded 19 deterministic reference implementation. 20 21 The authoritative package charter is the 22 [`radroots_storage` section of the Release V1 specification](../../contracts/crates/release_v1/radroots_crates_release_v1.toml). 23 24 ## Typical flow 25 26 1. A host selects a concrete backend and owns its lifecycle. 27 2. The host records operation preparation in [`Journal`] with a caller-owned 28 idempotency key. 29 3. Verified events enter [`EventStore`] with explicit source provenance. 30 4. [`Outbox`] stores transport-neutral delivery intent and evidence. 31 5. [`ProjectionStore`] records checkpoints and rebuild coordination while 32 domain reducers remain outside storage. 33 6. Advanced hosts use [`atomic::AtomicStorage`] to commit related journal, 34 event, outbox, and projection transitions as one local durable operation. 35 7. [`BackupSource`] exposes explicit backup, staged restore, integrity, status, 36 and close operations without leaking backend handles. 37 38 [`Journal`]: crate::Journal 39 [`EventStore`]: crate::EventStore 40 [`Outbox`]: crate::Outbox 41 [`ProjectionStore`]: crate::ProjectionStore 42 [`BackupSource`]: crate::BackupSource 43 44 ```rust 45 use futures_executor::block_on; 46 use radroots_storage::{ 47 BackupSource, 48 event::SourceGeneration, 49 memory::MemoryStorage, 50 status::{ShutdownState, StorageBackend}, 51 }; 52 53 let generation = SourceGeneration::new([1; 32])?; 54 let storage = MemoryStorage::new(generation); 55 let status = block_on(BackupSource::status(&storage))?; 56 57 assert_eq!(status.backend(), StorageBackend::Memory); 58 assert_eq!(status.shutdown(), ShutdownState::Open); 59 # Ok::<(), radroots_storage::Error>(()) 60 ``` 61 62 The same program is available as 63 [`examples/memory_status.rs`](examples/memory_status.rs). 64 65 ## Public capability boundary 66 67 The crate root exposes the ordinary aggregate [`Storage`] capability plus 68 `EventStore`, `Journal`, `Outbox`, `ProjectionStore`, `BackupSource`, 69 `StorageStatus`, and `Error`. Advanced contracts remain in their owning 70 modules so ordinary consumers do not accidentally depend on workflow or 71 protected-record internals. 72 73 All SPIs are externally implementable and dyn-compatible. Native traits retain 74 `Send + Sync` and boxed `Future + Send`; browser traits and futures may stay 75 local. Every SPI uses the same target-aware transport host bounds and future 76 alias, allowing the host to choose the async executor. SPI declarations and 77 pure value imports do not 78 install an executor, spawn hidden 79 workers, read a clock, generate identities, or perform implicit retries. 80 81 The opt-in `protected_outbox` feature supplies pure local buyer/seller-outbox 82 enrollment at `workspace::local_outbox` and the validated shared protected-grant 83 codec at `workspace::grant`. The separate `workspace::seller_stock` policy 84 selects fresh stock-owner enrollment for a qualified native host; pure policy 85 values confer no durable authority. 86 `LocalOutboxConfig::new` selects a fresh buyer outbox; `seller` explicitly 87 selects a fresh seller public-authoring/communication outbox. Authenticated 88 bindings retain that exact scope. Neither scope grants firm stock, delegation, 89 organization access or recovery of old obligations. Scope separates route, 90 genesis and credential commitments; changing an actor cannot upgrade a grant. 91 Concrete native/browser hosts share these rules and exact envelope context. 92 Shared `inventory::StockAbortRequest`, `StockAbortOutcome` and `StockAbortReceipt` 93 carry explicit original-preparation cleanup selectors and reports. 94 `InventoryStore::abort_proven_unissued` and `stock_preparation_abort` are typed 95 transaction/query ports; unsupported hosts return `UnsupportedOperation`, whose 96 stable code is `capability_unavailable`. 97 Report reconstruction checks namespace, committed anchor, progress and bounded 98 canonical after-images, and supplies no credential, absence or commit proof. 99 Qualified stores must authenticate all original claims and exact release effects 100 inside their owned atomic transaction. The native implementation uses its same 101 bounded connection owner; ordinary outboxes acquire no stock authority. 102 The codec's Serde wire stays private; decoding enforces the 16 KiB limit and 103 validates current scope. Enrollment takes qualified host identities and proof 104 without acquiring a backend or establishing durable authority. Concrete hosts 105 must protect, commit and authenticate that state through their explicit owned 106 lifecycle. This feature does not enable a database, provider or runtime. 107 Its `command::protected` module supplies the shared native/browser immutable 108 Prepared codec and pure binding/commitment checks. The complete codec is bounded 109 to 64 KiB; a concrete host must split it to meet any smaller private-record 110 budget, protect each record and atomically retain the full set. Decoding is a 111 historical observation and supplies no current grant, actual commit or issuance. 112 New preparations retain scope explicitly in version 3. Historical version 2 113 preparations remain readable only with their original buyer-outbox authority; 114 no actor descriptor can infer a seller grant during recovery. 115 116 Common admission retains a typed command family. Fresh public listing T0 117 requires an authenticated seller outbox and valid bounded event inputs. The 118 current history and approval prerequisites remain explicit refusals. Chat-only 119 ports require actual chat data and return `capability_unavailable` for a public 120 preparation; no synthetic peer or receipt bridges the families. 121 122 `BackupSource::settle_backup_writes` waits for earlier owner writes, including 123 work whose caller was cancelled. The host excludes new writes before settling 124 and retains that exclusion through related inventory and capture. Settling does 125 not create a snapshot or an ongoing reservation. 126 127 `BackupSource::capture_backup`, `verify_backup`, and `finalize_backup` invoke 128 actual owner operations. Their default implementation returns typed 129 `BackupCapabilityError::Unsupported`; reliability metadata transitions cannot 130 substitute for a snapshot. The boundary returns manifests and completion, never 131 paths or database handles. Hosts coordinate application state and referenced 132 files separately; per-member snapshots do not imply a global transaction. 133 134 Actual backup capture can retain incomplete staging after interruption. It 135 does not replace live data or report that staging as finalized. The host must 136 retain the original plan and manifest and reconcile an ambiguous result before 137 retrying; cancellation never authorizes deleting retained evidence. 138 139 ## Cancellation and commit points 140 141 Dropping a returned future requests cancellation. Read operations may stop 142 without side effects. Transactional record operations preserve their local 143 durable commit point: 144 145 - before the commit point, cancellation or failure leaves no partial state; 146 - after a successful commit, cancellation cannot claim rollback; 147 - replaying the same identity and canonical input returns the original result; 148 - reusing an identity with different input fails as a conflict; 149 - atomic workflow commits publish every requested mutation or none of them; 150 - rollback failure never replaces the primary operation failure. 151 152 `JournalState::Committed` is the durable operation-journal boundary. 153 `AtomicStorage::commit` is the aggregate local workflow boundary. Network 154 publication is outside this crate and is not implied by either state. 155 156 `AuthoredAtomicCommand::RecordSigned` retains an already-created, cryptographically 157 verified signature after its original signing lease expires or is superseded. 158 The backend retrieves and checks its immutable signing claim receipt, including 159 the full claim, operation, artifact and exact retained plan. This command performs 160 no signing and supplies no permission to schedule another phase. Active 161 `ApplySigned` and work claims retain their strict fences. 162 163 The first exact signed bytes are immutable. Repeated identical evidence is 164 idempotent; different raw bytes conflict. Cancelled or terminally failed signing 165 may retain valid bytes while preserving its stop and failure state, with no 166 admission or delivery claim. Receipt replay returns historical state, so hosts 167 must query current durable status before deciding what work may follow. 168 169 ## Events, journal, outbox, and projections 170 171 Event storage preserves the exact signed event, verification/admission stage, 172 source generation, monotonically increasing position, and every unique 173 transport provenance observation. Queries are bounded and generation-aware; 174 backends fail closed on corrupt rows or source changes. 175 176 Current replacement heads are selected from verified and visible admissions 177 before interpreting application payloads. A verified-only winner supersedes an 178 older visible record without itself becoming visible. Raw records cannot select 179 heads, and only visible contract-valid author-authorized deletion requests 180 suppress events. Deleted winners never revive predecessors. Consumers use the 181 shared visibility snapshot digest to detect admission changes even when the raw 182 event count remains unchanged. 183 184 The journal records a command lifecycle under a validated idempotency key and 185 optimistic revision. The outbox persists explicit multi-target delivery plans, 186 leases, attempts, normalized receipts, partial success, and satisfaction 187 evidence without owning a transport adapter. Projection storage owns only 188 checkpoints, invalidation/rebuild state, and event-index manifests; reducer 189 algorithms and projected domain rows stay with their domain owners. 190 191 Idempotency-key construction validates borrowed input before allocating its 192 bounded owned representation, so rejected oversized input cannot force a 193 second attacker-sized allocation at this public boundary. 194 195 ## Draft queries and atomic submission 196 197 `AuthoredDraftStore::query_authored_drafts` returns bounded current-head pages 198 under an independently selected author, payload schema and optional immutable 199 scope. Continue using the returned scoped cursor. Corrupt rows have individual 200 repair locators; applications retain their evidence and continue other work. 201 Pages scan stable draft IDs, so a later sweep must revisit new IDs inserted 202 behind the cursor. A page holds at most 256 records and 4 MiB of decoded payload. 203 204 `AuthoredAtomicCommand::PrepareFromDraft` joins a captured source revision, 205 a distinct initial intent and the existing authored preparation in one commit. 206 The application puts the complete frozen semantic request in the intent payload 207 and owns strict payload validation. Reserve a stable command ID before effects. 208 The author and command identify the receipt; context, source, intent and every 209 preparation field are compared for exact replay. Replay precedes fresh source 210 CAS and survives later editing. The source remains editable. The stored 211 submission receipt retains the immutable source-to-intent association, and the 212 ordinary Prepare receipt allows existing signing orchestration to resume. 213 214 No signing or transport effect occurs in this storage transaction. Public 215 backends must implement the new query method and submission command explicitly. 216 217 ## Protected metadata and security 218 219 `private_artifact` stores bounded metadata and opaque durable secret references. 220 It never accepts domain plaintext, ciphertext bytes, or an active secret 221 capability. Encryption, wrapping, and provider access belong to 222 `radroots_secrets` and the concrete backend. 223 224 Identifiers, paths, query sizes, revisions, timestamps, and result sets are 225 bounded and validated before backend work. Secret references and idempotency 226 keys have redacted diagnostics. Public errors are stable and do not expose SQL, 227 filesystem, key-provider, or transport implementation messages. The crate 228 forbids unsafe code. 229 230 ## Backup, restore, integrity, and close 231 232 Backup and restore are explicit multi-stage operations. A backend captures a 233 versioned member plan, verifies exact member digests, and finalizes only after 234 all expected members are present. Restore uses isolated staging and cannot 235 replace live state before complete verification. Relative member paths reject 236 absolute paths, traversal, duplicates, and unsafe separators. 237 238 `BackupSource::stage_restore` and `finalize_restore` invoke actual owner 239 operations. Unsupported backends return `RestoreCapabilityError::Unsupported`; 240 reliability metadata cannot substitute for restored storage. Staging retains 241 live state and refuses existing staging. Finalization verifies, closes the 242 owner and installs through its recovery protocol. Hosts must reopen explicitly 243 and reconcile historical operations before delivery. Identity, related media 244 and durable application delivery guards remain host responsibilities. 245 246 Canceling SQLite finalization retains writer authority. A subsequent explicit 247 close drains both pools before releasing that authority, permitting guarded 248 reopen. Cancellation is not evidence of either successful restore or rollback. 249 250 Status and integrity inspection are passive. `close` is explicit and 251 idempotent; once closed, an implementation rejects ordinary operations. 252 Backend-specific durability fields are discriminated by `StorageBackend`: 253 memory does not pretend to use WAL or a process writer lock, while writable 254 SQLite status requires its governed lock, WAL, and busy-timeout contract. 255 256 ## Serialization 257 258 `AuthoredDraftQuery::new` selects an exact author, schema and optional scope; 259 an absent scope selects only unscoped drafts. `AuthoredDraftQuery::for_author` 260 explicitly traverses that author's schema across all scopes using the same 261 bounded pages. Its version-2 continuation includes an explicit selection marker 262 and cannot be used for an exact-scope query or another author/schema. Existing 263 version-1 continuations keep their original bytes and meaning. Both traversals 264 are live views: callers must revisit earlier IDs when new work can be inserted. 265 266 The optional `serde` feature serializes passive identities, requests, records, 267 receipts, status values, manifests, and coordination metadata. Deserialization 268 revalidates invariants rather than trusting encoded revisions, digests, paths, 269 cardinality, lifecycle transitions, or derived state. 270 271 Serialization is not a database schema or wire-protocol authority. Backend 272 schemas are private to their implementation, and cross-language runtime DTOs 273 remain owned by `radroots_protocol`. 274 275 ## Features 276 277 | Feature | Default | Contract | 278 | --- | --- | --- | 279 | `command_journal` | no | Enables typed protected command admission and complete chat-plan bind/query contracts through the portable Protocol profile. It starts no worker and grants no execution authority. | 280 | `memory` | yes | Enables the deterministic bounded in-memory reference backend. It installs no task, clock, entropy source, filesystem, or global state. | 281 | `serde` | yes | Adds validated serialization to passive storage values. It does not serialize backend handles, active secret capabilities, or transactions. | 282 283 Features are additive. `--no-default-features` exposes the backend-neutral SPI 284 without an implementation. `memory` and `serde` are supported independently, 285 and `--all-features` enables all declared features. 286 287 ## Paired authored revisions 288 289 `AuthoredDraftStore::append_authored_draft_pair` installs exactly two distinct 290 opaque draft revisions for one author in one atomic commit. Both expected heads 291 and existing per-draft bounds apply. A partial existing pair or either conflict 292 leaves both heads unchanged. Unsupported backends return `BackendUnavailable`; 293 there is no sequential-write fallback. 294 295 Exact replay returns both historical snapshots even after their heads advance. 296 It proves the requested snapshots exist, not current ownership or permission to 297 sign or deliver. Callers retain application policy and must recheck current 298 heads before effects. Losing the result after commit cannot establish rollback. 299 300 ## Capacity failures 301 302 `Error::SpaceInsufficient` reports exhausted storage capacity without exposing 303 backend details. It does not prove rollback or absence of earlier effects. 304 Retain pending requests, original operation identities and ambiguous receipts; 305 reconcile existing state before retrying. Capacity diagnosis grants no eviction 306 or automatic retry authority. Backend implementations that cannot distinguish 307 capacity failures may continue returning `BackendUnavailable`. 308 309 ## Intended consumers 310 311 - `radroots_storage_sqlite` implements the contracts for native durable state. 312 - `radroots_sync` coordinates bounded ingest, projection, enqueue, and delivery. 313 - `radroots_sdk` composes storage with signing and transport implementations. 314 - Service and application hosts may implement or inject a backend while 315 retaining ownership of paths, runtime, cancellation, clocks, and lifecycle. 316 317 Applications that only need ordinary Radroots operations should normally use 318 `radroots` or `radroots_sdk`. Implement this package directly when providing a 319 storage backend or advanced host composition. 320 321 ## Copyright 322 323 Except as otherwise noted, all files in the `radroots_storage` distribution are 324 325 Copyright (c) 2025 Tyson Lupul 326 327 For information on usage and redistribution, and for a DISCLAIMER OF ALL 328 WARRANTIES, see LICENSE included in the `radroots_storage` distribution. 329 330 ## Protected command and signing-plan transactions 331 332 With `command_journal`, `command::OperationJournal` defines admission/query and 333 complete immutable signing-plan bind/query, first-claim and obtained-evidence 334 capture/query transactions. Concrete adapters own 335 current authentication, exact indexed-fact comparisons, fixed first-issue 336 windows and observed commit/rollback completion. `PlanBindingOutcome::Unknown` 337 retains the original workspace, operation and command anchor for reconciliation; 338 a missing observer cannot establish rollback. `BoundSigningPlan` is a structural 339 store report, separate from signature, business installation and delivery. 340 341 342 `ClaimAttemptOutcome::Claimed` reports an actual newly committed claim; 343 `Retained` and `Unknown` never establish permission to invoke again. Structural 344 constructors do not qualify an adapter's persistence or dispatch behavior. 345 `SealEvidence::verify` authenticates the bounded original signature bytes and 346 every expected field; native capture must additionally compare its own retained 347 claim. Capture borrows the caller's evidence and credential so failure cannot 348 consume the sole obtained copy. It preserves the first optional observation and 349 does not apply a new issuance deadline. Queries still require current access. 350 351 352 `OperationJournal::operation_snapshot` queries the complete original T0/T1/S 353 graph coherently. The concrete adapter authenticates every retained claim and 354 obtained artifact, then compares both presence and absence together before 355 returning. Per-recipient reports share one owned plan; bounded boxed collections 356 discard spare caller capacity. The scoped progress digest excludes query time 357 and diagnostic correlation. Structural snapshot construction supplies no 358 proof of backend qualification, dispatch permission or business installation. 359 360 `install_transition_and_outbox` requires original operation/command/plan 361 selectors. Concrete owners atomically install one logical chat result and retain 362 separate publication work for actual obtained recipient artifacts. Original 363 installation and later work transactions remain distinct. A coherent snapshot 364 may carry the owned bounded message reconstructed from the original protected 365 command and plan, with validated installation/basis/work reports. Structural 366 constructors supply no authentication or persistence guarantee. Uncertain T2 367 completion never releases claims or obtained evidence. 368 369 Public T2 reports use `ListingInstallation` and `PublicPublicationWork`. The 370 snapshot exposes the original coordinate/revision and exact evidence/work 371 references without inventing a message or private recipient. Concrete owners 372 must fence the initial coordinate before issuance and atomically retain the 373 revision and work once. First-issue expiry does not erase authenticated obtained 374 evidence or impose a new T2 deadline; current access still gates installation. 375 376 Delivery preparation retains an exact outer representation and complete bounded 377 target intent against actual original publication work before transport. Only 378 an actual fresh confirmed claim may proceed once through its qualified owner; 379 replay and unknown outcomes grant no dispatch right. Retries retain new attempt 380 identities while reusing original exact bytes. Canonical endpoint budgets cannot 381 be reset by changing a target scope. 382 383 The first full target identity set and completion policy stay fixed for each 384 recipient's work. Each attempt independently retains a nonempty exact subset; 385 only those endpoints spend attempt slots. Shared acceptance evaluation accumulates 386 original target facts across retries without weakening the full criterion. 387 An exhausted optional endpoint cannot block a required endpoint. A distinct 388 current-access attempt may retain a new delivery deadline, while replay keeps 389 its original deadline and never repeats participant issuance. 390 391 `record_delivery_observation` appends independently scoped accepted, rejected, 392 timeout, disconnect, unknown and unattempted facts. Original credential and 393 actual retained attempt linkage are required; late capture grants no new lease 394 or read rights. The first original optional time in each class remains unchanged. 395 Snapshots carry bounded summaries and original observations without duplicating 396 raw outer bytes. Their v3 progress digest includes these facts; structural 397 construction and an adapter's report establish no backend qualification. 398 399 The shared inventory boundary owns `StockAcceptanceRequest`, 400 `StockAcceptanceReceipt` and `StockAcceptanceOutcome`. Its 401 `InventoryStore::prepare_stock_acceptance` port requires the concrete host's 402 actual protected T0 transaction. It does not sign or install an agreement. 403 Requests copy bounded captured commands and retain independent inventory, 404 private-source and public-listing minima. Report construction checks bounded 405 metadata, not consent, authority, actual holds or a commit. Unsupported hosts 406 refuse explicitly; no memory or runtime fallback is constructed. The native 407 adapter uses its selected existing worker. Signing/Nostr remain outside Storage's 408 dependency graph. Complete ordinary commerce ownership remains separate work. 409 410 `InventoryStore::capture_stock_evidence` borrows a bounded zeroizing 411 `StockEvidenceRequest`; the caller retains original bytes across errors or an 412 unknown append. The request contains untrusted original selectors and at most 413 64 KiB of wire bytes. A concrete host authenticates the actual retained claim 414 and verifies its operation, command digest, recipient, intent and signature. 415 `StockEvidenceReceipt` and `StockEvidenceOutcome` report append identity only. 416 Their constructors provide no protected claim, authority, installation or safe 417 release proof. Capture neither signs nor renews current issuing permission. 418 Unsupported hosts refuse. An ordinary owner must separately retain admitted 419 tasks, failed captures and resource leases until durable settlement. 420 421 `InventoryStore::reconcile_stock_evidence` explicitly resolves original capture 422 after an error or unknown append. A qualified host authenticates retained capture 423 ownership before replacing an uncertain connection, keeps its namespace writer 424 lease through read-only preflight and validates the existing namespace without 425 initialization or migration. Confirmation permits only the matching original 426 capture; no first signature, new command or renewed issuing grant is supplied. 427 The borrowed evidence remains available after failure. Unsupported hosts refuse; 428 ordinary capture is not an implicit recovery fallback.