README.md (14887B)
1 # radroots_storage 2 3 `radroots_storage` defines the backend-neutral persistence contracts used by 4 Radroots hosts. It owns canonical event persistence, durable operation journal 5 state, outbox and delivery evidence, projection coordination, protected-record 6 metadata, reliability operations, and high-level atomic workflow commits. 7 8 Projection document inventory selects one projection and explicitly selects 9 one generation or all generations. Pages contain at most 256 records and 10 16 MiB of opaque values, with independently scoped continuations and explicit 11 corrupt-record locators. This is a live scan, not a frozen snapshot. Callers 12 must fence their own mutations before treating a traversal as complete. 13 Storage neither interprets document payloads nor grants deletion authority. 14 15 The package does not expose SQL, filesystem handles, database pools, raw 16 transactions, encryption keys, transport clients, schedulers, or application 17 state. Concrete backends implement these contracts; `radroots_storage_sqlite` 18 is the native durable backend and the opt-in [`memory`] module is the bounded 19 deterministic reference implementation. 20 21 The authoritative package charter is the 22 [`radroots_storage` section of the Release V1 specification](../../contracts/crates/release_v1/radroots_crates_release_v1.toml). 23 24 ## Typical flow 25 26 1. A host selects a concrete backend and owns its lifecycle. 27 2. The host records operation preparation in [`Journal`] with a caller-owned 28 idempotency key. 29 3. Verified events enter [`EventStore`] with explicit source provenance. 30 4. [`Outbox`] stores transport-neutral delivery intent and evidence. 31 5. [`ProjectionStore`] records checkpoints and rebuild coordination while 32 domain reducers remain outside storage. 33 6. Advanced hosts use [`atomic::AtomicStorage`] to commit related journal, 34 event, outbox, and projection transitions as one local durable operation. 35 7. [`BackupSource`] exposes explicit backup, staged restore, integrity, status, 36 and close operations without leaking backend handles. 37 38 [`Journal`]: crate::Journal 39 [`EventStore`]: crate::EventStore 40 [`Outbox`]: crate::Outbox 41 [`ProjectionStore`]: crate::ProjectionStore 42 [`BackupSource`]: crate::BackupSource 43 44 ```rust 45 use futures_executor::block_on; 46 use radroots_storage::{ 47 BackupSource, 48 event::SourceGeneration, 49 memory::MemoryStorage, 50 status::{ShutdownState, StorageBackend}, 51 }; 52 53 let generation = SourceGeneration::new([1; 32])?; 54 let storage = MemoryStorage::new(generation); 55 let status = block_on(BackupSource::status(&storage))?; 56 57 assert_eq!(status.backend(), StorageBackend::Memory); 58 assert_eq!(status.shutdown(), ShutdownState::Open); 59 # Ok::<(), radroots_storage::Error>(()) 60 ``` 61 62 The same program is available as 63 [`examples/memory_status.rs`](examples/memory_status.rs). 64 65 ## Public capability boundary 66 67 The crate root exposes the ordinary aggregate [`Storage`] capability plus 68 `EventStore`, `Journal`, `Outbox`, `ProjectionStore`, `BackupSource`, 69 `StorageStatus`, and `Error`. Advanced contracts remain in their owning 70 modules so ordinary consumers do not accidentally depend on workflow or 71 protected-record internals. 72 73 All SPIs are externally implementable, dyn-compatible `Send + Sync` traits. 74 Their methods return boxed `Future + Send` values, allowing the host to choose 75 the async executor. Implementations must not install an executor, spawn hidden 76 workers, read a clock, generate identities, or perform implicit retries. 77 78 `BackupSource::settle_backup_writes` waits for earlier owner writes, including 79 work whose caller was cancelled. The host excludes new writes before settling 80 and retains that exclusion through related inventory and capture. Settling does 81 not create a snapshot or an ongoing reservation. 82 83 `BackupSource::capture_backup`, `verify_backup`, and `finalize_backup` invoke 84 actual owner operations. Their default implementation returns typed 85 `BackupCapabilityError::Unsupported`; reliability metadata transitions cannot 86 substitute for a snapshot. The boundary returns manifests and completion, never 87 paths or database handles. Hosts coordinate application state and referenced 88 files separately; per-member snapshots do not imply a global transaction. 89 90 Actual backup capture can retain incomplete staging after interruption. It 91 does not replace live data or report that staging as finalized. The host must 92 retain the original plan and manifest and reconcile an ambiguous result before 93 retrying; cancellation never authorizes deleting retained evidence. 94 95 ## Cancellation and commit points 96 97 Dropping a returned future requests cancellation. Read operations may stop 98 without side effects. Transactional record operations preserve their local 99 durable commit point: 100 101 - before the commit point, cancellation or failure leaves no partial state; 102 - after a successful commit, cancellation cannot claim rollback; 103 - replaying the same identity and canonical input returns the original result; 104 - reusing an identity with different input fails as a conflict; 105 - atomic workflow commits publish every requested mutation or none of them; 106 - rollback failure never replaces the primary operation failure. 107 108 `JournalState::Committed` is the durable operation-journal boundary. 109 `AtomicStorage::commit` is the aggregate local workflow boundary. Network 110 publication is outside this crate and is not implied by either state. 111 112 `AuthoredAtomicCommand::RecordSigned` retains an already-created, cryptographically 113 verified signature after its original signing lease expires or is superseded. 114 The backend retrieves and checks its immutable signing claim receipt, including 115 the full claim, operation, artifact and exact retained plan. This command performs 116 no signing and supplies no permission to schedule another phase. Active 117 `ApplySigned` and work claims retain their strict fences. 118 119 The first exact signed bytes are immutable. Repeated identical evidence is 120 idempotent; different raw bytes conflict. Cancelled or terminally failed signing 121 may retain valid bytes while preserving its stop and failure state, with no 122 admission or delivery claim. Receipt replay returns historical state, so hosts 123 must query current durable status before deciding what work may follow. 124 125 ## Events, journal, outbox, and projections 126 127 Event storage preserves the exact signed event, verification/admission stage, 128 source generation, monotonically increasing position, and every unique 129 transport provenance observation. Queries are bounded and generation-aware; 130 backends fail closed on corrupt rows or source changes. 131 132 Current replacement heads are selected from verified and visible admissions 133 before interpreting application payloads. A verified-only winner supersedes an 134 older visible record without itself becoming visible. Raw records cannot select 135 heads, and only visible contract-valid author-authorized deletion requests 136 suppress events. Deleted winners never revive predecessors. Consumers use the 137 shared visibility snapshot digest to detect admission changes even when the raw 138 event count remains unchanged. 139 140 The journal records a command lifecycle under a validated idempotency key and 141 optimistic revision. The outbox persists explicit multi-target delivery plans, 142 leases, attempts, normalized receipts, partial success, and satisfaction 143 evidence without owning a transport adapter. Projection storage owns only 144 checkpoints, invalidation/rebuild state, and event-index manifests; reducer 145 algorithms and projected domain rows stay with their domain owners. 146 147 Idempotency-key construction validates borrowed input before allocating its 148 bounded owned representation, so rejected oversized input cannot force a 149 second attacker-sized allocation at this public boundary. 150 151 ## Draft queries and atomic submission 152 153 `AuthoredDraftStore::query_authored_drafts` returns bounded current-head pages 154 under an independently selected author, payload schema and optional immutable 155 scope. Continue using the returned scoped cursor. Corrupt rows have individual 156 repair locators; applications retain their evidence and continue other work. 157 Pages scan stable draft IDs, so a later sweep must revisit new IDs inserted 158 behind the cursor. A page holds at most 256 records and 4 MiB of decoded payload. 159 160 `AuthoredAtomicCommand::PrepareFromDraft` joins a captured source revision, 161 a distinct initial intent and the existing authored preparation in one commit. 162 The application puts the complete frozen semantic request in the intent payload 163 and owns strict payload validation. Reserve a stable command ID before effects. 164 The author and command identify the receipt; context, source, intent and every 165 preparation field are compared for exact replay. Replay precedes fresh source 166 CAS and survives later editing. The source remains editable. The stored 167 submission receipt retains the immutable source-to-intent association, and the 168 ordinary Prepare receipt allows existing signing orchestration to resume. 169 170 No signing or transport effect occurs in this storage transaction. Public 171 backends must implement the new query method and submission command explicitly. 172 173 ## Protected metadata and security 174 175 `private_artifact` stores bounded metadata and opaque durable secret references. 176 It never accepts domain plaintext, ciphertext bytes, or an active secret 177 capability. Encryption, wrapping, and provider access belong to 178 `radroots_secrets` and the concrete backend. 179 180 Identifiers, paths, query sizes, revisions, timestamps, and result sets are 181 bounded and validated before backend work. Secret references and idempotency 182 keys have redacted diagnostics. Public errors are stable and do not expose SQL, 183 filesystem, key-provider, or transport implementation messages. The crate 184 forbids unsafe code. 185 186 ## Backup, restore, integrity, and close 187 188 Backup and restore are explicit multi-stage operations. A backend captures a 189 versioned member plan, verifies exact member digests, and finalizes only after 190 all expected members are present. Restore uses isolated staging and cannot 191 replace live state before complete verification. Relative member paths reject 192 absolute paths, traversal, duplicates, and unsafe separators. 193 194 `BackupSource::stage_restore` and `finalize_restore` invoke actual owner 195 operations. Unsupported backends return `RestoreCapabilityError::Unsupported`; 196 reliability metadata cannot substitute for restored storage. Staging retains 197 live state and refuses existing staging. Finalization verifies, closes the 198 owner and installs through its recovery protocol. Hosts must reopen explicitly 199 and reconcile historical operations before delivery. Identity, related media 200 and durable application delivery guards remain host responsibilities. 201 202 Canceling SQLite finalization retains writer authority. A subsequent explicit 203 close drains both pools before releasing that authority, permitting guarded 204 reopen. Cancellation is not evidence of either successful restore or rollback. 205 206 Status and integrity inspection are passive. `close` is explicit and 207 idempotent; once closed, an implementation rejects ordinary operations. 208 Backend-specific durability fields are discriminated by `StorageBackend`: 209 memory does not pretend to use WAL or a process writer lock, while writable 210 SQLite status requires its governed lock, WAL, and busy-timeout contract. 211 212 ## Serialization 213 214 `AuthoredDraftQuery::new` selects an exact author, schema and optional scope; 215 an absent scope selects only unscoped drafts. `AuthoredDraftQuery::for_author` 216 explicitly traverses that author's schema across all scopes using the same 217 bounded pages. Its version-2 continuation includes an explicit selection marker 218 and cannot be used for an exact-scope query or another author/schema. Existing 219 version-1 continuations keep their original bytes and meaning. Both traversals 220 are live views: callers must revisit earlier IDs when new work can be inserted. 221 222 The optional `serde` feature serializes passive identities, requests, records, 223 receipts, status values, manifests, and coordination metadata. Deserialization 224 revalidates invariants rather than trusting encoded revisions, digests, paths, 225 cardinality, lifecycle transitions, or derived state. 226 227 Serialization is not a database schema or wire-protocol authority. Backend 228 schemas are private to their implementation, and cross-language runtime DTOs 229 remain owned by `radroots_protocol`. 230 231 ## Features 232 233 | Feature | Default | Contract | 234 | --- | --- | --- | 235 | `memory` | yes | Enables the deterministic bounded in-memory reference backend. It installs no task, clock, entropy source, filesystem, or global state. | 236 | `serde` | yes | Adds validated serialization to passive storage values. It does not serialize backend handles, active secret capabilities, or transactions. | 237 238 Features are additive. `--no-default-features` exposes the backend-neutral SPI 239 without an implementation. `memory` and `serde` are supported independently, 240 and `--all-features` enables both. 241 242 ## Paired authored revisions 243 244 `AuthoredDraftStore::append_authored_draft_pair` installs exactly two distinct 245 opaque draft revisions for one author in one atomic commit. Both expected heads 246 and existing per-draft bounds apply. A partial existing pair or either conflict 247 leaves both heads unchanged. Unsupported backends return `BackendUnavailable`; 248 there is no sequential-write fallback. 249 250 Exact replay returns both historical snapshots even after their heads advance. 251 It proves the requested snapshots exist, not current ownership or permission to 252 sign or deliver. Callers retain application policy and must recheck current 253 heads before effects. Losing the result after commit cannot establish rollback. 254 255 ## Capacity failures 256 257 `Error::SpaceInsufficient` reports exhausted storage capacity without exposing 258 backend details. It does not prove rollback or absence of earlier effects. 259 Retain pending requests, original operation identities and ambiguous receipts; 260 reconcile existing state before retrying. Capacity diagnosis grants no eviction 261 or automatic retry authority. Backend implementations that cannot distinguish 262 capacity failures may continue returning `BackendUnavailable`. 263 264 ## Intended consumers 265 266 - `radroots_storage_sqlite` implements the contracts for native durable state. 267 - `radroots_sync` coordinates bounded ingest, projection, enqueue, and delivery. 268 - `radroots_sdk` composes storage with signing and transport implementations. 269 - Service and application hosts may implement or inject a backend while 270 retaining ownership of paths, runtime, cancellation, clocks, and lifecycle. 271 272 Applications that only need ordinary Radroots operations should normally use 273 `radroots` or `radroots_sdk`. Implement this package directly when providing a 274 storage backend or advanced host composition. 275 276 ## Copyright 277 278 Except as otherwise noted, all files in the `radroots_storage` distribution are 279 280 Copyright (c) 2025 Tyson Lupul 281 282 For information on usage and redistribution, and for a DISCLAIMER OF ALL 283 WARRANTIES, see LICENSE included in the `radroots_storage` distribution.