README.md (23476B)
1 # radroots_service_sqlite 2 3 `radroots_service_sqlite` is the unpublished, native SQLite-mechanism crate for 4 Radroots services. It provides narrow, reusable building blocks for exclusive 5 writer authority, instance locking, versioned schema mechanics, bounded 6 transactions, immutable service-instance database identity, integrity checks, 7 backup, restore, and passive storage status. 8 9 `ServiceSqliteHost` is the only public connection host. Its SQLx pool and raw 10 connections are sealed inside the crate. Services run typed SQLx queries through 11 the borrowed `ServiceSqliteTransaction` executor supplied by 12 `ServiceSqliteHost::transaction`; transaction begin, commit, rollback, policy 13 validation, attached-database exclusion, and cancellation quarantine remain 14 runner-owned. Writable host opening finishes every pending governed migration 15 before returning, and read-only inspection opens only current migration and 16 schema state. 17 18 Create-new state uses the borrowed `ServiceSqliteInitializer` executor rather 19 than a database path or raw connection. The runner reserves and retains the 20 exact canonical file, opens SQLite only through that retained descriptor, owns 21 `BEGIN IMMEDIATE`, and uses a private memory journal so descriptor-bound 22 initialization creates no path-derived SQLite sidecar. It commits the product 23 schema, shared `radroots_service_metadata`, empty v1 `schema_migrations` 24 ledger, and exact schema-catalog verification in one transaction. The 25 initializer screens the same closed transaction-control and attachment 26 inventory as host transactions; an ignored rejection still prevents commit. 27 Callback failure or cancellation cannot return a reusable connection or 28 publish a partial database. 29 30 Interactive callers may use `ServiceSqliteHost::open_or_initialize`. One held 31 writer authority and an exclusive create decide whether the sealed initializer 32 runs or the exact existing database is opened. The existing branch never runs 33 the initializer. Callers therefore do not use pathname probes, error-text 34 matching, recursive directory creation, permission repair, or direct SQLx 35 connections to choose the bootstrap branch. Success returns an 36 `OpenedServiceDatabase` that binds the retained host to the actual verified 37 metadata from the selected branch, including the persisted source generation 38 when state already existed. 39 40 Existing databases can be admitted without a caller guessing their stored 41 source generation. `ExistingServiceDatabaseIntent` seals the canonical 42 service and instance, supported schema ceiling, and SQLite application ID; 43 `ServiceSqliteHost::open_read_write_existing_with_intent` and 44 `ServiceSqliteHost::open_read_only_inspection_with_intent` discover and verify 45 the actual immutable metadata while retaining the corresponding writer or 46 inspection authority. Success returns an `OpenedServiceDatabase`, 47 which keeps the host and verified metadata inseparable until the caller 48 consumes them together. Recovery remains fail closed and may use this intent 49 only to discover the marker-bound generation; every other identity dimension, 50 artifact binding, migration prefix, and schema catalog remains governed. 51 52 Service-controlled SQL is screened before SQLite compilation through both the 53 borrowed transaction executor and migration callback executor. The closed 54 statement-control inventory is `PRAGMA`, `ATTACH`, `DETACH`, `BEGIN`, `COMMIT`, 55 `END`, `ROLLBACK`, `SAVEPOINT`, and `RELEASE`, regardless of case, whitespace, 56 comments, multiple statements, or prepared-query entry point. Rejection is 57 sticky for the transaction, so ignoring the immediate SQL error cannot permit 58 commit. Runner-owned setup remains private, and the complete connection policy 59 is revalidated before commit and before a connection can return to the pool. 60 61 Persisted SQLite text and blob values are admitted through bounded projections 62 before Rust decoding. Service and instance identifiers, source generations, 63 migration names, checksums, build identity, schema text, and database inventory 64 values carry an exact SQLite type, reported byte length, capped byte prefix, 65 and bounded row count. Integrity diagnostics use a borrowed-byte cap over the 66 exact `PRAGMA integrity_check(1)` operation and never convert an unbounded 67 diagnostic to UTF-8. 68 69 Initialization reports success only after its governed commit and cleanup 70 succeed. Hook removal and connection close are attempted even when commit or 71 rollback fails; successful cleanup cannot hide an earlier failure. A failed 72 commit's SQLite cause survives successful cleanup and cannot grant initialized 73 state or writer authority. 74 75 Cancelling a host transaction before the runner enables outer commit 76 quarantines its connection and leaves no authoritative transaction effect. A 77 service-operation error is returned only after rollback is confirmed; an 78 unconfirmed rollback is reported as `RollbackFailed`. Cancelling once outer 79 commit begins yields no result and must be treated as an unknown commit outcome. 80 Both that case and `CommitOutcomeUnknown` require rereading authoritative state 81 before an idempotent retry. 82 83 Every host must be closed explicitly with `ServiceSqliteHost::close`. Close 84 permanently stops new transaction admission, drains transactions that were 85 already admitted, and is safe to call sequentially or concurrently. Writable 86 close applies the fixed `PRAGMA wal_checkpoint(TRUNCATE)` policy, requires an 87 unblocked checkpoint, closes the private checkpoint connection, and explicitly 88 releases writer authority. Read-only inspection close drains its pool and 89 releases its shared inspection guard without checkpointing or mutating the 90 database or filesystem. Cancelling close before terminal completion leaves the 91 host non-admitting and retains authority; the private connect, checkpoint, and 92 explicit connection-close driver remains host-owned so a later call resumes 93 close without losing the SQLite handle or its close proof. Once authority 94 release is proven, the stable outer result is cached for every later call. 95 Dropping a host performs no asynchronous close work and is not proof that the 96 governed checkpoint and authority-release sequence completed. 97 98 `ServiceBackupManifest` is the stable model-only v1 backup identity. It admits 99 only compact canonical UTF-8 JSON in the frozen field order, capped at 1,024 100 bytes, and computes the external manifest SHA-256 over those exact bytes. The 101 v1 member array contains exactly one `state.sqlite` member with a nonzero byte 102 length and lowercase SHA-256; service, instance, nonzero source generation, 103 state schema, and injected creation time are explicit. SQLite and foreign-key 104 integrity are exactly `ok`, and protected material is always excluded. 105 Parsing proves only the strict structural and canonical contract. It rejects 106 unknown, duplicate, null, reordered, whitespace-altered, or version-drifted 107 input; member bytes, digest, SQLite identity, and actual integrity remain the 108 separate backup-verification boundary. Constructing or parsing the manifest 109 model performs no filesystem or SQLite work. 110 111 Writable hosts provide `ServiceSqliteHost::capture_online_backup` for one 112 incremental, point-in-time SQLite capture at a time. The caller supplies an 113 injected creation time and the exact new absolute staging-directory path under 114 an existing owner-controlled parent; capture creates that directory with mode 115 `0700` and its sole `state.sqlite` member with mode `0600`. It uses SQLite's 116 online-backup API without checkpointing or copying the live source file, then 117 requires exact service metadata, bounded `integrity_check`, an empty 118 `foreign_key_check`, a singleton member inventory, SHA-256, and file, staging, 119 and parent synchronization before returning the canonical manifest in memory. 120 No manifest file, bundle identifier, credential, or protected material is 121 written to the staging directory. 122 123 Capture rejects read-only, closing, unsupported, colliding, or concurrent 124 admission before publishing a result. Dropping the capture future requests 125 cancellation; the blocking worker retains its checked-out pool admission, 126 writer authority, and exact staging identities until SQLite handles are closed 127 and cleanup completes. Host close therefore drains capture and cancellation 128 cleanup before it checkpoints or releases authority. Capture has no hidden 129 timeout: callers own any deadline by cancelling the future. A completed capture 130 is still untrusted backup input until the separate verifier binds its manifest, 131 member bytes, expected intent, application metadata, and integrity; capture 132 does not provide restore or replacement behavior. 133 134 `verify_backup_bundle` is the synchronous, task-free boundary for an untrusted 135 manifest and bundle. The caller supplies the independently protected manifest 136 SHA-256, expected service database identity, and a positive maximum state-file 137 size. Verification requires canonical manifest bytes, exact service, instance, 138 source-generation, schema, and application intent, a restrictive owner-only 139 directory containing only `state.sqlite`, the exact bounded member length and 140 digest, immutable read-only/query-only SQLite access, main-only attachment, 141 bounded application metadata, `integrity_check(1)`, and an empty foreign-key 142 check. It performs no filesystem mutation and does not create a task or hidden 143 deadline. 144 145 Success returns a non-forgeable `VerifiedServiceBackup` that retains the exact 146 verified directory and member descriptors while exposing only the canonical 147 manifest and actual database metadata. It is not restore or replacement 148 authority, and it exposes no path or raw handle. Later restore work must copy 149 from the retained member and reverify the staged copy under its own supervised 150 blocking worker and deadline; pathname verification alone is insufficient. 151 152 Restore crash recovery uses a private sealed v1 marker stored beside canonical 153 service state. Its fixed layout names the live `state.sqlite`, staged 154 `state.restore-staged.sqlite`, retained `state.restore-backup.sqlite`, durable 155 `state.restore-marker.v1`, and create-new update scratch 156 `state.restore-marker.v1.next`. The compact canonical JSON is capped at 2,048 157 bytes and binds typed database intent, the protected source-manifest digest, 158 and exact live, staged, and retained-backup device, inode, length, and SHA-256 159 expectations. A domain-separated checksum binds the canonical fields and 160 detects corruption; it is not an authenticity credential. 161 162 The only legal durable sequence is `prepared` to `live_retained` to 163 `replacement_installed`; repeating the current phase is byte-idempotent and 164 every skip, reversal, or post-install transition fails closed. Marker files are 165 descriptor-relative, no-follow, single-link, owner-owned regular files with 166 mode `0600`. Creation synchronizes the file and state directory. Advancement 167 compare-and-reloads the current bytes, writes and synchronizes the fixed 168 create-new scratch, atomically replaces the marker, synchronizes the directory, 169 and reopens the exact new bytes. Stale scratch, tamper, collision, binding 170 replacement, insecure directory, or malformed marker remains evidence and 171 fails closed; reads do not repair or remove it. 172 173 This marker checkpoint does not stage, copy, open, rename, replace, or delete a 174 database. Later restore staging must consume a retained verified backup; 175 replacement must invoke the marker sequence around its governed renames; and 176 open-time recovery must reconcile durable marker and artifact identities before 177 removing any recovery evidence. No marker type, path, raw descriptor, or store 178 operation is public API. 179 180 `stage_verified_restore` is the offline boundary between retained backup proof 181 and live-state replacement. It first validates the expected identity plus exact 182 migration and schema catalogs, then acquires exclusive writer authority. A live 183 writable or read-only host, a live WAL/shared-memory/journal sidecar, an existing 184 stage, or any marker/retained-backup evidence fails closed. The only created 185 artifact is the fixed adjacent `state.restore-staged.sqlite`, opened create-new, 186 no-follow, owner-only, and single-link with mode `0600`. 187 188 Staging copies the exact manifest-bound bytes from the verifier's retained 189 member descriptor with a fixed-size buffer and digest, synchronizes the staged 190 file, and opens SQLite only through the retained staged descriptor. It then 191 rechecks immutable application metadata, the exact applied migration prefix and 192 schema-object catalog at the backup's actual supported version, main-only 193 read-only/query-only connection policy, bounded `integrity_check(1)`, and empty 194 `foreign_key_check`. A final retained-descriptor hash and file plus state- 195 directory synchronization precede success. Live database bytes, identity, 196 permissions, and timestamps remain untouched. 197 198 Success returns a sealed non-cloneable `StagedServiceRestore` that retains 199 writer authority and the exact staging identities. Dropping it attempts an 200 identity-checked unlink and state-directory synchronization before releasing 201 authority; cleanup failure leaves staging or recovery evidence that later 202 admission rejects. Cancelling the async operation requests bounded copy 203 cancellation; any detached work retains authority and exact cleanup ownership 204 until it ends. 205 The operation has no hidden timeout. It does not create or advance a recovery 206 marker, rename or retain live state, install a replacement, or authorize reopen; 207 those operations remain the finalization and recovery checkpoints. 208 209 `finalize_staged_restore` consumes that sealed stage in an owned blocking 210 worker. The stage has already bound the exact live inode, length, and digest 211 that will be retained. Finalization revalidates both retained descriptors, 212 creates and synchronizes the `prepared` marker, and only then disarms automatic 213 stage cleanup. It renames live to `state.restore-backup.sqlite` and staged to 214 live with descriptor-relative no-replace operations. Each rename is followed 215 by exact inode and hash verification, state-directory synchronization, and the 216 corresponding marker advance to `live_retained` or 217 `replacement_installed`. 218 219 Cancellation observed before the worker's atomic commit-ownership handoff 220 leaves live state untouched and attempts exact stage cleanup. Caller-task loss 221 after that handoff has an unknown immediate outcome, including the short 222 interval before `prepared` becomes durable; the worker retains writer 223 authority until it either fails before durability or establishes recovery 224 evidence and continues. Once `prepared` is durable, staged-artifact cleanup is 225 disarmed and the bound stage remains available after every later error. 226 Success returns no database host, retains the old live database and final 227 marker, and requires a new open. Read-write-existing open is the sole recovery 228 path. Under exclusive writer authority and before opening SQLite, it validates 229 the marker's exact service, instance, source generation, application ID, schema 230 ceiling, artifact identities, lengths, digests, restrictive modes, and the 231 absence of database sidecars. Read-only inspection, initialization, and an 232 initialized open never recover; they reject any stage, backup, marker, or marker 233 scratch as `Recovery` without mutation. 234 235 Recovery uses exact topology as the durable authority. `prepared` with the old 236 live database still installed rolls back by removing only the exact stage and 237 then the marker. Once the exact old live inode has reached the backup name, 238 recovery advances and rolls forward. A lagging `live_retained` phase installs or 239 recognizes the exact replacement, advances to `replacement_installed`, then 240 removes the exact old backup before retiring the marker. Interrupted rollback 241 and final cleanup accept only the corresponding already-absent exact artifact, 242 so repeated recovery is idempotent. Every other topology, sidecar, replacement, 243 link, mode, owner, length, digest, identity, or directory-authority mismatch 244 fails closed and preserves the evidence. 245 246 A marker scratch is admitted only when it is the canonical one-edge successor 247 of the current marker and the artifact topology already proves that successor. 248 Recovery removes only the exact bound scratch inode, synchronizes that removal, 249 then reproduces the transition through the governed marker-advance path. This 250 preserves the valid current marker if the scratch pathname was replaced. 251 Orphaned, malformed, skipped, same-phase, terminal, mismatched, or 252 topology-inconsistent scratch is never deleted or reinterpreted. Recovery has no 253 await point or hidden task: once a writable open is polled, each synchronous 254 filesystem step and its authority checks complete before the open can be 255 cancelled. A later cancelled SQLite open is retried by rereading the already 256 durable, marker-free state. Finalization itself does not reconcile or reopen the 257 database. 258 259 `ServiceSqliteHost::inspect_integrity` is the explicit active operator check. 260 It is available on initialized, writable-existing, and read-only inspection 261 hosts, admits at most one check per host, and uses one deferred read transaction 262 as the SQLite snapshot. The caller injects a positive wall-clock 263 `IntegrityCheckedAtUnixMs`; the library does not read an ambient clock or 264 create a timer. The completed report contains only `verified` or `failed` for 265 SQLite integrity and foreign keys, plus at most the fixed 266 `sqlite_integrity_failed` and `foreign_key_violation` diagnostic codes in that 267 canonical order. It can be projected to the passive `StorageIntegrity` 268 vocabulary, but the library does not persist or cache the report. 269 270 The check never publishes raw SQLite diagnostics, table or row identity, 271 filesystem paths, SQL, or dependency errors. Inability to execute, decode, or 272 finish either bounded check is an `Integrity` error rather than a fabricated 273 completed result. Authority is revalidated after every await and has precedence 274 over integrity classification. The operation has no hidden timeout or task. 275 Callers own a positive monotonic deadline by dropping the future; cancellation 276 returns no report, writes nothing, quarantines the checked-out connection, and 277 leaves it in a host-owned close driver. Retry or host close explicitly awaits 278 that retained close future until the prior SQLite worker terminates before any 279 new check or authority release. A retry uses a newly injected wall-clock time. 280 The strict backup and restore integrity verifier remains a separate fail-closed 281 boundary. 282 283 State-filesystem capacity inspection is an explicit synchronous input for 284 doctor checks and authoritative admission. `MinimumFreeBytes` must be supplied 285 and is constrained to `1..=i64::MAX`; it has no default. The value 286 `268435456` is the exact governed configuration and test vector, not an 287 implicit universal threshold. On Linux and macOS the platform adapter opens the 288 owner-owned state directory that is not group/other writable without following 289 links, retains and revalidates its identity, and uses `fstatvfs` to measure 290 bytes available to the unprivileged service user. The current native 291 qualification matrix is macOS aarch64 and Linux x86_64; other platforms fail 292 closed and successful compilation outside that matrix is not support evidence. 293 294 A successful immutable snapshot is `ready` when available bytes are greater 295 than or equal to the configured minimum and `low_disk` when they are below it. 296 Low disk rejects or pauses new authoritative admission; measurement failure is 297 a typed unavailable error and is never fabricated as low-disk evidence. A 298 consumer may cache the successful snapshot and later project low disk to the 299 stable `database_low_disk` readiness reason. `/readyz` remains passive and must 300 read only that caller-owned cached state; it never invokes the capacity 301 adapter. The measurement is advisory rather than a space reservation and does 302 not guarantee a later write. 303 304 Capacity inspection is host-independent and performs no database open, pool 305 operation, SQLite query, filesystem mutation, ambient time read, timer, task, 306 or hidden sampling. Service configuration, threshold defaults, cache refresh, 307 status persistence, admission wiring, and route projection remain consumer 308 responsibilities. 309 310 Durability fault injection is a private test-only mechanism. A closed 311 instance-scoped controller can arm exactly one named before/after boundary and 312 returns one injected error the first time that boundary is reached; later hits 313 are no-ops. The complete inventory covers database initialization, runner-owned 314 transaction begin and commit, online-backup creation/copy/synchronization, 315 restore-marker creation and advancement, both restore rename/synchronization 316 steps, and explicit host drain/checkpoint/connection-close/authority-release. 317 An ordinary controller has zero behavior, and no failpoint type or selector is 318 exported from the crate root. There is no process-global failpoint state, 319 environment or configuration selector, Cargo feature, hidden task, timer, 320 panic, or process-exit behavior. These deterministic in-process edges qualify 321 error ordering, rollback, cleanup, recovery evidence, and one-shot retry 322 semantics. 323 324 Process-crash qualification remains test-only and reuses Cargo's private 325 library and integration-test binaries; the crate ships no helper executable or 326 signal handler. A parent sends one bounded temporary root over stdin, waits for 327 a fixed stdout readiness token from an occurrence-aware failpoint barrier, and 328 then issues `SIGKILL`. The suite proves cross-process writer contention and 329 lock release plus five restore boundaries: orphan-stage refusal before a 330 durable marker, prepared rollback, interrupted marker-scratch promotion, 331 installed-replacement recovery, and terminal-marker cleanup. A permissive 332 child umask cannot broaden the fixed `0700` state directory or `0600` lock, 333 database, stage, backup, marker, and marker-scratch artifacts. Linux execution 334 on x86_64 is required for OS-level qualification; macOS aarch64 execution on 335 the current machine is developer evidence. No other platform or architecture 336 is an active qualification gate. These tests exercise process death at named 337 durable edges and do not claim abrupt power-loss or storage-device durability 338 behavior. 339 340 The crate owns mechanics only. Service-specific tables, SQL, repositories, 341 backup content policy, identity material, process lifecycle, and readiness 342 policy remain with the consuming service. The crate does not provide callers 343 with raw database authority. 344 345 Publication is disabled. The package is not part of the public Radroots crate 346 release closure. 347 348 ## Public API and package boundary 349 350 This unpublished `private_runtime`, `package_private` crate owns only 351 service-neutral SQLite mechanics. Its built-in persistent objects are the 352 shared `radroots_service_metadata` and `schema_migrations` tables plus their 353 governed immutability triggers. Every service-owned table, index, trigger, 354 migration statement, and schema policy is supplied through the caller-owned 355 migration and schema catalogs; no product identifier or product table belongs 356 in this package. 357 358 The crate-root exports are frozen in the reviewed 359 [service-SQLite API baseline](../../contracts/api_baselines/radroots_service_sqlite.txt). 360 Raw pools, pooled or direct connections, transaction-control handles, and 361 dependency re-exports are forbidden. The deliberate narrow exception is the 362 `sqlx::Executor` implementation for borrowed 363 `&mut ServiceSqliteInitializer<'_>` and `&mut ServiceSqliteTransaction<'_>` 364 values. They permit compile-time typed queries. The crate retains connection 365 ownership and sole begin, commit, rollback, policy, and cancellation authority.