lib

Core libraries for Radroots
git clone https://radroots.dev/git/lib.git
Log | Files | Refs | README

README.md (23476B)


      1 # radroots_service_sqlite
      2 
      3 `radroots_service_sqlite` is the unpublished, native SQLite-mechanism crate for
      4 Radroots services. It provides narrow, reusable building blocks for exclusive
      5 writer authority, instance locking, versioned schema mechanics, bounded
      6 transactions, immutable service-instance database identity, integrity checks,
      7 backup, restore, and passive storage status.
      8 
      9 `ServiceSqliteHost` is the only public connection host. Its SQLx pool and raw
     10 connections are sealed inside the crate. Services run typed SQLx queries through
     11 the borrowed `ServiceSqliteTransaction` executor supplied by
     12 `ServiceSqliteHost::transaction`; transaction begin, commit, rollback, policy
     13 validation, attached-database exclusion, and cancellation quarantine remain
     14 runner-owned. Writable host opening finishes every pending governed migration
     15 before returning, and read-only inspection opens only current migration and
     16 schema state.
     17 
     18 Create-new state uses the borrowed `ServiceSqliteInitializer` executor rather
     19 than a database path or raw connection. The runner reserves and retains the
     20 exact canonical file, opens SQLite only through that retained descriptor, owns
     21 `BEGIN IMMEDIATE`, and uses a private memory journal so descriptor-bound
     22 initialization creates no path-derived SQLite sidecar. It commits the product
     23 schema, shared `radroots_service_metadata`, empty v1 `schema_migrations`
     24 ledger, and exact schema-catalog verification in one transaction. The
     25 initializer screens the same closed transaction-control and attachment
     26 inventory as host transactions; an ignored rejection still prevents commit.
     27 Callback failure or cancellation cannot return a reusable connection or
     28 publish a partial database.
     29 
     30 Interactive callers may use `ServiceSqliteHost::open_or_initialize`. One held
     31 writer authority and an exclusive create decide whether the sealed initializer
     32 runs or the exact existing database is opened. The existing branch never runs
     33 the initializer. Callers therefore do not use pathname probes, error-text
     34 matching, recursive directory creation, permission repair, or direct SQLx
     35 connections to choose the bootstrap branch. Success returns an
     36 `OpenedServiceDatabase` that binds the retained host to the actual verified
     37 metadata from the selected branch, including the persisted source generation
     38 when state already existed.
     39 
     40 Existing databases can be admitted without a caller guessing their stored
     41 source generation. `ExistingServiceDatabaseIntent` seals the canonical
     42 service and instance, supported schema ceiling, and SQLite application ID;
     43 `ServiceSqliteHost::open_read_write_existing_with_intent` and
     44 `ServiceSqliteHost::open_read_only_inspection_with_intent` discover and verify
     45 the actual immutable metadata while retaining the corresponding writer or
     46 inspection authority. Success returns an `OpenedServiceDatabase`,
     47 which keeps the host and verified metadata inseparable until the caller
     48 consumes them together. Recovery remains fail closed and may use this intent
     49 only to discover the marker-bound generation; every other identity dimension,
     50 artifact binding, migration prefix, and schema catalog remains governed.
     51 
     52 Service-controlled SQL is screened before SQLite compilation through both the
     53 borrowed transaction executor and migration callback executor. The closed
     54 statement-control inventory is `PRAGMA`, `ATTACH`, `DETACH`, `BEGIN`, `COMMIT`,
     55 `END`, `ROLLBACK`, `SAVEPOINT`, and `RELEASE`, regardless of case, whitespace,
     56 comments, multiple statements, or prepared-query entry point. Rejection is
     57 sticky for the transaction, so ignoring the immediate SQL error cannot permit
     58 commit. Runner-owned setup remains private, and the complete connection policy
     59 is revalidated before commit and before a connection can return to the pool.
     60 
     61 Persisted SQLite text and blob values are admitted through bounded projections
     62 before Rust decoding. Service and instance identifiers, source generations,
     63 migration names, checksums, build identity, schema text, and database inventory
     64 values carry an exact SQLite type, reported byte length, capped byte prefix,
     65 and bounded row count. Integrity diagnostics use a borrowed-byte cap over the
     66 exact `PRAGMA integrity_check(1)` operation and never convert an unbounded
     67 diagnostic to UTF-8.
     68 
     69 Initialization reports success only after its governed commit and cleanup
     70 succeed. Hook removal and connection close are attempted even when commit or
     71 rollback fails; successful cleanup cannot hide an earlier failure. A failed
     72 commit's SQLite cause survives successful cleanup and cannot grant initialized
     73 state or writer authority.
     74 
     75 Cancelling a host transaction before the runner enables outer commit
     76 quarantines its connection and leaves no authoritative transaction effect. A
     77 service-operation error is returned only after rollback is confirmed; an
     78 unconfirmed rollback is reported as `RollbackFailed`. Cancelling once outer
     79 commit begins yields no result and must be treated as an unknown commit outcome.
     80 Both that case and `CommitOutcomeUnknown` require rereading authoritative state
     81 before an idempotent retry.
     82 
     83 Every host must be closed explicitly with `ServiceSqliteHost::close`. Close
     84 permanently stops new transaction admission, drains transactions that were
     85 already admitted, and is safe to call sequentially or concurrently. Writable
     86 close applies the fixed `PRAGMA wal_checkpoint(TRUNCATE)` policy, requires an
     87 unblocked checkpoint, closes the private checkpoint connection, and explicitly
     88 releases writer authority. Read-only inspection close drains its pool and
     89 releases its shared inspection guard without checkpointing or mutating the
     90 database or filesystem. Cancelling close before terminal completion leaves the
     91 host non-admitting and retains authority; the private connect, checkpoint, and
     92 explicit connection-close driver remains host-owned so a later call resumes
     93 close without losing the SQLite handle or its close proof. Once authority
     94 release is proven, the stable outer result is cached for every later call.
     95 Dropping a host performs no asynchronous close work and is not proof that the
     96 governed checkpoint and authority-release sequence completed.
     97 
     98 `ServiceBackupManifest` is the stable model-only v1 backup identity. It admits
     99 only compact canonical UTF-8 JSON in the frozen field order, capped at 1,024
    100 bytes, and computes the external manifest SHA-256 over those exact bytes. The
    101 v1 member array contains exactly one `state.sqlite` member with a nonzero byte
    102 length and lowercase SHA-256; service, instance, nonzero source generation,
    103 state schema, and injected creation time are explicit. SQLite and foreign-key
    104 integrity are exactly `ok`, and protected material is always excluded.
    105 Parsing proves only the strict structural and canonical contract. It rejects
    106 unknown, duplicate, null, reordered, whitespace-altered, or version-drifted
    107 input; member bytes, digest, SQLite identity, and actual integrity remain the
    108 separate backup-verification boundary. Constructing or parsing the manifest
    109 model performs no filesystem or SQLite work.
    110 
    111 Writable hosts provide `ServiceSqliteHost::capture_online_backup` for one
    112 incremental, point-in-time SQLite capture at a time. The caller supplies an
    113 injected creation time and the exact new absolute staging-directory path under
    114 an existing owner-controlled parent; capture creates that directory with mode
    115 `0700` and its sole `state.sqlite` member with mode `0600`. It uses SQLite's
    116 online-backup API without checkpointing or copying the live source file, then
    117 requires exact service metadata, bounded `integrity_check`, an empty
    118 `foreign_key_check`, a singleton member inventory, SHA-256, and file, staging,
    119 and parent synchronization before returning the canonical manifest in memory.
    120 No manifest file, bundle identifier, credential, or protected material is
    121 written to the staging directory.
    122 
    123 Capture rejects read-only, closing, unsupported, colliding, or concurrent
    124 admission before publishing a result. Dropping the capture future requests
    125 cancellation; the blocking worker retains its checked-out pool admission,
    126 writer authority, and exact staging identities until SQLite handles are closed
    127 and cleanup completes. Host close therefore drains capture and cancellation
    128 cleanup before it checkpoints or releases authority. Capture has no hidden
    129 timeout: callers own any deadline by cancelling the future. A completed capture
    130 is still untrusted backup input until the separate verifier binds its manifest,
    131 member bytes, expected intent, application metadata, and integrity; capture
    132 does not provide restore or replacement behavior.
    133 
    134 `verify_backup_bundle` is the synchronous, task-free boundary for an untrusted
    135 manifest and bundle. The caller supplies the independently protected manifest
    136 SHA-256, expected service database identity, and a positive maximum state-file
    137 size. Verification requires canonical manifest bytes, exact service, instance,
    138 source-generation, schema, and application intent, a restrictive owner-only
    139 directory containing only `state.sqlite`, the exact bounded member length and
    140 digest, immutable read-only/query-only SQLite access, main-only attachment,
    141 bounded application metadata, `integrity_check(1)`, and an empty foreign-key
    142 check. It performs no filesystem mutation and does not create a task or hidden
    143 deadline.
    144 
    145 Success returns a non-forgeable `VerifiedServiceBackup` that retains the exact
    146 verified directory and member descriptors while exposing only the canonical
    147 manifest and actual database metadata. It is not restore or replacement
    148 authority, and it exposes no path or raw handle. Later restore work must copy
    149 from the retained member and reverify the staged copy under its own supervised
    150 blocking worker and deadline; pathname verification alone is insufficient.
    151 
    152 Restore crash recovery uses a private sealed v1 marker stored beside canonical
    153 service state. Its fixed layout names the live `state.sqlite`, staged
    154 `state.restore-staged.sqlite`, retained `state.restore-backup.sqlite`, durable
    155 `state.restore-marker.v1`, and create-new update scratch
    156 `state.restore-marker.v1.next`. The compact canonical JSON is capped at 2,048
    157 bytes and binds typed database intent, the protected source-manifest digest,
    158 and exact live, staged, and retained-backup device, inode, length, and SHA-256
    159 expectations. A domain-separated checksum binds the canonical fields and
    160 detects corruption; it is not an authenticity credential.
    161 
    162 The only legal durable sequence is `prepared` to `live_retained` to
    163 `replacement_installed`; repeating the current phase is byte-idempotent and
    164 every skip, reversal, or post-install transition fails closed. Marker files are
    165 descriptor-relative, no-follow, single-link, owner-owned regular files with
    166 mode `0600`. Creation synchronizes the file and state directory. Advancement
    167 compare-and-reloads the current bytes, writes and synchronizes the fixed
    168 create-new scratch, atomically replaces the marker, synchronizes the directory,
    169 and reopens the exact new bytes. Stale scratch, tamper, collision, binding
    170 replacement, insecure directory, or malformed marker remains evidence and
    171 fails closed; reads do not repair or remove it.
    172 
    173 This marker checkpoint does not stage, copy, open, rename, replace, or delete a
    174 database. Later restore staging must consume a retained verified backup;
    175 replacement must invoke the marker sequence around its governed renames; and
    176 open-time recovery must reconcile durable marker and artifact identities before
    177 removing any recovery evidence. No marker type, path, raw descriptor, or store
    178 operation is public API.
    179 
    180 `stage_verified_restore` is the offline boundary between retained backup proof
    181 and live-state replacement. It first validates the expected identity plus exact
    182 migration and schema catalogs, then acquires exclusive writer authority. A live
    183 writable or read-only host, a live WAL/shared-memory/journal sidecar, an existing
    184 stage, or any marker/retained-backup evidence fails closed. The only created
    185 artifact is the fixed adjacent `state.restore-staged.sqlite`, opened create-new,
    186 no-follow, owner-only, and single-link with mode `0600`.
    187 
    188 Staging copies the exact manifest-bound bytes from the verifier's retained
    189 member descriptor with a fixed-size buffer and digest, synchronizes the staged
    190 file, and opens SQLite only through the retained staged descriptor. It then
    191 rechecks immutable application metadata, the exact applied migration prefix and
    192 schema-object catalog at the backup's actual supported version, main-only
    193 read-only/query-only connection policy, bounded `integrity_check(1)`, and empty
    194 `foreign_key_check`. A final retained-descriptor hash and file plus state-
    195 directory synchronization precede success. Live database bytes, identity,
    196 permissions, and timestamps remain untouched.
    197 
    198 Success returns a sealed non-cloneable `StagedServiceRestore` that retains
    199 writer authority and the exact staging identities. Dropping it attempts an
    200 identity-checked unlink and state-directory synchronization before releasing
    201 authority; cleanup failure leaves staging or recovery evidence that later
    202 admission rejects. Cancelling the async operation requests bounded copy
    203 cancellation; any detached work retains authority and exact cleanup ownership
    204 until it ends.
    205 The operation has no hidden timeout. It does not create or advance a recovery
    206 marker, rename or retain live state, install a replacement, or authorize reopen;
    207 those operations remain the finalization and recovery checkpoints.
    208 
    209 `finalize_staged_restore` consumes that sealed stage in an owned blocking
    210 worker. The stage has already bound the exact live inode, length, and digest
    211 that will be retained. Finalization revalidates both retained descriptors,
    212 creates and synchronizes the `prepared` marker, and only then disarms automatic
    213 stage cleanup. It renames live to `state.restore-backup.sqlite` and staged to
    214 live with descriptor-relative no-replace operations. Each rename is followed
    215 by exact inode and hash verification, state-directory synchronization, and the
    216 corresponding marker advance to `live_retained` or
    217 `replacement_installed`.
    218 
    219 Cancellation observed before the worker's atomic commit-ownership handoff
    220 leaves live state untouched and attempts exact stage cleanup. Caller-task loss
    221 after that handoff has an unknown immediate outcome, including the short
    222 interval before `prepared` becomes durable; the worker retains writer
    223 authority until it either fails before durability or establishes recovery
    224 evidence and continues. Once `prepared` is durable, staged-artifact cleanup is
    225 disarmed and the bound stage remains available after every later error.
    226 Success returns no database host, retains the old live database and final
    227 marker, and requires a new open. Read-write-existing open is the sole recovery
    228 path. Under exclusive writer authority and before opening SQLite, it validates
    229 the marker's exact service, instance, source generation, application ID, schema
    230 ceiling, artifact identities, lengths, digests, restrictive modes, and the
    231 absence of database sidecars. Read-only inspection, initialization, and an
    232 initialized open never recover; they reject any stage, backup, marker, or marker
    233 scratch as `Recovery` without mutation.
    234 
    235 Recovery uses exact topology as the durable authority. `prepared` with the old
    236 live database still installed rolls back by removing only the exact stage and
    237 then the marker. Once the exact old live inode has reached the backup name,
    238 recovery advances and rolls forward. A lagging `live_retained` phase installs or
    239 recognizes the exact replacement, advances to `replacement_installed`, then
    240 removes the exact old backup before retiring the marker. Interrupted rollback
    241 and final cleanup accept only the corresponding already-absent exact artifact,
    242 so repeated recovery is idempotent. Every other topology, sidecar, replacement,
    243 link, mode, owner, length, digest, identity, or directory-authority mismatch
    244 fails closed and preserves the evidence.
    245 
    246 A marker scratch is admitted only when it is the canonical one-edge successor
    247 of the current marker and the artifact topology already proves that successor.
    248 Recovery removes only the exact bound scratch inode, synchronizes that removal,
    249 then reproduces the transition through the governed marker-advance path. This
    250 preserves the valid current marker if the scratch pathname was replaced.
    251 Orphaned, malformed, skipped, same-phase, terminal, mismatched, or
    252 topology-inconsistent scratch is never deleted or reinterpreted. Recovery has no
    253 await point or hidden task: once a writable open is polled, each synchronous
    254 filesystem step and its authority checks complete before the open can be
    255 cancelled. A later cancelled SQLite open is retried by rereading the already
    256 durable, marker-free state. Finalization itself does not reconcile or reopen the
    257 database.
    258 
    259 `ServiceSqliteHost::inspect_integrity` is the explicit active operator check.
    260 It is available on initialized, writable-existing, and read-only inspection
    261 hosts, admits at most one check per host, and uses one deferred read transaction
    262 as the SQLite snapshot. The caller injects a positive wall-clock
    263 `IntegrityCheckedAtUnixMs`; the library does not read an ambient clock or
    264 create a timer. The completed report contains only `verified` or `failed` for
    265 SQLite integrity and foreign keys, plus at most the fixed
    266 `sqlite_integrity_failed` and `foreign_key_violation` diagnostic codes in that
    267 canonical order. It can be projected to the passive `StorageIntegrity`
    268 vocabulary, but the library does not persist or cache the report.
    269 
    270 The check never publishes raw SQLite diagnostics, table or row identity,
    271 filesystem paths, SQL, or dependency errors. Inability to execute, decode, or
    272 finish either bounded check is an `Integrity` error rather than a fabricated
    273 completed result. Authority is revalidated after every await and has precedence
    274 over integrity classification. The operation has no hidden timeout or task.
    275 Callers own a positive monotonic deadline by dropping the future; cancellation
    276 returns no report, writes nothing, quarantines the checked-out connection, and
    277 leaves it in a host-owned close driver. Retry or host close explicitly awaits
    278 that retained close future until the prior SQLite worker terminates before any
    279 new check or authority release. A retry uses a newly injected wall-clock time.
    280 The strict backup and restore integrity verifier remains a separate fail-closed
    281 boundary.
    282 
    283 State-filesystem capacity inspection is an explicit synchronous input for
    284 doctor checks and authoritative admission. `MinimumFreeBytes` must be supplied
    285 and is constrained to `1..=i64::MAX`; it has no default. The value
    286 `268435456` is the exact governed configuration and test vector, not an
    287 implicit universal threshold. On Linux and macOS the platform adapter opens the
    288 owner-owned state directory that is not group/other writable without following
    289 links, retains and revalidates its identity, and uses `fstatvfs` to measure
    290 bytes available to the unprivileged service user. The current native
    291 qualification matrix is macOS aarch64 and Linux x86_64; other platforms fail
    292 closed and successful compilation outside that matrix is not support evidence.
    293 
    294 A successful immutable snapshot is `ready` when available bytes are greater
    295 than or equal to the configured minimum and `low_disk` when they are below it.
    296 Low disk rejects or pauses new authoritative admission; measurement failure is
    297 a typed unavailable error and is never fabricated as low-disk evidence. A
    298 consumer may cache the successful snapshot and later project low disk to the
    299 stable `database_low_disk` readiness reason. `/readyz` remains passive and must
    300 read only that caller-owned cached state; it never invokes the capacity
    301 adapter. The measurement is advisory rather than a space reservation and does
    302 not guarantee a later write.
    303 
    304 Capacity inspection is host-independent and performs no database open, pool
    305 operation, SQLite query, filesystem mutation, ambient time read, timer, task,
    306 or hidden sampling. Service configuration, threshold defaults, cache refresh,
    307 status persistence, admission wiring, and route projection remain consumer
    308 responsibilities.
    309 
    310 Durability fault injection is a private test-only mechanism. A closed
    311 instance-scoped controller can arm exactly one named before/after boundary and
    312 returns one injected error the first time that boundary is reached; later hits
    313 are no-ops. The complete inventory covers database initialization, runner-owned
    314 transaction begin and commit, online-backup creation/copy/synchronization,
    315 restore-marker creation and advancement, both restore rename/synchronization
    316 steps, and explicit host drain/checkpoint/connection-close/authority-release.
    317 An ordinary controller has zero behavior, and no failpoint type or selector is
    318 exported from the crate root. There is no process-global failpoint state,
    319 environment or configuration selector, Cargo feature, hidden task, timer,
    320 panic, or process-exit behavior. These deterministic in-process edges qualify
    321 error ordering, rollback, cleanup, recovery evidence, and one-shot retry
    322 semantics.
    323 
    324 Process-crash qualification remains test-only and reuses Cargo's private
    325 library and integration-test binaries; the crate ships no helper executable or
    326 signal handler. A parent sends one bounded temporary root over stdin, waits for
    327 a fixed stdout readiness token from an occurrence-aware failpoint barrier, and
    328 then issues `SIGKILL`. The suite proves cross-process writer contention and
    329 lock release plus five restore boundaries: orphan-stage refusal before a
    330 durable marker, prepared rollback, interrupted marker-scratch promotion,
    331 installed-replacement recovery, and terminal-marker cleanup. A permissive
    332 child umask cannot broaden the fixed `0700` state directory or `0600` lock,
    333 database, stage, backup, marker, and marker-scratch artifacts. Linux execution
    334 on x86_64 is required for OS-level qualification; macOS aarch64 execution on
    335 the current machine is developer evidence. No other platform or architecture
    336 is an active qualification gate. These tests exercise process death at named
    337 durable edges and do not claim abrupt power-loss or storage-device durability
    338 behavior.
    339 
    340 The crate owns mechanics only. Service-specific tables, SQL, repositories,
    341 backup content policy, identity material, process lifecycle, and readiness
    342 policy remain with the consuming service. The crate does not provide callers
    343 with raw database authority.
    344 
    345 Publication is disabled. The package is not part of the public Radroots crate
    346 release closure.
    347 
    348 ## Public API and package boundary
    349 
    350 This unpublished `private_runtime`, `package_private` crate owns only
    351 service-neutral SQLite mechanics. Its built-in persistent objects are the
    352 shared `radroots_service_metadata` and `schema_migrations` tables plus their
    353 governed immutability triggers. Every service-owned table, index, trigger,
    354 migration statement, and schema policy is supplied through the caller-owned
    355 migration and schema catalogs; no product identifier or product table belongs
    356 in this package.
    357 
    358 The crate-root exports are frozen in the reviewed
    359 [service-SQLite API baseline](../../contracts/api_baselines/radroots_service_sqlite.txt).
    360 Raw pools, pooled or direct connections, transaction-control handles, and
    361 dependency re-exports are forbidden. The deliberate narrow exception is the
    362 `sqlx::Executor` implementation for borrowed
    363 `&mut ServiceSqliteInitializer<'_>` and `&mut ServiceSqliteTransaction<'_>`
    364 values. They permit compile-time typed queries. The crate retains connection
    365 ownership and sole begin, commit, rollback, policy, and cancellation authority.