# ADR 000: Retain and control the private mail archive - HTML version: https://robbiepalmer.me/projects/inbox-signal/adrs/000-private-mail-data-lifecycle - Project: Inbox Signal (https://robbiepalmer.me/projects/inbox-signal.md) - Status: Accepted - Date: 2026-10-25 - Initiatives: Digital Twins for Everyday Life (https://robbiepalmer.me/initiatives/digital-twins-for-everyday-life.md) ## Context Inbox Signal needs raw MIME, attachments, provider identifiers, and derived records to inventory old mail and explain proposed actions. Those copies are more sensitive than the mailbox metadata normally exposed by a provider UI. They can contain identity documents, financial records, account links, health information, and private correspondence. The project plans to put large objects in Cloudflare R2, operational records in Neon PostgreSQL, and bounded evaluation samples in a private DVC remote. Each store has a different deletion mechanism. Provider-side deletion also has its own recovery and hold behavior. Treating a mailbox delete as complete while a raw object, model sample, restored database branch, or derived assertion remains would break an explicit archive-deletion promise. Mailbox cleanup and archive deletion are separate actions: removing a provider copy must not silently erase the durable archive, and removing the archive must account for every project-controlled copy. [R2 encrypts all objects at rest with provider-managed keys](https://developers.cloudflare.com/r2/reference/data-security/). That protects storage media but does not separate mail content from a compromised Cloudflare account or an over-broad application credential. R2 bucket locks can prevent premature deletion, while [lifecycle expiry is asynchronous](https://developers.cloudflare.com/r2/buckets/object-lifecycles/) and may occur after the recorded expiry time. Explicit deletion is therefore required for user-directed purges. R2's [direct APIs are strongly consistent](https://developers.cloudflare.com/r2/reference/consistency/) for object deletion and listing, so the application can verify that an object is absent after a delete. [Neon history can restore a branch](https://neon.com/blog/announcing-point-in-time-restore) inside its configured history window. DVC keeps content-addressed objects until [remote garbage collection](https://dvc.org/doc/command-reference/gc) removes objects that are no longer referenced. Neither mechanism understands that an Inbox Signal archive record has been deleted. The application needs one inventory and distinct state transitions for provider cleanup and archive deletion. ## Decision Keep a machine-readable inventory of every store that may contain private mail data. Each inventory entry must name the data class, owner, location, encryption boundary, retention rule, restore path, deletion operation, and deletion verification. A new store cannot receive private mail until it appears in the inventory and passes the lifecycle rehearsal defined below. ### Encrypt content before object storage Encrypt every raw MIME object, attachment, retained document, export, and DVC sample before it reaches R2. R2's automatic encryption and TLS remain additional controls, not the content-encryption boundary. Use envelope encryption in the Workers runtime: 1. Generate a random 256-bit data-encryption key and 96-bit nonce for each object. 2. Encrypt the object with AES-256-GCM. Bind the account-scoped object ID, object version, content class, and envelope version as authenticated additional data. 3. Wrap the data-encryption key with a versioned AES-256 key-encryption key. Store only the ciphertext, nonce, wrapped key, algorithms, and key version in R2 and Neon. 4. Keep key-encryption keys in a dedicated Doppler production config. Deliver them only to the mail content service. Provider adapters, analytics jobs, browser automation, and general CI do not receive decryption keys. Cloudflare Workers supports AES-GCM, AES-KW, key generation, wrapping, and unwrapping through its [Web Crypto implementation](https://developers.cloudflare.com/workers/runtime-apis/web-crypto/). The implementation must use those runtime primitives rather than a handwritten cipher or a reusable object nonce. Each key version has an activation time and status. Rotate the key-encryption key after suspected exposure and at least once every 12 months. Rotation rewraps data-encryption keys without decrypting and rewriting the content. Retire an old key only after the inventory proves that every live envelope has been rewrapped or deleted and the longest backup window has elapsed. [Doppler retains secret version history](https://docs.doppler.com/docs/versioning), so key deletion is not treated as proof that mail was erased. Deletion evidence comes from removing every ciphertext copy. ### Retain the encrypted archive until explicit deletion Retention follows the content class wherever the project stores it. Raw mail and ordinary attachments form the durable archive and have no automatic expiry. The other periods are maxima unless an approved exception names a shorter or longer period. | Data class | Retention rule | End-of-life action | | ----------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------- | | Raw MIME and ordinary attachments | Indefinitely while the archive record remains active; provider cleanup does not end archive retention | Start the archive deletion workflow only after a separate explicit approval | | Documents or messages explicitly selected for long-term retention | Keep while the recorded retention reason remains active; review the reason and next review date at least every 365 days | Start a 30-day recovery window when the owner removes the reason, then purge | | Provider sync tokens and active message metadata | While the provider connection and source message remain active | Revoke the token on disconnect and remove message-level metadata during purge | | Derived classifications, entities, relationships, and assertions | While at least one live item of supporting evidence remains | Delete within 24 hours after the last supporting item is purged, unless the owner separately retains the derived record | | Approved action manifests and reversible action details | 30 days after execution | Remove message IDs, destinations, and content; retain only the content-free receipt described below | | Operational logs and traces | 14 days | Expire automatically; logs must never contain bodies, headers, attachment names, addresses, or stable message IDs | | Aggregate telemetry | 30 days | Expire automatically; dimensions must not identify an account, sender, message, document, or action manifest | | DVC evaluation sample content | 180 days from sample creation | Remove its references, then garbage-collect both private remote objects and every managed cache | | Generated exports and temporary processing objects | 24 hours | Delete explicitly and use lifecycle expiry as a backstop | | Neon point-in-time history | 6 hours for the first release, never more than 7 days without a later decision | Let provider history expire; delete temporary restore branches within 24 hours | | Content-free deletion receipts | 400 days | Expire automatically | A content-free deletion receipt contains a random operation ID, data-class counts, requested and completed timestamps, provider outcome classes, and failed store names. It does not contain account addresses, provider IDs, message IDs, content hashes, sender data, subjects, filenames, or object keys. Use R2 lifecycle rules as maximum-age backstops for evaluation, export, and temporary prefixes. The raw archive prefix must not have an expiry rule. Lifecycle expiry can lag by about a day, so the purge worker must call the direct object API for scheduled and user-directed deletion. Use a [seven-day R2 bucket lock](https://developers.cloudflare.com/r2/buckets/bucket-locks/) on newly ingested raw and retained objects to protect against immediate accidental deletion. The recovery workflow must show the resulting earliest purge time before accepting a request that would be delayed by an active lock. R2 is the independent archive of raw mail and ordinary attachments. It allows future parsers to reprocess historical mail and preserves a recovery source after provider cleanup. The source mailbox remains authoritative for provider state, but deleting from it does not delete the archive. Database history protects recent metadata changes but is not an additional permanent mail archive. ### Separate mailbox cleanup from archive deletion Provider state and archive state advance independently. Provider state is `active`, `pending_mailbox_deletion`, or `mailbox_purged`. Archive state is `retained`, `pending_archive_deletion`, or `archive_purged`. Moving a provider message to trash starts `pending_mailbox_deletion` and records the provider's verified recovery deadline. The default recovery period is 30 days. If a provider guarantees less time, the review UI must show the shorter deadline before approval. During this state, the system may restore the provider message. The encrypted archive and its evidence links remain intact. Provider-side permanent deletion needs a separate approval after the recovery deadline. The worker verifies that the approval, provider version, and recovery deadline still match, requests permanent deletion when the provider exposes it, and invalidates provider action manifests. It does not delete the encrypted archive or derived knowledge supported by that archive. Deleting from the archive is another explicit action. It starts `pending_archive_deletion`, presents the project-controlled copies that will be removed, and provides a 30-day recovery window. After a separate final approval, the archive purge worker: 1. verifies that both archive-deletion approvals, the archive object version, and recovery deadline still match; 2. deletes raw objects, attachments, exports, and retained document copies from R2; 3. deletes content-bearing Neon rows and disconnects derived records that have no other live evidence; 4. removes DVC references and garbage-collects the private remote and managed caches; 5. invalidates cached reads and outstanding action manifests; and 6. writes the content-free receipt only after every project-controlled store verifies absence. For R2, verification requires both a failed direct object lookup and absence from a strongly consistent prefix listing. A lifecycle rule alone is not verification. For Neon, verification queries every inventoried content table by the internal deletion operation before committing the receipt. DVC deletion uses remote garbage collection because removing a `.dvc` reference does not remove the content-addressed object. Git may retain the content hash in old pointer history, but the private object and all managed cache copies must be gone. The service reports a partial archive purge instead of claiming success when a store is unavailable or rejects deletion. It retries idempotently with the same operation ID. A provider cleanup success response records that the provider accepted the operation, not that its physical media or Recoverable Items area was erased. For example, [Microsoft Graph's permanent-delete operation](https://learn.microsoft.com/en-us/graph/api/message-permanentdelete?view=graph-rest-1.0) can move a message to the mailbox's hidden purges area before Exchange removes it according to the mailbox retention policy. The receipt must show that distinction. ### Limit and record exceptions Only the mailbox owner may add a legal, contractual, tax, warranty, dispute, or security exception to an archive-deletion request. The exception names the affected items or data class, reason, approver, start time, and next review date. It cannot be inferred from a classifier result. An exception may postpone application deletion, but it does not widen telemetry or agent access. When a provider hold prevents final deletion, remove all project-controlled copies that are not covered by the exception and report the provider hold as the remaining copy. Review every exception at least annually. ### Gate ingestion on restore and deletion rehearsals Do not import private mail until an isolated environment passes a synthetic lifecycle rehearsal. The fixture contains two threaded messages, duplicate provider delivery, a text and binary attachment, a retained document, derived records, an action manifest, telemetry, and a DVC sample. The rehearsal must: 1. encrypt and upload every object, prove that stored bytes and logs contain no plaintext fixture marker, and decrypt with the active key; 2. rotate the key-encryption key, rewrap the fixture keys, and decrypt with only the new active version; 3. move one fixture message to `pending_mailbox_deletion`, permanently delete its provider copy, and prove that the encrypted archive, attachments, evidence links, and action idempotency survive without duplicate effects; 4. move the other fixture to `pending_archive_deletion`, restore it, repeat the request with shortened test-only periods, then verify absence from R2 lookup and listing, all content-bearing Neon tables, the DVC remote, managed DVC caches, exports, logs, and application reads; 5. restore the Neon database to an isolated branch from a point before the fixture purge, run integrity checks, and delete the branch within 24 hours; 6. prove that the production history, bucket-lock, lifecycle, log, and telemetry settings match this decision; and 7. emit a content-free rehearsal receipt with the tested schema and envelope versions. Run the full rehearsal before first import, after any store, envelope, key, backup, or deletion-path change, and at least every 90 days while the system holds raw mail. A production restore rehearsal uses synthetic canary data. It must never restore private mailbox data into CI or a developer laptop. ## Consequences Inbox Signal has separate, explicit contracts for provider cleanup and archive deletion. Cleaning up a mailbox preserves the encrypted R2 archive, while an archive deletion propagates across R2, Neon, DVC, caches, exports, and derived knowledge. The 30-day reversible actions and seven-day R2 lock make accidental deletion recoverable. Explicit purge verification prevents a lifecycle configuration or removed DVC pointer from being mistaken for deletion. Client-side encryption reduces the value of an R2-only compromise. The mail content service still receives a decryption key and can read authorized objects, so its identity, logs, and capability checks remain part of the trusted boundary. Doppler becomes a required production dependency for key delivery and rotation. Indefinite raw retention allows future parsers to reprocess the full historical archive and makes the archive useful after provider cleanup. It also creates a second durable mailbox, increases storage cost and breach impact, and makes key custody a long-lived responsibility. The owner must explicitly delete the archive when those costs outweigh its value. The 180-day evaluation window still makes some old experiments irreproducible after their private examples expire. A fresh sample can be generated from the retained raw archive. Metrics, model artifacts, redacted error categories, and content-free provenance may remain after archive deletion, but they cannot reconstruct deleted examples. When provider cleanup and archive deletion are both requested, provider holds and backup windows may leave the provider copy in place after the project-controlled purge. The UI and receipt must say which boundary remains and its expected expiry. A failed or unverifiable store keeps the operation open. ## Alternatives considered ### Rely on provider-managed encryption R2 and Neon already encrypt stored data. This is simpler, but the same provider account that stores the objects can authorize reads. Client-side envelopes add an independent application key and allow key rotation without rewriting large objects. ### Encrypt the entire bucket with one application key One key and nonce discipline would be easier to implement. It would make rotation rewrite every object and would give one key direct access to every ciphertext. Per-object data keys contain compromise and make rotation a bounded rewrap operation. ### Expire raw mail after a bounded processing window A bounded window would reduce storage cost and the impact of a future compromise. It would also turn R2 into a processing cache, prevent later parsers from reprocessing the historical corpus, and remove the project's independent recovery source after provider cleanup. Explicit archive deletion provides the owner-controlled boundary instead. ### Treat a removed database row or DVC pointer as deletion This ignores R2 objects, DVC caches, database history, temporary exports, and provider recovery areas. The store inventory and verified purge state cost more to operate, but they are the only way to make a deletion receipt meaningful. --- Markdown index of this site: https://robbiepalmer.me/llms.txt