Cloud sync, accounts and end-to-end encryption (design ideas)

Status: future plan, not a decision. This document explores options only. Nothing here is agreed or built. Decisions live in docs/project-plan/05-decisions-and-open-questions.md; the one relevant so far is D-27 (OS protection on the device, end-to-end encryption for anything that leaves it). Update 2026-10-06: the backend is now decided by D-33: Jotry’s own end-to-end encrypted server (section 5). The managed-backend options in addendum A are kept for the record only. D-35 confirms that hosted sync carries the full revision graph (section 2). Update 2026-10-08: D-45 makes hosted sync personal, with account-free sync groups, on the owner’s backend infrastructure behind HTTPS; section 10 compares server frameworks. Self-hosted Supabase (D-44, refined by D-45) is kept as the alternative backend in section 9; the crypto, key and sync-engine design applies to either. How sync fits paid tiers and store release, and the proposal to use account-free sync groups with QR pairing instead of sign-in, are in monetization-and-sync-tiers.md. Everything else here is still a proposal: if any of it is adopted, record it as a new D-xx decision and update docs/project-plan/.

Context

Jotry is local-first (Kotlin Multiplatform, SQLDelight/SQLite, native Android/iOS UI, a Ktor server/ stub). Release 1 has no account or sync. docs/project-plan/03-sync-and-extension-platform.md and 05-decisions-and-open-questions.md already reserve the seams: stable note/revision IDs, immutable revisions with parentRevisionId + contentHash, tombstones (deletedAt), a destination capability contract, and an idempotent outbox. They also list open decisions: E2EE requirement, key recovery, device enrollment, metadata leakage. This is a design for Phase 4 (history-aware cloud sync) that fits those seams. No code is changed now.

1. Core principle

The server is a dumb, encrypted blob store plus an auth service. It never sees plaintext, titles, notebook names or note structure. Everything below follows from that.

2. What gets synced

Sync the revision graph, not “current note state”. Each unit is an immutable, encrypted record:

• revision (noteId, revisionId, parentRevisionId, markdown snapshot, timestamps, titleSource)
• noteMeta (notebookId, pinnedAt, deletedAt) as a small mutable-by-supersession record
• notebook (name, order, archived)

Revisions are immutable, so they sync as append-only: no merge on the wire, and conflicts only show up as divergent branches in the graph. That matches the existing rule “divergent branches must be preserved, never silently discarded”. For a divergence, keep both heads, auto-create a “conflict copy” note (or a merge revision with two parents later), and surface it in the existing History UI.

Mutable metadata (pin, notebook move, rename, order) uses per-field last-writer-wins with a hybrid logical clock (HLC), not wall clock. Deletion is a tombstone that wins over older edits but is overridden by a later restore.

3. Encryption design

• Account root key (RK): 256-bit random, generated on the first device, never leaves devices in plaintext.
• Per-record keys: derive with HKDF from RK (info = "jotry/v1/record" || recordId), or wrap a random per-note data key under RK. Per-note keys make future note sharing possible without exposing RK.
• Cipher: XChaCha20-Poly1305 (or AES-256-GCM). Bind recordId, noteId, schemaVersion and contentHash as AAD so the server cannot swap or replay ciphertexts across records.
• Envelope format: {v, keyId, nonce, ciphertext} with a payload version from day one (the doc already requires versioned transfer payloads).
• Library: libsodium via a KMP binding (e.g. libsodium-bindings/ionspin) for shared code; fall back to platform crypto (Android Keystore/Tink, iOS CryptoKit) behind an expect/actual Crypto interface. Do not roll your own primitives.
• Local key storage: RK wrapped by a key in Android Keystore / iOS Keychain (ThisDeviceOnly, no iCloud keychain sync, consistent with D-10/D-11).

Login vs. encryption must be separate

Use two independent secrets so the server can authenticate you without being able to decrypt:

• Auth: an opaque PAKE such as OPAQUE (or SRP) so the password is never sent to the server. Simpler fallback: email magic link / passkeys (WebAuthn) for auth only.
• Encryption: RK, protected by the user’s recovery material (below). Never derive RK from the same server-visible value used to log in.

4. Key management and recovery (the hard part)

Pick one primary model; offer the second as opt-in.

Decided in D-48: C for adding devices and a 24-word recovery key (A) as the break-glass recovery, no password option. Original suggestion: C for adding devices + A as the break-glass recovery, with optional passphrase on top (B) for people who will not store a recovery key. Include key rotation (rewrap the RK-wrapped keys, and lazily re-encrypt records) and device revocation (rotate RK when a device is removed; revoked devices lose server access immediately even if they cannot lose the old key).

5. Server design (server/, Ktor)

Replace the hello-world with:

• Auth: account registration, OPAQUE/passkey login, short-lived access tokens + rotating refresh tokens, per-device tokens, device list and revoke.
• Storage: Postgres for accounts/devices/records metadata; object storage (S3-compatible) for large blobs if needed.
• Record API (all payloads opaque):
  • GET /v1/changes?cursor=… returns encrypted records after a server-assigned monotonically increasing sequence number (the changeTokens capability).
  • PUT /v1/records/{id} with If-Match / expected-version precondition (the conflictSignals + atomicWrite capabilities); idempotent by recordId (revisions are write-once, so replay is a no-op).
  • Tombstones are records, not row deletes.
  • Batched push/pull, with 412 mapped to “fetch then re-reconcile”.
• Realtime nudge: websocket/SSE or push notification only says “something changed”; the client then pulls. No content in pushes.
• Server-visible metadata (be honest about this): account/email, device list, record IDs, sizes, timestamps, sync frequency. Mitigate with padding to size buckets, opaque random record IDs (HMAC of note ID under RK, so the server cannot link records to notes), and no plaintext titles/names. Document this as the metadata threat model.
• Export/delete-account endpoint; server operators can only delete ciphertext.

6. Client architecture (fits existing seams)

• SyncCoordinator + CloudDestination in a new KMP module (e.g. shared:sync), implementing the adapter contract already specified: discover(cursor), fetch, push(change, precondition), delete, getStatus, with capabilities remoteHistory, remoteTombstones, stableItemIds, changeTokens, atomicWrite, conflictSignals, bidirectional, backgroundSync.
• Add to the SQLDelight schema: outbox(eventId, recordId, op, createdAt, attempts), sync_state(destinationId, cursor, lastSuccessAt), remote_map(recordId, serverVersion). Outbox rows are written in the same transaction as the revision (already the plan).
• Pipeline: local commit → outbox → encrypt → push with precondition → ack (clear outbox only after success) → pull changes → decrypt → reconcile by parent/hash → fast-forward or stage conflict.
• Background: WorkManager (Android) and BGTaskScheduler (iOS) plus a foreground sync on app open and after edits (debounced). Everything idempotent and resumable.
• Search stays fully local, which is a privacy win: no server-side search. Remote data is indexed only after decrypt.
• Settings UI: Account (sign in/out; or, with account-free sync groups as proposed in monetization-and-sync-tiers.md, “Add a device” with a QR code and “Leave sync”), Sync status (pending count, last success, errors), Devices, Recovery key, “Delete cloud data”. Keep the app fully usable signed out (sync stays opt-in; the README promise “never needs an account” stays true).
• Update README.md privacy section and docs/project-plan/05-decisions-and-open-questions.md (new decisions: E2EE required, key model, auth method, conflict model).

7. Phased delivery

1. Decisions + threat model (E2EE required; recovery model; PAKE vs passkeys; what metadata is acceptable). Write them into doc 05.
2. Crypto + envelope module with test vectors, expect/actual platform key storage, versioned payload format.
3. Server MVP: auth, devices, change-feed + conditional put, Postgres, contract tests.
4. Sync engine on one platform (Android first) with fault-injection tests: interrupted upload, duplicate delivery, two devices editing offline, clock skew, tombstone vs edit.
5. Device enrollment + recovery key + revocation/rotation.
6. iOS parity, then desktop/web reusing the same protocol (Phase 5 in the roadmap).
7. Hardening: external crypto review, rate limiting, account deletion, metadata padding.

8. Open questions to settle before building

• Is E2EE mandatory for all sync, or is there a non-E2EE tier? (Strong recommendation: mandatory, single mode.)
• Recovery: is “lose all devices and the recovery key = lose the data” acceptable?
• Self-hostable server (users run their own) vs hosted only? Self-hosting reduces trust in the operator and suits the privacy brand.
• Sharing/collaboration later? If yes, adopt per-note keys now.
• Free vs paid storage limits; how much history to sync by default (full graph vs recent N revisions + lazy fetch).
• Attachments/images: chunked + encrypted blobs, content-addressed by hash of ciphertext.

9. Self-hosted Supabase (the alternative to an own server, D-45)

D-45 leaves the backend open between a self-hosted Supabase stack on the backend infrastructure and an own Kotlin server (section 10). The server is a dumb store for ciphertext either way, and sections 1–4 and 6 apply to both. Facts about Supabase come from its docs and have not been run by us.

What Supabase would give, and what it wouldn’t

• No server code for the first version: Postgres tables, row-level security (RLS), a SQL function for the conditional put, Supabase Auth, and Realtime for the “something changed” nudge. Section 5’s GET /v1/changes and PUT /v1/records/{id} become PostgREST calls and one RPC.
• Not covered: account-free sync groups (Supabase Auth is built on users, so groups need a membership table and rules around it), OPAQUE, store receipt checks (an Edge Function or a small service), and per-device revocation beyond Auth sessions.
• Shared code keeps the adapter rule from addendum A: a SupabaseDestination behind the CloudDestination contract, using the Ktor client against the REST endpoints, with no Supabase SDK in shared code.

Services to run

The official self-hosting setup is a Docker Compose file routing everything through a Kong gateway. The minimum (inferred from the docs):

Five always-on services are well over the few hundred MB the backend infrastructure can spare (D-46), which is a large part of why section 10 recommends an own server.

Data model sketch (not final)

-- all payload columns are ciphertext; ids are opaque
create table sync_group (id uuid primary key default gen_random_uuid(), created_at timestamptz default now());
create table sync_member (group_id uuid references sync_group, user_id uuid references auth.users, device_label text, added_at timestamptz default now(), primary key (group_id, user_id));
create table record (
  group_id uuid not null references sync_group,
  record_id text not null,
  seq bigint generated always as identity,  -- change cursor
  version int not null default 1,
  envelope bytea not null,
  created_at timestamptz default now(),
  primary key (group_id, record_id)
);
-- RLS: a row is visible only to members of its group
-- rpc put_record(group, id, expected_version, envelope): inserts or updates when the version matches,
--   returns the new seq, or a conflict marker the client maps to "fetch then reconcile"

Secrets: the default Supabase .env ships demo keys. Generate our own JWT secret and keys, keep them out of git, and never put the service-role key in an app.

10. Server framework comparison (D-45, 2026-10-08)

D-45 fixes the scope (the owner’s own devices), the identity model (account-free sync groups) and the reach (the backend infrastructure behind HTTPS). It leaves the backend open between a self-hosted Supabase (section 9) and an own Kotlin server. Account-free groups are the case Supabase Auth fits worst, so this section compares Kotlin servers. The requirements are lean, a Kotlin codebase, and an easy path to scale if the app takes off. Framework facts come from docs and community posts; I found no published benchmark that compares all of them on the same load, so the numbers below are directional. Step K4 measures them, late, on the real hosting.

What the server has to do

About ten endpoints: register a device into a sync group, authenticate, GET changes?cursor, conditional PUT record, batch push and pull, a change nudge (SSE), delete a group. Every payload is opaque ciphertext. There is no templating, no ORM-heavy domain model and no admin UI. Throughput is bound by Postgres, not by the framework.

Candidates

Recommendation (to confirm)

Ktor with Postgres. It is already in the repo, it is Kotlin-first and coroutine-based, JetBrains maintains it (the same company as the KMP and Compose stack we use), and ten endpoints don’t need Spring’s batteries. The framework is not what limits scale; these are:

• Stateless instances. Verify device signatures or short-lived tokens locally, with no server-side sessions, so more instances sit behind Caddy.
• The change cursor. A plain Postgres sequence can expose rows out of commit order, so a client could skip a record it should have received. Use a per-group counter assigned under a row lock (or a commit-safe cutoff) so the cursor never skips. This is the part of the design to test hardest (step S2).
• The nudge. One instance can keep SSE connections in memory. With several instances, publish through Postgres LISTEN/NOTIFY (or Redis). Not needed for one server.
• Memory. The backend infrastructure is amd64 and shared with many other services, so the budget is a few hundred MB for the sync server. Cap the JVM heap (for example -Xmx128m to -Xmx192m, plus -XX:MaxMetaspaceSize and a small thread stack) and measure RSS, not heap. Postgres needs its own share (tune shared_buffers to tens of MB), or reuse a Postgres instance that already runs there (open question). A plain JAR is architecture-neutral, so building on the Mac (arm64) and running on amd64 needs no cross-compile; a native image would have to be built on an amd64 Linux host. If memory per instance ever matters, move to Quarkus or a native image; the endpoint code is small enough to port.

Runner-up: http4k, for the leanest code and easiest tests. Take Quarkus instead if native-image memory matters more than staying with Ktor. Take Spring Boot only if the server grows beyond the sync API (accounts, billing, admin).

Server structure (D-60)

The logic sits in a framework-free module behind interfaces, and the web framework, the Postgres store and the store verifiers are adapters. That keeps this section’s framework choice cheap to reverse. The layout, rules and local Docker Compose environments are in server/README.md.

Account-free auth sketch (to design in step K2)

• A group is a random 128-bit id plus a recovery-key-protected root key held by devices only.
• Each device creates an Ed25519 key pair. Enrolling a device means an existing device approves it over the QR handshake (section 4, option C), and the server stores the new device’s public key under the group.
• A device requests a short-lived token by signing a server challenge with its private key. Revoking a device removes its public key; rotating the root key follows section 4.
• The server never sees a password, an email or any key that decrypts a record.

Work order (replaces section 9’s order if Ktor is confirmed)

Verification (when implemented)

• Crypto: known-answer test vectors; tamper tests (flip a byte, swap record IDs → decrypt must fail).
• Server: contract tests for preconditions, idempotent puts, cursor pagination; confirm the DB contains no plaintext (grep fixtures for sentinel strings).
• Sync: two-emulator offline-edit scenarios; kill the app mid-push; replay duplicate batches; verify no silent data loss and conflicts appear as preserved branches.
• Security review of auth and key flows; run the security-review skill on the branch before release.

Addendum: answers to follow-up questions

A. Do I have to write and run the server? (Firebase / Supabase / similar)

No. With E2EE the server only authenticates users and stores opaque encrypted records, so a managed backend fits well. The “dumb blob store + auth” model in section 5 is exactly what BaaS products provide.

What changes versus the custom-server design:

• Login: BaaS auth is normally password-to-server (bcrypt), email link, OAuth or passkeys. That is fine as long as login is decoupled from encryption (already required in section 3). The password must not be what unlocks the data; use the recovery key / device enrollment for RK. Avoid OPAQUE initially; use passkeys or email magic link for auth.
• Metadata: the provider sees email, IPs, timestamps, sizes. Same threat model as section 5, but also a third party. State it in the privacy docs.
• Client: a KMP client talks to a thin CloudDestination adapter (REST via Ktor client against Supabase PostgREST/RPC, or the Firebase REST API), so the provider can be swapped without touching the sync engine. Do not adopt provider SDKs inside shared code; keep them behind the adapter.
• Leaning (not decided): Supabase (hosted) behind a thin adapter, with schema/RPC definitions kept in-repo as migrations. Revisit the Ktor server only if a requirement (OPAQUE, self-host packaging, compliance) demands it. The backend choice is explicitly deferred.
• Reopened by D-44 and D-45 (2026-10-08): self-hosted Supabase, on the owner’s own server, answers the third-party objection below; it is the alternative in section 9.
• Superseded by D-33 (2026-10-06): the own Ktor server was chosen instead, so that no third party sees sync metadata and so that account-free sync groups and server-side receipt checks stay under our control.

B. Are notes encrypted on the device at rest, and is it needed? (Outcome: recorded as D-27, no encryption at rest beyond the OS)

Current state (from docs/project-plan/05-decisions-and-open-questions.md): notes live in SQLite in app-private storage, protected only by platform encryption (Android file-based encryption, iOS Data Protection). App-level database encryption (SQLCipher) is explicitly an open question (“Is on-device database encryption required?”), to be decided before the schema is stable.

What the OS protects against: another app reading your data (sandbox), a lost/stolen locked phone (FBE/Data Protection keys are tied to the screen lock; on iOS default class is “until first unlock”, so data is readable after the first unlock until reboot), and cold-boot/flash-dump attacks. What it does not protect against: a unlocked phone, someone who knows the PIN, malware with root/jailbreak, forensic tools on some unlocked states, and (importantly for sync) any backup or extraction path that bypasses the sandbox. Android devices with no screen lock get effectively no protection.

Recommendation:

• Not required for launch/local-only. For a typical notes app the OS protection is the accepted baseline, and the README already tells users this. Adding SQLCipher now costs per-platform driver work, loses system SQLite/FTS options, and creates a recovery problem (lost key = lost notes), for modest extra benefit.
• Becomes more relevant with E2EE sync. Once the user holds an RK that makes the cloud ciphertext-only, a plaintext local DB is the weakest link and the thing users will point at. Reasonable stance: E2EE protects data off the device; the OS protects it on the device.
• Cheap middle path: make Keystore/Keychain keys ThisDeviceOnly, require the OS biometric/passcode for opening RK if the user turns on “App lock”, and exclude from backups (already D-10/D-11). Encrypt only the RK and tokens at rest in Keystore/Keychain (required anyway), not the whole DB.
• Optional later: an “Encrypt notes on this device” setting using SQLCipher with a Keystore/Keychain-held key, if the threat model (shared/rooted devices, compliance) demands it. Decide it before the schema stabilizes, because converting an existing plaintext DB later needs a migration (export, re-create encrypted DB, verify, swap).
• If we do encrypt the local DB, we can reuse the same envelope/crypto module from the sync work, but whole-DB (SQLCipher) is simpler than per-field encryption, which would break the local LIKE search.

Decision to record in doc 05: “On-device encryption: rely on OS protection for Release 1; E2EE applies to sync; revisit SQLCipher with an explicit threat model before schema freeze.”

11. S3 result: the :crypto module (2026-10-08)

A first prototype used cryptography-kotlin with AES-256-GCM and per-record HKDF keys. The owner then chose libsodium, XChaCha20-Poly1305 and per-note data keys wrapped under the root key (D-47), and the module was rebuilt on that. The design, formats, auth and enrollment flows are in sync-system-design.md. Tests: NoteKeysTest, RecordCipherTest and DeviceEnrollmentTest, 21 in all, pass on JVM, iOS simulator and an Android emulator (API 37); known answers come from an independent Python script (HChaCha20 checked against draft-irtf-cfrg-xchacha, plus the cryptography package and hashlib).