Folder sync and Markdown interoperability (design plan)

Status: future plan, not a decision. This is a proposal for the folder-sync phase (Phase 3 in 04-roadmap.md) and for the Markdown file format that export and folder sync share. The only parts that are agreed are recorded in 05-decisions-and-open-questions.md: D-06 (Android local-folder sync is the first sync extension), D-34 (folder sync and export are never paid features), D-35 (folder sync carries current notes only; full history is a hosted-sync feature) and D-36 (note links are exported as relative Markdown links). Everything else below is open until it is recorded as a new D-xx.

Goal

Let people who use other Markdown tools (Obsidian, VS Code, iA Writer, Typora, git, Syncthing, iCloud Drive) keep a folder of their Jotry notes that those tools can read and edit, with changes flowing both ways, without Jotry giving up its own guarantees: every change is a revision, nothing is silently lost, and identity survives renames.

For most users this feature is invisible. It lives under Settings → Advanced and needs a deliberate choice of folder. People who never open it are not affected.

How Obsidian does it (reference)

Obsidian is file-first: the folder (a “vault”) is the only source of truth.

• Identity is the file path. There are no note IDs. The .obsidian/ metadata cache is derived and can be rebuilt.
• When a file is renamed inside Obsidian, it rewrites every link that points to it. A rename done in Finder or another app looks like a delete plus a create, and links to it break.
• Links default to [[wikilinks]] resolved by the shortest unique path. Users can switch to standard Markdown links with relative paths (“Use [[Wikilinks]]” off, “New link format: Relative path to file”). People who care about interoperability usually do.
• Front matter is optional. Obsidian shows it as “Properties”.
• Obsidian does not move files between devices itself: its paid Sync service, iCloud, Syncthing or git do. Obsidian watches the folder and reloads changes. Conflict handling belongs to the sync tool (often a “conflicted copy” file); Obsidian Sync tries to merge text.
• History is a bolt-on: the File Recovery plugin keeps local snapshots, and Sync keeps version history.

Jotry is the opposite: database-first, ID-based identity, immutable revision history. Once another app can edit the folder, the folder is a second source of truth, and folder sync is a reconciliation problem, not an export problem. Jotry’s history model is the advantage here: an external change becomes an import revision, so nothing is overwritten.

(The Obsidian description is from general knowledge, not checked against its current documentation. Verify before relying on details such as setting names.)

Principles

1. The database stays the source of truth for identity and history. The folder holds current notes only (D-35).
2. Files look native. Human file names, standard Markdown, standard relative links, no required metadata block. A file written by Jotry should look like one written by hand.
3. Never lose either side. If both Jotry and the folder changed a note, both versions are kept.
4. Simple by default. One folder, one switch. Format options exist but are not needed.
5. The same file format as export. Export is a one-shot, one-way folder sync. Both use the same writer, reader and link translator.

1. File format (shared with export)

Layout

<chosen folder>/
  .jotry/                 managed-folder marker and format version (see "Managed folder marker")
  Q3 Plan.md              notes in the default notebook (D-15) sit at the root
  Work/
    Standup notes.md
    Q3 Plan 2.md          a second "Q3 Plan" in the same notebook
  Personal/
    Ideas.md

• Folder = notebook. Notebooks cannot nest (D-15), so only one folder level is used. Deeper folders created by other apps: see open question Q4.
• The default notebook is the root folder. This matches what the user sees at the top level.
• Archived notebooks: exported like any other notebook. Whether they also sync is open (Q9).

File names

• File name = note title, kept as typed: case, spaces, accents and most punctuation stay. Only characters that are unsafe on common file systems are replaced (/ \ : * ? " < > |, control characters, leading dots, trailing dots and spaces), and the name is capped at a safe length (for example 120 UTF-8 bytes before .md).
• A blank automatic title (D-23) uses the display title, otherwise “Untitled”.
• Collisions inside one folder get 2, 3 … (Obsidian’s style), assigned by note creation order so they stay stable.
• No --<uuid> suffix. Identity lives in the database (section 2), not in the name.
• The title comes back from the file name on import. Because names keep case and most punctuation, the round trip is lossless except for replaced characters.

Body

• The note’s Markdown exactly as stored, UTF-8, \n line endings, ending in a newline.
• No front matter by default. Front matter is not needed when the database holds identity (section 2), and files without it look native in every editor. Open question Q1 covers an opt-in “Write Jotry properties” setting.
• Front matter in files Jotry reads is preserved, never stripped. Front matter written by another app (Obsidian properties, tags, aliases) is kept byte-for-byte when Jotry rewrites the file. Jotry does not show or edit it in Release 1 of this feature (Q2).

Note links (D-36)

Inside the app, links stay jotry://note/<noteId> (D-25). At the file boundary they are translated:

• Writing: [label](jotry://note/<id>) becomes [label](<relative path>), for example [Ideas](../Personal/Ideas.md). Spaces and reserved characters are percent-encoded (Q3%20Plan.md), which Obsidian, VS Code, GitHub and most renderers understand. A link to a note that isn’t in the folder (trashed, or in an excluded notebook) is written as its plain label, keeping the text readable.
• Reading: a relative .md link that resolves to a mapped file becomes jotry://note/<id>. [[Wikilinks]] (with optional |label and #heading) are also accepted and resolved Obsidian’s way: exact path first, then the unique file with that name anywhere in the folder. Ambiguous or unresolved links are left untouched as text so nothing is lost.
• Renames inside Jotry: when a note’s title or notebook changes, Jotry renames the file and rewrites links to it in the other managed files, as Obsidian does. Each rewrite is a normal file write. It does not create revisions of the linking notes, because their stored Markdown (jotry:// links) has not changed.

The same translator is used by export (D-36), so export can ship first.

2. Identity and the sync map

A local table in the shared core holds the link between notes and files. The core owns it; the extension never reads the notes database directly.

Doc 03 already says: keep a local mapping of note ID to provider document, and do not depend on file names as identity. This table is that mapping.

Rename detection for changes made in other apps. In one scan, if a mapped file is missing and an unmapped file appears whose content hash equals the missing file’s last synced hash (or is very similar, with a clear threshold), treat it as a rename or move. Update relativePath and, if the folder changed, move the note to the matching notebook. Otherwise the missing file is a possible deletion (section 3) and the new file is a new note.

3. Reconcile (three-way)

Each scan compares, for every mapped note, the file now, the note now and the synced point (syncedRevisionId / syncedContentHash).

Rules that apply throughout:

• Only Jotry’s own writes, imports and moves touch notes. Every change goes through the normal transactional revision path (D-04).
• Imported changes respect D-14: imports always add a version.
• A file that cannot be parsed or decoded (not UTF-8, binary) is skipped with a warning. It is never deleted or rewritten.
• Files that aren’t Markdown are left alone and never deleted (attachments: Q7).
• Every step is idempotent, so an interrupted scan can simply run again.

4. Platforms

On desktop, the flow for power users becomes: phone ⇄ hosted sync ⇄ desktop app ⇄ folder ⇄ Obsidian / VS Code / git. Phones stay simple, and the desktop app is the bridge to other tools.

Large folders: compare size and modification date first and hash only when they differ, so repeated scans stay fast.

5. Core vs extension

The split follows doc 03 (destinations declare capabilities; extensions are built into the app, not downloaded plugins, which iOS does not allow anyway).

Shared core (correctness, one implementation):

• the file format writer and reader, including the link translator (also used by export)
• the sync map, three-way reconcile, rename detection and deletion checks
• turning file changes into revisions, and conflict branches
• the status model (pending, last success, warnings, actions needed)

Folder extension (transport, per platform):

• choosing the folder and keeping access (SAF permission, iOS bookmark, desktop path)
• listing, reading and writing files, atomic replace where available
• change detection and scheduling (WorkManager, foreground triggers, desktop watcher)
• declaring its capabilities: bidirectional; no remoteHistory; stableItemIds only through the sync map; atomicWrite and changeTokens depending on the provider

The same core is reused by hosted sync (see monetization-and-sync-tiers.md), which adds history transport on top.

6. Managed folder marker

Doc 03 proposes a manifest file in the folder root. Proposed shape: a .jotry/ folder holding folder.json with the format version, the device that set the folder up and the date. Its purposes:

• recognize a folder Jotry already manages (re-selecting it after reinstalling the app, or from a second device)
• refuse to manage a folder written by a newer, unknown format version
• avoid treating a folder another tool owns as Jotry’s

It holds no note data. A dot-folder is hidden by Obsidian and most file browsers.

7. User experience

• Settings → Advanced → Folder sync: “Choose folder”, then a status line (“Synced 2 minutes ago”, “3 changes waiting”, “1 conflict”). Turning it off stops sync and leaves the files where they are.
• Before turning it on, one short explanation: the folder holds your current notes; version history stays in the app; other apps can edit these files and Jotry will pick up their changes.
• A non-empty folder that isn’t managed yet: offer to import its Markdown files (the existing import summary) and then manage it.
• Conflicts appear on the note (a small banner) and in History, which already shows branches and restore. Choosing a version is a normal restore.
• Optional format settings (only if Q1/Q3 need them): link style for new links (relative or wikilink), “Write Jotry properties to files”.
• A short guide for Obsidian users: turn off wikilinks and choose relative paths, so links written in Obsidian also work in Jotry.

8. Export changes that fall out of this

Export becomes “write the folder format once”, so it gains the same benefits before folder sync exists:

1. Relative note links (D-36), using the shared translator.
2. Human file names, as in section 1. Decided for export by D-42.
3. The front-matter question (Q1) applies to export too. Decided for export by D-42: export always writes front matter (note_id and the rest), because an export folder has no sync map to recognize its files on re-import. Q1 remains open for folder sync only.

Import keeps reading today’s exports (front matter with note_id, --<uuid> file names), so old exports keep working.

9. Test plan (when implemented)

• Round trip: export → import into an empty app gives identical notes, notebooks and links.
• Obsidian-style edits: rename in another app, move between folders, add a wikilink, add front matter properties; each lands as the expected revision, with properties preserved.
• Conflicts: edit the same note in Jotry and in the folder between scans; both versions survive.
• Deletion safety: an unreachable folder, a partial scan, a revoked permission, and a folder emptied by a sync tool; none of these trash notes on the first scan.
• Collisions and unsafe names: titles that differ only in case, titles with / or :, very long titles, accented characters (Unicode normalization on macOS and iOS).
• Interruptions: kill the app mid-write and mid-scan; the next run converges and creates no duplicates.
• Scale: a folder with thousands of notes; repeated scans stay fast.

Open questions