Vibecode Roam Research
track this build5 phases, 15 steps, beginner friendly0%YES · one-shottable
price $15/moyou'd save $180/yrbuild time one sittingcategory 🧠 notes & knowledgereplaced by 0 people
Roam Research's solo core is compact: build a local-first networked notes app with fast capture, full-text search, backlinks, tags, and Markdown export. A competent builder can reach a useful personal version in one sitting, while the paid product mainly wins on polish, sync, importers.
the project pack5 files · written for this build, step by step
You are building a lean indie version of Roam Research. Create the following project files first, then implement the application by following them. Keep the files updated as decisions change. Do not collapse this into a single README or prompt. ===== README.md ===== # Roam Research · indie build A desktop notes application whose storage is a directory of Markdown files you can read without it: daily notes for capture, wiki-style links that build a backlink index, block references, full text search that stays instant at ten thousand notes, and an export that is simply the folder you already have. Estimated effort: **weekend**. Work `BUILD_PLAN.md` top to bottom · every phase ends in a check that has to pass before the next one starts. ## Stack | Part | Choice | Why | | --- | --- | --- | | Shell | Tauri with a TypeScript front end | a native file watcher and real filesystem access, in a binary a fraction of the size of the alternative | | Storage | One Markdown file per note in a directory you choose | plain files mean the notes outlive the application, which is the entire point of local-first | | Index | SQLite with full text search, treated as a disposable cache | the files are the truth and the index is rebuildable, so a corrupt index is an inconvenience rather than data loss | | Parsing | An incremental Markdown parser that extracts links and blocks | reparsing the whole vault on every keystroke does not scale past a few hundred notes | | Watching | A filesystem watcher with debounce | edits made in another editor must show up, because plain files invite exactly that | ## Before you start Have every one of these ready. The plan assumes them from step one. - [ ] **Node.js 22 or newer** · free - Why: Builds the front end and runs the tooling. - Get it: nodejs.org, or nvm install 22. - Verify: node --version prints v22 or higher - [ ] **The Rust toolchain** · free - Why: Tauri compiles a native binary, and the filesystem watcher lives on that side. - Get it: Install with rustup from rustup.rs. On Linux you will also need the platform packages Tauri lists for your distribution. - Verify: cargo --version prints a version and a Tauri hello world builds - [ ] **A directory for the vault, and a decision about backups** · free - Why: These are your notes in plain files. The application does not protect them; your backup does. - Get it: Choose a directory covered by your existing backup, or set one up now. If you intend to sync it, decide between git and a file sync drive before Phase 6. - Verify: The chosen directory is included in a backup that has been restored at least once - [ ] **A second Markdown editor** · free - Why: Phase 5 verifies that the vault is readable without this application, which is the whole claim of local-first. You cannot test that with the application that wrote it. - Get it: Any general Markdown editor you already have. - Verify: You can open the vault and read notes in it - [ ] **A generated vault of ten thousand notes** · free - Why: Performance claims made against two hundred notes are meaningless, and the design that fails at scale feels fine until it does not. - Get it: Write the generator in Phase 3 before optimising anything. Plausible link density matters more than realistic prose. - Verify: The generator produces ten thousand linked notes in a scratch directory ## Quick start ```sh npm create tauri-app@latest ``` Then copy `.env.example` to `.env` and fill in the values it documents. ## Honest limits This build deliberately does not replace: - Conflict-free sync and real-time collaboration. - The query language, which is a substantial feature in its own right. - Mature importers from other notes tools. - Sharing, publishing and collaborative editing. - conflict-safe multi-device sync - mature importers and exporters - collaborative editing and sharing - native mobile clients If one of those is essential to you, that is the reason to keep paying for Roam Research, and the README should say so rather than pretend. ===== BRIEF.md ===== # Build brief · Roam Research The one-shot brief this plan expands. `BUILD_PLAN.md` (or `MILESTONES.md`) is the same sequence broken into steps and checks; where the two disagree, the plan wins. Build me a local-first networked notes application with fast capture, backlinks and instant search over plain Markdown files to replace Roam Research. Build it in phases, in the order below. Do not write the whole thing in one pass. Finish a phase, run its "Done when" check, fix what fails, and only then start the next phase. ### Stack (fixed, do not substitute) - Shell: Tauri with a TypeScript front end · a native file watcher and real filesystem access, in a binary a fraction of the size of the alternative - Storage: One Markdown file per note in a directory you choose · plain files mean the notes outlive the application, which is the entire point of local-first - Index: SQLite with full text search, treated as a disposable cache · the files are the truth and the index is rebuildable, so a corrupt index is an inconvenience rather than data loss - Parsing: An incremental Markdown parser that extracts links and blocks · reparsing the whole vault on every keystroke does not scale past a few hundred notes - Watching: A filesystem watcher with debounce · edits made in another editor must show up, because plain files invite exactly that ### Data model (create this before Phase 1) - Files are the source of truth. Everything below is an index that can be deleted and rebuilt. - notes(id, path, title, mtime, size, content_hash) - blocks(id, note_id, ordinal, depth, text, block_uid) · block_uid is a stable identifier written into the file so references survive edits - links(from_note_id, to_title, from_block_id, kind) · kind is wikilink, tag or block_ref; to_title is resolved lazily so a link to a note that does not exist yet still works - notes_fts · a full text index over block text - A link to a non-existent note is normal and useful, not an error. Resolving it late is what makes capture fast. ### Phase 1 · Files, watcher and the daily note Build: Choose a vault directory. Create today's daily note on launch. Write changes to disk with a debounce. Watch the directory so a file edited elsewhere updates the view. No index yet, no links yet: just fast, reliable capture into plain files. Trap: Your own writes will trigger your own file watcher and can produce a loop that reloads the note under the user's cursor. Track the hash you last wrote and ignore watcher events matching it, rather than trying to suppress the watcher around writes, which races. Done when: Typing into the daily note and quitting immediately leaves the text on disk. Editing the same file in another editor updates the view within a second without losing the cursor position. The application never reloads a note in response to its own write. Do not build yet: No links, no search, no backlinks. Capture that does not lose text. ### Phase 2 · Links and backlinks Build: Parse wiki-style links and tags out of each note into the links table. Show a backlinks panel listing the blocks that reference the current note, with enough surrounding text to be readable. Links to notes that do not exist yet are valid and create the note when followed. Trap: Resolving a link to a note identifier at parse time means every link to a not-yet-created note is either an error or a silently created empty file. Store the link target as text and resolve when rendering, which also makes renaming a note a cheap operation. Done when: Typing a link to a note that does not exist renders as a link, and following it creates the note. The target note's backlinks panel shows the referring block with surrounding context. Renaming a note updates what its backlinks display without rewriting every file that links to it. Do not build yet: No search and no block references yet. ### Phase 3 · Search that stays instant Build: Index block text into SQLite full text search, updating only changed notes on save. Search as you type with results grouped by note. Rebuild the whole index from the files on demand and prove that the index is genuinely disposable. Trap: Test at the size you intend to reach, not the size you have. A vault of two hundred notes is fast with any approach, including a naive grep, and the design that fails at ten thousand feels perfect until the day it does not. Generate ten thousand notes and search them before you commit to the approach. Done when: Search across ten thousand generated notes returns results in well under a hundred milliseconds as you type. Deleting the index file and relaunching rebuilds it from the Markdown files with identical results. A note edited externally is searchable within a second. Do not build yet: No block references yet. ### Phase 4 · Block references and transclusion Build: Give blocks stable identifiers written into the file as an unobtrusive suffix. A block reference renders the referenced block inline where it is written. Editing the source block updates every place it appears. Trap: A block identifier derived from position changes every time a line is inserted above it, silently repointing every reference. Write a generated identifier into the file the first time a block is referenced, and never derive identity from position or content. Done when: Referencing a block and then inserting three paragraphs above it in the source note leaves the reference pointing at the same text. Editing the source block updates the rendered reference. Deleting the source block leaves a visible broken reference rather than an empty space. Do not build yet: No graph view or export yet. ### Phase 5 · Graph, export and the honest limits Build: A graph view of notes and links, filtered to a neighbourhood rather than rendering everything. Export is the vault directory itself, which is the point, so the work is confirming that another application can read it. Document what your syntax does that plain Markdown does not. Trap: A force-directed graph of ten thousand nodes renders as a hairball and takes several seconds. Show a neighbourhood at a chosen depth from the current note. It is more useful and it is fast. Done when: The graph renders a neighbourhood at a chosen depth in under a second on the ten thousand note vault. Opening the vault in a general Markdown editor shows readable notes, with only the block identifiers as visible extras. The README documents every syntax extension. Do not build yet: Sync is the product phase and is the hard part. ### Phase 6 · Sync, and being honest about conflicts Build: Sync the vault directory with a file sync tool or a git repository. Detect conflicts and surface them as notes you can resolve. Do not attempt real-time collaborative editing. Trap: File sync tools resolve a simultaneous edit by keeping one version and, at best, leaving the other beside it with a mangled filename. Watch for conflict files, surface them in the application as something to resolve, and never assume the sync tool told the user anything. Done when: A simultaneous edit on two machines produces a visible conflict inside the application rather than a silently lost edit. A vault synced through git can be committed and pulled without the application corrupting the index. The interface states that simultaneous editing of one note is not supported. Do not build yet: Nothing after this. ### Out of scope (and why) - Conflict-free multi-device sync with real-time collaboration. That is the genuinely hard part and a naive version loses work quietly. - The query language. Roam's queries are a substantial feature and reproducing them well is a project of comparable size to everything above. - Mature importers for other tools' formats, each of which is a week of edge cases. - Collaborative editing and sharing, which need a server, accounts and a permission model. ### README must contain - That the Markdown files are the truth and the index is a disposable cache - Every syntax extension, particularly block identifiers, and how to strip them if you leave - That search is tested against a ten thousand note vault, with the measured figure - Which sync mechanism is supported and how conflicts appear - That simultaneous editing of one note is not supported ===== AGENTS.md ===== # Agent instructions · Roam Research indie build - Read `README.md` and `BUILD_PLAN.md` before writing code. The stack is fixed: Tauri with a TypeScript front end, One Markdown file per note in a directory you choose, SQLite with full text search, treated as a disposable cache, An incremental Markdown parser that extracts links and blocks, A filesystem watcher with debounce. Do not substitute. - Work one phase at a time, in order. Do not start a phase until every "Done when" item of the previous one passes. - Prefer the fewest moving parts that satisfy the step. No frameworks, services or dependencies the plan does not name. - Secrets live in `.env`, never in source or logs. Keep `.env.example` current when a variable is introduced. - Do not invent cryptography, security guarantees, APIs or compliance claims. - Add a focused test for every destructive, security-sensitive or data-loss path the plan names. - Run the project checks before declaring a phase complete, and record any deliberate shortcut in the README under "Tradeoffs". ## Known traps - Your own writes will trigger your own file watcher and can produce a loop that reloads the note under the user's cursor. Track the hash you last wrote and ignore watcher events matching it, rather than trying to suppress the watcher around writes, which races. - Resolving a link to a note identifier at parse time means every link to a not-yet-created note is either an error or a silently created empty file. Store the link target as text and resolve when rendering, which also makes renaming a note a cheap operation. - Test at the size you intend to reach, not the size you have. A vault of two hundred notes is fast with any approach, including a naive grep, and the design that fails at ten thousand feels perfect until the day it does not. Generate ten thousand notes and search them before you commit to the approach. - A block identifier derived from position changes every time a line is inserted above it, silently repointing every reference. Write a generated identifier into the file the first time a block is referenced, and never derive identity from position or content. - A force-directed graph of ten thousand nodes renders as a hairball and takes several seconds. Show a neighbourhood at a chosen depth from the current note. It is more useful and it is fast. ===== BUILD_PLAN.md ===== # Build plan · Roam Research A desktop notes application whose storage is a directory of Markdown files you can read without it: daily notes for capture, wiki-style links that build a backlink index, block references, full text search that stays instant at ten thousand notes, and an export that is simply the folder you already have. Phases are in dependency order. Each ends in a "Done when" list; treat an unticked item as a blocker, not a note. ## Phase 1 · Files, watcher and the daily note Capture is instant, and everything you type lands in a file you could open in any editor. ### Steps 1. Set up Tauri and pick a vault A directory chooser stored in settings. Everything else is relative to it. Files: `src-tauri/`, `src/vault.ts` ```sh npm create tauri-app@latest ``` 2. Write with debounce and atomic replace Debounce keystrokes, then write to a temporary file and rename over the target. A partial write during a crash is how people lose a day of notes. Files: `src/storage.ts` 3. Watch without looping Record the hash of every write. Ignore watcher events whose content hash matches what you just wrote. Files: `src/watcher.ts` ### Done when - [ ] Text typed and immediately quit is present on disk - [ ] An external edit appears within a second - [ ] The application never reloads in response to its own write - [ ] A crash mid-write leaves either the old file or the new one, never a truncated one ### Watch out - Your own writes will trigger your own file watcher and can produce a loop that reloads the note under the user's cursor. Track the hash you last wrote and ignore watcher events matching it, rather than trying to suppress the watcher around writes, which races. ## Phase 2 · Links and backlinks Typing a link creates a connection, and the target note shows what points at it. ### Steps 1. Parse links and tags incrementally On save, reparse only the changed note and replace its rows in the links table. Full vault reparse belongs to index rebuild, not to editing. Files: `src/parse.ts` 2. Store link targets as text Resolve to a note at render time. This is what makes links to future notes work and renames cheap. Files: `src/links.ts` 3. Build the backlinks panel Group by source note, show the referring block plus its parent for context. A bare list of note titles is much less useful than it looks. Files: `src/backlinks.ts` ### Done when - [ ] A link to a non-existent note renders and creates it when followed - [ ] Backlinks show the referring block with surrounding context - [ ] Renaming a note does not require rewriting files that link to it - [ ] A note linked from fifty places lists all fifty ### Watch out - Resolving a link to a note identifier at parse time means every link to a not-yet-created note is either an error or a silently created empty file. Store the link target as text and resolve when rendering, which also makes renaming a note a cheap operation. ## Phase 3 · Search that stays instant Search is fast enough to be the primary way you navigate, at ten thousand notes. ### Steps 1. Generate a realistic vault for testing Ten thousand notes with plausible link density. This fixture is what makes every later performance claim meaningful. Files: `scripts/generate-vault.ts` 2. Index incrementally on save Replace only the changed note's rows. A full reindex on every keystroke is the obvious implementation and the wrong one. Files: `src/index.ts` 3. Make rebuild a first-class action A menu item that deletes and rebuilds. Run it regularly during development so index corruption is never frightening. Files: `src/index.ts` ### Done when - [ ] Search over ten thousand notes returns in well under a hundred milliseconds - [ ] Deleting the index and relaunching rebuilds it with identical results - [ ] An externally edited note is searchable within a second - [ ] Indexing a single note edit does not reindex the vault ### Watch out - Test at the size you intend to reach, not the size you have. A vault of two hundred notes is fast with any approach, including a naive grep, and the design that fails at ten thousand feels perfect until the day it does not. Generate ten thousand notes and search them before you commit to the approach. ## Phase 4 · Block references and transclusion You can point at one paragraph rather than a whole note, and see it in place. ### Steps 1. Assign identifiers on first reference Generate an identifier and append it to the source line in a form that reads acceptably in a plain text editor. Do not identify every block eagerly; most are never referenced. Files: `src/blocks.ts` 2. Render references inline Look up by identifier and render the block's text with an affordance to jump to its source. Files: `src/transclude.ts` 3. Handle the broken case visibly A reference to a deleted block shows as broken, naming the identifier. Silence here produces documents with holes in them. Files: `src/transclude.ts` ### Done when - [ ] Inserting text above a referenced block does not break the reference - [ ] Editing the source block updates every rendered reference - [ ] A deleted source block leaves a visible broken reference - [ ] Identifiers are readable in a plain text editor without being obtrusive ### Watch out - A block identifier derived from position changes every time a line is inserted above it, silently repointing every reference. Write a generated identifier into the file the first time a block is referenced, and never derive identity from position or content. ## Phase 5 · Graph, export and the honest limits You can see the shape of the vault, and leaving costs nothing. ### Steps 1. Render a neighbourhood graph Nodes within a chosen link distance of the current note, not the entire vault. Files: `src/graph.ts` 2. Verify the vault reads elsewhere Open it in a different Markdown editor and read several notes. This is the test that local-first is real rather than a slogan. 3. Document the syntax extensions Block identifiers and any non-standard link syntax, so someone leaving knows exactly what to strip. Files: `README.md` ### Done when - [ ] The neighbourhood graph renders in under a second on the large vault - [ ] The vault is readable in a different Markdown editor - [ ] Every syntax extension is documented - [ ] No note requires this application to be understood ### Watch out - A force-directed graph of ten thousand nodes renders as a hairball and takes several seconds. Show a neighbourhood at a chosen depth from the current note. It is more useful and it is fast. ## Not in this build - Conflict-free sync and real-time collaboration. - The query language, which is a substantial feature in its own right. - Mature importers from other notes tools. - Sharing, publishing and collaborative editing. ## After v1, if you want it - A simple query syntax over the links table, starting with all notes linking to two given notes - Incremental parsing on keystroke rather than on save, once the save path is proven - A publish command that renders a subset of the vault as static HTML ===== .env.example ===== # Copy to .env and fill in. Never commit .env; this file documents it. # Required. The directory of Markdown files. This is your data and it belongs in your backup. VAULT_PATH=~/Notes # Required. The search index. Deliberately outside the vault so syncing cannot corrupt it. Safe to delete. INDEX_PATH=~/.local/share/notes/index.db # Required. How long to wait after typing before writing. Lower loses less on a crash and writes more often. WRITE_DEBOUNCE_MS=400 # Required. How many link hops the graph view shows from the current note. GRAPH_DEPTH=2 # Required. Filename format for daily notes. Changing it later does not rename existing notes. DAILY_NOTE_FORMAT=YYYY-MM-DD
You are building a lean indie version of Roam Research. Create the following project files first, then implement the application by following them. Keep the files updated as decisions change. Do not collapse this into a single README or prompt. ===== README.md ===== # Roam Research · indie build A desktop notes application whose storage is a directory of Markdown files you can read without it: daily notes for capture, wiki-style links that build a backlink index, block references, full text search that stays instant at ten thousand notes, and an export that is simply the folder you already have. Estimated effort: **weekend**. Work `BUILD_PLAN.md` top to bottom · every phase ends in a check that has to pass before the next one starts. ## Stack | Part | Choice | Why | | --- | --- | --- | | Shell | Tauri with a TypeScript front end | a native file watcher and real filesystem access, in a binary a fraction of the size of the alternative | | Storage | One Markdown file per note in a directory you choose | plain files mean the notes outlive the application, which is the entire point of local-first | | Index | SQLite with full text search, treated as a disposable cache | the files are the truth and the index is rebuildable, so a corrupt index is an inconvenience rather than data loss | | Parsing | An incremental Markdown parser that extracts links and blocks | reparsing the whole vault on every keystroke does not scale past a few hundred notes | | Watching | A filesystem watcher with debounce | edits made in another editor must show up, because plain files invite exactly that | ## Before you start Have every one of these ready. The plan assumes them from step one. - [ ] **Node.js 22 or newer** · free - Why: Builds the front end and runs the tooling. - Get it: nodejs.org, or nvm install 22. - Verify: node --version prints v22 or higher - [ ] **The Rust toolchain** · free - Why: Tauri compiles a native binary, and the filesystem watcher lives on that side. - Get it: Install with rustup from rustup.rs. On Linux you will also need the platform packages Tauri lists for your distribution. - Verify: cargo --version prints a version and a Tauri hello world builds - [ ] **A directory for the vault, and a decision about backups** · free - Why: These are your notes in plain files. The application does not protect them; your backup does. - Get it: Choose a directory covered by your existing backup, or set one up now. If you intend to sync it, decide between git and a file sync drive before Phase 6. - Verify: The chosen directory is included in a backup that has been restored at least once - [ ] **A second Markdown editor** · free - Why: Phase 5 verifies that the vault is readable without this application, which is the whole claim of local-first. You cannot test that with the application that wrote it. - Get it: Any general Markdown editor you already have. - Verify: You can open the vault and read notes in it - [ ] **A generated vault of ten thousand notes** · free - Why: Performance claims made against two hundred notes are meaningless, and the design that fails at scale feels fine until it does not. - Get it: Write the generator in Phase 3 before optimising anything. Plausible link density matters more than realistic prose. - Verify: The generator produces ten thousand linked notes in a scratch directory ## Quick start ```sh npm create tauri-app@latest ``` Then copy `.env.example` to `.env` and fill in the values it documents. ## Honest limits This build deliberately does not replace: - Conflict-free sync and real-time collaboration. - The query language, which is a substantial feature in its own right. - Mature importers from other notes tools. - Sharing, publishing and collaborative editing. - conflict-safe multi-device sync - mature importers and exporters - collaborative editing and sharing - native mobile clients If one of those is essential to you, that is the reason to keep paying for Roam Research, and the README should say so rather than pretend. ===== BRIEF.md ===== # Build brief · Roam Research The one-shot brief this plan expands. `BUILD_PLAN.md` (or `MILESTONES.md`) is the same sequence broken into steps and checks; where the two disagree, the plan wins. Build me a local-first networked notes application with fast capture, backlinks and instant search over plain Markdown files to replace Roam Research. Build it in phases, in the order below. Do not write the whole thing in one pass. Finish a phase, run its "Done when" check, fix what fails, and only then start the next phase. ### Stack (fixed, do not substitute) - Shell: Tauri with a TypeScript front end · a native file watcher and real filesystem access, in a binary a fraction of the size of the alternative - Storage: One Markdown file per note in a directory you choose · plain files mean the notes outlive the application, which is the entire point of local-first - Index: SQLite with full text search, treated as a disposable cache · the files are the truth and the index is rebuildable, so a corrupt index is an inconvenience rather than data loss - Parsing: An incremental Markdown parser that extracts links and blocks · reparsing the whole vault on every keystroke does not scale past a few hundred notes - Watching: A filesystem watcher with debounce · edits made in another editor must show up, because plain files invite exactly that ### Data model (create this before Phase 1) - Files are the source of truth. Everything below is an index that can be deleted and rebuilt. - notes(id, path, title, mtime, size, content_hash) - blocks(id, note_id, ordinal, depth, text, block_uid) · block_uid is a stable identifier written into the file so references survive edits - links(from_note_id, to_title, from_block_id, kind) · kind is wikilink, tag or block_ref; to_title is resolved lazily so a link to a note that does not exist yet still works - notes_fts · a full text index over block text - A link to a non-existent note is normal and useful, not an error. Resolving it late is what makes capture fast. ### Phase 1 · Files, watcher and the daily note Build: Choose a vault directory. Create today's daily note on launch. Write changes to disk with a debounce. Watch the directory so a file edited elsewhere updates the view. No index yet, no links yet: just fast, reliable capture into plain files. Trap: Your own writes will trigger your own file watcher and can produce a loop that reloads the note under the user's cursor. Track the hash you last wrote and ignore watcher events matching it, rather than trying to suppress the watcher around writes, which races. Done when: Typing into the daily note and quitting immediately leaves the text on disk. Editing the same file in another editor updates the view within a second without losing the cursor position. The application never reloads a note in response to its own write. Do not build yet: No links, no search, no backlinks. Capture that does not lose text. ### Phase 2 · Links and backlinks Build: Parse wiki-style links and tags out of each note into the links table. Show a backlinks panel listing the blocks that reference the current note, with enough surrounding text to be readable. Links to notes that do not exist yet are valid and create the note when followed. Trap: Resolving a link to a note identifier at parse time means every link to a not-yet-created note is either an error or a silently created empty file. Store the link target as text and resolve when rendering, which also makes renaming a note a cheap operation. Done when: Typing a link to a note that does not exist renders as a link, and following it creates the note. The target note's backlinks panel shows the referring block with surrounding context. Renaming a note updates what its backlinks display without rewriting every file that links to it. Do not build yet: No search and no block references yet. ### Phase 3 · Search that stays instant Build: Index block text into SQLite full text search, updating only changed notes on save. Search as you type with results grouped by note. Rebuild the whole index from the files on demand and prove that the index is genuinely disposable. Trap: Test at the size you intend to reach, not the size you have. A vault of two hundred notes is fast with any approach, including a naive grep, and the design that fails at ten thousand feels perfect until the day it does not. Generate ten thousand notes and search them before you commit to the approach. Done when: Search across ten thousand generated notes returns results in well under a hundred milliseconds as you type. Deleting the index file and relaunching rebuilds it from the Markdown files with identical results. A note edited externally is searchable within a second. Do not build yet: No block references yet. ### Phase 4 · Block references and transclusion Build: Give blocks stable identifiers written into the file as an unobtrusive suffix. A block reference renders the referenced block inline where it is written. Editing the source block updates every place it appears. Trap: A block identifier derived from position changes every time a line is inserted above it, silently repointing every reference. Write a generated identifier into the file the first time a block is referenced, and never derive identity from position or content. Done when: Referencing a block and then inserting three paragraphs above it in the source note leaves the reference pointing at the same text. Editing the source block updates the rendered reference. Deleting the source block leaves a visible broken reference rather than an empty space. Do not build yet: No graph view or export yet. ### Phase 5 · Graph, export and the honest limits Build: A graph view of notes and links, filtered to a neighbourhood rather than rendering everything. Export is the vault directory itself, which is the point, so the work is confirming that another application can read it. Document what your syntax does that plain Markdown does not. Trap: A force-directed graph of ten thousand nodes renders as a hairball and takes several seconds. Show a neighbourhood at a chosen depth from the current note. It is more useful and it is fast. Done when: The graph renders a neighbourhood at a chosen depth in under a second on the ten thousand note vault. Opening the vault in a general Markdown editor shows readable notes, with only the block identifiers as visible extras. The README documents every syntax extension. Do not build yet: Sync is the product phase and is the hard part. ### Phase 6 · Sync, and being honest about conflicts Build: Sync the vault directory with a file sync tool or a git repository. Detect conflicts and surface them as notes you can resolve. Do not attempt real-time collaborative editing. Trap: File sync tools resolve a simultaneous edit by keeping one version and, at best, leaving the other beside it with a mangled filename. Watch for conflict files, surface them in the application as something to resolve, and never assume the sync tool told the user anything. Done when: A simultaneous edit on two machines produces a visible conflict inside the application rather than a silently lost edit. A vault synced through git can be committed and pulled without the application corrupting the index. The interface states that simultaneous editing of one note is not supported. Do not build yet: Nothing after this. ### Out of scope (and why) - Conflict-free multi-device sync with real-time collaboration. That is the genuinely hard part and a naive version loses work quietly. - The query language. Roam's queries are a substantial feature and reproducing them well is a project of comparable size to everything above. - Mature importers for other tools' formats, each of which is a week of edge cases. - Collaborative editing and sharing, which need a server, accounts and a permission model. ### README must contain - That the Markdown files are the truth and the index is a disposable cache - Every syntax extension, particularly block identifiers, and how to strip them if you leave - That search is tested against a ten thousand note vault, with the measured figure - Which sync mechanism is supported and how conflicts appear - That simultaneous editing of one note is not supported ===== AGENTS.md ===== # Agent instructions · Roam Research indie build - Read `README.md` and `BUILD_PLAN.md` before writing code. The stack is fixed: Tauri with a TypeScript front end, One Markdown file per note in a directory you choose, SQLite with full text search, treated as a disposable cache, An incremental Markdown parser that extracts links and blocks, A filesystem watcher with debounce. Do not substitute. - Work one phase at a time, in order. Do not start a phase until every "Done when" item of the previous one passes. - Prefer the fewest moving parts that satisfy the step. No frameworks, services or dependencies the plan does not name. - Secrets live in `.env`, never in source or logs. Keep `.env.example` current when a variable is introduced. - Do not invent cryptography, security guarantees, APIs or compliance claims. - Add a focused test for every destructive, security-sensitive or data-loss path the plan names. - Run the project checks before declaring a phase complete, and record any deliberate shortcut in the README under "Tradeoffs". ## Known traps - Your own writes will trigger your own file watcher and can produce a loop that reloads the note under the user's cursor. Track the hash you last wrote and ignore watcher events matching it, rather than trying to suppress the watcher around writes, which races. - Resolving a link to a note identifier at parse time means every link to a not-yet-created note is either an error or a silently created empty file. Store the link target as text and resolve when rendering, which also makes renaming a note a cheap operation. - Test at the size you intend to reach, not the size you have. A vault of two hundred notes is fast with any approach, including a naive grep, and the design that fails at ten thousand feels perfect until the day it does not. Generate ten thousand notes and search them before you commit to the approach. - A block identifier derived from position changes every time a line is inserted above it, silently repointing every reference. Write a generated identifier into the file the first time a block is referenced, and never derive identity from position or content. - A force-directed graph of ten thousand nodes renders as a hairball and takes several seconds. Show a neighbourhood at a chosen depth from the current note. It is more useful and it is fast. ===== BUILD_PLAN.md ===== # Build plan · Roam Research A desktop notes application whose storage is a directory of Markdown files you can read without it: daily notes for capture, wiki-style links that build a backlink index, block references, full text search that stays instant at ten thousand notes, and an export that is simply the folder you already have. Phases are in dependency order. Each ends in a "Done when" list; treat an unticked item as a blocker, not a note. ## Phase 1 · Files, watcher and the daily note Capture is instant, and everything you type lands in a file you could open in any editor. ### Steps 1. Set up Tauri and pick a vault A directory chooser stored in settings. Everything else is relative to it. Files: `src-tauri/`, `src/vault.ts` ```sh npm create tauri-app@latest ``` 2. Write with debounce and atomic replace Debounce keystrokes, then write to a temporary file and rename over the target. A partial write during a crash is how people lose a day of notes. Files: `src/storage.ts` 3. Watch without looping Record the hash of every write. Ignore watcher events whose content hash matches what you just wrote. Files: `src/watcher.ts` ### Done when - [ ] Text typed and immediately quit is present on disk - [ ] An external edit appears within a second - [ ] The application never reloads in response to its own write - [ ] A crash mid-write leaves either the old file or the new one, never a truncated one ### Watch out - Your own writes will trigger your own file watcher and can produce a loop that reloads the note under the user's cursor. Track the hash you last wrote and ignore watcher events matching it, rather than trying to suppress the watcher around writes, which races. ## Phase 2 · Links and backlinks Typing a link creates a connection, and the target note shows what points at it. ### Steps 1. Parse links and tags incrementally On save, reparse only the changed note and replace its rows in the links table. Full vault reparse belongs to index rebuild, not to editing. Files: `src/parse.ts` 2. Store link targets as text Resolve to a note at render time. This is what makes links to future notes work and renames cheap. Files: `src/links.ts` 3. Build the backlinks panel Group by source note, show the referring block plus its parent for context. A bare list of note titles is much less useful than it looks. Files: `src/backlinks.ts` ### Done when - [ ] A link to a non-existent note renders and creates it when followed - [ ] Backlinks show the referring block with surrounding context - [ ] Renaming a note does not require rewriting files that link to it - [ ] A note linked from fifty places lists all fifty ### Watch out - Resolving a link to a note identifier at parse time means every link to a not-yet-created note is either an error or a silently created empty file. Store the link target as text and resolve when rendering, which also makes renaming a note a cheap operation. ## Phase 3 · Search that stays instant Search is fast enough to be the primary way you navigate, at ten thousand notes. ### Steps 1. Generate a realistic vault for testing Ten thousand notes with plausible link density. This fixture is what makes every later performance claim meaningful. Files: `scripts/generate-vault.ts` 2. Index incrementally on save Replace only the changed note's rows. A full reindex on every keystroke is the obvious implementation and the wrong one. Files: `src/index.ts` 3. Make rebuild a first-class action A menu item that deletes and rebuilds. Run it regularly during development so index corruption is never frightening. Files: `src/index.ts` ### Done when - [ ] Search over ten thousand notes returns in well under a hundred milliseconds - [ ] Deleting the index and relaunching rebuilds it with identical results - [ ] An externally edited note is searchable within a second - [ ] Indexing a single note edit does not reindex the vault ### Watch out - Test at the size you intend to reach, not the size you have. A vault of two hundred notes is fast with any approach, including a naive grep, and the design that fails at ten thousand feels perfect until the day it does not. Generate ten thousand notes and search them before you commit to the approach. ## Phase 4 · Block references and transclusion You can point at one paragraph rather than a whole note, and see it in place. ### Steps 1. Assign identifiers on first reference Generate an identifier and append it to the source line in a form that reads acceptably in a plain text editor. Do not identify every block eagerly; most are never referenced. Files: `src/blocks.ts` 2. Render references inline Look up by identifier and render the block's text with an affordance to jump to its source. Files: `src/transclude.ts` 3. Handle the broken case visibly A reference to a deleted block shows as broken, naming the identifier. Silence here produces documents with holes in them. Files: `src/transclude.ts` ### Done when - [ ] Inserting text above a referenced block does not break the reference - [ ] Editing the source block updates every rendered reference - [ ] A deleted source block leaves a visible broken reference - [ ] Identifiers are readable in a plain text editor without being obtrusive ### Watch out - A block identifier derived from position changes every time a line is inserted above it, silently repointing every reference. Write a generated identifier into the file the first time a block is referenced, and never derive identity from position or content. ## Phase 5 · Graph, export and the honest limits You can see the shape of the vault, and leaving costs nothing. ### Steps 1. Render a neighbourhood graph Nodes within a chosen link distance of the current note, not the entire vault. Files: `src/graph.ts` 2. Verify the vault reads elsewhere Open it in a different Markdown editor and read several notes. This is the test that local-first is real rather than a slogan. 3. Document the syntax extensions Block identifiers and any non-standard link syntax, so someone leaving knows exactly what to strip. Files: `README.md` ### Done when - [ ] The neighbourhood graph renders in under a second on the large vault - [ ] The vault is readable in a different Markdown editor - [ ] Every syntax extension is documented - [ ] No note requires this application to be understood ### Watch out - A force-directed graph of ten thousand nodes renders as a hairball and takes several seconds. Show a neighbourhood at a chosen depth from the current note. It is more useful and it is fast. ## Not in this build - Conflict-free sync and real-time collaboration. - The query language, which is a substantial feature in its own right. - Mature importers from other notes tools. - Sharing, publishing and collaborative editing. ## After v1, if you want it - A simple query syntax over the links table, starting with all notes linking to two given notes - Incremental parsing on keystroke rather than on save, once the save path is proven - A publish command that renders a subset of the vault as static HTML ===== .env.example ===== # Copy to .env and fill in. Never commit .env; this file documents it. # Required. The directory of Markdown files. This is your data and it belongs in your backup. VAULT_PATH=~/Notes # Required. The search index. Deliberately outside the vault so syncing cannot corrupt it. Safe to delete. INDEX_PATH=~/.local/share/notes/index.db # Required. How long to wait after typing before writing. Lower loses less on a crash and writes more often. WRITE_DEBOUNCE_MS=400 # Required. How many link hops the graph view shows from the current note. GRAPH_DEPTH=2 # Required. Filename format for daily notes. Changing it later does not rename existing notes. DAILY_NOTE_FORMAT=YYYY-MM-DD
You are building a production product version of Roam Research. Create the following project files first, then implement the application by following them. Keep the files updated as decisions change. Do not collapse this into a single README or prompt. ===== PRODUCT.md ===== # Roam Research · product brief ## Problem Roam Research's solo core is compact: build a local-first networked notes app with fast capture, full-text search, backlinks, tags, and Markdown export. A competent builder can reach a useful personal version in one sitting, while the paid product mainly wins on polish, sync, importers. ## Product outcome A notes application whose data is a folder of Markdown you could read in any editor, with backlinks and block references that survive editing, search that is instant at ten thousand notes, and conflicts that are visible rather than silent. ## Target user A builder who needs a maintainable product foundation, not a one-off demo. ## Required capabilities - desktop development machine - Node.js 22 and Rust toolchain - local filesystem access ## Explicit non-goals for v1 - Conflict-free sync and real-time collaboration. - The query language, which is a substantial feature in its own right. - Mature importers from other notes tools. - Sharing, publishing and collaborative editing. - conflict-safe multi-device sync - mature importers and exporters - collaborative editing and sharing - native mobile clients ## Success criteria - Search latency on the ten thousand note fixture is measured and has not regressed - Index rebuild produces identical results to the incremental index - The vault opens and reads correctly in a different Markdown editor - A block reference survives text inserted above its source - A simulated sync conflict is visible in the interface ===== BRIEF.md ===== # Build brief · Roam Research The one-shot brief this plan expands. `BUILD_PLAN.md` (or `MILESTONES.md`) is the same sequence broken into steps and checks; where the two disagree, the plan wins. Build me a local-first networked notes application with fast capture, backlinks and instant search over plain Markdown files to replace Roam Research. Build it in phases, in the order below. Do not write the whole thing in one pass. Finish a phase, run its "Done when" check, fix what fails, and only then start the next phase. ### Stack (fixed, do not substitute) - Shell: Tauri with a TypeScript front end · a native file watcher and real filesystem access, in a binary a fraction of the size of the alternative - Storage: One Markdown file per note in a directory you choose · plain files mean the notes outlive the application, which is the entire point of local-first - Index: SQLite with full text search, treated as a disposable cache · the files are the truth and the index is rebuildable, so a corrupt index is an inconvenience rather than data loss - Parsing: An incremental Markdown parser that extracts links and blocks · reparsing the whole vault on every keystroke does not scale past a few hundred notes - Watching: A filesystem watcher with debounce · edits made in another editor must show up, because plain files invite exactly that ### Data model (create this before Phase 1) - Files are the source of truth. Everything below is an index that can be deleted and rebuilt. - notes(id, path, title, mtime, size, content_hash) - blocks(id, note_id, ordinal, depth, text, block_uid) · block_uid is a stable identifier written into the file so references survive edits - links(from_note_id, to_title, from_block_id, kind) · kind is wikilink, tag or block_ref; to_title is resolved lazily so a link to a note that does not exist yet still works - notes_fts · a full text index over block text - A link to a non-existent note is normal and useful, not an error. Resolving it late is what makes capture fast. ### Phase 1 · Files, watcher and the daily note Build: Choose a vault directory. Create today's daily note on launch. Write changes to disk with a debounce. Watch the directory so a file edited elsewhere updates the view. No index yet, no links yet: just fast, reliable capture into plain files. Trap: Your own writes will trigger your own file watcher and can produce a loop that reloads the note under the user's cursor. Track the hash you last wrote and ignore watcher events matching it, rather than trying to suppress the watcher around writes, which races. Done when: Typing into the daily note and quitting immediately leaves the text on disk. Editing the same file in another editor updates the view within a second without losing the cursor position. The application never reloads a note in response to its own write. Do not build yet: No links, no search, no backlinks. Capture that does not lose text. ### Phase 2 · Links and backlinks Build: Parse wiki-style links and tags out of each note into the links table. Show a backlinks panel listing the blocks that reference the current note, with enough surrounding text to be readable. Links to notes that do not exist yet are valid and create the note when followed. Trap: Resolving a link to a note identifier at parse time means every link to a not-yet-created note is either an error or a silently created empty file. Store the link target as text and resolve when rendering, which also makes renaming a note a cheap operation. Done when: Typing a link to a note that does not exist renders as a link, and following it creates the note. The target note's backlinks panel shows the referring block with surrounding context. Renaming a note updates what its backlinks display without rewriting every file that links to it. Do not build yet: No search and no block references yet. ### Phase 3 · Search that stays instant Build: Index block text into SQLite full text search, updating only changed notes on save. Search as you type with results grouped by note. Rebuild the whole index from the files on demand and prove that the index is genuinely disposable. Trap: Test at the size you intend to reach, not the size you have. A vault of two hundred notes is fast with any approach, including a naive grep, and the design that fails at ten thousand feels perfect until the day it does not. Generate ten thousand notes and search them before you commit to the approach. Done when: Search across ten thousand generated notes returns results in well under a hundred milliseconds as you type. Deleting the index file and relaunching rebuilds it from the Markdown files with identical results. A note edited externally is searchable within a second. Do not build yet: No block references yet. ### Phase 4 · Block references and transclusion Build: Give blocks stable identifiers written into the file as an unobtrusive suffix. A block reference renders the referenced block inline where it is written. Editing the source block updates every place it appears. Trap: A block identifier derived from position changes every time a line is inserted above it, silently repointing every reference. Write a generated identifier into the file the first time a block is referenced, and never derive identity from position or content. Done when: Referencing a block and then inserting three paragraphs above it in the source note leaves the reference pointing at the same text. Editing the source block updates the rendered reference. Deleting the source block leaves a visible broken reference rather than an empty space. Do not build yet: No graph view or export yet. ### Phase 5 · Graph, export and the honest limits Build: A graph view of notes and links, filtered to a neighbourhood rather than rendering everything. Export is the vault directory itself, which is the point, so the work is confirming that another application can read it. Document what your syntax does that plain Markdown does not. Trap: A force-directed graph of ten thousand nodes renders as a hairball and takes several seconds. Show a neighbourhood at a chosen depth from the current note. It is more useful and it is fast. Done when: The graph renders a neighbourhood at a chosen depth in under a second on the ten thousand note vault. Opening the vault in a general Markdown editor shows readable notes, with only the block identifiers as visible extras. The README documents every syntax extension. Do not build yet: Sync is the product phase and is the hard part. ### Phase 6 · Sync, and being honest about conflicts Build: Sync the vault directory with a file sync tool or a git repository. Detect conflicts and surface them as notes you can resolve. Do not attempt real-time collaborative editing. Trap: File sync tools resolve a simultaneous edit by keeping one version and, at best, leaving the other beside it with a mangled filename. Watch for conflict files, surface them in the application as something to resolve, and never assume the sync tool told the user anything. Done when: A simultaneous edit on two machines produces a visible conflict inside the application rather than a silently lost edit. A vault synced through git can be committed and pulled without the application corrupting the index. The interface states that simultaneous editing of one note is not supported. Do not build yet: Nothing after this. ### Out of scope (and why) - Conflict-free multi-device sync with real-time collaboration. That is the genuinely hard part and a naive version loses work quietly. - The query language. Roam's queries are a substantial feature and reproducing them well is a project of comparable size to everything above. - Mature importers for other tools' formats, each of which is a week of edge cases. - Collaborative editing and sharing, which need a server, accounts and a permission model. ### README must contain - That the Markdown files are the truth and the index is a disposable cache - Every syntax extension, particularly block identifiers, and how to strip them if you leave - That search is tested against a ten thousand note vault, with the measured figure - Which sync mechanism is supported and how conflicts appear - That simultaneous editing of one note is not supported ===== ARCHITECTURE.md ===== # Architecture · Roam Research ## Stack | Part | Choice | Why | | --- | --- | --- | | Shell | Tauri with a TypeScript front end | a native file watcher and real filesystem access, in a binary a fraction of the size of the alternative | | Storage | One Markdown file per note in a directory you choose | plain files mean the notes outlive the application, which is the entire point of local-first | | Index | SQLite with full text search, treated as a disposable cache | the files are the truth and the index is rebuildable, so a corrupt index is an inconvenience rather than data loss | | Parsing | An incremental Markdown parser that extracts links and blocks | reparsing the whole vault on every keystroke does not scale past a few hundred notes | | Watching | A filesystem watcher with debounce | edits made in another editor must show up, because plain files invite exactly that | ## Modules Each module has one owner concern and a documented way to replace it. | Module | Owns | How to replace it | | --- | --- | --- | | Storage | Reading and writing Markdown files atomically. | Not swappable without abandoning the local-first claim. Everything else is built to be rebuildable from here. | | Index | Search, links and backlinks as a cache. | Entirely replaceable, and the rebuild action is what proves it. If a swap would lose data, the boundary has been drawn wrong. | | Parser | Extracting links, tags and blocks from Markdown. | A different parser is fine provided identifiers stay stable, because references are written into the files. | | Watcher | Noticing external edits without looping. | Platform-specific behind one interface. The hash-based loop prevention must survive any change. | | Sync | Moving the vault between machines and surfacing conflicts. | Git or a file sync drive. What must not change is that the index stays outside the synced directory. | ## Configuration Every runtime setting is an environment variable documented in `.env.example`, validated at startup, with a safe local default wherever one exists. - `VAULT_PATH` · required · The directory of Markdown files. This is your data and it belongs in your backup. - `INDEX_PATH` · required · The search index. Deliberately outside the vault so syncing cannot corrupt it. Safe to delete. - `WRITE_DEBOUNCE_MS` · required · How long to wait after typing before writing. Lower loses less on a crash and writes more often. - `GRAPH_DEPTH` · required · How many link hops the graph view shows from the current note. - `DAILY_NOTE_FORMAT` · required · Filename format for daily notes. Changing it later does not rename existing notes. ## Production baseline - Security: least privilege, input validation at every boundary, secret redaction in logs, rate limits on abuse-prone paths, no invented security primitives. - Data: explicit schema and migrations, transactional writes where integrity matters, backup and restore procedures that have been exercised. - Integrations: adapters around third-party providers, idempotent webhook or job processing, bounded retries, timeouts. - Observability: structured logs with request or operation ids, an error-tracking hook, and health and readiness checks where a server exists. - Quality: unit tests for domain rules, integration tests at module boundaries, one end-to-end test of the critical path. ## Decision records For each dependency in the stack table, keep a short note: why it was chosen, its failure mode, and how it is replaced. Do not add infrastructure until a requirement in `PRODUCT.md` justifies it. ===== AGENTS.md ===== # Agent instructions · Roam Research product build - Read `PRODUCT.md` and `ARCHITECTURE.md` before changing code. The stack is fixed: Tauri with a TypeScript front end, One Markdown file per note in a directory you choose, SQLite with full text search, treated as a disposable cache, An incremental Markdown parser that extracts links and blocks, A filesystem watcher with debounce. - Implement milestone by milestone from `MILESTONES.md`; keep each change reviewable and leave the application runnable at every commit. - Treat authentication, payments, encryption, imports, webhooks and destructive actions as high-risk boundaries when present. - Never invent cryptography or silently weaken a requirement to make a check pass. - Put every external service behind an interface with a deterministic fake for tests. - Add migrations and rollback or recovery notes for every persistent data change. - Log useful operational context without credentials, tokens, passwords or personal data. - Update documentation and run every check before completing a milestone. ## Known traps - Your own writes will trigger your own file watcher and can produce a loop that reloads the note under the user's cursor. Track the hash you last wrote and ignore watcher events matching it, rather than trying to suppress the watcher around writes, which races. - Resolving a link to a note identifier at parse time means every link to a not-yet-created note is either an error or a silently created empty file. Store the link target as text and resolve when rendering, which also makes renaming a note a cheap operation. - Test at the size you intend to reach, not the size you have. A vault of two hundred notes is fast with any approach, including a naive grep, and the design that fails at ten thousand feels perfect until the day it does not. Generate ten thousand notes and search them before you commit to the approach. - A block identifier derived from position changes every time a line is inserted above it, silently repointing every reference. Write a generated identifier into the file the first time a block is referenced, and never derive identity from position or content. - A force-directed graph of ten thousand nodes renders as a hairball and takes several seconds. Show a neighbourhood at a chosen depth from the current note. It is more useful and it is fast. - File sync tools resolve a simultaneous edit by keeping one version and, at best, leaving the other beside it with a mangled filename. Watch for conflict files, surface them in the application as something to resolve, and never assume the sync tool told the user anything. ===== MILESTONES.md ===== # Delivery milestones · Roam Research Estimated effort: **weekend** for the indie phases; the production-only milestones add the trust and operability layer. ## M1 · Files, watcher and the daily note Capture is instant, and everything you type lands in a file you could open in any editor. ### Steps 1. Set up Tauri and pick a vault A directory chooser stored in settings. Everything else is relative to it. Files: `src-tauri/`, `src/vault.ts` ```sh npm create tauri-app@latest ``` 2. Write with debounce and atomic replace Debounce keystrokes, then write to a temporary file and rename over the target. A partial write during a crash is how people lose a day of notes. Files: `src/storage.ts` 3. Watch without looping Record the hash of every write. Ignore watcher events whose content hash matches what you just wrote. Files: `src/watcher.ts` ### Done when - [ ] Text typed and immediately quit is present on disk - [ ] An external edit appears within a second - [ ] The application never reloads in response to its own write - [ ] A crash mid-write leaves either the old file or the new one, never a truncated one ### Watch out - Your own writes will trigger your own file watcher and can produce a loop that reloads the note under the user's cursor. Track the hash you last wrote and ignore watcher events matching it, rather than trying to suppress the watcher around writes, which races. ## M2 · Links and backlinks Typing a link creates a connection, and the target note shows what points at it. ### Steps 1. Parse links and tags incrementally On save, reparse only the changed note and replace its rows in the links table. Full vault reparse belongs to index rebuild, not to editing. Files: `src/parse.ts` 2. Store link targets as text Resolve to a note at render time. This is what makes links to future notes work and renames cheap. Files: `src/links.ts` 3. Build the backlinks panel Group by source note, show the referring block plus its parent for context. A bare list of note titles is much less useful than it looks. Files: `src/backlinks.ts` ### Done when - [ ] A link to a non-existent note renders and creates it when followed - [ ] Backlinks show the referring block with surrounding context - [ ] Renaming a note does not require rewriting files that link to it - [ ] A note linked from fifty places lists all fifty ### Watch out - Resolving a link to a note identifier at parse time means every link to a not-yet-created note is either an error or a silently created empty file. Store the link target as text and resolve when rendering, which also makes renaming a note a cheap operation. ## M3 · Search that stays instant Search is fast enough to be the primary way you navigate, at ten thousand notes. ### Steps 1. Generate a realistic vault for testing Ten thousand notes with plausible link density. This fixture is what makes every later performance claim meaningful. Files: `scripts/generate-vault.ts` 2. Index incrementally on save Replace only the changed note's rows. A full reindex on every keystroke is the obvious implementation and the wrong one. Files: `src/index.ts` 3. Make rebuild a first-class action A menu item that deletes and rebuilds. Run it regularly during development so index corruption is never frightening. Files: `src/index.ts` ### Done when - [ ] Search over ten thousand notes returns in well under a hundred milliseconds - [ ] Deleting the index and relaunching rebuilds it with identical results - [ ] An externally edited note is searchable within a second - [ ] Indexing a single note edit does not reindex the vault ### Watch out - Test at the size you intend to reach, not the size you have. A vault of two hundred notes is fast with any approach, including a naive grep, and the design that fails at ten thousand feels perfect until the day it does not. Generate ten thousand notes and search them before you commit to the approach. ## M4 · Block references and transclusion You can point at one paragraph rather than a whole note, and see it in place. ### Steps 1. Assign identifiers on first reference Generate an identifier and append it to the source line in a form that reads acceptably in a plain text editor. Do not identify every block eagerly; most are never referenced. Files: `src/blocks.ts` 2. Render references inline Look up by identifier and render the block's text with an affordance to jump to its source. Files: `src/transclude.ts` 3. Handle the broken case visibly A reference to a deleted block shows as broken, naming the identifier. Silence here produces documents with holes in them. Files: `src/transclude.ts` ### Done when - [ ] Inserting text above a referenced block does not break the reference - [ ] Editing the source block updates every rendered reference - [ ] A deleted source block leaves a visible broken reference - [ ] Identifiers are readable in a plain text editor without being obtrusive ### Watch out - A block identifier derived from position changes every time a line is inserted above it, silently repointing every reference. Write a generated identifier into the file the first time a block is referenced, and never derive identity from position or content. ## M5 · Graph, export and the honest limits You can see the shape of the vault, and leaving costs nothing. ### Steps 1. Render a neighbourhood graph Nodes within a chosen link distance of the current note, not the entire vault. Files: `src/graph.ts` 2. Verify the vault reads elsewhere Open it in a different Markdown editor and read several notes. This is the test that local-first is real rather than a slogan. 3. Document the syntax extensions Block identifiers and any non-standard link syntax, so someone leaving knows exactly what to strip. Files: `README.md` ### Done when - [ ] The neighbourhood graph renders in under a second on the large vault - [ ] The vault is readable in a different Markdown editor - [ ] Every syntax extension is documented - [ ] No note requires this application to be understood ### Watch out - A force-directed graph of ten thousand nodes renders as a hairball and takes several seconds. Show a neighbourhood at a chosen depth from the current note. It is more useful and it is fast. ## M6 · Sync, and being honest about conflicts (production only) Two machines can edit the vault, and a conflict is visible rather than silent. ### Steps 1. Choose and document the sync mechanism Git gives history and explicit conflicts; a file sync drive is simpler and hides them. Pick one and say which in the README. Files: `README.md` 2. Detect and surface conflict files Watch for the naming patterns sync tools produce and list them in the application as notes needing attention. Files: `src/conflicts.ts` 3. Make the index resilient to sync The index must never be inside the synced directory, or two machines will fight over it and corrupt it. Files: `src/index.ts` ### Done when - [ ] A simultaneous edit surfaces as a visible conflict - [ ] The index lives outside the synced directory - [ ] A git-synced vault survives commit and pull without index corruption - [ ] The single-editor limitation is stated in the interface ### Watch out - File sync tools resolve a simultaneous edit by keeping one version and, at best, leaving the other beside it with a mangled filename. Watch for conflict files, surface them in the application as something to resolve, and never assume the sync tool told the user anything. ===== OPERATIONS.md ===== # Operations · Roam Research ## Backup The vault directory is everything. Ordinary file backup covers it, which is the practical benefit of plain files. The index is explicitly not worth backing up. ## Restore Copy the vault back and launch. The index rebuilds. If a restore requires anything else, something has leaked out of the vault that should not have. Do a restore drill before the first real user, and write the date here when it passes. ## Monitoring Nothing runs as a service. What is worth watching during development is index rebuild time and search latency against the ten thousand note fixture, tracked over releases so a regression is caught. ## Incident checklist The realistic incidents are a sync conflict and a corrupt index. The first is surfaced in Phase 6, the second is fixed by the rebuild action, which is why rebuild is a menu item rather than a hidden repair path. 1. Contain the issue without destroying evidence or user data. 2. Record the timeline and affected scope. 3. Rotate exposed secrets and revoke compromised sessions or credentials. 4. Restore from a verified backup when needed. 5. Document the root cause, the remediation and the regression test. ## Release gate - [ ] Search latency on the ten thousand note fixture is measured and has not regressed - [ ] Index rebuild produces identical results to the incremental index - [ ] The vault opens and reads correctly in a different Markdown editor - [ ] A block reference survives text inserted above its source - [ ] A simulated sync conflict is visible in the interface ## Launch constraint Do not market omitted Roam Research capabilities as implemented. The non-goals in `PRODUCT.md` remain user-visible limitations until they are deliberately delivered. ===== .env.example ===== # Copy to .env and fill in. Never commit .env; this file documents it. # Required. The directory of Markdown files. This is your data and it belongs in your backup. VAULT_PATH=~/Notes # Required. The search index. Deliberately outside the vault so syncing cannot corrupt it. Safe to delete. INDEX_PATH=~/.local/share/notes/index.db # Required. How long to wait after typing before writing. Lower loses less on a crash and writes more often. WRITE_DEBOUNCE_MS=400 # Required. How many link hops the graph view shows from the current note. GRAPH_DEPTH=2 # Required. Filename format for daily notes. Changing it later does not rename existing notes. DAILY_NOTE_FORMAT=YYYY-MM-DD
README.md
# Roam Research · indie build A desktop notes application whose storage is a directory of Markdown files you can read without it: daily notes for capture, wiki-style links that build a backlink index, block references, full text search that stays instant at ten thousand notes, and an export that is simply the folder you already have. Estimated effort: **weekend**. Work `BUILD_PLAN.md` top to bottom · every phase ends in a check that has to pass before the next one starts. ## Stack | Part | Choice | Why | | --- | --- | --- | | Shell | Tauri with a TypeScript front end | a native file watcher and real filesystem access, in a binary a fraction of the size of the alternative | | Storage | One Markdown file per note in a directory you choose | plain files mean the notes outlive the application, which is the entire point of local-first | | Index | SQLite with full text search, treated as a disposable cache | the files are the truth and the index is rebuildable, so a corrupt index is an inconvenience rather than data loss | | Parsing | An incremental Markdown parser that extracts links and blocks | reparsing the whole vault on every keystroke does not scale past a few hundred notes | | Watching | A filesystem watcher with debounce | edits made in another editor must show up, because plain files invite exactly that | ## Before you start Have every one of these ready. The plan assumes them from step one. - [ ] **Node.js 22 or newer** · free - Why: Builds the front end and runs the tooling. - Get it: nodejs.org, or nvm install 22. - Verify: node --version prints v22 or higher - [ ] **The Rust toolchain** · free - Why: Tauri compiles a native binary, and the filesystem watcher lives on that side. - Get it: Install with rustup from rustup.rs. On Linux you will also need the platform packages Tauri lists for your distribution. - Verify: cargo --version prints a version and a Tauri hello world builds - [ ] **A directory for the vault, and a decision about backups** · free - Why: These are your notes in plain files. The application does not protect them; your backup does. - Get it: Choose a directory covered by your existing backup, or set one up now. If you intend to sync it, decide between git and a file sync drive before Phase 6. - Verify: The chosen directory is included in a backup that has been restored at least once - [ ] **A second Markdown editor** · free - Why: Phase 5 verifies that the vault is readable without this application, which is the whole claim of local-first. You cannot test that with the application that wrote it. - Get it: Any general Markdown editor you already have. - Verify: You can open the vault and read notes in it - [ ] **A generated vault of ten thousand notes** · free - Why: Performance claims made against two hundred notes are meaningless, and the design that fails at scale feels fine until it does not. - Get it: Write the generator in Phase 3 before optimising anything. Plausible link density matters more than realistic prose. - Verify: The generator produces ten thousand linked notes in a scratch directory ## Quick start ```sh npm create tauri-app@latest ``` Then copy `.env.example` to `.env` and fill in the values it documents. ## Honest limits This build deliberately does not replace: - Conflict-free sync and real-time collaboration. - The query language, which is a substantial feature in its own right. - Mature importers from other notes tools. - Sharing, publishing and collaborative editing. - conflict-safe multi-device sync - mature importers and exporters - collaborative editing and sharing - native mobile clients If one of those is essential to you, that is the reason to keep paying for Roam Research, and the README should say so rather than pretend.
BRIEF.md
# Build brief · Roam Research The one-shot brief this plan expands. `BUILD_PLAN.md` (or `MILESTONES.md`) is the same sequence broken into steps and checks; where the two disagree, the plan wins. Build me a local-first networked notes application with fast capture, backlinks and instant search over plain Markdown files to replace Roam Research. Build it in phases, in the order below. Do not write the whole thing in one pass. Finish a phase, run its "Done when" check, fix what fails, and only then start the next phase. ### Stack (fixed, do not substitute) - Shell: Tauri with a TypeScript front end · a native file watcher and real filesystem access, in a binary a fraction of the size of the alternative - Storage: One Markdown file per note in a directory you choose · plain files mean the notes outlive the application, which is the entire point of local-first - Index: SQLite with full text search, treated as a disposable cache · the files are the truth and the index is rebuildable, so a corrupt index is an inconvenience rather than data loss - Parsing: An incremental Markdown parser that extracts links and blocks · reparsing the whole vault on every keystroke does not scale past a few hundred notes - Watching: A filesystem watcher with debounce · edits made in another editor must show up, because plain files invite exactly that ### Data model (create this before Phase 1) - Files are the source of truth. Everything below is an index that can be deleted and rebuilt. - notes(id, path, title, mtime, size, content_hash) - blocks(id, note_id, ordinal, depth, text, block_uid) · block_uid is a stable identifier written into the file so references survive edits - links(from_note_id, to_title, from_block_id, kind) · kind is wikilink, tag or block_ref; to_title is resolved lazily so a link to a note that does not exist yet still works - notes_fts · a full text index over block text - A link to a non-existent note is normal and useful, not an error. Resolving it late is what makes capture fast. ### Phase 1 · Files, watcher and the daily note Build: Choose a vault directory. Create today's daily note on launch. Write changes to disk with a debounce. Watch the directory so a file edited elsewhere updates the view. No index yet, no links yet: just fast, reliable capture into plain files. Trap: Your own writes will trigger your own file watcher and can produce a loop that reloads the note under the user's cursor. Track the hash you last wrote and ignore watcher events matching it, rather than trying to suppress the watcher around writes, which races. Done when: Typing into the daily note and quitting immediately leaves the text on disk. Editing the same file in another editor updates the view within a second without losing the cursor position. The application never reloads a note in response to its own write. Do not build yet: No links, no search, no backlinks. Capture that does not lose text. ### Phase 2 · Links and backlinks Build: Parse wiki-style links and tags out of each note into the links table. Show a backlinks panel listing the blocks that reference the current note, with enough surrounding text to be readable. Links to notes that do not exist yet are valid and create the note when followed. Trap: Resolving a link to a note identifier at parse time means every link to a not-yet-created note is either an error or a silently created empty file. Store the link target as text and resolve when rendering, which also makes renaming a note a cheap operation. Done when: Typing a link to a note that does not exist renders as a link, and following it creates the note. The target note's backlinks panel shows the referring block with surrounding context. Renaming a note updates what its backlinks display without rewriting every file that links to it. Do not build yet: No search and no block references yet. ### Phase 3 · Search that stays instant Build: Index block text into SQLite full text search, updating only changed notes on save. Search as you type with results grouped by note. Rebuild the whole index from the files on demand and prove that the index is genuinely disposable. Trap: Test at the size you intend to reach, not the size you have. A vault of two hundred notes is fast with any approach, including a naive grep, and the design that fails at ten thousand feels perfect until the day it does not. Generate ten thousand notes and search them before you commit to the approach. Done when: Search across ten thousand generated notes returns results in well under a hundred milliseconds as you type. Deleting the index file and relaunching rebuilds it from the Markdown files with identical results. A note edited externally is searchable within a second. Do not build yet: No block references yet. ### Phase 4 · Block references and transclusion Build: Give blocks stable identifiers written into the file as an unobtrusive suffix. A block reference renders the referenced block inline where it is written. Editing the source block updates every place it appears. Trap: A block identifier derived from position changes every time a line is inserted above it, silently repointing every reference. Write a generated identifier into the file the first time a block is referenced, and never derive identity from position or content. Done when: Referencing a block and then inserting three paragraphs above it in the source note leaves the reference pointing at the same text. Editing the source block updates the rendered reference. Deleting the source block leaves a visible broken reference rather than an empty space. Do not build yet: No graph view or export yet. ### Phase 5 · Graph, export and the honest limits Build: A graph view of notes and links, filtered to a neighbourhood rather than rendering everything. Export is the vault directory itself, which is the point, so the work is confirming that another application can read it. Document what your syntax does that plain Markdown does not. Trap: A force-directed graph of ten thousand nodes renders as a hairball and takes several seconds. Show a neighbourhood at a chosen depth from the current note. It is more useful and it is fast. Done when: The graph renders a neighbourhood at a chosen depth in under a second on the ten thousand note vault. Opening the vault in a general Markdown editor shows readable notes, with only the block identifiers as visible extras. The README documents every syntax extension. Do not build yet: Sync is the product phase and is the hard part. ### Phase 6 · Sync, and being honest about conflicts Build: Sync the vault directory with a file sync tool or a git repository. Detect conflicts and surface them as notes you can resolve. Do not attempt real-time collaborative editing. Trap: File sync tools resolve a simultaneous edit by keeping one version and, at best, leaving the other beside it with a mangled filename. Watch for conflict files, surface them in the application as something to resolve, and never assume the sync tool told the user anything. Done when: A simultaneous edit on two machines produces a visible conflict inside the application rather than a silently lost edit. A vault synced through git can be committed and pulled without the application corrupting the index. The interface states that simultaneous editing of one note is not supported. Do not build yet: Nothing after this. ### Out of scope (and why) - Conflict-free multi-device sync with real-time collaboration. That is the genuinely hard part and a naive version loses work quietly. - The query language. Roam's queries are a substantial feature and reproducing them well is a project of comparable size to everything above. - Mature importers for other tools' formats, each of which is a week of edge cases. - Collaborative editing and sharing, which need a server, accounts and a permission model. ### README must contain - That the Markdown files are the truth and the index is a disposable cache - Every syntax extension, particularly block identifiers, and how to strip them if you leave - That search is tested against a ten thousand note vault, with the measured figure - Which sync mechanism is supported and how conflicts appear - That simultaneous editing of one note is not supported
AGENTS.md
# Agent instructions · Roam Research indie build - Read `README.md` and `BUILD_PLAN.md` before writing code. The stack is fixed: Tauri with a TypeScript front end, One Markdown file per note in a directory you choose, SQLite with full text search, treated as a disposable cache, An incremental Markdown parser that extracts links and blocks, A filesystem watcher with debounce. Do not substitute. - Work one phase at a time, in order. Do not start a phase until every "Done when" item of the previous one passes. - Prefer the fewest moving parts that satisfy the step. No frameworks, services or dependencies the plan does not name. - Secrets live in `.env`, never in source or logs. Keep `.env.example` current when a variable is introduced. - Do not invent cryptography, security guarantees, APIs or compliance claims. - Add a focused test for every destructive, security-sensitive or data-loss path the plan names. - Run the project checks before declaring a phase complete, and record any deliberate shortcut in the README under "Tradeoffs". ## Known traps - Your own writes will trigger your own file watcher and can produce a loop that reloads the note under the user's cursor. Track the hash you last wrote and ignore watcher events matching it, rather than trying to suppress the watcher around writes, which races. - Resolving a link to a note identifier at parse time means every link to a not-yet-created note is either an error or a silently created empty file. Store the link target as text and resolve when rendering, which also makes renaming a note a cheap operation. - Test at the size you intend to reach, not the size you have. A vault of two hundred notes is fast with any approach, including a naive grep, and the design that fails at ten thousand feels perfect until the day it does not. Generate ten thousand notes and search them before you commit to the approach. - A block identifier derived from position changes every time a line is inserted above it, silently repointing every reference. Write a generated identifier into the file the first time a block is referenced, and never derive identity from position or content. - A force-directed graph of ten thousand nodes renders as a hairball and takes several seconds. Show a neighbourhood at a chosen depth from the current note. It is more useful and it is fast.
BUILD_PLAN.md
# Build plan · Roam Research A desktop notes application whose storage is a directory of Markdown files you can read without it: daily notes for capture, wiki-style links that build a backlink index, block references, full text search that stays instant at ten thousand notes, and an export that is simply the folder you already have. Phases are in dependency order. Each ends in a "Done when" list; treat an unticked item as a blocker, not a note. ## Phase 1 · Files, watcher and the daily note Capture is instant, and everything you type lands in a file you could open in any editor. ### Steps 1. Set up Tauri and pick a vault A directory chooser stored in settings. Everything else is relative to it. Files: `src-tauri/`, `src/vault.ts` ```sh npm create tauri-app@latest ``` 2. Write with debounce and atomic replace Debounce keystrokes, then write to a temporary file and rename over the target. A partial write during a crash is how people lose a day of notes. Files: `src/storage.ts` 3. Watch without looping Record the hash of every write. Ignore watcher events whose content hash matches what you just wrote. Files: `src/watcher.ts` ### Done when - [ ] Text typed and immediately quit is present on disk - [ ] An external edit appears within a second - [ ] The application never reloads in response to its own write - [ ] A crash mid-write leaves either the old file or the new one, never a truncated one ### Watch out - Your own writes will trigger your own file watcher and can produce a loop that reloads the note under the user's cursor. Track the hash you last wrote and ignore watcher events matching it, rather than trying to suppress the watcher around writes, which races. ## Phase 2 · Links and backlinks Typing a link creates a connection, and the target note shows what points at it. ### Steps 1. Parse links and tags incrementally On save, reparse only the changed note and replace its rows in the links table. Full vault reparse belongs to index rebuild, not to editing. Files: `src/parse.ts` 2. Store link targets as text Resolve to a note at render time. This is what makes links to future notes work and renames cheap. Files: `src/links.ts` 3. Build the backlinks panel Group by source note, show the referring block plus its parent for context. A bare list of note titles is much less useful than it looks. Files: `src/backlinks.ts` ### Done when - [ ] A link to a non-existent note renders and creates it when followed - [ ] Backlinks show the referring block with surrounding context - [ ] Renaming a note does not require rewriting files that link to it - [ ] A note linked from fifty places lists all fifty ### Watch out - Resolving a link to a note identifier at parse time means every link to a not-yet-created note is either an error or a silently created empty file. Store the link target as text and resolve when rendering, which also makes renaming a note a cheap operation. ## Phase 3 · Search that stays instant Search is fast enough to be the primary way you navigate, at ten thousand notes. ### Steps 1. Generate a realistic vault for testing Ten thousand notes with plausible link density. This fixture is what makes every later performance claim meaningful. Files: `scripts/generate-vault.ts` 2. Index incrementally on save Replace only the changed note's rows. A full reindex on every keystroke is the obvious implementation and the wrong one. Files: `src/index.ts` 3. Make rebuild a first-class action A menu item that deletes and rebuilds. Run it regularly during development so index corruption is never frightening. Files: `src/index.ts` ### Done when - [ ] Search over ten thousand notes returns in well under a hundred milliseconds - [ ] Deleting the index and relaunching rebuilds it with identical results - [ ] An externally edited note is searchable within a second - [ ] Indexing a single note edit does not reindex the vault ### Watch out - Test at the size you intend to reach, not the size you have. A vault of two hundred notes is fast with any approach, including a naive grep, and the design that fails at ten thousand feels perfect until the day it does not. Generate ten thousand notes and search them before you commit to the approach. ## Phase 4 · Block references and transclusion You can point at one paragraph rather than a whole note, and see it in place. ### Steps 1. Assign identifiers on first reference Generate an identifier and append it to the source line in a form that reads acceptably in a plain text editor. Do not identify every block eagerly; most are never referenced. Files: `src/blocks.ts` 2. Render references inline Look up by identifier and render the block's text with an affordance to jump to its source. Files: `src/transclude.ts` 3. Handle the broken case visibly A reference to a deleted block shows as broken, naming the identifier. Silence here produces documents with holes in them. Files: `src/transclude.ts` ### Done when - [ ] Inserting text above a referenced block does not break the reference - [ ] Editing the source block updates every rendered reference - [ ] A deleted source block leaves a visible broken reference - [ ] Identifiers are readable in a plain text editor without being obtrusive ### Watch out - A block identifier derived from position changes every time a line is inserted above it, silently repointing every reference. Write a generated identifier into the file the first time a block is referenced, and never derive identity from position or content. ## Phase 5 · Graph, export and the honest limits You can see the shape of the vault, and leaving costs nothing. ### Steps 1. Render a neighbourhood graph Nodes within a chosen link distance of the current note, not the entire vault. Files: `src/graph.ts` 2. Verify the vault reads elsewhere Open it in a different Markdown editor and read several notes. This is the test that local-first is real rather than a slogan. 3. Document the syntax extensions Block identifiers and any non-standard link syntax, so someone leaving knows exactly what to strip. Files: `README.md` ### Done when - [ ] The neighbourhood graph renders in under a second on the large vault - [ ] The vault is readable in a different Markdown editor - [ ] Every syntax extension is documented - [ ] No note requires this application to be understood ### Watch out - A force-directed graph of ten thousand nodes renders as a hairball and takes several seconds. Show a neighbourhood at a chosen depth from the current note. It is more useful and it is fast. ## Not in this build - Conflict-free sync and real-time collaboration. - The query language, which is a substantial feature in its own right. - Mature importers from other notes tools. - Sharing, publishing and collaborative editing. ## After v1, if you want it - A simple query syntax over the links table, starting with all notes linking to two given notes - Incremental parsing on keystroke rather than on save, once the save path is proven - A publish command that renders a subset of the vault as static HTML
.env.example
# Copy to .env and fill in. Never commit .env; this file documents it. # Required. The directory of Markdown files. This is your data and it belongs in your backup. VAULT_PATH=~/Notes # Required. The search index. Deliberately outside the vault so syncing cannot corrupt it. Safe to delete. INDEX_PATH=~/.local/share/notes/index.db # Required. How long to wait after typing before writing. Lower loses less on a crash and writes more often. WRITE_DEBOUNCE_MS=400 # Required. How many link hops the graph view shows from the current note. GRAPH_DEPTH=2 # Required. Filename format for daily notes. Changing it later does not rename existing notes. DAILY_NOTE_FORMAT=YYYY-MM-DD
PRODUCT.md
# Roam Research · product brief ## Problem Roam Research's solo core is compact: build a local-first networked notes app with fast capture, full-text search, backlinks, tags, and Markdown export. A competent builder can reach a useful personal version in one sitting, while the paid product mainly wins on polish, sync, importers. ## Product outcome A notes application whose data is a folder of Markdown you could read in any editor, with backlinks and block references that survive editing, search that is instant at ten thousand notes, and conflicts that are visible rather than silent. ## Target user A builder who needs a maintainable product foundation, not a one-off demo. ## Required capabilities - desktop development machine - Node.js 22 and Rust toolchain - local filesystem access ## Explicit non-goals for v1 - Conflict-free sync and real-time collaboration. - The query language, which is a substantial feature in its own right. - Mature importers from other notes tools. - Sharing, publishing and collaborative editing. - conflict-safe multi-device sync - mature importers and exporters - collaborative editing and sharing - native mobile clients ## Success criteria - Search latency on the ten thousand note fixture is measured and has not regressed - Index rebuild produces identical results to the incremental index - The vault opens and reads correctly in a different Markdown editor - A block reference survives text inserted above its source - A simulated sync conflict is visible in the interface
BRIEF.md
# Build brief · Roam Research The one-shot brief this plan expands. `BUILD_PLAN.md` (or `MILESTONES.md`) is the same sequence broken into steps and checks; where the two disagree, the plan wins. Build me a local-first networked notes application with fast capture, backlinks and instant search over plain Markdown files to replace Roam Research. Build it in phases, in the order below. Do not write the whole thing in one pass. Finish a phase, run its "Done when" check, fix what fails, and only then start the next phase. ### Stack (fixed, do not substitute) - Shell: Tauri with a TypeScript front end · a native file watcher and real filesystem access, in a binary a fraction of the size of the alternative - Storage: One Markdown file per note in a directory you choose · plain files mean the notes outlive the application, which is the entire point of local-first - Index: SQLite with full text search, treated as a disposable cache · the files are the truth and the index is rebuildable, so a corrupt index is an inconvenience rather than data loss - Parsing: An incremental Markdown parser that extracts links and blocks · reparsing the whole vault on every keystroke does not scale past a few hundred notes - Watching: A filesystem watcher with debounce · edits made in another editor must show up, because plain files invite exactly that ### Data model (create this before Phase 1) - Files are the source of truth. Everything below is an index that can be deleted and rebuilt. - notes(id, path, title, mtime, size, content_hash) - blocks(id, note_id, ordinal, depth, text, block_uid) · block_uid is a stable identifier written into the file so references survive edits - links(from_note_id, to_title, from_block_id, kind) · kind is wikilink, tag or block_ref; to_title is resolved lazily so a link to a note that does not exist yet still works - notes_fts · a full text index over block text - A link to a non-existent note is normal and useful, not an error. Resolving it late is what makes capture fast. ### Phase 1 · Files, watcher and the daily note Build: Choose a vault directory. Create today's daily note on launch. Write changes to disk with a debounce. Watch the directory so a file edited elsewhere updates the view. No index yet, no links yet: just fast, reliable capture into plain files. Trap: Your own writes will trigger your own file watcher and can produce a loop that reloads the note under the user's cursor. Track the hash you last wrote and ignore watcher events matching it, rather than trying to suppress the watcher around writes, which races. Done when: Typing into the daily note and quitting immediately leaves the text on disk. Editing the same file in another editor updates the view within a second without losing the cursor position. The application never reloads a note in response to its own write. Do not build yet: No links, no search, no backlinks. Capture that does not lose text. ### Phase 2 · Links and backlinks Build: Parse wiki-style links and tags out of each note into the links table. Show a backlinks panel listing the blocks that reference the current note, with enough surrounding text to be readable. Links to notes that do not exist yet are valid and create the note when followed. Trap: Resolving a link to a note identifier at parse time means every link to a not-yet-created note is either an error or a silently created empty file. Store the link target as text and resolve when rendering, which also makes renaming a note a cheap operation. Done when: Typing a link to a note that does not exist renders as a link, and following it creates the note. The target note's backlinks panel shows the referring block with surrounding context. Renaming a note updates what its backlinks display without rewriting every file that links to it. Do not build yet: No search and no block references yet. ### Phase 3 · Search that stays instant Build: Index block text into SQLite full text search, updating only changed notes on save. Search as you type with results grouped by note. Rebuild the whole index from the files on demand and prove that the index is genuinely disposable. Trap: Test at the size you intend to reach, not the size you have. A vault of two hundred notes is fast with any approach, including a naive grep, and the design that fails at ten thousand feels perfect until the day it does not. Generate ten thousand notes and search them before you commit to the approach. Done when: Search across ten thousand generated notes returns results in well under a hundred milliseconds as you type. Deleting the index file and relaunching rebuilds it from the Markdown files with identical results. A note edited externally is searchable within a second. Do not build yet: No block references yet. ### Phase 4 · Block references and transclusion Build: Give blocks stable identifiers written into the file as an unobtrusive suffix. A block reference renders the referenced block inline where it is written. Editing the source block updates every place it appears. Trap: A block identifier derived from position changes every time a line is inserted above it, silently repointing every reference. Write a generated identifier into the file the first time a block is referenced, and never derive identity from position or content. Done when: Referencing a block and then inserting three paragraphs above it in the source note leaves the reference pointing at the same text. Editing the source block updates the rendered reference. Deleting the source block leaves a visible broken reference rather than an empty space. Do not build yet: No graph view or export yet. ### Phase 5 · Graph, export and the honest limits Build: A graph view of notes and links, filtered to a neighbourhood rather than rendering everything. Export is the vault directory itself, which is the point, so the work is confirming that another application can read it. Document what your syntax does that plain Markdown does not. Trap: A force-directed graph of ten thousand nodes renders as a hairball and takes several seconds. Show a neighbourhood at a chosen depth from the current note. It is more useful and it is fast. Done when: The graph renders a neighbourhood at a chosen depth in under a second on the ten thousand note vault. Opening the vault in a general Markdown editor shows readable notes, with only the block identifiers as visible extras. The README documents every syntax extension. Do not build yet: Sync is the product phase and is the hard part. ### Phase 6 · Sync, and being honest about conflicts Build: Sync the vault directory with a file sync tool or a git repository. Detect conflicts and surface them as notes you can resolve. Do not attempt real-time collaborative editing. Trap: File sync tools resolve a simultaneous edit by keeping one version and, at best, leaving the other beside it with a mangled filename. Watch for conflict files, surface them in the application as something to resolve, and never assume the sync tool told the user anything. Done when: A simultaneous edit on two machines produces a visible conflict inside the application rather than a silently lost edit. A vault synced through git can be committed and pulled without the application corrupting the index. The interface states that simultaneous editing of one note is not supported. Do not build yet: Nothing after this. ### Out of scope (and why) - Conflict-free multi-device sync with real-time collaboration. That is the genuinely hard part and a naive version loses work quietly. - The query language. Roam's queries are a substantial feature and reproducing them well is a project of comparable size to everything above. - Mature importers for other tools' formats, each of which is a week of edge cases. - Collaborative editing and sharing, which need a server, accounts and a permission model. ### README must contain - That the Markdown files are the truth and the index is a disposable cache - Every syntax extension, particularly block identifiers, and how to strip them if you leave - That search is tested against a ten thousand note vault, with the measured figure - Which sync mechanism is supported and how conflicts appear - That simultaneous editing of one note is not supported
ARCHITECTURE.md
# Architecture · Roam Research ## Stack | Part | Choice | Why | | --- | --- | --- | | Shell | Tauri with a TypeScript front end | a native file watcher and real filesystem access, in a binary a fraction of the size of the alternative | | Storage | One Markdown file per note in a directory you choose | plain files mean the notes outlive the application, which is the entire point of local-first | | Index | SQLite with full text search, treated as a disposable cache | the files are the truth and the index is rebuildable, so a corrupt index is an inconvenience rather than data loss | | Parsing | An incremental Markdown parser that extracts links and blocks | reparsing the whole vault on every keystroke does not scale past a few hundred notes | | Watching | A filesystem watcher with debounce | edits made in another editor must show up, because plain files invite exactly that | ## Modules Each module has one owner concern and a documented way to replace it. | Module | Owns | How to replace it | | --- | --- | --- | | Storage | Reading and writing Markdown files atomically. | Not swappable without abandoning the local-first claim. Everything else is built to be rebuildable from here. | | Index | Search, links and backlinks as a cache. | Entirely replaceable, and the rebuild action is what proves it. If a swap would lose data, the boundary has been drawn wrong. | | Parser | Extracting links, tags and blocks from Markdown. | A different parser is fine provided identifiers stay stable, because references are written into the files. | | Watcher | Noticing external edits without looping. | Platform-specific behind one interface. The hash-based loop prevention must survive any change. | | Sync | Moving the vault between machines and surfacing conflicts. | Git or a file sync drive. What must not change is that the index stays outside the synced directory. | ## Configuration Every runtime setting is an environment variable documented in `.env.example`, validated at startup, with a safe local default wherever one exists. - `VAULT_PATH` · required · The directory of Markdown files. This is your data and it belongs in your backup. - `INDEX_PATH` · required · The search index. Deliberately outside the vault so syncing cannot corrupt it. Safe to delete. - `WRITE_DEBOUNCE_MS` · required · How long to wait after typing before writing. Lower loses less on a crash and writes more often. - `GRAPH_DEPTH` · required · How many link hops the graph view shows from the current note. - `DAILY_NOTE_FORMAT` · required · Filename format for daily notes. Changing it later does not rename existing notes. ## Production baseline - Security: least privilege, input validation at every boundary, secret redaction in logs, rate limits on abuse-prone paths, no invented security primitives. - Data: explicit schema and migrations, transactional writes where integrity matters, backup and restore procedures that have been exercised. - Integrations: adapters around third-party providers, idempotent webhook or job processing, bounded retries, timeouts. - Observability: structured logs with request or operation ids, an error-tracking hook, and health and readiness checks where a server exists. - Quality: unit tests for domain rules, integration tests at module boundaries, one end-to-end test of the critical path. ## Decision records For each dependency in the stack table, keep a short note: why it was chosen, its failure mode, and how it is replaced. Do not add infrastructure until a requirement in `PRODUCT.md` justifies it.
AGENTS.md
# Agent instructions · Roam Research product build - Read `PRODUCT.md` and `ARCHITECTURE.md` before changing code. The stack is fixed: Tauri with a TypeScript front end, One Markdown file per note in a directory you choose, SQLite with full text search, treated as a disposable cache, An incremental Markdown parser that extracts links and blocks, A filesystem watcher with debounce. - Implement milestone by milestone from `MILESTONES.md`; keep each change reviewable and leave the application runnable at every commit. - Treat authentication, payments, encryption, imports, webhooks and destructive actions as high-risk boundaries when present. - Never invent cryptography or silently weaken a requirement to make a check pass. - Put every external service behind an interface with a deterministic fake for tests. - Add migrations and rollback or recovery notes for every persistent data change. - Log useful operational context without credentials, tokens, passwords or personal data. - Update documentation and run every check before completing a milestone. ## Known traps - Your own writes will trigger your own file watcher and can produce a loop that reloads the note under the user's cursor. Track the hash you last wrote and ignore watcher events matching it, rather than trying to suppress the watcher around writes, which races. - Resolving a link to a note identifier at parse time means every link to a not-yet-created note is either an error or a silently created empty file. Store the link target as text and resolve when rendering, which also makes renaming a note a cheap operation. - Test at the size you intend to reach, not the size you have. A vault of two hundred notes is fast with any approach, including a naive grep, and the design that fails at ten thousand feels perfect until the day it does not. Generate ten thousand notes and search them before you commit to the approach. - A block identifier derived from position changes every time a line is inserted above it, silently repointing every reference. Write a generated identifier into the file the first time a block is referenced, and never derive identity from position or content. - A force-directed graph of ten thousand nodes renders as a hairball and takes several seconds. Show a neighbourhood at a chosen depth from the current note. It is more useful and it is fast. - File sync tools resolve a simultaneous edit by keeping one version and, at best, leaving the other beside it with a mangled filename. Watch for conflict files, surface them in the application as something to resolve, and never assume the sync tool told the user anything.
MILESTONES.md
# Delivery milestones · Roam Research Estimated effort: **weekend** for the indie phases; the production-only milestones add the trust and operability layer. ## M1 · Files, watcher and the daily note Capture is instant, and everything you type lands in a file you could open in any editor. ### Steps 1. Set up Tauri and pick a vault A directory chooser stored in settings. Everything else is relative to it. Files: `src-tauri/`, `src/vault.ts` ```sh npm create tauri-app@latest ``` 2. Write with debounce and atomic replace Debounce keystrokes, then write to a temporary file and rename over the target. A partial write during a crash is how people lose a day of notes. Files: `src/storage.ts` 3. Watch without looping Record the hash of every write. Ignore watcher events whose content hash matches what you just wrote. Files: `src/watcher.ts` ### Done when - [ ] Text typed and immediately quit is present on disk - [ ] An external edit appears within a second - [ ] The application never reloads in response to its own write - [ ] A crash mid-write leaves either the old file or the new one, never a truncated one ### Watch out - Your own writes will trigger your own file watcher and can produce a loop that reloads the note under the user's cursor. Track the hash you last wrote and ignore watcher events matching it, rather than trying to suppress the watcher around writes, which races. ## M2 · Links and backlinks Typing a link creates a connection, and the target note shows what points at it. ### Steps 1. Parse links and tags incrementally On save, reparse only the changed note and replace its rows in the links table. Full vault reparse belongs to index rebuild, not to editing. Files: `src/parse.ts` 2. Store link targets as text Resolve to a note at render time. This is what makes links to future notes work and renames cheap. Files: `src/links.ts` 3. Build the backlinks panel Group by source note, show the referring block plus its parent for context. A bare list of note titles is much less useful than it looks. Files: `src/backlinks.ts` ### Done when - [ ] A link to a non-existent note renders and creates it when followed - [ ] Backlinks show the referring block with surrounding context - [ ] Renaming a note does not require rewriting files that link to it - [ ] A note linked from fifty places lists all fifty ### Watch out - Resolving a link to a note identifier at parse time means every link to a not-yet-created note is either an error or a silently created empty file. Store the link target as text and resolve when rendering, which also makes renaming a note a cheap operation. ## M3 · Search that stays instant Search is fast enough to be the primary way you navigate, at ten thousand notes. ### Steps 1. Generate a realistic vault for testing Ten thousand notes with plausible link density. This fixture is what makes every later performance claim meaningful. Files: `scripts/generate-vault.ts` 2. Index incrementally on save Replace only the changed note's rows. A full reindex on every keystroke is the obvious implementation and the wrong one. Files: `src/index.ts` 3. Make rebuild a first-class action A menu item that deletes and rebuilds. Run it regularly during development so index corruption is never frightening. Files: `src/index.ts` ### Done when - [ ] Search over ten thousand notes returns in well under a hundred milliseconds - [ ] Deleting the index and relaunching rebuilds it with identical results - [ ] An externally edited note is searchable within a second - [ ] Indexing a single note edit does not reindex the vault ### Watch out - Test at the size you intend to reach, not the size you have. A vault of two hundred notes is fast with any approach, including a naive grep, and the design that fails at ten thousand feels perfect until the day it does not. Generate ten thousand notes and search them before you commit to the approach. ## M4 · Block references and transclusion You can point at one paragraph rather than a whole note, and see it in place. ### Steps 1. Assign identifiers on first reference Generate an identifier and append it to the source line in a form that reads acceptably in a plain text editor. Do not identify every block eagerly; most are never referenced. Files: `src/blocks.ts` 2. Render references inline Look up by identifier and render the block's text with an affordance to jump to its source. Files: `src/transclude.ts` 3. Handle the broken case visibly A reference to a deleted block shows as broken, naming the identifier. Silence here produces documents with holes in them. Files: `src/transclude.ts` ### Done when - [ ] Inserting text above a referenced block does not break the reference - [ ] Editing the source block updates every rendered reference - [ ] A deleted source block leaves a visible broken reference - [ ] Identifiers are readable in a plain text editor without being obtrusive ### Watch out - A block identifier derived from position changes every time a line is inserted above it, silently repointing every reference. Write a generated identifier into the file the first time a block is referenced, and never derive identity from position or content. ## M5 · Graph, export and the honest limits You can see the shape of the vault, and leaving costs nothing. ### Steps 1. Render a neighbourhood graph Nodes within a chosen link distance of the current note, not the entire vault. Files: `src/graph.ts` 2. Verify the vault reads elsewhere Open it in a different Markdown editor and read several notes. This is the test that local-first is real rather than a slogan. 3. Document the syntax extensions Block identifiers and any non-standard link syntax, so someone leaving knows exactly what to strip. Files: `README.md` ### Done when - [ ] The neighbourhood graph renders in under a second on the large vault - [ ] The vault is readable in a different Markdown editor - [ ] Every syntax extension is documented - [ ] No note requires this application to be understood ### Watch out - A force-directed graph of ten thousand nodes renders as a hairball and takes several seconds. Show a neighbourhood at a chosen depth from the current note. It is more useful and it is fast. ## M6 · Sync, and being honest about conflicts (production only) Two machines can edit the vault, and a conflict is visible rather than silent. ### Steps 1. Choose and document the sync mechanism Git gives history and explicit conflicts; a file sync drive is simpler and hides them. Pick one and say which in the README. Files: `README.md` 2. Detect and surface conflict files Watch for the naming patterns sync tools produce and list them in the application as notes needing attention. Files: `src/conflicts.ts` 3. Make the index resilient to sync The index must never be inside the synced directory, or two machines will fight over it and corrupt it. Files: `src/index.ts` ### Done when - [ ] A simultaneous edit surfaces as a visible conflict - [ ] The index lives outside the synced directory - [ ] A git-synced vault survives commit and pull without index corruption - [ ] The single-editor limitation is stated in the interface ### Watch out - File sync tools resolve a simultaneous edit by keeping one version and, at best, leaving the other beside it with a mangled filename. Watch for conflict files, surface them in the application as something to resolve, and never assume the sync tool told the user anything.
OPERATIONS.md
# Operations · Roam Research ## Backup The vault directory is everything. Ordinary file backup covers it, which is the practical benefit of plain files. The index is explicitly not worth backing up. ## Restore Copy the vault back and launch. The index rebuilds. If a restore requires anything else, something has leaked out of the vault that should not have. Do a restore drill before the first real user, and write the date here when it passes. ## Monitoring Nothing runs as a service. What is worth watching during development is index rebuild time and search latency against the ten thousand note fixture, tracked over releases so a regression is caught. ## Incident checklist The realistic incidents are a sync conflict and a corrupt index. The first is surfaced in Phase 6, the second is fixed by the rebuild action, which is why rebuild is a menu item rather than a hidden repair path. 1. Contain the issue without destroying evidence or user data. 2. Record the timeline and affected scope. 3. Rotate exposed secrets and revoke compromised sessions or credentials. 4. Restore from a verified backup when needed. 5. Document the root cause, the remediation and the regression test. ## Release gate - [ ] Search latency on the ten thousand note fixture is measured and has not regressed - [ ] Index rebuild produces identical results to the incremental index - [ ] The vault opens and reads correctly in a different Markdown editor - [ ] A block reference survives text inserted above its source - [ ] A simulated sync conflict is visible in the interface ## Launch constraint Do not market omitted Roam Research capabilities as implemented. The non-goals in `PRODUCT.md` remain user-visible limitations until they are deliberately delivered.
.env.example
# Copy to .env and fill in. Never commit .env; this file documents it. # Required. The directory of Markdown files. This is your data and it belongs in your backup. VAULT_PATH=~/Notes # Required. The search index. Deliberately outside the vault so syncing cannot corrupt it. Safe to delete. INDEX_PATH=~/.local/share/notes/index.db # Required. How long to wait after typing before writing. Lower loses less on a crash and writes more often. WRITE_DEBOUNCE_MS=400 # Required. How many link hops the graph view shows from the current note. GRAPH_DEPTH=2 # Required. Filename format for daily notes. Changing it later does not rename existing notes. DAILY_NOTE_FORMAT=YYYY-MM-DD
$ choose a build depth, inspect the files, then open the complete pack in your agent
Roam Research: People pay for a writing surface they trust for years, plus migration tools, sync, and tiny interaction details that disappear into habit.
xconflict-safe multi-device sync
xmature importers and exporters
xcollaborative editing and sharing
xnative mobile clients
share on X ↗"I just replaced Roam Research ($15/mo) with one prompt"
Don't feel like building it? These folks already made it free.
SiYuanBlock references, backlinks and outlines without making the outliner your entire personality.open source↗
LogseqRoam's block graph, minus the monthly tribute and plus a folder you can inspect.open source↗
ObsidianBidirectional links and block references in plain files; the outline comes from plugins.free↗all 3 free alternatives to Roam Research →· no votes, no pay-to-list · just what's real
Vibecode Roam Research
Yes. A competent AI coding agent (Claude Code, Codex, Cursor) can build a usable personal Roam Research replacement in one session with the prompt on this page. It runs on your own machine or server with no subscription.
How much does Roam Research cost?
Roam Research costs about $15/month (Pro, checked 2026-07-31), which is $180 per year. That's what you save by replacing it with one prompt.
What do I lose by replacing Roam Research?
Honestly: conflict-safe multi-device sync; mature importers and exporters; collaborative editing and sharing; native mobile clients. If any of those are load-bearing for you, keep paying.
Is there an open-source alternative to Roam Research?
Yes: SiYuan (Block references, backlinks and outlines without making the outliner your entire personality.) Logseq (Roam's block graph, minus the monthly tribute and plus a folder you can inspect.) Obsidian (Bidirectional links and block references in plain files; the outline comes from plugins.) All 3 curated free alternatives are at vibecodeit.com/roam-research/alternatives. The prompt is for when you want it exactly your way.