Build Fabric

0%0 of 17 items done

Saved on this device only. Tick prerequisites first, then work the phases in order · do not start one until the checks above it pass.

A local file workspace whose hard problems are keeping the originals untouched so the index is always disposable, getting text out of formats that resist it, and making search find the thing you half remember rather than returning nothing.

estimated effort multi-daythe files for this build are in the project pack

ShellTauri 2 with React and TypeScriptIndexSQLite with FTS5, stored beside the library but deletableOriginalsLeft in place, never moved or renamed by the appExtractionOne module per format, run in a worker, failures recorded not swallowedEmbeddingsOptional, local or key-provided, added only after keyword search works

Before step 1

Everything below is assumed from the first step. Tick each one when you actually have it, not when you plan to.

installfree
Why Tauri needs both, and the extraction workers live on the Rust side where the time and memory ceilings are enforceable.
Get it rustup.rs for Rust, nodejs.org or nvm for Node. open ↗
Verify cargo --version and node --version both print, and a Tauri dev build launches
have readyfree
Why Every hard part here is a format question, and a folder of sample text files will pass tests that your actual PDFs, scans and exports fail.
Get it Point the app at a folder you genuinely use, on a copy first if you are nervous.
Verify The scan finds at least a few hundred files across at least four formats
have readyfree
Why The empty-extraction case is the single most common silent failure in this category and it needs a real fixture rather than a synthetic one.
Get it Scan a page, or find one in your own archive.
Verify Extraction yields zero characters and the file appears in the failure list
have readyfree
Why Scan, rebuild and search times are the claims this build makes, and they are only credible measured at a size that hurts.
Get it Your real library if it is large enough, otherwise generate one from public documents.
Verify A full rebuild completes and the time is recorded
decidefree
Why It decides whether search can match inside a word and what the minimum query length is, and changing it later means a full reindex.
Get it Look at what you actually store, read the tokenizer documentation, and write down the choice with the reason. open ↗
Verify The choice is in the schema and the README and is covered by a substring test

Data model

Create these before the first phase that stores anything. Changing a table later is the expensive kind of change.

- files(id, path, size, mtime, content_sha256, first_seen_at, last_seen_at, missing_since) · path is a reference, never a copy
- documents(file_id, kind, text, extracted_at, extractor, extractor_version, char_count) · the text pulled out, with what pulled it
- extraction_failures(file_id, at, extractor, reason, retryable) · visible in the interface, not only in a log
- fts(document rowid, text) · an FTS5 table whose tokenizer choice is recorded in the schema and in the README
- notes(id, file_id, body, created_at, updated_at) · your own writing, which is the one thing here that is not derived
- tags(file_id, tag, source, confidence) · source is manual or suggested, and a suggested tag never becomes manual by itself
- embeddings(document_id, model, dim, vector) · present only if you enabled them, and rebuildable like everything else

Environment variables

These go in a .env file the app reads at startup. The pack's .env.example is this table as a file · copy it, never commit the filled-in version.

Variable	Needed	Example	Where the value comes from
`LIBRARY_PATHS`	required	`/home/you/Documents,/home/you/Archive`	Watched folders. Read only as far as this app is concerned.
`INDEX_PATH`	required	`~/.local/share/fabric/index.db`	Derived data. Deleting it should cost only rebuild time.
`EXTRACT_TIMEOUT_SECONDS`	required	`60`	Per file. Exceeding it records a failure rather than blocking the queue.
`EXTRACT_MEMORY_MB`	required	`512`	Per worker. A malformed file should not take the app down.
`FTS_TOKENIZER`	required	`trigram`	Recorded here and in the README because it decides what search can find.
`EMBEDDINGS_ENABLED`	required	`false`	Off by default. The app is fully usable without it.
`EMBEDDINGS_API_KEY`secret	optional	`sk-...redacted`	Only if you enabled a hosted model. Never required for keyword search.
`EMBEDDINGS_COST_CEILING`	optional	`5.00`	Maximum estimated spend for one indexing pass before it refuses to start.

The build, in order

1
Files stay files
The library is a folder you already understand, and the index can be deleted without losing anything.
1. A hash survives a rename; a path does not.
  Files src-tauri/src/index/scan.rssrc/db/schema.sql
2. It is the test that the index is really disposable.
  Files src-tauri/src/index/rebuild.rstests/rebuild.test.ts
3. An unplugged drive is not a decision to delete your notes.
  Files src-tauri/src/index/scan.rs
done when · tick each as it passes
watch out
- An app that takes ownership of files by copying them into its own store makes the store the only real copy, and then a bug in the store is a bug in your documents. Keep the originals where the user put them, treat the database as a cache, and add a rebuild command in this phase so the claim is tested from the start.
next phase
2
Getting the text out, and admitting when you cannot
Every file either has extracted text or a visible reason why it does not.
1. Silently indexing nothing is the worst available outcome.
  Files src-tauri/src/extract/mod.rstests/empty-extraction.test.ts
2. One malformed file should not stop the queue.
  Files src-tauri/src/extract/worker.rs
3. So a fixed extractor can requeue exactly its own failures.
  Files src/db/schema.sql
done when · tick each as it passes
watch out
- A scanned PDF with no text layer extracts to an empty string, and unless you treat that as a failure it silently becomes a file that search will never find, which the user discovers months later while looking for the one document they needed. Treat a zero-length extraction from a format that should have text as a failure with a stated reason, and offer optical recognition as an explicit, opt-in, per-file action.
next phase
3
Search that finds the thing you half remember
Keyword search returns the right document for partial words, and the tokenizer choice is deliberate.
1. This single decision determines what search can ever find.
  Files src/db/schema.sqldocs/search.md
2. An empty list reads as a missing document.
  Files src/search/query.tstests/short-query.test.ts
3. Ten documents tells you nothing about ten thousand.
  Files tests/search-perf.test.ts
done when · tick each as it passes
watch out
- Accepting the default tokenizer and then wondering why searching for part of a product code finds nothing is the most common search complaint in an app like this, and the second is a trigram index that returns nothing for a two-character query without saying why. Pick the tokenizer for the content you actually store, and when a query is shorter than the tokenizer's floor, say so in the interface instead of showing zero results.
next phase
4
Suggestions that never move anything
Automatic tags help you find things and never reorganise your library behind your back.
1. The distinction is what stops guesses becoming facts.
  Files src/db/schema.sqlsrc/tags/suggest.ts
2. A second ranker is useful; a replacement is unpredictable.
  Files src/search/rerank.ts
3. A library-wide embedding pass is a real invoice.
  Files src/embeddings/estimate.ts
done when · tick each as it passes
watch out
- An app that files documents into folders on its own guess is delightful for a week and then you cannot find a contract because it decided the contract was a receipt, and by then you have stopped remembering where you put things. Tags are additive metadata, never a move, and a suggested tag stays labelled as suggested until you accept it.
5
Leaving, rebuilding and surviving a big libraryproduct builder
Everything you added is exportable, the index is genuinely disposable, and the app is usable at real size.
1. Without the hash the export is a pile of loose text.
  Files src/export/markdown.tstests/export-roundtrip.test.ts
2. An improved extractor should not mean a full reindex.
  Files src-tauri/src/extract/requeue.rs
3. Every claim about speed here is a number or it is nothing.
  Files tests/scale.test.tsREADME.md
done when · tick each as it passes
watch out
- Exporting notes without recording which file each one belonged to produces a folder of orphaned text that is worse than no export, because it looks like a migration path until you try to use it. Put the original path and the content hash in the exported front matter, and write an import that reattaches notes by hash so a rebuilt library keeps them.

what this build does not replace

Multi-device sync.
Collaborative editing and sharing.
Native mobile clients.
Automatic filing that moves files.
collaborative editing and sharing
native mobile clients
conflict-safe multi-device sync
mature importers and exporters

after v1, if you want it

Optical character recognition as an opt-in action on the files already in the failure list
Saved searches, which need nothing new in the index and a small amount in the interface
A second library root with its own index, once rebuild times at fifty thousand files are known

Need the files? The project pack on the verdict page hands your agent the whole brief · more notes & knowledge.