Vibecode Afforai
track this build4 phases, 12 steps, beginner friendly0%KINDA · weekend project
The visible document research assistant loop is buildable, but a credible replacement needs more than the first screen. Afforai earns its keep through data, import reliability, so expect a weekend or multi-day build and a narrower personal scope.
the project pack5 files · written for this build, step by step
You are building a lean indie version of Afforai. Create the following project files first, then implement the application by following them. Keep the files updated as decisions change. Do not collapse this into a single README or prompt. ===== README.md ===== # Afforai · indie build A local document research library whose hard problems are getting text out of a PDF with positions you can point at, fetching correct metadata from the scholarly APIs politely, and never producing a citation that cannot be resolved back to a real span on a real page. Estimated effort: **multi-day**. Work `BUILD_PLAN.md` top to bottom · every phase ends in a check that has to pass before the next one starts. ## Stack | Part | Choice | Why | | --- | --- | --- | | Runtime | One local web application with a background worker, no account system | a personal library with logins is a service you now operate | | Extraction | A PDF text extractor that returns page numbers and character positions, with OCR as an explicit fallback | a citation you cannot point at is not a citation | | Metadata | Crossref and other open scholarly APIs, identified politely, with the user's value winning | guessing an author from a filename produces a library of confident errors | | Retrieval | Keyword search and vector search together over structure-aware chunks | one of them finds the exact term and the other finds the paraphrase | | Verification | Every generated claim carries a quote that must match the source text exactly | this is the feature, not a safeguard bolted on afterwards | | Storage | Documents and the index on local disk, with an explicit record of what was sent where | research documents are often things you are not free to upload | ## Before you start Have every one of these ready. The plan assumes them from step one. - [ ] **Node.js 22 or newer** · free - Why: Runs the local application, the worker and the index. - Get it: nodejs.org, or nvm install 22. - Verify: node --version prints v22 or higher - [ ] **An OCR engine for scanned documents** (optional) · free locally - Why: Scanned PDFs are common in older literature and the honest alternative to OCR is refusing the document, which you should still do when OCR is absent. - Get it: Tesseract from your package manager, or a hosted OCR you have a key for. - Verify: A scanned page produces text with page numbers preserved - [ ] **A key for an answer-generating model** (optional) · pence per question - Why: Answers and comparisons need one, and it is also the boundary at which document text leaves the machine. - Get it: Create a key with its own spending limit. - Verify: A short question over one document returns sentences with quotes - [ ] **A test corpus of awkward documents** · free - Why: Two-column papers, scanned reports and a document containing a results table are the three cases every part of this build is judged on. - Get it: Open-access papers plus one deliberately scanned document. - Verify: The corpus contains a two-column paper, a scanned file and a table - [ ] **Your citation, retention and egress rules** · free - Why: Whether an unverified claim is ever shown, how long cached extractions are kept, and what may be sent to a hosted model are decisions the code enforces. - Get it: Write down all three before the first answer is generated. - Verify: Each appears in the configuration and in the README ## Quick start Follow Phase 1 in `BUILD_PLAN.md`; it creates the project. Then copy `.env.example` to `.env` and fill in the values it documents. ## Honest limits This build deliberately does not replace: - Automated publisher logins and paywall access. - Licensed metadata and citation graphs. - Shared team libraries with permissions. - Automatic full-text acquisition from arbitrary sites. - licensed scholarly metadata - publisher-specific import reliability - citation graph scale - team libraries and institutional access If one of those is essential to you, that is the reason to keep paying for Afforai, and the README should say so rather than pretend. ===== BRIEF.md ===== # Build brief · Afforai The one-shot brief this plan expands. `BUILD_PLAN.md` (or `MILESTONES.md`) is the same sequence broken into steps and checks; where the two disagree, the plan wins. Build me a private research library that answers questions about your own documents and shows you the exact passage to replace Afforai. Build it in phases, in the order below. Do not write the whole thing in one pass. Finish a phase, run its "Done when" check, fix what fails, and only then start the next phase. ### Rule zero An answer without a locatable citation is not an answer, it is a guess with formatting. Every sentence the assistant produces carries a document, a page and a quoted span that exists verbatim in the extracted text. The application checks that the span is really there before the sentence is displayed, and drops the sentence when it is not, saying so. This is not politeness: people paste these answers into literature reviews, grant applications and clinical summaries, and a fabricated citation that looks like the real thing survives every reading until someone tries to follow it. The corollary is that a document whose text could not be extracted is marked unreadable rather than indexed as empty. A scanned page that yields no characters silently becomes a document the assistant believes says nothing, and 'the papers do not mention it' is then a lie. ### Stack (fixed, do not substitute) - Runtime: One local web application with a background worker, no account system · a personal library with logins is a service you now operate - Extraction: A PDF text extractor that returns page numbers and character positions, with OCR as an explicit fallback · a citation you cannot point at is not a citation - Metadata: Crossref and other open scholarly APIs, identified politely, with the user's value winning · guessing an author from a filename produces a library of confident errors - Retrieval: Keyword search and vector search together over structure-aware chunks · one of them finds the exact term and the other finds the paraphrase - Verification: Every generated claim carries a quote that must match the source text exactly · this is the feature, not a safeguard bolted on afterwards - Storage: Documents and the index on local disk, with an explicit record of what was sent where · research documents are often things you are not free to upload ### Data model (create this before Phase 1) - documents(id, title, authors_json, year, doi, source_kind, source_ref, sha256, pages, extraction_state, added_at) - pages(document_id, page_number, text, char_count, extraction_method) · extraction_method is native or ocr, and an empty page with neither is a failure rather than a blank - spans(id, document_id, page_number, start_char, end_char, text) · the unit a citation resolves to - chunks(id, document_id, first_span_id, last_span_id, text, heading_path, token_count, embedding_ref) - answers(id, question, created_at, model, prompt_tokens, completion_tokens, cost_cents) - claims(id, answer_id, sentence, document_id, page_number, span_id, quote, verified) · an unverified claim is stored for the record and never shown - egress(id, at, destination, purpose, bytes, document_ids_json) · what left this machine, when, and why ### Phase 1 · Text out of a PDF, with the page and the position Build: Extract per page, keeping character positions so a span can be highlighted later. Detect a page that yields no text and mark the document as needing OCR rather than indexing it empty. Handle two-column layouts so the reading order does not interleave the columns, and normalise ligatures and hyphenation at line ends so a quoted span matches what a person would search for. Trap: A two-column paper extracted in raw layout order interleaves the left and right columns line by line, so every chunk is a sentence from one column followed by an unrelated sentence from the other, and both the search results and the quoted citations are nonsense that reads almost like English. Detect columns, extract per column, and check the result by reading one page of output beside the original. Done when: A two-column paper extracts in reading order, verified by comparing a page of output with the original. A scanned document with no text layer is marked unreadable and does not enter the index, tested. A quoted span resolves back to a page number and a character range that highlights the right text. Hyphenated line breaks and ligatures are normalised so a search for the plain word finds the passage. Do not build yet: No metadata fetching, retrieval, answers or export. ### Phase 2 · Metadata that is correct, fetched politely Build: Look up a DOI against Crossref and put a contact address in the request so you land in their polite pool, which they describe as a separate pool for clients that identify themselves by a mailto in the User-Agent header or as a query parameter. Their published limits from December 2025 are ten requests a second for a single record and three a second for lists in the polite pool, against five and one anonymously, with concurrency limits alongside (checked), so rate-limit yourself and back off on a refusal. Deduplicate by DOI first and content hash second, and let a value the user typed win over a fetched one. Trap: Guessing metadata from the filename gives you a library where half the entries are called 'main' by an author called 'Elsevier', and once those are in your notes and exports they are extremely hard to find again. Fetch by identifier, keep the source of every field, and mark a document as having unverified metadata rather than inventing a plausible title. Done when: A lookup sends a contact address and is rate-limited below the published polite figures, verified in the request log. A refusal backs off and retries rather than hammering, tested against a stub. The same paper added twice by DOI and once by file produces one entry, tested. A field the user edited is never overwritten by a later fetch, tested. Do not build yet: No retrieval, answers or export yet. ### Phase 3 · Retrieval you can inspect before you trust it Build: Chunk on document structure, headings and paragraphs, with each chunk recording its first and last span so it can be resolved to pages. Run keyword search and vector search together and merge the results. Show the retrieved passages with their documents and pages before any answer is generated, and let the person exclude a document from the question. Trap: Chunking at a fixed token count cuts tables and numbered findings in half, so a retrieved passage contains the first two rows of a results table and the answer confidently reports a figure whose units were in the row that got cut. Chunk on structure, keep tables whole or mark them as tables, and show the passage to the person before the answer. Done when: Retrieved passages are shown with document, page and heading before an answer is produced. A query for an exact term present in one paper retrieves that paper, tested where vector search alone misses it. A table is not split across chunk boundaries, tested with a paper containing one. Excluding a document removes it from retrieval for that question, verified. Do not build yet: No generated answers, comparison or export yet. ### Phase 4 · Answers pinned to real spans, and comparisons that can say nothing Build: Ask for an answer as sentences, each with a document identifier and a verbatim quote. Verify each quote by exact match against the page text after normalising whitespace and case, drop any sentence that fails, and show how many were dropped. Add a comparison table with one row per document and a cell that is either a verified quote or an explicit 'not stated'. Trap: A comparison table with no way to say 'not stated' pushes the model to fill every cell, so a paper that never mentions the outcome acquires an opinion about it, and the table is exactly the artefact people screenshot into a slide. Make 'not stated' a first-class cell value, verified the same way as a quote, and count it in the summary. Done when: A sentence whose quote is not present in the cited document is dropped and the drop is reported, tested by injecting a false quote. Every displayed citation opens the page and highlights the span. A comparison cell reads 'not stated' when no verified quote supports it, tested with a paper that does not address the question. The cost of each answer is recorded and shown. Do not build yet: No export, deletion or egress log yet. ### Phase 5 · Export, deletion, and knowing what left the machine Build: Export the library as BibTeX and CSL JSON with notes and verified citations attached, plus the original files. Deleting a document removes its pages, spans, chunks, embeddings and any cached extraction, then proves it. Log every outbound call with its destination, purpose and which documents it touched. Offer a local-only mode that keeps the library usable for search and citation with no external calls at all. Trap: Sending chunk text to a hosted model without recording it leaves a researcher unable to answer the only question their ethics committee or their publisher will ask, which is what left the machine and where it went. Log every outbound call with the document identifiers involved, and make local-only mode a switch rather than an aspiration. Done when: Export produces BibTeX and CSL JSON that a reference manager imports without hand-editing, tested by importing it. A deleted document leaves no text in search, no vector in the index and no file on disk, asserted in a test. Local-only mode answers search and citation queries with no network call, verified by capturing traffic. The egress log shows destination, purpose and documents for every outbound call. Do not build yet: Nothing after this. ### Out of scope (and why) - Automating a login to a publisher to fetch articles behind a paywall, which breaks the terms your institution agreed to and is the fastest way to have your library's access cut off for everyone. - Licensed scholarly metadata and citation graphs at scale, which are commercial datasets and not something a crawler substitutes for. - Shared team libraries with permissions, which need a server, an account model and a decision about whose copy of a document is authoritative. - Automatic full-text acquisition from arbitrary sites, since what you may download differs per publisher and per subscription and is not a thing code can infer. ### README must contain - That every displayed claim carries a verified verbatim quote and that unverifiable claims are dropped and counted - That a document with no extractable text is marked unreadable rather than indexed as empty - How metadata is fetched, that requests identify themselves politely, and that a value you typed always wins - What local-only mode does, and where the egress log records everything that left the machine - How to export the library into a reference manager, and what deleting a document actually removes ===== AGENTS.md ===== # Agent instructions · Afforai indie build - Read `README.md` and `BUILD_PLAN.md` before writing code. The stack is fixed: One local web application with a background worker, no account system, A PDF text extractor that returns page numbers and character positions, with OCR as an explicit fallback, Crossref and other open scholarly APIs, identified politely, with the user's value winning, Keyword search and vector search together over structure-aware chunks, Every generated claim carries a quote that must match the source text exactly, Documents and the index on local disk, with an explicit record of what was sent where. Do not substitute. - Work one phase at a time, in order. Do not start a phase until every "Done when" item of the previous one passes. - Prefer the fewest moving parts that satisfy the step. No frameworks, services or dependencies the plan does not name. - Secrets live in `.env`, never in source or logs. Keep `.env.example` current when a variable is introduced. - Do not invent cryptography, security guarantees, APIs or compliance claims. - Add a focused test for every destructive, security-sensitive or data-loss path the plan names. - Run the project checks before declaring a phase complete, and record any deliberate shortcut in the README under "Tradeoffs". ## Known traps - A two-column paper extracted in raw layout order interleaves the left and right columns line by line, so every chunk is a sentence from one column followed by an unrelated sentence from the other, and both the search results and the quoted citations are nonsense that reads almost like English. Detect columns, extract per column, and check the result by reading one page of output beside the original. - Guessing metadata from the filename gives you a library where half the entries are called 'main' by an author called 'Elsevier', and once those are in your notes and exports they are extremely hard to find again. Fetch by identifier, keep the source of every field, and mark a document as having unverified metadata rather than inventing a plausible title. - Chunking at a fixed token count cuts tables and numbered findings in half, so a retrieved passage contains the first two rows of a results table and the answer confidently reports a figure whose units were in the row that got cut. Chunk on structure, keep tables whole or mark them as tables, and show the passage to the person before the answer. - A comparison table with no way to say 'not stated' pushes the model to fill every cell, so a paper that never mentions the outcome acquires an opinion about it, and the table is exactly the artefact people screenshot into a slide. Make 'not stated' a first-class cell value, verified the same way as a quote, and count it in the summary. ===== BUILD_PLAN.md ===== # Build plan · Afforai A local document research library whose hard problems are getting text out of a PDF with positions you can point at, fetching correct metadata from the scholarly APIs politely, and never producing a citation that cannot be resolved back to a real span on a real page. Phases are in dependency order. Each ends in a "Done when" list; treat an unticked item as a blocker, not a note. ## Phase 1 · Text out of a PDF, with the page and the position Every character you index can be pointed at on a page of the original. ### Steps 1. Keep page numbers and character offsets Everything downstream depends on being able to point at the source. Files: `src/extract.ts`, `tests/extract.test.ts` 2. Detect and refuse empty extractions An unreadable document must never look like a document that says nothing. Files: `src/extract.ts` 3. Handle columns, ligatures and hyphenation Read one page of output beside the original before believing any of it. Files: `src/layout.ts`, `tests/layout.test.ts` ### Done when - [ ] Two-column papers extract in reading order - [ ] Scanned documents are marked unreadable, not empty - [ ] A span highlights the right text on the right page - [ ] Hyphenation and ligatures are normalised ### Watch out - A two-column paper extracted in raw layout order interleaves the left and right columns line by line, so every chunk is a sentence from one column followed by an unrelated sentence from the other, and both the search results and the quoted citations are nonsense that reads almost like English. Detect columns, extract per column, and check the result by reading one page of output beside the original. ## Phase 2 · Metadata that is correct, fetched politely Titles, authors and years come from the record, and the APIs are used the way they ask. ### Steps 1. Identify yourself and stay under the published rate Check the current figures rather than copying the ones here. Files: `src/crossref.ts` 2. Deduplicate by identifier then by content hash The same paper arrives three times with three filenames. Files: `src/dedupe.ts`, `tests/dedupe.test.ts` 3. Record the provenance of every field Fetched, typed or unknown, and the user's value wins. Files: `src/metadata.ts` ### Done when - [ ] Requests carry a contact address and stay under the rate - [ ] Refusals back off rather than repeat - [ ] Duplicate additions produce one entry - [ ] User-edited fields survive a refetch ### Watch out - Guessing metadata from the filename gives you a library where half the entries are called 'main' by an author called 'Elsevier', and once those are in your notes and exports they are extremely hard to find again. Fetch by identifier, keep the source of every field, and mark a document as having unverified metadata rather than inventing a plausible title. ## Phase 3 · Retrieval you can inspect before you trust it The passages behind an answer are visible first, and they are whole thoughts. ### Steps 1. Chunk on structure and record the span range A chunk that cannot resolve to pages cannot be cited. Files: `src/chunk.ts`, `tests/chunk.test.ts` 2. Run keyword and vector search together Exact terms and paraphrases are different queries. Files: `src/retrieve.ts` 3. Show the passages before the answer Reading the sources is the fastest way to catch a bad retrieval. Files: `src/views/passages.tsx` ### Done when - [ ] Passages are shown with document, page and heading - [ ] Exact-term queries retrieve the right paper - [ ] Tables survive chunking intact - [ ] Excluding a document removes it from retrieval ### Watch out - Chunking at a fixed token count cuts tables and numbered findings in half, so a retrieved passage contains the first two rows of a results table and the answer confidently reports a figure whose units were in the row that got cut. Chunk on structure, keep tables whole or mark them as tables, and show the passage to the person before the answer. ## Phase 4 · Answers pinned to real spans, and comparisons that can say nothing Every displayed sentence has a quote that exists, and a document that does not address the question says so. ### Steps 1. Require a verbatim quote per sentence Then check it by exact match rather than by similarity. Files: `src/answer.ts`, `tests/verify.test.ts` 2. Drop unverified sentences and count them A visible drop count tells you when retrieval is failing. Files: `src/answer.ts` 3. Make 'not stated' a real cell value The alternative is a table that invents agreement. Files: `src/compare.ts`, `tests/compare.test.ts` ### Done when - [ ] False quotes are dropped and reported - [ ] Citations highlight the span on the page - [ ] Comparison cells can read 'not stated' - [ ] Answer cost is recorded and shown ### Watch out - A comparison table with no way to say 'not stated' pushes the model to fill every cell, so a paper that never mentions the outcome acquires an opinion about it, and the table is exactly the artefact people screenshot into a slide. Make 'not stated' a first-class cell value, verified the same way as a quote, and count it in the summary. ## Not in this build - Automated publisher logins and paywall access. - Licensed metadata and citation graphs. - Shared team libraries with permissions. - Automatic full-text acquisition from arbitrary sites. ## After v1, if you want it - A reading view with highlights synced back to spans, which the span table already supports - Local embeddings and a local answer model, once the verification layer has been exercised on real questions - Reference list extraction, which is a second parsing problem and deserves its own phase ===== .env.example ===== # Copy to .env and fill in. Never commit .env; this file documents it. # Required. Localhost only. This library holds documents that are often not yours to publish. APP_PORT=3210 # Required. Original files. Deleting a document deletes from here too. LIBRARY_DIR=./data/library # Required. Documents, pages, spans, chunks, answers and the egress log. DATABASE_PATH=./data/library.db # Required. Sent with metadata lookups so they reach the polite pool. CROSSREF_CONTACT=you@example.com # Optional · secret. Only used when local-only mode is off. Every call is logged. MODEL_API_KEY=sk-...redacted # Required. When true, search and citation work and no request leaves the machine. LOCAL_ONLY=false # Required. A refusal with a reason beats an extraction that runs for an hour. MAX_UPLOAD_MB=200
You are building a lean indie version of Afforai. Create the following project files first, then implement the application by following them. Keep the files updated as decisions change. Do not collapse this into a single README or prompt. ===== README.md ===== # Afforai · indie build A local document research library whose hard problems are getting text out of a PDF with positions you can point at, fetching correct metadata from the scholarly APIs politely, and never producing a citation that cannot be resolved back to a real span on a real page. Estimated effort: **multi-day**. Work `BUILD_PLAN.md` top to bottom · every phase ends in a check that has to pass before the next one starts. ## Stack | Part | Choice | Why | | --- | --- | --- | | Runtime | One local web application with a background worker, no account system | a personal library with logins is a service you now operate | | Extraction | A PDF text extractor that returns page numbers and character positions, with OCR as an explicit fallback | a citation you cannot point at is not a citation | | Metadata | Crossref and other open scholarly APIs, identified politely, with the user's value winning | guessing an author from a filename produces a library of confident errors | | Retrieval | Keyword search and vector search together over structure-aware chunks | one of them finds the exact term and the other finds the paraphrase | | Verification | Every generated claim carries a quote that must match the source text exactly | this is the feature, not a safeguard bolted on afterwards | | Storage | Documents and the index on local disk, with an explicit record of what was sent where | research documents are often things you are not free to upload | ## Before you start Have every one of these ready. The plan assumes them from step one. - [ ] **Node.js 22 or newer** · free - Why: Runs the local application, the worker and the index. - Get it: nodejs.org, or nvm install 22. - Verify: node --version prints v22 or higher - [ ] **An OCR engine for scanned documents** (optional) · free locally - Why: Scanned PDFs are common in older literature and the honest alternative to OCR is refusing the document, which you should still do when OCR is absent. - Get it: Tesseract from your package manager, or a hosted OCR you have a key for. - Verify: A scanned page produces text with page numbers preserved - [ ] **A key for an answer-generating model** (optional) · pence per question - Why: Answers and comparisons need one, and it is also the boundary at which document text leaves the machine. - Get it: Create a key with its own spending limit. - Verify: A short question over one document returns sentences with quotes - [ ] **A test corpus of awkward documents** · free - Why: Two-column papers, scanned reports and a document containing a results table are the three cases every part of this build is judged on. - Get it: Open-access papers plus one deliberately scanned document. - Verify: The corpus contains a two-column paper, a scanned file and a table - [ ] **Your citation, retention and egress rules** · free - Why: Whether an unverified claim is ever shown, how long cached extractions are kept, and what may be sent to a hosted model are decisions the code enforces. - Get it: Write down all three before the first answer is generated. - Verify: Each appears in the configuration and in the README ## Quick start Follow Phase 1 in `BUILD_PLAN.md`; it creates the project. Then copy `.env.example` to `.env` and fill in the values it documents. ## Honest limits This build deliberately does not replace: - Automated publisher logins and paywall access. - Licensed metadata and citation graphs. - Shared team libraries with permissions. - Automatic full-text acquisition from arbitrary sites. - licensed scholarly metadata - publisher-specific import reliability - citation graph scale - team libraries and institutional access If one of those is essential to you, that is the reason to keep paying for Afforai, and the README should say so rather than pretend. ===== BRIEF.md ===== # Build brief · Afforai The one-shot brief this plan expands. `BUILD_PLAN.md` (or `MILESTONES.md`) is the same sequence broken into steps and checks; where the two disagree, the plan wins. Build me a private research library that answers questions about your own documents and shows you the exact passage to replace Afforai. Build it in phases, in the order below. Do not write the whole thing in one pass. Finish a phase, run its "Done when" check, fix what fails, and only then start the next phase. ### Rule zero An answer without a locatable citation is not an answer, it is a guess with formatting. Every sentence the assistant produces carries a document, a page and a quoted span that exists verbatim in the extracted text. The application checks that the span is really there before the sentence is displayed, and drops the sentence when it is not, saying so. This is not politeness: people paste these answers into literature reviews, grant applications and clinical summaries, and a fabricated citation that looks like the real thing survives every reading until someone tries to follow it. The corollary is that a document whose text could not be extracted is marked unreadable rather than indexed as empty. A scanned page that yields no characters silently becomes a document the assistant believes says nothing, and 'the papers do not mention it' is then a lie. ### Stack (fixed, do not substitute) - Runtime: One local web application with a background worker, no account system · a personal library with logins is a service you now operate - Extraction: A PDF text extractor that returns page numbers and character positions, with OCR as an explicit fallback · a citation you cannot point at is not a citation - Metadata: Crossref and other open scholarly APIs, identified politely, with the user's value winning · guessing an author from a filename produces a library of confident errors - Retrieval: Keyword search and vector search together over structure-aware chunks · one of them finds the exact term and the other finds the paraphrase - Verification: Every generated claim carries a quote that must match the source text exactly · this is the feature, not a safeguard bolted on afterwards - Storage: Documents and the index on local disk, with an explicit record of what was sent where · research documents are often things you are not free to upload ### Data model (create this before Phase 1) - documents(id, title, authors_json, year, doi, source_kind, source_ref, sha256, pages, extraction_state, added_at) - pages(document_id, page_number, text, char_count, extraction_method) · extraction_method is native or ocr, and an empty page with neither is a failure rather than a blank - spans(id, document_id, page_number, start_char, end_char, text) · the unit a citation resolves to - chunks(id, document_id, first_span_id, last_span_id, text, heading_path, token_count, embedding_ref) - answers(id, question, created_at, model, prompt_tokens, completion_tokens, cost_cents) - claims(id, answer_id, sentence, document_id, page_number, span_id, quote, verified) · an unverified claim is stored for the record and never shown - egress(id, at, destination, purpose, bytes, document_ids_json) · what left this machine, when, and why ### Phase 1 · Text out of a PDF, with the page and the position Build: Extract per page, keeping character positions so a span can be highlighted later. Detect a page that yields no text and mark the document as needing OCR rather than indexing it empty. Handle two-column layouts so the reading order does not interleave the columns, and normalise ligatures and hyphenation at line ends so a quoted span matches what a person would search for. Trap: A two-column paper extracted in raw layout order interleaves the left and right columns line by line, so every chunk is a sentence from one column followed by an unrelated sentence from the other, and both the search results and the quoted citations are nonsense that reads almost like English. Detect columns, extract per column, and check the result by reading one page of output beside the original. Done when: A two-column paper extracts in reading order, verified by comparing a page of output with the original. A scanned document with no text layer is marked unreadable and does not enter the index, tested. A quoted span resolves back to a page number and a character range that highlights the right text. Hyphenated line breaks and ligatures are normalised so a search for the plain word finds the passage. Do not build yet: No metadata fetching, retrieval, answers or export. ### Phase 2 · Metadata that is correct, fetched politely Build: Look up a DOI against Crossref and put a contact address in the request so you land in their polite pool, which they describe as a separate pool for clients that identify themselves by a mailto in the User-Agent header or as a query parameter. Their published limits from December 2025 are ten requests a second for a single record and three a second for lists in the polite pool, against five and one anonymously, with concurrency limits alongside (checked), so rate-limit yourself and back off on a refusal. Deduplicate by DOI first and content hash second, and let a value the user typed win over a fetched one. Trap: Guessing metadata from the filename gives you a library where half the entries are called 'main' by an author called 'Elsevier', and once those are in your notes and exports they are extremely hard to find again. Fetch by identifier, keep the source of every field, and mark a document as having unverified metadata rather than inventing a plausible title. Done when: A lookup sends a contact address and is rate-limited below the published polite figures, verified in the request log. A refusal backs off and retries rather than hammering, tested against a stub. The same paper added twice by DOI and once by file produces one entry, tested. A field the user edited is never overwritten by a later fetch, tested. Do not build yet: No retrieval, answers or export yet. ### Phase 3 · Retrieval you can inspect before you trust it Build: Chunk on document structure, headings and paragraphs, with each chunk recording its first and last span so it can be resolved to pages. Run keyword search and vector search together and merge the results. Show the retrieved passages with their documents and pages before any answer is generated, and let the person exclude a document from the question. Trap: Chunking at a fixed token count cuts tables and numbered findings in half, so a retrieved passage contains the first two rows of a results table and the answer confidently reports a figure whose units were in the row that got cut. Chunk on structure, keep tables whole or mark them as tables, and show the passage to the person before the answer. Done when: Retrieved passages are shown with document, page and heading before an answer is produced. A query for an exact term present in one paper retrieves that paper, tested where vector search alone misses it. A table is not split across chunk boundaries, tested with a paper containing one. Excluding a document removes it from retrieval for that question, verified. Do not build yet: No generated answers, comparison or export yet. ### Phase 4 · Answers pinned to real spans, and comparisons that can say nothing Build: Ask for an answer as sentences, each with a document identifier and a verbatim quote. Verify each quote by exact match against the page text after normalising whitespace and case, drop any sentence that fails, and show how many were dropped. Add a comparison table with one row per document and a cell that is either a verified quote or an explicit 'not stated'. Trap: A comparison table with no way to say 'not stated' pushes the model to fill every cell, so a paper that never mentions the outcome acquires an opinion about it, and the table is exactly the artefact people screenshot into a slide. Make 'not stated' a first-class cell value, verified the same way as a quote, and count it in the summary. Done when: A sentence whose quote is not present in the cited document is dropped and the drop is reported, tested by injecting a false quote. Every displayed citation opens the page and highlights the span. A comparison cell reads 'not stated' when no verified quote supports it, tested with a paper that does not address the question. The cost of each answer is recorded and shown. Do not build yet: No export, deletion or egress log yet. ### Phase 5 · Export, deletion, and knowing what left the machine Build: Export the library as BibTeX and CSL JSON with notes and verified citations attached, plus the original files. Deleting a document removes its pages, spans, chunks, embeddings and any cached extraction, then proves it. Log every outbound call with its destination, purpose and which documents it touched. Offer a local-only mode that keeps the library usable for search and citation with no external calls at all. Trap: Sending chunk text to a hosted model without recording it leaves a researcher unable to answer the only question their ethics committee or their publisher will ask, which is what left the machine and where it went. Log every outbound call with the document identifiers involved, and make local-only mode a switch rather than an aspiration. Done when: Export produces BibTeX and CSL JSON that a reference manager imports without hand-editing, tested by importing it. A deleted document leaves no text in search, no vector in the index and no file on disk, asserted in a test. Local-only mode answers search and citation queries with no network call, verified by capturing traffic. The egress log shows destination, purpose and documents for every outbound call. Do not build yet: Nothing after this. ### Out of scope (and why) - Automating a login to a publisher to fetch articles behind a paywall, which breaks the terms your institution agreed to and is the fastest way to have your library's access cut off for everyone. - Licensed scholarly metadata and citation graphs at scale, which are commercial datasets and not something a crawler substitutes for. - Shared team libraries with permissions, which need a server, an account model and a decision about whose copy of a document is authoritative. - Automatic full-text acquisition from arbitrary sites, since what you may download differs per publisher and per subscription and is not a thing code can infer. ### README must contain - That every displayed claim carries a verified verbatim quote and that unverifiable claims are dropped and counted - That a document with no extractable text is marked unreadable rather than indexed as empty - How metadata is fetched, that requests identify themselves politely, and that a value you typed always wins - What local-only mode does, and where the egress log records everything that left the machine - How to export the library into a reference manager, and what deleting a document actually removes ===== AGENTS.md ===== # Agent instructions · Afforai indie build - Read `README.md` and `BUILD_PLAN.md` before writing code. The stack is fixed: One local web application with a background worker, no account system, A PDF text extractor that returns page numbers and character positions, with OCR as an explicit fallback, Crossref and other open scholarly APIs, identified politely, with the user's value winning, Keyword search and vector search together over structure-aware chunks, Every generated claim carries a quote that must match the source text exactly, Documents and the index on local disk, with an explicit record of what was sent where. Do not substitute. - Work one phase at a time, in order. Do not start a phase until every "Done when" item of the previous one passes. - Prefer the fewest moving parts that satisfy the step. No frameworks, services or dependencies the plan does not name. - Secrets live in `.env`, never in source or logs. Keep `.env.example` current when a variable is introduced. - Do not invent cryptography, security guarantees, APIs or compliance claims. - Add a focused test for every destructive, security-sensitive or data-loss path the plan names. - Run the project checks before declaring a phase complete, and record any deliberate shortcut in the README under "Tradeoffs". ## Known traps - A two-column paper extracted in raw layout order interleaves the left and right columns line by line, so every chunk is a sentence from one column followed by an unrelated sentence from the other, and both the search results and the quoted citations are nonsense that reads almost like English. Detect columns, extract per column, and check the result by reading one page of output beside the original. - Guessing metadata from the filename gives you a library where half the entries are called 'main' by an author called 'Elsevier', and once those are in your notes and exports they are extremely hard to find again. Fetch by identifier, keep the source of every field, and mark a document as having unverified metadata rather than inventing a plausible title. - Chunking at a fixed token count cuts tables and numbered findings in half, so a retrieved passage contains the first two rows of a results table and the answer confidently reports a figure whose units were in the row that got cut. Chunk on structure, keep tables whole or mark them as tables, and show the passage to the person before the answer. - A comparison table with no way to say 'not stated' pushes the model to fill every cell, so a paper that never mentions the outcome acquires an opinion about it, and the table is exactly the artefact people screenshot into a slide. Make 'not stated' a first-class cell value, verified the same way as a quote, and count it in the summary. ===== BUILD_PLAN.md ===== # Build plan · Afforai A local document research library whose hard problems are getting text out of a PDF with positions you can point at, fetching correct metadata from the scholarly APIs politely, and never producing a citation that cannot be resolved back to a real span on a real page. Phases are in dependency order. Each ends in a "Done when" list; treat an unticked item as a blocker, not a note. ## Phase 1 · Text out of a PDF, with the page and the position Every character you index can be pointed at on a page of the original. ### Steps 1. Keep page numbers and character offsets Everything downstream depends on being able to point at the source. Files: `src/extract.ts`, `tests/extract.test.ts` 2. Detect and refuse empty extractions An unreadable document must never look like a document that says nothing. Files: `src/extract.ts` 3. Handle columns, ligatures and hyphenation Read one page of output beside the original before believing any of it. Files: `src/layout.ts`, `tests/layout.test.ts` ### Done when - [ ] Two-column papers extract in reading order - [ ] Scanned documents are marked unreadable, not empty - [ ] A span highlights the right text on the right page - [ ] Hyphenation and ligatures are normalised ### Watch out - A two-column paper extracted in raw layout order interleaves the left and right columns line by line, so every chunk is a sentence from one column followed by an unrelated sentence from the other, and both the search results and the quoted citations are nonsense that reads almost like English. Detect columns, extract per column, and check the result by reading one page of output beside the original. ## Phase 2 · Metadata that is correct, fetched politely Titles, authors and years come from the record, and the APIs are used the way they ask. ### Steps 1. Identify yourself and stay under the published rate Check the current figures rather than copying the ones here. Files: `src/crossref.ts` 2. Deduplicate by identifier then by content hash The same paper arrives three times with three filenames. Files: `src/dedupe.ts`, `tests/dedupe.test.ts` 3. Record the provenance of every field Fetched, typed or unknown, and the user's value wins. Files: `src/metadata.ts` ### Done when - [ ] Requests carry a contact address and stay under the rate - [ ] Refusals back off rather than repeat - [ ] Duplicate additions produce one entry - [ ] User-edited fields survive a refetch ### Watch out - Guessing metadata from the filename gives you a library where half the entries are called 'main' by an author called 'Elsevier', and once those are in your notes and exports they are extremely hard to find again. Fetch by identifier, keep the source of every field, and mark a document as having unverified metadata rather than inventing a plausible title. ## Phase 3 · Retrieval you can inspect before you trust it The passages behind an answer are visible first, and they are whole thoughts. ### Steps 1. Chunk on structure and record the span range A chunk that cannot resolve to pages cannot be cited. Files: `src/chunk.ts`, `tests/chunk.test.ts` 2. Run keyword and vector search together Exact terms and paraphrases are different queries. Files: `src/retrieve.ts` 3. Show the passages before the answer Reading the sources is the fastest way to catch a bad retrieval. Files: `src/views/passages.tsx` ### Done when - [ ] Passages are shown with document, page and heading - [ ] Exact-term queries retrieve the right paper - [ ] Tables survive chunking intact - [ ] Excluding a document removes it from retrieval ### Watch out - Chunking at a fixed token count cuts tables and numbered findings in half, so a retrieved passage contains the first two rows of a results table and the answer confidently reports a figure whose units were in the row that got cut. Chunk on structure, keep tables whole or mark them as tables, and show the passage to the person before the answer. ## Phase 4 · Answers pinned to real spans, and comparisons that can say nothing Every displayed sentence has a quote that exists, and a document that does not address the question says so. ### Steps 1. Require a verbatim quote per sentence Then check it by exact match rather than by similarity. Files: `src/answer.ts`, `tests/verify.test.ts` 2. Drop unverified sentences and count them A visible drop count tells you when retrieval is failing. Files: `src/answer.ts` 3. Make 'not stated' a real cell value The alternative is a table that invents agreement. Files: `src/compare.ts`, `tests/compare.test.ts` ### Done when - [ ] False quotes are dropped and reported - [ ] Citations highlight the span on the page - [ ] Comparison cells can read 'not stated' - [ ] Answer cost is recorded and shown ### Watch out - A comparison table with no way to say 'not stated' pushes the model to fill every cell, so a paper that never mentions the outcome acquires an opinion about it, and the table is exactly the artefact people screenshot into a slide. Make 'not stated' a first-class cell value, verified the same way as a quote, and count it in the summary. ## Not in this build - Automated publisher logins and paywall access. - Licensed metadata and citation graphs. - Shared team libraries with permissions. - Automatic full-text acquisition from arbitrary sites. ## After v1, if you want it - A reading view with highlights synced back to spans, which the span table already supports - Local embeddings and a local answer model, once the verification layer has been exercised on real questions - Reference list extraction, which is a second parsing problem and deserves its own phase ===== .env.example ===== # Copy to .env and fill in. Never commit .env; this file documents it. # Required. Localhost only. This library holds documents that are often not yours to publish. APP_PORT=3210 # Required. Original files. Deleting a document deletes from here too. LIBRARY_DIR=./data/library # Required. Documents, pages, spans, chunks, answers and the egress log. DATABASE_PATH=./data/library.db # Required. Sent with metadata lookups so they reach the polite pool. CROSSREF_CONTACT=you@example.com # Optional · secret. Only used when local-only mode is off. Every call is logged. MODEL_API_KEY=sk-...redacted # Required. When true, search and citation work and no request leaves the machine. LOCAL_ONLY=false # Required. A refusal with a reason beats an extraction that runs for an hour. MAX_UPLOAD_MB=200
You are building a production product version of Afforai. Create the following project files first, then implement the application by following them. Keep the files updated as decisions change. Do not collapse this into a single README or prompt. ===== PRODUCT.md ===== # Afforai · product brief ## Problem The visible document research assistant loop is buildable, but a credible replacement needs more than the first screen. Afforai earns its keep through data, import reliability, so expect a weekend or multi-day build and a narrower personal scope. ## Product outcome A research library where every answer can be followed to a highlighted passage on a real page, metadata comes from the record rather than a filename, and you can say exactly what left the machine and what was deleted. ## Target user A builder who needs a maintainable product foundation, not a one-off demo. ## Required capabilities - Node.js 22 - browser automation for user-authorized imports - optional OpenAI API key stored in .env ## Explicit non-goals for v1 - Automated publisher logins and paywall access. - Licensed metadata and citation graphs. - Shared team libraries with permissions. - Automatic full-text acquisition from arbitrary sites. - licensed scholarly metadata - publisher-specific import reliability - citation graph scale - team libraries and institutional access ## Success criteria - Two-column extraction is confirmed in reading order - A scanned document is confirmed marked unreadable - An injected false quote is confirmed dropped - Deletion is confirmed to remove text, vectors and files - Local-only mode is confirmed to make no network call ===== BRIEF.md ===== # Build brief · Afforai The one-shot brief this plan expands. `BUILD_PLAN.md` (or `MILESTONES.md`) is the same sequence broken into steps and checks; where the two disagree, the plan wins. Build me a private research library that answers questions about your own documents and shows you the exact passage to replace Afforai. Build it in phases, in the order below. Do not write the whole thing in one pass. Finish a phase, run its "Done when" check, fix what fails, and only then start the next phase. ### Rule zero An answer without a locatable citation is not an answer, it is a guess with formatting. Every sentence the assistant produces carries a document, a page and a quoted span that exists verbatim in the extracted text. The application checks that the span is really there before the sentence is displayed, and drops the sentence when it is not, saying so. This is not politeness: people paste these answers into literature reviews, grant applications and clinical summaries, and a fabricated citation that looks like the real thing survives every reading until someone tries to follow it. The corollary is that a document whose text could not be extracted is marked unreadable rather than indexed as empty. A scanned page that yields no characters silently becomes a document the assistant believes says nothing, and 'the papers do not mention it' is then a lie. ### Stack (fixed, do not substitute) - Runtime: One local web application with a background worker, no account system · a personal library with logins is a service you now operate - Extraction: A PDF text extractor that returns page numbers and character positions, with OCR as an explicit fallback · a citation you cannot point at is not a citation - Metadata: Crossref and other open scholarly APIs, identified politely, with the user's value winning · guessing an author from a filename produces a library of confident errors - Retrieval: Keyword search and vector search together over structure-aware chunks · one of them finds the exact term and the other finds the paraphrase - Verification: Every generated claim carries a quote that must match the source text exactly · this is the feature, not a safeguard bolted on afterwards - Storage: Documents and the index on local disk, with an explicit record of what was sent where · research documents are often things you are not free to upload ### Data model (create this before Phase 1) - documents(id, title, authors_json, year, doi, source_kind, source_ref, sha256, pages, extraction_state, added_at) - pages(document_id, page_number, text, char_count, extraction_method) · extraction_method is native or ocr, and an empty page with neither is a failure rather than a blank - spans(id, document_id, page_number, start_char, end_char, text) · the unit a citation resolves to - chunks(id, document_id, first_span_id, last_span_id, text, heading_path, token_count, embedding_ref) - answers(id, question, created_at, model, prompt_tokens, completion_tokens, cost_cents) - claims(id, answer_id, sentence, document_id, page_number, span_id, quote, verified) · an unverified claim is stored for the record and never shown - egress(id, at, destination, purpose, bytes, document_ids_json) · what left this machine, when, and why ### Phase 1 · Text out of a PDF, with the page and the position Build: Extract per page, keeping character positions so a span can be highlighted later. Detect a page that yields no text and mark the document as needing OCR rather than indexing it empty. Handle two-column layouts so the reading order does not interleave the columns, and normalise ligatures and hyphenation at line ends so a quoted span matches what a person would search for. Trap: A two-column paper extracted in raw layout order interleaves the left and right columns line by line, so every chunk is a sentence from one column followed by an unrelated sentence from the other, and both the search results and the quoted citations are nonsense that reads almost like English. Detect columns, extract per column, and check the result by reading one page of output beside the original. Done when: A two-column paper extracts in reading order, verified by comparing a page of output with the original. A scanned document with no text layer is marked unreadable and does not enter the index, tested. A quoted span resolves back to a page number and a character range that highlights the right text. Hyphenated line breaks and ligatures are normalised so a search for the plain word finds the passage. Do not build yet: No metadata fetching, retrieval, answers or export. ### Phase 2 · Metadata that is correct, fetched politely Build: Look up a DOI against Crossref and put a contact address in the request so you land in their polite pool, which they describe as a separate pool for clients that identify themselves by a mailto in the User-Agent header or as a query parameter. Their published limits from December 2025 are ten requests a second for a single record and three a second for lists in the polite pool, against five and one anonymously, with concurrency limits alongside (checked), so rate-limit yourself and back off on a refusal. Deduplicate by DOI first and content hash second, and let a value the user typed win over a fetched one. Trap: Guessing metadata from the filename gives you a library where half the entries are called 'main' by an author called 'Elsevier', and once those are in your notes and exports they are extremely hard to find again. Fetch by identifier, keep the source of every field, and mark a document as having unverified metadata rather than inventing a plausible title. Done when: A lookup sends a contact address and is rate-limited below the published polite figures, verified in the request log. A refusal backs off and retries rather than hammering, tested against a stub. The same paper added twice by DOI and once by file produces one entry, tested. A field the user edited is never overwritten by a later fetch, tested. Do not build yet: No retrieval, answers or export yet. ### Phase 3 · Retrieval you can inspect before you trust it Build: Chunk on document structure, headings and paragraphs, with each chunk recording its first and last span so it can be resolved to pages. Run keyword search and vector search together and merge the results. Show the retrieved passages with their documents and pages before any answer is generated, and let the person exclude a document from the question. Trap: Chunking at a fixed token count cuts tables and numbered findings in half, so a retrieved passage contains the first two rows of a results table and the answer confidently reports a figure whose units were in the row that got cut. Chunk on structure, keep tables whole or mark them as tables, and show the passage to the person before the answer. Done when: Retrieved passages are shown with document, page and heading before an answer is produced. A query for an exact term present in one paper retrieves that paper, tested where vector search alone misses it. A table is not split across chunk boundaries, tested with a paper containing one. Excluding a document removes it from retrieval for that question, verified. Do not build yet: No generated answers, comparison or export yet. ### Phase 4 · Answers pinned to real spans, and comparisons that can say nothing Build: Ask for an answer as sentences, each with a document identifier and a verbatim quote. Verify each quote by exact match against the page text after normalising whitespace and case, drop any sentence that fails, and show how many were dropped. Add a comparison table with one row per document and a cell that is either a verified quote or an explicit 'not stated'. Trap: A comparison table with no way to say 'not stated' pushes the model to fill every cell, so a paper that never mentions the outcome acquires an opinion about it, and the table is exactly the artefact people screenshot into a slide. Make 'not stated' a first-class cell value, verified the same way as a quote, and count it in the summary. Done when: A sentence whose quote is not present in the cited document is dropped and the drop is reported, tested by injecting a false quote. Every displayed citation opens the page and highlights the span. A comparison cell reads 'not stated' when no verified quote supports it, tested with a paper that does not address the question. The cost of each answer is recorded and shown. Do not build yet: No export, deletion or egress log yet. ### Phase 5 · Export, deletion, and knowing what left the machine Build: Export the library as BibTeX and CSL JSON with notes and verified citations attached, plus the original files. Deleting a document removes its pages, spans, chunks, embeddings and any cached extraction, then proves it. Log every outbound call with its destination, purpose and which documents it touched. Offer a local-only mode that keeps the library usable for search and citation with no external calls at all. Trap: Sending chunk text to a hosted model without recording it leaves a researcher unable to answer the only question their ethics committee or their publisher will ask, which is what left the machine and where it went. Log every outbound call with the document identifiers involved, and make local-only mode a switch rather than an aspiration. Done when: Export produces BibTeX and CSL JSON that a reference manager imports without hand-editing, tested by importing it. A deleted document leaves no text in search, no vector in the index and no file on disk, asserted in a test. Local-only mode answers search and citation queries with no network call, verified by capturing traffic. The egress log shows destination, purpose and documents for every outbound call. Do not build yet: Nothing after this. ### Out of scope (and why) - Automating a login to a publisher to fetch articles behind a paywall, which breaks the terms your institution agreed to and is the fastest way to have your library's access cut off for everyone. - Licensed scholarly metadata and citation graphs at scale, which are commercial datasets and not something a crawler substitutes for. - Shared team libraries with permissions, which need a server, an account model and a decision about whose copy of a document is authoritative. - Automatic full-text acquisition from arbitrary sites, since what you may download differs per publisher and per subscription and is not a thing code can infer. ### README must contain - That every displayed claim carries a verified verbatim quote and that unverifiable claims are dropped and counted - That a document with no extractable text is marked unreadable rather than indexed as empty - How metadata is fetched, that requests identify themselves politely, and that a value you typed always wins - What local-only mode does, and where the egress log records everything that left the machine - How to export the library into a reference manager, and what deleting a document actually removes ===== ARCHITECTURE.md ===== # Architecture · Afforai ## Stack | Part | Choice | Why | | --- | --- | --- | | Runtime | One local web application with a background worker, no account system | a personal library with logins is a service you now operate | | Extraction | A PDF text extractor that returns page numbers and character positions, with OCR as an explicit fallback | a citation you cannot point at is not a citation | | Metadata | Crossref and other open scholarly APIs, identified politely, with the user's value winning | guessing an author from a filename produces a library of confident errors | | Retrieval | Keyword search and vector search together over structure-aware chunks | one of them finds the exact term and the other finds the paraphrase | | Verification | Every generated claim carries a quote that must match the source text exactly | this is the feature, not a safeguard bolted on afterwards | | Storage | Documents and the index on local disk, with an explicit record of what was sent where | research documents are often things you are not free to upload | ## Modules Each module has one owner concern and a documented way to replace it. | Module | Owns | How to replace it | | --- | --- | --- | | Extraction | Per-page text, spans, columns, OCR fallback. | Any extractor returning character positions. Never one that returns text alone. | | Metadata | Identifier lookups, provenance, deduplication. | Any source, provided requests identify themselves and the user's value wins. | | Retrieval | Structure-aware chunks and merged search. | Any index, provided chunks still resolve to pages. | | Verification | Exact quote matching and the drop count. | Not swappable for similarity scoring, which is how fabricated citations get through. | | Lifecycle | Export, deletion, local-only mode, the egress log. | Any export format a reference manager reads, tested by importing it. | ## Configuration Every runtime setting is an environment variable documented in `.env.example`, validated at startup, with a safe local default wherever one exists. - `APP_PORT` · required · Localhost only. This library holds documents that are often not yours to publish. - `LIBRARY_DIR` · required · Original files. Deleting a document deletes from here too. - `DATABASE_PATH` · required · Documents, pages, spans, chunks, answers and the egress log. - `CROSSREF_CONTACT` · required · Sent with metadata lookups so they reach the polite pool. - `MODEL_API_KEY` · optional, secret · Only used when local-only mode is off. Every call is logged. - `LOCAL_ONLY` · required · When true, search and citation work and no request leaves the machine. - `MAX_UPLOAD_MB` · required · A refusal with a reason beats an extraction that runs for an hour. ## Production baseline - Security: least privilege, input validation at every boundary, secret redaction in logs, rate limits on abuse-prone paths, no invented security primitives. - Data: explicit schema and migrations, transactional writes where integrity matters, backup and restore procedures that have been exercised. - Integrations: adapters around third-party providers, idempotent webhook or job processing, bounded retries, timeouts. - Observability: structured logs with request or operation ids, an error-tracking hook, and health and readiness checks where a server exists. - Quality: unit tests for domain rules, integration tests at module boundaries, one end-to-end test of the critical path. ## Decision records For each dependency in the stack table, keep a short note: why it was chosen, its failure mode, and how it is replaced. Do not add infrastructure until a requirement in `PRODUCT.md` justifies it. ===== AGENTS.md ===== # Agent instructions · Afforai product build - Read `PRODUCT.md` and `ARCHITECTURE.md` before changing code. The stack is fixed: One local web application with a background worker, no account system, A PDF text extractor that returns page numbers and character positions, with OCR as an explicit fallback, Crossref and other open scholarly APIs, identified politely, with the user's value winning, Keyword search and vector search together over structure-aware chunks, Every generated claim carries a quote that must match the source text exactly, Documents and the index on local disk, with an explicit record of what was sent where. - Implement milestone by milestone from `MILESTONES.md`; keep each change reviewable and leave the application runnable at every commit. - Treat authentication, payments, encryption, imports, webhooks and destructive actions as high-risk boundaries when present. - Never invent cryptography or silently weaken a requirement to make a check pass. - Put every external service behind an interface with a deterministic fake for tests. - Add migrations and rollback or recovery notes for every persistent data change. - Log useful operational context without credentials, tokens, passwords or personal data. - Update documentation and run every check before completing a milestone. ## Known traps - A two-column paper extracted in raw layout order interleaves the left and right columns line by line, so every chunk is a sentence from one column followed by an unrelated sentence from the other, and both the search results and the quoted citations are nonsense that reads almost like English. Detect columns, extract per column, and check the result by reading one page of output beside the original. - Guessing metadata from the filename gives you a library where half the entries are called 'main' by an author called 'Elsevier', and once those are in your notes and exports they are extremely hard to find again. Fetch by identifier, keep the source of every field, and mark a document as having unverified metadata rather than inventing a plausible title. - Chunking at a fixed token count cuts tables and numbered findings in half, so a retrieved passage contains the first two rows of a results table and the answer confidently reports a figure whose units were in the row that got cut. Chunk on structure, keep tables whole or mark them as tables, and show the passage to the person before the answer. - A comparison table with no way to say 'not stated' pushes the model to fill every cell, so a paper that never mentions the outcome acquires an opinion about it, and the table is exactly the artefact people screenshot into a slide. Make 'not stated' a first-class cell value, verified the same way as a quote, and count it in the summary. - Sending chunk text to a hosted model without recording it leaves a researcher unable to answer the only question their ethics committee or their publisher will ask, which is what left the machine and where it went. Log every outbound call with the document identifiers involved, and make local-only mode a switch rather than an aspiration. ===== MILESTONES.md ===== # Delivery milestones · Afforai Estimated effort: **multi-day** for the indie phases; the production-only milestones add the trust and operability layer. ## M1 · Text out of a PDF, with the page and the position Every character you index can be pointed at on a page of the original. ### Steps 1. Keep page numbers and character offsets Everything downstream depends on being able to point at the source. Files: `src/extract.ts`, `tests/extract.test.ts` 2. Detect and refuse empty extractions An unreadable document must never look like a document that says nothing. Files: `src/extract.ts` 3. Handle columns, ligatures and hyphenation Read one page of output beside the original before believing any of it. Files: `src/layout.ts`, `tests/layout.test.ts` ### Done when - [ ] Two-column papers extract in reading order - [ ] Scanned documents are marked unreadable, not empty - [ ] A span highlights the right text on the right page - [ ] Hyphenation and ligatures are normalised ### Watch out - A two-column paper extracted in raw layout order interleaves the left and right columns line by line, so every chunk is a sentence from one column followed by an unrelated sentence from the other, and both the search results and the quoted citations are nonsense that reads almost like English. Detect columns, extract per column, and check the result by reading one page of output beside the original. ## M2 · Metadata that is correct, fetched politely Titles, authors and years come from the record, and the APIs are used the way they ask. ### Steps 1. Identify yourself and stay under the published rate Check the current figures rather than copying the ones here. Files: `src/crossref.ts` 2. Deduplicate by identifier then by content hash The same paper arrives three times with three filenames. Files: `src/dedupe.ts`, `tests/dedupe.test.ts` 3. Record the provenance of every field Fetched, typed or unknown, and the user's value wins. Files: `src/metadata.ts` ### Done when - [ ] Requests carry a contact address and stay under the rate - [ ] Refusals back off rather than repeat - [ ] Duplicate additions produce one entry - [ ] User-edited fields survive a refetch ### Watch out - Guessing metadata from the filename gives you a library where half the entries are called 'main' by an author called 'Elsevier', and once those are in your notes and exports they are extremely hard to find again. Fetch by identifier, keep the source of every field, and mark a document as having unverified metadata rather than inventing a plausible title. ## M3 · Retrieval you can inspect before you trust it The passages behind an answer are visible first, and they are whole thoughts. ### Steps 1. Chunk on structure and record the span range A chunk that cannot resolve to pages cannot be cited. Files: `src/chunk.ts`, `tests/chunk.test.ts` 2. Run keyword and vector search together Exact terms and paraphrases are different queries. Files: `src/retrieve.ts` 3. Show the passages before the answer Reading the sources is the fastest way to catch a bad retrieval. Files: `src/views/passages.tsx` ### Done when - [ ] Passages are shown with document, page and heading - [ ] Exact-term queries retrieve the right paper - [ ] Tables survive chunking intact - [ ] Excluding a document removes it from retrieval ### Watch out - Chunking at a fixed token count cuts tables and numbered findings in half, so a retrieved passage contains the first two rows of a results table and the answer confidently reports a figure whose units were in the row that got cut. Chunk on structure, keep tables whole or mark them as tables, and show the passage to the person before the answer. ## M4 · Answers pinned to real spans, and comparisons that can say nothing Every displayed sentence has a quote that exists, and a document that does not address the question says so. ### Steps 1. Require a verbatim quote per sentence Then check it by exact match rather than by similarity. Files: `src/answer.ts`, `tests/verify.test.ts` 2. Drop unverified sentences and count them A visible drop count tells you when retrieval is failing. Files: `src/answer.ts` 3. Make 'not stated' a real cell value The alternative is a table that invents agreement. Files: `src/compare.ts`, `tests/compare.test.ts` ### Done when - [ ] False quotes are dropped and reported - [ ] Citations highlight the span on the page - [ ] Comparison cells can read 'not stated' - [ ] Answer cost is recorded and shown ### Watch out - A comparison table with no way to say 'not stated' pushes the model to fill every cell, so a paper that never mentions the outcome acquires an opinion about it, and the table is exactly the artefact people screenshot into a slide. Make 'not stated' a first-class cell value, verified the same way as a quote, and count it in the summary. ## M5 · Export, deletion, and knowing what left the machine (production only) The library walks out in standard formats, deletion is complete, and you can see what was sent to whom. ### Steps 1. Export in formats a reference manager reads Then import it into one before calling it finished. Files: `src/export.ts`, `tests/export.test.ts` 2. Delete text, vectors and files together Then search for a phrase unique to that document and expect nothing. Files: `src/delete.ts`, `tests/delete.test.ts` 3. Log every outbound call with its documents It is the answer to the only compliance question that matters here. Files: `src/egress.ts` ### Done when - [ ] Export imports cleanly into a reference manager - [ ] Deletion removes text, vectors and files - [ ] Local-only mode makes no network call - [ ] The egress log records destination and documents ### Watch out - Sending chunk text to a hosted model without recording it leaves a researcher unable to answer the only question their ethics committee or their publisher will ask, which is what left the machine and where it went. Log every outbound call with the document identifiers involved, and make local-only mode a switch rather than an aspiration. ===== OPERATIONS.md ===== # Operations · Afforai ## Backup The original files and the database together. Chunks and embeddings are derived and can be rebuilt, so a smaller backup of files plus metadata and notes is the sensible daily one, with a full copy less often. ## Restore Restore files and database, then re-extract anything whose extraction state is incomplete. Verify by asking a question whose answer you know and checking the citation lands on the right page. Do a restore drill before the first real user, and write the date here when it passes. ## Monitoring Documents stuck in extraction, the proportion of generated sentences being dropped as unverifiable, month-to-date model spend, and index size against disk. ## Incident checklist A rising drop count means retrieval or chunking has regressed and the passage view shows why. A citation landing on the wrong page means extraction offsets drifted, which the two-column test catches first. 1. Contain the issue without destroying evidence or user data. 2. Record the timeline and affected scope. 3. Rotate exposed secrets and revoke compromised sessions or credentials. 4. Restore from a verified backup when needed. 5. Document the root cause, the remediation and the regression test. ## Release gate - [ ] Two-column extraction is confirmed in reading order - [ ] A scanned document is confirmed marked unreadable - [ ] An injected false quote is confirmed dropped - [ ] Deletion is confirmed to remove text, vectors and files - [ ] Local-only mode is confirmed to make no network call ## Launch constraint Do not market omitted Afforai capabilities as implemented. The non-goals in `PRODUCT.md` remain user-visible limitations until they are deliberately delivered. ===== .env.example ===== # Copy to .env and fill in. Never commit .env; this file documents it. # Required. Localhost only. This library holds documents that are often not yours to publish. APP_PORT=3210 # Required. Original files. Deleting a document deletes from here too. LIBRARY_DIR=./data/library # Required. Documents, pages, spans, chunks, answers and the egress log. DATABASE_PATH=./data/library.db # Required. Sent with metadata lookups so they reach the polite pool. CROSSREF_CONTACT=you@example.com # Optional · secret. Only used when local-only mode is off. Every call is logged. MODEL_API_KEY=sk-...redacted # Required. When true, search and citation work and no request leaves the machine. LOCAL_ONLY=false # Required. A refusal with a reason beats an extraction that runs for an hour. MAX_UPLOAD_MB=200
README.md
# Afforai · indie build A local document research library whose hard problems are getting text out of a PDF with positions you can point at, fetching correct metadata from the scholarly APIs politely, and never producing a citation that cannot be resolved back to a real span on a real page. Estimated effort: **multi-day**. Work `BUILD_PLAN.md` top to bottom · every phase ends in a check that has to pass before the next one starts. ## Stack | Part | Choice | Why | | --- | --- | --- | | Runtime | One local web application with a background worker, no account system | a personal library with logins is a service you now operate | | Extraction | A PDF text extractor that returns page numbers and character positions, with OCR as an explicit fallback | a citation you cannot point at is not a citation | | Metadata | Crossref and other open scholarly APIs, identified politely, with the user's value winning | guessing an author from a filename produces a library of confident errors | | Retrieval | Keyword search and vector search together over structure-aware chunks | one of them finds the exact term and the other finds the paraphrase | | Verification | Every generated claim carries a quote that must match the source text exactly | this is the feature, not a safeguard bolted on afterwards | | Storage | Documents and the index on local disk, with an explicit record of what was sent where | research documents are often things you are not free to upload | ## Before you start Have every one of these ready. The plan assumes them from step one. - [ ] **Node.js 22 or newer** · free - Why: Runs the local application, the worker and the index. - Get it: nodejs.org, or nvm install 22. - Verify: node --version prints v22 or higher - [ ] **An OCR engine for scanned documents** (optional) · free locally - Why: Scanned PDFs are common in older literature and the honest alternative to OCR is refusing the document, which you should still do when OCR is absent. - Get it: Tesseract from your package manager, or a hosted OCR you have a key for. - Verify: A scanned page produces text with page numbers preserved - [ ] **A key for an answer-generating model** (optional) · pence per question - Why: Answers and comparisons need one, and it is also the boundary at which document text leaves the machine. - Get it: Create a key with its own spending limit. - Verify: A short question over one document returns sentences with quotes - [ ] **A test corpus of awkward documents** · free - Why: Two-column papers, scanned reports and a document containing a results table are the three cases every part of this build is judged on. - Get it: Open-access papers plus one deliberately scanned document. - Verify: The corpus contains a two-column paper, a scanned file and a table - [ ] **Your citation, retention and egress rules** · free - Why: Whether an unverified claim is ever shown, how long cached extractions are kept, and what may be sent to a hosted model are decisions the code enforces. - Get it: Write down all three before the first answer is generated. - Verify: Each appears in the configuration and in the README ## Quick start Follow Phase 1 in `BUILD_PLAN.md`; it creates the project. Then copy `.env.example` to `.env` and fill in the values it documents. ## Honest limits This build deliberately does not replace: - Automated publisher logins and paywall access. - Licensed metadata and citation graphs. - Shared team libraries with permissions. - Automatic full-text acquisition from arbitrary sites. - licensed scholarly metadata - publisher-specific import reliability - citation graph scale - team libraries and institutional access If one of those is essential to you, that is the reason to keep paying for Afforai, and the README should say so rather than pretend.
BRIEF.md
# Build brief · Afforai The one-shot brief this plan expands. `BUILD_PLAN.md` (or `MILESTONES.md`) is the same sequence broken into steps and checks; where the two disagree, the plan wins. Build me a private research library that answers questions about your own documents and shows you the exact passage to replace Afforai. Build it in phases, in the order below. Do not write the whole thing in one pass. Finish a phase, run its "Done when" check, fix what fails, and only then start the next phase. ### Rule zero An answer without a locatable citation is not an answer, it is a guess with formatting. Every sentence the assistant produces carries a document, a page and a quoted span that exists verbatim in the extracted text. The application checks that the span is really there before the sentence is displayed, and drops the sentence when it is not, saying so. This is not politeness: people paste these answers into literature reviews, grant applications and clinical summaries, and a fabricated citation that looks like the real thing survives every reading until someone tries to follow it. The corollary is that a document whose text could not be extracted is marked unreadable rather than indexed as empty. A scanned page that yields no characters silently becomes a document the assistant believes says nothing, and 'the papers do not mention it' is then a lie. ### Stack (fixed, do not substitute) - Runtime: One local web application with a background worker, no account system · a personal library with logins is a service you now operate - Extraction: A PDF text extractor that returns page numbers and character positions, with OCR as an explicit fallback · a citation you cannot point at is not a citation - Metadata: Crossref and other open scholarly APIs, identified politely, with the user's value winning · guessing an author from a filename produces a library of confident errors - Retrieval: Keyword search and vector search together over structure-aware chunks · one of them finds the exact term and the other finds the paraphrase - Verification: Every generated claim carries a quote that must match the source text exactly · this is the feature, not a safeguard bolted on afterwards - Storage: Documents and the index on local disk, with an explicit record of what was sent where · research documents are often things you are not free to upload ### Data model (create this before Phase 1) - documents(id, title, authors_json, year, doi, source_kind, source_ref, sha256, pages, extraction_state, added_at) - pages(document_id, page_number, text, char_count, extraction_method) · extraction_method is native or ocr, and an empty page with neither is a failure rather than a blank - spans(id, document_id, page_number, start_char, end_char, text) · the unit a citation resolves to - chunks(id, document_id, first_span_id, last_span_id, text, heading_path, token_count, embedding_ref) - answers(id, question, created_at, model, prompt_tokens, completion_tokens, cost_cents) - claims(id, answer_id, sentence, document_id, page_number, span_id, quote, verified) · an unverified claim is stored for the record and never shown - egress(id, at, destination, purpose, bytes, document_ids_json) · what left this machine, when, and why ### Phase 1 · Text out of a PDF, with the page and the position Build: Extract per page, keeping character positions so a span can be highlighted later. Detect a page that yields no text and mark the document as needing OCR rather than indexing it empty. Handle two-column layouts so the reading order does not interleave the columns, and normalise ligatures and hyphenation at line ends so a quoted span matches what a person would search for. Trap: A two-column paper extracted in raw layout order interleaves the left and right columns line by line, so every chunk is a sentence from one column followed by an unrelated sentence from the other, and both the search results and the quoted citations are nonsense that reads almost like English. Detect columns, extract per column, and check the result by reading one page of output beside the original. Done when: A two-column paper extracts in reading order, verified by comparing a page of output with the original. A scanned document with no text layer is marked unreadable and does not enter the index, tested. A quoted span resolves back to a page number and a character range that highlights the right text. Hyphenated line breaks and ligatures are normalised so a search for the plain word finds the passage. Do not build yet: No metadata fetching, retrieval, answers or export. ### Phase 2 · Metadata that is correct, fetched politely Build: Look up a DOI against Crossref and put a contact address in the request so you land in their polite pool, which they describe as a separate pool for clients that identify themselves by a mailto in the User-Agent header or as a query parameter. Their published limits from December 2025 are ten requests a second for a single record and three a second for lists in the polite pool, against five and one anonymously, with concurrency limits alongside (checked), so rate-limit yourself and back off on a refusal. Deduplicate by DOI first and content hash second, and let a value the user typed win over a fetched one. Trap: Guessing metadata from the filename gives you a library where half the entries are called 'main' by an author called 'Elsevier', and once those are in your notes and exports they are extremely hard to find again. Fetch by identifier, keep the source of every field, and mark a document as having unverified metadata rather than inventing a plausible title. Done when: A lookup sends a contact address and is rate-limited below the published polite figures, verified in the request log. A refusal backs off and retries rather than hammering, tested against a stub. The same paper added twice by DOI and once by file produces one entry, tested. A field the user edited is never overwritten by a later fetch, tested. Do not build yet: No retrieval, answers or export yet. ### Phase 3 · Retrieval you can inspect before you trust it Build: Chunk on document structure, headings and paragraphs, with each chunk recording its first and last span so it can be resolved to pages. Run keyword search and vector search together and merge the results. Show the retrieved passages with their documents and pages before any answer is generated, and let the person exclude a document from the question. Trap: Chunking at a fixed token count cuts tables and numbered findings in half, so a retrieved passage contains the first two rows of a results table and the answer confidently reports a figure whose units were in the row that got cut. Chunk on structure, keep tables whole or mark them as tables, and show the passage to the person before the answer. Done when: Retrieved passages are shown with document, page and heading before an answer is produced. A query for an exact term present in one paper retrieves that paper, tested where vector search alone misses it. A table is not split across chunk boundaries, tested with a paper containing one. Excluding a document removes it from retrieval for that question, verified. Do not build yet: No generated answers, comparison or export yet. ### Phase 4 · Answers pinned to real spans, and comparisons that can say nothing Build: Ask for an answer as sentences, each with a document identifier and a verbatim quote. Verify each quote by exact match against the page text after normalising whitespace and case, drop any sentence that fails, and show how many were dropped. Add a comparison table with one row per document and a cell that is either a verified quote or an explicit 'not stated'. Trap: A comparison table with no way to say 'not stated' pushes the model to fill every cell, so a paper that never mentions the outcome acquires an opinion about it, and the table is exactly the artefact people screenshot into a slide. Make 'not stated' a first-class cell value, verified the same way as a quote, and count it in the summary. Done when: A sentence whose quote is not present in the cited document is dropped and the drop is reported, tested by injecting a false quote. Every displayed citation opens the page and highlights the span. A comparison cell reads 'not stated' when no verified quote supports it, tested with a paper that does not address the question. The cost of each answer is recorded and shown. Do not build yet: No export, deletion or egress log yet. ### Phase 5 · Export, deletion, and knowing what left the machine Build: Export the library as BibTeX and CSL JSON with notes and verified citations attached, plus the original files. Deleting a document removes its pages, spans, chunks, embeddings and any cached extraction, then proves it. Log every outbound call with its destination, purpose and which documents it touched. Offer a local-only mode that keeps the library usable for search and citation with no external calls at all. Trap: Sending chunk text to a hosted model without recording it leaves a researcher unable to answer the only question their ethics committee or their publisher will ask, which is what left the machine and where it went. Log every outbound call with the document identifiers involved, and make local-only mode a switch rather than an aspiration. Done when: Export produces BibTeX and CSL JSON that a reference manager imports without hand-editing, tested by importing it. A deleted document leaves no text in search, no vector in the index and no file on disk, asserted in a test. Local-only mode answers search and citation queries with no network call, verified by capturing traffic. The egress log shows destination, purpose and documents for every outbound call. Do not build yet: Nothing after this. ### Out of scope (and why) - Automating a login to a publisher to fetch articles behind a paywall, which breaks the terms your institution agreed to and is the fastest way to have your library's access cut off for everyone. - Licensed scholarly metadata and citation graphs at scale, which are commercial datasets and not something a crawler substitutes for. - Shared team libraries with permissions, which need a server, an account model and a decision about whose copy of a document is authoritative. - Automatic full-text acquisition from arbitrary sites, since what you may download differs per publisher and per subscription and is not a thing code can infer. ### README must contain - That every displayed claim carries a verified verbatim quote and that unverifiable claims are dropped and counted - That a document with no extractable text is marked unreadable rather than indexed as empty - How metadata is fetched, that requests identify themselves politely, and that a value you typed always wins - What local-only mode does, and where the egress log records everything that left the machine - How to export the library into a reference manager, and what deleting a document actually removes
AGENTS.md
# Agent instructions · Afforai indie build - Read `README.md` and `BUILD_PLAN.md` before writing code. The stack is fixed: One local web application with a background worker, no account system, A PDF text extractor that returns page numbers and character positions, with OCR as an explicit fallback, Crossref and other open scholarly APIs, identified politely, with the user's value winning, Keyword search and vector search together over structure-aware chunks, Every generated claim carries a quote that must match the source text exactly, Documents and the index on local disk, with an explicit record of what was sent where. Do not substitute. - Work one phase at a time, in order. Do not start a phase until every "Done when" item of the previous one passes. - Prefer the fewest moving parts that satisfy the step. No frameworks, services or dependencies the plan does not name. - Secrets live in `.env`, never in source or logs. Keep `.env.example` current when a variable is introduced. - Do not invent cryptography, security guarantees, APIs or compliance claims. - Add a focused test for every destructive, security-sensitive or data-loss path the plan names. - Run the project checks before declaring a phase complete, and record any deliberate shortcut in the README under "Tradeoffs". ## Known traps - A two-column paper extracted in raw layout order interleaves the left and right columns line by line, so every chunk is a sentence from one column followed by an unrelated sentence from the other, and both the search results and the quoted citations are nonsense that reads almost like English. Detect columns, extract per column, and check the result by reading one page of output beside the original. - Guessing metadata from the filename gives you a library where half the entries are called 'main' by an author called 'Elsevier', and once those are in your notes and exports they are extremely hard to find again. Fetch by identifier, keep the source of every field, and mark a document as having unverified metadata rather than inventing a plausible title. - Chunking at a fixed token count cuts tables and numbered findings in half, so a retrieved passage contains the first two rows of a results table and the answer confidently reports a figure whose units were in the row that got cut. Chunk on structure, keep tables whole or mark them as tables, and show the passage to the person before the answer. - A comparison table with no way to say 'not stated' pushes the model to fill every cell, so a paper that never mentions the outcome acquires an opinion about it, and the table is exactly the artefact people screenshot into a slide. Make 'not stated' a first-class cell value, verified the same way as a quote, and count it in the summary.
BUILD_PLAN.md
# Build plan · Afforai A local document research library whose hard problems are getting text out of a PDF with positions you can point at, fetching correct metadata from the scholarly APIs politely, and never producing a citation that cannot be resolved back to a real span on a real page. Phases are in dependency order. Each ends in a "Done when" list; treat an unticked item as a blocker, not a note. ## Phase 1 · Text out of a PDF, with the page and the position Every character you index can be pointed at on a page of the original. ### Steps 1. Keep page numbers and character offsets Everything downstream depends on being able to point at the source. Files: `src/extract.ts`, `tests/extract.test.ts` 2. Detect and refuse empty extractions An unreadable document must never look like a document that says nothing. Files: `src/extract.ts` 3. Handle columns, ligatures and hyphenation Read one page of output beside the original before believing any of it. Files: `src/layout.ts`, `tests/layout.test.ts` ### Done when - [ ] Two-column papers extract in reading order - [ ] Scanned documents are marked unreadable, not empty - [ ] A span highlights the right text on the right page - [ ] Hyphenation and ligatures are normalised ### Watch out - A two-column paper extracted in raw layout order interleaves the left and right columns line by line, so every chunk is a sentence from one column followed by an unrelated sentence from the other, and both the search results and the quoted citations are nonsense that reads almost like English. Detect columns, extract per column, and check the result by reading one page of output beside the original. ## Phase 2 · Metadata that is correct, fetched politely Titles, authors and years come from the record, and the APIs are used the way they ask. ### Steps 1. Identify yourself and stay under the published rate Check the current figures rather than copying the ones here. Files: `src/crossref.ts` 2. Deduplicate by identifier then by content hash The same paper arrives three times with three filenames. Files: `src/dedupe.ts`, `tests/dedupe.test.ts` 3. Record the provenance of every field Fetched, typed or unknown, and the user's value wins. Files: `src/metadata.ts` ### Done when - [ ] Requests carry a contact address and stay under the rate - [ ] Refusals back off rather than repeat - [ ] Duplicate additions produce one entry - [ ] User-edited fields survive a refetch ### Watch out - Guessing metadata from the filename gives you a library where half the entries are called 'main' by an author called 'Elsevier', and once those are in your notes and exports they are extremely hard to find again. Fetch by identifier, keep the source of every field, and mark a document as having unverified metadata rather than inventing a plausible title. ## Phase 3 · Retrieval you can inspect before you trust it The passages behind an answer are visible first, and they are whole thoughts. ### Steps 1. Chunk on structure and record the span range A chunk that cannot resolve to pages cannot be cited. Files: `src/chunk.ts`, `tests/chunk.test.ts` 2. Run keyword and vector search together Exact terms and paraphrases are different queries. Files: `src/retrieve.ts` 3. Show the passages before the answer Reading the sources is the fastest way to catch a bad retrieval. Files: `src/views/passages.tsx` ### Done when - [ ] Passages are shown with document, page and heading - [ ] Exact-term queries retrieve the right paper - [ ] Tables survive chunking intact - [ ] Excluding a document removes it from retrieval ### Watch out - Chunking at a fixed token count cuts tables and numbered findings in half, so a retrieved passage contains the first two rows of a results table and the answer confidently reports a figure whose units were in the row that got cut. Chunk on structure, keep tables whole or mark them as tables, and show the passage to the person before the answer. ## Phase 4 · Answers pinned to real spans, and comparisons that can say nothing Every displayed sentence has a quote that exists, and a document that does not address the question says so. ### Steps 1. Require a verbatim quote per sentence Then check it by exact match rather than by similarity. Files: `src/answer.ts`, `tests/verify.test.ts` 2. Drop unverified sentences and count them A visible drop count tells you when retrieval is failing. Files: `src/answer.ts` 3. Make 'not stated' a real cell value The alternative is a table that invents agreement. Files: `src/compare.ts`, `tests/compare.test.ts` ### Done when - [ ] False quotes are dropped and reported - [ ] Citations highlight the span on the page - [ ] Comparison cells can read 'not stated' - [ ] Answer cost is recorded and shown ### Watch out - A comparison table with no way to say 'not stated' pushes the model to fill every cell, so a paper that never mentions the outcome acquires an opinion about it, and the table is exactly the artefact people screenshot into a slide. Make 'not stated' a first-class cell value, verified the same way as a quote, and count it in the summary. ## Not in this build - Automated publisher logins and paywall access. - Licensed metadata and citation graphs. - Shared team libraries with permissions. - Automatic full-text acquisition from arbitrary sites. ## After v1, if you want it - A reading view with highlights synced back to spans, which the span table already supports - Local embeddings and a local answer model, once the verification layer has been exercised on real questions - Reference list extraction, which is a second parsing problem and deserves its own phase
.env.example
# Copy to .env and fill in. Never commit .env; this file documents it. # Required. Localhost only. This library holds documents that are often not yours to publish. APP_PORT=3210 # Required. Original files. Deleting a document deletes from here too. LIBRARY_DIR=./data/library # Required. Documents, pages, spans, chunks, answers and the egress log. DATABASE_PATH=./data/library.db # Required. Sent with metadata lookups so they reach the polite pool. CROSSREF_CONTACT=you@example.com # Optional · secret. Only used when local-only mode is off. Every call is logged. MODEL_API_KEY=sk-...redacted # Required. When true, search and citation work and no request leaves the machine. LOCAL_ONLY=false # Required. A refusal with a reason beats an extraction that runs for an hour. MAX_UPLOAD_MB=200
PRODUCT.md
# Afforai · product brief ## Problem The visible document research assistant loop is buildable, but a credible replacement needs more than the first screen. Afforai earns its keep through data, import reliability, so expect a weekend or multi-day build and a narrower personal scope. ## Product outcome A research library where every answer can be followed to a highlighted passage on a real page, metadata comes from the record rather than a filename, and you can say exactly what left the machine and what was deleted. ## Target user A builder who needs a maintainable product foundation, not a one-off demo. ## Required capabilities - Node.js 22 - browser automation for user-authorized imports - optional OpenAI API key stored in .env ## Explicit non-goals for v1 - Automated publisher logins and paywall access. - Licensed metadata and citation graphs. - Shared team libraries with permissions. - Automatic full-text acquisition from arbitrary sites. - licensed scholarly metadata - publisher-specific import reliability - citation graph scale - team libraries and institutional access ## Success criteria - Two-column extraction is confirmed in reading order - A scanned document is confirmed marked unreadable - An injected false quote is confirmed dropped - Deletion is confirmed to remove text, vectors and files - Local-only mode is confirmed to make no network call
BRIEF.md
# Build brief · Afforai The one-shot brief this plan expands. `BUILD_PLAN.md` (or `MILESTONES.md`) is the same sequence broken into steps and checks; where the two disagree, the plan wins. Build me a private research library that answers questions about your own documents and shows you the exact passage to replace Afforai. Build it in phases, in the order below. Do not write the whole thing in one pass. Finish a phase, run its "Done when" check, fix what fails, and only then start the next phase. ### Rule zero An answer without a locatable citation is not an answer, it is a guess with formatting. Every sentence the assistant produces carries a document, a page and a quoted span that exists verbatim in the extracted text. The application checks that the span is really there before the sentence is displayed, and drops the sentence when it is not, saying so. This is not politeness: people paste these answers into literature reviews, grant applications and clinical summaries, and a fabricated citation that looks like the real thing survives every reading until someone tries to follow it. The corollary is that a document whose text could not be extracted is marked unreadable rather than indexed as empty. A scanned page that yields no characters silently becomes a document the assistant believes says nothing, and 'the papers do not mention it' is then a lie. ### Stack (fixed, do not substitute) - Runtime: One local web application with a background worker, no account system · a personal library with logins is a service you now operate - Extraction: A PDF text extractor that returns page numbers and character positions, with OCR as an explicit fallback · a citation you cannot point at is not a citation - Metadata: Crossref and other open scholarly APIs, identified politely, with the user's value winning · guessing an author from a filename produces a library of confident errors - Retrieval: Keyword search and vector search together over structure-aware chunks · one of them finds the exact term and the other finds the paraphrase - Verification: Every generated claim carries a quote that must match the source text exactly · this is the feature, not a safeguard bolted on afterwards - Storage: Documents and the index on local disk, with an explicit record of what was sent where · research documents are often things you are not free to upload ### Data model (create this before Phase 1) - documents(id, title, authors_json, year, doi, source_kind, source_ref, sha256, pages, extraction_state, added_at) - pages(document_id, page_number, text, char_count, extraction_method) · extraction_method is native or ocr, and an empty page with neither is a failure rather than a blank - spans(id, document_id, page_number, start_char, end_char, text) · the unit a citation resolves to - chunks(id, document_id, first_span_id, last_span_id, text, heading_path, token_count, embedding_ref) - answers(id, question, created_at, model, prompt_tokens, completion_tokens, cost_cents) - claims(id, answer_id, sentence, document_id, page_number, span_id, quote, verified) · an unverified claim is stored for the record and never shown - egress(id, at, destination, purpose, bytes, document_ids_json) · what left this machine, when, and why ### Phase 1 · Text out of a PDF, with the page and the position Build: Extract per page, keeping character positions so a span can be highlighted later. Detect a page that yields no text and mark the document as needing OCR rather than indexing it empty. Handle two-column layouts so the reading order does not interleave the columns, and normalise ligatures and hyphenation at line ends so a quoted span matches what a person would search for. Trap: A two-column paper extracted in raw layout order interleaves the left and right columns line by line, so every chunk is a sentence from one column followed by an unrelated sentence from the other, and both the search results and the quoted citations are nonsense that reads almost like English. Detect columns, extract per column, and check the result by reading one page of output beside the original. Done when: A two-column paper extracts in reading order, verified by comparing a page of output with the original. A scanned document with no text layer is marked unreadable and does not enter the index, tested. A quoted span resolves back to a page number and a character range that highlights the right text. Hyphenated line breaks and ligatures are normalised so a search for the plain word finds the passage. Do not build yet: No metadata fetching, retrieval, answers or export. ### Phase 2 · Metadata that is correct, fetched politely Build: Look up a DOI against Crossref and put a contact address in the request so you land in their polite pool, which they describe as a separate pool for clients that identify themselves by a mailto in the User-Agent header or as a query parameter. Their published limits from December 2025 are ten requests a second for a single record and three a second for lists in the polite pool, against five and one anonymously, with concurrency limits alongside (checked), so rate-limit yourself and back off on a refusal. Deduplicate by DOI first and content hash second, and let a value the user typed win over a fetched one. Trap: Guessing metadata from the filename gives you a library where half the entries are called 'main' by an author called 'Elsevier', and once those are in your notes and exports they are extremely hard to find again. Fetch by identifier, keep the source of every field, and mark a document as having unverified metadata rather than inventing a plausible title. Done when: A lookup sends a contact address and is rate-limited below the published polite figures, verified in the request log. A refusal backs off and retries rather than hammering, tested against a stub. The same paper added twice by DOI and once by file produces one entry, tested. A field the user edited is never overwritten by a later fetch, tested. Do not build yet: No retrieval, answers or export yet. ### Phase 3 · Retrieval you can inspect before you trust it Build: Chunk on document structure, headings and paragraphs, with each chunk recording its first and last span so it can be resolved to pages. Run keyword search and vector search together and merge the results. Show the retrieved passages with their documents and pages before any answer is generated, and let the person exclude a document from the question. Trap: Chunking at a fixed token count cuts tables and numbered findings in half, so a retrieved passage contains the first two rows of a results table and the answer confidently reports a figure whose units were in the row that got cut. Chunk on structure, keep tables whole or mark them as tables, and show the passage to the person before the answer. Done when: Retrieved passages are shown with document, page and heading before an answer is produced. A query for an exact term present in one paper retrieves that paper, tested where vector search alone misses it. A table is not split across chunk boundaries, tested with a paper containing one. Excluding a document removes it from retrieval for that question, verified. Do not build yet: No generated answers, comparison or export yet. ### Phase 4 · Answers pinned to real spans, and comparisons that can say nothing Build: Ask for an answer as sentences, each with a document identifier and a verbatim quote. Verify each quote by exact match against the page text after normalising whitespace and case, drop any sentence that fails, and show how many were dropped. Add a comparison table with one row per document and a cell that is either a verified quote or an explicit 'not stated'. Trap: A comparison table with no way to say 'not stated' pushes the model to fill every cell, so a paper that never mentions the outcome acquires an opinion about it, and the table is exactly the artefact people screenshot into a slide. Make 'not stated' a first-class cell value, verified the same way as a quote, and count it in the summary. Done when: A sentence whose quote is not present in the cited document is dropped and the drop is reported, tested by injecting a false quote. Every displayed citation opens the page and highlights the span. A comparison cell reads 'not stated' when no verified quote supports it, tested with a paper that does not address the question. The cost of each answer is recorded and shown. Do not build yet: No export, deletion or egress log yet. ### Phase 5 · Export, deletion, and knowing what left the machine Build: Export the library as BibTeX and CSL JSON with notes and verified citations attached, plus the original files. Deleting a document removes its pages, spans, chunks, embeddings and any cached extraction, then proves it. Log every outbound call with its destination, purpose and which documents it touched. Offer a local-only mode that keeps the library usable for search and citation with no external calls at all. Trap: Sending chunk text to a hosted model without recording it leaves a researcher unable to answer the only question their ethics committee or their publisher will ask, which is what left the machine and where it went. Log every outbound call with the document identifiers involved, and make local-only mode a switch rather than an aspiration. Done when: Export produces BibTeX and CSL JSON that a reference manager imports without hand-editing, tested by importing it. A deleted document leaves no text in search, no vector in the index and no file on disk, asserted in a test. Local-only mode answers search and citation queries with no network call, verified by capturing traffic. The egress log shows destination, purpose and documents for every outbound call. Do not build yet: Nothing after this. ### Out of scope (and why) - Automating a login to a publisher to fetch articles behind a paywall, which breaks the terms your institution agreed to and is the fastest way to have your library's access cut off for everyone. - Licensed scholarly metadata and citation graphs at scale, which are commercial datasets and not something a crawler substitutes for. - Shared team libraries with permissions, which need a server, an account model and a decision about whose copy of a document is authoritative. - Automatic full-text acquisition from arbitrary sites, since what you may download differs per publisher and per subscription and is not a thing code can infer. ### README must contain - That every displayed claim carries a verified verbatim quote and that unverifiable claims are dropped and counted - That a document with no extractable text is marked unreadable rather than indexed as empty - How metadata is fetched, that requests identify themselves politely, and that a value you typed always wins - What local-only mode does, and where the egress log records everything that left the machine - How to export the library into a reference manager, and what deleting a document actually removes
ARCHITECTURE.md
# Architecture · Afforai ## Stack | Part | Choice | Why | | --- | --- | --- | | Runtime | One local web application with a background worker, no account system | a personal library with logins is a service you now operate | | Extraction | A PDF text extractor that returns page numbers and character positions, with OCR as an explicit fallback | a citation you cannot point at is not a citation | | Metadata | Crossref and other open scholarly APIs, identified politely, with the user's value winning | guessing an author from a filename produces a library of confident errors | | Retrieval | Keyword search and vector search together over structure-aware chunks | one of them finds the exact term and the other finds the paraphrase | | Verification | Every generated claim carries a quote that must match the source text exactly | this is the feature, not a safeguard bolted on afterwards | | Storage | Documents and the index on local disk, with an explicit record of what was sent where | research documents are often things you are not free to upload | ## Modules Each module has one owner concern and a documented way to replace it. | Module | Owns | How to replace it | | --- | --- | --- | | Extraction | Per-page text, spans, columns, OCR fallback. | Any extractor returning character positions. Never one that returns text alone. | | Metadata | Identifier lookups, provenance, deduplication. | Any source, provided requests identify themselves and the user's value wins. | | Retrieval | Structure-aware chunks and merged search. | Any index, provided chunks still resolve to pages. | | Verification | Exact quote matching and the drop count. | Not swappable for similarity scoring, which is how fabricated citations get through. | | Lifecycle | Export, deletion, local-only mode, the egress log. | Any export format a reference manager reads, tested by importing it. | ## Configuration Every runtime setting is an environment variable documented in `.env.example`, validated at startup, with a safe local default wherever one exists. - `APP_PORT` · required · Localhost only. This library holds documents that are often not yours to publish. - `LIBRARY_DIR` · required · Original files. Deleting a document deletes from here too. - `DATABASE_PATH` · required · Documents, pages, spans, chunks, answers and the egress log. - `CROSSREF_CONTACT` · required · Sent with metadata lookups so they reach the polite pool. - `MODEL_API_KEY` · optional, secret · Only used when local-only mode is off. Every call is logged. - `LOCAL_ONLY` · required · When true, search and citation work and no request leaves the machine. - `MAX_UPLOAD_MB` · required · A refusal with a reason beats an extraction that runs for an hour. ## Production baseline - Security: least privilege, input validation at every boundary, secret redaction in logs, rate limits on abuse-prone paths, no invented security primitives. - Data: explicit schema and migrations, transactional writes where integrity matters, backup and restore procedures that have been exercised. - Integrations: adapters around third-party providers, idempotent webhook or job processing, bounded retries, timeouts. - Observability: structured logs with request or operation ids, an error-tracking hook, and health and readiness checks where a server exists. - Quality: unit tests for domain rules, integration tests at module boundaries, one end-to-end test of the critical path. ## Decision records For each dependency in the stack table, keep a short note: why it was chosen, its failure mode, and how it is replaced. Do not add infrastructure until a requirement in `PRODUCT.md` justifies it.
AGENTS.md
# Agent instructions · Afforai product build - Read `PRODUCT.md` and `ARCHITECTURE.md` before changing code. The stack is fixed: One local web application with a background worker, no account system, A PDF text extractor that returns page numbers and character positions, with OCR as an explicit fallback, Crossref and other open scholarly APIs, identified politely, with the user's value winning, Keyword search and vector search together over structure-aware chunks, Every generated claim carries a quote that must match the source text exactly, Documents and the index on local disk, with an explicit record of what was sent where. - Implement milestone by milestone from `MILESTONES.md`; keep each change reviewable and leave the application runnable at every commit. - Treat authentication, payments, encryption, imports, webhooks and destructive actions as high-risk boundaries when present. - Never invent cryptography or silently weaken a requirement to make a check pass. - Put every external service behind an interface with a deterministic fake for tests. - Add migrations and rollback or recovery notes for every persistent data change. - Log useful operational context without credentials, tokens, passwords or personal data. - Update documentation and run every check before completing a milestone. ## Known traps - A two-column paper extracted in raw layout order interleaves the left and right columns line by line, so every chunk is a sentence from one column followed by an unrelated sentence from the other, and both the search results and the quoted citations are nonsense that reads almost like English. Detect columns, extract per column, and check the result by reading one page of output beside the original. - Guessing metadata from the filename gives you a library where half the entries are called 'main' by an author called 'Elsevier', and once those are in your notes and exports they are extremely hard to find again. Fetch by identifier, keep the source of every field, and mark a document as having unverified metadata rather than inventing a plausible title. - Chunking at a fixed token count cuts tables and numbered findings in half, so a retrieved passage contains the first two rows of a results table and the answer confidently reports a figure whose units were in the row that got cut. Chunk on structure, keep tables whole or mark them as tables, and show the passage to the person before the answer. - A comparison table with no way to say 'not stated' pushes the model to fill every cell, so a paper that never mentions the outcome acquires an opinion about it, and the table is exactly the artefact people screenshot into a slide. Make 'not stated' a first-class cell value, verified the same way as a quote, and count it in the summary. - Sending chunk text to a hosted model without recording it leaves a researcher unable to answer the only question their ethics committee or their publisher will ask, which is what left the machine and where it went. Log every outbound call with the document identifiers involved, and make local-only mode a switch rather than an aspiration.
MILESTONES.md
# Delivery milestones · Afforai Estimated effort: **multi-day** for the indie phases; the production-only milestones add the trust and operability layer. ## M1 · Text out of a PDF, with the page and the position Every character you index can be pointed at on a page of the original. ### Steps 1. Keep page numbers and character offsets Everything downstream depends on being able to point at the source. Files: `src/extract.ts`, `tests/extract.test.ts` 2. Detect and refuse empty extractions An unreadable document must never look like a document that says nothing. Files: `src/extract.ts` 3. Handle columns, ligatures and hyphenation Read one page of output beside the original before believing any of it. Files: `src/layout.ts`, `tests/layout.test.ts` ### Done when - [ ] Two-column papers extract in reading order - [ ] Scanned documents are marked unreadable, not empty - [ ] A span highlights the right text on the right page - [ ] Hyphenation and ligatures are normalised ### Watch out - A two-column paper extracted in raw layout order interleaves the left and right columns line by line, so every chunk is a sentence from one column followed by an unrelated sentence from the other, and both the search results and the quoted citations are nonsense that reads almost like English. Detect columns, extract per column, and check the result by reading one page of output beside the original. ## M2 · Metadata that is correct, fetched politely Titles, authors and years come from the record, and the APIs are used the way they ask. ### Steps 1. Identify yourself and stay under the published rate Check the current figures rather than copying the ones here. Files: `src/crossref.ts` 2. Deduplicate by identifier then by content hash The same paper arrives three times with three filenames. Files: `src/dedupe.ts`, `tests/dedupe.test.ts` 3. Record the provenance of every field Fetched, typed or unknown, and the user's value wins. Files: `src/metadata.ts` ### Done when - [ ] Requests carry a contact address and stay under the rate - [ ] Refusals back off rather than repeat - [ ] Duplicate additions produce one entry - [ ] User-edited fields survive a refetch ### Watch out - Guessing metadata from the filename gives you a library where half the entries are called 'main' by an author called 'Elsevier', and once those are in your notes and exports they are extremely hard to find again. Fetch by identifier, keep the source of every field, and mark a document as having unverified metadata rather than inventing a plausible title. ## M3 · Retrieval you can inspect before you trust it The passages behind an answer are visible first, and they are whole thoughts. ### Steps 1. Chunk on structure and record the span range A chunk that cannot resolve to pages cannot be cited. Files: `src/chunk.ts`, `tests/chunk.test.ts` 2. Run keyword and vector search together Exact terms and paraphrases are different queries. Files: `src/retrieve.ts` 3. Show the passages before the answer Reading the sources is the fastest way to catch a bad retrieval. Files: `src/views/passages.tsx` ### Done when - [ ] Passages are shown with document, page and heading - [ ] Exact-term queries retrieve the right paper - [ ] Tables survive chunking intact - [ ] Excluding a document removes it from retrieval ### Watch out - Chunking at a fixed token count cuts tables and numbered findings in half, so a retrieved passage contains the first two rows of a results table and the answer confidently reports a figure whose units were in the row that got cut. Chunk on structure, keep tables whole or mark them as tables, and show the passage to the person before the answer. ## M4 · Answers pinned to real spans, and comparisons that can say nothing Every displayed sentence has a quote that exists, and a document that does not address the question says so. ### Steps 1. Require a verbatim quote per sentence Then check it by exact match rather than by similarity. Files: `src/answer.ts`, `tests/verify.test.ts` 2. Drop unverified sentences and count them A visible drop count tells you when retrieval is failing. Files: `src/answer.ts` 3. Make 'not stated' a real cell value The alternative is a table that invents agreement. Files: `src/compare.ts`, `tests/compare.test.ts` ### Done when - [ ] False quotes are dropped and reported - [ ] Citations highlight the span on the page - [ ] Comparison cells can read 'not stated' - [ ] Answer cost is recorded and shown ### Watch out - A comparison table with no way to say 'not stated' pushes the model to fill every cell, so a paper that never mentions the outcome acquires an opinion about it, and the table is exactly the artefact people screenshot into a slide. Make 'not stated' a first-class cell value, verified the same way as a quote, and count it in the summary. ## M5 · Export, deletion, and knowing what left the machine (production only) The library walks out in standard formats, deletion is complete, and you can see what was sent to whom. ### Steps 1. Export in formats a reference manager reads Then import it into one before calling it finished. Files: `src/export.ts`, `tests/export.test.ts` 2. Delete text, vectors and files together Then search for a phrase unique to that document and expect nothing. Files: `src/delete.ts`, `tests/delete.test.ts` 3. Log every outbound call with its documents It is the answer to the only compliance question that matters here. Files: `src/egress.ts` ### Done when - [ ] Export imports cleanly into a reference manager - [ ] Deletion removes text, vectors and files - [ ] Local-only mode makes no network call - [ ] The egress log records destination and documents ### Watch out - Sending chunk text to a hosted model without recording it leaves a researcher unable to answer the only question their ethics committee or their publisher will ask, which is what left the machine and where it went. Log every outbound call with the document identifiers involved, and make local-only mode a switch rather than an aspiration.
OPERATIONS.md
# Operations · Afforai ## Backup The original files and the database together. Chunks and embeddings are derived and can be rebuilt, so a smaller backup of files plus metadata and notes is the sensible daily one, with a full copy less often. ## Restore Restore files and database, then re-extract anything whose extraction state is incomplete. Verify by asking a question whose answer you know and checking the citation lands on the right page. Do a restore drill before the first real user, and write the date here when it passes. ## Monitoring Documents stuck in extraction, the proportion of generated sentences being dropped as unverifiable, month-to-date model spend, and index size against disk. ## Incident checklist A rising drop count means retrieval or chunking has regressed and the passage view shows why. A citation landing on the wrong page means extraction offsets drifted, which the two-column test catches first. 1. Contain the issue without destroying evidence or user data. 2. Record the timeline and affected scope. 3. Rotate exposed secrets and revoke compromised sessions or credentials. 4. Restore from a verified backup when needed. 5. Document the root cause, the remediation and the regression test. ## Release gate - [ ] Two-column extraction is confirmed in reading order - [ ] A scanned document is confirmed marked unreadable - [ ] An injected false quote is confirmed dropped - [ ] Deletion is confirmed to remove text, vectors and files - [ ] Local-only mode is confirmed to make no network call ## Launch constraint Do not market omitted Afforai capabilities as implemented. The non-goals in `PRODUCT.md` remain user-visible limitations until they are deliberately delivered.
.env.example
# Copy to .env and fill in. Never commit .env; this file documents it. # Required. Localhost only. This library holds documents that are often not yours to publish. APP_PORT=3210 # Required. Original files. Deleting a document deletes from here too. LIBRARY_DIR=./data/library # Required. Documents, pages, spans, chunks, answers and the egress log. DATABASE_PATH=./data/library.db # Required. Sent with metadata lookups so they reach the polite pool. CROSSREF_CONTACT=you@example.com # Optional · secret. Only used when local-only mode is off. Every call is logged. MODEL_API_KEY=sk-...redacted # Required. When true, search and citation work and no request leaves the machine. LOCAL_ONLY=false # Required. A refusal with a reason beats an extraction that runs for an hour. MAX_UPLOAD_MB=200
$ choose a build depth, inspect the files, then open the complete pack in your agent
Afforai: Researchers pay for correct metadata, resilient importers, citation coverage, and workflows that survive publisher and browser changes.
xlicensed scholarly metadata
xpublisher-specific import reliability
xcitation graph scale
xteam libraries and institutional access
share on X ↗"I just replaced Afforai with one prompt"
Don't feel like building it? These folks already made it free.
AnythingLLMPoint it at your files and bring your own model; the privacy is free, the compute is not.open source↗OOpen NotebookNotebookLM on your own server; bring Docker, a model and realistic expectations about citations.open source↗
NotebookLMUpload the papers and interrogate them; Google pays the model bill and keeps the limits.free↗all 3 free alternatives to Afforai →· no votes, no pay-to-list · just what's real
Afforai pricing
| plan | monthly | annual (per mo) | what you get |
|---|---|---|---|
| logically free | $0/user | $0/user | Permanent free entry point; numeric document, storage and AI-use caps were not exposed by the JavaScript-only live page. |
| logically unlimited | custom | — | Unlimited storage and unlimited AI usage are advertised; numeric fair-use thresholds and price were not exposed. |
free tierfree plan exists, but numeric document, storage and AI-use caps could not be verified from the accessible JavaScript-only page
billingpaid Unlimited plan exists; exact monthly/annual cadence and price were not publicly exposed; 30-day money-back guarantee
hidden coststhe product and billing identity changed from Afforai to Logically; any unpublished fair-use threshold remains unknown
verified 2026-08-13 · source ↗
Vibecode Afforai
Kinda. The core of Afforai is buildable in a weekend with the prompt on this page, but there are real gaps: licensed scholarly metadata, publisher-specific import reliability. Read the honest list above before committing.
How much does Afforai cost?
Afforai's pricing is usage-based or varies by plan · The official pricing URL is recorded, but a stable current amount was not safely recoverable in this pass. Do not publish a number until rechecked..
What do I lose by replacing Afforai?
Honestly: licensed scholarly metadata; publisher-specific import reliability; citation graph scale; team libraries and institutional access. If any of those are load-bearing for you, keep paying.
Is there an open-source alternative to Afforai?
Yes: AnythingLLM (Point it at your files and bring your own model; the privacy is free, the compute is not.) Open Notebook (NotebookLM on your own server; bring Docker, a model and realistic expectations about citations.) NotebookLM (Upload the papers and interrogate them; Google pays the model bill and keeps the limits.) All 3 curated free alternatives are at vibecodeit.com/afforai/alternatives. The prompt is for when you want it exactly your way.