Vibecode Readwise Reader

You are building a lean indie version of Readwise Reader.

Create the following project files first, then implement the application by following them. Keep the files updated as decisions change. Do not collapse this into a single README or prompt.

===== README.md =====
# Readwise Reader · indie build

Save a page from anywhere, extract a clean reader view, highlight and annotate, and have those highlights still point at the right text when the extractor improves, because they are anchored by quoted text and its surroundings rather than by a character offset.

Estimated effort: **multi-day**. Work `BUILD_PLAN.md` top to bottom · every phase ends in a check that has to pass before the next one starts.

## Stack

| Part | Choice | Why |
| --- | --- | --- |
| Runtime | Node 22 with SQLite | one person's reading queue, where the interesting work is extraction and anchoring |
| Extraction | Readability plus a sanitiser | the article without the navigation, and inert so opening your own archive runs nobody's scripts |
| Anchoring | Quoted text with prefix and suffix context, not offsets | the single decision that determines whether highlights survive anything |
| Capture | A bookmarklet and a share endpoint | no browser extension to maintain across browsers and their review processes |
| Export | Markdown per document with highlights inline | highlights are the durable artefact and must outlive this application |

## Before you start

Have every one of these ready. The plan assumes them from step one.

- [ ] **Node.js 22 or newer** · free
  - Why: Runs the server, the extractor and the review job.
  - Get it: nodejs.org, or nvm install 22.
  - Verify: node --version prints v22 or higher
- [ ] **Five awkward article URLs** · free
  - Why: Phase 1's extraction and Phase 3's re-extraction are both only meaningful against real pages: a clean article, a JavaScript-only page, one with an aggressive content security policy, a very long piece and one that gets edited.
  - Get it: Collect them from your own reading and keep them as fixtures with their expected outcomes.
  - Verify: All five are saved with expected extraction results recorded
- [ ] **An article with repeated phrases** · free
  - Why: Phase 2's anchoring is only tested by a document where the same words appear twice, which is what the surrounding context exists to disambiguate.
  - Get it: Any long article will contain repeated phrases. Highlight one deliberately and check it resolves to the right occurrence.
  - Verify: A highlight on a repeated phrase resolves to the correct occurrence
- [ ] **An email provider for the review digest** (optional) · free tiers cover this
  - Why: Phase 5 only. Everything else works without delivery.
  - Get it: A transactional provider with a verified sending domain.
  - Verify: A digest arrives in an inbox
- [ ] **Somewhere to run it, for the product phase** (optional) · 5 to 10 US dollars a month
  - Why: Saving from a phone needs a reachable endpoint, and the review digest needs something running on a schedule.
  - Get it: A small virtual server, or a machine at home behind a tunnel. Authenticate before exposing it.
  - Verify: A save from a phone reaches the endpoint over HTTPS

## Quick start

```sh
npm install @mozilla/readability jsdom dompurify
```

Then copy `.env.example` to `.env` and fill in the values it documents.

## Honest limits

This build deliberately does not replace:

- Extraction quality matching the commercial product.
- Mobile applications with offline reading.
- Reading device integrations and their highlight formats.
- Access to paywalled material.
- excellent parsing
- mobile/offline apps
- Kindle integrations
- RSS/PDF workflows
- spaced repetition
- exports

If one of those is essential to you, that is the reason to keep paying for Readwise Reader, and the README should say so rather than pretend.

===== BRIEF.md =====
# Build brief · Readwise Reader

The one-shot brief this plan expands. `BUILD_PLAN.md` (or `MILESTONES.md`) is the same sequence broken into steps and checks; where the two disagree, the plan wins.

Build me a read-later queue where highlights survive the article being re-extracted to replace Readwise Reader. Build it in phases, in the order below.
Do not write the whole thing in one pass. Finish a phase, run its "Done when"
check, fix what fails, and only then start the next phase.

### Stack (fixed, do not substitute)
- Runtime: Node 22 with SQLite · one person's reading queue, where the interesting work is extraction and anchoring
- Extraction: Readability plus a sanitiser · the article without the navigation, and inert so opening your own archive runs nobody's scripts
- Anchoring: Quoted text with prefix and suffix context, not offsets · the single decision that determines whether highlights survive anything
- Capture: A bookmarklet and a share endpoint · no browser extension to maintain across browsers and their review processes
- Export: Markdown per document with highlights inline · highlights are the durable artefact and must outlive this application

### Data model (create this before Phase 1)
- documents(id, url, canonical_url, title, author, saved_at, published_at, state, word_count) · state is inbox, later, archived
- extractions(id, document_id, extractor_version, text, html, extracted_at, content_hash)
- highlights(id, document_id, quote, prefix, suffix, note, colour, created_at, resolved_offset, resolve_state)
- tags(id, name), document_tags(document_id, tag_id)
- reviews(highlight_id, last_shown_at, next_due_at, times_shown)
- A highlight belongs to a document, not to an extraction. That is what lets the extraction be replaced.

### Phase 1 · Save and extract
Build: A save endpoint taking a URL. Fetch with a timeout and a byte cap, extract the article,
  sanitise it so it makes no outbound requests, and store both text and cleaned HTML with
  the extractor version. Record failures as failures.
Trap: Storing raw fetched HTML and rendering it later means every time you open your own archive you load the publisher's scripts and trackers. Sanitise at capture: strip scripts, event handlers and remote references, so the stored copy is inert.
Done when: A saved article is readable offline. Opening a stored article makes no outbound network request, verified in developer tools. A page requiring JavaScript is recorded as a failed extraction with a reason rather than an empty document. The extractor version is stored with every extraction.
Do not build yet: No highlights, tags or review.

### Phase 2 · Highlights anchored to text, not to positions
Build: On highlight, store the quoted text plus a short prefix and suffix from the surrounding
  content. Resolve a highlight by searching for that quote with its context. Record whether
  resolution succeeded, was approximate, or failed.
Trap: Storing a character offset into the extracted text means every highlight breaks the moment the extraction changes by one character, which happens whenever the extractor improves or the publisher edits the page. Quote plus context resolves against new text and degrades gracefully when it cannot.
Done when: A highlight resolves correctly in the stored text. Inserting a paragraph before it in the extraction still resolves it to the same words. A highlight whose text has been deleted from the article is marked unresolvable rather than silently landing somewhere else. Resolution state is visible per highlight.
Do not build yet: No re-extraction or review yet.

### Phase 3 · Re-extraction, which is the point
Build: Re-extract a document with a newer extractor, keeping the previous extraction. Re-resolve
  every highlight against the new text and report the outcome. Let a failed re-extraction
  be rolled back to the previous version.
Trap: Replacing an extraction without re-resolving highlights leaves them pointing at text that has moved, and the damage is silent and cumulative. Re-resolve as part of re-extraction, report the counts, and keep the old extraction so a bad one can be reverted.
Done when: Re-extracting a document re-resolves every highlight and reports how many resolved exactly, approximately and not at all. A re-extraction that resolves fewer highlights can be rolled back to the previous version. The previous extraction is retained. A document with no highlights re-extracts without incident.
Do not build yet: No tags, reading queue or export yet.

### Phase 4 · A queue you can actually clear
Build: An inbox, a later list and an archive, with one action to move between them. Sort by
  estimated reading time as well as by date. Show the age of the oldest unread item
  honestly rather than hiding it.
Trap: A read-later queue with no triage becomes a list of a thousand articles nobody opens, and the tool stops being used rather than the queue being cleared. Make archiving without reading a first-class action, since deciding not to read something is a legitimate outcome and the queue depends on it.
Done when: Moving between inbox, later and archive is one action each. The list can be sorted by estimated reading time. Archiving without reading is as easy as reading, and is recorded distinctly. The age of the oldest unread item is shown.
Do not build yet: No review or export yet.

### Phase 5 · Highlight review
Build: A review that surfaces highlights on a schedule, weighted towards recent saves and
  highlights you have annotated. A daily email digest. Every review shows the source
  article and links back to it.
Trap: Showing highlights at random from a large archive produces mostly forgettable ones and trains you to skip the review. Weight towards annotated highlights and recent reading, and cap the daily count so the review takes a minute rather than being abandoned.
Done when: A review surfaces a capped number of highlights daily. Annotated highlights appear more often than unannotated ones, verified over a simulated month. Each review item links back to its source article. Marking a highlight as not useful stops it recurring.
Do not build yet: Export and capture tooling come last.

### Phase 6 · Capture everywhere, and export
Build: A bookmarklet and an endpoint that accepts a shared URL from a phone. Export every
  document as Markdown with its highlights inline and its metadata in front matter.
  Authenticate anything hosted, because a reading list is revealing.
Trap: A reading queue and its highlights are a detailed record of what you are thinking about, which is more revealing than most people expect. Authenticate before exposing it anywhere, and be deliberate about whether the highlights export includes your notes.
Done when: The bookmarklet saves from three different sites and reports clearly on one that blocks it. A shared URL from a phone saves through the endpoint. Export produces one Markdown file per document with highlights inline and front matter metadata. Hosted access requires authentication.
Do not build yet: Nothing after this.

### Out of scope (and why)
- Extraction quality at the level of the commercial product, which handles a long tail of sites through per-site work that never finishes.
- Mobile applications with offline reading, which is how most read-later use actually happens.
- Integrations with reading devices and their highlight formats, each of which is its own extraction problem.
- Access to material behind paywalls and logins, which this build does not attempt.

### README must contain
- That highlights are anchored by quoted text with context rather than by offsets, and why
- That re-extraction re-resolves highlights and reports the outcome before becoming current
- That stored articles are sanitised and make no outbound requests
- That archiving without reading is a first-class action
- That the Markdown export is the format designed to outlive this application

===== AGENTS.md =====
# Agent instructions · Readwise Reader indie build

- Read `README.md` and `BUILD_PLAN.md` before writing code. The stack is fixed: Node 22 with SQLite, Readability plus a sanitiser, Quoted text with prefix and suffix context, not offsets, A bookmarklet and a share endpoint, Markdown per document with highlights inline. Do not substitute.
- Work one phase at a time, in order. Do not start a phase until every "Done when" item of the previous one passes.
- Prefer the fewest moving parts that satisfy the step. No frameworks, services or dependencies the plan does not name.
- Secrets live in `.env`, never in source or logs. Keep `.env.example` current when a variable is introduced.
- Do not invent cryptography, security guarantees, APIs or compliance claims.
- Add a focused test for every destructive, security-sensitive or data-loss path the plan names.
- Run the project checks before declaring a phase complete, and record any deliberate shortcut in the README under "Tradeoffs".

## Known traps

- Storing raw fetched HTML and rendering it later means every time you open your own archive you load the publisher's scripts and trackers. Sanitise at capture: strip scripts, event handlers and remote references, so the stored copy is inert.
- Storing a character offset into the extracted text means every highlight breaks the moment the extraction changes by one character, which happens whenever the extractor improves or the publisher edits the page. Quote plus context resolves against new text and degrades gracefully when it cannot.
- Replacing an extraction without re-resolving highlights leaves them pointing at text that has moved, and the damage is silent and cumulative. Re-resolve as part of re-extraction, report the counts, and keep the old extraction so a bad one can be reverted.
- A read-later queue with no triage becomes a list of a thousand articles nobody opens, and the tool stops being used rather than the queue being cleared. Make archiving without reading a first-class action, since deciding not to read something is a legitimate outcome and the queue depends on it.
- Showing highlights at random from a large archive produces mostly forgettable ones and trains you to skip the review. Weight towards annotated highlights and recent reading, and cap the daily count so the review takes a minute rather than being abandoned.

===== BUILD_PLAN.md =====
# Build plan · Readwise Reader

Save a page from anywhere, extract a clean reader view, highlight and annotate, and have those highlights still point at the right text when the extractor improves, because they are anchored by quoted text and its surroundings rather than by a character offset.

Phases are in dependency order. Each ends in a "Done when" list; treat an unticked item as a blocker, not a note.

## Phase 1 · Save and extract

A saved page becomes readable text, and a page that cannot be extracted says so.

### Steps

1. Fetch with limits
   Timeout, streaming byte cap, robots respected. You are a well-behaved client saving for yourself.
   Files: `src/fetch.js`
2. Extract and sanitise
   Readability for the article, then strip scripts and remote references so the copy is inert.
   Files: `src/extract.js`

   ```sh
   npm install @mozilla/readability jsdom dompurify
   ```

3. Version every extraction
   So you know which extractor produced a stored copy, which matters in Phase 3.
   Files: `src/extract.js`

### Done when

- [ ] A saved article reads offline
- [ ] A stored article makes no outbound requests
- [ ] A JavaScript-only page is recorded as a failure with a reason
- [ ] Extractor version is stored with each extraction

### Watch out

- Storing raw fetched HTML and rendering it later means every time you open your own archive you load the publisher's scripts and trackers. Sanitise at capture: strip scripts, event handlers and remote references, so the stored copy is inert.

## Phase 2 · Highlights anchored to text, not to positions

A highlight knows what it selected, not where in a string it was.

### Steps

1. Store quote with context
   The selected text plus a short prefix and suffix. Enough context to disambiguate a repeated phrase.
   Files: `src/anchor.js`
2. Resolve by search
   Exact match first, then approximate. Record which strategy succeeded.
   Files: `src/anchor.js`
3. Handle the unresolvable case visibly
   Marked, still readable as a quote, never silently attached to the wrong place.
   Files: `src/anchor.js`

### Done when

- [ ] A highlight resolves correctly in stored text
- [ ] Inserting text before it does not move it
- [ ] A deleted quote is marked unresolvable rather than misplaced
- [ ] Resolution state is visible per highlight

### Watch out

- Storing a character offset into the extracted text means every highlight breaks the moment the extraction changes by one character, which happens whenever the extractor improves or the publisher edits the page. Quote plus context resolves against new text and degrades gracefully when it cannot.

## Phase 3 · Re-extraction, which is the point

Improving the extractor does not lose a year of highlights.

### Steps

1. Keep extractions as versions
   A document has several; one is current. Rollback is then trivial.
   Files: `src/extractions.js`
2. Re-resolve and report
   Counts by outcome, shown before the new extraction becomes current.
   Files: `src/reextract.js`
3. Support rollback
   A re-extraction that loses highlights should be reversible in one action.
   Files: `src/extractions.js`

### Done when

- [ ] Re-extraction reports resolution outcomes by count
- [ ] A worse extraction can be rolled back
- [ ] The previous extraction is retained
- [ ] A document without highlights re-extracts cleanly

### Watch out

- Replacing an extraction without re-resolving highlights leaves them pointing at text that has moved, and the damage is silent and cumulative. Re-resolve as part of re-extraction, report the counts, and keep the old extraction so a bad one can be reverted.

## Phase 4 · A queue you can actually clear

Saved articles are triaged rather than accumulating into a wall of guilt.

### Steps

1. Implement the three states
   Inbox, later, archive. One action between them, keyboard reachable.
   Files: `src/queue.js`
2. Estimate reading time
   From word count. It is what lets you pick something that fits the time you have.
   Files: `src/queue.js`
3. Make declining first class
   Archive without reading, recorded as such. It is the action that keeps the queue survivable.
   Files: `src/queue.js`

### Done when

- [ ] State changes are one action each
- [ ] Sorting by reading time works
- [ ] Archiving unread is recorded distinctly
- [ ] The oldest unread age is shown

### Watch out

- A read-later queue with no triage becomes a list of a thousand articles nobody opens, and the tool stops being used rather than the queue being cleared. Make archiving without reading a first-class action, since deciding not to read something is a legitimate outcome and the queue depends on it.

## Phase 5 · Highlight review

What you highlighted comes back, spaced out, so it is not written once and never seen again.

### Steps

1. Schedule and weight the selection
   Recency and annotation as weights, with a daily cap.
   Files: `src/review.js`
2. Send a digest
   Email with the highlights and their sources. This is where a review actually gets read.
   Files: `src/digest.js`
3. Let a highlight be retired
   Marked not useful and never shown again. Without it, the review degrades over time.
   Files: `src/review.js`

### Done when

- [ ] The daily review is capped
- [ ] Annotated highlights surface more often over a simulated month
- [ ] Each item links to its source
- [ ] A retired highlight stops recurring

### Watch out

- Showing highlights at random from a large archive produces mostly forgettable ones and trains you to skip the review. Weight towards annotated highlights and recent reading, and cap the daily count so the review takes a minute rather than being abandoned.

## Not in this build

- Extraction quality matching the commercial product.
- Mobile applications with offline reading.
- Reading device integrations and their highlight formats.
- Access to paywalled material.

## After v1, if you want it

- Full text search across every document and highlight
- Automatic tagging from article content, proposed rather than applied
- A weekly digest of what you saved and did not read, which is the honest metric

===== .env.example =====
# Copy to .env and fill in. Never commit .env; this file documents it.

# Required. Documents, extractions and highlights.
DB_PATH=./data/reader.db

# Required. Per-page fetch timeout. A slow page should fail rather than hold up a save.
FETCH_TIMEOUT_MS=10000

# Required. Streaming byte cap, enforced during download.
FETCH_MAX_BYTES=5000000

# Required. How much prefix and suffix is stored with a quote. Enough to disambiguate a repeated phrase.
ANCHOR_CONTEXT_CHARS=40

# Required. Highlights per daily review. Small enough that the review actually happens.
REVIEW_DAILY_CAP=5

# Required · secret. Used by the bookmarklet and the share endpoint. Scoped to saving only.
SAVE_TOKEN=replace-with-a-random-token

You are building a lean indie version of Readwise Reader.

Create the following project files first, then implement the application by following them. Keep the files updated as decisions change. Do not collapse this into a single README or prompt.

===== README.md =====
# Readwise Reader · indie build

Save a page from anywhere, extract a clean reader view, highlight and annotate, and have those highlights still point at the right text when the extractor improves, because they are anchored by quoted text and its surroundings rather than by a character offset.

Estimated effort: **multi-day**. Work `BUILD_PLAN.md` top to bottom · every phase ends in a check that has to pass before the next one starts.

## Stack

| Part | Choice | Why |
| --- | --- | --- |
| Runtime | Node 22 with SQLite | one person's reading queue, where the interesting work is extraction and anchoring |
| Extraction | Readability plus a sanitiser | the article without the navigation, and inert so opening your own archive runs nobody's scripts |
| Anchoring | Quoted text with prefix and suffix context, not offsets | the single decision that determines whether highlights survive anything |
| Capture | A bookmarklet and a share endpoint | no browser extension to maintain across browsers and their review processes |
| Export | Markdown per document with highlights inline | highlights are the durable artefact and must outlive this application |

## Before you start

Have every one of these ready. The plan assumes them from step one.

- [ ] **Node.js 22 or newer** · free
  - Why: Runs the server, the extractor and the review job.
  - Get it: nodejs.org, or nvm install 22.
  - Verify: node --version prints v22 or higher
- [ ] **Five awkward article URLs** · free
  - Why: Phase 1's extraction and Phase 3's re-extraction are both only meaningful against real pages: a clean article, a JavaScript-only page, one with an aggressive content security policy, a very long piece and one that gets edited.
  - Get it: Collect them from your own reading and keep them as fixtures with their expected outcomes.
  - Verify: All five are saved with expected extraction results recorded
- [ ] **An article with repeated phrases** · free
  - Why: Phase 2's anchoring is only tested by a document where the same words appear twice, which is what the surrounding context exists to disambiguate.
  - Get it: Any long article will contain repeated phrases. Highlight one deliberately and check it resolves to the right occurrence.
  - Verify: A highlight on a repeated phrase resolves to the correct occurrence
- [ ] **An email provider for the review digest** (optional) · free tiers cover this
  - Why: Phase 5 only. Everything else works without delivery.
  - Get it: A transactional provider with a verified sending domain.
  - Verify: A digest arrives in an inbox
- [ ] **Somewhere to run it, for the product phase** (optional) · 5 to 10 US dollars a month
  - Why: Saving from a phone needs a reachable endpoint, and the review digest needs something running on a schedule.
  - Get it: A small virtual server, or a machine at home behind a tunnel. Authenticate before exposing it.
  - Verify: A save from a phone reaches the endpoint over HTTPS

## Quick start

```sh
npm install @mozilla/readability jsdom dompurify
```

Then copy `.env.example` to `.env` and fill in the values it documents.

## Honest limits

This build deliberately does not replace:

- Extraction quality matching the commercial product.
- Mobile applications with offline reading.
- Reading device integrations and their highlight formats.
- Access to paywalled material.
- excellent parsing
- mobile/offline apps
- Kindle integrations
- RSS/PDF workflows
- spaced repetition
- exports

If one of those is essential to you, that is the reason to keep paying for Readwise Reader, and the README should say so rather than pretend.

===== BRIEF.md =====
# Build brief · Readwise Reader

The one-shot brief this plan expands. `BUILD_PLAN.md` (or `MILESTONES.md`) is the same sequence broken into steps and checks; where the two disagree, the plan wins.

Build me a read-later queue where highlights survive the article being re-extracted to replace Readwise Reader. Build it in phases, in the order below.
Do not write the whole thing in one pass. Finish a phase, run its "Done when"
check, fix what fails, and only then start the next phase.

### Stack (fixed, do not substitute)
- Runtime: Node 22 with SQLite · one person's reading queue, where the interesting work is extraction and anchoring
- Extraction: Readability plus a sanitiser · the article without the navigation, and inert so opening your own archive runs nobody's scripts
- Anchoring: Quoted text with prefix and suffix context, not offsets · the single decision that determines whether highlights survive anything
- Capture: A bookmarklet and a share endpoint · no browser extension to maintain across browsers and their review processes
- Export: Markdown per document with highlights inline · highlights are the durable artefact and must outlive this application

### Data model (create this before Phase 1)
- documents(id, url, canonical_url, title, author, saved_at, published_at, state, word_count) · state is inbox, later, archived
- extractions(id, document_id, extractor_version, text, html, extracted_at, content_hash)
- highlights(id, document_id, quote, prefix, suffix, note, colour, created_at, resolved_offset, resolve_state)
- tags(id, name), document_tags(document_id, tag_id)
- reviews(highlight_id, last_shown_at, next_due_at, times_shown)
- A highlight belongs to a document, not to an extraction. That is what lets the extraction be replaced.

### Phase 1 · Save and extract
Build: A save endpoint taking a URL. Fetch with a timeout and a byte cap, extract the article,
  sanitise it so it makes no outbound requests, and store both text and cleaned HTML with
  the extractor version. Record failures as failures.
Trap: Storing raw fetched HTML and rendering it later means every time you open your own archive you load the publisher's scripts and trackers. Sanitise at capture: strip scripts, event handlers and remote references, so the stored copy is inert.
Done when: A saved article is readable offline. Opening a stored article makes no outbound network request, verified in developer tools. A page requiring JavaScript is recorded as a failed extraction with a reason rather than an empty document. The extractor version is stored with every extraction.
Do not build yet: No highlights, tags or review.

### Phase 2 · Highlights anchored to text, not to positions
Build: On highlight, store the quoted text plus a short prefix and suffix from the surrounding
  content. Resolve a highlight by searching for that quote with its context. Record whether
  resolution succeeded, was approximate, or failed.
Trap: Storing a character offset into the extracted text means every highlight breaks the moment the extraction changes by one character, which happens whenever the extractor improves or the publisher edits the page. Quote plus context resolves against new text and degrades gracefully when it cannot.
Done when: A highlight resolves correctly in the stored text. Inserting a paragraph before it in the extraction still resolves it to the same words. A highlight whose text has been deleted from the article is marked unresolvable rather than silently landing somewhere else. Resolution state is visible per highlight.
Do not build yet: No re-extraction or review yet.

### Phase 3 · Re-extraction, which is the point
Build: Re-extract a document with a newer extractor, keeping the previous extraction. Re-resolve
  every highlight against the new text and report the outcome. Let a failed re-extraction
  be rolled back to the previous version.
Trap: Replacing an extraction without re-resolving highlights leaves them pointing at text that has moved, and the damage is silent and cumulative. Re-resolve as part of re-extraction, report the counts, and keep the old extraction so a bad one can be reverted.
Done when: Re-extracting a document re-resolves every highlight and reports how many resolved exactly, approximately and not at all. A re-extraction that resolves fewer highlights can be rolled back to the previous version. The previous extraction is retained. A document with no highlights re-extracts without incident.
Do not build yet: No tags, reading queue or export yet.

### Phase 4 · A queue you can actually clear
Build: An inbox, a later list and an archive, with one action to move between them. Sort by
  estimated reading time as well as by date. Show the age of the oldest unread item
  honestly rather than hiding it.
Trap: A read-later queue with no triage becomes a list of a thousand articles nobody opens, and the tool stops being used rather than the queue being cleared. Make archiving without reading a first-class action, since deciding not to read something is a legitimate outcome and the queue depends on it.
Done when: Moving between inbox, later and archive is one action each. The list can be sorted by estimated reading time. Archiving without reading is as easy as reading, and is recorded distinctly. The age of the oldest unread item is shown.
Do not build yet: No review or export yet.

### Phase 5 · Highlight review
Build: A review that surfaces highlights on a schedule, weighted towards recent saves and
  highlights you have annotated. A daily email digest. Every review shows the source
  article and links back to it.
Trap: Showing highlights at random from a large archive produces mostly forgettable ones and trains you to skip the review. Weight towards annotated highlights and recent reading, and cap the daily count so the review takes a minute rather than being abandoned.
Done when: A review surfaces a capped number of highlights daily. Annotated highlights appear more often than unannotated ones, verified over a simulated month. Each review item links back to its source article. Marking a highlight as not useful stops it recurring.
Do not build yet: Export and capture tooling come last.

### Phase 6 · Capture everywhere, and export
Build: A bookmarklet and an endpoint that accepts a shared URL from a phone. Export every
  document as Markdown with its highlights inline and its metadata in front matter.
  Authenticate anything hosted, because a reading list is revealing.
Trap: A reading queue and its highlights are a detailed record of what you are thinking about, which is more revealing than most people expect. Authenticate before exposing it anywhere, and be deliberate about whether the highlights export includes your notes.
Done when: The bookmarklet saves from three different sites and reports clearly on one that blocks it. A shared URL from a phone saves through the endpoint. Export produces one Markdown file per document with highlights inline and front matter metadata. Hosted access requires authentication.
Do not build yet: Nothing after this.

### Out of scope (and why)
- Extraction quality at the level of the commercial product, which handles a long tail of sites through per-site work that never finishes.
- Mobile applications with offline reading, which is how most read-later use actually happens.
- Integrations with reading devices and their highlight formats, each of which is its own extraction problem.
- Access to material behind paywalls and logins, which this build does not attempt.

### README must contain
- That highlights are anchored by quoted text with context rather than by offsets, and why
- That re-extraction re-resolves highlights and reports the outcome before becoming current
- That stored articles are sanitised and make no outbound requests
- That archiving without reading is a first-class action
- That the Markdown export is the format designed to outlive this application

===== AGENTS.md =====
# Agent instructions · Readwise Reader indie build

- Read `README.md` and `BUILD_PLAN.md` before writing code. The stack is fixed: Node 22 with SQLite, Readability plus a sanitiser, Quoted text with prefix and suffix context, not offsets, A bookmarklet and a share endpoint, Markdown per document with highlights inline. Do not substitute.
- Work one phase at a time, in order. Do not start a phase until every "Done when" item of the previous one passes.
- Prefer the fewest moving parts that satisfy the step. No frameworks, services or dependencies the plan does not name.
- Secrets live in `.env`, never in source or logs. Keep `.env.example` current when a variable is introduced.
- Do not invent cryptography, security guarantees, APIs or compliance claims.
- Add a focused test for every destructive, security-sensitive or data-loss path the plan names.
- Run the project checks before declaring a phase complete, and record any deliberate shortcut in the README under "Tradeoffs".

## Known traps

- Storing raw fetched HTML and rendering it later means every time you open your own archive you load the publisher's scripts and trackers. Sanitise at capture: strip scripts, event handlers and remote references, so the stored copy is inert.
- Storing a character offset into the extracted text means every highlight breaks the moment the extraction changes by one character, which happens whenever the extractor improves or the publisher edits the page. Quote plus context resolves against new text and degrades gracefully when it cannot.
- Replacing an extraction without re-resolving highlights leaves them pointing at text that has moved, and the damage is silent and cumulative. Re-resolve as part of re-extraction, report the counts, and keep the old extraction so a bad one can be reverted.
- A read-later queue with no triage becomes a list of a thousand articles nobody opens, and the tool stops being used rather than the queue being cleared. Make archiving without reading a first-class action, since deciding not to read something is a legitimate outcome and the queue depends on it.
- Showing highlights at random from a large archive produces mostly forgettable ones and trains you to skip the review. Weight towards annotated highlights and recent reading, and cap the daily count so the review takes a minute rather than being abandoned.

===== BUILD_PLAN.md =====
# Build plan · Readwise Reader

Save a page from anywhere, extract a clean reader view, highlight and annotate, and have those highlights still point at the right text when the extractor improves, because they are anchored by quoted text and its surroundings rather than by a character offset.

Phases are in dependency order. Each ends in a "Done when" list; treat an unticked item as a blocker, not a note.

## Phase 1 · Save and extract

A saved page becomes readable text, and a page that cannot be extracted says so.

### Steps

1. Fetch with limits
   Timeout, streaming byte cap, robots respected. You are a well-behaved client saving for yourself.
   Files: `src/fetch.js`
2. Extract and sanitise
   Readability for the article, then strip scripts and remote references so the copy is inert.
   Files: `src/extract.js`

   ```sh
   npm install @mozilla/readability jsdom dompurify
   ```

3. Version every extraction
   So you know which extractor produced a stored copy, which matters in Phase 3.
   Files: `src/extract.js`

### Done when

- [ ] A saved article reads offline
- [ ] A stored article makes no outbound requests
- [ ] A JavaScript-only page is recorded as a failure with a reason
- [ ] Extractor version is stored with each extraction

### Watch out

- Storing raw fetched HTML and rendering it later means every time you open your own archive you load the publisher's scripts and trackers. Sanitise at capture: strip scripts, event handlers and remote references, so the stored copy is inert.

## Phase 2 · Highlights anchored to text, not to positions

A highlight knows what it selected, not where in a string it was.

### Steps

1. Store quote with context
   The selected text plus a short prefix and suffix. Enough context to disambiguate a repeated phrase.
   Files: `src/anchor.js`
2. Resolve by search
   Exact match first, then approximate. Record which strategy succeeded.
   Files: `src/anchor.js`
3. Handle the unresolvable case visibly
   Marked, still readable as a quote, never silently attached to the wrong place.
   Files: `src/anchor.js`

### Done when

- [ ] A highlight resolves correctly in stored text
- [ ] Inserting text before it does not move it
- [ ] A deleted quote is marked unresolvable rather than misplaced
- [ ] Resolution state is visible per highlight

### Watch out

- Storing a character offset into the extracted text means every highlight breaks the moment the extraction changes by one character, which happens whenever the extractor improves or the publisher edits the page. Quote plus context resolves against new text and degrades gracefully when it cannot.

## Phase 3 · Re-extraction, which is the point

Improving the extractor does not lose a year of highlights.

### Steps

1. Keep extractions as versions
   A document has several; one is current. Rollback is then trivial.
   Files: `src/extractions.js`
2. Re-resolve and report
   Counts by outcome, shown before the new extraction becomes current.
   Files: `src/reextract.js`
3. Support rollback
   A re-extraction that loses highlights should be reversible in one action.
   Files: `src/extractions.js`

### Done when

- [ ] Re-extraction reports resolution outcomes by count
- [ ] A worse extraction can be rolled back
- [ ] The previous extraction is retained
- [ ] A document without highlights re-extracts cleanly

### Watch out

- Replacing an extraction without re-resolving highlights leaves them pointing at text that has moved, and the damage is silent and cumulative. Re-resolve as part of re-extraction, report the counts, and keep the old extraction so a bad one can be reverted.

## Phase 4 · A queue you can actually clear

Saved articles are triaged rather than accumulating into a wall of guilt.

### Steps

1. Implement the three states
   Inbox, later, archive. One action between them, keyboard reachable.
   Files: `src/queue.js`
2. Estimate reading time
   From word count. It is what lets you pick something that fits the time you have.
   Files: `src/queue.js`
3. Make declining first class
   Archive without reading, recorded as such. It is the action that keeps the queue survivable.
   Files: `src/queue.js`

### Done when

- [ ] State changes are one action each
- [ ] Sorting by reading time works
- [ ] Archiving unread is recorded distinctly
- [ ] The oldest unread age is shown

### Watch out

- A read-later queue with no triage becomes a list of a thousand articles nobody opens, and the tool stops being used rather than the queue being cleared. Make archiving without reading a first-class action, since deciding not to read something is a legitimate outcome and the queue depends on it.

## Phase 5 · Highlight review

What you highlighted comes back, spaced out, so it is not written once and never seen again.

### Steps

1. Schedule and weight the selection
   Recency and annotation as weights, with a daily cap.
   Files: `src/review.js`
2. Send a digest
   Email with the highlights and their sources. This is where a review actually gets read.
   Files: `src/digest.js`
3. Let a highlight be retired
   Marked not useful and never shown again. Without it, the review degrades over time.
   Files: `src/review.js`

### Done when

- [ ] The daily review is capped
- [ ] Annotated highlights surface more often over a simulated month
- [ ] Each item links to its source
- [ ] A retired highlight stops recurring

### Watch out

- Showing highlights at random from a large archive produces mostly forgettable ones and trains you to skip the review. Weight towards annotated highlights and recent reading, and cap the daily count so the review takes a minute rather than being abandoned.

## Not in this build

- Extraction quality matching the commercial product.
- Mobile applications with offline reading.
- Reading device integrations and their highlight formats.
- Access to paywalled material.

## After v1, if you want it

- Full text search across every document and highlight
- Automatic tagging from article content, proposed rather than applied
- A weekly digest of what you saved and did not read, which is the honest metric

===== .env.example =====
# Copy to .env and fill in. Never commit .env; this file documents it.

# Required. Documents, extractions and highlights.
DB_PATH=./data/reader.db

# Required. Per-page fetch timeout. A slow page should fail rather than hold up a save.
FETCH_TIMEOUT_MS=10000

# Required. Streaming byte cap, enforced during download.
FETCH_MAX_BYTES=5000000

# Required. How much prefix and suffix is stored with a quote. Enough to disambiguate a repeated phrase.
ANCHOR_CONTEXT_CHARS=40

# Required. Highlights per daily review. Small enough that the review actually happens.
REVIEW_DAILY_CAP=5

# Required · secret. Used by the bookmarklet and the share endpoint. Scoped to saving only.
SAVE_TOKEN=replace-with-a-random-token

You are building a production product version of Readwise Reader.

Create the following project files first, then implement the application by following them. Keep the files updated as decisions change. Do not collapse this into a single README or prompt.

===== PRODUCT.md =====
# Readwise Reader · product brief

## Problem

Saving articles and highlighting is easy; reliable extraction, mobile/offline reading, sync, Kindle/import integrations, RSS/PDF support, and review loops are real work.

## Product outcome

A reading queue where highlights are anchored to what you selected rather than where it was, so the extractor can improve without losing anything, and where everything exports to Markdown you could read without this software.

## Target user

A builder who needs a maintainable product foundation, not a one-off demo.

## Required capabilities

- hosted app
- browser extension or share sheet
- parser/extractor
- database
- email or notification service
- optional OCR/PDF tools

## Explicit non-goals for v1

- Extraction quality matching the commercial product.
- Mobile applications with offline reading.
- Reading device integrations and their highlight formats.
- Access to paywalled material.
- excellent parsing
- mobile/offline apps
- Kindle integrations
- RSS/PDF workflows
- spaced repetition
- exports

## Success criteria

- A highlight on a repeated phrase resolves to the correct occurrence
- Re-extraction reports resolution outcomes before becoming current
- A stored article is confirmed to make no outbound requests
- The Markdown export includes highlights and front matter
- Hosted access is confirmed to require authentication

===== BRIEF.md =====
# Build brief · Readwise Reader

The one-shot brief this plan expands. `BUILD_PLAN.md` (or `MILESTONES.md`) is the same sequence broken into steps and checks; where the two disagree, the plan wins.

Build me a read-later queue where highlights survive the article being re-extracted to replace Readwise Reader. Build it in phases, in the order below.
Do not write the whole thing in one pass. Finish a phase, run its "Done when"
check, fix what fails, and only then start the next phase.

### Stack (fixed, do not substitute)
- Runtime: Node 22 with SQLite · one person's reading queue, where the interesting work is extraction and anchoring
- Extraction: Readability plus a sanitiser · the article without the navigation, and inert so opening your own archive runs nobody's scripts
- Anchoring: Quoted text with prefix and suffix context, not offsets · the single decision that determines whether highlights survive anything
- Capture: A bookmarklet and a share endpoint · no browser extension to maintain across browsers and their review processes
- Export: Markdown per document with highlights inline · highlights are the durable artefact and must outlive this application

### Data model (create this before Phase 1)
- documents(id, url, canonical_url, title, author, saved_at, published_at, state, word_count) · state is inbox, later, archived
- extractions(id, document_id, extractor_version, text, html, extracted_at, content_hash)
- highlights(id, document_id, quote, prefix, suffix, note, colour, created_at, resolved_offset, resolve_state)
- tags(id, name), document_tags(document_id, tag_id)
- reviews(highlight_id, last_shown_at, next_due_at, times_shown)
- A highlight belongs to a document, not to an extraction. That is what lets the extraction be replaced.

### Phase 1 · Save and extract
Build: A save endpoint taking a URL. Fetch with a timeout and a byte cap, extract the article,
  sanitise it so it makes no outbound requests, and store both text and cleaned HTML with
  the extractor version. Record failures as failures.
Trap: Storing raw fetched HTML and rendering it later means every time you open your own archive you load the publisher's scripts and trackers. Sanitise at capture: strip scripts, event handlers and remote references, so the stored copy is inert.
Done when: A saved article is readable offline. Opening a stored article makes no outbound network request, verified in developer tools. A page requiring JavaScript is recorded as a failed extraction with a reason rather than an empty document. The extractor version is stored with every extraction.
Do not build yet: No highlights, tags or review.

### Phase 2 · Highlights anchored to text, not to positions
Build: On highlight, store the quoted text plus a short prefix and suffix from the surrounding
  content. Resolve a highlight by searching for that quote with its context. Record whether
  resolution succeeded, was approximate, or failed.
Trap: Storing a character offset into the extracted text means every highlight breaks the moment the extraction changes by one character, which happens whenever the extractor improves or the publisher edits the page. Quote plus context resolves against new text and degrades gracefully when it cannot.
Done when: A highlight resolves correctly in the stored text. Inserting a paragraph before it in the extraction still resolves it to the same words. A highlight whose text has been deleted from the article is marked unresolvable rather than silently landing somewhere else. Resolution state is visible per highlight.
Do not build yet: No re-extraction or review yet.

### Phase 3 · Re-extraction, which is the point
Build: Re-extract a document with a newer extractor, keeping the previous extraction. Re-resolve
  every highlight against the new text and report the outcome. Let a failed re-extraction
  be rolled back to the previous version.
Trap: Replacing an extraction without re-resolving highlights leaves them pointing at text that has moved, and the damage is silent and cumulative. Re-resolve as part of re-extraction, report the counts, and keep the old extraction so a bad one can be reverted.
Done when: Re-extracting a document re-resolves every highlight and reports how many resolved exactly, approximately and not at all. A re-extraction that resolves fewer highlights can be rolled back to the previous version. The previous extraction is retained. A document with no highlights re-extracts without incident.
Do not build yet: No tags, reading queue or export yet.

### Phase 4 · A queue you can actually clear
Build: An inbox, a later list and an archive, with one action to move between them. Sort by
  estimated reading time as well as by date. Show the age of the oldest unread item
  honestly rather than hiding it.
Trap: A read-later queue with no triage becomes a list of a thousand articles nobody opens, and the tool stops being used rather than the queue being cleared. Make archiving without reading a first-class action, since deciding not to read something is a legitimate outcome and the queue depends on it.
Done when: Moving between inbox, later and archive is one action each. The list can be sorted by estimated reading time. Archiving without reading is as easy as reading, and is recorded distinctly. The age of the oldest unread item is shown.
Do not build yet: No review or export yet.

### Phase 5 · Highlight review
Build: A review that surfaces highlights on a schedule, weighted towards recent saves and
  highlights you have annotated. A daily email digest. Every review shows the source
  article and links back to it.
Trap: Showing highlights at random from a large archive produces mostly forgettable ones and trains you to skip the review. Weight towards annotated highlights and recent reading, and cap the daily count so the review takes a minute rather than being abandoned.
Done when: A review surfaces a capped number of highlights daily. Annotated highlights appear more often than unannotated ones, verified over a simulated month. Each review item links back to its source article. Marking a highlight as not useful stops it recurring.
Do not build yet: Export and capture tooling come last.

### Phase 6 · Capture everywhere, and export
Build: A bookmarklet and an endpoint that accepts a shared URL from a phone. Export every
  document as Markdown with its highlights inline and its metadata in front matter.
  Authenticate anything hosted, because a reading list is revealing.
Trap: A reading queue and its highlights are a detailed record of what you are thinking about, which is more revealing than most people expect. Authenticate before exposing it anywhere, and be deliberate about whether the highlights export includes your notes.
Done when: The bookmarklet saves from three different sites and reports clearly on one that blocks it. A shared URL from a phone saves through the endpoint. Export produces one Markdown file per document with highlights inline and front matter metadata. Hosted access requires authentication.
Do not build yet: Nothing after this.

### Out of scope (and why)
- Extraction quality at the level of the commercial product, which handles a long tail of sites through per-site work that never finishes.
- Mobile applications with offline reading, which is how most read-later use actually happens.
- Integrations with reading devices and their highlight formats, each of which is its own extraction problem.
- Access to material behind paywalls and logins, which this build does not attempt.

### README must contain
- That highlights are anchored by quoted text with context rather than by offsets, and why
- That re-extraction re-resolves highlights and reports the outcome before becoming current
- That stored articles are sanitised and make no outbound requests
- That archiving without reading is a first-class action
- That the Markdown export is the format designed to outlive this application

===== ARCHITECTURE.md =====
# Architecture · Readwise Reader

## Stack

| Part | Choice | Why |
| --- | --- | --- |
| Runtime | Node 22 with SQLite | one person's reading queue, where the interesting work is extraction and anchoring |
| Extraction | Readability plus a sanitiser | the article without the navigation, and inert so opening your own archive runs nobody's scripts |
| Anchoring | Quoted text with prefix and suffix context, not offsets | the single decision that determines whether highlights survive anything |
| Capture | A bookmarklet and a share endpoint | no browser extension to maintain across browsers and their review processes |
| Export | Markdown per document with highlights inline | highlights are the durable artefact and must outlive this application |

## Modules

Each module has one owner concern and a documented way to replace it.

| Module | Owns | How to replace it |
| --- | --- | --- |
| Fetch and extract | Turning a URL into inert readable text. | A headless browser would handle more sites at a large cost in speed. The sanitisation must survive any change. |
| Anchor | Locating a highlight in text. | This is the module the whole design exists around. Offsets are the alternative and they are why highlights break elsewhere. |
| Extractions | Versioned copies of a document. | Keeping the previous version is what makes re-extraction safe rather than destructive. |
| Queue | Triage states. | Simple, and the declining-is-first-class rule is the part that keeps it usable. |
| Review | Resurfacing highlights. | Any scheduling approach, provided it is weighted and capped rather than random and unbounded. |

## Configuration

Every runtime setting is an environment variable documented in `.env.example`, validated at startup, with a safe local default wherever one exists.

- `DB_PATH` · required · Documents, extractions and highlights.
- `FETCH_TIMEOUT_MS` · required · Per-page fetch timeout. A slow page should fail rather than hold up a save.
- `FETCH_MAX_BYTES` · required · Streaming byte cap, enforced during download.
- `ANCHOR_CONTEXT_CHARS` · required · How much prefix and suffix is stored with a quote. Enough to disambiguate a repeated phrase.
- `REVIEW_DAILY_CAP` · required · Highlights per daily review. Small enough that the review actually happens.
- `SAVE_TOKEN` · required, secret · Used by the bookmarklet and the share endpoint. Scoped to saving only.

## Production baseline

- Security: least privilege, input validation at every boundary, secret redaction in logs, rate limits on abuse-prone paths, no invented security primitives.
- Data: explicit schema and migrations, transactional writes where integrity matters, backup and restore procedures that have been exercised.
- Integrations: adapters around third-party providers, idempotent webhook or job processing, bounded retries, timeouts.
- Observability: structured logs with request or operation ids, an error-tracking hook, and health and readiness checks where a server exists.
- Quality: unit tests for domain rules, integration tests at module boundaries, one end-to-end test of the critical path.

## Decision records

For each dependency in the stack table, keep a short note: why it was chosen, its failure mode, and how it is replaced. Do not add infrastructure until a requirement in `PRODUCT.md` justifies it.

===== AGENTS.md =====
# Agent instructions · Readwise Reader product build

- Read `PRODUCT.md` and `ARCHITECTURE.md` before changing code. The stack is fixed: Node 22 with SQLite, Readability plus a sanitiser, Quoted text with prefix and suffix context, not offsets, A bookmarklet and a share endpoint, Markdown per document with highlights inline.
- Implement milestone by milestone from `MILESTONES.md`; keep each change reviewable and leave the application runnable at every commit.
- Treat authentication, payments, encryption, imports, webhooks and destructive actions as high-risk boundaries when present.
- Never invent cryptography or silently weaken a requirement to make a check pass.
- Put every external service behind an interface with a deterministic fake for tests.
- Add migrations and rollback or recovery notes for every persistent data change.
- Log useful operational context without credentials, tokens, passwords or personal data.
- Update documentation and run every check before completing a milestone.

## Known traps

- Storing raw fetched HTML and rendering it later means every time you open your own archive you load the publisher's scripts and trackers. Sanitise at capture: strip scripts, event handlers and remote references, so the stored copy is inert.
- Storing a character offset into the extracted text means every highlight breaks the moment the extraction changes by one character, which happens whenever the extractor improves or the publisher edits the page. Quote plus context resolves against new text and degrades gracefully when it cannot.
- Replacing an extraction without re-resolving highlights leaves them pointing at text that has moved, and the damage is silent and cumulative. Re-resolve as part of re-extraction, report the counts, and keep the old extraction so a bad one can be reverted.
- A read-later queue with no triage becomes a list of a thousand articles nobody opens, and the tool stops being used rather than the queue being cleared. Make archiving without reading a first-class action, since deciding not to read something is a legitimate outcome and the queue depends on it.
- Showing highlights at random from a large archive produces mostly forgettable ones and trains you to skip the review. Weight towards annotated highlights and recent reading, and cap the daily count so the review takes a minute rather than being abandoned.
- A reading queue and its highlights are a detailed record of what you are thinking about, which is more revealing than most people expect. Authenticate before exposing it anywhere, and be deliberate about whether the highlights export includes your notes.

===== MILESTONES.md =====
# Delivery milestones · Readwise Reader

Estimated effort: **multi-day** for the indie phases; the production-only milestones add the trust and operability layer.

## M1 · Save and extract

A saved page becomes readable text, and a page that cannot be extracted says so.

### Steps

1. Fetch with limits
   Timeout, streaming byte cap, robots respected. You are a well-behaved client saving for yourself.
   Files: `src/fetch.js`
2. Extract and sanitise
   Readability for the article, then strip scripts and remote references so the copy is inert.
   Files: `src/extract.js`

   ```sh
   npm install @mozilla/readability jsdom dompurify
   ```

3. Version every extraction
   So you know which extractor produced a stored copy, which matters in Phase 3.
   Files: `src/extract.js`

### Done when

- [ ] A saved article reads offline
- [ ] A stored article makes no outbound requests
- [ ] A JavaScript-only page is recorded as a failure with a reason
- [ ] Extractor version is stored with each extraction

### Watch out

- Storing raw fetched HTML and rendering it later means every time you open your own archive you load the publisher's scripts and trackers. Sanitise at capture: strip scripts, event handlers and remote references, so the stored copy is inert.

## M2 · Highlights anchored to text, not to positions

A highlight knows what it selected, not where in a string it was.

### Steps

1. Store quote with context
   The selected text plus a short prefix and suffix. Enough context to disambiguate a repeated phrase.
   Files: `src/anchor.js`
2. Resolve by search
   Exact match first, then approximate. Record which strategy succeeded.
   Files: `src/anchor.js`
3. Handle the unresolvable case visibly
   Marked, still readable as a quote, never silently attached to the wrong place.
   Files: `src/anchor.js`

### Done when

- [ ] A highlight resolves correctly in stored text
- [ ] Inserting text before it does not move it
- [ ] A deleted quote is marked unresolvable rather than misplaced
- [ ] Resolution state is visible per highlight

### Watch out

- Storing a character offset into the extracted text means every highlight breaks the moment the extraction changes by one character, which happens whenever the extractor improves or the publisher edits the page. Quote plus context resolves against new text and degrades gracefully when it cannot.

## M3 · Re-extraction, which is the point

Improving the extractor does not lose a year of highlights.

### Steps

1. Keep extractions as versions
   A document has several; one is current. Rollback is then trivial.
   Files: `src/extractions.js`
2. Re-resolve and report
   Counts by outcome, shown before the new extraction becomes current.
   Files: `src/reextract.js`
3. Support rollback
   A re-extraction that loses highlights should be reversible in one action.
   Files: `src/extractions.js`

### Done when

- [ ] Re-extraction reports resolution outcomes by count
- [ ] A worse extraction can be rolled back
- [ ] The previous extraction is retained
- [ ] A document without highlights re-extracts cleanly

### Watch out

- Replacing an extraction without re-resolving highlights leaves them pointing at text that has moved, and the damage is silent and cumulative. Re-resolve as part of re-extraction, report the counts, and keep the old extraction so a bad one can be reverted.

## M4 · A queue you can actually clear

Saved articles are triaged rather than accumulating into a wall of guilt.

### Steps

1. Implement the three states
   Inbox, later, archive. One action between them, keyboard reachable.
   Files: `src/queue.js`
2. Estimate reading time
   From word count. It is what lets you pick something that fits the time you have.
   Files: `src/queue.js`
3. Make declining first class
   Archive without reading, recorded as such. It is the action that keeps the queue survivable.
   Files: `src/queue.js`

### Done when

- [ ] State changes are one action each
- [ ] Sorting by reading time works
- [ ] Archiving unread is recorded distinctly
- [ ] The oldest unread age is shown

### Watch out

- A read-later queue with no triage becomes a list of a thousand articles nobody opens, and the tool stops being used rather than the queue being cleared. Make archiving without reading a first-class action, since deciding not to read something is a legitimate outcome and the queue depends on it.

## M5 · Highlight review

What you highlighted comes back, spaced out, so it is not written once and never seen again.

### Steps

1. Schedule and weight the selection
   Recency and annotation as weights, with a daily cap.
   Files: `src/review.js`
2. Send a digest
   Email with the highlights and their sources. This is where a review actually gets read.
   Files: `src/digest.js`
3. Let a highlight be retired
   Marked not useful and never shown again. Without it, the review degrades over time.
   Files: `src/review.js`

### Done when

- [ ] The daily review is capped
- [ ] Annotated highlights surface more often over a simulated month
- [ ] Each item links to its source
- [ ] A retired highlight stops recurring

### Watch out

- Showing highlights at random from a large archive produces mostly forgettable ones and trains you to skip the review. Weight towards annotated highlights and recent reading, and cap the daily count so the review takes a minute rather than being abandoned.

## M6 · Capture everywhere, and export (production only)

Saving takes one action from anywhere, and nothing is trapped.

### Steps

1. Ship a bookmarklet and a share endpoint
   No extension to maintain. The endpoint covers the phone case.
   Files: `public/bookmarklet.js`, `src/save.js`
2. Export Markdown with highlights inline
   One file per document. This is the format that outlives the application.
   Files: `src/export.js`
3. Authenticate hosted access
   A reading list is personal. Put a login in front before it leaves your machine.
   Files: `src/auth.js`

### Done when

- [ ] The bookmarklet saves from three sites and reports on a blocked one
- [ ] A shared URL from a phone saves
- [ ] Export produces Markdown with inline highlights and front matter
- [ ] Hosted access requires authentication

### Watch out

- A reading queue and its highlights are a detailed record of what you are thinking about, which is more revealing than most people expect. Authenticate before exposing it anywhere, and be deliberate about whether the highlights export includes your notes.

===== OPERATIONS.md =====
# Operations · Readwise Reader

## Backup

The database holds documents, extractions and highlights. Highlights are the irreplaceable part; extractions can be refetched while the source still exists, which is exactly the assumption archiving exists to avoid relying on.

## Restore

Restore the database and open a document whose source site is gone. That is the test that the archive is real rather than a bookmark list.

Do a restore drill before the first real user, and write the date here when it passes.

## Monitoring

Extraction failure rate and unresolvable highlight count. A jump in the second after a re-extraction means the new extractor changed text more than expected and should be rolled back.

## Incident checklist

A re-extraction that silently moves highlights is the failure that matters, and reporting outcomes before making it current is the control. If highlights drift anyway, the anchoring context is too short.

1. Contain the issue without destroying evidence or user data.
2. Record the timeline and affected scope.
3. Rotate exposed secrets and revoke compromised sessions or credentials.
4. Restore from a verified backup when needed.
5. Document the root cause, the remediation and the regression test.

## Release gate

- [ ] A highlight on a repeated phrase resolves to the correct occurrence
- [ ] Re-extraction reports resolution outcomes before becoming current
- [ ] A stored article is confirmed to make no outbound requests
- [ ] The Markdown export includes highlights and front matter
- [ ] Hosted access is confirmed to require authentication

## Launch constraint

Do not market omitted Readwise Reader capabilities as implemented. The non-goals in `PRODUCT.md` remain user-visible limitations until they are deliberately delivered.

===== .env.example =====
# Copy to .env and fill in. Never commit .env; this file documents it.

# Required. Documents, extractions and highlights.
DB_PATH=./data/reader.db

# Required. Per-page fetch timeout. A slow page should fail rather than hold up a save.
FETCH_TIMEOUT_MS=10000

# Required. Streaming byte cap, enforced during download.
FETCH_MAX_BYTES=5000000

# Required. How much prefix and suffix is stored with a quote. Enough to disambiguate a repeated phrase.
ANCHOR_CONTEXT_CHARS=40

# Required. Highlights per daily review. Small enough that the review actually happens.
REVIEW_DAILY_CAP=5

# Required · secret. Used by the bookmarklet and the share endpoint. Scoped to saving only.
SAVE_TOKEN=replace-with-a-random-token

# Readwise Reader · indie build

Save a page from anywhere, extract a clean reader view, highlight and annotate, and have those highlights still point at the right text when the extractor improves, because they are anchored by quoted text and its surroundings rather than by a character offset.

Estimated effort: **multi-day**. Work `BUILD_PLAN.md` top to bottom · every phase ends in a check that has to pass before the next one starts.

## Stack

| Part | Choice | Why |
| --- | --- | --- |
| Runtime | Node 22 with SQLite | one person's reading queue, where the interesting work is extraction and anchoring |
| Extraction | Readability plus a sanitiser | the article without the navigation, and inert so opening your own archive runs nobody's scripts |
| Anchoring | Quoted text with prefix and suffix context, not offsets | the single decision that determines whether highlights survive anything |
| Capture | A bookmarklet and a share endpoint | no browser extension to maintain across browsers and their review processes |
| Export | Markdown per document with highlights inline | highlights are the durable artefact and must outlive this application |

## Before you start

Have every one of these ready. The plan assumes them from step one.

- [ ] **Node.js 22 or newer** · free
  - Why: Runs the server, the extractor and the review job.
  - Get it: nodejs.org, or nvm install 22.
  - Verify: node --version prints v22 or higher
- [ ] **Five awkward article URLs** · free
  - Why: Phase 1's extraction and Phase 3's re-extraction are both only meaningful against real pages: a clean article, a JavaScript-only page, one with an aggressive content security policy, a very long piece and one that gets edited.
  - Get it: Collect them from your own reading and keep them as fixtures with their expected outcomes.
  - Verify: All five are saved with expected extraction results recorded
- [ ] **An article with repeated phrases** · free
  - Why: Phase 2's anchoring is only tested by a document where the same words appear twice, which is what the surrounding context exists to disambiguate.
  - Get it: Any long article will contain repeated phrases. Highlight one deliberately and check it resolves to the right occurrence.
  - Verify: A highlight on a repeated phrase resolves to the correct occurrence
- [ ] **An email provider for the review digest** (optional) · free tiers cover this
  - Why: Phase 5 only. Everything else works without delivery.
  - Get it: A transactional provider with a verified sending domain.
  - Verify: A digest arrives in an inbox
- [ ] **Somewhere to run it, for the product phase** (optional) · 5 to 10 US dollars a month
  - Why: Saving from a phone needs a reachable endpoint, and the review digest needs something running on a schedule.
  - Get it: A small virtual server, or a machine at home behind a tunnel. Authenticate before exposing it.
  - Verify: A save from a phone reaches the endpoint over HTTPS

## Quick start

```sh
npm install @mozilla/readability jsdom dompurify
```

Then copy `.env.example` to `.env` and fill in the values it documents.

## Honest limits

This build deliberately does not replace:

- Extraction quality matching the commercial product.
- Mobile applications with offline reading.
- Reading device integrations and their highlight formats.
- Access to paywalled material.
- excellent parsing
- mobile/offline apps
- Kindle integrations
- RSS/PDF workflows
- spaced repetition
- exports

If one of those is essential to you, that is the reason to keep paying for Readwise Reader, and the README should say so rather than pretend.

# Build brief · Readwise Reader

The one-shot brief this plan expands. `BUILD_PLAN.md` (or `MILESTONES.md`) is the same sequence broken into steps and checks; where the two disagree, the plan wins.

Build me a read-later queue where highlights survive the article being re-extracted to replace Readwise Reader. Build it in phases, in the order below.
Do not write the whole thing in one pass. Finish a phase, run its "Done when"
check, fix what fails, and only then start the next phase.

### Stack (fixed, do not substitute)
- Runtime: Node 22 with SQLite · one person's reading queue, where the interesting work is extraction and anchoring
- Extraction: Readability plus a sanitiser · the article without the navigation, and inert so opening your own archive runs nobody's scripts
- Anchoring: Quoted text with prefix and suffix context, not offsets · the single decision that determines whether highlights survive anything
- Capture: A bookmarklet and a share endpoint · no browser extension to maintain across browsers and their review processes
- Export: Markdown per document with highlights inline · highlights are the durable artefact and must outlive this application

### Data model (create this before Phase 1)
- documents(id, url, canonical_url, title, author, saved_at, published_at, state, word_count) · state is inbox, later, archived
- extractions(id, document_id, extractor_version, text, html, extracted_at, content_hash)
- highlights(id, document_id, quote, prefix, suffix, note, colour, created_at, resolved_offset, resolve_state)
- tags(id, name), document_tags(document_id, tag_id)
- reviews(highlight_id, last_shown_at, next_due_at, times_shown)
- A highlight belongs to a document, not to an extraction. That is what lets the extraction be replaced.

### Phase 1 · Save and extract
Build: A save endpoint taking a URL. Fetch with a timeout and a byte cap, extract the article,
  sanitise it so it makes no outbound requests, and store both text and cleaned HTML with
  the extractor version. Record failures as failures.
Trap: Storing raw fetched HTML and rendering it later means every time you open your own archive you load the publisher's scripts and trackers. Sanitise at capture: strip scripts, event handlers and remote references, so the stored copy is inert.
Done when: A saved article is readable offline. Opening a stored article makes no outbound network request, verified in developer tools. A page requiring JavaScript is recorded as a failed extraction with a reason rather than an empty document. The extractor version is stored with every extraction.
Do not build yet: No highlights, tags or review.

### Phase 2 · Highlights anchored to text, not to positions
Build: On highlight, store the quoted text plus a short prefix and suffix from the surrounding
  content. Resolve a highlight by searching for that quote with its context. Record whether
  resolution succeeded, was approximate, or failed.
Trap: Storing a character offset into the extracted text means every highlight breaks the moment the extraction changes by one character, which happens whenever the extractor improves or the publisher edits the page. Quote plus context resolves against new text and degrades gracefully when it cannot.
Done when: A highlight resolves correctly in the stored text. Inserting a paragraph before it in the extraction still resolves it to the same words. A highlight whose text has been deleted from the article is marked unresolvable rather than silently landing somewhere else. Resolution state is visible per highlight.
Do not build yet: No re-extraction or review yet.

### Phase 3 · Re-extraction, which is the point
Build: Re-extract a document with a newer extractor, keeping the previous extraction. Re-resolve
  every highlight against the new text and report the outcome. Let a failed re-extraction
  be rolled back to the previous version.
Trap: Replacing an extraction without re-resolving highlights leaves them pointing at text that has moved, and the damage is silent and cumulative. Re-resolve as part of re-extraction, report the counts, and keep the old extraction so a bad one can be reverted.
Done when: Re-extracting a document re-resolves every highlight and reports how many resolved exactly, approximately and not at all. A re-extraction that resolves fewer highlights can be rolled back to the previous version. The previous extraction is retained. A document with no highlights re-extracts without incident.
Do not build yet: No tags, reading queue or export yet.

### Phase 4 · A queue you can actually clear
Build: An inbox, a later list and an archive, with one action to move between them. Sort by
  estimated reading time as well as by date. Show the age of the oldest unread item
  honestly rather than hiding it.
Trap: A read-later queue with no triage becomes a list of a thousand articles nobody opens, and the tool stops being used rather than the queue being cleared. Make archiving without reading a first-class action, since deciding not to read something is a legitimate outcome and the queue depends on it.
Done when: Moving between inbox, later and archive is one action each. The list can be sorted by estimated reading time. Archiving without reading is as easy as reading, and is recorded distinctly. The age of the oldest unread item is shown.
Do not build yet: No review or export yet.

### Phase 5 · Highlight review
Build: A review that surfaces highlights on a schedule, weighted towards recent saves and
  highlights you have annotated. A daily email digest. Every review shows the source
  article and links back to it.
Trap: Showing highlights at random from a large archive produces mostly forgettable ones and trains you to skip the review. Weight towards annotated highlights and recent reading, and cap the daily count so the review takes a minute rather than being abandoned.
Done when: A review surfaces a capped number of highlights daily. Annotated highlights appear more often than unannotated ones, verified over a simulated month. Each review item links back to its source article. Marking a highlight as not useful stops it recurring.
Do not build yet: Export and capture tooling come last.

### Phase 6 · Capture everywhere, and export
Build: A bookmarklet and an endpoint that accepts a shared URL from a phone. Export every
  document as Markdown with its highlights inline and its metadata in front matter.
  Authenticate anything hosted, because a reading list is revealing.
Trap: A reading queue and its highlights are a detailed record of what you are thinking about, which is more revealing than most people expect. Authenticate before exposing it anywhere, and be deliberate about whether the highlights export includes your notes.
Done when: The bookmarklet saves from three different sites and reports clearly on one that blocks it. A shared URL from a phone saves through the endpoint. Export produces one Markdown file per document with highlights inline and front matter metadata. Hosted access requires authentication.
Do not build yet: Nothing after this.

### Out of scope (and why)
- Extraction quality at the level of the commercial product, which handles a long tail of sites through per-site work that never finishes.
- Mobile applications with offline reading, which is how most read-later use actually happens.
- Integrations with reading devices and their highlight formats, each of which is its own extraction problem.
- Access to material behind paywalls and logins, which this build does not attempt.

### README must contain
- That highlights are anchored by quoted text with context rather than by offsets, and why
- That re-extraction re-resolves highlights and reports the outcome before becoming current
- That stored articles are sanitised and make no outbound requests
- That archiving without reading is a first-class action
- That the Markdown export is the format designed to outlive this application

# Agent instructions · Readwise Reader indie build

- Read `README.md` and `BUILD_PLAN.md` before writing code. The stack is fixed: Node 22 with SQLite, Readability plus a sanitiser, Quoted text with prefix and suffix context, not offsets, A bookmarklet and a share endpoint, Markdown per document with highlights inline. Do not substitute.
- Work one phase at a time, in order. Do not start a phase until every "Done when" item of the previous one passes.
- Prefer the fewest moving parts that satisfy the step. No frameworks, services or dependencies the plan does not name.
- Secrets live in `.env`, never in source or logs. Keep `.env.example` current when a variable is introduced.
- Do not invent cryptography, security guarantees, APIs or compliance claims.
- Add a focused test for every destructive, security-sensitive or data-loss path the plan names.
- Run the project checks before declaring a phase complete, and record any deliberate shortcut in the README under "Tradeoffs".

## Known traps

- Storing raw fetched HTML and rendering it later means every time you open your own archive you load the publisher's scripts and trackers. Sanitise at capture: strip scripts, event handlers and remote references, so the stored copy is inert.
- Storing a character offset into the extracted text means every highlight breaks the moment the extraction changes by one character, which happens whenever the extractor improves or the publisher edits the page. Quote plus context resolves against new text and degrades gracefully when it cannot.
- Replacing an extraction without re-resolving highlights leaves them pointing at text that has moved, and the damage is silent and cumulative. Re-resolve as part of re-extraction, report the counts, and keep the old extraction so a bad one can be reverted.
- A read-later queue with no triage becomes a list of a thousand articles nobody opens, and the tool stops being used rather than the queue being cleared. Make archiving without reading a first-class action, since deciding not to read something is a legitimate outcome and the queue depends on it.
- Showing highlights at random from a large archive produces mostly forgettable ones and trains you to skip the review. Weight towards annotated highlights and recent reading, and cap the daily count so the review takes a minute rather than being abandoned.

# Build plan · Readwise Reader

Save a page from anywhere, extract a clean reader view, highlight and annotate, and have those highlights still point at the right text when the extractor improves, because they are anchored by quoted text and its surroundings rather than by a character offset.

Phases are in dependency order. Each ends in a "Done when" list; treat an unticked item as a blocker, not a note.

## Phase 1 · Save and extract

A saved page becomes readable text, and a page that cannot be extracted says so.

### Steps

1. Fetch with limits
   Timeout, streaming byte cap, robots respected. You are a well-behaved client saving for yourself.
   Files: `src/fetch.js`
2. Extract and sanitise
   Readability for the article, then strip scripts and remote references so the copy is inert.
   Files: `src/extract.js`

   ```sh
   npm install @mozilla/readability jsdom dompurify
   ```

3. Version every extraction
   So you know which extractor produced a stored copy, which matters in Phase 3.
   Files: `src/extract.js`

### Done when

- [ ] A saved article reads offline
- [ ] A stored article makes no outbound requests
- [ ] A JavaScript-only page is recorded as a failure with a reason
- [ ] Extractor version is stored with each extraction

### Watch out

- Storing raw fetched HTML and rendering it later means every time you open your own archive you load the publisher's scripts and trackers. Sanitise at capture: strip scripts, event handlers and remote references, so the stored copy is inert.

## Phase 2 · Highlights anchored to text, not to positions

A highlight knows what it selected, not where in a string it was.

### Steps

1. Store quote with context
   The selected text plus a short prefix and suffix. Enough context to disambiguate a repeated phrase.
   Files: `src/anchor.js`
2. Resolve by search
   Exact match first, then approximate. Record which strategy succeeded.
   Files: `src/anchor.js`
3. Handle the unresolvable case visibly
   Marked, still readable as a quote, never silently attached to the wrong place.
   Files: `src/anchor.js`

### Done when

- [ ] A highlight resolves correctly in stored text
- [ ] Inserting text before it does not move it
- [ ] A deleted quote is marked unresolvable rather than misplaced
- [ ] Resolution state is visible per highlight

### Watch out

- Storing a character offset into the extracted text means every highlight breaks the moment the extraction changes by one character, which happens whenever the extractor improves or the publisher edits the page. Quote plus context resolves against new text and degrades gracefully when it cannot.

## Phase 3 · Re-extraction, which is the point

Improving the extractor does not lose a year of highlights.

### Steps

1. Keep extractions as versions
   A document has several; one is current. Rollback is then trivial.
   Files: `src/extractions.js`
2. Re-resolve and report
   Counts by outcome, shown before the new extraction becomes current.
   Files: `src/reextract.js`
3. Support rollback
   A re-extraction that loses highlights should be reversible in one action.
   Files: `src/extractions.js`

### Done when

- [ ] Re-extraction reports resolution outcomes by count
- [ ] A worse extraction can be rolled back
- [ ] The previous extraction is retained
- [ ] A document without highlights re-extracts cleanly

### Watch out

- Replacing an extraction without re-resolving highlights leaves them pointing at text that has moved, and the damage is silent and cumulative. Re-resolve as part of re-extraction, report the counts, and keep the old extraction so a bad one can be reverted.

## Phase 4 · A queue you can actually clear

Saved articles are triaged rather than accumulating into a wall of guilt.

### Steps

1. Implement the three states
   Inbox, later, archive. One action between them, keyboard reachable.
   Files: `src/queue.js`
2. Estimate reading time
   From word count. It is what lets you pick something that fits the time you have.
   Files: `src/queue.js`
3. Make declining first class
   Archive without reading, recorded as such. It is the action that keeps the queue survivable.
   Files: `src/queue.js`

### Done when

- [ ] State changes are one action each
- [ ] Sorting by reading time works
- [ ] Archiving unread is recorded distinctly
- [ ] The oldest unread age is shown

### Watch out

- A read-later queue with no triage becomes a list of a thousand articles nobody opens, and the tool stops being used rather than the queue being cleared. Make archiving without reading a first-class action, since deciding not to read something is a legitimate outcome and the queue depends on it.

## Phase 5 · Highlight review

What you highlighted comes back, spaced out, so it is not written once and never seen again.

### Steps

1. Schedule and weight the selection
   Recency and annotation as weights, with a daily cap.
   Files: `src/review.js`
2. Send a digest
   Email with the highlights and their sources. This is where a review actually gets read.
   Files: `src/digest.js`
3. Let a highlight be retired
   Marked not useful and never shown again. Without it, the review degrades over time.
   Files: `src/review.js`

### Done when

- [ ] The daily review is capped
- [ ] Annotated highlights surface more often over a simulated month
- [ ] Each item links to its source
- [ ] A retired highlight stops recurring

### Watch out

- Showing highlights at random from a large archive produces mostly forgettable ones and trains you to skip the review. Weight towards annotated highlights and recent reading, and cap the daily count so the review takes a minute rather than being abandoned.

## Not in this build

- Extraction quality matching the commercial product.
- Mobile applications with offline reading.
- Reading device integrations and their highlight formats.
- Access to paywalled material.

## After v1, if you want it

- Full text search across every document and highlight
- Automatic tagging from article content, proposed rather than applied
- A weekly digest of what you saved and did not read, which is the honest metric

# Copy to .env and fill in. Never commit .env; this file documents it.

# Required. Documents, extractions and highlights.
DB_PATH=./data/reader.db

# Required. Per-page fetch timeout. A slow page should fail rather than hold up a save.
FETCH_TIMEOUT_MS=10000

# Required. Streaming byte cap, enforced during download.
FETCH_MAX_BYTES=5000000

# Required. How much prefix and suffix is stored with a quote. Enough to disambiguate a repeated phrase.
ANCHOR_CONTEXT_CHARS=40

# Required. Highlights per daily review. Small enough that the review actually happens.
REVIEW_DAILY_CAP=5

# Required · secret. Used by the bookmarklet and the share endpoint. Scoped to saving only.
SAVE_TOKEN=replace-with-a-random-token

# Readwise Reader · product brief

## Problem

Saving articles and highlighting is easy; reliable extraction, mobile/offline reading, sync, Kindle/import integrations, RSS/PDF support, and review loops are real work.

## Product outcome

A reading queue where highlights are anchored to what you selected rather than where it was, so the extractor can improve without losing anything, and where everything exports to Markdown you could read without this software.

## Target user

A builder who needs a maintainable product foundation, not a one-off demo.

## Required capabilities

- hosted app
- browser extension or share sheet
- parser/extractor
- database
- email or notification service
- optional OCR/PDF tools

## Explicit non-goals for v1

- Extraction quality matching the commercial product.
- Mobile applications with offline reading.
- Reading device integrations and their highlight formats.
- Access to paywalled material.
- excellent parsing
- mobile/offline apps
- Kindle integrations
- RSS/PDF workflows
- spaced repetition
- exports

## Success criteria

- A highlight on a repeated phrase resolves to the correct occurrence
- Re-extraction reports resolution outcomes before becoming current
- A stored article is confirmed to make no outbound requests
- The Markdown export includes highlights and front matter
- Hosted access is confirmed to require authentication

# Build brief · Readwise Reader

The one-shot brief this plan expands. `BUILD_PLAN.md` (or `MILESTONES.md`) is the same sequence broken into steps and checks; where the two disagree, the plan wins.

Build me a read-later queue where highlights survive the article being re-extracted to replace Readwise Reader. Build it in phases, in the order below.
Do not write the whole thing in one pass. Finish a phase, run its "Done when"
check, fix what fails, and only then start the next phase.

### Stack (fixed, do not substitute)
- Runtime: Node 22 with SQLite · one person's reading queue, where the interesting work is extraction and anchoring
- Extraction: Readability plus a sanitiser · the article without the navigation, and inert so opening your own archive runs nobody's scripts
- Anchoring: Quoted text with prefix and suffix context, not offsets · the single decision that determines whether highlights survive anything
- Capture: A bookmarklet and a share endpoint · no browser extension to maintain across browsers and their review processes
- Export: Markdown per document with highlights inline · highlights are the durable artefact and must outlive this application

### Data model (create this before Phase 1)
- documents(id, url, canonical_url, title, author, saved_at, published_at, state, word_count) · state is inbox, later, archived
- extractions(id, document_id, extractor_version, text, html, extracted_at, content_hash)
- highlights(id, document_id, quote, prefix, suffix, note, colour, created_at, resolved_offset, resolve_state)
- tags(id, name), document_tags(document_id, tag_id)
- reviews(highlight_id, last_shown_at, next_due_at, times_shown)
- A highlight belongs to a document, not to an extraction. That is what lets the extraction be replaced.

### Phase 1 · Save and extract
Build: A save endpoint taking a URL. Fetch with a timeout and a byte cap, extract the article,
  sanitise it so it makes no outbound requests, and store both text and cleaned HTML with
  the extractor version. Record failures as failures.
Trap: Storing raw fetched HTML and rendering it later means every time you open your own archive you load the publisher's scripts and trackers. Sanitise at capture: strip scripts, event handlers and remote references, so the stored copy is inert.
Done when: A saved article is readable offline. Opening a stored article makes no outbound network request, verified in developer tools. A page requiring JavaScript is recorded as a failed extraction with a reason rather than an empty document. The extractor version is stored with every extraction.
Do not build yet: No highlights, tags or review.

### Phase 2 · Highlights anchored to text, not to positions
Build: On highlight, store the quoted text plus a short prefix and suffix from the surrounding
  content. Resolve a highlight by searching for that quote with its context. Record whether
  resolution succeeded, was approximate, or failed.
Trap: Storing a character offset into the extracted text means every highlight breaks the moment the extraction changes by one character, which happens whenever the extractor improves or the publisher edits the page. Quote plus context resolves against new text and degrades gracefully when it cannot.
Done when: A highlight resolves correctly in the stored text. Inserting a paragraph before it in the extraction still resolves it to the same words. A highlight whose text has been deleted from the article is marked unresolvable rather than silently landing somewhere else. Resolution state is visible per highlight.
Do not build yet: No re-extraction or review yet.

### Phase 3 · Re-extraction, which is the point
Build: Re-extract a document with a newer extractor, keeping the previous extraction. Re-resolve
  every highlight against the new text and report the outcome. Let a failed re-extraction
  be rolled back to the previous version.
Trap: Replacing an extraction without re-resolving highlights leaves them pointing at text that has moved, and the damage is silent and cumulative. Re-resolve as part of re-extraction, report the counts, and keep the old extraction so a bad one can be reverted.
Done when: Re-extracting a document re-resolves every highlight and reports how many resolved exactly, approximately and not at all. A re-extraction that resolves fewer highlights can be rolled back to the previous version. The previous extraction is retained. A document with no highlights re-extracts without incident.
Do not build yet: No tags, reading queue or export yet.

### Phase 4 · A queue you can actually clear
Build: An inbox, a later list and an archive, with one action to move between them. Sort by
  estimated reading time as well as by date. Show the age of the oldest unread item
  honestly rather than hiding it.
Trap: A read-later queue with no triage becomes a list of a thousand articles nobody opens, and the tool stops being used rather than the queue being cleared. Make archiving without reading a first-class action, since deciding not to read something is a legitimate outcome and the queue depends on it.
Done when: Moving between inbox, later and archive is one action each. The list can be sorted by estimated reading time. Archiving without reading is as easy as reading, and is recorded distinctly. The age of the oldest unread item is shown.
Do not build yet: No review or export yet.

### Phase 5 · Highlight review
Build: A review that surfaces highlights on a schedule, weighted towards recent saves and
  highlights you have annotated. A daily email digest. Every review shows the source
  article and links back to it.
Trap: Showing highlights at random from a large archive produces mostly forgettable ones and trains you to skip the review. Weight towards annotated highlights and recent reading, and cap the daily count so the review takes a minute rather than being abandoned.
Done when: A review surfaces a capped number of highlights daily. Annotated highlights appear more often than unannotated ones, verified over a simulated month. Each review item links back to its source article. Marking a highlight as not useful stops it recurring.
Do not build yet: Export and capture tooling come last.

### Phase 6 · Capture everywhere, and export
Build: A bookmarklet and an endpoint that accepts a shared URL from a phone. Export every
  document as Markdown with its highlights inline and its metadata in front matter.
  Authenticate anything hosted, because a reading list is revealing.
Trap: A reading queue and its highlights are a detailed record of what you are thinking about, which is more revealing than most people expect. Authenticate before exposing it anywhere, and be deliberate about whether the highlights export includes your notes.
Done when: The bookmarklet saves from three different sites and reports clearly on one that blocks it. A shared URL from a phone saves through the endpoint. Export produces one Markdown file per document with highlights inline and front matter metadata. Hosted access requires authentication.
Do not build yet: Nothing after this.

### Out of scope (and why)
- Extraction quality at the level of the commercial product, which handles a long tail of sites through per-site work that never finishes.
- Mobile applications with offline reading, which is how most read-later use actually happens.
- Integrations with reading devices and their highlight formats, each of which is its own extraction problem.
- Access to material behind paywalls and logins, which this build does not attempt.

### README must contain
- That highlights are anchored by quoted text with context rather than by offsets, and why
- That re-extraction re-resolves highlights and reports the outcome before becoming current
- That stored articles are sanitised and make no outbound requests
- That archiving without reading is a first-class action
- That the Markdown export is the format designed to outlive this application

# Architecture · Readwise Reader

## Stack

| Part | Choice | Why |
| --- | --- | --- |
| Runtime | Node 22 with SQLite | one person's reading queue, where the interesting work is extraction and anchoring |
| Extraction | Readability plus a sanitiser | the article without the navigation, and inert so opening your own archive runs nobody's scripts |
| Anchoring | Quoted text with prefix and suffix context, not offsets | the single decision that determines whether highlights survive anything |
| Capture | A bookmarklet and a share endpoint | no browser extension to maintain across browsers and their review processes |
| Export | Markdown per document with highlights inline | highlights are the durable artefact and must outlive this application |

## Modules

Each module has one owner concern and a documented way to replace it.

| Module | Owns | How to replace it |
| --- | --- | --- |
| Fetch and extract | Turning a URL into inert readable text. | A headless browser would handle more sites at a large cost in speed. The sanitisation must survive any change. |
| Anchor | Locating a highlight in text. | This is the module the whole design exists around. Offsets are the alternative and they are why highlights break elsewhere. |
| Extractions | Versioned copies of a document. | Keeping the previous version is what makes re-extraction safe rather than destructive. |
| Queue | Triage states. | Simple, and the declining-is-first-class rule is the part that keeps it usable. |
| Review | Resurfacing highlights. | Any scheduling approach, provided it is weighted and capped rather than random and unbounded. |

## Configuration

Every runtime setting is an environment variable documented in `.env.example`, validated at startup, with a safe local default wherever one exists.

- `DB_PATH` · required · Documents, extractions and highlights.
- `FETCH_TIMEOUT_MS` · required · Per-page fetch timeout. A slow page should fail rather than hold up a save.
- `FETCH_MAX_BYTES` · required · Streaming byte cap, enforced during download.
- `ANCHOR_CONTEXT_CHARS` · required · How much prefix and suffix is stored with a quote. Enough to disambiguate a repeated phrase.
- `REVIEW_DAILY_CAP` · required · Highlights per daily review. Small enough that the review actually happens.
- `SAVE_TOKEN` · required, secret · Used by the bookmarklet and the share endpoint. Scoped to saving only.

## Production baseline

- Security: least privilege, input validation at every boundary, secret redaction in logs, rate limits on abuse-prone paths, no invented security primitives.
- Data: explicit schema and migrations, transactional writes where integrity matters, backup and restore procedures that have been exercised.
- Integrations: adapters around third-party providers, idempotent webhook or job processing, bounded retries, timeouts.
- Observability: structured logs with request or operation ids, an error-tracking hook, and health and readiness checks where a server exists.
- Quality: unit tests for domain rules, integration tests at module boundaries, one end-to-end test of the critical path.

## Decision records

For each dependency in the stack table, keep a short note: why it was chosen, its failure mode, and how it is replaced. Do not add infrastructure until a requirement in `PRODUCT.md` justifies it.

# Agent instructions · Readwise Reader product build

- Read `PRODUCT.md` and `ARCHITECTURE.md` before changing code. The stack is fixed: Node 22 with SQLite, Readability plus a sanitiser, Quoted text with prefix and suffix context, not offsets, A bookmarklet and a share endpoint, Markdown per document with highlights inline.
- Implement milestone by milestone from `MILESTONES.md`; keep each change reviewable and leave the application runnable at every commit.
- Treat authentication, payments, encryption, imports, webhooks and destructive actions as high-risk boundaries when present.
- Never invent cryptography or silently weaken a requirement to make a check pass.
- Put every external service behind an interface with a deterministic fake for tests.
- Add migrations and rollback or recovery notes for every persistent data change.
- Log useful operational context without credentials, tokens, passwords or personal data.
- Update documentation and run every check before completing a milestone.

## Known traps

- Storing raw fetched HTML and rendering it later means every time you open your own archive you load the publisher's scripts and trackers. Sanitise at capture: strip scripts, event handlers and remote references, so the stored copy is inert.
- Storing a character offset into the extracted text means every highlight breaks the moment the extraction changes by one character, which happens whenever the extractor improves or the publisher edits the page. Quote plus context resolves against new text and degrades gracefully when it cannot.
- Replacing an extraction without re-resolving highlights leaves them pointing at text that has moved, and the damage is silent and cumulative. Re-resolve as part of re-extraction, report the counts, and keep the old extraction so a bad one can be reverted.
- A read-later queue with no triage becomes a list of a thousand articles nobody opens, and the tool stops being used rather than the queue being cleared. Make archiving without reading a first-class action, since deciding not to read something is a legitimate outcome and the queue depends on it.
- Showing highlights at random from a large archive produces mostly forgettable ones and trains you to skip the review. Weight towards annotated highlights and recent reading, and cap the daily count so the review takes a minute rather than being abandoned.
- A reading queue and its highlights are a detailed record of what you are thinking about, which is more revealing than most people expect. Authenticate before exposing it anywhere, and be deliberate about whether the highlights export includes your notes.

# Delivery milestones · Readwise Reader

Estimated effort: **multi-day** for the indie phases; the production-only milestones add the trust and operability layer.

## M1 · Save and extract

A saved page becomes readable text, and a page that cannot be extracted says so.

### Steps

1. Fetch with limits
   Timeout, streaming byte cap, robots respected. You are a well-behaved client saving for yourself.
   Files: `src/fetch.js`
2. Extract and sanitise
   Readability for the article, then strip scripts and remote references so the copy is inert.
   Files: `src/extract.js`

   ```sh
   npm install @mozilla/readability jsdom dompurify
   ```

3. Version every extraction
   So you know which extractor produced a stored copy, which matters in Phase 3.
   Files: `src/extract.js`

### Done when

- [ ] A saved article reads offline
- [ ] A stored article makes no outbound requests
- [ ] A JavaScript-only page is recorded as a failure with a reason
- [ ] Extractor version is stored with each extraction

### Watch out

- Storing raw fetched HTML and rendering it later means every time you open your own archive you load the publisher's scripts and trackers. Sanitise at capture: strip scripts, event handlers and remote references, so the stored copy is inert.

## M2 · Highlights anchored to text, not to positions

A highlight knows what it selected, not where in a string it was.

### Steps

1. Store quote with context
   The selected text plus a short prefix and suffix. Enough context to disambiguate a repeated phrase.
   Files: `src/anchor.js`
2. Resolve by search
   Exact match first, then approximate. Record which strategy succeeded.
   Files: `src/anchor.js`
3. Handle the unresolvable case visibly
   Marked, still readable as a quote, never silently attached to the wrong place.
   Files: `src/anchor.js`

### Done when

- [ ] A highlight resolves correctly in stored text
- [ ] Inserting text before it does not move it
- [ ] A deleted quote is marked unresolvable rather than misplaced
- [ ] Resolution state is visible per highlight

### Watch out

- Storing a character offset into the extracted text means every highlight breaks the moment the extraction changes by one character, which happens whenever the extractor improves or the publisher edits the page. Quote plus context resolves against new text and degrades gracefully when it cannot.

## M3 · Re-extraction, which is the point

Improving the extractor does not lose a year of highlights.

### Steps

1. Keep extractions as versions
   A document has several; one is current. Rollback is then trivial.
   Files: `src/extractions.js`
2. Re-resolve and report
   Counts by outcome, shown before the new extraction becomes current.
   Files: `src/reextract.js`
3. Support rollback
   A re-extraction that loses highlights should be reversible in one action.
   Files: `src/extractions.js`

### Done when

- [ ] Re-extraction reports resolution outcomes by count
- [ ] A worse extraction can be rolled back
- [ ] The previous extraction is retained
- [ ] A document without highlights re-extracts cleanly

### Watch out

- Replacing an extraction without re-resolving highlights leaves them pointing at text that has moved, and the damage is silent and cumulative. Re-resolve as part of re-extraction, report the counts, and keep the old extraction so a bad one can be reverted.

## M4 · A queue you can actually clear

Saved articles are triaged rather than accumulating into a wall of guilt.

### Steps

1. Implement the three states
   Inbox, later, archive. One action between them, keyboard reachable.
   Files: `src/queue.js`
2. Estimate reading time
   From word count. It is what lets you pick something that fits the time you have.
   Files: `src/queue.js`
3. Make declining first class
   Archive without reading, recorded as such. It is the action that keeps the queue survivable.
   Files: `src/queue.js`

### Done when

- [ ] State changes are one action each
- [ ] Sorting by reading time works
- [ ] Archiving unread is recorded distinctly
- [ ] The oldest unread age is shown

### Watch out

- A read-later queue with no triage becomes a list of a thousand articles nobody opens, and the tool stops being used rather than the queue being cleared. Make archiving without reading a first-class action, since deciding not to read something is a legitimate outcome and the queue depends on it.

## M5 · Highlight review

What you highlighted comes back, spaced out, so it is not written once and never seen again.

### Steps

1. Schedule and weight the selection
   Recency and annotation as weights, with a daily cap.
   Files: `src/review.js`
2. Send a digest
   Email with the highlights and their sources. This is where a review actually gets read.
   Files: `src/digest.js`
3. Let a highlight be retired
   Marked not useful and never shown again. Without it, the review degrades over time.
   Files: `src/review.js`

### Done when

- [ ] The daily review is capped
- [ ] Annotated highlights surface more often over a simulated month
- [ ] Each item links to its source
- [ ] A retired highlight stops recurring

### Watch out

- Showing highlights at random from a large archive produces mostly forgettable ones and trains you to skip the review. Weight towards annotated highlights and recent reading, and cap the daily count so the review takes a minute rather than being abandoned.

## M6 · Capture everywhere, and export (production only)

Saving takes one action from anywhere, and nothing is trapped.

### Steps

1. Ship a bookmarklet and a share endpoint
   No extension to maintain. The endpoint covers the phone case.
   Files: `public/bookmarklet.js`, `src/save.js`
2. Export Markdown with highlights inline
   One file per document. This is the format that outlives the application.
   Files: `src/export.js`
3. Authenticate hosted access
   A reading list is personal. Put a login in front before it leaves your machine.
   Files: `src/auth.js`

### Done when

- [ ] The bookmarklet saves from three sites and reports on a blocked one
- [ ] A shared URL from a phone saves
- [ ] Export produces Markdown with inline highlights and front matter
- [ ] Hosted access requires authentication

### Watch out

- A reading queue and its highlights are a detailed record of what you are thinking about, which is more revealing than most people expect. Authenticate before exposing it anywhere, and be deliberate about whether the highlights export includes your notes.

# Operations · Readwise Reader

## Backup

The database holds documents, extractions and highlights. Highlights are the irreplaceable part; extractions can be refetched while the source still exists, which is exactly the assumption archiving exists to avoid relying on.

## Restore

Restore the database and open a document whose source site is gone. That is the test that the archive is real rather than a bookmark list.

Do a restore drill before the first real user, and write the date here when it passes.

## Monitoring

Extraction failure rate and unresolvable highlight count. A jump in the second after a re-extraction means the new extractor changed text more than expected and should be rolled back.

## Incident checklist

A re-extraction that silently moves highlights is the failure that matters, and reporting outcomes before making it current is the control. If highlights drift anyway, the anchoring context is too short.

1. Contain the issue without destroying evidence or user data.
2. Record the timeline and affected scope.
3. Rotate exposed secrets and revoke compromised sessions or credentials.
4. Restore from a verified backup when needed.
5. Document the root cause, the remediation and the regression test.

## Release gate

- [ ] A highlight on a repeated phrase resolves to the correct occurrence
- [ ] Re-extraction reports resolution outcomes before becoming current
- [ ] A stored article is confirmed to make no outbound requests
- [ ] The Markdown export includes highlights and front matter
- [ ] Hosted access is confirmed to require authentication

## Launch constraint

Do not market omitted Readwise Reader capabilities as implemented. The non-goals in `PRODUCT.md` remain user-visible limitations until they are deliberately delivered.

# Copy to .env and fill in. Never commit .env; this file documents it.

# Required. Documents, extractions and highlights.
DB_PATH=./data/reader.db

# Required. Per-page fetch timeout. A slow page should fail rather than hold up a save.
FETCH_TIMEOUT_MS=10000

# Required. Streaming byte cap, enforced during download.
FETCH_MAX_BYTES=5000000

# Required. How much prefix and suffix is stored with a quote. Enough to disambiguate a repeated phrase.
ANCHOR_CONTEXT_CHARS=40

# Required. Highlights per daily review. Small enough that the review actually happens.
REVIEW_DAILY_CAP=5

# Required · secret. Used by the bookmarklet and the share endpoint. Scoped to saving only.
SAVE_TOKEN=replace-with-a-random-token

$ choose a build depth, inspect the files, then open the complete pack in your agent