Vibecode Speechify

You are building a lean indie version of Speechify.

Create the following project files first, then implement the application by following them. Keep the files updated as decisions change. Do not collapse this into a single README or prompt.

===== README.md =====
# Speechify · indie build

Import PDFs, EPUBs and web pages, extract clean readable text, speak it through a local neural text-to-speech server, highlight the sentence being spoken, control speed, and remember where you stopped in every document.

Estimated effort: **weekend**. Work `BUILD_PLAN.md` top to bottom · every phase ends in a check that has to pass before the next one starts.

## Stack

| Part | Choice | Why |
| --- | --- | --- |
| Runtime | Node 22 with a small HTTP server and a browser front end | the work is parsing files and orchestrating audio, and the browser is already a good audio player |
| Speech | Kokoro served behind an OpenAI-compatible endpoint, with Piper as the lighter fallback | neural quality locally, and an interface shape that lets you swap engines without touching the reader |
| Extraction | pdfjs for PDF, epub parsing for EPUB, and Readability for web pages | three formats, three mature parsers, none of them worth writing |
| Segmentation | Sentence splitting with Intl.Segmenter | the highlight unit is the sentence, and the browser already knows where sentences end in many languages |
| Storage | SQLite for the library and progress | documents, positions and cached audio are rows |

## Before you start

Have every one of these ready. The plan assumes them from step one.

- [ ] **Node.js 22 or newer** · free
  - Why: Runs the server and the import pipeline.
  - Get it: nodejs.org, or nvm install 22.
  - Verify: node --version prints v22 or higher
- [ ] **A local neural text-to-speech server** · free, both are open source; the cost is memory and CPU or GPU time
  - Why: The whole point. Kokoro sounds considerably better; Piper is far lighter and runs comfortably on a laptop with no GPU.
  - Get it: Run Kokoro behind its OpenAI-compatible server image, or install Piper and a voice model. Try both before choosing, because the quality difference is large and so is the resource difference.
  - Verify: A curl to the speech endpoint returns playable audio
- [ ] **Enough memory for the chosen voice model** · free
  - Why: Kokoro wants materially more than Piper, and a model that swaps produces synthesis slower than real time, which changes the design.
  - Get it: Check available memory before choosing. If the ratio measured in Phase 1 is above one, use the lighter engine.
  - Verify: The synthesis ratio measured in Phase 1 is below one
- [ ] **Three awkward test documents** · free
  - Why: Phase 2 is only meaningful against real files: a two column academic PDF, an EPUB with chapters, and a web article behind a cookie banner.
  - Get it: A paper from an open archive, a public domain EPUB, and any news article. Keep them as fixtures.
  - Verify: All three are saved in a fixtures directory
- [ ] **Several gigabytes of free disk for cached audio** · free
  - Why: Cached sentence audio for a long book adds up quickly, and a full disk mid-playback is an unpleasant failure.
  - Get it: Check free space and set the cache limit in configuration accordingly.
  - Verify: The configured cache limit is below your free space

## Quick start

```sh
curl -s http://localhost:8880/v1/audio/speech -H 'Content-Type: application/json' -d '{"model":"kokoro","input":"Hello there.","voice":"af_sky"}' --output test.wav
```

Then copy `.env.example` to `.env` and fill in the values it documents.

## Honest limits

This build deliberately does not replace:

- The hosted voice catalogue and its consistency across devices.
- Mobile scanning and optical character recognition capture.
- Cross-device sync of library and position.
- Native mobile applications with background audio.
- Speechify's 1000+ hosted voices and consistent quality across devices
- mobile scanning and polished OCR capture
- cross-device sync and offline native apps
- Google Drive, Dropbox, and OneDrive integrations
- voice typing, AI podcasts, and document chat

If one of those is essential to you, that is the reason to keep paying for Speechify, and the README should say so rather than pretend.

===== BRIEF.md =====
# Build brief · Speechify

The one-shot brief this plan expands. `BUILD_PLAN.md` (or `MILESTONES.md`) is the same sequence broken into steps and checks; where the two disagree, the plan wins.

Build me a reader that turns documents and web pages into speech with a local neural voice and synchronised highlighting to replace Speechify. Build it in phases, in the order below.
Do not write the whole thing in one pass. Finish a phase, run its "Done when"
check, fix what fails, and only then start the next phase.

### Stack (fixed, do not substitute)
- Runtime: Node 22 with a small HTTP server and a browser front end · the work is parsing files and orchestrating audio, and the browser is already a good audio player
- Speech: Kokoro served behind an OpenAI-compatible endpoint, with Piper as the lighter fallback · neural quality locally, and an interface shape that lets you swap engines without touching the reader
- Extraction: pdfjs for PDF, epub parsing for EPUB, and Readability for web pages · three formats, three mature parsers, none of them worth writing
- Segmentation: Sentence splitting with Intl.Segmenter · the highlight unit is the sentence, and the browser already knows where sentences end in many languages
- Storage: SQLite for the library and progress · documents, positions and cached audio are rows

### Data model (create this before Phase 1)
- documents(id, title, source_kind, source_path, imported_at, language) · source_kind is pdf, epub, web or text
- blocks(id, document_id, ordinal, kind, text) · kind is heading, paragraph or caption, in reading order
- sentences(id, block_id, ordinal, text, char_start, char_end) · the highlight and playback unit
- audio_cache(sentence_id, voice, speed, path, duration_ms) · synthesis is slow enough to be worth caching
- progress(document_id, sentence_id, updated_at) · one row per document, so resuming is exact
- Extraction happens once at import. Playback never reparses the source file.

### Phase 1 · A local voice that speaks a sentence
Build: Run a local text-to-speech server exposing an OpenAI-compatible speech endpoint. Send
  one sentence and play the result. Measure how long synthesis takes relative to the
  audio it produces, because that ratio decides the whole design.
Trap: If synthesis is slower than real time, streaming sentence by sentence will stutter and you must synthesise ahead. Measure the ratio now rather than discovering it during Phase 4, when the fix is a queue you did not plan for.
Done when: A sentence sent to the local server returns audio that plays. The ratio of synthesis time to audio duration is measured and written down. Stopping the server produces a clear error in the application rather than silence.
Do not build yet: No documents, no highlighting, no library. One sentence spoken.

### Phase 2 · Getting clean text out of real files
Build: Import each format into blocks in reading order. Strip headers, footers, page numbers
  and navigation. Preserve headings, because a reader that does not pause at a chapter
  boundary is exhausting.
Trap: PDF has no concept of reading order. Text is positioned glyphs, and a two column layout extracted naively interleaves the columns into nonsense. Detect columns by clustering horizontal positions, and test on an academic paper rather than a single column report.
Done when: A two column PDF extracts in correct reading order, verified by reading the first three paragraphs. Repeated page headers and footers appear at most once in the output. An EPUB preserves chapter boundaries as headings. A web page import contains the article and not the navigation or the cookie banner.
Do not build yet: No speech playback of documents yet.

### Phase 3 · Sentences and synchronised highlighting
Build: Split blocks into sentences with Intl.Segmenter, keeping character offsets into the
  original text so the highlight can be drawn precisely. Play one sentence at a time and
  advance the highlight on audio completion rather than on a timer.
Trap: Driving the highlight from a timer and an estimated duration drifts, and by the third chapter the highlight is a paragraph away from the audio. Advance on the audio element's ended event, which cannot drift because it is the audio itself telling you.
Done when: The highlight matches the spoken sentence after twenty minutes of continuous playback, checked by pausing and reading. Abbreviations such as Dr and e.g. do not split a sentence in the middle. Clicking a sentence starts playback there and the highlight follows correctly from that point.
Do not build yet: No prefetching or library yet, so playback may pause between sentences.

### Phase 4 · Playing ahead so it never stutters
Build: Synthesise several sentences ahead of the one playing and cache the audio by sentence,
  voice and speed. Discard the queue when the user seeks. Handle speed either through the
  audio element's playback rate or by re-synthesising, whichever your engine does better.
Trap: Changing playback rate on the audio element is instant and free but raises pitch on some engines' output; re-synthesising at a different speed sounds better and costs a delay. Pick one deliberately, and if you use the element's rate, check whether your engine's audio survives it.
Done when: Ten minutes of continuous playback contains no audible gap between sentences. Seeking to a distant sentence starts playing in under a second rather than after synthesising everything in between. Changing speed mid-document takes effect within one sentence.
Do not build yet: No library or progress yet.

### Phase 5 · Library, progress and export
Build: A library listing imported documents with progress. Resume at the exact sentence.
  Export a document's audio as a single file for offline listening, and let the cache be
  cleared without losing the documents.
Trap: Storing progress as a character offset breaks the moment a document is re-imported and extraction changes slightly. Store the sentence identity, and if re-import produces different sentences, say so rather than silently resuming somewhere wrong.
Done when: Closing the application mid-document and reopening it resumes at the same sentence. Clearing the audio cache frees disk without losing documents or progress. Exporting a chapter produces one audio file whose duration matches the sum of its sentences.
Do not build yet: Sync and mobile are out of scope entirely.

### Phase 6 · Sharing it, and the licence question
Build: Put authentication in front of it. Then read the licence of every voice model you ship
  and record what it permits, because voice models vary a great deal and some forbid
  redistribution or commercial use.
Trap: Turning a purchased book into an audio file is fine for you and is redistribution the moment you share it. Similarly, a voice model licensed for personal use only does not become shareable because your application is free. Both questions have to be answered before anyone else connects.
Done when: Access requires authentication. Every bundled voice model has its licence recorded with what it permits. The interface states that exported audio is for personal use and does not offer a share link.
Do not build yet: Nothing after this.

### Out of scope (and why)
- The thousand hosted voices and their consistency across devices, which is a licensing and infrastructure operation rather than a feature.
- Mobile applications with camera scanning and polished optical character recognition, which is the way most people actually use the paid product.
- Cross-device sync of position and library, which needs accounts and a server.
- Offline native applications on phones, and the background audio behaviour that platform gives them.

### README must contain
- The measured ratio of synthesis time to audio duration on your machine, and what it implies for prefetching
- That highlighting advances on audio completion rather than on a timer, and why
- How PDF column detection works and which documents it still gets wrong
- Where the audio cache lives, how large it grows and that clearing it is safe
- The licence of every bundled voice and the personal use position on exported audio

===== AGENTS.md =====
# Agent instructions · Speechify indie build

- Read `README.md` and `BUILD_PLAN.md` before writing code. The stack is fixed: Node 22 with a small HTTP server and a browser front end, Kokoro served behind an OpenAI-compatible endpoint, with Piper as the lighter fallback, pdfjs for PDF, epub parsing for EPUB, and Readability for web pages, Sentence splitting with Intl.Segmenter, SQLite for the library and progress. Do not substitute.
- Work one phase at a time, in order. Do not start a phase until every "Done when" item of the previous one passes.
- Prefer the fewest moving parts that satisfy the step. No frameworks, services or dependencies the plan does not name.
- Secrets live in `.env`, never in source or logs. Keep `.env.example` current when a variable is introduced.
- Do not invent cryptography, security guarantees, APIs or compliance claims.
- Add a focused test for every destructive, security-sensitive or data-loss path the plan names.
- Run the project checks before declaring a phase complete, and record any deliberate shortcut in the README under "Tradeoffs".

## Known traps

- If synthesis is slower than real time, streaming sentence by sentence will stutter and you must synthesise ahead. Measure the ratio now rather than discovering it during Phase 4, when the fix is a queue you did not plan for.
- PDF has no concept of reading order. Text is positioned glyphs, and a two column layout extracted naively interleaves the columns into nonsense. Detect columns by clustering horizontal positions, and test on an academic paper rather than a single column report.
- Driving the highlight from a timer and an estimated duration drifts, and by the third chapter the highlight is a paragraph away from the audio. Advance on the audio element's ended event, which cannot drift because it is the audio itself telling you.
- Changing playback rate on the audio element is instant and free but raises pitch on some engines' output; re-synthesising at a different speed sounds better and costs a delay. Pick one deliberately, and if you use the element's rate, check whether your engine's audio survives it.
- Storing progress as a character offset breaks the moment a document is re-imported and extraction changes slightly. Store the sentence identity, and if re-import produces different sentences, say so rather than silently resuming somewhere wrong.

===== BUILD_PLAN.md =====
# Build plan · Speechify

Import PDFs, EPUBs and web pages, extract clean readable text, speak it through a local neural text-to-speech server, highlight the sentence being spoken, control speed, and remember where you stopped in every document.

Phases are in dependency order. Each ends in a "Done when" list; treat an unticked item as a blocker, not a note.

## Phase 1 · A local voice that speaks a sentence

Text goes in, audio comes out, from a server on your own machine.

### Steps

1. Run the speech server
   Start Kokoro behind its OpenAI-compatible endpoint, or Piper if your machine is modest. Confirm with a curl before writing any application code.

   ```sh
   curl -s http://localhost:8880/v1/audio/speech -H 'Content-Type: application/json' -d '{"model":"kokoro","input":"Hello there.","voice":"af_sky"}' --output test.wav
   ```

2. Wrap it behind one interface
   A single synthesize function taking text, voice and speed. Every engine difference lives behind it, which is what makes Piper a fallback rather than a rewrite.
   Files: `src/tts.ts`
3. Measure the synthesis ratio
   Time synthesis against the duration of the audio produced, across ten sentences of varying length. Write the ratio in the README.
   Files: `README.md`

### Done when

- [ ] A sentence returns playable audio from the local server
- [ ] The synthesis to duration ratio is measured and recorded
- [ ] A stopped server produces a clear error rather than silence
- [ ] Switching to the fallback engine requires changing one configuration value

### Watch out

- If synthesis is slower than real time, streaming sentence by sentence will stutter and you must synthesise ahead. Measure the ratio now rather than discovering it during Phase 4, when the fix is a queue you did not plan for.

## Phase 2 · Getting clean text out of real files

A PDF, an EPUB and a web page all become the same clean block structure.

### Steps

1. Extract PDF with column detection
   pdfjs gives positioned text items. Cluster their x positions per page to detect columns, then order within each column before moving to the next.
   Files: `src/import/pdf.ts`
2. Strip repeating furniture
   Text appearing at the same position on many pages is a header or a footer. Detect by repetition across pages rather than by guessing margins.
   Files: `src/import/pdf.ts`
3. Import EPUB and web pages
   EPUB is structured already, so preserve its heading levels. For web pages, use Readability to get the article, then convert to the same block shape.
   Files: `src/import/epub.ts`, `src/import/web.ts`

### Done when

- [ ] A two column academic PDF reads in the correct order
- [ ] Page headers and footers do not repeat through the extracted text
- [ ] EPUB chapter boundaries survive as headings
- [ ] A web import contains the article and not the navigation

### Watch out

- PDF has no concept of reading order. Text is positioned glyphs, and a two column layout extracted naively interleaves the columns into nonsense. Detect columns by clustering horizontal positions, and test on an academic paper rather than a single column report.

## Phase 3 · Sentences and synchronised highlighting

The highlighted sentence is the sentence you are hearing, all the way through a long document.

### Steps

1. Segment with offsets preserved
   Intl.Segmenter with the sentence granularity, storing character start and end so the highlight maps back to the rendered text exactly.
   Files: `src/segment.ts`
2. Advance on audio completion
   Play a sentence, wait for the ended event, then advance. No timers anywhere in the playback loop.
   Files: `src/player.ts`
3. Make sentences clickable
   Clicking sets the position and starts from there, which is also how you will test that offsets are right.
   Files: `src/reader.ts`

### Done when

- [ ] Highlight and audio still agree after twenty minutes of playback
- [ ] Common abbreviations do not cause a mid-sentence split
- [ ] Clicking a sentence starts playback at that sentence
- [ ] Pausing and resuming continues from the same sentence

### Watch out

- Driving the highlight from a timer and an estimated duration drifts, and by the third chapter the highlight is a paragraph away from the audio. Advance on the audio element's ended event, which cannot drift because it is the audio itself telling you.

## Phase 4 · Playing ahead so it never stutters

Playback is continuous, and speed changes take effect without a gap.

### Steps

1. Prefetch a small queue
   Keep a few sentences ahead synthesised. The right number is derived from the ratio you measured in Phase 1, not guessed.
   Files: `src/prefetch.ts`
2. Cache by sentence, voice and speed
   The cache key must include voice and speed or a speed change will replay stale audio.
   Files: `src/cache.ts`
3. Cancel work on seek
   Abort in-flight synthesis when the user jumps. Otherwise the queue delivers audio for a position nobody is at any more.
   Files: `src/prefetch.ts`

### Done when

- [ ] Ten minutes of playback has no audible gaps
- [ ] Seeking far ahead begins playing in under a second
- [ ] A speed change takes effect within one sentence
- [ ] Cached audio is keyed by voice and speed

### Watch out

- Changing playback rate on the audio element is instant and free but raises pitch on some engines' output; re-synthesising at a different speed sounds better and costs a delay. Pick one deliberately, and if you use the element's rate, check whether your engine's audio survives it.

## Phase 5 · Library, progress and export

You can leave mid-chapter and come back a week later exactly where you were.

### Steps

1. Build the library view
   Documents with progress as a proportion and the last read time. Sorted by recency, because that is how people actually return to reading.
   Files: `src/library.ts`
2. Persist progress by sentence identity
   Store the sentence id. On re-import, detect that ids changed and tell the user rather than guessing.
   Files: `src/progress.ts`
3. Export audio and make the cache disposable
   Concatenate cached sentence audio into one file. Clearing the cache must never touch documents or progress.
   Files: `src/export.ts`

### Done when

- [ ] Reopening resumes at the same sentence
- [ ] Clearing the audio cache preserves documents and progress
- [ ] An exported chapter's duration matches the sum of its sentences
- [ ] Re-importing a changed document reports that progress may have shifted

### Watch out

- Storing progress as a character offset breaks the moment a document is re-imported and extraction changes slightly. Store the sentence identity, and if re-import produces different sentences, say so rather than silently resuming somewhere wrong.

## Not in this build

- The hosted voice catalogue and its consistency across devices.
- Mobile scanning and optical character recognition capture.
- Cross-device sync of library and position.
- Native mobile applications with background audio.

## After v1, if you want it

- Optical character recognition for scanned PDFs that contain no text layer
- A word-level highlight, which needs timestamps your engine may not provide
- Automatic skipping of reference lists and footnotes in academic papers

===== .env.example =====
# Copy to .env and fill in. Never commit .env; this file documents it.

# Required. Your local speech server. The OpenAI-compatible shape is what makes engines swappable.
TTS_ENDPOINT=http://localhost:8880/v1/audio/speech

# Required. Voice identifier from your engine's voice list.
TTS_VOICE=af_sky

# Required. kokoro or piper. Chosen by the ratio measured in Phase 1.
TTS_ENGINE=kokoro

# Required. How far ahead to synthesise. Derived from the synthesis ratio, not guessed.
PREFETCH_SENTENCES=4

# Required. Where cached sentence audio is written. Safe to delete.
AUDIO_CACHE_DIR=./data/audio

# Required. Cache ceiling. Oldest entries are evicted first.
AUDIO_CACHE_MAX_GB=5

# Required. Library, documents, sentences and progress. This is the backup.
DB_PATH=./data/reader.db

You are building a lean indie version of Speechify.

Create the following project files first, then implement the application by following them. Keep the files updated as decisions change. Do not collapse this into a single README or prompt.

===== README.md =====
# Speechify · indie build

Import PDFs, EPUBs and web pages, extract clean readable text, speak it through a local neural text-to-speech server, highlight the sentence being spoken, control speed, and remember where you stopped in every document.

Estimated effort: **weekend**. Work `BUILD_PLAN.md` top to bottom · every phase ends in a check that has to pass before the next one starts.

## Stack

| Part | Choice | Why |
| --- | --- | --- |
| Runtime | Node 22 with a small HTTP server and a browser front end | the work is parsing files and orchestrating audio, and the browser is already a good audio player |
| Speech | Kokoro served behind an OpenAI-compatible endpoint, with Piper as the lighter fallback | neural quality locally, and an interface shape that lets you swap engines without touching the reader |
| Extraction | pdfjs for PDF, epub parsing for EPUB, and Readability for web pages | three formats, three mature parsers, none of them worth writing |
| Segmentation | Sentence splitting with Intl.Segmenter | the highlight unit is the sentence, and the browser already knows where sentences end in many languages |
| Storage | SQLite for the library and progress | documents, positions and cached audio are rows |

## Before you start

Have every one of these ready. The plan assumes them from step one.

- [ ] **Node.js 22 or newer** · free
  - Why: Runs the server and the import pipeline.
  - Get it: nodejs.org, or nvm install 22.
  - Verify: node --version prints v22 or higher
- [ ] **A local neural text-to-speech server** · free, both are open source; the cost is memory and CPU or GPU time
  - Why: The whole point. Kokoro sounds considerably better; Piper is far lighter and runs comfortably on a laptop with no GPU.
  - Get it: Run Kokoro behind its OpenAI-compatible server image, or install Piper and a voice model. Try both before choosing, because the quality difference is large and so is the resource difference.
  - Verify: A curl to the speech endpoint returns playable audio
- [ ] **Enough memory for the chosen voice model** · free
  - Why: Kokoro wants materially more than Piper, and a model that swaps produces synthesis slower than real time, which changes the design.
  - Get it: Check available memory before choosing. If the ratio measured in Phase 1 is above one, use the lighter engine.
  - Verify: The synthesis ratio measured in Phase 1 is below one
- [ ] **Three awkward test documents** · free
  - Why: Phase 2 is only meaningful against real files: a two column academic PDF, an EPUB with chapters, and a web article behind a cookie banner.
  - Get it: A paper from an open archive, a public domain EPUB, and any news article. Keep them as fixtures.
  - Verify: All three are saved in a fixtures directory
- [ ] **Several gigabytes of free disk for cached audio** · free
  - Why: Cached sentence audio for a long book adds up quickly, and a full disk mid-playback is an unpleasant failure.
  - Get it: Check free space and set the cache limit in configuration accordingly.
  - Verify: The configured cache limit is below your free space

## Quick start

```sh
curl -s http://localhost:8880/v1/audio/speech -H 'Content-Type: application/json' -d '{"model":"kokoro","input":"Hello there.","voice":"af_sky"}' --output test.wav
```

Then copy `.env.example` to `.env` and fill in the values it documents.

## Honest limits

This build deliberately does not replace:

- The hosted voice catalogue and its consistency across devices.
- Mobile scanning and optical character recognition capture.
- Cross-device sync of library and position.
- Native mobile applications with background audio.
- Speechify's 1000+ hosted voices and consistent quality across devices
- mobile scanning and polished OCR capture
- cross-device sync and offline native apps
- Google Drive, Dropbox, and OneDrive integrations
- voice typing, AI podcasts, and document chat

If one of those is essential to you, that is the reason to keep paying for Speechify, and the README should say so rather than pretend.

===== BRIEF.md =====
# Build brief · Speechify

The one-shot brief this plan expands. `BUILD_PLAN.md` (or `MILESTONES.md`) is the same sequence broken into steps and checks; where the two disagree, the plan wins.

Build me a reader that turns documents and web pages into speech with a local neural voice and synchronised highlighting to replace Speechify. Build it in phases, in the order below.
Do not write the whole thing in one pass. Finish a phase, run its "Done when"
check, fix what fails, and only then start the next phase.

### Stack (fixed, do not substitute)
- Runtime: Node 22 with a small HTTP server and a browser front end · the work is parsing files and orchestrating audio, and the browser is already a good audio player
- Speech: Kokoro served behind an OpenAI-compatible endpoint, with Piper as the lighter fallback · neural quality locally, and an interface shape that lets you swap engines without touching the reader
- Extraction: pdfjs for PDF, epub parsing for EPUB, and Readability for web pages · three formats, three mature parsers, none of them worth writing
- Segmentation: Sentence splitting with Intl.Segmenter · the highlight unit is the sentence, and the browser already knows where sentences end in many languages
- Storage: SQLite for the library and progress · documents, positions and cached audio are rows

### Data model (create this before Phase 1)
- documents(id, title, source_kind, source_path, imported_at, language) · source_kind is pdf, epub, web or text
- blocks(id, document_id, ordinal, kind, text) · kind is heading, paragraph or caption, in reading order
- sentences(id, block_id, ordinal, text, char_start, char_end) · the highlight and playback unit
- audio_cache(sentence_id, voice, speed, path, duration_ms) · synthesis is slow enough to be worth caching
- progress(document_id, sentence_id, updated_at) · one row per document, so resuming is exact
- Extraction happens once at import. Playback never reparses the source file.

### Phase 1 · A local voice that speaks a sentence
Build: Run a local text-to-speech server exposing an OpenAI-compatible speech endpoint. Send
  one sentence and play the result. Measure how long synthesis takes relative to the
  audio it produces, because that ratio decides the whole design.
Trap: If synthesis is slower than real time, streaming sentence by sentence will stutter and you must synthesise ahead. Measure the ratio now rather than discovering it during Phase 4, when the fix is a queue you did not plan for.
Done when: A sentence sent to the local server returns audio that plays. The ratio of synthesis time to audio duration is measured and written down. Stopping the server produces a clear error in the application rather than silence.
Do not build yet: No documents, no highlighting, no library. One sentence spoken.

### Phase 2 · Getting clean text out of real files
Build: Import each format into blocks in reading order. Strip headers, footers, page numbers
  and navigation. Preserve headings, because a reader that does not pause at a chapter
  boundary is exhausting.
Trap: PDF has no concept of reading order. Text is positioned glyphs, and a two column layout extracted naively interleaves the columns into nonsense. Detect columns by clustering horizontal positions, and test on an academic paper rather than a single column report.
Done when: A two column PDF extracts in correct reading order, verified by reading the first three paragraphs. Repeated page headers and footers appear at most once in the output. An EPUB preserves chapter boundaries as headings. A web page import contains the article and not the navigation or the cookie banner.
Do not build yet: No speech playback of documents yet.

### Phase 3 · Sentences and synchronised highlighting
Build: Split blocks into sentences with Intl.Segmenter, keeping character offsets into the
  original text so the highlight can be drawn precisely. Play one sentence at a time and
  advance the highlight on audio completion rather than on a timer.
Trap: Driving the highlight from a timer and an estimated duration drifts, and by the third chapter the highlight is a paragraph away from the audio. Advance on the audio element's ended event, which cannot drift because it is the audio itself telling you.
Done when: The highlight matches the spoken sentence after twenty minutes of continuous playback, checked by pausing and reading. Abbreviations such as Dr and e.g. do not split a sentence in the middle. Clicking a sentence starts playback there and the highlight follows correctly from that point.
Do not build yet: No prefetching or library yet, so playback may pause between sentences.

### Phase 4 · Playing ahead so it never stutters
Build: Synthesise several sentences ahead of the one playing and cache the audio by sentence,
  voice and speed. Discard the queue when the user seeks. Handle speed either through the
  audio element's playback rate or by re-synthesising, whichever your engine does better.
Trap: Changing playback rate on the audio element is instant and free but raises pitch on some engines' output; re-synthesising at a different speed sounds better and costs a delay. Pick one deliberately, and if you use the element's rate, check whether your engine's audio survives it.
Done when: Ten minutes of continuous playback contains no audible gap between sentences. Seeking to a distant sentence starts playing in under a second rather than after synthesising everything in between. Changing speed mid-document takes effect within one sentence.
Do not build yet: No library or progress yet.

### Phase 5 · Library, progress and export
Build: A library listing imported documents with progress. Resume at the exact sentence.
  Export a document's audio as a single file for offline listening, and let the cache be
  cleared without losing the documents.
Trap: Storing progress as a character offset breaks the moment a document is re-imported and extraction changes slightly. Store the sentence identity, and if re-import produces different sentences, say so rather than silently resuming somewhere wrong.
Done when: Closing the application mid-document and reopening it resumes at the same sentence. Clearing the audio cache frees disk without losing documents or progress. Exporting a chapter produces one audio file whose duration matches the sum of its sentences.
Do not build yet: Sync and mobile are out of scope entirely.

### Phase 6 · Sharing it, and the licence question
Build: Put authentication in front of it. Then read the licence of every voice model you ship
  and record what it permits, because voice models vary a great deal and some forbid
  redistribution or commercial use.
Trap: Turning a purchased book into an audio file is fine for you and is redistribution the moment you share it. Similarly, a voice model licensed for personal use only does not become shareable because your application is free. Both questions have to be answered before anyone else connects.
Done when: Access requires authentication. Every bundled voice model has its licence recorded with what it permits. The interface states that exported audio is for personal use and does not offer a share link.
Do not build yet: Nothing after this.

### Out of scope (and why)
- The thousand hosted voices and their consistency across devices, which is a licensing and infrastructure operation rather than a feature.
- Mobile applications with camera scanning and polished optical character recognition, which is the way most people actually use the paid product.
- Cross-device sync of position and library, which needs accounts and a server.
- Offline native applications on phones, and the background audio behaviour that platform gives them.

### README must contain
- The measured ratio of synthesis time to audio duration on your machine, and what it implies for prefetching
- That highlighting advances on audio completion rather than on a timer, and why
- How PDF column detection works and which documents it still gets wrong
- Where the audio cache lives, how large it grows and that clearing it is safe
- The licence of every bundled voice and the personal use position on exported audio

===== AGENTS.md =====
# Agent instructions · Speechify indie build

- Read `README.md` and `BUILD_PLAN.md` before writing code. The stack is fixed: Node 22 with a small HTTP server and a browser front end, Kokoro served behind an OpenAI-compatible endpoint, with Piper as the lighter fallback, pdfjs for PDF, epub parsing for EPUB, and Readability for web pages, Sentence splitting with Intl.Segmenter, SQLite for the library and progress. Do not substitute.
- Work one phase at a time, in order. Do not start a phase until every "Done when" item of the previous one passes.
- Prefer the fewest moving parts that satisfy the step. No frameworks, services or dependencies the plan does not name.
- Secrets live in `.env`, never in source or logs. Keep `.env.example` current when a variable is introduced.
- Do not invent cryptography, security guarantees, APIs or compliance claims.
- Add a focused test for every destructive, security-sensitive or data-loss path the plan names.
- Run the project checks before declaring a phase complete, and record any deliberate shortcut in the README under "Tradeoffs".

## Known traps

- If synthesis is slower than real time, streaming sentence by sentence will stutter and you must synthesise ahead. Measure the ratio now rather than discovering it during Phase 4, when the fix is a queue you did not plan for.
- PDF has no concept of reading order. Text is positioned glyphs, and a two column layout extracted naively interleaves the columns into nonsense. Detect columns by clustering horizontal positions, and test on an academic paper rather than a single column report.
- Driving the highlight from a timer and an estimated duration drifts, and by the third chapter the highlight is a paragraph away from the audio. Advance on the audio element's ended event, which cannot drift because it is the audio itself telling you.
- Changing playback rate on the audio element is instant and free but raises pitch on some engines' output; re-synthesising at a different speed sounds better and costs a delay. Pick one deliberately, and if you use the element's rate, check whether your engine's audio survives it.
- Storing progress as a character offset breaks the moment a document is re-imported and extraction changes slightly. Store the sentence identity, and if re-import produces different sentences, say so rather than silently resuming somewhere wrong.

===== BUILD_PLAN.md =====
# Build plan · Speechify

Import PDFs, EPUBs and web pages, extract clean readable text, speak it through a local neural text-to-speech server, highlight the sentence being spoken, control speed, and remember where you stopped in every document.

Phases are in dependency order. Each ends in a "Done when" list; treat an unticked item as a blocker, not a note.

## Phase 1 · A local voice that speaks a sentence

Text goes in, audio comes out, from a server on your own machine.

### Steps

1. Run the speech server
   Start Kokoro behind its OpenAI-compatible endpoint, or Piper if your machine is modest. Confirm with a curl before writing any application code.

   ```sh
   curl -s http://localhost:8880/v1/audio/speech -H 'Content-Type: application/json' -d '{"model":"kokoro","input":"Hello there.","voice":"af_sky"}' --output test.wav
   ```

2. Wrap it behind one interface
   A single synthesize function taking text, voice and speed. Every engine difference lives behind it, which is what makes Piper a fallback rather than a rewrite.
   Files: `src/tts.ts`
3. Measure the synthesis ratio
   Time synthesis against the duration of the audio produced, across ten sentences of varying length. Write the ratio in the README.
   Files: `README.md`

### Done when

- [ ] A sentence returns playable audio from the local server
- [ ] The synthesis to duration ratio is measured and recorded
- [ ] A stopped server produces a clear error rather than silence
- [ ] Switching to the fallback engine requires changing one configuration value

### Watch out

- If synthesis is slower than real time, streaming sentence by sentence will stutter and you must synthesise ahead. Measure the ratio now rather than discovering it during Phase 4, when the fix is a queue you did not plan for.

## Phase 2 · Getting clean text out of real files

A PDF, an EPUB and a web page all become the same clean block structure.

### Steps

1. Extract PDF with column detection
   pdfjs gives positioned text items. Cluster their x positions per page to detect columns, then order within each column before moving to the next.
   Files: `src/import/pdf.ts`
2. Strip repeating furniture
   Text appearing at the same position on many pages is a header or a footer. Detect by repetition across pages rather than by guessing margins.
   Files: `src/import/pdf.ts`
3. Import EPUB and web pages
   EPUB is structured already, so preserve its heading levels. For web pages, use Readability to get the article, then convert to the same block shape.
   Files: `src/import/epub.ts`, `src/import/web.ts`

### Done when

- [ ] A two column academic PDF reads in the correct order
- [ ] Page headers and footers do not repeat through the extracted text
- [ ] EPUB chapter boundaries survive as headings
- [ ] A web import contains the article and not the navigation

### Watch out

- PDF has no concept of reading order. Text is positioned glyphs, and a two column layout extracted naively interleaves the columns into nonsense. Detect columns by clustering horizontal positions, and test on an academic paper rather than a single column report.

## Phase 3 · Sentences and synchronised highlighting

The highlighted sentence is the sentence you are hearing, all the way through a long document.

### Steps

1. Segment with offsets preserved
   Intl.Segmenter with the sentence granularity, storing character start and end so the highlight maps back to the rendered text exactly.
   Files: `src/segment.ts`
2. Advance on audio completion
   Play a sentence, wait for the ended event, then advance. No timers anywhere in the playback loop.
   Files: `src/player.ts`
3. Make sentences clickable
   Clicking sets the position and starts from there, which is also how you will test that offsets are right.
   Files: `src/reader.ts`

### Done when

- [ ] Highlight and audio still agree after twenty minutes of playback
- [ ] Common abbreviations do not cause a mid-sentence split
- [ ] Clicking a sentence starts playback at that sentence
- [ ] Pausing and resuming continues from the same sentence

### Watch out

- Driving the highlight from a timer and an estimated duration drifts, and by the third chapter the highlight is a paragraph away from the audio. Advance on the audio element's ended event, which cannot drift because it is the audio itself telling you.

## Phase 4 · Playing ahead so it never stutters

Playback is continuous, and speed changes take effect without a gap.

### Steps

1. Prefetch a small queue
   Keep a few sentences ahead synthesised. The right number is derived from the ratio you measured in Phase 1, not guessed.
   Files: `src/prefetch.ts`
2. Cache by sentence, voice and speed
   The cache key must include voice and speed or a speed change will replay stale audio.
   Files: `src/cache.ts`
3. Cancel work on seek
   Abort in-flight synthesis when the user jumps. Otherwise the queue delivers audio for a position nobody is at any more.
   Files: `src/prefetch.ts`

### Done when

- [ ] Ten minutes of playback has no audible gaps
- [ ] Seeking far ahead begins playing in under a second
- [ ] A speed change takes effect within one sentence
- [ ] Cached audio is keyed by voice and speed

### Watch out

- Changing playback rate on the audio element is instant and free but raises pitch on some engines' output; re-synthesising at a different speed sounds better and costs a delay. Pick one deliberately, and if you use the element's rate, check whether your engine's audio survives it.

## Phase 5 · Library, progress and export

You can leave mid-chapter and come back a week later exactly where you were.

### Steps

1. Build the library view
   Documents with progress as a proportion and the last read time. Sorted by recency, because that is how people actually return to reading.
   Files: `src/library.ts`
2. Persist progress by sentence identity
   Store the sentence id. On re-import, detect that ids changed and tell the user rather than guessing.
   Files: `src/progress.ts`
3. Export audio and make the cache disposable
   Concatenate cached sentence audio into one file. Clearing the cache must never touch documents or progress.
   Files: `src/export.ts`

### Done when

- [ ] Reopening resumes at the same sentence
- [ ] Clearing the audio cache preserves documents and progress
- [ ] An exported chapter's duration matches the sum of its sentences
- [ ] Re-importing a changed document reports that progress may have shifted

### Watch out

- Storing progress as a character offset breaks the moment a document is re-imported and extraction changes slightly. Store the sentence identity, and if re-import produces different sentences, say so rather than silently resuming somewhere wrong.

## Not in this build

- The hosted voice catalogue and its consistency across devices.
- Mobile scanning and optical character recognition capture.
- Cross-device sync of library and position.
- Native mobile applications with background audio.

## After v1, if you want it

- Optical character recognition for scanned PDFs that contain no text layer
- A word-level highlight, which needs timestamps your engine may not provide
- Automatic skipping of reference lists and footnotes in academic papers

===== .env.example =====
# Copy to .env and fill in. Never commit .env; this file documents it.

# Required. Your local speech server. The OpenAI-compatible shape is what makes engines swappable.
TTS_ENDPOINT=http://localhost:8880/v1/audio/speech

# Required. Voice identifier from your engine's voice list.
TTS_VOICE=af_sky

# Required. kokoro or piper. Chosen by the ratio measured in Phase 1.
TTS_ENGINE=kokoro

# Required. How far ahead to synthesise. Derived from the synthesis ratio, not guessed.
PREFETCH_SENTENCES=4

# Required. Where cached sentence audio is written. Safe to delete.
AUDIO_CACHE_DIR=./data/audio

# Required. Cache ceiling. Oldest entries are evicted first.
AUDIO_CACHE_MAX_GB=5

# Required. Library, documents, sentences and progress. This is the backup.
DB_PATH=./data/reader.db

You are building a production product version of Speechify.

Create the following project files first, then implement the application by following them. Keep the files updated as decisions change. Do not collapse this into a single README or prompt.

===== PRODUCT.md =====
# Speechify · product brief

## Problem

Speechify's core reading loop is a weekend build: import documents, extract text, read it aloud with a natural local voice, highlight the current sentence, and save progress. The gap is product depth, including the size and consistency of Speechify's hosted voice catalog, OCR and mobile capture, cross-device sync, cloud-drive integrations, voice typing, AI podcasts, and document chat.

## Product outcome

A reader that turns awkward real documents into clean speech with a highlight that stays honest through a whole book, resumes exactly where you left off, and has settled the voice licensing question before anyone else uses it.

## Target user

A builder who needs a maintainable product foundation, not a one-off demo.

## Required capabilities

- Node.js 22
- local OpenAI-compatible Kokoro TTS server
- PDF, EPUB, and OCR parsing libraries

## Explicit non-goals for v1

- The hosted voice catalogue and its consistency across devices.
- Mobile scanning and optical character recognition capture.
- Cross-device sync of library and position.
- Native mobile applications with background audio.
- Speechify's 1000+ hosted voices and consistent quality across devices
- mobile scanning and polished OCR capture
- cross-device sync and offline native apps
- Google Drive, Dropbox, and OneDrive integrations
- voice typing, AI podcasts, and document chat

## Success criteria

- Highlight and audio still agree after twenty minutes of continuous playback
- The two column PDF fixture extracts in correct reading order
- Ten minutes of playback has no audible gaps
- Every bundled voice licence is recorded and permits the use you are making
- Clearing the audio cache is proved not to affect documents or progress

===== BRIEF.md =====
# Build brief · Speechify

The one-shot brief this plan expands. `BUILD_PLAN.md` (or `MILESTONES.md`) is the same sequence broken into steps and checks; where the two disagree, the plan wins.

Build me a reader that turns documents and web pages into speech with a local neural voice and synchronised highlighting to replace Speechify. Build it in phases, in the order below.
Do not write the whole thing in one pass. Finish a phase, run its "Done when"
check, fix what fails, and only then start the next phase.

### Stack (fixed, do not substitute)
- Runtime: Node 22 with a small HTTP server and a browser front end · the work is parsing files and orchestrating audio, and the browser is already a good audio player
- Speech: Kokoro served behind an OpenAI-compatible endpoint, with Piper as the lighter fallback · neural quality locally, and an interface shape that lets you swap engines without touching the reader
- Extraction: pdfjs for PDF, epub parsing for EPUB, and Readability for web pages · three formats, three mature parsers, none of them worth writing
- Segmentation: Sentence splitting with Intl.Segmenter · the highlight unit is the sentence, and the browser already knows where sentences end in many languages
- Storage: SQLite for the library and progress · documents, positions and cached audio are rows

### Data model (create this before Phase 1)
- documents(id, title, source_kind, source_path, imported_at, language) · source_kind is pdf, epub, web or text
- blocks(id, document_id, ordinal, kind, text) · kind is heading, paragraph or caption, in reading order
- sentences(id, block_id, ordinal, text, char_start, char_end) · the highlight and playback unit
- audio_cache(sentence_id, voice, speed, path, duration_ms) · synthesis is slow enough to be worth caching
- progress(document_id, sentence_id, updated_at) · one row per document, so resuming is exact
- Extraction happens once at import. Playback never reparses the source file.

### Phase 1 · A local voice that speaks a sentence
Build: Run a local text-to-speech server exposing an OpenAI-compatible speech endpoint. Send
  one sentence and play the result. Measure how long synthesis takes relative to the
  audio it produces, because that ratio decides the whole design.
Trap: If synthesis is slower than real time, streaming sentence by sentence will stutter and you must synthesise ahead. Measure the ratio now rather than discovering it during Phase 4, when the fix is a queue you did not plan for.
Done when: A sentence sent to the local server returns audio that plays. The ratio of synthesis time to audio duration is measured and written down. Stopping the server produces a clear error in the application rather than silence.
Do not build yet: No documents, no highlighting, no library. One sentence spoken.

### Phase 2 · Getting clean text out of real files
Build: Import each format into blocks in reading order. Strip headers, footers, page numbers
  and navigation. Preserve headings, because a reader that does not pause at a chapter
  boundary is exhausting.
Trap: PDF has no concept of reading order. Text is positioned glyphs, and a two column layout extracted naively interleaves the columns into nonsense. Detect columns by clustering horizontal positions, and test on an academic paper rather than a single column report.
Done when: A two column PDF extracts in correct reading order, verified by reading the first three paragraphs. Repeated page headers and footers appear at most once in the output. An EPUB preserves chapter boundaries as headings. A web page import contains the article and not the navigation or the cookie banner.
Do not build yet: No speech playback of documents yet.

### Phase 3 · Sentences and synchronised highlighting
Build: Split blocks into sentences with Intl.Segmenter, keeping character offsets into the
  original text so the highlight can be drawn precisely. Play one sentence at a time and
  advance the highlight on audio completion rather than on a timer.
Trap: Driving the highlight from a timer and an estimated duration drifts, and by the third chapter the highlight is a paragraph away from the audio. Advance on the audio element's ended event, which cannot drift because it is the audio itself telling you.
Done when: The highlight matches the spoken sentence after twenty minutes of continuous playback, checked by pausing and reading. Abbreviations such as Dr and e.g. do not split a sentence in the middle. Clicking a sentence starts playback there and the highlight follows correctly from that point.
Do not build yet: No prefetching or library yet, so playback may pause between sentences.

### Phase 4 · Playing ahead so it never stutters
Build: Synthesise several sentences ahead of the one playing and cache the audio by sentence,
  voice and speed. Discard the queue when the user seeks. Handle speed either through the
  audio element's playback rate or by re-synthesising, whichever your engine does better.
Trap: Changing playback rate on the audio element is instant and free but raises pitch on some engines' output; re-synthesising at a different speed sounds better and costs a delay. Pick one deliberately, and if you use the element's rate, check whether your engine's audio survives it.
Done when: Ten minutes of continuous playback contains no audible gap between sentences. Seeking to a distant sentence starts playing in under a second rather than after synthesising everything in between. Changing speed mid-document takes effect within one sentence.
Do not build yet: No library or progress yet.

### Phase 5 · Library, progress and export
Build: A library listing imported documents with progress. Resume at the exact sentence.
  Export a document's audio as a single file for offline listening, and let the cache be
  cleared without losing the documents.
Trap: Storing progress as a character offset breaks the moment a document is re-imported and extraction changes slightly. Store the sentence identity, and if re-import produces different sentences, say so rather than silently resuming somewhere wrong.
Done when: Closing the application mid-document and reopening it resumes at the same sentence. Clearing the audio cache frees disk without losing documents or progress. Exporting a chapter produces one audio file whose duration matches the sum of its sentences.
Do not build yet: Sync and mobile are out of scope entirely.

### Phase 6 · Sharing it, and the licence question
Build: Put authentication in front of it. Then read the licence of every voice model you ship
  and record what it permits, because voice models vary a great deal and some forbid
  redistribution or commercial use.
Trap: Turning a purchased book into an audio file is fine for you and is redistribution the moment you share it. Similarly, a voice model licensed for personal use only does not become shareable because your application is free. Both questions have to be answered before anyone else connects.
Done when: Access requires authentication. Every bundled voice model has its licence recorded with what it permits. The interface states that exported audio is for personal use and does not offer a share link.
Do not build yet: Nothing after this.

### Out of scope (and why)
- The thousand hosted voices and their consistency across devices, which is a licensing and infrastructure operation rather than a feature.
- Mobile applications with camera scanning and polished optical character recognition, which is the way most people actually use the paid product.
- Cross-device sync of position and library, which needs accounts and a server.
- Offline native applications on phones, and the background audio behaviour that platform gives them.

### README must contain
- The measured ratio of synthesis time to audio duration on your machine, and what it implies for prefetching
- That highlighting advances on audio completion rather than on a timer, and why
- How PDF column detection works and which documents it still gets wrong
- Where the audio cache lives, how large it grows and that clearing it is safe
- The licence of every bundled voice and the personal use position on exported audio

===== ARCHITECTURE.md =====
# Architecture · Speechify

## Stack

| Part | Choice | Why |
| --- | --- | --- |
| Runtime | Node 22 with a small HTTP server and a browser front end | the work is parsing files and orchestrating audio, and the browser is already a good audio player |
| Speech | Kokoro served behind an OpenAI-compatible endpoint, with Piper as the lighter fallback | neural quality locally, and an interface shape that lets you swap engines without touching the reader |
| Extraction | pdfjs for PDF, epub parsing for EPUB, and Readability for web pages | three formats, three mature parsers, none of them worth writing |
| Segmentation | Sentence splitting with Intl.Segmenter | the highlight unit is the sentence, and the browser already knows where sentences end in many languages |
| Storage | SQLite for the library and progress | documents, positions and cached audio are rows |

## Modules

Each module has one owner concern and a documented way to replace it.

| Module | Owns | How to replace it |
| --- | --- | --- |
| Import | Turning files into ordered blocks. | Each format is an independent module behind one interface, so adding a format touches nothing else. PDF is the hard one and always will be. |
| Segment | Splitting blocks into sentences with offsets. | Intl.Segmenter is built in and good. A specialised splitter would improve abbreviation handling at the cost of a dependency. |
| Speech | Synthesis behind one interface. | Kokoro and Piper are already interchangeable through the OpenAI-compatible shape, which is the reason to adopt that shape rather than a native API. |
| Player | Playback, highlighting and seeking. | The rule that advancement comes from the audio ended event, never a timer, must survive any rewrite. |
| Cache | Prefetched audio and eviction. | Disposable by design. Losing it costs synthesis time and nothing else. |

## Configuration

Every runtime setting is an environment variable documented in `.env.example`, validated at startup, with a safe local default wherever one exists.

- `TTS_ENDPOINT` · required · Your local speech server. The OpenAI-compatible shape is what makes engines swappable.
- `TTS_VOICE` · required · Voice identifier from your engine's voice list.
- `TTS_ENGINE` · required · kokoro or piper. Chosen by the ratio measured in Phase 1.
- `PREFETCH_SENTENCES` · required · How far ahead to synthesise. Derived from the synthesis ratio, not guessed.
- `AUDIO_CACHE_DIR` · required · Where cached sentence audio is written. Safe to delete.
- `AUDIO_CACHE_MAX_GB` · required · Cache ceiling. Oldest entries are evicted first.
- `DB_PATH` · required · Library, documents, sentences and progress. This is the backup.

## Production baseline

- Security: least privilege, input validation at every boundary, secret redaction in logs, rate limits on abuse-prone paths, no invented security primitives.
- Data: explicit schema and migrations, transactional writes where integrity matters, backup and restore procedures that have been exercised.
- Integrations: adapters around third-party providers, idempotent webhook or job processing, bounded retries, timeouts.
- Observability: structured logs with request or operation ids, an error-tracking hook, and health and readiness checks where a server exists.
- Quality: unit tests for domain rules, integration tests at module boundaries, one end-to-end test of the critical path.

## Decision records

For each dependency in the stack table, keep a short note: why it was chosen, its failure mode, and how it is replaced. Do not add infrastructure until a requirement in `PRODUCT.md` justifies it.

===== AGENTS.md =====
# Agent instructions · Speechify product build

- Read `PRODUCT.md` and `ARCHITECTURE.md` before changing code. The stack is fixed: Node 22 with a small HTTP server and a browser front end, Kokoro served behind an OpenAI-compatible endpoint, with Piper as the lighter fallback, pdfjs for PDF, epub parsing for EPUB, and Readability for web pages, Sentence splitting with Intl.Segmenter, SQLite for the library and progress.
- Implement milestone by milestone from `MILESTONES.md`; keep each change reviewable and leave the application runnable at every commit.
- Treat authentication, payments, encryption, imports, webhooks and destructive actions as high-risk boundaries when present.
- Never invent cryptography or silently weaken a requirement to make a check pass.
- Put every external service behind an interface with a deterministic fake for tests.
- Add migrations and rollback or recovery notes for every persistent data change.
- Log useful operational context without credentials, tokens, passwords or personal data.
- Update documentation and run every check before completing a milestone.

## Known traps

- If synthesis is slower than real time, streaming sentence by sentence will stutter and you must synthesise ahead. Measure the ratio now rather than discovering it during Phase 4, when the fix is a queue you did not plan for.
- PDF has no concept of reading order. Text is positioned glyphs, and a two column layout extracted naively interleaves the columns into nonsense. Detect columns by clustering horizontal positions, and test on an academic paper rather than a single column report.
- Driving the highlight from a timer and an estimated duration drifts, and by the third chapter the highlight is a paragraph away from the audio. Advance on the audio element's ended event, which cannot drift because it is the audio itself telling you.
- Changing playback rate on the audio element is instant and free but raises pitch on some engines' output; re-synthesising at a different speed sounds better and costs a delay. Pick one deliberately, and if you use the element's rate, check whether your engine's audio survives it.
- Storing progress as a character offset breaks the moment a document is re-imported and extraction changes slightly. Store the sentence identity, and if re-import produces different sentences, say so rather than silently resuming somewhere wrong.
- Turning a purchased book into an audio file is fine for you and is redistribution the moment you share it. Similarly, a voice model licensed for personal use only does not become shareable because your application is free. Both questions have to be answered before anyone else connects.

===== MILESTONES.md =====
# Delivery milestones · Speechify

Estimated effort: **weekend** for the indie phases; the production-only milestones add the trust and operability layer.

## M1 · A local voice that speaks a sentence

Text goes in, audio comes out, from a server on your own machine.

### Steps

1. Run the speech server
   Start Kokoro behind its OpenAI-compatible endpoint, or Piper if your machine is modest. Confirm with a curl before writing any application code.

   ```sh
   curl -s http://localhost:8880/v1/audio/speech -H 'Content-Type: application/json' -d '{"model":"kokoro","input":"Hello there.","voice":"af_sky"}' --output test.wav
   ```

2. Wrap it behind one interface
   A single synthesize function taking text, voice and speed. Every engine difference lives behind it, which is what makes Piper a fallback rather than a rewrite.
   Files: `src/tts.ts`
3. Measure the synthesis ratio
   Time synthesis against the duration of the audio produced, across ten sentences of varying length. Write the ratio in the README.
   Files: `README.md`

### Done when

- [ ] A sentence returns playable audio from the local server
- [ ] The synthesis to duration ratio is measured and recorded
- [ ] A stopped server produces a clear error rather than silence
- [ ] Switching to the fallback engine requires changing one configuration value

### Watch out

- If synthesis is slower than real time, streaming sentence by sentence will stutter and you must synthesise ahead. Measure the ratio now rather than discovering it during Phase 4, when the fix is a queue you did not plan for.

## M2 · Getting clean text out of real files

A PDF, an EPUB and a web page all become the same clean block structure.

### Steps

1. Extract PDF with column detection
   pdfjs gives positioned text items. Cluster their x positions per page to detect columns, then order within each column before moving to the next.
   Files: `src/import/pdf.ts`
2. Strip repeating furniture
   Text appearing at the same position on many pages is a header or a footer. Detect by repetition across pages rather than by guessing margins.
   Files: `src/import/pdf.ts`
3. Import EPUB and web pages
   EPUB is structured already, so preserve its heading levels. For web pages, use Readability to get the article, then convert to the same block shape.
   Files: `src/import/epub.ts`, `src/import/web.ts`

### Done when

- [ ] A two column academic PDF reads in the correct order
- [ ] Page headers and footers do not repeat through the extracted text
- [ ] EPUB chapter boundaries survive as headings
- [ ] A web import contains the article and not the navigation

### Watch out

- PDF has no concept of reading order. Text is positioned glyphs, and a two column layout extracted naively interleaves the columns into nonsense. Detect columns by clustering horizontal positions, and test on an academic paper rather than a single column report.

## M3 · Sentences and synchronised highlighting

The highlighted sentence is the sentence you are hearing, all the way through a long document.

### Steps

1. Segment with offsets preserved
   Intl.Segmenter with the sentence granularity, storing character start and end so the highlight maps back to the rendered text exactly.
   Files: `src/segment.ts`
2. Advance on audio completion
   Play a sentence, wait for the ended event, then advance. No timers anywhere in the playback loop.
   Files: `src/player.ts`
3. Make sentences clickable
   Clicking sets the position and starts from there, which is also how you will test that offsets are right.
   Files: `src/reader.ts`

### Done when

- [ ] Highlight and audio still agree after twenty minutes of playback
- [ ] Common abbreviations do not cause a mid-sentence split
- [ ] Clicking a sentence starts playback at that sentence
- [ ] Pausing and resuming continues from the same sentence

### Watch out

- Driving the highlight from a timer and an estimated duration drifts, and by the third chapter the highlight is a paragraph away from the audio. Advance on the audio element's ended event, which cannot drift because it is the audio itself telling you.

## M4 · Playing ahead so it never stutters

Playback is continuous, and speed changes take effect without a gap.

### Steps

1. Prefetch a small queue
   Keep a few sentences ahead synthesised. The right number is derived from the ratio you measured in Phase 1, not guessed.
   Files: `src/prefetch.ts`
2. Cache by sentence, voice and speed
   The cache key must include voice and speed or a speed change will replay stale audio.
   Files: `src/cache.ts`
3. Cancel work on seek
   Abort in-flight synthesis when the user jumps. Otherwise the queue delivers audio for a position nobody is at any more.
   Files: `src/prefetch.ts`

### Done when

- [ ] Ten minutes of playback has no audible gaps
- [ ] Seeking far ahead begins playing in under a second
- [ ] A speed change takes effect within one sentence
- [ ] Cached audio is keyed by voice and speed

### Watch out

- Changing playback rate on the audio element is instant and free but raises pitch on some engines' output; re-synthesising at a different speed sounds better and costs a delay. Pick one deliberately, and if you use the element's rate, check whether your engine's audio survives it.

## M5 · Library, progress and export

You can leave mid-chapter and come back a week later exactly where you were.

### Steps

1. Build the library view
   Documents with progress as a proportion and the last read time. Sorted by recency, because that is how people actually return to reading.
   Files: `src/library.ts`
2. Persist progress by sentence identity
   Store the sentence id. On re-import, detect that ids changed and tell the user rather than guessing.
   Files: `src/progress.ts`
3. Export audio and make the cache disposable
   Concatenate cached sentence audio into one file. Clearing the cache must never touch documents or progress.
   Files: `src/export.ts`

### Done when

- [ ] Reopening resumes at the same sentence
- [ ] Clearing the audio cache preserves documents and progress
- [ ] An exported chapter's duration matches the sum of its sentences
- [ ] Re-importing a changed document reports that progress may have shifted

### Watch out

- Storing progress as a character offset breaks the moment a document is re-imported and extraction changes slightly. Store the sentence identity, and if re-import produces different sentences, say so rather than silently resuming somewhere wrong.

## M6 · Sharing it, and the licence question (production only)

If anyone else uses it, the voice licences and the copyright position are settled.

### Steps

1. Add authentication
   Any session mechanism. The point is that an open reader on the internet is a copyright problem waiting to be someone else's.
   Files: `src/auth.ts`
2. Record every voice licence
   A table of model, source and licence terms. Some are permissive, some are personal use only, and the difference matters.
   Files: `docs/voices.md`
3. State the position in the interface
   One sentence about personal use next to the export button. Better there than in a document nobody opens.
   Files: `src/reader.ts`

### Done when

- [ ] Access requires authentication
- [ ] Every bundled voice has its licence recorded
- [ ] The export control states the personal use position
- [ ] No sharing or public link feature exists

### Watch out

- Turning a purchased book into an audio file is fine for you and is redistribution the moment you share it. Similarly, a voice model licensed for personal use only does not become shareable because your application is free. Both questions have to be answered before anyone else connects.

===== OPERATIONS.md =====
# Operations · Speechify

## Backup

The database holds documents, extracted text and progress, and is the only thing worth backing up. The audio cache is regenerable and usually larger than everything else combined.

## Restore

Restore the database and open a document. Playback should resume at the recorded sentence and re-synthesise as it goes, which is also the test that the cache really is disposable.

Do a restore drill before the first real user, and write the date here when it passes.

## Monitoring

For personal use there is nothing to monitor beyond disk. If shared, watch synthesis queue depth, because a slow engine under concurrent use shows up as stuttering rather than as an error.

## Incident checklist

A full disk from the audio cache is the realistic failure. Enforce the cache ceiling rather than trusting eviction to keep up, and fail synthesis with a clear message rather than filling the disk.

1. Contain the issue without destroying evidence or user data.
2. Record the timeline and affected scope.
3. Rotate exposed secrets and revoke compromised sessions or credentials.
4. Restore from a verified backup when needed.
5. Document the root cause, the remediation and the regression test.

## Release gate

- [ ] Highlight and audio still agree after twenty minutes of continuous playback
- [ ] The two column PDF fixture extracts in correct reading order
- [ ] Ten minutes of playback has no audible gaps
- [ ] Every bundled voice licence is recorded and permits the use you are making
- [ ] Clearing the audio cache is proved not to affect documents or progress

## Launch constraint

Do not market omitted Speechify capabilities as implemented. The non-goals in `PRODUCT.md` remain user-visible limitations until they are deliberately delivered.

===== .env.example =====
# Copy to .env and fill in. Never commit .env; this file documents it.

# Required. Your local speech server. The OpenAI-compatible shape is what makes engines swappable.
TTS_ENDPOINT=http://localhost:8880/v1/audio/speech

# Required. Voice identifier from your engine's voice list.
TTS_VOICE=af_sky

# Required. kokoro or piper. Chosen by the ratio measured in Phase 1.
TTS_ENGINE=kokoro

# Required. How far ahead to synthesise. Derived from the synthesis ratio, not guessed.
PREFETCH_SENTENCES=4

# Required. Where cached sentence audio is written. Safe to delete.
AUDIO_CACHE_DIR=./data/audio

# Required. Cache ceiling. Oldest entries are evicted first.
AUDIO_CACHE_MAX_GB=5

# Required. Library, documents, sentences and progress. This is the backup.
DB_PATH=./data/reader.db

# Speechify · indie build

Import PDFs, EPUBs and web pages, extract clean readable text, speak it through a local neural text-to-speech server, highlight the sentence being spoken, control speed, and remember where you stopped in every document.

Estimated effort: **weekend**. Work `BUILD_PLAN.md` top to bottom · every phase ends in a check that has to pass before the next one starts.

## Stack

| Part | Choice | Why |
| --- | --- | --- |
| Runtime | Node 22 with a small HTTP server and a browser front end | the work is parsing files and orchestrating audio, and the browser is already a good audio player |
| Speech | Kokoro served behind an OpenAI-compatible endpoint, with Piper as the lighter fallback | neural quality locally, and an interface shape that lets you swap engines without touching the reader |
| Extraction | pdfjs for PDF, epub parsing for EPUB, and Readability for web pages | three formats, three mature parsers, none of them worth writing |
| Segmentation | Sentence splitting with Intl.Segmenter | the highlight unit is the sentence, and the browser already knows where sentences end in many languages |
| Storage | SQLite for the library and progress | documents, positions and cached audio are rows |

## Before you start

Have every one of these ready. The plan assumes them from step one.

- [ ] **Node.js 22 or newer** · free
  - Why: Runs the server and the import pipeline.
  - Get it: nodejs.org, or nvm install 22.
  - Verify: node --version prints v22 or higher
- [ ] **A local neural text-to-speech server** · free, both are open source; the cost is memory and CPU or GPU time
  - Why: The whole point. Kokoro sounds considerably better; Piper is far lighter and runs comfortably on a laptop with no GPU.
  - Get it: Run Kokoro behind its OpenAI-compatible server image, or install Piper and a voice model. Try both before choosing, because the quality difference is large and so is the resource difference.
  - Verify: A curl to the speech endpoint returns playable audio
- [ ] **Enough memory for the chosen voice model** · free
  - Why: Kokoro wants materially more than Piper, and a model that swaps produces synthesis slower than real time, which changes the design.
  - Get it: Check available memory before choosing. If the ratio measured in Phase 1 is above one, use the lighter engine.
  - Verify: The synthesis ratio measured in Phase 1 is below one
- [ ] **Three awkward test documents** · free
  - Why: Phase 2 is only meaningful against real files: a two column academic PDF, an EPUB with chapters, and a web article behind a cookie banner.
  - Get it: A paper from an open archive, a public domain EPUB, and any news article. Keep them as fixtures.
  - Verify: All three are saved in a fixtures directory
- [ ] **Several gigabytes of free disk for cached audio** · free
  - Why: Cached sentence audio for a long book adds up quickly, and a full disk mid-playback is an unpleasant failure.
  - Get it: Check free space and set the cache limit in configuration accordingly.
  - Verify: The configured cache limit is below your free space

## Quick start

```sh
curl -s http://localhost:8880/v1/audio/speech -H 'Content-Type: application/json' -d '{"model":"kokoro","input":"Hello there.","voice":"af_sky"}' --output test.wav
```

Then copy `.env.example` to `.env` and fill in the values it documents.

## Honest limits

This build deliberately does not replace:

- The hosted voice catalogue and its consistency across devices.
- Mobile scanning and optical character recognition capture.
- Cross-device sync of library and position.
- Native mobile applications with background audio.
- Speechify's 1000+ hosted voices and consistent quality across devices
- mobile scanning and polished OCR capture
- cross-device sync and offline native apps
- Google Drive, Dropbox, and OneDrive integrations
- voice typing, AI podcasts, and document chat

If one of those is essential to you, that is the reason to keep paying for Speechify, and the README should say so rather than pretend.

# Build brief · Speechify

The one-shot brief this plan expands. `BUILD_PLAN.md` (or `MILESTONES.md`) is the same sequence broken into steps and checks; where the two disagree, the plan wins.

Build me a reader that turns documents and web pages into speech with a local neural voice and synchronised highlighting to replace Speechify. Build it in phases, in the order below.
Do not write the whole thing in one pass. Finish a phase, run its "Done when"
check, fix what fails, and only then start the next phase.

### Stack (fixed, do not substitute)
- Runtime: Node 22 with a small HTTP server and a browser front end · the work is parsing files and orchestrating audio, and the browser is already a good audio player
- Speech: Kokoro served behind an OpenAI-compatible endpoint, with Piper as the lighter fallback · neural quality locally, and an interface shape that lets you swap engines without touching the reader
- Extraction: pdfjs for PDF, epub parsing for EPUB, and Readability for web pages · three formats, three mature parsers, none of them worth writing
- Segmentation: Sentence splitting with Intl.Segmenter · the highlight unit is the sentence, and the browser already knows where sentences end in many languages
- Storage: SQLite for the library and progress · documents, positions and cached audio are rows

### Data model (create this before Phase 1)
- documents(id, title, source_kind, source_path, imported_at, language) · source_kind is pdf, epub, web or text
- blocks(id, document_id, ordinal, kind, text) · kind is heading, paragraph or caption, in reading order
- sentences(id, block_id, ordinal, text, char_start, char_end) · the highlight and playback unit
- audio_cache(sentence_id, voice, speed, path, duration_ms) · synthesis is slow enough to be worth caching
- progress(document_id, sentence_id, updated_at) · one row per document, so resuming is exact
- Extraction happens once at import. Playback never reparses the source file.

### Phase 1 · A local voice that speaks a sentence
Build: Run a local text-to-speech server exposing an OpenAI-compatible speech endpoint. Send
  one sentence and play the result. Measure how long synthesis takes relative to the
  audio it produces, because that ratio decides the whole design.
Trap: If synthesis is slower than real time, streaming sentence by sentence will stutter and you must synthesise ahead. Measure the ratio now rather than discovering it during Phase 4, when the fix is a queue you did not plan for.
Done when: A sentence sent to the local server returns audio that plays. The ratio of synthesis time to audio duration is measured and written down. Stopping the server produces a clear error in the application rather than silence.
Do not build yet: No documents, no highlighting, no library. One sentence spoken.

### Phase 2 · Getting clean text out of real files
Build: Import each format into blocks in reading order. Strip headers, footers, page numbers
  and navigation. Preserve headings, because a reader that does not pause at a chapter
  boundary is exhausting.
Trap: PDF has no concept of reading order. Text is positioned glyphs, and a two column layout extracted naively interleaves the columns into nonsense. Detect columns by clustering horizontal positions, and test on an academic paper rather than a single column report.
Done when: A two column PDF extracts in correct reading order, verified by reading the first three paragraphs. Repeated page headers and footers appear at most once in the output. An EPUB preserves chapter boundaries as headings. A web page import contains the article and not the navigation or the cookie banner.
Do not build yet: No speech playback of documents yet.

### Phase 3 · Sentences and synchronised highlighting
Build: Split blocks into sentences with Intl.Segmenter, keeping character offsets into the
  original text so the highlight can be drawn precisely. Play one sentence at a time and
  advance the highlight on audio completion rather than on a timer.
Trap: Driving the highlight from a timer and an estimated duration drifts, and by the third chapter the highlight is a paragraph away from the audio. Advance on the audio element's ended event, which cannot drift because it is the audio itself telling you.
Done when: The highlight matches the spoken sentence after twenty minutes of continuous playback, checked by pausing and reading. Abbreviations such as Dr and e.g. do not split a sentence in the middle. Clicking a sentence starts playback there and the highlight follows correctly from that point.
Do not build yet: No prefetching or library yet, so playback may pause between sentences.

### Phase 4 · Playing ahead so it never stutters
Build: Synthesise several sentences ahead of the one playing and cache the audio by sentence,
  voice and speed. Discard the queue when the user seeks. Handle speed either through the
  audio element's playback rate or by re-synthesising, whichever your engine does better.
Trap: Changing playback rate on the audio element is instant and free but raises pitch on some engines' output; re-synthesising at a different speed sounds better and costs a delay. Pick one deliberately, and if you use the element's rate, check whether your engine's audio survives it.
Done when: Ten minutes of continuous playback contains no audible gap between sentences. Seeking to a distant sentence starts playing in under a second rather than after synthesising everything in between. Changing speed mid-document takes effect within one sentence.
Do not build yet: No library or progress yet.

### Phase 5 · Library, progress and export
Build: A library listing imported documents with progress. Resume at the exact sentence.
  Export a document's audio as a single file for offline listening, and let the cache be
  cleared without losing the documents.
Trap: Storing progress as a character offset breaks the moment a document is re-imported and extraction changes slightly. Store the sentence identity, and if re-import produces different sentences, say so rather than silently resuming somewhere wrong.
Done when: Closing the application mid-document and reopening it resumes at the same sentence. Clearing the audio cache frees disk without losing documents or progress. Exporting a chapter produces one audio file whose duration matches the sum of its sentences.
Do not build yet: Sync and mobile are out of scope entirely.

### Phase 6 · Sharing it, and the licence question
Build: Put authentication in front of it. Then read the licence of every voice model you ship
  and record what it permits, because voice models vary a great deal and some forbid
  redistribution or commercial use.
Trap: Turning a purchased book into an audio file is fine for you and is redistribution the moment you share it. Similarly, a voice model licensed for personal use only does not become shareable because your application is free. Both questions have to be answered before anyone else connects.
Done when: Access requires authentication. Every bundled voice model has its licence recorded with what it permits. The interface states that exported audio is for personal use and does not offer a share link.
Do not build yet: Nothing after this.

### Out of scope (and why)
- The thousand hosted voices and their consistency across devices, which is a licensing and infrastructure operation rather than a feature.
- Mobile applications with camera scanning and polished optical character recognition, which is the way most people actually use the paid product.
- Cross-device sync of position and library, which needs accounts and a server.
- Offline native applications on phones, and the background audio behaviour that platform gives them.

### README must contain
- The measured ratio of synthesis time to audio duration on your machine, and what it implies for prefetching
- That highlighting advances on audio completion rather than on a timer, and why
- How PDF column detection works and which documents it still gets wrong
- Where the audio cache lives, how large it grows and that clearing it is safe
- The licence of every bundled voice and the personal use position on exported audio

# Agent instructions · Speechify indie build

- Read `README.md` and `BUILD_PLAN.md` before writing code. The stack is fixed: Node 22 with a small HTTP server and a browser front end, Kokoro served behind an OpenAI-compatible endpoint, with Piper as the lighter fallback, pdfjs for PDF, epub parsing for EPUB, and Readability for web pages, Sentence splitting with Intl.Segmenter, SQLite for the library and progress. Do not substitute.
- Work one phase at a time, in order. Do not start a phase until every "Done when" item of the previous one passes.
- Prefer the fewest moving parts that satisfy the step. No frameworks, services or dependencies the plan does not name.
- Secrets live in `.env`, never in source or logs. Keep `.env.example` current when a variable is introduced.
- Do not invent cryptography, security guarantees, APIs or compliance claims.
- Add a focused test for every destructive, security-sensitive or data-loss path the plan names.
- Run the project checks before declaring a phase complete, and record any deliberate shortcut in the README under "Tradeoffs".

## Known traps

- If synthesis is slower than real time, streaming sentence by sentence will stutter and you must synthesise ahead. Measure the ratio now rather than discovering it during Phase 4, when the fix is a queue you did not plan for.
- PDF has no concept of reading order. Text is positioned glyphs, and a two column layout extracted naively interleaves the columns into nonsense. Detect columns by clustering horizontal positions, and test on an academic paper rather than a single column report.
- Driving the highlight from a timer and an estimated duration drifts, and by the third chapter the highlight is a paragraph away from the audio. Advance on the audio element's ended event, which cannot drift because it is the audio itself telling you.
- Changing playback rate on the audio element is instant and free but raises pitch on some engines' output; re-synthesising at a different speed sounds better and costs a delay. Pick one deliberately, and if you use the element's rate, check whether your engine's audio survives it.
- Storing progress as a character offset breaks the moment a document is re-imported and extraction changes slightly. Store the sentence identity, and if re-import produces different sentences, say so rather than silently resuming somewhere wrong.

# Build plan · Speechify

Import PDFs, EPUBs and web pages, extract clean readable text, speak it through a local neural text-to-speech server, highlight the sentence being spoken, control speed, and remember where you stopped in every document.

Phases are in dependency order. Each ends in a "Done when" list; treat an unticked item as a blocker, not a note.

## Phase 1 · A local voice that speaks a sentence

Text goes in, audio comes out, from a server on your own machine.

### Steps

1. Run the speech server
   Start Kokoro behind its OpenAI-compatible endpoint, or Piper if your machine is modest. Confirm with a curl before writing any application code.

   ```sh
   curl -s http://localhost:8880/v1/audio/speech -H 'Content-Type: application/json' -d '{"model":"kokoro","input":"Hello there.","voice":"af_sky"}' --output test.wav
   ```

2. Wrap it behind one interface
   A single synthesize function taking text, voice and speed. Every engine difference lives behind it, which is what makes Piper a fallback rather than a rewrite.
   Files: `src/tts.ts`
3. Measure the synthesis ratio
   Time synthesis against the duration of the audio produced, across ten sentences of varying length. Write the ratio in the README.
   Files: `README.md`

### Done when

- [ ] A sentence returns playable audio from the local server
- [ ] The synthesis to duration ratio is measured and recorded
- [ ] A stopped server produces a clear error rather than silence
- [ ] Switching to the fallback engine requires changing one configuration value

### Watch out

- If synthesis is slower than real time, streaming sentence by sentence will stutter and you must synthesise ahead. Measure the ratio now rather than discovering it during Phase 4, when the fix is a queue you did not plan for.

## Phase 2 · Getting clean text out of real files

A PDF, an EPUB and a web page all become the same clean block structure.

### Steps

1. Extract PDF with column detection
   pdfjs gives positioned text items. Cluster their x positions per page to detect columns, then order within each column before moving to the next.
   Files: `src/import/pdf.ts`
2. Strip repeating furniture
   Text appearing at the same position on many pages is a header or a footer. Detect by repetition across pages rather than by guessing margins.
   Files: `src/import/pdf.ts`
3. Import EPUB and web pages
   EPUB is structured already, so preserve its heading levels. For web pages, use Readability to get the article, then convert to the same block shape.
   Files: `src/import/epub.ts`, `src/import/web.ts`

### Done when

- [ ] A two column academic PDF reads in the correct order
- [ ] Page headers and footers do not repeat through the extracted text
- [ ] EPUB chapter boundaries survive as headings
- [ ] A web import contains the article and not the navigation

### Watch out

- PDF has no concept of reading order. Text is positioned glyphs, and a two column layout extracted naively interleaves the columns into nonsense. Detect columns by clustering horizontal positions, and test on an academic paper rather than a single column report.

## Phase 3 · Sentences and synchronised highlighting

The highlighted sentence is the sentence you are hearing, all the way through a long document.

### Steps

1. Segment with offsets preserved
   Intl.Segmenter with the sentence granularity, storing character start and end so the highlight maps back to the rendered text exactly.
   Files: `src/segment.ts`
2. Advance on audio completion
   Play a sentence, wait for the ended event, then advance. No timers anywhere in the playback loop.
   Files: `src/player.ts`
3. Make sentences clickable
   Clicking sets the position and starts from there, which is also how you will test that offsets are right.
   Files: `src/reader.ts`

### Done when

- [ ] Highlight and audio still agree after twenty minutes of playback
- [ ] Common abbreviations do not cause a mid-sentence split
- [ ] Clicking a sentence starts playback at that sentence
- [ ] Pausing and resuming continues from the same sentence

### Watch out

- Driving the highlight from a timer and an estimated duration drifts, and by the third chapter the highlight is a paragraph away from the audio. Advance on the audio element's ended event, which cannot drift because it is the audio itself telling you.

## Phase 4 · Playing ahead so it never stutters

Playback is continuous, and speed changes take effect without a gap.

### Steps

1. Prefetch a small queue
   Keep a few sentences ahead synthesised. The right number is derived from the ratio you measured in Phase 1, not guessed.
   Files: `src/prefetch.ts`
2. Cache by sentence, voice and speed
   The cache key must include voice and speed or a speed change will replay stale audio.
   Files: `src/cache.ts`
3. Cancel work on seek
   Abort in-flight synthesis when the user jumps. Otherwise the queue delivers audio for a position nobody is at any more.
   Files: `src/prefetch.ts`

### Done when

- [ ] Ten minutes of playback has no audible gaps
- [ ] Seeking far ahead begins playing in under a second
- [ ] A speed change takes effect within one sentence
- [ ] Cached audio is keyed by voice and speed

### Watch out

- Changing playback rate on the audio element is instant and free but raises pitch on some engines' output; re-synthesising at a different speed sounds better and costs a delay. Pick one deliberately, and if you use the element's rate, check whether your engine's audio survives it.

## Phase 5 · Library, progress and export

You can leave mid-chapter and come back a week later exactly where you were.

### Steps

1. Build the library view
   Documents with progress as a proportion and the last read time. Sorted by recency, because that is how people actually return to reading.
   Files: `src/library.ts`
2. Persist progress by sentence identity
   Store the sentence id. On re-import, detect that ids changed and tell the user rather than guessing.
   Files: `src/progress.ts`
3. Export audio and make the cache disposable
   Concatenate cached sentence audio into one file. Clearing the cache must never touch documents or progress.
   Files: `src/export.ts`

### Done when

- [ ] Reopening resumes at the same sentence
- [ ] Clearing the audio cache preserves documents and progress
- [ ] An exported chapter's duration matches the sum of its sentences
- [ ] Re-importing a changed document reports that progress may have shifted

### Watch out

- Storing progress as a character offset breaks the moment a document is re-imported and extraction changes slightly. Store the sentence identity, and if re-import produces different sentences, say so rather than silently resuming somewhere wrong.

## Not in this build

- The hosted voice catalogue and its consistency across devices.
- Mobile scanning and optical character recognition capture.
- Cross-device sync of library and position.
- Native mobile applications with background audio.

## After v1, if you want it

- Optical character recognition for scanned PDFs that contain no text layer
- A word-level highlight, which needs timestamps your engine may not provide
- Automatic skipping of reference lists and footnotes in academic papers

# Copy to .env and fill in. Never commit .env; this file documents it.

# Required. Your local speech server. The OpenAI-compatible shape is what makes engines swappable.
TTS_ENDPOINT=http://localhost:8880/v1/audio/speech

# Required. Voice identifier from your engine's voice list.
TTS_VOICE=af_sky

# Required. kokoro or piper. Chosen by the ratio measured in Phase 1.
TTS_ENGINE=kokoro

# Required. How far ahead to synthesise. Derived from the synthesis ratio, not guessed.
PREFETCH_SENTENCES=4

# Required. Where cached sentence audio is written. Safe to delete.
AUDIO_CACHE_DIR=./data/audio

# Required. Cache ceiling. Oldest entries are evicted first.
AUDIO_CACHE_MAX_GB=5

# Required. Library, documents, sentences and progress. This is the backup.
DB_PATH=./data/reader.db

# Speechify · product brief

## Problem

Speechify's core reading loop is a weekend build: import documents, extract text, read it aloud with a natural local voice, highlight the current sentence, and save progress. The gap is product depth, including the size and consistency of Speechify's hosted voice catalog, OCR and mobile capture, cross-device sync, cloud-drive integrations, voice typing, AI podcasts, and document chat.

## Product outcome

A reader that turns awkward real documents into clean speech with a highlight that stays honest through a whole book, resumes exactly where you left off, and has settled the voice licensing question before anyone else uses it.

## Target user

A builder who needs a maintainable product foundation, not a one-off demo.

## Required capabilities

- Node.js 22
- local OpenAI-compatible Kokoro TTS server
- PDF, EPUB, and OCR parsing libraries

## Explicit non-goals for v1

- The hosted voice catalogue and its consistency across devices.
- Mobile scanning and optical character recognition capture.
- Cross-device sync of library and position.
- Native mobile applications with background audio.
- Speechify's 1000+ hosted voices and consistent quality across devices
- mobile scanning and polished OCR capture
- cross-device sync and offline native apps
- Google Drive, Dropbox, and OneDrive integrations
- voice typing, AI podcasts, and document chat

## Success criteria

- Highlight and audio still agree after twenty minutes of continuous playback
- The two column PDF fixture extracts in correct reading order
- Ten minutes of playback has no audible gaps
- Every bundled voice licence is recorded and permits the use you are making
- Clearing the audio cache is proved not to affect documents or progress

# Build brief · Speechify

The one-shot brief this plan expands. `BUILD_PLAN.md` (or `MILESTONES.md`) is the same sequence broken into steps and checks; where the two disagree, the plan wins.

Build me a reader that turns documents and web pages into speech with a local neural voice and synchronised highlighting to replace Speechify. Build it in phases, in the order below.
Do not write the whole thing in one pass. Finish a phase, run its "Done when"
check, fix what fails, and only then start the next phase.

### Stack (fixed, do not substitute)
- Runtime: Node 22 with a small HTTP server and a browser front end · the work is parsing files and orchestrating audio, and the browser is already a good audio player
- Speech: Kokoro served behind an OpenAI-compatible endpoint, with Piper as the lighter fallback · neural quality locally, and an interface shape that lets you swap engines without touching the reader
- Extraction: pdfjs for PDF, epub parsing for EPUB, and Readability for web pages · three formats, three mature parsers, none of them worth writing
- Segmentation: Sentence splitting with Intl.Segmenter · the highlight unit is the sentence, and the browser already knows where sentences end in many languages
- Storage: SQLite for the library and progress · documents, positions and cached audio are rows

### Data model (create this before Phase 1)
- documents(id, title, source_kind, source_path, imported_at, language) · source_kind is pdf, epub, web or text
- blocks(id, document_id, ordinal, kind, text) · kind is heading, paragraph or caption, in reading order
- sentences(id, block_id, ordinal, text, char_start, char_end) · the highlight and playback unit
- audio_cache(sentence_id, voice, speed, path, duration_ms) · synthesis is slow enough to be worth caching
- progress(document_id, sentence_id, updated_at) · one row per document, so resuming is exact
- Extraction happens once at import. Playback never reparses the source file.

### Phase 1 · A local voice that speaks a sentence
Build: Run a local text-to-speech server exposing an OpenAI-compatible speech endpoint. Send
  one sentence and play the result. Measure how long synthesis takes relative to the
  audio it produces, because that ratio decides the whole design.
Trap: If synthesis is slower than real time, streaming sentence by sentence will stutter and you must synthesise ahead. Measure the ratio now rather than discovering it during Phase 4, when the fix is a queue you did not plan for.
Done when: A sentence sent to the local server returns audio that plays. The ratio of synthesis time to audio duration is measured and written down. Stopping the server produces a clear error in the application rather than silence.
Do not build yet: No documents, no highlighting, no library. One sentence spoken.

### Phase 2 · Getting clean text out of real files
Build: Import each format into blocks in reading order. Strip headers, footers, page numbers
  and navigation. Preserve headings, because a reader that does not pause at a chapter
  boundary is exhausting.
Trap: PDF has no concept of reading order. Text is positioned glyphs, and a two column layout extracted naively interleaves the columns into nonsense. Detect columns by clustering horizontal positions, and test on an academic paper rather than a single column report.
Done when: A two column PDF extracts in correct reading order, verified by reading the first three paragraphs. Repeated page headers and footers appear at most once in the output. An EPUB preserves chapter boundaries as headings. A web page import contains the article and not the navigation or the cookie banner.
Do not build yet: No speech playback of documents yet.

### Phase 3 · Sentences and synchronised highlighting
Build: Split blocks into sentences with Intl.Segmenter, keeping character offsets into the
  original text so the highlight can be drawn precisely. Play one sentence at a time and
  advance the highlight on audio completion rather than on a timer.
Trap: Driving the highlight from a timer and an estimated duration drifts, and by the third chapter the highlight is a paragraph away from the audio. Advance on the audio element's ended event, which cannot drift because it is the audio itself telling you.
Done when: The highlight matches the spoken sentence after twenty minutes of continuous playback, checked by pausing and reading. Abbreviations such as Dr and e.g. do not split a sentence in the middle. Clicking a sentence starts playback there and the highlight follows correctly from that point.
Do not build yet: No prefetching or library yet, so playback may pause between sentences.

### Phase 4 · Playing ahead so it never stutters
Build: Synthesise several sentences ahead of the one playing and cache the audio by sentence,
  voice and speed. Discard the queue when the user seeks. Handle speed either through the
  audio element's playback rate or by re-synthesising, whichever your engine does better.
Trap: Changing playback rate on the audio element is instant and free but raises pitch on some engines' output; re-synthesising at a different speed sounds better and costs a delay. Pick one deliberately, and if you use the element's rate, check whether your engine's audio survives it.
Done when: Ten minutes of continuous playback contains no audible gap between sentences. Seeking to a distant sentence starts playing in under a second rather than after synthesising everything in between. Changing speed mid-document takes effect within one sentence.
Do not build yet: No library or progress yet.

### Phase 5 · Library, progress and export
Build: A library listing imported documents with progress. Resume at the exact sentence.
  Export a document's audio as a single file for offline listening, and let the cache be
  cleared without losing the documents.
Trap: Storing progress as a character offset breaks the moment a document is re-imported and extraction changes slightly. Store the sentence identity, and if re-import produces different sentences, say so rather than silently resuming somewhere wrong.
Done when: Closing the application mid-document and reopening it resumes at the same sentence. Clearing the audio cache frees disk without losing documents or progress. Exporting a chapter produces one audio file whose duration matches the sum of its sentences.
Do not build yet: Sync and mobile are out of scope entirely.

### Phase 6 · Sharing it, and the licence question
Build: Put authentication in front of it. Then read the licence of every voice model you ship
  and record what it permits, because voice models vary a great deal and some forbid
  redistribution or commercial use.
Trap: Turning a purchased book into an audio file is fine for you and is redistribution the moment you share it. Similarly, a voice model licensed for personal use only does not become shareable because your application is free. Both questions have to be answered before anyone else connects.
Done when: Access requires authentication. Every bundled voice model has its licence recorded with what it permits. The interface states that exported audio is for personal use and does not offer a share link.
Do not build yet: Nothing after this.

### Out of scope (and why)
- The thousand hosted voices and their consistency across devices, which is a licensing and infrastructure operation rather than a feature.
- Mobile applications with camera scanning and polished optical character recognition, which is the way most people actually use the paid product.
- Cross-device sync of position and library, which needs accounts and a server.
- Offline native applications on phones, and the background audio behaviour that platform gives them.

### README must contain
- The measured ratio of synthesis time to audio duration on your machine, and what it implies for prefetching
- That highlighting advances on audio completion rather than on a timer, and why
- How PDF column detection works and which documents it still gets wrong
- Where the audio cache lives, how large it grows and that clearing it is safe
- The licence of every bundled voice and the personal use position on exported audio

# Architecture · Speechify

## Stack

| Part | Choice | Why |
| --- | --- | --- |
| Runtime | Node 22 with a small HTTP server and a browser front end | the work is parsing files and orchestrating audio, and the browser is already a good audio player |
| Speech | Kokoro served behind an OpenAI-compatible endpoint, with Piper as the lighter fallback | neural quality locally, and an interface shape that lets you swap engines without touching the reader |
| Extraction | pdfjs for PDF, epub parsing for EPUB, and Readability for web pages | three formats, three mature parsers, none of them worth writing |
| Segmentation | Sentence splitting with Intl.Segmenter | the highlight unit is the sentence, and the browser already knows where sentences end in many languages |
| Storage | SQLite for the library and progress | documents, positions and cached audio are rows |

## Modules

Each module has one owner concern and a documented way to replace it.

| Module | Owns | How to replace it |
| --- | --- | --- |
| Import | Turning files into ordered blocks. | Each format is an independent module behind one interface, so adding a format touches nothing else. PDF is the hard one and always will be. |
| Segment | Splitting blocks into sentences with offsets. | Intl.Segmenter is built in and good. A specialised splitter would improve abbreviation handling at the cost of a dependency. |
| Speech | Synthesis behind one interface. | Kokoro and Piper are already interchangeable through the OpenAI-compatible shape, which is the reason to adopt that shape rather than a native API. |
| Player | Playback, highlighting and seeking. | The rule that advancement comes from the audio ended event, never a timer, must survive any rewrite. |
| Cache | Prefetched audio and eviction. | Disposable by design. Losing it costs synthesis time and nothing else. |

## Configuration

Every runtime setting is an environment variable documented in `.env.example`, validated at startup, with a safe local default wherever one exists.

- `TTS_ENDPOINT` · required · Your local speech server. The OpenAI-compatible shape is what makes engines swappable.
- `TTS_VOICE` · required · Voice identifier from your engine's voice list.
- `TTS_ENGINE` · required · kokoro or piper. Chosen by the ratio measured in Phase 1.
- `PREFETCH_SENTENCES` · required · How far ahead to synthesise. Derived from the synthesis ratio, not guessed.
- `AUDIO_CACHE_DIR` · required · Where cached sentence audio is written. Safe to delete.
- `AUDIO_CACHE_MAX_GB` · required · Cache ceiling. Oldest entries are evicted first.
- `DB_PATH` · required · Library, documents, sentences and progress. This is the backup.

## Production baseline

- Security: least privilege, input validation at every boundary, secret redaction in logs, rate limits on abuse-prone paths, no invented security primitives.
- Data: explicit schema and migrations, transactional writes where integrity matters, backup and restore procedures that have been exercised.
- Integrations: adapters around third-party providers, idempotent webhook or job processing, bounded retries, timeouts.
- Observability: structured logs with request or operation ids, an error-tracking hook, and health and readiness checks where a server exists.
- Quality: unit tests for domain rules, integration tests at module boundaries, one end-to-end test of the critical path.

## Decision records

For each dependency in the stack table, keep a short note: why it was chosen, its failure mode, and how it is replaced. Do not add infrastructure until a requirement in `PRODUCT.md` justifies it.

# Agent instructions · Speechify product build

- Read `PRODUCT.md` and `ARCHITECTURE.md` before changing code. The stack is fixed: Node 22 with a small HTTP server and a browser front end, Kokoro served behind an OpenAI-compatible endpoint, with Piper as the lighter fallback, pdfjs for PDF, epub parsing for EPUB, and Readability for web pages, Sentence splitting with Intl.Segmenter, SQLite for the library and progress.
- Implement milestone by milestone from `MILESTONES.md`; keep each change reviewable and leave the application runnable at every commit.
- Treat authentication, payments, encryption, imports, webhooks and destructive actions as high-risk boundaries when present.
- Never invent cryptography or silently weaken a requirement to make a check pass.
- Put every external service behind an interface with a deterministic fake for tests.
- Add migrations and rollback or recovery notes for every persistent data change.
- Log useful operational context without credentials, tokens, passwords or personal data.
- Update documentation and run every check before completing a milestone.

## Known traps

- If synthesis is slower than real time, streaming sentence by sentence will stutter and you must synthesise ahead. Measure the ratio now rather than discovering it during Phase 4, when the fix is a queue you did not plan for.
- PDF has no concept of reading order. Text is positioned glyphs, and a two column layout extracted naively interleaves the columns into nonsense. Detect columns by clustering horizontal positions, and test on an academic paper rather than a single column report.
- Driving the highlight from a timer and an estimated duration drifts, and by the third chapter the highlight is a paragraph away from the audio. Advance on the audio element's ended event, which cannot drift because it is the audio itself telling you.
- Changing playback rate on the audio element is instant and free but raises pitch on some engines' output; re-synthesising at a different speed sounds better and costs a delay. Pick one deliberately, and if you use the element's rate, check whether your engine's audio survives it.
- Storing progress as a character offset breaks the moment a document is re-imported and extraction changes slightly. Store the sentence identity, and if re-import produces different sentences, say so rather than silently resuming somewhere wrong.
- Turning a purchased book into an audio file is fine for you and is redistribution the moment you share it. Similarly, a voice model licensed for personal use only does not become shareable because your application is free. Both questions have to be answered before anyone else connects.

# Delivery milestones · Speechify

Estimated effort: **weekend** for the indie phases; the production-only milestones add the trust and operability layer.

## M1 · A local voice that speaks a sentence

Text goes in, audio comes out, from a server on your own machine.

### Steps

1. Run the speech server
   Start Kokoro behind its OpenAI-compatible endpoint, or Piper if your machine is modest. Confirm with a curl before writing any application code.

   ```sh
   curl -s http://localhost:8880/v1/audio/speech -H 'Content-Type: application/json' -d '{"model":"kokoro","input":"Hello there.","voice":"af_sky"}' --output test.wav
   ```

2. Wrap it behind one interface
   A single synthesize function taking text, voice and speed. Every engine difference lives behind it, which is what makes Piper a fallback rather than a rewrite.
   Files: `src/tts.ts`
3. Measure the synthesis ratio
   Time synthesis against the duration of the audio produced, across ten sentences of varying length. Write the ratio in the README.
   Files: `README.md`

### Done when

- [ ] A sentence returns playable audio from the local server
- [ ] The synthesis to duration ratio is measured and recorded
- [ ] A stopped server produces a clear error rather than silence
- [ ] Switching to the fallback engine requires changing one configuration value

### Watch out

- If synthesis is slower than real time, streaming sentence by sentence will stutter and you must synthesise ahead. Measure the ratio now rather than discovering it during Phase 4, when the fix is a queue you did not plan for.

## M2 · Getting clean text out of real files

A PDF, an EPUB and a web page all become the same clean block structure.

### Steps

1. Extract PDF with column detection
   pdfjs gives positioned text items. Cluster their x positions per page to detect columns, then order within each column before moving to the next.
   Files: `src/import/pdf.ts`
2. Strip repeating furniture
   Text appearing at the same position on many pages is a header or a footer. Detect by repetition across pages rather than by guessing margins.
   Files: `src/import/pdf.ts`
3. Import EPUB and web pages
   EPUB is structured already, so preserve its heading levels. For web pages, use Readability to get the article, then convert to the same block shape.
   Files: `src/import/epub.ts`, `src/import/web.ts`

### Done when

- [ ] A two column academic PDF reads in the correct order
- [ ] Page headers and footers do not repeat through the extracted text
- [ ] EPUB chapter boundaries survive as headings
- [ ] A web import contains the article and not the navigation

### Watch out

- PDF has no concept of reading order. Text is positioned glyphs, and a two column layout extracted naively interleaves the columns into nonsense. Detect columns by clustering horizontal positions, and test on an academic paper rather than a single column report.

## M3 · Sentences and synchronised highlighting

The highlighted sentence is the sentence you are hearing, all the way through a long document.

### Steps

1. Segment with offsets preserved
   Intl.Segmenter with the sentence granularity, storing character start and end so the highlight maps back to the rendered text exactly.
   Files: `src/segment.ts`
2. Advance on audio completion
   Play a sentence, wait for the ended event, then advance. No timers anywhere in the playback loop.
   Files: `src/player.ts`
3. Make sentences clickable
   Clicking sets the position and starts from there, which is also how you will test that offsets are right.
   Files: `src/reader.ts`

### Done when

- [ ] Highlight and audio still agree after twenty minutes of playback
- [ ] Common abbreviations do not cause a mid-sentence split
- [ ] Clicking a sentence starts playback at that sentence
- [ ] Pausing and resuming continues from the same sentence

### Watch out

- Driving the highlight from a timer and an estimated duration drifts, and by the third chapter the highlight is a paragraph away from the audio. Advance on the audio element's ended event, which cannot drift because it is the audio itself telling you.

## M4 · Playing ahead so it never stutters

Playback is continuous, and speed changes take effect without a gap.

### Steps

1. Prefetch a small queue
   Keep a few sentences ahead synthesised. The right number is derived from the ratio you measured in Phase 1, not guessed.
   Files: `src/prefetch.ts`
2. Cache by sentence, voice and speed
   The cache key must include voice and speed or a speed change will replay stale audio.
   Files: `src/cache.ts`
3. Cancel work on seek
   Abort in-flight synthesis when the user jumps. Otherwise the queue delivers audio for a position nobody is at any more.
   Files: `src/prefetch.ts`

### Done when

- [ ] Ten minutes of playback has no audible gaps
- [ ] Seeking far ahead begins playing in under a second
- [ ] A speed change takes effect within one sentence
- [ ] Cached audio is keyed by voice and speed

### Watch out

- Changing playback rate on the audio element is instant and free but raises pitch on some engines' output; re-synthesising at a different speed sounds better and costs a delay. Pick one deliberately, and if you use the element's rate, check whether your engine's audio survives it.

## M5 · Library, progress and export

You can leave mid-chapter and come back a week later exactly where you were.

### Steps

1. Build the library view
   Documents with progress as a proportion and the last read time. Sorted by recency, because that is how people actually return to reading.
   Files: `src/library.ts`
2. Persist progress by sentence identity
   Store the sentence id. On re-import, detect that ids changed and tell the user rather than guessing.
   Files: `src/progress.ts`
3. Export audio and make the cache disposable
   Concatenate cached sentence audio into one file. Clearing the cache must never touch documents or progress.
   Files: `src/export.ts`

### Done when

- [ ] Reopening resumes at the same sentence
- [ ] Clearing the audio cache preserves documents and progress
- [ ] An exported chapter's duration matches the sum of its sentences
- [ ] Re-importing a changed document reports that progress may have shifted

### Watch out

- Storing progress as a character offset breaks the moment a document is re-imported and extraction changes slightly. Store the sentence identity, and if re-import produces different sentences, say so rather than silently resuming somewhere wrong.

## M6 · Sharing it, and the licence question (production only)

If anyone else uses it, the voice licences and the copyright position are settled.

### Steps

1. Add authentication
   Any session mechanism. The point is that an open reader on the internet is a copyright problem waiting to be someone else's.
   Files: `src/auth.ts`
2. Record every voice licence
   A table of model, source and licence terms. Some are permissive, some are personal use only, and the difference matters.
   Files: `docs/voices.md`
3. State the position in the interface
   One sentence about personal use next to the export button. Better there than in a document nobody opens.
   Files: `src/reader.ts`

### Done when

- [ ] Access requires authentication
- [ ] Every bundled voice has its licence recorded
- [ ] The export control states the personal use position
- [ ] No sharing or public link feature exists

### Watch out

- Turning a purchased book into an audio file is fine for you and is redistribution the moment you share it. Similarly, a voice model licensed for personal use only does not become shareable because your application is free. Both questions have to be answered before anyone else connects.

# Operations · Speechify

## Backup

The database holds documents, extracted text and progress, and is the only thing worth backing up. The audio cache is regenerable and usually larger than everything else combined.

## Restore

Restore the database and open a document. Playback should resume at the recorded sentence and re-synthesise as it goes, which is also the test that the cache really is disposable.

Do a restore drill before the first real user, and write the date here when it passes.

## Monitoring

For personal use there is nothing to monitor beyond disk. If shared, watch synthesis queue depth, because a slow engine under concurrent use shows up as stuttering rather than as an error.

## Incident checklist

A full disk from the audio cache is the realistic failure. Enforce the cache ceiling rather than trusting eviction to keep up, and fail synthesis with a clear message rather than filling the disk.

1. Contain the issue without destroying evidence or user data.
2. Record the timeline and affected scope.
3. Rotate exposed secrets and revoke compromised sessions or credentials.
4. Restore from a verified backup when needed.
5. Document the root cause, the remediation and the regression test.

## Release gate

- [ ] Highlight and audio still agree after twenty minutes of continuous playback
- [ ] The two column PDF fixture extracts in correct reading order
- [ ] Ten minutes of playback has no audible gaps
- [ ] Every bundled voice licence is recorded and permits the use you are making
- [ ] Clearing the audio cache is proved not to affect documents or progress

## Launch constraint

Do not market omitted Speechify capabilities as implemented. The non-goals in `PRODUCT.md` remain user-visible limitations until they are deliberately delivered.

# Copy to .env and fill in. Never commit .env; this file documents it.

# Required. Your local speech server. The OpenAI-compatible shape is what makes engines swappable.
TTS_ENDPOINT=http://localhost:8880/v1/audio/speech

# Required. Voice identifier from your engine's voice list.
TTS_VOICE=af_sky

# Required. kokoro or piper. Chosen by the ratio measured in Phase 1.
TTS_ENGINE=kokoro

# Required. How far ahead to synthesise. Derived from the synthesis ratio, not guessed.
PREFETCH_SENTENCES=4

# Required. Where cached sentence audio is written. Safe to delete.
AUDIO_CACHE_DIR=./data/audio

# Required. Cache ceiling. Oldest entries are evicted first.
AUDIO_CACHE_MAX_GB=5

# Required. Library, documents, sentences and progress. This is the backup.
DB_PATH=./data/reader.db

$ choose a build depth, inspect the files, then open the complete pack in your agent