Vibecode Grain

You are building a lean indie version of Grain.

Create the following project files first, then implement the application by following them. Keep the files updated as decisions change. Do not collapse this into a single README or prompt.

===== README.md =====
# Grain · indie build

A local meeting archive whose hard problems are refusing to process a recording nobody consented to, transcribing long audio in chunks without losing or duplicating words at the seams, and producing notes where every claim cites a timestamp you can play.

Estimated effort: **multi-day**. Work `BUILD_PLAN.md` top to bottom · every phase ends in a check that has to pass before the next one starts.

## Stack

| Part | Choice | Why |
| --- | --- | --- |
| Runtime | Python 3.12 with FastAPI bound to localhost only | recordings of private conversations should not sit on a public port |
| Transcription | whisper.cpp run locally with segment and word timestamps | the audio never leaves the machine and the clock is the whole product |
| Media | FFmpeg as a subprocess for probing, chunking and clipping | one tool that already understands every container somebody will hand you |
| Storage | SQLite for meetings, transcripts and notes, media in a folder beside it | a transcript is small, a two hour recording is not |
| Notes | An optional model behind one interface, with an extractive fallback | the archive has to stay useful when no model is available |

## Before you start

Have every one of these ready. The plan assumes them from step one.

- [ ] **Python 3.12 or newer** · free
  - Why: Runs the pipeline, the job runner and the local interface.
  - Get it: python.org, or your package manager.
  - Verify: python3 --version prints 3.12 or higher
- [ ] **FFmpeg with ffprobe** · free
  - Why: Probing, chunking and clipping all go through it, and a build without ffprobe fails at the first import rather than at the first clip.
  - Get it: Your package manager, or a static build from the official downloads page.
  - Verify: ffprobe -version and ffmpeg -version both print a version
- [ ] **whisper.cpp with a downloaded model file** · free
  - Why: Transcription runs locally, and the model file is a separate download that is easy to forget until the first job fails.
  - Get it: Build whisper.cpp and download a model file of a size your machine can actually run.
  - Verify: The binary transcribes the bundled sample and prints timestamps
- [ ] **One real recording over ninety minutes, with consent** · free
  - Why: Chunking, resume and timestamp offsets are all fine on a five minute file and all break on a long one, which is the only length that matters.
  - Get it: A recording you made of a meeting where everyone was told, or a long public talk you are permitted to use.
  - Verify: It transcribes end to end and three random citations play the right words
- [ ] **Your consent and retention policy** · free
  - Why: Both are enforced by code and asserted by tests, and writing them after the fact means writing whatever the code already does.
  - Get it: Write down who must be informed, how, and how long recordings and transcripts are kept.
  - Verify: Both appear in the README and their values are the ones the tests use

## Quick start

Follow Phase 1 in `BUILD_PLAN.md`; it creates the project.

Then copy `.env.example` to `.env` and fill in the values it documents.

## Honest limits

This build deliberately does not replace:

- A bot that joins calls.
- Live transcription.
- CRM and calendar integrations.
- Cross meeting coaching scores.
- live multi-speaker accuracy
- calendar and CRM integrations
- cross-call team analytics
- meeting-bot auto-join

If one of those is essential to you, that is the reason to keep paying for Grain, and the README should say so rather than pretend.

===== BRIEF.md =====
# Build brief · Grain

The one-shot brief this plan expands. `BUILD_PLAN.md` (or `MILESTONES.md`) is the same sequence broken into steps and checks; where the two disagree, the plan wins.

Build me a private meeting archive that turns a recording into notes and clips, every line pointing at the second it came from to replace Grain. Build it in phases, in the order below.
Do not write the whole thing in one pass. Finish a phase, run its "Done when"
check, fix what fails, and only then start the next phase.

### Rule zero
Two rules, and the first one is law where some of your colleagues live.
Nothing here records people who were not told. Several jurisdictions require the consent of
every party rather than one: California Penal Code section 632 makes it an offence to record a
confidential communication without the consent of all parties, with a fine of up to 2,500
dollars per violation, and the rules differ by country and by state. So there is no bot that
joins a call, no hidden capture, and no import without a stored record of who was told, how and
when. The consent record is written before transcription starts, not after.
Second, every line of a generated note points at a timestamp and quotes the transcript. A
summary sentence that cannot be traced to something somebody actually said gets dropped rather
than softened, because the only reason to trust meeting notes is that you can check them.

### Stack (fixed, do not substitute)
- Runtime: Python 3.12 with FastAPI bound to localhost only · recordings of private conversations should not sit on a public port
- Transcription: whisper.cpp run locally with segment and word timestamps · the audio never leaves the machine and the clock is the whole product
- Media: FFmpeg as a subprocess for probing, chunking and clipping · one tool that already understands every container somebody will hand you
- Storage: SQLite for meetings, transcripts and notes, media in a folder beside it · a transcript is small, a two hour recording is not
- Notes: An optional model behind one interface, with an extractive fallback · the archive has to stay useful when no model is available

### Data model (create this before Phase 1)
- meetings(id, title, occurred_at, source_path, media_sha256, duration_seconds, imported_at, delete_after)
- consent(meeting_id, parties_json, informed_how, informed_at, recorded_by, note) · written before transcription is allowed to start
- segments(meeting_id, ordinal, start_ms, end_ms, speaker_label, text, confidence) · timestamps are absolute in the meeting, never per chunk
- words(segment_id, start_ms, end_ms, text) · what makes a clip cut land between words instead of through one
- notes(id, meeting_id, kind, text, source_segment_id, quote, created_at) · kind is decision, action, question or summary, and every row cites a segment
- clips(id, meeting_id, start_ms, end_ms, path, created_at)
- jobs(id, meeting_id, kind, state, chunk_index, attempts, last_error) · transcription is resumable, because a two hour file fails at minute ninety

### Phase 1 · Intake that will not start without consent
Build: Import a file, probe it with FFmpeg for duration, streams and codecs, and record a checksum.
  Require a consent record naming the parties, how they were informed and when, before any job
  is queued. Set a retention date at import. Refuse unreadable or zero length media loudly.
Trap: Making consent a checkbox that is already ticked turns the whole record into a lie that looks like compliance, and a consent row written after transcription proves nothing at all. Require the parties and the method to be typed, block the transcription job until the row exists, and store the time it was written rather than the time the meeting happened.
Done when: Queueing transcription without a consent record fails with a clear message, tested. The consent row records the parties, the method of informing them and a written timestamp that is set by the system rather than the user. A file that FFmpeg cannot probe is rejected at import with the reason shown, not queued and failed later. Every meeting has a retention date from the moment it is imported.
Do not build yet: No transcription, notes, clips or search.

### Phase 2 · Long audio without losing the seams
Build: Split the audio into chunks with a deliberate overlap, transcribe each one, then offset every
  timestamp by its chunk start and reconcile the overlap so no word appears twice and none is
  dropped. Record chunk progress so a failed run resumes rather than restarting.
Trap: Cutting a file into fixed ten minute chunks and concatenating the results slices words in half at every boundary and leaves each chunk's timestamps starting again at zero, so the transcript reads fine and every citation in it points at the wrong moment. Overlap the chunks, offset the timestamps by the chunk start, and de-duplicate across the overlap by matching text.
Done when: Timestamps are absolute in the meeting, checked by playing three random citations and hearing the quoted words. A chunk boundary placed mid sentence produces the sentence once, tested with a fixture that straddles one. Killing the process mid transcription resumes from the last finished chunk, tested. The transcript of a known short fixture matches a stored expected text within a stated tolerance.
Do not build yet: No speaker labels, notes or clips yet.

### Phase 3 · Speakers labelled as honestly as you can label them
Build: Assign provisional speaker labels with a confidence, let the person rename a label once and
  have it apply to every segment, and never infer a real name from context. Show the label as
  provisional until it has been renamed by a human.
Trap: Guessing that the voice saying 'I will do it' belongs to the person whose name was said earlier produces an action item assigned to the wrong colleague, which is discovered in the following meeting. Keep labels anonymous until a person renames them, and carry the confidence into the notes so an attribution never looks more certain than it is.
Done when: Speaker labels are anonymous until renamed, and renaming one updates every segment it covers. No code path derives a name from the transcript text, verified by review. Segment confidence is stored and shown wherever a quote is attributed. A recording with one speaker does not produce three, checked against a single speaker fixture.
Do not build yet: No notes, clips, search or export yet.

### Phase 4 · Notes that cite the second they came from
Build: Produce notes as typed rows: decision, action, question, summary. Each row stores a quote and
  the segment it came from. Verify every generated quote against the stored transcript text and
  drop rows that fail. Provide an extractive fallback that works with no model at all.
Trap: A model asked to summarise a meeting will produce a clean, plausible action item that nobody raised, and it will be the one item somebody acts on. Require a quote and a segment id on every row, check the quote actually appears in that segment before saving, and delete the rows that fail rather than keeping them with a caveat.
Done when: Every note row has a segment id and a quote, and the quote is confirmed to appear in that segment's text before the row is written. A deliberately fabricated quote is rejected by the verification step, tested with a fixture. With no model configured, the extractive fallback still produces usable notes and says which mode was used. Clicking a note line plays the recording from its timestamp.
Do not build yet: No clips, cross meeting search or export yet.

### Phase 5 · Clips, search, and deletion that actually deletes
Build: Cut clips on word boundaries using the word timestamps, re-encoding the clip rather than
  stream copying when the cut point is not on a keyframe. Search across meetings over transcript
  text. Delete a meeting by removing media, chunks, transcript, notes and clips together, and
  run retention as a scheduled job that reports what it removed.
Trap: Deleting the meeting row and leaving the audio, the chunk files and the exported clips on disk means a person who asked for a recording to be removed still has it sitting in three folders, and nobody notices until an audit. Make deletion a single operation over every artefact, and assert in the suite that no file with that meeting's identifier survives it.
Done when: Deleting a meeting leaves no file containing its identifier anywhere under the media directory, asserted by a test that walks the tree. A clip cut mid word starts and ends on a word boundary, checked against the word timestamps. Search returns the segment and plays from its timestamp. Retention runs on a schedule and reports the number of meetings and bytes removed.
Do not build yet: Nothing after this.

### Out of scope (and why)
- A bot that joins calls, which is the feature people actually pay for and also the one that records people before they have agreed to it.
- Live transcription during a meeting, which is a streaming and latency problem rather than a batch one and needs a different engine entirely.
- CRM and calendar integrations, where each one is a credential, a schema and a permission model that outlives your interest in it.
- Cross meeting coaching scores, which rank colleagues on numbers derived from an approximate transcript and should not exist without a lot more care than a weekend allows.

### README must contain
- That a consent record is required before transcription and what it stores, with a plain note that all party consent is the rule in some jurisdictions
- That timestamps are absolute in the meeting and how chunk overlap and de-duplication work
- That speaker labels are provisional until a person renames them, and that no name is ever inferred
- That every note row is verified against the transcript and unverifiable rows are dropped
- How deletion and retention work, including that media, chunks, transcript, notes and clips go together

===== AGENTS.md =====
# Agent instructions · Grain indie build

- Read `README.md` and `BUILD_PLAN.md` before writing code. The stack is fixed: Python 3.12 with FastAPI bound to localhost only, whisper.cpp run locally with segment and word timestamps, FFmpeg as a subprocess for probing, chunking and clipping, SQLite for meetings, transcripts and notes, media in a folder beside it, An optional model behind one interface, with an extractive fallback. Do not substitute.
- Work one phase at a time, in order. Do not start a phase until every "Done when" item of the previous one passes.
- Prefer the fewest moving parts that satisfy the step. No frameworks, services or dependencies the plan does not name.
- Secrets live in `.env`, never in source or logs. Keep `.env.example` current when a variable is introduced.
- Do not invent cryptography, security guarantees, APIs or compliance claims.
- Add a focused test for every destructive, security-sensitive or data-loss path the plan names.
- Run the project checks before declaring a phase complete, and record any deliberate shortcut in the README under "Tradeoffs".

## Known traps

- Making consent a checkbox that is already ticked turns the whole record into a lie that looks like compliance, and a consent row written after transcription proves nothing at all. Require the parties and the method to be typed, block the transcription job until the row exists, and store the time it was written rather than the time the meeting happened.
- Cutting a file into fixed ten minute chunks and concatenating the results slices words in half at every boundary and leaves each chunk's timestamps starting again at zero, so the transcript reads fine and every citation in it points at the wrong moment. Overlap the chunks, offset the timestamps by the chunk start, and de-duplicate across the overlap by matching text.
- Guessing that the voice saying 'I will do it' belongs to the person whose name was said earlier produces an action item assigned to the wrong colleague, which is discovered in the following meeting. Keep labels anonymous until a person renames them, and carry the confidence into the notes so an attribution never looks more certain than it is.
- A model asked to summarise a meeting will produce a clean, plausible action item that nobody raised, and it will be the one item somebody acts on. Require a quote and a segment id on every row, check the quote actually appears in that segment before saving, and delete the rows that fail rather than keeping them with a caveat.

===== BUILD_PLAN.md =====
# Build plan · Grain

A local meeting archive whose hard problems are refusing to process a recording nobody consented to, transcribing long audio in chunks without losing or duplicating words at the seams, and producing notes where every claim cites a timestamp you can play.

Phases are in dependency order. Each ends in a "Done when" list; treat an unticked item as a blocker, not a note.

## Phase 1 · Intake that will not start without consent

A recording cannot be transcribed until who was told is recorded.

### Steps

1. Probe before accepting
   Duration, streams and codecs decide whether the rest of the pipeline can work at all.
   Files: `src/intake.py`, `src/media/probe.py`
2. Block the job on the consent row
   In the queueing code, so there is one place it can be bypassed and it is visible.
   Files: `src/jobs.py`, `tests/test_consent_gate.py`
3. Set the retention date at import
   Decided once, when nobody is attached to the recording yet.
   Files: `src/retention.py`

### Done when

- [ ] Transcription without consent is refused
- [ ] Consent records parties, method and a system timestamp
- [ ] Unprobeable media is rejected at import with a reason
- [ ] Every meeting carries a retention date

### Watch out

- Making consent a checkbox that is already ticked turns the whole record into a lie that looks like compliance, and a consent row written after transcription proves nothing at all. Require the parties and the method to be typed, block the transcription job until the row exists, and store the time it was written rather than the time the meeting happened.

## Phase 2 · Long audio without losing the seams

A two hour recording transcribes in chunks with one continuous clock.

### Steps

1. Overlap the chunks deliberately
   A few seconds of overlap is the cheapest fix for words cut in half.
   Files: `src/transcribe/chunk.py`
2. Offset timestamps by chunk start
   This one line is the difference between citations that work and citations that lie.
   Files: `src/transcribe/merge.py`, `tests/test_timestamps.py`
3. Record chunk progress for resume
   Long jobs fail, and restarting a ninety minute transcription is how people give up.
   Files: `src/jobs.py`

### Done when

- [ ] Citations play the words they quote
- [ ] A sentence across a boundary appears exactly once
- [ ] Transcription resumes after a kill
- [ ] A known fixture matches its expected transcript

### Watch out

- Cutting a file into fixed ten minute chunks and concatenating the results slices words in half at every boundary and leaves each chunk's timestamps starting again at zero, so the transcript reads fine and every citation in it points at the wrong moment. Overlap the chunks, offset the timestamps by the chunk start, and de-duplicate across the overlap by matching text.

## Phase 3 · Speakers labelled as honestly as you can label them

Attribution is either evidenced or clearly provisional.

### Steps

1. Keep labels provisional by default
   A letter is honest; a guessed name is a claim.
   Files: `src/speakers.py`
2. Rename once, apply everywhere
   Otherwise people rename the first three segments and give up.
   Files: `src/speakers.py`, `tests/test_rename.py`
3. Carry confidence into every quote
   So the note can show that the attribution is weak.
   Files: `src/notes/render.py`

### Done when

- [ ] Labels stay anonymous until renamed
- [ ] Renaming updates every covered segment
- [ ] Confidence is stored and displayed with quotes
- [ ] A single speaker file yields one label

### Watch out

- Guessing that the voice saying 'I will do it' belongs to the person whose name was said earlier produces an action item assigned to the wrong colleague, which is discovered in the following meeting. Keep labels anonymous until a person renames them, and carry the confidence into the notes so an attribution never looks more certain than it is.

## Phase 4 · Notes that cite the second they came from

Every decision, action and question quotes the transcript and links to a timestamp.

### Steps

1. Verify each quote against its segment
   The check is a substring match, and it catches almost everything invented.
   Files: `src/notes/verify.py`, `tests/test_quote_verification.py`
2. Drop unverifiable rows entirely
   A caveat next to a wrong action item does not stop anyone acting on it.
   Files: `src/notes/build.py`
3. Ship the extractive fallback first
   Then the model is an improvement rather than a dependency.
   Files: `src/notes/extractive.py`

### Done when

- [ ] Every note row cites a segment and a verified quote
- [ ] A fabricated quote is rejected
- [ ] The no-model fallback produces notes and says so
- [ ] A note line plays from its timestamp

### Watch out

- A model asked to summarise a meeting will produce a clean, plausible action item that nobody raised, and it will be the one item somebody acts on. Require a quote and a segment id on every row, check the quote actually appears in that segment before saving, and delete the rows that fail rather than keeping them with a caveat.

## Not in this build

- A bot that joins calls.
- Live transcription.
- CRM and calendar integrations.
- Cross meeting coaching scores.

## After v1, if you want it

- Topic grouping across meetings, once you trust the transcripts enough to aggregate them
- A shareable read only page per meeting, which is a rendering job plus a decision about who sees it
- Translation of the transcript, kept beside the original rather than replacing it

===== .env.example =====
# Copy to .env and fill in. Never commit .env; this file documents it.

# Required. Meetings, consent, transcripts and notes.
DATABASE_PATH=./data/meetings.db

# Required. Source recordings, chunks and clips. Deletion must clear all three.
MEDIA_DIR=./data/media

# Required. The local transcription binary.
WHISPER_BINARY=/usr/local/bin/whisper-cli

# Required. The model file. Bigger is slower and usually better; measure on your own machine.
WHISPER_MODEL_PATH=./models/ggml-base.en.bin

# Required. Chunk length for long audio. Timestamps are offset by the chunk start.
CHUNK_SECONDS=600

# Required. Overlap between chunks, de-duplicated on merge. Zero here means words lost at every seam.
CHUNK_OVERLAP_SECONDS=5

# Required. Set at import. The scheduled job reports what it removed.
RETENTION_DAYS=365

# Optional. Optional. Unset means the extractive fallback, which must stay usable.
NOTES_MODEL=none

You are building a lean indie version of Grain.

Create the following project files first, then implement the application by following them. Keep the files updated as decisions change. Do not collapse this into a single README or prompt.

===== README.md =====
# Grain · indie build

A local meeting archive whose hard problems are refusing to process a recording nobody consented to, transcribing long audio in chunks without losing or duplicating words at the seams, and producing notes where every claim cites a timestamp you can play.

Estimated effort: **multi-day**. Work `BUILD_PLAN.md` top to bottom · every phase ends in a check that has to pass before the next one starts.

## Stack

| Part | Choice | Why |
| --- | --- | --- |
| Runtime | Python 3.12 with FastAPI bound to localhost only | recordings of private conversations should not sit on a public port |
| Transcription | whisper.cpp run locally with segment and word timestamps | the audio never leaves the machine and the clock is the whole product |
| Media | FFmpeg as a subprocess for probing, chunking and clipping | one tool that already understands every container somebody will hand you |
| Storage | SQLite for meetings, transcripts and notes, media in a folder beside it | a transcript is small, a two hour recording is not |
| Notes | An optional model behind one interface, with an extractive fallback | the archive has to stay useful when no model is available |

## Before you start

Have every one of these ready. The plan assumes them from step one.

- [ ] **Python 3.12 or newer** · free
  - Why: Runs the pipeline, the job runner and the local interface.
  - Get it: python.org, or your package manager.
  - Verify: python3 --version prints 3.12 or higher
- [ ] **FFmpeg with ffprobe** · free
  - Why: Probing, chunking and clipping all go through it, and a build without ffprobe fails at the first import rather than at the first clip.
  - Get it: Your package manager, or a static build from the official downloads page.
  - Verify: ffprobe -version and ffmpeg -version both print a version
- [ ] **whisper.cpp with a downloaded model file** · free
  - Why: Transcription runs locally, and the model file is a separate download that is easy to forget until the first job fails.
  - Get it: Build whisper.cpp and download a model file of a size your machine can actually run.
  - Verify: The binary transcribes the bundled sample and prints timestamps
- [ ] **One real recording over ninety minutes, with consent** · free
  - Why: Chunking, resume and timestamp offsets are all fine on a five minute file and all break on a long one, which is the only length that matters.
  - Get it: A recording you made of a meeting where everyone was told, or a long public talk you are permitted to use.
  - Verify: It transcribes end to end and three random citations play the right words
- [ ] **Your consent and retention policy** · free
  - Why: Both are enforced by code and asserted by tests, and writing them after the fact means writing whatever the code already does.
  - Get it: Write down who must be informed, how, and how long recordings and transcripts are kept.
  - Verify: Both appear in the README and their values are the ones the tests use

## Quick start

Follow Phase 1 in `BUILD_PLAN.md`; it creates the project.

Then copy `.env.example` to `.env` and fill in the values it documents.

## Honest limits

This build deliberately does not replace:

- A bot that joins calls.
- Live transcription.
- CRM and calendar integrations.
- Cross meeting coaching scores.
- live multi-speaker accuracy
- calendar and CRM integrations
- cross-call team analytics
- meeting-bot auto-join

If one of those is essential to you, that is the reason to keep paying for Grain, and the README should say so rather than pretend.

===== BRIEF.md =====
# Build brief · Grain

The one-shot brief this plan expands. `BUILD_PLAN.md` (or `MILESTONES.md`) is the same sequence broken into steps and checks; where the two disagree, the plan wins.

Build me a private meeting archive that turns a recording into notes and clips, every line pointing at the second it came from to replace Grain. Build it in phases, in the order below.
Do not write the whole thing in one pass. Finish a phase, run its "Done when"
check, fix what fails, and only then start the next phase.

### Rule zero
Two rules, and the first one is law where some of your colleagues live.
Nothing here records people who were not told. Several jurisdictions require the consent of
every party rather than one: California Penal Code section 632 makes it an offence to record a
confidential communication without the consent of all parties, with a fine of up to 2,500
dollars per violation, and the rules differ by country and by state. So there is no bot that
joins a call, no hidden capture, and no import without a stored record of who was told, how and
when. The consent record is written before transcription starts, not after.
Second, every line of a generated note points at a timestamp and quotes the transcript. A
summary sentence that cannot be traced to something somebody actually said gets dropped rather
than softened, because the only reason to trust meeting notes is that you can check them.

### Stack (fixed, do not substitute)
- Runtime: Python 3.12 with FastAPI bound to localhost only · recordings of private conversations should not sit on a public port
- Transcription: whisper.cpp run locally with segment and word timestamps · the audio never leaves the machine and the clock is the whole product
- Media: FFmpeg as a subprocess for probing, chunking and clipping · one tool that already understands every container somebody will hand you
- Storage: SQLite for meetings, transcripts and notes, media in a folder beside it · a transcript is small, a two hour recording is not
- Notes: An optional model behind one interface, with an extractive fallback · the archive has to stay useful when no model is available

### Data model (create this before Phase 1)
- meetings(id, title, occurred_at, source_path, media_sha256, duration_seconds, imported_at, delete_after)
- consent(meeting_id, parties_json, informed_how, informed_at, recorded_by, note) · written before transcription is allowed to start
- segments(meeting_id, ordinal, start_ms, end_ms, speaker_label, text, confidence) · timestamps are absolute in the meeting, never per chunk
- words(segment_id, start_ms, end_ms, text) · what makes a clip cut land between words instead of through one
- notes(id, meeting_id, kind, text, source_segment_id, quote, created_at) · kind is decision, action, question or summary, and every row cites a segment
- clips(id, meeting_id, start_ms, end_ms, path, created_at)
- jobs(id, meeting_id, kind, state, chunk_index, attempts, last_error) · transcription is resumable, because a two hour file fails at minute ninety

### Phase 1 · Intake that will not start without consent
Build: Import a file, probe it with FFmpeg for duration, streams and codecs, and record a checksum.
  Require a consent record naming the parties, how they were informed and when, before any job
  is queued. Set a retention date at import. Refuse unreadable or zero length media loudly.
Trap: Making consent a checkbox that is already ticked turns the whole record into a lie that looks like compliance, and a consent row written after transcription proves nothing at all. Require the parties and the method to be typed, block the transcription job until the row exists, and store the time it was written rather than the time the meeting happened.
Done when: Queueing transcription without a consent record fails with a clear message, tested. The consent row records the parties, the method of informing them and a written timestamp that is set by the system rather than the user. A file that FFmpeg cannot probe is rejected at import with the reason shown, not queued and failed later. Every meeting has a retention date from the moment it is imported.
Do not build yet: No transcription, notes, clips or search.

### Phase 2 · Long audio without losing the seams
Build: Split the audio into chunks with a deliberate overlap, transcribe each one, then offset every
  timestamp by its chunk start and reconcile the overlap so no word appears twice and none is
  dropped. Record chunk progress so a failed run resumes rather than restarting.
Trap: Cutting a file into fixed ten minute chunks and concatenating the results slices words in half at every boundary and leaves each chunk's timestamps starting again at zero, so the transcript reads fine and every citation in it points at the wrong moment. Overlap the chunks, offset the timestamps by the chunk start, and de-duplicate across the overlap by matching text.
Done when: Timestamps are absolute in the meeting, checked by playing three random citations and hearing the quoted words. A chunk boundary placed mid sentence produces the sentence once, tested with a fixture that straddles one. Killing the process mid transcription resumes from the last finished chunk, tested. The transcript of a known short fixture matches a stored expected text within a stated tolerance.
Do not build yet: No speaker labels, notes or clips yet.

### Phase 3 · Speakers labelled as honestly as you can label them
Build: Assign provisional speaker labels with a confidence, let the person rename a label once and
  have it apply to every segment, and never infer a real name from context. Show the label as
  provisional until it has been renamed by a human.
Trap: Guessing that the voice saying 'I will do it' belongs to the person whose name was said earlier produces an action item assigned to the wrong colleague, which is discovered in the following meeting. Keep labels anonymous until a person renames them, and carry the confidence into the notes so an attribution never looks more certain than it is.
Done when: Speaker labels are anonymous until renamed, and renaming one updates every segment it covers. No code path derives a name from the transcript text, verified by review. Segment confidence is stored and shown wherever a quote is attributed. A recording with one speaker does not produce three, checked against a single speaker fixture.
Do not build yet: No notes, clips, search or export yet.

### Phase 4 · Notes that cite the second they came from
Build: Produce notes as typed rows: decision, action, question, summary. Each row stores a quote and
  the segment it came from. Verify every generated quote against the stored transcript text and
  drop rows that fail. Provide an extractive fallback that works with no model at all.
Trap: A model asked to summarise a meeting will produce a clean, plausible action item that nobody raised, and it will be the one item somebody acts on. Require a quote and a segment id on every row, check the quote actually appears in that segment before saving, and delete the rows that fail rather than keeping them with a caveat.
Done when: Every note row has a segment id and a quote, and the quote is confirmed to appear in that segment's text before the row is written. A deliberately fabricated quote is rejected by the verification step, tested with a fixture. With no model configured, the extractive fallback still produces usable notes and says which mode was used. Clicking a note line plays the recording from its timestamp.
Do not build yet: No clips, cross meeting search or export yet.

### Phase 5 · Clips, search, and deletion that actually deletes
Build: Cut clips on word boundaries using the word timestamps, re-encoding the clip rather than
  stream copying when the cut point is not on a keyframe. Search across meetings over transcript
  text. Delete a meeting by removing media, chunks, transcript, notes and clips together, and
  run retention as a scheduled job that reports what it removed.
Trap: Deleting the meeting row and leaving the audio, the chunk files and the exported clips on disk means a person who asked for a recording to be removed still has it sitting in three folders, and nobody notices until an audit. Make deletion a single operation over every artefact, and assert in the suite that no file with that meeting's identifier survives it.
Done when: Deleting a meeting leaves no file containing its identifier anywhere under the media directory, asserted by a test that walks the tree. A clip cut mid word starts and ends on a word boundary, checked against the word timestamps. Search returns the segment and plays from its timestamp. Retention runs on a schedule and reports the number of meetings and bytes removed.
Do not build yet: Nothing after this.

### Out of scope (and why)
- A bot that joins calls, which is the feature people actually pay for and also the one that records people before they have agreed to it.
- Live transcription during a meeting, which is a streaming and latency problem rather than a batch one and needs a different engine entirely.
- CRM and calendar integrations, where each one is a credential, a schema and a permission model that outlives your interest in it.
- Cross meeting coaching scores, which rank colleagues on numbers derived from an approximate transcript and should not exist without a lot more care than a weekend allows.

### README must contain
- That a consent record is required before transcription and what it stores, with a plain note that all party consent is the rule in some jurisdictions
- That timestamps are absolute in the meeting and how chunk overlap and de-duplication work
- That speaker labels are provisional until a person renames them, and that no name is ever inferred
- That every note row is verified against the transcript and unverifiable rows are dropped
- How deletion and retention work, including that media, chunks, transcript, notes and clips go together

===== AGENTS.md =====
# Agent instructions · Grain indie build

- Read `README.md` and `BUILD_PLAN.md` before writing code. The stack is fixed: Python 3.12 with FastAPI bound to localhost only, whisper.cpp run locally with segment and word timestamps, FFmpeg as a subprocess for probing, chunking and clipping, SQLite for meetings, transcripts and notes, media in a folder beside it, An optional model behind one interface, with an extractive fallback. Do not substitute.
- Work one phase at a time, in order. Do not start a phase until every "Done when" item of the previous one passes.
- Prefer the fewest moving parts that satisfy the step. No frameworks, services or dependencies the plan does not name.
- Secrets live in `.env`, never in source or logs. Keep `.env.example` current when a variable is introduced.
- Do not invent cryptography, security guarantees, APIs or compliance claims.
- Add a focused test for every destructive, security-sensitive or data-loss path the plan names.
- Run the project checks before declaring a phase complete, and record any deliberate shortcut in the README under "Tradeoffs".

## Known traps

- Making consent a checkbox that is already ticked turns the whole record into a lie that looks like compliance, and a consent row written after transcription proves nothing at all. Require the parties and the method to be typed, block the transcription job until the row exists, and store the time it was written rather than the time the meeting happened.
- Cutting a file into fixed ten minute chunks and concatenating the results slices words in half at every boundary and leaves each chunk's timestamps starting again at zero, so the transcript reads fine and every citation in it points at the wrong moment. Overlap the chunks, offset the timestamps by the chunk start, and de-duplicate across the overlap by matching text.
- Guessing that the voice saying 'I will do it' belongs to the person whose name was said earlier produces an action item assigned to the wrong colleague, which is discovered in the following meeting. Keep labels anonymous until a person renames them, and carry the confidence into the notes so an attribution never looks more certain than it is.
- A model asked to summarise a meeting will produce a clean, plausible action item that nobody raised, and it will be the one item somebody acts on. Require a quote and a segment id on every row, check the quote actually appears in that segment before saving, and delete the rows that fail rather than keeping them with a caveat.

===== BUILD_PLAN.md =====
# Build plan · Grain

A local meeting archive whose hard problems are refusing to process a recording nobody consented to, transcribing long audio in chunks without losing or duplicating words at the seams, and producing notes where every claim cites a timestamp you can play.

Phases are in dependency order. Each ends in a "Done when" list; treat an unticked item as a blocker, not a note.

## Phase 1 · Intake that will not start without consent

A recording cannot be transcribed until who was told is recorded.

### Steps

1. Probe before accepting
   Duration, streams and codecs decide whether the rest of the pipeline can work at all.
   Files: `src/intake.py`, `src/media/probe.py`
2. Block the job on the consent row
   In the queueing code, so there is one place it can be bypassed and it is visible.
   Files: `src/jobs.py`, `tests/test_consent_gate.py`
3. Set the retention date at import
   Decided once, when nobody is attached to the recording yet.
   Files: `src/retention.py`

### Done when

- [ ] Transcription without consent is refused
- [ ] Consent records parties, method and a system timestamp
- [ ] Unprobeable media is rejected at import with a reason
- [ ] Every meeting carries a retention date

### Watch out

- Making consent a checkbox that is already ticked turns the whole record into a lie that looks like compliance, and a consent row written after transcription proves nothing at all. Require the parties and the method to be typed, block the transcription job until the row exists, and store the time it was written rather than the time the meeting happened.

## Phase 2 · Long audio without losing the seams

A two hour recording transcribes in chunks with one continuous clock.

### Steps

1. Overlap the chunks deliberately
   A few seconds of overlap is the cheapest fix for words cut in half.
   Files: `src/transcribe/chunk.py`
2. Offset timestamps by chunk start
   This one line is the difference between citations that work and citations that lie.
   Files: `src/transcribe/merge.py`, `tests/test_timestamps.py`
3. Record chunk progress for resume
   Long jobs fail, and restarting a ninety minute transcription is how people give up.
   Files: `src/jobs.py`

### Done when

- [ ] Citations play the words they quote
- [ ] A sentence across a boundary appears exactly once
- [ ] Transcription resumes after a kill
- [ ] A known fixture matches its expected transcript

### Watch out

- Cutting a file into fixed ten minute chunks and concatenating the results slices words in half at every boundary and leaves each chunk's timestamps starting again at zero, so the transcript reads fine and every citation in it points at the wrong moment. Overlap the chunks, offset the timestamps by the chunk start, and de-duplicate across the overlap by matching text.

## Phase 3 · Speakers labelled as honestly as you can label them

Attribution is either evidenced or clearly provisional.

### Steps

1. Keep labels provisional by default
   A letter is honest; a guessed name is a claim.
   Files: `src/speakers.py`
2. Rename once, apply everywhere
   Otherwise people rename the first three segments and give up.
   Files: `src/speakers.py`, `tests/test_rename.py`
3. Carry confidence into every quote
   So the note can show that the attribution is weak.
   Files: `src/notes/render.py`

### Done when

- [ ] Labels stay anonymous until renamed
- [ ] Renaming updates every covered segment
- [ ] Confidence is stored and displayed with quotes
- [ ] A single speaker file yields one label

### Watch out

- Guessing that the voice saying 'I will do it' belongs to the person whose name was said earlier produces an action item assigned to the wrong colleague, which is discovered in the following meeting. Keep labels anonymous until a person renames them, and carry the confidence into the notes so an attribution never looks more certain than it is.

## Phase 4 · Notes that cite the second they came from

Every decision, action and question quotes the transcript and links to a timestamp.

### Steps

1. Verify each quote against its segment
   The check is a substring match, and it catches almost everything invented.
   Files: `src/notes/verify.py`, `tests/test_quote_verification.py`
2. Drop unverifiable rows entirely
   A caveat next to a wrong action item does not stop anyone acting on it.
   Files: `src/notes/build.py`
3. Ship the extractive fallback first
   Then the model is an improvement rather than a dependency.
   Files: `src/notes/extractive.py`

### Done when

- [ ] Every note row cites a segment and a verified quote
- [ ] A fabricated quote is rejected
- [ ] The no-model fallback produces notes and says so
- [ ] A note line plays from its timestamp

### Watch out

- A model asked to summarise a meeting will produce a clean, plausible action item that nobody raised, and it will be the one item somebody acts on. Require a quote and a segment id on every row, check the quote actually appears in that segment before saving, and delete the rows that fail rather than keeping them with a caveat.

## Not in this build

- A bot that joins calls.
- Live transcription.
- CRM and calendar integrations.
- Cross meeting coaching scores.

## After v1, if you want it

- Topic grouping across meetings, once you trust the transcripts enough to aggregate them
- A shareable read only page per meeting, which is a rendering job plus a decision about who sees it
- Translation of the transcript, kept beside the original rather than replacing it

===== .env.example =====
# Copy to .env and fill in. Never commit .env; this file documents it.

# Required. Meetings, consent, transcripts and notes.
DATABASE_PATH=./data/meetings.db

# Required. Source recordings, chunks and clips. Deletion must clear all three.
MEDIA_DIR=./data/media

# Required. The local transcription binary.
WHISPER_BINARY=/usr/local/bin/whisper-cli

# Required. The model file. Bigger is slower and usually better; measure on your own machine.
WHISPER_MODEL_PATH=./models/ggml-base.en.bin

# Required. Chunk length for long audio. Timestamps are offset by the chunk start.
CHUNK_SECONDS=600

# Required. Overlap between chunks, de-duplicated on merge. Zero here means words lost at every seam.
CHUNK_OVERLAP_SECONDS=5

# Required. Set at import. The scheduled job reports what it removed.
RETENTION_DAYS=365

# Optional. Optional. Unset means the extractive fallback, which must stay usable.
NOTES_MODEL=none

You are building a production product version of Grain.

Create the following project files first, then implement the application by following them. Keep the files updated as decisions change. Do not collapse this into a single README or prompt.

===== PRODUCT.md =====
# Grain · product brief

## Problem

The visible meeting clips + intelligence loop is buildable, but a credible replacement needs more than the first screen. Grain earns its keep through capture, integrations, reliability, so expect a weekend or multi-day build and a narrower personal scope.

## Product outcome

An archive that will not touch a recording without a consent record, transcribes long audio with one continuous clock, produces notes whose every line quotes something that was actually said, and deletes a meeting completely when asked.

## Target user

A builder who needs a maintainable product foundation, not a one-off demo.

## Required capabilities

- Python 3.12
- FFmpeg
- local Whisper model or optional API key in .env

## Explicit non-goals for v1

- A bot that joins calls.
- Live transcription.
- CRM and calendar integrations.
- Cross meeting coaching scores.
- live multi-speaker accuracy
- calendar and CRM integrations
- cross-call team analytics
- meeting-bot auto-join

## Success criteria

- Transcription without consent is confirmed refused
- Citations are confirmed to play the quoted words
- Speaker names are confirmed never inferred
- Fabricated quotes are confirmed rejected
- Deletion is confirmed to leave no file behind

===== BRIEF.md =====
# Build brief · Grain

The one-shot brief this plan expands. `BUILD_PLAN.md` (or `MILESTONES.md`) is the same sequence broken into steps and checks; where the two disagree, the plan wins.

Build me a private meeting archive that turns a recording into notes and clips, every line pointing at the second it came from to replace Grain. Build it in phases, in the order below.
Do not write the whole thing in one pass. Finish a phase, run its "Done when"
check, fix what fails, and only then start the next phase.

### Rule zero
Two rules, and the first one is law where some of your colleagues live.
Nothing here records people who were not told. Several jurisdictions require the consent of
every party rather than one: California Penal Code section 632 makes it an offence to record a
confidential communication without the consent of all parties, with a fine of up to 2,500
dollars per violation, and the rules differ by country and by state. So there is no bot that
joins a call, no hidden capture, and no import without a stored record of who was told, how and
when. The consent record is written before transcription starts, not after.
Second, every line of a generated note points at a timestamp and quotes the transcript. A
summary sentence that cannot be traced to something somebody actually said gets dropped rather
than softened, because the only reason to trust meeting notes is that you can check them.

### Stack (fixed, do not substitute)
- Runtime: Python 3.12 with FastAPI bound to localhost only · recordings of private conversations should not sit on a public port
- Transcription: whisper.cpp run locally with segment and word timestamps · the audio never leaves the machine and the clock is the whole product
- Media: FFmpeg as a subprocess for probing, chunking and clipping · one tool that already understands every container somebody will hand you
- Storage: SQLite for meetings, transcripts and notes, media in a folder beside it · a transcript is small, a two hour recording is not
- Notes: An optional model behind one interface, with an extractive fallback · the archive has to stay useful when no model is available

### Data model (create this before Phase 1)
- meetings(id, title, occurred_at, source_path, media_sha256, duration_seconds, imported_at, delete_after)
- consent(meeting_id, parties_json, informed_how, informed_at, recorded_by, note) · written before transcription is allowed to start
- segments(meeting_id, ordinal, start_ms, end_ms, speaker_label, text, confidence) · timestamps are absolute in the meeting, never per chunk
- words(segment_id, start_ms, end_ms, text) · what makes a clip cut land between words instead of through one
- notes(id, meeting_id, kind, text, source_segment_id, quote, created_at) · kind is decision, action, question or summary, and every row cites a segment
- clips(id, meeting_id, start_ms, end_ms, path, created_at)
- jobs(id, meeting_id, kind, state, chunk_index, attempts, last_error) · transcription is resumable, because a two hour file fails at minute ninety

### Phase 1 · Intake that will not start without consent
Build: Import a file, probe it with FFmpeg for duration, streams and codecs, and record a checksum.
  Require a consent record naming the parties, how they were informed and when, before any job
  is queued. Set a retention date at import. Refuse unreadable or zero length media loudly.
Trap: Making consent a checkbox that is already ticked turns the whole record into a lie that looks like compliance, and a consent row written after transcription proves nothing at all. Require the parties and the method to be typed, block the transcription job until the row exists, and store the time it was written rather than the time the meeting happened.
Done when: Queueing transcription without a consent record fails with a clear message, tested. The consent row records the parties, the method of informing them and a written timestamp that is set by the system rather than the user. A file that FFmpeg cannot probe is rejected at import with the reason shown, not queued and failed later. Every meeting has a retention date from the moment it is imported.
Do not build yet: No transcription, notes, clips or search.

### Phase 2 · Long audio without losing the seams
Build: Split the audio into chunks with a deliberate overlap, transcribe each one, then offset every
  timestamp by its chunk start and reconcile the overlap so no word appears twice and none is
  dropped. Record chunk progress so a failed run resumes rather than restarting.
Trap: Cutting a file into fixed ten minute chunks and concatenating the results slices words in half at every boundary and leaves each chunk's timestamps starting again at zero, so the transcript reads fine and every citation in it points at the wrong moment. Overlap the chunks, offset the timestamps by the chunk start, and de-duplicate across the overlap by matching text.
Done when: Timestamps are absolute in the meeting, checked by playing three random citations and hearing the quoted words. A chunk boundary placed mid sentence produces the sentence once, tested with a fixture that straddles one. Killing the process mid transcription resumes from the last finished chunk, tested. The transcript of a known short fixture matches a stored expected text within a stated tolerance.
Do not build yet: No speaker labels, notes or clips yet.

### Phase 3 · Speakers labelled as honestly as you can label them
Build: Assign provisional speaker labels with a confidence, let the person rename a label once and
  have it apply to every segment, and never infer a real name from context. Show the label as
  provisional until it has been renamed by a human.
Trap: Guessing that the voice saying 'I will do it' belongs to the person whose name was said earlier produces an action item assigned to the wrong colleague, which is discovered in the following meeting. Keep labels anonymous until a person renames them, and carry the confidence into the notes so an attribution never looks more certain than it is.
Done when: Speaker labels are anonymous until renamed, and renaming one updates every segment it covers. No code path derives a name from the transcript text, verified by review. Segment confidence is stored and shown wherever a quote is attributed. A recording with one speaker does not produce three, checked against a single speaker fixture.
Do not build yet: No notes, clips, search or export yet.

### Phase 4 · Notes that cite the second they came from
Build: Produce notes as typed rows: decision, action, question, summary. Each row stores a quote and
  the segment it came from. Verify every generated quote against the stored transcript text and
  drop rows that fail. Provide an extractive fallback that works with no model at all.
Trap: A model asked to summarise a meeting will produce a clean, plausible action item that nobody raised, and it will be the one item somebody acts on. Require a quote and a segment id on every row, check the quote actually appears in that segment before saving, and delete the rows that fail rather than keeping them with a caveat.
Done when: Every note row has a segment id and a quote, and the quote is confirmed to appear in that segment's text before the row is written. A deliberately fabricated quote is rejected by the verification step, tested with a fixture. With no model configured, the extractive fallback still produces usable notes and says which mode was used. Clicking a note line plays the recording from its timestamp.
Do not build yet: No clips, cross meeting search or export yet.

### Phase 5 · Clips, search, and deletion that actually deletes
Build: Cut clips on word boundaries using the word timestamps, re-encoding the clip rather than
  stream copying when the cut point is not on a keyframe. Search across meetings over transcript
  text. Delete a meeting by removing media, chunks, transcript, notes and clips together, and
  run retention as a scheduled job that reports what it removed.
Trap: Deleting the meeting row and leaving the audio, the chunk files and the exported clips on disk means a person who asked for a recording to be removed still has it sitting in three folders, and nobody notices until an audit. Make deletion a single operation over every artefact, and assert in the suite that no file with that meeting's identifier survives it.
Done when: Deleting a meeting leaves no file containing its identifier anywhere under the media directory, asserted by a test that walks the tree. A clip cut mid word starts and ends on a word boundary, checked against the word timestamps. Search returns the segment and plays from its timestamp. Retention runs on a schedule and reports the number of meetings and bytes removed.
Do not build yet: Nothing after this.

### Out of scope (and why)
- A bot that joins calls, which is the feature people actually pay for and also the one that records people before they have agreed to it.
- Live transcription during a meeting, which is a streaming and latency problem rather than a batch one and needs a different engine entirely.
- CRM and calendar integrations, where each one is a credential, a schema and a permission model that outlives your interest in it.
- Cross meeting coaching scores, which rank colleagues on numbers derived from an approximate transcript and should not exist without a lot more care than a weekend allows.

### README must contain
- That a consent record is required before transcription and what it stores, with a plain note that all party consent is the rule in some jurisdictions
- That timestamps are absolute in the meeting and how chunk overlap and de-duplication work
- That speaker labels are provisional until a person renames them, and that no name is ever inferred
- That every note row is verified against the transcript and unverifiable rows are dropped
- How deletion and retention work, including that media, chunks, transcript, notes and clips go together

===== ARCHITECTURE.md =====
# Architecture · Grain

## Stack

| Part | Choice | Why |
| --- | --- | --- |
| Runtime | Python 3.12 with FastAPI bound to localhost only | recordings of private conversations should not sit on a public port |
| Transcription | whisper.cpp run locally with segment and word timestamps | the audio never leaves the machine and the clock is the whole product |
| Media | FFmpeg as a subprocess for probing, chunking and clipping | one tool that already understands every container somebody will hand you |
| Storage | SQLite for meetings, transcripts and notes, media in a folder beside it | a transcript is small, a two hour recording is not |
| Notes | An optional model behind one interface, with an extractive fallback | the archive has to stay useful when no model is available |

## Modules

Each module has one owner concern and a documented way to replace it.

| Module | Owns | How to replace it |
| --- | --- | --- |
| Intake | Probing, checksums and the consent gate. | Any importer, provided the consent gate stays in the queueing path. |
| Transcription | Chunking, offsets, overlap merge and resume. | Any engine that returns timestamps, provided offsets stay absolute. |
| Speakers | Provisional labels and confidence. | Any diarisation, provided no real name is ever inferred. |
| Notes | Typed rows with verified quotes. | Any model, provided the verification step still drops unverifiable rows. |
| Lifecycle | Clips, search, deletion and retention. | Not swappable for a soft delete that leaves media on disk. |

## Configuration

Every runtime setting is an environment variable documented in `.env.example`, validated at startup, with a safe local default wherever one exists.

- `DATABASE_PATH` · required · Meetings, consent, transcripts and notes.
- `MEDIA_DIR` · required · Source recordings, chunks and clips. Deletion must clear all three.
- `WHISPER_BINARY` · required · The local transcription binary.
- `WHISPER_MODEL_PATH` · required · The model file. Bigger is slower and usually better; measure on your own machine.
- `CHUNK_SECONDS` · required · Chunk length for long audio. Timestamps are offset by the chunk start.
- `CHUNK_OVERLAP_SECONDS` · required · Overlap between chunks, de-duplicated on merge. Zero here means words lost at every seam.
- `RETENTION_DAYS` · required · Set at import. The scheduled job reports what it removed.
- `NOTES_MODEL` · optional · Optional. Unset means the extractive fallback, which must stay usable.

## Production baseline

- Security: least privilege, input validation at every boundary, secret redaction in logs, rate limits on abuse-prone paths, no invented security primitives.
- Data: explicit schema and migrations, transactional writes where integrity matters, backup and restore procedures that have been exercised.
- Integrations: adapters around third-party providers, idempotent webhook or job processing, bounded retries, timeouts.
- Observability: structured logs with request or operation ids, an error-tracking hook, and health and readiness checks where a server exists.
- Quality: unit tests for domain rules, integration tests at module boundaries, one end-to-end test of the critical path.

## Decision records

For each dependency in the stack table, keep a short note: why it was chosen, its failure mode, and how it is replaced. Do not add infrastructure until a requirement in `PRODUCT.md` justifies it.

===== AGENTS.md =====
# Agent instructions · Grain product build

- Read `PRODUCT.md` and `ARCHITECTURE.md` before changing code. The stack is fixed: Python 3.12 with FastAPI bound to localhost only, whisper.cpp run locally with segment and word timestamps, FFmpeg as a subprocess for probing, chunking and clipping, SQLite for meetings, transcripts and notes, media in a folder beside it, An optional model behind one interface, with an extractive fallback.
- Implement milestone by milestone from `MILESTONES.md`; keep each change reviewable and leave the application runnable at every commit.
- Treat authentication, payments, encryption, imports, webhooks and destructive actions as high-risk boundaries when present.
- Never invent cryptography or silently weaken a requirement to make a check pass.
- Put every external service behind an interface with a deterministic fake for tests.
- Add migrations and rollback or recovery notes for every persistent data change.
- Log useful operational context without credentials, tokens, passwords or personal data.
- Update documentation and run every check before completing a milestone.

## Known traps

- Making consent a checkbox that is already ticked turns the whole record into a lie that looks like compliance, and a consent row written after transcription proves nothing at all. Require the parties and the method to be typed, block the transcription job until the row exists, and store the time it was written rather than the time the meeting happened.
- Cutting a file into fixed ten minute chunks and concatenating the results slices words in half at every boundary and leaves each chunk's timestamps starting again at zero, so the transcript reads fine and every citation in it points at the wrong moment. Overlap the chunks, offset the timestamps by the chunk start, and de-duplicate across the overlap by matching text.
- Guessing that the voice saying 'I will do it' belongs to the person whose name was said earlier produces an action item assigned to the wrong colleague, which is discovered in the following meeting. Keep labels anonymous until a person renames them, and carry the confidence into the notes so an attribution never looks more certain than it is.
- A model asked to summarise a meeting will produce a clean, plausible action item that nobody raised, and it will be the one item somebody acts on. Require a quote and a segment id on every row, check the quote actually appears in that segment before saving, and delete the rows that fail rather than keeping them with a caveat.
- Deleting the meeting row and leaving the audio, the chunk files and the exported clips on disk means a person who asked for a recording to be removed still has it sitting in three folders, and nobody notices until an audit. Make deletion a single operation over every artefact, and assert in the suite that no file with that meeting's identifier survives it.

===== MILESTONES.md =====
# Delivery milestones · Grain

Estimated effort: **multi-day** for the indie phases; the production-only milestones add the trust and operability layer.

## M1 · Intake that will not start without consent

A recording cannot be transcribed until who was told is recorded.

### Steps

1. Probe before accepting
   Duration, streams and codecs decide whether the rest of the pipeline can work at all.
   Files: `src/intake.py`, `src/media/probe.py`
2. Block the job on the consent row
   In the queueing code, so there is one place it can be bypassed and it is visible.
   Files: `src/jobs.py`, `tests/test_consent_gate.py`
3. Set the retention date at import
   Decided once, when nobody is attached to the recording yet.
   Files: `src/retention.py`

### Done when

- [ ] Transcription without consent is refused
- [ ] Consent records parties, method and a system timestamp
- [ ] Unprobeable media is rejected at import with a reason
- [ ] Every meeting carries a retention date

### Watch out

- Making consent a checkbox that is already ticked turns the whole record into a lie that looks like compliance, and a consent row written after transcription proves nothing at all. Require the parties and the method to be typed, block the transcription job until the row exists, and store the time it was written rather than the time the meeting happened.

## M2 · Long audio without losing the seams

A two hour recording transcribes in chunks with one continuous clock.

### Steps

1. Overlap the chunks deliberately
   A few seconds of overlap is the cheapest fix for words cut in half.
   Files: `src/transcribe/chunk.py`
2. Offset timestamps by chunk start
   This one line is the difference between citations that work and citations that lie.
   Files: `src/transcribe/merge.py`, `tests/test_timestamps.py`
3. Record chunk progress for resume
   Long jobs fail, and restarting a ninety minute transcription is how people give up.
   Files: `src/jobs.py`

### Done when

- [ ] Citations play the words they quote
- [ ] A sentence across a boundary appears exactly once
- [ ] Transcription resumes after a kill
- [ ] A known fixture matches its expected transcript

### Watch out

- Cutting a file into fixed ten minute chunks and concatenating the results slices words in half at every boundary and leaves each chunk's timestamps starting again at zero, so the transcript reads fine and every citation in it points at the wrong moment. Overlap the chunks, offset the timestamps by the chunk start, and de-duplicate across the overlap by matching text.

## M3 · Speakers labelled as honestly as you can label them

Attribution is either evidenced or clearly provisional.

### Steps

1. Keep labels provisional by default
   A letter is honest; a guessed name is a claim.
   Files: `src/speakers.py`
2. Rename once, apply everywhere
   Otherwise people rename the first three segments and give up.
   Files: `src/speakers.py`, `tests/test_rename.py`
3. Carry confidence into every quote
   So the note can show that the attribution is weak.
   Files: `src/notes/render.py`

### Done when

- [ ] Labels stay anonymous until renamed
- [ ] Renaming updates every covered segment
- [ ] Confidence is stored and displayed with quotes
- [ ] A single speaker file yields one label

### Watch out

- Guessing that the voice saying 'I will do it' belongs to the person whose name was said earlier produces an action item assigned to the wrong colleague, which is discovered in the following meeting. Keep labels anonymous until a person renames them, and carry the confidence into the notes so an attribution never looks more certain than it is.

## M4 · Notes that cite the second they came from

Every decision, action and question quotes the transcript and links to a timestamp.

### Steps

1. Verify each quote against its segment
   The check is a substring match, and it catches almost everything invented.
   Files: `src/notes/verify.py`, `tests/test_quote_verification.py`
2. Drop unverifiable rows entirely
   A caveat next to a wrong action item does not stop anyone acting on it.
   Files: `src/notes/build.py`
3. Ship the extractive fallback first
   Then the model is an improvement rather than a dependency.
   Files: `src/notes/extractive.py`

### Done when

- [ ] Every note row cites a segment and a verified quote
- [ ] A fabricated quote is rejected
- [ ] The no-model fallback produces notes and says so
- [ ] A note line plays from its timestamp

### Watch out

- A model asked to summarise a meeting will produce a clean, plausible action item that nobody raised, and it will be the one item somebody acts on. Require a quote and a segment id on every row, check the quote actually appears in that segment before saving, and delete the rows that fail rather than keeping them with a caveat.

## M5 · Clips, search, and deletion that actually deletes (production only)

Getting a moment out is easy and getting a meeting gone is complete.

### Steps

1. Cut on word boundaries
   Word timestamps exist precisely so a clip does not start mid syllable.
   Files: `src/clips.py`
2. Delete every artefact in one operation
   Media, chunks, transcript, notes, clips, in one function with one test.
   Files: `src/delete.py`, `tests/test_delete_complete.py`
3. Run retention on a schedule and report
   A retention policy nobody has seen run is a paragraph, not a policy.
   Files: `src/retention.py`

### Done when

- [ ] Deletion leaves no file for that meeting
- [ ] Clips start and end on word boundaries
- [ ] Search returns segments and plays from them
- [ ] Retention runs on a schedule and reports removals

### Watch out

- Deleting the meeting row and leaving the audio, the chunk files and the exported clips on disk means a person who asked for a recording to be removed still has it sitting in three folders, and nobody notices until an audit. Make deletion a single operation over every artefact, and assert in the suite that no file with that meeting's identifier survives it.

===== OPERATIONS.md =====
# Operations · Grain

## Backup

The database and the media directory together, since a transcript whose audio is gone cannot be checked and audio with no transcript has to be processed again. Encrypt the backup, because it contains recordings of private conversations.

## Restore

Restore both, then confirm three random note citations still play the right words. If timestamps drift after a restore, the chunk offsets were rebuilt rather than restored.

Do a restore drill before the first real user, and write the date here when it passes.

## Monitoring

Failed and stalled jobs, meetings past their retention date, and the count of note rows dropped by quote verification. A sudden rise in dropped rows usually means a model or prompt change, not a bad meeting.

## Incident checklist

The serious incident is a recording that should not exist, either because consent was never recorded or because a deletion did not remove everything. Both are answered by the consent table and by the deletion test that walks the media tree.

1. Contain the issue without destroying evidence or user data.
2. Record the timeline and affected scope.
3. Rotate exposed secrets and revoke compromised sessions or credentials.
4. Restore from a verified backup when needed.
5. Document the root cause, the remediation and the regression test.

## Release gate

- [ ] Transcription without consent is confirmed refused
- [ ] Citations are confirmed to play the quoted words
- [ ] Speaker names are confirmed never inferred
- [ ] Fabricated quotes are confirmed rejected
- [ ] Deletion is confirmed to leave no file behind

## Launch constraint

Do not market omitted Grain capabilities as implemented. The non-goals in `PRODUCT.md` remain user-visible limitations until they are deliberately delivered.

===== .env.example =====
# Copy to .env and fill in. Never commit .env; this file documents it.

# Required. Meetings, consent, transcripts and notes.
DATABASE_PATH=./data/meetings.db

# Required. Source recordings, chunks and clips. Deletion must clear all three.
MEDIA_DIR=./data/media

# Required. The local transcription binary.
WHISPER_BINARY=/usr/local/bin/whisper-cli

# Required. The model file. Bigger is slower and usually better; measure on your own machine.
WHISPER_MODEL_PATH=./models/ggml-base.en.bin

# Required. Chunk length for long audio. Timestamps are offset by the chunk start.
CHUNK_SECONDS=600

# Required. Overlap between chunks, de-duplicated on merge. Zero here means words lost at every seam.
CHUNK_OVERLAP_SECONDS=5

# Required. Set at import. The scheduled job reports what it removed.
RETENTION_DAYS=365

# Optional. Optional. Unset means the extractive fallback, which must stay usable.
NOTES_MODEL=none

# Grain · indie build

A local meeting archive whose hard problems are refusing to process a recording nobody consented to, transcribing long audio in chunks without losing or duplicating words at the seams, and producing notes where every claim cites a timestamp you can play.

Estimated effort: **multi-day**. Work `BUILD_PLAN.md` top to bottom · every phase ends in a check that has to pass before the next one starts.

## Stack

| Part | Choice | Why |
| --- | --- | --- |
| Runtime | Python 3.12 with FastAPI bound to localhost only | recordings of private conversations should not sit on a public port |
| Transcription | whisper.cpp run locally with segment and word timestamps | the audio never leaves the machine and the clock is the whole product |
| Media | FFmpeg as a subprocess for probing, chunking and clipping | one tool that already understands every container somebody will hand you |
| Storage | SQLite for meetings, transcripts and notes, media in a folder beside it | a transcript is small, a two hour recording is not |
| Notes | An optional model behind one interface, with an extractive fallback | the archive has to stay useful when no model is available |

## Before you start

Have every one of these ready. The plan assumes them from step one.

- [ ] **Python 3.12 or newer** · free
  - Why: Runs the pipeline, the job runner and the local interface.
  - Get it: python.org, or your package manager.
  - Verify: python3 --version prints 3.12 or higher
- [ ] **FFmpeg with ffprobe** · free
  - Why: Probing, chunking and clipping all go through it, and a build without ffprobe fails at the first import rather than at the first clip.
  - Get it: Your package manager, or a static build from the official downloads page.
  - Verify: ffprobe -version and ffmpeg -version both print a version
- [ ] **whisper.cpp with a downloaded model file** · free
  - Why: Transcription runs locally, and the model file is a separate download that is easy to forget until the first job fails.
  - Get it: Build whisper.cpp and download a model file of a size your machine can actually run.
  - Verify: The binary transcribes the bundled sample and prints timestamps
- [ ] **One real recording over ninety minutes, with consent** · free
  - Why: Chunking, resume and timestamp offsets are all fine on a five minute file and all break on a long one, which is the only length that matters.
  - Get it: A recording you made of a meeting where everyone was told, or a long public talk you are permitted to use.
  - Verify: It transcribes end to end and three random citations play the right words
- [ ] **Your consent and retention policy** · free
  - Why: Both are enforced by code and asserted by tests, and writing them after the fact means writing whatever the code already does.
  - Get it: Write down who must be informed, how, and how long recordings and transcripts are kept.
  - Verify: Both appear in the README and their values are the ones the tests use

## Quick start

Follow Phase 1 in `BUILD_PLAN.md`; it creates the project.

Then copy `.env.example` to `.env` and fill in the values it documents.

## Honest limits

This build deliberately does not replace:

- A bot that joins calls.
- Live transcription.
- CRM and calendar integrations.
- Cross meeting coaching scores.
- live multi-speaker accuracy
- calendar and CRM integrations
- cross-call team analytics
- meeting-bot auto-join

If one of those is essential to you, that is the reason to keep paying for Grain, and the README should say so rather than pretend.

# Build brief · Grain

The one-shot brief this plan expands. `BUILD_PLAN.md` (or `MILESTONES.md`) is the same sequence broken into steps and checks; where the two disagree, the plan wins.

Build me a private meeting archive that turns a recording into notes and clips, every line pointing at the second it came from to replace Grain. Build it in phases, in the order below.
Do not write the whole thing in one pass. Finish a phase, run its "Done when"
check, fix what fails, and only then start the next phase.

### Rule zero
Two rules, and the first one is law where some of your colleagues live.
Nothing here records people who were not told. Several jurisdictions require the consent of
every party rather than one: California Penal Code section 632 makes it an offence to record a
confidential communication without the consent of all parties, with a fine of up to 2,500
dollars per violation, and the rules differ by country and by state. So there is no bot that
joins a call, no hidden capture, and no import without a stored record of who was told, how and
when. The consent record is written before transcription starts, not after.
Second, every line of a generated note points at a timestamp and quotes the transcript. A
summary sentence that cannot be traced to something somebody actually said gets dropped rather
than softened, because the only reason to trust meeting notes is that you can check them.

### Stack (fixed, do not substitute)
- Runtime: Python 3.12 with FastAPI bound to localhost only · recordings of private conversations should not sit on a public port
- Transcription: whisper.cpp run locally with segment and word timestamps · the audio never leaves the machine and the clock is the whole product
- Media: FFmpeg as a subprocess for probing, chunking and clipping · one tool that already understands every container somebody will hand you
- Storage: SQLite for meetings, transcripts and notes, media in a folder beside it · a transcript is small, a two hour recording is not
- Notes: An optional model behind one interface, with an extractive fallback · the archive has to stay useful when no model is available

### Data model (create this before Phase 1)
- meetings(id, title, occurred_at, source_path, media_sha256, duration_seconds, imported_at, delete_after)
- consent(meeting_id, parties_json, informed_how, informed_at, recorded_by, note) · written before transcription is allowed to start
- segments(meeting_id, ordinal, start_ms, end_ms, speaker_label, text, confidence) · timestamps are absolute in the meeting, never per chunk
- words(segment_id, start_ms, end_ms, text) · what makes a clip cut land between words instead of through one
- notes(id, meeting_id, kind, text, source_segment_id, quote, created_at) · kind is decision, action, question or summary, and every row cites a segment
- clips(id, meeting_id, start_ms, end_ms, path, created_at)
- jobs(id, meeting_id, kind, state, chunk_index, attempts, last_error) · transcription is resumable, because a two hour file fails at minute ninety

### Phase 1 · Intake that will not start without consent
Build: Import a file, probe it with FFmpeg for duration, streams and codecs, and record a checksum.
  Require a consent record naming the parties, how they were informed and when, before any job
  is queued. Set a retention date at import. Refuse unreadable or zero length media loudly.
Trap: Making consent a checkbox that is already ticked turns the whole record into a lie that looks like compliance, and a consent row written after transcription proves nothing at all. Require the parties and the method to be typed, block the transcription job until the row exists, and store the time it was written rather than the time the meeting happened.
Done when: Queueing transcription without a consent record fails with a clear message, tested. The consent row records the parties, the method of informing them and a written timestamp that is set by the system rather than the user. A file that FFmpeg cannot probe is rejected at import with the reason shown, not queued and failed later. Every meeting has a retention date from the moment it is imported.
Do not build yet: No transcription, notes, clips or search.

### Phase 2 · Long audio without losing the seams
Build: Split the audio into chunks with a deliberate overlap, transcribe each one, then offset every
  timestamp by its chunk start and reconcile the overlap so no word appears twice and none is
  dropped. Record chunk progress so a failed run resumes rather than restarting.
Trap: Cutting a file into fixed ten minute chunks and concatenating the results slices words in half at every boundary and leaves each chunk's timestamps starting again at zero, so the transcript reads fine and every citation in it points at the wrong moment. Overlap the chunks, offset the timestamps by the chunk start, and de-duplicate across the overlap by matching text.
Done when: Timestamps are absolute in the meeting, checked by playing three random citations and hearing the quoted words. A chunk boundary placed mid sentence produces the sentence once, tested with a fixture that straddles one. Killing the process mid transcription resumes from the last finished chunk, tested. The transcript of a known short fixture matches a stored expected text within a stated tolerance.
Do not build yet: No speaker labels, notes or clips yet.

### Phase 3 · Speakers labelled as honestly as you can label them
Build: Assign provisional speaker labels with a confidence, let the person rename a label once and
  have it apply to every segment, and never infer a real name from context. Show the label as
  provisional until it has been renamed by a human.
Trap: Guessing that the voice saying 'I will do it' belongs to the person whose name was said earlier produces an action item assigned to the wrong colleague, which is discovered in the following meeting. Keep labels anonymous until a person renames them, and carry the confidence into the notes so an attribution never looks more certain than it is.
Done when: Speaker labels are anonymous until renamed, and renaming one updates every segment it covers. No code path derives a name from the transcript text, verified by review. Segment confidence is stored and shown wherever a quote is attributed. A recording with one speaker does not produce three, checked against a single speaker fixture.
Do not build yet: No notes, clips, search or export yet.

### Phase 4 · Notes that cite the second they came from
Build: Produce notes as typed rows: decision, action, question, summary. Each row stores a quote and
  the segment it came from. Verify every generated quote against the stored transcript text and
  drop rows that fail. Provide an extractive fallback that works with no model at all.
Trap: A model asked to summarise a meeting will produce a clean, plausible action item that nobody raised, and it will be the one item somebody acts on. Require a quote and a segment id on every row, check the quote actually appears in that segment before saving, and delete the rows that fail rather than keeping them with a caveat.
Done when: Every note row has a segment id and a quote, and the quote is confirmed to appear in that segment's text before the row is written. A deliberately fabricated quote is rejected by the verification step, tested with a fixture. With no model configured, the extractive fallback still produces usable notes and says which mode was used. Clicking a note line plays the recording from its timestamp.
Do not build yet: No clips, cross meeting search or export yet.

### Phase 5 · Clips, search, and deletion that actually deletes
Build: Cut clips on word boundaries using the word timestamps, re-encoding the clip rather than
  stream copying when the cut point is not on a keyframe. Search across meetings over transcript
  text. Delete a meeting by removing media, chunks, transcript, notes and clips together, and
  run retention as a scheduled job that reports what it removed.
Trap: Deleting the meeting row and leaving the audio, the chunk files and the exported clips on disk means a person who asked for a recording to be removed still has it sitting in three folders, and nobody notices until an audit. Make deletion a single operation over every artefact, and assert in the suite that no file with that meeting's identifier survives it.
Done when: Deleting a meeting leaves no file containing its identifier anywhere under the media directory, asserted by a test that walks the tree. A clip cut mid word starts and ends on a word boundary, checked against the word timestamps. Search returns the segment and plays from its timestamp. Retention runs on a schedule and reports the number of meetings and bytes removed.
Do not build yet: Nothing after this.

### Out of scope (and why)
- A bot that joins calls, which is the feature people actually pay for and also the one that records people before they have agreed to it.
- Live transcription during a meeting, which is a streaming and latency problem rather than a batch one and needs a different engine entirely.
- CRM and calendar integrations, where each one is a credential, a schema and a permission model that outlives your interest in it.
- Cross meeting coaching scores, which rank colleagues on numbers derived from an approximate transcript and should not exist without a lot more care than a weekend allows.

### README must contain
- That a consent record is required before transcription and what it stores, with a plain note that all party consent is the rule in some jurisdictions
- That timestamps are absolute in the meeting and how chunk overlap and de-duplication work
- That speaker labels are provisional until a person renames them, and that no name is ever inferred
- That every note row is verified against the transcript and unverifiable rows are dropped
- How deletion and retention work, including that media, chunks, transcript, notes and clips go together

# Agent instructions · Grain indie build

- Read `README.md` and `BUILD_PLAN.md` before writing code. The stack is fixed: Python 3.12 with FastAPI bound to localhost only, whisper.cpp run locally with segment and word timestamps, FFmpeg as a subprocess for probing, chunking and clipping, SQLite for meetings, transcripts and notes, media in a folder beside it, An optional model behind one interface, with an extractive fallback. Do not substitute.
- Work one phase at a time, in order. Do not start a phase until every "Done when" item of the previous one passes.
- Prefer the fewest moving parts that satisfy the step. No frameworks, services or dependencies the plan does not name.
- Secrets live in `.env`, never in source or logs. Keep `.env.example` current when a variable is introduced.
- Do not invent cryptography, security guarantees, APIs or compliance claims.
- Add a focused test for every destructive, security-sensitive or data-loss path the plan names.
- Run the project checks before declaring a phase complete, and record any deliberate shortcut in the README under "Tradeoffs".

## Known traps

- Making consent a checkbox that is already ticked turns the whole record into a lie that looks like compliance, and a consent row written after transcription proves nothing at all. Require the parties and the method to be typed, block the transcription job until the row exists, and store the time it was written rather than the time the meeting happened.
- Cutting a file into fixed ten minute chunks and concatenating the results slices words in half at every boundary and leaves each chunk's timestamps starting again at zero, so the transcript reads fine and every citation in it points at the wrong moment. Overlap the chunks, offset the timestamps by the chunk start, and de-duplicate across the overlap by matching text.
- Guessing that the voice saying 'I will do it' belongs to the person whose name was said earlier produces an action item assigned to the wrong colleague, which is discovered in the following meeting. Keep labels anonymous until a person renames them, and carry the confidence into the notes so an attribution never looks more certain than it is.
- A model asked to summarise a meeting will produce a clean, plausible action item that nobody raised, and it will be the one item somebody acts on. Require a quote and a segment id on every row, check the quote actually appears in that segment before saving, and delete the rows that fail rather than keeping them with a caveat.

# Build plan · Grain

A local meeting archive whose hard problems are refusing to process a recording nobody consented to, transcribing long audio in chunks without losing or duplicating words at the seams, and producing notes where every claim cites a timestamp you can play.

Phases are in dependency order. Each ends in a "Done when" list; treat an unticked item as a blocker, not a note.

## Phase 1 · Intake that will not start without consent

A recording cannot be transcribed until who was told is recorded.

### Steps

1. Probe before accepting
   Duration, streams and codecs decide whether the rest of the pipeline can work at all.
   Files: `src/intake.py`, `src/media/probe.py`
2. Block the job on the consent row
   In the queueing code, so there is one place it can be bypassed and it is visible.
   Files: `src/jobs.py`, `tests/test_consent_gate.py`
3. Set the retention date at import
   Decided once, when nobody is attached to the recording yet.
   Files: `src/retention.py`

### Done when

- [ ] Transcription without consent is refused
- [ ] Consent records parties, method and a system timestamp
- [ ] Unprobeable media is rejected at import with a reason
- [ ] Every meeting carries a retention date

### Watch out

- Making consent a checkbox that is already ticked turns the whole record into a lie that looks like compliance, and a consent row written after transcription proves nothing at all. Require the parties and the method to be typed, block the transcription job until the row exists, and store the time it was written rather than the time the meeting happened.

## Phase 2 · Long audio without losing the seams

A two hour recording transcribes in chunks with one continuous clock.

### Steps

1. Overlap the chunks deliberately
   A few seconds of overlap is the cheapest fix for words cut in half.
   Files: `src/transcribe/chunk.py`
2. Offset timestamps by chunk start
   This one line is the difference between citations that work and citations that lie.
   Files: `src/transcribe/merge.py`, `tests/test_timestamps.py`
3. Record chunk progress for resume
   Long jobs fail, and restarting a ninety minute transcription is how people give up.
   Files: `src/jobs.py`

### Done when

- [ ] Citations play the words they quote
- [ ] A sentence across a boundary appears exactly once
- [ ] Transcription resumes after a kill
- [ ] A known fixture matches its expected transcript

### Watch out

- Cutting a file into fixed ten minute chunks and concatenating the results slices words in half at every boundary and leaves each chunk's timestamps starting again at zero, so the transcript reads fine and every citation in it points at the wrong moment. Overlap the chunks, offset the timestamps by the chunk start, and de-duplicate across the overlap by matching text.

## Phase 3 · Speakers labelled as honestly as you can label them

Attribution is either evidenced or clearly provisional.

### Steps

1. Keep labels provisional by default
   A letter is honest; a guessed name is a claim.
   Files: `src/speakers.py`
2. Rename once, apply everywhere
   Otherwise people rename the first three segments and give up.
   Files: `src/speakers.py`, `tests/test_rename.py`
3. Carry confidence into every quote
   So the note can show that the attribution is weak.
   Files: `src/notes/render.py`

### Done when

- [ ] Labels stay anonymous until renamed
- [ ] Renaming updates every covered segment
- [ ] Confidence is stored and displayed with quotes
- [ ] A single speaker file yields one label

### Watch out

- Guessing that the voice saying 'I will do it' belongs to the person whose name was said earlier produces an action item assigned to the wrong colleague, which is discovered in the following meeting. Keep labels anonymous until a person renames them, and carry the confidence into the notes so an attribution never looks more certain than it is.

## Phase 4 · Notes that cite the second they came from

Every decision, action and question quotes the transcript and links to a timestamp.

### Steps

1. Verify each quote against its segment
   The check is a substring match, and it catches almost everything invented.
   Files: `src/notes/verify.py`, `tests/test_quote_verification.py`
2. Drop unverifiable rows entirely
   A caveat next to a wrong action item does not stop anyone acting on it.
   Files: `src/notes/build.py`
3. Ship the extractive fallback first
   Then the model is an improvement rather than a dependency.
   Files: `src/notes/extractive.py`

### Done when

- [ ] Every note row cites a segment and a verified quote
- [ ] A fabricated quote is rejected
- [ ] The no-model fallback produces notes and says so
- [ ] A note line plays from its timestamp

### Watch out

- A model asked to summarise a meeting will produce a clean, plausible action item that nobody raised, and it will be the one item somebody acts on. Require a quote and a segment id on every row, check the quote actually appears in that segment before saving, and delete the rows that fail rather than keeping them with a caveat.

## Not in this build

- A bot that joins calls.
- Live transcription.
- CRM and calendar integrations.
- Cross meeting coaching scores.

## After v1, if you want it

- Topic grouping across meetings, once you trust the transcripts enough to aggregate them
- A shareable read only page per meeting, which is a rendering job plus a decision about who sees it
- Translation of the transcript, kept beside the original rather than replacing it

# Copy to .env and fill in. Never commit .env; this file documents it.

# Required. Meetings, consent, transcripts and notes.
DATABASE_PATH=./data/meetings.db

# Required. Source recordings, chunks and clips. Deletion must clear all three.
MEDIA_DIR=./data/media

# Required. The local transcription binary.
WHISPER_BINARY=/usr/local/bin/whisper-cli

# Required. The model file. Bigger is slower and usually better; measure on your own machine.
WHISPER_MODEL_PATH=./models/ggml-base.en.bin

# Required. Chunk length for long audio. Timestamps are offset by the chunk start.
CHUNK_SECONDS=600

# Required. Overlap between chunks, de-duplicated on merge. Zero here means words lost at every seam.
CHUNK_OVERLAP_SECONDS=5

# Required. Set at import. The scheduled job reports what it removed.
RETENTION_DAYS=365

# Optional. Optional. Unset means the extractive fallback, which must stay usable.
NOTES_MODEL=none

# Grain · product brief

## Problem

The visible meeting clips + intelligence loop is buildable, but a credible replacement needs more than the first screen. Grain earns its keep through capture, integrations, reliability, so expect a weekend or multi-day build and a narrower personal scope.

## Product outcome

An archive that will not touch a recording without a consent record, transcribes long audio with one continuous clock, produces notes whose every line quotes something that was actually said, and deletes a meeting completely when asked.

## Target user

A builder who needs a maintainable product foundation, not a one-off demo.

## Required capabilities

- Python 3.12
- FFmpeg
- local Whisper model or optional API key in .env

## Explicit non-goals for v1

- A bot that joins calls.
- Live transcription.
- CRM and calendar integrations.
- Cross meeting coaching scores.
- live multi-speaker accuracy
- calendar and CRM integrations
- cross-call team analytics
- meeting-bot auto-join

## Success criteria

- Transcription without consent is confirmed refused
- Citations are confirmed to play the quoted words
- Speaker names are confirmed never inferred
- Fabricated quotes are confirmed rejected
- Deletion is confirmed to leave no file behind

# Build brief · Grain

The one-shot brief this plan expands. `BUILD_PLAN.md` (or `MILESTONES.md`) is the same sequence broken into steps and checks; where the two disagree, the plan wins.

Build me a private meeting archive that turns a recording into notes and clips, every line pointing at the second it came from to replace Grain. Build it in phases, in the order below.
Do not write the whole thing in one pass. Finish a phase, run its "Done when"
check, fix what fails, and only then start the next phase.

### Rule zero
Two rules, and the first one is law where some of your colleagues live.
Nothing here records people who were not told. Several jurisdictions require the consent of
every party rather than one: California Penal Code section 632 makes it an offence to record a
confidential communication without the consent of all parties, with a fine of up to 2,500
dollars per violation, and the rules differ by country and by state. So there is no bot that
joins a call, no hidden capture, and no import without a stored record of who was told, how and
when. The consent record is written before transcription starts, not after.
Second, every line of a generated note points at a timestamp and quotes the transcript. A
summary sentence that cannot be traced to something somebody actually said gets dropped rather
than softened, because the only reason to trust meeting notes is that you can check them.

### Stack (fixed, do not substitute)
- Runtime: Python 3.12 with FastAPI bound to localhost only · recordings of private conversations should not sit on a public port
- Transcription: whisper.cpp run locally with segment and word timestamps · the audio never leaves the machine and the clock is the whole product
- Media: FFmpeg as a subprocess for probing, chunking and clipping · one tool that already understands every container somebody will hand you
- Storage: SQLite for meetings, transcripts and notes, media in a folder beside it · a transcript is small, a two hour recording is not
- Notes: An optional model behind one interface, with an extractive fallback · the archive has to stay useful when no model is available

### Data model (create this before Phase 1)
- meetings(id, title, occurred_at, source_path, media_sha256, duration_seconds, imported_at, delete_after)
- consent(meeting_id, parties_json, informed_how, informed_at, recorded_by, note) · written before transcription is allowed to start
- segments(meeting_id, ordinal, start_ms, end_ms, speaker_label, text, confidence) · timestamps are absolute in the meeting, never per chunk
- words(segment_id, start_ms, end_ms, text) · what makes a clip cut land between words instead of through one
- notes(id, meeting_id, kind, text, source_segment_id, quote, created_at) · kind is decision, action, question or summary, and every row cites a segment
- clips(id, meeting_id, start_ms, end_ms, path, created_at)
- jobs(id, meeting_id, kind, state, chunk_index, attempts, last_error) · transcription is resumable, because a two hour file fails at minute ninety

### Phase 1 · Intake that will not start without consent
Build: Import a file, probe it with FFmpeg for duration, streams and codecs, and record a checksum.
  Require a consent record naming the parties, how they were informed and when, before any job
  is queued. Set a retention date at import. Refuse unreadable or zero length media loudly.
Trap: Making consent a checkbox that is already ticked turns the whole record into a lie that looks like compliance, and a consent row written after transcription proves nothing at all. Require the parties and the method to be typed, block the transcription job until the row exists, and store the time it was written rather than the time the meeting happened.
Done when: Queueing transcription without a consent record fails with a clear message, tested. The consent row records the parties, the method of informing them and a written timestamp that is set by the system rather than the user. A file that FFmpeg cannot probe is rejected at import with the reason shown, not queued and failed later. Every meeting has a retention date from the moment it is imported.
Do not build yet: No transcription, notes, clips or search.

### Phase 2 · Long audio without losing the seams
Build: Split the audio into chunks with a deliberate overlap, transcribe each one, then offset every
  timestamp by its chunk start and reconcile the overlap so no word appears twice and none is
  dropped. Record chunk progress so a failed run resumes rather than restarting.
Trap: Cutting a file into fixed ten minute chunks and concatenating the results slices words in half at every boundary and leaves each chunk's timestamps starting again at zero, so the transcript reads fine and every citation in it points at the wrong moment. Overlap the chunks, offset the timestamps by the chunk start, and de-duplicate across the overlap by matching text.
Done when: Timestamps are absolute in the meeting, checked by playing three random citations and hearing the quoted words. A chunk boundary placed mid sentence produces the sentence once, tested with a fixture that straddles one. Killing the process mid transcription resumes from the last finished chunk, tested. The transcript of a known short fixture matches a stored expected text within a stated tolerance.
Do not build yet: No speaker labels, notes or clips yet.

### Phase 3 · Speakers labelled as honestly as you can label them
Build: Assign provisional speaker labels with a confidence, let the person rename a label once and
  have it apply to every segment, and never infer a real name from context. Show the label as
  provisional until it has been renamed by a human.
Trap: Guessing that the voice saying 'I will do it' belongs to the person whose name was said earlier produces an action item assigned to the wrong colleague, which is discovered in the following meeting. Keep labels anonymous until a person renames them, and carry the confidence into the notes so an attribution never looks more certain than it is.
Done when: Speaker labels are anonymous until renamed, and renaming one updates every segment it covers. No code path derives a name from the transcript text, verified by review. Segment confidence is stored and shown wherever a quote is attributed. A recording with one speaker does not produce three, checked against a single speaker fixture.
Do not build yet: No notes, clips, search or export yet.

### Phase 4 · Notes that cite the second they came from
Build: Produce notes as typed rows: decision, action, question, summary. Each row stores a quote and
  the segment it came from. Verify every generated quote against the stored transcript text and
  drop rows that fail. Provide an extractive fallback that works with no model at all.
Trap: A model asked to summarise a meeting will produce a clean, plausible action item that nobody raised, and it will be the one item somebody acts on. Require a quote and a segment id on every row, check the quote actually appears in that segment before saving, and delete the rows that fail rather than keeping them with a caveat.
Done when: Every note row has a segment id and a quote, and the quote is confirmed to appear in that segment's text before the row is written. A deliberately fabricated quote is rejected by the verification step, tested with a fixture. With no model configured, the extractive fallback still produces usable notes and says which mode was used. Clicking a note line plays the recording from its timestamp.
Do not build yet: No clips, cross meeting search or export yet.

### Phase 5 · Clips, search, and deletion that actually deletes
Build: Cut clips on word boundaries using the word timestamps, re-encoding the clip rather than
  stream copying when the cut point is not on a keyframe. Search across meetings over transcript
  text. Delete a meeting by removing media, chunks, transcript, notes and clips together, and
  run retention as a scheduled job that reports what it removed.
Trap: Deleting the meeting row and leaving the audio, the chunk files and the exported clips on disk means a person who asked for a recording to be removed still has it sitting in three folders, and nobody notices until an audit. Make deletion a single operation over every artefact, and assert in the suite that no file with that meeting's identifier survives it.
Done when: Deleting a meeting leaves no file containing its identifier anywhere under the media directory, asserted by a test that walks the tree. A clip cut mid word starts and ends on a word boundary, checked against the word timestamps. Search returns the segment and plays from its timestamp. Retention runs on a schedule and reports the number of meetings and bytes removed.
Do not build yet: Nothing after this.

### Out of scope (and why)
- A bot that joins calls, which is the feature people actually pay for and also the one that records people before they have agreed to it.
- Live transcription during a meeting, which is a streaming and latency problem rather than a batch one and needs a different engine entirely.
- CRM and calendar integrations, where each one is a credential, a schema and a permission model that outlives your interest in it.
- Cross meeting coaching scores, which rank colleagues on numbers derived from an approximate transcript and should not exist without a lot more care than a weekend allows.

### README must contain
- That a consent record is required before transcription and what it stores, with a plain note that all party consent is the rule in some jurisdictions
- That timestamps are absolute in the meeting and how chunk overlap and de-duplication work
- That speaker labels are provisional until a person renames them, and that no name is ever inferred
- That every note row is verified against the transcript and unverifiable rows are dropped
- How deletion and retention work, including that media, chunks, transcript, notes and clips go together

# Architecture · Grain

## Stack

| Part | Choice | Why |
| --- | --- | --- |
| Runtime | Python 3.12 with FastAPI bound to localhost only | recordings of private conversations should not sit on a public port |
| Transcription | whisper.cpp run locally with segment and word timestamps | the audio never leaves the machine and the clock is the whole product |
| Media | FFmpeg as a subprocess for probing, chunking and clipping | one tool that already understands every container somebody will hand you |
| Storage | SQLite for meetings, transcripts and notes, media in a folder beside it | a transcript is small, a two hour recording is not |
| Notes | An optional model behind one interface, with an extractive fallback | the archive has to stay useful when no model is available |

## Modules

Each module has one owner concern and a documented way to replace it.

| Module | Owns | How to replace it |
| --- | --- | --- |
| Intake | Probing, checksums and the consent gate. | Any importer, provided the consent gate stays in the queueing path. |
| Transcription | Chunking, offsets, overlap merge and resume. | Any engine that returns timestamps, provided offsets stay absolute. |
| Speakers | Provisional labels and confidence. | Any diarisation, provided no real name is ever inferred. |
| Notes | Typed rows with verified quotes. | Any model, provided the verification step still drops unverifiable rows. |
| Lifecycle | Clips, search, deletion and retention. | Not swappable for a soft delete that leaves media on disk. |

## Configuration

Every runtime setting is an environment variable documented in `.env.example`, validated at startup, with a safe local default wherever one exists.

- `DATABASE_PATH` · required · Meetings, consent, transcripts and notes.
- `MEDIA_DIR` · required · Source recordings, chunks and clips. Deletion must clear all three.
- `WHISPER_BINARY` · required · The local transcription binary.
- `WHISPER_MODEL_PATH` · required · The model file. Bigger is slower and usually better; measure on your own machine.
- `CHUNK_SECONDS` · required · Chunk length for long audio. Timestamps are offset by the chunk start.
- `CHUNK_OVERLAP_SECONDS` · required · Overlap between chunks, de-duplicated on merge. Zero here means words lost at every seam.
- `RETENTION_DAYS` · required · Set at import. The scheduled job reports what it removed.
- `NOTES_MODEL` · optional · Optional. Unset means the extractive fallback, which must stay usable.

## Production baseline

- Security: least privilege, input validation at every boundary, secret redaction in logs, rate limits on abuse-prone paths, no invented security primitives.
- Data: explicit schema and migrations, transactional writes where integrity matters, backup and restore procedures that have been exercised.
- Integrations: adapters around third-party providers, idempotent webhook or job processing, bounded retries, timeouts.
- Observability: structured logs with request or operation ids, an error-tracking hook, and health and readiness checks where a server exists.
- Quality: unit tests for domain rules, integration tests at module boundaries, one end-to-end test of the critical path.

## Decision records

For each dependency in the stack table, keep a short note: why it was chosen, its failure mode, and how it is replaced. Do not add infrastructure until a requirement in `PRODUCT.md` justifies it.

# Agent instructions · Grain product build

- Read `PRODUCT.md` and `ARCHITECTURE.md` before changing code. The stack is fixed: Python 3.12 with FastAPI bound to localhost only, whisper.cpp run locally with segment and word timestamps, FFmpeg as a subprocess for probing, chunking and clipping, SQLite for meetings, transcripts and notes, media in a folder beside it, An optional model behind one interface, with an extractive fallback.
- Implement milestone by milestone from `MILESTONES.md`; keep each change reviewable and leave the application runnable at every commit.
- Treat authentication, payments, encryption, imports, webhooks and destructive actions as high-risk boundaries when present.
- Never invent cryptography or silently weaken a requirement to make a check pass.
- Put every external service behind an interface with a deterministic fake for tests.
- Add migrations and rollback or recovery notes for every persistent data change.
- Log useful operational context without credentials, tokens, passwords or personal data.
- Update documentation and run every check before completing a milestone.

## Known traps

- Making consent a checkbox that is already ticked turns the whole record into a lie that looks like compliance, and a consent row written after transcription proves nothing at all. Require the parties and the method to be typed, block the transcription job until the row exists, and store the time it was written rather than the time the meeting happened.
- Cutting a file into fixed ten minute chunks and concatenating the results slices words in half at every boundary and leaves each chunk's timestamps starting again at zero, so the transcript reads fine and every citation in it points at the wrong moment. Overlap the chunks, offset the timestamps by the chunk start, and de-duplicate across the overlap by matching text.
- Guessing that the voice saying 'I will do it' belongs to the person whose name was said earlier produces an action item assigned to the wrong colleague, which is discovered in the following meeting. Keep labels anonymous until a person renames them, and carry the confidence into the notes so an attribution never looks more certain than it is.
- A model asked to summarise a meeting will produce a clean, plausible action item that nobody raised, and it will be the one item somebody acts on. Require a quote and a segment id on every row, check the quote actually appears in that segment before saving, and delete the rows that fail rather than keeping them with a caveat.
- Deleting the meeting row and leaving the audio, the chunk files and the exported clips on disk means a person who asked for a recording to be removed still has it sitting in three folders, and nobody notices until an audit. Make deletion a single operation over every artefact, and assert in the suite that no file with that meeting's identifier survives it.

# Delivery milestones · Grain

Estimated effort: **multi-day** for the indie phases; the production-only milestones add the trust and operability layer.

## M1 · Intake that will not start without consent

A recording cannot be transcribed until who was told is recorded.

### Steps

1. Probe before accepting
   Duration, streams and codecs decide whether the rest of the pipeline can work at all.
   Files: `src/intake.py`, `src/media/probe.py`
2. Block the job on the consent row
   In the queueing code, so there is one place it can be bypassed and it is visible.
   Files: `src/jobs.py`, `tests/test_consent_gate.py`
3. Set the retention date at import
   Decided once, when nobody is attached to the recording yet.
   Files: `src/retention.py`

### Done when

- [ ] Transcription without consent is refused
- [ ] Consent records parties, method and a system timestamp
- [ ] Unprobeable media is rejected at import with a reason
- [ ] Every meeting carries a retention date

### Watch out

- Making consent a checkbox that is already ticked turns the whole record into a lie that looks like compliance, and a consent row written after transcription proves nothing at all. Require the parties and the method to be typed, block the transcription job until the row exists, and store the time it was written rather than the time the meeting happened.

## M2 · Long audio without losing the seams

A two hour recording transcribes in chunks with one continuous clock.

### Steps

1. Overlap the chunks deliberately
   A few seconds of overlap is the cheapest fix for words cut in half.
   Files: `src/transcribe/chunk.py`
2. Offset timestamps by chunk start
   This one line is the difference between citations that work and citations that lie.
   Files: `src/transcribe/merge.py`, `tests/test_timestamps.py`
3. Record chunk progress for resume
   Long jobs fail, and restarting a ninety minute transcription is how people give up.
   Files: `src/jobs.py`

### Done when

- [ ] Citations play the words they quote
- [ ] A sentence across a boundary appears exactly once
- [ ] Transcription resumes after a kill
- [ ] A known fixture matches its expected transcript

### Watch out

- Cutting a file into fixed ten minute chunks and concatenating the results slices words in half at every boundary and leaves each chunk's timestamps starting again at zero, so the transcript reads fine and every citation in it points at the wrong moment. Overlap the chunks, offset the timestamps by the chunk start, and de-duplicate across the overlap by matching text.

## M3 · Speakers labelled as honestly as you can label them

Attribution is either evidenced or clearly provisional.

### Steps

1. Keep labels provisional by default
   A letter is honest; a guessed name is a claim.
   Files: `src/speakers.py`
2. Rename once, apply everywhere
   Otherwise people rename the first three segments and give up.
   Files: `src/speakers.py`, `tests/test_rename.py`
3. Carry confidence into every quote
   So the note can show that the attribution is weak.
   Files: `src/notes/render.py`

### Done when

- [ ] Labels stay anonymous until renamed
- [ ] Renaming updates every covered segment
- [ ] Confidence is stored and displayed with quotes
- [ ] A single speaker file yields one label

### Watch out

- Guessing that the voice saying 'I will do it' belongs to the person whose name was said earlier produces an action item assigned to the wrong colleague, which is discovered in the following meeting. Keep labels anonymous until a person renames them, and carry the confidence into the notes so an attribution never looks more certain than it is.

## M4 · Notes that cite the second they came from

Every decision, action and question quotes the transcript and links to a timestamp.

### Steps

1. Verify each quote against its segment
   The check is a substring match, and it catches almost everything invented.
   Files: `src/notes/verify.py`, `tests/test_quote_verification.py`
2. Drop unverifiable rows entirely
   A caveat next to a wrong action item does not stop anyone acting on it.
   Files: `src/notes/build.py`
3. Ship the extractive fallback first
   Then the model is an improvement rather than a dependency.
   Files: `src/notes/extractive.py`

### Done when

- [ ] Every note row cites a segment and a verified quote
- [ ] A fabricated quote is rejected
- [ ] The no-model fallback produces notes and says so
- [ ] A note line plays from its timestamp

### Watch out

- A model asked to summarise a meeting will produce a clean, plausible action item that nobody raised, and it will be the one item somebody acts on. Require a quote and a segment id on every row, check the quote actually appears in that segment before saving, and delete the rows that fail rather than keeping them with a caveat.

## M5 · Clips, search, and deletion that actually deletes (production only)

Getting a moment out is easy and getting a meeting gone is complete.

### Steps

1. Cut on word boundaries
   Word timestamps exist precisely so a clip does not start mid syllable.
   Files: `src/clips.py`
2. Delete every artefact in one operation
   Media, chunks, transcript, notes, clips, in one function with one test.
   Files: `src/delete.py`, `tests/test_delete_complete.py`
3. Run retention on a schedule and report
   A retention policy nobody has seen run is a paragraph, not a policy.
   Files: `src/retention.py`

### Done when

- [ ] Deletion leaves no file for that meeting
- [ ] Clips start and end on word boundaries
- [ ] Search returns segments and plays from them
- [ ] Retention runs on a schedule and reports removals

### Watch out

- Deleting the meeting row and leaving the audio, the chunk files and the exported clips on disk means a person who asked for a recording to be removed still has it sitting in three folders, and nobody notices until an audit. Make deletion a single operation over every artefact, and assert in the suite that no file with that meeting's identifier survives it.

# Operations · Grain

## Backup

The database and the media directory together, since a transcript whose audio is gone cannot be checked and audio with no transcript has to be processed again. Encrypt the backup, because it contains recordings of private conversations.

## Restore

Restore both, then confirm three random note citations still play the right words. If timestamps drift after a restore, the chunk offsets were rebuilt rather than restored.

Do a restore drill before the first real user, and write the date here when it passes.

## Monitoring

Failed and stalled jobs, meetings past their retention date, and the count of note rows dropped by quote verification. A sudden rise in dropped rows usually means a model or prompt change, not a bad meeting.

## Incident checklist

The serious incident is a recording that should not exist, either because consent was never recorded or because a deletion did not remove everything. Both are answered by the consent table and by the deletion test that walks the media tree.

1. Contain the issue without destroying evidence or user data.
2. Record the timeline and affected scope.
3. Rotate exposed secrets and revoke compromised sessions or credentials.
4. Restore from a verified backup when needed.
5. Document the root cause, the remediation and the regression test.

## Release gate

- [ ] Transcription without consent is confirmed refused
- [ ] Citations are confirmed to play the quoted words
- [ ] Speaker names are confirmed never inferred
- [ ] Fabricated quotes are confirmed rejected
- [ ] Deletion is confirmed to leave no file behind

## Launch constraint

Do not market omitted Grain capabilities as implemented. The non-goals in `PRODUCT.md` remain user-visible limitations until they are deliberately delivered.

# Copy to .env and fill in. Never commit .env; this file documents it.

# Required. Meetings, consent, transcripts and notes.
DATABASE_PATH=./data/meetings.db

# Required. Source recordings, chunks and clips. Deletion must clear all three.
MEDIA_DIR=./data/media

# Required. The local transcription binary.
WHISPER_BINARY=/usr/local/bin/whisper-cli

# Required. The model file. Bigger is slower and usually better; measure on your own machine.
WHISPER_MODEL_PATH=./models/ggml-base.en.bin

# Required. Chunk length for long audio. Timestamps are offset by the chunk start.
CHUNK_SECONDS=600

# Required. Overlap between chunks, de-duplicated on merge. Zero here means words lost at every seam.
CHUNK_OVERLAP_SECONDS=5

# Required. Set at import. The scheduled job reports what it removed.
RETENTION_DAYS=365

# Optional. Optional. Unset means the extractive fallback, which must stay usable.
NOTES_MODEL=none

$ choose a build depth, inspect the files, then open the complete pack in your agent