Vibecode OpusClip

You are building a lean indie version of OpusClip.

Create the following project files first, then implement the application by following them. Keep the files updated as decisions change. Do not collapse this into a single README or prompt.

===== README.md =====
# OpusClip · indie build

A clipping pipeline where the selection problem is honest and the rendering problem is exact: finding passages that stand alone is a judgement the tool should assist rather than claim, and reframing to vertical without decapitating the speaker is a tracking problem with a measurable answer.

Estimated effort: **multi-day**. Work `BUILD_PLAN.md` top to bottom · every phase ends in a check that has to pass before the next one starts.

## Stack

| Part | Choice | Why |
| --- | --- | --- |
| Transcription | Local recognition with word timings | every cut point comes from a word boundary, so the timings are the foundation |
| Selection | Candidate passages scored on stated signals, proposed not chosen | what makes a clip work is a judgement and the tool should say so |
| Framing | Subject tracking with a smoothed crop path | a crop that jumps between speakers is unwatchable |
| Captions | Timings from the transcript, never estimated | captions estimated from reading speed drift within a minute |
| Runtime | Python 3.12 with ffmpeg | the work is timing arithmetic and video processing |

## Before you start

Have every one of these ready. The plan assumes them from step one.

- [ ] **Python 3.12 or newer** · free
  - Why: Orchestrates transcription, selection and rendering.
  - Get it: python.org or your package manager, then a virtual environment.
  - Verify: python3 --version prints 3.12 or higher
- [ ] **ffmpeg with the codecs you need** · free
  - Why: Cutting, cropping and rendering all run through it, and reimplementing any of it is not the project.
  - Get it: Your package manager, confirming it handles your source format and the vertical output your targets require.
  - Verify: A clip cuts and re-encodes at the target aspect without quality loss beyond expectation
- [ ] **Local speech recognition with word timings** · free, needs machine time
  - Why: Every cut point and every caption derives from word timings, and segment-level timings are not precise enough to cut on.
  - Get it: whisper.cpp or an equivalent producing word-level timings, and check it does before building on it.
  - Verify: A five minute clip produces per-word timings
- [ ] **Face detection and tracking** · free
  - Why: Reframing to vertical requires knowing where the speaker is in every frame, and a static crop is usually wrong.
  - Get it: Any detector with tracking across frames, evaluated on your own footage rather than on a benchmark.
  - Verify: A speaker moving across frame is tracked without losing them
- [ ] **A real long recording with two speakers** · free if you have one
  - Why: Seam losses, speaker changes and framing failures only appear in real material of realistic length.
  - Get it: A two hour recording with two people, including a stretch where they talk over each other and one where the speaker leaves frame.
  - Verify: A two hour two-speaker source is available with both cases present

## Quick start

Follow Phase 1 in `BUILD_PLAN.md`; it creates the project.

Then copy `.env.example` to `.env` and fill in the values it documents.

## Honest limits

This build deliberately does not replace:

- Predicting which clips will perform.
- Publishing to platforms.
- Music, effects and heavy editing.
- Multi-camera switching.
- professional codecs, effects, and collaboration
- production codecs, rendering speed, and media templates
- hosted recording reliability
- large-scale rendering
- distribution and podcast hosting

If one of those is essential to you, that is the reason to keep paying for OpusClip, and the README should say so rather than pretend.

===== BRIEF.md =====
# Build brief · OpusClip

The one-shot brief this plan expands. `BUILD_PLAN.md` (or `MILESTONES.md`) is the same sequence broken into steps and checks; where the two disagree, the plan wins.

Build me short clips cut from a long video, where the cut points come from what was said to replace OpusClip. Build it in phases, in the order below.
Do not write the whole thing in one pass. Finish a phase, run its "Done when"
check, fix what fails, and only then start the next phase.

### Stack (fixed, do not substitute)
- Transcription: Local recognition with word timings · every cut point comes from a word boundary, so the timings are the foundation
- Selection: Candidate passages scored on stated signals, proposed not chosen · what makes a clip work is a judgement and the tool should say so
- Framing: Subject tracking with a smoothed crop path · a crop that jumps between speakers is unwatchable
- Captions: Timings from the transcript, never estimated · captions estimated from reading speed drift within a minute
- Runtime: Python 3.12 with ffmpeg · the work is timing arithmetic and video processing

### Data model (create this before Phase 1)
- sources(id, path, duration_ms, width, height, fps, imported_at)
- words(source_id, ordinal, text, start_ms, end_ms, speaker, confidence)
- candidates(id, source_id, start_ms, end_ms, signals_json, score, rationale, accepted_by)
- clips(id, candidate_id, start_ms, end_ms, aspect, crop_path_json, rendered_path, rendered_at)
- captions(clip_id, ordinal, start_ms, end_ms, text) · derived from words, never estimated
- speakers(source_id, id, label, face_track_id) · because reframing needs to know who is talking
- A candidate stores the signals behind its score rather than only the number, so a user can see why a passage was suggested and disagree with it.

### Phase 1 · Words with times you can cut on
Build: Transcribe locally with word-level timings and speaker labels where available. Chunk long
  recordings with overlap and merge without losing words at the seams. Verify the final
  timestamp against the source duration.
Trap: Chunking a two hour recording without overlap loses words at every boundary, and every cut point after that is derived from a transcript that has drifted from the audio. Overlap, merge on matching text, and check the word count and final timestamp against a single-pass transcription of a shorter excerpt.
Done when: A two hour source transcribes with no words lost or duplicated at chunk seams, verified against a single-pass excerpt. The final word's timestamp matches the source duration within tolerance. Speaker labels are produced where the source supports them and their absence is recorded rather than guessed. Transcription runs with networking disabled.
Do not build yet: No selection, framing, captions or rendering.

### Phase 2 · Candidates proposed, with reasons
Build: Score candidate passages on stated signals such as a complete thought, a question and
  answer pair, or a self-contained explanation. Store the signals alongside the score.
  Present candidates for acceptance rather than producing clips automatically.
Trap: Producing a score with no explanation makes disagreeing with the tool impossible, and the user either accepts everything or ignores the feature. Store which signals fired for each candidate and show them, so a suggestion is an argument the person can reject.
Done when: Every candidate shows the signals that produced its score rather than only a number. Candidates are proposed and require acceptance before rendering. A source with no self-contained passages produces few candidates rather than filling a quota. Candidate boundaries fall on word boundaries and start at the beginning of a sentence.
Do not build yet: No framing, captions or rendering yet.

### Phase 3 · Reframing without decapitating anyone
Build: Track faces and derive a crop path, smoothed so it does not jitter. Handle a cut between
  two speakers, a speaker who leaves the frame, and a shot with nobody in it. Verify the
  head stays within the crop throughout.
Trap: A crop that follows the detected face position frame by frame jitters constantly and is more distracting than a static crop, while one that snaps between two speakers is nauseating. Smooth the path, move only when the subject has genuinely moved, and cut rather than pan when the speaker changes.
Done when: A vertical crop keeps the speaker's head fully in frame throughout, verified by inspecting the output rather than the tracking data. The crop path is smooth rather than jittery. A speaker change produces a cut rather than a pan. A shot with no detected subject falls back to a stated behaviour rather than drifting.
Do not build yet: No captions or rendering yet.

### Phase 4 · Captions from the transcript
Build: Derive caption timings from the transcript's word timings, offset for the clip's start.
  Break lines at natural boundaries within a character limit. Verify synchronisation at the
  end of the clip rather than the start.
Trap: Estimating caption timing from word count and reading speed drifts audibly within the first minute, and checking only the opening seconds always looks fine. Use the word timings you already have, and verify at the end where drift accumulates.
Done when: Captions match the speech at the end of a sixty second clip, verified by watching. Line breaks fall at natural boundaries within a stated character limit. Caption timings are offset correctly for a clip cut from the middle of a source. A word with low recognition confidence is marked rather than presented as certain.
Do not build yet: No rendering or batch yet.

### Phase 5 · Rendering and keeping the source
Build: Render from a declarative clip description so the same inputs produce the same file.
  Open sources read only and checksum them in the suite. Batch render with per-clip failure
  isolation and a manifest.
Trap: Any path that writes to the source file, including a helpfully added normalisation pass, damages an original recording that may be irreplaceable, and the loss is discovered when someone wants to re-clip it. Open read only, checksum before and after, and fail the build if they differ.
Done when: The same clip description renders byte-identically twice. Source checksums are unchanged after a full batch, asserted in the suite. A batch of thirty clips isolates per-clip failures and reports them individually. The manifest records inputs, settings and outputs so a render is reproducible from it.
Do not build yet: Nothing after this.

### Out of scope (and why)
- Predicting which clips will perform well, which no tool can do and which every product in this category implies.
- Publishing to platforms, which has its own rate limits, costs and disclosure requirements.
- Music, effects and heavy editing, which is production rather than clipping.
- Multi-camera switching, which is a substantially harder framing problem than tracking one speaker.

### README must contain
- That every cut point comes from a word timing, so transcription accuracy is the foundation
- That candidates carry the signals behind their score, so a suggestion can be argued with
- That the crop path is smoothed and a speaker change cuts rather than pans
- That captions come from the transcript's timings and are verified at the end of a clip
- That sources are opened read only and checksummed in the test suite

===== AGENTS.md =====
# Agent instructions · OpusClip indie build

- Read `README.md` and `BUILD_PLAN.md` before writing code. The stack is fixed: Local recognition with word timings, Candidate passages scored on stated signals, proposed not chosen, Subject tracking with a smoothed crop path, Timings from the transcript, never estimated, Python 3.12 with ffmpeg. Do not substitute.
- Work one phase at a time, in order. Do not start a phase until every "Done when" item of the previous one passes.
- Prefer the fewest moving parts that satisfy the step. No frameworks, services or dependencies the plan does not name.
- Secrets live in `.env`, never in source or logs. Keep `.env.example` current when a variable is introduced.
- Do not invent cryptography, security guarantees, APIs or compliance claims.
- Add a focused test for every destructive, security-sensitive or data-loss path the plan names.
- Run the project checks before declaring a phase complete, and record any deliberate shortcut in the README under "Tradeoffs".

## Known traps

- Chunking a two hour recording without overlap loses words at every boundary, and every cut point after that is derived from a transcript that has drifted from the audio. Overlap, merge on matching text, and check the word count and final timestamp against a single-pass transcription of a shorter excerpt.
- Producing a score with no explanation makes disagreeing with the tool impossible, and the user either accepts everything or ignores the feature. Store which signals fired for each candidate and show them, so a suggestion is an argument the person can reject.
- A crop that follows the detected face position frame by frame jitters constantly and is more distracting than a static crop, while one that snaps between two speakers is nauseating. Smooth the path, move only when the subject has genuinely moved, and cut rather than pan when the speaker changes.
- Estimating caption timing from word count and reading speed drifts audibly within the first minute, and checking only the opening seconds always looks fine. Use the word timings you already have, and verify at the end where drift accumulates.

===== BUILD_PLAN.md =====
# Build plan · OpusClip

A clipping pipeline where the selection problem is honest and the rendering problem is exact: finding passages that stand alone is a judgement the tool should assist rather than claim, and reframing to vertical without decapitating the speaker is a tracking problem with a measurable answer.

Phases are in dependency order. Each ends in a "Done when" list; treat an unticked item as a blocker, not a note.

## Phase 1 · Words with times you can cut on

A long recording transcribes with timings accurate enough to cut by.

### Steps

1. Chunk with overlap and merge on text
   The seams are where the words go missing and everything downstream drifts.
   Files: `src/transcribe.py`, `tests/test_seams.py`
2. Verify against the source duration
   A transcript ending early means every later cut point is wrong.
   Files: `src/verify.py`
3. Record absent speaker labels as absent
   Guessing who spoke is worse than saying you do not know.
   Files: `src/speakers.py`

### Done when

- [ ] A two hour source transcribes with no seam losses
- [ ] The final timestamp matches the source duration
- [ ] Absent speaker labels are recorded, not guessed
- [ ] Transcription works offline

### Watch out

- Chunking a two hour recording without overlap loses words at every boundary, and every cut point after that is derived from a transcript that has drifted from the audio. Overlap, merge on matching text, and check the word count and final timestamp against a single-pass transcription of a shorter excerpt.

## Phase 2 · Candidates proposed, with reasons

Suggested passages carry the signals behind them.

### Steps

1. Score on stated signals
   A number with no reasons cannot be argued with.
   Files: `src/candidates.py`
2. Require acceptance before rendering
   Automatic clipping produces volume rather than clips.
   Files: `src/review.py`
3. Snap boundaries to sentences
   A clip starting mid-sentence is discarded by whoever watches it.
   Files: `src/boundaries.py`

### Done when

- [ ] Candidates show their signals
- [ ] Rendering requires acceptance
- [ ] A thin source produces few candidates
- [ ] Boundaries fall on sentence starts

### Watch out

- Producing a score with no explanation makes disagreeing with the tool impossible, and the user either accepts everything or ignores the feature. Store which signals fired for each candidate and show them, so a suggestion is an argument the person can reject.

## Phase 3 · Reframing without decapitating anyone

A vertical crop follows the speaker smoothly and keeps their head in frame.

### Steps

1. Smooth the crop path
   Frame-by-frame following is jitter, which is worse than a static crop.
   Files: `src/framing.py`
2. Cut rather than pan on speaker change
   A fast pan between two people is nauseating.
   Files: `src/framing.py`
3. Assert the head stays in frame
   Checked against the output, not against the tracking confidence.
   Files: `tests/test_framing.py`

### Done when

- [ ] The head stays fully in frame throughout
- [ ] The crop path is smooth rather than jittery
- [ ] Speaker changes cut rather than pan
- [ ] No detected subject has a stated fallback

### Watch out

- A crop that follows the detected face position frame by frame jitters constantly and is more distracting than a static crop, while one that snaps between two speakers is nauseating. Smooth the path, move only when the subject has genuinely moved, and cut rather than pan when the speaker changes.

## Phase 4 · Captions from the transcript

Captions match the speech exactly at the end of a clip.

### Steps

1. Use the transcript's word timings
   You already have them. Estimation drifts within a minute.
   Files: `src/captions.py`
2. Break lines at natural boundaries
   A caption breaking mid-phrase is harder to read than a longer line.
   Files: `src/captions.py`
3. Verify at the end of the clip
   The opening seconds always look correct.
   Files: `tests/test_sync.py`

### Done when

- [ ] Captions match speech at the end of a clip
- [ ] Line breaks fall at natural boundaries
- [ ] Offsets are correct for mid-source clips
- [ ] Low confidence words are marked

### Watch out

- Estimating caption timing from word count and reading speed drifts audibly within the first minute, and checking only the opening seconds always looks fine. Use the word timings you already have, and verify at the end where drift accumulates.

## Not in this build

- Predicting which clips will perform.
- Publishing to platforms.
- Music, effects and heavy editing.
- Multi-camera switching.

## After v1, if you want it

- Speaker-aware candidate selection, so a clip does not start mid-answer
- A review interface that plays candidates back to back, which is how the accepting actually happens
- Alternative aspect ratios from one render description, since targets differ

===== .env.example =====
# Copy to .env and fill in. Never commit .env; this file documents it.

# Required. Original recordings, opened read only.
SOURCE_DIR=./sources

# Required. Rendered clips and manifests. Never the source directory.
OUTPUT_DIR=./clips

# Required. Timing quality matters more here than word accuracy.
ASR_MODEL=./models/ggml-medium.en.bin

# Required. The output aspect, which drives the whole framing problem.
TARGET_ASPECT=9:16

# Required. Too little jitters, too much lags behind the speaker.
CROP_SMOOTHING_MS=400

# Required. Present as a constant. Automatic clipping produces volume rather than clips.
AUTO_ACCEPT_CANDIDATES=never

You are building a lean indie version of OpusClip.

Create the following project files first, then implement the application by following them. Keep the files updated as decisions change. Do not collapse this into a single README or prompt.

===== README.md =====
# OpusClip · indie build

A clipping pipeline where the selection problem is honest and the rendering problem is exact: finding passages that stand alone is a judgement the tool should assist rather than claim, and reframing to vertical without decapitating the speaker is a tracking problem with a measurable answer.

Estimated effort: **multi-day**. Work `BUILD_PLAN.md` top to bottom · every phase ends in a check that has to pass before the next one starts.

## Stack

| Part | Choice | Why |
| --- | --- | --- |
| Transcription | Local recognition with word timings | every cut point comes from a word boundary, so the timings are the foundation |
| Selection | Candidate passages scored on stated signals, proposed not chosen | what makes a clip work is a judgement and the tool should say so |
| Framing | Subject tracking with a smoothed crop path | a crop that jumps between speakers is unwatchable |
| Captions | Timings from the transcript, never estimated | captions estimated from reading speed drift within a minute |
| Runtime | Python 3.12 with ffmpeg | the work is timing arithmetic and video processing |

## Before you start

Have every one of these ready. The plan assumes them from step one.

- [ ] **Python 3.12 or newer** · free
  - Why: Orchestrates transcription, selection and rendering.
  - Get it: python.org or your package manager, then a virtual environment.
  - Verify: python3 --version prints 3.12 or higher
- [ ] **ffmpeg with the codecs you need** · free
  - Why: Cutting, cropping and rendering all run through it, and reimplementing any of it is not the project.
  - Get it: Your package manager, confirming it handles your source format and the vertical output your targets require.
  - Verify: A clip cuts and re-encodes at the target aspect without quality loss beyond expectation
- [ ] **Local speech recognition with word timings** · free, needs machine time
  - Why: Every cut point and every caption derives from word timings, and segment-level timings are not precise enough to cut on.
  - Get it: whisper.cpp or an equivalent producing word-level timings, and check it does before building on it.
  - Verify: A five minute clip produces per-word timings
- [ ] **Face detection and tracking** · free
  - Why: Reframing to vertical requires knowing where the speaker is in every frame, and a static crop is usually wrong.
  - Get it: Any detector with tracking across frames, evaluated on your own footage rather than on a benchmark.
  - Verify: A speaker moving across frame is tracked without losing them
- [ ] **A real long recording with two speakers** · free if you have one
  - Why: Seam losses, speaker changes and framing failures only appear in real material of realistic length.
  - Get it: A two hour recording with two people, including a stretch where they talk over each other and one where the speaker leaves frame.
  - Verify: A two hour two-speaker source is available with both cases present

## Quick start

Follow Phase 1 in `BUILD_PLAN.md`; it creates the project.

Then copy `.env.example` to `.env` and fill in the values it documents.

## Honest limits

This build deliberately does not replace:

- Predicting which clips will perform.
- Publishing to platforms.
- Music, effects and heavy editing.
- Multi-camera switching.
- professional codecs, effects, and collaboration
- production codecs, rendering speed, and media templates
- hosted recording reliability
- large-scale rendering
- distribution and podcast hosting

If one of those is essential to you, that is the reason to keep paying for OpusClip, and the README should say so rather than pretend.

===== BRIEF.md =====
# Build brief · OpusClip

The one-shot brief this plan expands. `BUILD_PLAN.md` (or `MILESTONES.md`) is the same sequence broken into steps and checks; where the two disagree, the plan wins.

Build me short clips cut from a long video, where the cut points come from what was said to replace OpusClip. Build it in phases, in the order below.
Do not write the whole thing in one pass. Finish a phase, run its "Done when"
check, fix what fails, and only then start the next phase.

### Stack (fixed, do not substitute)
- Transcription: Local recognition with word timings · every cut point comes from a word boundary, so the timings are the foundation
- Selection: Candidate passages scored on stated signals, proposed not chosen · what makes a clip work is a judgement and the tool should say so
- Framing: Subject tracking with a smoothed crop path · a crop that jumps between speakers is unwatchable
- Captions: Timings from the transcript, never estimated · captions estimated from reading speed drift within a minute
- Runtime: Python 3.12 with ffmpeg · the work is timing arithmetic and video processing

### Data model (create this before Phase 1)
- sources(id, path, duration_ms, width, height, fps, imported_at)
- words(source_id, ordinal, text, start_ms, end_ms, speaker, confidence)
- candidates(id, source_id, start_ms, end_ms, signals_json, score, rationale, accepted_by)
- clips(id, candidate_id, start_ms, end_ms, aspect, crop_path_json, rendered_path, rendered_at)
- captions(clip_id, ordinal, start_ms, end_ms, text) · derived from words, never estimated
- speakers(source_id, id, label, face_track_id) · because reframing needs to know who is talking
- A candidate stores the signals behind its score rather than only the number, so a user can see why a passage was suggested and disagree with it.

### Phase 1 · Words with times you can cut on
Build: Transcribe locally with word-level timings and speaker labels where available. Chunk long
  recordings with overlap and merge without losing words at the seams. Verify the final
  timestamp against the source duration.
Trap: Chunking a two hour recording without overlap loses words at every boundary, and every cut point after that is derived from a transcript that has drifted from the audio. Overlap, merge on matching text, and check the word count and final timestamp against a single-pass transcription of a shorter excerpt.
Done when: A two hour source transcribes with no words lost or duplicated at chunk seams, verified against a single-pass excerpt. The final word's timestamp matches the source duration within tolerance. Speaker labels are produced where the source supports them and their absence is recorded rather than guessed. Transcription runs with networking disabled.
Do not build yet: No selection, framing, captions or rendering.

### Phase 2 · Candidates proposed, with reasons
Build: Score candidate passages on stated signals such as a complete thought, a question and
  answer pair, or a self-contained explanation. Store the signals alongside the score.
  Present candidates for acceptance rather than producing clips automatically.
Trap: Producing a score with no explanation makes disagreeing with the tool impossible, and the user either accepts everything or ignores the feature. Store which signals fired for each candidate and show them, so a suggestion is an argument the person can reject.
Done when: Every candidate shows the signals that produced its score rather than only a number. Candidates are proposed and require acceptance before rendering. A source with no self-contained passages produces few candidates rather than filling a quota. Candidate boundaries fall on word boundaries and start at the beginning of a sentence.
Do not build yet: No framing, captions or rendering yet.

### Phase 3 · Reframing without decapitating anyone
Build: Track faces and derive a crop path, smoothed so it does not jitter. Handle a cut between
  two speakers, a speaker who leaves the frame, and a shot with nobody in it. Verify the
  head stays within the crop throughout.
Trap: A crop that follows the detected face position frame by frame jitters constantly and is more distracting than a static crop, while one that snaps between two speakers is nauseating. Smooth the path, move only when the subject has genuinely moved, and cut rather than pan when the speaker changes.
Done when: A vertical crop keeps the speaker's head fully in frame throughout, verified by inspecting the output rather than the tracking data. The crop path is smooth rather than jittery. A speaker change produces a cut rather than a pan. A shot with no detected subject falls back to a stated behaviour rather than drifting.
Do not build yet: No captions or rendering yet.

### Phase 4 · Captions from the transcript
Build: Derive caption timings from the transcript's word timings, offset for the clip's start.
  Break lines at natural boundaries within a character limit. Verify synchronisation at the
  end of the clip rather than the start.
Trap: Estimating caption timing from word count and reading speed drifts audibly within the first minute, and checking only the opening seconds always looks fine. Use the word timings you already have, and verify at the end where drift accumulates.
Done when: Captions match the speech at the end of a sixty second clip, verified by watching. Line breaks fall at natural boundaries within a stated character limit. Caption timings are offset correctly for a clip cut from the middle of a source. A word with low recognition confidence is marked rather than presented as certain.
Do not build yet: No rendering or batch yet.

### Phase 5 · Rendering and keeping the source
Build: Render from a declarative clip description so the same inputs produce the same file.
  Open sources read only and checksum them in the suite. Batch render with per-clip failure
  isolation and a manifest.
Trap: Any path that writes to the source file, including a helpfully added normalisation pass, damages an original recording that may be irreplaceable, and the loss is discovered when someone wants to re-clip it. Open read only, checksum before and after, and fail the build if they differ.
Done when: The same clip description renders byte-identically twice. Source checksums are unchanged after a full batch, asserted in the suite. A batch of thirty clips isolates per-clip failures and reports them individually. The manifest records inputs, settings and outputs so a render is reproducible from it.
Do not build yet: Nothing after this.

### Out of scope (and why)
- Predicting which clips will perform well, which no tool can do and which every product in this category implies.
- Publishing to platforms, which has its own rate limits, costs and disclosure requirements.
- Music, effects and heavy editing, which is production rather than clipping.
- Multi-camera switching, which is a substantially harder framing problem than tracking one speaker.

### README must contain
- That every cut point comes from a word timing, so transcription accuracy is the foundation
- That candidates carry the signals behind their score, so a suggestion can be argued with
- That the crop path is smoothed and a speaker change cuts rather than pans
- That captions come from the transcript's timings and are verified at the end of a clip
- That sources are opened read only and checksummed in the test suite

===== AGENTS.md =====
# Agent instructions · OpusClip indie build

- Read `README.md` and `BUILD_PLAN.md` before writing code. The stack is fixed: Local recognition with word timings, Candidate passages scored on stated signals, proposed not chosen, Subject tracking with a smoothed crop path, Timings from the transcript, never estimated, Python 3.12 with ffmpeg. Do not substitute.
- Work one phase at a time, in order. Do not start a phase until every "Done when" item of the previous one passes.
- Prefer the fewest moving parts that satisfy the step. No frameworks, services or dependencies the plan does not name.
- Secrets live in `.env`, never in source or logs. Keep `.env.example` current when a variable is introduced.
- Do not invent cryptography, security guarantees, APIs or compliance claims.
- Add a focused test for every destructive, security-sensitive or data-loss path the plan names.
- Run the project checks before declaring a phase complete, and record any deliberate shortcut in the README under "Tradeoffs".

## Known traps

- Chunking a two hour recording without overlap loses words at every boundary, and every cut point after that is derived from a transcript that has drifted from the audio. Overlap, merge on matching text, and check the word count and final timestamp against a single-pass transcription of a shorter excerpt.
- Producing a score with no explanation makes disagreeing with the tool impossible, and the user either accepts everything or ignores the feature. Store which signals fired for each candidate and show them, so a suggestion is an argument the person can reject.
- A crop that follows the detected face position frame by frame jitters constantly and is more distracting than a static crop, while one that snaps between two speakers is nauseating. Smooth the path, move only when the subject has genuinely moved, and cut rather than pan when the speaker changes.
- Estimating caption timing from word count and reading speed drifts audibly within the first minute, and checking only the opening seconds always looks fine. Use the word timings you already have, and verify at the end where drift accumulates.

===== BUILD_PLAN.md =====
# Build plan · OpusClip

A clipping pipeline where the selection problem is honest and the rendering problem is exact: finding passages that stand alone is a judgement the tool should assist rather than claim, and reframing to vertical without decapitating the speaker is a tracking problem with a measurable answer.

Phases are in dependency order. Each ends in a "Done when" list; treat an unticked item as a blocker, not a note.

## Phase 1 · Words with times you can cut on

A long recording transcribes with timings accurate enough to cut by.

### Steps

1. Chunk with overlap and merge on text
   The seams are where the words go missing and everything downstream drifts.
   Files: `src/transcribe.py`, `tests/test_seams.py`
2. Verify against the source duration
   A transcript ending early means every later cut point is wrong.
   Files: `src/verify.py`
3. Record absent speaker labels as absent
   Guessing who spoke is worse than saying you do not know.
   Files: `src/speakers.py`

### Done when

- [ ] A two hour source transcribes with no seam losses
- [ ] The final timestamp matches the source duration
- [ ] Absent speaker labels are recorded, not guessed
- [ ] Transcription works offline

### Watch out

- Chunking a two hour recording without overlap loses words at every boundary, and every cut point after that is derived from a transcript that has drifted from the audio. Overlap, merge on matching text, and check the word count and final timestamp against a single-pass transcription of a shorter excerpt.

## Phase 2 · Candidates proposed, with reasons

Suggested passages carry the signals behind them.

### Steps

1. Score on stated signals
   A number with no reasons cannot be argued with.
   Files: `src/candidates.py`
2. Require acceptance before rendering
   Automatic clipping produces volume rather than clips.
   Files: `src/review.py`
3. Snap boundaries to sentences
   A clip starting mid-sentence is discarded by whoever watches it.
   Files: `src/boundaries.py`

### Done when

- [ ] Candidates show their signals
- [ ] Rendering requires acceptance
- [ ] A thin source produces few candidates
- [ ] Boundaries fall on sentence starts

### Watch out

- Producing a score with no explanation makes disagreeing with the tool impossible, and the user either accepts everything or ignores the feature. Store which signals fired for each candidate and show them, so a suggestion is an argument the person can reject.

## Phase 3 · Reframing without decapitating anyone

A vertical crop follows the speaker smoothly and keeps their head in frame.

### Steps

1. Smooth the crop path
   Frame-by-frame following is jitter, which is worse than a static crop.
   Files: `src/framing.py`
2. Cut rather than pan on speaker change
   A fast pan between two people is nauseating.
   Files: `src/framing.py`
3. Assert the head stays in frame
   Checked against the output, not against the tracking confidence.
   Files: `tests/test_framing.py`

### Done when

- [ ] The head stays fully in frame throughout
- [ ] The crop path is smooth rather than jittery
- [ ] Speaker changes cut rather than pan
- [ ] No detected subject has a stated fallback

### Watch out

- A crop that follows the detected face position frame by frame jitters constantly and is more distracting than a static crop, while one that snaps between two speakers is nauseating. Smooth the path, move only when the subject has genuinely moved, and cut rather than pan when the speaker changes.

## Phase 4 · Captions from the transcript

Captions match the speech exactly at the end of a clip.

### Steps

1. Use the transcript's word timings
   You already have them. Estimation drifts within a minute.
   Files: `src/captions.py`
2. Break lines at natural boundaries
   A caption breaking mid-phrase is harder to read than a longer line.
   Files: `src/captions.py`
3. Verify at the end of the clip
   The opening seconds always look correct.
   Files: `tests/test_sync.py`

### Done when

- [ ] Captions match speech at the end of a clip
- [ ] Line breaks fall at natural boundaries
- [ ] Offsets are correct for mid-source clips
- [ ] Low confidence words are marked

### Watch out

- Estimating caption timing from word count and reading speed drifts audibly within the first minute, and checking only the opening seconds always looks fine. Use the word timings you already have, and verify at the end where drift accumulates.

## Not in this build

- Predicting which clips will perform.
- Publishing to platforms.
- Music, effects and heavy editing.
- Multi-camera switching.

## After v1, if you want it

- Speaker-aware candidate selection, so a clip does not start mid-answer
- A review interface that plays candidates back to back, which is how the accepting actually happens
- Alternative aspect ratios from one render description, since targets differ

===== .env.example =====
# Copy to .env and fill in. Never commit .env; this file documents it.

# Required. Original recordings, opened read only.
SOURCE_DIR=./sources

# Required. Rendered clips and manifests. Never the source directory.
OUTPUT_DIR=./clips

# Required. Timing quality matters more here than word accuracy.
ASR_MODEL=./models/ggml-medium.en.bin

# Required. The output aspect, which drives the whole framing problem.
TARGET_ASPECT=9:16

# Required. Too little jitters, too much lags behind the speaker.
CROP_SMOOTHING_MS=400

# Required. Present as a constant. Automatic clipping produces volume rather than clips.
AUTO_ACCEPT_CANDIDATES=never

You are building a production product version of OpusClip.

Create the following project files first, then implement the application by following them. Keep the files updated as decisions change. Do not collapse this into a single README or prompt.

===== PRODUCT.md =====
# OpusClip · product brief

## Problem

The visible AI video repurposing loop is buildable, but a credible replacement needs more than the first screen. OpusClip earns its keep through media pipeline, hosting, distribution, so expect a weekend or multi-day build and a narrower personal scope.

## Product outcome

Clips cut on word boundaries from passages you accepted, reframed so nobody loses their head, with captions that stay in sync to the last second and originals that were never touched.

## Target user

A builder who needs a maintainable product foundation, not a one-off demo.

## Required capabilities

- Python 3.12
- FFmpeg
- desktop with sufficient storage
- optional model or API key in .env

## Explicit non-goals for v1

- Predicting which clips will perform.
- Publishing to platforms.
- Music, effects and heavy editing.
- Multi-camera switching.
- professional codecs, effects, and collaboration
- production codecs, rendering speed, and media templates
- hosted recording reliability
- large-scale rendering
- distribution and podcast hosting

## Success criteria

- A two hour source is confirmed to transcribe without seam losses
- Candidates are confirmed to show their signals
- The head is confirmed to stay in frame throughout
- Captions are confirmed in sync at the end of a clip
- Source checksums are confirmed unchanged

===== BRIEF.md =====
# Build brief · OpusClip

The one-shot brief this plan expands. `BUILD_PLAN.md` (or `MILESTONES.md`) is the same sequence broken into steps and checks; where the two disagree, the plan wins.

Build me short clips cut from a long video, where the cut points come from what was said to replace OpusClip. Build it in phases, in the order below.
Do not write the whole thing in one pass. Finish a phase, run its "Done when"
check, fix what fails, and only then start the next phase.

### Stack (fixed, do not substitute)
- Transcription: Local recognition with word timings · every cut point comes from a word boundary, so the timings are the foundation
- Selection: Candidate passages scored on stated signals, proposed not chosen · what makes a clip work is a judgement and the tool should say so
- Framing: Subject tracking with a smoothed crop path · a crop that jumps between speakers is unwatchable
- Captions: Timings from the transcript, never estimated · captions estimated from reading speed drift within a minute
- Runtime: Python 3.12 with ffmpeg · the work is timing arithmetic and video processing

### Data model (create this before Phase 1)
- sources(id, path, duration_ms, width, height, fps, imported_at)
- words(source_id, ordinal, text, start_ms, end_ms, speaker, confidence)
- candidates(id, source_id, start_ms, end_ms, signals_json, score, rationale, accepted_by)
- clips(id, candidate_id, start_ms, end_ms, aspect, crop_path_json, rendered_path, rendered_at)
- captions(clip_id, ordinal, start_ms, end_ms, text) · derived from words, never estimated
- speakers(source_id, id, label, face_track_id) · because reframing needs to know who is talking
- A candidate stores the signals behind its score rather than only the number, so a user can see why a passage was suggested and disagree with it.

### Phase 1 · Words with times you can cut on
Build: Transcribe locally with word-level timings and speaker labels where available. Chunk long
  recordings with overlap and merge without losing words at the seams. Verify the final
  timestamp against the source duration.
Trap: Chunking a two hour recording without overlap loses words at every boundary, and every cut point after that is derived from a transcript that has drifted from the audio. Overlap, merge on matching text, and check the word count and final timestamp against a single-pass transcription of a shorter excerpt.
Done when: A two hour source transcribes with no words lost or duplicated at chunk seams, verified against a single-pass excerpt. The final word's timestamp matches the source duration within tolerance. Speaker labels are produced where the source supports them and their absence is recorded rather than guessed. Transcription runs with networking disabled.
Do not build yet: No selection, framing, captions or rendering.

### Phase 2 · Candidates proposed, with reasons
Build: Score candidate passages on stated signals such as a complete thought, a question and
  answer pair, or a self-contained explanation. Store the signals alongside the score.
  Present candidates for acceptance rather than producing clips automatically.
Trap: Producing a score with no explanation makes disagreeing with the tool impossible, and the user either accepts everything or ignores the feature. Store which signals fired for each candidate and show them, so a suggestion is an argument the person can reject.
Done when: Every candidate shows the signals that produced its score rather than only a number. Candidates are proposed and require acceptance before rendering. A source with no self-contained passages produces few candidates rather than filling a quota. Candidate boundaries fall on word boundaries and start at the beginning of a sentence.
Do not build yet: No framing, captions or rendering yet.

### Phase 3 · Reframing without decapitating anyone
Build: Track faces and derive a crop path, smoothed so it does not jitter. Handle a cut between
  two speakers, a speaker who leaves the frame, and a shot with nobody in it. Verify the
  head stays within the crop throughout.
Trap: A crop that follows the detected face position frame by frame jitters constantly and is more distracting than a static crop, while one that snaps between two speakers is nauseating. Smooth the path, move only when the subject has genuinely moved, and cut rather than pan when the speaker changes.
Done when: A vertical crop keeps the speaker's head fully in frame throughout, verified by inspecting the output rather than the tracking data. The crop path is smooth rather than jittery. A speaker change produces a cut rather than a pan. A shot with no detected subject falls back to a stated behaviour rather than drifting.
Do not build yet: No captions or rendering yet.

### Phase 4 · Captions from the transcript
Build: Derive caption timings from the transcript's word timings, offset for the clip's start.
  Break lines at natural boundaries within a character limit. Verify synchronisation at the
  end of the clip rather than the start.
Trap: Estimating caption timing from word count and reading speed drifts audibly within the first minute, and checking only the opening seconds always looks fine. Use the word timings you already have, and verify at the end where drift accumulates.
Done when: Captions match the speech at the end of a sixty second clip, verified by watching. Line breaks fall at natural boundaries within a stated character limit. Caption timings are offset correctly for a clip cut from the middle of a source. A word with low recognition confidence is marked rather than presented as certain.
Do not build yet: No rendering or batch yet.

### Phase 5 · Rendering and keeping the source
Build: Render from a declarative clip description so the same inputs produce the same file.
  Open sources read only and checksum them in the suite. Batch render with per-clip failure
  isolation and a manifest.
Trap: Any path that writes to the source file, including a helpfully added normalisation pass, damages an original recording that may be irreplaceable, and the loss is discovered when someone wants to re-clip it. Open read only, checksum before and after, and fail the build if they differ.
Done when: The same clip description renders byte-identically twice. Source checksums are unchanged after a full batch, asserted in the suite. A batch of thirty clips isolates per-clip failures and reports them individually. The manifest records inputs, settings and outputs so a render is reproducible from it.
Do not build yet: Nothing after this.

### Out of scope (and why)
- Predicting which clips will perform well, which no tool can do and which every product in this category implies.
- Publishing to platforms, which has its own rate limits, costs and disclosure requirements.
- Music, effects and heavy editing, which is production rather than clipping.
- Multi-camera switching, which is a substantially harder framing problem than tracking one speaker.

### README must contain
- That every cut point comes from a word timing, so transcription accuracy is the foundation
- That candidates carry the signals behind their score, so a suggestion can be argued with
- That the crop path is smoothed and a speaker change cuts rather than pans
- That captions come from the transcript's timings and are verified at the end of a clip
- That sources are opened read only and checksummed in the test suite

===== ARCHITECTURE.md =====
# Architecture · OpusClip

## Stack

| Part | Choice | Why |
| --- | --- | --- |
| Transcription | Local recognition with word timings | every cut point comes from a word boundary, so the timings are the foundation |
| Selection | Candidate passages scored on stated signals, proposed not chosen | what makes a clip work is a judgement and the tool should say so |
| Framing | Subject tracking with a smoothed crop path | a crop that jumps between speakers is unwatchable |
| Captions | Timings from the transcript, never estimated | captions estimated from reading speed drift within a minute |
| Runtime | Python 3.12 with ffmpeg | the work is timing arithmetic and video processing |

## Modules

Each module has one owner concern and a documented way to replace it.

| Module | Owns | How to replace it |
| --- | --- | --- |
| Transcription | Word timings with verified seams. | Any engine producing word-level timings. Segment timings are not sufficient. |
| Selection | Scored candidates with visible signals. | Any scoring, provided the signals are stored and shown. |
| Framing | Smoothed crop paths and speaker cuts. | Any tracker, provided the output is checked rather than the confidence. |
| Captions | Transcript-derived timings. | Any renderer, provided timings are never estimated. |
| Render | Reproducible output from read-only sources. | Any pipeline, provided source checksums are asserted. |

## Configuration

Every runtime setting is an environment variable documented in `.env.example`, validated at startup, with a safe local default wherever one exists.

- `SOURCE_DIR` · required · Original recordings, opened read only.
- `OUTPUT_DIR` · required · Rendered clips and manifests. Never the source directory.
- `ASR_MODEL` · required · Timing quality matters more here than word accuracy.
- `TARGET_ASPECT` · required · The output aspect, which drives the whole framing problem.
- `CROP_SMOOTHING_MS` · required · Too little jitters, too much lags behind the speaker.
- `AUTO_ACCEPT_CANDIDATES` · required · Present as a constant. Automatic clipping produces volume rather than clips.

## Production baseline

- Security: least privilege, input validation at every boundary, secret redaction in logs, rate limits on abuse-prone paths, no invented security primitives.
- Data: explicit schema and migrations, transactional writes where integrity matters, backup and restore procedures that have been exercised.
- Integrations: adapters around third-party providers, idempotent webhook or job processing, bounded retries, timeouts.
- Observability: structured logs with request or operation ids, an error-tracking hook, and health and readiness checks where a server exists.
- Quality: unit tests for domain rules, integration tests at module boundaries, one end-to-end test of the critical path.

## Decision records

For each dependency in the stack table, keep a short note: why it was chosen, its failure mode, and how it is replaced. Do not add infrastructure until a requirement in `PRODUCT.md` justifies it.

===== AGENTS.md =====
# Agent instructions · OpusClip product build

- Read `PRODUCT.md` and `ARCHITECTURE.md` before changing code. The stack is fixed: Local recognition with word timings, Candidate passages scored on stated signals, proposed not chosen, Subject tracking with a smoothed crop path, Timings from the transcript, never estimated, Python 3.12 with ffmpeg.
- Implement milestone by milestone from `MILESTONES.md`; keep each change reviewable and leave the application runnable at every commit.
- Treat authentication, payments, encryption, imports, webhooks and destructive actions as high-risk boundaries when present.
- Never invent cryptography or silently weaken a requirement to make a check pass.
- Put every external service behind an interface with a deterministic fake for tests.
- Add migrations and rollback or recovery notes for every persistent data change.
- Log useful operational context without credentials, tokens, passwords or personal data.
- Update documentation and run every check before completing a milestone.

## Known traps

- Chunking a two hour recording without overlap loses words at every boundary, and every cut point after that is derived from a transcript that has drifted from the audio. Overlap, merge on matching text, and check the word count and final timestamp against a single-pass transcription of a shorter excerpt.
- Producing a score with no explanation makes disagreeing with the tool impossible, and the user either accepts everything or ignores the feature. Store which signals fired for each candidate and show them, so a suggestion is an argument the person can reject.
- A crop that follows the detected face position frame by frame jitters constantly and is more distracting than a static crop, while one that snaps between two speakers is nauseating. Smooth the path, move only when the subject has genuinely moved, and cut rather than pan when the speaker changes.
- Estimating caption timing from word count and reading speed drifts audibly within the first minute, and checking only the opening seconds always looks fine. Use the word timings you already have, and verify at the end where drift accumulates.
- Any path that writes to the source file, including a helpfully added normalisation pass, damages an original recording that may be irreplaceable, and the loss is discovered when someone wants to re-clip it. Open read only, checksum before and after, and fail the build if they differ.

===== MILESTONES.md =====
# Delivery milestones · OpusClip

Estimated effort: **multi-day** for the indie phases; the production-only milestones add the trust and operability layer.

## M1 · Words with times you can cut on

A long recording transcribes with timings accurate enough to cut by.

### Steps

1. Chunk with overlap and merge on text
   The seams are where the words go missing and everything downstream drifts.
   Files: `src/transcribe.py`, `tests/test_seams.py`
2. Verify against the source duration
   A transcript ending early means every later cut point is wrong.
   Files: `src/verify.py`
3. Record absent speaker labels as absent
   Guessing who spoke is worse than saying you do not know.
   Files: `src/speakers.py`

### Done when

- [ ] A two hour source transcribes with no seam losses
- [ ] The final timestamp matches the source duration
- [ ] Absent speaker labels are recorded, not guessed
- [ ] Transcription works offline

### Watch out

- Chunking a two hour recording without overlap loses words at every boundary, and every cut point after that is derived from a transcript that has drifted from the audio. Overlap, merge on matching text, and check the word count and final timestamp against a single-pass transcription of a shorter excerpt.

## M2 · Candidates proposed, with reasons

Suggested passages carry the signals behind them.

### Steps

1. Score on stated signals
   A number with no reasons cannot be argued with.
   Files: `src/candidates.py`
2. Require acceptance before rendering
   Automatic clipping produces volume rather than clips.
   Files: `src/review.py`
3. Snap boundaries to sentences
   A clip starting mid-sentence is discarded by whoever watches it.
   Files: `src/boundaries.py`

### Done when

- [ ] Candidates show their signals
- [ ] Rendering requires acceptance
- [ ] A thin source produces few candidates
- [ ] Boundaries fall on sentence starts

### Watch out

- Producing a score with no explanation makes disagreeing with the tool impossible, and the user either accepts everything or ignores the feature. Store which signals fired for each candidate and show them, so a suggestion is an argument the person can reject.

## M3 · Reframing without decapitating anyone

A vertical crop follows the speaker smoothly and keeps their head in frame.

### Steps

1. Smooth the crop path
   Frame-by-frame following is jitter, which is worse than a static crop.
   Files: `src/framing.py`
2. Cut rather than pan on speaker change
   A fast pan between two people is nauseating.
   Files: `src/framing.py`
3. Assert the head stays in frame
   Checked against the output, not against the tracking confidence.
   Files: `tests/test_framing.py`

### Done when

- [ ] The head stays fully in frame throughout
- [ ] The crop path is smooth rather than jittery
- [ ] Speaker changes cut rather than pan
- [ ] No detected subject has a stated fallback

### Watch out

- A crop that follows the detected face position frame by frame jitters constantly and is more distracting than a static crop, while one that snaps between two speakers is nauseating. Smooth the path, move only when the subject has genuinely moved, and cut rather than pan when the speaker changes.

## M4 · Captions from the transcript

Captions match the speech exactly at the end of a clip.

### Steps

1. Use the transcript's word timings
   You already have them. Estimation drifts within a minute.
   Files: `src/captions.py`
2. Break lines at natural boundaries
   A caption breaking mid-phrase is harder to read than a longer line.
   Files: `src/captions.py`
3. Verify at the end of the clip
   The opening seconds always look correct.
   Files: `tests/test_sync.py`

### Done when

- [ ] Captions match speech at the end of a clip
- [ ] Line breaks fall at natural boundaries
- [ ] Offsets are correct for mid-source clips
- [ ] Low confidence words are marked

### Watch out

- Estimating caption timing from word count and reading speed drifts audibly within the first minute, and checking only the opening seconds always looks fine. Use the word timings you already have, and verify at the end where drift accumulates.

## M5 · Rendering and keeping the source (production only)

Clips render reproducibly and the original is untouched.

### Steps

1. Render from a declarative description
   Reproducible, and reviewable as text before it runs.
   Files: `src/render.py`
2. Checksum sources in the suite
   A write to an original is an irreversible loss.
   Files: `tests/test_sources_untouched.py`
3. Isolate per-clip failures
   One bad clip must not end a thirty clip batch.
   Files: `src/batch.py`

### Done when

- [ ] The same description renders identically twice
- [ ] Source checksums are unchanged after a batch
- [ ] Per-clip failures are isolated and reported
- [ ] The manifest makes a render reproducible

### Watch out

- Any path that writes to the source file, including a helpfully added normalisation pass, damages an original recording that may be irreplaceable, and the loss is discovered when someone wants to re-clip it. Open read only, checksum before and after, and fail the build if they differ.

===== OPERATIONS.md =====
# Operations · OpusClip

## Backup

Source recordings are irreplaceable and large. Clips regenerate from sources plus their descriptions, so back up sources and descriptions and treat rendered clips as disposable.

## Restore

Restore sources and descriptions, re-render and compare against a kept clip. Byte equality proves the pipeline is deterministic and the restore complete.

Do a restore drill before the first real user, and write the date here when it passes.

## Monitoring

For a local pipeline the checks live in the suite: seam integrity, source checksums and end-of-clip caption sync. All three fail silently in ways that look like a quality problem rather than a bug.

## Incident checklist

A published clip where the speaker's head is cropped off is the visible failure, and it means the framing was verified against tracking confidence rather than against the rendered output.

1. Contain the issue without destroying evidence or user data.
2. Record the timeline and affected scope.
3. Rotate exposed secrets and revoke compromised sessions or credentials.
4. Restore from a verified backup when needed.
5. Document the root cause, the remediation and the regression test.

## Release gate

- [ ] A two hour source is confirmed to transcribe without seam losses
- [ ] Candidates are confirmed to show their signals
- [ ] The head is confirmed to stay in frame throughout
- [ ] Captions are confirmed in sync at the end of a clip
- [ ] Source checksums are confirmed unchanged

## Launch constraint

Do not market omitted OpusClip capabilities as implemented. The non-goals in `PRODUCT.md` remain user-visible limitations until they are deliberately delivered.

===== .env.example =====
# Copy to .env and fill in. Never commit .env; this file documents it.

# Required. Original recordings, opened read only.
SOURCE_DIR=./sources

# Required. Rendered clips and manifests. Never the source directory.
OUTPUT_DIR=./clips

# Required. Timing quality matters more here than word accuracy.
ASR_MODEL=./models/ggml-medium.en.bin

# Required. The output aspect, which drives the whole framing problem.
TARGET_ASPECT=9:16

# Required. Too little jitters, too much lags behind the speaker.
CROP_SMOOTHING_MS=400

# Required. Present as a constant. Automatic clipping produces volume rather than clips.
AUTO_ACCEPT_CANDIDATES=never

# OpusClip · indie build

A clipping pipeline where the selection problem is honest and the rendering problem is exact: finding passages that stand alone is a judgement the tool should assist rather than claim, and reframing to vertical without decapitating the speaker is a tracking problem with a measurable answer.

Estimated effort: **multi-day**. Work `BUILD_PLAN.md` top to bottom · every phase ends in a check that has to pass before the next one starts.

## Stack

| Part | Choice | Why |
| --- | --- | --- |
| Transcription | Local recognition with word timings | every cut point comes from a word boundary, so the timings are the foundation |
| Selection | Candidate passages scored on stated signals, proposed not chosen | what makes a clip work is a judgement and the tool should say so |
| Framing | Subject tracking with a smoothed crop path | a crop that jumps between speakers is unwatchable |
| Captions | Timings from the transcript, never estimated | captions estimated from reading speed drift within a minute |
| Runtime | Python 3.12 with ffmpeg | the work is timing arithmetic and video processing |

## Before you start

Have every one of these ready. The plan assumes them from step one.

- [ ] **Python 3.12 or newer** · free
  - Why: Orchestrates transcription, selection and rendering.
  - Get it: python.org or your package manager, then a virtual environment.
  - Verify: python3 --version prints 3.12 or higher
- [ ] **ffmpeg with the codecs you need** · free
  - Why: Cutting, cropping and rendering all run through it, and reimplementing any of it is not the project.
  - Get it: Your package manager, confirming it handles your source format and the vertical output your targets require.
  - Verify: A clip cuts and re-encodes at the target aspect without quality loss beyond expectation
- [ ] **Local speech recognition with word timings** · free, needs machine time
  - Why: Every cut point and every caption derives from word timings, and segment-level timings are not precise enough to cut on.
  - Get it: whisper.cpp or an equivalent producing word-level timings, and check it does before building on it.
  - Verify: A five minute clip produces per-word timings
- [ ] **Face detection and tracking** · free
  - Why: Reframing to vertical requires knowing where the speaker is in every frame, and a static crop is usually wrong.
  - Get it: Any detector with tracking across frames, evaluated on your own footage rather than on a benchmark.
  - Verify: A speaker moving across frame is tracked without losing them
- [ ] **A real long recording with two speakers** · free if you have one
  - Why: Seam losses, speaker changes and framing failures only appear in real material of realistic length.
  - Get it: A two hour recording with two people, including a stretch where they talk over each other and one where the speaker leaves frame.
  - Verify: A two hour two-speaker source is available with both cases present

## Quick start

Follow Phase 1 in `BUILD_PLAN.md`; it creates the project.

Then copy `.env.example` to `.env` and fill in the values it documents.

## Honest limits

This build deliberately does not replace:

- Predicting which clips will perform.
- Publishing to platforms.
- Music, effects and heavy editing.
- Multi-camera switching.
- professional codecs, effects, and collaboration
- production codecs, rendering speed, and media templates
- hosted recording reliability
- large-scale rendering
- distribution and podcast hosting

If one of those is essential to you, that is the reason to keep paying for OpusClip, and the README should say so rather than pretend.

# Build brief · OpusClip

The one-shot brief this plan expands. `BUILD_PLAN.md` (or `MILESTONES.md`) is the same sequence broken into steps and checks; where the two disagree, the plan wins.

Build me short clips cut from a long video, where the cut points come from what was said to replace OpusClip. Build it in phases, in the order below.
Do not write the whole thing in one pass. Finish a phase, run its "Done when"
check, fix what fails, and only then start the next phase.

### Stack (fixed, do not substitute)
- Transcription: Local recognition with word timings · every cut point comes from a word boundary, so the timings are the foundation
- Selection: Candidate passages scored on stated signals, proposed not chosen · what makes a clip work is a judgement and the tool should say so
- Framing: Subject tracking with a smoothed crop path · a crop that jumps between speakers is unwatchable
- Captions: Timings from the transcript, never estimated · captions estimated from reading speed drift within a minute
- Runtime: Python 3.12 with ffmpeg · the work is timing arithmetic and video processing

### Data model (create this before Phase 1)
- sources(id, path, duration_ms, width, height, fps, imported_at)
- words(source_id, ordinal, text, start_ms, end_ms, speaker, confidence)
- candidates(id, source_id, start_ms, end_ms, signals_json, score, rationale, accepted_by)
- clips(id, candidate_id, start_ms, end_ms, aspect, crop_path_json, rendered_path, rendered_at)
- captions(clip_id, ordinal, start_ms, end_ms, text) · derived from words, never estimated
- speakers(source_id, id, label, face_track_id) · because reframing needs to know who is talking
- A candidate stores the signals behind its score rather than only the number, so a user can see why a passage was suggested and disagree with it.

### Phase 1 · Words with times you can cut on
Build: Transcribe locally with word-level timings and speaker labels where available. Chunk long
  recordings with overlap and merge without losing words at the seams. Verify the final
  timestamp against the source duration.
Trap: Chunking a two hour recording without overlap loses words at every boundary, and every cut point after that is derived from a transcript that has drifted from the audio. Overlap, merge on matching text, and check the word count and final timestamp against a single-pass transcription of a shorter excerpt.
Done when: A two hour source transcribes with no words lost or duplicated at chunk seams, verified against a single-pass excerpt. The final word's timestamp matches the source duration within tolerance. Speaker labels are produced where the source supports them and their absence is recorded rather than guessed. Transcription runs with networking disabled.
Do not build yet: No selection, framing, captions or rendering.

### Phase 2 · Candidates proposed, with reasons
Build: Score candidate passages on stated signals such as a complete thought, a question and
  answer pair, or a self-contained explanation. Store the signals alongside the score.
  Present candidates for acceptance rather than producing clips automatically.
Trap: Producing a score with no explanation makes disagreeing with the tool impossible, and the user either accepts everything or ignores the feature. Store which signals fired for each candidate and show them, so a suggestion is an argument the person can reject.
Done when: Every candidate shows the signals that produced its score rather than only a number. Candidates are proposed and require acceptance before rendering. A source with no self-contained passages produces few candidates rather than filling a quota. Candidate boundaries fall on word boundaries and start at the beginning of a sentence.
Do not build yet: No framing, captions or rendering yet.

### Phase 3 · Reframing without decapitating anyone
Build: Track faces and derive a crop path, smoothed so it does not jitter. Handle a cut between
  two speakers, a speaker who leaves the frame, and a shot with nobody in it. Verify the
  head stays within the crop throughout.
Trap: A crop that follows the detected face position frame by frame jitters constantly and is more distracting than a static crop, while one that snaps between two speakers is nauseating. Smooth the path, move only when the subject has genuinely moved, and cut rather than pan when the speaker changes.
Done when: A vertical crop keeps the speaker's head fully in frame throughout, verified by inspecting the output rather than the tracking data. The crop path is smooth rather than jittery. A speaker change produces a cut rather than a pan. A shot with no detected subject falls back to a stated behaviour rather than drifting.
Do not build yet: No captions or rendering yet.

### Phase 4 · Captions from the transcript
Build: Derive caption timings from the transcript's word timings, offset for the clip's start.
  Break lines at natural boundaries within a character limit. Verify synchronisation at the
  end of the clip rather than the start.
Trap: Estimating caption timing from word count and reading speed drifts audibly within the first minute, and checking only the opening seconds always looks fine. Use the word timings you already have, and verify at the end where drift accumulates.
Done when: Captions match the speech at the end of a sixty second clip, verified by watching. Line breaks fall at natural boundaries within a stated character limit. Caption timings are offset correctly for a clip cut from the middle of a source. A word with low recognition confidence is marked rather than presented as certain.
Do not build yet: No rendering or batch yet.

### Phase 5 · Rendering and keeping the source
Build: Render from a declarative clip description so the same inputs produce the same file.
  Open sources read only and checksum them in the suite. Batch render with per-clip failure
  isolation and a manifest.
Trap: Any path that writes to the source file, including a helpfully added normalisation pass, damages an original recording that may be irreplaceable, and the loss is discovered when someone wants to re-clip it. Open read only, checksum before and after, and fail the build if they differ.
Done when: The same clip description renders byte-identically twice. Source checksums are unchanged after a full batch, asserted in the suite. A batch of thirty clips isolates per-clip failures and reports them individually. The manifest records inputs, settings and outputs so a render is reproducible from it.
Do not build yet: Nothing after this.

### Out of scope (and why)
- Predicting which clips will perform well, which no tool can do and which every product in this category implies.
- Publishing to platforms, which has its own rate limits, costs and disclosure requirements.
- Music, effects and heavy editing, which is production rather than clipping.
- Multi-camera switching, which is a substantially harder framing problem than tracking one speaker.

### README must contain
- That every cut point comes from a word timing, so transcription accuracy is the foundation
- That candidates carry the signals behind their score, so a suggestion can be argued with
- That the crop path is smoothed and a speaker change cuts rather than pans
- That captions come from the transcript's timings and are verified at the end of a clip
- That sources are opened read only and checksummed in the test suite

# Agent instructions · OpusClip indie build

- Read `README.md` and `BUILD_PLAN.md` before writing code. The stack is fixed: Local recognition with word timings, Candidate passages scored on stated signals, proposed not chosen, Subject tracking with a smoothed crop path, Timings from the transcript, never estimated, Python 3.12 with ffmpeg. Do not substitute.
- Work one phase at a time, in order. Do not start a phase until every "Done when" item of the previous one passes.
- Prefer the fewest moving parts that satisfy the step. No frameworks, services or dependencies the plan does not name.
- Secrets live in `.env`, never in source or logs. Keep `.env.example` current when a variable is introduced.
- Do not invent cryptography, security guarantees, APIs or compliance claims.
- Add a focused test for every destructive, security-sensitive or data-loss path the plan names.
- Run the project checks before declaring a phase complete, and record any deliberate shortcut in the README under "Tradeoffs".

## Known traps

- Chunking a two hour recording without overlap loses words at every boundary, and every cut point after that is derived from a transcript that has drifted from the audio. Overlap, merge on matching text, and check the word count and final timestamp against a single-pass transcription of a shorter excerpt.
- Producing a score with no explanation makes disagreeing with the tool impossible, and the user either accepts everything or ignores the feature. Store which signals fired for each candidate and show them, so a suggestion is an argument the person can reject.
- A crop that follows the detected face position frame by frame jitters constantly and is more distracting than a static crop, while one that snaps between two speakers is nauseating. Smooth the path, move only when the subject has genuinely moved, and cut rather than pan when the speaker changes.
- Estimating caption timing from word count and reading speed drifts audibly within the first minute, and checking only the opening seconds always looks fine. Use the word timings you already have, and verify at the end where drift accumulates.

# Build plan · OpusClip

A clipping pipeline where the selection problem is honest and the rendering problem is exact: finding passages that stand alone is a judgement the tool should assist rather than claim, and reframing to vertical without decapitating the speaker is a tracking problem with a measurable answer.

Phases are in dependency order. Each ends in a "Done when" list; treat an unticked item as a blocker, not a note.

## Phase 1 · Words with times you can cut on

A long recording transcribes with timings accurate enough to cut by.

### Steps

1. Chunk with overlap and merge on text
   The seams are where the words go missing and everything downstream drifts.
   Files: `src/transcribe.py`, `tests/test_seams.py`
2. Verify against the source duration
   A transcript ending early means every later cut point is wrong.
   Files: `src/verify.py`
3. Record absent speaker labels as absent
   Guessing who spoke is worse than saying you do not know.
   Files: `src/speakers.py`

### Done when

- [ ] A two hour source transcribes with no seam losses
- [ ] The final timestamp matches the source duration
- [ ] Absent speaker labels are recorded, not guessed
- [ ] Transcription works offline

### Watch out

- Chunking a two hour recording without overlap loses words at every boundary, and every cut point after that is derived from a transcript that has drifted from the audio. Overlap, merge on matching text, and check the word count and final timestamp against a single-pass transcription of a shorter excerpt.

## Phase 2 · Candidates proposed, with reasons

Suggested passages carry the signals behind them.

### Steps

1. Score on stated signals
   A number with no reasons cannot be argued with.
   Files: `src/candidates.py`
2. Require acceptance before rendering
   Automatic clipping produces volume rather than clips.
   Files: `src/review.py`
3. Snap boundaries to sentences
   A clip starting mid-sentence is discarded by whoever watches it.
   Files: `src/boundaries.py`

### Done when

- [ ] Candidates show their signals
- [ ] Rendering requires acceptance
- [ ] A thin source produces few candidates
- [ ] Boundaries fall on sentence starts

### Watch out

- Producing a score with no explanation makes disagreeing with the tool impossible, and the user either accepts everything or ignores the feature. Store which signals fired for each candidate and show them, so a suggestion is an argument the person can reject.

## Phase 3 · Reframing without decapitating anyone

A vertical crop follows the speaker smoothly and keeps their head in frame.

### Steps

1. Smooth the crop path
   Frame-by-frame following is jitter, which is worse than a static crop.
   Files: `src/framing.py`
2. Cut rather than pan on speaker change
   A fast pan between two people is nauseating.
   Files: `src/framing.py`
3. Assert the head stays in frame
   Checked against the output, not against the tracking confidence.
   Files: `tests/test_framing.py`

### Done when

- [ ] The head stays fully in frame throughout
- [ ] The crop path is smooth rather than jittery
- [ ] Speaker changes cut rather than pan
- [ ] No detected subject has a stated fallback

### Watch out

- A crop that follows the detected face position frame by frame jitters constantly and is more distracting than a static crop, while one that snaps between two speakers is nauseating. Smooth the path, move only when the subject has genuinely moved, and cut rather than pan when the speaker changes.

## Phase 4 · Captions from the transcript

Captions match the speech exactly at the end of a clip.

### Steps

1. Use the transcript's word timings
   You already have them. Estimation drifts within a minute.
   Files: `src/captions.py`
2. Break lines at natural boundaries
   A caption breaking mid-phrase is harder to read than a longer line.
   Files: `src/captions.py`
3. Verify at the end of the clip
   The opening seconds always look correct.
   Files: `tests/test_sync.py`

### Done when

- [ ] Captions match speech at the end of a clip
- [ ] Line breaks fall at natural boundaries
- [ ] Offsets are correct for mid-source clips
- [ ] Low confidence words are marked

### Watch out

- Estimating caption timing from word count and reading speed drifts audibly within the first minute, and checking only the opening seconds always looks fine. Use the word timings you already have, and verify at the end where drift accumulates.

## Not in this build

- Predicting which clips will perform.
- Publishing to platforms.
- Music, effects and heavy editing.
- Multi-camera switching.

## After v1, if you want it

- Speaker-aware candidate selection, so a clip does not start mid-answer
- A review interface that plays candidates back to back, which is how the accepting actually happens
- Alternative aspect ratios from one render description, since targets differ

# Copy to .env and fill in. Never commit .env; this file documents it.

# Required. Original recordings, opened read only.
SOURCE_DIR=./sources

# Required. Rendered clips and manifests. Never the source directory.
OUTPUT_DIR=./clips

# Required. Timing quality matters more here than word accuracy.
ASR_MODEL=./models/ggml-medium.en.bin

# Required. The output aspect, which drives the whole framing problem.
TARGET_ASPECT=9:16

# Required. Too little jitters, too much lags behind the speaker.
CROP_SMOOTHING_MS=400

# Required. Present as a constant. Automatic clipping produces volume rather than clips.
AUTO_ACCEPT_CANDIDATES=never

# OpusClip · product brief

## Problem

The visible AI video repurposing loop is buildable, but a credible replacement needs more than the first screen. OpusClip earns its keep through media pipeline, hosting, distribution, so expect a weekend or multi-day build and a narrower personal scope.

## Product outcome

Clips cut on word boundaries from passages you accepted, reframed so nobody loses their head, with captions that stay in sync to the last second and originals that were never touched.

## Target user

A builder who needs a maintainable product foundation, not a one-off demo.

## Required capabilities

- Python 3.12
- FFmpeg
- desktop with sufficient storage
- optional model or API key in .env

## Explicit non-goals for v1

- Predicting which clips will perform.
- Publishing to platforms.
- Music, effects and heavy editing.
- Multi-camera switching.
- professional codecs, effects, and collaboration
- production codecs, rendering speed, and media templates
- hosted recording reliability
- large-scale rendering
- distribution and podcast hosting

## Success criteria

- A two hour source is confirmed to transcribe without seam losses
- Candidates are confirmed to show their signals
- The head is confirmed to stay in frame throughout
- Captions are confirmed in sync at the end of a clip
- Source checksums are confirmed unchanged

# Build brief · OpusClip

The one-shot brief this plan expands. `BUILD_PLAN.md` (or `MILESTONES.md`) is the same sequence broken into steps and checks; where the two disagree, the plan wins.

Build me short clips cut from a long video, where the cut points come from what was said to replace OpusClip. Build it in phases, in the order below.
Do not write the whole thing in one pass. Finish a phase, run its "Done when"
check, fix what fails, and only then start the next phase.

### Stack (fixed, do not substitute)
- Transcription: Local recognition with word timings · every cut point comes from a word boundary, so the timings are the foundation
- Selection: Candidate passages scored on stated signals, proposed not chosen · what makes a clip work is a judgement and the tool should say so
- Framing: Subject tracking with a smoothed crop path · a crop that jumps between speakers is unwatchable
- Captions: Timings from the transcript, never estimated · captions estimated from reading speed drift within a minute
- Runtime: Python 3.12 with ffmpeg · the work is timing arithmetic and video processing

### Data model (create this before Phase 1)
- sources(id, path, duration_ms, width, height, fps, imported_at)
- words(source_id, ordinal, text, start_ms, end_ms, speaker, confidence)
- candidates(id, source_id, start_ms, end_ms, signals_json, score, rationale, accepted_by)
- clips(id, candidate_id, start_ms, end_ms, aspect, crop_path_json, rendered_path, rendered_at)
- captions(clip_id, ordinal, start_ms, end_ms, text) · derived from words, never estimated
- speakers(source_id, id, label, face_track_id) · because reframing needs to know who is talking
- A candidate stores the signals behind its score rather than only the number, so a user can see why a passage was suggested and disagree with it.

### Phase 1 · Words with times you can cut on
Build: Transcribe locally with word-level timings and speaker labels where available. Chunk long
  recordings with overlap and merge without losing words at the seams. Verify the final
  timestamp against the source duration.
Trap: Chunking a two hour recording without overlap loses words at every boundary, and every cut point after that is derived from a transcript that has drifted from the audio. Overlap, merge on matching text, and check the word count and final timestamp against a single-pass transcription of a shorter excerpt.
Done when: A two hour source transcribes with no words lost or duplicated at chunk seams, verified against a single-pass excerpt. The final word's timestamp matches the source duration within tolerance. Speaker labels are produced where the source supports them and their absence is recorded rather than guessed. Transcription runs with networking disabled.
Do not build yet: No selection, framing, captions or rendering.

### Phase 2 · Candidates proposed, with reasons
Build: Score candidate passages on stated signals such as a complete thought, a question and
  answer pair, or a self-contained explanation. Store the signals alongside the score.
  Present candidates for acceptance rather than producing clips automatically.
Trap: Producing a score with no explanation makes disagreeing with the tool impossible, and the user either accepts everything or ignores the feature. Store which signals fired for each candidate and show them, so a suggestion is an argument the person can reject.
Done when: Every candidate shows the signals that produced its score rather than only a number. Candidates are proposed and require acceptance before rendering. A source with no self-contained passages produces few candidates rather than filling a quota. Candidate boundaries fall on word boundaries and start at the beginning of a sentence.
Do not build yet: No framing, captions or rendering yet.

### Phase 3 · Reframing without decapitating anyone
Build: Track faces and derive a crop path, smoothed so it does not jitter. Handle a cut between
  two speakers, a speaker who leaves the frame, and a shot with nobody in it. Verify the
  head stays within the crop throughout.
Trap: A crop that follows the detected face position frame by frame jitters constantly and is more distracting than a static crop, while one that snaps between two speakers is nauseating. Smooth the path, move only when the subject has genuinely moved, and cut rather than pan when the speaker changes.
Done when: A vertical crop keeps the speaker's head fully in frame throughout, verified by inspecting the output rather than the tracking data. The crop path is smooth rather than jittery. A speaker change produces a cut rather than a pan. A shot with no detected subject falls back to a stated behaviour rather than drifting.
Do not build yet: No captions or rendering yet.

### Phase 4 · Captions from the transcript
Build: Derive caption timings from the transcript's word timings, offset for the clip's start.
  Break lines at natural boundaries within a character limit. Verify synchronisation at the
  end of the clip rather than the start.
Trap: Estimating caption timing from word count and reading speed drifts audibly within the first minute, and checking only the opening seconds always looks fine. Use the word timings you already have, and verify at the end where drift accumulates.
Done when: Captions match the speech at the end of a sixty second clip, verified by watching. Line breaks fall at natural boundaries within a stated character limit. Caption timings are offset correctly for a clip cut from the middle of a source. A word with low recognition confidence is marked rather than presented as certain.
Do not build yet: No rendering or batch yet.

### Phase 5 · Rendering and keeping the source
Build: Render from a declarative clip description so the same inputs produce the same file.
  Open sources read only and checksum them in the suite. Batch render with per-clip failure
  isolation and a manifest.
Trap: Any path that writes to the source file, including a helpfully added normalisation pass, damages an original recording that may be irreplaceable, and the loss is discovered when someone wants to re-clip it. Open read only, checksum before and after, and fail the build if they differ.
Done when: The same clip description renders byte-identically twice. Source checksums are unchanged after a full batch, asserted in the suite. A batch of thirty clips isolates per-clip failures and reports them individually. The manifest records inputs, settings and outputs so a render is reproducible from it.
Do not build yet: Nothing after this.

### Out of scope (and why)
- Predicting which clips will perform well, which no tool can do and which every product in this category implies.
- Publishing to platforms, which has its own rate limits, costs and disclosure requirements.
- Music, effects and heavy editing, which is production rather than clipping.
- Multi-camera switching, which is a substantially harder framing problem than tracking one speaker.

### README must contain
- That every cut point comes from a word timing, so transcription accuracy is the foundation
- That candidates carry the signals behind their score, so a suggestion can be argued with
- That the crop path is smoothed and a speaker change cuts rather than pans
- That captions come from the transcript's timings and are verified at the end of a clip
- That sources are opened read only and checksummed in the test suite

# Architecture · OpusClip

## Stack

| Part | Choice | Why |
| --- | --- | --- |
| Transcription | Local recognition with word timings | every cut point comes from a word boundary, so the timings are the foundation |
| Selection | Candidate passages scored on stated signals, proposed not chosen | what makes a clip work is a judgement and the tool should say so |
| Framing | Subject tracking with a smoothed crop path | a crop that jumps between speakers is unwatchable |
| Captions | Timings from the transcript, never estimated | captions estimated from reading speed drift within a minute |
| Runtime | Python 3.12 with ffmpeg | the work is timing arithmetic and video processing |

## Modules

Each module has one owner concern and a documented way to replace it.

| Module | Owns | How to replace it |
| --- | --- | --- |
| Transcription | Word timings with verified seams. | Any engine producing word-level timings. Segment timings are not sufficient. |
| Selection | Scored candidates with visible signals. | Any scoring, provided the signals are stored and shown. |
| Framing | Smoothed crop paths and speaker cuts. | Any tracker, provided the output is checked rather than the confidence. |
| Captions | Transcript-derived timings. | Any renderer, provided timings are never estimated. |
| Render | Reproducible output from read-only sources. | Any pipeline, provided source checksums are asserted. |

## Configuration

Every runtime setting is an environment variable documented in `.env.example`, validated at startup, with a safe local default wherever one exists.

- `SOURCE_DIR` · required · Original recordings, opened read only.
- `OUTPUT_DIR` · required · Rendered clips and manifests. Never the source directory.
- `ASR_MODEL` · required · Timing quality matters more here than word accuracy.
- `TARGET_ASPECT` · required · The output aspect, which drives the whole framing problem.
- `CROP_SMOOTHING_MS` · required · Too little jitters, too much lags behind the speaker.
- `AUTO_ACCEPT_CANDIDATES` · required · Present as a constant. Automatic clipping produces volume rather than clips.

## Production baseline

- Security: least privilege, input validation at every boundary, secret redaction in logs, rate limits on abuse-prone paths, no invented security primitives.
- Data: explicit schema and migrations, transactional writes where integrity matters, backup and restore procedures that have been exercised.
- Integrations: adapters around third-party providers, idempotent webhook or job processing, bounded retries, timeouts.
- Observability: structured logs with request or operation ids, an error-tracking hook, and health and readiness checks where a server exists.
- Quality: unit tests for domain rules, integration tests at module boundaries, one end-to-end test of the critical path.

## Decision records

For each dependency in the stack table, keep a short note: why it was chosen, its failure mode, and how it is replaced. Do not add infrastructure until a requirement in `PRODUCT.md` justifies it.

# Agent instructions · OpusClip product build

- Read `PRODUCT.md` and `ARCHITECTURE.md` before changing code. The stack is fixed: Local recognition with word timings, Candidate passages scored on stated signals, proposed not chosen, Subject tracking with a smoothed crop path, Timings from the transcript, never estimated, Python 3.12 with ffmpeg.
- Implement milestone by milestone from `MILESTONES.md`; keep each change reviewable and leave the application runnable at every commit.
- Treat authentication, payments, encryption, imports, webhooks and destructive actions as high-risk boundaries when present.
- Never invent cryptography or silently weaken a requirement to make a check pass.
- Put every external service behind an interface with a deterministic fake for tests.
- Add migrations and rollback or recovery notes for every persistent data change.
- Log useful operational context without credentials, tokens, passwords or personal data.
- Update documentation and run every check before completing a milestone.

## Known traps

- Chunking a two hour recording without overlap loses words at every boundary, and every cut point after that is derived from a transcript that has drifted from the audio. Overlap, merge on matching text, and check the word count and final timestamp against a single-pass transcription of a shorter excerpt.
- Producing a score with no explanation makes disagreeing with the tool impossible, and the user either accepts everything or ignores the feature. Store which signals fired for each candidate and show them, so a suggestion is an argument the person can reject.
- A crop that follows the detected face position frame by frame jitters constantly and is more distracting than a static crop, while one that snaps between two speakers is nauseating. Smooth the path, move only when the subject has genuinely moved, and cut rather than pan when the speaker changes.
- Estimating caption timing from word count and reading speed drifts audibly within the first minute, and checking only the opening seconds always looks fine. Use the word timings you already have, and verify at the end where drift accumulates.
- Any path that writes to the source file, including a helpfully added normalisation pass, damages an original recording that may be irreplaceable, and the loss is discovered when someone wants to re-clip it. Open read only, checksum before and after, and fail the build if they differ.

# Delivery milestones · OpusClip

Estimated effort: **multi-day** for the indie phases; the production-only milestones add the trust and operability layer.

## M1 · Words with times you can cut on

A long recording transcribes with timings accurate enough to cut by.

### Steps

1. Chunk with overlap and merge on text
   The seams are where the words go missing and everything downstream drifts.
   Files: `src/transcribe.py`, `tests/test_seams.py`
2. Verify against the source duration
   A transcript ending early means every later cut point is wrong.
   Files: `src/verify.py`
3. Record absent speaker labels as absent
   Guessing who spoke is worse than saying you do not know.
   Files: `src/speakers.py`

### Done when

- [ ] A two hour source transcribes with no seam losses
- [ ] The final timestamp matches the source duration
- [ ] Absent speaker labels are recorded, not guessed
- [ ] Transcription works offline

### Watch out

- Chunking a two hour recording without overlap loses words at every boundary, and every cut point after that is derived from a transcript that has drifted from the audio. Overlap, merge on matching text, and check the word count and final timestamp against a single-pass transcription of a shorter excerpt.

## M2 · Candidates proposed, with reasons

Suggested passages carry the signals behind them.

### Steps

1. Score on stated signals
   A number with no reasons cannot be argued with.
   Files: `src/candidates.py`
2. Require acceptance before rendering
   Automatic clipping produces volume rather than clips.
   Files: `src/review.py`
3. Snap boundaries to sentences
   A clip starting mid-sentence is discarded by whoever watches it.
   Files: `src/boundaries.py`

### Done when

- [ ] Candidates show their signals
- [ ] Rendering requires acceptance
- [ ] A thin source produces few candidates
- [ ] Boundaries fall on sentence starts

### Watch out

- Producing a score with no explanation makes disagreeing with the tool impossible, and the user either accepts everything or ignores the feature. Store which signals fired for each candidate and show them, so a suggestion is an argument the person can reject.

## M3 · Reframing without decapitating anyone

A vertical crop follows the speaker smoothly and keeps their head in frame.

### Steps

1. Smooth the crop path
   Frame-by-frame following is jitter, which is worse than a static crop.
   Files: `src/framing.py`
2. Cut rather than pan on speaker change
   A fast pan between two people is nauseating.
   Files: `src/framing.py`
3. Assert the head stays in frame
   Checked against the output, not against the tracking confidence.
   Files: `tests/test_framing.py`

### Done when

- [ ] The head stays fully in frame throughout
- [ ] The crop path is smooth rather than jittery
- [ ] Speaker changes cut rather than pan
- [ ] No detected subject has a stated fallback

### Watch out

- A crop that follows the detected face position frame by frame jitters constantly and is more distracting than a static crop, while one that snaps between two speakers is nauseating. Smooth the path, move only when the subject has genuinely moved, and cut rather than pan when the speaker changes.

## M4 · Captions from the transcript

Captions match the speech exactly at the end of a clip.

### Steps

1. Use the transcript's word timings
   You already have them. Estimation drifts within a minute.
   Files: `src/captions.py`
2. Break lines at natural boundaries
   A caption breaking mid-phrase is harder to read than a longer line.
   Files: `src/captions.py`
3. Verify at the end of the clip
   The opening seconds always look correct.
   Files: `tests/test_sync.py`

### Done when

- [ ] Captions match speech at the end of a clip
- [ ] Line breaks fall at natural boundaries
- [ ] Offsets are correct for mid-source clips
- [ ] Low confidence words are marked

### Watch out

- Estimating caption timing from word count and reading speed drifts audibly within the first minute, and checking only the opening seconds always looks fine. Use the word timings you already have, and verify at the end where drift accumulates.

## M5 · Rendering and keeping the source (production only)

Clips render reproducibly and the original is untouched.

### Steps

1. Render from a declarative description
   Reproducible, and reviewable as text before it runs.
   Files: `src/render.py`
2. Checksum sources in the suite
   A write to an original is an irreversible loss.
   Files: `tests/test_sources_untouched.py`
3. Isolate per-clip failures
   One bad clip must not end a thirty clip batch.
   Files: `src/batch.py`

### Done when

- [ ] The same description renders identically twice
- [ ] Source checksums are unchanged after a batch
- [ ] Per-clip failures are isolated and reported
- [ ] The manifest makes a render reproducible

### Watch out

- Any path that writes to the source file, including a helpfully added normalisation pass, damages an original recording that may be irreplaceable, and the loss is discovered when someone wants to re-clip it. Open read only, checksum before and after, and fail the build if they differ.

# Operations · OpusClip

## Backup

Source recordings are irreplaceable and large. Clips regenerate from sources plus their descriptions, so back up sources and descriptions and treat rendered clips as disposable.

## Restore

Restore sources and descriptions, re-render and compare against a kept clip. Byte equality proves the pipeline is deterministic and the restore complete.

Do a restore drill before the first real user, and write the date here when it passes.

## Monitoring

For a local pipeline the checks live in the suite: seam integrity, source checksums and end-of-clip caption sync. All three fail silently in ways that look like a quality problem rather than a bug.

## Incident checklist

A published clip where the speaker's head is cropped off is the visible failure, and it means the framing was verified against tracking confidence rather than against the rendered output.

1. Contain the issue without destroying evidence or user data.
2. Record the timeline and affected scope.
3. Rotate exposed secrets and revoke compromised sessions or credentials.
4. Restore from a verified backup when needed.
5. Document the root cause, the remediation and the regression test.

## Release gate

- [ ] A two hour source is confirmed to transcribe without seam losses
- [ ] Candidates are confirmed to show their signals
- [ ] The head is confirmed to stay in frame throughout
- [ ] Captions are confirmed in sync at the end of a clip
- [ ] Source checksums are confirmed unchanged

## Launch constraint

Do not market omitted OpusClip capabilities as implemented. The non-goals in `PRODUCT.md` remain user-visible limitations until they are deliberately delivered.

# Copy to .env and fill in. Never commit .env; this file documents it.

# Required. Original recordings, opened read only.
SOURCE_DIR=./sources

# Required. Rendered clips and manifests. Never the source directory.
OUTPUT_DIR=./clips

# Required. Timing quality matters more here than word accuracy.
ASR_MODEL=./models/ggml-medium.en.bin

# Required. The output aspect, which drives the whole framing problem.
TARGET_ASPECT=9:16

# Required. Too little jitters, too much lags behind the speaker.
CROP_SMOOTHING_MS=400

# Required. Present as a constant. Automatic clipping produces volume rather than clips.
AUTO_ACCEPT_CANDIDATES=never

$ choose a build depth, inspect the files, then open the complete pack in your agent