Vibecode VidNotes
track this build4 phases, 12 steps, beginner friendly0%KINDA · weekend project
The narrow DIY loop is achievable in a weekend: import a file or YouTube URL, transcribe it, generate notes, and save the result locally. VidNotes as a product is broader than that prototype. It maintains ingestion across changing video sites, runs long jobs reliably, ships iOS, Android, web, and a Chrome extension, keeps account state consistent, and exposes the workflow to agents through an API, CLI, and MCP server. Rebuilding all of that is a multi-day integration project with an ongoing operations burden.
the project pack5 files · written for this build, step by step
You are building a lean indie version of VidNotes. Create the following project files first, then implement the application by following them. Keep the files updated as decisions change. Do not collapse this into a single README or prompt. ===== README.md ===== # VidNotes · indie build A local workspace turning long recordings into timestamped transcripts and study material, whose hard problems are cutting audio into pieces a transcription API will accept without losing words at the seams, surviving a two-hour job that fails at minute ninety, and never showing a generated claim that cannot be found in the recording. Estimated effort: **weekend**. Work `BUILD_PLAN.md` top to bottom · every phase ends in a check that has to pass before the next one starts. ## Stack | Part | Choice | Why | | --- | --- | --- | | Runtime | One Python process serving localhost with a background worker, SQLite with full-text search | a study tool is not a service and should not need one | | Audio | ffmpeg to mono at speech sample rate in a compressed codec, split at silence | the input is a two-hour video and the API takes megabytes | | Transcription | A hosted model with published limits, chunked under them, key in the environment | local models are a fine later swap and a poor first dependency | | Jobs | A durable queue with per-chunk checkpoints and a recorded cost per call | a long job that restarts from zero costs money twice | | Notes | Generated per section, each claim verified against the transcript before it is shown | an unverified flashcard is a wrong answer with a confident face | | Storage | Media in a per-project directory beside the database, with a disk ceiling and a real delete | video is large and nobody remembers what they imported in March | ## Before you start Have every one of these ready. The plan assumes them from step one. - [ ] **Python 3.12 or newer** · free - Why: Runs the server and the worker. - Get it: python.org or your package manager. - Verify: python3 --version prints 3.12 or higher - [ ] **ffmpeg with an Opus or MP3 encoder** · free - Why: Extracts and compresses the audio, and a build without the encoder you chose fails only at the first real import. - Get it: Your package manager, or a static build from ffmpeg.org. - Verify: ffmpeg -encoders lists the encoder you intend to use - [ ] **An API key for a transcription and text model** · per minute of audio, a few pence per hour of recording - Why: Transcription is the one part not worth writing, and the key is also the thing that can quietly cost money. - Get it: Create a key with its own spending limit rather than reusing an unlimited one. - Verify: A one-minute sample returns timestamped text and the call appears on the usage page - [ ] **A recording whose true timings you know** · free - Why: Chunk offsets and seek accuracy cannot be tested against a recording nobody has listened to. - Get it: Record two minutes saying a distinct word at known times, or annotate a lecture you own. - Verify: The known words appear in the transcript at the times you spoke them - [ ] **Your disk ceiling, spend cap and import policy** · free - Why: Each is enforced by code and asserted by a test, and deciding them later means accepting whatever the first month produced. - Get it: Write down the media directory limit, the monthly spend cap, and whether the downloader is enabled and on what basis. - Verify: All three appear in the configuration and in the README ## Quick start ```sh ffmpeg -i input.mp4 -ac 1 -ar 16000 -c:a libopus prepared.opus ``` Then copy `.env.example` to `.env` and fill in the values it documents. ## Honest limits This build deliberately does not replace: - Native mobile apps and a browser extension. - Extraction from sites whose terms forbid it. - Accounts and cross-device sync. - A public API, CLI or agent server. - native iOS and Android apps, a Chrome extension, and cross-device Premium state - managed extraction for TikTok, Instagram, Vimeo, and changing source sites - background processing, queue reliability, and long-video handling - 30+ language polish and prebuilt flashcards, quizzes, and chat - hosted web, API, CLI, and MCP access for agents If one of those is essential to you, that is the reason to keep paying for VidNotes, and the README should say so rather than pretend. ===== BRIEF.md ===== # Build brief · VidNotes The one-shot brief this plan expands. `BUILD_PLAN.md` (or `MILESTONES.md`) is the same sequence broken into steps and checks; where the two disagree, the plan wins. Build me a local transcription and study-notes workspace for recordings you already hold to replace VidNotes. Build it in phases, in the order below. Do not write the whole thing in one pass. Finish a phase, run its "Done when" check, fix what fails, and only then start the next phase. ### Rule zero Two rules. The first is about what comes in, the second about what goes out. The default import is a file on your disk. A downloader pointed at a video site is a separate, off-by-default switch with the site's terms quoted beside it, because YouTube's terms forbid accessing, reproducing or downloading any part of the Service or its Content except as expressly authorised by the Service or with prior written permission (checked). Build the switch if you have that permission or the licence allows it, and make the app fully useful without it. Second, generated text never stands in for the recording. Every summary bullet, flashcard and quiz answer carries a timestamp and a span that exists verbatim in the transcript, and anything the verifier cannot locate is dropped with the gap shown rather than displayed anyway. Someone revising from a confidently invented date is worse off than someone with no notes at all. ### Stack (fixed, do not substitute) - Runtime: One Python process serving localhost with a background worker, SQLite with full-text search · a study tool is not a service and should not need one - Audio: ffmpeg to mono at speech sample rate in a compressed codec, split at silence · the input is a two-hour video and the API takes megabytes - Transcription: A hosted model with published limits, chunked under them, key in the environment · local models are a fine later swap and a poor first dependency - Jobs: A durable queue with per-chunk checkpoints and a recorded cost per call · a long job that restarts from zero costs money twice - Notes: Generated per section, each claim verified against the transcript before it is shown · an unverified flashcard is a wrong answer with a confident face - Storage: Media in a per-project directory beside the database, with a disk ceiling and a real delete · video is large and nobody remembers what they imported in March ### Data model (create this before Phase 1) - projects(id, title, source_kind, source_ref, duration_ms, created_at, deleted_at) · source_kind is file or url and source_ref is kept so you know where it came from - media(project_id, path, sha256, bytes, codec, sample_rate, channels, prepared_path) - chunks(id, project_id, ordinal, start_ms, end_ms, path, bytes, state, attempts, cost_cents, transcribed_at) · offsets are absolute in the original recording - segments(id, project_id, chunk_id, start_ms, end_ms, text) · one row per timed span, and the only thing search ever reads - notes(id, project_id, kind, section_start_ms, section_end_ms, content, quote, quote_segment_id, generated_at) · kind is summary, key_point, flashcard or quiz - jobs(id, project_id, stage, state, attempts, last_error, updated_at) · stage is extracting, chunking, transcribing or generating - A note without a resolvable quote_segment_id is never displayed, which makes the join the honesty check rather than a nicety. ### Phase 1 · Audio the API will actually accept Build: Extract audio with ffmpeg to mono at a speech sample rate in a compressed codec. Check the provider's published upload limit before choosing a chunk size: the widely used hosted API documents files up to 25 MB, lists mp3, mp4, mpeg, mpga, m4a, wav and webm, and says to compress or split into chunks of 25 MB or less while avoiding a split in the middle of a sentence (checked). Split at a silence near each boundary, store each chunk's absolute start offset, and overlap slightly so a word spanning a cut appears in one of the two. Trap: Cutting the file at a fixed byte offset drops a word at every seam and, worse, each chunk's returned timestamps start at zero, so unless you add the chunk's own start offset back every timestamp after the first chunk is wrong by a growing amount and every link in the app lands in the wrong place. Cut at silence, store absolute offsets, and assert against a recording whose true times you know. Done when: A two-hour recording produces chunks each under the published limit, verified by file size. Timestamps in the last chunk match the true position in the original within a stated tolerance, tested against a recording with known spoken markers. Every cut falls in a silent span, or the overlap covers the word, checked on a dense-speech sample. A file in an unsupported container is converted before upload rather than rejected at the API. Do not build yet: No job queue, notes, search or export. ### Phase 2 · A long job that survives being interrupted Build: Give every chunk its own state and attempt count. The worker takes one chunk at a time, records the call's cost, and retries only failures that could not have produced a transcript. On restart, chunks already transcribed are skipped. Show progress as chunks completed out of total, with the running cost, and let a project be cancelled mid-run without corrupting what it has. Trap: Treating transcription as one long call means a network blip, a rate limit or a laptop lid closing at minute ninety throws away every chunk already paid for, and the user retries the whole two hours. Checkpoint per chunk, persist before and after each call, and make resume the default path rather than an option. Done when: Killing the process mid-transcription and restarting resumes at the first unfinished chunk with no chunk transcribed twice, tested. A rate-limited call backs off and eventually succeeds without duplicating a chunk. The per-project cost is recorded from actual calls and shown before and after the run. Cancelling leaves completed chunks intact and the project resumable. Do not build yet: No search, notes or export yet. ### Phase 3 · Search that lands on the moment Build: Index segments rather than whole transcripts in full-text search. Return matches as a list of timed spans with surrounding context, and make each one seek the player. Group segments into readable paragraphs for display without losing the underlying times. Handle a query for a name or a number, which naive tokenisation tends to lose. Trap: Storing the transcript as one text blob and searching it gives you a hit that says the phrase is somewhere in a ninety-minute recording, which is the same as no answer; and rebuilding paragraphs for display without carrying the segment times means the clickable timestamp drifts from the words shown. Index per segment, and keep the time on every displayed line. Done when: A search for a phrase returns the segments containing it with their times, tested against a known transcript. Clicking a result seeks the player to within a stated tolerance of the spoken phrase. A query containing a number or a proper noun returns the right segment, tested explicitly. Paragraph grouping preserves the start time of each displayed line. Do not build yet: No generated notes or export yet. ### Phase 4 · Notes that can be checked against the recording Build: Generate summaries, key points, flashcards and quiz questions per section rather than over the whole transcript, asking for a verbatim quote with each claim. Verify each quote by exact match against the segment text after normalising whitespace and case, resolve it to a segment id, and drop anything that fails with the gap shown. Offer regeneration per section. Trap: Passing a ninety-minute transcript in one request and asking for ten flashcards produces confident cards about things nobody said, because the model is compressing far too much and has no obligation to point at anything. Generate per section, demand a quote, verify the quote exists, and show a section as unsummarised rather than inventing a summary of it. Done when: A card whose quote cannot be found in any segment is dropped and counted, tested by feeding a deliberately wrong quote. Every displayed note resolves to a segment id and its timestamp is clickable. Regenerating one section leaves other sections untouched, tested. A section that produces nothing verifiable is shown as such rather than filled in. Do not build yet: No export, deletion flow or ceilings yet. ### Phase 5 · Getting it out, and keeping the disk and the bill in hand Build: Export the transcript and notes as Markdown, plain text, subtitles and JSON, with timestamps preserved in each. Make deleting a project remove its media, its rows, its search index entries and its chunk files, then prove it. Set a disk ceiling for the media directory and a monthly spend cap that refuses new transcription rather than exceeding it. Keep the API key in the environment and never in the database or a log. Trap: Deleting a project by marking a row deleted leaves gigabytes of video and every transcript line in the search index, so the disk fills and a search still surfaces text from a recording the person believes they removed. Delete the files, the rows and the index entries in one operation, then assert the search returns nothing for a phrase that was unique to that project. Done when: Export produces all four formats with timestamps intact, opened and checked in another tool. After deleting a project, a phrase unique to it returns no search results and its directory is gone, both asserted in a test. Importing past the disk ceiling is refused with the current usage shown. Passing the monthly cap refuses new transcription and says so, tested by lowering the cap. Do not build yet: Nothing after this. ### Out of scope (and why) - Native mobile applications and a browser extension, each of which is its own release process and its own way for results to disagree with the desktop. - Extraction from sites whose terms forbid it, which is the feature that gets the tool taken down rather than the feature that makes it good. - Cross-device sync and accounts, which turn a local tool into a service with other people's recordings in it. - A public API, command line and agent server, which are a stable interface promise on top of a personal tool. ### README must contain - That the default import is a local file, and that any downloader is off by default with the source site's terms quoted beside it - That every generated claim is verified against a transcript span and dropped when it cannot be found - How chunking works, that offsets are absolute, and how to check timestamps against a known recording - Where media lives, what deleting a project actually removes, and how that is tested - That transcription costs money per recording, where the running total is shown, and how the monthly cap behaves ===== AGENTS.md ===== # Agent instructions · VidNotes indie build - Read `README.md` and `BUILD_PLAN.md` before writing code. The stack is fixed: One Python process serving localhost with a background worker, SQLite with full-text search, ffmpeg to mono at speech sample rate in a compressed codec, split at silence, A hosted model with published limits, chunked under them, key in the environment, A durable queue with per-chunk checkpoints and a recorded cost per call, Generated per section, each claim verified against the transcript before it is shown, Media in a per-project directory beside the database, with a disk ceiling and a real delete. Do not substitute. - Work one phase at a time, in order. Do not start a phase until every "Done when" item of the previous one passes. - Prefer the fewest moving parts that satisfy the step. No frameworks, services or dependencies the plan does not name. - Secrets live in `.env`, never in source or logs. Keep `.env.example` current when a variable is introduced. - Do not invent cryptography, security guarantees, APIs or compliance claims. - Add a focused test for every destructive, security-sensitive or data-loss path the plan names. - Run the project checks before declaring a phase complete, and record any deliberate shortcut in the README under "Tradeoffs". ## Known traps - Cutting the file at a fixed byte offset drops a word at every seam and, worse, each chunk's returned timestamps start at zero, so unless you add the chunk's own start offset back every timestamp after the first chunk is wrong by a growing amount and every link in the app lands in the wrong place. Cut at silence, store absolute offsets, and assert against a recording whose true times you know. - Treating transcription as one long call means a network blip, a rate limit or a laptop lid closing at minute ninety throws away every chunk already paid for, and the user retries the whole two hours. Checkpoint per chunk, persist before and after each call, and make resume the default path rather than an option. - Storing the transcript as one text blob and searching it gives you a hit that says the phrase is somewhere in a ninety-minute recording, which is the same as no answer; and rebuilding paragraphs for display without carrying the segment times means the clickable timestamp drifts from the words shown. Index per segment, and keep the time on every displayed line. - Passing a ninety-minute transcript in one request and asking for ten flashcards produces confident cards about things nobody said, because the model is compressing far too much and has no obligation to point at anything. Generate per section, demand a quote, verify the quote exists, and show a section as unsummarised rather than inventing a summary of it. ===== BUILD_PLAN.md ===== # Build plan · VidNotes A local workspace turning long recordings into timestamped transcripts and study material, whose hard problems are cutting audio into pieces a transcription API will accept without losing words at the seams, surviving a two-hour job that fails at minute ninety, and never showing a generated claim that cannot be found in the recording. Phases are in dependency order. Each ends in a "Done when" list; treat an unticked item as a blocker, not a note. ## Phase 1 · Audio the API will actually accept Any recording becomes chunks under the published size limit with offsets that still line up. ### Steps 1. Normalise to mono speech audio first Stereo at music sample rates doubles the bytes and adds nothing a transcriber uses. Files: `src/audio.py` ```sh ffmpeg -i input.mp4 -ac 1 -ar 16000 -c:a libopus prepared.opus ``` 2. Split at silence, not at a byte count Detect quiet spans near each boundary and cut there. Files: `src/chunking.py`, `tests/test_offsets.py` 3. Store absolute offsets and add them back Every returned timestamp is relative to its chunk until you fix it. Files: `src/chunking.py` ### Done when - [ ] Every chunk is under the published size limit - [ ] Late timestamps match true positions within tolerance - [ ] Cuts land in silence or are covered by overlap - [ ] Unsupported containers are converted, not refused ### Watch out - Cutting the file at a fixed byte offset drops a word at every seam and, worse, each chunk's returned timestamps start at zero, so unless you add the chunk's own start offset back every timestamp after the first chunk is wrong by a growing amount and every link in the app lands in the wrong place. Cut at silence, store absolute offsets, and assert against a recording whose true times you know. ## Phase 2 · A long job that survives being interrupted A failure at chunk forty resumes at chunk forty, and you can see what it cost. ### Steps 1. One chunk, one state, one transaction Write the state before the call and the result after it. Files: `src/worker.py`, `tests/test_resume.py` 2. Record cost per call as it happens A three-hour lecture is an unpleasant surprise unless the number is on screen. Files: `src/worker.py`, `src/templates/project.html` 3. Back off on rate limits, retry only pre-response failures A timeout after the audio was accepted may already have been charged. Files: `src/transcribe.py` ### Done when - [ ] A kill mid-run resumes without repeating a chunk - [ ] Rate limits back off and complete - [ ] Cost is recorded per call and displayed - [ ] Cancelling leaves completed work intact ### Watch out - Treating transcription as one long call means a network blip, a rate limit or a laptop lid closing at minute ninety throws away every chunk already paid for, and the user retries the whole two hours. Checkpoint per chunk, persist before and after each call, and make resume the default path rather than an option. ## Phase 3 · Search that lands on the moment A query returns spans with times, and clicking one moves the player there. ### Steps 1. Index segments, not transcripts The unit of search should be the unit you can seek to. Files: `src/search.py`, `tests/test_search.py` 2. Carry the time onto every rendered line Display grouping must never be allowed to lose it. Files: `src/templates/transcript.html` 3. Test names and numbers deliberately They are what people actually search a lecture for. Files: `tests/test_search.py` ### Done when - [ ] Phrase search returns timed segments - [ ] Clicking a result seeks accurately - [ ] Numbers and proper nouns are findable - [ ] Grouped lines keep their start times ### Watch out - Storing the transcript as one text blob and searching it gives you a hit that says the phrase is somewhere in a ninety-minute recording, which is the same as no answer; and rebuilding paragraphs for display without carrying the segment times means the clickable timestamp drifts from the words shown. Index per segment, and keep the time on every displayed line. ## Phase 4 · Notes that can be checked against the recording Every generated claim points at a span that really says it, or it is not shown. ### Steps 1. Generate per section with a required quote A claim with nowhere to point is a claim you cannot check. Files: `src/notes.py` 2. Verify the quote by exact match, then resolve it Normalise whitespace and case, nothing more forgiving than that. Files: `src/verify.py`, `tests/test_verify.py` 3. Show gaps instead of filling them An honest blank is more useful than a plausible sentence. Files: `src/templates/notes.html` ### Done when - [ ] Unverifiable claims are dropped and counted - [ ] Every shown note resolves to a segment - [ ] Section regeneration is isolated - [ ] Empty sections are shown as empty ### Watch out - Passing a ninety-minute transcript in one request and asking for ten flashcards produces confident cards about things nobody said, because the model is compressing far too much and has no obligation to point at anything. Generate per section, demand a quote, verify the quote exists, and show a section as unsummarised rather than inventing a summary of it. ## Not in this build - Native mobile apps and a browser extension. - Extraction from sites whose terms forbid it. - Accounts and cross-device sync. - A public API, CLI or agent server. ## After v1, if you want it - A local transcription model as a second backend, chosen per project, once the chunking is proven - Speaker labelling, which changes the segment table and should not be retrofitted casually - A revision mode that schedules flashcards over time, which needs nothing new from the pipeline ===== .env.example ===== # Copy to .env and fill in. Never commit .env; this file documents it. # Required. Bound to localhost only. This is a personal tool. APP_PORT=8787 # Required. Projects, segments, notes and the job state. DATABASE_PATH=./data/vidnotes.db # Required. One directory per project. Deleted in full when a project is deleted. MEDIA_DIR=./data/projects # Required. The disk ceiling. Imports past it are refused with usage shown. MEDIA_MAX_BYTES=53687091200 # Required · secret. Environment only. Never in the database, never in a log line. TRANSCRIBE_API_KEY=sk-...redacted # Required. Set below the provider's published limit, which you check rather than assume. CHUNK_MAX_BYTES=24000000 # Required. New transcription is refused past this rather than billed through it. MONTHLY_SPEND_CAP_CENTS=2000 # Required. Off by default. Turning it on is a statement about the source's terms. ENABLE_URL_IMPORT=false
You are building a lean indie version of VidNotes. Create the following project files first, then implement the application by following them. Keep the files updated as decisions change. Do not collapse this into a single README or prompt. ===== README.md ===== # VidNotes · indie build A local workspace turning long recordings into timestamped transcripts and study material, whose hard problems are cutting audio into pieces a transcription API will accept without losing words at the seams, surviving a two-hour job that fails at minute ninety, and never showing a generated claim that cannot be found in the recording. Estimated effort: **weekend**. Work `BUILD_PLAN.md` top to bottom · every phase ends in a check that has to pass before the next one starts. ## Stack | Part | Choice | Why | | --- | --- | --- | | Runtime | One Python process serving localhost with a background worker, SQLite with full-text search | a study tool is not a service and should not need one | | Audio | ffmpeg to mono at speech sample rate in a compressed codec, split at silence | the input is a two-hour video and the API takes megabytes | | Transcription | A hosted model with published limits, chunked under them, key in the environment | local models are a fine later swap and a poor first dependency | | Jobs | A durable queue with per-chunk checkpoints and a recorded cost per call | a long job that restarts from zero costs money twice | | Notes | Generated per section, each claim verified against the transcript before it is shown | an unverified flashcard is a wrong answer with a confident face | | Storage | Media in a per-project directory beside the database, with a disk ceiling and a real delete | video is large and nobody remembers what they imported in March | ## Before you start Have every one of these ready. The plan assumes them from step one. - [ ] **Python 3.12 or newer** · free - Why: Runs the server and the worker. - Get it: python.org or your package manager. - Verify: python3 --version prints 3.12 or higher - [ ] **ffmpeg with an Opus or MP3 encoder** · free - Why: Extracts and compresses the audio, and a build without the encoder you chose fails only at the first real import. - Get it: Your package manager, or a static build from ffmpeg.org. - Verify: ffmpeg -encoders lists the encoder you intend to use - [ ] **An API key for a transcription and text model** · per minute of audio, a few pence per hour of recording - Why: Transcription is the one part not worth writing, and the key is also the thing that can quietly cost money. - Get it: Create a key with its own spending limit rather than reusing an unlimited one. - Verify: A one-minute sample returns timestamped text and the call appears on the usage page - [ ] **A recording whose true timings you know** · free - Why: Chunk offsets and seek accuracy cannot be tested against a recording nobody has listened to. - Get it: Record two minutes saying a distinct word at known times, or annotate a lecture you own. - Verify: The known words appear in the transcript at the times you spoke them - [ ] **Your disk ceiling, spend cap and import policy** · free - Why: Each is enforced by code and asserted by a test, and deciding them later means accepting whatever the first month produced. - Get it: Write down the media directory limit, the monthly spend cap, and whether the downloader is enabled and on what basis. - Verify: All three appear in the configuration and in the README ## Quick start ```sh ffmpeg -i input.mp4 -ac 1 -ar 16000 -c:a libopus prepared.opus ``` Then copy `.env.example` to `.env` and fill in the values it documents. ## Honest limits This build deliberately does not replace: - Native mobile apps and a browser extension. - Extraction from sites whose terms forbid it. - Accounts and cross-device sync. - A public API, CLI or agent server. - native iOS and Android apps, a Chrome extension, and cross-device Premium state - managed extraction for TikTok, Instagram, Vimeo, and changing source sites - background processing, queue reliability, and long-video handling - 30+ language polish and prebuilt flashcards, quizzes, and chat - hosted web, API, CLI, and MCP access for agents If one of those is essential to you, that is the reason to keep paying for VidNotes, and the README should say so rather than pretend. ===== BRIEF.md ===== # Build brief · VidNotes The one-shot brief this plan expands. `BUILD_PLAN.md` (or `MILESTONES.md`) is the same sequence broken into steps and checks; where the two disagree, the plan wins. Build me a local transcription and study-notes workspace for recordings you already hold to replace VidNotes. Build it in phases, in the order below. Do not write the whole thing in one pass. Finish a phase, run its "Done when" check, fix what fails, and only then start the next phase. ### Rule zero Two rules. The first is about what comes in, the second about what goes out. The default import is a file on your disk. A downloader pointed at a video site is a separate, off-by-default switch with the site's terms quoted beside it, because YouTube's terms forbid accessing, reproducing or downloading any part of the Service or its Content except as expressly authorised by the Service or with prior written permission (checked). Build the switch if you have that permission or the licence allows it, and make the app fully useful without it. Second, generated text never stands in for the recording. Every summary bullet, flashcard and quiz answer carries a timestamp and a span that exists verbatim in the transcript, and anything the verifier cannot locate is dropped with the gap shown rather than displayed anyway. Someone revising from a confidently invented date is worse off than someone with no notes at all. ### Stack (fixed, do not substitute) - Runtime: One Python process serving localhost with a background worker, SQLite with full-text search · a study tool is not a service and should not need one - Audio: ffmpeg to mono at speech sample rate in a compressed codec, split at silence · the input is a two-hour video and the API takes megabytes - Transcription: A hosted model with published limits, chunked under them, key in the environment · local models are a fine later swap and a poor first dependency - Jobs: A durable queue with per-chunk checkpoints and a recorded cost per call · a long job that restarts from zero costs money twice - Notes: Generated per section, each claim verified against the transcript before it is shown · an unverified flashcard is a wrong answer with a confident face - Storage: Media in a per-project directory beside the database, with a disk ceiling and a real delete · video is large and nobody remembers what they imported in March ### Data model (create this before Phase 1) - projects(id, title, source_kind, source_ref, duration_ms, created_at, deleted_at) · source_kind is file or url and source_ref is kept so you know where it came from - media(project_id, path, sha256, bytes, codec, sample_rate, channels, prepared_path) - chunks(id, project_id, ordinal, start_ms, end_ms, path, bytes, state, attempts, cost_cents, transcribed_at) · offsets are absolute in the original recording - segments(id, project_id, chunk_id, start_ms, end_ms, text) · one row per timed span, and the only thing search ever reads - notes(id, project_id, kind, section_start_ms, section_end_ms, content, quote, quote_segment_id, generated_at) · kind is summary, key_point, flashcard or quiz - jobs(id, project_id, stage, state, attempts, last_error, updated_at) · stage is extracting, chunking, transcribing or generating - A note without a resolvable quote_segment_id is never displayed, which makes the join the honesty check rather than a nicety. ### Phase 1 · Audio the API will actually accept Build: Extract audio with ffmpeg to mono at a speech sample rate in a compressed codec. Check the provider's published upload limit before choosing a chunk size: the widely used hosted API documents files up to 25 MB, lists mp3, mp4, mpeg, mpga, m4a, wav and webm, and says to compress or split into chunks of 25 MB or less while avoiding a split in the middle of a sentence (checked). Split at a silence near each boundary, store each chunk's absolute start offset, and overlap slightly so a word spanning a cut appears in one of the two. Trap: Cutting the file at a fixed byte offset drops a word at every seam and, worse, each chunk's returned timestamps start at zero, so unless you add the chunk's own start offset back every timestamp after the first chunk is wrong by a growing amount and every link in the app lands in the wrong place. Cut at silence, store absolute offsets, and assert against a recording whose true times you know. Done when: A two-hour recording produces chunks each under the published limit, verified by file size. Timestamps in the last chunk match the true position in the original within a stated tolerance, tested against a recording with known spoken markers. Every cut falls in a silent span, or the overlap covers the word, checked on a dense-speech sample. A file in an unsupported container is converted before upload rather than rejected at the API. Do not build yet: No job queue, notes, search or export. ### Phase 2 · A long job that survives being interrupted Build: Give every chunk its own state and attempt count. The worker takes one chunk at a time, records the call's cost, and retries only failures that could not have produced a transcript. On restart, chunks already transcribed are skipped. Show progress as chunks completed out of total, with the running cost, and let a project be cancelled mid-run without corrupting what it has. Trap: Treating transcription as one long call means a network blip, a rate limit or a laptop lid closing at minute ninety throws away every chunk already paid for, and the user retries the whole two hours. Checkpoint per chunk, persist before and after each call, and make resume the default path rather than an option. Done when: Killing the process mid-transcription and restarting resumes at the first unfinished chunk with no chunk transcribed twice, tested. A rate-limited call backs off and eventually succeeds without duplicating a chunk. The per-project cost is recorded from actual calls and shown before and after the run. Cancelling leaves completed chunks intact and the project resumable. Do not build yet: No search, notes or export yet. ### Phase 3 · Search that lands on the moment Build: Index segments rather than whole transcripts in full-text search. Return matches as a list of timed spans with surrounding context, and make each one seek the player. Group segments into readable paragraphs for display without losing the underlying times. Handle a query for a name or a number, which naive tokenisation tends to lose. Trap: Storing the transcript as one text blob and searching it gives you a hit that says the phrase is somewhere in a ninety-minute recording, which is the same as no answer; and rebuilding paragraphs for display without carrying the segment times means the clickable timestamp drifts from the words shown. Index per segment, and keep the time on every displayed line. Done when: A search for a phrase returns the segments containing it with their times, tested against a known transcript. Clicking a result seeks the player to within a stated tolerance of the spoken phrase. A query containing a number or a proper noun returns the right segment, tested explicitly. Paragraph grouping preserves the start time of each displayed line. Do not build yet: No generated notes or export yet. ### Phase 4 · Notes that can be checked against the recording Build: Generate summaries, key points, flashcards and quiz questions per section rather than over the whole transcript, asking for a verbatim quote with each claim. Verify each quote by exact match against the segment text after normalising whitespace and case, resolve it to a segment id, and drop anything that fails with the gap shown. Offer regeneration per section. Trap: Passing a ninety-minute transcript in one request and asking for ten flashcards produces confident cards about things nobody said, because the model is compressing far too much and has no obligation to point at anything. Generate per section, demand a quote, verify the quote exists, and show a section as unsummarised rather than inventing a summary of it. Done when: A card whose quote cannot be found in any segment is dropped and counted, tested by feeding a deliberately wrong quote. Every displayed note resolves to a segment id and its timestamp is clickable. Regenerating one section leaves other sections untouched, tested. A section that produces nothing verifiable is shown as such rather than filled in. Do not build yet: No export, deletion flow or ceilings yet. ### Phase 5 · Getting it out, and keeping the disk and the bill in hand Build: Export the transcript and notes as Markdown, plain text, subtitles and JSON, with timestamps preserved in each. Make deleting a project remove its media, its rows, its search index entries and its chunk files, then prove it. Set a disk ceiling for the media directory and a monthly spend cap that refuses new transcription rather than exceeding it. Keep the API key in the environment and never in the database or a log. Trap: Deleting a project by marking a row deleted leaves gigabytes of video and every transcript line in the search index, so the disk fills and a search still surfaces text from a recording the person believes they removed. Delete the files, the rows and the index entries in one operation, then assert the search returns nothing for a phrase that was unique to that project. Done when: Export produces all four formats with timestamps intact, opened and checked in another tool. After deleting a project, a phrase unique to it returns no search results and its directory is gone, both asserted in a test. Importing past the disk ceiling is refused with the current usage shown. Passing the monthly cap refuses new transcription and says so, tested by lowering the cap. Do not build yet: Nothing after this. ### Out of scope (and why) - Native mobile applications and a browser extension, each of which is its own release process and its own way for results to disagree with the desktop. - Extraction from sites whose terms forbid it, which is the feature that gets the tool taken down rather than the feature that makes it good. - Cross-device sync and accounts, which turn a local tool into a service with other people's recordings in it. - A public API, command line and agent server, which are a stable interface promise on top of a personal tool. ### README must contain - That the default import is a local file, and that any downloader is off by default with the source site's terms quoted beside it - That every generated claim is verified against a transcript span and dropped when it cannot be found - How chunking works, that offsets are absolute, and how to check timestamps against a known recording - Where media lives, what deleting a project actually removes, and how that is tested - That transcription costs money per recording, where the running total is shown, and how the monthly cap behaves ===== AGENTS.md ===== # Agent instructions · VidNotes indie build - Read `README.md` and `BUILD_PLAN.md` before writing code. The stack is fixed: One Python process serving localhost with a background worker, SQLite with full-text search, ffmpeg to mono at speech sample rate in a compressed codec, split at silence, A hosted model with published limits, chunked under them, key in the environment, A durable queue with per-chunk checkpoints and a recorded cost per call, Generated per section, each claim verified against the transcript before it is shown, Media in a per-project directory beside the database, with a disk ceiling and a real delete. Do not substitute. - Work one phase at a time, in order. Do not start a phase until every "Done when" item of the previous one passes. - Prefer the fewest moving parts that satisfy the step. No frameworks, services or dependencies the plan does not name. - Secrets live in `.env`, never in source or logs. Keep `.env.example` current when a variable is introduced. - Do not invent cryptography, security guarantees, APIs or compliance claims. - Add a focused test for every destructive, security-sensitive or data-loss path the plan names. - Run the project checks before declaring a phase complete, and record any deliberate shortcut in the README under "Tradeoffs". ## Known traps - Cutting the file at a fixed byte offset drops a word at every seam and, worse, each chunk's returned timestamps start at zero, so unless you add the chunk's own start offset back every timestamp after the first chunk is wrong by a growing amount and every link in the app lands in the wrong place. Cut at silence, store absolute offsets, and assert against a recording whose true times you know. - Treating transcription as one long call means a network blip, a rate limit or a laptop lid closing at minute ninety throws away every chunk already paid for, and the user retries the whole two hours. Checkpoint per chunk, persist before and after each call, and make resume the default path rather than an option. - Storing the transcript as one text blob and searching it gives you a hit that says the phrase is somewhere in a ninety-minute recording, which is the same as no answer; and rebuilding paragraphs for display without carrying the segment times means the clickable timestamp drifts from the words shown. Index per segment, and keep the time on every displayed line. - Passing a ninety-minute transcript in one request and asking for ten flashcards produces confident cards about things nobody said, because the model is compressing far too much and has no obligation to point at anything. Generate per section, demand a quote, verify the quote exists, and show a section as unsummarised rather than inventing a summary of it. ===== BUILD_PLAN.md ===== # Build plan · VidNotes A local workspace turning long recordings into timestamped transcripts and study material, whose hard problems are cutting audio into pieces a transcription API will accept without losing words at the seams, surviving a two-hour job that fails at minute ninety, and never showing a generated claim that cannot be found in the recording. Phases are in dependency order. Each ends in a "Done when" list; treat an unticked item as a blocker, not a note. ## Phase 1 · Audio the API will actually accept Any recording becomes chunks under the published size limit with offsets that still line up. ### Steps 1. Normalise to mono speech audio first Stereo at music sample rates doubles the bytes and adds nothing a transcriber uses. Files: `src/audio.py` ```sh ffmpeg -i input.mp4 -ac 1 -ar 16000 -c:a libopus prepared.opus ``` 2. Split at silence, not at a byte count Detect quiet spans near each boundary and cut there. Files: `src/chunking.py`, `tests/test_offsets.py` 3. Store absolute offsets and add them back Every returned timestamp is relative to its chunk until you fix it. Files: `src/chunking.py` ### Done when - [ ] Every chunk is under the published size limit - [ ] Late timestamps match true positions within tolerance - [ ] Cuts land in silence or are covered by overlap - [ ] Unsupported containers are converted, not refused ### Watch out - Cutting the file at a fixed byte offset drops a word at every seam and, worse, each chunk's returned timestamps start at zero, so unless you add the chunk's own start offset back every timestamp after the first chunk is wrong by a growing amount and every link in the app lands in the wrong place. Cut at silence, store absolute offsets, and assert against a recording whose true times you know. ## Phase 2 · A long job that survives being interrupted A failure at chunk forty resumes at chunk forty, and you can see what it cost. ### Steps 1. One chunk, one state, one transaction Write the state before the call and the result after it. Files: `src/worker.py`, `tests/test_resume.py` 2. Record cost per call as it happens A three-hour lecture is an unpleasant surprise unless the number is on screen. Files: `src/worker.py`, `src/templates/project.html` 3. Back off on rate limits, retry only pre-response failures A timeout after the audio was accepted may already have been charged. Files: `src/transcribe.py` ### Done when - [ ] A kill mid-run resumes without repeating a chunk - [ ] Rate limits back off and complete - [ ] Cost is recorded per call and displayed - [ ] Cancelling leaves completed work intact ### Watch out - Treating transcription as one long call means a network blip, a rate limit or a laptop lid closing at minute ninety throws away every chunk already paid for, and the user retries the whole two hours. Checkpoint per chunk, persist before and after each call, and make resume the default path rather than an option. ## Phase 3 · Search that lands on the moment A query returns spans with times, and clicking one moves the player there. ### Steps 1. Index segments, not transcripts The unit of search should be the unit you can seek to. Files: `src/search.py`, `tests/test_search.py` 2. Carry the time onto every rendered line Display grouping must never be allowed to lose it. Files: `src/templates/transcript.html` 3. Test names and numbers deliberately They are what people actually search a lecture for. Files: `tests/test_search.py` ### Done when - [ ] Phrase search returns timed segments - [ ] Clicking a result seeks accurately - [ ] Numbers and proper nouns are findable - [ ] Grouped lines keep their start times ### Watch out - Storing the transcript as one text blob and searching it gives you a hit that says the phrase is somewhere in a ninety-minute recording, which is the same as no answer; and rebuilding paragraphs for display without carrying the segment times means the clickable timestamp drifts from the words shown. Index per segment, and keep the time on every displayed line. ## Phase 4 · Notes that can be checked against the recording Every generated claim points at a span that really says it, or it is not shown. ### Steps 1. Generate per section with a required quote A claim with nowhere to point is a claim you cannot check. Files: `src/notes.py` 2. Verify the quote by exact match, then resolve it Normalise whitespace and case, nothing more forgiving than that. Files: `src/verify.py`, `tests/test_verify.py` 3. Show gaps instead of filling them An honest blank is more useful than a plausible sentence. Files: `src/templates/notes.html` ### Done when - [ ] Unverifiable claims are dropped and counted - [ ] Every shown note resolves to a segment - [ ] Section regeneration is isolated - [ ] Empty sections are shown as empty ### Watch out - Passing a ninety-minute transcript in one request and asking for ten flashcards produces confident cards about things nobody said, because the model is compressing far too much and has no obligation to point at anything. Generate per section, demand a quote, verify the quote exists, and show a section as unsummarised rather than inventing a summary of it. ## Not in this build - Native mobile apps and a browser extension. - Extraction from sites whose terms forbid it. - Accounts and cross-device sync. - A public API, CLI or agent server. ## After v1, if you want it - A local transcription model as a second backend, chosen per project, once the chunking is proven - Speaker labelling, which changes the segment table and should not be retrofitted casually - A revision mode that schedules flashcards over time, which needs nothing new from the pipeline ===== .env.example ===== # Copy to .env and fill in. Never commit .env; this file documents it. # Required. Bound to localhost only. This is a personal tool. APP_PORT=8787 # Required. Projects, segments, notes and the job state. DATABASE_PATH=./data/vidnotes.db # Required. One directory per project. Deleted in full when a project is deleted. MEDIA_DIR=./data/projects # Required. The disk ceiling. Imports past it are refused with usage shown. MEDIA_MAX_BYTES=53687091200 # Required · secret. Environment only. Never in the database, never in a log line. TRANSCRIBE_API_KEY=sk-...redacted # Required. Set below the provider's published limit, which you check rather than assume. CHUNK_MAX_BYTES=24000000 # Required. New transcription is refused past this rather than billed through it. MONTHLY_SPEND_CAP_CENTS=2000 # Required. Off by default. Turning it on is a statement about the source's terms. ENABLE_URL_IMPORT=false
You are building a production product version of VidNotes. Create the following project files first, then implement the application by following them. Keep the files updated as decisions change. Do not collapse this into a single README or prompt. ===== PRODUCT.md ===== # VidNotes · product brief ## Problem The narrow DIY loop is achievable in a weekend: import a file or YouTube URL, transcribe it, generate notes, and save the result locally. VidNotes as a product is broader than that prototype. It maintains ingestion across changing video sites, runs long jobs reliably, ships iOS, Android, web, and a Chrome extension, keeps account state consistent, and exposes the workflow to agents through an API, CLI, and MCP server. Rebuilding all of that is a multi-day integration project with an ongoing operations burden. ## Product outcome A workspace that turns a long recording into a searchable timestamped transcript and study material where every claim can be checked against what was actually said, resumes a failed job instead of paying twice, and deletes what you tell it to delete. ## Target user A builder who needs a maintainable product foundation, not a one-off demo. ## Required capabilities - Python 3.12 - ffmpeg - yt-dlp - OpenAI API key - local disk ## Explicit non-goals for v1 - Native mobile apps and a browser extension. - Extraction from sites whose terms forbid it. - Accounts and cross-device sync. - A public API, CLI or agent server. - native iOS and Android apps, a Chrome extension, and cross-device Premium state - managed extraction for TikTok, Instagram, Vimeo, and changing source sites - background processing, queue reliability, and long-video handling - 30+ language polish and prebuilt flashcards, quizzes, and chat - hosted web, API, CLI, and MCP access for agents ## Success criteria - Late-chunk timestamps are confirmed against a known recording - A kill mid-run is confirmed to resume without repeating a chunk - An unverifiable generated claim is confirmed dropped - A deleted project is confirmed to leave nothing findable - The monthly spend cap is confirmed to refuse new work ===== BRIEF.md ===== # Build brief · VidNotes The one-shot brief this plan expands. `BUILD_PLAN.md` (or `MILESTONES.md`) is the same sequence broken into steps and checks; where the two disagree, the plan wins. Build me a local transcription and study-notes workspace for recordings you already hold to replace VidNotes. Build it in phases, in the order below. Do not write the whole thing in one pass. Finish a phase, run its "Done when" check, fix what fails, and only then start the next phase. ### Rule zero Two rules. The first is about what comes in, the second about what goes out. The default import is a file on your disk. A downloader pointed at a video site is a separate, off-by-default switch with the site's terms quoted beside it, because YouTube's terms forbid accessing, reproducing or downloading any part of the Service or its Content except as expressly authorised by the Service or with prior written permission (checked). Build the switch if you have that permission or the licence allows it, and make the app fully useful without it. Second, generated text never stands in for the recording. Every summary bullet, flashcard and quiz answer carries a timestamp and a span that exists verbatim in the transcript, and anything the verifier cannot locate is dropped with the gap shown rather than displayed anyway. Someone revising from a confidently invented date is worse off than someone with no notes at all. ### Stack (fixed, do not substitute) - Runtime: One Python process serving localhost with a background worker, SQLite with full-text search · a study tool is not a service and should not need one - Audio: ffmpeg to mono at speech sample rate in a compressed codec, split at silence · the input is a two-hour video and the API takes megabytes - Transcription: A hosted model with published limits, chunked under them, key in the environment · local models are a fine later swap and a poor first dependency - Jobs: A durable queue with per-chunk checkpoints and a recorded cost per call · a long job that restarts from zero costs money twice - Notes: Generated per section, each claim verified against the transcript before it is shown · an unverified flashcard is a wrong answer with a confident face - Storage: Media in a per-project directory beside the database, with a disk ceiling and a real delete · video is large and nobody remembers what they imported in March ### Data model (create this before Phase 1) - projects(id, title, source_kind, source_ref, duration_ms, created_at, deleted_at) · source_kind is file or url and source_ref is kept so you know where it came from - media(project_id, path, sha256, bytes, codec, sample_rate, channels, prepared_path) - chunks(id, project_id, ordinal, start_ms, end_ms, path, bytes, state, attempts, cost_cents, transcribed_at) · offsets are absolute in the original recording - segments(id, project_id, chunk_id, start_ms, end_ms, text) · one row per timed span, and the only thing search ever reads - notes(id, project_id, kind, section_start_ms, section_end_ms, content, quote, quote_segment_id, generated_at) · kind is summary, key_point, flashcard or quiz - jobs(id, project_id, stage, state, attempts, last_error, updated_at) · stage is extracting, chunking, transcribing or generating - A note without a resolvable quote_segment_id is never displayed, which makes the join the honesty check rather than a nicety. ### Phase 1 · Audio the API will actually accept Build: Extract audio with ffmpeg to mono at a speech sample rate in a compressed codec. Check the provider's published upload limit before choosing a chunk size: the widely used hosted API documents files up to 25 MB, lists mp3, mp4, mpeg, mpga, m4a, wav and webm, and says to compress or split into chunks of 25 MB or less while avoiding a split in the middle of a sentence (checked). Split at a silence near each boundary, store each chunk's absolute start offset, and overlap slightly so a word spanning a cut appears in one of the two. Trap: Cutting the file at a fixed byte offset drops a word at every seam and, worse, each chunk's returned timestamps start at zero, so unless you add the chunk's own start offset back every timestamp after the first chunk is wrong by a growing amount and every link in the app lands in the wrong place. Cut at silence, store absolute offsets, and assert against a recording whose true times you know. Done when: A two-hour recording produces chunks each under the published limit, verified by file size. Timestamps in the last chunk match the true position in the original within a stated tolerance, tested against a recording with known spoken markers. Every cut falls in a silent span, or the overlap covers the word, checked on a dense-speech sample. A file in an unsupported container is converted before upload rather than rejected at the API. Do not build yet: No job queue, notes, search or export. ### Phase 2 · A long job that survives being interrupted Build: Give every chunk its own state and attempt count. The worker takes one chunk at a time, records the call's cost, and retries only failures that could not have produced a transcript. On restart, chunks already transcribed are skipped. Show progress as chunks completed out of total, with the running cost, and let a project be cancelled mid-run without corrupting what it has. Trap: Treating transcription as one long call means a network blip, a rate limit or a laptop lid closing at minute ninety throws away every chunk already paid for, and the user retries the whole two hours. Checkpoint per chunk, persist before and after each call, and make resume the default path rather than an option. Done when: Killing the process mid-transcription and restarting resumes at the first unfinished chunk with no chunk transcribed twice, tested. A rate-limited call backs off and eventually succeeds without duplicating a chunk. The per-project cost is recorded from actual calls and shown before and after the run. Cancelling leaves completed chunks intact and the project resumable. Do not build yet: No search, notes or export yet. ### Phase 3 · Search that lands on the moment Build: Index segments rather than whole transcripts in full-text search. Return matches as a list of timed spans with surrounding context, and make each one seek the player. Group segments into readable paragraphs for display without losing the underlying times. Handle a query for a name or a number, which naive tokenisation tends to lose. Trap: Storing the transcript as one text blob and searching it gives you a hit that says the phrase is somewhere in a ninety-minute recording, which is the same as no answer; and rebuilding paragraphs for display without carrying the segment times means the clickable timestamp drifts from the words shown. Index per segment, and keep the time on every displayed line. Done when: A search for a phrase returns the segments containing it with their times, tested against a known transcript. Clicking a result seeks the player to within a stated tolerance of the spoken phrase. A query containing a number or a proper noun returns the right segment, tested explicitly. Paragraph grouping preserves the start time of each displayed line. Do not build yet: No generated notes or export yet. ### Phase 4 · Notes that can be checked against the recording Build: Generate summaries, key points, flashcards and quiz questions per section rather than over the whole transcript, asking for a verbatim quote with each claim. Verify each quote by exact match against the segment text after normalising whitespace and case, resolve it to a segment id, and drop anything that fails with the gap shown. Offer regeneration per section. Trap: Passing a ninety-minute transcript in one request and asking for ten flashcards produces confident cards about things nobody said, because the model is compressing far too much and has no obligation to point at anything. Generate per section, demand a quote, verify the quote exists, and show a section as unsummarised rather than inventing a summary of it. Done when: A card whose quote cannot be found in any segment is dropped and counted, tested by feeding a deliberately wrong quote. Every displayed note resolves to a segment id and its timestamp is clickable. Regenerating one section leaves other sections untouched, tested. A section that produces nothing verifiable is shown as such rather than filled in. Do not build yet: No export, deletion flow or ceilings yet. ### Phase 5 · Getting it out, and keeping the disk and the bill in hand Build: Export the transcript and notes as Markdown, plain text, subtitles and JSON, with timestamps preserved in each. Make deleting a project remove its media, its rows, its search index entries and its chunk files, then prove it. Set a disk ceiling for the media directory and a monthly spend cap that refuses new transcription rather than exceeding it. Keep the API key in the environment and never in the database or a log. Trap: Deleting a project by marking a row deleted leaves gigabytes of video and every transcript line in the search index, so the disk fills and a search still surfaces text from a recording the person believes they removed. Delete the files, the rows and the index entries in one operation, then assert the search returns nothing for a phrase that was unique to that project. Done when: Export produces all four formats with timestamps intact, opened and checked in another tool. After deleting a project, a phrase unique to it returns no search results and its directory is gone, both asserted in a test. Importing past the disk ceiling is refused with the current usage shown. Passing the monthly cap refuses new transcription and says so, tested by lowering the cap. Do not build yet: Nothing after this. ### Out of scope (and why) - Native mobile applications and a browser extension, each of which is its own release process and its own way for results to disagree with the desktop. - Extraction from sites whose terms forbid it, which is the feature that gets the tool taken down rather than the feature that makes it good. - Cross-device sync and accounts, which turn a local tool into a service with other people's recordings in it. - A public API, command line and agent server, which are a stable interface promise on top of a personal tool. ### README must contain - That the default import is a local file, and that any downloader is off by default with the source site's terms quoted beside it - That every generated claim is verified against a transcript span and dropped when it cannot be found - How chunking works, that offsets are absolute, and how to check timestamps against a known recording - Where media lives, what deleting a project actually removes, and how that is tested - That transcription costs money per recording, where the running total is shown, and how the monthly cap behaves ===== ARCHITECTURE.md ===== # Architecture · VidNotes ## Stack | Part | Choice | Why | | --- | --- | --- | | Runtime | One Python process serving localhost with a background worker, SQLite with full-text search | a study tool is not a service and should not need one | | Audio | ffmpeg to mono at speech sample rate in a compressed codec, split at silence | the input is a two-hour video and the API takes megabytes | | Transcription | A hosted model with published limits, chunked under them, key in the environment | local models are a fine later swap and a poor first dependency | | Jobs | A durable queue with per-chunk checkpoints and a recorded cost per call | a long job that restarts from zero costs money twice | | Notes | Generated per section, each claim verified against the transcript before it is shown | an unverified flashcard is a wrong answer with a confident face | | Storage | Media in a per-project directory beside the database, with a disk ceiling and a real delete | video is large and nobody remembers what they imported in March | ## Modules Each module has one owner concern and a documented way to replace it. | Module | Owns | How to replace it | | --- | --- | --- | | Audio | Normalisation, silence-aware chunking, absolute offsets. | Any encoder, provided offsets are still asserted against a known recording. | | Worker | Per-chunk state, retries, cost recording. | Any queue, provided resume remains the default path. | | Search | Segment-level indexing and seeking. | Any index, provided the unit stays the segment. | | Notes | Per-section generation and quote verification. | Any model, provided the verifier still drops what it cannot locate. | | Lifecycle | Export, deletion, disk and spend ceilings. | Not swappable for soft deletion, which leaves the media and the index behind. | ## Configuration Every runtime setting is an environment variable documented in `.env.example`, validated at startup, with a safe local default wherever one exists. - `APP_PORT` · required · Bound to localhost only. This is a personal tool. - `DATABASE_PATH` · required · Projects, segments, notes and the job state. - `MEDIA_DIR` · required · One directory per project. Deleted in full when a project is deleted. - `MEDIA_MAX_BYTES` · required · The disk ceiling. Imports past it are refused with usage shown. - `TRANSCRIBE_API_KEY` · required, secret · Environment only. Never in the database, never in a log line. - `CHUNK_MAX_BYTES` · required · Set below the provider's published limit, which you check rather than assume. - `MONTHLY_SPEND_CAP_CENTS` · required · New transcription is refused past this rather than billed through it. - `ENABLE_URL_IMPORT` · required · Off by default. Turning it on is a statement about the source's terms. ## Production baseline - Security: least privilege, input validation at every boundary, secret redaction in logs, rate limits on abuse-prone paths, no invented security primitives. - Data: explicit schema and migrations, transactional writes where integrity matters, backup and restore procedures that have been exercised. - Integrations: adapters around third-party providers, idempotent webhook or job processing, bounded retries, timeouts. - Observability: structured logs with request or operation ids, an error-tracking hook, and health and readiness checks where a server exists. - Quality: unit tests for domain rules, integration tests at module boundaries, one end-to-end test of the critical path. ## Decision records For each dependency in the stack table, keep a short note: why it was chosen, its failure mode, and how it is replaced. Do not add infrastructure until a requirement in `PRODUCT.md` justifies it. ===== AGENTS.md ===== # Agent instructions · VidNotes product build - Read `PRODUCT.md` and `ARCHITECTURE.md` before changing code. The stack is fixed: One Python process serving localhost with a background worker, SQLite with full-text search, ffmpeg to mono at speech sample rate in a compressed codec, split at silence, A hosted model with published limits, chunked under them, key in the environment, A durable queue with per-chunk checkpoints and a recorded cost per call, Generated per section, each claim verified against the transcript before it is shown, Media in a per-project directory beside the database, with a disk ceiling and a real delete. - Implement milestone by milestone from `MILESTONES.md`; keep each change reviewable and leave the application runnable at every commit. - Treat authentication, payments, encryption, imports, webhooks and destructive actions as high-risk boundaries when present. - Never invent cryptography or silently weaken a requirement to make a check pass. - Put every external service behind an interface with a deterministic fake for tests. - Add migrations and rollback or recovery notes for every persistent data change. - Log useful operational context without credentials, tokens, passwords or personal data. - Update documentation and run every check before completing a milestone. ## Known traps - Cutting the file at a fixed byte offset drops a word at every seam and, worse, each chunk's returned timestamps start at zero, so unless you add the chunk's own start offset back every timestamp after the first chunk is wrong by a growing amount and every link in the app lands in the wrong place. Cut at silence, store absolute offsets, and assert against a recording whose true times you know. - Treating transcription as one long call means a network blip, a rate limit or a laptop lid closing at minute ninety throws away every chunk already paid for, and the user retries the whole two hours. Checkpoint per chunk, persist before and after each call, and make resume the default path rather than an option. - Storing the transcript as one text blob and searching it gives you a hit that says the phrase is somewhere in a ninety-minute recording, which is the same as no answer; and rebuilding paragraphs for display without carrying the segment times means the clickable timestamp drifts from the words shown. Index per segment, and keep the time on every displayed line. - Passing a ninety-minute transcript in one request and asking for ten flashcards produces confident cards about things nobody said, because the model is compressing far too much and has no obligation to point at anything. Generate per section, demand a quote, verify the quote exists, and show a section as unsummarised rather than inventing a summary of it. - Deleting a project by marking a row deleted leaves gigabytes of video and every transcript line in the search index, so the disk fills and a search still surfaces text from a recording the person believes they removed. Delete the files, the rows and the index entries in one operation, then assert the search returns nothing for a phrase that was unique to that project. ===== MILESTONES.md ===== # Delivery milestones · VidNotes Estimated effort: **weekend** for the indie phases; the production-only milestones add the trust and operability layer. ## M1 · Audio the API will actually accept Any recording becomes chunks under the published size limit with offsets that still line up. ### Steps 1. Normalise to mono speech audio first Stereo at music sample rates doubles the bytes and adds nothing a transcriber uses. Files: `src/audio.py` ```sh ffmpeg -i input.mp4 -ac 1 -ar 16000 -c:a libopus prepared.opus ``` 2. Split at silence, not at a byte count Detect quiet spans near each boundary and cut there. Files: `src/chunking.py`, `tests/test_offsets.py` 3. Store absolute offsets and add them back Every returned timestamp is relative to its chunk until you fix it. Files: `src/chunking.py` ### Done when - [ ] Every chunk is under the published size limit - [ ] Late timestamps match true positions within tolerance - [ ] Cuts land in silence or are covered by overlap - [ ] Unsupported containers are converted, not refused ### Watch out - Cutting the file at a fixed byte offset drops a word at every seam and, worse, each chunk's returned timestamps start at zero, so unless you add the chunk's own start offset back every timestamp after the first chunk is wrong by a growing amount and every link in the app lands in the wrong place. Cut at silence, store absolute offsets, and assert against a recording whose true times you know. ## M2 · A long job that survives being interrupted A failure at chunk forty resumes at chunk forty, and you can see what it cost. ### Steps 1. One chunk, one state, one transaction Write the state before the call and the result after it. Files: `src/worker.py`, `tests/test_resume.py` 2. Record cost per call as it happens A three-hour lecture is an unpleasant surprise unless the number is on screen. Files: `src/worker.py`, `src/templates/project.html` 3. Back off on rate limits, retry only pre-response failures A timeout after the audio was accepted may already have been charged. Files: `src/transcribe.py` ### Done when - [ ] A kill mid-run resumes without repeating a chunk - [ ] Rate limits back off and complete - [ ] Cost is recorded per call and displayed - [ ] Cancelling leaves completed work intact ### Watch out - Treating transcription as one long call means a network blip, a rate limit or a laptop lid closing at minute ninety throws away every chunk already paid for, and the user retries the whole two hours. Checkpoint per chunk, persist before and after each call, and make resume the default path rather than an option. ## M3 · Search that lands on the moment A query returns spans with times, and clicking one moves the player there. ### Steps 1. Index segments, not transcripts The unit of search should be the unit you can seek to. Files: `src/search.py`, `tests/test_search.py` 2. Carry the time onto every rendered line Display grouping must never be allowed to lose it. Files: `src/templates/transcript.html` 3. Test names and numbers deliberately They are what people actually search a lecture for. Files: `tests/test_search.py` ### Done when - [ ] Phrase search returns timed segments - [ ] Clicking a result seeks accurately - [ ] Numbers and proper nouns are findable - [ ] Grouped lines keep their start times ### Watch out - Storing the transcript as one text blob and searching it gives you a hit that says the phrase is somewhere in a ninety-minute recording, which is the same as no answer; and rebuilding paragraphs for display without carrying the segment times means the clickable timestamp drifts from the words shown. Index per segment, and keep the time on every displayed line. ## M4 · Notes that can be checked against the recording Every generated claim points at a span that really says it, or it is not shown. ### Steps 1. Generate per section with a required quote A claim with nowhere to point is a claim you cannot check. Files: `src/notes.py` 2. Verify the quote by exact match, then resolve it Normalise whitespace and case, nothing more forgiving than that. Files: `src/verify.py`, `tests/test_verify.py` 3. Show gaps instead of filling them An honest blank is more useful than a plausible sentence. Files: `src/templates/notes.html` ### Done when - [ ] Unverifiable claims are dropped and counted - [ ] Every shown note resolves to a segment - [ ] Section regeneration is isolated - [ ] Empty sections are shown as empty ### Watch out - Passing a ninety-minute transcript in one request and asking for ten flashcards produces confident cards about things nobody said, because the model is compressing far too much and has no obligation to point at anything. Generate per section, demand a quote, verify the quote exists, and show a section as unsummarised rather than inventing a summary of it. ## M5 · Getting it out, and keeping the disk and the bill in hand (production only) Everything exports, deletion really deletes, and neither disk nor spend can run away unwatched. ### Steps 1. Export four formats with times preserved Subtitles and JSON are what other tools can read. Files: `src/export.py`, `tests/test_export.py` 2. Delete files, rows and index together, then assert Then search for the unique phrase and expect nothing. Files: `src/delete.py`, `tests/test_delete.py` 3. Cap disk and monthly spend, and refuse past them A refusal you can see beats a bill you cannot. Files: `src/limits.py` ### Done when - [ ] All four export formats keep timestamps - [ ] A deleted project leaves nothing findable - [ ] The disk ceiling refuses new imports - [ ] The spend cap refuses new transcription ### Watch out - Deleting a project by marking a row deleted leaves gigabytes of video and every transcript line in the search index, so the disk fills and a search still surfaces text from a recording the person believes they removed. Delete the files, the rows and the index entries in one operation, then assert the search returns nothing for a phrase that was unique to that project. ===== OPERATIONS.md ===== # Operations · VidNotes ## Backup The database and the media directory together, since one without the other leaves projects pointing at recordings that are gone. Transcripts and notes are small and worth backing up far more often than the media, which the person usually still has. ## Restore Restore both, then run a consistency pass that marks any project whose media is missing rather than failing on open. A restored job in mid-transcription resumes from its completed chunks, which is the same path as a crash. Do a restore drill before the first real user, and write the date here when it passes. ## Monitoring For a local tool this lives in the interface: jobs stuck in a stage, chunks with repeated attempts, month-to-date spend against the cap, and media directory usage against the ceiling. ## Incident checklist Wrong timestamps mean the chunk offset arithmetic broke and the test against the known recording is the first thing to run. A surprising bill means a job retried a chunk that had already been charged, and the per-chunk cost rows show which. 1. Contain the issue without destroying evidence or user data. 2. Record the timeline and affected scope. 3. Rotate exposed secrets and revoke compromised sessions or credentials. 4. Restore from a verified backup when needed. 5. Document the root cause, the remediation and the regression test. ## Release gate - [ ] Late-chunk timestamps are confirmed against a known recording - [ ] A kill mid-run is confirmed to resume without repeating a chunk - [ ] An unverifiable generated claim is confirmed dropped - [ ] A deleted project is confirmed to leave nothing findable - [ ] The monthly spend cap is confirmed to refuse new work ## Launch constraint Do not market omitted VidNotes capabilities as implemented. The non-goals in `PRODUCT.md` remain user-visible limitations until they are deliberately delivered. ===== .env.example ===== # Copy to .env and fill in. Never commit .env; this file documents it. # Required. Bound to localhost only. This is a personal tool. APP_PORT=8787 # Required. Projects, segments, notes and the job state. DATABASE_PATH=./data/vidnotes.db # Required. One directory per project. Deleted in full when a project is deleted. MEDIA_DIR=./data/projects # Required. The disk ceiling. Imports past it are refused with usage shown. MEDIA_MAX_BYTES=53687091200 # Required · secret. Environment only. Never in the database, never in a log line. TRANSCRIBE_API_KEY=sk-...redacted # Required. Set below the provider's published limit, which you check rather than assume. CHUNK_MAX_BYTES=24000000 # Required. New transcription is refused past this rather than billed through it. MONTHLY_SPEND_CAP_CENTS=2000 # Required. Off by default. Turning it on is a statement about the source's terms. ENABLE_URL_IMPORT=false
README.md
# VidNotes · indie build A local workspace turning long recordings into timestamped transcripts and study material, whose hard problems are cutting audio into pieces a transcription API will accept without losing words at the seams, surviving a two-hour job that fails at minute ninety, and never showing a generated claim that cannot be found in the recording. Estimated effort: **weekend**. Work `BUILD_PLAN.md` top to bottom · every phase ends in a check that has to pass before the next one starts. ## Stack | Part | Choice | Why | | --- | --- | --- | | Runtime | One Python process serving localhost with a background worker, SQLite with full-text search | a study tool is not a service and should not need one | | Audio | ffmpeg to mono at speech sample rate in a compressed codec, split at silence | the input is a two-hour video and the API takes megabytes | | Transcription | A hosted model with published limits, chunked under them, key in the environment | local models are a fine later swap and a poor first dependency | | Jobs | A durable queue with per-chunk checkpoints and a recorded cost per call | a long job that restarts from zero costs money twice | | Notes | Generated per section, each claim verified against the transcript before it is shown | an unverified flashcard is a wrong answer with a confident face | | Storage | Media in a per-project directory beside the database, with a disk ceiling and a real delete | video is large and nobody remembers what they imported in March | ## Before you start Have every one of these ready. The plan assumes them from step one. - [ ] **Python 3.12 or newer** · free - Why: Runs the server and the worker. - Get it: python.org or your package manager. - Verify: python3 --version prints 3.12 or higher - [ ] **ffmpeg with an Opus or MP3 encoder** · free - Why: Extracts and compresses the audio, and a build without the encoder you chose fails only at the first real import. - Get it: Your package manager, or a static build from ffmpeg.org. - Verify: ffmpeg -encoders lists the encoder you intend to use - [ ] **An API key for a transcription and text model** · per minute of audio, a few pence per hour of recording - Why: Transcription is the one part not worth writing, and the key is also the thing that can quietly cost money. - Get it: Create a key with its own spending limit rather than reusing an unlimited one. - Verify: A one-minute sample returns timestamped text and the call appears on the usage page - [ ] **A recording whose true timings you know** · free - Why: Chunk offsets and seek accuracy cannot be tested against a recording nobody has listened to. - Get it: Record two minutes saying a distinct word at known times, or annotate a lecture you own. - Verify: The known words appear in the transcript at the times you spoke them - [ ] **Your disk ceiling, spend cap and import policy** · free - Why: Each is enforced by code and asserted by a test, and deciding them later means accepting whatever the first month produced. - Get it: Write down the media directory limit, the monthly spend cap, and whether the downloader is enabled and on what basis. - Verify: All three appear in the configuration and in the README ## Quick start ```sh ffmpeg -i input.mp4 -ac 1 -ar 16000 -c:a libopus prepared.opus ``` Then copy `.env.example` to `.env` and fill in the values it documents. ## Honest limits This build deliberately does not replace: - Native mobile apps and a browser extension. - Extraction from sites whose terms forbid it. - Accounts and cross-device sync. - A public API, CLI or agent server. - native iOS and Android apps, a Chrome extension, and cross-device Premium state - managed extraction for TikTok, Instagram, Vimeo, and changing source sites - background processing, queue reliability, and long-video handling - 30+ language polish and prebuilt flashcards, quizzes, and chat - hosted web, API, CLI, and MCP access for agents If one of those is essential to you, that is the reason to keep paying for VidNotes, and the README should say so rather than pretend.
BRIEF.md
# Build brief · VidNotes The one-shot brief this plan expands. `BUILD_PLAN.md` (or `MILESTONES.md`) is the same sequence broken into steps and checks; where the two disagree, the plan wins. Build me a local transcription and study-notes workspace for recordings you already hold to replace VidNotes. Build it in phases, in the order below. Do not write the whole thing in one pass. Finish a phase, run its "Done when" check, fix what fails, and only then start the next phase. ### Rule zero Two rules. The first is about what comes in, the second about what goes out. The default import is a file on your disk. A downloader pointed at a video site is a separate, off-by-default switch with the site's terms quoted beside it, because YouTube's terms forbid accessing, reproducing or downloading any part of the Service or its Content except as expressly authorised by the Service or with prior written permission (checked). Build the switch if you have that permission or the licence allows it, and make the app fully useful without it. Second, generated text never stands in for the recording. Every summary bullet, flashcard and quiz answer carries a timestamp and a span that exists verbatim in the transcript, and anything the verifier cannot locate is dropped with the gap shown rather than displayed anyway. Someone revising from a confidently invented date is worse off than someone with no notes at all. ### Stack (fixed, do not substitute) - Runtime: One Python process serving localhost with a background worker, SQLite with full-text search · a study tool is not a service and should not need one - Audio: ffmpeg to mono at speech sample rate in a compressed codec, split at silence · the input is a two-hour video and the API takes megabytes - Transcription: A hosted model with published limits, chunked under them, key in the environment · local models are a fine later swap and a poor first dependency - Jobs: A durable queue with per-chunk checkpoints and a recorded cost per call · a long job that restarts from zero costs money twice - Notes: Generated per section, each claim verified against the transcript before it is shown · an unverified flashcard is a wrong answer with a confident face - Storage: Media in a per-project directory beside the database, with a disk ceiling and a real delete · video is large and nobody remembers what they imported in March ### Data model (create this before Phase 1) - projects(id, title, source_kind, source_ref, duration_ms, created_at, deleted_at) · source_kind is file or url and source_ref is kept so you know where it came from - media(project_id, path, sha256, bytes, codec, sample_rate, channels, prepared_path) - chunks(id, project_id, ordinal, start_ms, end_ms, path, bytes, state, attempts, cost_cents, transcribed_at) · offsets are absolute in the original recording - segments(id, project_id, chunk_id, start_ms, end_ms, text) · one row per timed span, and the only thing search ever reads - notes(id, project_id, kind, section_start_ms, section_end_ms, content, quote, quote_segment_id, generated_at) · kind is summary, key_point, flashcard or quiz - jobs(id, project_id, stage, state, attempts, last_error, updated_at) · stage is extracting, chunking, transcribing or generating - A note without a resolvable quote_segment_id is never displayed, which makes the join the honesty check rather than a nicety. ### Phase 1 · Audio the API will actually accept Build: Extract audio with ffmpeg to mono at a speech sample rate in a compressed codec. Check the provider's published upload limit before choosing a chunk size: the widely used hosted API documents files up to 25 MB, lists mp3, mp4, mpeg, mpga, m4a, wav and webm, and says to compress or split into chunks of 25 MB or less while avoiding a split in the middle of a sentence (checked). Split at a silence near each boundary, store each chunk's absolute start offset, and overlap slightly so a word spanning a cut appears in one of the two. Trap: Cutting the file at a fixed byte offset drops a word at every seam and, worse, each chunk's returned timestamps start at zero, so unless you add the chunk's own start offset back every timestamp after the first chunk is wrong by a growing amount and every link in the app lands in the wrong place. Cut at silence, store absolute offsets, and assert against a recording whose true times you know. Done when: A two-hour recording produces chunks each under the published limit, verified by file size. Timestamps in the last chunk match the true position in the original within a stated tolerance, tested against a recording with known spoken markers. Every cut falls in a silent span, or the overlap covers the word, checked on a dense-speech sample. A file in an unsupported container is converted before upload rather than rejected at the API. Do not build yet: No job queue, notes, search or export. ### Phase 2 · A long job that survives being interrupted Build: Give every chunk its own state and attempt count. The worker takes one chunk at a time, records the call's cost, and retries only failures that could not have produced a transcript. On restart, chunks already transcribed are skipped. Show progress as chunks completed out of total, with the running cost, and let a project be cancelled mid-run without corrupting what it has. Trap: Treating transcription as one long call means a network blip, a rate limit or a laptop lid closing at minute ninety throws away every chunk already paid for, and the user retries the whole two hours. Checkpoint per chunk, persist before and after each call, and make resume the default path rather than an option. Done when: Killing the process mid-transcription and restarting resumes at the first unfinished chunk with no chunk transcribed twice, tested. A rate-limited call backs off and eventually succeeds without duplicating a chunk. The per-project cost is recorded from actual calls and shown before and after the run. Cancelling leaves completed chunks intact and the project resumable. Do not build yet: No search, notes or export yet. ### Phase 3 · Search that lands on the moment Build: Index segments rather than whole transcripts in full-text search. Return matches as a list of timed spans with surrounding context, and make each one seek the player. Group segments into readable paragraphs for display without losing the underlying times. Handle a query for a name or a number, which naive tokenisation tends to lose. Trap: Storing the transcript as one text blob and searching it gives you a hit that says the phrase is somewhere in a ninety-minute recording, which is the same as no answer; and rebuilding paragraphs for display without carrying the segment times means the clickable timestamp drifts from the words shown. Index per segment, and keep the time on every displayed line. Done when: A search for a phrase returns the segments containing it with their times, tested against a known transcript. Clicking a result seeks the player to within a stated tolerance of the spoken phrase. A query containing a number or a proper noun returns the right segment, tested explicitly. Paragraph grouping preserves the start time of each displayed line. Do not build yet: No generated notes or export yet. ### Phase 4 · Notes that can be checked against the recording Build: Generate summaries, key points, flashcards and quiz questions per section rather than over the whole transcript, asking for a verbatim quote with each claim. Verify each quote by exact match against the segment text after normalising whitespace and case, resolve it to a segment id, and drop anything that fails with the gap shown. Offer regeneration per section. Trap: Passing a ninety-minute transcript in one request and asking for ten flashcards produces confident cards about things nobody said, because the model is compressing far too much and has no obligation to point at anything. Generate per section, demand a quote, verify the quote exists, and show a section as unsummarised rather than inventing a summary of it. Done when: A card whose quote cannot be found in any segment is dropped and counted, tested by feeding a deliberately wrong quote. Every displayed note resolves to a segment id and its timestamp is clickable. Regenerating one section leaves other sections untouched, tested. A section that produces nothing verifiable is shown as such rather than filled in. Do not build yet: No export, deletion flow or ceilings yet. ### Phase 5 · Getting it out, and keeping the disk and the bill in hand Build: Export the transcript and notes as Markdown, plain text, subtitles and JSON, with timestamps preserved in each. Make deleting a project remove its media, its rows, its search index entries and its chunk files, then prove it. Set a disk ceiling for the media directory and a monthly spend cap that refuses new transcription rather than exceeding it. Keep the API key in the environment and never in the database or a log. Trap: Deleting a project by marking a row deleted leaves gigabytes of video and every transcript line in the search index, so the disk fills and a search still surfaces text from a recording the person believes they removed. Delete the files, the rows and the index entries in one operation, then assert the search returns nothing for a phrase that was unique to that project. Done when: Export produces all four formats with timestamps intact, opened and checked in another tool. After deleting a project, a phrase unique to it returns no search results and its directory is gone, both asserted in a test. Importing past the disk ceiling is refused with the current usage shown. Passing the monthly cap refuses new transcription and says so, tested by lowering the cap. Do not build yet: Nothing after this. ### Out of scope (and why) - Native mobile applications and a browser extension, each of which is its own release process and its own way for results to disagree with the desktop. - Extraction from sites whose terms forbid it, which is the feature that gets the tool taken down rather than the feature that makes it good. - Cross-device sync and accounts, which turn a local tool into a service with other people's recordings in it. - A public API, command line and agent server, which are a stable interface promise on top of a personal tool. ### README must contain - That the default import is a local file, and that any downloader is off by default with the source site's terms quoted beside it - That every generated claim is verified against a transcript span and dropped when it cannot be found - How chunking works, that offsets are absolute, and how to check timestamps against a known recording - Where media lives, what deleting a project actually removes, and how that is tested - That transcription costs money per recording, where the running total is shown, and how the monthly cap behaves
AGENTS.md
# Agent instructions · VidNotes indie build - Read `README.md` and `BUILD_PLAN.md` before writing code. The stack is fixed: One Python process serving localhost with a background worker, SQLite with full-text search, ffmpeg to mono at speech sample rate in a compressed codec, split at silence, A hosted model with published limits, chunked under them, key in the environment, A durable queue with per-chunk checkpoints and a recorded cost per call, Generated per section, each claim verified against the transcript before it is shown, Media in a per-project directory beside the database, with a disk ceiling and a real delete. Do not substitute. - Work one phase at a time, in order. Do not start a phase until every "Done when" item of the previous one passes. - Prefer the fewest moving parts that satisfy the step. No frameworks, services or dependencies the plan does not name. - Secrets live in `.env`, never in source or logs. Keep `.env.example` current when a variable is introduced. - Do not invent cryptography, security guarantees, APIs or compliance claims. - Add a focused test for every destructive, security-sensitive or data-loss path the plan names. - Run the project checks before declaring a phase complete, and record any deliberate shortcut in the README under "Tradeoffs". ## Known traps - Cutting the file at a fixed byte offset drops a word at every seam and, worse, each chunk's returned timestamps start at zero, so unless you add the chunk's own start offset back every timestamp after the first chunk is wrong by a growing amount and every link in the app lands in the wrong place. Cut at silence, store absolute offsets, and assert against a recording whose true times you know. - Treating transcription as one long call means a network blip, a rate limit or a laptop lid closing at minute ninety throws away every chunk already paid for, and the user retries the whole two hours. Checkpoint per chunk, persist before and after each call, and make resume the default path rather than an option. - Storing the transcript as one text blob and searching it gives you a hit that says the phrase is somewhere in a ninety-minute recording, which is the same as no answer; and rebuilding paragraphs for display without carrying the segment times means the clickable timestamp drifts from the words shown. Index per segment, and keep the time on every displayed line. - Passing a ninety-minute transcript in one request and asking for ten flashcards produces confident cards about things nobody said, because the model is compressing far too much and has no obligation to point at anything. Generate per section, demand a quote, verify the quote exists, and show a section as unsummarised rather than inventing a summary of it.
BUILD_PLAN.md
# Build plan · VidNotes A local workspace turning long recordings into timestamped transcripts and study material, whose hard problems are cutting audio into pieces a transcription API will accept without losing words at the seams, surviving a two-hour job that fails at minute ninety, and never showing a generated claim that cannot be found in the recording. Phases are in dependency order. Each ends in a "Done when" list; treat an unticked item as a blocker, not a note. ## Phase 1 · Audio the API will actually accept Any recording becomes chunks under the published size limit with offsets that still line up. ### Steps 1. Normalise to mono speech audio first Stereo at music sample rates doubles the bytes and adds nothing a transcriber uses. Files: `src/audio.py` ```sh ffmpeg -i input.mp4 -ac 1 -ar 16000 -c:a libopus prepared.opus ``` 2. Split at silence, not at a byte count Detect quiet spans near each boundary and cut there. Files: `src/chunking.py`, `tests/test_offsets.py` 3. Store absolute offsets and add them back Every returned timestamp is relative to its chunk until you fix it. Files: `src/chunking.py` ### Done when - [ ] Every chunk is under the published size limit - [ ] Late timestamps match true positions within tolerance - [ ] Cuts land in silence or are covered by overlap - [ ] Unsupported containers are converted, not refused ### Watch out - Cutting the file at a fixed byte offset drops a word at every seam and, worse, each chunk's returned timestamps start at zero, so unless you add the chunk's own start offset back every timestamp after the first chunk is wrong by a growing amount and every link in the app lands in the wrong place. Cut at silence, store absolute offsets, and assert against a recording whose true times you know. ## Phase 2 · A long job that survives being interrupted A failure at chunk forty resumes at chunk forty, and you can see what it cost. ### Steps 1. One chunk, one state, one transaction Write the state before the call and the result after it. Files: `src/worker.py`, `tests/test_resume.py` 2. Record cost per call as it happens A three-hour lecture is an unpleasant surprise unless the number is on screen. Files: `src/worker.py`, `src/templates/project.html` 3. Back off on rate limits, retry only pre-response failures A timeout after the audio was accepted may already have been charged. Files: `src/transcribe.py` ### Done when - [ ] A kill mid-run resumes without repeating a chunk - [ ] Rate limits back off and complete - [ ] Cost is recorded per call and displayed - [ ] Cancelling leaves completed work intact ### Watch out - Treating transcription as one long call means a network blip, a rate limit or a laptop lid closing at minute ninety throws away every chunk already paid for, and the user retries the whole two hours. Checkpoint per chunk, persist before and after each call, and make resume the default path rather than an option. ## Phase 3 · Search that lands on the moment A query returns spans with times, and clicking one moves the player there. ### Steps 1. Index segments, not transcripts The unit of search should be the unit you can seek to. Files: `src/search.py`, `tests/test_search.py` 2. Carry the time onto every rendered line Display grouping must never be allowed to lose it. Files: `src/templates/transcript.html` 3. Test names and numbers deliberately They are what people actually search a lecture for. Files: `tests/test_search.py` ### Done when - [ ] Phrase search returns timed segments - [ ] Clicking a result seeks accurately - [ ] Numbers and proper nouns are findable - [ ] Grouped lines keep their start times ### Watch out - Storing the transcript as one text blob and searching it gives you a hit that says the phrase is somewhere in a ninety-minute recording, which is the same as no answer; and rebuilding paragraphs for display without carrying the segment times means the clickable timestamp drifts from the words shown. Index per segment, and keep the time on every displayed line. ## Phase 4 · Notes that can be checked against the recording Every generated claim points at a span that really says it, or it is not shown. ### Steps 1. Generate per section with a required quote A claim with nowhere to point is a claim you cannot check. Files: `src/notes.py` 2. Verify the quote by exact match, then resolve it Normalise whitespace and case, nothing more forgiving than that. Files: `src/verify.py`, `tests/test_verify.py` 3. Show gaps instead of filling them An honest blank is more useful than a plausible sentence. Files: `src/templates/notes.html` ### Done when - [ ] Unverifiable claims are dropped and counted - [ ] Every shown note resolves to a segment - [ ] Section regeneration is isolated - [ ] Empty sections are shown as empty ### Watch out - Passing a ninety-minute transcript in one request and asking for ten flashcards produces confident cards about things nobody said, because the model is compressing far too much and has no obligation to point at anything. Generate per section, demand a quote, verify the quote exists, and show a section as unsummarised rather than inventing a summary of it. ## Not in this build - Native mobile apps and a browser extension. - Extraction from sites whose terms forbid it. - Accounts and cross-device sync. - A public API, CLI or agent server. ## After v1, if you want it - A local transcription model as a second backend, chosen per project, once the chunking is proven - Speaker labelling, which changes the segment table and should not be retrofitted casually - A revision mode that schedules flashcards over time, which needs nothing new from the pipeline
.env.example
# Copy to .env and fill in. Never commit .env; this file documents it. # Required. Bound to localhost only. This is a personal tool. APP_PORT=8787 # Required. Projects, segments, notes and the job state. DATABASE_PATH=./data/vidnotes.db # Required. One directory per project. Deleted in full when a project is deleted. MEDIA_DIR=./data/projects # Required. The disk ceiling. Imports past it are refused with usage shown. MEDIA_MAX_BYTES=53687091200 # Required · secret. Environment only. Never in the database, never in a log line. TRANSCRIBE_API_KEY=sk-...redacted # Required. Set below the provider's published limit, which you check rather than assume. CHUNK_MAX_BYTES=24000000 # Required. New transcription is refused past this rather than billed through it. MONTHLY_SPEND_CAP_CENTS=2000 # Required. Off by default. Turning it on is a statement about the source's terms. ENABLE_URL_IMPORT=false
PRODUCT.md
# VidNotes · product brief ## Problem The narrow DIY loop is achievable in a weekend: import a file or YouTube URL, transcribe it, generate notes, and save the result locally. VidNotes as a product is broader than that prototype. It maintains ingestion across changing video sites, runs long jobs reliably, ships iOS, Android, web, and a Chrome extension, keeps account state consistent, and exposes the workflow to agents through an API, CLI, and MCP server. Rebuilding all of that is a multi-day integration project with an ongoing operations burden. ## Product outcome A workspace that turns a long recording into a searchable timestamped transcript and study material where every claim can be checked against what was actually said, resumes a failed job instead of paying twice, and deletes what you tell it to delete. ## Target user A builder who needs a maintainable product foundation, not a one-off demo. ## Required capabilities - Python 3.12 - ffmpeg - yt-dlp - OpenAI API key - local disk ## Explicit non-goals for v1 - Native mobile apps and a browser extension. - Extraction from sites whose terms forbid it. - Accounts and cross-device sync. - A public API, CLI or agent server. - native iOS and Android apps, a Chrome extension, and cross-device Premium state - managed extraction for TikTok, Instagram, Vimeo, and changing source sites - background processing, queue reliability, and long-video handling - 30+ language polish and prebuilt flashcards, quizzes, and chat - hosted web, API, CLI, and MCP access for agents ## Success criteria - Late-chunk timestamps are confirmed against a known recording - A kill mid-run is confirmed to resume without repeating a chunk - An unverifiable generated claim is confirmed dropped - A deleted project is confirmed to leave nothing findable - The monthly spend cap is confirmed to refuse new work
BRIEF.md
# Build brief · VidNotes The one-shot brief this plan expands. `BUILD_PLAN.md` (or `MILESTONES.md`) is the same sequence broken into steps and checks; where the two disagree, the plan wins. Build me a local transcription and study-notes workspace for recordings you already hold to replace VidNotes. Build it in phases, in the order below. Do not write the whole thing in one pass. Finish a phase, run its "Done when" check, fix what fails, and only then start the next phase. ### Rule zero Two rules. The first is about what comes in, the second about what goes out. The default import is a file on your disk. A downloader pointed at a video site is a separate, off-by-default switch with the site's terms quoted beside it, because YouTube's terms forbid accessing, reproducing or downloading any part of the Service or its Content except as expressly authorised by the Service or with prior written permission (checked). Build the switch if you have that permission or the licence allows it, and make the app fully useful without it. Second, generated text never stands in for the recording. Every summary bullet, flashcard and quiz answer carries a timestamp and a span that exists verbatim in the transcript, and anything the verifier cannot locate is dropped with the gap shown rather than displayed anyway. Someone revising from a confidently invented date is worse off than someone with no notes at all. ### Stack (fixed, do not substitute) - Runtime: One Python process serving localhost with a background worker, SQLite with full-text search · a study tool is not a service and should not need one - Audio: ffmpeg to mono at speech sample rate in a compressed codec, split at silence · the input is a two-hour video and the API takes megabytes - Transcription: A hosted model with published limits, chunked under them, key in the environment · local models are a fine later swap and a poor first dependency - Jobs: A durable queue with per-chunk checkpoints and a recorded cost per call · a long job that restarts from zero costs money twice - Notes: Generated per section, each claim verified against the transcript before it is shown · an unverified flashcard is a wrong answer with a confident face - Storage: Media in a per-project directory beside the database, with a disk ceiling and a real delete · video is large and nobody remembers what they imported in March ### Data model (create this before Phase 1) - projects(id, title, source_kind, source_ref, duration_ms, created_at, deleted_at) · source_kind is file or url and source_ref is kept so you know where it came from - media(project_id, path, sha256, bytes, codec, sample_rate, channels, prepared_path) - chunks(id, project_id, ordinal, start_ms, end_ms, path, bytes, state, attempts, cost_cents, transcribed_at) · offsets are absolute in the original recording - segments(id, project_id, chunk_id, start_ms, end_ms, text) · one row per timed span, and the only thing search ever reads - notes(id, project_id, kind, section_start_ms, section_end_ms, content, quote, quote_segment_id, generated_at) · kind is summary, key_point, flashcard or quiz - jobs(id, project_id, stage, state, attempts, last_error, updated_at) · stage is extracting, chunking, transcribing or generating - A note without a resolvable quote_segment_id is never displayed, which makes the join the honesty check rather than a nicety. ### Phase 1 · Audio the API will actually accept Build: Extract audio with ffmpeg to mono at a speech sample rate in a compressed codec. Check the provider's published upload limit before choosing a chunk size: the widely used hosted API documents files up to 25 MB, lists mp3, mp4, mpeg, mpga, m4a, wav and webm, and says to compress or split into chunks of 25 MB or less while avoiding a split in the middle of a sentence (checked). Split at a silence near each boundary, store each chunk's absolute start offset, and overlap slightly so a word spanning a cut appears in one of the two. Trap: Cutting the file at a fixed byte offset drops a word at every seam and, worse, each chunk's returned timestamps start at zero, so unless you add the chunk's own start offset back every timestamp after the first chunk is wrong by a growing amount and every link in the app lands in the wrong place. Cut at silence, store absolute offsets, and assert against a recording whose true times you know. Done when: A two-hour recording produces chunks each under the published limit, verified by file size. Timestamps in the last chunk match the true position in the original within a stated tolerance, tested against a recording with known spoken markers. Every cut falls in a silent span, or the overlap covers the word, checked on a dense-speech sample. A file in an unsupported container is converted before upload rather than rejected at the API. Do not build yet: No job queue, notes, search or export. ### Phase 2 · A long job that survives being interrupted Build: Give every chunk its own state and attempt count. The worker takes one chunk at a time, records the call's cost, and retries only failures that could not have produced a transcript. On restart, chunks already transcribed are skipped. Show progress as chunks completed out of total, with the running cost, and let a project be cancelled mid-run without corrupting what it has. Trap: Treating transcription as one long call means a network blip, a rate limit or a laptop lid closing at minute ninety throws away every chunk already paid for, and the user retries the whole two hours. Checkpoint per chunk, persist before and after each call, and make resume the default path rather than an option. Done when: Killing the process mid-transcription and restarting resumes at the first unfinished chunk with no chunk transcribed twice, tested. A rate-limited call backs off and eventually succeeds without duplicating a chunk. The per-project cost is recorded from actual calls and shown before and after the run. Cancelling leaves completed chunks intact and the project resumable. Do not build yet: No search, notes or export yet. ### Phase 3 · Search that lands on the moment Build: Index segments rather than whole transcripts in full-text search. Return matches as a list of timed spans with surrounding context, and make each one seek the player. Group segments into readable paragraphs for display without losing the underlying times. Handle a query for a name or a number, which naive tokenisation tends to lose. Trap: Storing the transcript as one text blob and searching it gives you a hit that says the phrase is somewhere in a ninety-minute recording, which is the same as no answer; and rebuilding paragraphs for display without carrying the segment times means the clickable timestamp drifts from the words shown. Index per segment, and keep the time on every displayed line. Done when: A search for a phrase returns the segments containing it with their times, tested against a known transcript. Clicking a result seeks the player to within a stated tolerance of the spoken phrase. A query containing a number or a proper noun returns the right segment, tested explicitly. Paragraph grouping preserves the start time of each displayed line. Do not build yet: No generated notes or export yet. ### Phase 4 · Notes that can be checked against the recording Build: Generate summaries, key points, flashcards and quiz questions per section rather than over the whole transcript, asking for a verbatim quote with each claim. Verify each quote by exact match against the segment text after normalising whitespace and case, resolve it to a segment id, and drop anything that fails with the gap shown. Offer regeneration per section. Trap: Passing a ninety-minute transcript in one request and asking for ten flashcards produces confident cards about things nobody said, because the model is compressing far too much and has no obligation to point at anything. Generate per section, demand a quote, verify the quote exists, and show a section as unsummarised rather than inventing a summary of it. Done when: A card whose quote cannot be found in any segment is dropped and counted, tested by feeding a deliberately wrong quote. Every displayed note resolves to a segment id and its timestamp is clickable. Regenerating one section leaves other sections untouched, tested. A section that produces nothing verifiable is shown as such rather than filled in. Do not build yet: No export, deletion flow or ceilings yet. ### Phase 5 · Getting it out, and keeping the disk and the bill in hand Build: Export the transcript and notes as Markdown, plain text, subtitles and JSON, with timestamps preserved in each. Make deleting a project remove its media, its rows, its search index entries and its chunk files, then prove it. Set a disk ceiling for the media directory and a monthly spend cap that refuses new transcription rather than exceeding it. Keep the API key in the environment and never in the database or a log. Trap: Deleting a project by marking a row deleted leaves gigabytes of video and every transcript line in the search index, so the disk fills and a search still surfaces text from a recording the person believes they removed. Delete the files, the rows and the index entries in one operation, then assert the search returns nothing for a phrase that was unique to that project. Done when: Export produces all four formats with timestamps intact, opened and checked in another tool. After deleting a project, a phrase unique to it returns no search results and its directory is gone, both asserted in a test. Importing past the disk ceiling is refused with the current usage shown. Passing the monthly cap refuses new transcription and says so, tested by lowering the cap. Do not build yet: Nothing after this. ### Out of scope (and why) - Native mobile applications and a browser extension, each of which is its own release process and its own way for results to disagree with the desktop. - Extraction from sites whose terms forbid it, which is the feature that gets the tool taken down rather than the feature that makes it good. - Cross-device sync and accounts, which turn a local tool into a service with other people's recordings in it. - A public API, command line and agent server, which are a stable interface promise on top of a personal tool. ### README must contain - That the default import is a local file, and that any downloader is off by default with the source site's terms quoted beside it - That every generated claim is verified against a transcript span and dropped when it cannot be found - How chunking works, that offsets are absolute, and how to check timestamps against a known recording - Where media lives, what deleting a project actually removes, and how that is tested - That transcription costs money per recording, where the running total is shown, and how the monthly cap behaves
ARCHITECTURE.md
# Architecture · VidNotes ## Stack | Part | Choice | Why | | --- | --- | --- | | Runtime | One Python process serving localhost with a background worker, SQLite with full-text search | a study tool is not a service and should not need one | | Audio | ffmpeg to mono at speech sample rate in a compressed codec, split at silence | the input is a two-hour video and the API takes megabytes | | Transcription | A hosted model with published limits, chunked under them, key in the environment | local models are a fine later swap and a poor first dependency | | Jobs | A durable queue with per-chunk checkpoints and a recorded cost per call | a long job that restarts from zero costs money twice | | Notes | Generated per section, each claim verified against the transcript before it is shown | an unverified flashcard is a wrong answer with a confident face | | Storage | Media in a per-project directory beside the database, with a disk ceiling and a real delete | video is large and nobody remembers what they imported in March | ## Modules Each module has one owner concern and a documented way to replace it. | Module | Owns | How to replace it | | --- | --- | --- | | Audio | Normalisation, silence-aware chunking, absolute offsets. | Any encoder, provided offsets are still asserted against a known recording. | | Worker | Per-chunk state, retries, cost recording. | Any queue, provided resume remains the default path. | | Search | Segment-level indexing and seeking. | Any index, provided the unit stays the segment. | | Notes | Per-section generation and quote verification. | Any model, provided the verifier still drops what it cannot locate. | | Lifecycle | Export, deletion, disk and spend ceilings. | Not swappable for soft deletion, which leaves the media and the index behind. | ## Configuration Every runtime setting is an environment variable documented in `.env.example`, validated at startup, with a safe local default wherever one exists. - `APP_PORT` · required · Bound to localhost only. This is a personal tool. - `DATABASE_PATH` · required · Projects, segments, notes and the job state. - `MEDIA_DIR` · required · One directory per project. Deleted in full when a project is deleted. - `MEDIA_MAX_BYTES` · required · The disk ceiling. Imports past it are refused with usage shown. - `TRANSCRIBE_API_KEY` · required, secret · Environment only. Never in the database, never in a log line. - `CHUNK_MAX_BYTES` · required · Set below the provider's published limit, which you check rather than assume. - `MONTHLY_SPEND_CAP_CENTS` · required · New transcription is refused past this rather than billed through it. - `ENABLE_URL_IMPORT` · required · Off by default. Turning it on is a statement about the source's terms. ## Production baseline - Security: least privilege, input validation at every boundary, secret redaction in logs, rate limits on abuse-prone paths, no invented security primitives. - Data: explicit schema and migrations, transactional writes where integrity matters, backup and restore procedures that have been exercised. - Integrations: adapters around third-party providers, idempotent webhook or job processing, bounded retries, timeouts. - Observability: structured logs with request or operation ids, an error-tracking hook, and health and readiness checks where a server exists. - Quality: unit tests for domain rules, integration tests at module boundaries, one end-to-end test of the critical path. ## Decision records For each dependency in the stack table, keep a short note: why it was chosen, its failure mode, and how it is replaced. Do not add infrastructure until a requirement in `PRODUCT.md` justifies it.
AGENTS.md
# Agent instructions · VidNotes product build - Read `PRODUCT.md` and `ARCHITECTURE.md` before changing code. The stack is fixed: One Python process serving localhost with a background worker, SQLite with full-text search, ffmpeg to mono at speech sample rate in a compressed codec, split at silence, A hosted model with published limits, chunked under them, key in the environment, A durable queue with per-chunk checkpoints and a recorded cost per call, Generated per section, each claim verified against the transcript before it is shown, Media in a per-project directory beside the database, with a disk ceiling and a real delete. - Implement milestone by milestone from `MILESTONES.md`; keep each change reviewable and leave the application runnable at every commit. - Treat authentication, payments, encryption, imports, webhooks and destructive actions as high-risk boundaries when present. - Never invent cryptography or silently weaken a requirement to make a check pass. - Put every external service behind an interface with a deterministic fake for tests. - Add migrations and rollback or recovery notes for every persistent data change. - Log useful operational context without credentials, tokens, passwords or personal data. - Update documentation and run every check before completing a milestone. ## Known traps - Cutting the file at a fixed byte offset drops a word at every seam and, worse, each chunk's returned timestamps start at zero, so unless you add the chunk's own start offset back every timestamp after the first chunk is wrong by a growing amount and every link in the app lands in the wrong place. Cut at silence, store absolute offsets, and assert against a recording whose true times you know. - Treating transcription as one long call means a network blip, a rate limit or a laptop lid closing at minute ninety throws away every chunk already paid for, and the user retries the whole two hours. Checkpoint per chunk, persist before and after each call, and make resume the default path rather than an option. - Storing the transcript as one text blob and searching it gives you a hit that says the phrase is somewhere in a ninety-minute recording, which is the same as no answer; and rebuilding paragraphs for display without carrying the segment times means the clickable timestamp drifts from the words shown. Index per segment, and keep the time on every displayed line. - Passing a ninety-minute transcript in one request and asking for ten flashcards produces confident cards about things nobody said, because the model is compressing far too much and has no obligation to point at anything. Generate per section, demand a quote, verify the quote exists, and show a section as unsummarised rather than inventing a summary of it. - Deleting a project by marking a row deleted leaves gigabytes of video and every transcript line in the search index, so the disk fills and a search still surfaces text from a recording the person believes they removed. Delete the files, the rows and the index entries in one operation, then assert the search returns nothing for a phrase that was unique to that project.
MILESTONES.md
# Delivery milestones · VidNotes Estimated effort: **weekend** for the indie phases; the production-only milestones add the trust and operability layer. ## M1 · Audio the API will actually accept Any recording becomes chunks under the published size limit with offsets that still line up. ### Steps 1. Normalise to mono speech audio first Stereo at music sample rates doubles the bytes and adds nothing a transcriber uses. Files: `src/audio.py` ```sh ffmpeg -i input.mp4 -ac 1 -ar 16000 -c:a libopus prepared.opus ``` 2. Split at silence, not at a byte count Detect quiet spans near each boundary and cut there. Files: `src/chunking.py`, `tests/test_offsets.py` 3. Store absolute offsets and add them back Every returned timestamp is relative to its chunk until you fix it. Files: `src/chunking.py` ### Done when - [ ] Every chunk is under the published size limit - [ ] Late timestamps match true positions within tolerance - [ ] Cuts land in silence or are covered by overlap - [ ] Unsupported containers are converted, not refused ### Watch out - Cutting the file at a fixed byte offset drops a word at every seam and, worse, each chunk's returned timestamps start at zero, so unless you add the chunk's own start offset back every timestamp after the first chunk is wrong by a growing amount and every link in the app lands in the wrong place. Cut at silence, store absolute offsets, and assert against a recording whose true times you know. ## M2 · A long job that survives being interrupted A failure at chunk forty resumes at chunk forty, and you can see what it cost. ### Steps 1. One chunk, one state, one transaction Write the state before the call and the result after it. Files: `src/worker.py`, `tests/test_resume.py` 2. Record cost per call as it happens A three-hour lecture is an unpleasant surprise unless the number is on screen. Files: `src/worker.py`, `src/templates/project.html` 3. Back off on rate limits, retry only pre-response failures A timeout after the audio was accepted may already have been charged. Files: `src/transcribe.py` ### Done when - [ ] A kill mid-run resumes without repeating a chunk - [ ] Rate limits back off and complete - [ ] Cost is recorded per call and displayed - [ ] Cancelling leaves completed work intact ### Watch out - Treating transcription as one long call means a network blip, a rate limit or a laptop lid closing at minute ninety throws away every chunk already paid for, and the user retries the whole two hours. Checkpoint per chunk, persist before and after each call, and make resume the default path rather than an option. ## M3 · Search that lands on the moment A query returns spans with times, and clicking one moves the player there. ### Steps 1. Index segments, not transcripts The unit of search should be the unit you can seek to. Files: `src/search.py`, `tests/test_search.py` 2. Carry the time onto every rendered line Display grouping must never be allowed to lose it. Files: `src/templates/transcript.html` 3. Test names and numbers deliberately They are what people actually search a lecture for. Files: `tests/test_search.py` ### Done when - [ ] Phrase search returns timed segments - [ ] Clicking a result seeks accurately - [ ] Numbers and proper nouns are findable - [ ] Grouped lines keep their start times ### Watch out - Storing the transcript as one text blob and searching it gives you a hit that says the phrase is somewhere in a ninety-minute recording, which is the same as no answer; and rebuilding paragraphs for display without carrying the segment times means the clickable timestamp drifts from the words shown. Index per segment, and keep the time on every displayed line. ## M4 · Notes that can be checked against the recording Every generated claim points at a span that really says it, or it is not shown. ### Steps 1. Generate per section with a required quote A claim with nowhere to point is a claim you cannot check. Files: `src/notes.py` 2. Verify the quote by exact match, then resolve it Normalise whitespace and case, nothing more forgiving than that. Files: `src/verify.py`, `tests/test_verify.py` 3. Show gaps instead of filling them An honest blank is more useful than a plausible sentence. Files: `src/templates/notes.html` ### Done when - [ ] Unverifiable claims are dropped and counted - [ ] Every shown note resolves to a segment - [ ] Section regeneration is isolated - [ ] Empty sections are shown as empty ### Watch out - Passing a ninety-minute transcript in one request and asking for ten flashcards produces confident cards about things nobody said, because the model is compressing far too much and has no obligation to point at anything. Generate per section, demand a quote, verify the quote exists, and show a section as unsummarised rather than inventing a summary of it. ## M5 · Getting it out, and keeping the disk and the bill in hand (production only) Everything exports, deletion really deletes, and neither disk nor spend can run away unwatched. ### Steps 1. Export four formats with times preserved Subtitles and JSON are what other tools can read. Files: `src/export.py`, `tests/test_export.py` 2. Delete files, rows and index together, then assert Then search for the unique phrase and expect nothing. Files: `src/delete.py`, `tests/test_delete.py` 3. Cap disk and monthly spend, and refuse past them A refusal you can see beats a bill you cannot. Files: `src/limits.py` ### Done when - [ ] All four export formats keep timestamps - [ ] A deleted project leaves nothing findable - [ ] The disk ceiling refuses new imports - [ ] The spend cap refuses new transcription ### Watch out - Deleting a project by marking a row deleted leaves gigabytes of video and every transcript line in the search index, so the disk fills and a search still surfaces text from a recording the person believes they removed. Delete the files, the rows and the index entries in one operation, then assert the search returns nothing for a phrase that was unique to that project.
OPERATIONS.md
# Operations · VidNotes ## Backup The database and the media directory together, since one without the other leaves projects pointing at recordings that are gone. Transcripts and notes are small and worth backing up far more often than the media, which the person usually still has. ## Restore Restore both, then run a consistency pass that marks any project whose media is missing rather than failing on open. A restored job in mid-transcription resumes from its completed chunks, which is the same path as a crash. Do a restore drill before the first real user, and write the date here when it passes. ## Monitoring For a local tool this lives in the interface: jobs stuck in a stage, chunks with repeated attempts, month-to-date spend against the cap, and media directory usage against the ceiling. ## Incident checklist Wrong timestamps mean the chunk offset arithmetic broke and the test against the known recording is the first thing to run. A surprising bill means a job retried a chunk that had already been charged, and the per-chunk cost rows show which. 1. Contain the issue without destroying evidence or user data. 2. Record the timeline and affected scope. 3. Rotate exposed secrets and revoke compromised sessions or credentials. 4. Restore from a verified backup when needed. 5. Document the root cause, the remediation and the regression test. ## Release gate - [ ] Late-chunk timestamps are confirmed against a known recording - [ ] A kill mid-run is confirmed to resume without repeating a chunk - [ ] An unverifiable generated claim is confirmed dropped - [ ] A deleted project is confirmed to leave nothing findable - [ ] The monthly spend cap is confirmed to refuse new work ## Launch constraint Do not market omitted VidNotes capabilities as implemented. The non-goals in `PRODUCT.md` remain user-visible limitations until they are deliberately delivered.
.env.example
# Copy to .env and fill in. Never commit .env; this file documents it. # Required. Bound to localhost only. This is a personal tool. APP_PORT=8787 # Required. Projects, segments, notes and the job state. DATABASE_PATH=./data/vidnotes.db # Required. One directory per project. Deleted in full when a project is deleted. MEDIA_DIR=./data/projects # Required. The disk ceiling. Imports past it are refused with usage shown. MEDIA_MAX_BYTES=53687091200 # Required · secret. Environment only. Never in the database, never in a log line. TRANSCRIBE_API_KEY=sk-...redacted # Required. Set below the provider's published limit, which you check rather than assume. CHUNK_MAX_BYTES=24000000 # Required. New transcription is refused past this rather than billed through it. MONTHLY_SPEND_CAP_CENTS=2000 # Required. Off by default. Turning it on is a statement about the source's terms. ENABLE_URL_IMPORT=false
$ choose a build depth, inspect the files, then open the complete pack in your agent
People pay for the maintained system around transcription: source adapters that keep working, long-job processing and retries, consistent results across native apps, web, and the extension, synced account state, and stable API, CLI, and MCP access for agents. The recurring value is avoiding integration breakage and operations work, not just getting speech-to-text.
xnative iOS and Android apps, a Chrome extension, and cross-device Premium state
xmanaged extraction for TikTok, Instagram, Vimeo, and changing source sites
xbackground processing, queue reliability, and long-video handling
x30+ language polish and prebuilt flashcards, quizzes, and chat
xhosted web, API, CLI, and MCP access for agents
share on X ↗"I just replaced VidNotes ($9.99/mo) with one prompt"
Don't feel like building it? These folks already made it free.
StoryToolkitAILocal transcription and semantic footage search from an app that still admits it is rough.open source↗
NotebookLMDrop in a YouTube video and get searchable, source-grounded study material; Google keeps the notebook.free↗no votes, no pay-to-list · just what's real
VidNotes pricing
| plan | monthly | annual (per mo) | what you get |
|---|---|---|---|
| monthly | $9.99 | $4.17 | Unlimited transcription, summaries, flashcards, chat and action items; 30+ languages |
free tierno free tier; any trial allowance is not numerically disclosed on the public pricing section
billingmonthly or $49.99/year; no per-minute transcription charge advertised
verified 2026-08-14 · source ↗
Vibecode VidNotes
Kinda. The core of VidNotes is buildable in a weekend with the prompt on this page, but there are real gaps: native iOS and Android apps, a Chrome extension, and cross-device Premium state, managed extraction for TikTok, Instagram, Vimeo, and changing source sites. Read the honest list above before committing.
How much does VidNotes cost?
VidNotes costs about $9.99/month (Monthly, checked 2026-07-31), which is $119.88 per year.
What do I lose by replacing VidNotes?
Honestly: native iOS and Android apps, a Chrome extension, and cross-device Premium state; managed extraction for TikTok, Instagram, Vimeo, and changing source sites; background processing, queue reliability, and long-video handling; 30+ language polish and prebuilt flashcards, quizzes, and chat; hosted web, API, CLI, and MCP access for agents. If any of those are load-bearing for you, keep paying.
Is there an open-source alternative to VidNotes?
Yes: StoryToolkitAI (Local transcription and semantic footage search from an app that still admits it is rough.) NotebookLM (Drop in a YouTube video and get searchable, source-grounded study material; Google keeps the notebook.) The prompt is for when you want it exactly your way.