Vibecode MacWhisper
track this build4 phases, 12 steps, beginner friendly0%YES · one-shottable
MacWhisper's solo core is compact: build a private local transcription pipeline that accepts an audio file, transcribes it, creates structured notes, and exports Markdown. A competent builder can reach a useful personal version in one sitting, while the paid product mainly wins on capture, integrations, reliability.
the project pack5 files · written for this build, step by step
You are building a lean indie version of MacWhisper. Create the following project files first, then implement the application by following them. Keep the files updated as decisions change. Do not collapse this into a single README or prompt. ===== README.md ===== # MacWhisper · indie build A dictation tool where the whole engineering problem is that the best local speech model is not a streaming model. It processes fixed windows, so live dictation means gating on speech activity, chunking with overlap, and reconciling text that the model revises as it hears more. Estimated effort: **multi-day**. Work `BUILD_PLAN.md` top to bottom · every phase ends in a check that has to pass before the next one starts. ## Stack | Part | Choice | Why | | --- | --- | --- | | Recognition | whisper.cpp with a model sized to the machine | the standard for offline recognition, and not natively a streaming one | | Gating | Voice activity detection ahead of the model | silence fed to the model produces invented sentences, not empty output | | Chunking | Overlapping windows reconciled on their overlap | the model works in fixed windows and the seams are where words are lost | | Insertion | Typing into the focused application, with a review buffer | dictation that goes somewhere you then have to copy from is not dictation | | Runtime | A native application with a global hotkey | it has to be available in whatever window you are already in | ## Before you start Have every one of these ready. The plan assumes them from step one. - [ ] **A Mac with enough memory for the model** · free if you own one - Why: Model size determines both accuracy and whether live dictation keeps up, and a machine that swaps will produce unusable latency. - Get it: Check available memory and choose a model that fits comfortably alongside your normal work. - Verify: A five minute file transcribes without the machine swapping - [ ] **whisper.cpp with a model** · free, and models are a large download - Why: The recognition engine, and the source of every timing and confidence the build depends on. - Get it: Build whisper.cpp and download a model. Recent versions include voice activity detection support, which phase 2 depends on. - Verify: A short file transcribes with word timings and the build includes voice activity detection - [ ] **Xcode and the command line tools** · free - Why: A global hotkey, accessibility insertion and audio capture are all native work. - Get it: Install from the App Store, then the command line tools. - Verify: A trivial application builds and registers a global hotkey - [ ] **Recordings of your actual working environment** · free - Why: Phase 2 is tuned against real background noise, and a quiet room is much easier than a real one with a fan, typing and a conversation next door. - Get it: Record thirty seconds of your workspace in silence, and thirty seconds while typing. - Verify: Both recordings exist and the detector gates on both - [ ] **A list of names and terms the model gets wrong** · free - Why: Vocabulary correction is only testable against terms that actually fail, and the ones that fail are specific to your work. - Get it: Dictate a few paragraphs of your own writing and note every proper noun and technical term that comes out wrong. - Verify: At least fifteen terms are listed, including one that appears as a substring of a common word ## Quick start Follow Phase 1 in `BUILD_PLAN.md`; it creates the project. Then copy `.env.example` to `.env` and fill in the values it documents. ## Honest limits This build deliberately does not replace: - Speaker diarisation. - Real-time translation. - Cloud recognition as a fallback. - Custom model training on your own voice. - cross-call team analytics - meeting-bot auto-join - live multi-speaker accuracy - calendar and CRM integrations If one of those is essential to you, that is the reason to keep paying for MacWhisper, and the README should say so rather than pretend. ===== BRIEF.md ===== # Build brief · MacWhisper The one-shot brief this plan expands. `BUILD_PLAN.md` (or `MILESTONES.md`) is the same sequence broken into steps and checks; where the two disagree, the plan wins. Build me local dictation and transcription that types into whatever window you are in to replace MacWhisper. Build it in phases, in the order below. Do not write the whole thing in one pass. Finish a phase, run its "Done when" check, fix what fails, and only then start the next phase. ### Stack (fixed, do not substitute) - Recognition: whisper.cpp with a model sized to the machine · the standard for offline recognition, and not natively a streaming one - Gating: Voice activity detection ahead of the model · silence fed to the model produces invented sentences, not empty output - Chunking: Overlapping windows reconciled on their overlap · the model works in fixed windows and the seams are where words are lost - Insertion: Typing into the focused application, with a review buffer · dictation that goes somewhere you then have to copy from is not dictation - Runtime: A native application with a global hotkey · it has to be available in whatever window you are already in ### Data model (create this before Phase 1) - sessions(id, started_at, ended_at, mode, target_app) · mode is dictation or file transcription - segments(id, session_id, ordinal, start_ms, end_ms, text, confidence, revised_from) · revised_from records that the model changed its mind - vocabulary(id, phrase, replacement, scope) · names and terms the model reliably gets wrong - files(id, path, duration_ms, model, transcribed_at, output_path) · batch transcription - settings(key, value) · hotkey, model, insertion behaviour - A segment can be superseded rather than only appended, because a streaming approximation over a non-streaming model produces revisions. ### Phase 1 · Transcribing a file correctly Build: Transcribe an audio file by chunking into windows with overlap and merging on the overlapping text. Verify against a shorter file transcribed in one pass. Report progress and allow cancellation. Trap: The model processes fixed-length windows, so chunking without overlap drops words at every boundary and chunking with naive overlap duplicates them. Merge by matching text across the overlap, and verify the word count and final timestamp against a single-pass transcription of a shorter file. Done when: An hour-long file transcribes with no words lost or duplicated at seams, verified against a single-pass transcription of a ten minute excerpt. Progress is reported and cancellation stops promptly. The final timestamp matches the audio duration. Transcription runs with networking disabled. Do not build yet: No live dictation, gating or insertion. ### Phase 2 · Not letting the model talk to itself Build: Voice activity detection ahead of the model, so windows containing no speech are never sent. Tune the detector against real background noise. Log how often it gates, since a detector that never gates is not working. Trap: Fed a window of silence or background noise, the model will confidently produce a plausible sentence, and in dictation that means text appearing in your document when nobody spoke. This is not an efficiency optimisation, it is the thing that makes the feature usable at all. Done when: Thirty seconds of room silence produces no text, tested directly. Background noise such as a fan or typing does not trigger recognition. Speech at a normal level is not gated out, tested with quiet speech. The gating rate is logged and is neither zero nor everything. Do not build yet: No live dictation or insertion yet. ### Phase 3 · Live dictation, with the model changing its mind Build: Run recognition on a rolling window while speaking, showing provisional text and committing it once stable. Handle the model revising an earlier phrase after hearing more. Measure the delay from speech to visible text. Trap: Committing every recognition result immediately produces text that visibly rewrites itself as the model revises, which is unusable for dictation because you cannot type around it. Hold recent text as provisional, commit only once a window has passed without it changing, and never revise text already committed to the target application. Done when: Speech appears as provisional text and commits once stable. Committed text is never rewritten, verified by speaking a phrase the model initially mishears and then corrects. The delay from speech to visible text is measured and stated. A long dictation does not accumulate growing latency. Do not build yet: No insertion into other applications yet. ### Phase 4 · Typing into whatever is in front of you Build: Insert committed text into the focused application, with a review buffer as an alternative. Handle applications that reject synthetic input. Document the accessibility permission the platform requires and what it grants. Trap: Inserting into whatever happens to be focused at the moment text commits will type into the wrong window when focus changes mid-sentence, which in the wrong application is anything from embarrassing to destructive. Capture the target at the start of dictation and stop rather than following focus. Done when: Text inserts into the application focused when dictation started. A focus change mid-dictation stops insertion rather than typing into the new window, tested by switching applications while speaking. An application rejecting synthetic input falls back to the review buffer with an explanation. The required permission is requested with a clear explanation of what it grants. Do not build yet: No vocabulary or batch processing yet. ### Phase 5 · Vocabulary, batch work and keeping it local Build: A vocabulary of terms and names with replacements applied after recognition. Batch transcription of a folder with subtitle output. Verify the whole application functions with networking disabled. Trap: Applying vocabulary replacements by simple string substitution will corrupt words that happen to contain the term, so replacing a short name mangles unrelated text throughout the transcript. Match on word boundaries with the surrounding context, and test against a transcript containing the term as a substring. Done when: Vocabulary replacements apply on word boundaries without corrupting words containing the term, tested with a crafted transcript. A folder of files transcribes unattended with subtitle output carrying correct timings. The entire application works with networking disabled, verified for every feature. Model files are the only large download and are fetched deliberately. Do not build yet: Nothing after this. ### Out of scope (and why) - Speaker diarisation, which is unreliable on a single mixed track and better solved by recording separately. - Real-time translation, which compounds recognition latency with translation latency. - Cloud recognition as a fallback, which would undo the reason for building this. - Custom model training on your own voice, which is a substantially larger project. ### README must contain - That the model is not a streaming model, so live dictation is an approximation over fixed windows - That speech activity gating exists because silence produces invented sentences rather than nothing - That committed text is never rewritten even when the model revises its own earlier output - That the insertion target is captured at the start so a focus change cannot misdirect text - That every feature works with networking disabled, and how to verify that ===== AGENTS.md ===== # Agent instructions · MacWhisper indie build - Read `README.md` and `BUILD_PLAN.md` before writing code. The stack is fixed: whisper.cpp with a model sized to the machine, Voice activity detection ahead of the model, Overlapping windows reconciled on their overlap, Typing into the focused application, with a review buffer, A native application with a global hotkey. Do not substitute. - Work one phase at a time, in order. Do not start a phase until every "Done when" item of the previous one passes. - Prefer the fewest moving parts that satisfy the step. No frameworks, services or dependencies the plan does not name. - Secrets live in `.env`, never in source or logs. Keep `.env.example` current when a variable is introduced. - Do not invent cryptography, security guarantees, APIs or compliance claims. - Add a focused test for every destructive, security-sensitive or data-loss path the plan names. - Run the project checks before declaring a phase complete, and record any deliberate shortcut in the README under "Tradeoffs". ## Known traps - The model processes fixed-length windows, so chunking without overlap drops words at every boundary and chunking with naive overlap duplicates them. Merge by matching text across the overlap, and verify the word count and final timestamp against a single-pass transcription of a shorter file. - Fed a window of silence or background noise, the model will confidently produce a plausible sentence, and in dictation that means text appearing in your document when nobody spoke. This is not an efficiency optimisation, it is the thing that makes the feature usable at all. - Committing every recognition result immediately produces text that visibly rewrites itself as the model revises, which is unusable for dictation because you cannot type around it. Hold recent text as provisional, commit only once a window has passed without it changing, and never revise text already committed to the target application. - Inserting into whatever happens to be focused at the moment text commits will type into the wrong window when focus changes mid-sentence, which in the wrong application is anything from embarrassing to destructive. Capture the target at the start of dictation and stop rather than following focus. ===== BUILD_PLAN.md ===== # Build plan · MacWhisper A dictation tool where the whole engineering problem is that the best local speech model is not a streaming model. It processes fixed windows, so live dictation means gating on speech activity, chunking with overlap, and reconciling text that the model revises as it hears more. Phases are in dependency order. Each ends in a "Done when" list; treat an unticked item as a blocker, not a note. ## Phase 1 · Transcribing a file correctly A long recording transcribes with nothing lost at the window seams. ### Steps 1. Chunk with overlap and merge on text The window is fixed. The seams are where the words go missing. Files: `src/chunk.swift`, `tests/ChunkTests.swift` 2. Verify against a single-pass transcription A shorter file transcribed whole is the reference. Compare word counts. Files: `tests/SeamTests.swift` 3. Report progress and allow cancellation An hour of audio is minutes of work. Silence during it feels broken. Files: `src/Transcriber.swift` ### Done when - [ ] An hour transcribes with no seam losses - [ ] Word count matches a single-pass reference - [ ] Progress reports and cancellation is prompt - [ ] Transcription works offline ### Watch out - The model processes fixed-length windows, so chunking without overlap drops words at every boundary and chunking with naive overlap duplicates them. Merge by matching text across the overlap, and verify the word count and final timestamp against a single-pass transcription of a shorter file. ## Phase 2 · Not letting the model talk to itself Silence produces silence rather than invented sentences. ### Steps 1. Gate on speech activity before the model Nothing reaches the model without speech in it. Files: `src/VAD.swift` 2. Tune against real background noise A fan, typing, a room. Synthetic silence is much easier than real quiet. Files: `tests/NoiseTests.swift` 3. Log the gating rate A detector that never gates is not working, and you will not notice otherwise. Files: `src/VAD.swift` ### Done when - [ ] Thirty seconds of silence produces no text - [ ] Background noise does not trigger recognition - [ ] Quiet speech is not gated out - [ ] The gating rate is logged and plausible ### Watch out - Fed a window of silence or background noise, the model will confidently produce a plausible sentence, and in dictation that means text appearing in your document when nobody spoke. This is not an efficiency optimisation, it is the thing that makes the feature usable at all. ## Phase 3 · Live dictation, with the model changing its mind Words appear as you speak and corrections are handled gracefully. ### Steps 1. Show provisional, commit when stable Stability over a window is the commit condition, not the model returning. Files: `src/Live.swift` 2. Never revise committed text Once it has left for the target application it is the user's problem to edit. Files: `src/Live.swift` 3. Measure speech-to-text delay A number, logged. It is what the feature is judged on. Files: `src/Metrics.swift` ### Done when - [ ] Provisional text commits once stable - [ ] Committed text is never rewritten - [ ] Speech-to-visible delay is measured - [ ] Long dictation does not accumulate latency ### Watch out - Committing every recognition result immediately produces text that visibly rewrites itself as the model revises, which is unusable for dictation because you cannot type around it. Hold recent text as provisional, commit only once a window has passed without it changing, and never revise text already committed to the target application. ## Phase 4 · Typing into whatever is in front of you Dictated text lands in the application you were already using. ### Steps 1. Capture the target at start Not at commit. Following focus types into the wrong window. Files: `src/Insert.swift` 2. Fall back to a review buffer Some applications reject synthetic input. Losing the text is not acceptable. Files: `src/Buffer.swift` 3. Explain the permission honestly Accessibility access is broad. Say what it is used for. Files: `src/Permissions.swift` ### Done when - [ ] Text lands in the application focused at start - [ ] A focus change stops insertion rather than misdirecting it - [ ] A rejecting application falls back to the buffer - [ ] The permission is explained before being requested ### Watch out - Inserting into whatever happens to be focused at the moment text commits will type into the wrong window when focus changes mid-sentence, which in the wrong application is anything from embarrassing to destructive. Capture the target at the start of dictation and stop rather than following focus. ## Not in this build - Speaker diarisation. - Real-time translation. - Cloud recognition as a fallback. - Custom model training on your own voice. ## After v1, if you want it - Commands mixed into dictation, such as spoken punctuation and new paragraph - Per-application settings, since dictating code and dictating prose want different handling - A correction pass that learns from what you edit afterwards ===== .env.example ===== # Copy to .env and fill in. Never commit .env; this file documents it. # Required. The recognition model. Larger is more accurate and slower, which matters for live dictation. MODEL_PATH=~/Library/Application Support/dictation/ggml-medium.en.bin # Required. The global hotkey. Choose something nothing else claims, which is harder than it sounds. DICTATION_HOTKEY=Fn+Fn # Required. Speech activity sensitivity. Too high loses quiet speech, too low lets the model invent. VAD_THRESHOLD=0.5 # Required. How long text must stay unchanged before committing. Shorter feels faster and rewrites more. COMMIT_STABLE_MS=600 # Required. Names and terms with replacements. Applied on word boundaries after recognition. VOCABULARY_PATH=./config/vocabulary.txt # Required. Present as a constant so the answer is visible in configuration. Nothing else may use the network. ALLOW_NETWORK=model_download_only
You are building a lean indie version of MacWhisper. Create the following project files first, then implement the application by following them. Keep the files updated as decisions change. Do not collapse this into a single README or prompt. ===== README.md ===== # MacWhisper · indie build A dictation tool where the whole engineering problem is that the best local speech model is not a streaming model. It processes fixed windows, so live dictation means gating on speech activity, chunking with overlap, and reconciling text that the model revises as it hears more. Estimated effort: **multi-day**. Work `BUILD_PLAN.md` top to bottom · every phase ends in a check that has to pass before the next one starts. ## Stack | Part | Choice | Why | | --- | --- | --- | | Recognition | whisper.cpp with a model sized to the machine | the standard for offline recognition, and not natively a streaming one | | Gating | Voice activity detection ahead of the model | silence fed to the model produces invented sentences, not empty output | | Chunking | Overlapping windows reconciled on their overlap | the model works in fixed windows and the seams are where words are lost | | Insertion | Typing into the focused application, with a review buffer | dictation that goes somewhere you then have to copy from is not dictation | | Runtime | A native application with a global hotkey | it has to be available in whatever window you are already in | ## Before you start Have every one of these ready. The plan assumes them from step one. - [ ] **A Mac with enough memory for the model** · free if you own one - Why: Model size determines both accuracy and whether live dictation keeps up, and a machine that swaps will produce unusable latency. - Get it: Check available memory and choose a model that fits comfortably alongside your normal work. - Verify: A five minute file transcribes without the machine swapping - [ ] **whisper.cpp with a model** · free, and models are a large download - Why: The recognition engine, and the source of every timing and confidence the build depends on. - Get it: Build whisper.cpp and download a model. Recent versions include voice activity detection support, which phase 2 depends on. - Verify: A short file transcribes with word timings and the build includes voice activity detection - [ ] **Xcode and the command line tools** · free - Why: A global hotkey, accessibility insertion and audio capture are all native work. - Get it: Install from the App Store, then the command line tools. - Verify: A trivial application builds and registers a global hotkey - [ ] **Recordings of your actual working environment** · free - Why: Phase 2 is tuned against real background noise, and a quiet room is much easier than a real one with a fan, typing and a conversation next door. - Get it: Record thirty seconds of your workspace in silence, and thirty seconds while typing. - Verify: Both recordings exist and the detector gates on both - [ ] **A list of names and terms the model gets wrong** · free - Why: Vocabulary correction is only testable against terms that actually fail, and the ones that fail are specific to your work. - Get it: Dictate a few paragraphs of your own writing and note every proper noun and technical term that comes out wrong. - Verify: At least fifteen terms are listed, including one that appears as a substring of a common word ## Quick start Follow Phase 1 in `BUILD_PLAN.md`; it creates the project. Then copy `.env.example` to `.env` and fill in the values it documents. ## Honest limits This build deliberately does not replace: - Speaker diarisation. - Real-time translation. - Cloud recognition as a fallback. - Custom model training on your own voice. - cross-call team analytics - meeting-bot auto-join - live multi-speaker accuracy - calendar and CRM integrations If one of those is essential to you, that is the reason to keep paying for MacWhisper, and the README should say so rather than pretend. ===== BRIEF.md ===== # Build brief · MacWhisper The one-shot brief this plan expands. `BUILD_PLAN.md` (or `MILESTONES.md`) is the same sequence broken into steps and checks; where the two disagree, the plan wins. Build me local dictation and transcription that types into whatever window you are in to replace MacWhisper. Build it in phases, in the order below. Do not write the whole thing in one pass. Finish a phase, run its "Done when" check, fix what fails, and only then start the next phase. ### Stack (fixed, do not substitute) - Recognition: whisper.cpp with a model sized to the machine · the standard for offline recognition, and not natively a streaming one - Gating: Voice activity detection ahead of the model · silence fed to the model produces invented sentences, not empty output - Chunking: Overlapping windows reconciled on their overlap · the model works in fixed windows and the seams are where words are lost - Insertion: Typing into the focused application, with a review buffer · dictation that goes somewhere you then have to copy from is not dictation - Runtime: A native application with a global hotkey · it has to be available in whatever window you are already in ### Data model (create this before Phase 1) - sessions(id, started_at, ended_at, mode, target_app) · mode is dictation or file transcription - segments(id, session_id, ordinal, start_ms, end_ms, text, confidence, revised_from) · revised_from records that the model changed its mind - vocabulary(id, phrase, replacement, scope) · names and terms the model reliably gets wrong - files(id, path, duration_ms, model, transcribed_at, output_path) · batch transcription - settings(key, value) · hotkey, model, insertion behaviour - A segment can be superseded rather than only appended, because a streaming approximation over a non-streaming model produces revisions. ### Phase 1 · Transcribing a file correctly Build: Transcribe an audio file by chunking into windows with overlap and merging on the overlapping text. Verify against a shorter file transcribed in one pass. Report progress and allow cancellation. Trap: The model processes fixed-length windows, so chunking without overlap drops words at every boundary and chunking with naive overlap duplicates them. Merge by matching text across the overlap, and verify the word count and final timestamp against a single-pass transcription of a shorter file. Done when: An hour-long file transcribes with no words lost or duplicated at seams, verified against a single-pass transcription of a ten minute excerpt. Progress is reported and cancellation stops promptly. The final timestamp matches the audio duration. Transcription runs with networking disabled. Do not build yet: No live dictation, gating or insertion. ### Phase 2 · Not letting the model talk to itself Build: Voice activity detection ahead of the model, so windows containing no speech are never sent. Tune the detector against real background noise. Log how often it gates, since a detector that never gates is not working. Trap: Fed a window of silence or background noise, the model will confidently produce a plausible sentence, and in dictation that means text appearing in your document when nobody spoke. This is not an efficiency optimisation, it is the thing that makes the feature usable at all. Done when: Thirty seconds of room silence produces no text, tested directly. Background noise such as a fan or typing does not trigger recognition. Speech at a normal level is not gated out, tested with quiet speech. The gating rate is logged and is neither zero nor everything. Do not build yet: No live dictation or insertion yet. ### Phase 3 · Live dictation, with the model changing its mind Build: Run recognition on a rolling window while speaking, showing provisional text and committing it once stable. Handle the model revising an earlier phrase after hearing more. Measure the delay from speech to visible text. Trap: Committing every recognition result immediately produces text that visibly rewrites itself as the model revises, which is unusable for dictation because you cannot type around it. Hold recent text as provisional, commit only once a window has passed without it changing, and never revise text already committed to the target application. Done when: Speech appears as provisional text and commits once stable. Committed text is never rewritten, verified by speaking a phrase the model initially mishears and then corrects. The delay from speech to visible text is measured and stated. A long dictation does not accumulate growing latency. Do not build yet: No insertion into other applications yet. ### Phase 4 · Typing into whatever is in front of you Build: Insert committed text into the focused application, with a review buffer as an alternative. Handle applications that reject synthetic input. Document the accessibility permission the platform requires and what it grants. Trap: Inserting into whatever happens to be focused at the moment text commits will type into the wrong window when focus changes mid-sentence, which in the wrong application is anything from embarrassing to destructive. Capture the target at the start of dictation and stop rather than following focus. Done when: Text inserts into the application focused when dictation started. A focus change mid-dictation stops insertion rather than typing into the new window, tested by switching applications while speaking. An application rejecting synthetic input falls back to the review buffer with an explanation. The required permission is requested with a clear explanation of what it grants. Do not build yet: No vocabulary or batch processing yet. ### Phase 5 · Vocabulary, batch work and keeping it local Build: A vocabulary of terms and names with replacements applied after recognition. Batch transcription of a folder with subtitle output. Verify the whole application functions with networking disabled. Trap: Applying vocabulary replacements by simple string substitution will corrupt words that happen to contain the term, so replacing a short name mangles unrelated text throughout the transcript. Match on word boundaries with the surrounding context, and test against a transcript containing the term as a substring. Done when: Vocabulary replacements apply on word boundaries without corrupting words containing the term, tested with a crafted transcript. A folder of files transcribes unattended with subtitle output carrying correct timings. The entire application works with networking disabled, verified for every feature. Model files are the only large download and are fetched deliberately. Do not build yet: Nothing after this. ### Out of scope (and why) - Speaker diarisation, which is unreliable on a single mixed track and better solved by recording separately. - Real-time translation, which compounds recognition latency with translation latency. - Cloud recognition as a fallback, which would undo the reason for building this. - Custom model training on your own voice, which is a substantially larger project. ### README must contain - That the model is not a streaming model, so live dictation is an approximation over fixed windows - That speech activity gating exists because silence produces invented sentences rather than nothing - That committed text is never rewritten even when the model revises its own earlier output - That the insertion target is captured at the start so a focus change cannot misdirect text - That every feature works with networking disabled, and how to verify that ===== AGENTS.md ===== # Agent instructions · MacWhisper indie build - Read `README.md` and `BUILD_PLAN.md` before writing code. The stack is fixed: whisper.cpp with a model sized to the machine, Voice activity detection ahead of the model, Overlapping windows reconciled on their overlap, Typing into the focused application, with a review buffer, A native application with a global hotkey. Do not substitute. - Work one phase at a time, in order. Do not start a phase until every "Done when" item of the previous one passes. - Prefer the fewest moving parts that satisfy the step. No frameworks, services or dependencies the plan does not name. - Secrets live in `.env`, never in source or logs. Keep `.env.example` current when a variable is introduced. - Do not invent cryptography, security guarantees, APIs or compliance claims. - Add a focused test for every destructive, security-sensitive or data-loss path the plan names. - Run the project checks before declaring a phase complete, and record any deliberate shortcut in the README under "Tradeoffs". ## Known traps - The model processes fixed-length windows, so chunking without overlap drops words at every boundary and chunking with naive overlap duplicates them. Merge by matching text across the overlap, and verify the word count and final timestamp against a single-pass transcription of a shorter file. - Fed a window of silence or background noise, the model will confidently produce a plausible sentence, and in dictation that means text appearing in your document when nobody spoke. This is not an efficiency optimisation, it is the thing that makes the feature usable at all. - Committing every recognition result immediately produces text that visibly rewrites itself as the model revises, which is unusable for dictation because you cannot type around it. Hold recent text as provisional, commit only once a window has passed without it changing, and never revise text already committed to the target application. - Inserting into whatever happens to be focused at the moment text commits will type into the wrong window when focus changes mid-sentence, which in the wrong application is anything from embarrassing to destructive. Capture the target at the start of dictation and stop rather than following focus. ===== BUILD_PLAN.md ===== # Build plan · MacWhisper A dictation tool where the whole engineering problem is that the best local speech model is not a streaming model. It processes fixed windows, so live dictation means gating on speech activity, chunking with overlap, and reconciling text that the model revises as it hears more. Phases are in dependency order. Each ends in a "Done when" list; treat an unticked item as a blocker, not a note. ## Phase 1 · Transcribing a file correctly A long recording transcribes with nothing lost at the window seams. ### Steps 1. Chunk with overlap and merge on text The window is fixed. The seams are where the words go missing. Files: `src/chunk.swift`, `tests/ChunkTests.swift` 2. Verify against a single-pass transcription A shorter file transcribed whole is the reference. Compare word counts. Files: `tests/SeamTests.swift` 3. Report progress and allow cancellation An hour of audio is minutes of work. Silence during it feels broken. Files: `src/Transcriber.swift` ### Done when - [ ] An hour transcribes with no seam losses - [ ] Word count matches a single-pass reference - [ ] Progress reports and cancellation is prompt - [ ] Transcription works offline ### Watch out - The model processes fixed-length windows, so chunking without overlap drops words at every boundary and chunking with naive overlap duplicates them. Merge by matching text across the overlap, and verify the word count and final timestamp against a single-pass transcription of a shorter file. ## Phase 2 · Not letting the model talk to itself Silence produces silence rather than invented sentences. ### Steps 1. Gate on speech activity before the model Nothing reaches the model without speech in it. Files: `src/VAD.swift` 2. Tune against real background noise A fan, typing, a room. Synthetic silence is much easier than real quiet. Files: `tests/NoiseTests.swift` 3. Log the gating rate A detector that never gates is not working, and you will not notice otherwise. Files: `src/VAD.swift` ### Done when - [ ] Thirty seconds of silence produces no text - [ ] Background noise does not trigger recognition - [ ] Quiet speech is not gated out - [ ] The gating rate is logged and plausible ### Watch out - Fed a window of silence or background noise, the model will confidently produce a plausible sentence, and in dictation that means text appearing in your document when nobody spoke. This is not an efficiency optimisation, it is the thing that makes the feature usable at all. ## Phase 3 · Live dictation, with the model changing its mind Words appear as you speak and corrections are handled gracefully. ### Steps 1. Show provisional, commit when stable Stability over a window is the commit condition, not the model returning. Files: `src/Live.swift` 2. Never revise committed text Once it has left for the target application it is the user's problem to edit. Files: `src/Live.swift` 3. Measure speech-to-text delay A number, logged. It is what the feature is judged on. Files: `src/Metrics.swift` ### Done when - [ ] Provisional text commits once stable - [ ] Committed text is never rewritten - [ ] Speech-to-visible delay is measured - [ ] Long dictation does not accumulate latency ### Watch out - Committing every recognition result immediately produces text that visibly rewrites itself as the model revises, which is unusable for dictation because you cannot type around it. Hold recent text as provisional, commit only once a window has passed without it changing, and never revise text already committed to the target application. ## Phase 4 · Typing into whatever is in front of you Dictated text lands in the application you were already using. ### Steps 1. Capture the target at start Not at commit. Following focus types into the wrong window. Files: `src/Insert.swift` 2. Fall back to a review buffer Some applications reject synthetic input. Losing the text is not acceptable. Files: `src/Buffer.swift` 3. Explain the permission honestly Accessibility access is broad. Say what it is used for. Files: `src/Permissions.swift` ### Done when - [ ] Text lands in the application focused at start - [ ] A focus change stops insertion rather than misdirecting it - [ ] A rejecting application falls back to the buffer - [ ] The permission is explained before being requested ### Watch out - Inserting into whatever happens to be focused at the moment text commits will type into the wrong window when focus changes mid-sentence, which in the wrong application is anything from embarrassing to destructive. Capture the target at the start of dictation and stop rather than following focus. ## Not in this build - Speaker diarisation. - Real-time translation. - Cloud recognition as a fallback. - Custom model training on your own voice. ## After v1, if you want it - Commands mixed into dictation, such as spoken punctuation and new paragraph - Per-application settings, since dictating code and dictating prose want different handling - A correction pass that learns from what you edit afterwards ===== .env.example ===== # Copy to .env and fill in. Never commit .env; this file documents it. # Required. The recognition model. Larger is more accurate and slower, which matters for live dictation. MODEL_PATH=~/Library/Application Support/dictation/ggml-medium.en.bin # Required. The global hotkey. Choose something nothing else claims, which is harder than it sounds. DICTATION_HOTKEY=Fn+Fn # Required. Speech activity sensitivity. Too high loses quiet speech, too low lets the model invent. VAD_THRESHOLD=0.5 # Required. How long text must stay unchanged before committing. Shorter feels faster and rewrites more. COMMIT_STABLE_MS=600 # Required. Names and terms with replacements. Applied on word boundaries after recognition. VOCABULARY_PATH=./config/vocabulary.txt # Required. Present as a constant so the answer is visible in configuration. Nothing else may use the network. ALLOW_NETWORK=model_download_only
You are building a production product version of MacWhisper. Create the following project files first, then implement the application by following them. Keep the files updated as decisions change. Do not collapse this into a single README or prompt. ===== PRODUCT.md ===== # MacWhisper · product brief ## Problem MacWhisper's solo core is compact: build a private local transcription pipeline that accepts an audio file, transcribes it, creates structured notes, and exports Markdown. A competent builder can reach a useful personal version in one sitting, while the paid product mainly wins on capture, integrations, reliability. ## Product outcome Dictation that types into the window you are already in, never invents a sentence during a pause, never rewrites what it already committed, and works with the network switched off. ## Target user A builder who needs a maintainable product foundation, not a one-off demo. ## Required capabilities - Python 3.12 - FFmpeg - local Whisper model or optional API key in .env ## Explicit non-goals for v1 - Speaker diarisation. - Real-time translation. - Cloud recognition as a fallback. - Custom model training on your own voice. - cross-call team analytics - meeting-bot auto-join - live multi-speaker accuracy - calendar and CRM integrations ## Success criteria - Seam merging is confirmed against a single-pass reference - Thirty seconds of silence is confirmed to produce no text - Committed text is confirmed never rewritten - A focus change is confirmed to stop insertion - Every feature is confirmed to work with networking disabled ===== BRIEF.md ===== # Build brief · MacWhisper The one-shot brief this plan expands. `BUILD_PLAN.md` (or `MILESTONES.md`) is the same sequence broken into steps and checks; where the two disagree, the plan wins. Build me local dictation and transcription that types into whatever window you are in to replace MacWhisper. Build it in phases, in the order below. Do not write the whole thing in one pass. Finish a phase, run its "Done when" check, fix what fails, and only then start the next phase. ### Stack (fixed, do not substitute) - Recognition: whisper.cpp with a model sized to the machine · the standard for offline recognition, and not natively a streaming one - Gating: Voice activity detection ahead of the model · silence fed to the model produces invented sentences, not empty output - Chunking: Overlapping windows reconciled on their overlap · the model works in fixed windows and the seams are where words are lost - Insertion: Typing into the focused application, with a review buffer · dictation that goes somewhere you then have to copy from is not dictation - Runtime: A native application with a global hotkey · it has to be available in whatever window you are already in ### Data model (create this before Phase 1) - sessions(id, started_at, ended_at, mode, target_app) · mode is dictation or file transcription - segments(id, session_id, ordinal, start_ms, end_ms, text, confidence, revised_from) · revised_from records that the model changed its mind - vocabulary(id, phrase, replacement, scope) · names and terms the model reliably gets wrong - files(id, path, duration_ms, model, transcribed_at, output_path) · batch transcription - settings(key, value) · hotkey, model, insertion behaviour - A segment can be superseded rather than only appended, because a streaming approximation over a non-streaming model produces revisions. ### Phase 1 · Transcribing a file correctly Build: Transcribe an audio file by chunking into windows with overlap and merging on the overlapping text. Verify against a shorter file transcribed in one pass. Report progress and allow cancellation. Trap: The model processes fixed-length windows, so chunking without overlap drops words at every boundary and chunking with naive overlap duplicates them. Merge by matching text across the overlap, and verify the word count and final timestamp against a single-pass transcription of a shorter file. Done when: An hour-long file transcribes with no words lost or duplicated at seams, verified against a single-pass transcription of a ten minute excerpt. Progress is reported and cancellation stops promptly. The final timestamp matches the audio duration. Transcription runs with networking disabled. Do not build yet: No live dictation, gating or insertion. ### Phase 2 · Not letting the model talk to itself Build: Voice activity detection ahead of the model, so windows containing no speech are never sent. Tune the detector against real background noise. Log how often it gates, since a detector that never gates is not working. Trap: Fed a window of silence or background noise, the model will confidently produce a plausible sentence, and in dictation that means text appearing in your document when nobody spoke. This is not an efficiency optimisation, it is the thing that makes the feature usable at all. Done when: Thirty seconds of room silence produces no text, tested directly. Background noise such as a fan or typing does not trigger recognition. Speech at a normal level is not gated out, tested with quiet speech. The gating rate is logged and is neither zero nor everything. Do not build yet: No live dictation or insertion yet. ### Phase 3 · Live dictation, with the model changing its mind Build: Run recognition on a rolling window while speaking, showing provisional text and committing it once stable. Handle the model revising an earlier phrase after hearing more. Measure the delay from speech to visible text. Trap: Committing every recognition result immediately produces text that visibly rewrites itself as the model revises, which is unusable for dictation because you cannot type around it. Hold recent text as provisional, commit only once a window has passed without it changing, and never revise text already committed to the target application. Done when: Speech appears as provisional text and commits once stable. Committed text is never rewritten, verified by speaking a phrase the model initially mishears and then corrects. The delay from speech to visible text is measured and stated. A long dictation does not accumulate growing latency. Do not build yet: No insertion into other applications yet. ### Phase 4 · Typing into whatever is in front of you Build: Insert committed text into the focused application, with a review buffer as an alternative. Handle applications that reject synthetic input. Document the accessibility permission the platform requires and what it grants. Trap: Inserting into whatever happens to be focused at the moment text commits will type into the wrong window when focus changes mid-sentence, which in the wrong application is anything from embarrassing to destructive. Capture the target at the start of dictation and stop rather than following focus. Done when: Text inserts into the application focused when dictation started. A focus change mid-dictation stops insertion rather than typing into the new window, tested by switching applications while speaking. An application rejecting synthetic input falls back to the review buffer with an explanation. The required permission is requested with a clear explanation of what it grants. Do not build yet: No vocabulary or batch processing yet. ### Phase 5 · Vocabulary, batch work and keeping it local Build: A vocabulary of terms and names with replacements applied after recognition. Batch transcription of a folder with subtitle output. Verify the whole application functions with networking disabled. Trap: Applying vocabulary replacements by simple string substitution will corrupt words that happen to contain the term, so replacing a short name mangles unrelated text throughout the transcript. Match on word boundaries with the surrounding context, and test against a transcript containing the term as a substring. Done when: Vocabulary replacements apply on word boundaries without corrupting words containing the term, tested with a crafted transcript. A folder of files transcribes unattended with subtitle output carrying correct timings. The entire application works with networking disabled, verified for every feature. Model files are the only large download and are fetched deliberately. Do not build yet: Nothing after this. ### Out of scope (and why) - Speaker diarisation, which is unreliable on a single mixed track and better solved by recording separately. - Real-time translation, which compounds recognition latency with translation latency. - Cloud recognition as a fallback, which would undo the reason for building this. - Custom model training on your own voice, which is a substantially larger project. ### README must contain - That the model is not a streaming model, so live dictation is an approximation over fixed windows - That speech activity gating exists because silence produces invented sentences rather than nothing - That committed text is never rewritten even when the model revises its own earlier output - That the insertion target is captured at the start so a focus change cannot misdirect text - That every feature works with networking disabled, and how to verify that ===== ARCHITECTURE.md ===== # Architecture · MacWhisper ## Stack | Part | Choice | Why | | --- | --- | --- | | Recognition | whisper.cpp with a model sized to the machine | the standard for offline recognition, and not natively a streaming one | | Gating | Voice activity detection ahead of the model | silence fed to the model produces invented sentences, not empty output | | Chunking | Overlapping windows reconciled on their overlap | the model works in fixed windows and the seams are where words are lost | | Insertion | Typing into the focused application, with a review buffer | dictation that goes somewhere you then have to copy from is not dictation | | Runtime | A native application with a global hotkey | it has to be available in whatever window you are already in | ## Modules Each module has one owner concern and a documented way to replace it. | Module | Owns | How to replace it | | --- | --- | --- | | Chunking | Overlapping windows and seam merging. | Any strategy, provided the seams are verified against a single-pass reference. | | Gating | Speech activity detection ahead of the model. | Any detector. Removing it entirely is what produces invented text. | | Live | Provisional text and stable commits. | Any policy, provided committed text is never rewritten. | | Insertion | Typing into the captured target. | Any mechanism with a review buffer fallback, since some applications refuse synthetic input. | | Vocabulary | Post-recognition corrections. | Any matcher that respects word boundaries. | ## Configuration Every runtime setting is an environment variable documented in `.env.example`, validated at startup, with a safe local default wherever one exists. - `MODEL_PATH` · required · The recognition model. Larger is more accurate and slower, which matters for live dictation. - `DICTATION_HOTKEY` · required · The global hotkey. Choose something nothing else claims, which is harder than it sounds. - `VAD_THRESHOLD` · required · Speech activity sensitivity. Too high loses quiet speech, too low lets the model invent. - `COMMIT_STABLE_MS` · required · How long text must stay unchanged before committing. Shorter feels faster and rewrites more. - `VOCABULARY_PATH` · required · Names and terms with replacements. Applied on word boundaries after recognition. - `ALLOW_NETWORK` · required · Present as a constant so the answer is visible in configuration. Nothing else may use the network. ## Production baseline - Security: least privilege, input validation at every boundary, secret redaction in logs, rate limits on abuse-prone paths, no invented security primitives. - Data: explicit schema and migrations, transactional writes where integrity matters, backup and restore procedures that have been exercised. - Integrations: adapters around third-party providers, idempotent webhook or job processing, bounded retries, timeouts. - Observability: structured logs with request or operation ids, an error-tracking hook, and health and readiness checks where a server exists. - Quality: unit tests for domain rules, integration tests at module boundaries, one end-to-end test of the critical path. ## Decision records For each dependency in the stack table, keep a short note: why it was chosen, its failure mode, and how it is replaced. Do not add infrastructure until a requirement in `PRODUCT.md` justifies it. ===== AGENTS.md ===== # Agent instructions · MacWhisper product build - Read `PRODUCT.md` and `ARCHITECTURE.md` before changing code. The stack is fixed: whisper.cpp with a model sized to the machine, Voice activity detection ahead of the model, Overlapping windows reconciled on their overlap, Typing into the focused application, with a review buffer, A native application with a global hotkey. - Implement milestone by milestone from `MILESTONES.md`; keep each change reviewable and leave the application runnable at every commit. - Treat authentication, payments, encryption, imports, webhooks and destructive actions as high-risk boundaries when present. - Never invent cryptography or silently weaken a requirement to make a check pass. - Put every external service behind an interface with a deterministic fake for tests. - Add migrations and rollback or recovery notes for every persistent data change. - Log useful operational context without credentials, tokens, passwords or personal data. - Update documentation and run every check before completing a milestone. ## Known traps - The model processes fixed-length windows, so chunking without overlap drops words at every boundary and chunking with naive overlap duplicates them. Merge by matching text across the overlap, and verify the word count and final timestamp against a single-pass transcription of a shorter file. - Fed a window of silence or background noise, the model will confidently produce a plausible sentence, and in dictation that means text appearing in your document when nobody spoke. This is not an efficiency optimisation, it is the thing that makes the feature usable at all. - Committing every recognition result immediately produces text that visibly rewrites itself as the model revises, which is unusable for dictation because you cannot type around it. Hold recent text as provisional, commit only once a window has passed without it changing, and never revise text already committed to the target application. - Inserting into whatever happens to be focused at the moment text commits will type into the wrong window when focus changes mid-sentence, which in the wrong application is anything from embarrassing to destructive. Capture the target at the start of dictation and stop rather than following focus. - Applying vocabulary replacements by simple string substitution will corrupt words that happen to contain the term, so replacing a short name mangles unrelated text throughout the transcript. Match on word boundaries with the surrounding context, and test against a transcript containing the term as a substring. ===== MILESTONES.md ===== # Delivery milestones · MacWhisper Estimated effort: **multi-day** for the indie phases; the production-only milestones add the trust and operability layer. ## M1 · Transcribing a file correctly A long recording transcribes with nothing lost at the window seams. ### Steps 1. Chunk with overlap and merge on text The window is fixed. The seams are where the words go missing. Files: `src/chunk.swift`, `tests/ChunkTests.swift` 2. Verify against a single-pass transcription A shorter file transcribed whole is the reference. Compare word counts. Files: `tests/SeamTests.swift` 3. Report progress and allow cancellation An hour of audio is minutes of work. Silence during it feels broken. Files: `src/Transcriber.swift` ### Done when - [ ] An hour transcribes with no seam losses - [ ] Word count matches a single-pass reference - [ ] Progress reports and cancellation is prompt - [ ] Transcription works offline ### Watch out - The model processes fixed-length windows, so chunking without overlap drops words at every boundary and chunking with naive overlap duplicates them. Merge by matching text across the overlap, and verify the word count and final timestamp against a single-pass transcription of a shorter file. ## M2 · Not letting the model talk to itself Silence produces silence rather than invented sentences. ### Steps 1. Gate on speech activity before the model Nothing reaches the model without speech in it. Files: `src/VAD.swift` 2. Tune against real background noise A fan, typing, a room. Synthetic silence is much easier than real quiet. Files: `tests/NoiseTests.swift` 3. Log the gating rate A detector that never gates is not working, and you will not notice otherwise. Files: `src/VAD.swift` ### Done when - [ ] Thirty seconds of silence produces no text - [ ] Background noise does not trigger recognition - [ ] Quiet speech is not gated out - [ ] The gating rate is logged and plausible ### Watch out - Fed a window of silence or background noise, the model will confidently produce a plausible sentence, and in dictation that means text appearing in your document when nobody spoke. This is not an efficiency optimisation, it is the thing that makes the feature usable at all. ## M3 · Live dictation, with the model changing its mind Words appear as you speak and corrections are handled gracefully. ### Steps 1. Show provisional, commit when stable Stability over a window is the commit condition, not the model returning. Files: `src/Live.swift` 2. Never revise committed text Once it has left for the target application it is the user's problem to edit. Files: `src/Live.swift` 3. Measure speech-to-text delay A number, logged. It is what the feature is judged on. Files: `src/Metrics.swift` ### Done when - [ ] Provisional text commits once stable - [ ] Committed text is never rewritten - [ ] Speech-to-visible delay is measured - [ ] Long dictation does not accumulate latency ### Watch out - Committing every recognition result immediately produces text that visibly rewrites itself as the model revises, which is unusable for dictation because you cannot type around it. Hold recent text as provisional, commit only once a window has passed without it changing, and never revise text already committed to the target application. ## M4 · Typing into whatever is in front of you Dictated text lands in the application you were already using. ### Steps 1. Capture the target at start Not at commit. Following focus types into the wrong window. Files: `src/Insert.swift` 2. Fall back to a review buffer Some applications reject synthetic input. Losing the text is not acceptable. Files: `src/Buffer.swift` 3. Explain the permission honestly Accessibility access is broad. Say what it is used for. Files: `src/Permissions.swift` ### Done when - [ ] Text lands in the application focused at start - [ ] A focus change stops insertion rather than misdirecting it - [ ] A rejecting application falls back to the buffer - [ ] The permission is explained before being requested ### Watch out - Inserting into whatever happens to be focused at the moment text commits will type into the wrong window when focus changes mid-sentence, which in the wrong application is anything from embarrassing to destructive. Capture the target at the start of dictation and stop rather than following focus. ## M5 · Vocabulary, batch work and keeping it local (production only) Names come out right, files process unattended, and nothing leaves the machine. ### Steps 1. Replace on word boundaries with context Simple substitution corrupts words containing the term. Files: `src/Vocabulary.swift`, `tests/VocabularyTests.swift` 2. Batch a folder with subtitle output Timings must be correct in the subtitle format, which is where they usually are not. Files: `src/Batch.swift` 3. Verify every feature offline The privacy claim is the reason to use this. Prove it rather than stating it. Files: `tests/OfflineTests.swift` ### Done when - [ ] Vocabulary replacement does not corrupt substrings - [ ] A folder transcribes unattended with correct subtitle timings - [ ] Every feature works with networking disabled - [ ] Model downloads are the only network use and are deliberate ### Watch out - Applying vocabulary replacements by simple string substitution will corrupt words that happen to contain the term, so replacing a short name mangles unrelated text throughout the transcript. Match on word boundaries with the surrounding context, and test against a transcript containing the term as a substring. ===== OPERATIONS.md ===== # Operations · MacWhisper ## Backup Transcripts of files are regenerable from the sources. Vocabulary and settings are small and worth backing up, since rebuilding a list of terms the model gets wrong takes weeks of noticing. ## Restore Restore vocabulary and settings and confirm the model path still resolves, since a restore onto a machine without the model file leaves an application that appears broken rather than one that says a model is missing. Do a restore drill before the first real user, and write the date here when it passes. ## Monitoring Speech-to-visible latency and the gating rate. Latency creeping up over a long session means the rolling window is growing rather than rolling, and a gating rate of zero means the detector has stopped working. ## Incident checklist Text appearing when nobody spoke means gating failed, which is the most alarming failure since it puts words in a document. Text landing in the wrong window means insertion is following focus rather than the captured target. 1. Contain the issue without destroying evidence or user data. 2. Record the timeline and affected scope. 3. Rotate exposed secrets and revoke compromised sessions or credentials. 4. Restore from a verified backup when needed. 5. Document the root cause, the remediation and the regression test. ## Release gate - [ ] Seam merging is confirmed against a single-pass reference - [ ] Thirty seconds of silence is confirmed to produce no text - [ ] Committed text is confirmed never rewritten - [ ] A focus change is confirmed to stop insertion - [ ] Every feature is confirmed to work with networking disabled ## Launch constraint Do not market omitted MacWhisper capabilities as implemented. The non-goals in `PRODUCT.md` remain user-visible limitations until they are deliberately delivered. ===== .env.example ===== # Copy to .env and fill in. Never commit .env; this file documents it. # Required. The recognition model. Larger is more accurate and slower, which matters for live dictation. MODEL_PATH=~/Library/Application Support/dictation/ggml-medium.en.bin # Required. The global hotkey. Choose something nothing else claims, which is harder than it sounds. DICTATION_HOTKEY=Fn+Fn # Required. Speech activity sensitivity. Too high loses quiet speech, too low lets the model invent. VAD_THRESHOLD=0.5 # Required. How long text must stay unchanged before committing. Shorter feels faster and rewrites more. COMMIT_STABLE_MS=600 # Required. Names and terms with replacements. Applied on word boundaries after recognition. VOCABULARY_PATH=./config/vocabulary.txt # Required. Present as a constant so the answer is visible in configuration. Nothing else may use the network. ALLOW_NETWORK=model_download_only
README.md
# MacWhisper · indie build A dictation tool where the whole engineering problem is that the best local speech model is not a streaming model. It processes fixed windows, so live dictation means gating on speech activity, chunking with overlap, and reconciling text that the model revises as it hears more. Estimated effort: **multi-day**. Work `BUILD_PLAN.md` top to bottom · every phase ends in a check that has to pass before the next one starts. ## Stack | Part | Choice | Why | | --- | --- | --- | | Recognition | whisper.cpp with a model sized to the machine | the standard for offline recognition, and not natively a streaming one | | Gating | Voice activity detection ahead of the model | silence fed to the model produces invented sentences, not empty output | | Chunking | Overlapping windows reconciled on their overlap | the model works in fixed windows and the seams are where words are lost | | Insertion | Typing into the focused application, with a review buffer | dictation that goes somewhere you then have to copy from is not dictation | | Runtime | A native application with a global hotkey | it has to be available in whatever window you are already in | ## Before you start Have every one of these ready. The plan assumes them from step one. - [ ] **A Mac with enough memory for the model** · free if you own one - Why: Model size determines both accuracy and whether live dictation keeps up, and a machine that swaps will produce unusable latency. - Get it: Check available memory and choose a model that fits comfortably alongside your normal work. - Verify: A five minute file transcribes without the machine swapping - [ ] **whisper.cpp with a model** · free, and models are a large download - Why: The recognition engine, and the source of every timing and confidence the build depends on. - Get it: Build whisper.cpp and download a model. Recent versions include voice activity detection support, which phase 2 depends on. - Verify: A short file transcribes with word timings and the build includes voice activity detection - [ ] **Xcode and the command line tools** · free - Why: A global hotkey, accessibility insertion and audio capture are all native work. - Get it: Install from the App Store, then the command line tools. - Verify: A trivial application builds and registers a global hotkey - [ ] **Recordings of your actual working environment** · free - Why: Phase 2 is tuned against real background noise, and a quiet room is much easier than a real one with a fan, typing and a conversation next door. - Get it: Record thirty seconds of your workspace in silence, and thirty seconds while typing. - Verify: Both recordings exist and the detector gates on both - [ ] **A list of names and terms the model gets wrong** · free - Why: Vocabulary correction is only testable against terms that actually fail, and the ones that fail are specific to your work. - Get it: Dictate a few paragraphs of your own writing and note every proper noun and technical term that comes out wrong. - Verify: At least fifteen terms are listed, including one that appears as a substring of a common word ## Quick start Follow Phase 1 in `BUILD_PLAN.md`; it creates the project. Then copy `.env.example` to `.env` and fill in the values it documents. ## Honest limits This build deliberately does not replace: - Speaker diarisation. - Real-time translation. - Cloud recognition as a fallback. - Custom model training on your own voice. - cross-call team analytics - meeting-bot auto-join - live multi-speaker accuracy - calendar and CRM integrations If one of those is essential to you, that is the reason to keep paying for MacWhisper, and the README should say so rather than pretend.
BRIEF.md
# Build brief · MacWhisper The one-shot brief this plan expands. `BUILD_PLAN.md` (or `MILESTONES.md`) is the same sequence broken into steps and checks; where the two disagree, the plan wins. Build me local dictation and transcription that types into whatever window you are in to replace MacWhisper. Build it in phases, in the order below. Do not write the whole thing in one pass. Finish a phase, run its "Done when" check, fix what fails, and only then start the next phase. ### Stack (fixed, do not substitute) - Recognition: whisper.cpp with a model sized to the machine · the standard for offline recognition, and not natively a streaming one - Gating: Voice activity detection ahead of the model · silence fed to the model produces invented sentences, not empty output - Chunking: Overlapping windows reconciled on their overlap · the model works in fixed windows and the seams are where words are lost - Insertion: Typing into the focused application, with a review buffer · dictation that goes somewhere you then have to copy from is not dictation - Runtime: A native application with a global hotkey · it has to be available in whatever window you are already in ### Data model (create this before Phase 1) - sessions(id, started_at, ended_at, mode, target_app) · mode is dictation or file transcription - segments(id, session_id, ordinal, start_ms, end_ms, text, confidence, revised_from) · revised_from records that the model changed its mind - vocabulary(id, phrase, replacement, scope) · names and terms the model reliably gets wrong - files(id, path, duration_ms, model, transcribed_at, output_path) · batch transcription - settings(key, value) · hotkey, model, insertion behaviour - A segment can be superseded rather than only appended, because a streaming approximation over a non-streaming model produces revisions. ### Phase 1 · Transcribing a file correctly Build: Transcribe an audio file by chunking into windows with overlap and merging on the overlapping text. Verify against a shorter file transcribed in one pass. Report progress and allow cancellation. Trap: The model processes fixed-length windows, so chunking without overlap drops words at every boundary and chunking with naive overlap duplicates them. Merge by matching text across the overlap, and verify the word count and final timestamp against a single-pass transcription of a shorter file. Done when: An hour-long file transcribes with no words lost or duplicated at seams, verified against a single-pass transcription of a ten minute excerpt. Progress is reported and cancellation stops promptly. The final timestamp matches the audio duration. Transcription runs with networking disabled. Do not build yet: No live dictation, gating or insertion. ### Phase 2 · Not letting the model talk to itself Build: Voice activity detection ahead of the model, so windows containing no speech are never sent. Tune the detector against real background noise. Log how often it gates, since a detector that never gates is not working. Trap: Fed a window of silence or background noise, the model will confidently produce a plausible sentence, and in dictation that means text appearing in your document when nobody spoke. This is not an efficiency optimisation, it is the thing that makes the feature usable at all. Done when: Thirty seconds of room silence produces no text, tested directly. Background noise such as a fan or typing does not trigger recognition. Speech at a normal level is not gated out, tested with quiet speech. The gating rate is logged and is neither zero nor everything. Do not build yet: No live dictation or insertion yet. ### Phase 3 · Live dictation, with the model changing its mind Build: Run recognition on a rolling window while speaking, showing provisional text and committing it once stable. Handle the model revising an earlier phrase after hearing more. Measure the delay from speech to visible text. Trap: Committing every recognition result immediately produces text that visibly rewrites itself as the model revises, which is unusable for dictation because you cannot type around it. Hold recent text as provisional, commit only once a window has passed without it changing, and never revise text already committed to the target application. Done when: Speech appears as provisional text and commits once stable. Committed text is never rewritten, verified by speaking a phrase the model initially mishears and then corrects. The delay from speech to visible text is measured and stated. A long dictation does not accumulate growing latency. Do not build yet: No insertion into other applications yet. ### Phase 4 · Typing into whatever is in front of you Build: Insert committed text into the focused application, with a review buffer as an alternative. Handle applications that reject synthetic input. Document the accessibility permission the platform requires and what it grants. Trap: Inserting into whatever happens to be focused at the moment text commits will type into the wrong window when focus changes mid-sentence, which in the wrong application is anything from embarrassing to destructive. Capture the target at the start of dictation and stop rather than following focus. Done when: Text inserts into the application focused when dictation started. A focus change mid-dictation stops insertion rather than typing into the new window, tested by switching applications while speaking. An application rejecting synthetic input falls back to the review buffer with an explanation. The required permission is requested with a clear explanation of what it grants. Do not build yet: No vocabulary or batch processing yet. ### Phase 5 · Vocabulary, batch work and keeping it local Build: A vocabulary of terms and names with replacements applied after recognition. Batch transcription of a folder with subtitle output. Verify the whole application functions with networking disabled. Trap: Applying vocabulary replacements by simple string substitution will corrupt words that happen to contain the term, so replacing a short name mangles unrelated text throughout the transcript. Match on word boundaries with the surrounding context, and test against a transcript containing the term as a substring. Done when: Vocabulary replacements apply on word boundaries without corrupting words containing the term, tested with a crafted transcript. A folder of files transcribes unattended with subtitle output carrying correct timings. The entire application works with networking disabled, verified for every feature. Model files are the only large download and are fetched deliberately. Do not build yet: Nothing after this. ### Out of scope (and why) - Speaker diarisation, which is unreliable on a single mixed track and better solved by recording separately. - Real-time translation, which compounds recognition latency with translation latency. - Cloud recognition as a fallback, which would undo the reason for building this. - Custom model training on your own voice, which is a substantially larger project. ### README must contain - That the model is not a streaming model, so live dictation is an approximation over fixed windows - That speech activity gating exists because silence produces invented sentences rather than nothing - That committed text is never rewritten even when the model revises its own earlier output - That the insertion target is captured at the start so a focus change cannot misdirect text - That every feature works with networking disabled, and how to verify that
AGENTS.md
# Agent instructions · MacWhisper indie build - Read `README.md` and `BUILD_PLAN.md` before writing code. The stack is fixed: whisper.cpp with a model sized to the machine, Voice activity detection ahead of the model, Overlapping windows reconciled on their overlap, Typing into the focused application, with a review buffer, A native application with a global hotkey. Do not substitute. - Work one phase at a time, in order. Do not start a phase until every "Done when" item of the previous one passes. - Prefer the fewest moving parts that satisfy the step. No frameworks, services or dependencies the plan does not name. - Secrets live in `.env`, never in source or logs. Keep `.env.example` current when a variable is introduced. - Do not invent cryptography, security guarantees, APIs or compliance claims. - Add a focused test for every destructive, security-sensitive or data-loss path the plan names. - Run the project checks before declaring a phase complete, and record any deliberate shortcut in the README under "Tradeoffs". ## Known traps - The model processes fixed-length windows, so chunking without overlap drops words at every boundary and chunking with naive overlap duplicates them. Merge by matching text across the overlap, and verify the word count and final timestamp against a single-pass transcription of a shorter file. - Fed a window of silence or background noise, the model will confidently produce a plausible sentence, and in dictation that means text appearing in your document when nobody spoke. This is not an efficiency optimisation, it is the thing that makes the feature usable at all. - Committing every recognition result immediately produces text that visibly rewrites itself as the model revises, which is unusable for dictation because you cannot type around it. Hold recent text as provisional, commit only once a window has passed without it changing, and never revise text already committed to the target application. - Inserting into whatever happens to be focused at the moment text commits will type into the wrong window when focus changes mid-sentence, which in the wrong application is anything from embarrassing to destructive. Capture the target at the start of dictation and stop rather than following focus.
BUILD_PLAN.md
# Build plan · MacWhisper A dictation tool where the whole engineering problem is that the best local speech model is not a streaming model. It processes fixed windows, so live dictation means gating on speech activity, chunking with overlap, and reconciling text that the model revises as it hears more. Phases are in dependency order. Each ends in a "Done when" list; treat an unticked item as a blocker, not a note. ## Phase 1 · Transcribing a file correctly A long recording transcribes with nothing lost at the window seams. ### Steps 1. Chunk with overlap and merge on text The window is fixed. The seams are where the words go missing. Files: `src/chunk.swift`, `tests/ChunkTests.swift` 2. Verify against a single-pass transcription A shorter file transcribed whole is the reference. Compare word counts. Files: `tests/SeamTests.swift` 3. Report progress and allow cancellation An hour of audio is minutes of work. Silence during it feels broken. Files: `src/Transcriber.swift` ### Done when - [ ] An hour transcribes with no seam losses - [ ] Word count matches a single-pass reference - [ ] Progress reports and cancellation is prompt - [ ] Transcription works offline ### Watch out - The model processes fixed-length windows, so chunking without overlap drops words at every boundary and chunking with naive overlap duplicates them. Merge by matching text across the overlap, and verify the word count and final timestamp against a single-pass transcription of a shorter file. ## Phase 2 · Not letting the model talk to itself Silence produces silence rather than invented sentences. ### Steps 1. Gate on speech activity before the model Nothing reaches the model without speech in it. Files: `src/VAD.swift` 2. Tune against real background noise A fan, typing, a room. Synthetic silence is much easier than real quiet. Files: `tests/NoiseTests.swift` 3. Log the gating rate A detector that never gates is not working, and you will not notice otherwise. Files: `src/VAD.swift` ### Done when - [ ] Thirty seconds of silence produces no text - [ ] Background noise does not trigger recognition - [ ] Quiet speech is not gated out - [ ] The gating rate is logged and plausible ### Watch out - Fed a window of silence or background noise, the model will confidently produce a plausible sentence, and in dictation that means text appearing in your document when nobody spoke. This is not an efficiency optimisation, it is the thing that makes the feature usable at all. ## Phase 3 · Live dictation, with the model changing its mind Words appear as you speak and corrections are handled gracefully. ### Steps 1. Show provisional, commit when stable Stability over a window is the commit condition, not the model returning. Files: `src/Live.swift` 2. Never revise committed text Once it has left for the target application it is the user's problem to edit. Files: `src/Live.swift` 3. Measure speech-to-text delay A number, logged. It is what the feature is judged on. Files: `src/Metrics.swift` ### Done when - [ ] Provisional text commits once stable - [ ] Committed text is never rewritten - [ ] Speech-to-visible delay is measured - [ ] Long dictation does not accumulate latency ### Watch out - Committing every recognition result immediately produces text that visibly rewrites itself as the model revises, which is unusable for dictation because you cannot type around it. Hold recent text as provisional, commit only once a window has passed without it changing, and never revise text already committed to the target application. ## Phase 4 · Typing into whatever is in front of you Dictated text lands in the application you were already using. ### Steps 1. Capture the target at start Not at commit. Following focus types into the wrong window. Files: `src/Insert.swift` 2. Fall back to a review buffer Some applications reject synthetic input. Losing the text is not acceptable. Files: `src/Buffer.swift` 3. Explain the permission honestly Accessibility access is broad. Say what it is used for. Files: `src/Permissions.swift` ### Done when - [ ] Text lands in the application focused at start - [ ] A focus change stops insertion rather than misdirecting it - [ ] A rejecting application falls back to the buffer - [ ] The permission is explained before being requested ### Watch out - Inserting into whatever happens to be focused at the moment text commits will type into the wrong window when focus changes mid-sentence, which in the wrong application is anything from embarrassing to destructive. Capture the target at the start of dictation and stop rather than following focus. ## Not in this build - Speaker diarisation. - Real-time translation. - Cloud recognition as a fallback. - Custom model training on your own voice. ## After v1, if you want it - Commands mixed into dictation, such as spoken punctuation and new paragraph - Per-application settings, since dictating code and dictating prose want different handling - A correction pass that learns from what you edit afterwards
.env.example
# Copy to .env and fill in. Never commit .env; this file documents it. # Required. The recognition model. Larger is more accurate and slower, which matters for live dictation. MODEL_PATH=~/Library/Application Support/dictation/ggml-medium.en.bin # Required. The global hotkey. Choose something nothing else claims, which is harder than it sounds. DICTATION_HOTKEY=Fn+Fn # Required. Speech activity sensitivity. Too high loses quiet speech, too low lets the model invent. VAD_THRESHOLD=0.5 # Required. How long text must stay unchanged before committing. Shorter feels faster and rewrites more. COMMIT_STABLE_MS=600 # Required. Names and terms with replacements. Applied on word boundaries after recognition. VOCABULARY_PATH=./config/vocabulary.txt # Required. Present as a constant so the answer is visible in configuration. Nothing else may use the network. ALLOW_NETWORK=model_download_only
PRODUCT.md
# MacWhisper · product brief ## Problem MacWhisper's solo core is compact: build a private local transcription pipeline that accepts an audio file, transcribes it, creates structured notes, and exports Markdown. A competent builder can reach a useful personal version in one sitting, while the paid product mainly wins on capture, integrations, reliability. ## Product outcome Dictation that types into the window you are already in, never invents a sentence during a pause, never rewrites what it already committed, and works with the network switched off. ## Target user A builder who needs a maintainable product foundation, not a one-off demo. ## Required capabilities - Python 3.12 - FFmpeg - local Whisper model or optional API key in .env ## Explicit non-goals for v1 - Speaker diarisation. - Real-time translation. - Cloud recognition as a fallback. - Custom model training on your own voice. - cross-call team analytics - meeting-bot auto-join - live multi-speaker accuracy - calendar and CRM integrations ## Success criteria - Seam merging is confirmed against a single-pass reference - Thirty seconds of silence is confirmed to produce no text - Committed text is confirmed never rewritten - A focus change is confirmed to stop insertion - Every feature is confirmed to work with networking disabled
BRIEF.md
# Build brief · MacWhisper The one-shot brief this plan expands. `BUILD_PLAN.md` (or `MILESTONES.md`) is the same sequence broken into steps and checks; where the two disagree, the plan wins. Build me local dictation and transcription that types into whatever window you are in to replace MacWhisper. Build it in phases, in the order below. Do not write the whole thing in one pass. Finish a phase, run its "Done when" check, fix what fails, and only then start the next phase. ### Stack (fixed, do not substitute) - Recognition: whisper.cpp with a model sized to the machine · the standard for offline recognition, and not natively a streaming one - Gating: Voice activity detection ahead of the model · silence fed to the model produces invented sentences, not empty output - Chunking: Overlapping windows reconciled on their overlap · the model works in fixed windows and the seams are where words are lost - Insertion: Typing into the focused application, with a review buffer · dictation that goes somewhere you then have to copy from is not dictation - Runtime: A native application with a global hotkey · it has to be available in whatever window you are already in ### Data model (create this before Phase 1) - sessions(id, started_at, ended_at, mode, target_app) · mode is dictation or file transcription - segments(id, session_id, ordinal, start_ms, end_ms, text, confidence, revised_from) · revised_from records that the model changed its mind - vocabulary(id, phrase, replacement, scope) · names and terms the model reliably gets wrong - files(id, path, duration_ms, model, transcribed_at, output_path) · batch transcription - settings(key, value) · hotkey, model, insertion behaviour - A segment can be superseded rather than only appended, because a streaming approximation over a non-streaming model produces revisions. ### Phase 1 · Transcribing a file correctly Build: Transcribe an audio file by chunking into windows with overlap and merging on the overlapping text. Verify against a shorter file transcribed in one pass. Report progress and allow cancellation. Trap: The model processes fixed-length windows, so chunking without overlap drops words at every boundary and chunking with naive overlap duplicates them. Merge by matching text across the overlap, and verify the word count and final timestamp against a single-pass transcription of a shorter file. Done when: An hour-long file transcribes with no words lost or duplicated at seams, verified against a single-pass transcription of a ten minute excerpt. Progress is reported and cancellation stops promptly. The final timestamp matches the audio duration. Transcription runs with networking disabled. Do not build yet: No live dictation, gating or insertion. ### Phase 2 · Not letting the model talk to itself Build: Voice activity detection ahead of the model, so windows containing no speech are never sent. Tune the detector against real background noise. Log how often it gates, since a detector that never gates is not working. Trap: Fed a window of silence or background noise, the model will confidently produce a plausible sentence, and in dictation that means text appearing in your document when nobody spoke. This is not an efficiency optimisation, it is the thing that makes the feature usable at all. Done when: Thirty seconds of room silence produces no text, tested directly. Background noise such as a fan or typing does not trigger recognition. Speech at a normal level is not gated out, tested with quiet speech. The gating rate is logged and is neither zero nor everything. Do not build yet: No live dictation or insertion yet. ### Phase 3 · Live dictation, with the model changing its mind Build: Run recognition on a rolling window while speaking, showing provisional text and committing it once stable. Handle the model revising an earlier phrase after hearing more. Measure the delay from speech to visible text. Trap: Committing every recognition result immediately produces text that visibly rewrites itself as the model revises, which is unusable for dictation because you cannot type around it. Hold recent text as provisional, commit only once a window has passed without it changing, and never revise text already committed to the target application. Done when: Speech appears as provisional text and commits once stable. Committed text is never rewritten, verified by speaking a phrase the model initially mishears and then corrects. The delay from speech to visible text is measured and stated. A long dictation does not accumulate growing latency. Do not build yet: No insertion into other applications yet. ### Phase 4 · Typing into whatever is in front of you Build: Insert committed text into the focused application, with a review buffer as an alternative. Handle applications that reject synthetic input. Document the accessibility permission the platform requires and what it grants. Trap: Inserting into whatever happens to be focused at the moment text commits will type into the wrong window when focus changes mid-sentence, which in the wrong application is anything from embarrassing to destructive. Capture the target at the start of dictation and stop rather than following focus. Done when: Text inserts into the application focused when dictation started. A focus change mid-dictation stops insertion rather than typing into the new window, tested by switching applications while speaking. An application rejecting synthetic input falls back to the review buffer with an explanation. The required permission is requested with a clear explanation of what it grants. Do not build yet: No vocabulary or batch processing yet. ### Phase 5 · Vocabulary, batch work and keeping it local Build: A vocabulary of terms and names with replacements applied after recognition. Batch transcription of a folder with subtitle output. Verify the whole application functions with networking disabled. Trap: Applying vocabulary replacements by simple string substitution will corrupt words that happen to contain the term, so replacing a short name mangles unrelated text throughout the transcript. Match on word boundaries with the surrounding context, and test against a transcript containing the term as a substring. Done when: Vocabulary replacements apply on word boundaries without corrupting words containing the term, tested with a crafted transcript. A folder of files transcribes unattended with subtitle output carrying correct timings. The entire application works with networking disabled, verified for every feature. Model files are the only large download and are fetched deliberately. Do not build yet: Nothing after this. ### Out of scope (and why) - Speaker diarisation, which is unreliable on a single mixed track and better solved by recording separately. - Real-time translation, which compounds recognition latency with translation latency. - Cloud recognition as a fallback, which would undo the reason for building this. - Custom model training on your own voice, which is a substantially larger project. ### README must contain - That the model is not a streaming model, so live dictation is an approximation over fixed windows - That speech activity gating exists because silence produces invented sentences rather than nothing - That committed text is never rewritten even when the model revises its own earlier output - That the insertion target is captured at the start so a focus change cannot misdirect text - That every feature works with networking disabled, and how to verify that
ARCHITECTURE.md
# Architecture · MacWhisper ## Stack | Part | Choice | Why | | --- | --- | --- | | Recognition | whisper.cpp with a model sized to the machine | the standard for offline recognition, and not natively a streaming one | | Gating | Voice activity detection ahead of the model | silence fed to the model produces invented sentences, not empty output | | Chunking | Overlapping windows reconciled on their overlap | the model works in fixed windows and the seams are where words are lost | | Insertion | Typing into the focused application, with a review buffer | dictation that goes somewhere you then have to copy from is not dictation | | Runtime | A native application with a global hotkey | it has to be available in whatever window you are already in | ## Modules Each module has one owner concern and a documented way to replace it. | Module | Owns | How to replace it | | --- | --- | --- | | Chunking | Overlapping windows and seam merging. | Any strategy, provided the seams are verified against a single-pass reference. | | Gating | Speech activity detection ahead of the model. | Any detector. Removing it entirely is what produces invented text. | | Live | Provisional text and stable commits. | Any policy, provided committed text is never rewritten. | | Insertion | Typing into the captured target. | Any mechanism with a review buffer fallback, since some applications refuse synthetic input. | | Vocabulary | Post-recognition corrections. | Any matcher that respects word boundaries. | ## Configuration Every runtime setting is an environment variable documented in `.env.example`, validated at startup, with a safe local default wherever one exists. - `MODEL_PATH` · required · The recognition model. Larger is more accurate and slower, which matters for live dictation. - `DICTATION_HOTKEY` · required · The global hotkey. Choose something nothing else claims, which is harder than it sounds. - `VAD_THRESHOLD` · required · Speech activity sensitivity. Too high loses quiet speech, too low lets the model invent. - `COMMIT_STABLE_MS` · required · How long text must stay unchanged before committing. Shorter feels faster and rewrites more. - `VOCABULARY_PATH` · required · Names and terms with replacements. Applied on word boundaries after recognition. - `ALLOW_NETWORK` · required · Present as a constant so the answer is visible in configuration. Nothing else may use the network. ## Production baseline - Security: least privilege, input validation at every boundary, secret redaction in logs, rate limits on abuse-prone paths, no invented security primitives. - Data: explicit schema and migrations, transactional writes where integrity matters, backup and restore procedures that have been exercised. - Integrations: adapters around third-party providers, idempotent webhook or job processing, bounded retries, timeouts. - Observability: structured logs with request or operation ids, an error-tracking hook, and health and readiness checks where a server exists. - Quality: unit tests for domain rules, integration tests at module boundaries, one end-to-end test of the critical path. ## Decision records For each dependency in the stack table, keep a short note: why it was chosen, its failure mode, and how it is replaced. Do not add infrastructure until a requirement in `PRODUCT.md` justifies it.
AGENTS.md
# Agent instructions · MacWhisper product build - Read `PRODUCT.md` and `ARCHITECTURE.md` before changing code. The stack is fixed: whisper.cpp with a model sized to the machine, Voice activity detection ahead of the model, Overlapping windows reconciled on their overlap, Typing into the focused application, with a review buffer, A native application with a global hotkey. - Implement milestone by milestone from `MILESTONES.md`; keep each change reviewable and leave the application runnable at every commit. - Treat authentication, payments, encryption, imports, webhooks and destructive actions as high-risk boundaries when present. - Never invent cryptography or silently weaken a requirement to make a check pass. - Put every external service behind an interface with a deterministic fake for tests. - Add migrations and rollback or recovery notes for every persistent data change. - Log useful operational context without credentials, tokens, passwords or personal data. - Update documentation and run every check before completing a milestone. ## Known traps - The model processes fixed-length windows, so chunking without overlap drops words at every boundary and chunking with naive overlap duplicates them. Merge by matching text across the overlap, and verify the word count and final timestamp against a single-pass transcription of a shorter file. - Fed a window of silence or background noise, the model will confidently produce a plausible sentence, and in dictation that means text appearing in your document when nobody spoke. This is not an efficiency optimisation, it is the thing that makes the feature usable at all. - Committing every recognition result immediately produces text that visibly rewrites itself as the model revises, which is unusable for dictation because you cannot type around it. Hold recent text as provisional, commit only once a window has passed without it changing, and never revise text already committed to the target application. - Inserting into whatever happens to be focused at the moment text commits will type into the wrong window when focus changes mid-sentence, which in the wrong application is anything from embarrassing to destructive. Capture the target at the start of dictation and stop rather than following focus. - Applying vocabulary replacements by simple string substitution will corrupt words that happen to contain the term, so replacing a short name mangles unrelated text throughout the transcript. Match on word boundaries with the surrounding context, and test against a transcript containing the term as a substring.
MILESTONES.md
# Delivery milestones · MacWhisper Estimated effort: **multi-day** for the indie phases; the production-only milestones add the trust and operability layer. ## M1 · Transcribing a file correctly A long recording transcribes with nothing lost at the window seams. ### Steps 1. Chunk with overlap and merge on text The window is fixed. The seams are where the words go missing. Files: `src/chunk.swift`, `tests/ChunkTests.swift` 2. Verify against a single-pass transcription A shorter file transcribed whole is the reference. Compare word counts. Files: `tests/SeamTests.swift` 3. Report progress and allow cancellation An hour of audio is minutes of work. Silence during it feels broken. Files: `src/Transcriber.swift` ### Done when - [ ] An hour transcribes with no seam losses - [ ] Word count matches a single-pass reference - [ ] Progress reports and cancellation is prompt - [ ] Transcription works offline ### Watch out - The model processes fixed-length windows, so chunking without overlap drops words at every boundary and chunking with naive overlap duplicates them. Merge by matching text across the overlap, and verify the word count and final timestamp against a single-pass transcription of a shorter file. ## M2 · Not letting the model talk to itself Silence produces silence rather than invented sentences. ### Steps 1. Gate on speech activity before the model Nothing reaches the model without speech in it. Files: `src/VAD.swift` 2. Tune against real background noise A fan, typing, a room. Synthetic silence is much easier than real quiet. Files: `tests/NoiseTests.swift` 3. Log the gating rate A detector that never gates is not working, and you will not notice otherwise. Files: `src/VAD.swift` ### Done when - [ ] Thirty seconds of silence produces no text - [ ] Background noise does not trigger recognition - [ ] Quiet speech is not gated out - [ ] The gating rate is logged and plausible ### Watch out - Fed a window of silence or background noise, the model will confidently produce a plausible sentence, and in dictation that means text appearing in your document when nobody spoke. This is not an efficiency optimisation, it is the thing that makes the feature usable at all. ## M3 · Live dictation, with the model changing its mind Words appear as you speak and corrections are handled gracefully. ### Steps 1. Show provisional, commit when stable Stability over a window is the commit condition, not the model returning. Files: `src/Live.swift` 2. Never revise committed text Once it has left for the target application it is the user's problem to edit. Files: `src/Live.swift` 3. Measure speech-to-text delay A number, logged. It is what the feature is judged on. Files: `src/Metrics.swift` ### Done when - [ ] Provisional text commits once stable - [ ] Committed text is never rewritten - [ ] Speech-to-visible delay is measured - [ ] Long dictation does not accumulate latency ### Watch out - Committing every recognition result immediately produces text that visibly rewrites itself as the model revises, which is unusable for dictation because you cannot type around it. Hold recent text as provisional, commit only once a window has passed without it changing, and never revise text already committed to the target application. ## M4 · Typing into whatever is in front of you Dictated text lands in the application you were already using. ### Steps 1. Capture the target at start Not at commit. Following focus types into the wrong window. Files: `src/Insert.swift` 2. Fall back to a review buffer Some applications reject synthetic input. Losing the text is not acceptable. Files: `src/Buffer.swift` 3. Explain the permission honestly Accessibility access is broad. Say what it is used for. Files: `src/Permissions.swift` ### Done when - [ ] Text lands in the application focused at start - [ ] A focus change stops insertion rather than misdirecting it - [ ] A rejecting application falls back to the buffer - [ ] The permission is explained before being requested ### Watch out - Inserting into whatever happens to be focused at the moment text commits will type into the wrong window when focus changes mid-sentence, which in the wrong application is anything from embarrassing to destructive. Capture the target at the start of dictation and stop rather than following focus. ## M5 · Vocabulary, batch work and keeping it local (production only) Names come out right, files process unattended, and nothing leaves the machine. ### Steps 1. Replace on word boundaries with context Simple substitution corrupts words containing the term. Files: `src/Vocabulary.swift`, `tests/VocabularyTests.swift` 2. Batch a folder with subtitle output Timings must be correct in the subtitle format, which is where they usually are not. Files: `src/Batch.swift` 3. Verify every feature offline The privacy claim is the reason to use this. Prove it rather than stating it. Files: `tests/OfflineTests.swift` ### Done when - [ ] Vocabulary replacement does not corrupt substrings - [ ] A folder transcribes unattended with correct subtitle timings - [ ] Every feature works with networking disabled - [ ] Model downloads are the only network use and are deliberate ### Watch out - Applying vocabulary replacements by simple string substitution will corrupt words that happen to contain the term, so replacing a short name mangles unrelated text throughout the transcript. Match on word boundaries with the surrounding context, and test against a transcript containing the term as a substring.
OPERATIONS.md
# Operations · MacWhisper ## Backup Transcripts of files are regenerable from the sources. Vocabulary and settings are small and worth backing up, since rebuilding a list of terms the model gets wrong takes weeks of noticing. ## Restore Restore vocabulary and settings and confirm the model path still resolves, since a restore onto a machine without the model file leaves an application that appears broken rather than one that says a model is missing. Do a restore drill before the first real user, and write the date here when it passes. ## Monitoring Speech-to-visible latency and the gating rate. Latency creeping up over a long session means the rolling window is growing rather than rolling, and a gating rate of zero means the detector has stopped working. ## Incident checklist Text appearing when nobody spoke means gating failed, which is the most alarming failure since it puts words in a document. Text landing in the wrong window means insertion is following focus rather than the captured target. 1. Contain the issue without destroying evidence or user data. 2. Record the timeline and affected scope. 3. Rotate exposed secrets and revoke compromised sessions or credentials. 4. Restore from a verified backup when needed. 5. Document the root cause, the remediation and the regression test. ## Release gate - [ ] Seam merging is confirmed against a single-pass reference - [ ] Thirty seconds of silence is confirmed to produce no text - [ ] Committed text is confirmed never rewritten - [ ] A focus change is confirmed to stop insertion - [ ] Every feature is confirmed to work with networking disabled ## Launch constraint Do not market omitted MacWhisper capabilities as implemented. The non-goals in `PRODUCT.md` remain user-visible limitations until they are deliberately delivered.
.env.example
# Copy to .env and fill in. Never commit .env; this file documents it. # Required. The recognition model. Larger is more accurate and slower, which matters for live dictation. MODEL_PATH=~/Library/Application Support/dictation/ggml-medium.en.bin # Required. The global hotkey. Choose something nothing else claims, which is harder than it sounds. DICTATION_HOTKEY=Fn+Fn # Required. Speech activity sensitivity. Too high loses quiet speech, too low lets the model invent. VAD_THRESHOLD=0.5 # Required. How long text must stay unchanged before committing. Shorter feels faster and rewrites more. COMMIT_STABLE_MS=600 # Required. Names and terms with replacements. Applied on word boundaries after recognition. VOCABULARY_PATH=./config/vocabulary.txt # Required. Present as a constant so the answer is visible in configuration. Nothing else may use the network. ALLOW_NETWORK=model_download_only
$ choose a build depth, inspect the files, then open the complete pack in your agent
MacWhisper: Customers pay for automatic capture, dependable speaker handling, search across calls, and notes arriving without manual file wrangling.
xcross-call team analytics
xmeeting-bot auto-join
xlive multi-speaker accuracy
xcalendar and CRM integrations
share on X ↗"I just replaced MacWhisper with one prompt"
Don't feel like building it? These folks already made it free.
BBuzzMacWhisper’s basic job, minus the Mac-only part and the invoice.open source↗SSubtitle EditA subtitle workbench with Whisper bolted in; ugly enough to be trustworthy, useful enough to stay installed.open source↗VVibeA cross-platform desktop transcriber with subtitles, batch jobs, translation and optional local summaries.open source↗all 4 free alternatives to MacWhisper →· no votes, no pay-to-list · just what's real
MacWhisper pricing
| plan | monthly | annual (per mo) | what you get |
|---|---|---|---|
| free for mac | $0/user | $0/user | Local transcription with smaller Whisper models and core export tools; no published numeric file/minute cap |
| macwhisper pro direct | custom | — | Lifetime updates; soft activation limit of 3 devices |
| mac app store subscription | custom | — | Weekly, monthly, and yearly subscription options exist; exact US prices were not exposed on the official public pages |
| macwhisper for ios | $0/user | $0/user | All iOS app features and local models are free; no subscription required |
free tierMac free tier: local transcription with smaller models and no published numeric file/minute cap; iOS features and local models are free.
billingdirect Mac license is one-time; Mac App Store edition offers weekly/monthly/yearly subscriptions; iOS is free
hidden costsCloud transcription and LLM actions can require the user's own paid API key or provider account. Direct Pro activation is softly limited to 3 devices; volume-license packs are sold separately.
verified 2026-08-14 · source ↗
Vibecode MacWhisper
Yes. A competent AI coding agent (Claude Code, Codex, Cursor) can build a usable personal MacWhisper replacement in one session with the prompt on this page. It runs on your own machine or server with no subscription.
How much does MacWhisper cost?
MacWhisper's pricing is usage-based or varies by plan · The official pricing URL is recorded, but a stable current amount was not safely recoverable in this pass. Do not publish a number until rechecked..
What do I lose by replacing MacWhisper?
Honestly: cross-call team analytics; meeting-bot auto-join; live multi-speaker accuracy; calendar and CRM integrations. If any of those are load-bearing for you, keep paying.
Is there an open-source alternative to MacWhisper?
Yes: Buzz (MacWhisper’s basic job, minus the Mac-only part and the invoice.) Subtitle Edit (A subtitle workbench with Whisper bolted in; ugly enough to be trustworthy, useful enough to stay installed.) Vibe (A cross-platform desktop transcriber with subtitles, batch jobs, translation and optional local summaries.) All 4 curated free alternatives are at vibecodeit.com/macwhisper/alternatives. The prompt is for when you want it exactly your way.