Vibecode Superwhisper
track this build5 phases, 15 steps, beginner friendly0%A push-to-talk recorder that transcribes and pastes text into the active app is very buildable for one person, especially on macOS.
You are building a lean indie version of Superwhisper. Create the following project files first, then implement the application by following them. Keep the files updated as decisions change. Do not collapse this into a single README or prompt. ===== README.md ===== # Superwhisper · indie build Hold a hotkey, speak, release, and the text appears in whatever field had focus, transcribed by a Whisper model running on your own machine, shaped by a mode that knows whether you are writing an email or a commit message, with a custom vocabulary so names and jargon come out right. Estimated effort: **weekend**. Work `BUILD_PLAN.md` top to bottom · every phase ends in a check that has to pass before the next one starts. ## Stack | Part | Choice | Why | | --- | --- | --- | | Language | Swift 6 and SwiftUI in Xcode | audio capture, hotkeys and text insertion are all AppKit | | Transcription | whisper.cpp with a locally downloaded model | runs offline, costs nothing per use, and the recording never leaves the machine | | Audio | AVAudioEngine capturing to 16 kHz mono 16 bit | whisper.cpp accepts that format specifically, and converting once at capture avoids a conversion step later | | Rewriting | An optional model call, local or hosted, per mode | a commit message and a message to a colleague want different shaping, and neither wants raw dictation | | Insertion | Pasteboard plus a synthetic Cmd-V, with the previous contents restored | per-character event posting drops characters and mangles anything outside ASCII | ## Before you start Have every one of these ready. The plan assumes them from step one. - [ ] **A Mac running macOS 14 or newer** · free if you already own one - Why: Audio capture, hotkeys and text insertion are all platform APIs. - Get it: Apple menu, then About This Mac. - Verify: About This Mac reports macOS 14 or higher - [ ] **Xcode 16 or newer** · free - Why: Builds and signs the application and links whisper.cpp. - Get it: Mac App Store or developer.apple.com/xcode. - Verify: xcodebuild -version prints 16 or higher - [ ] **whisper.cpp and a downloaded model** · free; models range from tens of megabytes to several gigabytes of disk - Why: It is the transcription engine. Without a model there is nothing to transcribe with. - Get it: Build whisper.cpp from its repository, then download a model. Start with a small English model to keep the ratio measurement quick, and try a larger one once the pipeline works. - Verify: whisper-cli transcribes a 16 kHz mono WAV correctly - [ ] **Microphone permission, and awareness of what it captures** · free - Why: Push-to-talk limits recording to when you hold the key, but the microphone still captures whatever is audible in that window. - Get it: System Settings, then Privacy and Security, then Microphone. The usage description string must be in Info.plist or the prompt never appears. - Verify: Recording produces audio with a level above silence - [ ] **Apple Developer Program membership** (optional) · 99 USD per year - Why: Product phase only. Without it there is no notarization and a Gatekeeper warning everywhere else. - Get it: Enrol at developer.apple.com/programs. - Verify: A Developer ID Application certificate appears in Xcode settings ## Quick start ```sh xcrun notarytool submit Dictate.zip --keychain-profile AC --wait ``` Then copy `.env.example` to `.env` and fill in the values it documents. ## Honest limits This build deliberately does not replace: - The native interface polish of the commercial application. - A curated library of context presets. - Per-speaker voice adaptation. - Mobile dictation. - beautiful native UX - presets - vocabulary/profile tuning - app-wide polish - support If one of those is essential to you, that is the reason to keep paying for Superwhisper, and the README should say so rather than pretend. ===== BRIEF.md ===== # Build brief · Superwhisper The one-shot brief this plan expands. `BUILD_PLAN.md` (or `MILESTONES.md`) is the same sequence broken into steps and checks; where the two disagree, the plan wins. Build me a local dictation tool with per-context modes that transcribes offline and optionally rewrites before pasting to replace Superwhisper. Build it in phases, in the order below. Do not write the whole thing in one pass. Finish a phase, run its "Done when" check, fix what fails, and only then start the next phase. ### Stack (fixed, do not substitute) - Language: Swift 6 and SwiftUI in Xcode · audio capture, hotkeys and text insertion are all AppKit - Transcription: whisper.cpp with a locally downloaded model · runs offline, costs nothing per use, and the recording never leaves the machine - Audio: AVAudioEngine capturing to 16 kHz mono 16 bit · whisper.cpp accepts that format specifically, and converting once at capture avoids a conversion step later - Rewriting: An optional model call, local or hosted, per mode · a commit message and a message to a colleague want different shaping, and neither wants raw dictation - Insertion: Pasteboard plus a synthetic Cmd-V, with the previous contents restored · per-character event posting drops characters and mangles anything outside ASCII ### Data model (create this before Phase 1) - modes(id, name, hotkey, model_size, rewrite_enabled, rewrite_prompt, target_apps_json) - transcripts(id, mode_id, at, audio_ms, raw_text, final_text, model, transcribe_ms, rewrite_ms) - vocabulary(id, term, sounds_like, mode_id) · names and jargon that Whisper gets wrong, with corrections applied after transcription - settings(key, value) · audio device, whether to keep recordings, retention period - Recordings are deleted after transcription by default. Keeping them is a setting the user turns on, not a default they discover. ### Phase 1 · Hotkey, capture and one transcription Build: Register a push-to-talk hotkey. Capture microphone audio directly at 16 kHz mono 16 bit. On release, run whisper.cpp over the buffer and display the result. Measure how long transcription takes relative to the audio length. Trap: whisper.cpp expects 16 kHz mono 16 bit PCM and behaves badly with anything else. Configure the audio engine to deliver that format directly rather than capturing at the device's native rate and resampling afterwards, which is an extra step that introduces its own artefacts. Done when: Holding the hotkey, speaking a sentence and releasing shows a correct transcription in a window. The captured audio is verified as 16 kHz mono 16 bit. The ratio of transcription time to audio length is measured and written down, because it decides whether larger models are usable. Do not build yet: No insertion, no modes, no rewriting. ### Phase 2 · Getting text into the field you were in Build: Save the pasteboard, write the transcription, send Cmd-V, then restore the previous contents after a short delay. Detect the case where insertion is refused and say so rather than leaving the user staring at an unchanged field. Trap: Do not type the text with one event per character. It drops characters under load, mangles anything outside ASCII, and behaves differently in every text engine. Trap: Restoring the pasteboard too quickly races the paste and restores before the target application has read it. Restore after a delay, and only if the pasteboard still holds what you wrote. Done when: Dictating into a text editor inserts the text and the pasteboard afterwards holds what it held before. Text containing an accented character and an emoji inserts intact. Dictating into a secure field is refused with a visible message rather than producing nothing. Do not build yet: No modes and no rewriting yet. ### Phase 3 · Modes Build: A mode carries its own hotkey, model size and rewrite behaviour. Optionally bind a mode to specific applications so the right one activates automatically. Show which mode is active during recording, because the wrong mode is otherwise invisible until too late. Trap: A larger model gives better transcription and a longer wait, and past a couple of seconds the pause breaks the flow that makes dictation worth using. Let each mode choose its own model size, so quick notes use a fast one and long-form uses a better one. Done when: Two modes with different hotkeys produce different handling of the same spoken input. A mode bound to a specific application activates automatically when that application is frontmost. The active mode is visible during recording. Switching modes does not require reopening settings. Do not build yet: No rewriting or vocabulary yet. ### Phase 4 · Vocabulary and rewriting Build: Apply a vocabulary of corrections after transcription, matching on what the model actually produces rather than on what you meant. Optionally send the text to a model with the mode's prompt to remove filler and apply formatting. Always keep the raw transcription so a bad rewrite is recoverable. Trap: A rewrite step can quietly change meaning as well as style, and once it has replaced the raw text you cannot tell. Store both, show the raw text when the user asks, and keep rewrite prompts narrow: remove filler and fix punctuation, not improve the message. Done when: A name Whisper consistently mishears is corrected by a vocabulary entry. The raw transcription is stored alongside the rewritten text and can be viewed. A rewrite that fails or times out inserts the raw transcription rather than nothing. Disabling rewrite for a mode makes insertion measurably faster. Do not build yet: No history or retention yet. ### Phase 5 · History, retention and privacy Build: A searchable history of transcriptions. Delete audio immediately after transcription by default, with retention as an explicit opt-in. Make deletion cover the audio, the transcript and the search index together. Trap: Dictation captures whatever was said near the microphone, including things the user did not intend to record. Default to deleting audio at once, make retention a deliberate choice, and never make the history the only place a transcription exists without saying so. Done when: Audio files are absent from disk after a transcription completes with default settings. Enabling retention keeps them and the setting states where they are. Deleting a history entry removes the transcript, the audio if kept and the search entry, verified by searching for the deleted text afterwards. Do not build yet: Distribution is the last phase. ### Phase 6 · Signing, notarization and updates Build: Hardened runtime, Developer ID signing, notarization, stapling and Sparkle updates. Ship or fetch the model deliberately, since models are large and licence terms vary. Trap: Whisper models are hundreds of megabytes to several gigabytes. Bundling one makes the download enormous and may not be permitted by its licence; downloading on first run needs a progress indicator, a checksum and a resumable transfer, because a failed model download leaves the application useless with no explanation. Done when: A notarized build opens on a fresh Mac with no Gatekeeper warning. First run downloads the model with visible progress and verifies a checksum. An interrupted download resumes rather than starting again. The README states what is recorded, what is deleted and what leaves the machine. Do not build yet: Nothing after this. ### Out of scope (and why) - The native interface polish of the commercial application, which is most of what people are paying for in this category. - The curated preset library covering dozens of contexts, which is accumulated tuning rather than code. - Per-speaker voice profile adaptation, which needs training infrastructure and data you do not have. - Mobile dictation, where the platform provides its own and the integration points are entirely different. ### README must contain - That transcription runs locally and audio does not leave the machine, unless a hosted rewrite model is configured - That audio is deleted immediately after transcription unless retention is enabled - The transcription time to audio ratio per model size, so mode choices are informed - That insertion uses the pasteboard and restores it, and that secure fields refuse input - Which model is used, its licence, and where it is downloaded from ===== AGENTS.md ===== # Agent instructions · Superwhisper indie build - Read `README.md` and `BUILD_PLAN.md` before writing code. The stack is fixed: Swift 6 and SwiftUI in Xcode, whisper.cpp with a locally downloaded model, AVAudioEngine capturing to 16 kHz mono 16 bit, An optional model call, local or hosted, per mode, Pasteboard plus a synthetic Cmd-V, with the previous contents restored. Do not substitute. - Work one phase at a time, in order. Do not start a phase until every "Done when" item of the previous one passes. - Prefer the fewest moving parts that satisfy the step. No frameworks, services or dependencies the plan does not name. - Secrets live in `.env`, never in source or logs. Keep `.env.example` current when a variable is introduced. - Do not invent cryptography, security guarantees, APIs or compliance claims. - Add a focused test for every destructive, security-sensitive or data-loss path the plan names. - Run the project checks before declaring a phase complete, and record any deliberate shortcut in the README under "Tradeoffs". ## Known traps - whisper.cpp expects 16 kHz mono 16 bit PCM and behaves badly with anything else. Configure the audio engine to deliver that format directly rather than capturing at the device's native rate and resampling afterwards, which is an extra step that introduces its own artefacts. - Do not type the text with one event per character. It drops characters under load, mangles anything outside ASCII, and behaves differently in every text engine. - Restoring the pasteboard too quickly races the paste and restores before the target application has read it. Restore after a delay, and only if the pasteboard still holds what you wrote. - A larger model gives better transcription and a longer wait, and past a couple of seconds the pause breaks the flow that makes dictation worth using. Let each mode choose its own model size, so quick notes use a fast one and long-form uses a better one. - A rewrite step can quietly change meaning as well as style, and once it has replaced the raw text you cannot tell. Store both, show the raw text when the user asks, and keep rewrite prompts narrow: remove filler and fix punctuation, not improve the message. - Dictation captures whatever was said near the microphone, including things the user did not intend to record. Default to deleting audio at once, make retention a deliberate choice, and never make the history the only place a transcription exists without saying so. ===== BUILD_PLAN.md ===== # Build plan · Superwhisper Hold a hotkey, speak, release, and the text appears in whatever field had focus, transcribed by a Whisper model running on your own machine, shaped by a mode that knows whether you are writing an email or a commit message, with a custom vocabulary so names and jargon come out right. Phases are in dependency order. Each ends in a "Done when" list; treat an unticked item as a blocker, not a note. ## Phase 1 · Hotkey, capture and one transcription Hold a key, speak, release, and see the text in a window. ### Steps 1. Register a push-to-talk hotkey RegisterEventHotKey for the press, and track the release. Push-to-talk avoids the whole problem of deciding when speech has ended. Files: `Hotkey.swift` 2. Capture in whisper's format directly AVAudioEngine with an output format of 16 kHz mono 16 bit. Microphone access is its own permission with its own usage string in Info.plist. Files: `Audio.swift`, `Info.plist` 3. Run whisper.cpp and time it Link the library or invoke the binary. Record transcription milliseconds against audio milliseconds for several utterances. Files: `Transcribe.swift` ### Done when - [ ] A spoken sentence transcribes correctly into a window - [ ] Captured audio is confirmed 16 kHz mono 16 bit - [ ] The transcription to audio time ratio is measured and recorded - [ ] Denying microphone access shows an explanation rather than failing silently ### Watch out - whisper.cpp expects 16 kHz mono 16 bit PCM and behaves badly with anything else. Configure the audio engine to deliver that format directly rather than capturing at the device's native rate and resampling afterwards, which is an extra step that introduces its own artefacts. ## Phase 2 · Getting text into the field you were in The transcription lands in whatever had focus, without losing your clipboard. ### Steps 1. Save, write, paste, restore Capture the existing pasteboard items first. Restore after a delay, and only if your value is still there. Files: `Insert.swift` 2. Handle refusal visibly Secure text fields refuse synthetic input by design. Detect and show a brief message. Files: `Insert.swift` 3. Test across text engines A native editor, a browser field and a terminal. They behave differently and one of them will surprise you. ### Done when - [ ] Text inserts into the previously focused field - [ ] The pasteboard afterwards holds its previous contents - [ ] Accented characters and emoji insert intact - [ ] A secure field produces a visible refusal ### Watch out - Do not type the text with one event per character. It drops characters under load, mangles anything outside ASCII, and behaves differently in every text engine. - Restoring the pasteboard too quickly races the paste and restores before the target application has read it. Restore after a delay, and only if the pasteboard still holds what you wrote. ## Phase 3 · Modes Dictating a commit message and dictating an email produce appropriately different text. ### Steps 1. Make modes first-class Hotkey, model, rewrite settings and application bindings per mode. Not a global setting with overrides. Files: `Modes.swift` 2. Bind modes to applications Match the frontmost bundle identifier when recording starts, not when the hotkey is registered. Files: `Modes.swift` 3. Show the active mode while recording A small indicator naming the mode. Discovering the wrong mode after speaking for a minute is the failure this prevents. Files: `Indicator.swift` ### Done when - [ ] Two modes handle the same input differently - [ ] An application-bound mode activates automatically - [ ] The active mode is visible during recording - [ ] Each mode uses its own configured model size ### Watch out - A larger model gives better transcription and a longer wait, and past a couple of seconds the pause breaks the flow that makes dictation worth using. Let each mode choose its own model size, so quick notes use a fast one and long-form uses a better one. ## Phase 4 · Vocabulary and rewriting Names and jargon come out right, and the text reads like writing rather than speech. ### Steps 1. Build the vocabulary on observed errors Record what the model produced, then map it. Guessing at misrecognitions is far less effective than reading your own transcripts. Files: `Vocabulary.swift` 2. Keep rewrite prompts narrow Filler removal and punctuation. A prompt that invites improvement will rewrite your meaning. Files: `Rewrite.swift` 3. Fall back to raw on failure A timeout or an error inserts the transcription. Losing dictated text because a rewrite failed is the worst outcome here. Files: `Rewrite.swift` ### Done when - [ ] A vocabulary entry corrects a consistent misrecognition - [ ] Raw and rewritten text are both stored and the raw is viewable - [ ] A failed rewrite inserts the raw transcription - [ ] Disabling rewrite makes insertion measurably faster ### Watch out - A rewrite step can quietly change meaning as well as style, and once it has replaced the raw text you cannot tell. Store both, show the raw text when the user asks, and keep rewrite prompts narrow: remove filler and fix punctuation, not improve the message. ## Phase 5 · History, retention and privacy You can find what you dictated last week, and nothing is kept that you did not choose to keep. ### Steps 1. Delete audio by default Immediately after transcription. Retention is opt-in and says where files are kept. Files: `Retention.swift` 2. Make history searchable Full text search over transcripts. This is the feature that turns dictation into a record you can use. Files: `History.swift` 3. Make deletion complete One function covering transcript, audio and index. Test by searching for the deleted text. Files: `Delete.swift` ### Done when - [ ] Audio is absent from disk after transcription with default settings - [ ] Retention is opt-in and states where audio is kept - [ ] Deleting an entry removes it from search - [ ] Deleting an entry removes its audio when retention was on ### Watch out - Dictation captures whatever was said near the microphone, including things the user did not intend to record. Default to deleting audio at once, make retention a deliberate choice, and never make the history the only place a transcription exists without saying so. ## Not in this build - The native interface polish of the commercial application. - A curated library of context presets. - Per-speaker voice adaptation. - Mobile dictation. ## After v1, if you want it - Streaming transcription so text appears while you are still speaking - A mode that dictates directly into a terminal with shell-aware punctuation - Automatic vocabulary suggestions from corrections you make by hand ===== .env.example ===== # Copy to .env and fill in. Never commit .env; this file documents it. # Required. Where the downloaded model lives. Modes may point at different sizes. WHISPER_MODEL_PATH=~/Library/Application Support/Dictate/models/ggml-base.en.bin # Required. Fixed by whisper.cpp. Capture at this rate directly rather than resampling. AUDIO_SAMPLE_RATE=16000 # Required. Whether recordings are retained after transcription. False by default and deliberately so. KEEP_AUDIO=false # Optional. Only used when retention is enabled. Older recordings are deleted. AUDIO_RETENTION_DAYS=7 # Optional. Optional rewrite model. A local endpoint keeps everything on the machine; a hosted one does not, and the README must say which you chose. REWRITE_ENDPOINT=http://localhost:11434/v1/chat/completions # Required. How long to wait before restoring the pasteboard. Too short races the paste. PASTE_RESTORE_DELAY_MS=250
You are building a lean indie version of Superwhisper. Create the following project files first, then implement the application by following them. Keep the files updated as decisions change. Do not collapse this into a single README or prompt. ===== README.md ===== # Superwhisper · indie build Hold a hotkey, speak, release, and the text appears in whatever field had focus, transcribed by a Whisper model running on your own machine, shaped by a mode that knows whether you are writing an email or a commit message, with a custom vocabulary so names and jargon come out right. Estimated effort: **weekend**. Work `BUILD_PLAN.md` top to bottom · every phase ends in a check that has to pass before the next one starts. ## Stack | Part | Choice | Why | | --- | --- | --- | | Language | Swift 6 and SwiftUI in Xcode | audio capture, hotkeys and text insertion are all AppKit | | Transcription | whisper.cpp with a locally downloaded model | runs offline, costs nothing per use, and the recording never leaves the machine | | Audio | AVAudioEngine capturing to 16 kHz mono 16 bit | whisper.cpp accepts that format specifically, and converting once at capture avoids a conversion step later | | Rewriting | An optional model call, local or hosted, per mode | a commit message and a message to a colleague want different shaping, and neither wants raw dictation | | Insertion | Pasteboard plus a synthetic Cmd-V, with the previous contents restored | per-character event posting drops characters and mangles anything outside ASCII | ## Before you start Have every one of these ready. The plan assumes them from step one. - [ ] **A Mac running macOS 14 or newer** · free if you already own one - Why: Audio capture, hotkeys and text insertion are all platform APIs. - Get it: Apple menu, then About This Mac. - Verify: About This Mac reports macOS 14 or higher - [ ] **Xcode 16 or newer** · free - Why: Builds and signs the application and links whisper.cpp. - Get it: Mac App Store or developer.apple.com/xcode. - Verify: xcodebuild -version prints 16 or higher - [ ] **whisper.cpp and a downloaded model** · free; models range from tens of megabytes to several gigabytes of disk - Why: It is the transcription engine. Without a model there is nothing to transcribe with. - Get it: Build whisper.cpp from its repository, then download a model. Start with a small English model to keep the ratio measurement quick, and try a larger one once the pipeline works. - Verify: whisper-cli transcribes a 16 kHz mono WAV correctly - [ ] **Microphone permission, and awareness of what it captures** · free - Why: Push-to-talk limits recording to when you hold the key, but the microphone still captures whatever is audible in that window. - Get it: System Settings, then Privacy and Security, then Microphone. The usage description string must be in Info.plist or the prompt never appears. - Verify: Recording produces audio with a level above silence - [ ] **Apple Developer Program membership** (optional) · 99 USD per year - Why: Product phase only. Without it there is no notarization and a Gatekeeper warning everywhere else. - Get it: Enrol at developer.apple.com/programs. - Verify: A Developer ID Application certificate appears in Xcode settings ## Quick start ```sh xcrun notarytool submit Dictate.zip --keychain-profile AC --wait ``` Then copy `.env.example` to `.env` and fill in the values it documents. ## Honest limits This build deliberately does not replace: - The native interface polish of the commercial application. - A curated library of context presets. - Per-speaker voice adaptation. - Mobile dictation. - beautiful native UX - presets - vocabulary/profile tuning - app-wide polish - support If one of those is essential to you, that is the reason to keep paying for Superwhisper, and the README should say so rather than pretend. ===== BRIEF.md ===== # Build brief · Superwhisper The one-shot brief this plan expands. `BUILD_PLAN.md` (or `MILESTONES.md`) is the same sequence broken into steps and checks; where the two disagree, the plan wins. Build me a local dictation tool with per-context modes that transcribes offline and optionally rewrites before pasting to replace Superwhisper. Build it in phases, in the order below. Do not write the whole thing in one pass. Finish a phase, run its "Done when" check, fix what fails, and only then start the next phase. ### Stack (fixed, do not substitute) - Language: Swift 6 and SwiftUI in Xcode · audio capture, hotkeys and text insertion are all AppKit - Transcription: whisper.cpp with a locally downloaded model · runs offline, costs nothing per use, and the recording never leaves the machine - Audio: AVAudioEngine capturing to 16 kHz mono 16 bit · whisper.cpp accepts that format specifically, and converting once at capture avoids a conversion step later - Rewriting: An optional model call, local or hosted, per mode · a commit message and a message to a colleague want different shaping, and neither wants raw dictation - Insertion: Pasteboard plus a synthetic Cmd-V, with the previous contents restored · per-character event posting drops characters and mangles anything outside ASCII ### Data model (create this before Phase 1) - modes(id, name, hotkey, model_size, rewrite_enabled, rewrite_prompt, target_apps_json) - transcripts(id, mode_id, at, audio_ms, raw_text, final_text, model, transcribe_ms, rewrite_ms) - vocabulary(id, term, sounds_like, mode_id) · names and jargon that Whisper gets wrong, with corrections applied after transcription - settings(key, value) · audio device, whether to keep recordings, retention period - Recordings are deleted after transcription by default. Keeping them is a setting the user turns on, not a default they discover. ### Phase 1 · Hotkey, capture and one transcription Build: Register a push-to-talk hotkey. Capture microphone audio directly at 16 kHz mono 16 bit. On release, run whisper.cpp over the buffer and display the result. Measure how long transcription takes relative to the audio length. Trap: whisper.cpp expects 16 kHz mono 16 bit PCM and behaves badly with anything else. Configure the audio engine to deliver that format directly rather than capturing at the device's native rate and resampling afterwards, which is an extra step that introduces its own artefacts. Done when: Holding the hotkey, speaking a sentence and releasing shows a correct transcription in a window. The captured audio is verified as 16 kHz mono 16 bit. The ratio of transcription time to audio length is measured and written down, because it decides whether larger models are usable. Do not build yet: No insertion, no modes, no rewriting. ### Phase 2 · Getting text into the field you were in Build: Save the pasteboard, write the transcription, send Cmd-V, then restore the previous contents after a short delay. Detect the case where insertion is refused and say so rather than leaving the user staring at an unchanged field. Trap: Do not type the text with one event per character. It drops characters under load, mangles anything outside ASCII, and behaves differently in every text engine. Trap: Restoring the pasteboard too quickly races the paste and restores before the target application has read it. Restore after a delay, and only if the pasteboard still holds what you wrote. Done when: Dictating into a text editor inserts the text and the pasteboard afterwards holds what it held before. Text containing an accented character and an emoji inserts intact. Dictating into a secure field is refused with a visible message rather than producing nothing. Do not build yet: No modes and no rewriting yet. ### Phase 3 · Modes Build: A mode carries its own hotkey, model size and rewrite behaviour. Optionally bind a mode to specific applications so the right one activates automatically. Show which mode is active during recording, because the wrong mode is otherwise invisible until too late. Trap: A larger model gives better transcription and a longer wait, and past a couple of seconds the pause breaks the flow that makes dictation worth using. Let each mode choose its own model size, so quick notes use a fast one and long-form uses a better one. Done when: Two modes with different hotkeys produce different handling of the same spoken input. A mode bound to a specific application activates automatically when that application is frontmost. The active mode is visible during recording. Switching modes does not require reopening settings. Do not build yet: No rewriting or vocabulary yet. ### Phase 4 · Vocabulary and rewriting Build: Apply a vocabulary of corrections after transcription, matching on what the model actually produces rather than on what you meant. Optionally send the text to a model with the mode's prompt to remove filler and apply formatting. Always keep the raw transcription so a bad rewrite is recoverable. Trap: A rewrite step can quietly change meaning as well as style, and once it has replaced the raw text you cannot tell. Store both, show the raw text when the user asks, and keep rewrite prompts narrow: remove filler and fix punctuation, not improve the message. Done when: A name Whisper consistently mishears is corrected by a vocabulary entry. The raw transcription is stored alongside the rewritten text and can be viewed. A rewrite that fails or times out inserts the raw transcription rather than nothing. Disabling rewrite for a mode makes insertion measurably faster. Do not build yet: No history or retention yet. ### Phase 5 · History, retention and privacy Build: A searchable history of transcriptions. Delete audio immediately after transcription by default, with retention as an explicit opt-in. Make deletion cover the audio, the transcript and the search index together. Trap: Dictation captures whatever was said near the microphone, including things the user did not intend to record. Default to deleting audio at once, make retention a deliberate choice, and never make the history the only place a transcription exists without saying so. Done when: Audio files are absent from disk after a transcription completes with default settings. Enabling retention keeps them and the setting states where they are. Deleting a history entry removes the transcript, the audio if kept and the search entry, verified by searching for the deleted text afterwards. Do not build yet: Distribution is the last phase. ### Phase 6 · Signing, notarization and updates Build: Hardened runtime, Developer ID signing, notarization, stapling and Sparkle updates. Ship or fetch the model deliberately, since models are large and licence terms vary. Trap: Whisper models are hundreds of megabytes to several gigabytes. Bundling one makes the download enormous and may not be permitted by its licence; downloading on first run needs a progress indicator, a checksum and a resumable transfer, because a failed model download leaves the application useless with no explanation. Done when: A notarized build opens on a fresh Mac with no Gatekeeper warning. First run downloads the model with visible progress and verifies a checksum. An interrupted download resumes rather than starting again. The README states what is recorded, what is deleted and what leaves the machine. Do not build yet: Nothing after this. ### Out of scope (and why) - The native interface polish of the commercial application, which is most of what people are paying for in this category. - The curated preset library covering dozens of contexts, which is accumulated tuning rather than code. - Per-speaker voice profile adaptation, which needs training infrastructure and data you do not have. - Mobile dictation, where the platform provides its own and the integration points are entirely different. ### README must contain - That transcription runs locally and audio does not leave the machine, unless a hosted rewrite model is configured - That audio is deleted immediately after transcription unless retention is enabled - The transcription time to audio ratio per model size, so mode choices are informed - That insertion uses the pasteboard and restores it, and that secure fields refuse input - Which model is used, its licence, and where it is downloaded from ===== AGENTS.md ===== # Agent instructions · Superwhisper indie build - Read `README.md` and `BUILD_PLAN.md` before writing code. The stack is fixed: Swift 6 and SwiftUI in Xcode, whisper.cpp with a locally downloaded model, AVAudioEngine capturing to 16 kHz mono 16 bit, An optional model call, local or hosted, per mode, Pasteboard plus a synthetic Cmd-V, with the previous contents restored. Do not substitute. - Work one phase at a time, in order. Do not start a phase until every "Done when" item of the previous one passes. - Prefer the fewest moving parts that satisfy the step. No frameworks, services or dependencies the plan does not name. - Secrets live in `.env`, never in source or logs. Keep `.env.example` current when a variable is introduced. - Do not invent cryptography, security guarantees, APIs or compliance claims. - Add a focused test for every destructive, security-sensitive or data-loss path the plan names. - Run the project checks before declaring a phase complete, and record any deliberate shortcut in the README under "Tradeoffs". ## Known traps - whisper.cpp expects 16 kHz mono 16 bit PCM and behaves badly with anything else. Configure the audio engine to deliver that format directly rather than capturing at the device's native rate and resampling afterwards, which is an extra step that introduces its own artefacts. - Do not type the text with one event per character. It drops characters under load, mangles anything outside ASCII, and behaves differently in every text engine. - Restoring the pasteboard too quickly races the paste and restores before the target application has read it. Restore after a delay, and only if the pasteboard still holds what you wrote. - A larger model gives better transcription and a longer wait, and past a couple of seconds the pause breaks the flow that makes dictation worth using. Let each mode choose its own model size, so quick notes use a fast one and long-form uses a better one. - A rewrite step can quietly change meaning as well as style, and once it has replaced the raw text you cannot tell. Store both, show the raw text when the user asks, and keep rewrite prompts narrow: remove filler and fix punctuation, not improve the message. - Dictation captures whatever was said near the microphone, including things the user did not intend to record. Default to deleting audio at once, make retention a deliberate choice, and never make the history the only place a transcription exists without saying so. ===== BUILD_PLAN.md ===== # Build plan · Superwhisper Hold a hotkey, speak, release, and the text appears in whatever field had focus, transcribed by a Whisper model running on your own machine, shaped by a mode that knows whether you are writing an email or a commit message, with a custom vocabulary so names and jargon come out right. Phases are in dependency order. Each ends in a "Done when" list; treat an unticked item as a blocker, not a note. ## Phase 1 · Hotkey, capture and one transcription Hold a key, speak, release, and see the text in a window. ### Steps 1. Register a push-to-talk hotkey RegisterEventHotKey for the press, and track the release. Push-to-talk avoids the whole problem of deciding when speech has ended. Files: `Hotkey.swift` 2. Capture in whisper's format directly AVAudioEngine with an output format of 16 kHz mono 16 bit. Microphone access is its own permission with its own usage string in Info.plist. Files: `Audio.swift`, `Info.plist` 3. Run whisper.cpp and time it Link the library or invoke the binary. Record transcription milliseconds against audio milliseconds for several utterances. Files: `Transcribe.swift` ### Done when - [ ] A spoken sentence transcribes correctly into a window - [ ] Captured audio is confirmed 16 kHz mono 16 bit - [ ] The transcription to audio time ratio is measured and recorded - [ ] Denying microphone access shows an explanation rather than failing silently ### Watch out - whisper.cpp expects 16 kHz mono 16 bit PCM and behaves badly with anything else. Configure the audio engine to deliver that format directly rather than capturing at the device's native rate and resampling afterwards, which is an extra step that introduces its own artefacts. ## Phase 2 · Getting text into the field you were in The transcription lands in whatever had focus, without losing your clipboard. ### Steps 1. Save, write, paste, restore Capture the existing pasteboard items first. Restore after a delay, and only if your value is still there. Files: `Insert.swift` 2. Handle refusal visibly Secure text fields refuse synthetic input by design. Detect and show a brief message. Files: `Insert.swift` 3. Test across text engines A native editor, a browser field and a terminal. They behave differently and one of them will surprise you. ### Done when - [ ] Text inserts into the previously focused field - [ ] The pasteboard afterwards holds its previous contents - [ ] Accented characters and emoji insert intact - [ ] A secure field produces a visible refusal ### Watch out - Do not type the text with one event per character. It drops characters under load, mangles anything outside ASCII, and behaves differently in every text engine. - Restoring the pasteboard too quickly races the paste and restores before the target application has read it. Restore after a delay, and only if the pasteboard still holds what you wrote. ## Phase 3 · Modes Dictating a commit message and dictating an email produce appropriately different text. ### Steps 1. Make modes first-class Hotkey, model, rewrite settings and application bindings per mode. Not a global setting with overrides. Files: `Modes.swift` 2. Bind modes to applications Match the frontmost bundle identifier when recording starts, not when the hotkey is registered. Files: `Modes.swift` 3. Show the active mode while recording A small indicator naming the mode. Discovering the wrong mode after speaking for a minute is the failure this prevents. Files: `Indicator.swift` ### Done when - [ ] Two modes handle the same input differently - [ ] An application-bound mode activates automatically - [ ] The active mode is visible during recording - [ ] Each mode uses its own configured model size ### Watch out - A larger model gives better transcription and a longer wait, and past a couple of seconds the pause breaks the flow that makes dictation worth using. Let each mode choose its own model size, so quick notes use a fast one and long-form uses a better one. ## Phase 4 · Vocabulary and rewriting Names and jargon come out right, and the text reads like writing rather than speech. ### Steps 1. Build the vocabulary on observed errors Record what the model produced, then map it. Guessing at misrecognitions is far less effective than reading your own transcripts. Files: `Vocabulary.swift` 2. Keep rewrite prompts narrow Filler removal and punctuation. A prompt that invites improvement will rewrite your meaning. Files: `Rewrite.swift` 3. Fall back to raw on failure A timeout or an error inserts the transcription. Losing dictated text because a rewrite failed is the worst outcome here. Files: `Rewrite.swift` ### Done when - [ ] A vocabulary entry corrects a consistent misrecognition - [ ] Raw and rewritten text are both stored and the raw is viewable - [ ] A failed rewrite inserts the raw transcription - [ ] Disabling rewrite makes insertion measurably faster ### Watch out - A rewrite step can quietly change meaning as well as style, and once it has replaced the raw text you cannot tell. Store both, show the raw text when the user asks, and keep rewrite prompts narrow: remove filler and fix punctuation, not improve the message. ## Phase 5 · History, retention and privacy You can find what you dictated last week, and nothing is kept that you did not choose to keep. ### Steps 1. Delete audio by default Immediately after transcription. Retention is opt-in and says where files are kept. Files: `Retention.swift` 2. Make history searchable Full text search over transcripts. This is the feature that turns dictation into a record you can use. Files: `History.swift` 3. Make deletion complete One function covering transcript, audio and index. Test by searching for the deleted text. Files: `Delete.swift` ### Done when - [ ] Audio is absent from disk after transcription with default settings - [ ] Retention is opt-in and states where audio is kept - [ ] Deleting an entry removes it from search - [ ] Deleting an entry removes its audio when retention was on ### Watch out - Dictation captures whatever was said near the microphone, including things the user did not intend to record. Default to deleting audio at once, make retention a deliberate choice, and never make the history the only place a transcription exists without saying so. ## Not in this build - The native interface polish of the commercial application. - A curated library of context presets. - Per-speaker voice adaptation. - Mobile dictation. ## After v1, if you want it - Streaming transcription so text appears while you are still speaking - A mode that dictates directly into a terminal with shell-aware punctuation - Automatic vocabulary suggestions from corrections you make by hand ===== .env.example ===== # Copy to .env and fill in. Never commit .env; this file documents it. # Required. Where the downloaded model lives. Modes may point at different sizes. WHISPER_MODEL_PATH=~/Library/Application Support/Dictate/models/ggml-base.en.bin # Required. Fixed by whisper.cpp. Capture at this rate directly rather than resampling. AUDIO_SAMPLE_RATE=16000 # Required. Whether recordings are retained after transcription. False by default and deliberately so. KEEP_AUDIO=false # Optional. Only used when retention is enabled. Older recordings are deleted. AUDIO_RETENTION_DAYS=7 # Optional. Optional rewrite model. A local endpoint keeps everything on the machine; a hosted one does not, and the README must say which you chose. REWRITE_ENDPOINT=http://localhost:11434/v1/chat/completions # Required. How long to wait before restoring the pasteboard. Too short races the paste. PASTE_RESTORE_DELAY_MS=250
You are building a production product version of Superwhisper. Create the following project files first, then implement the application by following them. Keep the files updated as decisions change. Do not collapse this into a single README or prompt. ===== PRODUCT.md ===== # Superwhisper · product brief ## Problem A push-to-talk recorder that transcribes and pastes text into the active app is very buildable for one person, especially on macOS. ## Product outcome A dictation tool that transcribes locally, inserts reliably into any text field without losing the clipboard, adapts to context through modes, corrects the words it always gets wrong, and deletes audio by default. ## Target user A builder who needs a maintainable product foundation, not a one-off demo. ## Required capabilities - macOS automation/accessibility permissions - speech API or local Whisper - optional LLM API - hotkey library ## Explicit non-goals for v1 - The native interface polish of the commercial application. - A curated library of context presets. - Per-speaker voice adaptation. - Mobile dictation. - beautiful native UX - presets - vocabulary/profile tuning - app-wide polish - support ## Success criteria - Insertion is verified in a native editor, a browser field and a terminal - The pasteboard is confirmed restored after insertion - Audio is confirmed absent from disk with default settings - The model download resumes after an interruption and verifies its checksum - The README states exactly what leaves the machine ===== BRIEF.md ===== # Build brief · Superwhisper The one-shot brief this plan expands. `BUILD_PLAN.md` (or `MILESTONES.md`) is the same sequence broken into steps and checks; where the two disagree, the plan wins. Build me a local dictation tool with per-context modes that transcribes offline and optionally rewrites before pasting to replace Superwhisper. Build it in phases, in the order below. Do not write the whole thing in one pass. Finish a phase, run its "Done when" check, fix what fails, and only then start the next phase. ### Stack (fixed, do not substitute) - Language: Swift 6 and SwiftUI in Xcode · audio capture, hotkeys and text insertion are all AppKit - Transcription: whisper.cpp with a locally downloaded model · runs offline, costs nothing per use, and the recording never leaves the machine - Audio: AVAudioEngine capturing to 16 kHz mono 16 bit · whisper.cpp accepts that format specifically, and converting once at capture avoids a conversion step later - Rewriting: An optional model call, local or hosted, per mode · a commit message and a message to a colleague want different shaping, and neither wants raw dictation - Insertion: Pasteboard plus a synthetic Cmd-V, with the previous contents restored · per-character event posting drops characters and mangles anything outside ASCII ### Data model (create this before Phase 1) - modes(id, name, hotkey, model_size, rewrite_enabled, rewrite_prompt, target_apps_json) - transcripts(id, mode_id, at, audio_ms, raw_text, final_text, model, transcribe_ms, rewrite_ms) - vocabulary(id, term, sounds_like, mode_id) · names and jargon that Whisper gets wrong, with corrections applied after transcription - settings(key, value) · audio device, whether to keep recordings, retention period - Recordings are deleted after transcription by default. Keeping them is a setting the user turns on, not a default they discover. ### Phase 1 · Hotkey, capture and one transcription Build: Register a push-to-talk hotkey. Capture microphone audio directly at 16 kHz mono 16 bit. On release, run whisper.cpp over the buffer and display the result. Measure how long transcription takes relative to the audio length. Trap: whisper.cpp expects 16 kHz mono 16 bit PCM and behaves badly with anything else. Configure the audio engine to deliver that format directly rather than capturing at the device's native rate and resampling afterwards, which is an extra step that introduces its own artefacts. Done when: Holding the hotkey, speaking a sentence and releasing shows a correct transcription in a window. The captured audio is verified as 16 kHz mono 16 bit. The ratio of transcription time to audio length is measured and written down, because it decides whether larger models are usable. Do not build yet: No insertion, no modes, no rewriting. ### Phase 2 · Getting text into the field you were in Build: Save the pasteboard, write the transcription, send Cmd-V, then restore the previous contents after a short delay. Detect the case where insertion is refused and say so rather than leaving the user staring at an unchanged field. Trap: Do not type the text with one event per character. It drops characters under load, mangles anything outside ASCII, and behaves differently in every text engine. Trap: Restoring the pasteboard too quickly races the paste and restores before the target application has read it. Restore after a delay, and only if the pasteboard still holds what you wrote. Done when: Dictating into a text editor inserts the text and the pasteboard afterwards holds what it held before. Text containing an accented character and an emoji inserts intact. Dictating into a secure field is refused with a visible message rather than producing nothing. Do not build yet: No modes and no rewriting yet. ### Phase 3 · Modes Build: A mode carries its own hotkey, model size and rewrite behaviour. Optionally bind a mode to specific applications so the right one activates automatically. Show which mode is active during recording, because the wrong mode is otherwise invisible until too late. Trap: A larger model gives better transcription and a longer wait, and past a couple of seconds the pause breaks the flow that makes dictation worth using. Let each mode choose its own model size, so quick notes use a fast one and long-form uses a better one. Done when: Two modes with different hotkeys produce different handling of the same spoken input. A mode bound to a specific application activates automatically when that application is frontmost. The active mode is visible during recording. Switching modes does not require reopening settings. Do not build yet: No rewriting or vocabulary yet. ### Phase 4 · Vocabulary and rewriting Build: Apply a vocabulary of corrections after transcription, matching on what the model actually produces rather than on what you meant. Optionally send the text to a model with the mode's prompt to remove filler and apply formatting. Always keep the raw transcription so a bad rewrite is recoverable. Trap: A rewrite step can quietly change meaning as well as style, and once it has replaced the raw text you cannot tell. Store both, show the raw text when the user asks, and keep rewrite prompts narrow: remove filler and fix punctuation, not improve the message. Done when: A name Whisper consistently mishears is corrected by a vocabulary entry. The raw transcription is stored alongside the rewritten text and can be viewed. A rewrite that fails or times out inserts the raw transcription rather than nothing. Disabling rewrite for a mode makes insertion measurably faster. Do not build yet: No history or retention yet. ### Phase 5 · History, retention and privacy Build: A searchable history of transcriptions. Delete audio immediately after transcription by default, with retention as an explicit opt-in. Make deletion cover the audio, the transcript and the search index together. Trap: Dictation captures whatever was said near the microphone, including things the user did not intend to record. Default to deleting audio at once, make retention a deliberate choice, and never make the history the only place a transcription exists without saying so. Done when: Audio files are absent from disk after a transcription completes with default settings. Enabling retention keeps them and the setting states where they are. Deleting a history entry removes the transcript, the audio if kept and the search entry, verified by searching for the deleted text afterwards. Do not build yet: Distribution is the last phase. ### Phase 6 · Signing, notarization and updates Build: Hardened runtime, Developer ID signing, notarization, stapling and Sparkle updates. Ship or fetch the model deliberately, since models are large and licence terms vary. Trap: Whisper models are hundreds of megabytes to several gigabytes. Bundling one makes the download enormous and may not be permitted by its licence; downloading on first run needs a progress indicator, a checksum and a resumable transfer, because a failed model download leaves the application useless with no explanation. Done when: A notarized build opens on a fresh Mac with no Gatekeeper warning. First run downloads the model with visible progress and verifies a checksum. An interrupted download resumes rather than starting again. The README states what is recorded, what is deleted and what leaves the machine. Do not build yet: Nothing after this. ### Out of scope (and why) - The native interface polish of the commercial application, which is most of what people are paying for in this category. - The curated preset library covering dozens of contexts, which is accumulated tuning rather than code. - Per-speaker voice profile adaptation, which needs training infrastructure and data you do not have. - Mobile dictation, where the platform provides its own and the integration points are entirely different. ### README must contain - That transcription runs locally and audio does not leave the machine, unless a hosted rewrite model is configured - That audio is deleted immediately after transcription unless retention is enabled - The transcription time to audio ratio per model size, so mode choices are informed - That insertion uses the pasteboard and restores it, and that secure fields refuse input - Which model is used, its licence, and where it is downloaded from ===== ARCHITECTURE.md ===== # Architecture · Superwhisper ## Stack | Part | Choice | Why | | --- | --- | --- | | Language | Swift 6 and SwiftUI in Xcode | audio capture, hotkeys and text insertion are all AppKit | | Transcription | whisper.cpp with a locally downloaded model | runs offline, costs nothing per use, and the recording never leaves the machine | | Audio | AVAudioEngine capturing to 16 kHz mono 16 bit | whisper.cpp accepts that format specifically, and converting once at capture avoids a conversion step later | | Rewriting | An optional model call, local or hosted, per mode | a commit message and a message to a colleague want different shaping, and neither wants raw dictation | | Insertion | Pasteboard plus a synthetic Cmd-V, with the previous contents restored | per-character event posting drops characters and mangles anything outside ASCII | ## Modules Each module has one owner concern and a documented way to replace it. | Module | Owns | How to replace it | | --- | --- | --- | | Capture | Hotkey, audio in whisper's format, and the recording lifecycle. | Capturing directly in the required format is the decision worth keeping whatever else changes. | | Transcribe | Running the model and timing it. | A hosted transcription service fits the same interface and gives up the offline and privacy properties, which is a product decision rather than a technical one. | | Insert | Pasteboard handling and the synthetic paste. | Not swappable in practice. Per-character insertion is the alternative and it is worse in every way. | | Modes | Per-context model, prompt and bindings. | Configuration rather than code, so new modes cost nothing. | | Rewrite | Optional shaping, with raw text preserved. | Any model endpoint. The rule that raw text survives a failed rewrite must not be lost. | ## Configuration Every runtime setting is an environment variable documented in `.env.example`, validated at startup, with a safe local default wherever one exists. - `WHISPER_MODEL_PATH` · required · Where the downloaded model lives. Modes may point at different sizes. - `AUDIO_SAMPLE_RATE` · required · Fixed by whisper.cpp. Capture at this rate directly rather than resampling. - `KEEP_AUDIO` · required · Whether recordings are retained after transcription. False by default and deliberately so. - `AUDIO_RETENTION_DAYS` · optional · Only used when retention is enabled. Older recordings are deleted. - `REWRITE_ENDPOINT` · optional · Optional rewrite model. A local endpoint keeps everything on the machine; a hosted one does not, and the README must say which you chose. - `PASTE_RESTORE_DELAY_MS` · required · How long to wait before restoring the pasteboard. Too short races the paste. ## Production baseline - Security: least privilege, input validation at every boundary, secret redaction in logs, rate limits on abuse-prone paths, no invented security primitives. - Data: explicit schema and migrations, transactional writes where integrity matters, backup and restore procedures that have been exercised. - Integrations: adapters around third-party providers, idempotent webhook or job processing, bounded retries, timeouts. - Observability: structured logs with request or operation ids, an error-tracking hook, and health and readiness checks where a server exists. - Quality: unit tests for domain rules, integration tests at module boundaries, one end-to-end test of the critical path. ## Decision records For each dependency in the stack table, keep a short note: why it was chosen, its failure mode, and how it is replaced. Do not add infrastructure until a requirement in `PRODUCT.md` justifies it. ===== AGENTS.md ===== # Agent instructions · Superwhisper product build - Read `PRODUCT.md` and `ARCHITECTURE.md` before changing code. The stack is fixed: Swift 6 and SwiftUI in Xcode, whisper.cpp with a locally downloaded model, AVAudioEngine capturing to 16 kHz mono 16 bit, An optional model call, local or hosted, per mode, Pasteboard plus a synthetic Cmd-V, with the previous contents restored. - Implement milestone by milestone from `MILESTONES.md`; keep each change reviewable and leave the application runnable at every commit. - Treat authentication, payments, encryption, imports, webhooks and destructive actions as high-risk boundaries when present. - Never invent cryptography or silently weaken a requirement to make a check pass. - Put every external service behind an interface with a deterministic fake for tests. - Add migrations and rollback or recovery notes for every persistent data change. - Log useful operational context without credentials, tokens, passwords or personal data. - Update documentation and run every check before completing a milestone. ## Known traps - whisper.cpp expects 16 kHz mono 16 bit PCM and behaves badly with anything else. Configure the audio engine to deliver that format directly rather than capturing at the device's native rate and resampling afterwards, which is an extra step that introduces its own artefacts. - Do not type the text with one event per character. It drops characters under load, mangles anything outside ASCII, and behaves differently in every text engine. - Restoring the pasteboard too quickly races the paste and restores before the target application has read it. Restore after a delay, and only if the pasteboard still holds what you wrote. - A larger model gives better transcription and a longer wait, and past a couple of seconds the pause breaks the flow that makes dictation worth using. Let each mode choose its own model size, so quick notes use a fast one and long-form uses a better one. - A rewrite step can quietly change meaning as well as style, and once it has replaced the raw text you cannot tell. Store both, show the raw text when the user asks, and keep rewrite prompts narrow: remove filler and fix punctuation, not improve the message. - Dictation captures whatever was said near the microphone, including things the user did not intend to record. Default to deleting audio at once, make retention a deliberate choice, and never make the history the only place a transcription exists without saying so. - Whisper models are hundreds of megabytes to several gigabytes. Bundling one makes the download enormous and may not be permitted by its licence; downloading on first run needs a progress indicator, a checksum and a resumable transfer, because a failed model download leaves the application useless with no explanation. ===== MILESTONES.md ===== # Delivery milestones · Superwhisper Estimated effort: **weekend** for the indie phases; the production-only milestones add the trust and operability layer. ## M1 · Hotkey, capture and one transcription Hold a key, speak, release, and see the text in a window. ### Steps 1. Register a push-to-talk hotkey RegisterEventHotKey for the press, and track the release. Push-to-talk avoids the whole problem of deciding when speech has ended. Files: `Hotkey.swift` 2. Capture in whisper's format directly AVAudioEngine with an output format of 16 kHz mono 16 bit. Microphone access is its own permission with its own usage string in Info.plist. Files: `Audio.swift`, `Info.plist` 3. Run whisper.cpp and time it Link the library or invoke the binary. Record transcription milliseconds against audio milliseconds for several utterances. Files: `Transcribe.swift` ### Done when - [ ] A spoken sentence transcribes correctly into a window - [ ] Captured audio is confirmed 16 kHz mono 16 bit - [ ] The transcription to audio time ratio is measured and recorded - [ ] Denying microphone access shows an explanation rather than failing silently ### Watch out - whisper.cpp expects 16 kHz mono 16 bit PCM and behaves badly with anything else. Configure the audio engine to deliver that format directly rather than capturing at the device's native rate and resampling afterwards, which is an extra step that introduces its own artefacts. ## M2 · Getting text into the field you were in The transcription lands in whatever had focus, without losing your clipboard. ### Steps 1. Save, write, paste, restore Capture the existing pasteboard items first. Restore after a delay, and only if your value is still there. Files: `Insert.swift` 2. Handle refusal visibly Secure text fields refuse synthetic input by design. Detect and show a brief message. Files: `Insert.swift` 3. Test across text engines A native editor, a browser field and a terminal. They behave differently and one of them will surprise you. ### Done when - [ ] Text inserts into the previously focused field - [ ] The pasteboard afterwards holds its previous contents - [ ] Accented characters and emoji insert intact - [ ] A secure field produces a visible refusal ### Watch out - Do not type the text with one event per character. It drops characters under load, mangles anything outside ASCII, and behaves differently in every text engine. - Restoring the pasteboard too quickly races the paste and restores before the target application has read it. Restore after a delay, and only if the pasteboard still holds what you wrote. ## M3 · Modes Dictating a commit message and dictating an email produce appropriately different text. ### Steps 1. Make modes first-class Hotkey, model, rewrite settings and application bindings per mode. Not a global setting with overrides. Files: `Modes.swift` 2. Bind modes to applications Match the frontmost bundle identifier when recording starts, not when the hotkey is registered. Files: `Modes.swift` 3. Show the active mode while recording A small indicator naming the mode. Discovering the wrong mode after speaking for a minute is the failure this prevents. Files: `Indicator.swift` ### Done when - [ ] Two modes handle the same input differently - [ ] An application-bound mode activates automatically - [ ] The active mode is visible during recording - [ ] Each mode uses its own configured model size ### Watch out - A larger model gives better transcription and a longer wait, and past a couple of seconds the pause breaks the flow that makes dictation worth using. Let each mode choose its own model size, so quick notes use a fast one and long-form uses a better one. ## M4 · Vocabulary and rewriting Names and jargon come out right, and the text reads like writing rather than speech. ### Steps 1. Build the vocabulary on observed errors Record what the model produced, then map it. Guessing at misrecognitions is far less effective than reading your own transcripts. Files: `Vocabulary.swift` 2. Keep rewrite prompts narrow Filler removal and punctuation. A prompt that invites improvement will rewrite your meaning. Files: `Rewrite.swift` 3. Fall back to raw on failure A timeout or an error inserts the transcription. Losing dictated text because a rewrite failed is the worst outcome here. Files: `Rewrite.swift` ### Done when - [ ] A vocabulary entry corrects a consistent misrecognition - [ ] Raw and rewritten text are both stored and the raw is viewable - [ ] A failed rewrite inserts the raw transcription - [ ] Disabling rewrite makes insertion measurably faster ### Watch out - A rewrite step can quietly change meaning as well as style, and once it has replaced the raw text you cannot tell. Store both, show the raw text when the user asks, and keep rewrite prompts narrow: remove filler and fix punctuation, not improve the message. ## M5 · History, retention and privacy You can find what you dictated last week, and nothing is kept that you did not choose to keep. ### Steps 1. Delete audio by default Immediately after transcription. Retention is opt-in and says where files are kept. Files: `Retention.swift` 2. Make history searchable Full text search over transcripts. This is the feature that turns dictation into a record you can use. Files: `History.swift` 3. Make deletion complete One function covering transcript, audio and index. Test by searching for the deleted text. Files: `Delete.swift` ### Done when - [ ] Audio is absent from disk after transcription with default settings - [ ] Retention is opt-in and states where audio is kept - [ ] Deleting an entry removes it from search - [ ] Deleting an entry removes its audio when retention was on ### Watch out - Dictation captures whatever was said near the microphone, including things the user did not intend to record. Default to deleting audio at once, make retention a deliberate choice, and never make the history the only place a transcription exists without saying so. ## M6 · Signing, notarization and updates (production only) Others can install it, and the permission story is stated plainly. ### Steps 1. Sign and notarize Hardened runtime, Developer ID, notarytool, stapler. Confirm the microphone usage string is present. ```sh xcrun notarytool submit Dictate.zip --keychain-profile AC --wait ``` 2. Fetch models with verification Progress, checksum and resume. Record each model's licence. Files: `ModelDownload.swift`, `docs/models.md` 3. State the privacy position What is recorded, what is deleted, and whether anything leaves the machine, which depends on whether rewriting uses a hosted model. Files: `README.md` ### Done when - [ ] spctl reports the notarized build accepted - [ ] First run downloads the model with progress and a checksum - [ ] An interrupted download resumes - [ ] The README states what leaves the machine and when ### Watch out - Whisper models are hundreds of megabytes to several gigabytes. Bundling one makes the download enormous and may not be permitted by its licence; downloading on first run needs a progress indicator, a checksum and a resumable transfer, because a failed model download leaves the application useless with no explanation. ===== OPERATIONS.md ===== # Operations · Superwhisper ## Backup Modes, vocabulary and history are the state worth keeping, and they are small. Audio, when retained, is large and rarely worth backing up. ## Restore Restore the configuration and history, then re-download the model, which is large and re-fetchable. Keeping models out of the backup is deliberate. Do a restore drill before the first real user, and write the date here when it passes. ## Monitoring Track transcription time against audio length per model. A rising ratio usually means thermal throttling or a larger model chosen in a mode where it does not belong. ## Incident checklist A macOS update changing audio or Accessibility behaviour is the recurring risk. The other realistic failure is a model file that is corrupt after an interrupted download, which is why the checksum exists. 1. Contain the issue without destroying evidence or user data. 2. Record the timeline and affected scope. 3. Rotate exposed secrets and revoke compromised sessions or credentials. 4. Restore from a verified backup when needed. 5. Document the root cause, the remediation and the regression test. ## Release gate - [ ] Insertion is verified in a native editor, a browser field and a terminal - [ ] The pasteboard is confirmed restored after insertion - [ ] Audio is confirmed absent from disk with default settings - [ ] The model download resumes after an interruption and verifies its checksum - [ ] The README states exactly what leaves the machine ## Launch constraint Do not market omitted Superwhisper capabilities as implemented. The non-goals in `PRODUCT.md` remain user-visible limitations until they are deliberately delivered. ===== .env.example ===== # Copy to .env and fill in. Never commit .env; this file documents it. # Required. Where the downloaded model lives. Modes may point at different sizes. WHISPER_MODEL_PATH=~/Library/Application Support/Dictate/models/ggml-base.en.bin # Required. Fixed by whisper.cpp. Capture at this rate directly rather than resampling. AUDIO_SAMPLE_RATE=16000 # Required. Whether recordings are retained after transcription. False by default and deliberately so. KEEP_AUDIO=false # Optional. Only used when retention is enabled. Older recordings are deleted. AUDIO_RETENTION_DAYS=7 # Optional. Optional rewrite model. A local endpoint keeps everything on the machine; a hosted one does not, and the README must say which you chose. REWRITE_ENDPOINT=http://localhost:11434/v1/chat/completions # Required. How long to wait before restoring the pasteboard. Too short races the paste. PASTE_RESTORE_DELAY_MS=250
# Superwhisper · indie build Hold a hotkey, speak, release, and the text appears in whatever field had focus, transcribed by a Whisper model running on your own machine, shaped by a mode that knows whether you are writing an email or a commit message, with a custom vocabulary so names and jargon come out right. Estimated effort: **weekend**. Work `BUILD_PLAN.md` top to bottom · every phase ends in a check that has to pass before the next one starts. ## Stack | Part | Choice | Why | | --- | --- | --- | | Language | Swift 6 and SwiftUI in Xcode | audio capture, hotkeys and text insertion are all AppKit | | Transcription | whisper.cpp with a locally downloaded model | runs offline, costs nothing per use, and the recording never leaves the machine | | Audio | AVAudioEngine capturing to 16 kHz mono 16 bit | whisper.cpp accepts that format specifically, and converting once at capture avoids a conversion step later | | Rewriting | An optional model call, local or hosted, per mode | a commit message and a message to a colleague want different shaping, and neither wants raw dictation | | Insertion | Pasteboard plus a synthetic Cmd-V, with the previous contents restored | per-character event posting drops characters and mangles anything outside ASCII | ## Before you start Have every one of these ready. The plan assumes them from step one. - [ ] **A Mac running macOS 14 or newer** · free if you already own one - Why: Audio capture, hotkeys and text insertion are all platform APIs. - Get it: Apple menu, then About This Mac. - Verify: About This Mac reports macOS 14 or higher - [ ] **Xcode 16 or newer** · free - Why: Builds and signs the application and links whisper.cpp. - Get it: Mac App Store or developer.apple.com/xcode. - Verify: xcodebuild -version prints 16 or higher - [ ] **whisper.cpp and a downloaded model** · free; models range from tens of megabytes to several gigabytes of disk - Why: It is the transcription engine. Without a model there is nothing to transcribe with. - Get it: Build whisper.cpp from its repository, then download a model. Start with a small English model to keep the ratio measurement quick, and try a larger one once the pipeline works. - Verify: whisper-cli transcribes a 16 kHz mono WAV correctly - [ ] **Microphone permission, and awareness of what it captures** · free - Why: Push-to-talk limits recording to when you hold the key, but the microphone still captures whatever is audible in that window. - Get it: System Settings, then Privacy and Security, then Microphone. The usage description string must be in Info.plist or the prompt never appears. - Verify: Recording produces audio with a level above silence - [ ] **Apple Developer Program membership** (optional) · 99 USD per year - Why: Product phase only. Without it there is no notarization and a Gatekeeper warning everywhere else. - Get it: Enrol at developer.apple.com/programs. - Verify: A Developer ID Application certificate appears in Xcode settings ## Quick start ```sh xcrun notarytool submit Dictate.zip --keychain-profile AC --wait ``` Then copy `.env.example` to `.env` and fill in the values it documents. ## Honest limits This build deliberately does not replace: - The native interface polish of the commercial application. - A curated library of context presets. - Per-speaker voice adaptation. - Mobile dictation. - beautiful native UX - presets - vocabulary/profile tuning - app-wide polish - support If one of those is essential to you, that is the reason to keep paying for Superwhisper, and the README should say so rather than pretend.
# Build brief · Superwhisper The one-shot brief this plan expands. `BUILD_PLAN.md` (or `MILESTONES.md`) is the same sequence broken into steps and checks; where the two disagree, the plan wins. Build me a local dictation tool with per-context modes that transcribes offline and optionally rewrites before pasting to replace Superwhisper. Build it in phases, in the order below. Do not write the whole thing in one pass. Finish a phase, run its "Done when" check, fix what fails, and only then start the next phase. ### Stack (fixed, do not substitute) - Language: Swift 6 and SwiftUI in Xcode · audio capture, hotkeys and text insertion are all AppKit - Transcription: whisper.cpp with a locally downloaded model · runs offline, costs nothing per use, and the recording never leaves the machine - Audio: AVAudioEngine capturing to 16 kHz mono 16 bit · whisper.cpp accepts that format specifically, and converting once at capture avoids a conversion step later - Rewriting: An optional model call, local or hosted, per mode · a commit message and a message to a colleague want different shaping, and neither wants raw dictation - Insertion: Pasteboard plus a synthetic Cmd-V, with the previous contents restored · per-character event posting drops characters and mangles anything outside ASCII ### Data model (create this before Phase 1) - modes(id, name, hotkey, model_size, rewrite_enabled, rewrite_prompt, target_apps_json) - transcripts(id, mode_id, at, audio_ms, raw_text, final_text, model, transcribe_ms, rewrite_ms) - vocabulary(id, term, sounds_like, mode_id) · names and jargon that Whisper gets wrong, with corrections applied after transcription - settings(key, value) · audio device, whether to keep recordings, retention period - Recordings are deleted after transcription by default. Keeping them is a setting the user turns on, not a default they discover. ### Phase 1 · Hotkey, capture and one transcription Build: Register a push-to-talk hotkey. Capture microphone audio directly at 16 kHz mono 16 bit. On release, run whisper.cpp over the buffer and display the result. Measure how long transcription takes relative to the audio length. Trap: whisper.cpp expects 16 kHz mono 16 bit PCM and behaves badly with anything else. Configure the audio engine to deliver that format directly rather than capturing at the device's native rate and resampling afterwards, which is an extra step that introduces its own artefacts. Done when: Holding the hotkey, speaking a sentence and releasing shows a correct transcription in a window. The captured audio is verified as 16 kHz mono 16 bit. The ratio of transcription time to audio length is measured and written down, because it decides whether larger models are usable. Do not build yet: No insertion, no modes, no rewriting. ### Phase 2 · Getting text into the field you were in Build: Save the pasteboard, write the transcription, send Cmd-V, then restore the previous contents after a short delay. Detect the case where insertion is refused and say so rather than leaving the user staring at an unchanged field. Trap: Do not type the text with one event per character. It drops characters under load, mangles anything outside ASCII, and behaves differently in every text engine. Trap: Restoring the pasteboard too quickly races the paste and restores before the target application has read it. Restore after a delay, and only if the pasteboard still holds what you wrote. Done when: Dictating into a text editor inserts the text and the pasteboard afterwards holds what it held before. Text containing an accented character and an emoji inserts intact. Dictating into a secure field is refused with a visible message rather than producing nothing. Do not build yet: No modes and no rewriting yet. ### Phase 3 · Modes Build: A mode carries its own hotkey, model size and rewrite behaviour. Optionally bind a mode to specific applications so the right one activates automatically. Show which mode is active during recording, because the wrong mode is otherwise invisible until too late. Trap: A larger model gives better transcription and a longer wait, and past a couple of seconds the pause breaks the flow that makes dictation worth using. Let each mode choose its own model size, so quick notes use a fast one and long-form uses a better one. Done when: Two modes with different hotkeys produce different handling of the same spoken input. A mode bound to a specific application activates automatically when that application is frontmost. The active mode is visible during recording. Switching modes does not require reopening settings. Do not build yet: No rewriting or vocabulary yet. ### Phase 4 · Vocabulary and rewriting Build: Apply a vocabulary of corrections after transcription, matching on what the model actually produces rather than on what you meant. Optionally send the text to a model with the mode's prompt to remove filler and apply formatting. Always keep the raw transcription so a bad rewrite is recoverable. Trap: A rewrite step can quietly change meaning as well as style, and once it has replaced the raw text you cannot tell. Store both, show the raw text when the user asks, and keep rewrite prompts narrow: remove filler and fix punctuation, not improve the message. Done when: A name Whisper consistently mishears is corrected by a vocabulary entry. The raw transcription is stored alongside the rewritten text and can be viewed. A rewrite that fails or times out inserts the raw transcription rather than nothing. Disabling rewrite for a mode makes insertion measurably faster. Do not build yet: No history or retention yet. ### Phase 5 · History, retention and privacy Build: A searchable history of transcriptions. Delete audio immediately after transcription by default, with retention as an explicit opt-in. Make deletion cover the audio, the transcript and the search index together. Trap: Dictation captures whatever was said near the microphone, including things the user did not intend to record. Default to deleting audio at once, make retention a deliberate choice, and never make the history the only place a transcription exists without saying so. Done when: Audio files are absent from disk after a transcription completes with default settings. Enabling retention keeps them and the setting states where they are. Deleting a history entry removes the transcript, the audio if kept and the search entry, verified by searching for the deleted text afterwards. Do not build yet: Distribution is the last phase. ### Phase 6 · Signing, notarization and updates Build: Hardened runtime, Developer ID signing, notarization, stapling and Sparkle updates. Ship or fetch the model deliberately, since models are large and licence terms vary. Trap: Whisper models are hundreds of megabytes to several gigabytes. Bundling one makes the download enormous and may not be permitted by its licence; downloading on first run needs a progress indicator, a checksum and a resumable transfer, because a failed model download leaves the application useless with no explanation. Done when: A notarized build opens on a fresh Mac with no Gatekeeper warning. First run downloads the model with visible progress and verifies a checksum. An interrupted download resumes rather than starting again. The README states what is recorded, what is deleted and what leaves the machine. Do not build yet: Nothing after this. ### Out of scope (and why) - The native interface polish of the commercial application, which is most of what people are paying for in this category. - The curated preset library covering dozens of contexts, which is accumulated tuning rather than code. - Per-speaker voice profile adaptation, which needs training infrastructure and data you do not have. - Mobile dictation, where the platform provides its own and the integration points are entirely different. ### README must contain - That transcription runs locally and audio does not leave the machine, unless a hosted rewrite model is configured - That audio is deleted immediately after transcription unless retention is enabled - The transcription time to audio ratio per model size, so mode choices are informed - That insertion uses the pasteboard and restores it, and that secure fields refuse input - Which model is used, its licence, and where it is downloaded from
# Agent instructions · Superwhisper indie build - Read `README.md` and `BUILD_PLAN.md` before writing code. The stack is fixed: Swift 6 and SwiftUI in Xcode, whisper.cpp with a locally downloaded model, AVAudioEngine capturing to 16 kHz mono 16 bit, An optional model call, local or hosted, per mode, Pasteboard plus a synthetic Cmd-V, with the previous contents restored. Do not substitute. - Work one phase at a time, in order. Do not start a phase until every "Done when" item of the previous one passes. - Prefer the fewest moving parts that satisfy the step. No frameworks, services or dependencies the plan does not name. - Secrets live in `.env`, never in source or logs. Keep `.env.example` current when a variable is introduced. - Do not invent cryptography, security guarantees, APIs or compliance claims. - Add a focused test for every destructive, security-sensitive or data-loss path the plan names. - Run the project checks before declaring a phase complete, and record any deliberate shortcut in the README under "Tradeoffs". ## Known traps - whisper.cpp expects 16 kHz mono 16 bit PCM and behaves badly with anything else. Configure the audio engine to deliver that format directly rather than capturing at the device's native rate and resampling afterwards, which is an extra step that introduces its own artefacts. - Do not type the text with one event per character. It drops characters under load, mangles anything outside ASCII, and behaves differently in every text engine. - Restoring the pasteboard too quickly races the paste and restores before the target application has read it. Restore after a delay, and only if the pasteboard still holds what you wrote. - A larger model gives better transcription and a longer wait, and past a couple of seconds the pause breaks the flow that makes dictation worth using. Let each mode choose its own model size, so quick notes use a fast one and long-form uses a better one. - A rewrite step can quietly change meaning as well as style, and once it has replaced the raw text you cannot tell. Store both, show the raw text when the user asks, and keep rewrite prompts narrow: remove filler and fix punctuation, not improve the message. - Dictation captures whatever was said near the microphone, including things the user did not intend to record. Default to deleting audio at once, make retention a deliberate choice, and never make the history the only place a transcription exists without saying so.
# Build plan · Superwhisper Hold a hotkey, speak, release, and the text appears in whatever field had focus, transcribed by a Whisper model running on your own machine, shaped by a mode that knows whether you are writing an email or a commit message, with a custom vocabulary so names and jargon come out right. Phases are in dependency order. Each ends in a "Done when" list; treat an unticked item as a blocker, not a note. ## Phase 1 · Hotkey, capture and one transcription Hold a key, speak, release, and see the text in a window. ### Steps 1. Register a push-to-talk hotkey RegisterEventHotKey for the press, and track the release. Push-to-talk avoids the whole problem of deciding when speech has ended. Files: `Hotkey.swift` 2. Capture in whisper's format directly AVAudioEngine with an output format of 16 kHz mono 16 bit. Microphone access is its own permission with its own usage string in Info.plist. Files: `Audio.swift`, `Info.plist` 3. Run whisper.cpp and time it Link the library or invoke the binary. Record transcription milliseconds against audio milliseconds for several utterances. Files: `Transcribe.swift` ### Done when - [ ] A spoken sentence transcribes correctly into a window - [ ] Captured audio is confirmed 16 kHz mono 16 bit - [ ] The transcription to audio time ratio is measured and recorded - [ ] Denying microphone access shows an explanation rather than failing silently ### Watch out - whisper.cpp expects 16 kHz mono 16 bit PCM and behaves badly with anything else. Configure the audio engine to deliver that format directly rather than capturing at the device's native rate and resampling afterwards, which is an extra step that introduces its own artefacts. ## Phase 2 · Getting text into the field you were in The transcription lands in whatever had focus, without losing your clipboard. ### Steps 1. Save, write, paste, restore Capture the existing pasteboard items first. Restore after a delay, and only if your value is still there. Files: `Insert.swift` 2. Handle refusal visibly Secure text fields refuse synthetic input by design. Detect and show a brief message. Files: `Insert.swift` 3. Test across text engines A native editor, a browser field and a terminal. They behave differently and one of them will surprise you. ### Done when - [ ] Text inserts into the previously focused field - [ ] The pasteboard afterwards holds its previous contents - [ ] Accented characters and emoji insert intact - [ ] A secure field produces a visible refusal ### Watch out - Do not type the text with one event per character. It drops characters under load, mangles anything outside ASCII, and behaves differently in every text engine. - Restoring the pasteboard too quickly races the paste and restores before the target application has read it. Restore after a delay, and only if the pasteboard still holds what you wrote. ## Phase 3 · Modes Dictating a commit message and dictating an email produce appropriately different text. ### Steps 1. Make modes first-class Hotkey, model, rewrite settings and application bindings per mode. Not a global setting with overrides. Files: `Modes.swift` 2. Bind modes to applications Match the frontmost bundle identifier when recording starts, not when the hotkey is registered. Files: `Modes.swift` 3. Show the active mode while recording A small indicator naming the mode. Discovering the wrong mode after speaking for a minute is the failure this prevents. Files: `Indicator.swift` ### Done when - [ ] Two modes handle the same input differently - [ ] An application-bound mode activates automatically - [ ] The active mode is visible during recording - [ ] Each mode uses its own configured model size ### Watch out - A larger model gives better transcription and a longer wait, and past a couple of seconds the pause breaks the flow that makes dictation worth using. Let each mode choose its own model size, so quick notes use a fast one and long-form uses a better one. ## Phase 4 · Vocabulary and rewriting Names and jargon come out right, and the text reads like writing rather than speech. ### Steps 1. Build the vocabulary on observed errors Record what the model produced, then map it. Guessing at misrecognitions is far less effective than reading your own transcripts. Files: `Vocabulary.swift` 2. Keep rewrite prompts narrow Filler removal and punctuation. A prompt that invites improvement will rewrite your meaning. Files: `Rewrite.swift` 3. Fall back to raw on failure A timeout or an error inserts the transcription. Losing dictated text because a rewrite failed is the worst outcome here. Files: `Rewrite.swift` ### Done when - [ ] A vocabulary entry corrects a consistent misrecognition - [ ] Raw and rewritten text are both stored and the raw is viewable - [ ] A failed rewrite inserts the raw transcription - [ ] Disabling rewrite makes insertion measurably faster ### Watch out - A rewrite step can quietly change meaning as well as style, and once it has replaced the raw text you cannot tell. Store both, show the raw text when the user asks, and keep rewrite prompts narrow: remove filler and fix punctuation, not improve the message. ## Phase 5 · History, retention and privacy You can find what you dictated last week, and nothing is kept that you did not choose to keep. ### Steps 1. Delete audio by default Immediately after transcription. Retention is opt-in and says where files are kept. Files: `Retention.swift` 2. Make history searchable Full text search over transcripts. This is the feature that turns dictation into a record you can use. Files: `History.swift` 3. Make deletion complete One function covering transcript, audio and index. Test by searching for the deleted text. Files: `Delete.swift` ### Done when - [ ] Audio is absent from disk after transcription with default settings - [ ] Retention is opt-in and states where audio is kept - [ ] Deleting an entry removes it from search - [ ] Deleting an entry removes its audio when retention was on ### Watch out - Dictation captures whatever was said near the microphone, including things the user did not intend to record. Default to deleting audio at once, make retention a deliberate choice, and never make the history the only place a transcription exists without saying so. ## Not in this build - The native interface polish of the commercial application. - A curated library of context presets. - Per-speaker voice adaptation. - Mobile dictation. ## After v1, if you want it - Streaming transcription so text appears while you are still speaking - A mode that dictates directly into a terminal with shell-aware punctuation - Automatic vocabulary suggestions from corrections you make by hand
# Copy to .env and fill in. Never commit .env; this file documents it. # Required. Where the downloaded model lives. Modes may point at different sizes. WHISPER_MODEL_PATH=~/Library/Application Support/Dictate/models/ggml-base.en.bin # Required. Fixed by whisper.cpp. Capture at this rate directly rather than resampling. AUDIO_SAMPLE_RATE=16000 # Required. Whether recordings are retained after transcription. False by default and deliberately so. KEEP_AUDIO=false # Optional. Only used when retention is enabled. Older recordings are deleted. AUDIO_RETENTION_DAYS=7 # Optional. Optional rewrite model. A local endpoint keeps everything on the machine; a hosted one does not, and the README must say which you chose. REWRITE_ENDPOINT=http://localhost:11434/v1/chat/completions # Required. How long to wait before restoring the pasteboard. Too short races the paste. PASTE_RESTORE_DELAY_MS=250
# Superwhisper · product brief ## Problem A push-to-talk recorder that transcribes and pastes text into the active app is very buildable for one person, especially on macOS. ## Product outcome A dictation tool that transcribes locally, inserts reliably into any text field without losing the clipboard, adapts to context through modes, corrects the words it always gets wrong, and deletes audio by default. ## Target user A builder who needs a maintainable product foundation, not a one-off demo. ## Required capabilities - macOS automation/accessibility permissions - speech API or local Whisper - optional LLM API - hotkey library ## Explicit non-goals for v1 - The native interface polish of the commercial application. - A curated library of context presets. - Per-speaker voice adaptation. - Mobile dictation. - beautiful native UX - presets - vocabulary/profile tuning - app-wide polish - support ## Success criteria - Insertion is verified in a native editor, a browser field and a terminal - The pasteboard is confirmed restored after insertion - Audio is confirmed absent from disk with default settings - The model download resumes after an interruption and verifies its checksum - The README states exactly what leaves the machine
# Build brief · Superwhisper The one-shot brief this plan expands. `BUILD_PLAN.md` (or `MILESTONES.md`) is the same sequence broken into steps and checks; where the two disagree, the plan wins. Build me a local dictation tool with per-context modes that transcribes offline and optionally rewrites before pasting to replace Superwhisper. Build it in phases, in the order below. Do not write the whole thing in one pass. Finish a phase, run its "Done when" check, fix what fails, and only then start the next phase. ### Stack (fixed, do not substitute) - Language: Swift 6 and SwiftUI in Xcode · audio capture, hotkeys and text insertion are all AppKit - Transcription: whisper.cpp with a locally downloaded model · runs offline, costs nothing per use, and the recording never leaves the machine - Audio: AVAudioEngine capturing to 16 kHz mono 16 bit · whisper.cpp accepts that format specifically, and converting once at capture avoids a conversion step later - Rewriting: An optional model call, local or hosted, per mode · a commit message and a message to a colleague want different shaping, and neither wants raw dictation - Insertion: Pasteboard plus a synthetic Cmd-V, with the previous contents restored · per-character event posting drops characters and mangles anything outside ASCII ### Data model (create this before Phase 1) - modes(id, name, hotkey, model_size, rewrite_enabled, rewrite_prompt, target_apps_json) - transcripts(id, mode_id, at, audio_ms, raw_text, final_text, model, transcribe_ms, rewrite_ms) - vocabulary(id, term, sounds_like, mode_id) · names and jargon that Whisper gets wrong, with corrections applied after transcription - settings(key, value) · audio device, whether to keep recordings, retention period - Recordings are deleted after transcription by default. Keeping them is a setting the user turns on, not a default they discover. ### Phase 1 · Hotkey, capture and one transcription Build: Register a push-to-talk hotkey. Capture microphone audio directly at 16 kHz mono 16 bit. On release, run whisper.cpp over the buffer and display the result. Measure how long transcription takes relative to the audio length. Trap: whisper.cpp expects 16 kHz mono 16 bit PCM and behaves badly with anything else. Configure the audio engine to deliver that format directly rather than capturing at the device's native rate and resampling afterwards, which is an extra step that introduces its own artefacts. Done when: Holding the hotkey, speaking a sentence and releasing shows a correct transcription in a window. The captured audio is verified as 16 kHz mono 16 bit. The ratio of transcription time to audio length is measured and written down, because it decides whether larger models are usable. Do not build yet: No insertion, no modes, no rewriting. ### Phase 2 · Getting text into the field you were in Build: Save the pasteboard, write the transcription, send Cmd-V, then restore the previous contents after a short delay. Detect the case where insertion is refused and say so rather than leaving the user staring at an unchanged field. Trap: Do not type the text with one event per character. It drops characters under load, mangles anything outside ASCII, and behaves differently in every text engine. Trap: Restoring the pasteboard too quickly races the paste and restores before the target application has read it. Restore after a delay, and only if the pasteboard still holds what you wrote. Done when: Dictating into a text editor inserts the text and the pasteboard afterwards holds what it held before. Text containing an accented character and an emoji inserts intact. Dictating into a secure field is refused with a visible message rather than producing nothing. Do not build yet: No modes and no rewriting yet. ### Phase 3 · Modes Build: A mode carries its own hotkey, model size and rewrite behaviour. Optionally bind a mode to specific applications so the right one activates automatically. Show which mode is active during recording, because the wrong mode is otherwise invisible until too late. Trap: A larger model gives better transcription and a longer wait, and past a couple of seconds the pause breaks the flow that makes dictation worth using. Let each mode choose its own model size, so quick notes use a fast one and long-form uses a better one. Done when: Two modes with different hotkeys produce different handling of the same spoken input. A mode bound to a specific application activates automatically when that application is frontmost. The active mode is visible during recording. Switching modes does not require reopening settings. Do not build yet: No rewriting or vocabulary yet. ### Phase 4 · Vocabulary and rewriting Build: Apply a vocabulary of corrections after transcription, matching on what the model actually produces rather than on what you meant. Optionally send the text to a model with the mode's prompt to remove filler and apply formatting. Always keep the raw transcription so a bad rewrite is recoverable. Trap: A rewrite step can quietly change meaning as well as style, and once it has replaced the raw text you cannot tell. Store both, show the raw text when the user asks, and keep rewrite prompts narrow: remove filler and fix punctuation, not improve the message. Done when: A name Whisper consistently mishears is corrected by a vocabulary entry. The raw transcription is stored alongside the rewritten text and can be viewed. A rewrite that fails or times out inserts the raw transcription rather than nothing. Disabling rewrite for a mode makes insertion measurably faster. Do not build yet: No history or retention yet. ### Phase 5 · History, retention and privacy Build: A searchable history of transcriptions. Delete audio immediately after transcription by default, with retention as an explicit opt-in. Make deletion cover the audio, the transcript and the search index together. Trap: Dictation captures whatever was said near the microphone, including things the user did not intend to record. Default to deleting audio at once, make retention a deliberate choice, and never make the history the only place a transcription exists without saying so. Done when: Audio files are absent from disk after a transcription completes with default settings. Enabling retention keeps them and the setting states where they are. Deleting a history entry removes the transcript, the audio if kept and the search entry, verified by searching for the deleted text afterwards. Do not build yet: Distribution is the last phase. ### Phase 6 · Signing, notarization and updates Build: Hardened runtime, Developer ID signing, notarization, stapling and Sparkle updates. Ship or fetch the model deliberately, since models are large and licence terms vary. Trap: Whisper models are hundreds of megabytes to several gigabytes. Bundling one makes the download enormous and may not be permitted by its licence; downloading on first run needs a progress indicator, a checksum and a resumable transfer, because a failed model download leaves the application useless with no explanation. Done when: A notarized build opens on a fresh Mac with no Gatekeeper warning. First run downloads the model with visible progress and verifies a checksum. An interrupted download resumes rather than starting again. The README states what is recorded, what is deleted and what leaves the machine. Do not build yet: Nothing after this. ### Out of scope (and why) - The native interface polish of the commercial application, which is most of what people are paying for in this category. - The curated preset library covering dozens of contexts, which is accumulated tuning rather than code. - Per-speaker voice profile adaptation, which needs training infrastructure and data you do not have. - Mobile dictation, where the platform provides its own and the integration points are entirely different. ### README must contain - That transcription runs locally and audio does not leave the machine, unless a hosted rewrite model is configured - That audio is deleted immediately after transcription unless retention is enabled - The transcription time to audio ratio per model size, so mode choices are informed - That insertion uses the pasteboard and restores it, and that secure fields refuse input - Which model is used, its licence, and where it is downloaded from
# Architecture · Superwhisper ## Stack | Part | Choice | Why | | --- | --- | --- | | Language | Swift 6 and SwiftUI in Xcode | audio capture, hotkeys and text insertion are all AppKit | | Transcription | whisper.cpp with a locally downloaded model | runs offline, costs nothing per use, and the recording never leaves the machine | | Audio | AVAudioEngine capturing to 16 kHz mono 16 bit | whisper.cpp accepts that format specifically, and converting once at capture avoids a conversion step later | | Rewriting | An optional model call, local or hosted, per mode | a commit message and a message to a colleague want different shaping, and neither wants raw dictation | | Insertion | Pasteboard plus a synthetic Cmd-V, with the previous contents restored | per-character event posting drops characters and mangles anything outside ASCII | ## Modules Each module has one owner concern and a documented way to replace it. | Module | Owns | How to replace it | | --- | --- | --- | | Capture | Hotkey, audio in whisper's format, and the recording lifecycle. | Capturing directly in the required format is the decision worth keeping whatever else changes. | | Transcribe | Running the model and timing it. | A hosted transcription service fits the same interface and gives up the offline and privacy properties, which is a product decision rather than a technical one. | | Insert | Pasteboard handling and the synthetic paste. | Not swappable in practice. Per-character insertion is the alternative and it is worse in every way. | | Modes | Per-context model, prompt and bindings. | Configuration rather than code, so new modes cost nothing. | | Rewrite | Optional shaping, with raw text preserved. | Any model endpoint. The rule that raw text survives a failed rewrite must not be lost. | ## Configuration Every runtime setting is an environment variable documented in `.env.example`, validated at startup, with a safe local default wherever one exists. - `WHISPER_MODEL_PATH` · required · Where the downloaded model lives. Modes may point at different sizes. - `AUDIO_SAMPLE_RATE` · required · Fixed by whisper.cpp. Capture at this rate directly rather than resampling. - `KEEP_AUDIO` · required · Whether recordings are retained after transcription. False by default and deliberately so. - `AUDIO_RETENTION_DAYS` · optional · Only used when retention is enabled. Older recordings are deleted. - `REWRITE_ENDPOINT` · optional · Optional rewrite model. A local endpoint keeps everything on the machine; a hosted one does not, and the README must say which you chose. - `PASTE_RESTORE_DELAY_MS` · required · How long to wait before restoring the pasteboard. Too short races the paste. ## Production baseline - Security: least privilege, input validation at every boundary, secret redaction in logs, rate limits on abuse-prone paths, no invented security primitives. - Data: explicit schema and migrations, transactional writes where integrity matters, backup and restore procedures that have been exercised. - Integrations: adapters around third-party providers, idempotent webhook or job processing, bounded retries, timeouts. - Observability: structured logs with request or operation ids, an error-tracking hook, and health and readiness checks where a server exists. - Quality: unit tests for domain rules, integration tests at module boundaries, one end-to-end test of the critical path. ## Decision records For each dependency in the stack table, keep a short note: why it was chosen, its failure mode, and how it is replaced. Do not add infrastructure until a requirement in `PRODUCT.md` justifies it.
# Agent instructions · Superwhisper product build - Read `PRODUCT.md` and `ARCHITECTURE.md` before changing code. The stack is fixed: Swift 6 and SwiftUI in Xcode, whisper.cpp with a locally downloaded model, AVAudioEngine capturing to 16 kHz mono 16 bit, An optional model call, local or hosted, per mode, Pasteboard plus a synthetic Cmd-V, with the previous contents restored. - Implement milestone by milestone from `MILESTONES.md`; keep each change reviewable and leave the application runnable at every commit. - Treat authentication, payments, encryption, imports, webhooks and destructive actions as high-risk boundaries when present. - Never invent cryptography or silently weaken a requirement to make a check pass. - Put every external service behind an interface with a deterministic fake for tests. - Add migrations and rollback or recovery notes for every persistent data change. - Log useful operational context without credentials, tokens, passwords or personal data. - Update documentation and run every check before completing a milestone. ## Known traps - whisper.cpp expects 16 kHz mono 16 bit PCM and behaves badly with anything else. Configure the audio engine to deliver that format directly rather than capturing at the device's native rate and resampling afterwards, which is an extra step that introduces its own artefacts. - Do not type the text with one event per character. It drops characters under load, mangles anything outside ASCII, and behaves differently in every text engine. - Restoring the pasteboard too quickly races the paste and restores before the target application has read it. Restore after a delay, and only if the pasteboard still holds what you wrote. - A larger model gives better transcription and a longer wait, and past a couple of seconds the pause breaks the flow that makes dictation worth using. Let each mode choose its own model size, so quick notes use a fast one and long-form uses a better one. - A rewrite step can quietly change meaning as well as style, and once it has replaced the raw text you cannot tell. Store both, show the raw text when the user asks, and keep rewrite prompts narrow: remove filler and fix punctuation, not improve the message. - Dictation captures whatever was said near the microphone, including things the user did not intend to record. Default to deleting audio at once, make retention a deliberate choice, and never make the history the only place a transcription exists without saying so. - Whisper models are hundreds of megabytes to several gigabytes. Bundling one makes the download enormous and may not be permitted by its licence; downloading on first run needs a progress indicator, a checksum and a resumable transfer, because a failed model download leaves the application useless with no explanation.
# Delivery milestones · Superwhisper Estimated effort: **weekend** for the indie phases; the production-only milestones add the trust and operability layer. ## M1 · Hotkey, capture and one transcription Hold a key, speak, release, and see the text in a window. ### Steps 1. Register a push-to-talk hotkey RegisterEventHotKey for the press, and track the release. Push-to-talk avoids the whole problem of deciding when speech has ended. Files: `Hotkey.swift` 2. Capture in whisper's format directly AVAudioEngine with an output format of 16 kHz mono 16 bit. Microphone access is its own permission with its own usage string in Info.plist. Files: `Audio.swift`, `Info.plist` 3. Run whisper.cpp and time it Link the library or invoke the binary. Record transcription milliseconds against audio milliseconds for several utterances. Files: `Transcribe.swift` ### Done when - [ ] A spoken sentence transcribes correctly into a window - [ ] Captured audio is confirmed 16 kHz mono 16 bit - [ ] The transcription to audio time ratio is measured and recorded - [ ] Denying microphone access shows an explanation rather than failing silently ### Watch out - whisper.cpp expects 16 kHz mono 16 bit PCM and behaves badly with anything else. Configure the audio engine to deliver that format directly rather than capturing at the device's native rate and resampling afterwards, which is an extra step that introduces its own artefacts. ## M2 · Getting text into the field you were in The transcription lands in whatever had focus, without losing your clipboard. ### Steps 1. Save, write, paste, restore Capture the existing pasteboard items first. Restore after a delay, and only if your value is still there. Files: `Insert.swift` 2. Handle refusal visibly Secure text fields refuse synthetic input by design. Detect and show a brief message. Files: `Insert.swift` 3. Test across text engines A native editor, a browser field and a terminal. They behave differently and one of them will surprise you. ### Done when - [ ] Text inserts into the previously focused field - [ ] The pasteboard afterwards holds its previous contents - [ ] Accented characters and emoji insert intact - [ ] A secure field produces a visible refusal ### Watch out - Do not type the text with one event per character. It drops characters under load, mangles anything outside ASCII, and behaves differently in every text engine. - Restoring the pasteboard too quickly races the paste and restores before the target application has read it. Restore after a delay, and only if the pasteboard still holds what you wrote. ## M3 · Modes Dictating a commit message and dictating an email produce appropriately different text. ### Steps 1. Make modes first-class Hotkey, model, rewrite settings and application bindings per mode. Not a global setting with overrides. Files: `Modes.swift` 2. Bind modes to applications Match the frontmost bundle identifier when recording starts, not when the hotkey is registered. Files: `Modes.swift` 3. Show the active mode while recording A small indicator naming the mode. Discovering the wrong mode after speaking for a minute is the failure this prevents. Files: `Indicator.swift` ### Done when - [ ] Two modes handle the same input differently - [ ] An application-bound mode activates automatically - [ ] The active mode is visible during recording - [ ] Each mode uses its own configured model size ### Watch out - A larger model gives better transcription and a longer wait, and past a couple of seconds the pause breaks the flow that makes dictation worth using. Let each mode choose its own model size, so quick notes use a fast one and long-form uses a better one. ## M4 · Vocabulary and rewriting Names and jargon come out right, and the text reads like writing rather than speech. ### Steps 1. Build the vocabulary on observed errors Record what the model produced, then map it. Guessing at misrecognitions is far less effective than reading your own transcripts. Files: `Vocabulary.swift` 2. Keep rewrite prompts narrow Filler removal and punctuation. A prompt that invites improvement will rewrite your meaning. Files: `Rewrite.swift` 3. Fall back to raw on failure A timeout or an error inserts the transcription. Losing dictated text because a rewrite failed is the worst outcome here. Files: `Rewrite.swift` ### Done when - [ ] A vocabulary entry corrects a consistent misrecognition - [ ] Raw and rewritten text are both stored and the raw is viewable - [ ] A failed rewrite inserts the raw transcription - [ ] Disabling rewrite makes insertion measurably faster ### Watch out - A rewrite step can quietly change meaning as well as style, and once it has replaced the raw text you cannot tell. Store both, show the raw text when the user asks, and keep rewrite prompts narrow: remove filler and fix punctuation, not improve the message. ## M5 · History, retention and privacy You can find what you dictated last week, and nothing is kept that you did not choose to keep. ### Steps 1. Delete audio by default Immediately after transcription. Retention is opt-in and says where files are kept. Files: `Retention.swift` 2. Make history searchable Full text search over transcripts. This is the feature that turns dictation into a record you can use. Files: `History.swift` 3. Make deletion complete One function covering transcript, audio and index. Test by searching for the deleted text. Files: `Delete.swift` ### Done when - [ ] Audio is absent from disk after transcription with default settings - [ ] Retention is opt-in and states where audio is kept - [ ] Deleting an entry removes it from search - [ ] Deleting an entry removes its audio when retention was on ### Watch out - Dictation captures whatever was said near the microphone, including things the user did not intend to record. Default to deleting audio at once, make retention a deliberate choice, and never make the history the only place a transcription exists without saying so. ## M6 · Signing, notarization and updates (production only) Others can install it, and the permission story is stated plainly. ### Steps 1. Sign and notarize Hardened runtime, Developer ID, notarytool, stapler. Confirm the microphone usage string is present. ```sh xcrun notarytool submit Dictate.zip --keychain-profile AC --wait ``` 2. Fetch models with verification Progress, checksum and resume. Record each model's licence. Files: `ModelDownload.swift`, `docs/models.md` 3. State the privacy position What is recorded, what is deleted, and whether anything leaves the machine, which depends on whether rewriting uses a hosted model. Files: `README.md` ### Done when - [ ] spctl reports the notarized build accepted - [ ] First run downloads the model with progress and a checksum - [ ] An interrupted download resumes - [ ] The README states what leaves the machine and when ### Watch out - Whisper models are hundreds of megabytes to several gigabytes. Bundling one makes the download enormous and may not be permitted by its licence; downloading on first run needs a progress indicator, a checksum and a resumable transfer, because a failed model download leaves the application useless with no explanation.
# Operations · Superwhisper ## Backup Modes, vocabulary and history are the state worth keeping, and they are small. Audio, when retained, is large and rarely worth backing up. ## Restore Restore the configuration and history, then re-download the model, which is large and re-fetchable. Keeping models out of the backup is deliberate. Do a restore drill before the first real user, and write the date here when it passes. ## Monitoring Track transcription time against audio length per model. A rising ratio usually means thermal throttling or a larger model chosen in a mode where it does not belong. ## Incident checklist A macOS update changing audio or Accessibility behaviour is the recurring risk. The other realistic failure is a model file that is corrupt after an interrupted download, which is why the checksum exists. 1. Contain the issue without destroying evidence or user data. 2. Record the timeline and affected scope. 3. Rotate exposed secrets and revoke compromised sessions or credentials. 4. Restore from a verified backup when needed. 5. Document the root cause, the remediation and the regression test. ## Release gate - [ ] Insertion is verified in a native editor, a browser field and a terminal - [ ] The pasteboard is confirmed restored after insertion - [ ] Audio is confirmed absent from disk with default settings - [ ] The model download resumes after an interruption and verifies its checksum - [ ] The README states exactly what leaves the machine ## Launch constraint Do not market omitted Superwhisper capabilities as implemented. The non-goals in `PRODUCT.md` remain user-visible limitations until they are deliberately delivered.
# Copy to .env and fill in. Never commit .env; this file documents it. # Required. Where the downloaded model lives. Modes may point at different sizes. WHISPER_MODEL_PATH=~/Library/Application Support/Dictate/models/ggml-base.en.bin # Required. Fixed by whisper.cpp. Capture at this rate directly rather than resampling. AUDIO_SAMPLE_RATE=16000 # Required. Whether recordings are retained after transcription. False by default and deliberately so. KEEP_AUDIO=false # Optional. Only used when retention is enabled. Older recordings are deleted. AUDIO_RETENTION_DAYS=7 # Optional. Optional rewrite model. A local endpoint keeps everything on the machine; a hosted one does not, and the README must say which you chose. REWRITE_ENDPOINT=http://localhost:11434/v1/chat/completions # Required. How long to wait before restoring the pasteboard. Too short races the paste. PASTE_RESTORE_DELAY_MS=250
$ choose a build depth, inspect the files, then open the complete pack in your agent
They pay for the low-friction menu-bar experience and dictation profiles.
xbeautiful native UX
xpresets
xvocabulary/profile tuning
xapp-wide polish
xsupport
Don't feel like building it? These folks already made it free.
all 8 free alternatives to Superwhisper →· no votes, no pay-to-list · just what's real
Superwhisper pricing
| plan | monthly | annual (per mo) | what you get |
|---|---|---|---|
| free | $0 | $0 | Basic dictation; no local models or custom modes; current numeric cloud-usage cap is not published. |
| pro monthly | $8.49 | — | Unlimited cloud and local models; license covers personal Mac, Windows, iPhone and iPad devices. |
| pro annual | — | $7.08 | Unlimited cloud and local models; same personal-device license. |
| pro lifetime | custom | — | One-time $249.99 purchase; unlimited cloud and local models under the lifetime license terms. |
free tierBasic dictation; no local models or custom modes; the current numeric cloud-use allowance is not published.
billingmonthly, annual and one-time lifetime purchase; direct-web prices
hidden costsApp Store pricing is higher because of Apple fees, but the current exact App Store amounts were not published in the official Pro documentation.
verified 2026-08-11 · source ↗
Is Superwhisper free?
A free version covers basic dictation; Pro adds unlimited cloud and local models. Paid is Pro at $8.49/mo (checked 2026-08-07).
Vibecode Superwhisper
Yes. A competent AI coding agent (Claude Code, Codex, Cursor) can build a usable personal Superwhisper replacement in one session with the prompt on this page. It runs on your own machine or server with no subscription.
How much does Superwhisper cost?
Superwhisper costs about $8.49/month (Pro, checked 2026-08-07), which is $101.88 per year. That's what you save by replacing it with one prompt.
What do I lose by replacing Superwhisper?
Honestly: beautiful native UX; presets; vocabulary/profile tuning; app-wide polish; support. If any of those are load-bearing for you, keep paying.
Is there an open-source alternative to Superwhisper?
Yes: Voicebox (Voice cloning studio attached to a global dictation hotkey; overkill, free overkill.) Handy (Press a key, talk and get plain text in the app you were already using. No cleanup magic, no invoice.) FluidVoice (Fast Mac dictation with local cleanup; even the fancy part stays on the machine.) All 8 curated free alternatives are at vibecodeit.com/superwhisper/alternatives. The prompt is for when you want it exactly your way.