Vibecode Krisp
track this build5 phases, 15 steps, beginner friendly0%KINDA · weekend project
You can build post-processing and maybe route audio through open models, but real-time low-latency virtual-device noise cancellation is not a one-sitting web app.
the project pack5 files · written for this build, step by step
You are building a lean indie version of Krisp. Create the following project files first, then implement the application by following them. Keep the files updated as decisions change. Do not collapse this into a single README or prompt. ===== README.md ===== # Krisp · indie build A virtual microphone that appears in every application's input list, pulls from your real microphone, runs a noise suppression model in the audio callback without ever missing a deadline, and offers a lower latency model for live calls and a heavier one for recordings. Estimated effort: **multi-day**. Work `BUILD_PLAN.md` top to bottom · every phase ends in a check that has to pass before the next one starts. ## Stack | Part | Choice | Why | | --- | --- | --- | | Language | Swift for the application, C or Rust for the audio path | the processing callback is real-time code and cannot afford allocation or garbage collection | | Virtual device | A Core Audio server plug-in | the supported way to publish an input device other applications can select | | Low latency model | RNNoise, BSD licensed, roughly 10 milliseconds and one core | good on steady noise such as fans and hum, and cheap enough to never miss a deadline | | High quality model | DeepFilterNet, roughly 40 milliseconds and more compute | clearly better on overlapping speech, clatter and reverberation, at a latency cost that matters live | | Buffering | A lock-free ring buffer between capture and processing | a mutex in an audio callback is a dropout waiting to happen | ## Before you start Have every one of these ready. The plan assumes them from step one. - [ ] **A Mac running macOS 13 or newer** · free if you own one - Why: The virtual device is a Core Audio server plug-in, which is a macOS concept. The equivalent on other platforms is a different project. - Get it: Apple menu, then About This Mac. - Verify: About This Mac reports macOS 13 or higher - [ ] **Xcode 16 or newer** · free - Why: Builds the application and the plug-in, and provides Instruments for the timing work in Phase 2. - Get it: Mac App Store or developer.apple.com/xcode. - Verify: xcodebuild -version prints 16 or higher - [ ] **Comfort with real-time audio constraints** · free - Why: An audio callback that allocates or blocks produces clicks the user hears and you cannot reproduce. This is the one part of the build where the usual habits are actively wrong. - Get it: Read about real-time safety before Phase 2. The rule is short: no allocation, no locks, no system calls, no unbounded work. - Verify: You can state why a mutex in the audio callback is a bug rather than a risk - [ ] **RNNoise and a DeepFilterNet build** · free, but check each licence for redistribution - Why: These are the two models the profiles use. Their latency and compute difference is the product decision in Phase 3. - Get it: Build RNNoise from its repository, which is small and BSD licensed. DeepFilterNet ships a real-time capable build; check its current licence before bundling. - Verify: Both process a test file and their added latency is measured - [ ] **Recordings with four kinds of noise** · free - Why: Steady fan noise is where every model succeeds. The differences appear on background speech, sudden clatter and reverberation. - Get it: Record them yourself: a fan, a café, a keyboard and a room with hard walls. Keep them as fixtures. - Verify: All four are saved with their noise floor measured ## Quick start ```sh sudo killall coreaudiod ``` Then copy `.env.example` to `.env` and fill in the values it documents. ## Honest limits This build deliberately does not replace: - The commercial model's quality on hard noise. - Echo cancellation. - Mobile applications and per-application switching polish. - Speaker-side suppression of what you hear. - low-latency virtual microphone - polished app switching - model quality - meeting integrations - admin controls If one of those is essential to you, that is the reason to keep paying for Krisp, and the README should say so rather than pretend. ===== BRIEF.md ===== # Build brief · Krisp The one-shot brief this plan expands. `BUILD_PLAN.md` (or `MILESTONES.md`) is the same sequence broken into steps and checks; where the two disagree, the plan wins. Build me a noise suppressing virtual microphone that other applications can select as an input device to replace Krisp. Build it in phases, in the order below. Do not write the whole thing in one pass. Finish a phase, run its "Done when" check, fix what fails, and only then start the next phase. ### Rule zero Do not train a noise suppression model. Two good open ones exist and the engineering that matters here is everything around them: a virtual audio device the operating system trusts, a processing loop that never misses its deadline, and an honest latency budget. Choose the model from the budget rather than from a quality comparison. RNNoise adds roughly 10 milliseconds and runs on one CPU core; DeepFilterNet sounds clearly better and adds roughly 40 milliseconds and wants real compute. On a live call that difference is audible as conversational lag, so the right answer depends on whether you are cleaning a call or a recording. Checked September 2026. ### Stack (fixed, do not substitute) - Language: Swift for the application, C or Rust for the audio path · the processing callback is real-time code and cannot afford allocation or garbage collection - Virtual device: A Core Audio server plug-in · the supported way to publish an input device other applications can select - Low latency model: RNNoise, BSD licensed, roughly 10 milliseconds and one core · good on steady noise such as fans and hum, and cheap enough to never miss a deadline - High quality model: DeepFilterNet, roughly 40 milliseconds and more compute · clearly better on overlapping speech, clatter and reverberation, at a latency cost that matters live - Buffering: A lock-free ring buffer between capture and processing · a mutex in an audio callback is a dropout waiting to happen ### Data model (create this before Phase 1) - This is a real-time audio path, so the state that matters is timing rather than tables. - device · the published virtual input, its sample rate, its buffer size and its channel count - profiles(id, name, model, expected_latency_ms, cpu_budget_pct) · one for calls, one for recordings - stats · rolling callback duration, deadline misses, dropouts, all visible in the interface - The deadline is the whole design: at a 48 kHz sample rate with a 256 frame buffer you have roughly 5.3 milliseconds to return, every time, without exception. ### Phase 1 · A virtual device that appears in other applications Build: Build a Core Audio server plug-in publishing one input device. Pass audio straight through from the real microphone with no processing at all. Getting the device recognised and stable is the whole phase, because nothing else can be tested until another application can select it. Trap: A server plug-in runs inside a system audio daemon rather than in your application, so a crash in it takes down audio for the whole machine and a debugger cannot attach the way you expect. Keep the plug-in minimal and put every piece of logic you can into your own process, communicating over a shared buffer. Done when: The virtual device appears in the input list of a video conferencing application and a recorder. Selecting it and speaking produces audio in that application. Running for an hour produces no dropouts. Quitting your application leaves the system's audio working rather than broken. Do not build yet: No noise suppression at all. A pass-through device that does not break anything. ### Phase 2 · Processing without missing a deadline Build: Insert RNNoise into the path. Measure callback duration on every call and count deadline misses. Do no allocation, no locking and no file access inside the callback. Show the measured headroom in the interface. Trap: Allocating memory, taking a lock or logging inside the audio callback will eventually block for longer than the deadline, and the result is a click that the user hears and you cannot reproduce. Preallocate everything, use a lock-free ring buffer, and move logging out to another thread. Done when: Noise suppression is audibly working on a recording with fan noise. Callback duration is measured and its worst case over ten minutes stays well inside the deadline. A deliberate allocation inserted into the callback shows up as measured deadline misses, proving the measurement is real. No dropouts during a ten minute call. Do not build yet: Only one model and no profiles yet. ### Phase 3 · Two models, two purposes Build: Add DeepFilterNet as a second profile. Show each profile's measured added latency and processor use, not the numbers from a paper. Let the user switch without restarting the device, and default to the low latency model for live use. Trap: Roughly 40 milliseconds of added latency is not noticeable when you listen back to a recording and is very noticeable in a conversation, where it sits on top of network delay and makes people talk over each other. Label the profiles by purpose rather than by quality, so the choice is obvious. Done when: Switching profiles takes effect without the device disappearing from other applications. Each profile reports its own measured added latency on this machine. The high quality profile audibly outperforms the low latency one on a recording with background speech. The default for live use is the low latency profile. Do not build yet: No per-application behaviour or recording mode yet. ### Phase 4 · Recording mode, where latency does not matter Build: An offline path that processes a file rather than a stream, using the heavier model with as much lookahead as it wants. Keep the original untouched and write a new file. Report a measurable before and after noise figure. Trap: Reusing the real-time path for files throws away the one advantage offline processing has, which is that it can look ahead. Run the file path without the deadline and without the streaming constraints, and it will beat the live path on the same recording. Done when: A file processed offline is measurably quieter in silent passages than the same file processed through the live path. The original file is unmodified, verified by checksum. A batch of ten files processes without intervention and reports each one's before and after figure. Do not build yet: No per-application rules yet. ### Phase 5 · Behaving well on a real machine Build: Handle the microphone being unplugged, the default device changing, sample rate changes, and the machine sleeping and waking. Under processor pressure, fall back to the lighter model rather than producing dropouts, and say that it happened. Trap: Audio device configuration changes at the worst moments: someone plugs in headphones mid-call. If the plug-in does not handle the change it either produces silence or crashes the audio daemon, and the user's call ends. Subscribe to device change notifications and reconfigure rather than assuming a stable configuration. Done when: Unplugging and replugging the microphone mid-stream recovers without restarting the application. Sleeping and waking the machine restores working audio. Under artificial processor load the application falls back to the lighter model and shows that it did, rather than producing dropouts. A sample rate change is handled without a crash. Do not build yet: Distribution comes last. ### Phase 6 · Signing, installation and model licences Build: Sign and notarize. A system audio plug-in needs an installer and needs the audio daemon restarted, so the installation is more involved than dragging an application. Record each model's licence, because they differ. Trap: Installing a server plug-in means writing into a system location and restarting the audio daemon, which briefly interrupts audio for everything running. Warn before doing it, and never do it silently during a call. Done when: A signed installer places the plug-in and the device appears after the documented step. Uninstalling removes the plug-in and leaves system audio working. Each bundled model's licence is recorded and permits redistribution. The installer warns before restarting the audio daemon. Do not build yet: Nothing after this. ### Out of scope (and why) - The commercial model's quality, which comes from training data and tuning you do not have. - Echo cancellation, which is a separate and harder problem than noise suppression and is what stops your own speakers feeding back into the microphone. - Per-application switching polish and the mobile applications. - Speaker-side suppression that cleans what you hear rather than what you send, which needs a virtual output device and its own set of problems. ### README must contain - The measured added latency of each profile on real hardware, and why the live default is the lighter model - That the audio callback allocates nothing and takes no locks, and how deadline misses are measured - That the offline path is deliberately separate and produces better results - How the application behaves when a device changes, the machine sleeps, or the processor is loaded - Every bundled model and its licence ===== AGENTS.md ===== # Agent instructions · Krisp indie build - Read `README.md` and `BUILD_PLAN.md` before writing code. The stack is fixed: Swift for the application, C or Rust for the audio path, A Core Audio server plug-in, RNNoise, BSD licensed, roughly 10 milliseconds and one core, DeepFilterNet, roughly 40 milliseconds and more compute, A lock-free ring buffer between capture and processing. Do not substitute. - Work one phase at a time, in order. Do not start a phase until every "Done when" item of the previous one passes. - Prefer the fewest moving parts that satisfy the step. No frameworks, services or dependencies the plan does not name. - Secrets live in `.env`, never in source or logs. Keep `.env.example` current when a variable is introduced. - Do not invent cryptography, security guarantees, APIs or compliance claims. - Add a focused test for every destructive, security-sensitive or data-loss path the plan names. - Run the project checks before declaring a phase complete, and record any deliberate shortcut in the README under "Tradeoffs". ## Known traps - A server plug-in runs inside a system audio daemon rather than in your application, so a crash in it takes down audio for the whole machine and a debugger cannot attach the way you expect. Keep the plug-in minimal and put every piece of logic you can into your own process, communicating over a shared buffer. - Allocating memory, taking a lock or logging inside the audio callback will eventually block for longer than the deadline, and the result is a click that the user hears and you cannot reproduce. Preallocate everything, use a lock-free ring buffer, and move logging out to another thread. - Roughly 40 milliseconds of added latency is not noticeable when you listen back to a recording and is very noticeable in a conversation, where it sits on top of network delay and makes people talk over each other. Label the profiles by purpose rather than by quality, so the choice is obvious. - Reusing the real-time path for files throws away the one advantage offline processing has, which is that it can look ahead. Run the file path without the deadline and without the streaming constraints, and it will beat the live path on the same recording. - Audio device configuration changes at the worst moments: someone plugs in headphones mid-call. If the plug-in does not handle the change it either produces silence or crashes the audio daemon, and the user's call ends. Subscribe to device change notifications and reconfigure rather than assuming a stable configuration. ===== BUILD_PLAN.md ===== # Build plan · Krisp A virtual microphone that appears in every application's input list, pulls from your real microphone, runs a noise suppression model in the audio callback without ever missing a deadline, and offers a lower latency model for live calls and a heavier one for recordings. Phases are in dependency order. Each ends in a "Done when" list; treat an unticked item as a blocker, not a note. ## Phase 1 · A virtual device that appears in other applications Your device shows up in the input list of an application you did not write, and passes audio through unchanged. ### Steps 1. Build the server plug-in A minimal Core Audio server plug-in publishing one input device with a fixed sample rate and channel count. Install it and restart the audio daemon to load it. Files: `plugin/` ```sh sudo killall coreaudiod ``` 2. Pass audio through untouched Real microphone in, virtual device out, no processing. Verify bit-for-bit where you can. Files: `plugin/passthrough.c` 3. Test in applications you did not write A conferencing application and a recorder. Your own test harness will not reveal the sample rate and buffer size assumptions real applications make. ### Done when - [ ] The device appears in another application's input list - [ ] Speaking through it produces audio in that application - [ ] An hour of use produces no dropouts - [ ] Quitting your application leaves system audio working ### Watch out - A server plug-in runs inside a system audio daemon rather than in your application, so a crash in it takes down audio for the whole machine and a debugger cannot attach the way you expect. Keep the plug-in minimal and put every piece of logic you can into your own process, communicating over a shared buffer. ## Phase 2 · Processing without missing a deadline Noise suppression runs in the audio path and the callback always returns in time. ### Steps 1. Preallocate and go lock-free Every buffer allocated at start-up, a lock-free ring buffer between threads, and no system calls in the callback. Files: `audio/ring.c` 2. Integrate RNNoise It expects a specific frame size, so buffer to that size rather than assuming the host's buffer matches. Files: `audio/denoise.c` 3. Measure every callback Duration and deadline misses, aggregated off the audio thread and shown in the interface. Files: `audio/stats.c` ### Done when - [ ] Fan noise is audibly suppressed - [ ] Worst case callback duration over ten minutes stays inside the deadline - [ ] A deliberately inserted allocation produces measured deadline misses - [ ] Ten minutes of use produces no dropouts ### Watch out - Allocating memory, taking a lock or logging inside the audio callback will eventually block for longer than the deadline, and the result is a click that the user hears and you cannot reproduce. Preallocate everything, use a lock-free ring buffer, and move logging out to another thread. ## Phase 3 · Two models, two purposes You can choose clean audio or low latency, and the interface tells you the cost. ### Steps 1. Add the second model behind one interface Same buffer contract, different implementation. Switching is swapping the processor, not rebuilding the path. Files: `audio/models/` 2. Measure latency per profile on this machine Round trip through the device, measured rather than quoted. Hardware varies enough that the paper figure is not your figure. Files: `audio/latency.c` 3. Label profiles by purpose Calls and recordings, with the measured numbers shown underneath. Not good and better. Files: `ui/Profiles.swift` ### Done when - [ ] Switching profiles does not remove the device from other applications - [ ] Each profile shows its measured latency on this machine - [ ] The high quality profile is audibly better on background speech - [ ] Live use defaults to the low latency profile ### Watch out - Roughly 40 milliseconds of added latency is not noticeable when you listen back to a recording and is very noticeable in a conversation, where it sits on top of network delay and makes people talk over each other. Label the profiles by purpose rather than by quality, so the choice is obvious. ## Phase 4 · Recording mode, where latency does not matter Files get the best available quality because nothing is waiting on them. ### Steps 1. Build the offline path No deadline, full lookahead, best model. This is a different code path on purpose. Files: `offline/process.c` 2. Never modify the input Write a new file alongside. Checksum the source before and after as a test. Files: `offline/process.c` 3. Report before and after A noise floor measurement from a silent passage, so the improvement is a number rather than an impression. Files: `offline/report.c` ### Done when - [ ] Offline processing beats the live path on the same file - [ ] Source files are unmodified by checksum - [ ] A batch of ten files completes unattended - [ ] Each file reports a before and after noise figure ### Watch out - Reusing the real-time path for files throws away the one advantage offline processing has, which is that it can look ahead. Run the file path without the deadline and without the streaming constraints, and it will beat the live path on the same recording. ## Phase 5 · Behaving well on a real machine It survives the things that actually happen: devices changing, sleep, and processor pressure. ### Steps 1. Handle device changes Subscribe to Core Audio device notifications and reconfigure the path. Test by unplugging things during a stream. Files: `audio/devices.c` 2. Survive sleep and wake Tear down and rebuild the path around sleep rather than assuming it survives. Files: `audio/lifecycle.c` 3. Degrade rather than drop If deadline misses rise, switch to the lighter model and tell the user. A dropout is worse than a quality reduction. Files: `audio/adaptive.c` ### Done when - [ ] Unplugging and replugging the microphone recovers automatically - [ ] Sleep and wake restores working audio - [ ] Processor pressure triggers a visible fallback rather than dropouts - [ ] A sample rate change is handled without a crash ### Watch out - Audio device configuration changes at the worst moments: someone plugs in headphones mid-call. If the plug-in does not handle the change it either produces silence or crashes the audio daemon, and the user's call ends. Subscribe to device change notifications and reconfigure rather than assuming a stable configuration. ## Not in this build - The commercial model's quality on hard noise. - Echo cancellation. - Mobile applications and per-application switching polish. - Speaker-side suppression of what you hear. ## After v1, if you want it - A meter showing suppressed noise in real time, which makes the effect visible - Echo cancellation, accepting that it is a larger problem than suppression - A Windows build, which needs an entirely different virtual device approach ===== .env.example ===== # Copy to .env and fill in. Never commit .env; this file documents it. # Required. The published device's sample rate. Applications will resample if they need something else. DEVICE_SAMPLE_RATE=48000 # Required. Frames per callback. This sets the deadline: 256 frames at 48 kHz is roughly 5.3 milliseconds. DEVICE_BUFFER_FRAMES=256 # Required. Which profile is active by default. Calls means the lighter, lower latency model. DEFAULT_PROFILE=calls # Required. Misses per minute before the application falls back to the lighter model and says so. DEADLINE_MISS_THRESHOLD=5 # Required. Where offline processed files are written. Never the source directory. OFFLINE_OUTPUT_DIR=~/Music/Denoised
You are building a lean indie version of Krisp. Create the following project files first, then implement the application by following them. Keep the files updated as decisions change. Do not collapse this into a single README or prompt. ===== README.md ===== # Krisp · indie build A virtual microphone that appears in every application's input list, pulls from your real microphone, runs a noise suppression model in the audio callback without ever missing a deadline, and offers a lower latency model for live calls and a heavier one for recordings. Estimated effort: **multi-day**. Work `BUILD_PLAN.md` top to bottom · every phase ends in a check that has to pass before the next one starts. ## Stack | Part | Choice | Why | | --- | --- | --- | | Language | Swift for the application, C or Rust for the audio path | the processing callback is real-time code and cannot afford allocation or garbage collection | | Virtual device | A Core Audio server plug-in | the supported way to publish an input device other applications can select | | Low latency model | RNNoise, BSD licensed, roughly 10 milliseconds and one core | good on steady noise such as fans and hum, and cheap enough to never miss a deadline | | High quality model | DeepFilterNet, roughly 40 milliseconds and more compute | clearly better on overlapping speech, clatter and reverberation, at a latency cost that matters live | | Buffering | A lock-free ring buffer between capture and processing | a mutex in an audio callback is a dropout waiting to happen | ## Before you start Have every one of these ready. The plan assumes them from step one. - [ ] **A Mac running macOS 13 or newer** · free if you own one - Why: The virtual device is a Core Audio server plug-in, which is a macOS concept. The equivalent on other platforms is a different project. - Get it: Apple menu, then About This Mac. - Verify: About This Mac reports macOS 13 or higher - [ ] **Xcode 16 or newer** · free - Why: Builds the application and the plug-in, and provides Instruments for the timing work in Phase 2. - Get it: Mac App Store or developer.apple.com/xcode. - Verify: xcodebuild -version prints 16 or higher - [ ] **Comfort with real-time audio constraints** · free - Why: An audio callback that allocates or blocks produces clicks the user hears and you cannot reproduce. This is the one part of the build where the usual habits are actively wrong. - Get it: Read about real-time safety before Phase 2. The rule is short: no allocation, no locks, no system calls, no unbounded work. - Verify: You can state why a mutex in the audio callback is a bug rather than a risk - [ ] **RNNoise and a DeepFilterNet build** · free, but check each licence for redistribution - Why: These are the two models the profiles use. Their latency and compute difference is the product decision in Phase 3. - Get it: Build RNNoise from its repository, which is small and BSD licensed. DeepFilterNet ships a real-time capable build; check its current licence before bundling. - Verify: Both process a test file and their added latency is measured - [ ] **Recordings with four kinds of noise** · free - Why: Steady fan noise is where every model succeeds. The differences appear on background speech, sudden clatter and reverberation. - Get it: Record them yourself: a fan, a café, a keyboard and a room with hard walls. Keep them as fixtures. - Verify: All four are saved with their noise floor measured ## Quick start ```sh sudo killall coreaudiod ``` Then copy `.env.example` to `.env` and fill in the values it documents. ## Honest limits This build deliberately does not replace: - The commercial model's quality on hard noise. - Echo cancellation. - Mobile applications and per-application switching polish. - Speaker-side suppression of what you hear. - low-latency virtual microphone - polished app switching - model quality - meeting integrations - admin controls If one of those is essential to you, that is the reason to keep paying for Krisp, and the README should say so rather than pretend. ===== BRIEF.md ===== # Build brief · Krisp The one-shot brief this plan expands. `BUILD_PLAN.md` (or `MILESTONES.md`) is the same sequence broken into steps and checks; where the two disagree, the plan wins. Build me a noise suppressing virtual microphone that other applications can select as an input device to replace Krisp. Build it in phases, in the order below. Do not write the whole thing in one pass. Finish a phase, run its "Done when" check, fix what fails, and only then start the next phase. ### Rule zero Do not train a noise suppression model. Two good open ones exist and the engineering that matters here is everything around them: a virtual audio device the operating system trusts, a processing loop that never misses its deadline, and an honest latency budget. Choose the model from the budget rather than from a quality comparison. RNNoise adds roughly 10 milliseconds and runs on one CPU core; DeepFilterNet sounds clearly better and adds roughly 40 milliseconds and wants real compute. On a live call that difference is audible as conversational lag, so the right answer depends on whether you are cleaning a call or a recording. Checked September 2026. ### Stack (fixed, do not substitute) - Language: Swift for the application, C or Rust for the audio path · the processing callback is real-time code and cannot afford allocation or garbage collection - Virtual device: A Core Audio server plug-in · the supported way to publish an input device other applications can select - Low latency model: RNNoise, BSD licensed, roughly 10 milliseconds and one core · good on steady noise such as fans and hum, and cheap enough to never miss a deadline - High quality model: DeepFilterNet, roughly 40 milliseconds and more compute · clearly better on overlapping speech, clatter and reverberation, at a latency cost that matters live - Buffering: A lock-free ring buffer between capture and processing · a mutex in an audio callback is a dropout waiting to happen ### Data model (create this before Phase 1) - This is a real-time audio path, so the state that matters is timing rather than tables. - device · the published virtual input, its sample rate, its buffer size and its channel count - profiles(id, name, model, expected_latency_ms, cpu_budget_pct) · one for calls, one for recordings - stats · rolling callback duration, deadline misses, dropouts, all visible in the interface - The deadline is the whole design: at a 48 kHz sample rate with a 256 frame buffer you have roughly 5.3 milliseconds to return, every time, without exception. ### Phase 1 · A virtual device that appears in other applications Build: Build a Core Audio server plug-in publishing one input device. Pass audio straight through from the real microphone with no processing at all. Getting the device recognised and stable is the whole phase, because nothing else can be tested until another application can select it. Trap: A server plug-in runs inside a system audio daemon rather than in your application, so a crash in it takes down audio for the whole machine and a debugger cannot attach the way you expect. Keep the plug-in minimal and put every piece of logic you can into your own process, communicating over a shared buffer. Done when: The virtual device appears in the input list of a video conferencing application and a recorder. Selecting it and speaking produces audio in that application. Running for an hour produces no dropouts. Quitting your application leaves the system's audio working rather than broken. Do not build yet: No noise suppression at all. A pass-through device that does not break anything. ### Phase 2 · Processing without missing a deadline Build: Insert RNNoise into the path. Measure callback duration on every call and count deadline misses. Do no allocation, no locking and no file access inside the callback. Show the measured headroom in the interface. Trap: Allocating memory, taking a lock or logging inside the audio callback will eventually block for longer than the deadline, and the result is a click that the user hears and you cannot reproduce. Preallocate everything, use a lock-free ring buffer, and move logging out to another thread. Done when: Noise suppression is audibly working on a recording with fan noise. Callback duration is measured and its worst case over ten minutes stays well inside the deadline. A deliberate allocation inserted into the callback shows up as measured deadline misses, proving the measurement is real. No dropouts during a ten minute call. Do not build yet: Only one model and no profiles yet. ### Phase 3 · Two models, two purposes Build: Add DeepFilterNet as a second profile. Show each profile's measured added latency and processor use, not the numbers from a paper. Let the user switch without restarting the device, and default to the low latency model for live use. Trap: Roughly 40 milliseconds of added latency is not noticeable when you listen back to a recording and is very noticeable in a conversation, where it sits on top of network delay and makes people talk over each other. Label the profiles by purpose rather than by quality, so the choice is obvious. Done when: Switching profiles takes effect without the device disappearing from other applications. Each profile reports its own measured added latency on this machine. The high quality profile audibly outperforms the low latency one on a recording with background speech. The default for live use is the low latency profile. Do not build yet: No per-application behaviour or recording mode yet. ### Phase 4 · Recording mode, where latency does not matter Build: An offline path that processes a file rather than a stream, using the heavier model with as much lookahead as it wants. Keep the original untouched and write a new file. Report a measurable before and after noise figure. Trap: Reusing the real-time path for files throws away the one advantage offline processing has, which is that it can look ahead. Run the file path without the deadline and without the streaming constraints, and it will beat the live path on the same recording. Done when: A file processed offline is measurably quieter in silent passages than the same file processed through the live path. The original file is unmodified, verified by checksum. A batch of ten files processes without intervention and reports each one's before and after figure. Do not build yet: No per-application rules yet. ### Phase 5 · Behaving well on a real machine Build: Handle the microphone being unplugged, the default device changing, sample rate changes, and the machine sleeping and waking. Under processor pressure, fall back to the lighter model rather than producing dropouts, and say that it happened. Trap: Audio device configuration changes at the worst moments: someone plugs in headphones mid-call. If the plug-in does not handle the change it either produces silence or crashes the audio daemon, and the user's call ends. Subscribe to device change notifications and reconfigure rather than assuming a stable configuration. Done when: Unplugging and replugging the microphone mid-stream recovers without restarting the application. Sleeping and waking the machine restores working audio. Under artificial processor load the application falls back to the lighter model and shows that it did, rather than producing dropouts. A sample rate change is handled without a crash. Do not build yet: Distribution comes last. ### Phase 6 · Signing, installation and model licences Build: Sign and notarize. A system audio plug-in needs an installer and needs the audio daemon restarted, so the installation is more involved than dragging an application. Record each model's licence, because they differ. Trap: Installing a server plug-in means writing into a system location and restarting the audio daemon, which briefly interrupts audio for everything running. Warn before doing it, and never do it silently during a call. Done when: A signed installer places the plug-in and the device appears after the documented step. Uninstalling removes the plug-in and leaves system audio working. Each bundled model's licence is recorded and permits redistribution. The installer warns before restarting the audio daemon. Do not build yet: Nothing after this. ### Out of scope (and why) - The commercial model's quality, which comes from training data and tuning you do not have. - Echo cancellation, which is a separate and harder problem than noise suppression and is what stops your own speakers feeding back into the microphone. - Per-application switching polish and the mobile applications. - Speaker-side suppression that cleans what you hear rather than what you send, which needs a virtual output device and its own set of problems. ### README must contain - The measured added latency of each profile on real hardware, and why the live default is the lighter model - That the audio callback allocates nothing and takes no locks, and how deadline misses are measured - That the offline path is deliberately separate and produces better results - How the application behaves when a device changes, the machine sleeps, or the processor is loaded - Every bundled model and its licence ===== AGENTS.md ===== # Agent instructions · Krisp indie build - Read `README.md` and `BUILD_PLAN.md` before writing code. The stack is fixed: Swift for the application, C or Rust for the audio path, A Core Audio server plug-in, RNNoise, BSD licensed, roughly 10 milliseconds and one core, DeepFilterNet, roughly 40 milliseconds and more compute, A lock-free ring buffer between capture and processing. Do not substitute. - Work one phase at a time, in order. Do not start a phase until every "Done when" item of the previous one passes. - Prefer the fewest moving parts that satisfy the step. No frameworks, services or dependencies the plan does not name. - Secrets live in `.env`, never in source or logs. Keep `.env.example` current when a variable is introduced. - Do not invent cryptography, security guarantees, APIs or compliance claims. - Add a focused test for every destructive, security-sensitive or data-loss path the plan names. - Run the project checks before declaring a phase complete, and record any deliberate shortcut in the README under "Tradeoffs". ## Known traps - A server plug-in runs inside a system audio daemon rather than in your application, so a crash in it takes down audio for the whole machine and a debugger cannot attach the way you expect. Keep the plug-in minimal and put every piece of logic you can into your own process, communicating over a shared buffer. - Allocating memory, taking a lock or logging inside the audio callback will eventually block for longer than the deadline, and the result is a click that the user hears and you cannot reproduce. Preallocate everything, use a lock-free ring buffer, and move logging out to another thread. - Roughly 40 milliseconds of added latency is not noticeable when you listen back to a recording and is very noticeable in a conversation, where it sits on top of network delay and makes people talk over each other. Label the profiles by purpose rather than by quality, so the choice is obvious. - Reusing the real-time path for files throws away the one advantage offline processing has, which is that it can look ahead. Run the file path without the deadline and without the streaming constraints, and it will beat the live path on the same recording. - Audio device configuration changes at the worst moments: someone plugs in headphones mid-call. If the plug-in does not handle the change it either produces silence or crashes the audio daemon, and the user's call ends. Subscribe to device change notifications and reconfigure rather than assuming a stable configuration. ===== BUILD_PLAN.md ===== # Build plan · Krisp A virtual microphone that appears in every application's input list, pulls from your real microphone, runs a noise suppression model in the audio callback without ever missing a deadline, and offers a lower latency model for live calls and a heavier one for recordings. Phases are in dependency order. Each ends in a "Done when" list; treat an unticked item as a blocker, not a note. ## Phase 1 · A virtual device that appears in other applications Your device shows up in the input list of an application you did not write, and passes audio through unchanged. ### Steps 1. Build the server plug-in A minimal Core Audio server plug-in publishing one input device with a fixed sample rate and channel count. Install it and restart the audio daemon to load it. Files: `plugin/` ```sh sudo killall coreaudiod ``` 2. Pass audio through untouched Real microphone in, virtual device out, no processing. Verify bit-for-bit where you can. Files: `plugin/passthrough.c` 3. Test in applications you did not write A conferencing application and a recorder. Your own test harness will not reveal the sample rate and buffer size assumptions real applications make. ### Done when - [ ] The device appears in another application's input list - [ ] Speaking through it produces audio in that application - [ ] An hour of use produces no dropouts - [ ] Quitting your application leaves system audio working ### Watch out - A server plug-in runs inside a system audio daemon rather than in your application, so a crash in it takes down audio for the whole machine and a debugger cannot attach the way you expect. Keep the plug-in minimal and put every piece of logic you can into your own process, communicating over a shared buffer. ## Phase 2 · Processing without missing a deadline Noise suppression runs in the audio path and the callback always returns in time. ### Steps 1. Preallocate and go lock-free Every buffer allocated at start-up, a lock-free ring buffer between threads, and no system calls in the callback. Files: `audio/ring.c` 2. Integrate RNNoise It expects a specific frame size, so buffer to that size rather than assuming the host's buffer matches. Files: `audio/denoise.c` 3. Measure every callback Duration and deadline misses, aggregated off the audio thread and shown in the interface. Files: `audio/stats.c` ### Done when - [ ] Fan noise is audibly suppressed - [ ] Worst case callback duration over ten minutes stays inside the deadline - [ ] A deliberately inserted allocation produces measured deadline misses - [ ] Ten minutes of use produces no dropouts ### Watch out - Allocating memory, taking a lock or logging inside the audio callback will eventually block for longer than the deadline, and the result is a click that the user hears and you cannot reproduce. Preallocate everything, use a lock-free ring buffer, and move logging out to another thread. ## Phase 3 · Two models, two purposes You can choose clean audio or low latency, and the interface tells you the cost. ### Steps 1. Add the second model behind one interface Same buffer contract, different implementation. Switching is swapping the processor, not rebuilding the path. Files: `audio/models/` 2. Measure latency per profile on this machine Round trip through the device, measured rather than quoted. Hardware varies enough that the paper figure is not your figure. Files: `audio/latency.c` 3. Label profiles by purpose Calls and recordings, with the measured numbers shown underneath. Not good and better. Files: `ui/Profiles.swift` ### Done when - [ ] Switching profiles does not remove the device from other applications - [ ] Each profile shows its measured latency on this machine - [ ] The high quality profile is audibly better on background speech - [ ] Live use defaults to the low latency profile ### Watch out - Roughly 40 milliseconds of added latency is not noticeable when you listen back to a recording and is very noticeable in a conversation, where it sits on top of network delay and makes people talk over each other. Label the profiles by purpose rather than by quality, so the choice is obvious. ## Phase 4 · Recording mode, where latency does not matter Files get the best available quality because nothing is waiting on them. ### Steps 1. Build the offline path No deadline, full lookahead, best model. This is a different code path on purpose. Files: `offline/process.c` 2. Never modify the input Write a new file alongside. Checksum the source before and after as a test. Files: `offline/process.c` 3. Report before and after A noise floor measurement from a silent passage, so the improvement is a number rather than an impression. Files: `offline/report.c` ### Done when - [ ] Offline processing beats the live path on the same file - [ ] Source files are unmodified by checksum - [ ] A batch of ten files completes unattended - [ ] Each file reports a before and after noise figure ### Watch out - Reusing the real-time path for files throws away the one advantage offline processing has, which is that it can look ahead. Run the file path without the deadline and without the streaming constraints, and it will beat the live path on the same recording. ## Phase 5 · Behaving well on a real machine It survives the things that actually happen: devices changing, sleep, and processor pressure. ### Steps 1. Handle device changes Subscribe to Core Audio device notifications and reconfigure the path. Test by unplugging things during a stream. Files: `audio/devices.c` 2. Survive sleep and wake Tear down and rebuild the path around sleep rather than assuming it survives. Files: `audio/lifecycle.c` 3. Degrade rather than drop If deadline misses rise, switch to the lighter model and tell the user. A dropout is worse than a quality reduction. Files: `audio/adaptive.c` ### Done when - [ ] Unplugging and replugging the microphone recovers automatically - [ ] Sleep and wake restores working audio - [ ] Processor pressure triggers a visible fallback rather than dropouts - [ ] A sample rate change is handled without a crash ### Watch out - Audio device configuration changes at the worst moments: someone plugs in headphones mid-call. If the plug-in does not handle the change it either produces silence or crashes the audio daemon, and the user's call ends. Subscribe to device change notifications and reconfigure rather than assuming a stable configuration. ## Not in this build - The commercial model's quality on hard noise. - Echo cancellation. - Mobile applications and per-application switching polish. - Speaker-side suppression of what you hear. ## After v1, if you want it - A meter showing suppressed noise in real time, which makes the effect visible - Echo cancellation, accepting that it is a larger problem than suppression - A Windows build, which needs an entirely different virtual device approach ===== .env.example ===== # Copy to .env and fill in. Never commit .env; this file documents it. # Required. The published device's sample rate. Applications will resample if they need something else. DEVICE_SAMPLE_RATE=48000 # Required. Frames per callback. This sets the deadline: 256 frames at 48 kHz is roughly 5.3 milliseconds. DEVICE_BUFFER_FRAMES=256 # Required. Which profile is active by default. Calls means the lighter, lower latency model. DEFAULT_PROFILE=calls # Required. Misses per minute before the application falls back to the lighter model and says so. DEADLINE_MISS_THRESHOLD=5 # Required. Where offline processed files are written. Never the source directory. OFFLINE_OUTPUT_DIR=~/Music/Denoised
You are building a production product version of Krisp. Create the following project files first, then implement the application by following them. Keep the files updated as decisions change. Do not collapse this into a single README or prompt. ===== PRODUCT.md ===== # Krisp · product brief ## Problem You can build post-processing and maybe route audio through open models, but real-time low-latency virtual-device noise cancellation is not a one-sitting web app. ## Product outcome A virtual microphone other applications trust, whose processing never misses its deadline, whose latency cost is measured rather than quoted, which degrades visibly under pressure instead of clicking, and which uninstalls cleanly. ## Target user A builder who needs a maintainable product foundation, not a one-off demo. ## Required capabilities - audio DSP library - OS-level virtual audio device - model/runtime optimized for low latency - macOS/Windows audio permissions ## Explicit non-goals for v1 - The commercial model's quality on hard noise. - Echo cancellation. - Mobile applications and per-application switching polish. - Speaker-side suppression of what you hear. - low-latency virtual microphone - polished app switching - model quality - meeting integrations - admin controls ## Success criteria - Worst case callback duration over an hour stays inside the deadline - Device change, sleep and wake are all verified by doing them - Each profile's latency is measured on real hardware and shown in the interface - Uninstalling is confirmed to leave system audio working - Every bundled model licence permits redistribution ===== BRIEF.md ===== # Build brief · Krisp The one-shot brief this plan expands. `BUILD_PLAN.md` (or `MILESTONES.md`) is the same sequence broken into steps and checks; where the two disagree, the plan wins. Build me a noise suppressing virtual microphone that other applications can select as an input device to replace Krisp. Build it in phases, in the order below. Do not write the whole thing in one pass. Finish a phase, run its "Done when" check, fix what fails, and only then start the next phase. ### Rule zero Do not train a noise suppression model. Two good open ones exist and the engineering that matters here is everything around them: a virtual audio device the operating system trusts, a processing loop that never misses its deadline, and an honest latency budget. Choose the model from the budget rather than from a quality comparison. RNNoise adds roughly 10 milliseconds and runs on one CPU core; DeepFilterNet sounds clearly better and adds roughly 40 milliseconds and wants real compute. On a live call that difference is audible as conversational lag, so the right answer depends on whether you are cleaning a call or a recording. Checked September 2026. ### Stack (fixed, do not substitute) - Language: Swift for the application, C or Rust for the audio path · the processing callback is real-time code and cannot afford allocation or garbage collection - Virtual device: A Core Audio server plug-in · the supported way to publish an input device other applications can select - Low latency model: RNNoise, BSD licensed, roughly 10 milliseconds and one core · good on steady noise such as fans and hum, and cheap enough to never miss a deadline - High quality model: DeepFilterNet, roughly 40 milliseconds and more compute · clearly better on overlapping speech, clatter and reverberation, at a latency cost that matters live - Buffering: A lock-free ring buffer between capture and processing · a mutex in an audio callback is a dropout waiting to happen ### Data model (create this before Phase 1) - This is a real-time audio path, so the state that matters is timing rather than tables. - device · the published virtual input, its sample rate, its buffer size and its channel count - profiles(id, name, model, expected_latency_ms, cpu_budget_pct) · one for calls, one for recordings - stats · rolling callback duration, deadline misses, dropouts, all visible in the interface - The deadline is the whole design: at a 48 kHz sample rate with a 256 frame buffer you have roughly 5.3 milliseconds to return, every time, without exception. ### Phase 1 · A virtual device that appears in other applications Build: Build a Core Audio server plug-in publishing one input device. Pass audio straight through from the real microphone with no processing at all. Getting the device recognised and stable is the whole phase, because nothing else can be tested until another application can select it. Trap: A server plug-in runs inside a system audio daemon rather than in your application, so a crash in it takes down audio for the whole machine and a debugger cannot attach the way you expect. Keep the plug-in minimal and put every piece of logic you can into your own process, communicating over a shared buffer. Done when: The virtual device appears in the input list of a video conferencing application and a recorder. Selecting it and speaking produces audio in that application. Running for an hour produces no dropouts. Quitting your application leaves the system's audio working rather than broken. Do not build yet: No noise suppression at all. A pass-through device that does not break anything. ### Phase 2 · Processing without missing a deadline Build: Insert RNNoise into the path. Measure callback duration on every call and count deadline misses. Do no allocation, no locking and no file access inside the callback. Show the measured headroom in the interface. Trap: Allocating memory, taking a lock or logging inside the audio callback will eventually block for longer than the deadline, and the result is a click that the user hears and you cannot reproduce. Preallocate everything, use a lock-free ring buffer, and move logging out to another thread. Done when: Noise suppression is audibly working on a recording with fan noise. Callback duration is measured and its worst case over ten minutes stays well inside the deadline. A deliberate allocation inserted into the callback shows up as measured deadline misses, proving the measurement is real. No dropouts during a ten minute call. Do not build yet: Only one model and no profiles yet. ### Phase 3 · Two models, two purposes Build: Add DeepFilterNet as a second profile. Show each profile's measured added latency and processor use, not the numbers from a paper. Let the user switch without restarting the device, and default to the low latency model for live use. Trap: Roughly 40 milliseconds of added latency is not noticeable when you listen back to a recording and is very noticeable in a conversation, where it sits on top of network delay and makes people talk over each other. Label the profiles by purpose rather than by quality, so the choice is obvious. Done when: Switching profiles takes effect without the device disappearing from other applications. Each profile reports its own measured added latency on this machine. The high quality profile audibly outperforms the low latency one on a recording with background speech. The default for live use is the low latency profile. Do not build yet: No per-application behaviour or recording mode yet. ### Phase 4 · Recording mode, where latency does not matter Build: An offline path that processes a file rather than a stream, using the heavier model with as much lookahead as it wants. Keep the original untouched and write a new file. Report a measurable before and after noise figure. Trap: Reusing the real-time path for files throws away the one advantage offline processing has, which is that it can look ahead. Run the file path without the deadline and without the streaming constraints, and it will beat the live path on the same recording. Done when: A file processed offline is measurably quieter in silent passages than the same file processed through the live path. The original file is unmodified, verified by checksum. A batch of ten files processes without intervention and reports each one's before and after figure. Do not build yet: No per-application rules yet. ### Phase 5 · Behaving well on a real machine Build: Handle the microphone being unplugged, the default device changing, sample rate changes, and the machine sleeping and waking. Under processor pressure, fall back to the lighter model rather than producing dropouts, and say that it happened. Trap: Audio device configuration changes at the worst moments: someone plugs in headphones mid-call. If the plug-in does not handle the change it either produces silence or crashes the audio daemon, and the user's call ends. Subscribe to device change notifications and reconfigure rather than assuming a stable configuration. Done when: Unplugging and replugging the microphone mid-stream recovers without restarting the application. Sleeping and waking the machine restores working audio. Under artificial processor load the application falls back to the lighter model and shows that it did, rather than producing dropouts. A sample rate change is handled without a crash. Do not build yet: Distribution comes last. ### Phase 6 · Signing, installation and model licences Build: Sign and notarize. A system audio plug-in needs an installer and needs the audio daemon restarted, so the installation is more involved than dragging an application. Record each model's licence, because they differ. Trap: Installing a server plug-in means writing into a system location and restarting the audio daemon, which briefly interrupts audio for everything running. Warn before doing it, and never do it silently during a call. Done when: A signed installer places the plug-in and the device appears after the documented step. Uninstalling removes the plug-in and leaves system audio working. Each bundled model's licence is recorded and permits redistribution. The installer warns before restarting the audio daemon. Do not build yet: Nothing after this. ### Out of scope (and why) - The commercial model's quality, which comes from training data and tuning you do not have. - Echo cancellation, which is a separate and harder problem than noise suppression and is what stops your own speakers feeding back into the microphone. - Per-application switching polish and the mobile applications. - Speaker-side suppression that cleans what you hear rather than what you send, which needs a virtual output device and its own set of problems. ### README must contain - The measured added latency of each profile on real hardware, and why the live default is the lighter model - That the audio callback allocates nothing and takes no locks, and how deadline misses are measured - That the offline path is deliberately separate and produces better results - How the application behaves when a device changes, the machine sleeps, or the processor is loaded - Every bundled model and its licence ===== ARCHITECTURE.md ===== # Architecture · Krisp ## Stack | Part | Choice | Why | | --- | --- | --- | | Language | Swift for the application, C or Rust for the audio path | the processing callback is real-time code and cannot afford allocation or garbage collection | | Virtual device | A Core Audio server plug-in | the supported way to publish an input device other applications can select | | Low latency model | RNNoise, BSD licensed, roughly 10 milliseconds and one core | good on steady noise such as fans and hum, and cheap enough to never miss a deadline | | High quality model | DeepFilterNet, roughly 40 milliseconds and more compute | clearly better on overlapping speech, clatter and reverberation, at a latency cost that matters live | | Buffering | A lock-free ring buffer between capture and processing | a mutex in an audio callback is a dropout waiting to happen | ## Modules Each module has one owner concern and a documented way to replace it. | Module | Owns | How to replace it | | --- | --- | --- | | Plug-in | Publishing the device and moving samples. | Not swappable on macOS. Keep it minimal, because a crash here takes down system audio. | | Transport | The lock-free path between capture and processing. | Implementation can change; the real-time safety rules cannot. | | Models | The actual suppression. | Both models sit behind one buffer contract, which is what makes profiles a configuration change rather than a rewrite. | | Stats | Callback timing and deadline misses. | This is what turns dropouts from a mystery into a number, and it must stay on the fast path's measurements rather than sampling occasionally. | | Offline | File processing without a deadline. | Deliberately separate from the live path so it can use lookahead. Merging the two would cost quality for no benefit. | ## Configuration Every runtime setting is an environment variable documented in `.env.example`, validated at startup, with a safe local default wherever one exists. - `DEVICE_SAMPLE_RATE` · required · The published device's sample rate. Applications will resample if they need something else. - `DEVICE_BUFFER_FRAMES` · required · Frames per callback. This sets the deadline: 256 frames at 48 kHz is roughly 5.3 milliseconds. - `DEFAULT_PROFILE` · required · Which profile is active by default. Calls means the lighter, lower latency model. - `DEADLINE_MISS_THRESHOLD` · required · Misses per minute before the application falls back to the lighter model and says so. - `OFFLINE_OUTPUT_DIR` · required · Where offline processed files are written. Never the source directory. ## Production baseline - Security: least privilege, input validation at every boundary, secret redaction in logs, rate limits on abuse-prone paths, no invented security primitives. - Data: explicit schema and migrations, transactional writes where integrity matters, backup and restore procedures that have been exercised. - Integrations: adapters around third-party providers, idempotent webhook or job processing, bounded retries, timeouts. - Observability: structured logs with request or operation ids, an error-tracking hook, and health and readiness checks where a server exists. - Quality: unit tests for domain rules, integration tests at module boundaries, one end-to-end test of the critical path. ## Decision records For each dependency in the stack table, keep a short note: why it was chosen, its failure mode, and how it is replaced. Do not add infrastructure until a requirement in `PRODUCT.md` justifies it. ===== AGENTS.md ===== # Agent instructions · Krisp product build - Read `PRODUCT.md` and `ARCHITECTURE.md` before changing code. The stack is fixed: Swift for the application, C or Rust for the audio path, A Core Audio server plug-in, RNNoise, BSD licensed, roughly 10 milliseconds and one core, DeepFilterNet, roughly 40 milliseconds and more compute, A lock-free ring buffer between capture and processing. - Implement milestone by milestone from `MILESTONES.md`; keep each change reviewable and leave the application runnable at every commit. - Treat authentication, payments, encryption, imports, webhooks and destructive actions as high-risk boundaries when present. - Never invent cryptography or silently weaken a requirement to make a check pass. - Put every external service behind an interface with a deterministic fake for tests. - Add migrations and rollback or recovery notes for every persistent data change. - Log useful operational context without credentials, tokens, passwords or personal data. - Update documentation and run every check before completing a milestone. ## Known traps - A server plug-in runs inside a system audio daemon rather than in your application, so a crash in it takes down audio for the whole machine and a debugger cannot attach the way you expect. Keep the plug-in minimal and put every piece of logic you can into your own process, communicating over a shared buffer. - Allocating memory, taking a lock or logging inside the audio callback will eventually block for longer than the deadline, and the result is a click that the user hears and you cannot reproduce. Preallocate everything, use a lock-free ring buffer, and move logging out to another thread. - Roughly 40 milliseconds of added latency is not noticeable when you listen back to a recording and is very noticeable in a conversation, where it sits on top of network delay and makes people talk over each other. Label the profiles by purpose rather than by quality, so the choice is obvious. - Reusing the real-time path for files throws away the one advantage offline processing has, which is that it can look ahead. Run the file path without the deadline and without the streaming constraints, and it will beat the live path on the same recording. - Audio device configuration changes at the worst moments: someone plugs in headphones mid-call. If the plug-in does not handle the change it either produces silence or crashes the audio daemon, and the user's call ends. Subscribe to device change notifications and reconfigure rather than assuming a stable configuration. - Installing a server plug-in means writing into a system location and restarting the audio daemon, which briefly interrupts audio for everything running. Warn before doing it, and never do it silently during a call. ===== MILESTONES.md ===== # Delivery milestones · Krisp Estimated effort: **multi-day** for the indie phases; the production-only milestones add the trust and operability layer. ## M1 · A virtual device that appears in other applications Your device shows up in the input list of an application you did not write, and passes audio through unchanged. ### Steps 1. Build the server plug-in A minimal Core Audio server plug-in publishing one input device with a fixed sample rate and channel count. Install it and restart the audio daemon to load it. Files: `plugin/` ```sh sudo killall coreaudiod ``` 2. Pass audio through untouched Real microphone in, virtual device out, no processing. Verify bit-for-bit where you can. Files: `plugin/passthrough.c` 3. Test in applications you did not write A conferencing application and a recorder. Your own test harness will not reveal the sample rate and buffer size assumptions real applications make. ### Done when - [ ] The device appears in another application's input list - [ ] Speaking through it produces audio in that application - [ ] An hour of use produces no dropouts - [ ] Quitting your application leaves system audio working ### Watch out - A server plug-in runs inside a system audio daemon rather than in your application, so a crash in it takes down audio for the whole machine and a debugger cannot attach the way you expect. Keep the plug-in minimal and put every piece of logic you can into your own process, communicating over a shared buffer. ## M2 · Processing without missing a deadline Noise suppression runs in the audio path and the callback always returns in time. ### Steps 1. Preallocate and go lock-free Every buffer allocated at start-up, a lock-free ring buffer between threads, and no system calls in the callback. Files: `audio/ring.c` 2. Integrate RNNoise It expects a specific frame size, so buffer to that size rather than assuming the host's buffer matches. Files: `audio/denoise.c` 3. Measure every callback Duration and deadline misses, aggregated off the audio thread and shown in the interface. Files: `audio/stats.c` ### Done when - [ ] Fan noise is audibly suppressed - [ ] Worst case callback duration over ten minutes stays inside the deadline - [ ] A deliberately inserted allocation produces measured deadline misses - [ ] Ten minutes of use produces no dropouts ### Watch out - Allocating memory, taking a lock or logging inside the audio callback will eventually block for longer than the deadline, and the result is a click that the user hears and you cannot reproduce. Preallocate everything, use a lock-free ring buffer, and move logging out to another thread. ## M3 · Two models, two purposes You can choose clean audio or low latency, and the interface tells you the cost. ### Steps 1. Add the second model behind one interface Same buffer contract, different implementation. Switching is swapping the processor, not rebuilding the path. Files: `audio/models/` 2. Measure latency per profile on this machine Round trip through the device, measured rather than quoted. Hardware varies enough that the paper figure is not your figure. Files: `audio/latency.c` 3. Label profiles by purpose Calls and recordings, with the measured numbers shown underneath. Not good and better. Files: `ui/Profiles.swift` ### Done when - [ ] Switching profiles does not remove the device from other applications - [ ] Each profile shows its measured latency on this machine - [ ] The high quality profile is audibly better on background speech - [ ] Live use defaults to the low latency profile ### Watch out - Roughly 40 milliseconds of added latency is not noticeable when you listen back to a recording and is very noticeable in a conversation, where it sits on top of network delay and makes people talk over each other. Label the profiles by purpose rather than by quality, so the choice is obvious. ## M4 · Recording mode, where latency does not matter Files get the best available quality because nothing is waiting on them. ### Steps 1. Build the offline path No deadline, full lookahead, best model. This is a different code path on purpose. Files: `offline/process.c` 2. Never modify the input Write a new file alongside. Checksum the source before and after as a test. Files: `offline/process.c` 3. Report before and after A noise floor measurement from a silent passage, so the improvement is a number rather than an impression. Files: `offline/report.c` ### Done when - [ ] Offline processing beats the live path on the same file - [ ] Source files are unmodified by checksum - [ ] A batch of ten files completes unattended - [ ] Each file reports a before and after noise figure ### Watch out - Reusing the real-time path for files throws away the one advantage offline processing has, which is that it can look ahead. Run the file path without the deadline and without the streaming constraints, and it will beat the live path on the same recording. ## M5 · Behaving well on a real machine It survives the things that actually happen: devices changing, sleep, and processor pressure. ### Steps 1. Handle device changes Subscribe to Core Audio device notifications and reconfigure the path. Test by unplugging things during a stream. Files: `audio/devices.c` 2. Survive sleep and wake Tear down and rebuild the path around sleep rather than assuming it survives. Files: `audio/lifecycle.c` 3. Degrade rather than drop If deadline misses rise, switch to the lighter model and tell the user. A dropout is worse than a quality reduction. Files: `audio/adaptive.c` ### Done when - [ ] Unplugging and replugging the microphone recovers automatically - [ ] Sleep and wake restores working audio - [ ] Processor pressure triggers a visible fallback rather than dropouts - [ ] A sample rate change is handled without a crash ### Watch out - Audio device configuration changes at the worst moments: someone plugs in headphones mid-call. If the plug-in does not handle the change it either produces silence or crashes the audio daemon, and the user's call ends. Subscribe to device change notifications and reconfigure rather than assuming a stable configuration. ## M6 · Signing, installation and model licences (production only) It installs on someone else's machine, and every model you ship is one you may ship. ### Steps 1. Sign, notarize and package Hardened runtime for the application, correct signing for the plug-in, and an installer package. ```sh xcrun notarytool submit Denoise.zip --keychain-profile AC --wait ``` 2. Write a real uninstaller Removing a system plug-in by hand is not something you should ask of anyone. Test that audio still works afterwards. Files: `scripts/uninstall.sh` 3. Record model licences RNNoise is BSD licensed. Check the terms of anything else you bundle, because permissive is not universal. Files: `docs/models.md` ### Done when - [ ] The signed installer produces a working device - [ ] Uninstalling leaves system audio working - [ ] Every bundled model's licence permits redistribution - [ ] The installer warns before restarting the audio daemon ### Watch out - Installing a server plug-in means writing into a system location and restarting the audio daemon, which briefly interrupts audio for everything running. Warn before doing it, and never do it silently during a call. ===== OPERATIONS.md ===== # Operations · Krisp ## Backup Configuration is small and the models are re-downloadable. There is no user data here worth backing up, which is unusual and worth stating. ## Restore Reinstall, reselect the device in each application. Applications remember input device by name, so keeping the device name stable across versions matters more than it looks. Do a restore drill before the first real user, and write the date here when it passes. ## Monitoring Deadline misses per minute and processor use per profile. A rising miss rate is the leading indicator of the dropouts users will report, and it is visible before they hear anything. ## Incident checklist If system audio breaks, the plug-in is the first suspect: uninstall it and restart the audio daemon. That is why the uninstaller must work and must be documented somewhere a user can find without the application running. 1. Contain the issue without destroying evidence or user data. 2. Record the timeline and affected scope. 3. Rotate exposed secrets and revoke compromised sessions or credentials. 4. Restore from a verified backup when needed. 5. Document the root cause, the remediation and the regression test. ## Release gate - [ ] Worst case callback duration over an hour stays inside the deadline - [ ] Device change, sleep and wake are all verified by doing them - [ ] Each profile's latency is measured on real hardware and shown in the interface - [ ] Uninstalling is confirmed to leave system audio working - [ ] Every bundled model licence permits redistribution ## Launch constraint Do not market omitted Krisp capabilities as implemented. The non-goals in `PRODUCT.md` remain user-visible limitations until they are deliberately delivered. ===== .env.example ===== # Copy to .env and fill in. Never commit .env; this file documents it. # Required. The published device's sample rate. Applications will resample if they need something else. DEVICE_SAMPLE_RATE=48000 # Required. Frames per callback. This sets the deadline: 256 frames at 48 kHz is roughly 5.3 milliseconds. DEVICE_BUFFER_FRAMES=256 # Required. Which profile is active by default. Calls means the lighter, lower latency model. DEFAULT_PROFILE=calls # Required. Misses per minute before the application falls back to the lighter model and says so. DEADLINE_MISS_THRESHOLD=5 # Required. Where offline processed files are written. Never the source directory. OFFLINE_OUTPUT_DIR=~/Music/Denoised
README.md
# Krisp · indie build A virtual microphone that appears in every application's input list, pulls from your real microphone, runs a noise suppression model in the audio callback without ever missing a deadline, and offers a lower latency model for live calls and a heavier one for recordings. Estimated effort: **multi-day**. Work `BUILD_PLAN.md` top to bottom · every phase ends in a check that has to pass before the next one starts. ## Stack | Part | Choice | Why | | --- | --- | --- | | Language | Swift for the application, C or Rust for the audio path | the processing callback is real-time code and cannot afford allocation or garbage collection | | Virtual device | A Core Audio server plug-in | the supported way to publish an input device other applications can select | | Low latency model | RNNoise, BSD licensed, roughly 10 milliseconds and one core | good on steady noise such as fans and hum, and cheap enough to never miss a deadline | | High quality model | DeepFilterNet, roughly 40 milliseconds and more compute | clearly better on overlapping speech, clatter and reverberation, at a latency cost that matters live | | Buffering | A lock-free ring buffer between capture and processing | a mutex in an audio callback is a dropout waiting to happen | ## Before you start Have every one of these ready. The plan assumes them from step one. - [ ] **A Mac running macOS 13 or newer** · free if you own one - Why: The virtual device is a Core Audio server plug-in, which is a macOS concept. The equivalent on other platforms is a different project. - Get it: Apple menu, then About This Mac. - Verify: About This Mac reports macOS 13 or higher - [ ] **Xcode 16 or newer** · free - Why: Builds the application and the plug-in, and provides Instruments for the timing work in Phase 2. - Get it: Mac App Store or developer.apple.com/xcode. - Verify: xcodebuild -version prints 16 or higher - [ ] **Comfort with real-time audio constraints** · free - Why: An audio callback that allocates or blocks produces clicks the user hears and you cannot reproduce. This is the one part of the build where the usual habits are actively wrong. - Get it: Read about real-time safety before Phase 2. The rule is short: no allocation, no locks, no system calls, no unbounded work. - Verify: You can state why a mutex in the audio callback is a bug rather than a risk - [ ] **RNNoise and a DeepFilterNet build** · free, but check each licence for redistribution - Why: These are the two models the profiles use. Their latency and compute difference is the product decision in Phase 3. - Get it: Build RNNoise from its repository, which is small and BSD licensed. DeepFilterNet ships a real-time capable build; check its current licence before bundling. - Verify: Both process a test file and their added latency is measured - [ ] **Recordings with four kinds of noise** · free - Why: Steady fan noise is where every model succeeds. The differences appear on background speech, sudden clatter and reverberation. - Get it: Record them yourself: a fan, a café, a keyboard and a room with hard walls. Keep them as fixtures. - Verify: All four are saved with their noise floor measured ## Quick start ```sh sudo killall coreaudiod ``` Then copy `.env.example` to `.env` and fill in the values it documents. ## Honest limits This build deliberately does not replace: - The commercial model's quality on hard noise. - Echo cancellation. - Mobile applications and per-application switching polish. - Speaker-side suppression of what you hear. - low-latency virtual microphone - polished app switching - model quality - meeting integrations - admin controls If one of those is essential to you, that is the reason to keep paying for Krisp, and the README should say so rather than pretend.
BRIEF.md
# Build brief · Krisp The one-shot brief this plan expands. `BUILD_PLAN.md` (or `MILESTONES.md`) is the same sequence broken into steps and checks; where the two disagree, the plan wins. Build me a noise suppressing virtual microphone that other applications can select as an input device to replace Krisp. Build it in phases, in the order below. Do not write the whole thing in one pass. Finish a phase, run its "Done when" check, fix what fails, and only then start the next phase. ### Rule zero Do not train a noise suppression model. Two good open ones exist and the engineering that matters here is everything around them: a virtual audio device the operating system trusts, a processing loop that never misses its deadline, and an honest latency budget. Choose the model from the budget rather than from a quality comparison. RNNoise adds roughly 10 milliseconds and runs on one CPU core; DeepFilterNet sounds clearly better and adds roughly 40 milliseconds and wants real compute. On a live call that difference is audible as conversational lag, so the right answer depends on whether you are cleaning a call or a recording. Checked September 2026. ### Stack (fixed, do not substitute) - Language: Swift for the application, C or Rust for the audio path · the processing callback is real-time code and cannot afford allocation or garbage collection - Virtual device: A Core Audio server plug-in · the supported way to publish an input device other applications can select - Low latency model: RNNoise, BSD licensed, roughly 10 milliseconds and one core · good on steady noise such as fans and hum, and cheap enough to never miss a deadline - High quality model: DeepFilterNet, roughly 40 milliseconds and more compute · clearly better on overlapping speech, clatter and reverberation, at a latency cost that matters live - Buffering: A lock-free ring buffer between capture and processing · a mutex in an audio callback is a dropout waiting to happen ### Data model (create this before Phase 1) - This is a real-time audio path, so the state that matters is timing rather than tables. - device · the published virtual input, its sample rate, its buffer size and its channel count - profiles(id, name, model, expected_latency_ms, cpu_budget_pct) · one for calls, one for recordings - stats · rolling callback duration, deadline misses, dropouts, all visible in the interface - The deadline is the whole design: at a 48 kHz sample rate with a 256 frame buffer you have roughly 5.3 milliseconds to return, every time, without exception. ### Phase 1 · A virtual device that appears in other applications Build: Build a Core Audio server plug-in publishing one input device. Pass audio straight through from the real microphone with no processing at all. Getting the device recognised and stable is the whole phase, because nothing else can be tested until another application can select it. Trap: A server plug-in runs inside a system audio daemon rather than in your application, so a crash in it takes down audio for the whole machine and a debugger cannot attach the way you expect. Keep the plug-in minimal and put every piece of logic you can into your own process, communicating over a shared buffer. Done when: The virtual device appears in the input list of a video conferencing application and a recorder. Selecting it and speaking produces audio in that application. Running for an hour produces no dropouts. Quitting your application leaves the system's audio working rather than broken. Do not build yet: No noise suppression at all. A pass-through device that does not break anything. ### Phase 2 · Processing without missing a deadline Build: Insert RNNoise into the path. Measure callback duration on every call and count deadline misses. Do no allocation, no locking and no file access inside the callback. Show the measured headroom in the interface. Trap: Allocating memory, taking a lock or logging inside the audio callback will eventually block for longer than the deadline, and the result is a click that the user hears and you cannot reproduce. Preallocate everything, use a lock-free ring buffer, and move logging out to another thread. Done when: Noise suppression is audibly working on a recording with fan noise. Callback duration is measured and its worst case over ten minutes stays well inside the deadline. A deliberate allocation inserted into the callback shows up as measured deadline misses, proving the measurement is real. No dropouts during a ten minute call. Do not build yet: Only one model and no profiles yet. ### Phase 3 · Two models, two purposes Build: Add DeepFilterNet as a second profile. Show each profile's measured added latency and processor use, not the numbers from a paper. Let the user switch without restarting the device, and default to the low latency model for live use. Trap: Roughly 40 milliseconds of added latency is not noticeable when you listen back to a recording and is very noticeable in a conversation, where it sits on top of network delay and makes people talk over each other. Label the profiles by purpose rather than by quality, so the choice is obvious. Done when: Switching profiles takes effect without the device disappearing from other applications. Each profile reports its own measured added latency on this machine. The high quality profile audibly outperforms the low latency one on a recording with background speech. The default for live use is the low latency profile. Do not build yet: No per-application behaviour or recording mode yet. ### Phase 4 · Recording mode, where latency does not matter Build: An offline path that processes a file rather than a stream, using the heavier model with as much lookahead as it wants. Keep the original untouched and write a new file. Report a measurable before and after noise figure. Trap: Reusing the real-time path for files throws away the one advantage offline processing has, which is that it can look ahead. Run the file path without the deadline and without the streaming constraints, and it will beat the live path on the same recording. Done when: A file processed offline is measurably quieter in silent passages than the same file processed through the live path. The original file is unmodified, verified by checksum. A batch of ten files processes without intervention and reports each one's before and after figure. Do not build yet: No per-application rules yet. ### Phase 5 · Behaving well on a real machine Build: Handle the microphone being unplugged, the default device changing, sample rate changes, and the machine sleeping and waking. Under processor pressure, fall back to the lighter model rather than producing dropouts, and say that it happened. Trap: Audio device configuration changes at the worst moments: someone plugs in headphones mid-call. If the plug-in does not handle the change it either produces silence or crashes the audio daemon, and the user's call ends. Subscribe to device change notifications and reconfigure rather than assuming a stable configuration. Done when: Unplugging and replugging the microphone mid-stream recovers without restarting the application. Sleeping and waking the machine restores working audio. Under artificial processor load the application falls back to the lighter model and shows that it did, rather than producing dropouts. A sample rate change is handled without a crash. Do not build yet: Distribution comes last. ### Phase 6 · Signing, installation and model licences Build: Sign and notarize. A system audio plug-in needs an installer and needs the audio daemon restarted, so the installation is more involved than dragging an application. Record each model's licence, because they differ. Trap: Installing a server plug-in means writing into a system location and restarting the audio daemon, which briefly interrupts audio for everything running. Warn before doing it, and never do it silently during a call. Done when: A signed installer places the plug-in and the device appears after the documented step. Uninstalling removes the plug-in and leaves system audio working. Each bundled model's licence is recorded and permits redistribution. The installer warns before restarting the audio daemon. Do not build yet: Nothing after this. ### Out of scope (and why) - The commercial model's quality, which comes from training data and tuning you do not have. - Echo cancellation, which is a separate and harder problem than noise suppression and is what stops your own speakers feeding back into the microphone. - Per-application switching polish and the mobile applications. - Speaker-side suppression that cleans what you hear rather than what you send, which needs a virtual output device and its own set of problems. ### README must contain - The measured added latency of each profile on real hardware, and why the live default is the lighter model - That the audio callback allocates nothing and takes no locks, and how deadline misses are measured - That the offline path is deliberately separate and produces better results - How the application behaves when a device changes, the machine sleeps, or the processor is loaded - Every bundled model and its licence
AGENTS.md
# Agent instructions · Krisp indie build - Read `README.md` and `BUILD_PLAN.md` before writing code. The stack is fixed: Swift for the application, C or Rust for the audio path, A Core Audio server plug-in, RNNoise, BSD licensed, roughly 10 milliseconds and one core, DeepFilterNet, roughly 40 milliseconds and more compute, A lock-free ring buffer between capture and processing. Do not substitute. - Work one phase at a time, in order. Do not start a phase until every "Done when" item of the previous one passes. - Prefer the fewest moving parts that satisfy the step. No frameworks, services or dependencies the plan does not name. - Secrets live in `.env`, never in source or logs. Keep `.env.example` current when a variable is introduced. - Do not invent cryptography, security guarantees, APIs or compliance claims. - Add a focused test for every destructive, security-sensitive or data-loss path the plan names. - Run the project checks before declaring a phase complete, and record any deliberate shortcut in the README under "Tradeoffs". ## Known traps - A server plug-in runs inside a system audio daemon rather than in your application, so a crash in it takes down audio for the whole machine and a debugger cannot attach the way you expect. Keep the plug-in minimal and put every piece of logic you can into your own process, communicating over a shared buffer. - Allocating memory, taking a lock or logging inside the audio callback will eventually block for longer than the deadline, and the result is a click that the user hears and you cannot reproduce. Preallocate everything, use a lock-free ring buffer, and move logging out to another thread. - Roughly 40 milliseconds of added latency is not noticeable when you listen back to a recording and is very noticeable in a conversation, where it sits on top of network delay and makes people talk over each other. Label the profiles by purpose rather than by quality, so the choice is obvious. - Reusing the real-time path for files throws away the one advantage offline processing has, which is that it can look ahead. Run the file path without the deadline and without the streaming constraints, and it will beat the live path on the same recording. - Audio device configuration changes at the worst moments: someone plugs in headphones mid-call. If the plug-in does not handle the change it either produces silence or crashes the audio daemon, and the user's call ends. Subscribe to device change notifications and reconfigure rather than assuming a stable configuration.
BUILD_PLAN.md
# Build plan · Krisp A virtual microphone that appears in every application's input list, pulls from your real microphone, runs a noise suppression model in the audio callback without ever missing a deadline, and offers a lower latency model for live calls and a heavier one for recordings. Phases are in dependency order. Each ends in a "Done when" list; treat an unticked item as a blocker, not a note. ## Phase 1 · A virtual device that appears in other applications Your device shows up in the input list of an application you did not write, and passes audio through unchanged. ### Steps 1. Build the server plug-in A minimal Core Audio server plug-in publishing one input device with a fixed sample rate and channel count. Install it and restart the audio daemon to load it. Files: `plugin/` ```sh sudo killall coreaudiod ``` 2. Pass audio through untouched Real microphone in, virtual device out, no processing. Verify bit-for-bit where you can. Files: `plugin/passthrough.c` 3. Test in applications you did not write A conferencing application and a recorder. Your own test harness will not reveal the sample rate and buffer size assumptions real applications make. ### Done when - [ ] The device appears in another application's input list - [ ] Speaking through it produces audio in that application - [ ] An hour of use produces no dropouts - [ ] Quitting your application leaves system audio working ### Watch out - A server plug-in runs inside a system audio daemon rather than in your application, so a crash in it takes down audio for the whole machine and a debugger cannot attach the way you expect. Keep the plug-in minimal and put every piece of logic you can into your own process, communicating over a shared buffer. ## Phase 2 · Processing without missing a deadline Noise suppression runs in the audio path and the callback always returns in time. ### Steps 1. Preallocate and go lock-free Every buffer allocated at start-up, a lock-free ring buffer between threads, and no system calls in the callback. Files: `audio/ring.c` 2. Integrate RNNoise It expects a specific frame size, so buffer to that size rather than assuming the host's buffer matches. Files: `audio/denoise.c` 3. Measure every callback Duration and deadline misses, aggregated off the audio thread and shown in the interface. Files: `audio/stats.c` ### Done when - [ ] Fan noise is audibly suppressed - [ ] Worst case callback duration over ten minutes stays inside the deadline - [ ] A deliberately inserted allocation produces measured deadline misses - [ ] Ten minutes of use produces no dropouts ### Watch out - Allocating memory, taking a lock or logging inside the audio callback will eventually block for longer than the deadline, and the result is a click that the user hears and you cannot reproduce. Preallocate everything, use a lock-free ring buffer, and move logging out to another thread. ## Phase 3 · Two models, two purposes You can choose clean audio or low latency, and the interface tells you the cost. ### Steps 1. Add the second model behind one interface Same buffer contract, different implementation. Switching is swapping the processor, not rebuilding the path. Files: `audio/models/` 2. Measure latency per profile on this machine Round trip through the device, measured rather than quoted. Hardware varies enough that the paper figure is not your figure. Files: `audio/latency.c` 3. Label profiles by purpose Calls and recordings, with the measured numbers shown underneath. Not good and better. Files: `ui/Profiles.swift` ### Done when - [ ] Switching profiles does not remove the device from other applications - [ ] Each profile shows its measured latency on this machine - [ ] The high quality profile is audibly better on background speech - [ ] Live use defaults to the low latency profile ### Watch out - Roughly 40 milliseconds of added latency is not noticeable when you listen back to a recording and is very noticeable in a conversation, where it sits on top of network delay and makes people talk over each other. Label the profiles by purpose rather than by quality, so the choice is obvious. ## Phase 4 · Recording mode, where latency does not matter Files get the best available quality because nothing is waiting on them. ### Steps 1. Build the offline path No deadline, full lookahead, best model. This is a different code path on purpose. Files: `offline/process.c` 2. Never modify the input Write a new file alongside. Checksum the source before and after as a test. Files: `offline/process.c` 3. Report before and after A noise floor measurement from a silent passage, so the improvement is a number rather than an impression. Files: `offline/report.c` ### Done when - [ ] Offline processing beats the live path on the same file - [ ] Source files are unmodified by checksum - [ ] A batch of ten files completes unattended - [ ] Each file reports a before and after noise figure ### Watch out - Reusing the real-time path for files throws away the one advantage offline processing has, which is that it can look ahead. Run the file path without the deadline and without the streaming constraints, and it will beat the live path on the same recording. ## Phase 5 · Behaving well on a real machine It survives the things that actually happen: devices changing, sleep, and processor pressure. ### Steps 1. Handle device changes Subscribe to Core Audio device notifications and reconfigure the path. Test by unplugging things during a stream. Files: `audio/devices.c` 2. Survive sleep and wake Tear down and rebuild the path around sleep rather than assuming it survives. Files: `audio/lifecycle.c` 3. Degrade rather than drop If deadline misses rise, switch to the lighter model and tell the user. A dropout is worse than a quality reduction. Files: `audio/adaptive.c` ### Done when - [ ] Unplugging and replugging the microphone recovers automatically - [ ] Sleep and wake restores working audio - [ ] Processor pressure triggers a visible fallback rather than dropouts - [ ] A sample rate change is handled without a crash ### Watch out - Audio device configuration changes at the worst moments: someone plugs in headphones mid-call. If the plug-in does not handle the change it either produces silence or crashes the audio daemon, and the user's call ends. Subscribe to device change notifications and reconfigure rather than assuming a stable configuration. ## Not in this build - The commercial model's quality on hard noise. - Echo cancellation. - Mobile applications and per-application switching polish. - Speaker-side suppression of what you hear. ## After v1, if you want it - A meter showing suppressed noise in real time, which makes the effect visible - Echo cancellation, accepting that it is a larger problem than suppression - A Windows build, which needs an entirely different virtual device approach
.env.example
# Copy to .env and fill in. Never commit .env; this file documents it. # Required. The published device's sample rate. Applications will resample if they need something else. DEVICE_SAMPLE_RATE=48000 # Required. Frames per callback. This sets the deadline: 256 frames at 48 kHz is roughly 5.3 milliseconds. DEVICE_BUFFER_FRAMES=256 # Required. Which profile is active by default. Calls means the lighter, lower latency model. DEFAULT_PROFILE=calls # Required. Misses per minute before the application falls back to the lighter model and says so. DEADLINE_MISS_THRESHOLD=5 # Required. Where offline processed files are written. Never the source directory. OFFLINE_OUTPUT_DIR=~/Music/Denoised
PRODUCT.md
# Krisp · product brief ## Problem You can build post-processing and maybe route audio through open models, but real-time low-latency virtual-device noise cancellation is not a one-sitting web app. ## Product outcome A virtual microphone other applications trust, whose processing never misses its deadline, whose latency cost is measured rather than quoted, which degrades visibly under pressure instead of clicking, and which uninstalls cleanly. ## Target user A builder who needs a maintainable product foundation, not a one-off demo. ## Required capabilities - audio DSP library - OS-level virtual audio device - model/runtime optimized for low latency - macOS/Windows audio permissions ## Explicit non-goals for v1 - The commercial model's quality on hard noise. - Echo cancellation. - Mobile applications and per-application switching polish. - Speaker-side suppression of what you hear. - low-latency virtual microphone - polished app switching - model quality - meeting integrations - admin controls ## Success criteria - Worst case callback duration over an hour stays inside the deadline - Device change, sleep and wake are all verified by doing them - Each profile's latency is measured on real hardware and shown in the interface - Uninstalling is confirmed to leave system audio working - Every bundled model licence permits redistribution
BRIEF.md
# Build brief · Krisp The one-shot brief this plan expands. `BUILD_PLAN.md` (or `MILESTONES.md`) is the same sequence broken into steps and checks; where the two disagree, the plan wins. Build me a noise suppressing virtual microphone that other applications can select as an input device to replace Krisp. Build it in phases, in the order below. Do not write the whole thing in one pass. Finish a phase, run its "Done when" check, fix what fails, and only then start the next phase. ### Rule zero Do not train a noise suppression model. Two good open ones exist and the engineering that matters here is everything around them: a virtual audio device the operating system trusts, a processing loop that never misses its deadline, and an honest latency budget. Choose the model from the budget rather than from a quality comparison. RNNoise adds roughly 10 milliseconds and runs on one CPU core; DeepFilterNet sounds clearly better and adds roughly 40 milliseconds and wants real compute. On a live call that difference is audible as conversational lag, so the right answer depends on whether you are cleaning a call or a recording. Checked September 2026. ### Stack (fixed, do not substitute) - Language: Swift for the application, C or Rust for the audio path · the processing callback is real-time code and cannot afford allocation or garbage collection - Virtual device: A Core Audio server plug-in · the supported way to publish an input device other applications can select - Low latency model: RNNoise, BSD licensed, roughly 10 milliseconds and one core · good on steady noise such as fans and hum, and cheap enough to never miss a deadline - High quality model: DeepFilterNet, roughly 40 milliseconds and more compute · clearly better on overlapping speech, clatter and reverberation, at a latency cost that matters live - Buffering: A lock-free ring buffer between capture and processing · a mutex in an audio callback is a dropout waiting to happen ### Data model (create this before Phase 1) - This is a real-time audio path, so the state that matters is timing rather than tables. - device · the published virtual input, its sample rate, its buffer size and its channel count - profiles(id, name, model, expected_latency_ms, cpu_budget_pct) · one for calls, one for recordings - stats · rolling callback duration, deadline misses, dropouts, all visible in the interface - The deadline is the whole design: at a 48 kHz sample rate with a 256 frame buffer you have roughly 5.3 milliseconds to return, every time, without exception. ### Phase 1 · A virtual device that appears in other applications Build: Build a Core Audio server plug-in publishing one input device. Pass audio straight through from the real microphone with no processing at all. Getting the device recognised and stable is the whole phase, because nothing else can be tested until another application can select it. Trap: A server plug-in runs inside a system audio daemon rather than in your application, so a crash in it takes down audio for the whole machine and a debugger cannot attach the way you expect. Keep the plug-in minimal and put every piece of logic you can into your own process, communicating over a shared buffer. Done when: The virtual device appears in the input list of a video conferencing application and a recorder. Selecting it and speaking produces audio in that application. Running for an hour produces no dropouts. Quitting your application leaves the system's audio working rather than broken. Do not build yet: No noise suppression at all. A pass-through device that does not break anything. ### Phase 2 · Processing without missing a deadline Build: Insert RNNoise into the path. Measure callback duration on every call and count deadline misses. Do no allocation, no locking and no file access inside the callback. Show the measured headroom in the interface. Trap: Allocating memory, taking a lock or logging inside the audio callback will eventually block for longer than the deadline, and the result is a click that the user hears and you cannot reproduce. Preallocate everything, use a lock-free ring buffer, and move logging out to another thread. Done when: Noise suppression is audibly working on a recording with fan noise. Callback duration is measured and its worst case over ten minutes stays well inside the deadline. A deliberate allocation inserted into the callback shows up as measured deadline misses, proving the measurement is real. No dropouts during a ten minute call. Do not build yet: Only one model and no profiles yet. ### Phase 3 · Two models, two purposes Build: Add DeepFilterNet as a second profile. Show each profile's measured added latency and processor use, not the numbers from a paper. Let the user switch without restarting the device, and default to the low latency model for live use. Trap: Roughly 40 milliseconds of added latency is not noticeable when you listen back to a recording and is very noticeable in a conversation, where it sits on top of network delay and makes people talk over each other. Label the profiles by purpose rather than by quality, so the choice is obvious. Done when: Switching profiles takes effect without the device disappearing from other applications. Each profile reports its own measured added latency on this machine. The high quality profile audibly outperforms the low latency one on a recording with background speech. The default for live use is the low latency profile. Do not build yet: No per-application behaviour or recording mode yet. ### Phase 4 · Recording mode, where latency does not matter Build: An offline path that processes a file rather than a stream, using the heavier model with as much lookahead as it wants. Keep the original untouched and write a new file. Report a measurable before and after noise figure. Trap: Reusing the real-time path for files throws away the one advantage offline processing has, which is that it can look ahead. Run the file path without the deadline and without the streaming constraints, and it will beat the live path on the same recording. Done when: A file processed offline is measurably quieter in silent passages than the same file processed through the live path. The original file is unmodified, verified by checksum. A batch of ten files processes without intervention and reports each one's before and after figure. Do not build yet: No per-application rules yet. ### Phase 5 · Behaving well on a real machine Build: Handle the microphone being unplugged, the default device changing, sample rate changes, and the machine sleeping and waking. Under processor pressure, fall back to the lighter model rather than producing dropouts, and say that it happened. Trap: Audio device configuration changes at the worst moments: someone plugs in headphones mid-call. If the plug-in does not handle the change it either produces silence or crashes the audio daemon, and the user's call ends. Subscribe to device change notifications and reconfigure rather than assuming a stable configuration. Done when: Unplugging and replugging the microphone mid-stream recovers without restarting the application. Sleeping and waking the machine restores working audio. Under artificial processor load the application falls back to the lighter model and shows that it did, rather than producing dropouts. A sample rate change is handled without a crash. Do not build yet: Distribution comes last. ### Phase 6 · Signing, installation and model licences Build: Sign and notarize. A system audio plug-in needs an installer and needs the audio daemon restarted, so the installation is more involved than dragging an application. Record each model's licence, because they differ. Trap: Installing a server plug-in means writing into a system location and restarting the audio daemon, which briefly interrupts audio for everything running. Warn before doing it, and never do it silently during a call. Done when: A signed installer places the plug-in and the device appears after the documented step. Uninstalling removes the plug-in and leaves system audio working. Each bundled model's licence is recorded and permits redistribution. The installer warns before restarting the audio daemon. Do not build yet: Nothing after this. ### Out of scope (and why) - The commercial model's quality, which comes from training data and tuning you do not have. - Echo cancellation, which is a separate and harder problem than noise suppression and is what stops your own speakers feeding back into the microphone. - Per-application switching polish and the mobile applications. - Speaker-side suppression that cleans what you hear rather than what you send, which needs a virtual output device and its own set of problems. ### README must contain - The measured added latency of each profile on real hardware, and why the live default is the lighter model - That the audio callback allocates nothing and takes no locks, and how deadline misses are measured - That the offline path is deliberately separate and produces better results - How the application behaves when a device changes, the machine sleeps, or the processor is loaded - Every bundled model and its licence
ARCHITECTURE.md
# Architecture · Krisp ## Stack | Part | Choice | Why | | --- | --- | --- | | Language | Swift for the application, C or Rust for the audio path | the processing callback is real-time code and cannot afford allocation or garbage collection | | Virtual device | A Core Audio server plug-in | the supported way to publish an input device other applications can select | | Low latency model | RNNoise, BSD licensed, roughly 10 milliseconds and one core | good on steady noise such as fans and hum, and cheap enough to never miss a deadline | | High quality model | DeepFilterNet, roughly 40 milliseconds and more compute | clearly better on overlapping speech, clatter and reverberation, at a latency cost that matters live | | Buffering | A lock-free ring buffer between capture and processing | a mutex in an audio callback is a dropout waiting to happen | ## Modules Each module has one owner concern and a documented way to replace it. | Module | Owns | How to replace it | | --- | --- | --- | | Plug-in | Publishing the device and moving samples. | Not swappable on macOS. Keep it minimal, because a crash here takes down system audio. | | Transport | The lock-free path between capture and processing. | Implementation can change; the real-time safety rules cannot. | | Models | The actual suppression. | Both models sit behind one buffer contract, which is what makes profiles a configuration change rather than a rewrite. | | Stats | Callback timing and deadline misses. | This is what turns dropouts from a mystery into a number, and it must stay on the fast path's measurements rather than sampling occasionally. | | Offline | File processing without a deadline. | Deliberately separate from the live path so it can use lookahead. Merging the two would cost quality for no benefit. | ## Configuration Every runtime setting is an environment variable documented in `.env.example`, validated at startup, with a safe local default wherever one exists. - `DEVICE_SAMPLE_RATE` · required · The published device's sample rate. Applications will resample if they need something else. - `DEVICE_BUFFER_FRAMES` · required · Frames per callback. This sets the deadline: 256 frames at 48 kHz is roughly 5.3 milliseconds. - `DEFAULT_PROFILE` · required · Which profile is active by default. Calls means the lighter, lower latency model. - `DEADLINE_MISS_THRESHOLD` · required · Misses per minute before the application falls back to the lighter model and says so. - `OFFLINE_OUTPUT_DIR` · required · Where offline processed files are written. Never the source directory. ## Production baseline - Security: least privilege, input validation at every boundary, secret redaction in logs, rate limits on abuse-prone paths, no invented security primitives. - Data: explicit schema and migrations, transactional writes where integrity matters, backup and restore procedures that have been exercised. - Integrations: adapters around third-party providers, idempotent webhook or job processing, bounded retries, timeouts. - Observability: structured logs with request or operation ids, an error-tracking hook, and health and readiness checks where a server exists. - Quality: unit tests for domain rules, integration tests at module boundaries, one end-to-end test of the critical path. ## Decision records For each dependency in the stack table, keep a short note: why it was chosen, its failure mode, and how it is replaced. Do not add infrastructure until a requirement in `PRODUCT.md` justifies it.
AGENTS.md
# Agent instructions · Krisp product build - Read `PRODUCT.md` and `ARCHITECTURE.md` before changing code. The stack is fixed: Swift for the application, C or Rust for the audio path, A Core Audio server plug-in, RNNoise, BSD licensed, roughly 10 milliseconds and one core, DeepFilterNet, roughly 40 milliseconds and more compute, A lock-free ring buffer between capture and processing. - Implement milestone by milestone from `MILESTONES.md`; keep each change reviewable and leave the application runnable at every commit. - Treat authentication, payments, encryption, imports, webhooks and destructive actions as high-risk boundaries when present. - Never invent cryptography or silently weaken a requirement to make a check pass. - Put every external service behind an interface with a deterministic fake for tests. - Add migrations and rollback or recovery notes for every persistent data change. - Log useful operational context without credentials, tokens, passwords or personal data. - Update documentation and run every check before completing a milestone. ## Known traps - A server plug-in runs inside a system audio daemon rather than in your application, so a crash in it takes down audio for the whole machine and a debugger cannot attach the way you expect. Keep the plug-in minimal and put every piece of logic you can into your own process, communicating over a shared buffer. - Allocating memory, taking a lock or logging inside the audio callback will eventually block for longer than the deadline, and the result is a click that the user hears and you cannot reproduce. Preallocate everything, use a lock-free ring buffer, and move logging out to another thread. - Roughly 40 milliseconds of added latency is not noticeable when you listen back to a recording and is very noticeable in a conversation, where it sits on top of network delay and makes people talk over each other. Label the profiles by purpose rather than by quality, so the choice is obvious. - Reusing the real-time path for files throws away the one advantage offline processing has, which is that it can look ahead. Run the file path without the deadline and without the streaming constraints, and it will beat the live path on the same recording. - Audio device configuration changes at the worst moments: someone plugs in headphones mid-call. If the plug-in does not handle the change it either produces silence or crashes the audio daemon, and the user's call ends. Subscribe to device change notifications and reconfigure rather than assuming a stable configuration. - Installing a server plug-in means writing into a system location and restarting the audio daemon, which briefly interrupts audio for everything running. Warn before doing it, and never do it silently during a call.
MILESTONES.md
# Delivery milestones · Krisp Estimated effort: **multi-day** for the indie phases; the production-only milestones add the trust and operability layer. ## M1 · A virtual device that appears in other applications Your device shows up in the input list of an application you did not write, and passes audio through unchanged. ### Steps 1. Build the server plug-in A minimal Core Audio server plug-in publishing one input device with a fixed sample rate and channel count. Install it and restart the audio daemon to load it. Files: `plugin/` ```sh sudo killall coreaudiod ``` 2. Pass audio through untouched Real microphone in, virtual device out, no processing. Verify bit-for-bit where you can. Files: `plugin/passthrough.c` 3. Test in applications you did not write A conferencing application and a recorder. Your own test harness will not reveal the sample rate and buffer size assumptions real applications make. ### Done when - [ ] The device appears in another application's input list - [ ] Speaking through it produces audio in that application - [ ] An hour of use produces no dropouts - [ ] Quitting your application leaves system audio working ### Watch out - A server plug-in runs inside a system audio daemon rather than in your application, so a crash in it takes down audio for the whole machine and a debugger cannot attach the way you expect. Keep the plug-in minimal and put every piece of logic you can into your own process, communicating over a shared buffer. ## M2 · Processing without missing a deadline Noise suppression runs in the audio path and the callback always returns in time. ### Steps 1. Preallocate and go lock-free Every buffer allocated at start-up, a lock-free ring buffer between threads, and no system calls in the callback. Files: `audio/ring.c` 2. Integrate RNNoise It expects a specific frame size, so buffer to that size rather than assuming the host's buffer matches. Files: `audio/denoise.c` 3. Measure every callback Duration and deadline misses, aggregated off the audio thread and shown in the interface. Files: `audio/stats.c` ### Done when - [ ] Fan noise is audibly suppressed - [ ] Worst case callback duration over ten minutes stays inside the deadline - [ ] A deliberately inserted allocation produces measured deadline misses - [ ] Ten minutes of use produces no dropouts ### Watch out - Allocating memory, taking a lock or logging inside the audio callback will eventually block for longer than the deadline, and the result is a click that the user hears and you cannot reproduce. Preallocate everything, use a lock-free ring buffer, and move logging out to another thread. ## M3 · Two models, two purposes You can choose clean audio or low latency, and the interface tells you the cost. ### Steps 1. Add the second model behind one interface Same buffer contract, different implementation. Switching is swapping the processor, not rebuilding the path. Files: `audio/models/` 2. Measure latency per profile on this machine Round trip through the device, measured rather than quoted. Hardware varies enough that the paper figure is not your figure. Files: `audio/latency.c` 3. Label profiles by purpose Calls and recordings, with the measured numbers shown underneath. Not good and better. Files: `ui/Profiles.swift` ### Done when - [ ] Switching profiles does not remove the device from other applications - [ ] Each profile shows its measured latency on this machine - [ ] The high quality profile is audibly better on background speech - [ ] Live use defaults to the low latency profile ### Watch out - Roughly 40 milliseconds of added latency is not noticeable when you listen back to a recording and is very noticeable in a conversation, where it sits on top of network delay and makes people talk over each other. Label the profiles by purpose rather than by quality, so the choice is obvious. ## M4 · Recording mode, where latency does not matter Files get the best available quality because nothing is waiting on them. ### Steps 1. Build the offline path No deadline, full lookahead, best model. This is a different code path on purpose. Files: `offline/process.c` 2. Never modify the input Write a new file alongside. Checksum the source before and after as a test. Files: `offline/process.c` 3. Report before and after A noise floor measurement from a silent passage, so the improvement is a number rather than an impression. Files: `offline/report.c` ### Done when - [ ] Offline processing beats the live path on the same file - [ ] Source files are unmodified by checksum - [ ] A batch of ten files completes unattended - [ ] Each file reports a before and after noise figure ### Watch out - Reusing the real-time path for files throws away the one advantage offline processing has, which is that it can look ahead. Run the file path without the deadline and without the streaming constraints, and it will beat the live path on the same recording. ## M5 · Behaving well on a real machine It survives the things that actually happen: devices changing, sleep, and processor pressure. ### Steps 1. Handle device changes Subscribe to Core Audio device notifications and reconfigure the path. Test by unplugging things during a stream. Files: `audio/devices.c` 2. Survive sleep and wake Tear down and rebuild the path around sleep rather than assuming it survives. Files: `audio/lifecycle.c` 3. Degrade rather than drop If deadline misses rise, switch to the lighter model and tell the user. A dropout is worse than a quality reduction. Files: `audio/adaptive.c` ### Done when - [ ] Unplugging and replugging the microphone recovers automatically - [ ] Sleep and wake restores working audio - [ ] Processor pressure triggers a visible fallback rather than dropouts - [ ] A sample rate change is handled without a crash ### Watch out - Audio device configuration changes at the worst moments: someone plugs in headphones mid-call. If the plug-in does not handle the change it either produces silence or crashes the audio daemon, and the user's call ends. Subscribe to device change notifications and reconfigure rather than assuming a stable configuration. ## M6 · Signing, installation and model licences (production only) It installs on someone else's machine, and every model you ship is one you may ship. ### Steps 1. Sign, notarize and package Hardened runtime for the application, correct signing for the plug-in, and an installer package. ```sh xcrun notarytool submit Denoise.zip --keychain-profile AC --wait ``` 2. Write a real uninstaller Removing a system plug-in by hand is not something you should ask of anyone. Test that audio still works afterwards. Files: `scripts/uninstall.sh` 3. Record model licences RNNoise is BSD licensed. Check the terms of anything else you bundle, because permissive is not universal. Files: `docs/models.md` ### Done when - [ ] The signed installer produces a working device - [ ] Uninstalling leaves system audio working - [ ] Every bundled model's licence permits redistribution - [ ] The installer warns before restarting the audio daemon ### Watch out - Installing a server plug-in means writing into a system location and restarting the audio daemon, which briefly interrupts audio for everything running. Warn before doing it, and never do it silently during a call.
OPERATIONS.md
# Operations · Krisp ## Backup Configuration is small and the models are re-downloadable. There is no user data here worth backing up, which is unusual and worth stating. ## Restore Reinstall, reselect the device in each application. Applications remember input device by name, so keeping the device name stable across versions matters more than it looks. Do a restore drill before the first real user, and write the date here when it passes. ## Monitoring Deadline misses per minute and processor use per profile. A rising miss rate is the leading indicator of the dropouts users will report, and it is visible before they hear anything. ## Incident checklist If system audio breaks, the plug-in is the first suspect: uninstall it and restart the audio daemon. That is why the uninstaller must work and must be documented somewhere a user can find without the application running. 1. Contain the issue without destroying evidence or user data. 2. Record the timeline and affected scope. 3. Rotate exposed secrets and revoke compromised sessions or credentials. 4. Restore from a verified backup when needed. 5. Document the root cause, the remediation and the regression test. ## Release gate - [ ] Worst case callback duration over an hour stays inside the deadline - [ ] Device change, sleep and wake are all verified by doing them - [ ] Each profile's latency is measured on real hardware and shown in the interface - [ ] Uninstalling is confirmed to leave system audio working - [ ] Every bundled model licence permits redistribution ## Launch constraint Do not market omitted Krisp capabilities as implemented. The non-goals in `PRODUCT.md` remain user-visible limitations until they are deliberately delivered.
.env.example
# Copy to .env and fill in. Never commit .env; this file documents it. # Required. The published device's sample rate. Applications will resample if they need something else. DEVICE_SAMPLE_RATE=48000 # Required. Frames per callback. This sets the deadline: 256 frames at 48 kHz is roughly 5.3 milliseconds. DEVICE_BUFFER_FRAMES=256 # Required. Which profile is active by default. Calls means the lighter, lower latency model. DEFAULT_PROFILE=calls # Required. Misses per minute before the application falls back to the lighter model and says so. DEADLINE_MISS_THRESHOLD=5 # Required. Where offline processed files are written. Never the source directory. OFFLINE_OUTPUT_DIR=~/Music/Denoised
$ choose a build depth, inspect the files, then open the complete pack in your agent
They pay because audio cleanup must be instant and invisible during real calls.
xlow-latency virtual microphone
xpolished app switching
xmodel quality
xmeeting integrations
xadmin controls
share on X ↗"I just replaced Krisp ($16/mo) with one prompt"
Don't feel like building it? These folks already made it free.
EEasyEffectsKrisp for Linux tinkerers: system-wide cleanup, zero magic account, several knobs.open source↗
NVIDIA BroadcastExcellent noise removal, free in money and expensive in RTX silicon.free↗no votes, no pay-to-list · just what's real
Krisp pricing
| plan | monthly | annual (per mo) | what you get |
|---|---|---|---|
| core | $16/user | $8/user | Unlimited transcription, recording, meeting notes and noise cancellation; accent conversion 1 hour/day; 10 GB storage |
| advanced | $30/user | $15/user | Unlimited meeting AI; speaker accent conversion 4 hours/day; listener accent conversion unlimited; 60 GB storage |
| enterprise | custom | — | Custom seats; unlimited storage; enterprise security, deployment and support |
| call center core | — | $10/user | Starts at $10/agent/month billed annually; public numeric usage caps are not disclosed |
| call center advanced | custom | — | Custom agent count, deployment, integrations and support |
free tierno free tier; 7-day trial includes unlimited transcription, noise cancellation, recording and AI notes, with accent conversion limited
billingmonthly + annual for Meeting AI; Call Center starts on annual billing
hidden costsEach user or agent is billed; BAA eligibility starts at 100 seats, while volume pricing is negotiated above roughly 50 seats.
verified 2026-08-14 · source ↗
Vibecode Krisp
Kinda. The core of Krisp is buildable in a weekend with the prompt on this page, but there are real gaps: low-latency virtual microphone, polished app switching. Read the honest list above before committing.
How much does Krisp cost?
Krisp costs about $16/month (Pro/Core, checked 2026-07-30), which is $192 per year.
What do I lose by replacing Krisp?
Honestly: low-latency virtual microphone; polished app switching; model quality; meeting integrations; admin controls. If any of those are load-bearing for you, keep paying.
Is there an open-source alternative to Krisp?
Yes: EasyEffects (Krisp for Linux tinkerers: system-wide cleanup, zero magic account, several knobs.) NVIDIA Broadcast (Excellent noise removal, free in money and expensive in RTX silicon.) The prompt is for when you want it exactly your way.