Vibecode Sonix
track this build4 phases, 12 steps, beginner friendly0%KINDA · weekend project
The visible automated transcription loop is buildable, but a credible replacement needs more than the first screen. Sonix earns its keep through capture, integrations, reliability, so expect a weekend or multi-day build and a narrower personal scope.
the project pack5 files · written for this build, step by step
You are building a lean indie version of Sonix. Create the following project files first, then implement the application by following them. Keep the files updated as decisions change. Do not collapse this into a single README or prompt. ===== README.md ===== # Sonix · indie build A local transcription pipeline whose hard problems are getting audio into the exact form the recogniser accepts, keeping timestamps honest across a long recording where the model hallucinates through silence, and exporting subtitle formats that players will actually load. Estimated effort: **weekend**. Work `BUILD_PLAN.md` top to bottom · every phase ends in a check that has to pass before the next one starts. ## Stack | Part | Choice | Why | | --- | --- | --- | | Runtime | Python 3.12 with FastAPI and a single background worker | transcription is a queue, not a request | | Recogniser | whisper.cpp invoked as a process, model path in configuration | a local model keeps recordings of other people off other people's servers | | Audio | FFmpeg to one canonical form, sixteen kilohertz mono signed sixteen bit | the command line tool accepts sixteen bit WAV and nothing else | | Storage | SQLite for segments and jobs, media beside it on disk | segments are rows because every one of them gets edited | | Editing | Corrections stored as rows against segments, never overwriting the original output | you will want to know what the machine actually said | ## Before you start Have every one of these ready. The plan assumes them from step one. - [ ] **Python 3.12 or newer** · free - Why: Runs the application and the worker. - Get it: python.org, or your package manager. - Verify: python3 --version prints 3.12 or higher - [ ] **FFmpeg** · free - Why: Produces the one audio form the recogniser accepts, and probes files before they are queued. - Get it: Your package manager, or a static build from the project site. - Verify: ffmpeg -i sample.m4a -ar 16000 -ac 1 -c:a pcm_s16le out.wav produces a playable file - [ ] **whisper.cpp with a downloaded model** · free - Why: The recogniser. Model size decides both quality and how long a ninety minute file takes on your machine. - Get it: Build the project and run its model download script for the size you want. - Verify: The command line tool transcribes the bundled sample - [ ] **A long recording and a silence fixture** · free - Why: Timestamp drift and hallucination through silence only appear on long real audio, never on a thirty second clip. - Get it: One recording of an hour or more that you have the right to use, plus a generated file with two minutes of silence in the middle. - Verify: Both are in the test fixtures and used by the timing tests - [ ] **Your retention period and timestamp tolerance** · free - Why: Both are asserted by tests, and a retention period chosen after a year of recordings have accumulated is not a decision. - Get it: Write down how long audio and transcripts are kept, and the tolerance in milliseconds the timing test enforces. - Verify: Both appear in configuration and in tests ## Quick start Follow Phase 1 in `BUILD_PLAN.md`; it creates the project. Then copy `.env.example` to `.env` and fill in the values it documents. ## Honest limits This build deliberately does not replace: - Meeting bots and call capture. - Live streaming transcription. - Translation and subtitle burn-in. - Cross recording analytics. - live multi-speaker accuracy - calendar and CRM integrations - cross-call team analytics - meeting-bot auto-join If one of those is essential to you, that is the reason to keep paying for Sonix, and the README should say so rather than pretend. ===== BRIEF.md ===== # Build brief · Sonix The one-shot brief this plan expands. `BUILD_PLAN.md` (or `MILESTONES.md`) is the same sequence broken into steps and checks; where the two disagree, the plan wins. Build me a local transcription pipeline that produces timed text you can correct and export to replace Sonix. Build it in phases, in the order below. Do not write the whole thing in one pass. Finish a phase, run its "Done when" check, fix what fails, and only then start the next phase. ### Rule zero Two rules, and both are about not claiming more than you have. Recording other people is not a technical decision. In several places all parties must agree before a conversation is recorded, and the rules differ by country and sometimes by state. So this build transcribes files you already hold, it never joins a meeting or captures a call, and every imported file carries a note of how consent was obtained. If that note is empty the file still imports, but the field is there so the omission is visible. Second, never display a number you did not measure. Recognisers emit token probabilities, not word accuracy, and presenting one as a confidence percentage tells the reader something untrue about a transcript they are about to act on. Show the model name, the settings and the audio quality instead, and if you want an accuracy figure, measure it against a transcript you typed yourself and report that sample size. ### Stack (fixed, do not substitute) - Runtime: Python 3.12 with FastAPI and a single background worker · transcription is a queue, not a request - Recogniser: whisper.cpp invoked as a process, model path in configuration · a local model keeps recordings of other people off other people's servers - Audio: FFmpeg to one canonical form, sixteen kilohertz mono signed sixteen bit · the command line tool accepts sixteen bit WAV and nothing else - Storage: SQLite for segments and jobs, media beside it on disk · segments are rows because every one of them gets edited - Editing: Corrections stored as rows against segments, never overwriting the original output · you will want to know what the machine actually said ### Data model (create this before Phase 1) - media(id, original_path, sha256, mime, duration_ms, imported_at, consent_note) - audio(media_id, canonical_path, sample_rate, channels, encoding) · the only file the recogniser sees - jobs(id, media_id, state, model_name, params_json, attempt, started_at, finished_at, error) - chunks(id, media_id, ordinal, start_ms, end_ms, silence_split) · where the file was cut and why - segments(id, media_id, ordinal, start_ms, end_ms, text, speaker_turn, chunk_id) - edits(id, segment_id, previous_text, new_text, at) · corrections are additive, the machine output stays - exports(id, media_id, format, path, at) · every timestamp in every export derives from segments, never from a chunk offset ### Phase 1 · One canonical audio file, produced before anything else runs Build: Copy the import beside the database, hash it, and convert with FFmpeg to sixteen kilohertz mono signed sixteen bit PCM: the whisper.cpp command line tool documents that it currently runs only with sixteen bit WAV files and gives that exact conversion. Record the duration and refuse a file FFmpeg cannot probe rather than queueing a job that will fail an hour later. Trap: Handing the recogniser the original file because it played fine in the browser gets you either an immediate failure or, worse, a run that produces plausible nonsense from a misread sample rate, and on a two hour recording you find out after two hours of processing. Convert first, assert the sample rate and channel count of the converted file, and treat probe failure as a rejected import. Done when: Every queued job points at a converted file asserted to be sixteen kilohertz, one channel and sixteen bit, tested for inputs in at least three container formats. A file FFmpeg cannot probe is rejected at import with the reason shown. The original import is retained and its hash matches after conversion. Importing the same file twice is detected by hash and offers the existing transcript rather than reprocessing. Do not build yet: No transcription, speakers, editing or export. ### Phase 2 · Long recordings, honest timestamps Build: Whisper models process audio in a sliding thirty second window, so long files are split. Split on detected silence near a target length rather than on a fixed boundary, carry a small overlap, and convert every timestamp back to an offset in the whole recording before it is stored. Detect long stretches of silence or music and mark them rather than transcribing them. Trap: Cutting a recording every thirty seconds on a fixed boundary slices words in half and, worse, feeding the model a passage of silence or music makes it emit fluent invented sentences, which are indistinguishable from real ones in the output because they carry timestamps too. Cut on silence, skip silent stretches explicitly, and mark anything the detector called non speech. Done when: Timestamps in the last segment of a ninety minute file are correct to within the stated tolerance against a known fixture, tested. Chunk boundaries fall on detected silence and the recorded boundary and reason are stored per chunk. A file containing two minutes of silence produces no transcript text for that stretch, tested with a generated fixture. Restarting the worker mid file resumes from the last completed chunk rather than from the beginning. Do not build yet: No speaker handling, editing or export yet. ### Phase 3 · Speakers, described as what they actually are Build: Use whisper.cpp's tinydiarize option where the matching model is installed: it is documented as experimental and it marks a speaker turn in the text rather than identifying anybody. Show turns as breaks, let the user name a run of turns once and apply that name forwards, and label the whole feature as turn detection rather than speaker identification in the interface. Trap: Presenting turn markers as 'Speaker 1' and 'Speaker 2' implies the tool tracked two voices across the recording, when it only noticed that the voice changed, so two people swapping back and forth get relabelled every few minutes and a reader trusts an attribution that was never computed. Show turns, let a person attach the names, and record who attached them. Done when: Turn markers appear only where the recogniser emitted one and no name is generated by the system, verified by review. Naming a turn applies to the run the user selected and is recorded as a human attribution with a timestamp. With no diarisation-capable model installed the feature is absent rather than degraded into guessing. The interface never uses the phrase speaker identification for this. Do not build yet: No editing surface or export yet. ### Phase 4 · Correcting a transcript without destroying its timing Build: Edit at segment level with the audio playable from the segment start. Store corrections as rows so the machine output remains readable. Support merging and splitting segments with the times recomputed from the audio positions rather than from the surrounding text. Search across segments and jump to the time. Trap: Letting the transcript be edited as one long document loses the mapping between text and time the moment someone reflows a paragraph, and the subtitles exported afterwards drift further out of sync the longer the recording runs. Keep the segment as the unit of editing, and derive the document view from segments rather than the other way round. Done when: Editing any segment leaves its start and end times unchanged, tested. Splitting a segment produces two segments whose times are taken from the chosen audio position and whose union equals the original range. The original machine output for every edited segment is still retrievable. Search returns segments and playing from a result starts at that segment's time, within the stated tolerance. Do not build yet: No exports or retention handling yet. ### Phase 5 · Exports that load, and recordings that do not linger Build: Export SubRip, WebVTT and Markdown. The WebVTT specification requires the body to be UTF-8, to begin with the string WEBVTT, and to write cue times with a full stop before the thousandths, while SubRip files conventionally use a comma, so the two formats are not the same file with a different extension. Add retention that deletes audio and transcripts together, and an export of everything for one recording. Trap: Writing SubRip timestamps into a file named with a WebVTT extension, or omitting the WEBVTT line, produces a file that every player accepts silently and displays no captions from, so the failure is discovered by an audience rather than by you. Write each format from its own function and validate an exported file by parsing it back before declaring the export complete. Done when: An exported WebVTT file begins with the required string and uses full stops in cue times, verified by parsing it back in the suite. An exported SubRip file uses commas and sequential cue numbers, verified the same way. Deleting a recording removes the original, the converted audio, the segments and the exports, checked by listing the directory afterwards. Retention runs on a schedule and logs what it removed without logging any transcript text. Do not build yet: Nothing after this. ### Out of scope (and why) - Joining meetings to record them, which is a consent problem before it is a technical one and is the part of the paid product you should think hardest about copying. - Live streaming transcription, which needs a different model configuration and a latency budget this build has no reason to carry. - Translation and subtitle burn-in, each of which is a separate quality problem with its own failure modes. - Cross recording analytics, which only mean anything once capture is automatic, and capture is exactly what is excluded here. ### README must contain - That the tool transcribes files you already hold and never joins or records a call, and why - The exact audio form the recogniser requires and the FFmpeg command that produces it - That long files are split on silence, silent stretches are marked rather than transcribed, and why that matters - That turn detection is not speaker identification, and that every name was attached by a person - That no confidence or accuracy percentage is displayed unless it was measured against a transcript you typed ===== AGENTS.md ===== # Agent instructions · Sonix indie build - Read `README.md` and `BUILD_PLAN.md` before writing code. The stack is fixed: Python 3.12 with FastAPI and a single background worker, whisper.cpp invoked as a process, model path in configuration, FFmpeg to one canonical form, sixteen kilohertz mono signed sixteen bit, SQLite for segments and jobs, media beside it on disk, Corrections stored as rows against segments, never overwriting the original output. Do not substitute. - Work one phase at a time, in order. Do not start a phase until every "Done when" item of the previous one passes. - Prefer the fewest moving parts that satisfy the step. No frameworks, services or dependencies the plan does not name. - Secrets live in `.env`, never in source or logs. Keep `.env.example` current when a variable is introduced. - Do not invent cryptography, security guarantees, APIs or compliance claims. - Add a focused test for every destructive, security-sensitive or data-loss path the plan names. - Run the project checks before declaring a phase complete, and record any deliberate shortcut in the README under "Tradeoffs". ## Known traps - Handing the recogniser the original file because it played fine in the browser gets you either an immediate failure or, worse, a run that produces plausible nonsense from a misread sample rate, and on a two hour recording you find out after two hours of processing. Convert first, assert the sample rate and channel count of the converted file, and treat probe failure as a rejected import. - Cutting a recording every thirty seconds on a fixed boundary slices words in half and, worse, feeding the model a passage of silence or music makes it emit fluent invented sentences, which are indistinguishable from real ones in the output because they carry timestamps too. Cut on silence, skip silent stretches explicitly, and mark anything the detector called non speech. - Presenting turn markers as 'Speaker 1' and 'Speaker 2' implies the tool tracked two voices across the recording, when it only noticed that the voice changed, so two people swapping back and forth get relabelled every few minutes and a reader trusts an attribution that was never computed. Show turns, let a person attach the names, and record who attached them. - Letting the transcript be edited as one long document loses the mapping between text and time the moment someone reflows a paragraph, and the subtitles exported afterwards drift further out of sync the longer the recording runs. Keep the segment as the unit of editing, and derive the document view from segments rather than the other way round. ===== BUILD_PLAN.md ===== # Build plan · Sonix A local transcription pipeline whose hard problems are getting audio into the exact form the recogniser accepts, keeping timestamps honest across a long recording where the model hallucinates through silence, and exporting subtitle formats that players will actually load. Phases are in dependency order. Each ends in a "Done when" list; treat an unticked item as a blocker, not a note. ## Phase 1 · One canonical audio file, produced before anything else runs Every recording becomes the exact format the recogniser accepts, and the original is kept. ### Steps 1. Convert to the documented form, then assert it The tool accepts one shape of audio; produce exactly that shape. Files: `src/audio/convert.py`, `tests/test_convert.py` 2. Probe before queueing An hour of processing is an expensive way to discover a broken file. Files: `src/audio/probe.py` 3. Deduplicate by hash The same recording gets imported twice more often than you would think. Files: `src/ingest.py` ### Done when - [ ] Every job's audio is asserted to be sixteen kilohertz mono sixteen bit - [ ] Unprobeable files are rejected at import - [ ] The original is retained with a matching hash - [ ] A duplicate import is detected by hash ### Watch out - Handing the recogniser the original file because it played fine in the browser gets you either an immediate failure or, worse, a run that produces plausible nonsense from a misread sample rate, and on a two hour recording you find out after two hours of processing. Convert first, assert the sample rate and channel count of the converted file, and treat probe failure as a rejected import. ## Phase 2 · Long recordings, honest timestamps A ninety minute file transcribes with timings that line up at the end as well as the start. ### Steps 1. Split on silence, not on a stopwatch A fixed cut lands in the middle of the most important sentence. Files: `src/transcribe/chunking.py` 2. Convert chunk time to file time once Every export inherits this arithmetic, so get it wrong in one place only. Files: `src/transcribe/timing.py`, `tests/test_timing.py` 3. Mark silence rather than transcribing it Fluent invented sentences are the failure mode nobody notices. Files: `src/transcribe/silence.py` ### Done when - [ ] End of file timestamps are within tolerance - [ ] Chunk boundaries land on detected silence and are recorded - [ ] Silent stretches produce no text - [ ] A worker restart resumes at the last chunk ### Watch out - Cutting a recording every thirty seconds on a fixed boundary slices words in half and, worse, feeding the model a passage of silence or music makes it emit fluent invented sentences, which are indistinguishable from real ones in the output because they carry timestamps too. Cut on silence, skip silent stretches explicitly, and mark anything the detector called non speech. ## Phase 3 · Speakers, described as what they actually are Speaker changes are marked where the tool can mark them, and never invented. ### Steps 1. Use turn detection only where the model supports it And say in the interface that it is experimental, because it is. Files: `src/transcribe/diarise.py` 2. Let a person attach every name And store that a person did it. Files: `src/app/segments.py` 3. Hide the feature when the model is missing A guessed speaker is worse than no speaker. Files: `src/config.py` ### Done when - [ ] No speaker name is generated by the system - [ ] Names are recorded as human attributions - [ ] The feature is absent without a capable model - [ ] The wording says turn detection ### Watch out - Presenting turn markers as 'Speaker 1' and 'Speaker 2' implies the tool tracked two voices across the recording, when it only noticed that the voice changed, so two people swapping back and forth get relabelled every few minutes and a reader trusts an attribution that was never computed. Show turns, let a person attach the names, and record who attached them. ## Phase 4 · Correcting a transcript without destroying its timing Editing text keeps every segment anchored to the audio. ### Steps 1. Make the segment the unit of editing The document view is a rendering of segments, never the source. Files: `src/app/editor.py` 2. Keep corrections additive You will want to compare what the machine said with what it meant. Files: `src/db/edits.py` 3. Recompute times from audio on split Not from character counts, which have nothing to do with speech. Files: `src/transcribe/split.py`, `tests/test_split.py` ### Done when - [ ] Editing text leaves segment times unchanged - [ ] A split produces times covering the original range - [ ] Machine output remains retrievable after edits - [ ] Search results play from the right position ### Watch out - Letting the transcript be edited as one long document loses the mapping between text and time the moment someone reflows a paragraph, and the subtitles exported afterwards drift further out of sync the longer the recording runs. Keep the segment as the unit of editing, and derive the document view from segments rather than the other way round. ## Not in this build - Meeting bots and call capture. - Live streaming transcription. - Translation and subtitle burn-in. - Cross recording analytics. ## After v1, if you want it - A measured accuracy report against transcripts you typed, with the sample size stated - Word level timing for a karaoke style review view, which the recogniser can already emit - A watched folder that imports and queues automatically, once retention has run for a month ===== .env.example ===== # Copy to .env and fill in. Never commit .env; this file documents it. # Required. Jobs, chunks, segments and edits. DATABASE_PATH=./data/transcripts.db # Required. Originals and converted audio, deleted together by retention. MEDIA_DIR=./data/media # Required. The recogniser binary invoked by the worker. WHISPER_BIN=/usr/local/bin/whisper-cli # Required. Model file. Its name is recorded on every job. WHISPER_MODEL=./models/ggml-base.en.bin # Required. Target chunk length. The actual cut lands on nearby silence. CHUNK_TARGET_SECONDS=240 # Required. Audio and transcripts are removed together after this. RETENTION_DAYS=180 # Required. What the long file timing test enforces at the end of a recording. TIMING_TOLERANCE_MS=250
You are building a lean indie version of Sonix. Create the following project files first, then implement the application by following them. Keep the files updated as decisions change. Do not collapse this into a single README or prompt. ===== README.md ===== # Sonix · indie build A local transcription pipeline whose hard problems are getting audio into the exact form the recogniser accepts, keeping timestamps honest across a long recording where the model hallucinates through silence, and exporting subtitle formats that players will actually load. Estimated effort: **weekend**. Work `BUILD_PLAN.md` top to bottom · every phase ends in a check that has to pass before the next one starts. ## Stack | Part | Choice | Why | | --- | --- | --- | | Runtime | Python 3.12 with FastAPI and a single background worker | transcription is a queue, not a request | | Recogniser | whisper.cpp invoked as a process, model path in configuration | a local model keeps recordings of other people off other people's servers | | Audio | FFmpeg to one canonical form, sixteen kilohertz mono signed sixteen bit | the command line tool accepts sixteen bit WAV and nothing else | | Storage | SQLite for segments and jobs, media beside it on disk | segments are rows because every one of them gets edited | | Editing | Corrections stored as rows against segments, never overwriting the original output | you will want to know what the machine actually said | ## Before you start Have every one of these ready. The plan assumes them from step one. - [ ] **Python 3.12 or newer** · free - Why: Runs the application and the worker. - Get it: python.org, or your package manager. - Verify: python3 --version prints 3.12 or higher - [ ] **FFmpeg** · free - Why: Produces the one audio form the recogniser accepts, and probes files before they are queued. - Get it: Your package manager, or a static build from the project site. - Verify: ffmpeg -i sample.m4a -ar 16000 -ac 1 -c:a pcm_s16le out.wav produces a playable file - [ ] **whisper.cpp with a downloaded model** · free - Why: The recogniser. Model size decides both quality and how long a ninety minute file takes on your machine. - Get it: Build the project and run its model download script for the size you want. - Verify: The command line tool transcribes the bundled sample - [ ] **A long recording and a silence fixture** · free - Why: Timestamp drift and hallucination through silence only appear on long real audio, never on a thirty second clip. - Get it: One recording of an hour or more that you have the right to use, plus a generated file with two minutes of silence in the middle. - Verify: Both are in the test fixtures and used by the timing tests - [ ] **Your retention period and timestamp tolerance** · free - Why: Both are asserted by tests, and a retention period chosen after a year of recordings have accumulated is not a decision. - Get it: Write down how long audio and transcripts are kept, and the tolerance in milliseconds the timing test enforces. - Verify: Both appear in configuration and in tests ## Quick start Follow Phase 1 in `BUILD_PLAN.md`; it creates the project. Then copy `.env.example` to `.env` and fill in the values it documents. ## Honest limits This build deliberately does not replace: - Meeting bots and call capture. - Live streaming transcription. - Translation and subtitle burn-in. - Cross recording analytics. - live multi-speaker accuracy - calendar and CRM integrations - cross-call team analytics - meeting-bot auto-join If one of those is essential to you, that is the reason to keep paying for Sonix, and the README should say so rather than pretend. ===== BRIEF.md ===== # Build brief · Sonix The one-shot brief this plan expands. `BUILD_PLAN.md` (or `MILESTONES.md`) is the same sequence broken into steps and checks; where the two disagree, the plan wins. Build me a local transcription pipeline that produces timed text you can correct and export to replace Sonix. Build it in phases, in the order below. Do not write the whole thing in one pass. Finish a phase, run its "Done when" check, fix what fails, and only then start the next phase. ### Rule zero Two rules, and both are about not claiming more than you have. Recording other people is not a technical decision. In several places all parties must agree before a conversation is recorded, and the rules differ by country and sometimes by state. So this build transcribes files you already hold, it never joins a meeting or captures a call, and every imported file carries a note of how consent was obtained. If that note is empty the file still imports, but the field is there so the omission is visible. Second, never display a number you did not measure. Recognisers emit token probabilities, not word accuracy, and presenting one as a confidence percentage tells the reader something untrue about a transcript they are about to act on. Show the model name, the settings and the audio quality instead, and if you want an accuracy figure, measure it against a transcript you typed yourself and report that sample size. ### Stack (fixed, do not substitute) - Runtime: Python 3.12 with FastAPI and a single background worker · transcription is a queue, not a request - Recogniser: whisper.cpp invoked as a process, model path in configuration · a local model keeps recordings of other people off other people's servers - Audio: FFmpeg to one canonical form, sixteen kilohertz mono signed sixteen bit · the command line tool accepts sixteen bit WAV and nothing else - Storage: SQLite for segments and jobs, media beside it on disk · segments are rows because every one of them gets edited - Editing: Corrections stored as rows against segments, never overwriting the original output · you will want to know what the machine actually said ### Data model (create this before Phase 1) - media(id, original_path, sha256, mime, duration_ms, imported_at, consent_note) - audio(media_id, canonical_path, sample_rate, channels, encoding) · the only file the recogniser sees - jobs(id, media_id, state, model_name, params_json, attempt, started_at, finished_at, error) - chunks(id, media_id, ordinal, start_ms, end_ms, silence_split) · where the file was cut and why - segments(id, media_id, ordinal, start_ms, end_ms, text, speaker_turn, chunk_id) - edits(id, segment_id, previous_text, new_text, at) · corrections are additive, the machine output stays - exports(id, media_id, format, path, at) · every timestamp in every export derives from segments, never from a chunk offset ### Phase 1 · One canonical audio file, produced before anything else runs Build: Copy the import beside the database, hash it, and convert with FFmpeg to sixteen kilohertz mono signed sixteen bit PCM: the whisper.cpp command line tool documents that it currently runs only with sixteen bit WAV files and gives that exact conversion. Record the duration and refuse a file FFmpeg cannot probe rather than queueing a job that will fail an hour later. Trap: Handing the recogniser the original file because it played fine in the browser gets you either an immediate failure or, worse, a run that produces plausible nonsense from a misread sample rate, and on a two hour recording you find out after two hours of processing. Convert first, assert the sample rate and channel count of the converted file, and treat probe failure as a rejected import. Done when: Every queued job points at a converted file asserted to be sixteen kilohertz, one channel and sixteen bit, tested for inputs in at least three container formats. A file FFmpeg cannot probe is rejected at import with the reason shown. The original import is retained and its hash matches after conversion. Importing the same file twice is detected by hash and offers the existing transcript rather than reprocessing. Do not build yet: No transcription, speakers, editing or export. ### Phase 2 · Long recordings, honest timestamps Build: Whisper models process audio in a sliding thirty second window, so long files are split. Split on detected silence near a target length rather than on a fixed boundary, carry a small overlap, and convert every timestamp back to an offset in the whole recording before it is stored. Detect long stretches of silence or music and mark them rather than transcribing them. Trap: Cutting a recording every thirty seconds on a fixed boundary slices words in half and, worse, feeding the model a passage of silence or music makes it emit fluent invented sentences, which are indistinguishable from real ones in the output because they carry timestamps too. Cut on silence, skip silent stretches explicitly, and mark anything the detector called non speech. Done when: Timestamps in the last segment of a ninety minute file are correct to within the stated tolerance against a known fixture, tested. Chunk boundaries fall on detected silence and the recorded boundary and reason are stored per chunk. A file containing two minutes of silence produces no transcript text for that stretch, tested with a generated fixture. Restarting the worker mid file resumes from the last completed chunk rather than from the beginning. Do not build yet: No speaker handling, editing or export yet. ### Phase 3 · Speakers, described as what they actually are Build: Use whisper.cpp's tinydiarize option where the matching model is installed: it is documented as experimental and it marks a speaker turn in the text rather than identifying anybody. Show turns as breaks, let the user name a run of turns once and apply that name forwards, and label the whole feature as turn detection rather than speaker identification in the interface. Trap: Presenting turn markers as 'Speaker 1' and 'Speaker 2' implies the tool tracked two voices across the recording, when it only noticed that the voice changed, so two people swapping back and forth get relabelled every few minutes and a reader trusts an attribution that was never computed. Show turns, let a person attach the names, and record who attached them. Done when: Turn markers appear only where the recogniser emitted one and no name is generated by the system, verified by review. Naming a turn applies to the run the user selected and is recorded as a human attribution with a timestamp. With no diarisation-capable model installed the feature is absent rather than degraded into guessing. The interface never uses the phrase speaker identification for this. Do not build yet: No editing surface or export yet. ### Phase 4 · Correcting a transcript without destroying its timing Build: Edit at segment level with the audio playable from the segment start. Store corrections as rows so the machine output remains readable. Support merging and splitting segments with the times recomputed from the audio positions rather than from the surrounding text. Search across segments and jump to the time. Trap: Letting the transcript be edited as one long document loses the mapping between text and time the moment someone reflows a paragraph, and the subtitles exported afterwards drift further out of sync the longer the recording runs. Keep the segment as the unit of editing, and derive the document view from segments rather than the other way round. Done when: Editing any segment leaves its start and end times unchanged, tested. Splitting a segment produces two segments whose times are taken from the chosen audio position and whose union equals the original range. The original machine output for every edited segment is still retrievable. Search returns segments and playing from a result starts at that segment's time, within the stated tolerance. Do not build yet: No exports or retention handling yet. ### Phase 5 · Exports that load, and recordings that do not linger Build: Export SubRip, WebVTT and Markdown. The WebVTT specification requires the body to be UTF-8, to begin with the string WEBVTT, and to write cue times with a full stop before the thousandths, while SubRip files conventionally use a comma, so the two formats are not the same file with a different extension. Add retention that deletes audio and transcripts together, and an export of everything for one recording. Trap: Writing SubRip timestamps into a file named with a WebVTT extension, or omitting the WEBVTT line, produces a file that every player accepts silently and displays no captions from, so the failure is discovered by an audience rather than by you. Write each format from its own function and validate an exported file by parsing it back before declaring the export complete. Done when: An exported WebVTT file begins with the required string and uses full stops in cue times, verified by parsing it back in the suite. An exported SubRip file uses commas and sequential cue numbers, verified the same way. Deleting a recording removes the original, the converted audio, the segments and the exports, checked by listing the directory afterwards. Retention runs on a schedule and logs what it removed without logging any transcript text. Do not build yet: Nothing after this. ### Out of scope (and why) - Joining meetings to record them, which is a consent problem before it is a technical one and is the part of the paid product you should think hardest about copying. - Live streaming transcription, which needs a different model configuration and a latency budget this build has no reason to carry. - Translation and subtitle burn-in, each of which is a separate quality problem with its own failure modes. - Cross recording analytics, which only mean anything once capture is automatic, and capture is exactly what is excluded here. ### README must contain - That the tool transcribes files you already hold and never joins or records a call, and why - The exact audio form the recogniser requires and the FFmpeg command that produces it - That long files are split on silence, silent stretches are marked rather than transcribed, and why that matters - That turn detection is not speaker identification, and that every name was attached by a person - That no confidence or accuracy percentage is displayed unless it was measured against a transcript you typed ===== AGENTS.md ===== # Agent instructions · Sonix indie build - Read `README.md` and `BUILD_PLAN.md` before writing code. The stack is fixed: Python 3.12 with FastAPI and a single background worker, whisper.cpp invoked as a process, model path in configuration, FFmpeg to one canonical form, sixteen kilohertz mono signed sixteen bit, SQLite for segments and jobs, media beside it on disk, Corrections stored as rows against segments, never overwriting the original output. Do not substitute. - Work one phase at a time, in order. Do not start a phase until every "Done when" item of the previous one passes. - Prefer the fewest moving parts that satisfy the step. No frameworks, services or dependencies the plan does not name. - Secrets live in `.env`, never in source or logs. Keep `.env.example` current when a variable is introduced. - Do not invent cryptography, security guarantees, APIs or compliance claims. - Add a focused test for every destructive, security-sensitive or data-loss path the plan names. - Run the project checks before declaring a phase complete, and record any deliberate shortcut in the README under "Tradeoffs". ## Known traps - Handing the recogniser the original file because it played fine in the browser gets you either an immediate failure or, worse, a run that produces plausible nonsense from a misread sample rate, and on a two hour recording you find out after two hours of processing. Convert first, assert the sample rate and channel count of the converted file, and treat probe failure as a rejected import. - Cutting a recording every thirty seconds on a fixed boundary slices words in half and, worse, feeding the model a passage of silence or music makes it emit fluent invented sentences, which are indistinguishable from real ones in the output because they carry timestamps too. Cut on silence, skip silent stretches explicitly, and mark anything the detector called non speech. - Presenting turn markers as 'Speaker 1' and 'Speaker 2' implies the tool tracked two voices across the recording, when it only noticed that the voice changed, so two people swapping back and forth get relabelled every few minutes and a reader trusts an attribution that was never computed. Show turns, let a person attach the names, and record who attached them. - Letting the transcript be edited as one long document loses the mapping between text and time the moment someone reflows a paragraph, and the subtitles exported afterwards drift further out of sync the longer the recording runs. Keep the segment as the unit of editing, and derive the document view from segments rather than the other way round. ===== BUILD_PLAN.md ===== # Build plan · Sonix A local transcription pipeline whose hard problems are getting audio into the exact form the recogniser accepts, keeping timestamps honest across a long recording where the model hallucinates through silence, and exporting subtitle formats that players will actually load. Phases are in dependency order. Each ends in a "Done when" list; treat an unticked item as a blocker, not a note. ## Phase 1 · One canonical audio file, produced before anything else runs Every recording becomes the exact format the recogniser accepts, and the original is kept. ### Steps 1. Convert to the documented form, then assert it The tool accepts one shape of audio; produce exactly that shape. Files: `src/audio/convert.py`, `tests/test_convert.py` 2. Probe before queueing An hour of processing is an expensive way to discover a broken file. Files: `src/audio/probe.py` 3. Deduplicate by hash The same recording gets imported twice more often than you would think. Files: `src/ingest.py` ### Done when - [ ] Every job's audio is asserted to be sixteen kilohertz mono sixteen bit - [ ] Unprobeable files are rejected at import - [ ] The original is retained with a matching hash - [ ] A duplicate import is detected by hash ### Watch out - Handing the recogniser the original file because it played fine in the browser gets you either an immediate failure or, worse, a run that produces plausible nonsense from a misread sample rate, and on a two hour recording you find out after two hours of processing. Convert first, assert the sample rate and channel count of the converted file, and treat probe failure as a rejected import. ## Phase 2 · Long recordings, honest timestamps A ninety minute file transcribes with timings that line up at the end as well as the start. ### Steps 1. Split on silence, not on a stopwatch A fixed cut lands in the middle of the most important sentence. Files: `src/transcribe/chunking.py` 2. Convert chunk time to file time once Every export inherits this arithmetic, so get it wrong in one place only. Files: `src/transcribe/timing.py`, `tests/test_timing.py` 3. Mark silence rather than transcribing it Fluent invented sentences are the failure mode nobody notices. Files: `src/transcribe/silence.py` ### Done when - [ ] End of file timestamps are within tolerance - [ ] Chunk boundaries land on detected silence and are recorded - [ ] Silent stretches produce no text - [ ] A worker restart resumes at the last chunk ### Watch out - Cutting a recording every thirty seconds on a fixed boundary slices words in half and, worse, feeding the model a passage of silence or music makes it emit fluent invented sentences, which are indistinguishable from real ones in the output because they carry timestamps too. Cut on silence, skip silent stretches explicitly, and mark anything the detector called non speech. ## Phase 3 · Speakers, described as what they actually are Speaker changes are marked where the tool can mark them, and never invented. ### Steps 1. Use turn detection only where the model supports it And say in the interface that it is experimental, because it is. Files: `src/transcribe/diarise.py` 2. Let a person attach every name And store that a person did it. Files: `src/app/segments.py` 3. Hide the feature when the model is missing A guessed speaker is worse than no speaker. Files: `src/config.py` ### Done when - [ ] No speaker name is generated by the system - [ ] Names are recorded as human attributions - [ ] The feature is absent without a capable model - [ ] The wording says turn detection ### Watch out - Presenting turn markers as 'Speaker 1' and 'Speaker 2' implies the tool tracked two voices across the recording, when it only noticed that the voice changed, so two people swapping back and forth get relabelled every few minutes and a reader trusts an attribution that was never computed. Show turns, let a person attach the names, and record who attached them. ## Phase 4 · Correcting a transcript without destroying its timing Editing text keeps every segment anchored to the audio. ### Steps 1. Make the segment the unit of editing The document view is a rendering of segments, never the source. Files: `src/app/editor.py` 2. Keep corrections additive You will want to compare what the machine said with what it meant. Files: `src/db/edits.py` 3. Recompute times from audio on split Not from character counts, which have nothing to do with speech. Files: `src/transcribe/split.py`, `tests/test_split.py` ### Done when - [ ] Editing text leaves segment times unchanged - [ ] A split produces times covering the original range - [ ] Machine output remains retrievable after edits - [ ] Search results play from the right position ### Watch out - Letting the transcript be edited as one long document loses the mapping between text and time the moment someone reflows a paragraph, and the subtitles exported afterwards drift further out of sync the longer the recording runs. Keep the segment as the unit of editing, and derive the document view from segments rather than the other way round. ## Not in this build - Meeting bots and call capture. - Live streaming transcription. - Translation and subtitle burn-in. - Cross recording analytics. ## After v1, if you want it - A measured accuracy report against transcripts you typed, with the sample size stated - Word level timing for a karaoke style review view, which the recogniser can already emit - A watched folder that imports and queues automatically, once retention has run for a month ===== .env.example ===== # Copy to .env and fill in. Never commit .env; this file documents it. # Required. Jobs, chunks, segments and edits. DATABASE_PATH=./data/transcripts.db # Required. Originals and converted audio, deleted together by retention. MEDIA_DIR=./data/media # Required. The recogniser binary invoked by the worker. WHISPER_BIN=/usr/local/bin/whisper-cli # Required. Model file. Its name is recorded on every job. WHISPER_MODEL=./models/ggml-base.en.bin # Required. Target chunk length. The actual cut lands on nearby silence. CHUNK_TARGET_SECONDS=240 # Required. Audio and transcripts are removed together after this. RETENTION_DAYS=180 # Required. What the long file timing test enforces at the end of a recording. TIMING_TOLERANCE_MS=250
You are building a production product version of Sonix. Create the following project files first, then implement the application by following them. Keep the files updated as decisions change. Do not collapse this into a single README or prompt. ===== PRODUCT.md ===== # Sonix · product brief ## Problem The visible automated transcription loop is buildable, but a credible replacement needs more than the first screen. Sonix earns its keep through capture, integrations, reliability, so expect a weekend or multi-day build and a narrower personal scope. ## Product outcome A pipeline that turns a file you already have into timed, correctable text, keeps the timestamps right ninety minutes in, refuses to invent speech during silence or names during a turn change, and exports subtitle files that actually display. ## Target user A builder who needs a maintainable product foundation, not a one-off demo. ## Required capabilities - Python 3.12 - FFmpeg - local Whisper model or optional API key in .env ## Explicit non-goals for v1 - Meeting bots and call capture. - Live streaming transcription. - Translation and subtitle burn-in. - Cross recording analytics. - live multi-speaker accuracy - calendar and CRM integrations - cross-call team analytics - meeting-bot auto-join ## Success criteria - Every job's audio is confirmed to be sixteen kilohertz mono sixteen bit - End of file timestamps are confirmed within tolerance on a long fixture - Silence is confirmed to produce no transcript text - No speaker name is confirmed to be generated by the system - Both subtitle exports are confirmed to parse back correctly ===== BRIEF.md ===== # Build brief · Sonix The one-shot brief this plan expands. `BUILD_PLAN.md` (or `MILESTONES.md`) is the same sequence broken into steps and checks; where the two disagree, the plan wins. Build me a local transcription pipeline that produces timed text you can correct and export to replace Sonix. Build it in phases, in the order below. Do not write the whole thing in one pass. Finish a phase, run its "Done when" check, fix what fails, and only then start the next phase. ### Rule zero Two rules, and both are about not claiming more than you have. Recording other people is not a technical decision. In several places all parties must agree before a conversation is recorded, and the rules differ by country and sometimes by state. So this build transcribes files you already hold, it never joins a meeting or captures a call, and every imported file carries a note of how consent was obtained. If that note is empty the file still imports, but the field is there so the omission is visible. Second, never display a number you did not measure. Recognisers emit token probabilities, not word accuracy, and presenting one as a confidence percentage tells the reader something untrue about a transcript they are about to act on. Show the model name, the settings and the audio quality instead, and if you want an accuracy figure, measure it against a transcript you typed yourself and report that sample size. ### Stack (fixed, do not substitute) - Runtime: Python 3.12 with FastAPI and a single background worker · transcription is a queue, not a request - Recogniser: whisper.cpp invoked as a process, model path in configuration · a local model keeps recordings of other people off other people's servers - Audio: FFmpeg to one canonical form, sixteen kilohertz mono signed sixteen bit · the command line tool accepts sixteen bit WAV and nothing else - Storage: SQLite for segments and jobs, media beside it on disk · segments are rows because every one of them gets edited - Editing: Corrections stored as rows against segments, never overwriting the original output · you will want to know what the machine actually said ### Data model (create this before Phase 1) - media(id, original_path, sha256, mime, duration_ms, imported_at, consent_note) - audio(media_id, canonical_path, sample_rate, channels, encoding) · the only file the recogniser sees - jobs(id, media_id, state, model_name, params_json, attempt, started_at, finished_at, error) - chunks(id, media_id, ordinal, start_ms, end_ms, silence_split) · where the file was cut and why - segments(id, media_id, ordinal, start_ms, end_ms, text, speaker_turn, chunk_id) - edits(id, segment_id, previous_text, new_text, at) · corrections are additive, the machine output stays - exports(id, media_id, format, path, at) · every timestamp in every export derives from segments, never from a chunk offset ### Phase 1 · One canonical audio file, produced before anything else runs Build: Copy the import beside the database, hash it, and convert with FFmpeg to sixteen kilohertz mono signed sixteen bit PCM: the whisper.cpp command line tool documents that it currently runs only with sixteen bit WAV files and gives that exact conversion. Record the duration and refuse a file FFmpeg cannot probe rather than queueing a job that will fail an hour later. Trap: Handing the recogniser the original file because it played fine in the browser gets you either an immediate failure or, worse, a run that produces plausible nonsense from a misread sample rate, and on a two hour recording you find out after two hours of processing. Convert first, assert the sample rate and channel count of the converted file, and treat probe failure as a rejected import. Done when: Every queued job points at a converted file asserted to be sixteen kilohertz, one channel and sixteen bit, tested for inputs in at least three container formats. A file FFmpeg cannot probe is rejected at import with the reason shown. The original import is retained and its hash matches after conversion. Importing the same file twice is detected by hash and offers the existing transcript rather than reprocessing. Do not build yet: No transcription, speakers, editing or export. ### Phase 2 · Long recordings, honest timestamps Build: Whisper models process audio in a sliding thirty second window, so long files are split. Split on detected silence near a target length rather than on a fixed boundary, carry a small overlap, and convert every timestamp back to an offset in the whole recording before it is stored. Detect long stretches of silence or music and mark them rather than transcribing them. Trap: Cutting a recording every thirty seconds on a fixed boundary slices words in half and, worse, feeding the model a passage of silence or music makes it emit fluent invented sentences, which are indistinguishable from real ones in the output because they carry timestamps too. Cut on silence, skip silent stretches explicitly, and mark anything the detector called non speech. Done when: Timestamps in the last segment of a ninety minute file are correct to within the stated tolerance against a known fixture, tested. Chunk boundaries fall on detected silence and the recorded boundary and reason are stored per chunk. A file containing two minutes of silence produces no transcript text for that stretch, tested with a generated fixture. Restarting the worker mid file resumes from the last completed chunk rather than from the beginning. Do not build yet: No speaker handling, editing or export yet. ### Phase 3 · Speakers, described as what they actually are Build: Use whisper.cpp's tinydiarize option where the matching model is installed: it is documented as experimental and it marks a speaker turn in the text rather than identifying anybody. Show turns as breaks, let the user name a run of turns once and apply that name forwards, and label the whole feature as turn detection rather than speaker identification in the interface. Trap: Presenting turn markers as 'Speaker 1' and 'Speaker 2' implies the tool tracked two voices across the recording, when it only noticed that the voice changed, so two people swapping back and forth get relabelled every few minutes and a reader trusts an attribution that was never computed. Show turns, let a person attach the names, and record who attached them. Done when: Turn markers appear only where the recogniser emitted one and no name is generated by the system, verified by review. Naming a turn applies to the run the user selected and is recorded as a human attribution with a timestamp. With no diarisation-capable model installed the feature is absent rather than degraded into guessing. The interface never uses the phrase speaker identification for this. Do not build yet: No editing surface or export yet. ### Phase 4 · Correcting a transcript without destroying its timing Build: Edit at segment level with the audio playable from the segment start. Store corrections as rows so the machine output remains readable. Support merging and splitting segments with the times recomputed from the audio positions rather than from the surrounding text. Search across segments and jump to the time. Trap: Letting the transcript be edited as one long document loses the mapping between text and time the moment someone reflows a paragraph, and the subtitles exported afterwards drift further out of sync the longer the recording runs. Keep the segment as the unit of editing, and derive the document view from segments rather than the other way round. Done when: Editing any segment leaves its start and end times unchanged, tested. Splitting a segment produces two segments whose times are taken from the chosen audio position and whose union equals the original range. The original machine output for every edited segment is still retrievable. Search returns segments and playing from a result starts at that segment's time, within the stated tolerance. Do not build yet: No exports or retention handling yet. ### Phase 5 · Exports that load, and recordings that do not linger Build: Export SubRip, WebVTT and Markdown. The WebVTT specification requires the body to be UTF-8, to begin with the string WEBVTT, and to write cue times with a full stop before the thousandths, while SubRip files conventionally use a comma, so the two formats are not the same file with a different extension. Add retention that deletes audio and transcripts together, and an export of everything for one recording. Trap: Writing SubRip timestamps into a file named with a WebVTT extension, or omitting the WEBVTT line, produces a file that every player accepts silently and displays no captions from, so the failure is discovered by an audience rather than by you. Write each format from its own function and validate an exported file by parsing it back before declaring the export complete. Done when: An exported WebVTT file begins with the required string and uses full stops in cue times, verified by parsing it back in the suite. An exported SubRip file uses commas and sequential cue numbers, verified the same way. Deleting a recording removes the original, the converted audio, the segments and the exports, checked by listing the directory afterwards. Retention runs on a schedule and logs what it removed without logging any transcript text. Do not build yet: Nothing after this. ### Out of scope (and why) - Joining meetings to record them, which is a consent problem before it is a technical one and is the part of the paid product you should think hardest about copying. - Live streaming transcription, which needs a different model configuration and a latency budget this build has no reason to carry. - Translation and subtitle burn-in, each of which is a separate quality problem with its own failure modes. - Cross recording analytics, which only mean anything once capture is automatic, and capture is exactly what is excluded here. ### README must contain - That the tool transcribes files you already hold and never joins or records a call, and why - The exact audio form the recogniser requires and the FFmpeg command that produces it - That long files are split on silence, silent stretches are marked rather than transcribed, and why that matters - That turn detection is not speaker identification, and that every name was attached by a person - That no confidence or accuracy percentage is displayed unless it was measured against a transcript you typed ===== ARCHITECTURE.md ===== # Architecture · Sonix ## Stack | Part | Choice | Why | | --- | --- | --- | | Runtime | Python 3.12 with FastAPI and a single background worker | transcription is a queue, not a request | | Recogniser | whisper.cpp invoked as a process, model path in configuration | a local model keeps recordings of other people off other people's servers | | Audio | FFmpeg to one canonical form, sixteen kilohertz mono signed sixteen bit | the command line tool accepts sixteen bit WAV and nothing else | | Storage | SQLite for segments and jobs, media beside it on disk | segments are rows because every one of them gets edited | | Editing | Corrections stored as rows against segments, never overwriting the original output | you will want to know what the machine actually said | ## Modules Each module has one owner concern and a documented way to replace it. | Module | Owns | How to replace it | | --- | --- | --- | | Ingest | Hashing, probing and conversion to the canonical form. | Any converter that produces the documented audio shape. | | Chunking | Silence based splitting and offset arithmetic. | Any splitter, provided times are converted to whole file offsets once. | | Recogniser | The process call and its recorded parameters. | Any engine, provided the model name and settings are stored per job. | | Editing | Segment level corrections kept additively. | Any interface, provided segments stay the unit of truth. | | Export | One function per format, each validated by reparsing. | Any format added the same way. | ## Configuration Every runtime setting is an environment variable documented in `.env.example`, validated at startup, with a safe local default wherever one exists. - `DATABASE_PATH` · required · Jobs, chunks, segments and edits. - `MEDIA_DIR` · required · Originals and converted audio, deleted together by retention. - `WHISPER_BIN` · required · The recogniser binary invoked by the worker. - `WHISPER_MODEL` · required · Model file. Its name is recorded on every job. - `CHUNK_TARGET_SECONDS` · required · Target chunk length. The actual cut lands on nearby silence. - `RETENTION_DAYS` · required · Audio and transcripts are removed together after this. - `TIMING_TOLERANCE_MS` · required · What the long file timing test enforces at the end of a recording. ## Production baseline - Security: least privilege, input validation at every boundary, secret redaction in logs, rate limits on abuse-prone paths, no invented security primitives. - Data: explicit schema and migrations, transactional writes where integrity matters, backup and restore procedures that have been exercised. - Integrations: adapters around third-party providers, idempotent webhook or job processing, bounded retries, timeouts. - Observability: structured logs with request or operation ids, an error-tracking hook, and health and readiness checks where a server exists. - Quality: unit tests for domain rules, integration tests at module boundaries, one end-to-end test of the critical path. ## Decision records For each dependency in the stack table, keep a short note: why it was chosen, its failure mode, and how it is replaced. Do not add infrastructure until a requirement in `PRODUCT.md` justifies it. ===== AGENTS.md ===== # Agent instructions · Sonix product build - Read `PRODUCT.md` and `ARCHITECTURE.md` before changing code. The stack is fixed: Python 3.12 with FastAPI and a single background worker, whisper.cpp invoked as a process, model path in configuration, FFmpeg to one canonical form, sixteen kilohertz mono signed sixteen bit, SQLite for segments and jobs, media beside it on disk, Corrections stored as rows against segments, never overwriting the original output. - Implement milestone by milestone from `MILESTONES.md`; keep each change reviewable and leave the application runnable at every commit. - Treat authentication, payments, encryption, imports, webhooks and destructive actions as high-risk boundaries when present. - Never invent cryptography or silently weaken a requirement to make a check pass. - Put every external service behind an interface with a deterministic fake for tests. - Add migrations and rollback or recovery notes for every persistent data change. - Log useful operational context without credentials, tokens, passwords or personal data. - Update documentation and run every check before completing a milestone. ## Known traps - Handing the recogniser the original file because it played fine in the browser gets you either an immediate failure or, worse, a run that produces plausible nonsense from a misread sample rate, and on a two hour recording you find out after two hours of processing. Convert first, assert the sample rate and channel count of the converted file, and treat probe failure as a rejected import. - Cutting a recording every thirty seconds on a fixed boundary slices words in half and, worse, feeding the model a passage of silence or music makes it emit fluent invented sentences, which are indistinguishable from real ones in the output because they carry timestamps too. Cut on silence, skip silent stretches explicitly, and mark anything the detector called non speech. - Presenting turn markers as 'Speaker 1' and 'Speaker 2' implies the tool tracked two voices across the recording, when it only noticed that the voice changed, so two people swapping back and forth get relabelled every few minutes and a reader trusts an attribution that was never computed. Show turns, let a person attach the names, and record who attached them. - Letting the transcript be edited as one long document loses the mapping between text and time the moment someone reflows a paragraph, and the subtitles exported afterwards drift further out of sync the longer the recording runs. Keep the segment as the unit of editing, and derive the document view from segments rather than the other way round. - Writing SubRip timestamps into a file named with a WebVTT extension, or omitting the WEBVTT line, produces a file that every player accepts silently and displays no captions from, so the failure is discovered by an audience rather than by you. Write each format from its own function and validate an exported file by parsing it back before declaring the export complete. ===== MILESTONES.md ===== # Delivery milestones · Sonix Estimated effort: **weekend** for the indie phases; the production-only milestones add the trust and operability layer. ## M1 · One canonical audio file, produced before anything else runs Every recording becomes the exact format the recogniser accepts, and the original is kept. ### Steps 1. Convert to the documented form, then assert it The tool accepts one shape of audio; produce exactly that shape. Files: `src/audio/convert.py`, `tests/test_convert.py` 2. Probe before queueing An hour of processing is an expensive way to discover a broken file. Files: `src/audio/probe.py` 3. Deduplicate by hash The same recording gets imported twice more often than you would think. Files: `src/ingest.py` ### Done when - [ ] Every job's audio is asserted to be sixteen kilohertz mono sixteen bit - [ ] Unprobeable files are rejected at import - [ ] The original is retained with a matching hash - [ ] A duplicate import is detected by hash ### Watch out - Handing the recogniser the original file because it played fine in the browser gets you either an immediate failure or, worse, a run that produces plausible nonsense from a misread sample rate, and on a two hour recording you find out after two hours of processing. Convert first, assert the sample rate and channel count of the converted file, and treat probe failure as a rejected import. ## M2 · Long recordings, honest timestamps A ninety minute file transcribes with timings that line up at the end as well as the start. ### Steps 1. Split on silence, not on a stopwatch A fixed cut lands in the middle of the most important sentence. Files: `src/transcribe/chunking.py` 2. Convert chunk time to file time once Every export inherits this arithmetic, so get it wrong in one place only. Files: `src/transcribe/timing.py`, `tests/test_timing.py` 3. Mark silence rather than transcribing it Fluent invented sentences are the failure mode nobody notices. Files: `src/transcribe/silence.py` ### Done when - [ ] End of file timestamps are within tolerance - [ ] Chunk boundaries land on detected silence and are recorded - [ ] Silent stretches produce no text - [ ] A worker restart resumes at the last chunk ### Watch out - Cutting a recording every thirty seconds on a fixed boundary slices words in half and, worse, feeding the model a passage of silence or music makes it emit fluent invented sentences, which are indistinguishable from real ones in the output because they carry timestamps too. Cut on silence, skip silent stretches explicitly, and mark anything the detector called non speech. ## M3 · Speakers, described as what they actually are Speaker changes are marked where the tool can mark them, and never invented. ### Steps 1. Use turn detection only where the model supports it And say in the interface that it is experimental, because it is. Files: `src/transcribe/diarise.py` 2. Let a person attach every name And store that a person did it. Files: `src/app/segments.py` 3. Hide the feature when the model is missing A guessed speaker is worse than no speaker. Files: `src/config.py` ### Done when - [ ] No speaker name is generated by the system - [ ] Names are recorded as human attributions - [ ] The feature is absent without a capable model - [ ] The wording says turn detection ### Watch out - Presenting turn markers as 'Speaker 1' and 'Speaker 2' implies the tool tracked two voices across the recording, when it only noticed that the voice changed, so two people swapping back and forth get relabelled every few minutes and a reader trusts an attribution that was never computed. Show turns, let a person attach the names, and record who attached them. ## M4 · Correcting a transcript without destroying its timing Editing text keeps every segment anchored to the audio. ### Steps 1. Make the segment the unit of editing The document view is a rendering of segments, never the source. Files: `src/app/editor.py` 2. Keep corrections additive You will want to compare what the machine said with what it meant. Files: `src/db/edits.py` 3. Recompute times from audio on split Not from character counts, which have nothing to do with speech. Files: `src/transcribe/split.py`, `tests/test_split.py` ### Done when - [ ] Editing text leaves segment times unchanged - [ ] A split produces times covering the original range - [ ] Machine output remains retrievable after edits - [ ] Search results play from the right position ### Watch out - Letting the transcript be edited as one long document loses the mapping between text and time the moment someone reflows a paragraph, and the subtitles exported afterwards drift further out of sync the longer the recording runs. Keep the segment as the unit of editing, and derive the document view from segments rather than the other way round. ## M5 · Exports that load, and recordings that do not linger (production only) Subtitle files open in a player, and a deletion removes the audio as well as the text. ### Steps 1. Write each subtitle format from its own function They differ in the header and in one punctuation mark, which is enough. Files: `src/export/vtt.py`, `src/export/srt.py` 2. Parse every export back before calling it done The cheapest possible test for a format you do not control. Files: `tests/test_export_roundtrip.py` 3. Delete audio and text together A deleted transcript with the recording still on disk has deleted nothing. Files: `src/retention.py` ### Done when - [ ] WebVTT exports parse and carry the required header - [ ] SubRip exports parse with comma times and cue numbers - [ ] Deletion removes audio, segments and exports - [ ] Retention logs removals without transcript text ### Watch out - Writing SubRip timestamps into a file named with a WebVTT extension, or omitting the WEBVTT line, produces a file that every player accepts silently and displays no captions from, so the failure is discovered by an audience rather than by you. Write each format from its own function and validate an exported file by parsing it back before declaring the export complete. ===== OPERATIONS.md ===== # Operations · Sonix ## Backup The database and the media directory together. Transcripts without their audio cannot be corrected, and audio without transcripts means reprocessing hours of recordings. ## Restore Restore and confirm that a job interrupted before the backup resumes at its last completed chunk rather than restarting, and that segment times still match the audio. Do a restore drill before the first real user, and write the date here when it passes. ## Monitoring Queue depth, jobs failed on their last attempt, and processing time per audio minute, which is the number that tells you whether the chosen model size still fits the machine. ## Incident checklist A job stuck in processing is the common one, usually a killed worker. The chunk table makes resumption safe, so the response is to reset the job to queued and let it continue rather than to delete and reimport. 1. Contain the issue without destroying evidence or user data. 2. Record the timeline and affected scope. 3. Rotate exposed secrets and revoke compromised sessions or credentials. 4. Restore from a verified backup when needed. 5. Document the root cause, the remediation and the regression test. ## Release gate - [ ] Every job's audio is confirmed to be sixteen kilohertz mono sixteen bit - [ ] End of file timestamps are confirmed within tolerance on a long fixture - [ ] Silence is confirmed to produce no transcript text - [ ] No speaker name is confirmed to be generated by the system - [ ] Both subtitle exports are confirmed to parse back correctly ## Launch constraint Do not market omitted Sonix capabilities as implemented. The non-goals in `PRODUCT.md` remain user-visible limitations until they are deliberately delivered. ===== .env.example ===== # Copy to .env and fill in. Never commit .env; this file documents it. # Required. Jobs, chunks, segments and edits. DATABASE_PATH=./data/transcripts.db # Required. Originals and converted audio, deleted together by retention. MEDIA_DIR=./data/media # Required. The recogniser binary invoked by the worker. WHISPER_BIN=/usr/local/bin/whisper-cli # Required. Model file. Its name is recorded on every job. WHISPER_MODEL=./models/ggml-base.en.bin # Required. Target chunk length. The actual cut lands on nearby silence. CHUNK_TARGET_SECONDS=240 # Required. Audio and transcripts are removed together after this. RETENTION_DAYS=180 # Required. What the long file timing test enforces at the end of a recording. TIMING_TOLERANCE_MS=250
README.md
# Sonix · indie build A local transcription pipeline whose hard problems are getting audio into the exact form the recogniser accepts, keeping timestamps honest across a long recording where the model hallucinates through silence, and exporting subtitle formats that players will actually load. Estimated effort: **weekend**. Work `BUILD_PLAN.md` top to bottom · every phase ends in a check that has to pass before the next one starts. ## Stack | Part | Choice | Why | | --- | --- | --- | | Runtime | Python 3.12 with FastAPI and a single background worker | transcription is a queue, not a request | | Recogniser | whisper.cpp invoked as a process, model path in configuration | a local model keeps recordings of other people off other people's servers | | Audio | FFmpeg to one canonical form, sixteen kilohertz mono signed sixteen bit | the command line tool accepts sixteen bit WAV and nothing else | | Storage | SQLite for segments and jobs, media beside it on disk | segments are rows because every one of them gets edited | | Editing | Corrections stored as rows against segments, never overwriting the original output | you will want to know what the machine actually said | ## Before you start Have every one of these ready. The plan assumes them from step one. - [ ] **Python 3.12 or newer** · free - Why: Runs the application and the worker. - Get it: python.org, or your package manager. - Verify: python3 --version prints 3.12 or higher - [ ] **FFmpeg** · free - Why: Produces the one audio form the recogniser accepts, and probes files before they are queued. - Get it: Your package manager, or a static build from the project site. - Verify: ffmpeg -i sample.m4a -ar 16000 -ac 1 -c:a pcm_s16le out.wav produces a playable file - [ ] **whisper.cpp with a downloaded model** · free - Why: The recogniser. Model size decides both quality and how long a ninety minute file takes on your machine. - Get it: Build the project and run its model download script for the size you want. - Verify: The command line tool transcribes the bundled sample - [ ] **A long recording and a silence fixture** · free - Why: Timestamp drift and hallucination through silence only appear on long real audio, never on a thirty second clip. - Get it: One recording of an hour or more that you have the right to use, plus a generated file with two minutes of silence in the middle. - Verify: Both are in the test fixtures and used by the timing tests - [ ] **Your retention period and timestamp tolerance** · free - Why: Both are asserted by tests, and a retention period chosen after a year of recordings have accumulated is not a decision. - Get it: Write down how long audio and transcripts are kept, and the tolerance in milliseconds the timing test enforces. - Verify: Both appear in configuration and in tests ## Quick start Follow Phase 1 in `BUILD_PLAN.md`; it creates the project. Then copy `.env.example` to `.env` and fill in the values it documents. ## Honest limits This build deliberately does not replace: - Meeting bots and call capture. - Live streaming transcription. - Translation and subtitle burn-in. - Cross recording analytics. - live multi-speaker accuracy - calendar and CRM integrations - cross-call team analytics - meeting-bot auto-join If one of those is essential to you, that is the reason to keep paying for Sonix, and the README should say so rather than pretend.
BRIEF.md
# Build brief · Sonix The one-shot brief this plan expands. `BUILD_PLAN.md` (or `MILESTONES.md`) is the same sequence broken into steps and checks; where the two disagree, the plan wins. Build me a local transcription pipeline that produces timed text you can correct and export to replace Sonix. Build it in phases, in the order below. Do not write the whole thing in one pass. Finish a phase, run its "Done when" check, fix what fails, and only then start the next phase. ### Rule zero Two rules, and both are about not claiming more than you have. Recording other people is not a technical decision. In several places all parties must agree before a conversation is recorded, and the rules differ by country and sometimes by state. So this build transcribes files you already hold, it never joins a meeting or captures a call, and every imported file carries a note of how consent was obtained. If that note is empty the file still imports, but the field is there so the omission is visible. Second, never display a number you did not measure. Recognisers emit token probabilities, not word accuracy, and presenting one as a confidence percentage tells the reader something untrue about a transcript they are about to act on. Show the model name, the settings and the audio quality instead, and if you want an accuracy figure, measure it against a transcript you typed yourself and report that sample size. ### Stack (fixed, do not substitute) - Runtime: Python 3.12 with FastAPI and a single background worker · transcription is a queue, not a request - Recogniser: whisper.cpp invoked as a process, model path in configuration · a local model keeps recordings of other people off other people's servers - Audio: FFmpeg to one canonical form, sixteen kilohertz mono signed sixteen bit · the command line tool accepts sixteen bit WAV and nothing else - Storage: SQLite for segments and jobs, media beside it on disk · segments are rows because every one of them gets edited - Editing: Corrections stored as rows against segments, never overwriting the original output · you will want to know what the machine actually said ### Data model (create this before Phase 1) - media(id, original_path, sha256, mime, duration_ms, imported_at, consent_note) - audio(media_id, canonical_path, sample_rate, channels, encoding) · the only file the recogniser sees - jobs(id, media_id, state, model_name, params_json, attempt, started_at, finished_at, error) - chunks(id, media_id, ordinal, start_ms, end_ms, silence_split) · where the file was cut and why - segments(id, media_id, ordinal, start_ms, end_ms, text, speaker_turn, chunk_id) - edits(id, segment_id, previous_text, new_text, at) · corrections are additive, the machine output stays - exports(id, media_id, format, path, at) · every timestamp in every export derives from segments, never from a chunk offset ### Phase 1 · One canonical audio file, produced before anything else runs Build: Copy the import beside the database, hash it, and convert with FFmpeg to sixteen kilohertz mono signed sixteen bit PCM: the whisper.cpp command line tool documents that it currently runs only with sixteen bit WAV files and gives that exact conversion. Record the duration and refuse a file FFmpeg cannot probe rather than queueing a job that will fail an hour later. Trap: Handing the recogniser the original file because it played fine in the browser gets you either an immediate failure or, worse, a run that produces plausible nonsense from a misread sample rate, and on a two hour recording you find out after two hours of processing. Convert first, assert the sample rate and channel count of the converted file, and treat probe failure as a rejected import. Done when: Every queued job points at a converted file asserted to be sixteen kilohertz, one channel and sixteen bit, tested for inputs in at least three container formats. A file FFmpeg cannot probe is rejected at import with the reason shown. The original import is retained and its hash matches after conversion. Importing the same file twice is detected by hash and offers the existing transcript rather than reprocessing. Do not build yet: No transcription, speakers, editing or export. ### Phase 2 · Long recordings, honest timestamps Build: Whisper models process audio in a sliding thirty second window, so long files are split. Split on detected silence near a target length rather than on a fixed boundary, carry a small overlap, and convert every timestamp back to an offset in the whole recording before it is stored. Detect long stretches of silence or music and mark them rather than transcribing them. Trap: Cutting a recording every thirty seconds on a fixed boundary slices words in half and, worse, feeding the model a passage of silence or music makes it emit fluent invented sentences, which are indistinguishable from real ones in the output because they carry timestamps too. Cut on silence, skip silent stretches explicitly, and mark anything the detector called non speech. Done when: Timestamps in the last segment of a ninety minute file are correct to within the stated tolerance against a known fixture, tested. Chunk boundaries fall on detected silence and the recorded boundary and reason are stored per chunk. A file containing two minutes of silence produces no transcript text for that stretch, tested with a generated fixture. Restarting the worker mid file resumes from the last completed chunk rather than from the beginning. Do not build yet: No speaker handling, editing or export yet. ### Phase 3 · Speakers, described as what they actually are Build: Use whisper.cpp's tinydiarize option where the matching model is installed: it is documented as experimental and it marks a speaker turn in the text rather than identifying anybody. Show turns as breaks, let the user name a run of turns once and apply that name forwards, and label the whole feature as turn detection rather than speaker identification in the interface. Trap: Presenting turn markers as 'Speaker 1' and 'Speaker 2' implies the tool tracked two voices across the recording, when it only noticed that the voice changed, so two people swapping back and forth get relabelled every few minutes and a reader trusts an attribution that was never computed. Show turns, let a person attach the names, and record who attached them. Done when: Turn markers appear only where the recogniser emitted one and no name is generated by the system, verified by review. Naming a turn applies to the run the user selected and is recorded as a human attribution with a timestamp. With no diarisation-capable model installed the feature is absent rather than degraded into guessing. The interface never uses the phrase speaker identification for this. Do not build yet: No editing surface or export yet. ### Phase 4 · Correcting a transcript without destroying its timing Build: Edit at segment level with the audio playable from the segment start. Store corrections as rows so the machine output remains readable. Support merging and splitting segments with the times recomputed from the audio positions rather than from the surrounding text. Search across segments and jump to the time. Trap: Letting the transcript be edited as one long document loses the mapping between text and time the moment someone reflows a paragraph, and the subtitles exported afterwards drift further out of sync the longer the recording runs. Keep the segment as the unit of editing, and derive the document view from segments rather than the other way round. Done when: Editing any segment leaves its start and end times unchanged, tested. Splitting a segment produces two segments whose times are taken from the chosen audio position and whose union equals the original range. The original machine output for every edited segment is still retrievable. Search returns segments and playing from a result starts at that segment's time, within the stated tolerance. Do not build yet: No exports or retention handling yet. ### Phase 5 · Exports that load, and recordings that do not linger Build: Export SubRip, WebVTT and Markdown. The WebVTT specification requires the body to be UTF-8, to begin with the string WEBVTT, and to write cue times with a full stop before the thousandths, while SubRip files conventionally use a comma, so the two formats are not the same file with a different extension. Add retention that deletes audio and transcripts together, and an export of everything for one recording. Trap: Writing SubRip timestamps into a file named with a WebVTT extension, or omitting the WEBVTT line, produces a file that every player accepts silently and displays no captions from, so the failure is discovered by an audience rather than by you. Write each format from its own function and validate an exported file by parsing it back before declaring the export complete. Done when: An exported WebVTT file begins with the required string and uses full stops in cue times, verified by parsing it back in the suite. An exported SubRip file uses commas and sequential cue numbers, verified the same way. Deleting a recording removes the original, the converted audio, the segments and the exports, checked by listing the directory afterwards. Retention runs on a schedule and logs what it removed without logging any transcript text. Do not build yet: Nothing after this. ### Out of scope (and why) - Joining meetings to record them, which is a consent problem before it is a technical one and is the part of the paid product you should think hardest about copying. - Live streaming transcription, which needs a different model configuration and a latency budget this build has no reason to carry. - Translation and subtitle burn-in, each of which is a separate quality problem with its own failure modes. - Cross recording analytics, which only mean anything once capture is automatic, and capture is exactly what is excluded here. ### README must contain - That the tool transcribes files you already hold and never joins or records a call, and why - The exact audio form the recogniser requires and the FFmpeg command that produces it - That long files are split on silence, silent stretches are marked rather than transcribed, and why that matters - That turn detection is not speaker identification, and that every name was attached by a person - That no confidence or accuracy percentage is displayed unless it was measured against a transcript you typed
AGENTS.md
# Agent instructions · Sonix indie build - Read `README.md` and `BUILD_PLAN.md` before writing code. The stack is fixed: Python 3.12 with FastAPI and a single background worker, whisper.cpp invoked as a process, model path in configuration, FFmpeg to one canonical form, sixteen kilohertz mono signed sixteen bit, SQLite for segments and jobs, media beside it on disk, Corrections stored as rows against segments, never overwriting the original output. Do not substitute. - Work one phase at a time, in order. Do not start a phase until every "Done when" item of the previous one passes. - Prefer the fewest moving parts that satisfy the step. No frameworks, services or dependencies the plan does not name. - Secrets live in `.env`, never in source or logs. Keep `.env.example` current when a variable is introduced. - Do not invent cryptography, security guarantees, APIs or compliance claims. - Add a focused test for every destructive, security-sensitive or data-loss path the plan names. - Run the project checks before declaring a phase complete, and record any deliberate shortcut in the README under "Tradeoffs". ## Known traps - Handing the recogniser the original file because it played fine in the browser gets you either an immediate failure or, worse, a run that produces plausible nonsense from a misread sample rate, and on a two hour recording you find out after two hours of processing. Convert first, assert the sample rate and channel count of the converted file, and treat probe failure as a rejected import. - Cutting a recording every thirty seconds on a fixed boundary slices words in half and, worse, feeding the model a passage of silence or music makes it emit fluent invented sentences, which are indistinguishable from real ones in the output because they carry timestamps too. Cut on silence, skip silent stretches explicitly, and mark anything the detector called non speech. - Presenting turn markers as 'Speaker 1' and 'Speaker 2' implies the tool tracked two voices across the recording, when it only noticed that the voice changed, so two people swapping back and forth get relabelled every few minutes and a reader trusts an attribution that was never computed. Show turns, let a person attach the names, and record who attached them. - Letting the transcript be edited as one long document loses the mapping between text and time the moment someone reflows a paragraph, and the subtitles exported afterwards drift further out of sync the longer the recording runs. Keep the segment as the unit of editing, and derive the document view from segments rather than the other way round.
BUILD_PLAN.md
# Build plan · Sonix A local transcription pipeline whose hard problems are getting audio into the exact form the recogniser accepts, keeping timestamps honest across a long recording where the model hallucinates through silence, and exporting subtitle formats that players will actually load. Phases are in dependency order. Each ends in a "Done when" list; treat an unticked item as a blocker, not a note. ## Phase 1 · One canonical audio file, produced before anything else runs Every recording becomes the exact format the recogniser accepts, and the original is kept. ### Steps 1. Convert to the documented form, then assert it The tool accepts one shape of audio; produce exactly that shape. Files: `src/audio/convert.py`, `tests/test_convert.py` 2. Probe before queueing An hour of processing is an expensive way to discover a broken file. Files: `src/audio/probe.py` 3. Deduplicate by hash The same recording gets imported twice more often than you would think. Files: `src/ingest.py` ### Done when - [ ] Every job's audio is asserted to be sixteen kilohertz mono sixteen bit - [ ] Unprobeable files are rejected at import - [ ] The original is retained with a matching hash - [ ] A duplicate import is detected by hash ### Watch out - Handing the recogniser the original file because it played fine in the browser gets you either an immediate failure or, worse, a run that produces plausible nonsense from a misread sample rate, and on a two hour recording you find out after two hours of processing. Convert first, assert the sample rate and channel count of the converted file, and treat probe failure as a rejected import. ## Phase 2 · Long recordings, honest timestamps A ninety minute file transcribes with timings that line up at the end as well as the start. ### Steps 1. Split on silence, not on a stopwatch A fixed cut lands in the middle of the most important sentence. Files: `src/transcribe/chunking.py` 2. Convert chunk time to file time once Every export inherits this arithmetic, so get it wrong in one place only. Files: `src/transcribe/timing.py`, `tests/test_timing.py` 3. Mark silence rather than transcribing it Fluent invented sentences are the failure mode nobody notices. Files: `src/transcribe/silence.py` ### Done when - [ ] End of file timestamps are within tolerance - [ ] Chunk boundaries land on detected silence and are recorded - [ ] Silent stretches produce no text - [ ] A worker restart resumes at the last chunk ### Watch out - Cutting a recording every thirty seconds on a fixed boundary slices words in half and, worse, feeding the model a passage of silence or music makes it emit fluent invented sentences, which are indistinguishable from real ones in the output because they carry timestamps too. Cut on silence, skip silent stretches explicitly, and mark anything the detector called non speech. ## Phase 3 · Speakers, described as what they actually are Speaker changes are marked where the tool can mark them, and never invented. ### Steps 1. Use turn detection only where the model supports it And say in the interface that it is experimental, because it is. Files: `src/transcribe/diarise.py` 2. Let a person attach every name And store that a person did it. Files: `src/app/segments.py` 3. Hide the feature when the model is missing A guessed speaker is worse than no speaker. Files: `src/config.py` ### Done when - [ ] No speaker name is generated by the system - [ ] Names are recorded as human attributions - [ ] The feature is absent without a capable model - [ ] The wording says turn detection ### Watch out - Presenting turn markers as 'Speaker 1' and 'Speaker 2' implies the tool tracked two voices across the recording, when it only noticed that the voice changed, so two people swapping back and forth get relabelled every few minutes and a reader trusts an attribution that was never computed. Show turns, let a person attach the names, and record who attached them. ## Phase 4 · Correcting a transcript without destroying its timing Editing text keeps every segment anchored to the audio. ### Steps 1. Make the segment the unit of editing The document view is a rendering of segments, never the source. Files: `src/app/editor.py` 2. Keep corrections additive You will want to compare what the machine said with what it meant. Files: `src/db/edits.py` 3. Recompute times from audio on split Not from character counts, which have nothing to do with speech. Files: `src/transcribe/split.py`, `tests/test_split.py` ### Done when - [ ] Editing text leaves segment times unchanged - [ ] A split produces times covering the original range - [ ] Machine output remains retrievable after edits - [ ] Search results play from the right position ### Watch out - Letting the transcript be edited as one long document loses the mapping between text and time the moment someone reflows a paragraph, and the subtitles exported afterwards drift further out of sync the longer the recording runs. Keep the segment as the unit of editing, and derive the document view from segments rather than the other way round. ## Not in this build - Meeting bots and call capture. - Live streaming transcription. - Translation and subtitle burn-in. - Cross recording analytics. ## After v1, if you want it - A measured accuracy report against transcripts you typed, with the sample size stated - Word level timing for a karaoke style review view, which the recogniser can already emit - A watched folder that imports and queues automatically, once retention has run for a month
.env.example
# Copy to .env and fill in. Never commit .env; this file documents it. # Required. Jobs, chunks, segments and edits. DATABASE_PATH=./data/transcripts.db # Required. Originals and converted audio, deleted together by retention. MEDIA_DIR=./data/media # Required. The recogniser binary invoked by the worker. WHISPER_BIN=/usr/local/bin/whisper-cli # Required. Model file. Its name is recorded on every job. WHISPER_MODEL=./models/ggml-base.en.bin # Required. Target chunk length. The actual cut lands on nearby silence. CHUNK_TARGET_SECONDS=240 # Required. Audio and transcripts are removed together after this. RETENTION_DAYS=180 # Required. What the long file timing test enforces at the end of a recording. TIMING_TOLERANCE_MS=250
PRODUCT.md
# Sonix · product brief ## Problem The visible automated transcription loop is buildable, but a credible replacement needs more than the first screen. Sonix earns its keep through capture, integrations, reliability, so expect a weekend or multi-day build and a narrower personal scope. ## Product outcome A pipeline that turns a file you already have into timed, correctable text, keeps the timestamps right ninety minutes in, refuses to invent speech during silence or names during a turn change, and exports subtitle files that actually display. ## Target user A builder who needs a maintainable product foundation, not a one-off demo. ## Required capabilities - Python 3.12 - FFmpeg - local Whisper model or optional API key in .env ## Explicit non-goals for v1 - Meeting bots and call capture. - Live streaming transcription. - Translation and subtitle burn-in. - Cross recording analytics. - live multi-speaker accuracy - calendar and CRM integrations - cross-call team analytics - meeting-bot auto-join ## Success criteria - Every job's audio is confirmed to be sixteen kilohertz mono sixteen bit - End of file timestamps are confirmed within tolerance on a long fixture - Silence is confirmed to produce no transcript text - No speaker name is confirmed to be generated by the system - Both subtitle exports are confirmed to parse back correctly
BRIEF.md
# Build brief · Sonix The one-shot brief this plan expands. `BUILD_PLAN.md` (or `MILESTONES.md`) is the same sequence broken into steps and checks; where the two disagree, the plan wins. Build me a local transcription pipeline that produces timed text you can correct and export to replace Sonix. Build it in phases, in the order below. Do not write the whole thing in one pass. Finish a phase, run its "Done when" check, fix what fails, and only then start the next phase. ### Rule zero Two rules, and both are about not claiming more than you have. Recording other people is not a technical decision. In several places all parties must agree before a conversation is recorded, and the rules differ by country and sometimes by state. So this build transcribes files you already hold, it never joins a meeting or captures a call, and every imported file carries a note of how consent was obtained. If that note is empty the file still imports, but the field is there so the omission is visible. Second, never display a number you did not measure. Recognisers emit token probabilities, not word accuracy, and presenting one as a confidence percentage tells the reader something untrue about a transcript they are about to act on. Show the model name, the settings and the audio quality instead, and if you want an accuracy figure, measure it against a transcript you typed yourself and report that sample size. ### Stack (fixed, do not substitute) - Runtime: Python 3.12 with FastAPI and a single background worker · transcription is a queue, not a request - Recogniser: whisper.cpp invoked as a process, model path in configuration · a local model keeps recordings of other people off other people's servers - Audio: FFmpeg to one canonical form, sixteen kilohertz mono signed sixteen bit · the command line tool accepts sixteen bit WAV and nothing else - Storage: SQLite for segments and jobs, media beside it on disk · segments are rows because every one of them gets edited - Editing: Corrections stored as rows against segments, never overwriting the original output · you will want to know what the machine actually said ### Data model (create this before Phase 1) - media(id, original_path, sha256, mime, duration_ms, imported_at, consent_note) - audio(media_id, canonical_path, sample_rate, channels, encoding) · the only file the recogniser sees - jobs(id, media_id, state, model_name, params_json, attempt, started_at, finished_at, error) - chunks(id, media_id, ordinal, start_ms, end_ms, silence_split) · where the file was cut and why - segments(id, media_id, ordinal, start_ms, end_ms, text, speaker_turn, chunk_id) - edits(id, segment_id, previous_text, new_text, at) · corrections are additive, the machine output stays - exports(id, media_id, format, path, at) · every timestamp in every export derives from segments, never from a chunk offset ### Phase 1 · One canonical audio file, produced before anything else runs Build: Copy the import beside the database, hash it, and convert with FFmpeg to sixteen kilohertz mono signed sixteen bit PCM: the whisper.cpp command line tool documents that it currently runs only with sixteen bit WAV files and gives that exact conversion. Record the duration and refuse a file FFmpeg cannot probe rather than queueing a job that will fail an hour later. Trap: Handing the recogniser the original file because it played fine in the browser gets you either an immediate failure or, worse, a run that produces plausible nonsense from a misread sample rate, and on a two hour recording you find out after two hours of processing. Convert first, assert the sample rate and channel count of the converted file, and treat probe failure as a rejected import. Done when: Every queued job points at a converted file asserted to be sixteen kilohertz, one channel and sixteen bit, tested for inputs in at least three container formats. A file FFmpeg cannot probe is rejected at import with the reason shown. The original import is retained and its hash matches after conversion. Importing the same file twice is detected by hash and offers the existing transcript rather than reprocessing. Do not build yet: No transcription, speakers, editing or export. ### Phase 2 · Long recordings, honest timestamps Build: Whisper models process audio in a sliding thirty second window, so long files are split. Split on detected silence near a target length rather than on a fixed boundary, carry a small overlap, and convert every timestamp back to an offset in the whole recording before it is stored. Detect long stretches of silence or music and mark them rather than transcribing them. Trap: Cutting a recording every thirty seconds on a fixed boundary slices words in half and, worse, feeding the model a passage of silence or music makes it emit fluent invented sentences, which are indistinguishable from real ones in the output because they carry timestamps too. Cut on silence, skip silent stretches explicitly, and mark anything the detector called non speech. Done when: Timestamps in the last segment of a ninety minute file are correct to within the stated tolerance against a known fixture, tested. Chunk boundaries fall on detected silence and the recorded boundary and reason are stored per chunk. A file containing two minutes of silence produces no transcript text for that stretch, tested with a generated fixture. Restarting the worker mid file resumes from the last completed chunk rather than from the beginning. Do not build yet: No speaker handling, editing or export yet. ### Phase 3 · Speakers, described as what they actually are Build: Use whisper.cpp's tinydiarize option where the matching model is installed: it is documented as experimental and it marks a speaker turn in the text rather than identifying anybody. Show turns as breaks, let the user name a run of turns once and apply that name forwards, and label the whole feature as turn detection rather than speaker identification in the interface. Trap: Presenting turn markers as 'Speaker 1' and 'Speaker 2' implies the tool tracked two voices across the recording, when it only noticed that the voice changed, so two people swapping back and forth get relabelled every few minutes and a reader trusts an attribution that was never computed. Show turns, let a person attach the names, and record who attached them. Done when: Turn markers appear only where the recogniser emitted one and no name is generated by the system, verified by review. Naming a turn applies to the run the user selected and is recorded as a human attribution with a timestamp. With no diarisation-capable model installed the feature is absent rather than degraded into guessing. The interface never uses the phrase speaker identification for this. Do not build yet: No editing surface or export yet. ### Phase 4 · Correcting a transcript without destroying its timing Build: Edit at segment level with the audio playable from the segment start. Store corrections as rows so the machine output remains readable. Support merging and splitting segments with the times recomputed from the audio positions rather than from the surrounding text. Search across segments and jump to the time. Trap: Letting the transcript be edited as one long document loses the mapping between text and time the moment someone reflows a paragraph, and the subtitles exported afterwards drift further out of sync the longer the recording runs. Keep the segment as the unit of editing, and derive the document view from segments rather than the other way round. Done when: Editing any segment leaves its start and end times unchanged, tested. Splitting a segment produces two segments whose times are taken from the chosen audio position and whose union equals the original range. The original machine output for every edited segment is still retrievable. Search returns segments and playing from a result starts at that segment's time, within the stated tolerance. Do not build yet: No exports or retention handling yet. ### Phase 5 · Exports that load, and recordings that do not linger Build: Export SubRip, WebVTT and Markdown. The WebVTT specification requires the body to be UTF-8, to begin with the string WEBVTT, and to write cue times with a full stop before the thousandths, while SubRip files conventionally use a comma, so the two formats are not the same file with a different extension. Add retention that deletes audio and transcripts together, and an export of everything for one recording. Trap: Writing SubRip timestamps into a file named with a WebVTT extension, or omitting the WEBVTT line, produces a file that every player accepts silently and displays no captions from, so the failure is discovered by an audience rather than by you. Write each format from its own function and validate an exported file by parsing it back before declaring the export complete. Done when: An exported WebVTT file begins with the required string and uses full stops in cue times, verified by parsing it back in the suite. An exported SubRip file uses commas and sequential cue numbers, verified the same way. Deleting a recording removes the original, the converted audio, the segments and the exports, checked by listing the directory afterwards. Retention runs on a schedule and logs what it removed without logging any transcript text. Do not build yet: Nothing after this. ### Out of scope (and why) - Joining meetings to record them, which is a consent problem before it is a technical one and is the part of the paid product you should think hardest about copying. - Live streaming transcription, which needs a different model configuration and a latency budget this build has no reason to carry. - Translation and subtitle burn-in, each of which is a separate quality problem with its own failure modes. - Cross recording analytics, which only mean anything once capture is automatic, and capture is exactly what is excluded here. ### README must contain - That the tool transcribes files you already hold and never joins or records a call, and why - The exact audio form the recogniser requires and the FFmpeg command that produces it - That long files are split on silence, silent stretches are marked rather than transcribed, and why that matters - That turn detection is not speaker identification, and that every name was attached by a person - That no confidence or accuracy percentage is displayed unless it was measured against a transcript you typed
ARCHITECTURE.md
# Architecture · Sonix ## Stack | Part | Choice | Why | | --- | --- | --- | | Runtime | Python 3.12 with FastAPI and a single background worker | transcription is a queue, not a request | | Recogniser | whisper.cpp invoked as a process, model path in configuration | a local model keeps recordings of other people off other people's servers | | Audio | FFmpeg to one canonical form, sixteen kilohertz mono signed sixteen bit | the command line tool accepts sixteen bit WAV and nothing else | | Storage | SQLite for segments and jobs, media beside it on disk | segments are rows because every one of them gets edited | | Editing | Corrections stored as rows against segments, never overwriting the original output | you will want to know what the machine actually said | ## Modules Each module has one owner concern and a documented way to replace it. | Module | Owns | How to replace it | | --- | --- | --- | | Ingest | Hashing, probing and conversion to the canonical form. | Any converter that produces the documented audio shape. | | Chunking | Silence based splitting and offset arithmetic. | Any splitter, provided times are converted to whole file offsets once. | | Recogniser | The process call and its recorded parameters. | Any engine, provided the model name and settings are stored per job. | | Editing | Segment level corrections kept additively. | Any interface, provided segments stay the unit of truth. | | Export | One function per format, each validated by reparsing. | Any format added the same way. | ## Configuration Every runtime setting is an environment variable documented in `.env.example`, validated at startup, with a safe local default wherever one exists. - `DATABASE_PATH` · required · Jobs, chunks, segments and edits. - `MEDIA_DIR` · required · Originals and converted audio, deleted together by retention. - `WHISPER_BIN` · required · The recogniser binary invoked by the worker. - `WHISPER_MODEL` · required · Model file. Its name is recorded on every job. - `CHUNK_TARGET_SECONDS` · required · Target chunk length. The actual cut lands on nearby silence. - `RETENTION_DAYS` · required · Audio and transcripts are removed together after this. - `TIMING_TOLERANCE_MS` · required · What the long file timing test enforces at the end of a recording. ## Production baseline - Security: least privilege, input validation at every boundary, secret redaction in logs, rate limits on abuse-prone paths, no invented security primitives. - Data: explicit schema and migrations, transactional writes where integrity matters, backup and restore procedures that have been exercised. - Integrations: adapters around third-party providers, idempotent webhook or job processing, bounded retries, timeouts. - Observability: structured logs with request or operation ids, an error-tracking hook, and health and readiness checks where a server exists. - Quality: unit tests for domain rules, integration tests at module boundaries, one end-to-end test of the critical path. ## Decision records For each dependency in the stack table, keep a short note: why it was chosen, its failure mode, and how it is replaced. Do not add infrastructure until a requirement in `PRODUCT.md` justifies it.
AGENTS.md
# Agent instructions · Sonix product build - Read `PRODUCT.md` and `ARCHITECTURE.md` before changing code. The stack is fixed: Python 3.12 with FastAPI and a single background worker, whisper.cpp invoked as a process, model path in configuration, FFmpeg to one canonical form, sixteen kilohertz mono signed sixteen bit, SQLite for segments and jobs, media beside it on disk, Corrections stored as rows against segments, never overwriting the original output. - Implement milestone by milestone from `MILESTONES.md`; keep each change reviewable and leave the application runnable at every commit. - Treat authentication, payments, encryption, imports, webhooks and destructive actions as high-risk boundaries when present. - Never invent cryptography or silently weaken a requirement to make a check pass. - Put every external service behind an interface with a deterministic fake for tests. - Add migrations and rollback or recovery notes for every persistent data change. - Log useful operational context without credentials, tokens, passwords or personal data. - Update documentation and run every check before completing a milestone. ## Known traps - Handing the recogniser the original file because it played fine in the browser gets you either an immediate failure or, worse, a run that produces plausible nonsense from a misread sample rate, and on a two hour recording you find out after two hours of processing. Convert first, assert the sample rate and channel count of the converted file, and treat probe failure as a rejected import. - Cutting a recording every thirty seconds on a fixed boundary slices words in half and, worse, feeding the model a passage of silence or music makes it emit fluent invented sentences, which are indistinguishable from real ones in the output because they carry timestamps too. Cut on silence, skip silent stretches explicitly, and mark anything the detector called non speech. - Presenting turn markers as 'Speaker 1' and 'Speaker 2' implies the tool tracked two voices across the recording, when it only noticed that the voice changed, so two people swapping back and forth get relabelled every few minutes and a reader trusts an attribution that was never computed. Show turns, let a person attach the names, and record who attached them. - Letting the transcript be edited as one long document loses the mapping between text and time the moment someone reflows a paragraph, and the subtitles exported afterwards drift further out of sync the longer the recording runs. Keep the segment as the unit of editing, and derive the document view from segments rather than the other way round. - Writing SubRip timestamps into a file named with a WebVTT extension, or omitting the WEBVTT line, produces a file that every player accepts silently and displays no captions from, so the failure is discovered by an audience rather than by you. Write each format from its own function and validate an exported file by parsing it back before declaring the export complete.
MILESTONES.md
# Delivery milestones · Sonix Estimated effort: **weekend** for the indie phases; the production-only milestones add the trust and operability layer. ## M1 · One canonical audio file, produced before anything else runs Every recording becomes the exact format the recogniser accepts, and the original is kept. ### Steps 1. Convert to the documented form, then assert it The tool accepts one shape of audio; produce exactly that shape. Files: `src/audio/convert.py`, `tests/test_convert.py` 2. Probe before queueing An hour of processing is an expensive way to discover a broken file. Files: `src/audio/probe.py` 3. Deduplicate by hash The same recording gets imported twice more often than you would think. Files: `src/ingest.py` ### Done when - [ ] Every job's audio is asserted to be sixteen kilohertz mono sixteen bit - [ ] Unprobeable files are rejected at import - [ ] The original is retained with a matching hash - [ ] A duplicate import is detected by hash ### Watch out - Handing the recogniser the original file because it played fine in the browser gets you either an immediate failure or, worse, a run that produces plausible nonsense from a misread sample rate, and on a two hour recording you find out after two hours of processing. Convert first, assert the sample rate and channel count of the converted file, and treat probe failure as a rejected import. ## M2 · Long recordings, honest timestamps A ninety minute file transcribes with timings that line up at the end as well as the start. ### Steps 1. Split on silence, not on a stopwatch A fixed cut lands in the middle of the most important sentence. Files: `src/transcribe/chunking.py` 2. Convert chunk time to file time once Every export inherits this arithmetic, so get it wrong in one place only. Files: `src/transcribe/timing.py`, `tests/test_timing.py` 3. Mark silence rather than transcribing it Fluent invented sentences are the failure mode nobody notices. Files: `src/transcribe/silence.py` ### Done when - [ ] End of file timestamps are within tolerance - [ ] Chunk boundaries land on detected silence and are recorded - [ ] Silent stretches produce no text - [ ] A worker restart resumes at the last chunk ### Watch out - Cutting a recording every thirty seconds on a fixed boundary slices words in half and, worse, feeding the model a passage of silence or music makes it emit fluent invented sentences, which are indistinguishable from real ones in the output because they carry timestamps too. Cut on silence, skip silent stretches explicitly, and mark anything the detector called non speech. ## M3 · Speakers, described as what they actually are Speaker changes are marked where the tool can mark them, and never invented. ### Steps 1. Use turn detection only where the model supports it And say in the interface that it is experimental, because it is. Files: `src/transcribe/diarise.py` 2. Let a person attach every name And store that a person did it. Files: `src/app/segments.py` 3. Hide the feature when the model is missing A guessed speaker is worse than no speaker. Files: `src/config.py` ### Done when - [ ] No speaker name is generated by the system - [ ] Names are recorded as human attributions - [ ] The feature is absent without a capable model - [ ] The wording says turn detection ### Watch out - Presenting turn markers as 'Speaker 1' and 'Speaker 2' implies the tool tracked two voices across the recording, when it only noticed that the voice changed, so two people swapping back and forth get relabelled every few minutes and a reader trusts an attribution that was never computed. Show turns, let a person attach the names, and record who attached them. ## M4 · Correcting a transcript without destroying its timing Editing text keeps every segment anchored to the audio. ### Steps 1. Make the segment the unit of editing The document view is a rendering of segments, never the source. Files: `src/app/editor.py` 2. Keep corrections additive You will want to compare what the machine said with what it meant. Files: `src/db/edits.py` 3. Recompute times from audio on split Not from character counts, which have nothing to do with speech. Files: `src/transcribe/split.py`, `tests/test_split.py` ### Done when - [ ] Editing text leaves segment times unchanged - [ ] A split produces times covering the original range - [ ] Machine output remains retrievable after edits - [ ] Search results play from the right position ### Watch out - Letting the transcript be edited as one long document loses the mapping between text and time the moment someone reflows a paragraph, and the subtitles exported afterwards drift further out of sync the longer the recording runs. Keep the segment as the unit of editing, and derive the document view from segments rather than the other way round. ## M5 · Exports that load, and recordings that do not linger (production only) Subtitle files open in a player, and a deletion removes the audio as well as the text. ### Steps 1. Write each subtitle format from its own function They differ in the header and in one punctuation mark, which is enough. Files: `src/export/vtt.py`, `src/export/srt.py` 2. Parse every export back before calling it done The cheapest possible test for a format you do not control. Files: `tests/test_export_roundtrip.py` 3. Delete audio and text together A deleted transcript with the recording still on disk has deleted nothing. Files: `src/retention.py` ### Done when - [ ] WebVTT exports parse and carry the required header - [ ] SubRip exports parse with comma times and cue numbers - [ ] Deletion removes audio, segments and exports - [ ] Retention logs removals without transcript text ### Watch out - Writing SubRip timestamps into a file named with a WebVTT extension, or omitting the WEBVTT line, produces a file that every player accepts silently and displays no captions from, so the failure is discovered by an audience rather than by you. Write each format from its own function and validate an exported file by parsing it back before declaring the export complete.
OPERATIONS.md
# Operations · Sonix ## Backup The database and the media directory together. Transcripts without their audio cannot be corrected, and audio without transcripts means reprocessing hours of recordings. ## Restore Restore and confirm that a job interrupted before the backup resumes at its last completed chunk rather than restarting, and that segment times still match the audio. Do a restore drill before the first real user, and write the date here when it passes. ## Monitoring Queue depth, jobs failed on their last attempt, and processing time per audio minute, which is the number that tells you whether the chosen model size still fits the machine. ## Incident checklist A job stuck in processing is the common one, usually a killed worker. The chunk table makes resumption safe, so the response is to reset the job to queued and let it continue rather than to delete and reimport. 1. Contain the issue without destroying evidence or user data. 2. Record the timeline and affected scope. 3. Rotate exposed secrets and revoke compromised sessions or credentials. 4. Restore from a verified backup when needed. 5. Document the root cause, the remediation and the regression test. ## Release gate - [ ] Every job's audio is confirmed to be sixteen kilohertz mono sixteen bit - [ ] End of file timestamps are confirmed within tolerance on a long fixture - [ ] Silence is confirmed to produce no transcript text - [ ] No speaker name is confirmed to be generated by the system - [ ] Both subtitle exports are confirmed to parse back correctly ## Launch constraint Do not market omitted Sonix capabilities as implemented. The non-goals in `PRODUCT.md` remain user-visible limitations until they are deliberately delivered.
.env.example
# Copy to .env and fill in. Never commit .env; this file documents it. # Required. Jobs, chunks, segments and edits. DATABASE_PATH=./data/transcripts.db # Required. Originals and converted audio, deleted together by retention. MEDIA_DIR=./data/media # Required. The recogniser binary invoked by the worker. WHISPER_BIN=/usr/local/bin/whisper-cli # Required. Model file. Its name is recorded on every job. WHISPER_MODEL=./models/ggml-base.en.bin # Required. Target chunk length. The actual cut lands on nearby silence. CHUNK_TARGET_SECONDS=240 # Required. Audio and transcripts are removed together after this. RETENTION_DAYS=180 # Required. What the long file timing test enforces at the end of a recording. TIMING_TOLERANCE_MS=250
$ choose a build depth, inspect the files, then open the complete pack in your agent
Sonix: Customers pay for automatic capture, dependable speaker handling, search across calls, and notes arriving without manual file wrangling.
xlive multi-speaker accuracy
xcalendar and CRM integrations
xcross-call team analytics
xmeeting-bot auto-join
share on X ↗"I just replaced Sonix with one prompt"
Sonix pricing
| plan | monthly | annual (per mo) | what you get |
|---|---|---|---|
| free trial | $0 | $0 | 30 transcription minutes total. |
| pay as you go | custom | — | 1 user; 5 GB storage; transcription charged at $10 per audio/video hour. |
| core | $25/workspace | $22.92/workspace | 5 transcription/translation hours and 5 AI Workspace hours per month; 25 GB storage; 1 included user. |
| advanced | $50/workspace | $45.83/workspace | 20 transcription/translation hours and 25 AI Workspace hours per month; 50 GB storage; 1 included user. |
| pro | $80/workspace | $73.33/workspace | 40 transcription/translation hours and 100 AI Workspace hours per month; 100 GB storage; 1 included user. |
| enterprise | custom | — | Custom hours; 1 TB or more storage; unlimited team members. |
free tierno free tier
billingpay as you go, monthly, or annual; annual subscriptions give 1 month free; 30-minute trial
hidden costsExtra transcription hours cost $10/hour on every subscription. Extra seats cost $25/month each but add no hours; automated translation, alignment, and burned-in subtitles can also consume separately charged hours.
verified 2026-08-14 · source ↗
Vibecode Sonix
Kinda. The core of Sonix is buildable in a weekend with the prompt on this page, but there are real gaps: live multi-speaker accuracy, calendar and CRM integrations. Read the honest list above before committing.
How much does Sonix cost?
Sonix's pricing is usage-based or varies by plan · The official pricing URL is recorded, but a stable current amount was not safely recoverable in this pass. Do not publish a number until rechecked..
What do I lose by replacing Sonix?
Honestly: live multi-speaker accuracy; calendar and CRM integrations; cross-call team analytics; meeting-bot auto-join. If any of those are load-bearing for you, keep paying.
Is there an open-source alternative to Sonix?
Yes: whisper.cpp (Local speech-to-text engine suitable for private transcription.). Using prior art is also vibecoding; the prompt is for when you want it exactly your way.