Vibecode timeOS
track this build4 phases, 12 steps, beginner friendly0%KINDA · weekend project
The visible meeting productivity loop is buildable, but a credible replacement needs more than the first screen. timeOS earns its keep through capture, integrations, reliability, so expect a weekend or multi-day build and a narrower personal scope.
the project pack5 files · written for this build, step by step
You are building a lean indie version of timeOS. Create the following project files first, then implement the application by following them. Keep the files updated as decisions change. Do not collapse this into a single README or prompt. ===== README.md ===== # timeOS · indie build A local meeting recorder whose hard problems are keeping the audio and the transcript joined so a claim can be checked, refusing to decode silence where speech models invent sentences, and being honest that speaker labels are guesses a human corrects. Estimated effort: **multi-day**. Work `BUILD_PLAN.md` top to bottom · every phase ends in a check that has to pass before the next one starts. ## Stack | Part | Choice | Why | | --- | --- | --- | | Runtime | Python 3.12 with FastAPI, one process, served on localhost | the recording and the audio never leave the machine that made them | | Transcription | whisper.cpp running locally against a downloaded model | a meeting is the most private audio most people own and it should not be uploaded to transcribe it | | Audio | ffmpeg for normalisation and segment extraction, originals never overwritten | the original file is the evidence behind every note | | Storage | SQLite beside the audio directory, one row per decoder segment | per-segment confidence is the only signal you get about a bad transcript | | Notes | An optional model call over the transcript, with every sentence bound to segments | a summary you cannot trace back is a rumour with timestamps | ## Before you start Have every one of these ready. The plan assumes them from step one. - [ ] **Python 3.12** · free - Why: Runs the application and the ingest pipeline. - Get it: python.org or your version manager. - Verify: python3 --version prints 3.12 or higher - [ ] **ffmpeg with silencedetect** · free - Why: Handles conversion and the silence detection that keeps non-speech out of the decoder. - Get it: Your package manager, or a static build. - Verify: ffmpeg -filters | grep silencedetect returns a row - [ ] **whisper.cpp with a downloaded model file** · free, plus the disk for the model - Why: Transcription runs locally, which is the whole reason this build exists rather than uploading a meeting recording. - Get it: Build it and fetch a model; start with a medium-sized one and measure speed on your own machine before choosing. - Verify: A one-minute sample transcribes and prints segment timings - [ ] **A real multi-speaker recording with cross-talk and silence** · free - Why: Every failure this build handles appears in a real meeting and in none of the clean test clips people usually try first. - Get it: Record a genuine conversation with everyone's agreement, including a long pause and two people talking over each other. - Verify: It contains at least one silence over ten seconds and one overlap - [ ] **How the room is told a recording is happening** · free - Why: Deciding this after the build means deciding it from whatever the interface happened to make easy. - Get it: Write down the wording you will use, when it is shown, and what you will check about local law. - Verify: The wording appears in the README and the timestamp gate is asserted in a test ## Quick start ```sh ffmpeg -i input.m4a -ac 1 -ar 16000 -c:a pcm_s16le work.wav ``` Then copy `.env.example` to `.env` and fill in the values it documents. ## Honest limits This build deliberately does not replace: - A meeting bot that joins calls on your behalf. - Real-time transcription during a call. - Voice prints and speaker identification across recordings. - Calendar and CRM synchronisation. - live multi-speaker accuracy - calendar and CRM integrations - cross-call team analytics - meeting-bot auto-join If one of those is essential to you, that is the reason to keep paying for timeOS, and the README should say so rather than pretend. ===== BRIEF.md ===== # Build brief · timeOS The one-shot brief this plan expands. `BUILD_PLAN.md` (or `MILESTONES.md`) is the same sequence broken into steps and checks; where the two disagree, the plan wins. Build me a private meeting recorder that transcribes on your own machine and turns a call into notes you can check against the audio to replace timeOS. Build it in phases, in the order below. Do not write the whole thing in one pass. Finish a phase, run its "Done when" check, fix what fails, and only then start the next phase. ### Rule zero A machine transcript is not a record of what was said, and a summary of a transcript is two steps away from it. Design for that or you will ship confident fiction. The 2024 FAccT paper 'Careless Whisper' measured this directly: roughly 1 percent of audio transcriptions contained entire hallucinated phrases or sentences which did not exist in any form in the underlying audio, and 38 percent of those hallucinations included explicit harms such as invented violence, false associations or implied authority. The hallucinations clustered on speakers with longer non-vocal stretches, which is a common symptom of aphasia, so the people most likely to be misquoted by this build are the ones least able to argue with it. So: the audio is kept for exactly as long as the notes are, every sentence in a generated note carries the segment identifiers it came from, and one click plays that second of audio. A note sentence with no source segment is not stored at all. Silence and non-speech are detected and dropped rather than decoded, because that is where the invented sentences appear. And tell the room. Recording a conversation is something the other people in it are told about before it starts, and what your jurisdiction requires on top of that is yours to look up. Nothing in this build joins a call by itself, sits in one quietly, or records without a visible indicator. ### Stack (fixed, do not substitute) - Runtime: Python 3.12 with FastAPI, one process, served on localhost · the recording and the audio never leave the machine that made them - Transcription: whisper.cpp running locally against a downloaded model · a meeting is the most private audio most people own and it should not be uploaded to transcribe it - Audio: ffmpeg for normalisation and segment extraction, originals never overwritten · the original file is the evidence behind every note - Storage: SQLite beside the audio directory, one row per decoder segment · per-segment confidence is the only signal you get about a bad transcript - Notes: An optional model call over the transcript, with every sentence bound to segments · a summary you cannot trace back is a rumour with timestamps ### Data model (create this before Phase 1) - recordings(id, title, started_at, ended_at, source, audio_path, sha256, duration_ms, sample_rate, notice_given_at) - segments(id, recording_id, ordinal, start_ms, end_ms, text, avg_logprob, no_speech_prob, compression_ratio, model_id, flagged_reason) · the decoder's own confidence fields are kept, not discarded - turns(id, recording_id, start_ms, end_ms, speaker_label, assigned_by) · assigned_by is heuristic or human, and the interface shows which - notes(id, recording_id, kind, body_md, model_id, prompt_version, created_at) · kind is summary, decisions or actions - note_claims(id, note_id, ordinal, text, segment_ids_json) · one row per sentence of a note and the segments behind it - actions(id, recording_id, text, owner_label, due_on, segment_id, state) · an action item with no segment is refused on insert - retention(recording_id, delete_audio_after_days, delete_all_after_days) · audio and notes expire together by default, because one without the other is unverifiable - There is no column anywhere holding a note sentence without a source segment. That constraint is what makes the rest of this honest. ### Phase 1 · Audio in, audio kept Build: Accept an audio or video file, extract audio with ffmpeg to a known sample rate into a working copy, and keep the original untouched alongside it. Hash both. Record a notice-given timestamp that the user has to set before a recording can be transcribed, and show a visible indicator whenever local capture is running. Trap: Transcoding the upload in place to save disk means the one artefact that can settle an argument about what was said has been through a lossy encoder chosen by you, and nobody can tell afterwards which distortions came from the microphone and which from your ffmpeg flags. Keep the original file, work from a derived copy, and store the hash of both. Done when: Importing a file stores the original unmodified, verified by comparing its hash before and after. A working copy exists at the declared sample rate and the ffmpeg command used is recorded with the row. A recording cannot be transcribed until a notice-given timestamp is set, tested by attempting it. Local capture shows an indicator for its whole duration, checked by review. Do not build yet: No transcription, speakers, notes or export yet. ### Phase 2 · A transcript that keeps its doubts Build: Run whisper.cpp with word or segment timestamps and capture avg_logprob, no_speech_prob and compression_ratio for every segment. Detect silence with ffmpeg before decoding and skip those spans entirely. Flag any segment whose no_speech_prob is above your threshold, whose compression ratio indicates a repetition loop, or whose text repeats the previous segment. Trap: Feeding the whole file to the decoder and keeping only the text throws away the three numbers that tell you which sentences to distrust, and the published research says the invented sentences appear precisely in the non-vocal stretches you just fed it. Cut silence out before decoding, keep every confidence field on the row, and mark flagged segments in the interface rather than in a log. Done when: Every segment row carries avg_logprob, no_speech_prob and compression_ratio from the decoder, asserted on a real file. A file consisting of thirty seconds of room tone produces no segments, tested directly. A repeated-phrase loop is flagged rather than stored as clean text, tested with a fixture containing a long silence. Flagged segments render visibly differently in the transcript view and are excluded from note generation unless the user overrides per segment. Do not build yet: No speaker labels, notes, search or export yet. ### Phase 3 · Speakers are a guess you can correct Build: Segment the recording into turns from pauses and energy changes, propose speaker labels, and mark every one as heuristic. Let the user rename a speaker once and have it apply across the whole recording. Show heuristic and human-assigned labels differently everywhere they appear. Trap: Presenting automatic speaker labels in the same typography as corrected ones means a quote gets attributed to the wrong person in the notes and then in an email, and nobody can tell from the interface that the attribution was a guess by a pause detector. Keep assigned_by on every turn and render the two states differently, including in exports. Done when: Every turn records whether its label came from the heuristic or from a person, and the two render differently in the transcript and in exports, checked by review. Renaming a speaker updates every turn with that label in one action, tested. A turn boundary can be split or merged by hand and the segment timings follow. No note or action item quotes a speaker name that is still heuristic without marking it as unconfirmed. Do not build yet: No notes, search or export yet. ### Phase 4 · Notes that cite the audio Build: Generate a summary, a decisions list and an action list from the flagged-clean transcript, and require the model to return each sentence with the segment identifiers it came from. Reject any sentence whose segment identifiers do not exist. Render every note sentence with a timestamp that seeks the audio player to that moment. Trap: Asking for a summary as prose and storing it produces text that reads beautifully and cannot be checked, so the first time someone disputes an action item you are re-listening to forty minutes of audio to find out whether the model made it up. Require segment identifiers per sentence, validate them against the table, and drop any sentence that cannot be placed. Done when: Every stored note sentence has at least one valid segment identifier, enforced by a database constraint and tested with a fabricated response. Clicking any note sentence seeks the audio player to the start of its first segment, tested end to end. A model response citing a segment identifier that does not exist stores nothing and is reported to the user. Action items with no source segment are refused on insert. Do not build yet: No retention, cost ceiling or export yet. ### Phase 5 · Expiry, export and a bill that cannot run away Build: Delete audio and notes together on the configured retention, and log the deletion. Export a recording as a folder containing the audio, a Markdown transcript with timestamps, the notes with their citations, and the raw segment table as JSON. Track any model spend against a monthly ceiling and fall back to transcript-only when it is reached. Trap: Expiring the audio while keeping the notes leaves a file of confident statements about a conversation that can no longer be checked against anything, which is the exact failure the citation work in the previous phase existed to prevent. Expire them together by default, and make keeping notes past the audio an explicit per-recording choice that stamps the notes as unverifiable. Done when: Retention deletes the audio and its notes in the same transaction by default and writes a deletion log entry, tested by advancing the clock. Keeping notes beyond the audio requires an explicit per-recording setting and stamps those notes as unverifiable in the interface and in exports. An exported folder opens and plays without this application present, tested on a second machine. Reaching the monthly model ceiling disables note generation and leaves transcription and playback working. Do not build yet: Nothing after this. ### Out of scope (and why) - A bot that joins calendar meetings by itself, which is the largest part of the paid product and depends on each platform's published participation rules rather than on your code. - Real-time transcription during the call, which changes the decoder configuration, the latency budget and the hardware requirement all at once. - Speaker identification across recordings, which means storing voice prints of colleagues and is a different consent conversation entirely. - Calendar and CRM synchronisation, which is the integration catalogue that the subscription is really selling. ### README must contain - The published hallucination findings for local speech models, with the study and the date, before any setup instructions - That silence is cut before decoding, and why that is where invented sentences come from - That every note sentence is stored with the audio segments it came from, and that unsourced sentences are dropped - That speaker labels are heuristic until a person corrects them, and how the two are distinguished in exports - That audio and notes expire together by default, and how to override that and what the override marks ===== AGENTS.md ===== # Agent instructions · timeOS indie build - Read `README.md` and `BUILD_PLAN.md` before writing code. The stack is fixed: Python 3.12 with FastAPI, one process, served on localhost, whisper.cpp running locally against a downloaded model, ffmpeg for normalisation and segment extraction, originals never overwritten, SQLite beside the audio directory, one row per decoder segment, An optional model call over the transcript, with every sentence bound to segments. Do not substitute. - Work one phase at a time, in order. Do not start a phase until every "Done when" item of the previous one passes. - Prefer the fewest moving parts that satisfy the step. No frameworks, services or dependencies the plan does not name. - Secrets live in `.env`, never in source or logs. Keep `.env.example` current when a variable is introduced. - Do not invent cryptography, security guarantees, APIs or compliance claims. - Add a focused test for every destructive, security-sensitive or data-loss path the plan names. - Run the project checks before declaring a phase complete, and record any deliberate shortcut in the README under "Tradeoffs". ## Known traps - Transcoding the upload in place to save disk means the one artefact that can settle an argument about what was said has been through a lossy encoder chosen by you, and nobody can tell afterwards which distortions came from the microphone and which from your ffmpeg flags. Keep the original file, work from a derived copy, and store the hash of both. - Feeding the whole file to the decoder and keeping only the text throws away the three numbers that tell you which sentences to distrust, and the published research says the invented sentences appear precisely in the non-vocal stretches you just fed it. Cut silence out before decoding, keep every confidence field on the row, and mark flagged segments in the interface rather than in a log. - Presenting automatic speaker labels in the same typography as corrected ones means a quote gets attributed to the wrong person in the notes and then in an email, and nobody can tell from the interface that the attribution was a guess by a pause detector. Keep assigned_by on every turn and render the two states differently, including in exports. - Asking for a summary as prose and storing it produces text that reads beautifully and cannot be checked, so the first time someone disputes an action item you are re-listening to forty minutes of audio to find out whether the model made it up. Require segment identifiers per sentence, validate them against the table, and drop any sentence that cannot be placed. ===== BUILD_PLAN.md ===== # Build plan · timeOS A local meeting recorder whose hard problems are keeping the audio and the transcript joined so a claim can be checked, refusing to decode silence where speech models invent sentences, and being honest that speaker labels are guesses a human corrects. Phases are in dependency order. Each ends in a "Done when" list; treat an unticked item as a blocker, not a note. ## Phase 1 · Audio in, audio kept A recording is imported or captured, hashed, and stored without being degraded on the way in. ### Steps 1. Keep the original and derive a working copy The original is the only thing that can settle a dispute about a note. Files: `app/ingest.py`, `app/audio.py` ```sh ffmpeg -i input.m4a -ac 1 -ar 16000 -c:a pcm_s16le work.wav ``` 2. Hash both files and store the command used So a strange transcript can be traced to a conversion rather than argued about. Files: `app/ingest.py`, `tests/test_ingest.py` 3. Gate transcription behind a notice timestamp The other people in the call are told before this runs, not after. Files: `app/models.py`, `app/routes/recordings.py` ### Done when - [ ] The original file is byte-identical after import - [ ] A working copy exists with its conversion command recorded - [ ] Transcription is refused until notice is recorded - [ ] Local capture shows a visible indicator throughout ### Watch out - Transcoding the upload in place to save disk means the one artefact that can settle an argument about what was said has been through a lossy encoder chosen by you, and nobody can tell afterwards which distortions came from the microphone and which from your ffmpeg flags. Keep the original file, work from a derived copy, and store the hash of both. ## Phase 2 · A transcript that keeps its doubts whisper.cpp output is stored per segment with its confidence fields, and non-speech is never decoded. ### Steps 1. Detect and cut silence before decoding This is where speech models invent whole sentences. Files: `app/silence.py`, `tests/test_silence.py` ```sh ffmpeg -i work.wav -af silencedetect=noise=-35dB:d=1.0 -f null - ``` 2. Store every confidence field the decoder returns Text alone gives you nothing to sort the doubtful segments by. Files: `app/transcribe.py`, `app/models.py` 3. Flag repetition loops and low-confidence segments Visibly, in the transcript, not in a log file nobody opens. Files: `app/flags.py`, `tests/test_flags.py` ### Done when - [ ] Confidence fields are stored per segment - [ ] Room tone produces zero segments - [ ] A repetition loop is flagged, not stored as clean text - [ ] Flagged segments are visible and excluded from notes by default ### Watch out - Feeding the whole file to the decoder and keeping only the text throws away the three numbers that tell you which sentences to distrust, and the published research says the invented sentences appear precisely in the non-vocal stretches you just fed it. Cut silence out before decoding, keep every confidence field on the row, and mark flagged segments in the interface rather than in a log. ## Phase 3 · Speakers are a guess you can correct Turns are proposed heuristically, labelled by a human, and the interface never pretends otherwise. ### Steps 1. Propose turns from pauses, not from a diarisation claim A pause detector is honest about being a pause detector. Files: `app/turns.py`, `tests/test_turns.py` 2. Record assigned_by on every turn The difference between a guess and a correction has to survive into the export. Files: `app/models.py` 3. Let one rename propagate across the recording Otherwise nobody corrects anything after the third turn. Files: `app/routes/turns.py` ### Done when - [ ] Heuristic and human labels render differently - [ ] One rename updates every matching turn - [ ] Turns can be split and merged by hand - [ ] Unconfirmed speaker names are marked in notes ### Watch out - Presenting automatic speaker labels in the same typography as corrected ones means a quote gets attributed to the wrong person in the notes and then in an email, and nobody can tell from the interface that the attribution was a guess by a pause detector. Keep assigned_by on every turn and render the two states differently, including in exports. ## Phase 4 · Notes that cite the audio Every sentence of a generated note is bound to segments and plays back from the recording. ### Steps 1. Require segment identifiers in the structured output A claim without a location in the audio is not a claim. Files: `app/notes.py`, `app/schemas.py` 2. Validate identifiers against the segments table Models cite things that do not exist, cheerfully. Files: `app/notes.py`, `tests/test_notes.py` 3. Wire every sentence to a seek in the player One click from the claim to the audio, or the citation is decoration. Files: `app/templates/recording.html`, `app/static/player.js` ### Done when - [ ] Every note sentence carries a valid segment identifier - [ ] Clicking a sentence seeks the audio to its segment - [ ] A fabricated citation stores nothing - [ ] Action items without a segment are refused ### Watch out - Asking for a summary as prose and storing it produces text that reads beautifully and cannot be checked, so the first time someone disputes an action item you are re-listening to forty minutes of audio to find out whether the model made it up. Require segment identifiers per sentence, validate them against the table, and drop any sentence that cannot be placed. ## Not in this build - A meeting bot that joins calls on your behalf. - Real-time transcription during a call. - Voice prints and speaker identification across recordings. - Calendar and CRM synchronisation. ## After v1, if you want it - A search across recordings that ranks by segment confidence as well as relevance - A weekly digest of action items whose source segments were flagged, which is where the errors hide - A second decoder pass on flagged segments only, comparing the two outputs before trusting either ===== .env.example ===== # Copy to .env and fill in. Never commit .env; this file documents it. # Required. Recordings, segments, turns, notes and citations. DATABASE_PATH=./data/meetings.db # Required. Originals and working copies. Sized for the retention you chose. AUDIO_DIR=./data/audio # Required. The local model file. Its identifier is stored with every segment. WHISPER_MODEL_PATH=./models/ggml-medium.bin # Required. Segments above this are flagged and excluded from note generation. NO_SPEECH_THRESHOLD=0.6 # Required. Passed to ffmpeg silencedetect. Tune it against your own recordings. SILENCE_DB=-35dB # Required. Audio and notes are deleted together at this age unless overridden per recording. RETENTION_DAYS=90 # Optional · secret. Optional. Only note generation uses it; transcription never leaves the machine. NOTES_MODEL_KEY=sk-...redacted # Required. Note generation stops here for the month. Transcription and playback continue. MONTHLY_COST_CEILING=3.00
You are building a lean indie version of timeOS. Create the following project files first, then implement the application by following them. Keep the files updated as decisions change. Do not collapse this into a single README or prompt. ===== README.md ===== # timeOS · indie build A local meeting recorder whose hard problems are keeping the audio and the transcript joined so a claim can be checked, refusing to decode silence where speech models invent sentences, and being honest that speaker labels are guesses a human corrects. Estimated effort: **multi-day**. Work `BUILD_PLAN.md` top to bottom · every phase ends in a check that has to pass before the next one starts. ## Stack | Part | Choice | Why | | --- | --- | --- | | Runtime | Python 3.12 with FastAPI, one process, served on localhost | the recording and the audio never leave the machine that made them | | Transcription | whisper.cpp running locally against a downloaded model | a meeting is the most private audio most people own and it should not be uploaded to transcribe it | | Audio | ffmpeg for normalisation and segment extraction, originals never overwritten | the original file is the evidence behind every note | | Storage | SQLite beside the audio directory, one row per decoder segment | per-segment confidence is the only signal you get about a bad transcript | | Notes | An optional model call over the transcript, with every sentence bound to segments | a summary you cannot trace back is a rumour with timestamps | ## Before you start Have every one of these ready. The plan assumes them from step one. - [ ] **Python 3.12** · free - Why: Runs the application and the ingest pipeline. - Get it: python.org or your version manager. - Verify: python3 --version prints 3.12 or higher - [ ] **ffmpeg with silencedetect** · free - Why: Handles conversion and the silence detection that keeps non-speech out of the decoder. - Get it: Your package manager, or a static build. - Verify: ffmpeg -filters | grep silencedetect returns a row - [ ] **whisper.cpp with a downloaded model file** · free, plus the disk for the model - Why: Transcription runs locally, which is the whole reason this build exists rather than uploading a meeting recording. - Get it: Build it and fetch a model; start with a medium-sized one and measure speed on your own machine before choosing. - Verify: A one-minute sample transcribes and prints segment timings - [ ] **A real multi-speaker recording with cross-talk and silence** · free - Why: Every failure this build handles appears in a real meeting and in none of the clean test clips people usually try first. - Get it: Record a genuine conversation with everyone's agreement, including a long pause and two people talking over each other. - Verify: It contains at least one silence over ten seconds and one overlap - [ ] **How the room is told a recording is happening** · free - Why: Deciding this after the build means deciding it from whatever the interface happened to make easy. - Get it: Write down the wording you will use, when it is shown, and what you will check about local law. - Verify: The wording appears in the README and the timestamp gate is asserted in a test ## Quick start ```sh ffmpeg -i input.m4a -ac 1 -ar 16000 -c:a pcm_s16le work.wav ``` Then copy `.env.example` to `.env` and fill in the values it documents. ## Honest limits This build deliberately does not replace: - A meeting bot that joins calls on your behalf. - Real-time transcription during a call. - Voice prints and speaker identification across recordings. - Calendar and CRM synchronisation. - live multi-speaker accuracy - calendar and CRM integrations - cross-call team analytics - meeting-bot auto-join If one of those is essential to you, that is the reason to keep paying for timeOS, and the README should say so rather than pretend. ===== BRIEF.md ===== # Build brief · timeOS The one-shot brief this plan expands. `BUILD_PLAN.md` (or `MILESTONES.md`) is the same sequence broken into steps and checks; where the two disagree, the plan wins. Build me a private meeting recorder that transcribes on your own machine and turns a call into notes you can check against the audio to replace timeOS. Build it in phases, in the order below. Do not write the whole thing in one pass. Finish a phase, run its "Done when" check, fix what fails, and only then start the next phase. ### Rule zero A machine transcript is not a record of what was said, and a summary of a transcript is two steps away from it. Design for that or you will ship confident fiction. The 2024 FAccT paper 'Careless Whisper' measured this directly: roughly 1 percent of audio transcriptions contained entire hallucinated phrases or sentences which did not exist in any form in the underlying audio, and 38 percent of those hallucinations included explicit harms such as invented violence, false associations or implied authority. The hallucinations clustered on speakers with longer non-vocal stretches, which is a common symptom of aphasia, so the people most likely to be misquoted by this build are the ones least able to argue with it. So: the audio is kept for exactly as long as the notes are, every sentence in a generated note carries the segment identifiers it came from, and one click plays that second of audio. A note sentence with no source segment is not stored at all. Silence and non-speech are detected and dropped rather than decoded, because that is where the invented sentences appear. And tell the room. Recording a conversation is something the other people in it are told about before it starts, and what your jurisdiction requires on top of that is yours to look up. Nothing in this build joins a call by itself, sits in one quietly, or records without a visible indicator. ### Stack (fixed, do not substitute) - Runtime: Python 3.12 with FastAPI, one process, served on localhost · the recording and the audio never leave the machine that made them - Transcription: whisper.cpp running locally against a downloaded model · a meeting is the most private audio most people own and it should not be uploaded to transcribe it - Audio: ffmpeg for normalisation and segment extraction, originals never overwritten · the original file is the evidence behind every note - Storage: SQLite beside the audio directory, one row per decoder segment · per-segment confidence is the only signal you get about a bad transcript - Notes: An optional model call over the transcript, with every sentence bound to segments · a summary you cannot trace back is a rumour with timestamps ### Data model (create this before Phase 1) - recordings(id, title, started_at, ended_at, source, audio_path, sha256, duration_ms, sample_rate, notice_given_at) - segments(id, recording_id, ordinal, start_ms, end_ms, text, avg_logprob, no_speech_prob, compression_ratio, model_id, flagged_reason) · the decoder's own confidence fields are kept, not discarded - turns(id, recording_id, start_ms, end_ms, speaker_label, assigned_by) · assigned_by is heuristic or human, and the interface shows which - notes(id, recording_id, kind, body_md, model_id, prompt_version, created_at) · kind is summary, decisions or actions - note_claims(id, note_id, ordinal, text, segment_ids_json) · one row per sentence of a note and the segments behind it - actions(id, recording_id, text, owner_label, due_on, segment_id, state) · an action item with no segment is refused on insert - retention(recording_id, delete_audio_after_days, delete_all_after_days) · audio and notes expire together by default, because one without the other is unverifiable - There is no column anywhere holding a note sentence without a source segment. That constraint is what makes the rest of this honest. ### Phase 1 · Audio in, audio kept Build: Accept an audio or video file, extract audio with ffmpeg to a known sample rate into a working copy, and keep the original untouched alongside it. Hash both. Record a notice-given timestamp that the user has to set before a recording can be transcribed, and show a visible indicator whenever local capture is running. Trap: Transcoding the upload in place to save disk means the one artefact that can settle an argument about what was said has been through a lossy encoder chosen by you, and nobody can tell afterwards which distortions came from the microphone and which from your ffmpeg flags. Keep the original file, work from a derived copy, and store the hash of both. Done when: Importing a file stores the original unmodified, verified by comparing its hash before and after. A working copy exists at the declared sample rate and the ffmpeg command used is recorded with the row. A recording cannot be transcribed until a notice-given timestamp is set, tested by attempting it. Local capture shows an indicator for its whole duration, checked by review. Do not build yet: No transcription, speakers, notes or export yet. ### Phase 2 · A transcript that keeps its doubts Build: Run whisper.cpp with word or segment timestamps and capture avg_logprob, no_speech_prob and compression_ratio for every segment. Detect silence with ffmpeg before decoding and skip those spans entirely. Flag any segment whose no_speech_prob is above your threshold, whose compression ratio indicates a repetition loop, or whose text repeats the previous segment. Trap: Feeding the whole file to the decoder and keeping only the text throws away the three numbers that tell you which sentences to distrust, and the published research says the invented sentences appear precisely in the non-vocal stretches you just fed it. Cut silence out before decoding, keep every confidence field on the row, and mark flagged segments in the interface rather than in a log. Done when: Every segment row carries avg_logprob, no_speech_prob and compression_ratio from the decoder, asserted on a real file. A file consisting of thirty seconds of room tone produces no segments, tested directly. A repeated-phrase loop is flagged rather than stored as clean text, tested with a fixture containing a long silence. Flagged segments render visibly differently in the transcript view and are excluded from note generation unless the user overrides per segment. Do not build yet: No speaker labels, notes, search or export yet. ### Phase 3 · Speakers are a guess you can correct Build: Segment the recording into turns from pauses and energy changes, propose speaker labels, and mark every one as heuristic. Let the user rename a speaker once and have it apply across the whole recording. Show heuristic and human-assigned labels differently everywhere they appear. Trap: Presenting automatic speaker labels in the same typography as corrected ones means a quote gets attributed to the wrong person in the notes and then in an email, and nobody can tell from the interface that the attribution was a guess by a pause detector. Keep assigned_by on every turn and render the two states differently, including in exports. Done when: Every turn records whether its label came from the heuristic or from a person, and the two render differently in the transcript and in exports, checked by review. Renaming a speaker updates every turn with that label in one action, tested. A turn boundary can be split or merged by hand and the segment timings follow. No note or action item quotes a speaker name that is still heuristic without marking it as unconfirmed. Do not build yet: No notes, search or export yet. ### Phase 4 · Notes that cite the audio Build: Generate a summary, a decisions list and an action list from the flagged-clean transcript, and require the model to return each sentence with the segment identifiers it came from. Reject any sentence whose segment identifiers do not exist. Render every note sentence with a timestamp that seeks the audio player to that moment. Trap: Asking for a summary as prose and storing it produces text that reads beautifully and cannot be checked, so the first time someone disputes an action item you are re-listening to forty minutes of audio to find out whether the model made it up. Require segment identifiers per sentence, validate them against the table, and drop any sentence that cannot be placed. Done when: Every stored note sentence has at least one valid segment identifier, enforced by a database constraint and tested with a fabricated response. Clicking any note sentence seeks the audio player to the start of its first segment, tested end to end. A model response citing a segment identifier that does not exist stores nothing and is reported to the user. Action items with no source segment are refused on insert. Do not build yet: No retention, cost ceiling or export yet. ### Phase 5 · Expiry, export and a bill that cannot run away Build: Delete audio and notes together on the configured retention, and log the deletion. Export a recording as a folder containing the audio, a Markdown transcript with timestamps, the notes with their citations, and the raw segment table as JSON. Track any model spend against a monthly ceiling and fall back to transcript-only when it is reached. Trap: Expiring the audio while keeping the notes leaves a file of confident statements about a conversation that can no longer be checked against anything, which is the exact failure the citation work in the previous phase existed to prevent. Expire them together by default, and make keeping notes past the audio an explicit per-recording choice that stamps the notes as unverifiable. Done when: Retention deletes the audio and its notes in the same transaction by default and writes a deletion log entry, tested by advancing the clock. Keeping notes beyond the audio requires an explicit per-recording setting and stamps those notes as unverifiable in the interface and in exports. An exported folder opens and plays without this application present, tested on a second machine. Reaching the monthly model ceiling disables note generation and leaves transcription and playback working. Do not build yet: Nothing after this. ### Out of scope (and why) - A bot that joins calendar meetings by itself, which is the largest part of the paid product and depends on each platform's published participation rules rather than on your code. - Real-time transcription during the call, which changes the decoder configuration, the latency budget and the hardware requirement all at once. - Speaker identification across recordings, which means storing voice prints of colleagues and is a different consent conversation entirely. - Calendar and CRM synchronisation, which is the integration catalogue that the subscription is really selling. ### README must contain - The published hallucination findings for local speech models, with the study and the date, before any setup instructions - That silence is cut before decoding, and why that is where invented sentences come from - That every note sentence is stored with the audio segments it came from, and that unsourced sentences are dropped - That speaker labels are heuristic until a person corrects them, and how the two are distinguished in exports - That audio and notes expire together by default, and how to override that and what the override marks ===== AGENTS.md ===== # Agent instructions · timeOS indie build - Read `README.md` and `BUILD_PLAN.md` before writing code. The stack is fixed: Python 3.12 with FastAPI, one process, served on localhost, whisper.cpp running locally against a downloaded model, ffmpeg for normalisation and segment extraction, originals never overwritten, SQLite beside the audio directory, one row per decoder segment, An optional model call over the transcript, with every sentence bound to segments. Do not substitute. - Work one phase at a time, in order. Do not start a phase until every "Done when" item of the previous one passes. - Prefer the fewest moving parts that satisfy the step. No frameworks, services or dependencies the plan does not name. - Secrets live in `.env`, never in source or logs. Keep `.env.example` current when a variable is introduced. - Do not invent cryptography, security guarantees, APIs or compliance claims. - Add a focused test for every destructive, security-sensitive or data-loss path the plan names. - Run the project checks before declaring a phase complete, and record any deliberate shortcut in the README under "Tradeoffs". ## Known traps - Transcoding the upload in place to save disk means the one artefact that can settle an argument about what was said has been through a lossy encoder chosen by you, and nobody can tell afterwards which distortions came from the microphone and which from your ffmpeg flags. Keep the original file, work from a derived copy, and store the hash of both. - Feeding the whole file to the decoder and keeping only the text throws away the three numbers that tell you which sentences to distrust, and the published research says the invented sentences appear precisely in the non-vocal stretches you just fed it. Cut silence out before decoding, keep every confidence field on the row, and mark flagged segments in the interface rather than in a log. - Presenting automatic speaker labels in the same typography as corrected ones means a quote gets attributed to the wrong person in the notes and then in an email, and nobody can tell from the interface that the attribution was a guess by a pause detector. Keep assigned_by on every turn and render the two states differently, including in exports. - Asking for a summary as prose and storing it produces text that reads beautifully and cannot be checked, so the first time someone disputes an action item you are re-listening to forty minutes of audio to find out whether the model made it up. Require segment identifiers per sentence, validate them against the table, and drop any sentence that cannot be placed. ===== BUILD_PLAN.md ===== # Build plan · timeOS A local meeting recorder whose hard problems are keeping the audio and the transcript joined so a claim can be checked, refusing to decode silence where speech models invent sentences, and being honest that speaker labels are guesses a human corrects. Phases are in dependency order. Each ends in a "Done when" list; treat an unticked item as a blocker, not a note. ## Phase 1 · Audio in, audio kept A recording is imported or captured, hashed, and stored without being degraded on the way in. ### Steps 1. Keep the original and derive a working copy The original is the only thing that can settle a dispute about a note. Files: `app/ingest.py`, `app/audio.py` ```sh ffmpeg -i input.m4a -ac 1 -ar 16000 -c:a pcm_s16le work.wav ``` 2. Hash both files and store the command used So a strange transcript can be traced to a conversion rather than argued about. Files: `app/ingest.py`, `tests/test_ingest.py` 3. Gate transcription behind a notice timestamp The other people in the call are told before this runs, not after. Files: `app/models.py`, `app/routes/recordings.py` ### Done when - [ ] The original file is byte-identical after import - [ ] A working copy exists with its conversion command recorded - [ ] Transcription is refused until notice is recorded - [ ] Local capture shows a visible indicator throughout ### Watch out - Transcoding the upload in place to save disk means the one artefact that can settle an argument about what was said has been through a lossy encoder chosen by you, and nobody can tell afterwards which distortions came from the microphone and which from your ffmpeg flags. Keep the original file, work from a derived copy, and store the hash of both. ## Phase 2 · A transcript that keeps its doubts whisper.cpp output is stored per segment with its confidence fields, and non-speech is never decoded. ### Steps 1. Detect and cut silence before decoding This is where speech models invent whole sentences. Files: `app/silence.py`, `tests/test_silence.py` ```sh ffmpeg -i work.wav -af silencedetect=noise=-35dB:d=1.0 -f null - ``` 2. Store every confidence field the decoder returns Text alone gives you nothing to sort the doubtful segments by. Files: `app/transcribe.py`, `app/models.py` 3. Flag repetition loops and low-confidence segments Visibly, in the transcript, not in a log file nobody opens. Files: `app/flags.py`, `tests/test_flags.py` ### Done when - [ ] Confidence fields are stored per segment - [ ] Room tone produces zero segments - [ ] A repetition loop is flagged, not stored as clean text - [ ] Flagged segments are visible and excluded from notes by default ### Watch out - Feeding the whole file to the decoder and keeping only the text throws away the three numbers that tell you which sentences to distrust, and the published research says the invented sentences appear precisely in the non-vocal stretches you just fed it. Cut silence out before decoding, keep every confidence field on the row, and mark flagged segments in the interface rather than in a log. ## Phase 3 · Speakers are a guess you can correct Turns are proposed heuristically, labelled by a human, and the interface never pretends otherwise. ### Steps 1. Propose turns from pauses, not from a diarisation claim A pause detector is honest about being a pause detector. Files: `app/turns.py`, `tests/test_turns.py` 2. Record assigned_by on every turn The difference between a guess and a correction has to survive into the export. Files: `app/models.py` 3. Let one rename propagate across the recording Otherwise nobody corrects anything after the third turn. Files: `app/routes/turns.py` ### Done when - [ ] Heuristic and human labels render differently - [ ] One rename updates every matching turn - [ ] Turns can be split and merged by hand - [ ] Unconfirmed speaker names are marked in notes ### Watch out - Presenting automatic speaker labels in the same typography as corrected ones means a quote gets attributed to the wrong person in the notes and then in an email, and nobody can tell from the interface that the attribution was a guess by a pause detector. Keep assigned_by on every turn and render the two states differently, including in exports. ## Phase 4 · Notes that cite the audio Every sentence of a generated note is bound to segments and plays back from the recording. ### Steps 1. Require segment identifiers in the structured output A claim without a location in the audio is not a claim. Files: `app/notes.py`, `app/schemas.py` 2. Validate identifiers against the segments table Models cite things that do not exist, cheerfully. Files: `app/notes.py`, `tests/test_notes.py` 3. Wire every sentence to a seek in the player One click from the claim to the audio, or the citation is decoration. Files: `app/templates/recording.html`, `app/static/player.js` ### Done when - [ ] Every note sentence carries a valid segment identifier - [ ] Clicking a sentence seeks the audio to its segment - [ ] A fabricated citation stores nothing - [ ] Action items without a segment are refused ### Watch out - Asking for a summary as prose and storing it produces text that reads beautifully and cannot be checked, so the first time someone disputes an action item you are re-listening to forty minutes of audio to find out whether the model made it up. Require segment identifiers per sentence, validate them against the table, and drop any sentence that cannot be placed. ## Not in this build - A meeting bot that joins calls on your behalf. - Real-time transcription during a call. - Voice prints and speaker identification across recordings. - Calendar and CRM synchronisation. ## After v1, if you want it - A search across recordings that ranks by segment confidence as well as relevance - A weekly digest of action items whose source segments were flagged, which is where the errors hide - A second decoder pass on flagged segments only, comparing the two outputs before trusting either ===== .env.example ===== # Copy to .env and fill in. Never commit .env; this file documents it. # Required. Recordings, segments, turns, notes and citations. DATABASE_PATH=./data/meetings.db # Required. Originals and working copies. Sized for the retention you chose. AUDIO_DIR=./data/audio # Required. The local model file. Its identifier is stored with every segment. WHISPER_MODEL_PATH=./models/ggml-medium.bin # Required. Segments above this are flagged and excluded from note generation. NO_SPEECH_THRESHOLD=0.6 # Required. Passed to ffmpeg silencedetect. Tune it against your own recordings. SILENCE_DB=-35dB # Required. Audio and notes are deleted together at this age unless overridden per recording. RETENTION_DAYS=90 # Optional · secret. Optional. Only note generation uses it; transcription never leaves the machine. NOTES_MODEL_KEY=sk-...redacted # Required. Note generation stops here for the month. Transcription and playback continue. MONTHLY_COST_CEILING=3.00
You are building a production product version of timeOS. Create the following project files first, then implement the application by following them. Keep the files updated as decisions change. Do not collapse this into a single README or prompt. ===== PRODUCT.md ===== # timeOS · product brief ## Problem The visible meeting productivity loop is buildable, but a credible replacement needs more than the first screen. timeOS earns its keep through capture, integrations, reliability, so expect a weekend or multi-day build and a narrower personal scope. ## Product outcome A meeting archive where every generated sentence can be played back from the audio it came from, the decoder never runs on silence, speaker labels admit to being guesses, and the notes never outlive the recording that would settle an argument about them. ## Target user A builder who needs a maintainable product foundation, not a one-off demo. ## Required capabilities - Python 3.12 - FFmpeg - local Whisper model or optional API key in .env ## Explicit non-goals for v1 - A meeting bot that joins calls on your behalf. - Real-time transcription during a call. - Voice prints and speaker identification across recordings. - Calendar and CRM synchronisation. - live multi-speaker accuracy - calendar and CRM integrations - cross-call team analytics - meeting-bot auto-join ## Success criteria - Room tone is confirmed to produce zero segments - Every note sentence is confirmed to carry a valid segment identifier - Heuristic speaker labels are confirmed visibly distinct from corrected ones - Retention is confirmed to delete audio and notes in one transaction - The cost ceiling is confirmed to stop notes and leave transcription working ===== BRIEF.md ===== # Build brief · timeOS The one-shot brief this plan expands. `BUILD_PLAN.md` (or `MILESTONES.md`) is the same sequence broken into steps and checks; where the two disagree, the plan wins. Build me a private meeting recorder that transcribes on your own machine and turns a call into notes you can check against the audio to replace timeOS. Build it in phases, in the order below. Do not write the whole thing in one pass. Finish a phase, run its "Done when" check, fix what fails, and only then start the next phase. ### Rule zero A machine transcript is not a record of what was said, and a summary of a transcript is two steps away from it. Design for that or you will ship confident fiction. The 2024 FAccT paper 'Careless Whisper' measured this directly: roughly 1 percent of audio transcriptions contained entire hallucinated phrases or sentences which did not exist in any form in the underlying audio, and 38 percent of those hallucinations included explicit harms such as invented violence, false associations or implied authority. The hallucinations clustered on speakers with longer non-vocal stretches, which is a common symptom of aphasia, so the people most likely to be misquoted by this build are the ones least able to argue with it. So: the audio is kept for exactly as long as the notes are, every sentence in a generated note carries the segment identifiers it came from, and one click plays that second of audio. A note sentence with no source segment is not stored at all. Silence and non-speech are detected and dropped rather than decoded, because that is where the invented sentences appear. And tell the room. Recording a conversation is something the other people in it are told about before it starts, and what your jurisdiction requires on top of that is yours to look up. Nothing in this build joins a call by itself, sits in one quietly, or records without a visible indicator. ### Stack (fixed, do not substitute) - Runtime: Python 3.12 with FastAPI, one process, served on localhost · the recording and the audio never leave the machine that made them - Transcription: whisper.cpp running locally against a downloaded model · a meeting is the most private audio most people own and it should not be uploaded to transcribe it - Audio: ffmpeg for normalisation and segment extraction, originals never overwritten · the original file is the evidence behind every note - Storage: SQLite beside the audio directory, one row per decoder segment · per-segment confidence is the only signal you get about a bad transcript - Notes: An optional model call over the transcript, with every sentence bound to segments · a summary you cannot trace back is a rumour with timestamps ### Data model (create this before Phase 1) - recordings(id, title, started_at, ended_at, source, audio_path, sha256, duration_ms, sample_rate, notice_given_at) - segments(id, recording_id, ordinal, start_ms, end_ms, text, avg_logprob, no_speech_prob, compression_ratio, model_id, flagged_reason) · the decoder's own confidence fields are kept, not discarded - turns(id, recording_id, start_ms, end_ms, speaker_label, assigned_by) · assigned_by is heuristic or human, and the interface shows which - notes(id, recording_id, kind, body_md, model_id, prompt_version, created_at) · kind is summary, decisions or actions - note_claims(id, note_id, ordinal, text, segment_ids_json) · one row per sentence of a note and the segments behind it - actions(id, recording_id, text, owner_label, due_on, segment_id, state) · an action item with no segment is refused on insert - retention(recording_id, delete_audio_after_days, delete_all_after_days) · audio and notes expire together by default, because one without the other is unverifiable - There is no column anywhere holding a note sentence without a source segment. That constraint is what makes the rest of this honest. ### Phase 1 · Audio in, audio kept Build: Accept an audio or video file, extract audio with ffmpeg to a known sample rate into a working copy, and keep the original untouched alongside it. Hash both. Record a notice-given timestamp that the user has to set before a recording can be transcribed, and show a visible indicator whenever local capture is running. Trap: Transcoding the upload in place to save disk means the one artefact that can settle an argument about what was said has been through a lossy encoder chosen by you, and nobody can tell afterwards which distortions came from the microphone and which from your ffmpeg flags. Keep the original file, work from a derived copy, and store the hash of both. Done when: Importing a file stores the original unmodified, verified by comparing its hash before and after. A working copy exists at the declared sample rate and the ffmpeg command used is recorded with the row. A recording cannot be transcribed until a notice-given timestamp is set, tested by attempting it. Local capture shows an indicator for its whole duration, checked by review. Do not build yet: No transcription, speakers, notes or export yet. ### Phase 2 · A transcript that keeps its doubts Build: Run whisper.cpp with word or segment timestamps and capture avg_logprob, no_speech_prob and compression_ratio for every segment. Detect silence with ffmpeg before decoding and skip those spans entirely. Flag any segment whose no_speech_prob is above your threshold, whose compression ratio indicates a repetition loop, or whose text repeats the previous segment. Trap: Feeding the whole file to the decoder and keeping only the text throws away the three numbers that tell you which sentences to distrust, and the published research says the invented sentences appear precisely in the non-vocal stretches you just fed it. Cut silence out before decoding, keep every confidence field on the row, and mark flagged segments in the interface rather than in a log. Done when: Every segment row carries avg_logprob, no_speech_prob and compression_ratio from the decoder, asserted on a real file. A file consisting of thirty seconds of room tone produces no segments, tested directly. A repeated-phrase loop is flagged rather than stored as clean text, tested with a fixture containing a long silence. Flagged segments render visibly differently in the transcript view and are excluded from note generation unless the user overrides per segment. Do not build yet: No speaker labels, notes, search or export yet. ### Phase 3 · Speakers are a guess you can correct Build: Segment the recording into turns from pauses and energy changes, propose speaker labels, and mark every one as heuristic. Let the user rename a speaker once and have it apply across the whole recording. Show heuristic and human-assigned labels differently everywhere they appear. Trap: Presenting automatic speaker labels in the same typography as corrected ones means a quote gets attributed to the wrong person in the notes and then in an email, and nobody can tell from the interface that the attribution was a guess by a pause detector. Keep assigned_by on every turn and render the two states differently, including in exports. Done when: Every turn records whether its label came from the heuristic or from a person, and the two render differently in the transcript and in exports, checked by review. Renaming a speaker updates every turn with that label in one action, tested. A turn boundary can be split or merged by hand and the segment timings follow. No note or action item quotes a speaker name that is still heuristic without marking it as unconfirmed. Do not build yet: No notes, search or export yet. ### Phase 4 · Notes that cite the audio Build: Generate a summary, a decisions list and an action list from the flagged-clean transcript, and require the model to return each sentence with the segment identifiers it came from. Reject any sentence whose segment identifiers do not exist. Render every note sentence with a timestamp that seeks the audio player to that moment. Trap: Asking for a summary as prose and storing it produces text that reads beautifully and cannot be checked, so the first time someone disputes an action item you are re-listening to forty minutes of audio to find out whether the model made it up. Require segment identifiers per sentence, validate them against the table, and drop any sentence that cannot be placed. Done when: Every stored note sentence has at least one valid segment identifier, enforced by a database constraint and tested with a fabricated response. Clicking any note sentence seeks the audio player to the start of its first segment, tested end to end. A model response citing a segment identifier that does not exist stores nothing and is reported to the user. Action items with no source segment are refused on insert. Do not build yet: No retention, cost ceiling or export yet. ### Phase 5 · Expiry, export and a bill that cannot run away Build: Delete audio and notes together on the configured retention, and log the deletion. Export a recording as a folder containing the audio, a Markdown transcript with timestamps, the notes with their citations, and the raw segment table as JSON. Track any model spend against a monthly ceiling and fall back to transcript-only when it is reached. Trap: Expiring the audio while keeping the notes leaves a file of confident statements about a conversation that can no longer be checked against anything, which is the exact failure the citation work in the previous phase existed to prevent. Expire them together by default, and make keeping notes past the audio an explicit per-recording choice that stamps the notes as unverifiable. Done when: Retention deletes the audio and its notes in the same transaction by default and writes a deletion log entry, tested by advancing the clock. Keeping notes beyond the audio requires an explicit per-recording setting and stamps those notes as unverifiable in the interface and in exports. An exported folder opens and plays without this application present, tested on a second machine. Reaching the monthly model ceiling disables note generation and leaves transcription and playback working. Do not build yet: Nothing after this. ### Out of scope (and why) - A bot that joins calendar meetings by itself, which is the largest part of the paid product and depends on each platform's published participation rules rather than on your code. - Real-time transcription during the call, which changes the decoder configuration, the latency budget and the hardware requirement all at once. - Speaker identification across recordings, which means storing voice prints of colleagues and is a different consent conversation entirely. - Calendar and CRM synchronisation, which is the integration catalogue that the subscription is really selling. ### README must contain - The published hallucination findings for local speech models, with the study and the date, before any setup instructions - That silence is cut before decoding, and why that is where invented sentences come from - That every note sentence is stored with the audio segments it came from, and that unsourced sentences are dropped - That speaker labels are heuristic until a person corrects them, and how the two are distinguished in exports - That audio and notes expire together by default, and how to override that and what the override marks ===== ARCHITECTURE.md ===== # Architecture · timeOS ## Stack | Part | Choice | Why | | --- | --- | --- | | Runtime | Python 3.12 with FastAPI, one process, served on localhost | the recording and the audio never leave the machine that made them | | Transcription | whisper.cpp running locally against a downloaded model | a meeting is the most private audio most people own and it should not be uploaded to transcribe it | | Audio | ffmpeg for normalisation and segment extraction, originals never overwritten | the original file is the evidence behind every note | | Storage | SQLite beside the audio directory, one row per decoder segment | per-segment confidence is the only signal you get about a bad transcript | | Notes | An optional model call over the transcript, with every sentence bound to segments | a summary you cannot trace back is a rumour with timestamps | ## Modules Each module has one owner concern and a documented way to replace it. | Module | Owns | How to replace it | | --- | --- | --- | | Ingest | Originals, working copies, hashes and the notice gate. | Any capture source, provided the original file survives untouched. | | Transcribe | Silence detection, decoding and per-segment confidence. | Any local engine that returns timings and confidence fields. | | Turns | Heuristic turn boundaries and human labels. | A real diarisation model, provided assigned_by still distinguishes the two. | | Notes | Structured generation and citation validation. | Any model, provided unsourced sentences are still refused. | | Retention | Joint expiry, deletion logging and export. | Not swappable for note-only retention without the unverifiable stamp. | ## Configuration Every runtime setting is an environment variable documented in `.env.example`, validated at startup, with a safe local default wherever one exists. - `DATABASE_PATH` · required · Recordings, segments, turns, notes and citations. - `AUDIO_DIR` · required · Originals and working copies. Sized for the retention you chose. - `WHISPER_MODEL_PATH` · required · The local model file. Its identifier is stored with every segment. - `NO_SPEECH_THRESHOLD` · required · Segments above this are flagged and excluded from note generation. - `SILENCE_DB` · required · Passed to ffmpeg silencedetect. Tune it against your own recordings. - `RETENTION_DAYS` · required · Audio and notes are deleted together at this age unless overridden per recording. - `NOTES_MODEL_KEY` · optional, secret · Optional. Only note generation uses it; transcription never leaves the machine. - `MONTHLY_COST_CEILING` · required · Note generation stops here for the month. Transcription and playback continue. ## Production baseline - Security: least privilege, input validation at every boundary, secret redaction in logs, rate limits on abuse-prone paths, no invented security primitives. - Data: explicit schema and migrations, transactional writes where integrity matters, backup and restore procedures that have been exercised. - Integrations: adapters around third-party providers, idempotent webhook or job processing, bounded retries, timeouts. - Observability: structured logs with request or operation ids, an error-tracking hook, and health and readiness checks where a server exists. - Quality: unit tests for domain rules, integration tests at module boundaries, one end-to-end test of the critical path. ## Decision records For each dependency in the stack table, keep a short note: why it was chosen, its failure mode, and how it is replaced. Do not add infrastructure until a requirement in `PRODUCT.md` justifies it. ===== AGENTS.md ===== # Agent instructions · timeOS product build - Read `PRODUCT.md` and `ARCHITECTURE.md` before changing code. The stack is fixed: Python 3.12 with FastAPI, one process, served on localhost, whisper.cpp running locally against a downloaded model, ffmpeg for normalisation and segment extraction, originals never overwritten, SQLite beside the audio directory, one row per decoder segment, An optional model call over the transcript, with every sentence bound to segments. - Implement milestone by milestone from `MILESTONES.md`; keep each change reviewable and leave the application runnable at every commit. - Treat authentication, payments, encryption, imports, webhooks and destructive actions as high-risk boundaries when present. - Never invent cryptography or silently weaken a requirement to make a check pass. - Put every external service behind an interface with a deterministic fake for tests. - Add migrations and rollback or recovery notes for every persistent data change. - Log useful operational context without credentials, tokens, passwords or personal data. - Update documentation and run every check before completing a milestone. ## Known traps - Transcoding the upload in place to save disk means the one artefact that can settle an argument about what was said has been through a lossy encoder chosen by you, and nobody can tell afterwards which distortions came from the microphone and which from your ffmpeg flags. Keep the original file, work from a derived copy, and store the hash of both. - Feeding the whole file to the decoder and keeping only the text throws away the three numbers that tell you which sentences to distrust, and the published research says the invented sentences appear precisely in the non-vocal stretches you just fed it. Cut silence out before decoding, keep every confidence field on the row, and mark flagged segments in the interface rather than in a log. - Presenting automatic speaker labels in the same typography as corrected ones means a quote gets attributed to the wrong person in the notes and then in an email, and nobody can tell from the interface that the attribution was a guess by a pause detector. Keep assigned_by on every turn and render the two states differently, including in exports. - Asking for a summary as prose and storing it produces text that reads beautifully and cannot be checked, so the first time someone disputes an action item you are re-listening to forty minutes of audio to find out whether the model made it up. Require segment identifiers per sentence, validate them against the table, and drop any sentence that cannot be placed. - Expiring the audio while keeping the notes leaves a file of confident statements about a conversation that can no longer be checked against anything, which is the exact failure the citation work in the previous phase existed to prevent. Expire them together by default, and make keeping notes past the audio an explicit per-recording choice that stamps the notes as unverifiable. ===== MILESTONES.md ===== # Delivery milestones · timeOS Estimated effort: **multi-day** for the indie phases; the production-only milestones add the trust and operability layer. ## M1 · Audio in, audio kept A recording is imported or captured, hashed, and stored without being degraded on the way in. ### Steps 1. Keep the original and derive a working copy The original is the only thing that can settle a dispute about a note. Files: `app/ingest.py`, `app/audio.py` ```sh ffmpeg -i input.m4a -ac 1 -ar 16000 -c:a pcm_s16le work.wav ``` 2. Hash both files and store the command used So a strange transcript can be traced to a conversion rather than argued about. Files: `app/ingest.py`, `tests/test_ingest.py` 3. Gate transcription behind a notice timestamp The other people in the call are told before this runs, not after. Files: `app/models.py`, `app/routes/recordings.py` ### Done when - [ ] The original file is byte-identical after import - [ ] A working copy exists with its conversion command recorded - [ ] Transcription is refused until notice is recorded - [ ] Local capture shows a visible indicator throughout ### Watch out - Transcoding the upload in place to save disk means the one artefact that can settle an argument about what was said has been through a lossy encoder chosen by you, and nobody can tell afterwards which distortions came from the microphone and which from your ffmpeg flags. Keep the original file, work from a derived copy, and store the hash of both. ## M2 · A transcript that keeps its doubts whisper.cpp output is stored per segment with its confidence fields, and non-speech is never decoded. ### Steps 1. Detect and cut silence before decoding This is where speech models invent whole sentences. Files: `app/silence.py`, `tests/test_silence.py` ```sh ffmpeg -i work.wav -af silencedetect=noise=-35dB:d=1.0 -f null - ``` 2. Store every confidence field the decoder returns Text alone gives you nothing to sort the doubtful segments by. Files: `app/transcribe.py`, `app/models.py` 3. Flag repetition loops and low-confidence segments Visibly, in the transcript, not in a log file nobody opens. Files: `app/flags.py`, `tests/test_flags.py` ### Done when - [ ] Confidence fields are stored per segment - [ ] Room tone produces zero segments - [ ] A repetition loop is flagged, not stored as clean text - [ ] Flagged segments are visible and excluded from notes by default ### Watch out - Feeding the whole file to the decoder and keeping only the text throws away the three numbers that tell you which sentences to distrust, and the published research says the invented sentences appear precisely in the non-vocal stretches you just fed it. Cut silence out before decoding, keep every confidence field on the row, and mark flagged segments in the interface rather than in a log. ## M3 · Speakers are a guess you can correct Turns are proposed heuristically, labelled by a human, and the interface never pretends otherwise. ### Steps 1. Propose turns from pauses, not from a diarisation claim A pause detector is honest about being a pause detector. Files: `app/turns.py`, `tests/test_turns.py` 2. Record assigned_by on every turn The difference between a guess and a correction has to survive into the export. Files: `app/models.py` 3. Let one rename propagate across the recording Otherwise nobody corrects anything after the third turn. Files: `app/routes/turns.py` ### Done when - [ ] Heuristic and human labels render differently - [ ] One rename updates every matching turn - [ ] Turns can be split and merged by hand - [ ] Unconfirmed speaker names are marked in notes ### Watch out - Presenting automatic speaker labels in the same typography as corrected ones means a quote gets attributed to the wrong person in the notes and then in an email, and nobody can tell from the interface that the attribution was a guess by a pause detector. Keep assigned_by on every turn and render the two states differently, including in exports. ## M4 · Notes that cite the audio Every sentence of a generated note is bound to segments and plays back from the recording. ### Steps 1. Require segment identifiers in the structured output A claim without a location in the audio is not a claim. Files: `app/notes.py`, `app/schemas.py` 2. Validate identifiers against the segments table Models cite things that do not exist, cheerfully. Files: `app/notes.py`, `tests/test_notes.py` 3. Wire every sentence to a seek in the player One click from the claim to the audio, or the citation is decoration. Files: `app/templates/recording.html`, `app/static/player.js` ### Done when - [ ] Every note sentence carries a valid segment identifier - [ ] Clicking a sentence seeks the audio to its segment - [ ] A fabricated citation stores nothing - [ ] Action items without a segment are refused ### Watch out - Asking for a summary as prose and storing it produces text that reads beautifully and cannot be checked, so the first time someone disputes an action item you are re-listening to forty minutes of audio to find out whether the model made it up. Require segment identifiers per sentence, validate them against the table, and drop any sentence that cannot be placed. ## M5 · Expiry, export and a bill that cannot run away (production only) Recordings expire on a schedule, leave in a readable format, and the optional model spend has a ceiling. ### Steps 1. Expire audio and notes together Notes that outlive their evidence are the thing this build avoids. Files: `app/retention.py`, `tests/test_retention.py` 2. Export a folder that works without the app Audio, Markdown transcript, notes with citations, segments as JSON. Files: `app/export.py` 3. Cap optional model spend and degrade to transcript The transcription is local and free, so the fallback is a good product. Files: `app/cost.py`, `tests/test_cost.py` ### Done when - [ ] Retention deletes audio and notes together and logs it - [ ] Surviving notes are stamped unverifiable - [ ] An exported folder is readable without the app - [ ] The cost ceiling stops notes, not transcription ### Watch out - Expiring the audio while keeping the notes leaves a file of confident statements about a conversation that can no longer be checked against anything, which is the exact failure the citation work in the previous phase existed to prevent. Expire them together by default, and make keeping notes past the audio an explicit per-recording choice that stamps the notes as unverifiable. ===== OPERATIONS.md ===== # Operations · timeOS ## Backup The database and the audio directory together, because either alone is useless. A backup of notes without audio is a set of claims nobody can check. ## Restore Restore both, then verify that a sample of note sentences still resolve to existing segments and that clicking one seeks correctly. A restore that leaves dangling citations has lost the property the build is for. Do a restore drill before the first real user, and write the date here when it passes. ## Monitoring Decode time per audio minute, the proportion of segments flagged, and the share of note sentences dropped for invalid citations. A rising drop rate usually means the transcript quality fell, not that the model got worse at citing. ## Incident checklist The incident here is someone acting on a note that the audio does not support. The segment citations and the retained originals are what let you find out what was actually said, which is why they expire together. 1. Contain the issue without destroying evidence or user data. 2. Record the timeline and affected scope. 3. Rotate exposed secrets and revoke compromised sessions or credentials. 4. Restore from a verified backup when needed. 5. Document the root cause, the remediation and the regression test. ## Release gate - [ ] Room tone is confirmed to produce zero segments - [ ] Every note sentence is confirmed to carry a valid segment identifier - [ ] Heuristic speaker labels are confirmed visibly distinct from corrected ones - [ ] Retention is confirmed to delete audio and notes in one transaction - [ ] The cost ceiling is confirmed to stop notes and leave transcription working ## Launch constraint Do not market omitted timeOS capabilities as implemented. The non-goals in `PRODUCT.md` remain user-visible limitations until they are deliberately delivered. ===== .env.example ===== # Copy to .env and fill in. Never commit .env; this file documents it. # Required. Recordings, segments, turns, notes and citations. DATABASE_PATH=./data/meetings.db # Required. Originals and working copies. Sized for the retention you chose. AUDIO_DIR=./data/audio # Required. The local model file. Its identifier is stored with every segment. WHISPER_MODEL_PATH=./models/ggml-medium.bin # Required. Segments above this are flagged and excluded from note generation. NO_SPEECH_THRESHOLD=0.6 # Required. Passed to ffmpeg silencedetect. Tune it against your own recordings. SILENCE_DB=-35dB # Required. Audio and notes are deleted together at this age unless overridden per recording. RETENTION_DAYS=90 # Optional · secret. Optional. Only note generation uses it; transcription never leaves the machine. NOTES_MODEL_KEY=sk-...redacted # Required. Note generation stops here for the month. Transcription and playback continue. MONTHLY_COST_CEILING=3.00
README.md
# timeOS · indie build A local meeting recorder whose hard problems are keeping the audio and the transcript joined so a claim can be checked, refusing to decode silence where speech models invent sentences, and being honest that speaker labels are guesses a human corrects. Estimated effort: **multi-day**. Work `BUILD_PLAN.md` top to bottom · every phase ends in a check that has to pass before the next one starts. ## Stack | Part | Choice | Why | | --- | --- | --- | | Runtime | Python 3.12 with FastAPI, one process, served on localhost | the recording and the audio never leave the machine that made them | | Transcription | whisper.cpp running locally against a downloaded model | a meeting is the most private audio most people own and it should not be uploaded to transcribe it | | Audio | ffmpeg for normalisation and segment extraction, originals never overwritten | the original file is the evidence behind every note | | Storage | SQLite beside the audio directory, one row per decoder segment | per-segment confidence is the only signal you get about a bad transcript | | Notes | An optional model call over the transcript, with every sentence bound to segments | a summary you cannot trace back is a rumour with timestamps | ## Before you start Have every one of these ready. The plan assumes them from step one. - [ ] **Python 3.12** · free - Why: Runs the application and the ingest pipeline. - Get it: python.org or your version manager. - Verify: python3 --version prints 3.12 or higher - [ ] **ffmpeg with silencedetect** · free - Why: Handles conversion and the silence detection that keeps non-speech out of the decoder. - Get it: Your package manager, or a static build. - Verify: ffmpeg -filters | grep silencedetect returns a row - [ ] **whisper.cpp with a downloaded model file** · free, plus the disk for the model - Why: Transcription runs locally, which is the whole reason this build exists rather than uploading a meeting recording. - Get it: Build it and fetch a model; start with a medium-sized one and measure speed on your own machine before choosing. - Verify: A one-minute sample transcribes and prints segment timings - [ ] **A real multi-speaker recording with cross-talk and silence** · free - Why: Every failure this build handles appears in a real meeting and in none of the clean test clips people usually try first. - Get it: Record a genuine conversation with everyone's agreement, including a long pause and two people talking over each other. - Verify: It contains at least one silence over ten seconds and one overlap - [ ] **How the room is told a recording is happening** · free - Why: Deciding this after the build means deciding it from whatever the interface happened to make easy. - Get it: Write down the wording you will use, when it is shown, and what you will check about local law. - Verify: The wording appears in the README and the timestamp gate is asserted in a test ## Quick start ```sh ffmpeg -i input.m4a -ac 1 -ar 16000 -c:a pcm_s16le work.wav ``` Then copy `.env.example` to `.env` and fill in the values it documents. ## Honest limits This build deliberately does not replace: - A meeting bot that joins calls on your behalf. - Real-time transcription during a call. - Voice prints and speaker identification across recordings. - Calendar and CRM synchronisation. - live multi-speaker accuracy - calendar and CRM integrations - cross-call team analytics - meeting-bot auto-join If one of those is essential to you, that is the reason to keep paying for timeOS, and the README should say so rather than pretend.
BRIEF.md
# Build brief · timeOS The one-shot brief this plan expands. `BUILD_PLAN.md` (or `MILESTONES.md`) is the same sequence broken into steps and checks; where the two disagree, the plan wins. Build me a private meeting recorder that transcribes on your own machine and turns a call into notes you can check against the audio to replace timeOS. Build it in phases, in the order below. Do not write the whole thing in one pass. Finish a phase, run its "Done when" check, fix what fails, and only then start the next phase. ### Rule zero A machine transcript is not a record of what was said, and a summary of a transcript is two steps away from it. Design for that or you will ship confident fiction. The 2024 FAccT paper 'Careless Whisper' measured this directly: roughly 1 percent of audio transcriptions contained entire hallucinated phrases or sentences which did not exist in any form in the underlying audio, and 38 percent of those hallucinations included explicit harms such as invented violence, false associations or implied authority. The hallucinations clustered on speakers with longer non-vocal stretches, which is a common symptom of aphasia, so the people most likely to be misquoted by this build are the ones least able to argue with it. So: the audio is kept for exactly as long as the notes are, every sentence in a generated note carries the segment identifiers it came from, and one click plays that second of audio. A note sentence with no source segment is not stored at all. Silence and non-speech are detected and dropped rather than decoded, because that is where the invented sentences appear. And tell the room. Recording a conversation is something the other people in it are told about before it starts, and what your jurisdiction requires on top of that is yours to look up. Nothing in this build joins a call by itself, sits in one quietly, or records without a visible indicator. ### Stack (fixed, do not substitute) - Runtime: Python 3.12 with FastAPI, one process, served on localhost · the recording and the audio never leave the machine that made them - Transcription: whisper.cpp running locally against a downloaded model · a meeting is the most private audio most people own and it should not be uploaded to transcribe it - Audio: ffmpeg for normalisation and segment extraction, originals never overwritten · the original file is the evidence behind every note - Storage: SQLite beside the audio directory, one row per decoder segment · per-segment confidence is the only signal you get about a bad transcript - Notes: An optional model call over the transcript, with every sentence bound to segments · a summary you cannot trace back is a rumour with timestamps ### Data model (create this before Phase 1) - recordings(id, title, started_at, ended_at, source, audio_path, sha256, duration_ms, sample_rate, notice_given_at) - segments(id, recording_id, ordinal, start_ms, end_ms, text, avg_logprob, no_speech_prob, compression_ratio, model_id, flagged_reason) · the decoder's own confidence fields are kept, not discarded - turns(id, recording_id, start_ms, end_ms, speaker_label, assigned_by) · assigned_by is heuristic or human, and the interface shows which - notes(id, recording_id, kind, body_md, model_id, prompt_version, created_at) · kind is summary, decisions or actions - note_claims(id, note_id, ordinal, text, segment_ids_json) · one row per sentence of a note and the segments behind it - actions(id, recording_id, text, owner_label, due_on, segment_id, state) · an action item with no segment is refused on insert - retention(recording_id, delete_audio_after_days, delete_all_after_days) · audio and notes expire together by default, because one without the other is unverifiable - There is no column anywhere holding a note sentence without a source segment. That constraint is what makes the rest of this honest. ### Phase 1 · Audio in, audio kept Build: Accept an audio or video file, extract audio with ffmpeg to a known sample rate into a working copy, and keep the original untouched alongside it. Hash both. Record a notice-given timestamp that the user has to set before a recording can be transcribed, and show a visible indicator whenever local capture is running. Trap: Transcoding the upload in place to save disk means the one artefact that can settle an argument about what was said has been through a lossy encoder chosen by you, and nobody can tell afterwards which distortions came from the microphone and which from your ffmpeg flags. Keep the original file, work from a derived copy, and store the hash of both. Done when: Importing a file stores the original unmodified, verified by comparing its hash before and after. A working copy exists at the declared sample rate and the ffmpeg command used is recorded with the row. A recording cannot be transcribed until a notice-given timestamp is set, tested by attempting it. Local capture shows an indicator for its whole duration, checked by review. Do not build yet: No transcription, speakers, notes or export yet. ### Phase 2 · A transcript that keeps its doubts Build: Run whisper.cpp with word or segment timestamps and capture avg_logprob, no_speech_prob and compression_ratio for every segment. Detect silence with ffmpeg before decoding and skip those spans entirely. Flag any segment whose no_speech_prob is above your threshold, whose compression ratio indicates a repetition loop, or whose text repeats the previous segment. Trap: Feeding the whole file to the decoder and keeping only the text throws away the three numbers that tell you which sentences to distrust, and the published research says the invented sentences appear precisely in the non-vocal stretches you just fed it. Cut silence out before decoding, keep every confidence field on the row, and mark flagged segments in the interface rather than in a log. Done when: Every segment row carries avg_logprob, no_speech_prob and compression_ratio from the decoder, asserted on a real file. A file consisting of thirty seconds of room tone produces no segments, tested directly. A repeated-phrase loop is flagged rather than stored as clean text, tested with a fixture containing a long silence. Flagged segments render visibly differently in the transcript view and are excluded from note generation unless the user overrides per segment. Do not build yet: No speaker labels, notes, search or export yet. ### Phase 3 · Speakers are a guess you can correct Build: Segment the recording into turns from pauses and energy changes, propose speaker labels, and mark every one as heuristic. Let the user rename a speaker once and have it apply across the whole recording. Show heuristic and human-assigned labels differently everywhere they appear. Trap: Presenting automatic speaker labels in the same typography as corrected ones means a quote gets attributed to the wrong person in the notes and then in an email, and nobody can tell from the interface that the attribution was a guess by a pause detector. Keep assigned_by on every turn and render the two states differently, including in exports. Done when: Every turn records whether its label came from the heuristic or from a person, and the two render differently in the transcript and in exports, checked by review. Renaming a speaker updates every turn with that label in one action, tested. A turn boundary can be split or merged by hand and the segment timings follow. No note or action item quotes a speaker name that is still heuristic without marking it as unconfirmed. Do not build yet: No notes, search or export yet. ### Phase 4 · Notes that cite the audio Build: Generate a summary, a decisions list and an action list from the flagged-clean transcript, and require the model to return each sentence with the segment identifiers it came from. Reject any sentence whose segment identifiers do not exist. Render every note sentence with a timestamp that seeks the audio player to that moment. Trap: Asking for a summary as prose and storing it produces text that reads beautifully and cannot be checked, so the first time someone disputes an action item you are re-listening to forty minutes of audio to find out whether the model made it up. Require segment identifiers per sentence, validate them against the table, and drop any sentence that cannot be placed. Done when: Every stored note sentence has at least one valid segment identifier, enforced by a database constraint and tested with a fabricated response. Clicking any note sentence seeks the audio player to the start of its first segment, tested end to end. A model response citing a segment identifier that does not exist stores nothing and is reported to the user. Action items with no source segment are refused on insert. Do not build yet: No retention, cost ceiling or export yet. ### Phase 5 · Expiry, export and a bill that cannot run away Build: Delete audio and notes together on the configured retention, and log the deletion. Export a recording as a folder containing the audio, a Markdown transcript with timestamps, the notes with their citations, and the raw segment table as JSON. Track any model spend against a monthly ceiling and fall back to transcript-only when it is reached. Trap: Expiring the audio while keeping the notes leaves a file of confident statements about a conversation that can no longer be checked against anything, which is the exact failure the citation work in the previous phase existed to prevent. Expire them together by default, and make keeping notes past the audio an explicit per-recording choice that stamps the notes as unverifiable. Done when: Retention deletes the audio and its notes in the same transaction by default and writes a deletion log entry, tested by advancing the clock. Keeping notes beyond the audio requires an explicit per-recording setting and stamps those notes as unverifiable in the interface and in exports. An exported folder opens and plays without this application present, tested on a second machine. Reaching the monthly model ceiling disables note generation and leaves transcription and playback working. Do not build yet: Nothing after this. ### Out of scope (and why) - A bot that joins calendar meetings by itself, which is the largest part of the paid product and depends on each platform's published participation rules rather than on your code. - Real-time transcription during the call, which changes the decoder configuration, the latency budget and the hardware requirement all at once. - Speaker identification across recordings, which means storing voice prints of colleagues and is a different consent conversation entirely. - Calendar and CRM synchronisation, which is the integration catalogue that the subscription is really selling. ### README must contain - The published hallucination findings for local speech models, with the study and the date, before any setup instructions - That silence is cut before decoding, and why that is where invented sentences come from - That every note sentence is stored with the audio segments it came from, and that unsourced sentences are dropped - That speaker labels are heuristic until a person corrects them, and how the two are distinguished in exports - That audio and notes expire together by default, and how to override that and what the override marks
AGENTS.md
# Agent instructions · timeOS indie build - Read `README.md` and `BUILD_PLAN.md` before writing code. The stack is fixed: Python 3.12 with FastAPI, one process, served on localhost, whisper.cpp running locally against a downloaded model, ffmpeg for normalisation and segment extraction, originals never overwritten, SQLite beside the audio directory, one row per decoder segment, An optional model call over the transcript, with every sentence bound to segments. Do not substitute. - Work one phase at a time, in order. Do not start a phase until every "Done when" item of the previous one passes. - Prefer the fewest moving parts that satisfy the step. No frameworks, services or dependencies the plan does not name. - Secrets live in `.env`, never in source or logs. Keep `.env.example` current when a variable is introduced. - Do not invent cryptography, security guarantees, APIs or compliance claims. - Add a focused test for every destructive, security-sensitive or data-loss path the plan names. - Run the project checks before declaring a phase complete, and record any deliberate shortcut in the README under "Tradeoffs". ## Known traps - Transcoding the upload in place to save disk means the one artefact that can settle an argument about what was said has been through a lossy encoder chosen by you, and nobody can tell afterwards which distortions came from the microphone and which from your ffmpeg flags. Keep the original file, work from a derived copy, and store the hash of both. - Feeding the whole file to the decoder and keeping only the text throws away the three numbers that tell you which sentences to distrust, and the published research says the invented sentences appear precisely in the non-vocal stretches you just fed it. Cut silence out before decoding, keep every confidence field on the row, and mark flagged segments in the interface rather than in a log. - Presenting automatic speaker labels in the same typography as corrected ones means a quote gets attributed to the wrong person in the notes and then in an email, and nobody can tell from the interface that the attribution was a guess by a pause detector. Keep assigned_by on every turn and render the two states differently, including in exports. - Asking for a summary as prose and storing it produces text that reads beautifully and cannot be checked, so the first time someone disputes an action item you are re-listening to forty minutes of audio to find out whether the model made it up. Require segment identifiers per sentence, validate them against the table, and drop any sentence that cannot be placed.
BUILD_PLAN.md
# Build plan · timeOS A local meeting recorder whose hard problems are keeping the audio and the transcript joined so a claim can be checked, refusing to decode silence where speech models invent sentences, and being honest that speaker labels are guesses a human corrects. Phases are in dependency order. Each ends in a "Done when" list; treat an unticked item as a blocker, not a note. ## Phase 1 · Audio in, audio kept A recording is imported or captured, hashed, and stored without being degraded on the way in. ### Steps 1. Keep the original and derive a working copy The original is the only thing that can settle a dispute about a note. Files: `app/ingest.py`, `app/audio.py` ```sh ffmpeg -i input.m4a -ac 1 -ar 16000 -c:a pcm_s16le work.wav ``` 2. Hash both files and store the command used So a strange transcript can be traced to a conversion rather than argued about. Files: `app/ingest.py`, `tests/test_ingest.py` 3. Gate transcription behind a notice timestamp The other people in the call are told before this runs, not after. Files: `app/models.py`, `app/routes/recordings.py` ### Done when - [ ] The original file is byte-identical after import - [ ] A working copy exists with its conversion command recorded - [ ] Transcription is refused until notice is recorded - [ ] Local capture shows a visible indicator throughout ### Watch out - Transcoding the upload in place to save disk means the one artefact that can settle an argument about what was said has been through a lossy encoder chosen by you, and nobody can tell afterwards which distortions came from the microphone and which from your ffmpeg flags. Keep the original file, work from a derived copy, and store the hash of both. ## Phase 2 · A transcript that keeps its doubts whisper.cpp output is stored per segment with its confidence fields, and non-speech is never decoded. ### Steps 1. Detect and cut silence before decoding This is where speech models invent whole sentences. Files: `app/silence.py`, `tests/test_silence.py` ```sh ffmpeg -i work.wav -af silencedetect=noise=-35dB:d=1.0 -f null - ``` 2. Store every confidence field the decoder returns Text alone gives you nothing to sort the doubtful segments by. Files: `app/transcribe.py`, `app/models.py` 3. Flag repetition loops and low-confidence segments Visibly, in the transcript, not in a log file nobody opens. Files: `app/flags.py`, `tests/test_flags.py` ### Done when - [ ] Confidence fields are stored per segment - [ ] Room tone produces zero segments - [ ] A repetition loop is flagged, not stored as clean text - [ ] Flagged segments are visible and excluded from notes by default ### Watch out - Feeding the whole file to the decoder and keeping only the text throws away the three numbers that tell you which sentences to distrust, and the published research says the invented sentences appear precisely in the non-vocal stretches you just fed it. Cut silence out before decoding, keep every confidence field on the row, and mark flagged segments in the interface rather than in a log. ## Phase 3 · Speakers are a guess you can correct Turns are proposed heuristically, labelled by a human, and the interface never pretends otherwise. ### Steps 1. Propose turns from pauses, not from a diarisation claim A pause detector is honest about being a pause detector. Files: `app/turns.py`, `tests/test_turns.py` 2. Record assigned_by on every turn The difference between a guess and a correction has to survive into the export. Files: `app/models.py` 3. Let one rename propagate across the recording Otherwise nobody corrects anything after the third turn. Files: `app/routes/turns.py` ### Done when - [ ] Heuristic and human labels render differently - [ ] One rename updates every matching turn - [ ] Turns can be split and merged by hand - [ ] Unconfirmed speaker names are marked in notes ### Watch out - Presenting automatic speaker labels in the same typography as corrected ones means a quote gets attributed to the wrong person in the notes and then in an email, and nobody can tell from the interface that the attribution was a guess by a pause detector. Keep assigned_by on every turn and render the two states differently, including in exports. ## Phase 4 · Notes that cite the audio Every sentence of a generated note is bound to segments and plays back from the recording. ### Steps 1. Require segment identifiers in the structured output A claim without a location in the audio is not a claim. Files: `app/notes.py`, `app/schemas.py` 2. Validate identifiers against the segments table Models cite things that do not exist, cheerfully. Files: `app/notes.py`, `tests/test_notes.py` 3. Wire every sentence to a seek in the player One click from the claim to the audio, or the citation is decoration. Files: `app/templates/recording.html`, `app/static/player.js` ### Done when - [ ] Every note sentence carries a valid segment identifier - [ ] Clicking a sentence seeks the audio to its segment - [ ] A fabricated citation stores nothing - [ ] Action items without a segment are refused ### Watch out - Asking for a summary as prose and storing it produces text that reads beautifully and cannot be checked, so the first time someone disputes an action item you are re-listening to forty minutes of audio to find out whether the model made it up. Require segment identifiers per sentence, validate them against the table, and drop any sentence that cannot be placed. ## Not in this build - A meeting bot that joins calls on your behalf. - Real-time transcription during a call. - Voice prints and speaker identification across recordings. - Calendar and CRM synchronisation. ## After v1, if you want it - A search across recordings that ranks by segment confidence as well as relevance - A weekly digest of action items whose source segments were flagged, which is where the errors hide - A second decoder pass on flagged segments only, comparing the two outputs before trusting either
.env.example
# Copy to .env and fill in. Never commit .env; this file documents it. # Required. Recordings, segments, turns, notes and citations. DATABASE_PATH=./data/meetings.db # Required. Originals and working copies. Sized for the retention you chose. AUDIO_DIR=./data/audio # Required. The local model file. Its identifier is stored with every segment. WHISPER_MODEL_PATH=./models/ggml-medium.bin # Required. Segments above this are flagged and excluded from note generation. NO_SPEECH_THRESHOLD=0.6 # Required. Passed to ffmpeg silencedetect. Tune it against your own recordings. SILENCE_DB=-35dB # Required. Audio and notes are deleted together at this age unless overridden per recording. RETENTION_DAYS=90 # Optional · secret. Optional. Only note generation uses it; transcription never leaves the machine. NOTES_MODEL_KEY=sk-...redacted # Required. Note generation stops here for the month. Transcription and playback continue. MONTHLY_COST_CEILING=3.00
PRODUCT.md
# timeOS · product brief ## Problem The visible meeting productivity loop is buildable, but a credible replacement needs more than the first screen. timeOS earns its keep through capture, integrations, reliability, so expect a weekend or multi-day build and a narrower personal scope. ## Product outcome A meeting archive where every generated sentence can be played back from the audio it came from, the decoder never runs on silence, speaker labels admit to being guesses, and the notes never outlive the recording that would settle an argument about them. ## Target user A builder who needs a maintainable product foundation, not a one-off demo. ## Required capabilities - Python 3.12 - FFmpeg - local Whisper model or optional API key in .env ## Explicit non-goals for v1 - A meeting bot that joins calls on your behalf. - Real-time transcription during a call. - Voice prints and speaker identification across recordings. - Calendar and CRM synchronisation. - live multi-speaker accuracy - calendar and CRM integrations - cross-call team analytics - meeting-bot auto-join ## Success criteria - Room tone is confirmed to produce zero segments - Every note sentence is confirmed to carry a valid segment identifier - Heuristic speaker labels are confirmed visibly distinct from corrected ones - Retention is confirmed to delete audio and notes in one transaction - The cost ceiling is confirmed to stop notes and leave transcription working
BRIEF.md
# Build brief · timeOS The one-shot brief this plan expands. `BUILD_PLAN.md` (or `MILESTONES.md`) is the same sequence broken into steps and checks; where the two disagree, the plan wins. Build me a private meeting recorder that transcribes on your own machine and turns a call into notes you can check against the audio to replace timeOS. Build it in phases, in the order below. Do not write the whole thing in one pass. Finish a phase, run its "Done when" check, fix what fails, and only then start the next phase. ### Rule zero A machine transcript is not a record of what was said, and a summary of a transcript is two steps away from it. Design for that or you will ship confident fiction. The 2024 FAccT paper 'Careless Whisper' measured this directly: roughly 1 percent of audio transcriptions contained entire hallucinated phrases or sentences which did not exist in any form in the underlying audio, and 38 percent of those hallucinations included explicit harms such as invented violence, false associations or implied authority. The hallucinations clustered on speakers with longer non-vocal stretches, which is a common symptom of aphasia, so the people most likely to be misquoted by this build are the ones least able to argue with it. So: the audio is kept for exactly as long as the notes are, every sentence in a generated note carries the segment identifiers it came from, and one click plays that second of audio. A note sentence with no source segment is not stored at all. Silence and non-speech are detected and dropped rather than decoded, because that is where the invented sentences appear. And tell the room. Recording a conversation is something the other people in it are told about before it starts, and what your jurisdiction requires on top of that is yours to look up. Nothing in this build joins a call by itself, sits in one quietly, or records without a visible indicator. ### Stack (fixed, do not substitute) - Runtime: Python 3.12 with FastAPI, one process, served on localhost · the recording and the audio never leave the machine that made them - Transcription: whisper.cpp running locally against a downloaded model · a meeting is the most private audio most people own and it should not be uploaded to transcribe it - Audio: ffmpeg for normalisation and segment extraction, originals never overwritten · the original file is the evidence behind every note - Storage: SQLite beside the audio directory, one row per decoder segment · per-segment confidence is the only signal you get about a bad transcript - Notes: An optional model call over the transcript, with every sentence bound to segments · a summary you cannot trace back is a rumour with timestamps ### Data model (create this before Phase 1) - recordings(id, title, started_at, ended_at, source, audio_path, sha256, duration_ms, sample_rate, notice_given_at) - segments(id, recording_id, ordinal, start_ms, end_ms, text, avg_logprob, no_speech_prob, compression_ratio, model_id, flagged_reason) · the decoder's own confidence fields are kept, not discarded - turns(id, recording_id, start_ms, end_ms, speaker_label, assigned_by) · assigned_by is heuristic or human, and the interface shows which - notes(id, recording_id, kind, body_md, model_id, prompt_version, created_at) · kind is summary, decisions or actions - note_claims(id, note_id, ordinal, text, segment_ids_json) · one row per sentence of a note and the segments behind it - actions(id, recording_id, text, owner_label, due_on, segment_id, state) · an action item with no segment is refused on insert - retention(recording_id, delete_audio_after_days, delete_all_after_days) · audio and notes expire together by default, because one without the other is unverifiable - There is no column anywhere holding a note sentence without a source segment. That constraint is what makes the rest of this honest. ### Phase 1 · Audio in, audio kept Build: Accept an audio or video file, extract audio with ffmpeg to a known sample rate into a working copy, and keep the original untouched alongside it. Hash both. Record a notice-given timestamp that the user has to set before a recording can be transcribed, and show a visible indicator whenever local capture is running. Trap: Transcoding the upload in place to save disk means the one artefact that can settle an argument about what was said has been through a lossy encoder chosen by you, and nobody can tell afterwards which distortions came from the microphone and which from your ffmpeg flags. Keep the original file, work from a derived copy, and store the hash of both. Done when: Importing a file stores the original unmodified, verified by comparing its hash before and after. A working copy exists at the declared sample rate and the ffmpeg command used is recorded with the row. A recording cannot be transcribed until a notice-given timestamp is set, tested by attempting it. Local capture shows an indicator for its whole duration, checked by review. Do not build yet: No transcription, speakers, notes or export yet. ### Phase 2 · A transcript that keeps its doubts Build: Run whisper.cpp with word or segment timestamps and capture avg_logprob, no_speech_prob and compression_ratio for every segment. Detect silence with ffmpeg before decoding and skip those spans entirely. Flag any segment whose no_speech_prob is above your threshold, whose compression ratio indicates a repetition loop, or whose text repeats the previous segment. Trap: Feeding the whole file to the decoder and keeping only the text throws away the three numbers that tell you which sentences to distrust, and the published research says the invented sentences appear precisely in the non-vocal stretches you just fed it. Cut silence out before decoding, keep every confidence field on the row, and mark flagged segments in the interface rather than in a log. Done when: Every segment row carries avg_logprob, no_speech_prob and compression_ratio from the decoder, asserted on a real file. A file consisting of thirty seconds of room tone produces no segments, tested directly. A repeated-phrase loop is flagged rather than stored as clean text, tested with a fixture containing a long silence. Flagged segments render visibly differently in the transcript view and are excluded from note generation unless the user overrides per segment. Do not build yet: No speaker labels, notes, search or export yet. ### Phase 3 · Speakers are a guess you can correct Build: Segment the recording into turns from pauses and energy changes, propose speaker labels, and mark every one as heuristic. Let the user rename a speaker once and have it apply across the whole recording. Show heuristic and human-assigned labels differently everywhere they appear. Trap: Presenting automatic speaker labels in the same typography as corrected ones means a quote gets attributed to the wrong person in the notes and then in an email, and nobody can tell from the interface that the attribution was a guess by a pause detector. Keep assigned_by on every turn and render the two states differently, including in exports. Done when: Every turn records whether its label came from the heuristic or from a person, and the two render differently in the transcript and in exports, checked by review. Renaming a speaker updates every turn with that label in one action, tested. A turn boundary can be split or merged by hand and the segment timings follow. No note or action item quotes a speaker name that is still heuristic without marking it as unconfirmed. Do not build yet: No notes, search or export yet. ### Phase 4 · Notes that cite the audio Build: Generate a summary, a decisions list and an action list from the flagged-clean transcript, and require the model to return each sentence with the segment identifiers it came from. Reject any sentence whose segment identifiers do not exist. Render every note sentence with a timestamp that seeks the audio player to that moment. Trap: Asking for a summary as prose and storing it produces text that reads beautifully and cannot be checked, so the first time someone disputes an action item you are re-listening to forty minutes of audio to find out whether the model made it up. Require segment identifiers per sentence, validate them against the table, and drop any sentence that cannot be placed. Done when: Every stored note sentence has at least one valid segment identifier, enforced by a database constraint and tested with a fabricated response. Clicking any note sentence seeks the audio player to the start of its first segment, tested end to end. A model response citing a segment identifier that does not exist stores nothing and is reported to the user. Action items with no source segment are refused on insert. Do not build yet: No retention, cost ceiling or export yet. ### Phase 5 · Expiry, export and a bill that cannot run away Build: Delete audio and notes together on the configured retention, and log the deletion. Export a recording as a folder containing the audio, a Markdown transcript with timestamps, the notes with their citations, and the raw segment table as JSON. Track any model spend against a monthly ceiling and fall back to transcript-only when it is reached. Trap: Expiring the audio while keeping the notes leaves a file of confident statements about a conversation that can no longer be checked against anything, which is the exact failure the citation work in the previous phase existed to prevent. Expire them together by default, and make keeping notes past the audio an explicit per-recording choice that stamps the notes as unverifiable. Done when: Retention deletes the audio and its notes in the same transaction by default and writes a deletion log entry, tested by advancing the clock. Keeping notes beyond the audio requires an explicit per-recording setting and stamps those notes as unverifiable in the interface and in exports. An exported folder opens and plays without this application present, tested on a second machine. Reaching the monthly model ceiling disables note generation and leaves transcription and playback working. Do not build yet: Nothing after this. ### Out of scope (and why) - A bot that joins calendar meetings by itself, which is the largest part of the paid product and depends on each platform's published participation rules rather than on your code. - Real-time transcription during the call, which changes the decoder configuration, the latency budget and the hardware requirement all at once. - Speaker identification across recordings, which means storing voice prints of colleagues and is a different consent conversation entirely. - Calendar and CRM synchronisation, which is the integration catalogue that the subscription is really selling. ### README must contain - The published hallucination findings for local speech models, with the study and the date, before any setup instructions - That silence is cut before decoding, and why that is where invented sentences come from - That every note sentence is stored with the audio segments it came from, and that unsourced sentences are dropped - That speaker labels are heuristic until a person corrects them, and how the two are distinguished in exports - That audio and notes expire together by default, and how to override that and what the override marks
ARCHITECTURE.md
# Architecture · timeOS ## Stack | Part | Choice | Why | | --- | --- | --- | | Runtime | Python 3.12 with FastAPI, one process, served on localhost | the recording and the audio never leave the machine that made them | | Transcription | whisper.cpp running locally against a downloaded model | a meeting is the most private audio most people own and it should not be uploaded to transcribe it | | Audio | ffmpeg for normalisation and segment extraction, originals never overwritten | the original file is the evidence behind every note | | Storage | SQLite beside the audio directory, one row per decoder segment | per-segment confidence is the only signal you get about a bad transcript | | Notes | An optional model call over the transcript, with every sentence bound to segments | a summary you cannot trace back is a rumour with timestamps | ## Modules Each module has one owner concern and a documented way to replace it. | Module | Owns | How to replace it | | --- | --- | --- | | Ingest | Originals, working copies, hashes and the notice gate. | Any capture source, provided the original file survives untouched. | | Transcribe | Silence detection, decoding and per-segment confidence. | Any local engine that returns timings and confidence fields. | | Turns | Heuristic turn boundaries and human labels. | A real diarisation model, provided assigned_by still distinguishes the two. | | Notes | Structured generation and citation validation. | Any model, provided unsourced sentences are still refused. | | Retention | Joint expiry, deletion logging and export. | Not swappable for note-only retention without the unverifiable stamp. | ## Configuration Every runtime setting is an environment variable documented in `.env.example`, validated at startup, with a safe local default wherever one exists. - `DATABASE_PATH` · required · Recordings, segments, turns, notes and citations. - `AUDIO_DIR` · required · Originals and working copies. Sized for the retention you chose. - `WHISPER_MODEL_PATH` · required · The local model file. Its identifier is stored with every segment. - `NO_SPEECH_THRESHOLD` · required · Segments above this are flagged and excluded from note generation. - `SILENCE_DB` · required · Passed to ffmpeg silencedetect. Tune it against your own recordings. - `RETENTION_DAYS` · required · Audio and notes are deleted together at this age unless overridden per recording. - `NOTES_MODEL_KEY` · optional, secret · Optional. Only note generation uses it; transcription never leaves the machine. - `MONTHLY_COST_CEILING` · required · Note generation stops here for the month. Transcription and playback continue. ## Production baseline - Security: least privilege, input validation at every boundary, secret redaction in logs, rate limits on abuse-prone paths, no invented security primitives. - Data: explicit schema and migrations, transactional writes where integrity matters, backup and restore procedures that have been exercised. - Integrations: adapters around third-party providers, idempotent webhook or job processing, bounded retries, timeouts. - Observability: structured logs with request or operation ids, an error-tracking hook, and health and readiness checks where a server exists. - Quality: unit tests for domain rules, integration tests at module boundaries, one end-to-end test of the critical path. ## Decision records For each dependency in the stack table, keep a short note: why it was chosen, its failure mode, and how it is replaced. Do not add infrastructure until a requirement in `PRODUCT.md` justifies it.
AGENTS.md
# Agent instructions · timeOS product build - Read `PRODUCT.md` and `ARCHITECTURE.md` before changing code. The stack is fixed: Python 3.12 with FastAPI, one process, served on localhost, whisper.cpp running locally against a downloaded model, ffmpeg for normalisation and segment extraction, originals never overwritten, SQLite beside the audio directory, one row per decoder segment, An optional model call over the transcript, with every sentence bound to segments. - Implement milestone by milestone from `MILESTONES.md`; keep each change reviewable and leave the application runnable at every commit. - Treat authentication, payments, encryption, imports, webhooks and destructive actions as high-risk boundaries when present. - Never invent cryptography or silently weaken a requirement to make a check pass. - Put every external service behind an interface with a deterministic fake for tests. - Add migrations and rollback or recovery notes for every persistent data change. - Log useful operational context without credentials, tokens, passwords or personal data. - Update documentation and run every check before completing a milestone. ## Known traps - Transcoding the upload in place to save disk means the one artefact that can settle an argument about what was said has been through a lossy encoder chosen by you, and nobody can tell afterwards which distortions came from the microphone and which from your ffmpeg flags. Keep the original file, work from a derived copy, and store the hash of both. - Feeding the whole file to the decoder and keeping only the text throws away the three numbers that tell you which sentences to distrust, and the published research says the invented sentences appear precisely in the non-vocal stretches you just fed it. Cut silence out before decoding, keep every confidence field on the row, and mark flagged segments in the interface rather than in a log. - Presenting automatic speaker labels in the same typography as corrected ones means a quote gets attributed to the wrong person in the notes and then in an email, and nobody can tell from the interface that the attribution was a guess by a pause detector. Keep assigned_by on every turn and render the two states differently, including in exports. - Asking for a summary as prose and storing it produces text that reads beautifully and cannot be checked, so the first time someone disputes an action item you are re-listening to forty minutes of audio to find out whether the model made it up. Require segment identifiers per sentence, validate them against the table, and drop any sentence that cannot be placed. - Expiring the audio while keeping the notes leaves a file of confident statements about a conversation that can no longer be checked against anything, which is the exact failure the citation work in the previous phase existed to prevent. Expire them together by default, and make keeping notes past the audio an explicit per-recording choice that stamps the notes as unverifiable.
MILESTONES.md
# Delivery milestones · timeOS Estimated effort: **multi-day** for the indie phases; the production-only milestones add the trust and operability layer. ## M1 · Audio in, audio kept A recording is imported or captured, hashed, and stored without being degraded on the way in. ### Steps 1. Keep the original and derive a working copy The original is the only thing that can settle a dispute about a note. Files: `app/ingest.py`, `app/audio.py` ```sh ffmpeg -i input.m4a -ac 1 -ar 16000 -c:a pcm_s16le work.wav ``` 2. Hash both files and store the command used So a strange transcript can be traced to a conversion rather than argued about. Files: `app/ingest.py`, `tests/test_ingest.py` 3. Gate transcription behind a notice timestamp The other people in the call are told before this runs, not after. Files: `app/models.py`, `app/routes/recordings.py` ### Done when - [ ] The original file is byte-identical after import - [ ] A working copy exists with its conversion command recorded - [ ] Transcription is refused until notice is recorded - [ ] Local capture shows a visible indicator throughout ### Watch out - Transcoding the upload in place to save disk means the one artefact that can settle an argument about what was said has been through a lossy encoder chosen by you, and nobody can tell afterwards which distortions came from the microphone and which from your ffmpeg flags. Keep the original file, work from a derived copy, and store the hash of both. ## M2 · A transcript that keeps its doubts whisper.cpp output is stored per segment with its confidence fields, and non-speech is never decoded. ### Steps 1. Detect and cut silence before decoding This is where speech models invent whole sentences. Files: `app/silence.py`, `tests/test_silence.py` ```sh ffmpeg -i work.wav -af silencedetect=noise=-35dB:d=1.0 -f null - ``` 2. Store every confidence field the decoder returns Text alone gives you nothing to sort the doubtful segments by. Files: `app/transcribe.py`, `app/models.py` 3. Flag repetition loops and low-confidence segments Visibly, in the transcript, not in a log file nobody opens. Files: `app/flags.py`, `tests/test_flags.py` ### Done when - [ ] Confidence fields are stored per segment - [ ] Room tone produces zero segments - [ ] A repetition loop is flagged, not stored as clean text - [ ] Flagged segments are visible and excluded from notes by default ### Watch out - Feeding the whole file to the decoder and keeping only the text throws away the three numbers that tell you which sentences to distrust, and the published research says the invented sentences appear precisely in the non-vocal stretches you just fed it. Cut silence out before decoding, keep every confidence field on the row, and mark flagged segments in the interface rather than in a log. ## M3 · Speakers are a guess you can correct Turns are proposed heuristically, labelled by a human, and the interface never pretends otherwise. ### Steps 1. Propose turns from pauses, not from a diarisation claim A pause detector is honest about being a pause detector. Files: `app/turns.py`, `tests/test_turns.py` 2. Record assigned_by on every turn The difference between a guess and a correction has to survive into the export. Files: `app/models.py` 3. Let one rename propagate across the recording Otherwise nobody corrects anything after the third turn. Files: `app/routes/turns.py` ### Done when - [ ] Heuristic and human labels render differently - [ ] One rename updates every matching turn - [ ] Turns can be split and merged by hand - [ ] Unconfirmed speaker names are marked in notes ### Watch out - Presenting automatic speaker labels in the same typography as corrected ones means a quote gets attributed to the wrong person in the notes and then in an email, and nobody can tell from the interface that the attribution was a guess by a pause detector. Keep assigned_by on every turn and render the two states differently, including in exports. ## M4 · Notes that cite the audio Every sentence of a generated note is bound to segments and plays back from the recording. ### Steps 1. Require segment identifiers in the structured output A claim without a location in the audio is not a claim. Files: `app/notes.py`, `app/schemas.py` 2. Validate identifiers against the segments table Models cite things that do not exist, cheerfully. Files: `app/notes.py`, `tests/test_notes.py` 3. Wire every sentence to a seek in the player One click from the claim to the audio, or the citation is decoration. Files: `app/templates/recording.html`, `app/static/player.js` ### Done when - [ ] Every note sentence carries a valid segment identifier - [ ] Clicking a sentence seeks the audio to its segment - [ ] A fabricated citation stores nothing - [ ] Action items without a segment are refused ### Watch out - Asking for a summary as prose and storing it produces text that reads beautifully and cannot be checked, so the first time someone disputes an action item you are re-listening to forty minutes of audio to find out whether the model made it up. Require segment identifiers per sentence, validate them against the table, and drop any sentence that cannot be placed. ## M5 · Expiry, export and a bill that cannot run away (production only) Recordings expire on a schedule, leave in a readable format, and the optional model spend has a ceiling. ### Steps 1. Expire audio and notes together Notes that outlive their evidence are the thing this build avoids. Files: `app/retention.py`, `tests/test_retention.py` 2. Export a folder that works without the app Audio, Markdown transcript, notes with citations, segments as JSON. Files: `app/export.py` 3. Cap optional model spend and degrade to transcript The transcription is local and free, so the fallback is a good product. Files: `app/cost.py`, `tests/test_cost.py` ### Done when - [ ] Retention deletes audio and notes together and logs it - [ ] Surviving notes are stamped unverifiable - [ ] An exported folder is readable without the app - [ ] The cost ceiling stops notes, not transcription ### Watch out - Expiring the audio while keeping the notes leaves a file of confident statements about a conversation that can no longer be checked against anything, which is the exact failure the citation work in the previous phase existed to prevent. Expire them together by default, and make keeping notes past the audio an explicit per-recording choice that stamps the notes as unverifiable.
OPERATIONS.md
# Operations · timeOS ## Backup The database and the audio directory together, because either alone is useless. A backup of notes without audio is a set of claims nobody can check. ## Restore Restore both, then verify that a sample of note sentences still resolve to existing segments and that clicking one seeks correctly. A restore that leaves dangling citations has lost the property the build is for. Do a restore drill before the first real user, and write the date here when it passes. ## Monitoring Decode time per audio minute, the proportion of segments flagged, and the share of note sentences dropped for invalid citations. A rising drop rate usually means the transcript quality fell, not that the model got worse at citing. ## Incident checklist The incident here is someone acting on a note that the audio does not support. The segment citations and the retained originals are what let you find out what was actually said, which is why they expire together. 1. Contain the issue without destroying evidence or user data. 2. Record the timeline and affected scope. 3. Rotate exposed secrets and revoke compromised sessions or credentials. 4. Restore from a verified backup when needed. 5. Document the root cause, the remediation and the regression test. ## Release gate - [ ] Room tone is confirmed to produce zero segments - [ ] Every note sentence is confirmed to carry a valid segment identifier - [ ] Heuristic speaker labels are confirmed visibly distinct from corrected ones - [ ] Retention is confirmed to delete audio and notes in one transaction - [ ] The cost ceiling is confirmed to stop notes and leave transcription working ## Launch constraint Do not market omitted timeOS capabilities as implemented. The non-goals in `PRODUCT.md` remain user-visible limitations until they are deliberately delivered.
.env.example
# Copy to .env and fill in. Never commit .env; this file documents it. # Required. Recordings, segments, turns, notes and citations. DATABASE_PATH=./data/meetings.db # Required. Originals and working copies. Sized for the retention you chose. AUDIO_DIR=./data/audio # Required. The local model file. Its identifier is stored with every segment. WHISPER_MODEL_PATH=./models/ggml-medium.bin # Required. Segments above this are flagged and excluded from note generation. NO_SPEECH_THRESHOLD=0.6 # Required. Passed to ffmpeg silencedetect. Tune it against your own recordings. SILENCE_DB=-35dB # Required. Audio and notes are deleted together at this age unless overridden per recording. RETENTION_DAYS=90 # Optional · secret. Optional. Only note generation uses it; transcription never leaves the machine. NOTES_MODEL_KEY=sk-...redacted # Required. Note generation stops here for the month. Transcription and playback continue. MONTHLY_COST_CEILING=3.00
$ choose a build depth, inspect the files, then open the complete pack in your agent
timeOS: Customers pay for automatic capture, dependable speaker handling, search across calls, and notes arriving without manual file wrangling.
xlive multi-speaker accuracy
xcalendar and CRM integrations
xcross-call team analytics
xmeeting-bot auto-join
share on X ↗"I just replaced timeOS with one prompt"
timeOS pricing
| plan | monthly | annual (per mo) | what you get |
|---|---|---|---|
| free | $0/user | $0/user | 5 AI meeting credits at sign-up; maximum 90 minutes per meeting. |
| pro | $29/user | $19/user | Unlimited meeting notes and storage. |
| max | $39/user | $29/user | Unlimited meeting notes and storage plus unlimited API, MCP, and webhooks. |
| team | $35/user | $25/user | Minimum 3 seats; unlimited meeting notes and storage. |
| business | $39/user | $29/user | Unlimited meeting notes, storage, API, MCP, and webhooks. |
| enterprise | custom | — | Custom administration, security, and support; no public price or numeric limits. |
free tier5 AI meeting credits granted at sign-up; each meeting can be up to 90 minutes.
billingmonthly + annual; annual saves up to 35%
hidden costsTeam requires at least 3 paid seats, and API/MCP/webhook access requires the individual Max tier or team Business tier.
verified 2026-08-14 · source ↗
Vibecode timeOS
Kinda. The core of timeOS is buildable in a weekend with the prompt on this page, but there are real gaps: live multi-speaker accuracy, calendar and CRM integrations. Read the honest list above before committing.
How much does timeOS cost?
timeOS's pricing is usage-based or varies by plan · The official pricing URL is recorded, but a stable current amount was not safely recoverable in this pass. Do not publish a number until rechecked..
What do I lose by replacing timeOS?
Honestly: live multi-speaker accuracy; calendar and CRM integrations; cross-call team analytics; meeting-bot auto-join. If any of those are load-bearing for you, keep paying.
Is there an open-source alternative to timeOS?
Yes: whisper.cpp (Local speech-to-text engine suitable for private transcription.). Using prior art is also vibecoding; the prompt is for when you want it exactly your way.