Vibecode Bevel
track this build4 phases, 12 steps, beginner friendly0%KINDA · weekend project
Bevel is not sitting on secret data: it reads the same HealthKit records your watch already wrote, then does math and draws charts. An agent can absolutely build you a local dashboard over an Apple Health export that computes rolling averages, sleep and HRV trends, and lag correlations between habits and recovery. The gap is delivery, not analysis: real HealthKit access means an actual iOS app, Xcode, a developer account and background sync, and manual export zips get stale fast. You also lose the part Bevel spends most of its effort on, which is turning noisy sensor data into something you glance at once a day and actually understand. Fine for a curious quantified-self person, annoying for anyone who wants a phone widget.
the project pack5 files · written for this build, step by step
You are building a lean indie version of Bevel. Create the following project files first, then implement the application by following them. Keep the files updated as decisions change. Do not collapse this into a single README or prompt. ===== README.md ===== # Bevel · indie build A local analytics dashboard over your own health export whose hard problems are ingesting a file too large to hold in memory, resolving two devices that both recorded the same day, and presenting each metric at the resolution the published validation supports rather than the resolution the file happens to have. Estimated effort: **weekend**. Work `BUILD_PLAN.md` top to bottom · every phase ends in a check that has to pass before the next one starts. ## Stack | Part | Choice | Why | | --- | --- | --- | | Runtime | Python 3.12 with a local web dashboard, nothing leaving the machine | this is a record of your body and it belongs on your disk | | Ingest | An incremental XML parser writing straight to SQLite in batches | the export is hundreds of megabytes and a document parser will not survive it | | Storage | SQLite, one row per exported record, source and device retained | you cannot resolve a double count without knowing which device wrote what | | Analysis | A declared set of comparisons with intervals and sample sizes | a dashboard that tries every pair is a machine for generating findings | | Presentation | Every figure carries its device, its sample size and its known agreement | the number is only as good as the classifier that produced it | ## Before you start Have every one of these ready. The plan assumes them from step one. - [ ] **Python 3.12 or newer** · free - Why: Runs the importer, the analysis and the local dashboard. - Get it: python.org, or your package manager. - Verify: python3 --version prints 3.12 or higher - [ ] **Your own health export archive** · free, plus the wait while it generates - Why: Everything here is built against the real file, and the real file is far larger and far messier than any sample you will find. - Get it: From the health application on your phone, request a full export and transfer the archive to the machine running this. - Verify: The archive extracts and the importer reports a record count and export date - [ ] **Enough disk for the archive, the extraction and the database** · free - Why: The extracted export and the database together are several times the archive, and running out part way through an import is the most common first failure. - Get it: Check free space against the archive size multiplied by a factor you measured on the first import. - Verify: A full import completes and the measured total size is written in the README - [ ] **What each metric is allowed to display** · free - Why: The published validation differs sharply by metric, and deciding presentation after the charts exist means deciding it from whatever looked best. - Get it: Write down, per metric, whether it is shown as a value, a band or a baseline, and which published figure justifies that. - Verify: The table is in the README and each rule is asserted in a test - [ ] **The comparison pairs and lags, declared in advance** · free - Why: Declaring them afterwards turns a dashboard into a search for whatever matches the story you already had. - Get it: List the metric pairs and lags you care about in the configuration file with today's date. - Verify: The dashboard shows only declared pairs and prints the total comparison count ## Quick start Follow Phase 1 in `BUILD_PLAN.md`; it creates the project. Then copy `.env.example` to `.env` and fill in the values it documents. ## Honest limits This build deliberately does not replace: - Live background synchronisation. - A composite readiness or recovery score. - Clinical interpretation of any kind. - Non-Apple device sources. - Live background sync; you are re-exporting a zip by hand - A phone app, widgets and notifications - Their opinionated composite scores and plain-English daily readouts - Non-Apple device integrations and whatever normalisation they do across sources - Charts that a designer looked at If one of those is essential to you, that is the reason to keep paying for Bevel, and the README should say so rather than pretend. ===== BRIEF.md ===== # Build brief · Bevel The one-shot brief this plan expands. `BUILD_PLAN.md` (or `MILESTONES.md`) is the same sequence broken into steps and checks; where the two disagree, the plan wins. Build me a local dashboard over your own health export that shows each metric at the resolution its device actually has to replace Bevel. Build it in phases, in the order below. Do not write the whole thing in one pass. Finish a phase, run its "Done when" check, fix what fails, and only then start the next phase. ### Rule zero This app displays physiological measurements, so what the published validation says decides what it is allowed to show. Read this before designing a single chart. Sleep stages are not measured, they are classified, and not well. A 2025 validation in SLEEP Advances put six wrist wearables against polysomnography in 62 adults. Four-stage agreement ran from a Cohen's kappa of 0.21 to 0.53, the best being the Apple Watch Series 8 at 0.53. Every device found over 90 percent of sleep epochs, sensitivity 91.68 to 96.27 percent, but specificity for wake ran only 29.39 to 52.15 percent and correctly classified deep sleep epochs ran 47.46 to 69.63 percent, the Apple Watch at 50.66 percent with REM at 68.57. A screen reading '1 h 12 m of deep sleep' is reporting a classifier close to a coin flip per epoch. Lead with total sleep time and sleep timing, which the same study found detected well, and show stages in coarse bands with the agreement figure beside them. Wrist heart rate variability is the same story: published validations find heart rate tracks an electrocardiogram closely while the beat-to-beat variability measures do not, with reported agreement approaching or exceeding the measure's own day-to-day variation. Show it as a rolling baseline with a band, never as tonight's number against yesterday's. Therefore: no composite score, because nobody has validated that a number called readiness or recovery predicts anything about you. And declare your correlations before you compute them: testing every habit against every metric over thirty days finds something by construction. ### Stack (fixed, do not substitute) - Runtime: Python 3.12 with a local web dashboard, nothing leaving the machine · this is a record of your body and it belongs on your disk - Ingest: An incremental XML parser writing straight to SQLite in batches · the export is hundreds of megabytes and a document parser will not survive it - Storage: SQLite, one row per exported record, source and device retained · you cannot resolve a double count without knowing which device wrote what - Analysis: A declared set of comparisons with intervals and sample sizes · a dashboard that tries every pair is a machine for generating findings - Presentation: Every figure carries its device, its sample size and its known agreement · the number is only as good as the classifier that produced it ### Data model (create this before Phase 1) - imports(id, at, zip_path, export_date, record_count, sources_json, state) · state is partial or complete, never assumed - records(id, import_id, type, source_name, source_version, device, unit, value, start_at, end_at) · the raw grain, insert only - sleep_segments(id, record_id, source_name, stage, start_at, end_at) · one row per interval exactly as exported - nights(date, sleep_start, sleep_end, total_sleep_min, stage_minutes_json, primary_source, overlap_dropped_min, conflict) - daily(date, metric, value, n_records, primary_source, conflict) · one chosen source per metric per day, and the choice is recorded - baselines(metric, date, window_days, median, p25, p75, n) · what any single value is compared against - comparisons(id, metric_a, metric_b, lag_days, window_days, n, coefficient, ci_low, ci_high, declared_at) - There is no column for a composite score, and stage minutes are stored so they can be shown in bands rather than to the minute. ### Phase 1 · An export that does not fit in memory Build: Stream the export with an incremental parser, clearing each element after handling it, and insert in batches inside a transaction. Record the export date, the record count and every source name and version found. Make the import atomic: an interrupted run leaves no partial import visible to the dashboard. Trap: Loading the export with a document parser builds the whole file as objects in memory, takes every gigabyte the machine has and dies several minutes in, and the obvious fix of raising the limit only moves the moment of death. Parse incrementally, clear each element as you finish with it, and batch the inserts so the transaction log does not become the new problem. Trap: Re-importing after adding a metric duplicates every record you already had, and because health data is full of legitimately repeated values you cannot spot it afterwards by eye. Key records on their natural identity and make a re-import a no-op for anything already present, then prove it by importing the same file twice and comparing counts. Done when: An export of a stated size imports within a stated memory ceiling, measured and written in the README. Importing the same export twice produces identical record counts, tested. Killing the process mid-import leaves either no import or a complete one, tested by killing it. The import records the export date, record count and the set of source names and versions, and the dashboard shows them. Do not build yet: No source resolution, charts, baselines or comparisons yet. ### Phase 2 · Two devices, one day, and the double count Build: Pick one source per metric per day by an explicit configured preference, record which one was used, and count the days where sources disagreed. Merge overlapping sleep intervals rather than summing them, and store the minutes of overlap you dropped. Changing the preference recomputes the derived tables and touches no raw record. Trap: Summing every step or sleep record for a day adds the phone's count to the watch's and gives somebody who carried both a day that never happened, which is the single most common error in analysis of these exports. Choose one source per metric per day, write down the rule, and show the count of days where the sources disagreed rather than hiding the decision. Trap: Summing sleep intervals from two sources that both recorded the same night produces more sleep than there were hours, and a chart that occasionally exceeds twenty-four hours is a chart nobody trusts again. Merge intervals by union, store the overlap you removed, and assert that no night exceeds the elapsed wall clock between its start and end. Done when: A crafted day with records from two sources yields one daily value and a recorded primary source, tested. Overlapping sleep intervals are merged rather than summed and no night's total sleep exceeds the elapsed time between its start and end, asserted as a property. Switching the source preference recomputes affected days and leaves every raw record unchanged, verified by hash. The number of days with conflicting sources is visible on the dashboard. Do not build yet: No sleep presentation, baselines or comparisons yet. ### Phase 3 · Sleep at the resolution the device actually has Build: The sleep page leads with total sleep time and sleep timing as a rolling series. Stage durations appear in coarse bands rather than to the minute, each labelled with the agreement figure from the validation literature and the study's date. No stage gets a trend line over a window shorter than the one you can defend, and the interface has no path to a bare minute count. Trap: Rendering stage durations to the minute implies a measurement the device does not make: the published validation reports four-stage agreement between kappa 0.21 and 0.53 across six wearables and correctly classified deep sleep epochs between about 47 and 70 percent, so the deep sleep figure is close to a coin flip per epoch. Show bands, put the agreement figure beside the number, and remove every code path that can display a stage minute count on its own. Trap: Comparing tonight's stage minutes against last night's and drawing an arrow presents the difference between two noisy classifications as a change in the person. Compare against a rolling baseline with its spread, and require a stated minimum number of nights before any comparison is drawn at all. Done when: No interface path displays a stage duration as a bare minute figure, verified by review and by a test over the rendered templates. Every stage figure carries the agreement statistic and the source study's citation and date. A night-to-night arrow appears nowhere; comparisons are against a baseline with its spread and its sample size. Total sleep time and sleep timing are the first things on the page, checked by review. Do not build yet: No correlations, export or staleness handling yet. ### Phase 4 · Comparisons declared before they are computed Build: Declare the metric pairs and lags in configuration with the date you declared them. Compute each with its confidence interval and its sample size, state how many comparisons the dashboard runs in total, and refuse to display one below a configured minimum sample. Every result is phrased as an association and the interface contains no causal wording. Trap: A panel that tries every metric against every other one over a month will find several convincing relationships, because that is what running forty tests does, and the one you remember is the one that matched a story you already had. Fix the pairs in configuration with a declaration date, show the total number of comparisons run beside the results, and treat anything you add afterwards as a new declaration rather than a finding. Trap: Reporting a coefficient without its interval and sample size makes a relationship computed from eleven nights look like one computed from three hundred. Show the interval and the count everywhere the coefficient appears, and refuse to render below the minimum. Done when: Comparisons come only from the declared configuration and each row carries its declaration date, asserted. A coefficient never appears without its confidence interval and sample size, verified over the templates. The total count of declared comparisons is displayed alongside the results. Any pair below the configured minimum sample renders as not yet computable with the count needed, tested at the boundary. Do not build yet: No staleness handling, export or deletion yet. ### Phase 5 · Data that goes stale, and saying so Build: Show the export date at the top of every page and the age of the data beside it. Stop every axis at the export date rather than at today. Grey out anything that would imply current information. Export the derived tables as CSV, and provide one command that removes the database, the extracted files and the original zip together. Trap: Drawing today's date on the axis when the data ends eleven days ago makes a stale number look current, and the whole failure mode of this build is a dashboard that quietly stops being about now. Stop the axis at the export date, print the data age on every page, and show a prompt to re-export once the age crosses a configured threshold. Trap: Leaving the extracted export files beside the database means a delete that removes the database leaves a complete copy of your health record on disk, usually in a directory nobody remembers. Make deletion one command covering the database, the extracted files and the original archive, and print what it removed. Done when: Every page shows the export date and the data age, and no chart axis extends past the export date, verified by review. Crossing the staleness threshold shows a re-export prompt naming the age, tested at the boundary. Exporting produces the derived tables as CSV and reimports into an empty database with identical row counts. The delete command removes the database, extracted files and the original archive, and prints each path it removed, tested. Do not build yet: Nothing after this. ### Out of scope (and why) - Live background synchronisation, which needs an actual phone application, a developer account and a review process, and is the entire reason the paid product exists. - A composite readiness or recovery score, because no published work establishes that such a number predicts anything about an individual, and presenting one would be the only dishonest thing in this build. - Clinical interpretation of any kind, including sleep disorder screening, since the same validation work that sets the limits here is explicit that these devices do not replace polysomnography for diagnosis. - Non-Apple device sources and whatever normalisation would be needed between them, which is a per-vendor data problem rather than a weekend of code. ### README must contain - The published agreement figures for wearable sleep staging, with the study, the devices and the date, stated before any instructions - That wrist heart rate variability is shown as a rolling baseline because its published agreement approaches the measure's own variation - That there is no composite score and why one is not provided - That one source is chosen per metric per day, what the rule is, and how many days had a conflict - That the export goes stale, how the dashboard shows that, and how to delete everything including the archive ===== AGENTS.md ===== # Agent instructions · Bevel indie build - Read `README.md` and `BUILD_PLAN.md` before writing code. The stack is fixed: Python 3.12 with a local web dashboard, nothing leaving the machine, An incremental XML parser writing straight to SQLite in batches, SQLite, one row per exported record, source and device retained, A declared set of comparisons with intervals and sample sizes, Every figure carries its device, its sample size and its known agreement. Do not substitute. - Work one phase at a time, in order. Do not start a phase until every "Done when" item of the previous one passes. - Prefer the fewest moving parts that satisfy the step. No frameworks, services or dependencies the plan does not name. - Secrets live in `.env`, never in source or logs. Keep `.env.example` current when a variable is introduced. - Do not invent cryptography, security guarantees, APIs or compliance claims. - Add a focused test for every destructive, security-sensitive or data-loss path the plan names. - Run the project checks before declaring a phase complete, and record any deliberate shortcut in the README under "Tradeoffs". ## Known traps - Loading the export with a document parser builds the whole file as objects in memory, takes every gigabyte the machine has and dies several minutes in, and the obvious fix of raising the limit only moves the moment of death. Parse incrementally, clear each element as you finish with it, and batch the inserts so the transaction log does not become the new problem. - Re-importing after adding a metric duplicates every record you already had, and because health data is full of legitimately repeated values you cannot spot it afterwards by eye. Key records on their natural identity and make a re-import a no-op for anything already present, then prove it by importing the same file twice and comparing counts. - Summing every step or sleep record for a day adds the phone's count to the watch's and gives somebody who carried both a day that never happened, which is the single most common error in analysis of these exports. Choose one source per metric per day, write down the rule, and show the count of days where the sources disagreed rather than hiding the decision. - Summing sleep intervals from two sources that both recorded the same night produces more sleep than there were hours, and a chart that occasionally exceeds twenty-four hours is a chart nobody trusts again. Merge intervals by union, store the overlap you removed, and assert that no night exceeds the elapsed wall clock between its start and end. - Rendering stage durations to the minute implies a measurement the device does not make: the published validation reports four-stage agreement between kappa 0.21 and 0.53 across six wearables and correctly classified deep sleep epochs between about 47 and 70 percent, so the deep sleep figure is close to a coin flip per epoch. Show bands, put the agreement figure beside the number, and remove every code path that can display a stage minute count on its own. - Comparing tonight's stage minutes against last night's and drawing an arrow presents the difference between two noisy classifications as a change in the person. Compare against a rolling baseline with its spread, and require a stated minimum number of nights before any comparison is drawn at all. - A panel that tries every metric against every other one over a month will find several convincing relationships, because that is what running forty tests does, and the one you remember is the one that matched a story you already had. Fix the pairs in configuration with a declaration date, show the total number of comparisons run beside the results, and treat anything you add afterwards as a new declaration rather than a finding. - Reporting a coefficient without its interval and sample size makes a relationship computed from eleven nights look like one computed from three hundred. Show the interval and the count everywhere the coefficient appears, and refuse to render below the minimum. ===== BUILD_PLAN.md ===== # Build plan · Bevel A local analytics dashboard over your own health export whose hard problems are ingesting a file too large to hold in memory, resolving two devices that both recorded the same day, and presenting each metric at the resolution the published validation supports rather than the resolution the file happens to have. Phases are in dependency order. Each ends in a "Done when" list; treat an unticked item as a blocker, not a note. ## Phase 1 · An export that does not fit in memory A multi-hundred-megabyte export imports within a stated memory ceiling and either completes or leaves nothing. ### Steps 1. Parse incrementally and clear as you go The file is bigger than the memory you want to spend. Files: `src/ingest/stream.py`, `tests/test_memory.py` 2. Make a re-import a no-op for existing records Duplicated health records cannot be spotted by eye later. Files: `src/ingest/dedupe.py` 3. Make the import atomic A half import is a dashboard quietly showing half a life. Files: `src/ingest/transaction.py` ### Done when - [ ] A large export imports under the stated memory ceiling - [ ] Importing twice yields identical record counts - [ ] An interrupted import leaves nothing partial - [ ] Export date, count and sources are recorded and shown ### Watch out - Loading the export with a document parser builds the whole file as objects in memory, takes every gigabyte the machine has and dies several minutes in, and the obvious fix of raising the limit only moves the moment of death. Parse incrementally, clear each element as you finish with it, and batch the inserts so the transaction log does not become the new problem. - Re-importing after adding a metric duplicates every record you already had, and because health data is full of legitimately repeated values you cannot spot it afterwards by eye. Key records on their natural identity and make a re-import a no-op for anything already present, then prove it by importing the same file twice and comparing counts. ## Phase 2 · Two devices, one day, and the double count Each metric on each day comes from one chosen source, and what was discarded is recorded. ### Steps 1. Choose one source per metric per day and record it Adding two devices together invents a day. Files: `src/resolve/sources.py`, `tests/test_double_count.py` 2. Merge sleep intervals by union, keep the overlap figure Two devices recording one night is not two nights. Files: `src/resolve/sleep.py` 3. Recompute derived tables without touching raw records The raw import is the only thing you cannot re-derive. Files: `src/resolve/recompute.py` ### Done when - [ ] A two-source day yields one value and a recorded source - [ ] No night's sleep exceeds its own elapsed time - [ ] Changing the preference leaves raw records untouched - [ ] Conflicting-source day count is visible ### Watch out - Summing every step or sleep record for a day adds the phone's count to the watch's and gives somebody who carried both a day that never happened, which is the single most common error in analysis of these exports. Choose one source per metric per day, write down the rule, and show the count of days where the sources disagreed rather than hiding the decision. - Summing sleep intervals from two sources that both recorded the same night produces more sleep than there were hours, and a chart that occasionally exceeds twenty-four hours is a chart nobody trusts again. Merge intervals by union, store the overlap you removed, and assert that no night exceeds the elapsed wall clock between its start and end. ## Phase 3 · Sleep at the resolution the device actually has Total sleep and timing lead, stages appear in bands with their published agreement attached. ### Steps 1. Lead with total sleep and timing They are the parts the devices are measured as detecting well. Files: `src/web/sleep.py`, `src/web/templates/sleep.html` 2. Show stages in bands with the agreement figure The number and its uncertainty travel together or not at all. Files: `src/analysis/stages.py`, `tests/test_no_bare_minutes.py` 3. Compare against a baseline, never against last night Two noisy classifications do not make a trend. Files: `src/analysis/baseline.py` ### Done when - [ ] No path shows a bare stage minute count - [ ] Every stage figure carries its agreement statistic and source - [ ] Comparisons are against a baseline with spread and n - [ ] Total sleep and timing lead the page ### Watch out - Rendering stage durations to the minute implies a measurement the device does not make: the published validation reports four-stage agreement between kappa 0.21 and 0.53 across six wearables and correctly classified deep sleep epochs between about 47 and 70 percent, so the deep sleep figure is close to a coin flip per epoch. Show bands, put the agreement figure beside the number, and remove every code path that can display a stage minute count on its own. - Comparing tonight's stage minutes against last night's and drawing an arrow presents the difference between two noisy classifications as a change in the person. Compare against a rolling baseline with its spread, and require a stated minimum number of nights before any comparison is drawn at all. ## Phase 4 · Comparisons declared before they are computed The correlations on the dashboard are the ones you chose in advance, reported with intervals and counts. ### Steps 1. Declare pairs and lags in configuration with a date A hypothesis chosen after the data is not a hypothesis. Files: `config/comparisons.toml`, `src/analysis/declare.py` 2. Report interval and sample size everywhere The coefficient alone hides how little is behind it. Files: `src/analysis/correlate.py`, `tests/test_intervals.py` 3. Say how many comparisons the dashboard runs It is the context that makes one result readable. Files: `src/web/templates/insights.html` ### Done when - [ ] Only declared comparisons appear, each with its date - [ ] No coefficient appears without interval and sample size - [ ] The total comparison count is displayed - [ ] Under-sampled pairs render as not yet computable ### Watch out - A panel that tries every metric against every other one over a month will find several convincing relationships, because that is what running forty tests does, and the one you remember is the one that matched a story you already had. Fix the pairs in configuration with a declaration date, show the total number of comparisons run beside the results, and treat anything you add afterwards as a new declaration rather than a finding. - Reporting a coefficient without its interval and sample size makes a relationship computed from eleven nights look like one computed from three hundred. Show the interval and the count everywhere the coefficient appears, and refuse to render below the minimum. ## Not in this build - Live background synchronisation. - A composite readiness or recovery score. - Clinical interpretation of any kind. - Non-Apple device sources. ## After v1, if you want it - A per-metric page showing what the published validation says about that metric, which is the honest version of an explanation screen - Incremental import of successive exports so the history accumulates rather than being replaced each time - A single annual summary at the timescale the agreement figures actually support ===== .env.example ===== # Copy to .env and fill in. Never commit .env; this file documents it. # Required. Raw records and derived tables. Everything about your body; keep it local. DATABASE_PATH=./data/health.db # Required. Where the archive is unpacked. Removed by the delete command along with the archive. EXTRACT_DIR=./data/extract # Required. Rows per insert batch. Larger is faster and uses more memory. IMPORT_BATCH_SIZE=5000 # Required. Order used to pick one source per metric per day. Changing it recomputes derived tables. SOURCE_PREFERENCE=watch,phone # Required. Rolling window any single value is compared against. BASELINE_WINDOW_DAYS=28 # Required. Below this many paired days, a comparison is shown as not yet computable. MIN_COMPARISON_N=30 # Required. Data age at which the dashboard prompts for a fresh export. STALE_AFTER_DAYS=14
You are building a lean indie version of Bevel. Create the following project files first, then implement the application by following them. Keep the files updated as decisions change. Do not collapse this into a single README or prompt. ===== README.md ===== # Bevel · indie build A local analytics dashboard over your own health export whose hard problems are ingesting a file too large to hold in memory, resolving two devices that both recorded the same day, and presenting each metric at the resolution the published validation supports rather than the resolution the file happens to have. Estimated effort: **weekend**. Work `BUILD_PLAN.md` top to bottom · every phase ends in a check that has to pass before the next one starts. ## Stack | Part | Choice | Why | | --- | --- | --- | | Runtime | Python 3.12 with a local web dashboard, nothing leaving the machine | this is a record of your body and it belongs on your disk | | Ingest | An incremental XML parser writing straight to SQLite in batches | the export is hundreds of megabytes and a document parser will not survive it | | Storage | SQLite, one row per exported record, source and device retained | you cannot resolve a double count without knowing which device wrote what | | Analysis | A declared set of comparisons with intervals and sample sizes | a dashboard that tries every pair is a machine for generating findings | | Presentation | Every figure carries its device, its sample size and its known agreement | the number is only as good as the classifier that produced it | ## Before you start Have every one of these ready. The plan assumes them from step one. - [ ] **Python 3.12 or newer** · free - Why: Runs the importer, the analysis and the local dashboard. - Get it: python.org, or your package manager. - Verify: python3 --version prints 3.12 or higher - [ ] **Your own health export archive** · free, plus the wait while it generates - Why: Everything here is built against the real file, and the real file is far larger and far messier than any sample you will find. - Get it: From the health application on your phone, request a full export and transfer the archive to the machine running this. - Verify: The archive extracts and the importer reports a record count and export date - [ ] **Enough disk for the archive, the extraction and the database** · free - Why: The extracted export and the database together are several times the archive, and running out part way through an import is the most common first failure. - Get it: Check free space against the archive size multiplied by a factor you measured on the first import. - Verify: A full import completes and the measured total size is written in the README - [ ] **What each metric is allowed to display** · free - Why: The published validation differs sharply by metric, and deciding presentation after the charts exist means deciding it from whatever looked best. - Get it: Write down, per metric, whether it is shown as a value, a band or a baseline, and which published figure justifies that. - Verify: The table is in the README and each rule is asserted in a test - [ ] **The comparison pairs and lags, declared in advance** · free - Why: Declaring them afterwards turns a dashboard into a search for whatever matches the story you already had. - Get it: List the metric pairs and lags you care about in the configuration file with today's date. - Verify: The dashboard shows only declared pairs and prints the total comparison count ## Quick start Follow Phase 1 in `BUILD_PLAN.md`; it creates the project. Then copy `.env.example` to `.env` and fill in the values it documents. ## Honest limits This build deliberately does not replace: - Live background synchronisation. - A composite readiness or recovery score. - Clinical interpretation of any kind. - Non-Apple device sources. - Live background sync; you are re-exporting a zip by hand - A phone app, widgets and notifications - Their opinionated composite scores and plain-English daily readouts - Non-Apple device integrations and whatever normalisation they do across sources - Charts that a designer looked at If one of those is essential to you, that is the reason to keep paying for Bevel, and the README should say so rather than pretend. ===== BRIEF.md ===== # Build brief · Bevel The one-shot brief this plan expands. `BUILD_PLAN.md` (or `MILESTONES.md`) is the same sequence broken into steps and checks; where the two disagree, the plan wins. Build me a local dashboard over your own health export that shows each metric at the resolution its device actually has to replace Bevel. Build it in phases, in the order below. Do not write the whole thing in one pass. Finish a phase, run its "Done when" check, fix what fails, and only then start the next phase. ### Rule zero This app displays physiological measurements, so what the published validation says decides what it is allowed to show. Read this before designing a single chart. Sleep stages are not measured, they are classified, and not well. A 2025 validation in SLEEP Advances put six wrist wearables against polysomnography in 62 adults. Four-stage agreement ran from a Cohen's kappa of 0.21 to 0.53, the best being the Apple Watch Series 8 at 0.53. Every device found over 90 percent of sleep epochs, sensitivity 91.68 to 96.27 percent, but specificity for wake ran only 29.39 to 52.15 percent and correctly classified deep sleep epochs ran 47.46 to 69.63 percent, the Apple Watch at 50.66 percent with REM at 68.57. A screen reading '1 h 12 m of deep sleep' is reporting a classifier close to a coin flip per epoch. Lead with total sleep time and sleep timing, which the same study found detected well, and show stages in coarse bands with the agreement figure beside them. Wrist heart rate variability is the same story: published validations find heart rate tracks an electrocardiogram closely while the beat-to-beat variability measures do not, with reported agreement approaching or exceeding the measure's own day-to-day variation. Show it as a rolling baseline with a band, never as tonight's number against yesterday's. Therefore: no composite score, because nobody has validated that a number called readiness or recovery predicts anything about you. And declare your correlations before you compute them: testing every habit against every metric over thirty days finds something by construction. ### Stack (fixed, do not substitute) - Runtime: Python 3.12 with a local web dashboard, nothing leaving the machine · this is a record of your body and it belongs on your disk - Ingest: An incremental XML parser writing straight to SQLite in batches · the export is hundreds of megabytes and a document parser will not survive it - Storage: SQLite, one row per exported record, source and device retained · you cannot resolve a double count without knowing which device wrote what - Analysis: A declared set of comparisons with intervals and sample sizes · a dashboard that tries every pair is a machine for generating findings - Presentation: Every figure carries its device, its sample size and its known agreement · the number is only as good as the classifier that produced it ### Data model (create this before Phase 1) - imports(id, at, zip_path, export_date, record_count, sources_json, state) · state is partial or complete, never assumed - records(id, import_id, type, source_name, source_version, device, unit, value, start_at, end_at) · the raw grain, insert only - sleep_segments(id, record_id, source_name, stage, start_at, end_at) · one row per interval exactly as exported - nights(date, sleep_start, sleep_end, total_sleep_min, stage_minutes_json, primary_source, overlap_dropped_min, conflict) - daily(date, metric, value, n_records, primary_source, conflict) · one chosen source per metric per day, and the choice is recorded - baselines(metric, date, window_days, median, p25, p75, n) · what any single value is compared against - comparisons(id, metric_a, metric_b, lag_days, window_days, n, coefficient, ci_low, ci_high, declared_at) - There is no column for a composite score, and stage minutes are stored so they can be shown in bands rather than to the minute. ### Phase 1 · An export that does not fit in memory Build: Stream the export with an incremental parser, clearing each element after handling it, and insert in batches inside a transaction. Record the export date, the record count and every source name and version found. Make the import atomic: an interrupted run leaves no partial import visible to the dashboard. Trap: Loading the export with a document parser builds the whole file as objects in memory, takes every gigabyte the machine has and dies several minutes in, and the obvious fix of raising the limit only moves the moment of death. Parse incrementally, clear each element as you finish with it, and batch the inserts so the transaction log does not become the new problem. Trap: Re-importing after adding a metric duplicates every record you already had, and because health data is full of legitimately repeated values you cannot spot it afterwards by eye. Key records on their natural identity and make a re-import a no-op for anything already present, then prove it by importing the same file twice and comparing counts. Done when: An export of a stated size imports within a stated memory ceiling, measured and written in the README. Importing the same export twice produces identical record counts, tested. Killing the process mid-import leaves either no import or a complete one, tested by killing it. The import records the export date, record count and the set of source names and versions, and the dashboard shows them. Do not build yet: No source resolution, charts, baselines or comparisons yet. ### Phase 2 · Two devices, one day, and the double count Build: Pick one source per metric per day by an explicit configured preference, record which one was used, and count the days where sources disagreed. Merge overlapping sleep intervals rather than summing them, and store the minutes of overlap you dropped. Changing the preference recomputes the derived tables and touches no raw record. Trap: Summing every step or sleep record for a day adds the phone's count to the watch's and gives somebody who carried both a day that never happened, which is the single most common error in analysis of these exports. Choose one source per metric per day, write down the rule, and show the count of days where the sources disagreed rather than hiding the decision. Trap: Summing sleep intervals from two sources that both recorded the same night produces more sleep than there were hours, and a chart that occasionally exceeds twenty-four hours is a chart nobody trusts again. Merge intervals by union, store the overlap you removed, and assert that no night exceeds the elapsed wall clock between its start and end. Done when: A crafted day with records from two sources yields one daily value and a recorded primary source, tested. Overlapping sleep intervals are merged rather than summed and no night's total sleep exceeds the elapsed time between its start and end, asserted as a property. Switching the source preference recomputes affected days and leaves every raw record unchanged, verified by hash. The number of days with conflicting sources is visible on the dashboard. Do not build yet: No sleep presentation, baselines or comparisons yet. ### Phase 3 · Sleep at the resolution the device actually has Build: The sleep page leads with total sleep time and sleep timing as a rolling series. Stage durations appear in coarse bands rather than to the minute, each labelled with the agreement figure from the validation literature and the study's date. No stage gets a trend line over a window shorter than the one you can defend, and the interface has no path to a bare minute count. Trap: Rendering stage durations to the minute implies a measurement the device does not make: the published validation reports four-stage agreement between kappa 0.21 and 0.53 across six wearables and correctly classified deep sleep epochs between about 47 and 70 percent, so the deep sleep figure is close to a coin flip per epoch. Show bands, put the agreement figure beside the number, and remove every code path that can display a stage minute count on its own. Trap: Comparing tonight's stage minutes against last night's and drawing an arrow presents the difference between two noisy classifications as a change in the person. Compare against a rolling baseline with its spread, and require a stated minimum number of nights before any comparison is drawn at all. Done when: No interface path displays a stage duration as a bare minute figure, verified by review and by a test over the rendered templates. Every stage figure carries the agreement statistic and the source study's citation and date. A night-to-night arrow appears nowhere; comparisons are against a baseline with its spread and its sample size. Total sleep time and sleep timing are the first things on the page, checked by review. Do not build yet: No correlations, export or staleness handling yet. ### Phase 4 · Comparisons declared before they are computed Build: Declare the metric pairs and lags in configuration with the date you declared them. Compute each with its confidence interval and its sample size, state how many comparisons the dashboard runs in total, and refuse to display one below a configured minimum sample. Every result is phrased as an association and the interface contains no causal wording. Trap: A panel that tries every metric against every other one over a month will find several convincing relationships, because that is what running forty tests does, and the one you remember is the one that matched a story you already had. Fix the pairs in configuration with a declaration date, show the total number of comparisons run beside the results, and treat anything you add afterwards as a new declaration rather than a finding. Trap: Reporting a coefficient without its interval and sample size makes a relationship computed from eleven nights look like one computed from three hundred. Show the interval and the count everywhere the coefficient appears, and refuse to render below the minimum. Done when: Comparisons come only from the declared configuration and each row carries its declaration date, asserted. A coefficient never appears without its confidence interval and sample size, verified over the templates. The total count of declared comparisons is displayed alongside the results. Any pair below the configured minimum sample renders as not yet computable with the count needed, tested at the boundary. Do not build yet: No staleness handling, export or deletion yet. ### Phase 5 · Data that goes stale, and saying so Build: Show the export date at the top of every page and the age of the data beside it. Stop every axis at the export date rather than at today. Grey out anything that would imply current information. Export the derived tables as CSV, and provide one command that removes the database, the extracted files and the original zip together. Trap: Drawing today's date on the axis when the data ends eleven days ago makes a stale number look current, and the whole failure mode of this build is a dashboard that quietly stops being about now. Stop the axis at the export date, print the data age on every page, and show a prompt to re-export once the age crosses a configured threshold. Trap: Leaving the extracted export files beside the database means a delete that removes the database leaves a complete copy of your health record on disk, usually in a directory nobody remembers. Make deletion one command covering the database, the extracted files and the original archive, and print what it removed. Done when: Every page shows the export date and the data age, and no chart axis extends past the export date, verified by review. Crossing the staleness threshold shows a re-export prompt naming the age, tested at the boundary. Exporting produces the derived tables as CSV and reimports into an empty database with identical row counts. The delete command removes the database, extracted files and the original archive, and prints each path it removed, tested. Do not build yet: Nothing after this. ### Out of scope (and why) - Live background synchronisation, which needs an actual phone application, a developer account and a review process, and is the entire reason the paid product exists. - A composite readiness or recovery score, because no published work establishes that such a number predicts anything about an individual, and presenting one would be the only dishonest thing in this build. - Clinical interpretation of any kind, including sleep disorder screening, since the same validation work that sets the limits here is explicit that these devices do not replace polysomnography for diagnosis. - Non-Apple device sources and whatever normalisation would be needed between them, which is a per-vendor data problem rather than a weekend of code. ### README must contain - The published agreement figures for wearable sleep staging, with the study, the devices and the date, stated before any instructions - That wrist heart rate variability is shown as a rolling baseline because its published agreement approaches the measure's own variation - That there is no composite score and why one is not provided - That one source is chosen per metric per day, what the rule is, and how many days had a conflict - That the export goes stale, how the dashboard shows that, and how to delete everything including the archive ===== AGENTS.md ===== # Agent instructions · Bevel indie build - Read `README.md` and `BUILD_PLAN.md` before writing code. The stack is fixed: Python 3.12 with a local web dashboard, nothing leaving the machine, An incremental XML parser writing straight to SQLite in batches, SQLite, one row per exported record, source and device retained, A declared set of comparisons with intervals and sample sizes, Every figure carries its device, its sample size and its known agreement. Do not substitute. - Work one phase at a time, in order. Do not start a phase until every "Done when" item of the previous one passes. - Prefer the fewest moving parts that satisfy the step. No frameworks, services or dependencies the plan does not name. - Secrets live in `.env`, never in source or logs. Keep `.env.example` current when a variable is introduced. - Do not invent cryptography, security guarantees, APIs or compliance claims. - Add a focused test for every destructive, security-sensitive or data-loss path the plan names. - Run the project checks before declaring a phase complete, and record any deliberate shortcut in the README under "Tradeoffs". ## Known traps - Loading the export with a document parser builds the whole file as objects in memory, takes every gigabyte the machine has and dies several minutes in, and the obvious fix of raising the limit only moves the moment of death. Parse incrementally, clear each element as you finish with it, and batch the inserts so the transaction log does not become the new problem. - Re-importing after adding a metric duplicates every record you already had, and because health data is full of legitimately repeated values you cannot spot it afterwards by eye. Key records on their natural identity and make a re-import a no-op for anything already present, then prove it by importing the same file twice and comparing counts. - Summing every step or sleep record for a day adds the phone's count to the watch's and gives somebody who carried both a day that never happened, which is the single most common error in analysis of these exports. Choose one source per metric per day, write down the rule, and show the count of days where the sources disagreed rather than hiding the decision. - Summing sleep intervals from two sources that both recorded the same night produces more sleep than there were hours, and a chart that occasionally exceeds twenty-four hours is a chart nobody trusts again. Merge intervals by union, store the overlap you removed, and assert that no night exceeds the elapsed wall clock between its start and end. - Rendering stage durations to the minute implies a measurement the device does not make: the published validation reports four-stage agreement between kappa 0.21 and 0.53 across six wearables and correctly classified deep sleep epochs between about 47 and 70 percent, so the deep sleep figure is close to a coin flip per epoch. Show bands, put the agreement figure beside the number, and remove every code path that can display a stage minute count on its own. - Comparing tonight's stage minutes against last night's and drawing an arrow presents the difference between two noisy classifications as a change in the person. Compare against a rolling baseline with its spread, and require a stated minimum number of nights before any comparison is drawn at all. - A panel that tries every metric against every other one over a month will find several convincing relationships, because that is what running forty tests does, and the one you remember is the one that matched a story you already had. Fix the pairs in configuration with a declaration date, show the total number of comparisons run beside the results, and treat anything you add afterwards as a new declaration rather than a finding. - Reporting a coefficient without its interval and sample size makes a relationship computed from eleven nights look like one computed from three hundred. Show the interval and the count everywhere the coefficient appears, and refuse to render below the minimum. ===== BUILD_PLAN.md ===== # Build plan · Bevel A local analytics dashboard over your own health export whose hard problems are ingesting a file too large to hold in memory, resolving two devices that both recorded the same day, and presenting each metric at the resolution the published validation supports rather than the resolution the file happens to have. Phases are in dependency order. Each ends in a "Done when" list; treat an unticked item as a blocker, not a note. ## Phase 1 · An export that does not fit in memory A multi-hundred-megabyte export imports within a stated memory ceiling and either completes or leaves nothing. ### Steps 1. Parse incrementally and clear as you go The file is bigger than the memory you want to spend. Files: `src/ingest/stream.py`, `tests/test_memory.py` 2. Make a re-import a no-op for existing records Duplicated health records cannot be spotted by eye later. Files: `src/ingest/dedupe.py` 3. Make the import atomic A half import is a dashboard quietly showing half a life. Files: `src/ingest/transaction.py` ### Done when - [ ] A large export imports under the stated memory ceiling - [ ] Importing twice yields identical record counts - [ ] An interrupted import leaves nothing partial - [ ] Export date, count and sources are recorded and shown ### Watch out - Loading the export with a document parser builds the whole file as objects in memory, takes every gigabyte the machine has and dies several minutes in, and the obvious fix of raising the limit only moves the moment of death. Parse incrementally, clear each element as you finish with it, and batch the inserts so the transaction log does not become the new problem. - Re-importing after adding a metric duplicates every record you already had, and because health data is full of legitimately repeated values you cannot spot it afterwards by eye. Key records on their natural identity and make a re-import a no-op for anything already present, then prove it by importing the same file twice and comparing counts. ## Phase 2 · Two devices, one day, and the double count Each metric on each day comes from one chosen source, and what was discarded is recorded. ### Steps 1. Choose one source per metric per day and record it Adding two devices together invents a day. Files: `src/resolve/sources.py`, `tests/test_double_count.py` 2. Merge sleep intervals by union, keep the overlap figure Two devices recording one night is not two nights. Files: `src/resolve/sleep.py` 3. Recompute derived tables without touching raw records The raw import is the only thing you cannot re-derive. Files: `src/resolve/recompute.py` ### Done when - [ ] A two-source day yields one value and a recorded source - [ ] No night's sleep exceeds its own elapsed time - [ ] Changing the preference leaves raw records untouched - [ ] Conflicting-source day count is visible ### Watch out - Summing every step or sleep record for a day adds the phone's count to the watch's and gives somebody who carried both a day that never happened, which is the single most common error in analysis of these exports. Choose one source per metric per day, write down the rule, and show the count of days where the sources disagreed rather than hiding the decision. - Summing sleep intervals from two sources that both recorded the same night produces more sleep than there were hours, and a chart that occasionally exceeds twenty-four hours is a chart nobody trusts again. Merge intervals by union, store the overlap you removed, and assert that no night exceeds the elapsed wall clock between its start and end. ## Phase 3 · Sleep at the resolution the device actually has Total sleep and timing lead, stages appear in bands with their published agreement attached. ### Steps 1. Lead with total sleep and timing They are the parts the devices are measured as detecting well. Files: `src/web/sleep.py`, `src/web/templates/sleep.html` 2. Show stages in bands with the agreement figure The number and its uncertainty travel together or not at all. Files: `src/analysis/stages.py`, `tests/test_no_bare_minutes.py` 3. Compare against a baseline, never against last night Two noisy classifications do not make a trend. Files: `src/analysis/baseline.py` ### Done when - [ ] No path shows a bare stage minute count - [ ] Every stage figure carries its agreement statistic and source - [ ] Comparisons are against a baseline with spread and n - [ ] Total sleep and timing lead the page ### Watch out - Rendering stage durations to the minute implies a measurement the device does not make: the published validation reports four-stage agreement between kappa 0.21 and 0.53 across six wearables and correctly classified deep sleep epochs between about 47 and 70 percent, so the deep sleep figure is close to a coin flip per epoch. Show bands, put the agreement figure beside the number, and remove every code path that can display a stage minute count on its own. - Comparing tonight's stage minutes against last night's and drawing an arrow presents the difference between two noisy classifications as a change in the person. Compare against a rolling baseline with its spread, and require a stated minimum number of nights before any comparison is drawn at all. ## Phase 4 · Comparisons declared before they are computed The correlations on the dashboard are the ones you chose in advance, reported with intervals and counts. ### Steps 1. Declare pairs and lags in configuration with a date A hypothesis chosen after the data is not a hypothesis. Files: `config/comparisons.toml`, `src/analysis/declare.py` 2. Report interval and sample size everywhere The coefficient alone hides how little is behind it. Files: `src/analysis/correlate.py`, `tests/test_intervals.py` 3. Say how many comparisons the dashboard runs It is the context that makes one result readable. Files: `src/web/templates/insights.html` ### Done when - [ ] Only declared comparisons appear, each with its date - [ ] No coefficient appears without interval and sample size - [ ] The total comparison count is displayed - [ ] Under-sampled pairs render as not yet computable ### Watch out - A panel that tries every metric against every other one over a month will find several convincing relationships, because that is what running forty tests does, and the one you remember is the one that matched a story you already had. Fix the pairs in configuration with a declaration date, show the total number of comparisons run beside the results, and treat anything you add afterwards as a new declaration rather than a finding. - Reporting a coefficient without its interval and sample size makes a relationship computed from eleven nights look like one computed from three hundred. Show the interval and the count everywhere the coefficient appears, and refuse to render below the minimum. ## Not in this build - Live background synchronisation. - A composite readiness or recovery score. - Clinical interpretation of any kind. - Non-Apple device sources. ## After v1, if you want it - A per-metric page showing what the published validation says about that metric, which is the honest version of an explanation screen - Incremental import of successive exports so the history accumulates rather than being replaced each time - A single annual summary at the timescale the agreement figures actually support ===== .env.example ===== # Copy to .env and fill in. Never commit .env; this file documents it. # Required. Raw records and derived tables. Everything about your body; keep it local. DATABASE_PATH=./data/health.db # Required. Where the archive is unpacked. Removed by the delete command along with the archive. EXTRACT_DIR=./data/extract # Required. Rows per insert batch. Larger is faster and uses more memory. IMPORT_BATCH_SIZE=5000 # Required. Order used to pick one source per metric per day. Changing it recomputes derived tables. SOURCE_PREFERENCE=watch,phone # Required. Rolling window any single value is compared against. BASELINE_WINDOW_DAYS=28 # Required. Below this many paired days, a comparison is shown as not yet computable. MIN_COMPARISON_N=30 # Required. Data age at which the dashboard prompts for a fresh export. STALE_AFTER_DAYS=14
You are building a production product version of Bevel. Create the following project files first, then implement the application by following them. Keep the files updated as decisions change. Do not collapse this into a single README or prompt. ===== PRODUCT.md ===== # Bevel · product brief ## Problem Bevel is not sitting on secret data: it reads the same HealthKit records your watch already wrote, then does math and draws charts. An agent can absolutely build you a local dashboard over an Apple Health export that computes rolling averages, sleep and HRV trends, and lag correlations between habits and recovery. The gap is delivery, not analysis: real HealthKit access means an actual iOS app, Xcode, a developer account and background sync, and manual export zips get stale fast. You also lose the part Bevel spends most of its effort on, which is turning noisy sensor data into something you glance at once a day and actually understand. Fine for a curious quantified-self person, annoying for anyone who wants a phone widget. ## Product outcome A dashboard that is honest about a measurement the literature says is rough: sleep stages in bands with their agreement attached, variability as a baseline rather than a nightly number, no composite score, only declared comparisons, and the data's age on every page. ## Target user A builder who needs a maintainable product foundation, not a one-off demo. ## Required capabilities - An Apple Health export zip (Health app, profile, Export All Health Data) - Python 3.12 locally - Patience with a multi-hundred-megabyte export.xml - An Apple Developer account and Xcode if you want live HealthKit sync instead of manual exports ## Explicit non-goals for v1 - Live background synchronisation. - A composite readiness or recovery score. - Clinical interpretation of any kind. - Non-Apple device sources. - Live background sync; you are re-exporting a zip by hand - A phone app, widgets and notifications - Their opinionated composite scores and plain-English daily readouts - Non-Apple device integrations and whatever normalisation they do across sources - Charts that a designer looked at ## Success criteria - A large export is confirmed to import under the stated memory ceiling - No night's total sleep is confirmed to exceed its own elapsed time - No interface path is confirmed to show a bare stage minute count - Only declared comparisons are confirmed to appear, with intervals and counts - Deletion is confirmed to remove database, extraction and archive ===== BRIEF.md ===== # Build brief · Bevel The one-shot brief this plan expands. `BUILD_PLAN.md` (or `MILESTONES.md`) is the same sequence broken into steps and checks; where the two disagree, the plan wins. Build me a local dashboard over your own health export that shows each metric at the resolution its device actually has to replace Bevel. Build it in phases, in the order below. Do not write the whole thing in one pass. Finish a phase, run its "Done when" check, fix what fails, and only then start the next phase. ### Rule zero This app displays physiological measurements, so what the published validation says decides what it is allowed to show. Read this before designing a single chart. Sleep stages are not measured, they are classified, and not well. A 2025 validation in SLEEP Advances put six wrist wearables against polysomnography in 62 adults. Four-stage agreement ran from a Cohen's kappa of 0.21 to 0.53, the best being the Apple Watch Series 8 at 0.53. Every device found over 90 percent of sleep epochs, sensitivity 91.68 to 96.27 percent, but specificity for wake ran only 29.39 to 52.15 percent and correctly classified deep sleep epochs ran 47.46 to 69.63 percent, the Apple Watch at 50.66 percent with REM at 68.57. A screen reading '1 h 12 m of deep sleep' is reporting a classifier close to a coin flip per epoch. Lead with total sleep time and sleep timing, which the same study found detected well, and show stages in coarse bands with the agreement figure beside them. Wrist heart rate variability is the same story: published validations find heart rate tracks an electrocardiogram closely while the beat-to-beat variability measures do not, with reported agreement approaching or exceeding the measure's own day-to-day variation. Show it as a rolling baseline with a band, never as tonight's number against yesterday's. Therefore: no composite score, because nobody has validated that a number called readiness or recovery predicts anything about you. And declare your correlations before you compute them: testing every habit against every metric over thirty days finds something by construction. ### Stack (fixed, do not substitute) - Runtime: Python 3.12 with a local web dashboard, nothing leaving the machine · this is a record of your body and it belongs on your disk - Ingest: An incremental XML parser writing straight to SQLite in batches · the export is hundreds of megabytes and a document parser will not survive it - Storage: SQLite, one row per exported record, source and device retained · you cannot resolve a double count without knowing which device wrote what - Analysis: A declared set of comparisons with intervals and sample sizes · a dashboard that tries every pair is a machine for generating findings - Presentation: Every figure carries its device, its sample size and its known agreement · the number is only as good as the classifier that produced it ### Data model (create this before Phase 1) - imports(id, at, zip_path, export_date, record_count, sources_json, state) · state is partial or complete, never assumed - records(id, import_id, type, source_name, source_version, device, unit, value, start_at, end_at) · the raw grain, insert only - sleep_segments(id, record_id, source_name, stage, start_at, end_at) · one row per interval exactly as exported - nights(date, sleep_start, sleep_end, total_sleep_min, stage_minutes_json, primary_source, overlap_dropped_min, conflict) - daily(date, metric, value, n_records, primary_source, conflict) · one chosen source per metric per day, and the choice is recorded - baselines(metric, date, window_days, median, p25, p75, n) · what any single value is compared against - comparisons(id, metric_a, metric_b, lag_days, window_days, n, coefficient, ci_low, ci_high, declared_at) - There is no column for a composite score, and stage minutes are stored so they can be shown in bands rather than to the minute. ### Phase 1 · An export that does not fit in memory Build: Stream the export with an incremental parser, clearing each element after handling it, and insert in batches inside a transaction. Record the export date, the record count and every source name and version found. Make the import atomic: an interrupted run leaves no partial import visible to the dashboard. Trap: Loading the export with a document parser builds the whole file as objects in memory, takes every gigabyte the machine has and dies several minutes in, and the obvious fix of raising the limit only moves the moment of death. Parse incrementally, clear each element as you finish with it, and batch the inserts so the transaction log does not become the new problem. Trap: Re-importing after adding a metric duplicates every record you already had, and because health data is full of legitimately repeated values you cannot spot it afterwards by eye. Key records on their natural identity and make a re-import a no-op for anything already present, then prove it by importing the same file twice and comparing counts. Done when: An export of a stated size imports within a stated memory ceiling, measured and written in the README. Importing the same export twice produces identical record counts, tested. Killing the process mid-import leaves either no import or a complete one, tested by killing it. The import records the export date, record count and the set of source names and versions, and the dashboard shows them. Do not build yet: No source resolution, charts, baselines or comparisons yet. ### Phase 2 · Two devices, one day, and the double count Build: Pick one source per metric per day by an explicit configured preference, record which one was used, and count the days where sources disagreed. Merge overlapping sleep intervals rather than summing them, and store the minutes of overlap you dropped. Changing the preference recomputes the derived tables and touches no raw record. Trap: Summing every step or sleep record for a day adds the phone's count to the watch's and gives somebody who carried both a day that never happened, which is the single most common error in analysis of these exports. Choose one source per metric per day, write down the rule, and show the count of days where the sources disagreed rather than hiding the decision. Trap: Summing sleep intervals from two sources that both recorded the same night produces more sleep than there were hours, and a chart that occasionally exceeds twenty-four hours is a chart nobody trusts again. Merge intervals by union, store the overlap you removed, and assert that no night exceeds the elapsed wall clock between its start and end. Done when: A crafted day with records from two sources yields one daily value and a recorded primary source, tested. Overlapping sleep intervals are merged rather than summed and no night's total sleep exceeds the elapsed time between its start and end, asserted as a property. Switching the source preference recomputes affected days and leaves every raw record unchanged, verified by hash. The number of days with conflicting sources is visible on the dashboard. Do not build yet: No sleep presentation, baselines or comparisons yet. ### Phase 3 · Sleep at the resolution the device actually has Build: The sleep page leads with total sleep time and sleep timing as a rolling series. Stage durations appear in coarse bands rather than to the minute, each labelled with the agreement figure from the validation literature and the study's date. No stage gets a trend line over a window shorter than the one you can defend, and the interface has no path to a bare minute count. Trap: Rendering stage durations to the minute implies a measurement the device does not make: the published validation reports four-stage agreement between kappa 0.21 and 0.53 across six wearables and correctly classified deep sleep epochs between about 47 and 70 percent, so the deep sleep figure is close to a coin flip per epoch. Show bands, put the agreement figure beside the number, and remove every code path that can display a stage minute count on its own. Trap: Comparing tonight's stage minutes against last night's and drawing an arrow presents the difference between two noisy classifications as a change in the person. Compare against a rolling baseline with its spread, and require a stated minimum number of nights before any comparison is drawn at all. Done when: No interface path displays a stage duration as a bare minute figure, verified by review and by a test over the rendered templates. Every stage figure carries the agreement statistic and the source study's citation and date. A night-to-night arrow appears nowhere; comparisons are against a baseline with its spread and its sample size. Total sleep time and sleep timing are the first things on the page, checked by review. Do not build yet: No correlations, export or staleness handling yet. ### Phase 4 · Comparisons declared before they are computed Build: Declare the metric pairs and lags in configuration with the date you declared them. Compute each with its confidence interval and its sample size, state how many comparisons the dashboard runs in total, and refuse to display one below a configured minimum sample. Every result is phrased as an association and the interface contains no causal wording. Trap: A panel that tries every metric against every other one over a month will find several convincing relationships, because that is what running forty tests does, and the one you remember is the one that matched a story you already had. Fix the pairs in configuration with a declaration date, show the total number of comparisons run beside the results, and treat anything you add afterwards as a new declaration rather than a finding. Trap: Reporting a coefficient without its interval and sample size makes a relationship computed from eleven nights look like one computed from three hundred. Show the interval and the count everywhere the coefficient appears, and refuse to render below the minimum. Done when: Comparisons come only from the declared configuration and each row carries its declaration date, asserted. A coefficient never appears without its confidence interval and sample size, verified over the templates. The total count of declared comparisons is displayed alongside the results. Any pair below the configured minimum sample renders as not yet computable with the count needed, tested at the boundary. Do not build yet: No staleness handling, export or deletion yet. ### Phase 5 · Data that goes stale, and saying so Build: Show the export date at the top of every page and the age of the data beside it. Stop every axis at the export date rather than at today. Grey out anything that would imply current information. Export the derived tables as CSV, and provide one command that removes the database, the extracted files and the original zip together. Trap: Drawing today's date on the axis when the data ends eleven days ago makes a stale number look current, and the whole failure mode of this build is a dashboard that quietly stops being about now. Stop the axis at the export date, print the data age on every page, and show a prompt to re-export once the age crosses a configured threshold. Trap: Leaving the extracted export files beside the database means a delete that removes the database leaves a complete copy of your health record on disk, usually in a directory nobody remembers. Make deletion one command covering the database, the extracted files and the original archive, and print what it removed. Done when: Every page shows the export date and the data age, and no chart axis extends past the export date, verified by review. Crossing the staleness threshold shows a re-export prompt naming the age, tested at the boundary. Exporting produces the derived tables as CSV and reimports into an empty database with identical row counts. The delete command removes the database, extracted files and the original archive, and prints each path it removed, tested. Do not build yet: Nothing after this. ### Out of scope (and why) - Live background synchronisation, which needs an actual phone application, a developer account and a review process, and is the entire reason the paid product exists. - A composite readiness or recovery score, because no published work establishes that such a number predicts anything about an individual, and presenting one would be the only dishonest thing in this build. - Clinical interpretation of any kind, including sleep disorder screening, since the same validation work that sets the limits here is explicit that these devices do not replace polysomnography for diagnosis. - Non-Apple device sources and whatever normalisation would be needed between them, which is a per-vendor data problem rather than a weekend of code. ### README must contain - The published agreement figures for wearable sleep staging, with the study, the devices and the date, stated before any instructions - That wrist heart rate variability is shown as a rolling baseline because its published agreement approaches the measure's own variation - That there is no composite score and why one is not provided - That one source is chosen per metric per day, what the rule is, and how many days had a conflict - That the export goes stale, how the dashboard shows that, and how to delete everything including the archive ===== ARCHITECTURE.md ===== # Architecture · Bevel ## Stack | Part | Choice | Why | | --- | --- | --- | | Runtime | Python 3.12 with a local web dashboard, nothing leaving the machine | this is a record of your body and it belongs on your disk | | Ingest | An incremental XML parser writing straight to SQLite in batches | the export is hundreds of megabytes and a document parser will not survive it | | Storage | SQLite, one row per exported record, source and device retained | you cannot resolve a double count without knowing which device wrote what | | Analysis | A declared set of comparisons with intervals and sample sizes | a dashboard that tries every pair is a machine for generating findings | | Presentation | Every figure carries its device, its sample size and its known agreement | the number is only as good as the classifier that produced it | ## Modules Each module has one owner concern and a documented way to replace it. | Module | Owns | How to replace it | | --- | --- | --- | | Ingest | Streaming parse, batching and atomic imports. | Any parser, provided it never builds the document in memory. | | Resolve | Source choice, interval merging and conflict counts. | Any rule, provided the choice is recorded and raw records are untouched. | | Presentation | Bands, baselines and the agreement labels. | Any charting, provided no bare stage minute count can be rendered. | | Comparisons | Declared pairs, intervals and sample sizes. | Any statistic, provided it is declared before it is computed. | | Lifecycle | Staleness, export and complete deletion. | Any storage, provided deletion covers the archive and the extraction. | ## Configuration Every runtime setting is an environment variable documented in `.env.example`, validated at startup, with a safe local default wherever one exists. - `DATABASE_PATH` · required · Raw records and derived tables. Everything about your body; keep it local. - `EXTRACT_DIR` · required · Where the archive is unpacked. Removed by the delete command along with the archive. - `IMPORT_BATCH_SIZE` · required · Rows per insert batch. Larger is faster and uses more memory. - `SOURCE_PREFERENCE` · required · Order used to pick one source per metric per day. Changing it recomputes derived tables. - `BASELINE_WINDOW_DAYS` · required · Rolling window any single value is compared against. - `MIN_COMPARISON_N` · required · Below this many paired days, a comparison is shown as not yet computable. - `STALE_AFTER_DAYS` · required · Data age at which the dashboard prompts for a fresh export. ## Production baseline - Security: least privilege, input validation at every boundary, secret redaction in logs, rate limits on abuse-prone paths, no invented security primitives. - Data: explicit schema and migrations, transactional writes where integrity matters, backup and restore procedures that have been exercised. - Integrations: adapters around third-party providers, idempotent webhook or job processing, bounded retries, timeouts. - Observability: structured logs with request or operation ids, an error-tracking hook, and health and readiness checks where a server exists. - Quality: unit tests for domain rules, integration tests at module boundaries, one end-to-end test of the critical path. ## Decision records For each dependency in the stack table, keep a short note: why it was chosen, its failure mode, and how it is replaced. Do not add infrastructure until a requirement in `PRODUCT.md` justifies it. ===== AGENTS.md ===== # Agent instructions · Bevel product build - Read `PRODUCT.md` and `ARCHITECTURE.md` before changing code. The stack is fixed: Python 3.12 with a local web dashboard, nothing leaving the machine, An incremental XML parser writing straight to SQLite in batches, SQLite, one row per exported record, source and device retained, A declared set of comparisons with intervals and sample sizes, Every figure carries its device, its sample size and its known agreement. - Implement milestone by milestone from `MILESTONES.md`; keep each change reviewable and leave the application runnable at every commit. - Treat authentication, payments, encryption, imports, webhooks and destructive actions as high-risk boundaries when present. - Never invent cryptography or silently weaken a requirement to make a check pass. - Put every external service behind an interface with a deterministic fake for tests. - Add migrations and rollback or recovery notes for every persistent data change. - Log useful operational context without credentials, tokens, passwords or personal data. - Update documentation and run every check before completing a milestone. ## Known traps - Loading the export with a document parser builds the whole file as objects in memory, takes every gigabyte the machine has and dies several minutes in, and the obvious fix of raising the limit only moves the moment of death. Parse incrementally, clear each element as you finish with it, and batch the inserts so the transaction log does not become the new problem. - Re-importing after adding a metric duplicates every record you already had, and because health data is full of legitimately repeated values you cannot spot it afterwards by eye. Key records on their natural identity and make a re-import a no-op for anything already present, then prove it by importing the same file twice and comparing counts. - Summing every step or sleep record for a day adds the phone's count to the watch's and gives somebody who carried both a day that never happened, which is the single most common error in analysis of these exports. Choose one source per metric per day, write down the rule, and show the count of days where the sources disagreed rather than hiding the decision. - Summing sleep intervals from two sources that both recorded the same night produces more sleep than there were hours, and a chart that occasionally exceeds twenty-four hours is a chart nobody trusts again. Merge intervals by union, store the overlap you removed, and assert that no night exceeds the elapsed wall clock between its start and end. - Rendering stage durations to the minute implies a measurement the device does not make: the published validation reports four-stage agreement between kappa 0.21 and 0.53 across six wearables and correctly classified deep sleep epochs between about 47 and 70 percent, so the deep sleep figure is close to a coin flip per epoch. Show bands, put the agreement figure beside the number, and remove every code path that can display a stage minute count on its own. - Comparing tonight's stage minutes against last night's and drawing an arrow presents the difference between two noisy classifications as a change in the person. Compare against a rolling baseline with its spread, and require a stated minimum number of nights before any comparison is drawn at all. - A panel that tries every metric against every other one over a month will find several convincing relationships, because that is what running forty tests does, and the one you remember is the one that matched a story you already had. Fix the pairs in configuration with a declaration date, show the total number of comparisons run beside the results, and treat anything you add afterwards as a new declaration rather than a finding. - Reporting a coefficient without its interval and sample size makes a relationship computed from eleven nights look like one computed from three hundred. Show the interval and the count everywhere the coefficient appears, and refuse to render below the minimum. - Drawing today's date on the axis when the data ends eleven days ago makes a stale number look current, and the whole failure mode of this build is a dashboard that quietly stops being about now. Stop the axis at the export date, print the data age on every page, and show a prompt to re-export once the age crosses a configured threshold. - Leaving the extracted export files beside the database means a delete that removes the database leaves a complete copy of your health record on disk, usually in a directory nobody remembers. Make deletion one command covering the database, the extracted files and the original archive, and print what it removed. ===== MILESTONES.md ===== # Delivery milestones · Bevel Estimated effort: **weekend** for the indie phases; the production-only milestones add the trust and operability layer. ## M1 · An export that does not fit in memory A multi-hundred-megabyte export imports within a stated memory ceiling and either completes or leaves nothing. ### Steps 1. Parse incrementally and clear as you go The file is bigger than the memory you want to spend. Files: `src/ingest/stream.py`, `tests/test_memory.py` 2. Make a re-import a no-op for existing records Duplicated health records cannot be spotted by eye later. Files: `src/ingest/dedupe.py` 3. Make the import atomic A half import is a dashboard quietly showing half a life. Files: `src/ingest/transaction.py` ### Done when - [ ] A large export imports under the stated memory ceiling - [ ] Importing twice yields identical record counts - [ ] An interrupted import leaves nothing partial - [ ] Export date, count and sources are recorded and shown ### Watch out - Loading the export with a document parser builds the whole file as objects in memory, takes every gigabyte the machine has and dies several minutes in, and the obvious fix of raising the limit only moves the moment of death. Parse incrementally, clear each element as you finish with it, and batch the inserts so the transaction log does not become the new problem. - Re-importing after adding a metric duplicates every record you already had, and because health data is full of legitimately repeated values you cannot spot it afterwards by eye. Key records on their natural identity and make a re-import a no-op for anything already present, then prove it by importing the same file twice and comparing counts. ## M2 · Two devices, one day, and the double count Each metric on each day comes from one chosen source, and what was discarded is recorded. ### Steps 1. Choose one source per metric per day and record it Adding two devices together invents a day. Files: `src/resolve/sources.py`, `tests/test_double_count.py` 2. Merge sleep intervals by union, keep the overlap figure Two devices recording one night is not two nights. Files: `src/resolve/sleep.py` 3. Recompute derived tables without touching raw records The raw import is the only thing you cannot re-derive. Files: `src/resolve/recompute.py` ### Done when - [ ] A two-source day yields one value and a recorded source - [ ] No night's sleep exceeds its own elapsed time - [ ] Changing the preference leaves raw records untouched - [ ] Conflicting-source day count is visible ### Watch out - Summing every step or sleep record for a day adds the phone's count to the watch's and gives somebody who carried both a day that never happened, which is the single most common error in analysis of these exports. Choose one source per metric per day, write down the rule, and show the count of days where the sources disagreed rather than hiding the decision. - Summing sleep intervals from two sources that both recorded the same night produces more sleep than there were hours, and a chart that occasionally exceeds twenty-four hours is a chart nobody trusts again. Merge intervals by union, store the overlap you removed, and assert that no night exceeds the elapsed wall clock between its start and end. ## M3 · Sleep at the resolution the device actually has Total sleep and timing lead, stages appear in bands with their published agreement attached. ### Steps 1. Lead with total sleep and timing They are the parts the devices are measured as detecting well. Files: `src/web/sleep.py`, `src/web/templates/sleep.html` 2. Show stages in bands with the agreement figure The number and its uncertainty travel together or not at all. Files: `src/analysis/stages.py`, `tests/test_no_bare_minutes.py` 3. Compare against a baseline, never against last night Two noisy classifications do not make a trend. Files: `src/analysis/baseline.py` ### Done when - [ ] No path shows a bare stage minute count - [ ] Every stage figure carries its agreement statistic and source - [ ] Comparisons are against a baseline with spread and n - [ ] Total sleep and timing lead the page ### Watch out - Rendering stage durations to the minute implies a measurement the device does not make: the published validation reports four-stage agreement between kappa 0.21 and 0.53 across six wearables and correctly classified deep sleep epochs between about 47 and 70 percent, so the deep sleep figure is close to a coin flip per epoch. Show bands, put the agreement figure beside the number, and remove every code path that can display a stage minute count on its own. - Comparing tonight's stage minutes against last night's and drawing an arrow presents the difference between two noisy classifications as a change in the person. Compare against a rolling baseline with its spread, and require a stated minimum number of nights before any comparison is drawn at all. ## M4 · Comparisons declared before they are computed The correlations on the dashboard are the ones you chose in advance, reported with intervals and counts. ### Steps 1. Declare pairs and lags in configuration with a date A hypothesis chosen after the data is not a hypothesis. Files: `config/comparisons.toml`, `src/analysis/declare.py` 2. Report interval and sample size everywhere The coefficient alone hides how little is behind it. Files: `src/analysis/correlate.py`, `tests/test_intervals.py` 3. Say how many comparisons the dashboard runs It is the context that makes one result readable. Files: `src/web/templates/insights.html` ### Done when - [ ] Only declared comparisons appear, each with its date - [ ] No coefficient appears without interval and sample size - [ ] The total comparison count is displayed - [ ] Under-sampled pairs render as not yet computable ### Watch out - A panel that tries every metric against every other one over a month will find several convincing relationships, because that is what running forty tests does, and the one you remember is the one that matched a story you already had. Fix the pairs in configuration with a declaration date, show the total number of comparisons run beside the results, and treat anything you add afterwards as a new declaration rather than a finding. - Reporting a coefficient without its interval and sample size makes a relationship computed from eleven nights look like one computed from three hundred. Show the interval and the count everywhere the coefficient appears, and refuse to render below the minimum. ## M5 · Data that goes stale, and saying so (production only) The dashboard is honest about how old the export is, everything exports, and deleting means deleting. ### Steps 1. Stop every axis at the export date An axis that runs to today is a lie about the data. Files: `src/web/charts.py` 2. Show the data age and prompt a re-export The staleness is the honest headline of this whole build. Files: `src/web/templates/base.html` 3. Make deletion cover the zip and the extraction A deleted database next to an extracted export is not deleted. Files: `src/admin/delete.py`, `tests/test_delete.py` ### Done when - [ ] Export date and data age appear on every page - [ ] No axis extends past the export date - [ ] Derived tables export and reimport with identical counts - [ ] Deletion removes database, extraction and archive ### Watch out - Drawing today's date on the axis when the data ends eleven days ago makes a stale number look current, and the whole failure mode of this build is a dashboard that quietly stops being about now. Stop the axis at the export date, print the data age on every page, and show a prompt to re-export once the age crosses a configured threshold. - Leaving the extracted export files beside the database means a delete that removes the database leaves a complete copy of your health record on disk, usually in a directory nobody remembers. Make deletion one command covering the database, the extracted files and the original archive, and print what it removed. ===== OPERATIONS.md ===== # Operations · Bevel ## Backup The original archive is the backup, since the database is derived from it and can be rebuilt. Keep the archives you have imported, because they are the only record of what your devices said at the time, and export requests are slow. ## Restore Re-import the archive into an empty database and confirm the record count and the conflicting-source day count match what the previous import reported. If they do not, the resolution rules changed rather than the data. Do a restore drill before the first real user, and write the date here when it passes. ## Monitoring For a local dashboard the checks live in the suite plus one number on the page: the data age. Beyond that, memory used by the import, the conflicting-source day count, and the proportion of nights where stage data came from a source you did not prefer. ## Incident checklist The incident here is a decision somebody made about their health from a figure the device does not actually measure. The agreement labels, the bands and the declared comparison list exist so that the dashboard cannot present that figure in the first place. 1. Contain the issue without destroying evidence or user data. 2. Record the timeline and affected scope. 3. Rotate exposed secrets and revoke compromised sessions or credentials. 4. Restore from a verified backup when needed. 5. Document the root cause, the remediation and the regression test. ## Release gate - [ ] A large export is confirmed to import under the stated memory ceiling - [ ] No night's total sleep is confirmed to exceed its own elapsed time - [ ] No interface path is confirmed to show a bare stage minute count - [ ] Only declared comparisons are confirmed to appear, with intervals and counts - [ ] Deletion is confirmed to remove database, extraction and archive ## Launch constraint Do not market omitted Bevel capabilities as implemented. The non-goals in `PRODUCT.md` remain user-visible limitations until they are deliberately delivered. ===== .env.example ===== # Copy to .env and fill in. Never commit .env; this file documents it. # Required. Raw records and derived tables. Everything about your body; keep it local. DATABASE_PATH=./data/health.db # Required. Where the archive is unpacked. Removed by the delete command along with the archive. EXTRACT_DIR=./data/extract # Required. Rows per insert batch. Larger is faster and uses more memory. IMPORT_BATCH_SIZE=5000 # Required. Order used to pick one source per metric per day. Changing it recomputes derived tables. SOURCE_PREFERENCE=watch,phone # Required. Rolling window any single value is compared against. BASELINE_WINDOW_DAYS=28 # Required. Below this many paired days, a comparison is shown as not yet computable. MIN_COMPARISON_N=30 # Required. Data age at which the dashboard prompts for a fresh export. STALE_AFTER_DAYS=14
README.md
# Bevel · indie build A local analytics dashboard over your own health export whose hard problems are ingesting a file too large to hold in memory, resolving two devices that both recorded the same day, and presenting each metric at the resolution the published validation supports rather than the resolution the file happens to have. Estimated effort: **weekend**. Work `BUILD_PLAN.md` top to bottom · every phase ends in a check that has to pass before the next one starts. ## Stack | Part | Choice | Why | | --- | --- | --- | | Runtime | Python 3.12 with a local web dashboard, nothing leaving the machine | this is a record of your body and it belongs on your disk | | Ingest | An incremental XML parser writing straight to SQLite in batches | the export is hundreds of megabytes and a document parser will not survive it | | Storage | SQLite, one row per exported record, source and device retained | you cannot resolve a double count without knowing which device wrote what | | Analysis | A declared set of comparisons with intervals and sample sizes | a dashboard that tries every pair is a machine for generating findings | | Presentation | Every figure carries its device, its sample size and its known agreement | the number is only as good as the classifier that produced it | ## Before you start Have every one of these ready. The plan assumes them from step one. - [ ] **Python 3.12 or newer** · free - Why: Runs the importer, the analysis and the local dashboard. - Get it: python.org, or your package manager. - Verify: python3 --version prints 3.12 or higher - [ ] **Your own health export archive** · free, plus the wait while it generates - Why: Everything here is built against the real file, and the real file is far larger and far messier than any sample you will find. - Get it: From the health application on your phone, request a full export and transfer the archive to the machine running this. - Verify: The archive extracts and the importer reports a record count and export date - [ ] **Enough disk for the archive, the extraction and the database** · free - Why: The extracted export and the database together are several times the archive, and running out part way through an import is the most common first failure. - Get it: Check free space against the archive size multiplied by a factor you measured on the first import. - Verify: A full import completes and the measured total size is written in the README - [ ] **What each metric is allowed to display** · free - Why: The published validation differs sharply by metric, and deciding presentation after the charts exist means deciding it from whatever looked best. - Get it: Write down, per metric, whether it is shown as a value, a band or a baseline, and which published figure justifies that. - Verify: The table is in the README and each rule is asserted in a test - [ ] **The comparison pairs and lags, declared in advance** · free - Why: Declaring them afterwards turns a dashboard into a search for whatever matches the story you already had. - Get it: List the metric pairs and lags you care about in the configuration file with today's date. - Verify: The dashboard shows only declared pairs and prints the total comparison count ## Quick start Follow Phase 1 in `BUILD_PLAN.md`; it creates the project. Then copy `.env.example` to `.env` and fill in the values it documents. ## Honest limits This build deliberately does not replace: - Live background synchronisation. - A composite readiness or recovery score. - Clinical interpretation of any kind. - Non-Apple device sources. - Live background sync; you are re-exporting a zip by hand - A phone app, widgets and notifications - Their opinionated composite scores and plain-English daily readouts - Non-Apple device integrations and whatever normalisation they do across sources - Charts that a designer looked at If one of those is essential to you, that is the reason to keep paying for Bevel, and the README should say so rather than pretend.
BRIEF.md
# Build brief · Bevel The one-shot brief this plan expands. `BUILD_PLAN.md` (or `MILESTONES.md`) is the same sequence broken into steps and checks; where the two disagree, the plan wins. Build me a local dashboard over your own health export that shows each metric at the resolution its device actually has to replace Bevel. Build it in phases, in the order below. Do not write the whole thing in one pass. Finish a phase, run its "Done when" check, fix what fails, and only then start the next phase. ### Rule zero This app displays physiological measurements, so what the published validation says decides what it is allowed to show. Read this before designing a single chart. Sleep stages are not measured, they are classified, and not well. A 2025 validation in SLEEP Advances put six wrist wearables against polysomnography in 62 adults. Four-stage agreement ran from a Cohen's kappa of 0.21 to 0.53, the best being the Apple Watch Series 8 at 0.53. Every device found over 90 percent of sleep epochs, sensitivity 91.68 to 96.27 percent, but specificity for wake ran only 29.39 to 52.15 percent and correctly classified deep sleep epochs ran 47.46 to 69.63 percent, the Apple Watch at 50.66 percent with REM at 68.57. A screen reading '1 h 12 m of deep sleep' is reporting a classifier close to a coin flip per epoch. Lead with total sleep time and sleep timing, which the same study found detected well, and show stages in coarse bands with the agreement figure beside them. Wrist heart rate variability is the same story: published validations find heart rate tracks an electrocardiogram closely while the beat-to-beat variability measures do not, with reported agreement approaching or exceeding the measure's own day-to-day variation. Show it as a rolling baseline with a band, never as tonight's number against yesterday's. Therefore: no composite score, because nobody has validated that a number called readiness or recovery predicts anything about you. And declare your correlations before you compute them: testing every habit against every metric over thirty days finds something by construction. ### Stack (fixed, do not substitute) - Runtime: Python 3.12 with a local web dashboard, nothing leaving the machine · this is a record of your body and it belongs on your disk - Ingest: An incremental XML parser writing straight to SQLite in batches · the export is hundreds of megabytes and a document parser will not survive it - Storage: SQLite, one row per exported record, source and device retained · you cannot resolve a double count without knowing which device wrote what - Analysis: A declared set of comparisons with intervals and sample sizes · a dashboard that tries every pair is a machine for generating findings - Presentation: Every figure carries its device, its sample size and its known agreement · the number is only as good as the classifier that produced it ### Data model (create this before Phase 1) - imports(id, at, zip_path, export_date, record_count, sources_json, state) · state is partial or complete, never assumed - records(id, import_id, type, source_name, source_version, device, unit, value, start_at, end_at) · the raw grain, insert only - sleep_segments(id, record_id, source_name, stage, start_at, end_at) · one row per interval exactly as exported - nights(date, sleep_start, sleep_end, total_sleep_min, stage_minutes_json, primary_source, overlap_dropped_min, conflict) - daily(date, metric, value, n_records, primary_source, conflict) · one chosen source per metric per day, and the choice is recorded - baselines(metric, date, window_days, median, p25, p75, n) · what any single value is compared against - comparisons(id, metric_a, metric_b, lag_days, window_days, n, coefficient, ci_low, ci_high, declared_at) - There is no column for a composite score, and stage minutes are stored so they can be shown in bands rather than to the minute. ### Phase 1 · An export that does not fit in memory Build: Stream the export with an incremental parser, clearing each element after handling it, and insert in batches inside a transaction. Record the export date, the record count and every source name and version found. Make the import atomic: an interrupted run leaves no partial import visible to the dashboard. Trap: Loading the export with a document parser builds the whole file as objects in memory, takes every gigabyte the machine has and dies several minutes in, and the obvious fix of raising the limit only moves the moment of death. Parse incrementally, clear each element as you finish with it, and batch the inserts so the transaction log does not become the new problem. Trap: Re-importing after adding a metric duplicates every record you already had, and because health data is full of legitimately repeated values you cannot spot it afterwards by eye. Key records on their natural identity and make a re-import a no-op for anything already present, then prove it by importing the same file twice and comparing counts. Done when: An export of a stated size imports within a stated memory ceiling, measured and written in the README. Importing the same export twice produces identical record counts, tested. Killing the process mid-import leaves either no import or a complete one, tested by killing it. The import records the export date, record count and the set of source names and versions, and the dashboard shows them. Do not build yet: No source resolution, charts, baselines or comparisons yet. ### Phase 2 · Two devices, one day, and the double count Build: Pick one source per metric per day by an explicit configured preference, record which one was used, and count the days where sources disagreed. Merge overlapping sleep intervals rather than summing them, and store the minutes of overlap you dropped. Changing the preference recomputes the derived tables and touches no raw record. Trap: Summing every step or sleep record for a day adds the phone's count to the watch's and gives somebody who carried both a day that never happened, which is the single most common error in analysis of these exports. Choose one source per metric per day, write down the rule, and show the count of days where the sources disagreed rather than hiding the decision. Trap: Summing sleep intervals from two sources that both recorded the same night produces more sleep than there were hours, and a chart that occasionally exceeds twenty-four hours is a chart nobody trusts again. Merge intervals by union, store the overlap you removed, and assert that no night exceeds the elapsed wall clock between its start and end. Done when: A crafted day with records from two sources yields one daily value and a recorded primary source, tested. Overlapping sleep intervals are merged rather than summed and no night's total sleep exceeds the elapsed time between its start and end, asserted as a property. Switching the source preference recomputes affected days and leaves every raw record unchanged, verified by hash. The number of days with conflicting sources is visible on the dashboard. Do not build yet: No sleep presentation, baselines or comparisons yet. ### Phase 3 · Sleep at the resolution the device actually has Build: The sleep page leads with total sleep time and sleep timing as a rolling series. Stage durations appear in coarse bands rather than to the minute, each labelled with the agreement figure from the validation literature and the study's date. No stage gets a trend line over a window shorter than the one you can defend, and the interface has no path to a bare minute count. Trap: Rendering stage durations to the minute implies a measurement the device does not make: the published validation reports four-stage agreement between kappa 0.21 and 0.53 across six wearables and correctly classified deep sleep epochs between about 47 and 70 percent, so the deep sleep figure is close to a coin flip per epoch. Show bands, put the agreement figure beside the number, and remove every code path that can display a stage minute count on its own. Trap: Comparing tonight's stage minutes against last night's and drawing an arrow presents the difference between two noisy classifications as a change in the person. Compare against a rolling baseline with its spread, and require a stated minimum number of nights before any comparison is drawn at all. Done when: No interface path displays a stage duration as a bare minute figure, verified by review and by a test over the rendered templates. Every stage figure carries the agreement statistic and the source study's citation and date. A night-to-night arrow appears nowhere; comparisons are against a baseline with its spread and its sample size. Total sleep time and sleep timing are the first things on the page, checked by review. Do not build yet: No correlations, export or staleness handling yet. ### Phase 4 · Comparisons declared before they are computed Build: Declare the metric pairs and lags in configuration with the date you declared them. Compute each with its confidence interval and its sample size, state how many comparisons the dashboard runs in total, and refuse to display one below a configured minimum sample. Every result is phrased as an association and the interface contains no causal wording. Trap: A panel that tries every metric against every other one over a month will find several convincing relationships, because that is what running forty tests does, and the one you remember is the one that matched a story you already had. Fix the pairs in configuration with a declaration date, show the total number of comparisons run beside the results, and treat anything you add afterwards as a new declaration rather than a finding. Trap: Reporting a coefficient without its interval and sample size makes a relationship computed from eleven nights look like one computed from three hundred. Show the interval and the count everywhere the coefficient appears, and refuse to render below the minimum. Done when: Comparisons come only from the declared configuration and each row carries its declaration date, asserted. A coefficient never appears without its confidence interval and sample size, verified over the templates. The total count of declared comparisons is displayed alongside the results. Any pair below the configured minimum sample renders as not yet computable with the count needed, tested at the boundary. Do not build yet: No staleness handling, export or deletion yet. ### Phase 5 · Data that goes stale, and saying so Build: Show the export date at the top of every page and the age of the data beside it. Stop every axis at the export date rather than at today. Grey out anything that would imply current information. Export the derived tables as CSV, and provide one command that removes the database, the extracted files and the original zip together. Trap: Drawing today's date on the axis when the data ends eleven days ago makes a stale number look current, and the whole failure mode of this build is a dashboard that quietly stops being about now. Stop the axis at the export date, print the data age on every page, and show a prompt to re-export once the age crosses a configured threshold. Trap: Leaving the extracted export files beside the database means a delete that removes the database leaves a complete copy of your health record on disk, usually in a directory nobody remembers. Make deletion one command covering the database, the extracted files and the original archive, and print what it removed. Done when: Every page shows the export date and the data age, and no chart axis extends past the export date, verified by review. Crossing the staleness threshold shows a re-export prompt naming the age, tested at the boundary. Exporting produces the derived tables as CSV and reimports into an empty database with identical row counts. The delete command removes the database, extracted files and the original archive, and prints each path it removed, tested. Do not build yet: Nothing after this. ### Out of scope (and why) - Live background synchronisation, which needs an actual phone application, a developer account and a review process, and is the entire reason the paid product exists. - A composite readiness or recovery score, because no published work establishes that such a number predicts anything about an individual, and presenting one would be the only dishonest thing in this build. - Clinical interpretation of any kind, including sleep disorder screening, since the same validation work that sets the limits here is explicit that these devices do not replace polysomnography for diagnosis. - Non-Apple device sources and whatever normalisation would be needed between them, which is a per-vendor data problem rather than a weekend of code. ### README must contain - The published agreement figures for wearable sleep staging, with the study, the devices and the date, stated before any instructions - That wrist heart rate variability is shown as a rolling baseline because its published agreement approaches the measure's own variation - That there is no composite score and why one is not provided - That one source is chosen per metric per day, what the rule is, and how many days had a conflict - That the export goes stale, how the dashboard shows that, and how to delete everything including the archive
AGENTS.md
# Agent instructions · Bevel indie build - Read `README.md` and `BUILD_PLAN.md` before writing code. The stack is fixed: Python 3.12 with a local web dashboard, nothing leaving the machine, An incremental XML parser writing straight to SQLite in batches, SQLite, one row per exported record, source and device retained, A declared set of comparisons with intervals and sample sizes, Every figure carries its device, its sample size and its known agreement. Do not substitute. - Work one phase at a time, in order. Do not start a phase until every "Done when" item of the previous one passes. - Prefer the fewest moving parts that satisfy the step. No frameworks, services or dependencies the plan does not name. - Secrets live in `.env`, never in source or logs. Keep `.env.example` current when a variable is introduced. - Do not invent cryptography, security guarantees, APIs or compliance claims. - Add a focused test for every destructive, security-sensitive or data-loss path the plan names. - Run the project checks before declaring a phase complete, and record any deliberate shortcut in the README under "Tradeoffs". ## Known traps - Loading the export with a document parser builds the whole file as objects in memory, takes every gigabyte the machine has and dies several minutes in, and the obvious fix of raising the limit only moves the moment of death. Parse incrementally, clear each element as you finish with it, and batch the inserts so the transaction log does not become the new problem. - Re-importing after adding a metric duplicates every record you already had, and because health data is full of legitimately repeated values you cannot spot it afterwards by eye. Key records on their natural identity and make a re-import a no-op for anything already present, then prove it by importing the same file twice and comparing counts. - Summing every step or sleep record for a day adds the phone's count to the watch's and gives somebody who carried both a day that never happened, which is the single most common error in analysis of these exports. Choose one source per metric per day, write down the rule, and show the count of days where the sources disagreed rather than hiding the decision. - Summing sleep intervals from two sources that both recorded the same night produces more sleep than there were hours, and a chart that occasionally exceeds twenty-four hours is a chart nobody trusts again. Merge intervals by union, store the overlap you removed, and assert that no night exceeds the elapsed wall clock between its start and end. - Rendering stage durations to the minute implies a measurement the device does not make: the published validation reports four-stage agreement between kappa 0.21 and 0.53 across six wearables and correctly classified deep sleep epochs between about 47 and 70 percent, so the deep sleep figure is close to a coin flip per epoch. Show bands, put the agreement figure beside the number, and remove every code path that can display a stage minute count on its own. - Comparing tonight's stage minutes against last night's and drawing an arrow presents the difference between two noisy classifications as a change in the person. Compare against a rolling baseline with its spread, and require a stated minimum number of nights before any comparison is drawn at all. - A panel that tries every metric against every other one over a month will find several convincing relationships, because that is what running forty tests does, and the one you remember is the one that matched a story you already had. Fix the pairs in configuration with a declaration date, show the total number of comparisons run beside the results, and treat anything you add afterwards as a new declaration rather than a finding. - Reporting a coefficient without its interval and sample size makes a relationship computed from eleven nights look like one computed from three hundred. Show the interval and the count everywhere the coefficient appears, and refuse to render below the minimum.
BUILD_PLAN.md
# Build plan · Bevel A local analytics dashboard over your own health export whose hard problems are ingesting a file too large to hold in memory, resolving two devices that both recorded the same day, and presenting each metric at the resolution the published validation supports rather than the resolution the file happens to have. Phases are in dependency order. Each ends in a "Done when" list; treat an unticked item as a blocker, not a note. ## Phase 1 · An export that does not fit in memory A multi-hundred-megabyte export imports within a stated memory ceiling and either completes or leaves nothing. ### Steps 1. Parse incrementally and clear as you go The file is bigger than the memory you want to spend. Files: `src/ingest/stream.py`, `tests/test_memory.py` 2. Make a re-import a no-op for existing records Duplicated health records cannot be spotted by eye later. Files: `src/ingest/dedupe.py` 3. Make the import atomic A half import is a dashboard quietly showing half a life. Files: `src/ingest/transaction.py` ### Done when - [ ] A large export imports under the stated memory ceiling - [ ] Importing twice yields identical record counts - [ ] An interrupted import leaves nothing partial - [ ] Export date, count and sources are recorded and shown ### Watch out - Loading the export with a document parser builds the whole file as objects in memory, takes every gigabyte the machine has and dies several minutes in, and the obvious fix of raising the limit only moves the moment of death. Parse incrementally, clear each element as you finish with it, and batch the inserts so the transaction log does not become the new problem. - Re-importing after adding a metric duplicates every record you already had, and because health data is full of legitimately repeated values you cannot spot it afterwards by eye. Key records on their natural identity and make a re-import a no-op for anything already present, then prove it by importing the same file twice and comparing counts. ## Phase 2 · Two devices, one day, and the double count Each metric on each day comes from one chosen source, and what was discarded is recorded. ### Steps 1. Choose one source per metric per day and record it Adding two devices together invents a day. Files: `src/resolve/sources.py`, `tests/test_double_count.py` 2. Merge sleep intervals by union, keep the overlap figure Two devices recording one night is not two nights. Files: `src/resolve/sleep.py` 3. Recompute derived tables without touching raw records The raw import is the only thing you cannot re-derive. Files: `src/resolve/recompute.py` ### Done when - [ ] A two-source day yields one value and a recorded source - [ ] No night's sleep exceeds its own elapsed time - [ ] Changing the preference leaves raw records untouched - [ ] Conflicting-source day count is visible ### Watch out - Summing every step or sleep record for a day adds the phone's count to the watch's and gives somebody who carried both a day that never happened, which is the single most common error in analysis of these exports. Choose one source per metric per day, write down the rule, and show the count of days where the sources disagreed rather than hiding the decision. - Summing sleep intervals from two sources that both recorded the same night produces more sleep than there were hours, and a chart that occasionally exceeds twenty-four hours is a chart nobody trusts again. Merge intervals by union, store the overlap you removed, and assert that no night exceeds the elapsed wall clock between its start and end. ## Phase 3 · Sleep at the resolution the device actually has Total sleep and timing lead, stages appear in bands with their published agreement attached. ### Steps 1. Lead with total sleep and timing They are the parts the devices are measured as detecting well. Files: `src/web/sleep.py`, `src/web/templates/sleep.html` 2. Show stages in bands with the agreement figure The number and its uncertainty travel together or not at all. Files: `src/analysis/stages.py`, `tests/test_no_bare_minutes.py` 3. Compare against a baseline, never against last night Two noisy classifications do not make a trend. Files: `src/analysis/baseline.py` ### Done when - [ ] No path shows a bare stage minute count - [ ] Every stage figure carries its agreement statistic and source - [ ] Comparisons are against a baseline with spread and n - [ ] Total sleep and timing lead the page ### Watch out - Rendering stage durations to the minute implies a measurement the device does not make: the published validation reports four-stage agreement between kappa 0.21 and 0.53 across six wearables and correctly classified deep sleep epochs between about 47 and 70 percent, so the deep sleep figure is close to a coin flip per epoch. Show bands, put the agreement figure beside the number, and remove every code path that can display a stage minute count on its own. - Comparing tonight's stage minutes against last night's and drawing an arrow presents the difference between two noisy classifications as a change in the person. Compare against a rolling baseline with its spread, and require a stated minimum number of nights before any comparison is drawn at all. ## Phase 4 · Comparisons declared before they are computed The correlations on the dashboard are the ones you chose in advance, reported with intervals and counts. ### Steps 1. Declare pairs and lags in configuration with a date A hypothesis chosen after the data is not a hypothesis. Files: `config/comparisons.toml`, `src/analysis/declare.py` 2. Report interval and sample size everywhere The coefficient alone hides how little is behind it. Files: `src/analysis/correlate.py`, `tests/test_intervals.py` 3. Say how many comparisons the dashboard runs It is the context that makes one result readable. Files: `src/web/templates/insights.html` ### Done when - [ ] Only declared comparisons appear, each with its date - [ ] No coefficient appears without interval and sample size - [ ] The total comparison count is displayed - [ ] Under-sampled pairs render as not yet computable ### Watch out - A panel that tries every metric against every other one over a month will find several convincing relationships, because that is what running forty tests does, and the one you remember is the one that matched a story you already had. Fix the pairs in configuration with a declaration date, show the total number of comparisons run beside the results, and treat anything you add afterwards as a new declaration rather than a finding. - Reporting a coefficient without its interval and sample size makes a relationship computed from eleven nights look like one computed from three hundred. Show the interval and the count everywhere the coefficient appears, and refuse to render below the minimum. ## Not in this build - Live background synchronisation. - A composite readiness or recovery score. - Clinical interpretation of any kind. - Non-Apple device sources. ## After v1, if you want it - A per-metric page showing what the published validation says about that metric, which is the honest version of an explanation screen - Incremental import of successive exports so the history accumulates rather than being replaced each time - A single annual summary at the timescale the agreement figures actually support
.env.example
# Copy to .env and fill in. Never commit .env; this file documents it. # Required. Raw records and derived tables. Everything about your body; keep it local. DATABASE_PATH=./data/health.db # Required. Where the archive is unpacked. Removed by the delete command along with the archive. EXTRACT_DIR=./data/extract # Required. Rows per insert batch. Larger is faster and uses more memory. IMPORT_BATCH_SIZE=5000 # Required. Order used to pick one source per metric per day. Changing it recomputes derived tables. SOURCE_PREFERENCE=watch,phone # Required. Rolling window any single value is compared against. BASELINE_WINDOW_DAYS=28 # Required. Below this many paired days, a comparison is shown as not yet computable. MIN_COMPARISON_N=30 # Required. Data age at which the dashboard prompts for a fresh export. STALE_AFTER_DAYS=14
PRODUCT.md
# Bevel · product brief ## Problem Bevel is not sitting on secret data: it reads the same HealthKit records your watch already wrote, then does math and draws charts. An agent can absolutely build you a local dashboard over an Apple Health export that computes rolling averages, sleep and HRV trends, and lag correlations between habits and recovery. The gap is delivery, not analysis: real HealthKit access means an actual iOS app, Xcode, a developer account and background sync, and manual export zips get stale fast. You also lose the part Bevel spends most of its effort on, which is turning noisy sensor data into something you glance at once a day and actually understand. Fine for a curious quantified-self person, annoying for anyone who wants a phone widget. ## Product outcome A dashboard that is honest about a measurement the literature says is rough: sleep stages in bands with their agreement attached, variability as a baseline rather than a nightly number, no composite score, only declared comparisons, and the data's age on every page. ## Target user A builder who needs a maintainable product foundation, not a one-off demo. ## Required capabilities - An Apple Health export zip (Health app, profile, Export All Health Data) - Python 3.12 locally - Patience with a multi-hundred-megabyte export.xml - An Apple Developer account and Xcode if you want live HealthKit sync instead of manual exports ## Explicit non-goals for v1 - Live background synchronisation. - A composite readiness or recovery score. - Clinical interpretation of any kind. - Non-Apple device sources. - Live background sync; you are re-exporting a zip by hand - A phone app, widgets and notifications - Their opinionated composite scores and plain-English daily readouts - Non-Apple device integrations and whatever normalisation they do across sources - Charts that a designer looked at ## Success criteria - A large export is confirmed to import under the stated memory ceiling - No night's total sleep is confirmed to exceed its own elapsed time - No interface path is confirmed to show a bare stage minute count - Only declared comparisons are confirmed to appear, with intervals and counts - Deletion is confirmed to remove database, extraction and archive
BRIEF.md
# Build brief · Bevel The one-shot brief this plan expands. `BUILD_PLAN.md` (or `MILESTONES.md`) is the same sequence broken into steps and checks; where the two disagree, the plan wins. Build me a local dashboard over your own health export that shows each metric at the resolution its device actually has to replace Bevel. Build it in phases, in the order below. Do not write the whole thing in one pass. Finish a phase, run its "Done when" check, fix what fails, and only then start the next phase. ### Rule zero This app displays physiological measurements, so what the published validation says decides what it is allowed to show. Read this before designing a single chart. Sleep stages are not measured, they are classified, and not well. A 2025 validation in SLEEP Advances put six wrist wearables against polysomnography in 62 adults. Four-stage agreement ran from a Cohen's kappa of 0.21 to 0.53, the best being the Apple Watch Series 8 at 0.53. Every device found over 90 percent of sleep epochs, sensitivity 91.68 to 96.27 percent, but specificity for wake ran only 29.39 to 52.15 percent and correctly classified deep sleep epochs ran 47.46 to 69.63 percent, the Apple Watch at 50.66 percent with REM at 68.57. A screen reading '1 h 12 m of deep sleep' is reporting a classifier close to a coin flip per epoch. Lead with total sleep time and sleep timing, which the same study found detected well, and show stages in coarse bands with the agreement figure beside them. Wrist heart rate variability is the same story: published validations find heart rate tracks an electrocardiogram closely while the beat-to-beat variability measures do not, with reported agreement approaching or exceeding the measure's own day-to-day variation. Show it as a rolling baseline with a band, never as tonight's number against yesterday's. Therefore: no composite score, because nobody has validated that a number called readiness or recovery predicts anything about you. And declare your correlations before you compute them: testing every habit against every metric over thirty days finds something by construction. ### Stack (fixed, do not substitute) - Runtime: Python 3.12 with a local web dashboard, nothing leaving the machine · this is a record of your body and it belongs on your disk - Ingest: An incremental XML parser writing straight to SQLite in batches · the export is hundreds of megabytes and a document parser will not survive it - Storage: SQLite, one row per exported record, source and device retained · you cannot resolve a double count without knowing which device wrote what - Analysis: A declared set of comparisons with intervals and sample sizes · a dashboard that tries every pair is a machine for generating findings - Presentation: Every figure carries its device, its sample size and its known agreement · the number is only as good as the classifier that produced it ### Data model (create this before Phase 1) - imports(id, at, zip_path, export_date, record_count, sources_json, state) · state is partial or complete, never assumed - records(id, import_id, type, source_name, source_version, device, unit, value, start_at, end_at) · the raw grain, insert only - sleep_segments(id, record_id, source_name, stage, start_at, end_at) · one row per interval exactly as exported - nights(date, sleep_start, sleep_end, total_sleep_min, stage_minutes_json, primary_source, overlap_dropped_min, conflict) - daily(date, metric, value, n_records, primary_source, conflict) · one chosen source per metric per day, and the choice is recorded - baselines(metric, date, window_days, median, p25, p75, n) · what any single value is compared against - comparisons(id, metric_a, metric_b, lag_days, window_days, n, coefficient, ci_low, ci_high, declared_at) - There is no column for a composite score, and stage minutes are stored so they can be shown in bands rather than to the minute. ### Phase 1 · An export that does not fit in memory Build: Stream the export with an incremental parser, clearing each element after handling it, and insert in batches inside a transaction. Record the export date, the record count and every source name and version found. Make the import atomic: an interrupted run leaves no partial import visible to the dashboard. Trap: Loading the export with a document parser builds the whole file as objects in memory, takes every gigabyte the machine has and dies several minutes in, and the obvious fix of raising the limit only moves the moment of death. Parse incrementally, clear each element as you finish with it, and batch the inserts so the transaction log does not become the new problem. Trap: Re-importing after adding a metric duplicates every record you already had, and because health data is full of legitimately repeated values you cannot spot it afterwards by eye. Key records on their natural identity and make a re-import a no-op for anything already present, then prove it by importing the same file twice and comparing counts. Done when: An export of a stated size imports within a stated memory ceiling, measured and written in the README. Importing the same export twice produces identical record counts, tested. Killing the process mid-import leaves either no import or a complete one, tested by killing it. The import records the export date, record count and the set of source names and versions, and the dashboard shows them. Do not build yet: No source resolution, charts, baselines or comparisons yet. ### Phase 2 · Two devices, one day, and the double count Build: Pick one source per metric per day by an explicit configured preference, record which one was used, and count the days where sources disagreed. Merge overlapping sleep intervals rather than summing them, and store the minutes of overlap you dropped. Changing the preference recomputes the derived tables and touches no raw record. Trap: Summing every step or sleep record for a day adds the phone's count to the watch's and gives somebody who carried both a day that never happened, which is the single most common error in analysis of these exports. Choose one source per metric per day, write down the rule, and show the count of days where the sources disagreed rather than hiding the decision. Trap: Summing sleep intervals from two sources that both recorded the same night produces more sleep than there were hours, and a chart that occasionally exceeds twenty-four hours is a chart nobody trusts again. Merge intervals by union, store the overlap you removed, and assert that no night exceeds the elapsed wall clock between its start and end. Done when: A crafted day with records from two sources yields one daily value and a recorded primary source, tested. Overlapping sleep intervals are merged rather than summed and no night's total sleep exceeds the elapsed time between its start and end, asserted as a property. Switching the source preference recomputes affected days and leaves every raw record unchanged, verified by hash. The number of days with conflicting sources is visible on the dashboard. Do not build yet: No sleep presentation, baselines or comparisons yet. ### Phase 3 · Sleep at the resolution the device actually has Build: The sleep page leads with total sleep time and sleep timing as a rolling series. Stage durations appear in coarse bands rather than to the minute, each labelled with the agreement figure from the validation literature and the study's date. No stage gets a trend line over a window shorter than the one you can defend, and the interface has no path to a bare minute count. Trap: Rendering stage durations to the minute implies a measurement the device does not make: the published validation reports four-stage agreement between kappa 0.21 and 0.53 across six wearables and correctly classified deep sleep epochs between about 47 and 70 percent, so the deep sleep figure is close to a coin flip per epoch. Show bands, put the agreement figure beside the number, and remove every code path that can display a stage minute count on its own. Trap: Comparing tonight's stage minutes against last night's and drawing an arrow presents the difference between two noisy classifications as a change in the person. Compare against a rolling baseline with its spread, and require a stated minimum number of nights before any comparison is drawn at all. Done when: No interface path displays a stage duration as a bare minute figure, verified by review and by a test over the rendered templates. Every stage figure carries the agreement statistic and the source study's citation and date. A night-to-night arrow appears nowhere; comparisons are against a baseline with its spread and its sample size. Total sleep time and sleep timing are the first things on the page, checked by review. Do not build yet: No correlations, export or staleness handling yet. ### Phase 4 · Comparisons declared before they are computed Build: Declare the metric pairs and lags in configuration with the date you declared them. Compute each with its confidence interval and its sample size, state how many comparisons the dashboard runs in total, and refuse to display one below a configured minimum sample. Every result is phrased as an association and the interface contains no causal wording. Trap: A panel that tries every metric against every other one over a month will find several convincing relationships, because that is what running forty tests does, and the one you remember is the one that matched a story you already had. Fix the pairs in configuration with a declaration date, show the total number of comparisons run beside the results, and treat anything you add afterwards as a new declaration rather than a finding. Trap: Reporting a coefficient without its interval and sample size makes a relationship computed from eleven nights look like one computed from three hundred. Show the interval and the count everywhere the coefficient appears, and refuse to render below the minimum. Done when: Comparisons come only from the declared configuration and each row carries its declaration date, asserted. A coefficient never appears without its confidence interval and sample size, verified over the templates. The total count of declared comparisons is displayed alongside the results. Any pair below the configured minimum sample renders as not yet computable with the count needed, tested at the boundary. Do not build yet: No staleness handling, export or deletion yet. ### Phase 5 · Data that goes stale, and saying so Build: Show the export date at the top of every page and the age of the data beside it. Stop every axis at the export date rather than at today. Grey out anything that would imply current information. Export the derived tables as CSV, and provide one command that removes the database, the extracted files and the original zip together. Trap: Drawing today's date on the axis when the data ends eleven days ago makes a stale number look current, and the whole failure mode of this build is a dashboard that quietly stops being about now. Stop the axis at the export date, print the data age on every page, and show a prompt to re-export once the age crosses a configured threshold. Trap: Leaving the extracted export files beside the database means a delete that removes the database leaves a complete copy of your health record on disk, usually in a directory nobody remembers. Make deletion one command covering the database, the extracted files and the original archive, and print what it removed. Done when: Every page shows the export date and the data age, and no chart axis extends past the export date, verified by review. Crossing the staleness threshold shows a re-export prompt naming the age, tested at the boundary. Exporting produces the derived tables as CSV and reimports into an empty database with identical row counts. The delete command removes the database, extracted files and the original archive, and prints each path it removed, tested. Do not build yet: Nothing after this. ### Out of scope (and why) - Live background synchronisation, which needs an actual phone application, a developer account and a review process, and is the entire reason the paid product exists. - A composite readiness or recovery score, because no published work establishes that such a number predicts anything about an individual, and presenting one would be the only dishonest thing in this build. - Clinical interpretation of any kind, including sleep disorder screening, since the same validation work that sets the limits here is explicit that these devices do not replace polysomnography for diagnosis. - Non-Apple device sources and whatever normalisation would be needed between them, which is a per-vendor data problem rather than a weekend of code. ### README must contain - The published agreement figures for wearable sleep staging, with the study, the devices and the date, stated before any instructions - That wrist heart rate variability is shown as a rolling baseline because its published agreement approaches the measure's own variation - That there is no composite score and why one is not provided - That one source is chosen per metric per day, what the rule is, and how many days had a conflict - That the export goes stale, how the dashboard shows that, and how to delete everything including the archive
ARCHITECTURE.md
# Architecture · Bevel ## Stack | Part | Choice | Why | | --- | --- | --- | | Runtime | Python 3.12 with a local web dashboard, nothing leaving the machine | this is a record of your body and it belongs on your disk | | Ingest | An incremental XML parser writing straight to SQLite in batches | the export is hundreds of megabytes and a document parser will not survive it | | Storage | SQLite, one row per exported record, source and device retained | you cannot resolve a double count without knowing which device wrote what | | Analysis | A declared set of comparisons with intervals and sample sizes | a dashboard that tries every pair is a machine for generating findings | | Presentation | Every figure carries its device, its sample size and its known agreement | the number is only as good as the classifier that produced it | ## Modules Each module has one owner concern and a documented way to replace it. | Module | Owns | How to replace it | | --- | --- | --- | | Ingest | Streaming parse, batching and atomic imports. | Any parser, provided it never builds the document in memory. | | Resolve | Source choice, interval merging and conflict counts. | Any rule, provided the choice is recorded and raw records are untouched. | | Presentation | Bands, baselines and the agreement labels. | Any charting, provided no bare stage minute count can be rendered. | | Comparisons | Declared pairs, intervals and sample sizes. | Any statistic, provided it is declared before it is computed. | | Lifecycle | Staleness, export and complete deletion. | Any storage, provided deletion covers the archive and the extraction. | ## Configuration Every runtime setting is an environment variable documented in `.env.example`, validated at startup, with a safe local default wherever one exists. - `DATABASE_PATH` · required · Raw records and derived tables. Everything about your body; keep it local. - `EXTRACT_DIR` · required · Where the archive is unpacked. Removed by the delete command along with the archive. - `IMPORT_BATCH_SIZE` · required · Rows per insert batch. Larger is faster and uses more memory. - `SOURCE_PREFERENCE` · required · Order used to pick one source per metric per day. Changing it recomputes derived tables. - `BASELINE_WINDOW_DAYS` · required · Rolling window any single value is compared against. - `MIN_COMPARISON_N` · required · Below this many paired days, a comparison is shown as not yet computable. - `STALE_AFTER_DAYS` · required · Data age at which the dashboard prompts for a fresh export. ## Production baseline - Security: least privilege, input validation at every boundary, secret redaction in logs, rate limits on abuse-prone paths, no invented security primitives. - Data: explicit schema and migrations, transactional writes where integrity matters, backup and restore procedures that have been exercised. - Integrations: adapters around third-party providers, idempotent webhook or job processing, bounded retries, timeouts. - Observability: structured logs with request or operation ids, an error-tracking hook, and health and readiness checks where a server exists. - Quality: unit tests for domain rules, integration tests at module boundaries, one end-to-end test of the critical path. ## Decision records For each dependency in the stack table, keep a short note: why it was chosen, its failure mode, and how it is replaced. Do not add infrastructure until a requirement in `PRODUCT.md` justifies it.
AGENTS.md
# Agent instructions · Bevel product build - Read `PRODUCT.md` and `ARCHITECTURE.md` before changing code. The stack is fixed: Python 3.12 with a local web dashboard, nothing leaving the machine, An incremental XML parser writing straight to SQLite in batches, SQLite, one row per exported record, source and device retained, A declared set of comparisons with intervals and sample sizes, Every figure carries its device, its sample size and its known agreement. - Implement milestone by milestone from `MILESTONES.md`; keep each change reviewable and leave the application runnable at every commit. - Treat authentication, payments, encryption, imports, webhooks and destructive actions as high-risk boundaries when present. - Never invent cryptography or silently weaken a requirement to make a check pass. - Put every external service behind an interface with a deterministic fake for tests. - Add migrations and rollback or recovery notes for every persistent data change. - Log useful operational context without credentials, tokens, passwords or personal data. - Update documentation and run every check before completing a milestone. ## Known traps - Loading the export with a document parser builds the whole file as objects in memory, takes every gigabyte the machine has and dies several minutes in, and the obvious fix of raising the limit only moves the moment of death. Parse incrementally, clear each element as you finish with it, and batch the inserts so the transaction log does not become the new problem. - Re-importing after adding a metric duplicates every record you already had, and because health data is full of legitimately repeated values you cannot spot it afterwards by eye. Key records on their natural identity and make a re-import a no-op for anything already present, then prove it by importing the same file twice and comparing counts. - Summing every step or sleep record for a day adds the phone's count to the watch's and gives somebody who carried both a day that never happened, which is the single most common error in analysis of these exports. Choose one source per metric per day, write down the rule, and show the count of days where the sources disagreed rather than hiding the decision. - Summing sleep intervals from two sources that both recorded the same night produces more sleep than there were hours, and a chart that occasionally exceeds twenty-four hours is a chart nobody trusts again. Merge intervals by union, store the overlap you removed, and assert that no night exceeds the elapsed wall clock between its start and end. - Rendering stage durations to the minute implies a measurement the device does not make: the published validation reports four-stage agreement between kappa 0.21 and 0.53 across six wearables and correctly classified deep sleep epochs between about 47 and 70 percent, so the deep sleep figure is close to a coin flip per epoch. Show bands, put the agreement figure beside the number, and remove every code path that can display a stage minute count on its own. - Comparing tonight's stage minutes against last night's and drawing an arrow presents the difference between two noisy classifications as a change in the person. Compare against a rolling baseline with its spread, and require a stated minimum number of nights before any comparison is drawn at all. - A panel that tries every metric against every other one over a month will find several convincing relationships, because that is what running forty tests does, and the one you remember is the one that matched a story you already had. Fix the pairs in configuration with a declaration date, show the total number of comparisons run beside the results, and treat anything you add afterwards as a new declaration rather than a finding. - Reporting a coefficient without its interval and sample size makes a relationship computed from eleven nights look like one computed from three hundred. Show the interval and the count everywhere the coefficient appears, and refuse to render below the minimum. - Drawing today's date on the axis when the data ends eleven days ago makes a stale number look current, and the whole failure mode of this build is a dashboard that quietly stops being about now. Stop the axis at the export date, print the data age on every page, and show a prompt to re-export once the age crosses a configured threshold. - Leaving the extracted export files beside the database means a delete that removes the database leaves a complete copy of your health record on disk, usually in a directory nobody remembers. Make deletion one command covering the database, the extracted files and the original archive, and print what it removed.
MILESTONES.md
# Delivery milestones · Bevel Estimated effort: **weekend** for the indie phases; the production-only milestones add the trust and operability layer. ## M1 · An export that does not fit in memory A multi-hundred-megabyte export imports within a stated memory ceiling and either completes or leaves nothing. ### Steps 1. Parse incrementally and clear as you go The file is bigger than the memory you want to spend. Files: `src/ingest/stream.py`, `tests/test_memory.py` 2. Make a re-import a no-op for existing records Duplicated health records cannot be spotted by eye later. Files: `src/ingest/dedupe.py` 3. Make the import atomic A half import is a dashboard quietly showing half a life. Files: `src/ingest/transaction.py` ### Done when - [ ] A large export imports under the stated memory ceiling - [ ] Importing twice yields identical record counts - [ ] An interrupted import leaves nothing partial - [ ] Export date, count and sources are recorded and shown ### Watch out - Loading the export with a document parser builds the whole file as objects in memory, takes every gigabyte the machine has and dies several minutes in, and the obvious fix of raising the limit only moves the moment of death. Parse incrementally, clear each element as you finish with it, and batch the inserts so the transaction log does not become the new problem. - Re-importing after adding a metric duplicates every record you already had, and because health data is full of legitimately repeated values you cannot spot it afterwards by eye. Key records on their natural identity and make a re-import a no-op for anything already present, then prove it by importing the same file twice and comparing counts. ## M2 · Two devices, one day, and the double count Each metric on each day comes from one chosen source, and what was discarded is recorded. ### Steps 1. Choose one source per metric per day and record it Adding two devices together invents a day. Files: `src/resolve/sources.py`, `tests/test_double_count.py` 2. Merge sleep intervals by union, keep the overlap figure Two devices recording one night is not two nights. Files: `src/resolve/sleep.py` 3. Recompute derived tables without touching raw records The raw import is the only thing you cannot re-derive. Files: `src/resolve/recompute.py` ### Done when - [ ] A two-source day yields one value and a recorded source - [ ] No night's sleep exceeds its own elapsed time - [ ] Changing the preference leaves raw records untouched - [ ] Conflicting-source day count is visible ### Watch out - Summing every step or sleep record for a day adds the phone's count to the watch's and gives somebody who carried both a day that never happened, which is the single most common error in analysis of these exports. Choose one source per metric per day, write down the rule, and show the count of days where the sources disagreed rather than hiding the decision. - Summing sleep intervals from two sources that both recorded the same night produces more sleep than there were hours, and a chart that occasionally exceeds twenty-four hours is a chart nobody trusts again. Merge intervals by union, store the overlap you removed, and assert that no night exceeds the elapsed wall clock between its start and end. ## M3 · Sleep at the resolution the device actually has Total sleep and timing lead, stages appear in bands with their published agreement attached. ### Steps 1. Lead with total sleep and timing They are the parts the devices are measured as detecting well. Files: `src/web/sleep.py`, `src/web/templates/sleep.html` 2. Show stages in bands with the agreement figure The number and its uncertainty travel together or not at all. Files: `src/analysis/stages.py`, `tests/test_no_bare_minutes.py` 3. Compare against a baseline, never against last night Two noisy classifications do not make a trend. Files: `src/analysis/baseline.py` ### Done when - [ ] No path shows a bare stage minute count - [ ] Every stage figure carries its agreement statistic and source - [ ] Comparisons are against a baseline with spread and n - [ ] Total sleep and timing lead the page ### Watch out - Rendering stage durations to the minute implies a measurement the device does not make: the published validation reports four-stage agreement between kappa 0.21 and 0.53 across six wearables and correctly classified deep sleep epochs between about 47 and 70 percent, so the deep sleep figure is close to a coin flip per epoch. Show bands, put the agreement figure beside the number, and remove every code path that can display a stage minute count on its own. - Comparing tonight's stage minutes against last night's and drawing an arrow presents the difference between two noisy classifications as a change in the person. Compare against a rolling baseline with its spread, and require a stated minimum number of nights before any comparison is drawn at all. ## M4 · Comparisons declared before they are computed The correlations on the dashboard are the ones you chose in advance, reported with intervals and counts. ### Steps 1. Declare pairs and lags in configuration with a date A hypothesis chosen after the data is not a hypothesis. Files: `config/comparisons.toml`, `src/analysis/declare.py` 2. Report interval and sample size everywhere The coefficient alone hides how little is behind it. Files: `src/analysis/correlate.py`, `tests/test_intervals.py` 3. Say how many comparisons the dashboard runs It is the context that makes one result readable. Files: `src/web/templates/insights.html` ### Done when - [ ] Only declared comparisons appear, each with its date - [ ] No coefficient appears without interval and sample size - [ ] The total comparison count is displayed - [ ] Under-sampled pairs render as not yet computable ### Watch out - A panel that tries every metric against every other one over a month will find several convincing relationships, because that is what running forty tests does, and the one you remember is the one that matched a story you already had. Fix the pairs in configuration with a declaration date, show the total number of comparisons run beside the results, and treat anything you add afterwards as a new declaration rather than a finding. - Reporting a coefficient without its interval and sample size makes a relationship computed from eleven nights look like one computed from three hundred. Show the interval and the count everywhere the coefficient appears, and refuse to render below the minimum. ## M5 · Data that goes stale, and saying so (production only) The dashboard is honest about how old the export is, everything exports, and deleting means deleting. ### Steps 1. Stop every axis at the export date An axis that runs to today is a lie about the data. Files: `src/web/charts.py` 2. Show the data age and prompt a re-export The staleness is the honest headline of this whole build. Files: `src/web/templates/base.html` 3. Make deletion cover the zip and the extraction A deleted database next to an extracted export is not deleted. Files: `src/admin/delete.py`, `tests/test_delete.py` ### Done when - [ ] Export date and data age appear on every page - [ ] No axis extends past the export date - [ ] Derived tables export and reimport with identical counts - [ ] Deletion removes database, extraction and archive ### Watch out - Drawing today's date on the axis when the data ends eleven days ago makes a stale number look current, and the whole failure mode of this build is a dashboard that quietly stops being about now. Stop the axis at the export date, print the data age on every page, and show a prompt to re-export once the age crosses a configured threshold. - Leaving the extracted export files beside the database means a delete that removes the database leaves a complete copy of your health record on disk, usually in a directory nobody remembers. Make deletion one command covering the database, the extracted files and the original archive, and print what it removed.
OPERATIONS.md
# Operations · Bevel ## Backup The original archive is the backup, since the database is derived from it and can be rebuilt. Keep the archives you have imported, because they are the only record of what your devices said at the time, and export requests are slow. ## Restore Re-import the archive into an empty database and confirm the record count and the conflicting-source day count match what the previous import reported. If they do not, the resolution rules changed rather than the data. Do a restore drill before the first real user, and write the date here when it passes. ## Monitoring For a local dashboard the checks live in the suite plus one number on the page: the data age. Beyond that, memory used by the import, the conflicting-source day count, and the proportion of nights where stage data came from a source you did not prefer. ## Incident checklist The incident here is a decision somebody made about their health from a figure the device does not actually measure. The agreement labels, the bands and the declared comparison list exist so that the dashboard cannot present that figure in the first place. 1. Contain the issue without destroying evidence or user data. 2. Record the timeline and affected scope. 3. Rotate exposed secrets and revoke compromised sessions or credentials. 4. Restore from a verified backup when needed. 5. Document the root cause, the remediation and the regression test. ## Release gate - [ ] A large export is confirmed to import under the stated memory ceiling - [ ] No night's total sleep is confirmed to exceed its own elapsed time - [ ] No interface path is confirmed to show a bare stage minute count - [ ] Only declared comparisons are confirmed to appear, with intervals and counts - [ ] Deletion is confirmed to remove database, extraction and archive ## Launch constraint Do not market omitted Bevel capabilities as implemented. The non-goals in `PRODUCT.md` remain user-visible limitations until they are deliberately delivered.
.env.example
# Copy to .env and fill in. Never commit .env; this file documents it. # Required. Raw records and derived tables. Everything about your body; keep it local. DATABASE_PATH=./data/health.db # Required. Where the archive is unpacked. Removed by the delete command along with the archive. EXTRACT_DIR=./data/extract # Required. Rows per insert batch. Larger is faster and uses more memory. IMPORT_BATCH_SIZE=5000 # Required. Order used to pick one source per metric per day. Changing it recomputes derived tables. SOURCE_PREFERENCE=watch,phone # Required. Rolling window any single value is compared against. BASELINE_WINDOW_DAYS=28 # Required. Below this many paired days, a comparison is shown as not yet computable. MIN_COMPARISON_N=30 # Required. Data age at which the dashboard prompts for a fresh export. STALE_AFTER_DAYS=14
$ choose a build depth, inspect the files, then open the complete pack in your agent
Because the value of health metrics collapses if you have to go fetch them. Bevel's job is to be already updated when you open your phone, with a number you trust and a sentence that explains it. A local notebook that needs a manual export every two weeks answers different questions: interesting once, abandoned by month two. People pay for the part that keeps running when their curiosity does not.
xLive background sync; you are re-exporting a zip by hand
xA phone app, widgets and notifications
xTheir opinionated composite scores and plain-English daily readouts
xNon-Apple device integrations and whatever normalisation they do across sources
xCharts that a designer looked at
Nothing worth pointing at. That's why the prompt exists.
share on X ↗"I just replaced Bevel ($14.99/mo) with one prompt"
Vibecode Bevel
Kinda. The core of Bevel is buildable in a weekend with the prompt on this page, but there are real gaps: Live background sync; you are re-exporting a zip by hand, A phone app, widgets and notifications. Read the honest list above before committing.
How much does Bevel cost?
Bevel costs about $14.99/month (Bevel Pro, checked 2026-08-18), which is $179.88 per year.
What do I lose by replacing Bevel?
Honestly: Live background sync; you are re-exporting a zip by hand; A phone app, widgets and notifications; Their opinionated composite scores and plain-English daily readouts; Non-Apple device integrations and whatever normalisation they do across sources; Charts that a designer looked at. If any of those are load-bearing for you, keep paying.
Is there an open-source alternative to Bevel?
No mature open-source alternative worth pointing at, which is exactly why the one-shot prompt on this page exists.