Vibecode Wholana
track this build5 phases, 15 steps, beginner friendly0%KINDA · weekend project
The core idea is simple arithmetic: a video's views divided by that creator's own median. An agent can build that for a watchlist of creators you pick, over a weekend, on top of a paid scraper API. What it cannot hand you is the corpus, hundreds of thousands of videos already scraped, deduped, and labeled against a curated craft taxonomy, which is what makes search across creators useful instead of a list of your own bookmarks. So: yes for watching 25 creators you already know, no for finding the ones you don't.
the project pack5 files · written for this build, step by step
You are building a lean indie version of Wholana. Create the following project files first, then implement the application by following them. Keep the files updated as decisions change. Do not collapse this into a single README or prompt. ===== README.md ===== # Wholana · indie build A nightly job that records view counts for a small watchlist of creators, computes each creator's rolling median so a video is judged against their own baseline rather than against a global one, surfaces the ones that outperformed, and labels what those videos did differently. Estimated effort: **weekend**. Work `BUILD_PLAN.md` top to bottom · every phase ends in a check that has to pass before the next one starts. ## Stack | Part | Choice | Why | | --- | --- | --- | | Runtime | Node 22 with SQLite through better-sqlite3 | a nightly job and a few thousand rows; anything larger is a scope problem rather than a database problem | | Collection | A single source behind one adapter, polled nightly | one place to change when the source changes, which it will | | Baseline | Rolling median of a creator's recent videos | median rather than mean, because one viral video would otherwise raise the bar for a year | | Labelling | A model applied to the outperformers only | labelling everything costs money for videos you will never look at | | Interface | A static page generated nightly | you read this once a day; a server is unnecessary | ## Before you start Have every one of these ready. The plan assumes them from step one. - [ ] **Node.js 22 or newer** · free - Why: Runs the nightly job and the page generator. - Get it: nodejs.org, or nvm install 22. - Verify: node --version prints v22 or higher - [ ] **A read of the source's terms, and your own position on them** · free, and it is the first task rather than a formality - Why: This decides whether you build at all and what shape it takes. TikTok's Research API is limited to non-profit academic researchers at qualifying institutions and excludes commercial interests; the platform terms prohibit automated collection. - Get it: Read the Research API eligibility page and the platform terms. Decide whether you qualify, and if not, decide deliberately what you are willing to do and write it down with the date. - Verify: A written note names the source, the permitted use and the date checked - [ ] **Access to a data source** · the Research API is free for those eligible; third party services charge per run and add up with a nightly schedule - Why: Without it there is nothing to collect. Third party scraping services are the usual route and are priced per run. - Get it: If you qualify for the Research API, apply; the process takes up to about four weeks. Otherwise, evaluate a third party service and understand that its terms exposure does not fully replace yours. - Verify: A test run returns video data for one creator - [ ] **A watchlist of creators with reasons** · free - Why: A small, reasoned watchlist is a different thing from broad collection, and the reasons are what keep it that way over time. - Get it: List the creators you already follow for a reason, and write the reason next to each. If you cannot write a reason, leave them off. - Verify: Every creator on the list has a recorded reason - [ ] **A model API key for labelling** (optional) · pay per token; labelling a handful of videos a night is negligible, labelling everything is not - Why: Phase 4 only. Detection works without it and labelling is a convenience on top. - Get it: A provider key with a spend limit set at the provider. - Verify: A test label returns and its cost is recorded ## Quick start Follow Phase 1 in `BUILD_PLAN.md`; it creates the project. Then copy `.env.example` to `.env` and fill in the values it documents. ## Honest limits This build deliberately does not replace: - A cross-creator corpus and search across it. - A curated taxonomy built from many accounts. - Any compliant route to platform data at commercial scale. - Coverage of creators you did not deliberately choose to watch. - the cross-creator corpus - search across videos you never chose to watch - a curated craft taxonomy instead of labels you invented - semantic and hybrid search - subject classification - creator equity and track-record views - someone else absorbing the scrape cost and keeping it running If one of those is essential to you, that is the reason to keep paying for Wholana, and the README should say so rather than pretend. ===== BRIEF.md ===== # Build brief · Wholana The one-shot brief this plan expands. `BUILD_PLAN.md` (or `MILESTONES.md`) is the same sequence broken into steps and checks; where the two disagree, the plan wins. Build me a watchlist that tells you which of a creator's videos beat their own usual numbers to replace Wholana. Build it in phases, in the order below. Do not write the whole thing in one pass. Finish a phase, run its "Done when" check, fix what fails, and only then start the next phase. ### Rule zero Read the access question before choosing an architecture, because it decides what you can build. TikTok's Research API is free and is restricted to non-profit academic researchers at qualifying institutions in a handful of jurisdictions; applicants must be independent of commercial interests. If you are a company or an individual builder, you do not qualify. TikTok's terms also prohibit automated collection, so scraping carries real terms exposure, and using a third party scraping service moves some of that exposure without removing yours. Checked September 2026. The honest consequence: keep the watchlist small and specific to creators you have a reason to follow, store aggregate metrics rather than profiles of people, poll rarely, and write your position down. A tool that quietly builds a database about individuals is a different product from one that tracks a dozen accounts you already watch. ### Stack (fixed, do not substitute) - Runtime: Node 22 with SQLite through better-sqlite3 · a nightly job and a few thousand rows; anything larger is a scope problem rather than a database problem - Collection: A single source behind one adapter, polled nightly · one place to change when the source changes, which it will - Baseline: Rolling median of a creator's recent videos · median rather than mean, because one viral video would otherwise raise the bar for a year - Labelling: A model applied to the outperformers only · labelling everything costs money for videos you will never look at - Interface: A static page generated nightly · you read this once a day; a server is unnecessary ### Data model (create this before Phase 1) - creators(id, handle, added_at, reason, enabled) · reason is why you are watching them, which keeps the watchlist honest - videos(id, creator_id, external_id, posted_at, caption, duration_s, first_seen_at) - metrics(video_id, observed_at, views, likes, comments) · one row per observation, so growth over time is visible - baselines(creator_id, computed_at, window_days, median_views, sample_size) - outperformers(video_id, ratio, detected_at, labels_json, reviewed) · ratio is views over the creator's median - Aggregate counts only. No follower lists, no commenter identities, no profiles of people who did not choose to be in your database. ### Phase 1 · Deciding what you may collect Build: Before any code, choose the source and read its terms. Write a note naming the source, what it permits, what you will collect and what you will deliberately not collect. Then design the schema to make the exclusions structural rather than a matter of remembering. Trap: Building the collector first and deciding the policy afterwards produces a system you are not comfortable running, and by then the schema already holds data you did not want. Deciding first is cheaper and the schema then enforces it: if there is no column for follower identities, nothing can accidentally store them. Done when: A written note names the source, the permitted use, the check date and the collection boundary. The schema has no column capable of holding individual identities beyond the creator handles you deliberately chose. The watchlist has a reason recorded for every creator. Do not build yet: No collection at all yet. The policy first, because it shapes the schema. ### Phase 2 · Collecting nightly, politely Build: One adapter for the chosen source, called on a schedule. Rate limit well below any documented ceiling. Back off on failure. Record every observation as a new row rather than updating, so growth over time survives. Trap: Updating a video's view count in place loses the growth curve, which is the interesting part: a video that reached its number in a day is a different signal from one that took three weeks. Append observations; never overwrite them. Done when: A nightly run records observations for every enabled creator. Running twice in one night does not create duplicate observations for the same day. A source failure backs off and the run reports it rather than failing silently. The observation history for one video shows its growth over several days. Do not build yet: No baselines or outperformer detection yet. ### Phase 3 · A baseline per creator Build: Compute a rolling median of recent view counts per creator over a window you choose. Require a minimum sample size before producing a baseline at all. Exclude the video being judged from its own baseline. Trap: A mean is dragged upward by one viral video and stays high for months, so nothing after it ever looks like an outperformer. Median is resistant to exactly that. Trap: Including the video in its own baseline dampens the very signal you are measuring, and it matters most for creators with few videos, which is where you most want the signal. Done when: A creator with a single viral video still has a baseline near their typical numbers, verified against hand calculation. A creator below the minimum sample size produces no baseline rather than an unreliable one. Excluding the judged video from its own baseline changes the ratio, proving the exclusion is applied. Do not build yet: No detection or labelling yet. ### Phase 4 · Detecting and labelling what worked Build: Flag videos exceeding a ratio threshold. Send only those to a model to label the hook, format and topic. Cap spending. Keep labels as suggestions you can correct, and use your corrections to improve the label set over time. Trap: Labelling every video means paying for thousands of labels you never read, and the ones that matter are a small fraction. Detect first, label second, and cap the number labelled per run so a source glitch that flags everything cannot produce a large bill. Done when: Only videos above the threshold are labelled, verified by comparing the label count to the flagged count. A per-run label cap prevents a spike from producing a large bill. Labels are editable and corrections persist. Cost per run is recorded. Do not build yet: No review interface yet. ### Phase 5 · The page you actually read Build: Generate a static page nightly: outperformers ranked by ratio, with the creator, the baseline, the caption and the labels. Group by label so patterns are visible. Keep it short enough that you read all of it. Trap: A page listing every video with a ratio above one is a page nobody reads, because most days nothing interesting happened. Set the threshold so a typical day shows a handful, and let an empty day be empty rather than padding it. Done when: The nightly page shows outperformers with their ratio and baseline. A day with nothing above the threshold produces a page that says so rather than lowering the bar. Grouping by label shows which formats recur. The page renders without a server. Do not build yet: Retention and review come last. ### Phase 6 · Retention, review and the terms check Build: Retention that deletes old observations while keeping aggregates. A periodic review of the watchlist against its recorded reasons. A scheduled reminder to re-read the source's terms, because they change and your note has a date on it. Trap: A collection policy written once and never revisited becomes inaccurate silently, and the terms of these platforms change more often than most. Put a date on the note and a reminder in the calendar, so the claim in your README stays true. Done when: Retention removes old observations and keeps computed aggregates, verified by row counts. A review lists creators whose recorded reason no longer applies. The terms note carries a date and a reminder exists to re-check it. Removing a creator deletes their videos, metrics and labels together. Do not build yet: Nothing after this. ### Out of scope (and why) - A cross-creator corpus. The hosted product's value is comparing against videos you never chose to watch, and building that means collecting at a scale this build deliberately refuses. - Search across the platform. That needs the corpus, which needs the collection, which is the thing rule zero is about. - A curated taxonomy built from many accounts' data. Your labels are your own and will be less useful for the first few months. - Any claim that this is a compliant route to platform data at commercial scale. It is not, and the terms note exists so you know what you are choosing. ### README must contain - Which source is used, what its terms permit, and the date that was checked - That collection is aggregate counts for a chosen watchlist and deliberately excludes individual identities - That baselines use a median per creator, and why a mean would fail after one viral video - That only outperformers are labelled, with the per-run cap - The retention window and how to remove a creator entirely ===== AGENTS.md ===== # Agent instructions · Wholana indie build - Read `README.md` and `BUILD_PLAN.md` before writing code. The stack is fixed: Node 22 with SQLite through better-sqlite3, A single source behind one adapter, polled nightly, Rolling median of a creator's recent videos, A model applied to the outperformers only, A static page generated nightly. Do not substitute. - Work one phase at a time, in order. Do not start a phase until every "Done when" item of the previous one passes. - Prefer the fewest moving parts that satisfy the step. No frameworks, services or dependencies the plan does not name. - Secrets live in `.env`, never in source or logs. Keep `.env.example` current when a variable is introduced. - Do not invent cryptography, security guarantees, APIs or compliance claims. - Add a focused test for every destructive, security-sensitive or data-loss path the plan names. - Run the project checks before declaring a phase complete, and record any deliberate shortcut in the README under "Tradeoffs". ## Known traps - Building the collector first and deciding the policy afterwards produces a system you are not comfortable running, and by then the schema already holds data you did not want. Deciding first is cheaper and the schema then enforces it: if there is no column for follower identities, nothing can accidentally store them. - Updating a video's view count in place loses the growth curve, which is the interesting part: a video that reached its number in a day is a different signal from one that took three weeks. Append observations; never overwrite them. - A mean is dragged upward by one viral video and stays high for months, so nothing after it ever looks like an outperformer. Median is resistant to exactly that. - Including the video in its own baseline dampens the very signal you are measuring, and it matters most for creators with few videos, which is where you most want the signal. - Labelling every video means paying for thousands of labels you never read, and the ones that matter are a small fraction. Detect first, label second, and cap the number labelled per run so a source glitch that flags everything cannot produce a large bill. - A page listing every video with a ratio above one is a page nobody reads, because most days nothing interesting happened. Set the threshold so a typical day shows a handful, and let an empty day be empty rather than padding it. ===== BUILD_PLAN.md ===== # Build plan · Wholana A nightly job that records view counts for a small watchlist of creators, computes each creator's rolling median so a video is judged against their own baseline rather than against a global one, surfaces the ones that outperformed, and labels what those videos did differently. Phases are in dependency order. Each ends in a "Done when" list; treat an unticked item as a blocker, not a note. ## Phase 1 · Deciding what you may collect You have written down what you are collecting, from where, and under what terms. ### Steps 1. Read the terms and write the note Source, permitted use, date checked, and what you will not collect. One page, in the repository. Files: `docs/collection-policy.md` 2. Design the schema around the exclusions Aggregate counts, no identity columns for anyone but the creators you chose. Structural rather than remembered. Files: `src/schema.sql` 3. Record a reason per creator Why you are watching them. A watchlist without reasons grows into surveillance by accident. Files: `src/creators.js` ### Done when - [ ] The collection policy is written with a check date - [ ] The schema cannot hold identities beyond chosen creators - [ ] Every creator has a recorded reason - [ ] The exclusions are enforced by structure rather than by discipline ### Watch out - Building the collector first and deciding the policy afterwards produces a system you are not comfortable running, and by then the schema already holds data you did not want. Deciding first is cheaper and the schema then enforces it: if there is no column for follower identities, nothing can accidentally store them. ## Phase 2 · Collecting nightly, politely View counts arrive once a night without hammering anything. ### Steps 1. Write the source adapter One interface. Everything source-specific lives behind it, because the source will change or break. Files: `src/source.js` 2. Append observations One row per video per run. Unique on video and date so a rerun is a no-op. Files: `src/collect.js` 3. Rate limit and back off Well below any ceiling, with exponential backoff. Politeness here is also what keeps access working. Files: `src/collect.js` ### Done when - [ ] A nightly run records observations for every enabled creator - [ ] A second run the same night creates no duplicates - [ ] A source failure backs off and is reported - [ ] Growth over several days is visible for one video ### Watch out - Updating a video's view count in place loses the growth curve, which is the interesting part: a video that reached its number in a day is a different signal from one that took three weeks. Append observations; never overwrite them. ## Phase 3 · A baseline per creator A video is judged against its own creator's usual numbers, not against everyone's. ### Steps 1. Compute a rolling median Over a window in days, per creator. Recompute nightly rather than incrementally, because it is cheap at this size. Files: `src/baseline.js` 2. Require a minimum sample Below it, no baseline. A ratio computed from three videos is noise presented as a number. Files: `src/baseline.js` 3. Exclude the subject video Test with a hand-calculated example so the exclusion is proven rather than assumed. Files: `test/baseline.test.js` ### Done when - [ ] One viral video does not distort the creator's baseline - [ ] Below the minimum sample no baseline is produced - [ ] The judged video is excluded from its own baseline - [ ] Baselines match a hand calculation on a fixture ### Watch out - A mean is dragged upward by one viral video and stays high for months, so nothing after it ever looks like an outperformer. Median is resistant to exactly that. - Including the video in its own baseline dampens the very signal you are measuring, and it matters most for creators with few videos, which is where you most want the signal. ## Phase 4 · Detecting and labelling what worked You see the videos that beat their creator's baseline, with a note on what they did differently. ### Steps 1. Flag by ratio Views over the creator's baseline, above a threshold you choose. Store the ratio, not just the flag. Files: `src/detect.js` 2. Label only the flagged One model call per outperformer, with a hard cap per run. Files: `src/label.js` 3. Make labels correctable Your corrections are the training signal for your own label set, and they are more accurate than the model's first pass. Files: `src/label.js` ### Done when - [ ] Labels are produced only for flagged videos - [ ] The per-run cap prevents a spike from spending freely - [ ] Label corrections persist - [ ] Cost per run is recorded ### Watch out - Labelling every video means paying for thousands of labels you never read, and the ones that matter are a small fraction. Detect first, label second, and cap the number labelled per run so a source glitch that flags everything cannot produce a large bill. ## Phase 5 · The page you actually read A daily page showing what outperformed and why, in under a minute of reading. ### Steps 1. Generate a static page Nightly, written to disk. No server, no framework. Files: `src/render.js` 2. Rank and group By ratio, grouped by label so patterns are visible across creators. Files: `src/render.js` 3. Let quiet days be quiet An empty page is a true statement. Padding it teaches you to ignore the page. Files: `src/render.js` ### Done when - [ ] The page shows outperformers with ratio and baseline - [ ] A quiet day produces an honest empty page - [ ] Grouping by label reveals recurring formats - [ ] The page renders with no server ### Watch out - A page listing every video with a ratio above one is a page nobody reads, because most days nothing interesting happened. Set the threshold so a typical day shows a handful, and let an empty day be empty rather than padding it. ## Not in this build - A cross-creator corpus and search across it. - A curated taxonomy built from many accounts. - Any compliant route to platform data at commercial scale. - Coverage of creators you did not deliberately choose to watch. ## After v1, if you want it - Growth curve shape as a second signal, since reaching a number in a day differs from three weeks - Your own label taxonomy replacing the model's, once corrections accumulate - A weekly digest instead of daily, if the daily page turns out to be mostly empty ===== .env.example ===== # Copy to .env and fill in. Never commit .env; this file documents it. # Required. Creators, videos, observations and labels. DB_PATH=./data/watch.db # Required · secret. Your data source's credential. Priced per run, so treat it as a spending instrument. SOURCE_API_KEY=replace-with-key # Required. How far back the rolling median looks. BASELINE_WINDOW_DAYS=90 # Required. Minimum videos before a baseline is computed at all. BASELINE_MIN_SAMPLE=8 # Required. How far above the median a video must be to be flagged. Set it so a typical day shows a handful. OUTPERFORM_RATIO=2.0 # Required. Maximum videos labelled per run, so a source glitch cannot produce a large bill. LABEL_CAP_PER_RUN=15 # Required. How long raw observations are kept. Aggregates survive beyond this. OBSERVATION_RETENTION_DAYS=365
You are building a lean indie version of Wholana. Create the following project files first, then implement the application by following them. Keep the files updated as decisions change. Do not collapse this into a single README or prompt. ===== README.md ===== # Wholana · indie build A nightly job that records view counts for a small watchlist of creators, computes each creator's rolling median so a video is judged against their own baseline rather than against a global one, surfaces the ones that outperformed, and labels what those videos did differently. Estimated effort: **weekend**. Work `BUILD_PLAN.md` top to bottom · every phase ends in a check that has to pass before the next one starts. ## Stack | Part | Choice | Why | | --- | --- | --- | | Runtime | Node 22 with SQLite through better-sqlite3 | a nightly job and a few thousand rows; anything larger is a scope problem rather than a database problem | | Collection | A single source behind one adapter, polled nightly | one place to change when the source changes, which it will | | Baseline | Rolling median of a creator's recent videos | median rather than mean, because one viral video would otherwise raise the bar for a year | | Labelling | A model applied to the outperformers only | labelling everything costs money for videos you will never look at | | Interface | A static page generated nightly | you read this once a day; a server is unnecessary | ## Before you start Have every one of these ready. The plan assumes them from step one. - [ ] **Node.js 22 or newer** · free - Why: Runs the nightly job and the page generator. - Get it: nodejs.org, or nvm install 22. - Verify: node --version prints v22 or higher - [ ] **A read of the source's terms, and your own position on them** · free, and it is the first task rather than a formality - Why: This decides whether you build at all and what shape it takes. TikTok's Research API is limited to non-profit academic researchers at qualifying institutions and excludes commercial interests; the platform terms prohibit automated collection. - Get it: Read the Research API eligibility page and the platform terms. Decide whether you qualify, and if not, decide deliberately what you are willing to do and write it down with the date. - Verify: A written note names the source, the permitted use and the date checked - [ ] **Access to a data source** · the Research API is free for those eligible; third party services charge per run and add up with a nightly schedule - Why: Without it there is nothing to collect. Third party scraping services are the usual route and are priced per run. - Get it: If you qualify for the Research API, apply; the process takes up to about four weeks. Otherwise, evaluate a third party service and understand that its terms exposure does not fully replace yours. - Verify: A test run returns video data for one creator - [ ] **A watchlist of creators with reasons** · free - Why: A small, reasoned watchlist is a different thing from broad collection, and the reasons are what keep it that way over time. - Get it: List the creators you already follow for a reason, and write the reason next to each. If you cannot write a reason, leave them off. - Verify: Every creator on the list has a recorded reason - [ ] **A model API key for labelling** (optional) · pay per token; labelling a handful of videos a night is negligible, labelling everything is not - Why: Phase 4 only. Detection works without it and labelling is a convenience on top. - Get it: A provider key with a spend limit set at the provider. - Verify: A test label returns and its cost is recorded ## Quick start Follow Phase 1 in `BUILD_PLAN.md`; it creates the project. Then copy `.env.example` to `.env` and fill in the values it documents. ## Honest limits This build deliberately does not replace: - A cross-creator corpus and search across it. - A curated taxonomy built from many accounts. - Any compliant route to platform data at commercial scale. - Coverage of creators you did not deliberately choose to watch. - the cross-creator corpus - search across videos you never chose to watch - a curated craft taxonomy instead of labels you invented - semantic and hybrid search - subject classification - creator equity and track-record views - someone else absorbing the scrape cost and keeping it running If one of those is essential to you, that is the reason to keep paying for Wholana, and the README should say so rather than pretend. ===== BRIEF.md ===== # Build brief · Wholana The one-shot brief this plan expands. `BUILD_PLAN.md` (or `MILESTONES.md`) is the same sequence broken into steps and checks; where the two disagree, the plan wins. Build me a watchlist that tells you which of a creator's videos beat their own usual numbers to replace Wholana. Build it in phases, in the order below. Do not write the whole thing in one pass. Finish a phase, run its "Done when" check, fix what fails, and only then start the next phase. ### Rule zero Read the access question before choosing an architecture, because it decides what you can build. TikTok's Research API is free and is restricted to non-profit academic researchers at qualifying institutions in a handful of jurisdictions; applicants must be independent of commercial interests. If you are a company or an individual builder, you do not qualify. TikTok's terms also prohibit automated collection, so scraping carries real terms exposure, and using a third party scraping service moves some of that exposure without removing yours. Checked September 2026. The honest consequence: keep the watchlist small and specific to creators you have a reason to follow, store aggregate metrics rather than profiles of people, poll rarely, and write your position down. A tool that quietly builds a database about individuals is a different product from one that tracks a dozen accounts you already watch. ### Stack (fixed, do not substitute) - Runtime: Node 22 with SQLite through better-sqlite3 · a nightly job and a few thousand rows; anything larger is a scope problem rather than a database problem - Collection: A single source behind one adapter, polled nightly · one place to change when the source changes, which it will - Baseline: Rolling median of a creator's recent videos · median rather than mean, because one viral video would otherwise raise the bar for a year - Labelling: A model applied to the outperformers only · labelling everything costs money for videos you will never look at - Interface: A static page generated nightly · you read this once a day; a server is unnecessary ### Data model (create this before Phase 1) - creators(id, handle, added_at, reason, enabled) · reason is why you are watching them, which keeps the watchlist honest - videos(id, creator_id, external_id, posted_at, caption, duration_s, first_seen_at) - metrics(video_id, observed_at, views, likes, comments) · one row per observation, so growth over time is visible - baselines(creator_id, computed_at, window_days, median_views, sample_size) - outperformers(video_id, ratio, detected_at, labels_json, reviewed) · ratio is views over the creator's median - Aggregate counts only. No follower lists, no commenter identities, no profiles of people who did not choose to be in your database. ### Phase 1 · Deciding what you may collect Build: Before any code, choose the source and read its terms. Write a note naming the source, what it permits, what you will collect and what you will deliberately not collect. Then design the schema to make the exclusions structural rather than a matter of remembering. Trap: Building the collector first and deciding the policy afterwards produces a system you are not comfortable running, and by then the schema already holds data you did not want. Deciding first is cheaper and the schema then enforces it: if there is no column for follower identities, nothing can accidentally store them. Done when: A written note names the source, the permitted use, the check date and the collection boundary. The schema has no column capable of holding individual identities beyond the creator handles you deliberately chose. The watchlist has a reason recorded for every creator. Do not build yet: No collection at all yet. The policy first, because it shapes the schema. ### Phase 2 · Collecting nightly, politely Build: One adapter for the chosen source, called on a schedule. Rate limit well below any documented ceiling. Back off on failure. Record every observation as a new row rather than updating, so growth over time survives. Trap: Updating a video's view count in place loses the growth curve, which is the interesting part: a video that reached its number in a day is a different signal from one that took three weeks. Append observations; never overwrite them. Done when: A nightly run records observations for every enabled creator. Running twice in one night does not create duplicate observations for the same day. A source failure backs off and the run reports it rather than failing silently. The observation history for one video shows its growth over several days. Do not build yet: No baselines or outperformer detection yet. ### Phase 3 · A baseline per creator Build: Compute a rolling median of recent view counts per creator over a window you choose. Require a minimum sample size before producing a baseline at all. Exclude the video being judged from its own baseline. Trap: A mean is dragged upward by one viral video and stays high for months, so nothing after it ever looks like an outperformer. Median is resistant to exactly that. Trap: Including the video in its own baseline dampens the very signal you are measuring, and it matters most for creators with few videos, which is where you most want the signal. Done when: A creator with a single viral video still has a baseline near their typical numbers, verified against hand calculation. A creator below the minimum sample size produces no baseline rather than an unreliable one. Excluding the judged video from its own baseline changes the ratio, proving the exclusion is applied. Do not build yet: No detection or labelling yet. ### Phase 4 · Detecting and labelling what worked Build: Flag videos exceeding a ratio threshold. Send only those to a model to label the hook, format and topic. Cap spending. Keep labels as suggestions you can correct, and use your corrections to improve the label set over time. Trap: Labelling every video means paying for thousands of labels you never read, and the ones that matter are a small fraction. Detect first, label second, and cap the number labelled per run so a source glitch that flags everything cannot produce a large bill. Done when: Only videos above the threshold are labelled, verified by comparing the label count to the flagged count. A per-run label cap prevents a spike from producing a large bill. Labels are editable and corrections persist. Cost per run is recorded. Do not build yet: No review interface yet. ### Phase 5 · The page you actually read Build: Generate a static page nightly: outperformers ranked by ratio, with the creator, the baseline, the caption and the labels. Group by label so patterns are visible. Keep it short enough that you read all of it. Trap: A page listing every video with a ratio above one is a page nobody reads, because most days nothing interesting happened. Set the threshold so a typical day shows a handful, and let an empty day be empty rather than padding it. Done when: The nightly page shows outperformers with their ratio and baseline. A day with nothing above the threshold produces a page that says so rather than lowering the bar. Grouping by label shows which formats recur. The page renders without a server. Do not build yet: Retention and review come last. ### Phase 6 · Retention, review and the terms check Build: Retention that deletes old observations while keeping aggregates. A periodic review of the watchlist against its recorded reasons. A scheduled reminder to re-read the source's terms, because they change and your note has a date on it. Trap: A collection policy written once and never revisited becomes inaccurate silently, and the terms of these platforms change more often than most. Put a date on the note and a reminder in the calendar, so the claim in your README stays true. Done when: Retention removes old observations and keeps computed aggregates, verified by row counts. A review lists creators whose recorded reason no longer applies. The terms note carries a date and a reminder exists to re-check it. Removing a creator deletes their videos, metrics and labels together. Do not build yet: Nothing after this. ### Out of scope (and why) - A cross-creator corpus. The hosted product's value is comparing against videos you never chose to watch, and building that means collecting at a scale this build deliberately refuses. - Search across the platform. That needs the corpus, which needs the collection, which is the thing rule zero is about. - A curated taxonomy built from many accounts' data. Your labels are your own and will be less useful for the first few months. - Any claim that this is a compliant route to platform data at commercial scale. It is not, and the terms note exists so you know what you are choosing. ### README must contain - Which source is used, what its terms permit, and the date that was checked - That collection is aggregate counts for a chosen watchlist and deliberately excludes individual identities - That baselines use a median per creator, and why a mean would fail after one viral video - That only outperformers are labelled, with the per-run cap - The retention window and how to remove a creator entirely ===== AGENTS.md ===== # Agent instructions · Wholana indie build - Read `README.md` and `BUILD_PLAN.md` before writing code. The stack is fixed: Node 22 with SQLite through better-sqlite3, A single source behind one adapter, polled nightly, Rolling median of a creator's recent videos, A model applied to the outperformers only, A static page generated nightly. Do not substitute. - Work one phase at a time, in order. Do not start a phase until every "Done when" item of the previous one passes. - Prefer the fewest moving parts that satisfy the step. No frameworks, services or dependencies the plan does not name. - Secrets live in `.env`, never in source or logs. Keep `.env.example` current when a variable is introduced. - Do not invent cryptography, security guarantees, APIs or compliance claims. - Add a focused test for every destructive, security-sensitive or data-loss path the plan names. - Run the project checks before declaring a phase complete, and record any deliberate shortcut in the README under "Tradeoffs". ## Known traps - Building the collector first and deciding the policy afterwards produces a system you are not comfortable running, and by then the schema already holds data you did not want. Deciding first is cheaper and the schema then enforces it: if there is no column for follower identities, nothing can accidentally store them. - Updating a video's view count in place loses the growth curve, which is the interesting part: a video that reached its number in a day is a different signal from one that took three weeks. Append observations; never overwrite them. - A mean is dragged upward by one viral video and stays high for months, so nothing after it ever looks like an outperformer. Median is resistant to exactly that. - Including the video in its own baseline dampens the very signal you are measuring, and it matters most for creators with few videos, which is where you most want the signal. - Labelling every video means paying for thousands of labels you never read, and the ones that matter are a small fraction. Detect first, label second, and cap the number labelled per run so a source glitch that flags everything cannot produce a large bill. - A page listing every video with a ratio above one is a page nobody reads, because most days nothing interesting happened. Set the threshold so a typical day shows a handful, and let an empty day be empty rather than padding it. ===== BUILD_PLAN.md ===== # Build plan · Wholana A nightly job that records view counts for a small watchlist of creators, computes each creator's rolling median so a video is judged against their own baseline rather than against a global one, surfaces the ones that outperformed, and labels what those videos did differently. Phases are in dependency order. Each ends in a "Done when" list; treat an unticked item as a blocker, not a note. ## Phase 1 · Deciding what you may collect You have written down what you are collecting, from where, and under what terms. ### Steps 1. Read the terms and write the note Source, permitted use, date checked, and what you will not collect. One page, in the repository. Files: `docs/collection-policy.md` 2. Design the schema around the exclusions Aggregate counts, no identity columns for anyone but the creators you chose. Structural rather than remembered. Files: `src/schema.sql` 3. Record a reason per creator Why you are watching them. A watchlist without reasons grows into surveillance by accident. Files: `src/creators.js` ### Done when - [ ] The collection policy is written with a check date - [ ] The schema cannot hold identities beyond chosen creators - [ ] Every creator has a recorded reason - [ ] The exclusions are enforced by structure rather than by discipline ### Watch out - Building the collector first and deciding the policy afterwards produces a system you are not comfortable running, and by then the schema already holds data you did not want. Deciding first is cheaper and the schema then enforces it: if there is no column for follower identities, nothing can accidentally store them. ## Phase 2 · Collecting nightly, politely View counts arrive once a night without hammering anything. ### Steps 1. Write the source adapter One interface. Everything source-specific lives behind it, because the source will change or break. Files: `src/source.js` 2. Append observations One row per video per run. Unique on video and date so a rerun is a no-op. Files: `src/collect.js` 3. Rate limit and back off Well below any ceiling, with exponential backoff. Politeness here is also what keeps access working. Files: `src/collect.js` ### Done when - [ ] A nightly run records observations for every enabled creator - [ ] A second run the same night creates no duplicates - [ ] A source failure backs off and is reported - [ ] Growth over several days is visible for one video ### Watch out - Updating a video's view count in place loses the growth curve, which is the interesting part: a video that reached its number in a day is a different signal from one that took three weeks. Append observations; never overwrite them. ## Phase 3 · A baseline per creator A video is judged against its own creator's usual numbers, not against everyone's. ### Steps 1. Compute a rolling median Over a window in days, per creator. Recompute nightly rather than incrementally, because it is cheap at this size. Files: `src/baseline.js` 2. Require a minimum sample Below it, no baseline. A ratio computed from three videos is noise presented as a number. Files: `src/baseline.js` 3. Exclude the subject video Test with a hand-calculated example so the exclusion is proven rather than assumed. Files: `test/baseline.test.js` ### Done when - [ ] One viral video does not distort the creator's baseline - [ ] Below the minimum sample no baseline is produced - [ ] The judged video is excluded from its own baseline - [ ] Baselines match a hand calculation on a fixture ### Watch out - A mean is dragged upward by one viral video and stays high for months, so nothing after it ever looks like an outperformer. Median is resistant to exactly that. - Including the video in its own baseline dampens the very signal you are measuring, and it matters most for creators with few videos, which is where you most want the signal. ## Phase 4 · Detecting and labelling what worked You see the videos that beat their creator's baseline, with a note on what they did differently. ### Steps 1. Flag by ratio Views over the creator's baseline, above a threshold you choose. Store the ratio, not just the flag. Files: `src/detect.js` 2. Label only the flagged One model call per outperformer, with a hard cap per run. Files: `src/label.js` 3. Make labels correctable Your corrections are the training signal for your own label set, and they are more accurate than the model's first pass. Files: `src/label.js` ### Done when - [ ] Labels are produced only for flagged videos - [ ] The per-run cap prevents a spike from spending freely - [ ] Label corrections persist - [ ] Cost per run is recorded ### Watch out - Labelling every video means paying for thousands of labels you never read, and the ones that matter are a small fraction. Detect first, label second, and cap the number labelled per run so a source glitch that flags everything cannot produce a large bill. ## Phase 5 · The page you actually read A daily page showing what outperformed and why, in under a minute of reading. ### Steps 1. Generate a static page Nightly, written to disk. No server, no framework. Files: `src/render.js` 2. Rank and group By ratio, grouped by label so patterns are visible across creators. Files: `src/render.js` 3. Let quiet days be quiet An empty page is a true statement. Padding it teaches you to ignore the page. Files: `src/render.js` ### Done when - [ ] The page shows outperformers with ratio and baseline - [ ] A quiet day produces an honest empty page - [ ] Grouping by label reveals recurring formats - [ ] The page renders with no server ### Watch out - A page listing every video with a ratio above one is a page nobody reads, because most days nothing interesting happened. Set the threshold so a typical day shows a handful, and let an empty day be empty rather than padding it. ## Not in this build - A cross-creator corpus and search across it. - A curated taxonomy built from many accounts. - Any compliant route to platform data at commercial scale. - Coverage of creators you did not deliberately choose to watch. ## After v1, if you want it - Growth curve shape as a second signal, since reaching a number in a day differs from three weeks - Your own label taxonomy replacing the model's, once corrections accumulate - A weekly digest instead of daily, if the daily page turns out to be mostly empty ===== .env.example ===== # Copy to .env and fill in. Never commit .env; this file documents it. # Required. Creators, videos, observations and labels. DB_PATH=./data/watch.db # Required · secret. Your data source's credential. Priced per run, so treat it as a spending instrument. SOURCE_API_KEY=replace-with-key # Required. How far back the rolling median looks. BASELINE_WINDOW_DAYS=90 # Required. Minimum videos before a baseline is computed at all. BASELINE_MIN_SAMPLE=8 # Required. How far above the median a video must be to be flagged. Set it so a typical day shows a handful. OUTPERFORM_RATIO=2.0 # Required. Maximum videos labelled per run, so a source glitch cannot produce a large bill. LABEL_CAP_PER_RUN=15 # Required. How long raw observations are kept. Aggregates survive beyond this. OBSERVATION_RETENTION_DAYS=365
You are building a production product version of Wholana. Create the following project files first, then implement the application by following them. Keep the files updated as decisions change. Do not collapse this into a single README or prompt. ===== PRODUCT.md ===== # Wholana · product brief ## Problem The core idea is simple arithmetic: a video's views divided by that creator's own median. An agent can build that for a watchlist of creators you pick, over a weekend, on top of a paid scraper API. What it cannot hand you is the corpus, hundreds of thousands of videos already scraped, deduped, and labeled against a curated craft taxonomy, which is what makes search across creators useful instead of a list of your own bookmarks. So: yes for watching 25 creators you already know, no for finding the ones you don't. ## Product outcome A small, reasoned watchlist that tells you each morning which videos beat their own creator's baseline and what they did differently, without accumulating a database about people who never chose to be in it. ## Target user A builder who needs a maintainable product foundation, not a one-off demo. ## Required capabilities - TikTok scraper API (Apify or similar, paid per run) - LLM API key for hook labeling - SQLite - a nightly cron job - a scrape budget that recurs every month ## Explicit non-goals for v1 - A cross-creator corpus and search across it. - A curated taxonomy built from many accounts. - Any compliant route to platform data at commercial scale. - Coverage of creators you did not deliberately choose to watch. - the cross-creator corpus - search across videos you never chose to watch - a curated craft taxonomy instead of labels you invented - semantic and hybrid search - subject classification - creator equity and track-record views - someone else absorbing the scrape cost and keeping it running ## Success criteria - The collection policy note is current and dated - The schema is confirmed unable to hold identities beyond chosen creators - Baseline calculations match hand-computed fixtures - The per-run label cap is confirmed to bound spending - Removing a creator is confirmed to delete all related rows ===== BRIEF.md ===== # Build brief · Wholana The one-shot brief this plan expands. `BUILD_PLAN.md` (or `MILESTONES.md`) is the same sequence broken into steps and checks; where the two disagree, the plan wins. Build me a watchlist that tells you which of a creator's videos beat their own usual numbers to replace Wholana. Build it in phases, in the order below. Do not write the whole thing in one pass. Finish a phase, run its "Done when" check, fix what fails, and only then start the next phase. ### Rule zero Read the access question before choosing an architecture, because it decides what you can build. TikTok's Research API is free and is restricted to non-profit academic researchers at qualifying institutions in a handful of jurisdictions; applicants must be independent of commercial interests. If you are a company or an individual builder, you do not qualify. TikTok's terms also prohibit automated collection, so scraping carries real terms exposure, and using a third party scraping service moves some of that exposure without removing yours. Checked September 2026. The honest consequence: keep the watchlist small and specific to creators you have a reason to follow, store aggregate metrics rather than profiles of people, poll rarely, and write your position down. A tool that quietly builds a database about individuals is a different product from one that tracks a dozen accounts you already watch. ### Stack (fixed, do not substitute) - Runtime: Node 22 with SQLite through better-sqlite3 · a nightly job and a few thousand rows; anything larger is a scope problem rather than a database problem - Collection: A single source behind one adapter, polled nightly · one place to change when the source changes, which it will - Baseline: Rolling median of a creator's recent videos · median rather than mean, because one viral video would otherwise raise the bar for a year - Labelling: A model applied to the outperformers only · labelling everything costs money for videos you will never look at - Interface: A static page generated nightly · you read this once a day; a server is unnecessary ### Data model (create this before Phase 1) - creators(id, handle, added_at, reason, enabled) · reason is why you are watching them, which keeps the watchlist honest - videos(id, creator_id, external_id, posted_at, caption, duration_s, first_seen_at) - metrics(video_id, observed_at, views, likes, comments) · one row per observation, so growth over time is visible - baselines(creator_id, computed_at, window_days, median_views, sample_size) - outperformers(video_id, ratio, detected_at, labels_json, reviewed) · ratio is views over the creator's median - Aggregate counts only. No follower lists, no commenter identities, no profiles of people who did not choose to be in your database. ### Phase 1 · Deciding what you may collect Build: Before any code, choose the source and read its terms. Write a note naming the source, what it permits, what you will collect and what you will deliberately not collect. Then design the schema to make the exclusions structural rather than a matter of remembering. Trap: Building the collector first and deciding the policy afterwards produces a system you are not comfortable running, and by then the schema already holds data you did not want. Deciding first is cheaper and the schema then enforces it: if there is no column for follower identities, nothing can accidentally store them. Done when: A written note names the source, the permitted use, the check date and the collection boundary. The schema has no column capable of holding individual identities beyond the creator handles you deliberately chose. The watchlist has a reason recorded for every creator. Do not build yet: No collection at all yet. The policy first, because it shapes the schema. ### Phase 2 · Collecting nightly, politely Build: One adapter for the chosen source, called on a schedule. Rate limit well below any documented ceiling. Back off on failure. Record every observation as a new row rather than updating, so growth over time survives. Trap: Updating a video's view count in place loses the growth curve, which is the interesting part: a video that reached its number in a day is a different signal from one that took three weeks. Append observations; never overwrite them. Done when: A nightly run records observations for every enabled creator. Running twice in one night does not create duplicate observations for the same day. A source failure backs off and the run reports it rather than failing silently. The observation history for one video shows its growth over several days. Do not build yet: No baselines or outperformer detection yet. ### Phase 3 · A baseline per creator Build: Compute a rolling median of recent view counts per creator over a window you choose. Require a minimum sample size before producing a baseline at all. Exclude the video being judged from its own baseline. Trap: A mean is dragged upward by one viral video and stays high for months, so nothing after it ever looks like an outperformer. Median is resistant to exactly that. Trap: Including the video in its own baseline dampens the very signal you are measuring, and it matters most for creators with few videos, which is where you most want the signal. Done when: A creator with a single viral video still has a baseline near their typical numbers, verified against hand calculation. A creator below the minimum sample size produces no baseline rather than an unreliable one. Excluding the judged video from its own baseline changes the ratio, proving the exclusion is applied. Do not build yet: No detection or labelling yet. ### Phase 4 · Detecting and labelling what worked Build: Flag videos exceeding a ratio threshold. Send only those to a model to label the hook, format and topic. Cap spending. Keep labels as suggestions you can correct, and use your corrections to improve the label set over time. Trap: Labelling every video means paying for thousands of labels you never read, and the ones that matter are a small fraction. Detect first, label second, and cap the number labelled per run so a source glitch that flags everything cannot produce a large bill. Done when: Only videos above the threshold are labelled, verified by comparing the label count to the flagged count. A per-run label cap prevents a spike from producing a large bill. Labels are editable and corrections persist. Cost per run is recorded. Do not build yet: No review interface yet. ### Phase 5 · The page you actually read Build: Generate a static page nightly: outperformers ranked by ratio, with the creator, the baseline, the caption and the labels. Group by label so patterns are visible. Keep it short enough that you read all of it. Trap: A page listing every video with a ratio above one is a page nobody reads, because most days nothing interesting happened. Set the threshold so a typical day shows a handful, and let an empty day be empty rather than padding it. Done when: The nightly page shows outperformers with their ratio and baseline. A day with nothing above the threshold produces a page that says so rather than lowering the bar. Grouping by label shows which formats recur. The page renders without a server. Do not build yet: Retention and review come last. ### Phase 6 · Retention, review and the terms check Build: Retention that deletes old observations while keeping aggregates. A periodic review of the watchlist against its recorded reasons. A scheduled reminder to re-read the source's terms, because they change and your note has a date on it. Trap: A collection policy written once and never revisited becomes inaccurate silently, and the terms of these platforms change more often than most. Put a date on the note and a reminder in the calendar, so the claim in your README stays true. Done when: Retention removes old observations and keeps computed aggregates, verified by row counts. A review lists creators whose recorded reason no longer applies. The terms note carries a date and a reminder exists to re-check it. Removing a creator deletes their videos, metrics and labels together. Do not build yet: Nothing after this. ### Out of scope (and why) - A cross-creator corpus. The hosted product's value is comparing against videos you never chose to watch, and building that means collecting at a scale this build deliberately refuses. - Search across the platform. That needs the corpus, which needs the collection, which is the thing rule zero is about. - A curated taxonomy built from many accounts' data. Your labels are your own and will be less useful for the first few months. - Any claim that this is a compliant route to platform data at commercial scale. It is not, and the terms note exists so you know what you are choosing. ### README must contain - Which source is used, what its terms permit, and the date that was checked - That collection is aggregate counts for a chosen watchlist and deliberately excludes individual identities - That baselines use a median per creator, and why a mean would fail after one viral video - That only outperformers are labelled, with the per-run cap - The retention window and how to remove a creator entirely ===== ARCHITECTURE.md ===== # Architecture · Wholana ## Stack | Part | Choice | Why | | --- | --- | --- | | Runtime | Node 22 with SQLite through better-sqlite3 | a nightly job and a few thousand rows; anything larger is a scope problem rather than a database problem | | Collection | A single source behind one adapter, polled nightly | one place to change when the source changes, which it will | | Baseline | Rolling median of a creator's recent videos | median rather than mean, because one viral video would otherwise raise the bar for a year | | Labelling | A model applied to the outperformers only | labelling everything costs money for videos you will never look at | | Interface | A static page generated nightly | you read this once a day; a server is unnecessary | ## Modules Each module has one owner concern and a documented way to replace it. | Module | Owns | How to replace it | | --- | --- | --- | | Policy | What may be collected and from where. | This is the module that shapes the schema. Changing it means revisiting the schema, which is the point of deciding first. | | Source | Fetching from one provider. | One adapter, because sources in this area change and break. Everything else is source-agnostic. | | Baseline | Per-creator medians. | Pure computation over observations, testable against hand calculations with no network. | | Detect and label | Flagging and describing outperformers. | Any model. The rule that labelling happens only after detection is what keeps the cost bounded. | | Render | The daily page. | Presentation only, and deliberately static. | ## Configuration Every runtime setting is an environment variable documented in `.env.example`, validated at startup, with a safe local default wherever one exists. - `DB_PATH` · required · Creators, videos, observations and labels. - `SOURCE_API_KEY` · required, secret · Your data source's credential. Priced per run, so treat it as a spending instrument. - `BASELINE_WINDOW_DAYS` · required · How far back the rolling median looks. - `BASELINE_MIN_SAMPLE` · required · Minimum videos before a baseline is computed at all. - `OUTPERFORM_RATIO` · required · How far above the median a video must be to be flagged. Set it so a typical day shows a handful. - `LABEL_CAP_PER_RUN` · required · Maximum videos labelled per run, so a source glitch cannot produce a large bill. - `OBSERVATION_RETENTION_DAYS` · required · How long raw observations are kept. Aggregates survive beyond this. ## Production baseline - Security: least privilege, input validation at every boundary, secret redaction in logs, rate limits on abuse-prone paths, no invented security primitives. - Data: explicit schema and migrations, transactional writes where integrity matters, backup and restore procedures that have been exercised. - Integrations: adapters around third-party providers, idempotent webhook or job processing, bounded retries, timeouts. - Observability: structured logs with request or operation ids, an error-tracking hook, and health and readiness checks where a server exists. - Quality: unit tests for domain rules, integration tests at module boundaries, one end-to-end test of the critical path. ## Decision records For each dependency in the stack table, keep a short note: why it was chosen, its failure mode, and how it is replaced. Do not add infrastructure until a requirement in `PRODUCT.md` justifies it. ===== AGENTS.md ===== # Agent instructions · Wholana product build - Read `PRODUCT.md` and `ARCHITECTURE.md` before changing code. The stack is fixed: Node 22 with SQLite through better-sqlite3, A single source behind one adapter, polled nightly, Rolling median of a creator's recent videos, A model applied to the outperformers only, A static page generated nightly. - Implement milestone by milestone from `MILESTONES.md`; keep each change reviewable and leave the application runnable at every commit. - Treat authentication, payments, encryption, imports, webhooks and destructive actions as high-risk boundaries when present. - Never invent cryptography or silently weaken a requirement to make a check pass. - Put every external service behind an interface with a deterministic fake for tests. - Add migrations and rollback or recovery notes for every persistent data change. - Log useful operational context without credentials, tokens, passwords or personal data. - Update documentation and run every check before completing a milestone. ## Known traps - Building the collector first and deciding the policy afterwards produces a system you are not comfortable running, and by then the schema already holds data you did not want. Deciding first is cheaper and the schema then enforces it: if there is no column for follower identities, nothing can accidentally store them. - Updating a video's view count in place loses the growth curve, which is the interesting part: a video that reached its number in a day is a different signal from one that took three weeks. Append observations; never overwrite them. - A mean is dragged upward by one viral video and stays high for months, so nothing after it ever looks like an outperformer. Median is resistant to exactly that. - Including the video in its own baseline dampens the very signal you are measuring, and it matters most for creators with few videos, which is where you most want the signal. - Labelling every video means paying for thousands of labels you never read, and the ones that matter are a small fraction. Detect first, label second, and cap the number labelled per run so a source glitch that flags everything cannot produce a large bill. - A page listing every video with a ratio above one is a page nobody reads, because most days nothing interesting happened. Set the threshold so a typical day shows a handful, and let an empty day be empty rather than padding it. - A collection policy written once and never revisited becomes inaccurate silently, and the terms of these platforms change more often than most. Put a date on the note and a reminder in the calendar, so the claim in your README stays true. ===== MILESTONES.md ===== # Delivery milestones · Wholana Estimated effort: **weekend** for the indie phases; the production-only milestones add the trust and operability layer. ## M1 · Deciding what you may collect You have written down what you are collecting, from where, and under what terms. ### Steps 1. Read the terms and write the note Source, permitted use, date checked, and what you will not collect. One page, in the repository. Files: `docs/collection-policy.md` 2. Design the schema around the exclusions Aggregate counts, no identity columns for anyone but the creators you chose. Structural rather than remembered. Files: `src/schema.sql` 3. Record a reason per creator Why you are watching them. A watchlist without reasons grows into surveillance by accident. Files: `src/creators.js` ### Done when - [ ] The collection policy is written with a check date - [ ] The schema cannot hold identities beyond chosen creators - [ ] Every creator has a recorded reason - [ ] The exclusions are enforced by structure rather than by discipline ### Watch out - Building the collector first and deciding the policy afterwards produces a system you are not comfortable running, and by then the schema already holds data you did not want. Deciding first is cheaper and the schema then enforces it: if there is no column for follower identities, nothing can accidentally store them. ## M2 · Collecting nightly, politely View counts arrive once a night without hammering anything. ### Steps 1. Write the source adapter One interface. Everything source-specific lives behind it, because the source will change or break. Files: `src/source.js` 2. Append observations One row per video per run. Unique on video and date so a rerun is a no-op. Files: `src/collect.js` 3. Rate limit and back off Well below any ceiling, with exponential backoff. Politeness here is also what keeps access working. Files: `src/collect.js` ### Done when - [ ] A nightly run records observations for every enabled creator - [ ] A second run the same night creates no duplicates - [ ] A source failure backs off and is reported - [ ] Growth over several days is visible for one video ### Watch out - Updating a video's view count in place loses the growth curve, which is the interesting part: a video that reached its number in a day is a different signal from one that took three weeks. Append observations; never overwrite them. ## M3 · A baseline per creator A video is judged against its own creator's usual numbers, not against everyone's. ### Steps 1. Compute a rolling median Over a window in days, per creator. Recompute nightly rather than incrementally, because it is cheap at this size. Files: `src/baseline.js` 2. Require a minimum sample Below it, no baseline. A ratio computed from three videos is noise presented as a number. Files: `src/baseline.js` 3. Exclude the subject video Test with a hand-calculated example so the exclusion is proven rather than assumed. Files: `test/baseline.test.js` ### Done when - [ ] One viral video does not distort the creator's baseline - [ ] Below the minimum sample no baseline is produced - [ ] The judged video is excluded from its own baseline - [ ] Baselines match a hand calculation on a fixture ### Watch out - A mean is dragged upward by one viral video and stays high for months, so nothing after it ever looks like an outperformer. Median is resistant to exactly that. - Including the video in its own baseline dampens the very signal you are measuring, and it matters most for creators with few videos, which is where you most want the signal. ## M4 · Detecting and labelling what worked You see the videos that beat their creator's baseline, with a note on what they did differently. ### Steps 1. Flag by ratio Views over the creator's baseline, above a threshold you choose. Store the ratio, not just the flag. Files: `src/detect.js` 2. Label only the flagged One model call per outperformer, with a hard cap per run. Files: `src/label.js` 3. Make labels correctable Your corrections are the training signal for your own label set, and they are more accurate than the model's first pass. Files: `src/label.js` ### Done when - [ ] Labels are produced only for flagged videos - [ ] The per-run cap prevents a spike from spending freely - [ ] Label corrections persist - [ ] Cost per run is recorded ### Watch out - Labelling every video means paying for thousands of labels you never read, and the ones that matter are a small fraction. Detect first, label second, and cap the number labelled per run so a source glitch that flags everything cannot produce a large bill. ## M5 · The page you actually read A daily page showing what outperformed and why, in under a minute of reading. ### Steps 1. Generate a static page Nightly, written to disk. No server, no framework. Files: `src/render.js` 2. Rank and group By ratio, grouped by label so patterns are visible across creators. Files: `src/render.js` 3. Let quiet days be quiet An empty page is a true statement. Padding it teaches you to ignore the page. Files: `src/render.js` ### Done when - [ ] The page shows outperformers with ratio and baseline - [ ] A quiet day produces an honest empty page - [ ] Grouping by label reveals recurring formats - [ ] The page renders with no server ### Watch out - A page listing every video with a ratio above one is a page nobody reads, because most days nothing interesting happened. Set the threshold so a typical day shows a handful, and let an empty day be empty rather than padding it. ## M6 · Retention, review and the terms check (production only) The database does not grow into something you would not want to explain, and the terms note stays current. ### Steps 1. Add retention Delete raw observations past a window; keep baselines and outperformer records, which are small. Files: `src/retention.js` 2. Review the watchlist Against the recorded reasons. A creator you no longer have a reason to watch comes off the list. Files: `src/review.js` 3. Schedule a terms re-check A reminder tied to the date on the note. Terms change and your README makes a claim about them. Files: `docs/collection-policy.md` ### Done when - [ ] Retention removes raw observations and keeps aggregates - [ ] The review surfaces creators without a current reason - [ ] The terms note carries a date and a re-check reminder exists - [ ] Removing a creator deletes all their related rows ### Watch out - A collection policy written once and never revisited becomes inaccurate silently, and the terms of these platforms change more often than most. Put a date on the note and a reminder in the calendar, so the claim in your README stays true. ===== OPERATIONS.md ===== # Operations · Wholana ## Backup The database is small and holds your watchlist, observations and labels. Back it up, and remember it contains data about other people's accounts even in aggregate form. ## Restore Restore the database and resume the nightly job. Missed nights are gaps in the observation history rather than errors, and the baseline handles gaps by design. Do a restore drill before the first real user, and write the date here when it passes. ## Monitoring Run success rate, per-run cost, and the count of flagged videos. A sudden jump in flagged videos usually means the baseline computation broke rather than that everyone had a good day. ## Incident checklist If the source's terms change or access is withdrawn, stop the job before deciding what to do. If the database has grown beyond what your policy describes, retention and the watchlist review are the tools, and the policy note is what tells you it happened. 1. Contain the issue without destroying evidence or user data. 2. Record the timeline and affected scope. 3. Rotate exposed secrets and revoke compromised sessions or credentials. 4. Restore from a verified backup when needed. 5. Document the root cause, the remediation and the regression test. ## Release gate - [ ] The collection policy note is current and dated - [ ] The schema is confirmed unable to hold identities beyond chosen creators - [ ] Baseline calculations match hand-computed fixtures - [ ] The per-run label cap is confirmed to bound spending - [ ] Removing a creator is confirmed to delete all related rows ## Launch constraint Do not market omitted Wholana capabilities as implemented. The non-goals in `PRODUCT.md` remain user-visible limitations until they are deliberately delivered. ===== .env.example ===== # Copy to .env and fill in. Never commit .env; this file documents it. # Required. Creators, videos, observations and labels. DB_PATH=./data/watch.db # Required · secret. Your data source's credential. Priced per run, so treat it as a spending instrument. SOURCE_API_KEY=replace-with-key # Required. How far back the rolling median looks. BASELINE_WINDOW_DAYS=90 # Required. Minimum videos before a baseline is computed at all. BASELINE_MIN_SAMPLE=8 # Required. How far above the median a video must be to be flagged. Set it so a typical day shows a handful. OUTPERFORM_RATIO=2.0 # Required. Maximum videos labelled per run, so a source glitch cannot produce a large bill. LABEL_CAP_PER_RUN=15 # Required. How long raw observations are kept. Aggregates survive beyond this. OBSERVATION_RETENTION_DAYS=365
README.md
# Wholana · indie build A nightly job that records view counts for a small watchlist of creators, computes each creator's rolling median so a video is judged against their own baseline rather than against a global one, surfaces the ones that outperformed, and labels what those videos did differently. Estimated effort: **weekend**. Work `BUILD_PLAN.md` top to bottom · every phase ends in a check that has to pass before the next one starts. ## Stack | Part | Choice | Why | | --- | --- | --- | | Runtime | Node 22 with SQLite through better-sqlite3 | a nightly job and a few thousand rows; anything larger is a scope problem rather than a database problem | | Collection | A single source behind one adapter, polled nightly | one place to change when the source changes, which it will | | Baseline | Rolling median of a creator's recent videos | median rather than mean, because one viral video would otherwise raise the bar for a year | | Labelling | A model applied to the outperformers only | labelling everything costs money for videos you will never look at | | Interface | A static page generated nightly | you read this once a day; a server is unnecessary | ## Before you start Have every one of these ready. The plan assumes them from step one. - [ ] **Node.js 22 or newer** · free - Why: Runs the nightly job and the page generator. - Get it: nodejs.org, or nvm install 22. - Verify: node --version prints v22 or higher - [ ] **A read of the source's terms, and your own position on them** · free, and it is the first task rather than a formality - Why: This decides whether you build at all and what shape it takes. TikTok's Research API is limited to non-profit academic researchers at qualifying institutions and excludes commercial interests; the platform terms prohibit automated collection. - Get it: Read the Research API eligibility page and the platform terms. Decide whether you qualify, and if not, decide deliberately what you are willing to do and write it down with the date. - Verify: A written note names the source, the permitted use and the date checked - [ ] **Access to a data source** · the Research API is free for those eligible; third party services charge per run and add up with a nightly schedule - Why: Without it there is nothing to collect. Third party scraping services are the usual route and are priced per run. - Get it: If you qualify for the Research API, apply; the process takes up to about four weeks. Otherwise, evaluate a third party service and understand that its terms exposure does not fully replace yours. - Verify: A test run returns video data for one creator - [ ] **A watchlist of creators with reasons** · free - Why: A small, reasoned watchlist is a different thing from broad collection, and the reasons are what keep it that way over time. - Get it: List the creators you already follow for a reason, and write the reason next to each. If you cannot write a reason, leave them off. - Verify: Every creator on the list has a recorded reason - [ ] **A model API key for labelling** (optional) · pay per token; labelling a handful of videos a night is negligible, labelling everything is not - Why: Phase 4 only. Detection works without it and labelling is a convenience on top. - Get it: A provider key with a spend limit set at the provider. - Verify: A test label returns and its cost is recorded ## Quick start Follow Phase 1 in `BUILD_PLAN.md`; it creates the project. Then copy `.env.example` to `.env` and fill in the values it documents. ## Honest limits This build deliberately does not replace: - A cross-creator corpus and search across it. - A curated taxonomy built from many accounts. - Any compliant route to platform data at commercial scale. - Coverage of creators you did not deliberately choose to watch. - the cross-creator corpus - search across videos you never chose to watch - a curated craft taxonomy instead of labels you invented - semantic and hybrid search - subject classification - creator equity and track-record views - someone else absorbing the scrape cost and keeping it running If one of those is essential to you, that is the reason to keep paying for Wholana, and the README should say so rather than pretend.
BRIEF.md
# Build brief · Wholana The one-shot brief this plan expands. `BUILD_PLAN.md` (or `MILESTONES.md`) is the same sequence broken into steps and checks; where the two disagree, the plan wins. Build me a watchlist that tells you which of a creator's videos beat their own usual numbers to replace Wholana. Build it in phases, in the order below. Do not write the whole thing in one pass. Finish a phase, run its "Done when" check, fix what fails, and only then start the next phase. ### Rule zero Read the access question before choosing an architecture, because it decides what you can build. TikTok's Research API is free and is restricted to non-profit academic researchers at qualifying institutions in a handful of jurisdictions; applicants must be independent of commercial interests. If you are a company or an individual builder, you do not qualify. TikTok's terms also prohibit automated collection, so scraping carries real terms exposure, and using a third party scraping service moves some of that exposure without removing yours. Checked September 2026. The honest consequence: keep the watchlist small and specific to creators you have a reason to follow, store aggregate metrics rather than profiles of people, poll rarely, and write your position down. A tool that quietly builds a database about individuals is a different product from one that tracks a dozen accounts you already watch. ### Stack (fixed, do not substitute) - Runtime: Node 22 with SQLite through better-sqlite3 · a nightly job and a few thousand rows; anything larger is a scope problem rather than a database problem - Collection: A single source behind one adapter, polled nightly · one place to change when the source changes, which it will - Baseline: Rolling median of a creator's recent videos · median rather than mean, because one viral video would otherwise raise the bar for a year - Labelling: A model applied to the outperformers only · labelling everything costs money for videos you will never look at - Interface: A static page generated nightly · you read this once a day; a server is unnecessary ### Data model (create this before Phase 1) - creators(id, handle, added_at, reason, enabled) · reason is why you are watching them, which keeps the watchlist honest - videos(id, creator_id, external_id, posted_at, caption, duration_s, first_seen_at) - metrics(video_id, observed_at, views, likes, comments) · one row per observation, so growth over time is visible - baselines(creator_id, computed_at, window_days, median_views, sample_size) - outperformers(video_id, ratio, detected_at, labels_json, reviewed) · ratio is views over the creator's median - Aggregate counts only. No follower lists, no commenter identities, no profiles of people who did not choose to be in your database. ### Phase 1 · Deciding what you may collect Build: Before any code, choose the source and read its terms. Write a note naming the source, what it permits, what you will collect and what you will deliberately not collect. Then design the schema to make the exclusions structural rather than a matter of remembering. Trap: Building the collector first and deciding the policy afterwards produces a system you are not comfortable running, and by then the schema already holds data you did not want. Deciding first is cheaper and the schema then enforces it: if there is no column for follower identities, nothing can accidentally store them. Done when: A written note names the source, the permitted use, the check date and the collection boundary. The schema has no column capable of holding individual identities beyond the creator handles you deliberately chose. The watchlist has a reason recorded for every creator. Do not build yet: No collection at all yet. The policy first, because it shapes the schema. ### Phase 2 · Collecting nightly, politely Build: One adapter for the chosen source, called on a schedule. Rate limit well below any documented ceiling. Back off on failure. Record every observation as a new row rather than updating, so growth over time survives. Trap: Updating a video's view count in place loses the growth curve, which is the interesting part: a video that reached its number in a day is a different signal from one that took three weeks. Append observations; never overwrite them. Done when: A nightly run records observations for every enabled creator. Running twice in one night does not create duplicate observations for the same day. A source failure backs off and the run reports it rather than failing silently. The observation history for one video shows its growth over several days. Do not build yet: No baselines or outperformer detection yet. ### Phase 3 · A baseline per creator Build: Compute a rolling median of recent view counts per creator over a window you choose. Require a minimum sample size before producing a baseline at all. Exclude the video being judged from its own baseline. Trap: A mean is dragged upward by one viral video and stays high for months, so nothing after it ever looks like an outperformer. Median is resistant to exactly that. Trap: Including the video in its own baseline dampens the very signal you are measuring, and it matters most for creators with few videos, which is where you most want the signal. Done when: A creator with a single viral video still has a baseline near their typical numbers, verified against hand calculation. A creator below the minimum sample size produces no baseline rather than an unreliable one. Excluding the judged video from its own baseline changes the ratio, proving the exclusion is applied. Do not build yet: No detection or labelling yet. ### Phase 4 · Detecting and labelling what worked Build: Flag videos exceeding a ratio threshold. Send only those to a model to label the hook, format and topic. Cap spending. Keep labels as suggestions you can correct, and use your corrections to improve the label set over time. Trap: Labelling every video means paying for thousands of labels you never read, and the ones that matter are a small fraction. Detect first, label second, and cap the number labelled per run so a source glitch that flags everything cannot produce a large bill. Done when: Only videos above the threshold are labelled, verified by comparing the label count to the flagged count. A per-run label cap prevents a spike from producing a large bill. Labels are editable and corrections persist. Cost per run is recorded. Do not build yet: No review interface yet. ### Phase 5 · The page you actually read Build: Generate a static page nightly: outperformers ranked by ratio, with the creator, the baseline, the caption and the labels. Group by label so patterns are visible. Keep it short enough that you read all of it. Trap: A page listing every video with a ratio above one is a page nobody reads, because most days nothing interesting happened. Set the threshold so a typical day shows a handful, and let an empty day be empty rather than padding it. Done when: The nightly page shows outperformers with their ratio and baseline. A day with nothing above the threshold produces a page that says so rather than lowering the bar. Grouping by label shows which formats recur. The page renders without a server. Do not build yet: Retention and review come last. ### Phase 6 · Retention, review and the terms check Build: Retention that deletes old observations while keeping aggregates. A periodic review of the watchlist against its recorded reasons. A scheduled reminder to re-read the source's terms, because they change and your note has a date on it. Trap: A collection policy written once and never revisited becomes inaccurate silently, and the terms of these platforms change more often than most. Put a date on the note and a reminder in the calendar, so the claim in your README stays true. Done when: Retention removes old observations and keeps computed aggregates, verified by row counts. A review lists creators whose recorded reason no longer applies. The terms note carries a date and a reminder exists to re-check it. Removing a creator deletes their videos, metrics and labels together. Do not build yet: Nothing after this. ### Out of scope (and why) - A cross-creator corpus. The hosted product's value is comparing against videos you never chose to watch, and building that means collecting at a scale this build deliberately refuses. - Search across the platform. That needs the corpus, which needs the collection, which is the thing rule zero is about. - A curated taxonomy built from many accounts' data. Your labels are your own and will be less useful for the first few months. - Any claim that this is a compliant route to platform data at commercial scale. It is not, and the terms note exists so you know what you are choosing. ### README must contain - Which source is used, what its terms permit, and the date that was checked - That collection is aggregate counts for a chosen watchlist and deliberately excludes individual identities - That baselines use a median per creator, and why a mean would fail after one viral video - That only outperformers are labelled, with the per-run cap - The retention window and how to remove a creator entirely
AGENTS.md
# Agent instructions · Wholana indie build - Read `README.md` and `BUILD_PLAN.md` before writing code. The stack is fixed: Node 22 with SQLite through better-sqlite3, A single source behind one adapter, polled nightly, Rolling median of a creator's recent videos, A model applied to the outperformers only, A static page generated nightly. Do not substitute. - Work one phase at a time, in order. Do not start a phase until every "Done when" item of the previous one passes. - Prefer the fewest moving parts that satisfy the step. No frameworks, services or dependencies the plan does not name. - Secrets live in `.env`, never in source or logs. Keep `.env.example` current when a variable is introduced. - Do not invent cryptography, security guarantees, APIs or compliance claims. - Add a focused test for every destructive, security-sensitive or data-loss path the plan names. - Run the project checks before declaring a phase complete, and record any deliberate shortcut in the README under "Tradeoffs". ## Known traps - Building the collector first and deciding the policy afterwards produces a system you are not comfortable running, and by then the schema already holds data you did not want. Deciding first is cheaper and the schema then enforces it: if there is no column for follower identities, nothing can accidentally store them. - Updating a video's view count in place loses the growth curve, which is the interesting part: a video that reached its number in a day is a different signal from one that took three weeks. Append observations; never overwrite them. - A mean is dragged upward by one viral video and stays high for months, so nothing after it ever looks like an outperformer. Median is resistant to exactly that. - Including the video in its own baseline dampens the very signal you are measuring, and it matters most for creators with few videos, which is where you most want the signal. - Labelling every video means paying for thousands of labels you never read, and the ones that matter are a small fraction. Detect first, label second, and cap the number labelled per run so a source glitch that flags everything cannot produce a large bill. - A page listing every video with a ratio above one is a page nobody reads, because most days nothing interesting happened. Set the threshold so a typical day shows a handful, and let an empty day be empty rather than padding it.
BUILD_PLAN.md
# Build plan · Wholana A nightly job that records view counts for a small watchlist of creators, computes each creator's rolling median so a video is judged against their own baseline rather than against a global one, surfaces the ones that outperformed, and labels what those videos did differently. Phases are in dependency order. Each ends in a "Done when" list; treat an unticked item as a blocker, not a note. ## Phase 1 · Deciding what you may collect You have written down what you are collecting, from where, and under what terms. ### Steps 1. Read the terms and write the note Source, permitted use, date checked, and what you will not collect. One page, in the repository. Files: `docs/collection-policy.md` 2. Design the schema around the exclusions Aggregate counts, no identity columns for anyone but the creators you chose. Structural rather than remembered. Files: `src/schema.sql` 3. Record a reason per creator Why you are watching them. A watchlist without reasons grows into surveillance by accident. Files: `src/creators.js` ### Done when - [ ] The collection policy is written with a check date - [ ] The schema cannot hold identities beyond chosen creators - [ ] Every creator has a recorded reason - [ ] The exclusions are enforced by structure rather than by discipline ### Watch out - Building the collector first and deciding the policy afterwards produces a system you are not comfortable running, and by then the schema already holds data you did not want. Deciding first is cheaper and the schema then enforces it: if there is no column for follower identities, nothing can accidentally store them. ## Phase 2 · Collecting nightly, politely View counts arrive once a night without hammering anything. ### Steps 1. Write the source adapter One interface. Everything source-specific lives behind it, because the source will change or break. Files: `src/source.js` 2. Append observations One row per video per run. Unique on video and date so a rerun is a no-op. Files: `src/collect.js` 3. Rate limit and back off Well below any ceiling, with exponential backoff. Politeness here is also what keeps access working. Files: `src/collect.js` ### Done when - [ ] A nightly run records observations for every enabled creator - [ ] A second run the same night creates no duplicates - [ ] A source failure backs off and is reported - [ ] Growth over several days is visible for one video ### Watch out - Updating a video's view count in place loses the growth curve, which is the interesting part: a video that reached its number in a day is a different signal from one that took three weeks. Append observations; never overwrite them. ## Phase 3 · A baseline per creator A video is judged against its own creator's usual numbers, not against everyone's. ### Steps 1. Compute a rolling median Over a window in days, per creator. Recompute nightly rather than incrementally, because it is cheap at this size. Files: `src/baseline.js` 2. Require a minimum sample Below it, no baseline. A ratio computed from three videos is noise presented as a number. Files: `src/baseline.js` 3. Exclude the subject video Test with a hand-calculated example so the exclusion is proven rather than assumed. Files: `test/baseline.test.js` ### Done when - [ ] One viral video does not distort the creator's baseline - [ ] Below the minimum sample no baseline is produced - [ ] The judged video is excluded from its own baseline - [ ] Baselines match a hand calculation on a fixture ### Watch out - A mean is dragged upward by one viral video and stays high for months, so nothing after it ever looks like an outperformer. Median is resistant to exactly that. - Including the video in its own baseline dampens the very signal you are measuring, and it matters most for creators with few videos, which is where you most want the signal. ## Phase 4 · Detecting and labelling what worked You see the videos that beat their creator's baseline, with a note on what they did differently. ### Steps 1. Flag by ratio Views over the creator's baseline, above a threshold you choose. Store the ratio, not just the flag. Files: `src/detect.js` 2. Label only the flagged One model call per outperformer, with a hard cap per run. Files: `src/label.js` 3. Make labels correctable Your corrections are the training signal for your own label set, and they are more accurate than the model's first pass. Files: `src/label.js` ### Done when - [ ] Labels are produced only for flagged videos - [ ] The per-run cap prevents a spike from spending freely - [ ] Label corrections persist - [ ] Cost per run is recorded ### Watch out - Labelling every video means paying for thousands of labels you never read, and the ones that matter are a small fraction. Detect first, label second, and cap the number labelled per run so a source glitch that flags everything cannot produce a large bill. ## Phase 5 · The page you actually read A daily page showing what outperformed and why, in under a minute of reading. ### Steps 1. Generate a static page Nightly, written to disk. No server, no framework. Files: `src/render.js` 2. Rank and group By ratio, grouped by label so patterns are visible across creators. Files: `src/render.js` 3. Let quiet days be quiet An empty page is a true statement. Padding it teaches you to ignore the page. Files: `src/render.js` ### Done when - [ ] The page shows outperformers with ratio and baseline - [ ] A quiet day produces an honest empty page - [ ] Grouping by label reveals recurring formats - [ ] The page renders with no server ### Watch out - A page listing every video with a ratio above one is a page nobody reads, because most days nothing interesting happened. Set the threshold so a typical day shows a handful, and let an empty day be empty rather than padding it. ## Not in this build - A cross-creator corpus and search across it. - A curated taxonomy built from many accounts. - Any compliant route to platform data at commercial scale. - Coverage of creators you did not deliberately choose to watch. ## After v1, if you want it - Growth curve shape as a second signal, since reaching a number in a day differs from three weeks - Your own label taxonomy replacing the model's, once corrections accumulate - A weekly digest instead of daily, if the daily page turns out to be mostly empty
.env.example
# Copy to .env and fill in. Never commit .env; this file documents it. # Required. Creators, videos, observations and labels. DB_PATH=./data/watch.db # Required · secret. Your data source's credential. Priced per run, so treat it as a spending instrument. SOURCE_API_KEY=replace-with-key # Required. How far back the rolling median looks. BASELINE_WINDOW_DAYS=90 # Required. Minimum videos before a baseline is computed at all. BASELINE_MIN_SAMPLE=8 # Required. How far above the median a video must be to be flagged. Set it so a typical day shows a handful. OUTPERFORM_RATIO=2.0 # Required. Maximum videos labelled per run, so a source glitch cannot produce a large bill. LABEL_CAP_PER_RUN=15 # Required. How long raw observations are kept. Aggregates survive beyond this. OBSERVATION_RETENTION_DAYS=365
PRODUCT.md
# Wholana · product brief ## Problem The core idea is simple arithmetic: a video's views divided by that creator's own median. An agent can build that for a watchlist of creators you pick, over a weekend, on top of a paid scraper API. What it cannot hand you is the corpus, hundreds of thousands of videos already scraped, deduped, and labeled against a curated craft taxonomy, which is what makes search across creators useful instead of a list of your own bookmarks. So: yes for watching 25 creators you already know, no for finding the ones you don't. ## Product outcome A small, reasoned watchlist that tells you each morning which videos beat their own creator's baseline and what they did differently, without accumulating a database about people who never chose to be in it. ## Target user A builder who needs a maintainable product foundation, not a one-off demo. ## Required capabilities - TikTok scraper API (Apify or similar, paid per run) - LLM API key for hook labeling - SQLite - a nightly cron job - a scrape budget that recurs every month ## Explicit non-goals for v1 - A cross-creator corpus and search across it. - A curated taxonomy built from many accounts. - Any compliant route to platform data at commercial scale. - Coverage of creators you did not deliberately choose to watch. - the cross-creator corpus - search across videos you never chose to watch - a curated craft taxonomy instead of labels you invented - semantic and hybrid search - subject classification - creator equity and track-record views - someone else absorbing the scrape cost and keeping it running ## Success criteria - The collection policy note is current and dated - The schema is confirmed unable to hold identities beyond chosen creators - Baseline calculations match hand-computed fixtures - The per-run label cap is confirmed to bound spending - Removing a creator is confirmed to delete all related rows
BRIEF.md
# Build brief · Wholana The one-shot brief this plan expands. `BUILD_PLAN.md` (or `MILESTONES.md`) is the same sequence broken into steps and checks; where the two disagree, the plan wins. Build me a watchlist that tells you which of a creator's videos beat their own usual numbers to replace Wholana. Build it in phases, in the order below. Do not write the whole thing in one pass. Finish a phase, run its "Done when" check, fix what fails, and only then start the next phase. ### Rule zero Read the access question before choosing an architecture, because it decides what you can build. TikTok's Research API is free and is restricted to non-profit academic researchers at qualifying institutions in a handful of jurisdictions; applicants must be independent of commercial interests. If you are a company or an individual builder, you do not qualify. TikTok's terms also prohibit automated collection, so scraping carries real terms exposure, and using a third party scraping service moves some of that exposure without removing yours. Checked September 2026. The honest consequence: keep the watchlist small and specific to creators you have a reason to follow, store aggregate metrics rather than profiles of people, poll rarely, and write your position down. A tool that quietly builds a database about individuals is a different product from one that tracks a dozen accounts you already watch. ### Stack (fixed, do not substitute) - Runtime: Node 22 with SQLite through better-sqlite3 · a nightly job and a few thousand rows; anything larger is a scope problem rather than a database problem - Collection: A single source behind one adapter, polled nightly · one place to change when the source changes, which it will - Baseline: Rolling median of a creator's recent videos · median rather than mean, because one viral video would otherwise raise the bar for a year - Labelling: A model applied to the outperformers only · labelling everything costs money for videos you will never look at - Interface: A static page generated nightly · you read this once a day; a server is unnecessary ### Data model (create this before Phase 1) - creators(id, handle, added_at, reason, enabled) · reason is why you are watching them, which keeps the watchlist honest - videos(id, creator_id, external_id, posted_at, caption, duration_s, first_seen_at) - metrics(video_id, observed_at, views, likes, comments) · one row per observation, so growth over time is visible - baselines(creator_id, computed_at, window_days, median_views, sample_size) - outperformers(video_id, ratio, detected_at, labels_json, reviewed) · ratio is views over the creator's median - Aggregate counts only. No follower lists, no commenter identities, no profiles of people who did not choose to be in your database. ### Phase 1 · Deciding what you may collect Build: Before any code, choose the source and read its terms. Write a note naming the source, what it permits, what you will collect and what you will deliberately not collect. Then design the schema to make the exclusions structural rather than a matter of remembering. Trap: Building the collector first and deciding the policy afterwards produces a system you are not comfortable running, and by then the schema already holds data you did not want. Deciding first is cheaper and the schema then enforces it: if there is no column for follower identities, nothing can accidentally store them. Done when: A written note names the source, the permitted use, the check date and the collection boundary. The schema has no column capable of holding individual identities beyond the creator handles you deliberately chose. The watchlist has a reason recorded for every creator. Do not build yet: No collection at all yet. The policy first, because it shapes the schema. ### Phase 2 · Collecting nightly, politely Build: One adapter for the chosen source, called on a schedule. Rate limit well below any documented ceiling. Back off on failure. Record every observation as a new row rather than updating, so growth over time survives. Trap: Updating a video's view count in place loses the growth curve, which is the interesting part: a video that reached its number in a day is a different signal from one that took three weeks. Append observations; never overwrite them. Done when: A nightly run records observations for every enabled creator. Running twice in one night does not create duplicate observations for the same day. A source failure backs off and the run reports it rather than failing silently. The observation history for one video shows its growth over several days. Do not build yet: No baselines or outperformer detection yet. ### Phase 3 · A baseline per creator Build: Compute a rolling median of recent view counts per creator over a window you choose. Require a minimum sample size before producing a baseline at all. Exclude the video being judged from its own baseline. Trap: A mean is dragged upward by one viral video and stays high for months, so nothing after it ever looks like an outperformer. Median is resistant to exactly that. Trap: Including the video in its own baseline dampens the very signal you are measuring, and it matters most for creators with few videos, which is where you most want the signal. Done when: A creator with a single viral video still has a baseline near their typical numbers, verified against hand calculation. A creator below the minimum sample size produces no baseline rather than an unreliable one. Excluding the judged video from its own baseline changes the ratio, proving the exclusion is applied. Do not build yet: No detection or labelling yet. ### Phase 4 · Detecting and labelling what worked Build: Flag videos exceeding a ratio threshold. Send only those to a model to label the hook, format and topic. Cap spending. Keep labels as suggestions you can correct, and use your corrections to improve the label set over time. Trap: Labelling every video means paying for thousands of labels you never read, and the ones that matter are a small fraction. Detect first, label second, and cap the number labelled per run so a source glitch that flags everything cannot produce a large bill. Done when: Only videos above the threshold are labelled, verified by comparing the label count to the flagged count. A per-run label cap prevents a spike from producing a large bill. Labels are editable and corrections persist. Cost per run is recorded. Do not build yet: No review interface yet. ### Phase 5 · The page you actually read Build: Generate a static page nightly: outperformers ranked by ratio, with the creator, the baseline, the caption and the labels. Group by label so patterns are visible. Keep it short enough that you read all of it. Trap: A page listing every video with a ratio above one is a page nobody reads, because most days nothing interesting happened. Set the threshold so a typical day shows a handful, and let an empty day be empty rather than padding it. Done when: The nightly page shows outperformers with their ratio and baseline. A day with nothing above the threshold produces a page that says so rather than lowering the bar. Grouping by label shows which formats recur. The page renders without a server. Do not build yet: Retention and review come last. ### Phase 6 · Retention, review and the terms check Build: Retention that deletes old observations while keeping aggregates. A periodic review of the watchlist against its recorded reasons. A scheduled reminder to re-read the source's terms, because they change and your note has a date on it. Trap: A collection policy written once and never revisited becomes inaccurate silently, and the terms of these platforms change more often than most. Put a date on the note and a reminder in the calendar, so the claim in your README stays true. Done when: Retention removes old observations and keeps computed aggregates, verified by row counts. A review lists creators whose recorded reason no longer applies. The terms note carries a date and a reminder exists to re-check it. Removing a creator deletes their videos, metrics and labels together. Do not build yet: Nothing after this. ### Out of scope (and why) - A cross-creator corpus. The hosted product's value is comparing against videos you never chose to watch, and building that means collecting at a scale this build deliberately refuses. - Search across the platform. That needs the corpus, which needs the collection, which is the thing rule zero is about. - A curated taxonomy built from many accounts' data. Your labels are your own and will be less useful for the first few months. - Any claim that this is a compliant route to platform data at commercial scale. It is not, and the terms note exists so you know what you are choosing. ### README must contain - Which source is used, what its terms permit, and the date that was checked - That collection is aggregate counts for a chosen watchlist and deliberately excludes individual identities - That baselines use a median per creator, and why a mean would fail after one viral video - That only outperformers are labelled, with the per-run cap - The retention window and how to remove a creator entirely
ARCHITECTURE.md
# Architecture · Wholana ## Stack | Part | Choice | Why | | --- | --- | --- | | Runtime | Node 22 with SQLite through better-sqlite3 | a nightly job and a few thousand rows; anything larger is a scope problem rather than a database problem | | Collection | A single source behind one adapter, polled nightly | one place to change when the source changes, which it will | | Baseline | Rolling median of a creator's recent videos | median rather than mean, because one viral video would otherwise raise the bar for a year | | Labelling | A model applied to the outperformers only | labelling everything costs money for videos you will never look at | | Interface | A static page generated nightly | you read this once a day; a server is unnecessary | ## Modules Each module has one owner concern and a documented way to replace it. | Module | Owns | How to replace it | | --- | --- | --- | | Policy | What may be collected and from where. | This is the module that shapes the schema. Changing it means revisiting the schema, which is the point of deciding first. | | Source | Fetching from one provider. | One adapter, because sources in this area change and break. Everything else is source-agnostic. | | Baseline | Per-creator medians. | Pure computation over observations, testable against hand calculations with no network. | | Detect and label | Flagging and describing outperformers. | Any model. The rule that labelling happens only after detection is what keeps the cost bounded. | | Render | The daily page. | Presentation only, and deliberately static. | ## Configuration Every runtime setting is an environment variable documented in `.env.example`, validated at startup, with a safe local default wherever one exists. - `DB_PATH` · required · Creators, videos, observations and labels. - `SOURCE_API_KEY` · required, secret · Your data source's credential. Priced per run, so treat it as a spending instrument. - `BASELINE_WINDOW_DAYS` · required · How far back the rolling median looks. - `BASELINE_MIN_SAMPLE` · required · Minimum videos before a baseline is computed at all. - `OUTPERFORM_RATIO` · required · How far above the median a video must be to be flagged. Set it so a typical day shows a handful. - `LABEL_CAP_PER_RUN` · required · Maximum videos labelled per run, so a source glitch cannot produce a large bill. - `OBSERVATION_RETENTION_DAYS` · required · How long raw observations are kept. Aggregates survive beyond this. ## Production baseline - Security: least privilege, input validation at every boundary, secret redaction in logs, rate limits on abuse-prone paths, no invented security primitives. - Data: explicit schema and migrations, transactional writes where integrity matters, backup and restore procedures that have been exercised. - Integrations: adapters around third-party providers, idempotent webhook or job processing, bounded retries, timeouts. - Observability: structured logs with request or operation ids, an error-tracking hook, and health and readiness checks where a server exists. - Quality: unit tests for domain rules, integration tests at module boundaries, one end-to-end test of the critical path. ## Decision records For each dependency in the stack table, keep a short note: why it was chosen, its failure mode, and how it is replaced. Do not add infrastructure until a requirement in `PRODUCT.md` justifies it.
AGENTS.md
# Agent instructions · Wholana product build - Read `PRODUCT.md` and `ARCHITECTURE.md` before changing code. The stack is fixed: Node 22 with SQLite through better-sqlite3, A single source behind one adapter, polled nightly, Rolling median of a creator's recent videos, A model applied to the outperformers only, A static page generated nightly. - Implement milestone by milestone from `MILESTONES.md`; keep each change reviewable and leave the application runnable at every commit. - Treat authentication, payments, encryption, imports, webhooks and destructive actions as high-risk boundaries when present. - Never invent cryptography or silently weaken a requirement to make a check pass. - Put every external service behind an interface with a deterministic fake for tests. - Add migrations and rollback or recovery notes for every persistent data change. - Log useful operational context without credentials, tokens, passwords or personal data. - Update documentation and run every check before completing a milestone. ## Known traps - Building the collector first and deciding the policy afterwards produces a system you are not comfortable running, and by then the schema already holds data you did not want. Deciding first is cheaper and the schema then enforces it: if there is no column for follower identities, nothing can accidentally store them. - Updating a video's view count in place loses the growth curve, which is the interesting part: a video that reached its number in a day is a different signal from one that took three weeks. Append observations; never overwrite them. - A mean is dragged upward by one viral video and stays high for months, so nothing after it ever looks like an outperformer. Median is resistant to exactly that. - Including the video in its own baseline dampens the very signal you are measuring, and it matters most for creators with few videos, which is where you most want the signal. - Labelling every video means paying for thousands of labels you never read, and the ones that matter are a small fraction. Detect first, label second, and cap the number labelled per run so a source glitch that flags everything cannot produce a large bill. - A page listing every video with a ratio above one is a page nobody reads, because most days nothing interesting happened. Set the threshold so a typical day shows a handful, and let an empty day be empty rather than padding it. - A collection policy written once and never revisited becomes inaccurate silently, and the terms of these platforms change more often than most. Put a date on the note and a reminder in the calendar, so the claim in your README stays true.
MILESTONES.md
# Delivery milestones · Wholana Estimated effort: **weekend** for the indie phases; the production-only milestones add the trust and operability layer. ## M1 · Deciding what you may collect You have written down what you are collecting, from where, and under what terms. ### Steps 1. Read the terms and write the note Source, permitted use, date checked, and what you will not collect. One page, in the repository. Files: `docs/collection-policy.md` 2. Design the schema around the exclusions Aggregate counts, no identity columns for anyone but the creators you chose. Structural rather than remembered. Files: `src/schema.sql` 3. Record a reason per creator Why you are watching them. A watchlist without reasons grows into surveillance by accident. Files: `src/creators.js` ### Done when - [ ] The collection policy is written with a check date - [ ] The schema cannot hold identities beyond chosen creators - [ ] Every creator has a recorded reason - [ ] The exclusions are enforced by structure rather than by discipline ### Watch out - Building the collector first and deciding the policy afterwards produces a system you are not comfortable running, and by then the schema already holds data you did not want. Deciding first is cheaper and the schema then enforces it: if there is no column for follower identities, nothing can accidentally store them. ## M2 · Collecting nightly, politely View counts arrive once a night without hammering anything. ### Steps 1. Write the source adapter One interface. Everything source-specific lives behind it, because the source will change or break. Files: `src/source.js` 2. Append observations One row per video per run. Unique on video and date so a rerun is a no-op. Files: `src/collect.js` 3. Rate limit and back off Well below any ceiling, with exponential backoff. Politeness here is also what keeps access working. Files: `src/collect.js` ### Done when - [ ] A nightly run records observations for every enabled creator - [ ] A second run the same night creates no duplicates - [ ] A source failure backs off and is reported - [ ] Growth over several days is visible for one video ### Watch out - Updating a video's view count in place loses the growth curve, which is the interesting part: a video that reached its number in a day is a different signal from one that took three weeks. Append observations; never overwrite them. ## M3 · A baseline per creator A video is judged against its own creator's usual numbers, not against everyone's. ### Steps 1. Compute a rolling median Over a window in days, per creator. Recompute nightly rather than incrementally, because it is cheap at this size. Files: `src/baseline.js` 2. Require a minimum sample Below it, no baseline. A ratio computed from three videos is noise presented as a number. Files: `src/baseline.js` 3. Exclude the subject video Test with a hand-calculated example so the exclusion is proven rather than assumed. Files: `test/baseline.test.js` ### Done when - [ ] One viral video does not distort the creator's baseline - [ ] Below the minimum sample no baseline is produced - [ ] The judged video is excluded from its own baseline - [ ] Baselines match a hand calculation on a fixture ### Watch out - A mean is dragged upward by one viral video and stays high for months, so nothing after it ever looks like an outperformer. Median is resistant to exactly that. - Including the video in its own baseline dampens the very signal you are measuring, and it matters most for creators with few videos, which is where you most want the signal. ## M4 · Detecting and labelling what worked You see the videos that beat their creator's baseline, with a note on what they did differently. ### Steps 1. Flag by ratio Views over the creator's baseline, above a threshold you choose. Store the ratio, not just the flag. Files: `src/detect.js` 2. Label only the flagged One model call per outperformer, with a hard cap per run. Files: `src/label.js` 3. Make labels correctable Your corrections are the training signal for your own label set, and they are more accurate than the model's first pass. Files: `src/label.js` ### Done when - [ ] Labels are produced only for flagged videos - [ ] The per-run cap prevents a spike from spending freely - [ ] Label corrections persist - [ ] Cost per run is recorded ### Watch out - Labelling every video means paying for thousands of labels you never read, and the ones that matter are a small fraction. Detect first, label second, and cap the number labelled per run so a source glitch that flags everything cannot produce a large bill. ## M5 · The page you actually read A daily page showing what outperformed and why, in under a minute of reading. ### Steps 1. Generate a static page Nightly, written to disk. No server, no framework. Files: `src/render.js` 2. Rank and group By ratio, grouped by label so patterns are visible across creators. Files: `src/render.js` 3. Let quiet days be quiet An empty page is a true statement. Padding it teaches you to ignore the page. Files: `src/render.js` ### Done when - [ ] The page shows outperformers with ratio and baseline - [ ] A quiet day produces an honest empty page - [ ] Grouping by label reveals recurring formats - [ ] The page renders with no server ### Watch out - A page listing every video with a ratio above one is a page nobody reads, because most days nothing interesting happened. Set the threshold so a typical day shows a handful, and let an empty day be empty rather than padding it. ## M6 · Retention, review and the terms check (production only) The database does not grow into something you would not want to explain, and the terms note stays current. ### Steps 1. Add retention Delete raw observations past a window; keep baselines and outperformer records, which are small. Files: `src/retention.js` 2. Review the watchlist Against the recorded reasons. A creator you no longer have a reason to watch comes off the list. Files: `src/review.js` 3. Schedule a terms re-check A reminder tied to the date on the note. Terms change and your README makes a claim about them. Files: `docs/collection-policy.md` ### Done when - [ ] Retention removes raw observations and keeps aggregates - [ ] The review surfaces creators without a current reason - [ ] The terms note carries a date and a re-check reminder exists - [ ] Removing a creator deletes all their related rows ### Watch out - A collection policy written once and never revisited becomes inaccurate silently, and the terms of these platforms change more often than most. Put a date on the note and a reminder in the calendar, so the claim in your README stays true.
OPERATIONS.md
# Operations · Wholana ## Backup The database is small and holds your watchlist, observations and labels. Back it up, and remember it contains data about other people's accounts even in aggregate form. ## Restore Restore the database and resume the nightly job. Missed nights are gaps in the observation history rather than errors, and the baseline handles gaps by design. Do a restore drill before the first real user, and write the date here when it passes. ## Monitoring Run success rate, per-run cost, and the count of flagged videos. A sudden jump in flagged videos usually means the baseline computation broke rather than that everyone had a good day. ## Incident checklist If the source's terms change or access is withdrawn, stop the job before deciding what to do. If the database has grown beyond what your policy describes, retention and the watchlist review are the tools, and the policy note is what tells you it happened. 1. Contain the issue without destroying evidence or user data. 2. Record the timeline and affected scope. 3. Rotate exposed secrets and revoke compromised sessions or credentials. 4. Restore from a verified backup when needed. 5. Document the root cause, the remediation and the regression test. ## Release gate - [ ] The collection policy note is current and dated - [ ] The schema is confirmed unable to hold identities beyond chosen creators - [ ] Baseline calculations match hand-computed fixtures - [ ] The per-run label cap is confirmed to bound spending - [ ] Removing a creator is confirmed to delete all related rows ## Launch constraint Do not market omitted Wholana capabilities as implemented. The non-goals in `PRODUCT.md` remain user-visible limitations until they are deliberately delivered.
.env.example
# Copy to .env and fill in. Never commit .env; this file documents it. # Required. Creators, videos, observations and labels. DB_PATH=./data/watch.db # Required · secret. Your data source's credential. Priced per run, so treat it as a spending instrument. SOURCE_API_KEY=replace-with-key # Required. How far back the rolling median looks. BASELINE_WINDOW_DAYS=90 # Required. Minimum videos before a baseline is computed at all. BASELINE_MIN_SAMPLE=8 # Required. How far above the median a video must be to be flagged. Set it so a typical day shows a handful. OUTPERFORM_RATIO=2.0 # Required. Maximum videos labelled per run, so a source glitch cannot produce a large bill. LABEL_CAP_PER_RUN=15 # Required. How long raw observations are kept. Aggregates survive beyond this. OBSERVATION_RETENTION_DAYS=365
$ choose a build depth, inspect the files, then open the complete pack in your agent
The arithmetic is free, the data is not. A personal build only ever knows about the creators you thought to add, and you pay the scraper bill every month to keep even that fresh. The subscription is renting a corpus that was already collected and labeled, plus the discovery that only exists once videos from creators you have never heard of are sitting in the same index.
xthe cross-creator corpus
xsearch across videos you never chose to watch
xa curated craft taxonomy instead of labels you invented
xsemantic and hybrid search
xsubject classification
xcreator equity and track-record views
xsomeone else absorbing the scrape cost and keeping it running
share on X ↗"I just replaced Wholana ($5/mo) with one prompt"
Wholana pricing
| plan | monthly | annual (per mo) | what you get |
|---|---|---|---|
| personal | $5 | — | 1 creator account for TikTok research and performance labeling |
| plus | $20 | — | 1 creator account plus ChatGPT and Claude analysis features |
| team | $20/user | — | Agency/team access; minimum 2 seats |
free tierno free tier
billingmonthly only, no annual plan published; card-backed first month free
hidden costsTeam has a 2-seat minimum ($40/month). The free month converts to paid unless cancelled; no annual discount or overage schedule is published.
verified 2026-08-14 · source ↗
Vibecode Wholana
Kinda. The core of Wholana is buildable in a weekend with the prompt on this page, but there are real gaps: the cross-creator corpus, search across videos you never chose to watch. Read the honest list above before committing.
How much does Wholana cost?
Wholana costs about $5/month (Personal, checked 2026-07-31), which is $60 per year.
What do I lose by replacing Wholana?
Honestly: the cross-creator corpus; search across videos you never chose to watch; a curated craft taxonomy instead of labels you invented; semantic and hybrid search; subject classification; creator equity and track-record views; someone else absorbing the scrape cost and keeping it running. If any of those are load-bearing for you, keep paying.
Is there an open-source alternative to Wholana?
Yes: TikTokApi (Unofficial Python wrapper for TikTok's web endpoints; gets you raw metrics, not a corpus, and breaks when TikTok changes). Using prior art is also vibecoding; the prompt is for when you want it exactly your way.