Vibecode Rankwise
track this build4 phases, 12 steps, beginner friendly0%KINDA · weekend project
The measurement half is a weekend project. Fire a fixed question set at the ChatGPT, Claude, Gemini and Perplexity APIs on a schedule, count whether your domain gets named, store every run, and you have the dashboard. What takes Rankwise past that is the acting half: ranking topics by demand and citation gap, drafting sourced articles, and publishing them into WordPress, Shopify, Webflow or Wix without breaking anything. You can hand-write that loop for one site and one CMS. Doing it across four engines and four CMSes, every week, while the APIs move under you, is the part that stays a product.
the project pack5 files · written for this build, step by step
You are building a lean indie version of Rankwise. Create the following project files first, then implement the application by following them. Keep the files updated as decisions change. Do not collapse this into a single README or prompt. ===== README.md ===== # Rankwise · indie build A local weekly citation tracker for one site across four answer engines, whose hard problems are an immutable run history, normalising citations so two spellings of one URL are one domain, and an on-page audit that gives correct advice about crawlers that do different jobs. Estimated effort: **weekend**. Work `BUILD_PLAN.md` top to bottom · every phase ends in a check that has to pass before the next one starts. ## Stack | Part | Choice | Why | | --- | --- | --- | | Runtime | Node 22 and TypeScript, one CLI plus a server-rendered dashboard on localhost | a scheduled job and a page to read it, nothing more | | Storage | SQLite with append-only run tables | history is the product, so the schema refuses updates | | Engines | The four vendors' own APIs with their search or grounding tools enabled | scripting the chat interfaces breaks their terms and the data is no better | | Scheduling | A cron entry calling the CLI, with the run recorded whether it finishes or not | a half-finished week is data, a missing week is a hole | | Budget | A per-run spend ceiling checked before the fan-out starts | questions times engines times weeks is how a hobby becomes an invoice | ## Before you start Have every one of these ready. The plan assumes them from step one. - [ ] **Node.js 22 or newer** · free - Why: Runs the CLI and the local dashboard. - Get it: nodejs.org, or nvm install 22. - Verify: node --version prints v22 or higher - [ ] **API keys for the four answer engines** · pay per call, and it is the main running cost of this build - Why: Each engine is a separate account with its own billing, and the search or grounding tool has to be enabled per provider or you are measuring the model's memory rather than its citations. - Get it: Create an account and a key with each provider, then confirm the web search or grounding tool is available on your plan. - Verify: One question returns an answer with citations from each of the four - [ ] **A per-run spend ceiling** · free to decide - Why: Questions times engines times weeks compounds quietly, and the ceiling is the only thing standing between a weekly job and a surprise invoice. - Get it: Estimate one run at current published prices, decide what a week is worth, and write the figure into the configuration. - Verify: A run that would exceed the figure stops and says so - [ ] **Up to forty buyer questions** · free - Why: The question set is the instrument, so changing it mid-history is like moving the goalposts and then measuring the distance. - Get it: Write the questions a buyer actually types before they know your name, and commit to leaving them alone for a quarter. - Verify: The file loads and every question is answerable by a source you could plausibly be - [ ] **A scheduler that runs weekly** · free, or the cost of the machine - Why: The value is the series, and a series with gaps where the laptop was shut is a series you cannot read. - Get it: A cron entry on an always-on machine, or your platform's equivalent. - Verify: A scheduled run appears in the runs table without anyone pressing anything ## Quick start Follow Phase 1 in `BUILD_PLAN.md`; it creates the project. Then copy `.env.example` to `.env` and fill in the values it documents. ## Honest limits This build deliberately does not replace: - Article drafting and CMS publishing. - Any scripting of the consumer chat interfaces. - A single blended visibility score. - Competitor site-wide tracking. - the CMS publishing matrix, and the upkeep as WordPress, Shopify, Webflow and Wix each change their APIs - the editorial layer: topics ranked by demand, citation gap and winnability, then drafted and reviewed rather than dumped - exact technical diffs (JSON-LD, canonicals, robots rules, llms.txt) instead of generic advice you still have to translate - months of score history, without which a single week's citation rate tells you nothing - per-engine weekly scoring across seven on-page dimensions, which costs real API spend every run If one of those is essential to you, that is the reason to keep paying for Rankwise, and the README should say so rather than pretend. ===== BRIEF.md ===== # Build brief · Rankwise The one-shot brief this plan expands. `BUILD_PLAN.md` (or `MILESTONES.md`) is the same sequence broken into steps and checks; where the two disagree, the plan wins. Build me a citation tracker that asks the answer engines your buyers' questions every week and keeps every run to replace Rankwise. Build it in phases, in the order below. Do not write the whole thing in one pass. Finish a phase, run its "Done when" check, fix what fails, and only then start the next phase. ### Rule zero This tool produces a number that people will make decisions with, so it has to be honest about what the number is. What you measure is what four APIs answered, with their search or grounding tools on, at a moment in time. It is not what a person sees in a chat window. So every row records the model identifier, whether the search tool was enabled, the timestamp and the raw answer, and the dashboard says in plain words that this approximates rather than reproduces the consumer experience. Two consequences. Rows are immutable: a run is never overwritten, never backfilled, never recomputed, because the whole value is a comparison against your own history and an edited past makes the comparison meaningless. And a failed call is a visible empty cell, never a zero, since a zero says you were not cited and an error says you do not know. Use the documented APIs. Do not script the consumer chat interfaces, whatever the numbers there would look like. ### Stack (fixed, do not substitute) - Runtime: Node 22 and TypeScript, one CLI plus a server-rendered dashboard on localhost · a scheduled job and a page to read it, nothing more - Storage: SQLite with append-only run tables · history is the product, so the schema refuses updates - Engines: The four vendors' own APIs with their search or grounding tools enabled · scripting the chat interfaces breaks their terms and the data is no better - Scheduling: A cron entry calling the CLI, with the run recorded whether it finishes or not · a half-finished week is data, a missing week is a hole - Budget: A per-run spend ceiling checked before the fan-out starts · questions times engines times weeks is how a hobby becomes an invoice ### Data model (create this before Phase 1) - site(domain, brand_aliases_json, competitor_domains_json) · one site, because the comparison only means anything for one - questions(id, text, added_at, retired_at) · retired rather than deleted, so old runs still make sense - runs(id, started_at, finished_at, questions_count, engines_json, estimated_cost, actual_cost, status) - answers(run_id, question_id, provider, model_id, search_enabled, raw_answer, latency_ms, error, created_at) · one immutable row per cell, errors included - citations(answer_id, ordinal, raw_url, host, canonical_path, is_mine, is_competitor) · normalised once at write time - audits(url, checked_at, findings_json) · the mechanical on-page checks, kept per run so a fix is visible as a change - No table in this schema has an update path. A correction is a new run, because the point of the history is that nobody edited it. ### Phase 1 · A question set and a run that cannot be rewritten Build: Load the site definition and up to forty questions from files. Create a run, write one row per question and provider with the raw answer, model identifier, whether the search tool was on, and any error. Make the tables append-only and prove it. Trap: Storing one current score per question and updating it each week feels tidier and destroys the only thing this tool is for, because a citation rate in week one means nothing and a trend over six months means everything. Append a row per run and never update one. Done when: Running twice produces two runs and zero updated rows, asserted by a test that attempts an update and expects it to fail. Every answer row records model identifier, search-enabled flag and timestamp. A provider error is stored as a row with an error and a null answer, not as an absent row. Retiring a question leaves every historical answer for it readable. Do not build yet: No scoring, no dashboard, no audit. ### Phase 2 · Fanning out without burning the budget Build: Cap concurrency per provider, retry rate limits and server errors with backoff, and stop the run when the estimated spend passes the ceiling. Estimate cost before the run from the current published prices, record the actual cost after, and show both. Trap: Retrying a rate limit immediately in a tight loop converts one throttled provider into a run that takes an hour and costs twice the estimate, and retrying a request that already produced an answer pays for the same cell twice. Back off, cap attempts, and record a cell as failed rather than retrying it forever. Done when: A run stops at the spend ceiling with the reason recorded and partial results kept. Rate limited calls back off rather than retrying immediately, tested against a fake provider that always throttles. The estimated and actual cost of each run are both stored and shown. Forty questions across four providers finish or fail within a stated wall clock time, measured. Do not build yet: No normalisation, scoring or audit yet. ### Phase 3 · Two spellings of one URL are one citation Build: Parse cited URLs out of each answer, strip tracking parameters and fragments, lower-case the host, drop a leading www, resolve one level of redirect where the provider gives you one, and store the normalised form alongside the raw one. Match your domain and your brand aliases. Trap: Counting citations by exact URL string splits one page into five entries because of a trailing slash, a tracking parameter, an http scheme and a www prefix, and your own site then appears to be cited a fifth as often as it is. Normalise at write time, keep the raw URL beside it, and test the five variants collapse to one. Done when: Five spellings of the same page collapse to one normalised citation, tested explicitly with a trailing slash, a tracking parameter, a scheme difference, a www prefix and a fragment. The raw URL is retained beside the normalised one for every citation. A brand alias mentioned without a link is counted separately from a linked citation and labelled as such. The top cited domains table per engine sums to the citation count. Do not build yet: No dashboard or audit yet. ### Phase 4 · An audit that gives correct crawler advice Build: Fetch one page and report only what is checkable: missing or invalid JSON-LD, a missing or conflicting canonical, heading structure, and which documented agents your robots rules allow or block. Print the JSON-LD block you would add. State each finding as a fact, not a promise that fixing it will win a citation. Trap: Reporting 'GPTBot is blocked, so you are invisible in ChatGPT' is wrong advice, and it is the single most common mistake in this category. OpenAI documents four separate agents with different jobs: GPTBot is the model training crawler, OAI-SearchBot is what surfaces sites in ChatGPT's search features, OAI-AdsBot validates pages submitted as ads, and ChatGPT-User serves user-initiated actions where, in OpenAI's own words, robots.txt rules may not apply. Report each token separately, say what blocking it actually costs, and re-read the vendor pages before you ship, because these tokens are new and they move. Done when: The audit lists each documented agent by token with its stated purpose and whether your rules allow it, and blocking the training crawler is never reported as removing you from search. JSON-LD is parsed and validated rather than merely detected, with the error shown. A conflicting canonical is reported with both values. Every finding is phrased as an observation and none of them promises a ranking outcome. Do not build yet: No publishing, drafting or CMS integration, ever. ### Phase 5 · History you can read and take with you Build: Render citation rate per engine over time, the questions where a competitor is cited and you are not, and a week over week difference that marks changed model identifiers. Export runs, answers and citations to CSV. Keep raw answers under a stated retention and say so. Trap: Showing a week over week movement without showing that the provider changed model version in between invites a team to celebrate or panic about a change they did not cause. Mark every diff whose model identifier differs from the previous run, and say plainly that the two weeks are not comparable. Done when: The dashboard shows citation rate per engine over every stored run, with a run count so a single week cannot be mistaken for a trend. A diff where the model identifier changed is flagged as not comparable. Export writes runs, answers and citations to CSV and the row counts match the database. The interface states, where the score is shown, that this measures API answers rather than what a user sees. Do not build yet: Nothing after this. ### Out of scope (and why) - Drafting articles and publishing them into a CMS, which is the half that makes Rankwise a product and a whole project on its own. - Scripting the consumer chat interfaces, which breaks the providers' terms and produces data no more trustworthy than the API. - A single overall visibility score, because averaging four engines with different answer shapes produces a number with no defined meaning. - Tracking competitors' whole sites, which multiplies the API spend and tells you less than one question they win and you lose. ### README must contain - That the tool measures API answers with search tools enabled, which approximates rather than reproduces what a person sees - The per-run cost estimate for your question count, and where the current prices are published - A cron line for the weekly run, and what a partially failed run looks like in the data - That run rows are append-only and why an edited history makes the whole tool pointless - Which crawler tokens the audit checks, what each one governs, and that these change often enough to re-read the vendor pages ===== AGENTS.md ===== # Agent instructions · Rankwise indie build - Read `README.md` and `BUILD_PLAN.md` before writing code. The stack is fixed: Node 22 and TypeScript, one CLI plus a server-rendered dashboard on localhost, SQLite with append-only run tables, The four vendors' own APIs with their search or grounding tools enabled, A cron entry calling the CLI, with the run recorded whether it finishes or not, A per-run spend ceiling checked before the fan-out starts. Do not substitute. - Work one phase at a time, in order. Do not start a phase until every "Done when" item of the previous one passes. - Prefer the fewest moving parts that satisfy the step. No frameworks, services or dependencies the plan does not name. - Secrets live in `.env`, never in source or logs. Keep `.env.example` current when a variable is introduced. - Do not invent cryptography, security guarantees, APIs or compliance claims. - Add a focused test for every destructive, security-sensitive or data-loss path the plan names. - Run the project checks before declaring a phase complete, and record any deliberate shortcut in the README under "Tradeoffs". ## Known traps - Storing one current score per question and updating it each week feels tidier and destroys the only thing this tool is for, because a citation rate in week one means nothing and a trend over six months means everything. Append a row per run and never update one. - Retrying a rate limit immediately in a tight loop converts one throttled provider into a run that takes an hour and costs twice the estimate, and retrying a request that already produced an answer pays for the same cell twice. Back off, cap attempts, and record a cell as failed rather than retrying it forever. - Counting citations by exact URL string splits one page into five entries because of a trailing slash, a tracking parameter, an http scheme and a www prefix, and your own site then appears to be cited a fifth as often as it is. Normalise at write time, keep the raw URL beside it, and test the five variants collapse to one. - Reporting 'GPTBot is blocked, so you are invisible in ChatGPT' is wrong advice, and it is the single most common mistake in this category. OpenAI documents four separate agents with different jobs: GPTBot is the model training crawler, OAI-SearchBot is what surfaces sites in ChatGPT's search features, OAI-AdsBot validates pages submitted as ads, and ChatGPT-User serves user-initiated actions where, in OpenAI's own words, robots.txt rules may not apply. Report each token separately, say what blocking it actually costs, and re-read the vendor pages before you ship, because these tokens are new and they move. ===== BUILD_PLAN.md ===== # Build plan · Rankwise A local weekly citation tracker for one site across four answer engines, whose hard problems are an immutable run history, normalising citations so two spellings of one URL are one domain, and an on-page audit that gives correct advice about crawlers that do different jobs. Phases are in dependency order. Each ends in a "Done when" list; treat an unticked item as a blocker, not a note. ## Phase 1 · A question set and a run that cannot be rewritten One run writes one immutable row per question and engine, including the failures. ### Steps 1. Make the tables append-only A trigger or a check that turns an update into a failed test rather than a lost week. Files: `src/db/schema.ts`, `test/append-only.test.ts` 2. Record the model identifier and tool state per cell Providers change defaults underneath you and the comparison has to survive that. Files: `src/providers/types.ts` 3. Store errors as rows An empty cell is information; a missing row is an unanswered question about your own data. Files: `src/run.ts` ### Done when - [ ] Two runs produce two rows, never an update - [ ] Model id and tool state are recorded per cell - [ ] Errors are stored as rows with the error text - [ ] Retired questions keep their history readable ### Watch out - Storing one current score per question and updating it each week feels tidier and destroys the only thing this tool is for, because a citation rate in week one means nothing and a trend over six months means everything. Append a row per run and never update one. ## Phase 2 · Fanning out without burning the budget Four providers are called concurrently under a spend ceiling with visible failures. ### Steps 1. Cap concurrency per provider Each has its own limits and one of them is always the slow one. Files: `src/pool.ts` 2. Check the ceiling before spending The estimate comes from the current published prices, which you re-check, not from a constant. Files: `src/budget.ts`, `docs/costs.md` 3. Back off and then give up A failed cell with an error beats a run that never ends. Files: `src/retry.ts`, `test/retry.test.ts` ### Done when - [ ] A run stops at the ceiling and keeps partial results - [ ] Throttled calls back off rather than hammering - [ ] Estimated and actual cost are both recorded - [ ] A full run completes within a measured time ### Watch out - Retrying a rate limit immediately in a tight loop converts one throttled provider into a run that takes an hour and costs twice the estimate, and retrying a request that already produced an answer pays for the same cell twice. Back off, cap attempts, and record a cell as failed rather than retrying it forever. ## Phase 3 · Two spellings of one URL are one citation Citations normalise to a host and a canonical path so counting means something. ### Steps 1. Normalise once, at write time Normalising at read time means every report defines its own truth. Files: `src/normalise.ts`, `test/normalise.test.ts` 2. Keep the raw URL When a count looks wrong the raw string is the only way to find out why. Files: `src/db/schema.ts` 3. Separate a mention from a citation Being named and being linked are different outcomes and mixing them flatters you. Files: `src/score.ts` ### Done when - [ ] Five URL variants collapse to one citation - [ ] Raw URLs are retained beside normalised ones - [ ] Mentions and citations are counted separately - [ ] Domain tables reconcile with citation counts ### Watch out - Counting citations by exact URL string splits one page into five entries because of a trailing slash, a tracking parameter, an http scheme and a www prefix, and your own site then appears to be cited a fifth as often as it is. Normalise at write time, keep the raw URL beside it, and test the five variants collapse to one. ## Phase 4 · An audit that gives correct crawler advice The on-page audit reports mechanical facts and names the right agent for the right job. ### Steps 1. Name each agent and what blocking it costs Training, search, ads and user-initiated fetching are four different decisions. Files: `src/audit/crawlers.ts`, `docs/crawlers.md` 2. Validate the JSON-LD, do not just find it A malformed block is worse than none and looks identical from the outside. Files: `src/audit/jsonld.ts` 3. Report observations, never promises Nobody can promise a citation, and a tool that implies it is lying politely. Files: `src/audit/report.ts` ### Done when - [ ] Each crawler token is reported separately with its purpose - [ ] JSON-LD is parsed and errors are shown - [ ] Conflicting canonicals show both values - [ ] No finding promises a ranking outcome ### Watch out - Reporting 'GPTBot is blocked, so you are invisible in ChatGPT' is wrong advice, and it is the single most common mistake in this category. OpenAI documents four separate agents with different jobs: GPTBot is the model training crawler, OAI-SearchBot is what surfaces sites in ChatGPT's search features, OAI-AdsBot validates pages submitted as ads, and ChatGPT-User serves user-initiated actions where, in OpenAI's own words, robots.txt rules may not apply. Report each token separately, say what blocking it actually costs, and re-read the vendor pages before you ship, because these tokens are new and they move. ## Not in this build - Article drafting and CMS publishing. - Any scripting of the consumer chat interfaces. - A single blended visibility score. - Competitor site-wide tracking. ## After v1, if you want it - A fifth engine added as an adapter, once the four have run for a quarter without a gap - Per-question cost reporting, so an expensive question with no movement can be retired honestly - An annotation layer marking the dates you published something, since a trend with no events is a line ===== .env.example ===== # Copy to .env and fill in. Never commit .env; this file documents it. # Required. Runs, answers and citations. Append-only and the whole asset. DATABASE_PATH=./data/rankwise.db # Required · secret. With the web search tool enabled on the account. OPENAI_API_KEY=sk-...redacted # Required · secret. With the web search tool enabled on the account. ANTHROPIC_API_KEY=sk-ant-...redacted # Required · secret. With grounding enabled on the account. GEMINI_API_KEY=AIza...redacted # Required · secret. Search is the default here, but record the flag anyway. PERPLEXITY_API_KEY=pplx-...redacted # Required. Checked before the fan-out. A run that would exceed it stops and records why. RUN_COST_CEILING_USD=12.00 # Required. Per provider, not in total. One of them is always the slow one. PROVIDER_CONCURRENCY=3 # Required. Raw answers are large. Scores and citations are kept forever regardless. RAW_ANSWER_RETENTION_DAYS=400
You are building a lean indie version of Rankwise. Create the following project files first, then implement the application by following them. Keep the files updated as decisions change. Do not collapse this into a single README or prompt. ===== README.md ===== # Rankwise · indie build A local weekly citation tracker for one site across four answer engines, whose hard problems are an immutable run history, normalising citations so two spellings of one URL are one domain, and an on-page audit that gives correct advice about crawlers that do different jobs. Estimated effort: **weekend**. Work `BUILD_PLAN.md` top to bottom · every phase ends in a check that has to pass before the next one starts. ## Stack | Part | Choice | Why | | --- | --- | --- | | Runtime | Node 22 and TypeScript, one CLI plus a server-rendered dashboard on localhost | a scheduled job and a page to read it, nothing more | | Storage | SQLite with append-only run tables | history is the product, so the schema refuses updates | | Engines | The four vendors' own APIs with their search or grounding tools enabled | scripting the chat interfaces breaks their terms and the data is no better | | Scheduling | A cron entry calling the CLI, with the run recorded whether it finishes or not | a half-finished week is data, a missing week is a hole | | Budget | A per-run spend ceiling checked before the fan-out starts | questions times engines times weeks is how a hobby becomes an invoice | ## Before you start Have every one of these ready. The plan assumes them from step one. - [ ] **Node.js 22 or newer** · free - Why: Runs the CLI and the local dashboard. - Get it: nodejs.org, or nvm install 22. - Verify: node --version prints v22 or higher - [ ] **API keys for the four answer engines** · pay per call, and it is the main running cost of this build - Why: Each engine is a separate account with its own billing, and the search or grounding tool has to be enabled per provider or you are measuring the model's memory rather than its citations. - Get it: Create an account and a key with each provider, then confirm the web search or grounding tool is available on your plan. - Verify: One question returns an answer with citations from each of the four - [ ] **A per-run spend ceiling** · free to decide - Why: Questions times engines times weeks compounds quietly, and the ceiling is the only thing standing between a weekly job and a surprise invoice. - Get it: Estimate one run at current published prices, decide what a week is worth, and write the figure into the configuration. - Verify: A run that would exceed the figure stops and says so - [ ] **Up to forty buyer questions** · free - Why: The question set is the instrument, so changing it mid-history is like moving the goalposts and then measuring the distance. - Get it: Write the questions a buyer actually types before they know your name, and commit to leaving them alone for a quarter. - Verify: The file loads and every question is answerable by a source you could plausibly be - [ ] **A scheduler that runs weekly** · free, or the cost of the machine - Why: The value is the series, and a series with gaps where the laptop was shut is a series you cannot read. - Get it: A cron entry on an always-on machine, or your platform's equivalent. - Verify: A scheduled run appears in the runs table without anyone pressing anything ## Quick start Follow Phase 1 in `BUILD_PLAN.md`; it creates the project. Then copy `.env.example` to `.env` and fill in the values it documents. ## Honest limits This build deliberately does not replace: - Article drafting and CMS publishing. - Any scripting of the consumer chat interfaces. - A single blended visibility score. - Competitor site-wide tracking. - the CMS publishing matrix, and the upkeep as WordPress, Shopify, Webflow and Wix each change their APIs - the editorial layer: topics ranked by demand, citation gap and winnability, then drafted and reviewed rather than dumped - exact technical diffs (JSON-LD, canonicals, robots rules, llms.txt) instead of generic advice you still have to translate - months of score history, without which a single week's citation rate tells you nothing - per-engine weekly scoring across seven on-page dimensions, which costs real API spend every run If one of those is essential to you, that is the reason to keep paying for Rankwise, and the README should say so rather than pretend. ===== BRIEF.md ===== # Build brief · Rankwise The one-shot brief this plan expands. `BUILD_PLAN.md` (or `MILESTONES.md`) is the same sequence broken into steps and checks; where the two disagree, the plan wins. Build me a citation tracker that asks the answer engines your buyers' questions every week and keeps every run to replace Rankwise. Build it in phases, in the order below. Do not write the whole thing in one pass. Finish a phase, run its "Done when" check, fix what fails, and only then start the next phase. ### Rule zero This tool produces a number that people will make decisions with, so it has to be honest about what the number is. What you measure is what four APIs answered, with their search or grounding tools on, at a moment in time. It is not what a person sees in a chat window. So every row records the model identifier, whether the search tool was enabled, the timestamp and the raw answer, and the dashboard says in plain words that this approximates rather than reproduces the consumer experience. Two consequences. Rows are immutable: a run is never overwritten, never backfilled, never recomputed, because the whole value is a comparison against your own history and an edited past makes the comparison meaningless. And a failed call is a visible empty cell, never a zero, since a zero says you were not cited and an error says you do not know. Use the documented APIs. Do not script the consumer chat interfaces, whatever the numbers there would look like. ### Stack (fixed, do not substitute) - Runtime: Node 22 and TypeScript, one CLI plus a server-rendered dashboard on localhost · a scheduled job and a page to read it, nothing more - Storage: SQLite with append-only run tables · history is the product, so the schema refuses updates - Engines: The four vendors' own APIs with their search or grounding tools enabled · scripting the chat interfaces breaks their terms and the data is no better - Scheduling: A cron entry calling the CLI, with the run recorded whether it finishes or not · a half-finished week is data, a missing week is a hole - Budget: A per-run spend ceiling checked before the fan-out starts · questions times engines times weeks is how a hobby becomes an invoice ### Data model (create this before Phase 1) - site(domain, brand_aliases_json, competitor_domains_json) · one site, because the comparison only means anything for one - questions(id, text, added_at, retired_at) · retired rather than deleted, so old runs still make sense - runs(id, started_at, finished_at, questions_count, engines_json, estimated_cost, actual_cost, status) - answers(run_id, question_id, provider, model_id, search_enabled, raw_answer, latency_ms, error, created_at) · one immutable row per cell, errors included - citations(answer_id, ordinal, raw_url, host, canonical_path, is_mine, is_competitor) · normalised once at write time - audits(url, checked_at, findings_json) · the mechanical on-page checks, kept per run so a fix is visible as a change - No table in this schema has an update path. A correction is a new run, because the point of the history is that nobody edited it. ### Phase 1 · A question set and a run that cannot be rewritten Build: Load the site definition and up to forty questions from files. Create a run, write one row per question and provider with the raw answer, model identifier, whether the search tool was on, and any error. Make the tables append-only and prove it. Trap: Storing one current score per question and updating it each week feels tidier and destroys the only thing this tool is for, because a citation rate in week one means nothing and a trend over six months means everything. Append a row per run and never update one. Done when: Running twice produces two runs and zero updated rows, asserted by a test that attempts an update and expects it to fail. Every answer row records model identifier, search-enabled flag and timestamp. A provider error is stored as a row with an error and a null answer, not as an absent row. Retiring a question leaves every historical answer for it readable. Do not build yet: No scoring, no dashboard, no audit. ### Phase 2 · Fanning out without burning the budget Build: Cap concurrency per provider, retry rate limits and server errors with backoff, and stop the run when the estimated spend passes the ceiling. Estimate cost before the run from the current published prices, record the actual cost after, and show both. Trap: Retrying a rate limit immediately in a tight loop converts one throttled provider into a run that takes an hour and costs twice the estimate, and retrying a request that already produced an answer pays for the same cell twice. Back off, cap attempts, and record a cell as failed rather than retrying it forever. Done when: A run stops at the spend ceiling with the reason recorded and partial results kept. Rate limited calls back off rather than retrying immediately, tested against a fake provider that always throttles. The estimated and actual cost of each run are both stored and shown. Forty questions across four providers finish or fail within a stated wall clock time, measured. Do not build yet: No normalisation, scoring or audit yet. ### Phase 3 · Two spellings of one URL are one citation Build: Parse cited URLs out of each answer, strip tracking parameters and fragments, lower-case the host, drop a leading www, resolve one level of redirect where the provider gives you one, and store the normalised form alongside the raw one. Match your domain and your brand aliases. Trap: Counting citations by exact URL string splits one page into five entries because of a trailing slash, a tracking parameter, an http scheme and a www prefix, and your own site then appears to be cited a fifth as often as it is. Normalise at write time, keep the raw URL beside it, and test the five variants collapse to one. Done when: Five spellings of the same page collapse to one normalised citation, tested explicitly with a trailing slash, a tracking parameter, a scheme difference, a www prefix and a fragment. The raw URL is retained beside the normalised one for every citation. A brand alias mentioned without a link is counted separately from a linked citation and labelled as such. The top cited domains table per engine sums to the citation count. Do not build yet: No dashboard or audit yet. ### Phase 4 · An audit that gives correct crawler advice Build: Fetch one page and report only what is checkable: missing or invalid JSON-LD, a missing or conflicting canonical, heading structure, and which documented agents your robots rules allow or block. Print the JSON-LD block you would add. State each finding as a fact, not a promise that fixing it will win a citation. Trap: Reporting 'GPTBot is blocked, so you are invisible in ChatGPT' is wrong advice, and it is the single most common mistake in this category. OpenAI documents four separate agents with different jobs: GPTBot is the model training crawler, OAI-SearchBot is what surfaces sites in ChatGPT's search features, OAI-AdsBot validates pages submitted as ads, and ChatGPT-User serves user-initiated actions where, in OpenAI's own words, robots.txt rules may not apply. Report each token separately, say what blocking it actually costs, and re-read the vendor pages before you ship, because these tokens are new and they move. Done when: The audit lists each documented agent by token with its stated purpose and whether your rules allow it, and blocking the training crawler is never reported as removing you from search. JSON-LD is parsed and validated rather than merely detected, with the error shown. A conflicting canonical is reported with both values. Every finding is phrased as an observation and none of them promises a ranking outcome. Do not build yet: No publishing, drafting or CMS integration, ever. ### Phase 5 · History you can read and take with you Build: Render citation rate per engine over time, the questions where a competitor is cited and you are not, and a week over week difference that marks changed model identifiers. Export runs, answers and citations to CSV. Keep raw answers under a stated retention and say so. Trap: Showing a week over week movement without showing that the provider changed model version in between invites a team to celebrate or panic about a change they did not cause. Mark every diff whose model identifier differs from the previous run, and say plainly that the two weeks are not comparable. Done when: The dashboard shows citation rate per engine over every stored run, with a run count so a single week cannot be mistaken for a trend. A diff where the model identifier changed is flagged as not comparable. Export writes runs, answers and citations to CSV and the row counts match the database. The interface states, where the score is shown, that this measures API answers rather than what a user sees. Do not build yet: Nothing after this. ### Out of scope (and why) - Drafting articles and publishing them into a CMS, which is the half that makes Rankwise a product and a whole project on its own. - Scripting the consumer chat interfaces, which breaks the providers' terms and produces data no more trustworthy than the API. - A single overall visibility score, because averaging four engines with different answer shapes produces a number with no defined meaning. - Tracking competitors' whole sites, which multiplies the API spend and tells you less than one question they win and you lose. ### README must contain - That the tool measures API answers with search tools enabled, which approximates rather than reproduces what a person sees - The per-run cost estimate for your question count, and where the current prices are published - A cron line for the weekly run, and what a partially failed run looks like in the data - That run rows are append-only and why an edited history makes the whole tool pointless - Which crawler tokens the audit checks, what each one governs, and that these change often enough to re-read the vendor pages ===== AGENTS.md ===== # Agent instructions · Rankwise indie build - Read `README.md` and `BUILD_PLAN.md` before writing code. The stack is fixed: Node 22 and TypeScript, one CLI plus a server-rendered dashboard on localhost, SQLite with append-only run tables, The four vendors' own APIs with their search or grounding tools enabled, A cron entry calling the CLI, with the run recorded whether it finishes or not, A per-run spend ceiling checked before the fan-out starts. Do not substitute. - Work one phase at a time, in order. Do not start a phase until every "Done when" item of the previous one passes. - Prefer the fewest moving parts that satisfy the step. No frameworks, services or dependencies the plan does not name. - Secrets live in `.env`, never in source or logs. Keep `.env.example` current when a variable is introduced. - Do not invent cryptography, security guarantees, APIs or compliance claims. - Add a focused test for every destructive, security-sensitive or data-loss path the plan names. - Run the project checks before declaring a phase complete, and record any deliberate shortcut in the README under "Tradeoffs". ## Known traps - Storing one current score per question and updating it each week feels tidier and destroys the only thing this tool is for, because a citation rate in week one means nothing and a trend over six months means everything. Append a row per run and never update one. - Retrying a rate limit immediately in a tight loop converts one throttled provider into a run that takes an hour and costs twice the estimate, and retrying a request that already produced an answer pays for the same cell twice. Back off, cap attempts, and record a cell as failed rather than retrying it forever. - Counting citations by exact URL string splits one page into five entries because of a trailing slash, a tracking parameter, an http scheme and a www prefix, and your own site then appears to be cited a fifth as often as it is. Normalise at write time, keep the raw URL beside it, and test the five variants collapse to one. - Reporting 'GPTBot is blocked, so you are invisible in ChatGPT' is wrong advice, and it is the single most common mistake in this category. OpenAI documents four separate agents with different jobs: GPTBot is the model training crawler, OAI-SearchBot is what surfaces sites in ChatGPT's search features, OAI-AdsBot validates pages submitted as ads, and ChatGPT-User serves user-initiated actions where, in OpenAI's own words, robots.txt rules may not apply. Report each token separately, say what blocking it actually costs, and re-read the vendor pages before you ship, because these tokens are new and they move. ===== BUILD_PLAN.md ===== # Build plan · Rankwise A local weekly citation tracker for one site across four answer engines, whose hard problems are an immutable run history, normalising citations so two spellings of one URL are one domain, and an on-page audit that gives correct advice about crawlers that do different jobs. Phases are in dependency order. Each ends in a "Done when" list; treat an unticked item as a blocker, not a note. ## Phase 1 · A question set and a run that cannot be rewritten One run writes one immutable row per question and engine, including the failures. ### Steps 1. Make the tables append-only A trigger or a check that turns an update into a failed test rather than a lost week. Files: `src/db/schema.ts`, `test/append-only.test.ts` 2. Record the model identifier and tool state per cell Providers change defaults underneath you and the comparison has to survive that. Files: `src/providers/types.ts` 3. Store errors as rows An empty cell is information; a missing row is an unanswered question about your own data. Files: `src/run.ts` ### Done when - [ ] Two runs produce two rows, never an update - [ ] Model id and tool state are recorded per cell - [ ] Errors are stored as rows with the error text - [ ] Retired questions keep their history readable ### Watch out - Storing one current score per question and updating it each week feels tidier and destroys the only thing this tool is for, because a citation rate in week one means nothing and a trend over six months means everything. Append a row per run and never update one. ## Phase 2 · Fanning out without burning the budget Four providers are called concurrently under a spend ceiling with visible failures. ### Steps 1. Cap concurrency per provider Each has its own limits and one of them is always the slow one. Files: `src/pool.ts` 2. Check the ceiling before spending The estimate comes from the current published prices, which you re-check, not from a constant. Files: `src/budget.ts`, `docs/costs.md` 3. Back off and then give up A failed cell with an error beats a run that never ends. Files: `src/retry.ts`, `test/retry.test.ts` ### Done when - [ ] A run stops at the ceiling and keeps partial results - [ ] Throttled calls back off rather than hammering - [ ] Estimated and actual cost are both recorded - [ ] A full run completes within a measured time ### Watch out - Retrying a rate limit immediately in a tight loop converts one throttled provider into a run that takes an hour and costs twice the estimate, and retrying a request that already produced an answer pays for the same cell twice. Back off, cap attempts, and record a cell as failed rather than retrying it forever. ## Phase 3 · Two spellings of one URL are one citation Citations normalise to a host and a canonical path so counting means something. ### Steps 1. Normalise once, at write time Normalising at read time means every report defines its own truth. Files: `src/normalise.ts`, `test/normalise.test.ts` 2. Keep the raw URL When a count looks wrong the raw string is the only way to find out why. Files: `src/db/schema.ts` 3. Separate a mention from a citation Being named and being linked are different outcomes and mixing them flatters you. Files: `src/score.ts` ### Done when - [ ] Five URL variants collapse to one citation - [ ] Raw URLs are retained beside normalised ones - [ ] Mentions and citations are counted separately - [ ] Domain tables reconcile with citation counts ### Watch out - Counting citations by exact URL string splits one page into five entries because of a trailing slash, a tracking parameter, an http scheme and a www prefix, and your own site then appears to be cited a fifth as often as it is. Normalise at write time, keep the raw URL beside it, and test the five variants collapse to one. ## Phase 4 · An audit that gives correct crawler advice The on-page audit reports mechanical facts and names the right agent for the right job. ### Steps 1. Name each agent and what blocking it costs Training, search, ads and user-initiated fetching are four different decisions. Files: `src/audit/crawlers.ts`, `docs/crawlers.md` 2. Validate the JSON-LD, do not just find it A malformed block is worse than none and looks identical from the outside. Files: `src/audit/jsonld.ts` 3. Report observations, never promises Nobody can promise a citation, and a tool that implies it is lying politely. Files: `src/audit/report.ts` ### Done when - [ ] Each crawler token is reported separately with its purpose - [ ] JSON-LD is parsed and errors are shown - [ ] Conflicting canonicals show both values - [ ] No finding promises a ranking outcome ### Watch out - Reporting 'GPTBot is blocked, so you are invisible in ChatGPT' is wrong advice, and it is the single most common mistake in this category. OpenAI documents four separate agents with different jobs: GPTBot is the model training crawler, OAI-SearchBot is what surfaces sites in ChatGPT's search features, OAI-AdsBot validates pages submitted as ads, and ChatGPT-User serves user-initiated actions where, in OpenAI's own words, robots.txt rules may not apply. Report each token separately, say what blocking it actually costs, and re-read the vendor pages before you ship, because these tokens are new and they move. ## Not in this build - Article drafting and CMS publishing. - Any scripting of the consumer chat interfaces. - A single blended visibility score. - Competitor site-wide tracking. ## After v1, if you want it - A fifth engine added as an adapter, once the four have run for a quarter without a gap - Per-question cost reporting, so an expensive question with no movement can be retired honestly - An annotation layer marking the dates you published something, since a trend with no events is a line ===== .env.example ===== # Copy to .env and fill in. Never commit .env; this file documents it. # Required. Runs, answers and citations. Append-only and the whole asset. DATABASE_PATH=./data/rankwise.db # Required · secret. With the web search tool enabled on the account. OPENAI_API_KEY=sk-...redacted # Required · secret. With the web search tool enabled on the account. ANTHROPIC_API_KEY=sk-ant-...redacted # Required · secret. With grounding enabled on the account. GEMINI_API_KEY=AIza...redacted # Required · secret. Search is the default here, but record the flag anyway. PERPLEXITY_API_KEY=pplx-...redacted # Required. Checked before the fan-out. A run that would exceed it stops and records why. RUN_COST_CEILING_USD=12.00 # Required. Per provider, not in total. One of them is always the slow one. PROVIDER_CONCURRENCY=3 # Required. Raw answers are large. Scores and citations are kept forever regardless. RAW_ANSWER_RETENTION_DAYS=400
You are building a production product version of Rankwise. Create the following project files first, then implement the application by following them. Keep the files updated as decisions change. Do not collapse this into a single README or prompt. ===== PRODUCT.md ===== # Rankwise · product brief ## Problem The measurement half is a weekend project. Fire a fixed question set at the ChatGPT, Claude, Gemini and Perplexity APIs on a schedule, count whether your domain gets named, store every run, and you have the dashboard. What takes Rankwise past that is the acting half: ranking topics by demand and citation gap, drafting sourced articles, and publishing them into WordPress, Shopify, Webflow or Wix without breaking anything. You can hand-write that loop for one site and one CMS. Doing it across four engines and four CMSes, every week, while the APIs move under you, is the part that stays a product. ## Product outcome A weekly job that builds a citation history nobody can edit, tells you which questions a competitor owns, and gives crawler advice that is correct about which agent does which job. ## Target user A builder who needs a maintainable product foundation, not a one-off demo. ## Required capabilities - OpenAI, Anthropic, Gemini and Perplexity API keys with web search or grounding enabled - a scheduler for the weekly run - durable per-run storage so history survives - an API budget that scales with questions x engines x weeks - CMS credentials if you want the publishing half too ## Explicit non-goals for v1 - Article drafting and CMS publishing. - Any scripting of the consumer chat interfaces. - A single blended visibility score. - Competitor site-wide tracking. - the CMS publishing matrix, and the upkeep as WordPress, Shopify, Webflow and Wix each change their APIs - the editorial layer: topics ranked by demand, citation gap and winnability, then drafted and reviewed rather than dumped - exact technical diffs (JSON-LD, canonicals, robots rules, llms.txt) instead of generic advice you still have to translate - months of score history, without which a single week's citation rate tells you nothing - per-engine weekly scoring across seven on-page dimensions, which costs real API spend every run ## Success criteria - An attempted update on a run row is confirmed to fail - Five URL variants are confirmed to collapse to one citation - A run is confirmed to stop at the spend ceiling with partial results kept - Each crawler token is confirmed reported separately with its purpose - A model-change diff is confirmed flagged as not comparable ===== BRIEF.md ===== # Build brief · Rankwise The one-shot brief this plan expands. `BUILD_PLAN.md` (or `MILESTONES.md`) is the same sequence broken into steps and checks; where the two disagree, the plan wins. Build me a citation tracker that asks the answer engines your buyers' questions every week and keeps every run to replace Rankwise. Build it in phases, in the order below. Do not write the whole thing in one pass. Finish a phase, run its "Done when" check, fix what fails, and only then start the next phase. ### Rule zero This tool produces a number that people will make decisions with, so it has to be honest about what the number is. What you measure is what four APIs answered, with their search or grounding tools on, at a moment in time. It is not what a person sees in a chat window. So every row records the model identifier, whether the search tool was enabled, the timestamp and the raw answer, and the dashboard says in plain words that this approximates rather than reproduces the consumer experience. Two consequences. Rows are immutable: a run is never overwritten, never backfilled, never recomputed, because the whole value is a comparison against your own history and an edited past makes the comparison meaningless. And a failed call is a visible empty cell, never a zero, since a zero says you were not cited and an error says you do not know. Use the documented APIs. Do not script the consumer chat interfaces, whatever the numbers there would look like. ### Stack (fixed, do not substitute) - Runtime: Node 22 and TypeScript, one CLI plus a server-rendered dashboard on localhost · a scheduled job and a page to read it, nothing more - Storage: SQLite with append-only run tables · history is the product, so the schema refuses updates - Engines: The four vendors' own APIs with their search or grounding tools enabled · scripting the chat interfaces breaks their terms and the data is no better - Scheduling: A cron entry calling the CLI, with the run recorded whether it finishes or not · a half-finished week is data, a missing week is a hole - Budget: A per-run spend ceiling checked before the fan-out starts · questions times engines times weeks is how a hobby becomes an invoice ### Data model (create this before Phase 1) - site(domain, brand_aliases_json, competitor_domains_json) · one site, because the comparison only means anything for one - questions(id, text, added_at, retired_at) · retired rather than deleted, so old runs still make sense - runs(id, started_at, finished_at, questions_count, engines_json, estimated_cost, actual_cost, status) - answers(run_id, question_id, provider, model_id, search_enabled, raw_answer, latency_ms, error, created_at) · one immutable row per cell, errors included - citations(answer_id, ordinal, raw_url, host, canonical_path, is_mine, is_competitor) · normalised once at write time - audits(url, checked_at, findings_json) · the mechanical on-page checks, kept per run so a fix is visible as a change - No table in this schema has an update path. A correction is a new run, because the point of the history is that nobody edited it. ### Phase 1 · A question set and a run that cannot be rewritten Build: Load the site definition and up to forty questions from files. Create a run, write one row per question and provider with the raw answer, model identifier, whether the search tool was on, and any error. Make the tables append-only and prove it. Trap: Storing one current score per question and updating it each week feels tidier and destroys the only thing this tool is for, because a citation rate in week one means nothing and a trend over six months means everything. Append a row per run and never update one. Done when: Running twice produces two runs and zero updated rows, asserted by a test that attempts an update and expects it to fail. Every answer row records model identifier, search-enabled flag and timestamp. A provider error is stored as a row with an error and a null answer, not as an absent row. Retiring a question leaves every historical answer for it readable. Do not build yet: No scoring, no dashboard, no audit. ### Phase 2 · Fanning out without burning the budget Build: Cap concurrency per provider, retry rate limits and server errors with backoff, and stop the run when the estimated spend passes the ceiling. Estimate cost before the run from the current published prices, record the actual cost after, and show both. Trap: Retrying a rate limit immediately in a tight loop converts one throttled provider into a run that takes an hour and costs twice the estimate, and retrying a request that already produced an answer pays for the same cell twice. Back off, cap attempts, and record a cell as failed rather than retrying it forever. Done when: A run stops at the spend ceiling with the reason recorded and partial results kept. Rate limited calls back off rather than retrying immediately, tested against a fake provider that always throttles. The estimated and actual cost of each run are both stored and shown. Forty questions across four providers finish or fail within a stated wall clock time, measured. Do not build yet: No normalisation, scoring or audit yet. ### Phase 3 · Two spellings of one URL are one citation Build: Parse cited URLs out of each answer, strip tracking parameters and fragments, lower-case the host, drop a leading www, resolve one level of redirect where the provider gives you one, and store the normalised form alongside the raw one. Match your domain and your brand aliases. Trap: Counting citations by exact URL string splits one page into five entries because of a trailing slash, a tracking parameter, an http scheme and a www prefix, and your own site then appears to be cited a fifth as often as it is. Normalise at write time, keep the raw URL beside it, and test the five variants collapse to one. Done when: Five spellings of the same page collapse to one normalised citation, tested explicitly with a trailing slash, a tracking parameter, a scheme difference, a www prefix and a fragment. The raw URL is retained beside the normalised one for every citation. A brand alias mentioned without a link is counted separately from a linked citation and labelled as such. The top cited domains table per engine sums to the citation count. Do not build yet: No dashboard or audit yet. ### Phase 4 · An audit that gives correct crawler advice Build: Fetch one page and report only what is checkable: missing or invalid JSON-LD, a missing or conflicting canonical, heading structure, and which documented agents your robots rules allow or block. Print the JSON-LD block you would add. State each finding as a fact, not a promise that fixing it will win a citation. Trap: Reporting 'GPTBot is blocked, so you are invisible in ChatGPT' is wrong advice, and it is the single most common mistake in this category. OpenAI documents four separate agents with different jobs: GPTBot is the model training crawler, OAI-SearchBot is what surfaces sites in ChatGPT's search features, OAI-AdsBot validates pages submitted as ads, and ChatGPT-User serves user-initiated actions where, in OpenAI's own words, robots.txt rules may not apply. Report each token separately, say what blocking it actually costs, and re-read the vendor pages before you ship, because these tokens are new and they move. Done when: The audit lists each documented agent by token with its stated purpose and whether your rules allow it, and blocking the training crawler is never reported as removing you from search. JSON-LD is parsed and validated rather than merely detected, with the error shown. A conflicting canonical is reported with both values. Every finding is phrased as an observation and none of them promises a ranking outcome. Do not build yet: No publishing, drafting or CMS integration, ever. ### Phase 5 · History you can read and take with you Build: Render citation rate per engine over time, the questions where a competitor is cited and you are not, and a week over week difference that marks changed model identifiers. Export runs, answers and citations to CSV. Keep raw answers under a stated retention and say so. Trap: Showing a week over week movement without showing that the provider changed model version in between invites a team to celebrate or panic about a change they did not cause. Mark every diff whose model identifier differs from the previous run, and say plainly that the two weeks are not comparable. Done when: The dashboard shows citation rate per engine over every stored run, with a run count so a single week cannot be mistaken for a trend. A diff where the model identifier changed is flagged as not comparable. Export writes runs, answers and citations to CSV and the row counts match the database. The interface states, where the score is shown, that this measures API answers rather than what a user sees. Do not build yet: Nothing after this. ### Out of scope (and why) - Drafting articles and publishing them into a CMS, which is the half that makes Rankwise a product and a whole project on its own. - Scripting the consumer chat interfaces, which breaks the providers' terms and produces data no more trustworthy than the API. - A single overall visibility score, because averaging four engines with different answer shapes produces a number with no defined meaning. - Tracking competitors' whole sites, which multiplies the API spend and tells you less than one question they win and you lose. ### README must contain - That the tool measures API answers with search tools enabled, which approximates rather than reproduces what a person sees - The per-run cost estimate for your question count, and where the current prices are published - A cron line for the weekly run, and what a partially failed run looks like in the data - That run rows are append-only and why an edited history makes the whole tool pointless - Which crawler tokens the audit checks, what each one governs, and that these change often enough to re-read the vendor pages ===== ARCHITECTURE.md ===== # Architecture · Rankwise ## Stack | Part | Choice | Why | | --- | --- | --- | | Runtime | Node 22 and TypeScript, one CLI plus a server-rendered dashboard on localhost | a scheduled job and a page to read it, nothing more | | Storage | SQLite with append-only run tables | history is the product, so the schema refuses updates | | Engines | The four vendors' own APIs with their search or grounding tools enabled | scripting the chat interfaces breaks their terms and the data is no better | | Scheduling | A cron entry calling the CLI, with the run recorded whether it finishes or not | a half-finished week is data, a missing week is a hole | | Budget | A per-run spend ceiling checked before the fan-out starts | questions times engines times weeks is how a hobby becomes an invoice | ## Modules Each module has one owner concern and a documented way to replace it. | Module | Owns | How to replace it | | --- | --- | --- | | Runner | Fan-out, concurrency and the spend ceiling. | Any scheduler, provided a partial run is still recorded. | | Providers | One adapter per engine returning answer, citations and model id. | Add an engine as an adapter; never by scripting a chat interface. | | Normalisation | URL canonicalisation at write time. | Any rules, provided the raw URL is kept beside the result. | | Audit | Mechanical on-page checks and crawler rules. | Any checks, provided each finding stays an observation. | | Dashboard | Trend, gaps and diffs. | Any presentation, provided run counts and model changes stay visible. | ## Configuration Every runtime setting is an environment variable documented in `.env.example`, validated at startup, with a safe local default wherever one exists. - `DATABASE_PATH` · required · Runs, answers and citations. Append-only and the whole asset. - `OPENAI_API_KEY` · required, secret · With the web search tool enabled on the account. - `ANTHROPIC_API_KEY` · required, secret · With the web search tool enabled on the account. - `GEMINI_API_KEY` · required, secret · With grounding enabled on the account. - `PERPLEXITY_API_KEY` · required, secret · Search is the default here, but record the flag anyway. - `RUN_COST_CEILING_USD` · required · Checked before the fan-out. A run that would exceed it stops and records why. - `PROVIDER_CONCURRENCY` · required · Per provider, not in total. One of them is always the slow one. - `RAW_ANSWER_RETENTION_DAYS` · required · Raw answers are large. Scores and citations are kept forever regardless. ## Production baseline - Security: least privilege, input validation at every boundary, secret redaction in logs, rate limits on abuse-prone paths, no invented security primitives. - Data: explicit schema and migrations, transactional writes where integrity matters, backup and restore procedures that have been exercised. - Integrations: adapters around third-party providers, idempotent webhook or job processing, bounded retries, timeouts. - Observability: structured logs with request or operation ids, an error-tracking hook, and health and readiness checks where a server exists. - Quality: unit tests for domain rules, integration tests at module boundaries, one end-to-end test of the critical path. ## Decision records For each dependency in the stack table, keep a short note: why it was chosen, its failure mode, and how it is replaced. Do not add infrastructure until a requirement in `PRODUCT.md` justifies it. ===== AGENTS.md ===== # Agent instructions · Rankwise product build - Read `PRODUCT.md` and `ARCHITECTURE.md` before changing code. The stack is fixed: Node 22 and TypeScript, one CLI plus a server-rendered dashboard on localhost, SQLite with append-only run tables, The four vendors' own APIs with their search or grounding tools enabled, A cron entry calling the CLI, with the run recorded whether it finishes or not, A per-run spend ceiling checked before the fan-out starts. - Implement milestone by milestone from `MILESTONES.md`; keep each change reviewable and leave the application runnable at every commit. - Treat authentication, payments, encryption, imports, webhooks and destructive actions as high-risk boundaries when present. - Never invent cryptography or silently weaken a requirement to make a check pass. - Put every external service behind an interface with a deterministic fake for tests. - Add migrations and rollback or recovery notes for every persistent data change. - Log useful operational context without credentials, tokens, passwords or personal data. - Update documentation and run every check before completing a milestone. ## Known traps - Storing one current score per question and updating it each week feels tidier and destroys the only thing this tool is for, because a citation rate in week one means nothing and a trend over six months means everything. Append a row per run and never update one. - Retrying a rate limit immediately in a tight loop converts one throttled provider into a run that takes an hour and costs twice the estimate, and retrying a request that already produced an answer pays for the same cell twice. Back off, cap attempts, and record a cell as failed rather than retrying it forever. - Counting citations by exact URL string splits one page into five entries because of a trailing slash, a tracking parameter, an http scheme and a www prefix, and your own site then appears to be cited a fifth as often as it is. Normalise at write time, keep the raw URL beside it, and test the five variants collapse to one. - Reporting 'GPTBot is blocked, so you are invisible in ChatGPT' is wrong advice, and it is the single most common mistake in this category. OpenAI documents four separate agents with different jobs: GPTBot is the model training crawler, OAI-SearchBot is what surfaces sites in ChatGPT's search features, OAI-AdsBot validates pages submitted as ads, and ChatGPT-User serves user-initiated actions where, in OpenAI's own words, robots.txt rules may not apply. Report each token separately, say what blocking it actually costs, and re-read the vendor pages before you ship, because these tokens are new and they move. - Showing a week over week movement without showing that the provider changed model version in between invites a team to celebrate or panic about a change they did not cause. Mark every diff whose model identifier differs from the previous run, and say plainly that the two weeks are not comparable. ===== MILESTONES.md ===== # Delivery milestones · Rankwise Estimated effort: **weekend** for the indie phases; the production-only milestones add the trust and operability layer. ## M1 · A question set and a run that cannot be rewritten One run writes one immutable row per question and engine, including the failures. ### Steps 1. Make the tables append-only A trigger or a check that turns an update into a failed test rather than a lost week. Files: `src/db/schema.ts`, `test/append-only.test.ts` 2. Record the model identifier and tool state per cell Providers change defaults underneath you and the comparison has to survive that. Files: `src/providers/types.ts` 3. Store errors as rows An empty cell is information; a missing row is an unanswered question about your own data. Files: `src/run.ts` ### Done when - [ ] Two runs produce two rows, never an update - [ ] Model id and tool state are recorded per cell - [ ] Errors are stored as rows with the error text - [ ] Retired questions keep their history readable ### Watch out - Storing one current score per question and updating it each week feels tidier and destroys the only thing this tool is for, because a citation rate in week one means nothing and a trend over six months means everything. Append a row per run and never update one. ## M2 · Fanning out without burning the budget Four providers are called concurrently under a spend ceiling with visible failures. ### Steps 1. Cap concurrency per provider Each has its own limits and one of them is always the slow one. Files: `src/pool.ts` 2. Check the ceiling before spending The estimate comes from the current published prices, which you re-check, not from a constant. Files: `src/budget.ts`, `docs/costs.md` 3. Back off and then give up A failed cell with an error beats a run that never ends. Files: `src/retry.ts`, `test/retry.test.ts` ### Done when - [ ] A run stops at the ceiling and keeps partial results - [ ] Throttled calls back off rather than hammering - [ ] Estimated and actual cost are both recorded - [ ] A full run completes within a measured time ### Watch out - Retrying a rate limit immediately in a tight loop converts one throttled provider into a run that takes an hour and costs twice the estimate, and retrying a request that already produced an answer pays for the same cell twice. Back off, cap attempts, and record a cell as failed rather than retrying it forever. ## M3 · Two spellings of one URL are one citation Citations normalise to a host and a canonical path so counting means something. ### Steps 1. Normalise once, at write time Normalising at read time means every report defines its own truth. Files: `src/normalise.ts`, `test/normalise.test.ts` 2. Keep the raw URL When a count looks wrong the raw string is the only way to find out why. Files: `src/db/schema.ts` 3. Separate a mention from a citation Being named and being linked are different outcomes and mixing them flatters you. Files: `src/score.ts` ### Done when - [ ] Five URL variants collapse to one citation - [ ] Raw URLs are retained beside normalised ones - [ ] Mentions and citations are counted separately - [ ] Domain tables reconcile with citation counts ### Watch out - Counting citations by exact URL string splits one page into five entries because of a trailing slash, a tracking parameter, an http scheme and a www prefix, and your own site then appears to be cited a fifth as often as it is. Normalise at write time, keep the raw URL beside it, and test the five variants collapse to one. ## M4 · An audit that gives correct crawler advice The on-page audit reports mechanical facts and names the right agent for the right job. ### Steps 1. Name each agent and what blocking it costs Training, search, ads and user-initiated fetching are four different decisions. Files: `src/audit/crawlers.ts`, `docs/crawlers.md` 2. Validate the JSON-LD, do not just find it A malformed block is worse than none and looks identical from the outside. Files: `src/audit/jsonld.ts` 3. Report observations, never promises Nobody can promise a citation, and a tool that implies it is lying politely. Files: `src/audit/report.ts` ### Done when - [ ] Each crawler token is reported separately with its purpose - [ ] JSON-LD is parsed and errors are shown - [ ] Conflicting canonicals show both values - [ ] No finding promises a ranking outcome ### Watch out - Reporting 'GPTBot is blocked, so you are invisible in ChatGPT' is wrong advice, and it is the single most common mistake in this category. OpenAI documents four separate agents with different jobs: GPTBot is the model training crawler, OAI-SearchBot is what surfaces sites in ChatGPT's search features, OAI-AdsBot validates pages submitted as ads, and ChatGPT-User serves user-initiated actions where, in OpenAI's own words, robots.txt rules may not apply. Report each token separately, say what blocking it actually costs, and re-read the vendor pages before you ship, because these tokens are new and they move. ## M5 · History you can read and take with you (production only) The dashboard shows trend and gaps, and everything exports without the app. ### Steps 1. Flag diffs across a model change The provider swapping a model is the most common false signal in this whole category. Files: `src/dashboard/diff.ts` 2. Show the run count beside every rate A rate from two runs is a coin toss with a percentage sign. Files: `src/dashboard/views.ts` 3. Export everything to CSV The history is the asset, so it must leave the tool intact. Files: `src/export.ts` ### Done when - [ ] Trend views show every stored run - [ ] Model-change diffs are flagged as not comparable - [ ] CSV export row counts match the database - [ ] The measurement caveat is shown beside the score ### Watch out - Showing a week over week movement without showing that the provider changed model version in between invites a team to celebrate or panic about a change they did not cause. Mark every diff whose model identifier differs from the previous run, and say plainly that the two weeks are not comparable. ===== OPERATIONS.md ===== # Operations · Rankwise ## Backup The database, weekly, off the machine that runs the job. It is a time series that cannot be regenerated at any price, because last March's answers no longer exist anywhere. ## Restore Restore and run the append-only test before the next scheduled run, so a restored database that somehow allows updates is caught before it silently rewrites history. Do a restore drill before the first real user, and write the date here when it passes. ## Monitoring Failed cells per run, actual against estimated cost, and the age of the last successful run. A quiet week usually means an expired key rather than a quiet week. ## Incident checklist A sudden citation collapse is a provider change until proven otherwise: check the recorded model identifiers and the failed cell count before anyone rewrites a content strategy. 1. Contain the issue without destroying evidence or user data. 2. Record the timeline and affected scope. 3. Rotate exposed secrets and revoke compromised sessions or credentials. 4. Restore from a verified backup when needed. 5. Document the root cause, the remediation and the regression test. ## Release gate - [ ] An attempted update on a run row is confirmed to fail - [ ] Five URL variants are confirmed to collapse to one citation - [ ] A run is confirmed to stop at the spend ceiling with partial results kept - [ ] Each crawler token is confirmed reported separately with its purpose - [ ] A model-change diff is confirmed flagged as not comparable ## Launch constraint Do not market omitted Rankwise capabilities as implemented. The non-goals in `PRODUCT.md` remain user-visible limitations until they are deliberately delivered. ===== .env.example ===== # Copy to .env and fill in. Never commit .env; this file documents it. # Required. Runs, answers and citations. Append-only and the whole asset. DATABASE_PATH=./data/rankwise.db # Required · secret. With the web search tool enabled on the account. OPENAI_API_KEY=sk-...redacted # Required · secret. With the web search tool enabled on the account. ANTHROPIC_API_KEY=sk-ant-...redacted # Required · secret. With grounding enabled on the account. GEMINI_API_KEY=AIza...redacted # Required · secret. Search is the default here, but record the flag anyway. PERPLEXITY_API_KEY=pplx-...redacted # Required. Checked before the fan-out. A run that would exceed it stops and records why. RUN_COST_CEILING_USD=12.00 # Required. Per provider, not in total. One of them is always the slow one. PROVIDER_CONCURRENCY=3 # Required. Raw answers are large. Scores and citations are kept forever regardless. RAW_ANSWER_RETENTION_DAYS=400
README.md
# Rankwise · indie build A local weekly citation tracker for one site across four answer engines, whose hard problems are an immutable run history, normalising citations so two spellings of one URL are one domain, and an on-page audit that gives correct advice about crawlers that do different jobs. Estimated effort: **weekend**. Work `BUILD_PLAN.md` top to bottom · every phase ends in a check that has to pass before the next one starts. ## Stack | Part | Choice | Why | | --- | --- | --- | | Runtime | Node 22 and TypeScript, one CLI plus a server-rendered dashboard on localhost | a scheduled job and a page to read it, nothing more | | Storage | SQLite with append-only run tables | history is the product, so the schema refuses updates | | Engines | The four vendors' own APIs with their search or grounding tools enabled | scripting the chat interfaces breaks their terms and the data is no better | | Scheduling | A cron entry calling the CLI, with the run recorded whether it finishes or not | a half-finished week is data, a missing week is a hole | | Budget | A per-run spend ceiling checked before the fan-out starts | questions times engines times weeks is how a hobby becomes an invoice | ## Before you start Have every one of these ready. The plan assumes them from step one. - [ ] **Node.js 22 or newer** · free - Why: Runs the CLI and the local dashboard. - Get it: nodejs.org, or nvm install 22. - Verify: node --version prints v22 or higher - [ ] **API keys for the four answer engines** · pay per call, and it is the main running cost of this build - Why: Each engine is a separate account with its own billing, and the search or grounding tool has to be enabled per provider or you are measuring the model's memory rather than its citations. - Get it: Create an account and a key with each provider, then confirm the web search or grounding tool is available on your plan. - Verify: One question returns an answer with citations from each of the four - [ ] **A per-run spend ceiling** · free to decide - Why: Questions times engines times weeks compounds quietly, and the ceiling is the only thing standing between a weekly job and a surprise invoice. - Get it: Estimate one run at current published prices, decide what a week is worth, and write the figure into the configuration. - Verify: A run that would exceed the figure stops and says so - [ ] **Up to forty buyer questions** · free - Why: The question set is the instrument, so changing it mid-history is like moving the goalposts and then measuring the distance. - Get it: Write the questions a buyer actually types before they know your name, and commit to leaving them alone for a quarter. - Verify: The file loads and every question is answerable by a source you could plausibly be - [ ] **A scheduler that runs weekly** · free, or the cost of the machine - Why: The value is the series, and a series with gaps where the laptop was shut is a series you cannot read. - Get it: A cron entry on an always-on machine, or your platform's equivalent. - Verify: A scheduled run appears in the runs table without anyone pressing anything ## Quick start Follow Phase 1 in `BUILD_PLAN.md`; it creates the project. Then copy `.env.example` to `.env` and fill in the values it documents. ## Honest limits This build deliberately does not replace: - Article drafting and CMS publishing. - Any scripting of the consumer chat interfaces. - A single blended visibility score. - Competitor site-wide tracking. - the CMS publishing matrix, and the upkeep as WordPress, Shopify, Webflow and Wix each change their APIs - the editorial layer: topics ranked by demand, citation gap and winnability, then drafted and reviewed rather than dumped - exact technical diffs (JSON-LD, canonicals, robots rules, llms.txt) instead of generic advice you still have to translate - months of score history, without which a single week's citation rate tells you nothing - per-engine weekly scoring across seven on-page dimensions, which costs real API spend every run If one of those is essential to you, that is the reason to keep paying for Rankwise, and the README should say so rather than pretend.
BRIEF.md
# Build brief · Rankwise The one-shot brief this plan expands. `BUILD_PLAN.md` (or `MILESTONES.md`) is the same sequence broken into steps and checks; where the two disagree, the plan wins. Build me a citation tracker that asks the answer engines your buyers' questions every week and keeps every run to replace Rankwise. Build it in phases, in the order below. Do not write the whole thing in one pass. Finish a phase, run its "Done when" check, fix what fails, and only then start the next phase. ### Rule zero This tool produces a number that people will make decisions with, so it has to be honest about what the number is. What you measure is what four APIs answered, with their search or grounding tools on, at a moment in time. It is not what a person sees in a chat window. So every row records the model identifier, whether the search tool was enabled, the timestamp and the raw answer, and the dashboard says in plain words that this approximates rather than reproduces the consumer experience. Two consequences. Rows are immutable: a run is never overwritten, never backfilled, never recomputed, because the whole value is a comparison against your own history and an edited past makes the comparison meaningless. And a failed call is a visible empty cell, never a zero, since a zero says you were not cited and an error says you do not know. Use the documented APIs. Do not script the consumer chat interfaces, whatever the numbers there would look like. ### Stack (fixed, do not substitute) - Runtime: Node 22 and TypeScript, one CLI plus a server-rendered dashboard on localhost · a scheduled job and a page to read it, nothing more - Storage: SQLite with append-only run tables · history is the product, so the schema refuses updates - Engines: The four vendors' own APIs with their search or grounding tools enabled · scripting the chat interfaces breaks their terms and the data is no better - Scheduling: A cron entry calling the CLI, with the run recorded whether it finishes or not · a half-finished week is data, a missing week is a hole - Budget: A per-run spend ceiling checked before the fan-out starts · questions times engines times weeks is how a hobby becomes an invoice ### Data model (create this before Phase 1) - site(domain, brand_aliases_json, competitor_domains_json) · one site, because the comparison only means anything for one - questions(id, text, added_at, retired_at) · retired rather than deleted, so old runs still make sense - runs(id, started_at, finished_at, questions_count, engines_json, estimated_cost, actual_cost, status) - answers(run_id, question_id, provider, model_id, search_enabled, raw_answer, latency_ms, error, created_at) · one immutable row per cell, errors included - citations(answer_id, ordinal, raw_url, host, canonical_path, is_mine, is_competitor) · normalised once at write time - audits(url, checked_at, findings_json) · the mechanical on-page checks, kept per run so a fix is visible as a change - No table in this schema has an update path. A correction is a new run, because the point of the history is that nobody edited it. ### Phase 1 · A question set and a run that cannot be rewritten Build: Load the site definition and up to forty questions from files. Create a run, write one row per question and provider with the raw answer, model identifier, whether the search tool was on, and any error. Make the tables append-only and prove it. Trap: Storing one current score per question and updating it each week feels tidier and destroys the only thing this tool is for, because a citation rate in week one means nothing and a trend over six months means everything. Append a row per run and never update one. Done when: Running twice produces two runs and zero updated rows, asserted by a test that attempts an update and expects it to fail. Every answer row records model identifier, search-enabled flag and timestamp. A provider error is stored as a row with an error and a null answer, not as an absent row. Retiring a question leaves every historical answer for it readable. Do not build yet: No scoring, no dashboard, no audit. ### Phase 2 · Fanning out without burning the budget Build: Cap concurrency per provider, retry rate limits and server errors with backoff, and stop the run when the estimated spend passes the ceiling. Estimate cost before the run from the current published prices, record the actual cost after, and show both. Trap: Retrying a rate limit immediately in a tight loop converts one throttled provider into a run that takes an hour and costs twice the estimate, and retrying a request that already produced an answer pays for the same cell twice. Back off, cap attempts, and record a cell as failed rather than retrying it forever. Done when: A run stops at the spend ceiling with the reason recorded and partial results kept. Rate limited calls back off rather than retrying immediately, tested against a fake provider that always throttles. The estimated and actual cost of each run are both stored and shown. Forty questions across four providers finish or fail within a stated wall clock time, measured. Do not build yet: No normalisation, scoring or audit yet. ### Phase 3 · Two spellings of one URL are one citation Build: Parse cited URLs out of each answer, strip tracking parameters and fragments, lower-case the host, drop a leading www, resolve one level of redirect where the provider gives you one, and store the normalised form alongside the raw one. Match your domain and your brand aliases. Trap: Counting citations by exact URL string splits one page into five entries because of a trailing slash, a tracking parameter, an http scheme and a www prefix, and your own site then appears to be cited a fifth as often as it is. Normalise at write time, keep the raw URL beside it, and test the five variants collapse to one. Done when: Five spellings of the same page collapse to one normalised citation, tested explicitly with a trailing slash, a tracking parameter, a scheme difference, a www prefix and a fragment. The raw URL is retained beside the normalised one for every citation. A brand alias mentioned without a link is counted separately from a linked citation and labelled as such. The top cited domains table per engine sums to the citation count. Do not build yet: No dashboard or audit yet. ### Phase 4 · An audit that gives correct crawler advice Build: Fetch one page and report only what is checkable: missing or invalid JSON-LD, a missing or conflicting canonical, heading structure, and which documented agents your robots rules allow or block. Print the JSON-LD block you would add. State each finding as a fact, not a promise that fixing it will win a citation. Trap: Reporting 'GPTBot is blocked, so you are invisible in ChatGPT' is wrong advice, and it is the single most common mistake in this category. OpenAI documents four separate agents with different jobs: GPTBot is the model training crawler, OAI-SearchBot is what surfaces sites in ChatGPT's search features, OAI-AdsBot validates pages submitted as ads, and ChatGPT-User serves user-initiated actions where, in OpenAI's own words, robots.txt rules may not apply. Report each token separately, say what blocking it actually costs, and re-read the vendor pages before you ship, because these tokens are new and they move. Done when: The audit lists each documented agent by token with its stated purpose and whether your rules allow it, and blocking the training crawler is never reported as removing you from search. JSON-LD is parsed and validated rather than merely detected, with the error shown. A conflicting canonical is reported with both values. Every finding is phrased as an observation and none of them promises a ranking outcome. Do not build yet: No publishing, drafting or CMS integration, ever. ### Phase 5 · History you can read and take with you Build: Render citation rate per engine over time, the questions where a competitor is cited and you are not, and a week over week difference that marks changed model identifiers. Export runs, answers and citations to CSV. Keep raw answers under a stated retention and say so. Trap: Showing a week over week movement without showing that the provider changed model version in between invites a team to celebrate or panic about a change they did not cause. Mark every diff whose model identifier differs from the previous run, and say plainly that the two weeks are not comparable. Done when: The dashboard shows citation rate per engine over every stored run, with a run count so a single week cannot be mistaken for a trend. A diff where the model identifier changed is flagged as not comparable. Export writes runs, answers and citations to CSV and the row counts match the database. The interface states, where the score is shown, that this measures API answers rather than what a user sees. Do not build yet: Nothing after this. ### Out of scope (and why) - Drafting articles and publishing them into a CMS, which is the half that makes Rankwise a product and a whole project on its own. - Scripting the consumer chat interfaces, which breaks the providers' terms and produces data no more trustworthy than the API. - A single overall visibility score, because averaging four engines with different answer shapes produces a number with no defined meaning. - Tracking competitors' whole sites, which multiplies the API spend and tells you less than one question they win and you lose. ### README must contain - That the tool measures API answers with search tools enabled, which approximates rather than reproduces what a person sees - The per-run cost estimate for your question count, and where the current prices are published - A cron line for the weekly run, and what a partially failed run looks like in the data - That run rows are append-only and why an edited history makes the whole tool pointless - Which crawler tokens the audit checks, what each one governs, and that these change often enough to re-read the vendor pages
AGENTS.md
# Agent instructions · Rankwise indie build - Read `README.md` and `BUILD_PLAN.md` before writing code. The stack is fixed: Node 22 and TypeScript, one CLI plus a server-rendered dashboard on localhost, SQLite with append-only run tables, The four vendors' own APIs with their search or grounding tools enabled, A cron entry calling the CLI, with the run recorded whether it finishes or not, A per-run spend ceiling checked before the fan-out starts. Do not substitute. - Work one phase at a time, in order. Do not start a phase until every "Done when" item of the previous one passes. - Prefer the fewest moving parts that satisfy the step. No frameworks, services or dependencies the plan does not name. - Secrets live in `.env`, never in source or logs. Keep `.env.example` current when a variable is introduced. - Do not invent cryptography, security guarantees, APIs or compliance claims. - Add a focused test for every destructive, security-sensitive or data-loss path the plan names. - Run the project checks before declaring a phase complete, and record any deliberate shortcut in the README under "Tradeoffs". ## Known traps - Storing one current score per question and updating it each week feels tidier and destroys the only thing this tool is for, because a citation rate in week one means nothing and a trend over six months means everything. Append a row per run and never update one. - Retrying a rate limit immediately in a tight loop converts one throttled provider into a run that takes an hour and costs twice the estimate, and retrying a request that already produced an answer pays for the same cell twice. Back off, cap attempts, and record a cell as failed rather than retrying it forever. - Counting citations by exact URL string splits one page into five entries because of a trailing slash, a tracking parameter, an http scheme and a www prefix, and your own site then appears to be cited a fifth as often as it is. Normalise at write time, keep the raw URL beside it, and test the five variants collapse to one. - Reporting 'GPTBot is blocked, so you are invisible in ChatGPT' is wrong advice, and it is the single most common mistake in this category. OpenAI documents four separate agents with different jobs: GPTBot is the model training crawler, OAI-SearchBot is what surfaces sites in ChatGPT's search features, OAI-AdsBot validates pages submitted as ads, and ChatGPT-User serves user-initiated actions where, in OpenAI's own words, robots.txt rules may not apply. Report each token separately, say what blocking it actually costs, and re-read the vendor pages before you ship, because these tokens are new and they move.
BUILD_PLAN.md
# Build plan · Rankwise A local weekly citation tracker for one site across four answer engines, whose hard problems are an immutable run history, normalising citations so two spellings of one URL are one domain, and an on-page audit that gives correct advice about crawlers that do different jobs. Phases are in dependency order. Each ends in a "Done when" list; treat an unticked item as a blocker, not a note. ## Phase 1 · A question set and a run that cannot be rewritten One run writes one immutable row per question and engine, including the failures. ### Steps 1. Make the tables append-only A trigger or a check that turns an update into a failed test rather than a lost week. Files: `src/db/schema.ts`, `test/append-only.test.ts` 2. Record the model identifier and tool state per cell Providers change defaults underneath you and the comparison has to survive that. Files: `src/providers/types.ts` 3. Store errors as rows An empty cell is information; a missing row is an unanswered question about your own data. Files: `src/run.ts` ### Done when - [ ] Two runs produce two rows, never an update - [ ] Model id and tool state are recorded per cell - [ ] Errors are stored as rows with the error text - [ ] Retired questions keep their history readable ### Watch out - Storing one current score per question and updating it each week feels tidier and destroys the only thing this tool is for, because a citation rate in week one means nothing and a trend over six months means everything. Append a row per run and never update one. ## Phase 2 · Fanning out without burning the budget Four providers are called concurrently under a spend ceiling with visible failures. ### Steps 1. Cap concurrency per provider Each has its own limits and one of them is always the slow one. Files: `src/pool.ts` 2. Check the ceiling before spending The estimate comes from the current published prices, which you re-check, not from a constant. Files: `src/budget.ts`, `docs/costs.md` 3. Back off and then give up A failed cell with an error beats a run that never ends. Files: `src/retry.ts`, `test/retry.test.ts` ### Done when - [ ] A run stops at the ceiling and keeps partial results - [ ] Throttled calls back off rather than hammering - [ ] Estimated and actual cost are both recorded - [ ] A full run completes within a measured time ### Watch out - Retrying a rate limit immediately in a tight loop converts one throttled provider into a run that takes an hour and costs twice the estimate, and retrying a request that already produced an answer pays for the same cell twice. Back off, cap attempts, and record a cell as failed rather than retrying it forever. ## Phase 3 · Two spellings of one URL are one citation Citations normalise to a host and a canonical path so counting means something. ### Steps 1. Normalise once, at write time Normalising at read time means every report defines its own truth. Files: `src/normalise.ts`, `test/normalise.test.ts` 2. Keep the raw URL When a count looks wrong the raw string is the only way to find out why. Files: `src/db/schema.ts` 3. Separate a mention from a citation Being named and being linked are different outcomes and mixing them flatters you. Files: `src/score.ts` ### Done when - [ ] Five URL variants collapse to one citation - [ ] Raw URLs are retained beside normalised ones - [ ] Mentions and citations are counted separately - [ ] Domain tables reconcile with citation counts ### Watch out - Counting citations by exact URL string splits one page into five entries because of a trailing slash, a tracking parameter, an http scheme and a www prefix, and your own site then appears to be cited a fifth as often as it is. Normalise at write time, keep the raw URL beside it, and test the five variants collapse to one. ## Phase 4 · An audit that gives correct crawler advice The on-page audit reports mechanical facts and names the right agent for the right job. ### Steps 1. Name each agent and what blocking it costs Training, search, ads and user-initiated fetching are four different decisions. Files: `src/audit/crawlers.ts`, `docs/crawlers.md` 2. Validate the JSON-LD, do not just find it A malformed block is worse than none and looks identical from the outside. Files: `src/audit/jsonld.ts` 3. Report observations, never promises Nobody can promise a citation, and a tool that implies it is lying politely. Files: `src/audit/report.ts` ### Done when - [ ] Each crawler token is reported separately with its purpose - [ ] JSON-LD is parsed and errors are shown - [ ] Conflicting canonicals show both values - [ ] No finding promises a ranking outcome ### Watch out - Reporting 'GPTBot is blocked, so you are invisible in ChatGPT' is wrong advice, and it is the single most common mistake in this category. OpenAI documents four separate agents with different jobs: GPTBot is the model training crawler, OAI-SearchBot is what surfaces sites in ChatGPT's search features, OAI-AdsBot validates pages submitted as ads, and ChatGPT-User serves user-initiated actions where, in OpenAI's own words, robots.txt rules may not apply. Report each token separately, say what blocking it actually costs, and re-read the vendor pages before you ship, because these tokens are new and they move. ## Not in this build - Article drafting and CMS publishing. - Any scripting of the consumer chat interfaces. - A single blended visibility score. - Competitor site-wide tracking. ## After v1, if you want it - A fifth engine added as an adapter, once the four have run for a quarter without a gap - Per-question cost reporting, so an expensive question with no movement can be retired honestly - An annotation layer marking the dates you published something, since a trend with no events is a line
.env.example
# Copy to .env and fill in. Never commit .env; this file documents it. # Required. Runs, answers and citations. Append-only and the whole asset. DATABASE_PATH=./data/rankwise.db # Required · secret. With the web search tool enabled on the account. OPENAI_API_KEY=sk-...redacted # Required · secret. With the web search tool enabled on the account. ANTHROPIC_API_KEY=sk-ant-...redacted # Required · secret. With grounding enabled on the account. GEMINI_API_KEY=AIza...redacted # Required · secret. Search is the default here, but record the flag anyway. PERPLEXITY_API_KEY=pplx-...redacted # Required. Checked before the fan-out. A run that would exceed it stops and records why. RUN_COST_CEILING_USD=12.00 # Required. Per provider, not in total. One of them is always the slow one. PROVIDER_CONCURRENCY=3 # Required. Raw answers are large. Scores and citations are kept forever regardless. RAW_ANSWER_RETENTION_DAYS=400
PRODUCT.md
# Rankwise · product brief ## Problem The measurement half is a weekend project. Fire a fixed question set at the ChatGPT, Claude, Gemini and Perplexity APIs on a schedule, count whether your domain gets named, store every run, and you have the dashboard. What takes Rankwise past that is the acting half: ranking topics by demand and citation gap, drafting sourced articles, and publishing them into WordPress, Shopify, Webflow or Wix without breaking anything. You can hand-write that loop for one site and one CMS. Doing it across four engines and four CMSes, every week, while the APIs move under you, is the part that stays a product. ## Product outcome A weekly job that builds a citation history nobody can edit, tells you which questions a competitor owns, and gives crawler advice that is correct about which agent does which job. ## Target user A builder who needs a maintainable product foundation, not a one-off demo. ## Required capabilities - OpenAI, Anthropic, Gemini and Perplexity API keys with web search or grounding enabled - a scheduler for the weekly run - durable per-run storage so history survives - an API budget that scales with questions x engines x weeks - CMS credentials if you want the publishing half too ## Explicit non-goals for v1 - Article drafting and CMS publishing. - Any scripting of the consumer chat interfaces. - A single blended visibility score. - Competitor site-wide tracking. - the CMS publishing matrix, and the upkeep as WordPress, Shopify, Webflow and Wix each change their APIs - the editorial layer: topics ranked by demand, citation gap and winnability, then drafted and reviewed rather than dumped - exact technical diffs (JSON-LD, canonicals, robots rules, llms.txt) instead of generic advice you still have to translate - months of score history, without which a single week's citation rate tells you nothing - per-engine weekly scoring across seven on-page dimensions, which costs real API spend every run ## Success criteria - An attempted update on a run row is confirmed to fail - Five URL variants are confirmed to collapse to one citation - A run is confirmed to stop at the spend ceiling with partial results kept - Each crawler token is confirmed reported separately with its purpose - A model-change diff is confirmed flagged as not comparable
BRIEF.md
# Build brief · Rankwise The one-shot brief this plan expands. `BUILD_PLAN.md` (or `MILESTONES.md`) is the same sequence broken into steps and checks; where the two disagree, the plan wins. Build me a citation tracker that asks the answer engines your buyers' questions every week and keeps every run to replace Rankwise. Build it in phases, in the order below. Do not write the whole thing in one pass. Finish a phase, run its "Done when" check, fix what fails, and only then start the next phase. ### Rule zero This tool produces a number that people will make decisions with, so it has to be honest about what the number is. What you measure is what four APIs answered, with their search or grounding tools on, at a moment in time. It is not what a person sees in a chat window. So every row records the model identifier, whether the search tool was enabled, the timestamp and the raw answer, and the dashboard says in plain words that this approximates rather than reproduces the consumer experience. Two consequences. Rows are immutable: a run is never overwritten, never backfilled, never recomputed, because the whole value is a comparison against your own history and an edited past makes the comparison meaningless. And a failed call is a visible empty cell, never a zero, since a zero says you were not cited and an error says you do not know. Use the documented APIs. Do not script the consumer chat interfaces, whatever the numbers there would look like. ### Stack (fixed, do not substitute) - Runtime: Node 22 and TypeScript, one CLI plus a server-rendered dashboard on localhost · a scheduled job and a page to read it, nothing more - Storage: SQLite with append-only run tables · history is the product, so the schema refuses updates - Engines: The four vendors' own APIs with their search or grounding tools enabled · scripting the chat interfaces breaks their terms and the data is no better - Scheduling: A cron entry calling the CLI, with the run recorded whether it finishes or not · a half-finished week is data, a missing week is a hole - Budget: A per-run spend ceiling checked before the fan-out starts · questions times engines times weeks is how a hobby becomes an invoice ### Data model (create this before Phase 1) - site(domain, brand_aliases_json, competitor_domains_json) · one site, because the comparison only means anything for one - questions(id, text, added_at, retired_at) · retired rather than deleted, so old runs still make sense - runs(id, started_at, finished_at, questions_count, engines_json, estimated_cost, actual_cost, status) - answers(run_id, question_id, provider, model_id, search_enabled, raw_answer, latency_ms, error, created_at) · one immutable row per cell, errors included - citations(answer_id, ordinal, raw_url, host, canonical_path, is_mine, is_competitor) · normalised once at write time - audits(url, checked_at, findings_json) · the mechanical on-page checks, kept per run so a fix is visible as a change - No table in this schema has an update path. A correction is a new run, because the point of the history is that nobody edited it. ### Phase 1 · A question set and a run that cannot be rewritten Build: Load the site definition and up to forty questions from files. Create a run, write one row per question and provider with the raw answer, model identifier, whether the search tool was on, and any error. Make the tables append-only and prove it. Trap: Storing one current score per question and updating it each week feels tidier and destroys the only thing this tool is for, because a citation rate in week one means nothing and a trend over six months means everything. Append a row per run and never update one. Done when: Running twice produces two runs and zero updated rows, asserted by a test that attempts an update and expects it to fail. Every answer row records model identifier, search-enabled flag and timestamp. A provider error is stored as a row with an error and a null answer, not as an absent row. Retiring a question leaves every historical answer for it readable. Do not build yet: No scoring, no dashboard, no audit. ### Phase 2 · Fanning out without burning the budget Build: Cap concurrency per provider, retry rate limits and server errors with backoff, and stop the run when the estimated spend passes the ceiling. Estimate cost before the run from the current published prices, record the actual cost after, and show both. Trap: Retrying a rate limit immediately in a tight loop converts one throttled provider into a run that takes an hour and costs twice the estimate, and retrying a request that already produced an answer pays for the same cell twice. Back off, cap attempts, and record a cell as failed rather than retrying it forever. Done when: A run stops at the spend ceiling with the reason recorded and partial results kept. Rate limited calls back off rather than retrying immediately, tested against a fake provider that always throttles. The estimated and actual cost of each run are both stored and shown. Forty questions across four providers finish or fail within a stated wall clock time, measured. Do not build yet: No normalisation, scoring or audit yet. ### Phase 3 · Two spellings of one URL are one citation Build: Parse cited URLs out of each answer, strip tracking parameters and fragments, lower-case the host, drop a leading www, resolve one level of redirect where the provider gives you one, and store the normalised form alongside the raw one. Match your domain and your brand aliases. Trap: Counting citations by exact URL string splits one page into five entries because of a trailing slash, a tracking parameter, an http scheme and a www prefix, and your own site then appears to be cited a fifth as often as it is. Normalise at write time, keep the raw URL beside it, and test the five variants collapse to one. Done when: Five spellings of the same page collapse to one normalised citation, tested explicitly with a trailing slash, a tracking parameter, a scheme difference, a www prefix and a fragment. The raw URL is retained beside the normalised one for every citation. A brand alias mentioned without a link is counted separately from a linked citation and labelled as such. The top cited domains table per engine sums to the citation count. Do not build yet: No dashboard or audit yet. ### Phase 4 · An audit that gives correct crawler advice Build: Fetch one page and report only what is checkable: missing or invalid JSON-LD, a missing or conflicting canonical, heading structure, and which documented agents your robots rules allow or block. Print the JSON-LD block you would add. State each finding as a fact, not a promise that fixing it will win a citation. Trap: Reporting 'GPTBot is blocked, so you are invisible in ChatGPT' is wrong advice, and it is the single most common mistake in this category. OpenAI documents four separate agents with different jobs: GPTBot is the model training crawler, OAI-SearchBot is what surfaces sites in ChatGPT's search features, OAI-AdsBot validates pages submitted as ads, and ChatGPT-User serves user-initiated actions where, in OpenAI's own words, robots.txt rules may not apply. Report each token separately, say what blocking it actually costs, and re-read the vendor pages before you ship, because these tokens are new and they move. Done when: The audit lists each documented agent by token with its stated purpose and whether your rules allow it, and blocking the training crawler is never reported as removing you from search. JSON-LD is parsed and validated rather than merely detected, with the error shown. A conflicting canonical is reported with both values. Every finding is phrased as an observation and none of them promises a ranking outcome. Do not build yet: No publishing, drafting or CMS integration, ever. ### Phase 5 · History you can read and take with you Build: Render citation rate per engine over time, the questions where a competitor is cited and you are not, and a week over week difference that marks changed model identifiers. Export runs, answers and citations to CSV. Keep raw answers under a stated retention and say so. Trap: Showing a week over week movement without showing that the provider changed model version in between invites a team to celebrate or panic about a change they did not cause. Mark every diff whose model identifier differs from the previous run, and say plainly that the two weeks are not comparable. Done when: The dashboard shows citation rate per engine over every stored run, with a run count so a single week cannot be mistaken for a trend. A diff where the model identifier changed is flagged as not comparable. Export writes runs, answers and citations to CSV and the row counts match the database. The interface states, where the score is shown, that this measures API answers rather than what a user sees. Do not build yet: Nothing after this. ### Out of scope (and why) - Drafting articles and publishing them into a CMS, which is the half that makes Rankwise a product and a whole project on its own. - Scripting the consumer chat interfaces, which breaks the providers' terms and produces data no more trustworthy than the API. - A single overall visibility score, because averaging four engines with different answer shapes produces a number with no defined meaning. - Tracking competitors' whole sites, which multiplies the API spend and tells you less than one question they win and you lose. ### README must contain - That the tool measures API answers with search tools enabled, which approximates rather than reproduces what a person sees - The per-run cost estimate for your question count, and where the current prices are published - A cron line for the weekly run, and what a partially failed run looks like in the data - That run rows are append-only and why an edited history makes the whole tool pointless - Which crawler tokens the audit checks, what each one governs, and that these change often enough to re-read the vendor pages
ARCHITECTURE.md
# Architecture · Rankwise ## Stack | Part | Choice | Why | | --- | --- | --- | | Runtime | Node 22 and TypeScript, one CLI plus a server-rendered dashboard on localhost | a scheduled job and a page to read it, nothing more | | Storage | SQLite with append-only run tables | history is the product, so the schema refuses updates | | Engines | The four vendors' own APIs with their search or grounding tools enabled | scripting the chat interfaces breaks their terms and the data is no better | | Scheduling | A cron entry calling the CLI, with the run recorded whether it finishes or not | a half-finished week is data, a missing week is a hole | | Budget | A per-run spend ceiling checked before the fan-out starts | questions times engines times weeks is how a hobby becomes an invoice | ## Modules Each module has one owner concern and a documented way to replace it. | Module | Owns | How to replace it | | --- | --- | --- | | Runner | Fan-out, concurrency and the spend ceiling. | Any scheduler, provided a partial run is still recorded. | | Providers | One adapter per engine returning answer, citations and model id. | Add an engine as an adapter; never by scripting a chat interface. | | Normalisation | URL canonicalisation at write time. | Any rules, provided the raw URL is kept beside the result. | | Audit | Mechanical on-page checks and crawler rules. | Any checks, provided each finding stays an observation. | | Dashboard | Trend, gaps and diffs. | Any presentation, provided run counts and model changes stay visible. | ## Configuration Every runtime setting is an environment variable documented in `.env.example`, validated at startup, with a safe local default wherever one exists. - `DATABASE_PATH` · required · Runs, answers and citations. Append-only and the whole asset. - `OPENAI_API_KEY` · required, secret · With the web search tool enabled on the account. - `ANTHROPIC_API_KEY` · required, secret · With the web search tool enabled on the account. - `GEMINI_API_KEY` · required, secret · With grounding enabled on the account. - `PERPLEXITY_API_KEY` · required, secret · Search is the default here, but record the flag anyway. - `RUN_COST_CEILING_USD` · required · Checked before the fan-out. A run that would exceed it stops and records why. - `PROVIDER_CONCURRENCY` · required · Per provider, not in total. One of them is always the slow one. - `RAW_ANSWER_RETENTION_DAYS` · required · Raw answers are large. Scores and citations are kept forever regardless. ## Production baseline - Security: least privilege, input validation at every boundary, secret redaction in logs, rate limits on abuse-prone paths, no invented security primitives. - Data: explicit schema and migrations, transactional writes where integrity matters, backup and restore procedures that have been exercised. - Integrations: adapters around third-party providers, idempotent webhook or job processing, bounded retries, timeouts. - Observability: structured logs with request or operation ids, an error-tracking hook, and health and readiness checks where a server exists. - Quality: unit tests for domain rules, integration tests at module boundaries, one end-to-end test of the critical path. ## Decision records For each dependency in the stack table, keep a short note: why it was chosen, its failure mode, and how it is replaced. Do not add infrastructure until a requirement in `PRODUCT.md` justifies it.
AGENTS.md
# Agent instructions · Rankwise product build - Read `PRODUCT.md` and `ARCHITECTURE.md` before changing code. The stack is fixed: Node 22 and TypeScript, one CLI plus a server-rendered dashboard on localhost, SQLite with append-only run tables, The four vendors' own APIs with their search or grounding tools enabled, A cron entry calling the CLI, with the run recorded whether it finishes or not, A per-run spend ceiling checked before the fan-out starts. - Implement milestone by milestone from `MILESTONES.md`; keep each change reviewable and leave the application runnable at every commit. - Treat authentication, payments, encryption, imports, webhooks and destructive actions as high-risk boundaries when present. - Never invent cryptography or silently weaken a requirement to make a check pass. - Put every external service behind an interface with a deterministic fake for tests. - Add migrations and rollback or recovery notes for every persistent data change. - Log useful operational context without credentials, tokens, passwords or personal data. - Update documentation and run every check before completing a milestone. ## Known traps - Storing one current score per question and updating it each week feels tidier and destroys the only thing this tool is for, because a citation rate in week one means nothing and a trend over six months means everything. Append a row per run and never update one. - Retrying a rate limit immediately in a tight loop converts one throttled provider into a run that takes an hour and costs twice the estimate, and retrying a request that already produced an answer pays for the same cell twice. Back off, cap attempts, and record a cell as failed rather than retrying it forever. - Counting citations by exact URL string splits one page into five entries because of a trailing slash, a tracking parameter, an http scheme and a www prefix, and your own site then appears to be cited a fifth as often as it is. Normalise at write time, keep the raw URL beside it, and test the five variants collapse to one. - Reporting 'GPTBot is blocked, so you are invisible in ChatGPT' is wrong advice, and it is the single most common mistake in this category. OpenAI documents four separate agents with different jobs: GPTBot is the model training crawler, OAI-SearchBot is what surfaces sites in ChatGPT's search features, OAI-AdsBot validates pages submitted as ads, and ChatGPT-User serves user-initiated actions where, in OpenAI's own words, robots.txt rules may not apply. Report each token separately, say what blocking it actually costs, and re-read the vendor pages before you ship, because these tokens are new and they move. - Showing a week over week movement without showing that the provider changed model version in between invites a team to celebrate or panic about a change they did not cause. Mark every diff whose model identifier differs from the previous run, and say plainly that the two weeks are not comparable.
MILESTONES.md
# Delivery milestones · Rankwise Estimated effort: **weekend** for the indie phases; the production-only milestones add the trust and operability layer. ## M1 · A question set and a run that cannot be rewritten One run writes one immutable row per question and engine, including the failures. ### Steps 1. Make the tables append-only A trigger or a check that turns an update into a failed test rather than a lost week. Files: `src/db/schema.ts`, `test/append-only.test.ts` 2. Record the model identifier and tool state per cell Providers change defaults underneath you and the comparison has to survive that. Files: `src/providers/types.ts` 3. Store errors as rows An empty cell is information; a missing row is an unanswered question about your own data. Files: `src/run.ts` ### Done when - [ ] Two runs produce two rows, never an update - [ ] Model id and tool state are recorded per cell - [ ] Errors are stored as rows with the error text - [ ] Retired questions keep their history readable ### Watch out - Storing one current score per question and updating it each week feels tidier and destroys the only thing this tool is for, because a citation rate in week one means nothing and a trend over six months means everything. Append a row per run and never update one. ## M2 · Fanning out without burning the budget Four providers are called concurrently under a spend ceiling with visible failures. ### Steps 1. Cap concurrency per provider Each has its own limits and one of them is always the slow one. Files: `src/pool.ts` 2. Check the ceiling before spending The estimate comes from the current published prices, which you re-check, not from a constant. Files: `src/budget.ts`, `docs/costs.md` 3. Back off and then give up A failed cell with an error beats a run that never ends. Files: `src/retry.ts`, `test/retry.test.ts` ### Done when - [ ] A run stops at the ceiling and keeps partial results - [ ] Throttled calls back off rather than hammering - [ ] Estimated and actual cost are both recorded - [ ] A full run completes within a measured time ### Watch out - Retrying a rate limit immediately in a tight loop converts one throttled provider into a run that takes an hour and costs twice the estimate, and retrying a request that already produced an answer pays for the same cell twice. Back off, cap attempts, and record a cell as failed rather than retrying it forever. ## M3 · Two spellings of one URL are one citation Citations normalise to a host and a canonical path so counting means something. ### Steps 1. Normalise once, at write time Normalising at read time means every report defines its own truth. Files: `src/normalise.ts`, `test/normalise.test.ts` 2. Keep the raw URL When a count looks wrong the raw string is the only way to find out why. Files: `src/db/schema.ts` 3. Separate a mention from a citation Being named and being linked are different outcomes and mixing them flatters you. Files: `src/score.ts` ### Done when - [ ] Five URL variants collapse to one citation - [ ] Raw URLs are retained beside normalised ones - [ ] Mentions and citations are counted separately - [ ] Domain tables reconcile with citation counts ### Watch out - Counting citations by exact URL string splits one page into five entries because of a trailing slash, a tracking parameter, an http scheme and a www prefix, and your own site then appears to be cited a fifth as often as it is. Normalise at write time, keep the raw URL beside it, and test the five variants collapse to one. ## M4 · An audit that gives correct crawler advice The on-page audit reports mechanical facts and names the right agent for the right job. ### Steps 1. Name each agent and what blocking it costs Training, search, ads and user-initiated fetching are four different decisions. Files: `src/audit/crawlers.ts`, `docs/crawlers.md` 2. Validate the JSON-LD, do not just find it A malformed block is worse than none and looks identical from the outside. Files: `src/audit/jsonld.ts` 3. Report observations, never promises Nobody can promise a citation, and a tool that implies it is lying politely. Files: `src/audit/report.ts` ### Done when - [ ] Each crawler token is reported separately with its purpose - [ ] JSON-LD is parsed and errors are shown - [ ] Conflicting canonicals show both values - [ ] No finding promises a ranking outcome ### Watch out - Reporting 'GPTBot is blocked, so you are invisible in ChatGPT' is wrong advice, and it is the single most common mistake in this category. OpenAI documents four separate agents with different jobs: GPTBot is the model training crawler, OAI-SearchBot is what surfaces sites in ChatGPT's search features, OAI-AdsBot validates pages submitted as ads, and ChatGPT-User serves user-initiated actions where, in OpenAI's own words, robots.txt rules may not apply. Report each token separately, say what blocking it actually costs, and re-read the vendor pages before you ship, because these tokens are new and they move. ## M5 · History you can read and take with you (production only) The dashboard shows trend and gaps, and everything exports without the app. ### Steps 1. Flag diffs across a model change The provider swapping a model is the most common false signal in this whole category. Files: `src/dashboard/diff.ts` 2. Show the run count beside every rate A rate from two runs is a coin toss with a percentage sign. Files: `src/dashboard/views.ts` 3. Export everything to CSV The history is the asset, so it must leave the tool intact. Files: `src/export.ts` ### Done when - [ ] Trend views show every stored run - [ ] Model-change diffs are flagged as not comparable - [ ] CSV export row counts match the database - [ ] The measurement caveat is shown beside the score ### Watch out - Showing a week over week movement without showing that the provider changed model version in between invites a team to celebrate or panic about a change they did not cause. Mark every diff whose model identifier differs from the previous run, and say plainly that the two weeks are not comparable.
OPERATIONS.md
# Operations · Rankwise ## Backup The database, weekly, off the machine that runs the job. It is a time series that cannot be regenerated at any price, because last March's answers no longer exist anywhere. ## Restore Restore and run the append-only test before the next scheduled run, so a restored database that somehow allows updates is caught before it silently rewrites history. Do a restore drill before the first real user, and write the date here when it passes. ## Monitoring Failed cells per run, actual against estimated cost, and the age of the last successful run. A quiet week usually means an expired key rather than a quiet week. ## Incident checklist A sudden citation collapse is a provider change until proven otherwise: check the recorded model identifiers and the failed cell count before anyone rewrites a content strategy. 1. Contain the issue without destroying evidence or user data. 2. Record the timeline and affected scope. 3. Rotate exposed secrets and revoke compromised sessions or credentials. 4. Restore from a verified backup when needed. 5. Document the root cause, the remediation and the regression test. ## Release gate - [ ] An attempted update on a run row is confirmed to fail - [ ] Five URL variants are confirmed to collapse to one citation - [ ] A run is confirmed to stop at the spend ceiling with partial results kept - [ ] Each crawler token is confirmed reported separately with its purpose - [ ] A model-change diff is confirmed flagged as not comparable ## Launch constraint Do not market omitted Rankwise capabilities as implemented. The non-goals in `PRODUCT.md` remain user-visible limitations until they are deliberately delivered.
.env.example
# Copy to .env and fill in. Never commit .env; this file documents it. # Required. Runs, answers and citations. Append-only and the whole asset. DATABASE_PATH=./data/rankwise.db # Required · secret. With the web search tool enabled on the account. OPENAI_API_KEY=sk-...redacted # Required · secret. With the web search tool enabled on the account. ANTHROPIC_API_KEY=sk-ant-...redacted # Required · secret. With grounding enabled on the account. GEMINI_API_KEY=AIza...redacted # Required · secret. Search is the default here, but record the flag anyway. PERPLEXITY_API_KEY=pplx-...redacted # Required. Checked before the fan-out. A run that would exceed it stops and records why. RUN_COST_CEILING_USD=12.00 # Required. Per provider, not in total. One of them is always the slow one. PROVIDER_CONCURRENCY=3 # Required. Raw answers are large. Scores and citations are kept forever regardless. RAW_ANSWER_RETENTION_DAYS=400
$ choose a build depth, inspect the files, then open the complete pack in your agent
Because measuring the gap is the easy half and closing it is the work. A personal script tells you Perplexity never mentions you. It does not decide which of forty questions is winnable, write a sourced article, get it live in your CMS, and then prove which citation that article won. The accumulated score history matters too: the number is meaningless in week one and load-bearing in month six, and starting over resets you to zero.
xthe CMS publishing matrix, and the upkeep as WordPress, Shopify, Webflow and Wix each change their APIs
xthe editorial layer: topics ranked by demand, citation gap and winnability, then drafted and reviewed rather than dumped
xexact technical diffs (JSON-LD, canonicals, robots rules, llms.txt) instead of generic advice you still have to translate
xmonths of score history, without which a single week's citation rate tells you nothing
xper-engine weekly scoring across seven on-page dimensions, which costs real API spend every run
share on X ↗"I just replaced Rankwise ($59/mo) with one prompt"
Don't feel like building it? These folks already made it free.
ElmoTracks the citations honestly. Writing the article and pushing it to your CMS is still your evening.open source↗no votes, no pay-to-list · just what's real
Rankwise pricing
starter$59/mo · monthly · $708/yr
free tierAfter the trial the account drops to a free plan that keeps monitoring 1 domain and 10 tracked questions against Perplexity, re-checked weekly, with the score history intact. Writing and publishing articles need a paid plan.
verified 2026-08-10 · source ↗
Is Rankwise free?
After the trial the account drops to a free plan that keeps monitoring 1 domain and 10 tracked questions against Perplexity, re-checked weekly, with the score history intact. Writing and publishing articles need a paid plan. Paid is Starter at $59/mo (checked 2026-08-10).
Vibecode Rankwise
Kinda. The core of Rankwise is buildable in a weekend with the prompt on this page, but there are real gaps: the CMS publishing matrix, and the upkeep as WordPress, Shopify, Webflow and Wix each change their APIs, the editorial layer: topics ranked by demand, citation gap and winnability, then drafted and reviewed rather than dumped. Read the honest list above before committing.
How much does Rankwise cost?
Rankwise costs about $59/month (Starter, checked 2026-08-10), which is $708 per year.
What do I lose by replacing Rankwise?
Honestly: the CMS publishing matrix, and the upkeep as WordPress, Shopify, Webflow and Wix each change their APIs; the editorial layer: topics ranked by demand, citation gap and winnability, then drafted and reviewed rather than dumped; exact technical diffs (JSON-LD, canonicals, robots rules, llms.txt) instead of generic advice you still have to translate; months of score history, without which a single week's citation rate tells you nothing; per-engine weekly scoring across seven on-page dimensions, which costs real API spend every run. If any of those are load-bearing for you, keep paying.
Is there an open-source alternative to Rankwise?
Yes: Elmo (Tracks the citations honestly. Writing the article and pushing it to your CMS is still your evening.) The prompt is for when you want it exactly your way.