Vibecode Admix
track this build4 phases, 12 steps, beginner friendly0%KINDA · weekend project
A local chat UI with side-by-side replies is a weekend project when it calls OpenRouter, but Admix's value depends on access to many hosted models, a current catalog, provider routing, and a maintained account and billing layer. You can reproduce the comparison workflow. You cannot reproduce the model access and service layer for a flat $10 subscription.
the project pack5 files · written for this build, step by step
You are building a lean indie version of Admix. Create the following project files first, then implement the application by following them. Keep the files updated as decisions change. Do not collapse this into a single README or prompt. ===== README.md ===== # Admix · indie build A multi-model chat client where the engineering is concurrent streaming from providers that behave differently, and the design constraint is that side-by-side comparison must not be presented as a form of verification. Estimated effort: **weekend**. Work `BUILD_PLAN.md` top to bottom · every phase ends in a check that has to pass before the next one starts. ## Stack | Part | Choice | Why | | --- | --- | --- | | Providers | A thin adapter per provider with a shared streaming interface | each provider differs in streaming format, errors and rate limits | | Concurrency | Requests issued in parallel with independent failure | one slow provider must not hold up the others | | Cost | Token counting per request against current published pricing | cost is the thing people are most surprised by | | Comparison | Difference highlighting, never a scored winner | agreement is not verification and should not look like it | | Storage | Local, with per-conversation provider choice | what you send to which provider should be a visible decision | ## Before you start Have every one of these ready. The plan assumes them from step one. - [ ] **Node.js 22 or newer** · free - Why: Runs the client and the concurrent streaming. - Get it: nodejs.org, or nvm install 22. - Verify: node --version prints v22 or higher - [ ] **API keys for at least three providers** · per token; budget for development being more expensive than you expect - Why: The comparison is uninteresting with fewer, and provider differences in streaming, errors and refusals only appear across several. - Get it: Accounts with three providers, using the smallest models while developing since every test costs money three times over. - Verify: All three stream a response successfully - [ ] **Current pricing for every model you will use** · free to obtain - Why: The cost display is only useful if it is right, and provider pricing changes without much notice. - Get it: Take current published pricing per model, record it with the date you checked and the source. - Verify: A dated pricing record exists per model with a source link - [ ] **Your spending limit, decided first** · free to decide - Why: This is a category where costs multiply by the number of providers and run away quietly, and the limit is enforced in code. - Get it: Decide a monthly figure you would be comfortable spending before you have sent anything. - Verify: A figure is written down and configured before the first request - [ ] **Ten prompts where you know the right answer** · free - Why: The build declines to judge correctness, but you need to see for yourself how often models agree confidently and wrongly, since that is the reason for the constraint. - Get it: Write ten prompts with verifiable answers, including two where the answer is obscure enough that models are likely to guess. - Verify: At least one prompt produces confident agreement on a wrong answer ## Quick start Follow Phase 1 in `BUILD_PLAN.md`; it creates the project. Then copy `.env.example` to `.env` and fill in the values it documents. ## Honest limits This build deliberately does not replace: - Any judgement about which answer is correct. - Model evaluation or benchmarking. - Automatic routing to the best model. - Self-hosting models. - Admix's bundled access to many providers under one subscription - A maintained model catalog and provider routing - Cross-device sync and account history - Built-in image, search, and document workflows - Usage limits, billing, and support handled for you If one of those is essential to you, that is the reason to keep paying for Admix, and the README should say so rather than pretend. ===== BRIEF.md ===== # Build brief · Admix The one-shot brief this plan expands. `BUILD_PLAN.md` (or `MILESTONES.md`) is the same sequence broken into steps and checks; where the two disagree, the plan wins. Build me one prompt sent to several models, compared side by side with honest cost accounting to replace Admix. Build it in phases, in the order below. Do not write the whole thing in one pass. Finish a phase, run its "Done when" check, fix what fails, and only then start the next phase. ### Rule zero There is a trap in this category that the interface encourages: showing three answers side by side implies that comparing them tells you which is right. It does not. Three models agreeing on a wrong fact is a common outcome, particularly when they share training data, and the visual symmetry makes agreement look like verification. So the build does not present agreement as confidence. Where models disagree it says so and shows where; where they agree it says they agree and nothing more. Any claim you intend to rely on still has to be checked against a source, and the interface says that rather than implying otherwise. The genuinely useful things here are cheaper: seeing how differently models interpret an ambiguous prompt, catching one that has gone off, and knowing what each request actually cost. Those are what the build is organised around. ### Stack (fixed, do not substitute) - Providers: A thin adapter per provider with a shared streaming interface · each provider differs in streaming format, errors and rate limits - Concurrency: Requests issued in parallel with independent failure · one slow provider must not hold up the others - Cost: Token counting per request against current published pricing · cost is the thing people are most surprised by - Comparison: Difference highlighting, never a scored winner · agreement is not verification and should not look like it - Storage: Local, with per-conversation provider choice · what you send to which provider should be a visible decision ### Data model (create this before Phase 1) - conversations(id, title, created_at, providers_json) - turns(id, conversation_id, ordinal, prompt, at) - responses(id, turn_id, provider, model, body, finished, error, latency_ms, tokens_in, tokens_out, cost_minor) - pricing(provider, model, input_per_million_minor, output_per_million_minor, effective_from, checked_on) - differences(turn_id, kind, detail_json) · kind is factual_disagreement, refusal_divergence or format_divergence - budgets(period, limit_minor, spent_minor) · because this is a category where costs run away quietly - There is deliberately no column scoring which response was best, because a stored score would be displayed and it would be invented. ### Phase 1 · Several providers, one interface Build: An adapter per provider behind a shared streaming interface. Issue requests in parallel with independent failure and timeout. Normalise the differences in streaming format without hiding differences in behaviour. Trap: Waiting for all providers before showing anything makes the whole interface as slow as the slowest one, which is often several times the fastest. Stream each independently and let them finish at their own pace, showing which are still running. Done when: Three providers stream concurrently and each renders as it arrives. One provider failing or timing out does not affect the others, tested by pointing one at an unreachable endpoint. Streaming differences between providers are normalised. A provider's refusal is shown as a refusal rather than as an error. Do not build yet: No cost, comparison or budgets. ### Phase 2 · Knowing what it cost Build: Count input and output tokens per request and price them against a pricing table with an effective date. Show cost per response and per conversation. Surface pricing entries that have not been rechecked recently. Trap: Hard-coding prices means the figures silently become wrong when a provider changes them, and a tool whose cost display is wrong is worse than one with no cost display, because it is trusted. Keep pricing as dated data with a checked date and flag entries that have gone stale. Done when: Each response shows its token counts and computed cost. Pricing is dated data with a checked date and a source. Stale pricing entries are surfaced rather than used silently. Conversation total cost is shown and matches the sum of its responses. Do not build yet: No comparison or budgets yet. ### Phase 3 · Showing difference without implying a winner Build: Highlight where responses differ substantively rather than only in wording. Present agreement as agreement, not as confidence. Provide no ranking, score or recommended answer anywhere. Trap: Any score, badge or ordering that suggests one answer is better will be read as a verdict, and there is no basis for one. Show the differences and let the reader judge, and if you cannot resist the urge to rank, remember the ranking would be produced by another model with the same limitations. Done when: Substantive differences are highlighted separately from wording differences. Agreement is stated as agreement with no implication of correctness, verified by reading the interface text. No score, ranking or recommended answer exists anywhere, verified by inspection. A factual disagreement is surfaced prominently since it is the most useful signal. Do not build yet: No budgets or export yet. ### Phase 4 · Spending that cannot run away Build: A spending limit per period, checked before issuing requests. Refuse rather than warn when it is reached. Show remaining budget continuously and account for the cost of querying several providers at once. Trap: Sending one prompt to four providers costs four times as much, which is obvious and still surprises people at the end of the month, particularly with long context. Show the multiplied cost before sending, not after. Done when: Reaching the budget refuses new requests rather than warning, tested by exhausting it. The estimated cost of a multi-provider request is shown before sending. Remaining budget is visible continuously. The budget cannot be raised for the current period without an explicit and recorded action. Do not build yet: No export or key handling yet. ### Phase 5 · Keys, storage and getting out Build: Store provider keys encrypted at rest with no path that logs them. Keep conversations locally with search. Export everything including which provider produced each response and what it cost. Trap: A provider key in a log or an error report is a credential someone can spend your money with, and it gets there through a well-meaning debug line printing a request. Redact at the logging boundary and assert in the suite that no key material can reach a log. Done when: Keys are encrypted at rest and a test fails if a key reaches a log, verified by deliberately adding one. Conversations are searchable locally. Export contains prompts, responses, providers, models and costs and re-imports. Deleting a conversation removes its responses and cost records together. Do not build yet: Nothing after this. ### Out of scope (and why) - Any judgement about which answer is correct, which is the feature the layout implies and which cannot be provided honestly. - Model evaluation or benchmarking, which needs a methodology rather than a side-by-side view. - Routing a prompt automatically to the best model, which requires the judgement this build declines to make. - Hosting models yourself, which is a different project entirely. ### README must contain - That agreement between models is not verification and the interface does not present it as one - That there is no ranking, score or recommended answer anywhere, deliberately - That pricing is dated data with a checked date, because hard-coded prices go quietly wrong - That the budget refuses rather than warns, and multiplied cost is shown before sending - That a test fails if a provider key can reach a log ===== AGENTS.md ===== # Agent instructions · Admix indie build - Read `README.md` and `BUILD_PLAN.md` before writing code. The stack is fixed: A thin adapter per provider with a shared streaming interface, Requests issued in parallel with independent failure, Token counting per request against current published pricing, Difference highlighting, never a scored winner, Local, with per-conversation provider choice. Do not substitute. - Work one phase at a time, in order. Do not start a phase until every "Done when" item of the previous one passes. - Prefer the fewest moving parts that satisfy the step. No frameworks, services or dependencies the plan does not name. - Secrets live in `.env`, never in source or logs. Keep `.env.example` current when a variable is introduced. - Do not invent cryptography, security guarantees, APIs or compliance claims. - Add a focused test for every destructive, security-sensitive or data-loss path the plan names. - Run the project checks before declaring a phase complete, and record any deliberate shortcut in the README under "Tradeoffs". ## Known traps - Waiting for all providers before showing anything makes the whole interface as slow as the slowest one, which is often several times the fastest. Stream each independently and let them finish at their own pace, showing which are still running. - Hard-coding prices means the figures silently become wrong when a provider changes them, and a tool whose cost display is wrong is worse than one with no cost display, because it is trusted. Keep pricing as dated data with a checked date and flag entries that have gone stale. - Any score, badge or ordering that suggests one answer is better will be read as a verdict, and there is no basis for one. Show the differences and let the reader judge, and if you cannot resist the urge to rank, remember the ranking would be produced by another model with the same limitations. - Sending one prompt to four providers costs four times as much, which is obvious and still surprises people at the end of the month, particularly with long context. Show the multiplied cost before sending, not after. ===== BUILD_PLAN.md ===== # Build plan · Admix A multi-model chat client where the engineering is concurrent streaming from providers that behave differently, and the design constraint is that side-by-side comparison must not be presented as a form of verification. Phases are in dependency order. Each ends in a "Done when" list; treat an unticked item as a blocker, not a note. ## Phase 1 · Several providers, one interface One prompt streams from three providers concurrently. ### Steps 1. One adapter per provider Thin. Each provider's streaming format and error shape differ. Files: `src/providers/` 2. Stream independently The slowest provider should not set the pace for the others. Files: `src/stream.ts` 3. Distinguish refusal from error They mean different things and conflating them hides useful information. Files: `src/normalise.ts` ### Done when - [ ] Three providers stream concurrently - [ ] One failing provider does not affect others - [ ] Streaming formats are normalised - [ ] Refusals are distinct from errors ### Watch out - Waiting for all providers before showing anything makes the whole interface as slow as the slowest one, which is often several times the fastest. Stream each independently and let them finish at their own pace, showing which are still running. ## Phase 2 · Knowing what it cost Every request shows its actual cost against current pricing. ### Steps 1. Count tokens per request Both directions. Output is usually the expensive half. Files: `src/tokens.ts` 2. Keep pricing as dated data Hard-coded prices go quietly wrong and are trusted anyway. Files: `config/pricing.json` 3. Surface stale pricing A cost display that is wrong is worse than none. Files: `src/pricing.ts` ### Done when - [ ] Each response shows tokens and cost - [ ] Pricing is dated with a source - [ ] Stale pricing entries are surfaced - [ ] Conversation totals match their responses ### Watch out - Hard-coding prices means the figures silently become wrong when a provider changes them, and a tool whose cost display is wrong is worse than one with no cost display, because it is trusted. Keep pricing as dated data with a checked date and flag entries that have gone stale. ## Phase 3 · Showing difference without implying a winner You can see where models disagree and are not told who is right. ### Steps 1. Distinguish substantive from wording differences Wording differs always. Substance differing is the interesting case. Files: `src/diff.ts` 2. State agreement plainly They agree. That is the whole claim. Files: `src/compare.ts` 3. Provide no ranking anywhere Its absence is the design. A score would be invented. Files: `tests/no-ranking.test.ts` ### Done when - [ ] Substantive and wording differences are distinguished - [ ] Agreement is stated without implying correctness - [ ] No ranking or score exists - [ ] Factual disagreements are prominent ### Watch out - Any score, badge or ordering that suggests one answer is better will be read as a verdict, and there is no basis for one. Show the differences and let the reader judge, and if you cannot resist the urge to rank, remember the ranking would be produced by another model with the same limitations. ## Phase 4 · Spending that cannot run away A budget stops requests before the bill surprises you. ### Steps 1. Check the budget before issuing Refusing is the point. A warning after the fact is a receipt. Files: `src/budget.ts` 2. Show the multiplied cost before sending Four providers is four times. Obvious and still surprising. Files: `src/estimate.ts` 3. Make raising the limit deliberate And recorded, so the pattern is visible to you. Files: `src/budget.ts` ### Done when - [ ] The budget refuses rather than warns - [ ] Multiplied cost is shown before sending - [ ] Remaining budget is continuously visible - [ ] Raising the limit is deliberate and recorded ### Watch out - Sending one prompt to four providers costs four times as much, which is obvious and still surprises people at the end of the month, particularly with long context. Show the multiplied cost before sending, not after. ## Not in this build - Any judgement about which answer is correct. - Model evaluation or benchmarking. - Automatic routing to the best model. - Self-hosting models. ## After v1, if you want it - Saved prompt sets you can rerun across models when a new one appears - Per-provider latency history, which is genuinely useful and honestly measurable - A cheap model first, expensive model on demand flow, once cost accounting is trustworthy ===== .env.example ===== # Copy to .env and fill in. Never commit .env; this file documents it. # Required. Conversations and cost records. Contains whatever you asked about. DB_PATH=~/.local/share/admix/chats.db # Required · secret. Encrypts provider keys at rest. Without it they sit in plain text. KEYS_ENCRYPTION_KEY=a keychain reference # Required. Dated pricing per model. The file that keeps the cost display honest. PRICING_PATH=./config/pricing.json # Required. After this a pricing entry is surfaced as stale. PRICING_RECHECK_DAYS=30 # Required. Checked before issuing. Refuses rather than warns. MONTHLY_BUDGET_MINOR=2000 # Required. Present as a constant so the answer is visible. A ranking would be invented. SHOW_RANKING=never
You are building a lean indie version of Admix. Create the following project files first, then implement the application by following them. Keep the files updated as decisions change. Do not collapse this into a single README or prompt. ===== README.md ===== # Admix · indie build A multi-model chat client where the engineering is concurrent streaming from providers that behave differently, and the design constraint is that side-by-side comparison must not be presented as a form of verification. Estimated effort: **weekend**. Work `BUILD_PLAN.md` top to bottom · every phase ends in a check that has to pass before the next one starts. ## Stack | Part | Choice | Why | | --- | --- | --- | | Providers | A thin adapter per provider with a shared streaming interface | each provider differs in streaming format, errors and rate limits | | Concurrency | Requests issued in parallel with independent failure | one slow provider must not hold up the others | | Cost | Token counting per request against current published pricing | cost is the thing people are most surprised by | | Comparison | Difference highlighting, never a scored winner | agreement is not verification and should not look like it | | Storage | Local, with per-conversation provider choice | what you send to which provider should be a visible decision | ## Before you start Have every one of these ready. The plan assumes them from step one. - [ ] **Node.js 22 or newer** · free - Why: Runs the client and the concurrent streaming. - Get it: nodejs.org, or nvm install 22. - Verify: node --version prints v22 or higher - [ ] **API keys for at least three providers** · per token; budget for development being more expensive than you expect - Why: The comparison is uninteresting with fewer, and provider differences in streaming, errors and refusals only appear across several. - Get it: Accounts with three providers, using the smallest models while developing since every test costs money three times over. - Verify: All three stream a response successfully - [ ] **Current pricing for every model you will use** · free to obtain - Why: The cost display is only useful if it is right, and provider pricing changes without much notice. - Get it: Take current published pricing per model, record it with the date you checked and the source. - Verify: A dated pricing record exists per model with a source link - [ ] **Your spending limit, decided first** · free to decide - Why: This is a category where costs multiply by the number of providers and run away quietly, and the limit is enforced in code. - Get it: Decide a monthly figure you would be comfortable spending before you have sent anything. - Verify: A figure is written down and configured before the first request - [ ] **Ten prompts where you know the right answer** · free - Why: The build declines to judge correctness, but you need to see for yourself how often models agree confidently and wrongly, since that is the reason for the constraint. - Get it: Write ten prompts with verifiable answers, including two where the answer is obscure enough that models are likely to guess. - Verify: At least one prompt produces confident agreement on a wrong answer ## Quick start Follow Phase 1 in `BUILD_PLAN.md`; it creates the project. Then copy `.env.example` to `.env` and fill in the values it documents. ## Honest limits This build deliberately does not replace: - Any judgement about which answer is correct. - Model evaluation or benchmarking. - Automatic routing to the best model. - Self-hosting models. - Admix's bundled access to many providers under one subscription - A maintained model catalog and provider routing - Cross-device sync and account history - Built-in image, search, and document workflows - Usage limits, billing, and support handled for you If one of those is essential to you, that is the reason to keep paying for Admix, and the README should say so rather than pretend. ===== BRIEF.md ===== # Build brief · Admix The one-shot brief this plan expands. `BUILD_PLAN.md` (or `MILESTONES.md`) is the same sequence broken into steps and checks; where the two disagree, the plan wins. Build me one prompt sent to several models, compared side by side with honest cost accounting to replace Admix. Build it in phases, in the order below. Do not write the whole thing in one pass. Finish a phase, run its "Done when" check, fix what fails, and only then start the next phase. ### Rule zero There is a trap in this category that the interface encourages: showing three answers side by side implies that comparing them tells you which is right. It does not. Three models agreeing on a wrong fact is a common outcome, particularly when they share training data, and the visual symmetry makes agreement look like verification. So the build does not present agreement as confidence. Where models disagree it says so and shows where; where they agree it says they agree and nothing more. Any claim you intend to rely on still has to be checked against a source, and the interface says that rather than implying otherwise. The genuinely useful things here are cheaper: seeing how differently models interpret an ambiguous prompt, catching one that has gone off, and knowing what each request actually cost. Those are what the build is organised around. ### Stack (fixed, do not substitute) - Providers: A thin adapter per provider with a shared streaming interface · each provider differs in streaming format, errors and rate limits - Concurrency: Requests issued in parallel with independent failure · one slow provider must not hold up the others - Cost: Token counting per request against current published pricing · cost is the thing people are most surprised by - Comparison: Difference highlighting, never a scored winner · agreement is not verification and should not look like it - Storage: Local, with per-conversation provider choice · what you send to which provider should be a visible decision ### Data model (create this before Phase 1) - conversations(id, title, created_at, providers_json) - turns(id, conversation_id, ordinal, prompt, at) - responses(id, turn_id, provider, model, body, finished, error, latency_ms, tokens_in, tokens_out, cost_minor) - pricing(provider, model, input_per_million_minor, output_per_million_minor, effective_from, checked_on) - differences(turn_id, kind, detail_json) · kind is factual_disagreement, refusal_divergence or format_divergence - budgets(period, limit_minor, spent_minor) · because this is a category where costs run away quietly - There is deliberately no column scoring which response was best, because a stored score would be displayed and it would be invented. ### Phase 1 · Several providers, one interface Build: An adapter per provider behind a shared streaming interface. Issue requests in parallel with independent failure and timeout. Normalise the differences in streaming format without hiding differences in behaviour. Trap: Waiting for all providers before showing anything makes the whole interface as slow as the slowest one, which is often several times the fastest. Stream each independently and let them finish at their own pace, showing which are still running. Done when: Three providers stream concurrently and each renders as it arrives. One provider failing or timing out does not affect the others, tested by pointing one at an unreachable endpoint. Streaming differences between providers are normalised. A provider's refusal is shown as a refusal rather than as an error. Do not build yet: No cost, comparison or budgets. ### Phase 2 · Knowing what it cost Build: Count input and output tokens per request and price them against a pricing table with an effective date. Show cost per response and per conversation. Surface pricing entries that have not been rechecked recently. Trap: Hard-coding prices means the figures silently become wrong when a provider changes them, and a tool whose cost display is wrong is worse than one with no cost display, because it is trusted. Keep pricing as dated data with a checked date and flag entries that have gone stale. Done when: Each response shows its token counts and computed cost. Pricing is dated data with a checked date and a source. Stale pricing entries are surfaced rather than used silently. Conversation total cost is shown and matches the sum of its responses. Do not build yet: No comparison or budgets yet. ### Phase 3 · Showing difference without implying a winner Build: Highlight where responses differ substantively rather than only in wording. Present agreement as agreement, not as confidence. Provide no ranking, score or recommended answer anywhere. Trap: Any score, badge or ordering that suggests one answer is better will be read as a verdict, and there is no basis for one. Show the differences and let the reader judge, and if you cannot resist the urge to rank, remember the ranking would be produced by another model with the same limitations. Done when: Substantive differences are highlighted separately from wording differences. Agreement is stated as agreement with no implication of correctness, verified by reading the interface text. No score, ranking or recommended answer exists anywhere, verified by inspection. A factual disagreement is surfaced prominently since it is the most useful signal. Do not build yet: No budgets or export yet. ### Phase 4 · Spending that cannot run away Build: A spending limit per period, checked before issuing requests. Refuse rather than warn when it is reached. Show remaining budget continuously and account for the cost of querying several providers at once. Trap: Sending one prompt to four providers costs four times as much, which is obvious and still surprises people at the end of the month, particularly with long context. Show the multiplied cost before sending, not after. Done when: Reaching the budget refuses new requests rather than warning, tested by exhausting it. The estimated cost of a multi-provider request is shown before sending. Remaining budget is visible continuously. The budget cannot be raised for the current period without an explicit and recorded action. Do not build yet: No export or key handling yet. ### Phase 5 · Keys, storage and getting out Build: Store provider keys encrypted at rest with no path that logs them. Keep conversations locally with search. Export everything including which provider produced each response and what it cost. Trap: A provider key in a log or an error report is a credential someone can spend your money with, and it gets there through a well-meaning debug line printing a request. Redact at the logging boundary and assert in the suite that no key material can reach a log. Done when: Keys are encrypted at rest and a test fails if a key reaches a log, verified by deliberately adding one. Conversations are searchable locally. Export contains prompts, responses, providers, models and costs and re-imports. Deleting a conversation removes its responses and cost records together. Do not build yet: Nothing after this. ### Out of scope (and why) - Any judgement about which answer is correct, which is the feature the layout implies and which cannot be provided honestly. - Model evaluation or benchmarking, which needs a methodology rather than a side-by-side view. - Routing a prompt automatically to the best model, which requires the judgement this build declines to make. - Hosting models yourself, which is a different project entirely. ### README must contain - That agreement between models is not verification and the interface does not present it as one - That there is no ranking, score or recommended answer anywhere, deliberately - That pricing is dated data with a checked date, because hard-coded prices go quietly wrong - That the budget refuses rather than warns, and multiplied cost is shown before sending - That a test fails if a provider key can reach a log ===== AGENTS.md ===== # Agent instructions · Admix indie build - Read `README.md` and `BUILD_PLAN.md` before writing code. The stack is fixed: A thin adapter per provider with a shared streaming interface, Requests issued in parallel with independent failure, Token counting per request against current published pricing, Difference highlighting, never a scored winner, Local, with per-conversation provider choice. Do not substitute. - Work one phase at a time, in order. Do not start a phase until every "Done when" item of the previous one passes. - Prefer the fewest moving parts that satisfy the step. No frameworks, services or dependencies the plan does not name. - Secrets live in `.env`, never in source or logs. Keep `.env.example` current when a variable is introduced. - Do not invent cryptography, security guarantees, APIs or compliance claims. - Add a focused test for every destructive, security-sensitive or data-loss path the plan names. - Run the project checks before declaring a phase complete, and record any deliberate shortcut in the README under "Tradeoffs". ## Known traps - Waiting for all providers before showing anything makes the whole interface as slow as the slowest one, which is often several times the fastest. Stream each independently and let them finish at their own pace, showing which are still running. - Hard-coding prices means the figures silently become wrong when a provider changes them, and a tool whose cost display is wrong is worse than one with no cost display, because it is trusted. Keep pricing as dated data with a checked date and flag entries that have gone stale. - Any score, badge or ordering that suggests one answer is better will be read as a verdict, and there is no basis for one. Show the differences and let the reader judge, and if you cannot resist the urge to rank, remember the ranking would be produced by another model with the same limitations. - Sending one prompt to four providers costs four times as much, which is obvious and still surprises people at the end of the month, particularly with long context. Show the multiplied cost before sending, not after. ===== BUILD_PLAN.md ===== # Build plan · Admix A multi-model chat client where the engineering is concurrent streaming from providers that behave differently, and the design constraint is that side-by-side comparison must not be presented as a form of verification. Phases are in dependency order. Each ends in a "Done when" list; treat an unticked item as a blocker, not a note. ## Phase 1 · Several providers, one interface One prompt streams from three providers concurrently. ### Steps 1. One adapter per provider Thin. Each provider's streaming format and error shape differ. Files: `src/providers/` 2. Stream independently The slowest provider should not set the pace for the others. Files: `src/stream.ts` 3. Distinguish refusal from error They mean different things and conflating them hides useful information. Files: `src/normalise.ts` ### Done when - [ ] Three providers stream concurrently - [ ] One failing provider does not affect others - [ ] Streaming formats are normalised - [ ] Refusals are distinct from errors ### Watch out - Waiting for all providers before showing anything makes the whole interface as slow as the slowest one, which is often several times the fastest. Stream each independently and let them finish at their own pace, showing which are still running. ## Phase 2 · Knowing what it cost Every request shows its actual cost against current pricing. ### Steps 1. Count tokens per request Both directions. Output is usually the expensive half. Files: `src/tokens.ts` 2. Keep pricing as dated data Hard-coded prices go quietly wrong and are trusted anyway. Files: `config/pricing.json` 3. Surface stale pricing A cost display that is wrong is worse than none. Files: `src/pricing.ts` ### Done when - [ ] Each response shows tokens and cost - [ ] Pricing is dated with a source - [ ] Stale pricing entries are surfaced - [ ] Conversation totals match their responses ### Watch out - Hard-coding prices means the figures silently become wrong when a provider changes them, and a tool whose cost display is wrong is worse than one with no cost display, because it is trusted. Keep pricing as dated data with a checked date and flag entries that have gone stale. ## Phase 3 · Showing difference without implying a winner You can see where models disagree and are not told who is right. ### Steps 1. Distinguish substantive from wording differences Wording differs always. Substance differing is the interesting case. Files: `src/diff.ts` 2. State agreement plainly They agree. That is the whole claim. Files: `src/compare.ts` 3. Provide no ranking anywhere Its absence is the design. A score would be invented. Files: `tests/no-ranking.test.ts` ### Done when - [ ] Substantive and wording differences are distinguished - [ ] Agreement is stated without implying correctness - [ ] No ranking or score exists - [ ] Factual disagreements are prominent ### Watch out - Any score, badge or ordering that suggests one answer is better will be read as a verdict, and there is no basis for one. Show the differences and let the reader judge, and if you cannot resist the urge to rank, remember the ranking would be produced by another model with the same limitations. ## Phase 4 · Spending that cannot run away A budget stops requests before the bill surprises you. ### Steps 1. Check the budget before issuing Refusing is the point. A warning after the fact is a receipt. Files: `src/budget.ts` 2. Show the multiplied cost before sending Four providers is four times. Obvious and still surprising. Files: `src/estimate.ts` 3. Make raising the limit deliberate And recorded, so the pattern is visible to you. Files: `src/budget.ts` ### Done when - [ ] The budget refuses rather than warns - [ ] Multiplied cost is shown before sending - [ ] Remaining budget is continuously visible - [ ] Raising the limit is deliberate and recorded ### Watch out - Sending one prompt to four providers costs four times as much, which is obvious and still surprises people at the end of the month, particularly with long context. Show the multiplied cost before sending, not after. ## Not in this build - Any judgement about which answer is correct. - Model evaluation or benchmarking. - Automatic routing to the best model. - Self-hosting models. ## After v1, if you want it - Saved prompt sets you can rerun across models when a new one appears - Per-provider latency history, which is genuinely useful and honestly measurable - A cheap model first, expensive model on demand flow, once cost accounting is trustworthy ===== .env.example ===== # Copy to .env and fill in. Never commit .env; this file documents it. # Required. Conversations and cost records. Contains whatever you asked about. DB_PATH=~/.local/share/admix/chats.db # Required · secret. Encrypts provider keys at rest. Without it they sit in plain text. KEYS_ENCRYPTION_KEY=a keychain reference # Required. Dated pricing per model. The file that keeps the cost display honest. PRICING_PATH=./config/pricing.json # Required. After this a pricing entry is surfaced as stale. PRICING_RECHECK_DAYS=30 # Required. Checked before issuing. Refuses rather than warns. MONTHLY_BUDGET_MINOR=2000 # Required. Present as a constant so the answer is visible. A ranking would be invented. SHOW_RANKING=never
You are building a production product version of Admix. Create the following project files first, then implement the application by following them. Keep the files updated as decisions change. Do not collapse this into a single README or prompt. ===== PRODUCT.md ===== # Admix · product brief ## Problem A local chat UI with side-by-side replies is a weekend project when it calls OpenRouter, but Admix's value depends on access to many hosted models, a current catalog, provider routing, and a maintained account and billing layer. You can reproduce the comparison workflow. You cannot reproduce the model access and service layer for a flat $10 subscription. ## Product outcome A comparison client that streams from several models at once, tells you exactly what each request cost, and never suggests that three models agreeing means they are right. ## Target user A builder who needs a maintainable product foundation, not a one-off demo. ## Required capabilities - Node 22 - OpenRouter API key in .env - SQLite - Provider and model IDs to compare ## Explicit non-goals for v1 - Any judgement about which answer is correct. - Model evaluation or benchmarking. - Automatic routing to the best model. - Self-hosting models. - Admix's bundled access to many providers under one subscription - A maintained model catalog and provider routing - Cross-device sync and account history - Built-in image, search, and document workflows - Usage limits, billing, and support handled for you ## Success criteria - Three providers are confirmed to stream concurrently with independent failure - Pricing is confirmed dated with stale entries surfaced - No ranking or score is confirmed to exist - The budget is confirmed to refuse rather than warn - A key is confirmed unable to reach a log ===== BRIEF.md ===== # Build brief · Admix The one-shot brief this plan expands. `BUILD_PLAN.md` (or `MILESTONES.md`) is the same sequence broken into steps and checks; where the two disagree, the plan wins. Build me one prompt sent to several models, compared side by side with honest cost accounting to replace Admix. Build it in phases, in the order below. Do not write the whole thing in one pass. Finish a phase, run its "Done when" check, fix what fails, and only then start the next phase. ### Rule zero There is a trap in this category that the interface encourages: showing three answers side by side implies that comparing them tells you which is right. It does not. Three models agreeing on a wrong fact is a common outcome, particularly when they share training data, and the visual symmetry makes agreement look like verification. So the build does not present agreement as confidence. Where models disagree it says so and shows where; where they agree it says they agree and nothing more. Any claim you intend to rely on still has to be checked against a source, and the interface says that rather than implying otherwise. The genuinely useful things here are cheaper: seeing how differently models interpret an ambiguous prompt, catching one that has gone off, and knowing what each request actually cost. Those are what the build is organised around. ### Stack (fixed, do not substitute) - Providers: A thin adapter per provider with a shared streaming interface · each provider differs in streaming format, errors and rate limits - Concurrency: Requests issued in parallel with independent failure · one slow provider must not hold up the others - Cost: Token counting per request against current published pricing · cost is the thing people are most surprised by - Comparison: Difference highlighting, never a scored winner · agreement is not verification and should not look like it - Storage: Local, with per-conversation provider choice · what you send to which provider should be a visible decision ### Data model (create this before Phase 1) - conversations(id, title, created_at, providers_json) - turns(id, conversation_id, ordinal, prompt, at) - responses(id, turn_id, provider, model, body, finished, error, latency_ms, tokens_in, tokens_out, cost_minor) - pricing(provider, model, input_per_million_minor, output_per_million_minor, effective_from, checked_on) - differences(turn_id, kind, detail_json) · kind is factual_disagreement, refusal_divergence or format_divergence - budgets(period, limit_minor, spent_minor) · because this is a category where costs run away quietly - There is deliberately no column scoring which response was best, because a stored score would be displayed and it would be invented. ### Phase 1 · Several providers, one interface Build: An adapter per provider behind a shared streaming interface. Issue requests in parallel with independent failure and timeout. Normalise the differences in streaming format without hiding differences in behaviour. Trap: Waiting for all providers before showing anything makes the whole interface as slow as the slowest one, which is often several times the fastest. Stream each independently and let them finish at their own pace, showing which are still running. Done when: Three providers stream concurrently and each renders as it arrives. One provider failing or timing out does not affect the others, tested by pointing one at an unreachable endpoint. Streaming differences between providers are normalised. A provider's refusal is shown as a refusal rather than as an error. Do not build yet: No cost, comparison or budgets. ### Phase 2 · Knowing what it cost Build: Count input and output tokens per request and price them against a pricing table with an effective date. Show cost per response and per conversation. Surface pricing entries that have not been rechecked recently. Trap: Hard-coding prices means the figures silently become wrong when a provider changes them, and a tool whose cost display is wrong is worse than one with no cost display, because it is trusted. Keep pricing as dated data with a checked date and flag entries that have gone stale. Done when: Each response shows its token counts and computed cost. Pricing is dated data with a checked date and a source. Stale pricing entries are surfaced rather than used silently. Conversation total cost is shown and matches the sum of its responses. Do not build yet: No comparison or budgets yet. ### Phase 3 · Showing difference without implying a winner Build: Highlight where responses differ substantively rather than only in wording. Present agreement as agreement, not as confidence. Provide no ranking, score or recommended answer anywhere. Trap: Any score, badge or ordering that suggests one answer is better will be read as a verdict, and there is no basis for one. Show the differences and let the reader judge, and if you cannot resist the urge to rank, remember the ranking would be produced by another model with the same limitations. Done when: Substantive differences are highlighted separately from wording differences. Agreement is stated as agreement with no implication of correctness, verified by reading the interface text. No score, ranking or recommended answer exists anywhere, verified by inspection. A factual disagreement is surfaced prominently since it is the most useful signal. Do not build yet: No budgets or export yet. ### Phase 4 · Spending that cannot run away Build: A spending limit per period, checked before issuing requests. Refuse rather than warn when it is reached. Show remaining budget continuously and account for the cost of querying several providers at once. Trap: Sending one prompt to four providers costs four times as much, which is obvious and still surprises people at the end of the month, particularly with long context. Show the multiplied cost before sending, not after. Done when: Reaching the budget refuses new requests rather than warning, tested by exhausting it. The estimated cost of a multi-provider request is shown before sending. Remaining budget is visible continuously. The budget cannot be raised for the current period without an explicit and recorded action. Do not build yet: No export or key handling yet. ### Phase 5 · Keys, storage and getting out Build: Store provider keys encrypted at rest with no path that logs them. Keep conversations locally with search. Export everything including which provider produced each response and what it cost. Trap: A provider key in a log or an error report is a credential someone can spend your money with, and it gets there through a well-meaning debug line printing a request. Redact at the logging boundary and assert in the suite that no key material can reach a log. Done when: Keys are encrypted at rest and a test fails if a key reaches a log, verified by deliberately adding one. Conversations are searchable locally. Export contains prompts, responses, providers, models and costs and re-imports. Deleting a conversation removes its responses and cost records together. Do not build yet: Nothing after this. ### Out of scope (and why) - Any judgement about which answer is correct, which is the feature the layout implies and which cannot be provided honestly. - Model evaluation or benchmarking, which needs a methodology rather than a side-by-side view. - Routing a prompt automatically to the best model, which requires the judgement this build declines to make. - Hosting models yourself, which is a different project entirely. ### README must contain - That agreement between models is not verification and the interface does not present it as one - That there is no ranking, score or recommended answer anywhere, deliberately - That pricing is dated data with a checked date, because hard-coded prices go quietly wrong - That the budget refuses rather than warns, and multiplied cost is shown before sending - That a test fails if a provider key can reach a log ===== ARCHITECTURE.md ===== # Architecture · Admix ## Stack | Part | Choice | Why | | --- | --- | --- | | Providers | A thin adapter per provider with a shared streaming interface | each provider differs in streaming format, errors and rate limits | | Concurrency | Requests issued in parallel with independent failure | one slow provider must not hold up the others | | Cost | Token counting per request against current published pricing | cost is the thing people are most surprised by | | Comparison | Difference highlighting, never a scored winner | agreement is not verification and should not look like it | | Storage | Local, with per-conversation provider choice | what you send to which provider should be a visible decision | ## Modules Each module has one owner concern and a documented way to replace it. | Module | Owns | How to replace it | | --- | --- | --- | | Providers | Adapters behind one streaming interface. | Any provider, provided refusals stay distinguishable from errors. | | Cost | Token counting against dated pricing. | Any implementation, provided pricing is data with a checked date. | | Comparison | Difference highlighting without ranking. | Any diffing, provided no score or recommended answer is produced. | | Budget | Pre-request refusal. | Any implementation, provided it refuses rather than warning. | | Keys | Encrypted storage and log redaction. | Any store, provided the log assertion stays in the suite. | ## Configuration Every runtime setting is an environment variable documented in `.env.example`, validated at startup, with a safe local default wherever one exists. - `DB_PATH` · required · Conversations and cost records. Contains whatever you asked about. - `KEYS_ENCRYPTION_KEY` · required, secret · Encrypts provider keys at rest. Without it they sit in plain text. - `PRICING_PATH` · required · Dated pricing per model. The file that keeps the cost display honest. - `PRICING_RECHECK_DAYS` · required · After this a pricing entry is surfaced as stale. - `MONTHLY_BUDGET_MINOR` · required · Checked before issuing. Refuses rather than warns. - `SHOW_RANKING` · required · Present as a constant so the answer is visible. A ranking would be invented. ## Production baseline - Security: least privilege, input validation at every boundary, secret redaction in logs, rate limits on abuse-prone paths, no invented security primitives. - Data: explicit schema and migrations, transactional writes where integrity matters, backup and restore procedures that have been exercised. - Integrations: adapters around third-party providers, idempotent webhook or job processing, bounded retries, timeouts. - Observability: structured logs with request or operation ids, an error-tracking hook, and health and readiness checks where a server exists. - Quality: unit tests for domain rules, integration tests at module boundaries, one end-to-end test of the critical path. ## Decision records For each dependency in the stack table, keep a short note: why it was chosen, its failure mode, and how it is replaced. Do not add infrastructure until a requirement in `PRODUCT.md` justifies it. ===== AGENTS.md ===== # Agent instructions · Admix product build - Read `PRODUCT.md` and `ARCHITECTURE.md` before changing code. The stack is fixed: A thin adapter per provider with a shared streaming interface, Requests issued in parallel with independent failure, Token counting per request against current published pricing, Difference highlighting, never a scored winner, Local, with per-conversation provider choice. - Implement milestone by milestone from `MILESTONES.md`; keep each change reviewable and leave the application runnable at every commit. - Treat authentication, payments, encryption, imports, webhooks and destructive actions as high-risk boundaries when present. - Never invent cryptography or silently weaken a requirement to make a check pass. - Put every external service behind an interface with a deterministic fake for tests. - Add migrations and rollback or recovery notes for every persistent data change. - Log useful operational context without credentials, tokens, passwords or personal data. - Update documentation and run every check before completing a milestone. ## Known traps - Waiting for all providers before showing anything makes the whole interface as slow as the slowest one, which is often several times the fastest. Stream each independently and let them finish at their own pace, showing which are still running. - Hard-coding prices means the figures silently become wrong when a provider changes them, and a tool whose cost display is wrong is worse than one with no cost display, because it is trusted. Keep pricing as dated data with a checked date and flag entries that have gone stale. - Any score, badge or ordering that suggests one answer is better will be read as a verdict, and there is no basis for one. Show the differences and let the reader judge, and if you cannot resist the urge to rank, remember the ranking would be produced by another model with the same limitations. - Sending one prompt to four providers costs four times as much, which is obvious and still surprises people at the end of the month, particularly with long context. Show the multiplied cost before sending, not after. - A provider key in a log or an error report is a credential someone can spend your money with, and it gets there through a well-meaning debug line printing a request. Redact at the logging boundary and assert in the suite that no key material can reach a log. ===== MILESTONES.md ===== # Delivery milestones · Admix Estimated effort: **weekend** for the indie phases; the production-only milestones add the trust and operability layer. ## M1 · Several providers, one interface One prompt streams from three providers concurrently. ### Steps 1. One adapter per provider Thin. Each provider's streaming format and error shape differ. Files: `src/providers/` 2. Stream independently The slowest provider should not set the pace for the others. Files: `src/stream.ts` 3. Distinguish refusal from error They mean different things and conflating them hides useful information. Files: `src/normalise.ts` ### Done when - [ ] Three providers stream concurrently - [ ] One failing provider does not affect others - [ ] Streaming formats are normalised - [ ] Refusals are distinct from errors ### Watch out - Waiting for all providers before showing anything makes the whole interface as slow as the slowest one, which is often several times the fastest. Stream each independently and let them finish at their own pace, showing which are still running. ## M2 · Knowing what it cost Every request shows its actual cost against current pricing. ### Steps 1. Count tokens per request Both directions. Output is usually the expensive half. Files: `src/tokens.ts` 2. Keep pricing as dated data Hard-coded prices go quietly wrong and are trusted anyway. Files: `config/pricing.json` 3. Surface stale pricing A cost display that is wrong is worse than none. Files: `src/pricing.ts` ### Done when - [ ] Each response shows tokens and cost - [ ] Pricing is dated with a source - [ ] Stale pricing entries are surfaced - [ ] Conversation totals match their responses ### Watch out - Hard-coding prices means the figures silently become wrong when a provider changes them, and a tool whose cost display is wrong is worse than one with no cost display, because it is trusted. Keep pricing as dated data with a checked date and flag entries that have gone stale. ## M3 · Showing difference without implying a winner You can see where models disagree and are not told who is right. ### Steps 1. Distinguish substantive from wording differences Wording differs always. Substance differing is the interesting case. Files: `src/diff.ts` 2. State agreement plainly They agree. That is the whole claim. Files: `src/compare.ts` 3. Provide no ranking anywhere Its absence is the design. A score would be invented. Files: `tests/no-ranking.test.ts` ### Done when - [ ] Substantive and wording differences are distinguished - [ ] Agreement is stated without implying correctness - [ ] No ranking or score exists - [ ] Factual disagreements are prominent ### Watch out - Any score, badge or ordering that suggests one answer is better will be read as a verdict, and there is no basis for one. Show the differences and let the reader judge, and if you cannot resist the urge to rank, remember the ranking would be produced by another model with the same limitations. ## M4 · Spending that cannot run away A budget stops requests before the bill surprises you. ### Steps 1. Check the budget before issuing Refusing is the point. A warning after the fact is a receipt. Files: `src/budget.ts` 2. Show the multiplied cost before sending Four providers is four times. Obvious and still surprising. Files: `src/estimate.ts` 3. Make raising the limit deliberate And recorded, so the pattern is visible to you. Files: `src/budget.ts` ### Done when - [ ] The budget refuses rather than warns - [ ] Multiplied cost is shown before sending - [ ] Remaining budget is continuously visible - [ ] Raising the limit is deliberate and recorded ### Watch out - Sending one prompt to four providers costs four times as much, which is obvious and still surprises people at the end of the month, particularly with long context. Show the multiplied cost before sending, not after. ## M5 · Keys, storage and getting out (production only) Your keys are safe, your conversations are yours, and both can leave. ### Steps 1. Encrypt keys and redact at the boundary A key in a log is money someone else can spend. Files: `src/keys.ts`, `tests/no-key-in-logs.test.ts` 2. Keep conversations local and searchable They contain whatever you asked about, which is a lot. Files: `src/search.ts` 3. Export with provider and cost attribution Which model said what, and what it cost. Files: `src/export.ts` ### Done when - [ ] Keys are encrypted and never reach logs - [ ] Conversations are searchable locally - [ ] Export attributes responses to providers with costs - [ ] Deleting removes responses and cost records ### Watch out - A provider key in a log or an error report is a credential someone can spend your money with, and it gets there through a well-meaning debug line printing a request. Redact at the logging boundary and assert in the suite that no key material can reach a log. ===== OPERATIONS.md ===== # Operations · Admix ## Backup Conversations are the data and they contain whatever you asked about, which is often a lot. Back them up encrypted, and keep provider keys out of the backup entirely rather than encrypted within it. ## Restore Restore and confirm cost records survived alongside their responses, since a restore that loses costs makes the budget figures wrong in a way that is invisible until the bill arrives. Do a restore drill before the first real user, and write the date here when it passes. ## Monitoring Spend against budget is the number, and stale pricing entries are the maintenance. Both go wrong quietly, which is why the recheck window exists. ## Incident checklist A provider key appearing in a log is the incident, since it is spendable by whoever reads it. The suite assertion is the control, and a debug line printing a full request is the usual cause. 1. Contain the issue without destroying evidence or user data. 2. Record the timeline and affected scope. 3. Rotate exposed secrets and revoke compromised sessions or credentials. 4. Restore from a verified backup when needed. 5. Document the root cause, the remediation and the regression test. ## Release gate - [ ] Three providers are confirmed to stream concurrently with independent failure - [ ] Pricing is confirmed dated with stale entries surfaced - [ ] No ranking or score is confirmed to exist - [ ] The budget is confirmed to refuse rather than warn - [ ] A key is confirmed unable to reach a log ## Launch constraint Do not market omitted Admix capabilities as implemented. The non-goals in `PRODUCT.md` remain user-visible limitations until they are deliberately delivered. ===== .env.example ===== # Copy to .env and fill in. Never commit .env; this file documents it. # Required. Conversations and cost records. Contains whatever you asked about. DB_PATH=~/.local/share/admix/chats.db # Required · secret. Encrypts provider keys at rest. Without it they sit in plain text. KEYS_ENCRYPTION_KEY=a keychain reference # Required. Dated pricing per model. The file that keeps the cost display honest. PRICING_PATH=./config/pricing.json # Required. After this a pricing entry is surfaced as stale. PRICING_RECHECK_DAYS=30 # Required. Checked before issuing. Refuses rather than warns. MONTHLY_BUDGET_MINOR=2000 # Required. Present as a constant so the answer is visible. A ranking would be invented. SHOW_RANKING=never
README.md
# Admix · indie build A multi-model chat client where the engineering is concurrent streaming from providers that behave differently, and the design constraint is that side-by-side comparison must not be presented as a form of verification. Estimated effort: **weekend**. Work `BUILD_PLAN.md` top to bottom · every phase ends in a check that has to pass before the next one starts. ## Stack | Part | Choice | Why | | --- | --- | --- | | Providers | A thin adapter per provider with a shared streaming interface | each provider differs in streaming format, errors and rate limits | | Concurrency | Requests issued in parallel with independent failure | one slow provider must not hold up the others | | Cost | Token counting per request against current published pricing | cost is the thing people are most surprised by | | Comparison | Difference highlighting, never a scored winner | agreement is not verification and should not look like it | | Storage | Local, with per-conversation provider choice | what you send to which provider should be a visible decision | ## Before you start Have every one of these ready. The plan assumes them from step one. - [ ] **Node.js 22 or newer** · free - Why: Runs the client and the concurrent streaming. - Get it: nodejs.org, or nvm install 22. - Verify: node --version prints v22 or higher - [ ] **API keys for at least three providers** · per token; budget for development being more expensive than you expect - Why: The comparison is uninteresting with fewer, and provider differences in streaming, errors and refusals only appear across several. - Get it: Accounts with three providers, using the smallest models while developing since every test costs money three times over. - Verify: All three stream a response successfully - [ ] **Current pricing for every model you will use** · free to obtain - Why: The cost display is only useful if it is right, and provider pricing changes without much notice. - Get it: Take current published pricing per model, record it with the date you checked and the source. - Verify: A dated pricing record exists per model with a source link - [ ] **Your spending limit, decided first** · free to decide - Why: This is a category where costs multiply by the number of providers and run away quietly, and the limit is enforced in code. - Get it: Decide a monthly figure you would be comfortable spending before you have sent anything. - Verify: A figure is written down and configured before the first request - [ ] **Ten prompts where you know the right answer** · free - Why: The build declines to judge correctness, but you need to see for yourself how often models agree confidently and wrongly, since that is the reason for the constraint. - Get it: Write ten prompts with verifiable answers, including two where the answer is obscure enough that models are likely to guess. - Verify: At least one prompt produces confident agreement on a wrong answer ## Quick start Follow Phase 1 in `BUILD_PLAN.md`; it creates the project. Then copy `.env.example` to `.env` and fill in the values it documents. ## Honest limits This build deliberately does not replace: - Any judgement about which answer is correct. - Model evaluation or benchmarking. - Automatic routing to the best model. - Self-hosting models. - Admix's bundled access to many providers under one subscription - A maintained model catalog and provider routing - Cross-device sync and account history - Built-in image, search, and document workflows - Usage limits, billing, and support handled for you If one of those is essential to you, that is the reason to keep paying for Admix, and the README should say so rather than pretend.
BRIEF.md
# Build brief · Admix The one-shot brief this plan expands. `BUILD_PLAN.md` (or `MILESTONES.md`) is the same sequence broken into steps and checks; where the two disagree, the plan wins. Build me one prompt sent to several models, compared side by side with honest cost accounting to replace Admix. Build it in phases, in the order below. Do not write the whole thing in one pass. Finish a phase, run its "Done when" check, fix what fails, and only then start the next phase. ### Rule zero There is a trap in this category that the interface encourages: showing three answers side by side implies that comparing them tells you which is right. It does not. Three models agreeing on a wrong fact is a common outcome, particularly when they share training data, and the visual symmetry makes agreement look like verification. So the build does not present agreement as confidence. Where models disagree it says so and shows where; where they agree it says they agree and nothing more. Any claim you intend to rely on still has to be checked against a source, and the interface says that rather than implying otherwise. The genuinely useful things here are cheaper: seeing how differently models interpret an ambiguous prompt, catching one that has gone off, and knowing what each request actually cost. Those are what the build is organised around. ### Stack (fixed, do not substitute) - Providers: A thin adapter per provider with a shared streaming interface · each provider differs in streaming format, errors and rate limits - Concurrency: Requests issued in parallel with independent failure · one slow provider must not hold up the others - Cost: Token counting per request against current published pricing · cost is the thing people are most surprised by - Comparison: Difference highlighting, never a scored winner · agreement is not verification and should not look like it - Storage: Local, with per-conversation provider choice · what you send to which provider should be a visible decision ### Data model (create this before Phase 1) - conversations(id, title, created_at, providers_json) - turns(id, conversation_id, ordinal, prompt, at) - responses(id, turn_id, provider, model, body, finished, error, latency_ms, tokens_in, tokens_out, cost_minor) - pricing(provider, model, input_per_million_minor, output_per_million_minor, effective_from, checked_on) - differences(turn_id, kind, detail_json) · kind is factual_disagreement, refusal_divergence or format_divergence - budgets(period, limit_minor, spent_minor) · because this is a category where costs run away quietly - There is deliberately no column scoring which response was best, because a stored score would be displayed and it would be invented. ### Phase 1 · Several providers, one interface Build: An adapter per provider behind a shared streaming interface. Issue requests in parallel with independent failure and timeout. Normalise the differences in streaming format without hiding differences in behaviour. Trap: Waiting for all providers before showing anything makes the whole interface as slow as the slowest one, which is often several times the fastest. Stream each independently and let them finish at their own pace, showing which are still running. Done when: Three providers stream concurrently and each renders as it arrives. One provider failing or timing out does not affect the others, tested by pointing one at an unreachable endpoint. Streaming differences between providers are normalised. A provider's refusal is shown as a refusal rather than as an error. Do not build yet: No cost, comparison or budgets. ### Phase 2 · Knowing what it cost Build: Count input and output tokens per request and price them against a pricing table with an effective date. Show cost per response and per conversation. Surface pricing entries that have not been rechecked recently. Trap: Hard-coding prices means the figures silently become wrong when a provider changes them, and a tool whose cost display is wrong is worse than one with no cost display, because it is trusted. Keep pricing as dated data with a checked date and flag entries that have gone stale. Done when: Each response shows its token counts and computed cost. Pricing is dated data with a checked date and a source. Stale pricing entries are surfaced rather than used silently. Conversation total cost is shown and matches the sum of its responses. Do not build yet: No comparison or budgets yet. ### Phase 3 · Showing difference without implying a winner Build: Highlight where responses differ substantively rather than only in wording. Present agreement as agreement, not as confidence. Provide no ranking, score or recommended answer anywhere. Trap: Any score, badge or ordering that suggests one answer is better will be read as a verdict, and there is no basis for one. Show the differences and let the reader judge, and if you cannot resist the urge to rank, remember the ranking would be produced by another model with the same limitations. Done when: Substantive differences are highlighted separately from wording differences. Agreement is stated as agreement with no implication of correctness, verified by reading the interface text. No score, ranking or recommended answer exists anywhere, verified by inspection. A factual disagreement is surfaced prominently since it is the most useful signal. Do not build yet: No budgets or export yet. ### Phase 4 · Spending that cannot run away Build: A spending limit per period, checked before issuing requests. Refuse rather than warn when it is reached. Show remaining budget continuously and account for the cost of querying several providers at once. Trap: Sending one prompt to four providers costs four times as much, which is obvious and still surprises people at the end of the month, particularly with long context. Show the multiplied cost before sending, not after. Done when: Reaching the budget refuses new requests rather than warning, tested by exhausting it. The estimated cost of a multi-provider request is shown before sending. Remaining budget is visible continuously. The budget cannot be raised for the current period without an explicit and recorded action. Do not build yet: No export or key handling yet. ### Phase 5 · Keys, storage and getting out Build: Store provider keys encrypted at rest with no path that logs them. Keep conversations locally with search. Export everything including which provider produced each response and what it cost. Trap: A provider key in a log or an error report is a credential someone can spend your money with, and it gets there through a well-meaning debug line printing a request. Redact at the logging boundary and assert in the suite that no key material can reach a log. Done when: Keys are encrypted at rest and a test fails if a key reaches a log, verified by deliberately adding one. Conversations are searchable locally. Export contains prompts, responses, providers, models and costs and re-imports. Deleting a conversation removes its responses and cost records together. Do not build yet: Nothing after this. ### Out of scope (and why) - Any judgement about which answer is correct, which is the feature the layout implies and which cannot be provided honestly. - Model evaluation or benchmarking, which needs a methodology rather than a side-by-side view. - Routing a prompt automatically to the best model, which requires the judgement this build declines to make. - Hosting models yourself, which is a different project entirely. ### README must contain - That agreement between models is not verification and the interface does not present it as one - That there is no ranking, score or recommended answer anywhere, deliberately - That pricing is dated data with a checked date, because hard-coded prices go quietly wrong - That the budget refuses rather than warns, and multiplied cost is shown before sending - That a test fails if a provider key can reach a log
AGENTS.md
# Agent instructions · Admix indie build - Read `README.md` and `BUILD_PLAN.md` before writing code. The stack is fixed: A thin adapter per provider with a shared streaming interface, Requests issued in parallel with independent failure, Token counting per request against current published pricing, Difference highlighting, never a scored winner, Local, with per-conversation provider choice. Do not substitute. - Work one phase at a time, in order. Do not start a phase until every "Done when" item of the previous one passes. - Prefer the fewest moving parts that satisfy the step. No frameworks, services or dependencies the plan does not name. - Secrets live in `.env`, never in source or logs. Keep `.env.example` current when a variable is introduced. - Do not invent cryptography, security guarantees, APIs or compliance claims. - Add a focused test for every destructive, security-sensitive or data-loss path the plan names. - Run the project checks before declaring a phase complete, and record any deliberate shortcut in the README under "Tradeoffs". ## Known traps - Waiting for all providers before showing anything makes the whole interface as slow as the slowest one, which is often several times the fastest. Stream each independently and let them finish at their own pace, showing which are still running. - Hard-coding prices means the figures silently become wrong when a provider changes them, and a tool whose cost display is wrong is worse than one with no cost display, because it is trusted. Keep pricing as dated data with a checked date and flag entries that have gone stale. - Any score, badge or ordering that suggests one answer is better will be read as a verdict, and there is no basis for one. Show the differences and let the reader judge, and if you cannot resist the urge to rank, remember the ranking would be produced by another model with the same limitations. - Sending one prompt to four providers costs four times as much, which is obvious and still surprises people at the end of the month, particularly with long context. Show the multiplied cost before sending, not after.
BUILD_PLAN.md
# Build plan · Admix A multi-model chat client where the engineering is concurrent streaming from providers that behave differently, and the design constraint is that side-by-side comparison must not be presented as a form of verification. Phases are in dependency order. Each ends in a "Done when" list; treat an unticked item as a blocker, not a note. ## Phase 1 · Several providers, one interface One prompt streams from three providers concurrently. ### Steps 1. One adapter per provider Thin. Each provider's streaming format and error shape differ. Files: `src/providers/` 2. Stream independently The slowest provider should not set the pace for the others. Files: `src/stream.ts` 3. Distinguish refusal from error They mean different things and conflating them hides useful information. Files: `src/normalise.ts` ### Done when - [ ] Three providers stream concurrently - [ ] One failing provider does not affect others - [ ] Streaming formats are normalised - [ ] Refusals are distinct from errors ### Watch out - Waiting for all providers before showing anything makes the whole interface as slow as the slowest one, which is often several times the fastest. Stream each independently and let them finish at their own pace, showing which are still running. ## Phase 2 · Knowing what it cost Every request shows its actual cost against current pricing. ### Steps 1. Count tokens per request Both directions. Output is usually the expensive half. Files: `src/tokens.ts` 2. Keep pricing as dated data Hard-coded prices go quietly wrong and are trusted anyway. Files: `config/pricing.json` 3. Surface stale pricing A cost display that is wrong is worse than none. Files: `src/pricing.ts` ### Done when - [ ] Each response shows tokens and cost - [ ] Pricing is dated with a source - [ ] Stale pricing entries are surfaced - [ ] Conversation totals match their responses ### Watch out - Hard-coding prices means the figures silently become wrong when a provider changes them, and a tool whose cost display is wrong is worse than one with no cost display, because it is trusted. Keep pricing as dated data with a checked date and flag entries that have gone stale. ## Phase 3 · Showing difference without implying a winner You can see where models disagree and are not told who is right. ### Steps 1. Distinguish substantive from wording differences Wording differs always. Substance differing is the interesting case. Files: `src/diff.ts` 2. State agreement plainly They agree. That is the whole claim. Files: `src/compare.ts` 3. Provide no ranking anywhere Its absence is the design. A score would be invented. Files: `tests/no-ranking.test.ts` ### Done when - [ ] Substantive and wording differences are distinguished - [ ] Agreement is stated without implying correctness - [ ] No ranking or score exists - [ ] Factual disagreements are prominent ### Watch out - Any score, badge or ordering that suggests one answer is better will be read as a verdict, and there is no basis for one. Show the differences and let the reader judge, and if you cannot resist the urge to rank, remember the ranking would be produced by another model with the same limitations. ## Phase 4 · Spending that cannot run away A budget stops requests before the bill surprises you. ### Steps 1. Check the budget before issuing Refusing is the point. A warning after the fact is a receipt. Files: `src/budget.ts` 2. Show the multiplied cost before sending Four providers is four times. Obvious and still surprising. Files: `src/estimate.ts` 3. Make raising the limit deliberate And recorded, so the pattern is visible to you. Files: `src/budget.ts` ### Done when - [ ] The budget refuses rather than warns - [ ] Multiplied cost is shown before sending - [ ] Remaining budget is continuously visible - [ ] Raising the limit is deliberate and recorded ### Watch out - Sending one prompt to four providers costs four times as much, which is obvious and still surprises people at the end of the month, particularly with long context. Show the multiplied cost before sending, not after. ## Not in this build - Any judgement about which answer is correct. - Model evaluation or benchmarking. - Automatic routing to the best model. - Self-hosting models. ## After v1, if you want it - Saved prompt sets you can rerun across models when a new one appears - Per-provider latency history, which is genuinely useful and honestly measurable - A cheap model first, expensive model on demand flow, once cost accounting is trustworthy
.env.example
# Copy to .env and fill in. Never commit .env; this file documents it. # Required. Conversations and cost records. Contains whatever you asked about. DB_PATH=~/.local/share/admix/chats.db # Required · secret. Encrypts provider keys at rest. Without it they sit in plain text. KEYS_ENCRYPTION_KEY=a keychain reference # Required. Dated pricing per model. The file that keeps the cost display honest. PRICING_PATH=./config/pricing.json # Required. After this a pricing entry is surfaced as stale. PRICING_RECHECK_DAYS=30 # Required. Checked before issuing. Refuses rather than warns. MONTHLY_BUDGET_MINOR=2000 # Required. Present as a constant so the answer is visible. A ranking would be invented. SHOW_RANKING=never
PRODUCT.md
# Admix · product brief ## Problem A local chat UI with side-by-side replies is a weekend project when it calls OpenRouter, but Admix's value depends on access to many hosted models, a current catalog, provider routing, and a maintained account and billing layer. You can reproduce the comparison workflow. You cannot reproduce the model access and service layer for a flat $10 subscription. ## Product outcome A comparison client that streams from several models at once, tells you exactly what each request cost, and never suggests that three models agreeing means they are right. ## Target user A builder who needs a maintainable product foundation, not a one-off demo. ## Required capabilities - Node 22 - OpenRouter API key in .env - SQLite - Provider and model IDs to compare ## Explicit non-goals for v1 - Any judgement about which answer is correct. - Model evaluation or benchmarking. - Automatic routing to the best model. - Self-hosting models. - Admix's bundled access to many providers under one subscription - A maintained model catalog and provider routing - Cross-device sync and account history - Built-in image, search, and document workflows - Usage limits, billing, and support handled for you ## Success criteria - Three providers are confirmed to stream concurrently with independent failure - Pricing is confirmed dated with stale entries surfaced - No ranking or score is confirmed to exist - The budget is confirmed to refuse rather than warn - A key is confirmed unable to reach a log
BRIEF.md
# Build brief · Admix The one-shot brief this plan expands. `BUILD_PLAN.md` (or `MILESTONES.md`) is the same sequence broken into steps and checks; where the two disagree, the plan wins. Build me one prompt sent to several models, compared side by side with honest cost accounting to replace Admix. Build it in phases, in the order below. Do not write the whole thing in one pass. Finish a phase, run its "Done when" check, fix what fails, and only then start the next phase. ### Rule zero There is a trap in this category that the interface encourages: showing three answers side by side implies that comparing them tells you which is right. It does not. Three models agreeing on a wrong fact is a common outcome, particularly when they share training data, and the visual symmetry makes agreement look like verification. So the build does not present agreement as confidence. Where models disagree it says so and shows where; where they agree it says they agree and nothing more. Any claim you intend to rely on still has to be checked against a source, and the interface says that rather than implying otherwise. The genuinely useful things here are cheaper: seeing how differently models interpret an ambiguous prompt, catching one that has gone off, and knowing what each request actually cost. Those are what the build is organised around. ### Stack (fixed, do not substitute) - Providers: A thin adapter per provider with a shared streaming interface · each provider differs in streaming format, errors and rate limits - Concurrency: Requests issued in parallel with independent failure · one slow provider must not hold up the others - Cost: Token counting per request against current published pricing · cost is the thing people are most surprised by - Comparison: Difference highlighting, never a scored winner · agreement is not verification and should not look like it - Storage: Local, with per-conversation provider choice · what you send to which provider should be a visible decision ### Data model (create this before Phase 1) - conversations(id, title, created_at, providers_json) - turns(id, conversation_id, ordinal, prompt, at) - responses(id, turn_id, provider, model, body, finished, error, latency_ms, tokens_in, tokens_out, cost_minor) - pricing(provider, model, input_per_million_minor, output_per_million_minor, effective_from, checked_on) - differences(turn_id, kind, detail_json) · kind is factual_disagreement, refusal_divergence or format_divergence - budgets(period, limit_minor, spent_minor) · because this is a category where costs run away quietly - There is deliberately no column scoring which response was best, because a stored score would be displayed and it would be invented. ### Phase 1 · Several providers, one interface Build: An adapter per provider behind a shared streaming interface. Issue requests in parallel with independent failure and timeout. Normalise the differences in streaming format without hiding differences in behaviour. Trap: Waiting for all providers before showing anything makes the whole interface as slow as the slowest one, which is often several times the fastest. Stream each independently and let them finish at their own pace, showing which are still running. Done when: Three providers stream concurrently and each renders as it arrives. One provider failing or timing out does not affect the others, tested by pointing one at an unreachable endpoint. Streaming differences between providers are normalised. A provider's refusal is shown as a refusal rather than as an error. Do not build yet: No cost, comparison or budgets. ### Phase 2 · Knowing what it cost Build: Count input and output tokens per request and price them against a pricing table with an effective date. Show cost per response and per conversation. Surface pricing entries that have not been rechecked recently. Trap: Hard-coding prices means the figures silently become wrong when a provider changes them, and a tool whose cost display is wrong is worse than one with no cost display, because it is trusted. Keep pricing as dated data with a checked date and flag entries that have gone stale. Done when: Each response shows its token counts and computed cost. Pricing is dated data with a checked date and a source. Stale pricing entries are surfaced rather than used silently. Conversation total cost is shown and matches the sum of its responses. Do not build yet: No comparison or budgets yet. ### Phase 3 · Showing difference without implying a winner Build: Highlight where responses differ substantively rather than only in wording. Present agreement as agreement, not as confidence. Provide no ranking, score or recommended answer anywhere. Trap: Any score, badge or ordering that suggests one answer is better will be read as a verdict, and there is no basis for one. Show the differences and let the reader judge, and if you cannot resist the urge to rank, remember the ranking would be produced by another model with the same limitations. Done when: Substantive differences are highlighted separately from wording differences. Agreement is stated as agreement with no implication of correctness, verified by reading the interface text. No score, ranking or recommended answer exists anywhere, verified by inspection. A factual disagreement is surfaced prominently since it is the most useful signal. Do not build yet: No budgets or export yet. ### Phase 4 · Spending that cannot run away Build: A spending limit per period, checked before issuing requests. Refuse rather than warn when it is reached. Show remaining budget continuously and account for the cost of querying several providers at once. Trap: Sending one prompt to four providers costs four times as much, which is obvious and still surprises people at the end of the month, particularly with long context. Show the multiplied cost before sending, not after. Done when: Reaching the budget refuses new requests rather than warning, tested by exhausting it. The estimated cost of a multi-provider request is shown before sending. Remaining budget is visible continuously. The budget cannot be raised for the current period without an explicit and recorded action. Do not build yet: No export or key handling yet. ### Phase 5 · Keys, storage and getting out Build: Store provider keys encrypted at rest with no path that logs them. Keep conversations locally with search. Export everything including which provider produced each response and what it cost. Trap: A provider key in a log or an error report is a credential someone can spend your money with, and it gets there through a well-meaning debug line printing a request. Redact at the logging boundary and assert in the suite that no key material can reach a log. Done when: Keys are encrypted at rest and a test fails if a key reaches a log, verified by deliberately adding one. Conversations are searchable locally. Export contains prompts, responses, providers, models and costs and re-imports. Deleting a conversation removes its responses and cost records together. Do not build yet: Nothing after this. ### Out of scope (and why) - Any judgement about which answer is correct, which is the feature the layout implies and which cannot be provided honestly. - Model evaluation or benchmarking, which needs a methodology rather than a side-by-side view. - Routing a prompt automatically to the best model, which requires the judgement this build declines to make. - Hosting models yourself, which is a different project entirely. ### README must contain - That agreement between models is not verification and the interface does not present it as one - That there is no ranking, score or recommended answer anywhere, deliberately - That pricing is dated data with a checked date, because hard-coded prices go quietly wrong - That the budget refuses rather than warns, and multiplied cost is shown before sending - That a test fails if a provider key can reach a log
ARCHITECTURE.md
# Architecture · Admix ## Stack | Part | Choice | Why | | --- | --- | --- | | Providers | A thin adapter per provider with a shared streaming interface | each provider differs in streaming format, errors and rate limits | | Concurrency | Requests issued in parallel with independent failure | one slow provider must not hold up the others | | Cost | Token counting per request against current published pricing | cost is the thing people are most surprised by | | Comparison | Difference highlighting, never a scored winner | agreement is not verification and should not look like it | | Storage | Local, with per-conversation provider choice | what you send to which provider should be a visible decision | ## Modules Each module has one owner concern and a documented way to replace it. | Module | Owns | How to replace it | | --- | --- | --- | | Providers | Adapters behind one streaming interface. | Any provider, provided refusals stay distinguishable from errors. | | Cost | Token counting against dated pricing. | Any implementation, provided pricing is data with a checked date. | | Comparison | Difference highlighting without ranking. | Any diffing, provided no score or recommended answer is produced. | | Budget | Pre-request refusal. | Any implementation, provided it refuses rather than warning. | | Keys | Encrypted storage and log redaction. | Any store, provided the log assertion stays in the suite. | ## Configuration Every runtime setting is an environment variable documented in `.env.example`, validated at startup, with a safe local default wherever one exists. - `DB_PATH` · required · Conversations and cost records. Contains whatever you asked about. - `KEYS_ENCRYPTION_KEY` · required, secret · Encrypts provider keys at rest. Without it they sit in plain text. - `PRICING_PATH` · required · Dated pricing per model. The file that keeps the cost display honest. - `PRICING_RECHECK_DAYS` · required · After this a pricing entry is surfaced as stale. - `MONTHLY_BUDGET_MINOR` · required · Checked before issuing. Refuses rather than warns. - `SHOW_RANKING` · required · Present as a constant so the answer is visible. A ranking would be invented. ## Production baseline - Security: least privilege, input validation at every boundary, secret redaction in logs, rate limits on abuse-prone paths, no invented security primitives. - Data: explicit schema and migrations, transactional writes where integrity matters, backup and restore procedures that have been exercised. - Integrations: adapters around third-party providers, idempotent webhook or job processing, bounded retries, timeouts. - Observability: structured logs with request or operation ids, an error-tracking hook, and health and readiness checks where a server exists. - Quality: unit tests for domain rules, integration tests at module boundaries, one end-to-end test of the critical path. ## Decision records For each dependency in the stack table, keep a short note: why it was chosen, its failure mode, and how it is replaced. Do not add infrastructure until a requirement in `PRODUCT.md` justifies it.
AGENTS.md
# Agent instructions · Admix product build - Read `PRODUCT.md` and `ARCHITECTURE.md` before changing code. The stack is fixed: A thin adapter per provider with a shared streaming interface, Requests issued in parallel with independent failure, Token counting per request against current published pricing, Difference highlighting, never a scored winner, Local, with per-conversation provider choice. - Implement milestone by milestone from `MILESTONES.md`; keep each change reviewable and leave the application runnable at every commit. - Treat authentication, payments, encryption, imports, webhooks and destructive actions as high-risk boundaries when present. - Never invent cryptography or silently weaken a requirement to make a check pass. - Put every external service behind an interface with a deterministic fake for tests. - Add migrations and rollback or recovery notes for every persistent data change. - Log useful operational context without credentials, tokens, passwords or personal data. - Update documentation and run every check before completing a milestone. ## Known traps - Waiting for all providers before showing anything makes the whole interface as slow as the slowest one, which is often several times the fastest. Stream each independently and let them finish at their own pace, showing which are still running. - Hard-coding prices means the figures silently become wrong when a provider changes them, and a tool whose cost display is wrong is worse than one with no cost display, because it is trusted. Keep pricing as dated data with a checked date and flag entries that have gone stale. - Any score, badge or ordering that suggests one answer is better will be read as a verdict, and there is no basis for one. Show the differences and let the reader judge, and if you cannot resist the urge to rank, remember the ranking would be produced by another model with the same limitations. - Sending one prompt to four providers costs four times as much, which is obvious and still surprises people at the end of the month, particularly with long context. Show the multiplied cost before sending, not after. - A provider key in a log or an error report is a credential someone can spend your money with, and it gets there through a well-meaning debug line printing a request. Redact at the logging boundary and assert in the suite that no key material can reach a log.
MILESTONES.md
# Delivery milestones · Admix Estimated effort: **weekend** for the indie phases; the production-only milestones add the trust and operability layer. ## M1 · Several providers, one interface One prompt streams from three providers concurrently. ### Steps 1. One adapter per provider Thin. Each provider's streaming format and error shape differ. Files: `src/providers/` 2. Stream independently The slowest provider should not set the pace for the others. Files: `src/stream.ts` 3. Distinguish refusal from error They mean different things and conflating them hides useful information. Files: `src/normalise.ts` ### Done when - [ ] Three providers stream concurrently - [ ] One failing provider does not affect others - [ ] Streaming formats are normalised - [ ] Refusals are distinct from errors ### Watch out - Waiting for all providers before showing anything makes the whole interface as slow as the slowest one, which is often several times the fastest. Stream each independently and let them finish at their own pace, showing which are still running. ## M2 · Knowing what it cost Every request shows its actual cost against current pricing. ### Steps 1. Count tokens per request Both directions. Output is usually the expensive half. Files: `src/tokens.ts` 2. Keep pricing as dated data Hard-coded prices go quietly wrong and are trusted anyway. Files: `config/pricing.json` 3. Surface stale pricing A cost display that is wrong is worse than none. Files: `src/pricing.ts` ### Done when - [ ] Each response shows tokens and cost - [ ] Pricing is dated with a source - [ ] Stale pricing entries are surfaced - [ ] Conversation totals match their responses ### Watch out - Hard-coding prices means the figures silently become wrong when a provider changes them, and a tool whose cost display is wrong is worse than one with no cost display, because it is trusted. Keep pricing as dated data with a checked date and flag entries that have gone stale. ## M3 · Showing difference without implying a winner You can see where models disagree and are not told who is right. ### Steps 1. Distinguish substantive from wording differences Wording differs always. Substance differing is the interesting case. Files: `src/diff.ts` 2. State agreement plainly They agree. That is the whole claim. Files: `src/compare.ts` 3. Provide no ranking anywhere Its absence is the design. A score would be invented. Files: `tests/no-ranking.test.ts` ### Done when - [ ] Substantive and wording differences are distinguished - [ ] Agreement is stated without implying correctness - [ ] No ranking or score exists - [ ] Factual disagreements are prominent ### Watch out - Any score, badge or ordering that suggests one answer is better will be read as a verdict, and there is no basis for one. Show the differences and let the reader judge, and if you cannot resist the urge to rank, remember the ranking would be produced by another model with the same limitations. ## M4 · Spending that cannot run away A budget stops requests before the bill surprises you. ### Steps 1. Check the budget before issuing Refusing is the point. A warning after the fact is a receipt. Files: `src/budget.ts` 2. Show the multiplied cost before sending Four providers is four times. Obvious and still surprising. Files: `src/estimate.ts` 3. Make raising the limit deliberate And recorded, so the pattern is visible to you. Files: `src/budget.ts` ### Done when - [ ] The budget refuses rather than warns - [ ] Multiplied cost is shown before sending - [ ] Remaining budget is continuously visible - [ ] Raising the limit is deliberate and recorded ### Watch out - Sending one prompt to four providers costs four times as much, which is obvious and still surprises people at the end of the month, particularly with long context. Show the multiplied cost before sending, not after. ## M5 · Keys, storage and getting out (production only) Your keys are safe, your conversations are yours, and both can leave. ### Steps 1. Encrypt keys and redact at the boundary A key in a log is money someone else can spend. Files: `src/keys.ts`, `tests/no-key-in-logs.test.ts` 2. Keep conversations local and searchable They contain whatever you asked about, which is a lot. Files: `src/search.ts` 3. Export with provider and cost attribution Which model said what, and what it cost. Files: `src/export.ts` ### Done when - [ ] Keys are encrypted and never reach logs - [ ] Conversations are searchable locally - [ ] Export attributes responses to providers with costs - [ ] Deleting removes responses and cost records ### Watch out - A provider key in a log or an error report is a credential someone can spend your money with, and it gets there through a well-meaning debug line printing a request. Redact at the logging boundary and assert in the suite that no key material can reach a log.
OPERATIONS.md
# Operations · Admix ## Backup Conversations are the data and they contain whatever you asked about, which is often a lot. Back them up encrypted, and keep provider keys out of the backup entirely rather than encrypted within it. ## Restore Restore and confirm cost records survived alongside their responses, since a restore that loses costs makes the budget figures wrong in a way that is invisible until the bill arrives. Do a restore drill before the first real user, and write the date here when it passes. ## Monitoring Spend against budget is the number, and stale pricing entries are the maintenance. Both go wrong quietly, which is why the recheck window exists. ## Incident checklist A provider key appearing in a log is the incident, since it is spendable by whoever reads it. The suite assertion is the control, and a debug line printing a full request is the usual cause. 1. Contain the issue without destroying evidence or user data. 2. Record the timeline and affected scope. 3. Rotate exposed secrets and revoke compromised sessions or credentials. 4. Restore from a verified backup when needed. 5. Document the root cause, the remediation and the regression test. ## Release gate - [ ] Three providers are confirmed to stream concurrently with independent failure - [ ] Pricing is confirmed dated with stale entries surfaced - [ ] No ranking or score is confirmed to exist - [ ] The budget is confirmed to refuse rather than warn - [ ] A key is confirmed unable to reach a log ## Launch constraint Do not market omitted Admix capabilities as implemented. The non-goals in `PRODUCT.md` remain user-visible limitations until they are deliberately delivered.
.env.example
# Copy to .env and fill in. Never commit .env; this file documents it. # Required. Conversations and cost records. Contains whatever you asked about. DB_PATH=~/.local/share/admix/chats.db # Required · secret. Encrypts provider keys at rest. Without it they sit in plain text. KEYS_ENCRYPTION_KEY=a keychain reference # Required. Dated pricing per model. The file that keeps the cost display honest. PRICING_PATH=./config/pricing.json # Required. After this a pricing entry is surfaced as stale. PRICING_RECHECK_DAYS=30 # Required. Checked before issuing. Refuses rather than warns. MONTHLY_BUDGET_MINOR=2000 # Required. Present as a constant so the answer is visible. A ranking would be invented. SHOW_RANKING=never
$ choose a build depth, inspect the files, then open the complete pack in your agent
They pay for one bill and one interface instead of managing several provider accounts, API keys, model IDs, and usage limits. The comparison grid is easy to copy; keeping model access and the account, billing, and storage layer working is the service.
xAdmix's bundled access to many providers under one subscription
xA maintained model catalog and provider routing
xCross-device sync and account history
xBuilt-in image, search, and document workflows
xUsage limits, billing, and support handled for you
share on X ↗"I just replaced Admix ($10/mo) with one prompt"
Don't feel like building it? These folks already made it free.
Cherry StudioA desktop model cockpit with parallel answers, files, agents, and rather more settings than restraint.open source↗
Cherry StudioSame prompt, several models, side by side; bring API keys or make local models do the arguing.open source↗
ChatALLThe exact compare-everyone desktop app; use API-backed bots, because scraped web connectors break whenever vendors sneeze.open source↗all 5 free alternatives to Admix →· no votes, no pay-to-list · just what's real
Admix pricing
| plan | monthly | annual (per mo) | what you get |
|---|---|---|---|
| free | $0 | $0 | 20 credits/day; Gemini 2.5 Flash access. |
| starter | $10 | $8 | 500 credits/month; 350+ models; 10 cross-chat memories. |
| basic | $19 | $15.20 | 1,500 credits/month; 350+ models; 30 cross-chat memories. |
| pro | $39 | $31.20 | 4,000 credits/month; 350+ models; 100 cross-chat memories; per-model memory. |
| max | $99 | $79.20 | 10,000 credits/month; 350+ models; 100 cross-chat memories. |
free tier20 credits/day; Gemini 2.5 Flash access.
billingmonthly + annual (-20%); paid plans begin with a $1.99 7-day trial that auto-renews
hidden costsCredits are model-dependent and multi-model/parallel comparisons consume multiple allocations; the paid trial auto-renews.
verified 2026-08-12 · source ↗
Vibecode Admix
Kinda. The core of Admix is buildable in a weekend with the prompt on this page, but there are real gaps: Admix's bundled access to many providers under one subscription, A maintained model catalog and provider routing. Read the honest list above before committing.
How much does Admix cost?
Admix costs about $10/month (Starter, checked 2026-08-03), which is $120 per year.
What do I lose by replacing Admix?
Honestly: Admix's bundled access to many providers under one subscription; A maintained model catalog and provider routing; Cross-device sync and account history; Built-in image, search, and document workflows; Usage limits, billing, and support handled for you. If any of those are load-bearing for you, keep paying.
Is there an open-source alternative to Admix?
Yes: Cherry Studio (A desktop model cockpit with parallel answers, files, agents, and rather more settings than restraint.) Cherry Studio (Same prompt, several models, side by side; bring API keys or make local models do the arguing.) ChatALL (The exact compare-everyone desktop app; use API-backed bots, because scraped web connectors break whenever vendors sneeze.) All 5 curated free alternatives are at vibecodeit.com/admix/alternatives. The prompt is for when you want it exactly your way.