Build Ideabrowser

KINDAreplaces $41.58/mosaves $498.96/yrback to the verdict

0%0 of 17 items done

Saved on this device only. Tick prerequisites first, then work the phases in order · do not start one until the checks above it pass.

A local scanner whose hard problems are collecting from sources at the rate they permit and keeping the raw evidence, recognising the same complaint reposted as one signal rather than ten, and producing briefs where every claim quotes a source and an unknown is allowed to stay unknown.

estimated effort weekendthe files for this build are in the project pack

RuntimeNext.js with TypeScript and SQLite, single user on the loopback interfaceSourcesA registry where each source records its terms and its published rate limitCacheRaw payloads stored with their fetch time and reused rather than re-requestedBriefsOne model call per candidate, every claim carrying a quote from the sourceCostAn estimate before the run and a hard ceiling during it

Before step 1

Everything below is assumed from the first step. Tick each one when you actually have it, not when you plan to.

installfree
Why Runs the application and the collection script.
Get it nodejs.org, or nvm install 22. open ↗
Verify node --version prints v22 or higher
decidefree
Why The registry refuses to enable a source without one, and writing the note is the step that makes you check.
Get it For each source, read its published terms and rate limits and write one sentence on what you may do and how often.
Verify Every enabled source has both notes recorded and the interval configured
API keypennies per brief, multiplied by however many candidates a day produces
Why Brief writing is the one part that needs a model, and the cost is per candidate.
Get it Create a key in your provider's console and put it in .env.
Verify One candidate produces a brief with verified quotes
decidefree
Why The candidate count is decided by the internet rather than by you, so the limit has to be in code.
Get it Pick a figure per run and put it in the environment.
Verify A run configured to exceed it stops and reports what it completed
have readyfree
Why The only way to judge whether the briefs are any good is to read the first batch about something whose reality you can check.
Get it Point the first run at a community you belong to and read every brief it produces.
Verify You can say which briefs are wrong and why

Data model

Create these before the first phase that stores anything. Changing a table later is the expensive kind of change.

- sources(id, name, endpoint, terms_note, rate_limit_note, user_agent, enabled) · a source with no terms note cannot be enabled
- fetches(id, source_id, url, requested_at, status, bytes, raw_path, from_cache) · raw payloads kept so a brief can be rechecked later
- candidates(id, source_id, external_id, url, author_handle, posted_at, title, text, normalised_hash, first_seen_at, seen_count)
- briefs(id, candidate_id, model_id, created_at, problem, target_user, mvp_scope, pricing_idea, market_size, market_size_source) · market size is a cited figure or the word unknown
- claims(brief_id, ordinal, text, quote, candidate_id) · every claim quotes the post it came from
- verdicts(brief_id, verdict, reason, decided_at) · your own judgement, recorded, because an unjudged backlog only grows
- runs(id, started_at, finished_at, fetch_count, candidates_new, briefs_written, estimated_cost, actual_cost)

Environment variables

These go in a .env file the app reads at startup. The pack's .env.example is this table as a file · copy it, never commit the filled-in version.

Variable	Needed	Example	Where the value comes from
`DATABASE_PATH`	required	`./data/ideas.db`	Sources, candidates, briefs and verdicts.
`RAW_CACHE_DIR`	required	`./data/raw`	Raw payloads with their fetch times. The evidence behind every brief.
`SOURCES_FILE`	required	`./sources.json`	Endpoints with their terms and rate limit notes.
`USER_AGENT`	required	`idea-scanner/0.1 (contact: you@example.com)`	Sent on every request. Identify yourself honestly.
`FETCH_MIN_INTERVAL_MS`	required	`2000`	Minimum gap between requests to one source, enforced in the client.
`COST_CEILING`	required	`5.00`	Per run, in your billing currency. Enforced, not remembered.
`RAW_RETENTION_DAYS`	required	`180`	How long payloads are kept. The cleanup reports what it removed.
`ANTHROPIC_API_KEY`secret	required	`sk-ant-...redacted`	Brief writing. Without it the tool still collects and deduplicates.

The build, in order

1
Sources you are allowed to read, cached once
Every fetch is permitted, rate limited, identified and kept.
1. If you cannot write the sentence, you have not read the terms.
  Files src/sources.ts
2. One place, so no code path can accidentally skip it.
  Files src/fetch.tstests/rate-limit.test.ts
3. It is the evidence, and re-fetching it later may not be possible.
  Files src/cache.ts
done when · tick each as it passes
watch out
- Fetching the same listing on every page load while developing turns a few hundred requests into a few thousand in an afternoon, and the first sign of trouble is a blanket block that also ends your reading of the site as a human. Cache raw payloads, enforce a minimum interval in the client rather than in your habits, and identify the tool honestly.
next phase
2
One complaint is one signal
The same thing said repeatedly is recognised as recurrence, not as volume.
1. Case, whitespace, punctuation and links, stated in the documentation.
  Files src/dedupe.tsdocs/normalisation.md
2. Recurrence is the signal, and a row count is not.
  Files src/candidates.ts
3. Two dates and a number make a feed honest.
  Files src/components/Candidate.tsx
done when · tick each as it passes
watch out
- Treating every post as a fresh candidate turns one grumble reposted in five places into five ideas, and the resulting feed looks like a trend when it is one person having a bad week. Normalise and hash the text, increment a seen count, and let recurrence be a number with dates behind it.
next phase
3
Briefs that cite, or say unknown
No sentence in a brief exists without a quote behind it.
1. A substring check, run before the row is written.
  Files src/briefs/verify.tstests/quote-check.test.ts
2. Enforced by the column and by validation, not by the prompt.
  Files src/briefs/schema.ts
3. So reading the brief is reading the evidence.
  Files src/components/Brief.tsx
done when · tick each as it passes
watch out
- A model asked to size a market will produce a plausible number with a currency symbol on it, and that number will be repeated in a pitch deck three weeks later by somebody who has forgotten where it came from. Make the field accept only a citation or the word unknown, and reject anything else before it is saved.
next phase
4
A backlog you can actually get through
Every brief ends up judged rather than scrolled past.
1. One sentence, and the backlog starts shrinking.
  Files src/verdicts.ts
2. A queue you can see the length of is a queue you finish.
  Files src/app/page.tsx
3. A brief built on a two year old post is a different thing.
  Files src/search.ts
done when · tick each as it passes
watch out
- An endless feed with a star button produces four hundred starred items and no decisions, which is the same state you were in before you built anything. Require a written reason with each verdict and put the unjudged count on the first screen, so the backlog is visibly a queue rather than a library.
5
What it cost, and what it is honestly worthproduct builder
Spending is bounded, evidence has a retention, and the tool states its own limits.
1. The multiplication before the money, every time.
  Files src/cost.ts
2. A brief without its evidence is worth nothing outside the tool.
  Files src/export.ts
3. In the interface, where somebody reading a brief will see them.
  Files src/app/limits/page.tsx
done when · tick each as it passes
watch out
- A run that briefs every candidate it finds costs whatever the feed happened to produce that day, which is fine on a quiet Tuesday and unpleasant after something went viral. Estimate from the candidate count before spending, confirm, and stop at the ceiling with a report of what was completed.

what this build does not replace

Editorial curation.
A historical trend database.
Paid trend data feeds.
Anything behind a login.
1,000+ already-curated and human-vetted ideas
historical trend database and idea archive
AI Research Agent for arbitrary custom ideas
community, coaching and Q&A strategist sessions
data exports and idea-suggest personalization

after v1, if you want it

A weekly digest of new candidates with high recurrence, which is a query and a Markdown file
A second source in a different community, once the deduplication rules have proven themselves
Tracking which pursued ideas went anywhere, which is the only feedback loop that would improve the briefs

Need the files? The project pack on the verdict page hands your agent the whole brief · more user research.