Build Dailygram

KINDAreplaces $5/mosaves $60/yrback to the verdict

0%0 of 17 items done

Saved on this device only. Tick prerequisites first, then work the phases in order · do not start one until the checks above it pass.

A personal digest pipeline whose hard problems are deduplicating items across feeds that rewrite their own identifiers, summarising without inventing anything, and sending exactly one digest a day even when the mail server times out.

estimated effort weekendthe files for this build are in the project pack

RuntimeNode 22 with Express and one scheduler in the same processStorageSQLite through better-sqlite3, with the digest state machine in the schemaIngestionRSS, Atom and JSON feeds that you supply, fetched with conditional requestsSummarisingOne model call per digest over new items only, with a daily cost ledgerDeliverySMTP through a provider you already have, one message per digest

Before step 1

Everything below is assumed from the first step. Tick each one when you actually have it, not when you plan to.

installfree
Why Runs the poller, the scheduler and the localhost dashboard in one process.
Get it nodejs.org, or nvm install 22. open ↗
Verify node --version prints v22 or higher
have readya few pounds a month, or free if you own the box
Why The digest fires on a schedule, and a laptop that sleeps turns a daily digest into a weekly surprise.
Get it A small server, a home box or a single-board computer.
Verify A digest scheduled for seven in the morning has arrived by breakfast
have readyfree, or whatever a feed service charges
Why This is the actual constraint on the whole product, and discovering that a network publishes none is better done before you build than after.
Get it Check each profile for an RSS, Atom or JSON feed, or a service that publishes one for it within that network's terms.
Verify Each URL parses and returns recent entries
API keypennies a day at a few dozen items
Why Summarising is the only external cost and it needs a ceiling from day one.
Get it Create a key with its own spend limit rather than reusing one you use elsewhere.
Verify A one item summary returns and the ledger records the tokens
API keyfree tier is ample for one message a day
Why Sending your own mail from a home address is a separate project in deliverability.
Get it Any transactional provider with a verified sending domain.
Verify A test digest arrives and the provider reports it accepted

Data model

Create these before the first phase that stores anything. Changing a table later is the expensive kind of change.

- sources(id, network, handle, profile_url, feed_url, feed_format, etag, last_modified, enabled, consecutive_failures, last_ok_at, paused_reason)
- items(id, source_id, remote_id, canonical_url, url_hash, content_hash, title, body_text, posted_at, first_seen_at)
- summaries(item_id, model, prompt_version, text, tokens_in, tokens_out, created_at) · keyed by item, so a rebuilt digest never pays twice
- digests(id, window_start, window_end, state, item_count, built_at, sent_at, message_id, last_error) · state is building, ready, sending, sent, needs_check or failed
- digest_items(digest_id, item_id, ordinal)
- cost_ledger(day, tokens_in, tokens_out, calls) · one row per day, checked against a ceiling before every call
- Identity is a ladder: remote identifier first, then canonical URL, then a hash of the text, because feeds rewrite their own identifiers and republish old posts more often than you would believe.

Environment variables

These go in a .env file the app reads at startup. The pack's .env.example is this table as a file · copy it, never commit the filled-in version.

Variable	Needed	Example	Where the value comes from
`DATABASE_PATH`	required	`./data/digest.db`	Sources, items, summaries and digests. The archive.
`POLL_INTERVAL_MINUTES`	required	`360`	Per source, subject to per-source backoff after failures.
`DIGEST_HOUR_LOCAL`	required	`07`	One window per day, named by date in this timezone.
`LLM_API_KEY`secret	required	`sk-...redacted`	Summarising only. Give it its own spend limit at the provider.
`DAILY_TOKEN_CEILING`	required	`120000`	Checked before every call. Refusing beats overspending.
`SMTP_URL`secret	required	`smtps://user:redacted@smtp.example:465`	Delivery. The returned message identifier is stored.
`DIGEST_TO`	required	`you@example.com`	One recipient. This is a personal digest, not a list.
`SOURCE_FAIL_PAUSE`	required	`5`	Consecutive failures before a source pauses and alerts once.

The build, in order

1
Sources are feeds, and feeds are checked when you add them
Adding a source means giving a feed URL that is fetched and parsed there and then.
1. The moment to discover a dead URL is while the person is looking at it.
  Files src/sources.jssrc/feeds/parse.js
2. Store what the server gave you and hand it back next time.
  Files src/feeds/fetch.jstests/conditional.test.js
3. A feed that is gone should go quiet, not louder.
  Files src/poller.js
done when · tick each as it passes
watch out
- Designing the add form around a handle, on the assumption that a handle can be turned into a feed, produces a product that works for nothing. There is no general API for reading a person's public posts on most networks: Meta deprecated the Instagram Basic Display API on 4 December 2024 and its replacements cover professional accounts the app owner controls, not arbitrary public profiles. Ask for a feed URL, validate it on save, and say plainly when a network has none.
next phase
2
The same post, twice, from three angles
An item already seen never appears in a second digest, whatever the feed does to its identifiers.
1. Seen, not sent. This is the difference between a tool and a spam incident.
  Files src/ingest.jstests/first-poll.test.js
2. Identifier, then canonical URL, then content hash.
  Files src/dedupe.jstests/dedupe.test.js
3. Two links to the same post differ only in the campaign tag.
  Files src/canonical.js
done when · tick each as it passes
watch out
- The first poll of a feed that carries a year of archive turns into a digest of three hundred posts, and the reaction to that email is to delete the tool. Watermark on the first poll: insert every item as already seen, and start the digest from the next one. The mirror of this trap is a feed that regenerates its identifiers on republish, which without the URL and content fallbacks resends last week as new.
next phase
3
Summaries that cannot invent a post
Every line in the digest points at an item that exists in your database.
1. An unattributable sentence is a hallucination with good grammar.
  Files src/summarise.jstests/citation.test.js
2. Model output never becomes a URL.
  Files src/render/digest.js
3. A ceiling you enforce beats a bill you discover.
  Files src/costs.jstests/ceiling.test.js
done when · tick each as it passes
watch out
- Asking for a nice five bullet overview and pasting the result into the email produces bullets about posts that do not exist, phrased confidently, and you only notice when you click one and there is nothing there. Require an item id per line, discard lines whose id you did not send, and render the link from your own database rather than from anything the model wrote.
next phase
4
One digest a day, once
A crash or a mail server timeout never produces two copies of the same digest.
1. The claim happens before any network call. That is the whole protection.
  Files src/digest.jstests/exactly-once.test.js
2. It is the only proof that accepted actually happened.
  Files src/mailer.js
3. Never as a reason to send again.
  Files src/reconcile.js
done when · tick each as it passes
watch out
- Retrying because the SMTP call timed out sends the digest twice whenever the server accepted the first one and the acknowledgement was lost, and the second copy arrives at four in the morning. Claim the window row first, treat an unknown outcome as a question, and only resend a digest whose row says it was never accepted.
5
Knowing when a feed quietly diedproduct builder
A broken source is visible within a day rather than showing up as a slowly emptying digest.
1. Last success, failure count, last error, items this week.
  Files src/views/dashboard.ejs
2. A repeated alert is an alert nobody reads.
  Files src/health.jstests/pause.test.js
3. Silence is the failure mode that does not raise an error.
  Files src/health.js
done when · tick each as it passes
watch out
- A feed that starts returning an error page with a 200 status parses as zero entries, so the source never fails, the digest just gets shorter, and three weeks later you realise you stopped hearing from the one person you cared about. Track items per source per week and flag a source whose rate falls to nothing, not only one that errors.

what this build does not replace

Any login, cookie or page scraping.
Feed discovery from a handle.
Maintained connectors for closed networks.
Multiple recipients and subscription management.
maintained ingestion across four social networks
resilience to rate limits, blocks, and page changes
automatic profile discovery and source setup
managed email delivery and monitoring

after v1, if you want it

A weekly digest alongside the daily one, reusing the same window machinery
Per-source importance so a quiet account is never crowded out by a busy one
Full text search over the archive, which you already store item by item

Need the files? The project pack on the verdict page hands your agent the whole brief · more social media.