Vibecode Grist
track this build5 phases, 15 steps, beginner friendly0%KINDA · weekend project
The core loop is buildable, but a dependable replacement becomes a real weekend or multi-day project. For Grist, combine spreadsheet formulas with relational records in a small self-hosted workspace. The hard boundary is mature formula engine, access rules, collaboration, integrations, and hosting, plus data model flexibility, collaboration, and integrations.
the project pack5 files · written for this build, step by step
You are building a lean indie version of Grist. Create the following project files first, then implement the application by following them. Keep the files updated as decisions change. Do not collapse this into a single README or prompt. ===== README.md ===== # Grist · indie build Tables with typed columns and formulas that reference records rather than cell addresses, several views over one table, an access rule layer that filters rows and hides columns on the server, and an API that respects the same rules as the interface. Estimated effort: **multi-day**. Work `BUILD_PLAN.md` top to bottom · every phase ends in a check that has to pass before the next one starts. ## Stack | Part | Choice | Why | | --- | --- | --- | | Runtime | Python 3.12 with PostgreSQL | records are relational and access rules become query conditions, which is what a database is for | | Formulas | A small expression language over records, with a written grammar | referencing records rather than cell addresses is what makes formulas survive row reordering | | Access | Rules compiled into query conditions, applied server side | the whole premise; a client-side filter is not access control | | Views | Grid, card and filtered views as queries over one table | copies diverge, so a view is never a copy | | API | The same rule layer as the interface | an API that bypasses access rules is the gap nobody notices until it matters | ## Before you start Have every one of these ready. The plan assumes them from step one. - [ ] **Python 3.12 or newer** · free - Why: Runs the application and the formula evaluator. - Get it: python.org or your package manager, then a virtual environment. - Verify: python3 --version prints 3.12 or higher - [ ] **PostgreSQL 16 or newer** · free - Why: Access rules become query conditions, which is exactly what a relational database is good at. - Get it: Install locally or run it in Docker. - Verify: psql connects and reports 16 or higher - [ ] **A real spreadsheet with messy columns** · free - Why: Phase 1's type handling is only meaningful against real data, which has mixed columns, inconsistent dates and numbers stored as text. - Get it: Use one you actually maintain. The messier the better for this purpose. - Verify: It imports and at least one mixed column is reported - [ ] **A real sharing situation with different visibility** · free - Why: Access rules are the reason to build this, and they can only be designed against a real case: who sees which rows, and which column must stay hidden from most people. - Get it: Write down the actual case: the table, the principals and what each may see. If there is no such case, a spreadsheet is the better tool and this build has no purpose. - Verify: The case is written down before Phase 3 - [ ] **Willingness to write the formula grammar first** · free - Why: A formula language grown by accretion becomes unpredictable, and unpredictable formulas over shared data is worse than no formulas. - Get it: Half a page before Phase 2: literals, references, precedence and a closed function list. - Verify: The grammar file exists and predates the parser ## Quick start Follow Phase 1 in `BUILD_PLAN.md`; it creates the project. Then copy `.env.example` to `.env` and fill in the values it documents. ## Honest limits This build deliberately does not replace: - A mature general formula engine. - Real-time multiplayer editing. - Automation and integration breadth. - Managed hosting and operations. - mature formula engine, access rules, collaboration, integrations, and hosting - real-time multiplayer - automation and integration breadth - scale and formula depth - enterprise permissions If one of those is essential to you, that is the reason to keep paying for Grist, and the README should say so rather than pretend. ===== BRIEF.md ===== # Build brief · Grist The one-shot brief this plan expands. `BUILD_PLAN.md` (or `MILESTONES.md`) is the same sequence broken into steps and checks; where the two disagree, the plan wins. Build me a spreadsheet whose rows are records, with formulas and access rules that work per row to replace Grist. Build it in phases, in the order below. Do not write the whole thing in one pass. Finish a phase, run its "Done when" check, fix what fails, and only then start the next phase. ### Rule zero The distinguishing feature here is not formulas, it is access control that operates on rows and columns rather than on whole documents. Sharing a table where each person sees only their own rows, and where one column is hidden from most of them, is the thing a spreadsheet cannot do and the reason to build this rather than use one. That has a consequence for the architecture: filtering must happen on the server before data is sent, never in the client. A client-side filter over a full dataset is not access control, it is a display preference, and anyone can read past it. ### Stack (fixed, do not substitute) - Runtime: Python 3.12 with PostgreSQL · records are relational and access rules become query conditions, which is what a database is for - Formulas: A small expression language over records, with a written grammar · referencing records rather than cell addresses is what makes formulas survive row reordering - Access: Rules compiled into query conditions, applied server side · the whole premise; a client-side filter is not access control - Views: Grid, card and filtered views as queries over one table · copies diverge, so a view is never a copy - API: The same rule layer as the interface · an API that bypasses access rules is the gap nobody notices until it matters ### Data model (create this before Phase 1) - docs(id, name, owner_id), tables(id, doc_id, name) - columns(id, table_id, name, kind, ordinal, formula, reference_table_id) · kind includes reference, which is what makes it relational - records(id, table_id, ordinal), cells(record_id, column_id, value_json, computed_json) - views(id, table_id, kind, filter_json, sort_json, visible_columns_json) - access_rules(id, table_id, principal, row_condition_json, column_visibility_json, permission) · compiled into query conditions - deps(from_column_id, to_column_id) · the formula dependency graph - An access rule is data compiled into a query condition. If it is ever evaluated after fetching, it has stopped being access control. ### Phase 1 · Typed columns and real imports Build: Import a spreadsheet, infer column types, and let the user correct them. Handle the values that break naive inference: dates in several formats, numbers with separators, and a column that is mostly numbers with three text entries. Trap: A column that is ninety-nine percent numbers and one percent text will be inferred as numeric and the text rows silently dropped or zeroed, which loses data quietly. Report mixed columns, show the offending values, and make the user decide rather than guessing. Done when: A real spreadsheet imports with types inferred and correctable. A mixed column is reported with its non-conforming values shown rather than coerced. Dates in two different formats within one column are flagged rather than parsed inconsistently. Re-importing the same file reports no changes. Do not build yet: No formulas, views or access rules. ### Phase 2 · Formulas over records Build: Write the expression grammar first: literals, column references, references across tables, arithmetic, comparison and a closed function list. Implement the parser and evaluator against it. Build a dependency graph and recompute only what changed. Trap: Cell address references are what make spreadsheet formulas break when rows move, and reproducing them here throws away the advantage of having records. Reference columns and related records, and reordering becomes a display concern rather than a correctness one. Done when: A formula referencing a column produces correct values and reordering rows changes nothing. A reference across tables resolves correctly. Editing one cell recomputes only its dependents, verified by counting evaluations. A cyclic formula is refused at save with the cycle named. Do not build yet: No views or access rules yet. ### Phase 3 · Access rules that run on the server Build: Rules as data: a row condition and a column visibility set per principal. Compile them into query conditions applied before data leaves the server. Test by requesting through the API with a restricted principal rather than through the interface. Trap: Filtering in the client means the full dataset was sent and anyone can read it in the network inspector, which is the difference between hiding and protecting. Compile rules into the query, and prove it by checking what the server actually returned rather than what the interface displayed. Done when: A restricted principal's API request returns only permitted rows, verified by inspecting the raw response rather than the interface. A hidden column is absent from the response entirely rather than blanked. A rule change takes effect immediately for an active session. An unrestricted principal is unaffected. Do not build yet: No views or API surface beyond this test. ### Phase 4 · Views over one table Build: Views as queries with filters, sorts and visible columns. Editing through a view edits the table. A record that stops matching a view's filter leaves it rather than becoming a stale copy. Views compose with access rules rather than bypassing them. Trap: A view that materialises its own rows makes editing ambiguous and diverges within a day, and worse, a view defined with a filter is not access control even though it looks like one. Views are convenience; rules are protection, and conflating them is how data leaks. Done when: Editing through a filtered view updates the table. A record edited so it no longer matches leaves the view and stays in the table. A view cannot show a record the access rules exclude, verified with a restricted principal. Two views of one table reflect an edit made in either. Do not build yet: No API or export yet. ### Phase 5 · An API with the same rules Build: A documented API for reading and writing records, using the same rule layer as the interface. Tokens scoped to a document and a principal. Rate limit writes. Document the rule behaviour so an integrator is not surprised. Trap: An API that bypasses access rules because it is for internal use is the gap that gets found later, usually by whoever integrated with it and did not realise. One rule layer, applied to both surfaces, tested with a restricted token. Done when: An API read with a restricted token returns the same rows the interface would show that principal. An API write to a forbidden row is refused. Tokens are scoped to a document and a principal rather than being global. The documented behaviour matches the tested behaviour. Do not build yet: Backups and operations come last. ### Phase 6 · Export, backups and rule review Build: Export a document as spreadsheet files plus a schema description including formulas. Back up the database and restore it. Then produce a rule report showing, for each principal, exactly which rows and columns they can see. Trap: Access rules accumulate and interact, and nobody can hold the combined effect in their head after the fifth rule. A report showing what each principal can actually see is the only way to review them, and it is worth building before the rules get complicated rather than after. Done when: A document exports as spreadsheets plus a readable schema including formulas. A restore reproduces the document and its rules, verified by a restricted principal seeing the same rows. The rule report shows per principal what they can see, and matches a hand check for one principal. Rules can be exported for review. Do not build yet: Nothing after this. ### Out of scope (and why) - The mature formula engine of the commercial product, which covers far more than a closed function list and has been debugged against real use. - Real-time multiplayer editing, which is a substantial project on top of everything here. - The automation and integration breadth that makes the hosted product part of a workflow. - Hosting and operations, which are now yours along with the backups and the upgrades. ### README must contain - That access rules are compiled into queries and applied server side, and how to verify that - That formulas reference columns and records rather than cell addresses - That views are queries and never widen access - That the API uses the same rule layer as the interface - That the rule report is how combined rule effects are reviewed ===== AGENTS.md ===== # Agent instructions · Grist indie build - Read `README.md` and `BUILD_PLAN.md` before writing code. The stack is fixed: Python 3.12 with PostgreSQL, A small expression language over records, with a written grammar, Rules compiled into query conditions, applied server side, Grid, card and filtered views as queries over one table, The same rule layer as the interface. Do not substitute. - Work one phase at a time, in order. Do not start a phase until every "Done when" item of the previous one passes. - Prefer the fewest moving parts that satisfy the step. No frameworks, services or dependencies the plan does not name. - Secrets live in `.env`, never in source or logs. Keep `.env.example` current when a variable is introduced. - Do not invent cryptography, security guarantees, APIs or compliance claims. - Add a focused test for every destructive, security-sensitive or data-loss path the plan names. - Run the project checks before declaring a phase complete, and record any deliberate shortcut in the README under "Tradeoffs". ## Known traps - A column that is ninety-nine percent numbers and one percent text will be inferred as numeric and the text rows silently dropped or zeroed, which loses data quietly. Report mixed columns, show the offending values, and make the user decide rather than guessing. - Cell address references are what make spreadsheet formulas break when rows move, and reproducing them here throws away the advantage of having records. Reference columns and related records, and reordering becomes a display concern rather than a correctness one. - Filtering in the client means the full dataset was sent and anyone can read it in the network inspector, which is the difference between hiding and protecting. Compile rules into the query, and prove it by checking what the server actually returned rather than what the interface displayed. - A view that materialises its own rows makes editing ambiguous and diverges within a day, and worse, a view defined with a filter is not access control even though it looks like one. Views are convenience; rules are protection, and conflating them is how data leaks. - An API that bypasses access rules because it is for internal use is the gap that gets found later, usually by whoever integrated with it and did not realise. One rule layer, applied to both surfaces, tested with a restricted token. ===== BUILD_PLAN.md ===== # Build plan · Grist Tables with typed columns and formulas that reference records rather than cell addresses, several views over one table, an access rule layer that filters rows and hides columns on the server, and an API that respects the same rules as the interface. Phases are in dependency order. Each ends in a "Done when" list; treat an unticked item as a blocker, not a note. ## Phase 1 · Typed columns and real imports A real spreadsheet imports with its types recognised and its awkward values handled. ### Steps 1. Infer types and let them be corrected Inference is a suggestion. The correction interface is what makes it safe. Files: `src/import.py` 2. Report mixed columns With the values that do not conform. Silent coercion loses data nobody notices. Files: `src/types.py` 3. Test with a real messy file Your own spreadsheet, not a generated one. Real files have all three problems. ### Done when - [ ] A real spreadsheet imports with correctable types - [ ] A mixed column is reported with its outliers shown - [ ] Inconsistent date formats are flagged rather than parsed - [ ] Re-importing reports no changes ### Watch out - A column that is ninety-nine percent numbers and one percent text will be inferred as numeric and the text rows silently dropped or zeroed, which loses data quietly. Report mixed columns, show the offending values, and make the user decide rather than guessing. ## Phase 2 · Formulas over records A formula references a column rather than a cell address, so reordering rows changes nothing. ### Steps 1. Write the grammar before the parser Closed function list, column and reference syntax, precedence. In a file, first. Files: `docs/formula-grammar.md` 2. Reference columns, not addresses Row order becomes presentation. This is the whole advantage over a spreadsheet. Files: `src/formula.py` 3. Build the dependency graph Recompute dependents only, with a cycle check at save time. Files: `src/deps.py` ### Done when - [ ] Reordering rows does not change formula results - [ ] Cross-table references resolve - [ ] One edit recomputes only dependents, proved by count - [ ] A cyclic formula is refused with the cycle named ### Watch out - Cell address references are what make spreadsheet formulas break when rows move, and reproducing them here throws away the advantage of having records. Reference columns and related records, and reordering becomes a display concern rather than a correctness one. ## Phase 3 · Access rules that run on the server Someone with restricted access cannot read rows they should not, even through the API. ### Steps 1. Model rules as data Row condition and column visibility per principal. Readable and reviewable. Files: `src/rules.py` 2. Compile into query conditions Applied in the database query. Never after fetching. Files: `src/compile.py` 3. Verify at the response level Raw API response with a restricted token. The interface is not the test. Files: `tests/test_access.py` ### Done when - [ ] A restricted API response contains only permitted rows - [ ] A hidden column is absent rather than blanked - [ ] A rule change applies immediately - [ ] An unrestricted principal is unaffected ### Watch out - Filtering in the client means the full dataset was sent and anyone can read it in the network inspector, which is the difference between hiding and protecting. Compile rules into the query, and prove it by checking what the server actually returned rather than what the interface displayed. ## Phase 4 · Views over one table Grid, card and filtered views all read the same records and never diverge. ### Steps 1. Evaluate views as queries Filter, sort and column selection at read time. Files: `src/views.py` 2. Compose with rules Rules apply first, then the view filter. A view can never widen access. Files: `src/views.py` 3. Test the leaving case A record edited out of a filter. This is where a materialised view reveals itself. Files: `tests/test_views.py` ### Done when - [ ] Editing through a view updates the table - [ ] A record leaving a filter stays in the table - [ ] A view cannot show a rule-excluded record - [ ] Two views reflect an edit made in either ### Watch out - A view that materialises its own rows makes editing ambiguous and diverges within a day, and worse, a view defined with a filter is not access control even though it looks like one. Views are convenience; rules are protection, and conflating them is how data leaks. ## Phase 5 · An API with the same rules Everything the interface can do is available programmatically, under identical restrictions. ### Steps 1. Reuse the rule layer The same code path as the interface. Two implementations diverge. Files: `src/api.py` 2. Scope tokens narrowly Document and principal, revocable individually. Files: `src/tokens.py` 3. Test with a restricted token Read and write. Both must be refused where the rules say so. Files: `tests/test_api.py` ### Done when - [ ] A restricted API read matches the interface for that principal - [ ] A forbidden API write is refused - [ ] Tokens are scoped and individually revocable - [ ] Documented and tested behaviour agree ### Watch out - An API that bypasses access rules because it is for internal use is the gap that gets found later, usually by whoever integrated with it and did not realise. One rule layer, applied to both surfaces, tested with a restricted token. ## Not in this build - A mature general formula engine. - Real-time multiplayer editing. - Automation and integration breadth. - Managed hosting and operations. ## After v1, if you want it - Column-level formulas that reference aggregates across a related table - A rule simulator that answers what a given principal would see before you save a rule - Change history per record, which shared data always eventually needs ===== .env.example ===== # Copy to .env and fill in. Never commit .env; this file documents it. # Required · secret. Your PostgreSQL connection string. DATABASE_URL=postgres://user:pass@localhost:5432/tables # Required. Maximum dependency chain length, catching pathological documents before they hang. MAX_FORMULA_DEPTH=20 # Required. Writes per minute per token. API_WRITE_RATE_LIMIT=60 # Required. Ceiling on an import, so a mistaken file does not consume the database. IMPORT_MAX_ROWS=200000 # Required. Where rule reports are written for review. RULE_REPORT_PATH=./reports
You are building a lean indie version of Grist. Create the following project files first, then implement the application by following them. Keep the files updated as decisions change. Do not collapse this into a single README or prompt. ===== README.md ===== # Grist · indie build Tables with typed columns and formulas that reference records rather than cell addresses, several views over one table, an access rule layer that filters rows and hides columns on the server, and an API that respects the same rules as the interface. Estimated effort: **multi-day**. Work `BUILD_PLAN.md` top to bottom · every phase ends in a check that has to pass before the next one starts. ## Stack | Part | Choice | Why | | --- | --- | --- | | Runtime | Python 3.12 with PostgreSQL | records are relational and access rules become query conditions, which is what a database is for | | Formulas | A small expression language over records, with a written grammar | referencing records rather than cell addresses is what makes formulas survive row reordering | | Access | Rules compiled into query conditions, applied server side | the whole premise; a client-side filter is not access control | | Views | Grid, card and filtered views as queries over one table | copies diverge, so a view is never a copy | | API | The same rule layer as the interface | an API that bypasses access rules is the gap nobody notices until it matters | ## Before you start Have every one of these ready. The plan assumes them from step one. - [ ] **Python 3.12 or newer** · free - Why: Runs the application and the formula evaluator. - Get it: python.org or your package manager, then a virtual environment. - Verify: python3 --version prints 3.12 or higher - [ ] **PostgreSQL 16 or newer** · free - Why: Access rules become query conditions, which is exactly what a relational database is good at. - Get it: Install locally or run it in Docker. - Verify: psql connects and reports 16 or higher - [ ] **A real spreadsheet with messy columns** · free - Why: Phase 1's type handling is only meaningful against real data, which has mixed columns, inconsistent dates and numbers stored as text. - Get it: Use one you actually maintain. The messier the better for this purpose. - Verify: It imports and at least one mixed column is reported - [ ] **A real sharing situation with different visibility** · free - Why: Access rules are the reason to build this, and they can only be designed against a real case: who sees which rows, and which column must stay hidden from most people. - Get it: Write down the actual case: the table, the principals and what each may see. If there is no such case, a spreadsheet is the better tool and this build has no purpose. - Verify: The case is written down before Phase 3 - [ ] **Willingness to write the formula grammar first** · free - Why: A formula language grown by accretion becomes unpredictable, and unpredictable formulas over shared data is worse than no formulas. - Get it: Half a page before Phase 2: literals, references, precedence and a closed function list. - Verify: The grammar file exists and predates the parser ## Quick start Follow Phase 1 in `BUILD_PLAN.md`; it creates the project. Then copy `.env.example` to `.env` and fill in the values it documents. ## Honest limits This build deliberately does not replace: - A mature general formula engine. - Real-time multiplayer editing. - Automation and integration breadth. - Managed hosting and operations. - mature formula engine, access rules, collaboration, integrations, and hosting - real-time multiplayer - automation and integration breadth - scale and formula depth - enterprise permissions If one of those is essential to you, that is the reason to keep paying for Grist, and the README should say so rather than pretend. ===== BRIEF.md ===== # Build brief · Grist The one-shot brief this plan expands. `BUILD_PLAN.md` (or `MILESTONES.md`) is the same sequence broken into steps and checks; where the two disagree, the plan wins. Build me a spreadsheet whose rows are records, with formulas and access rules that work per row to replace Grist. Build it in phases, in the order below. Do not write the whole thing in one pass. Finish a phase, run its "Done when" check, fix what fails, and only then start the next phase. ### Rule zero The distinguishing feature here is not formulas, it is access control that operates on rows and columns rather than on whole documents. Sharing a table where each person sees only their own rows, and where one column is hidden from most of them, is the thing a spreadsheet cannot do and the reason to build this rather than use one. That has a consequence for the architecture: filtering must happen on the server before data is sent, never in the client. A client-side filter over a full dataset is not access control, it is a display preference, and anyone can read past it. ### Stack (fixed, do not substitute) - Runtime: Python 3.12 with PostgreSQL · records are relational and access rules become query conditions, which is what a database is for - Formulas: A small expression language over records, with a written grammar · referencing records rather than cell addresses is what makes formulas survive row reordering - Access: Rules compiled into query conditions, applied server side · the whole premise; a client-side filter is not access control - Views: Grid, card and filtered views as queries over one table · copies diverge, so a view is never a copy - API: The same rule layer as the interface · an API that bypasses access rules is the gap nobody notices until it matters ### Data model (create this before Phase 1) - docs(id, name, owner_id), tables(id, doc_id, name) - columns(id, table_id, name, kind, ordinal, formula, reference_table_id) · kind includes reference, which is what makes it relational - records(id, table_id, ordinal), cells(record_id, column_id, value_json, computed_json) - views(id, table_id, kind, filter_json, sort_json, visible_columns_json) - access_rules(id, table_id, principal, row_condition_json, column_visibility_json, permission) · compiled into query conditions - deps(from_column_id, to_column_id) · the formula dependency graph - An access rule is data compiled into a query condition. If it is ever evaluated after fetching, it has stopped being access control. ### Phase 1 · Typed columns and real imports Build: Import a spreadsheet, infer column types, and let the user correct them. Handle the values that break naive inference: dates in several formats, numbers with separators, and a column that is mostly numbers with three text entries. Trap: A column that is ninety-nine percent numbers and one percent text will be inferred as numeric and the text rows silently dropped or zeroed, which loses data quietly. Report mixed columns, show the offending values, and make the user decide rather than guessing. Done when: A real spreadsheet imports with types inferred and correctable. A mixed column is reported with its non-conforming values shown rather than coerced. Dates in two different formats within one column are flagged rather than parsed inconsistently. Re-importing the same file reports no changes. Do not build yet: No formulas, views or access rules. ### Phase 2 · Formulas over records Build: Write the expression grammar first: literals, column references, references across tables, arithmetic, comparison and a closed function list. Implement the parser and evaluator against it. Build a dependency graph and recompute only what changed. Trap: Cell address references are what make spreadsheet formulas break when rows move, and reproducing them here throws away the advantage of having records. Reference columns and related records, and reordering becomes a display concern rather than a correctness one. Done when: A formula referencing a column produces correct values and reordering rows changes nothing. A reference across tables resolves correctly. Editing one cell recomputes only its dependents, verified by counting evaluations. A cyclic formula is refused at save with the cycle named. Do not build yet: No views or access rules yet. ### Phase 3 · Access rules that run on the server Build: Rules as data: a row condition and a column visibility set per principal. Compile them into query conditions applied before data leaves the server. Test by requesting through the API with a restricted principal rather than through the interface. Trap: Filtering in the client means the full dataset was sent and anyone can read it in the network inspector, which is the difference between hiding and protecting. Compile rules into the query, and prove it by checking what the server actually returned rather than what the interface displayed. Done when: A restricted principal's API request returns only permitted rows, verified by inspecting the raw response rather than the interface. A hidden column is absent from the response entirely rather than blanked. A rule change takes effect immediately for an active session. An unrestricted principal is unaffected. Do not build yet: No views or API surface beyond this test. ### Phase 4 · Views over one table Build: Views as queries with filters, sorts and visible columns. Editing through a view edits the table. A record that stops matching a view's filter leaves it rather than becoming a stale copy. Views compose with access rules rather than bypassing them. Trap: A view that materialises its own rows makes editing ambiguous and diverges within a day, and worse, a view defined with a filter is not access control even though it looks like one. Views are convenience; rules are protection, and conflating them is how data leaks. Done when: Editing through a filtered view updates the table. A record edited so it no longer matches leaves the view and stays in the table. A view cannot show a record the access rules exclude, verified with a restricted principal. Two views of one table reflect an edit made in either. Do not build yet: No API or export yet. ### Phase 5 · An API with the same rules Build: A documented API for reading and writing records, using the same rule layer as the interface. Tokens scoped to a document and a principal. Rate limit writes. Document the rule behaviour so an integrator is not surprised. Trap: An API that bypasses access rules because it is for internal use is the gap that gets found later, usually by whoever integrated with it and did not realise. One rule layer, applied to both surfaces, tested with a restricted token. Done when: An API read with a restricted token returns the same rows the interface would show that principal. An API write to a forbidden row is refused. Tokens are scoped to a document and a principal rather than being global. The documented behaviour matches the tested behaviour. Do not build yet: Backups and operations come last. ### Phase 6 · Export, backups and rule review Build: Export a document as spreadsheet files plus a schema description including formulas. Back up the database and restore it. Then produce a rule report showing, for each principal, exactly which rows and columns they can see. Trap: Access rules accumulate and interact, and nobody can hold the combined effect in their head after the fifth rule. A report showing what each principal can actually see is the only way to review them, and it is worth building before the rules get complicated rather than after. Done when: A document exports as spreadsheets plus a readable schema including formulas. A restore reproduces the document and its rules, verified by a restricted principal seeing the same rows. The rule report shows per principal what they can see, and matches a hand check for one principal. Rules can be exported for review. Do not build yet: Nothing after this. ### Out of scope (and why) - The mature formula engine of the commercial product, which covers far more than a closed function list and has been debugged against real use. - Real-time multiplayer editing, which is a substantial project on top of everything here. - The automation and integration breadth that makes the hosted product part of a workflow. - Hosting and operations, which are now yours along with the backups and the upgrades. ### README must contain - That access rules are compiled into queries and applied server side, and how to verify that - That formulas reference columns and records rather than cell addresses - That views are queries and never widen access - That the API uses the same rule layer as the interface - That the rule report is how combined rule effects are reviewed ===== AGENTS.md ===== # Agent instructions · Grist indie build - Read `README.md` and `BUILD_PLAN.md` before writing code. The stack is fixed: Python 3.12 with PostgreSQL, A small expression language over records, with a written grammar, Rules compiled into query conditions, applied server side, Grid, card and filtered views as queries over one table, The same rule layer as the interface. Do not substitute. - Work one phase at a time, in order. Do not start a phase until every "Done when" item of the previous one passes. - Prefer the fewest moving parts that satisfy the step. No frameworks, services or dependencies the plan does not name. - Secrets live in `.env`, never in source or logs. Keep `.env.example` current when a variable is introduced. - Do not invent cryptography, security guarantees, APIs or compliance claims. - Add a focused test for every destructive, security-sensitive or data-loss path the plan names. - Run the project checks before declaring a phase complete, and record any deliberate shortcut in the README under "Tradeoffs". ## Known traps - A column that is ninety-nine percent numbers and one percent text will be inferred as numeric and the text rows silently dropped or zeroed, which loses data quietly. Report mixed columns, show the offending values, and make the user decide rather than guessing. - Cell address references are what make spreadsheet formulas break when rows move, and reproducing them here throws away the advantage of having records. Reference columns and related records, and reordering becomes a display concern rather than a correctness one. - Filtering in the client means the full dataset was sent and anyone can read it in the network inspector, which is the difference between hiding and protecting. Compile rules into the query, and prove it by checking what the server actually returned rather than what the interface displayed. - A view that materialises its own rows makes editing ambiguous and diverges within a day, and worse, a view defined with a filter is not access control even though it looks like one. Views are convenience; rules are protection, and conflating them is how data leaks. - An API that bypasses access rules because it is for internal use is the gap that gets found later, usually by whoever integrated with it and did not realise. One rule layer, applied to both surfaces, tested with a restricted token. ===== BUILD_PLAN.md ===== # Build plan · Grist Tables with typed columns and formulas that reference records rather than cell addresses, several views over one table, an access rule layer that filters rows and hides columns on the server, and an API that respects the same rules as the interface. Phases are in dependency order. Each ends in a "Done when" list; treat an unticked item as a blocker, not a note. ## Phase 1 · Typed columns and real imports A real spreadsheet imports with its types recognised and its awkward values handled. ### Steps 1. Infer types and let them be corrected Inference is a suggestion. The correction interface is what makes it safe. Files: `src/import.py` 2. Report mixed columns With the values that do not conform. Silent coercion loses data nobody notices. Files: `src/types.py` 3. Test with a real messy file Your own spreadsheet, not a generated one. Real files have all three problems. ### Done when - [ ] A real spreadsheet imports with correctable types - [ ] A mixed column is reported with its outliers shown - [ ] Inconsistent date formats are flagged rather than parsed - [ ] Re-importing reports no changes ### Watch out - A column that is ninety-nine percent numbers and one percent text will be inferred as numeric and the text rows silently dropped or zeroed, which loses data quietly. Report mixed columns, show the offending values, and make the user decide rather than guessing. ## Phase 2 · Formulas over records A formula references a column rather than a cell address, so reordering rows changes nothing. ### Steps 1. Write the grammar before the parser Closed function list, column and reference syntax, precedence. In a file, first. Files: `docs/formula-grammar.md` 2. Reference columns, not addresses Row order becomes presentation. This is the whole advantage over a spreadsheet. Files: `src/formula.py` 3. Build the dependency graph Recompute dependents only, with a cycle check at save time. Files: `src/deps.py` ### Done when - [ ] Reordering rows does not change formula results - [ ] Cross-table references resolve - [ ] One edit recomputes only dependents, proved by count - [ ] A cyclic formula is refused with the cycle named ### Watch out - Cell address references are what make spreadsheet formulas break when rows move, and reproducing them here throws away the advantage of having records. Reference columns and related records, and reordering becomes a display concern rather than a correctness one. ## Phase 3 · Access rules that run on the server Someone with restricted access cannot read rows they should not, even through the API. ### Steps 1. Model rules as data Row condition and column visibility per principal. Readable and reviewable. Files: `src/rules.py` 2. Compile into query conditions Applied in the database query. Never after fetching. Files: `src/compile.py` 3. Verify at the response level Raw API response with a restricted token. The interface is not the test. Files: `tests/test_access.py` ### Done when - [ ] A restricted API response contains only permitted rows - [ ] A hidden column is absent rather than blanked - [ ] A rule change applies immediately - [ ] An unrestricted principal is unaffected ### Watch out - Filtering in the client means the full dataset was sent and anyone can read it in the network inspector, which is the difference between hiding and protecting. Compile rules into the query, and prove it by checking what the server actually returned rather than what the interface displayed. ## Phase 4 · Views over one table Grid, card and filtered views all read the same records and never diverge. ### Steps 1. Evaluate views as queries Filter, sort and column selection at read time. Files: `src/views.py` 2. Compose with rules Rules apply first, then the view filter. A view can never widen access. Files: `src/views.py` 3. Test the leaving case A record edited out of a filter. This is where a materialised view reveals itself. Files: `tests/test_views.py` ### Done when - [ ] Editing through a view updates the table - [ ] A record leaving a filter stays in the table - [ ] A view cannot show a rule-excluded record - [ ] Two views reflect an edit made in either ### Watch out - A view that materialises its own rows makes editing ambiguous and diverges within a day, and worse, a view defined with a filter is not access control even though it looks like one. Views are convenience; rules are protection, and conflating them is how data leaks. ## Phase 5 · An API with the same rules Everything the interface can do is available programmatically, under identical restrictions. ### Steps 1. Reuse the rule layer The same code path as the interface. Two implementations diverge. Files: `src/api.py` 2. Scope tokens narrowly Document and principal, revocable individually. Files: `src/tokens.py` 3. Test with a restricted token Read and write. Both must be refused where the rules say so. Files: `tests/test_api.py` ### Done when - [ ] A restricted API read matches the interface for that principal - [ ] A forbidden API write is refused - [ ] Tokens are scoped and individually revocable - [ ] Documented and tested behaviour agree ### Watch out - An API that bypasses access rules because it is for internal use is the gap that gets found later, usually by whoever integrated with it and did not realise. One rule layer, applied to both surfaces, tested with a restricted token. ## Not in this build - A mature general formula engine. - Real-time multiplayer editing. - Automation and integration breadth. - Managed hosting and operations. ## After v1, if you want it - Column-level formulas that reference aggregates across a related table - A rule simulator that answers what a given principal would see before you save a rule - Change history per record, which shared data always eventually needs ===== .env.example ===== # Copy to .env and fill in. Never commit .env; this file documents it. # Required · secret. Your PostgreSQL connection string. DATABASE_URL=postgres://user:pass@localhost:5432/tables # Required. Maximum dependency chain length, catching pathological documents before they hang. MAX_FORMULA_DEPTH=20 # Required. Writes per minute per token. API_WRITE_RATE_LIMIT=60 # Required. Ceiling on an import, so a mistaken file does not consume the database. IMPORT_MAX_ROWS=200000 # Required. Where rule reports are written for review. RULE_REPORT_PATH=./reports
You are building a production product version of Grist. Create the following project files first, then implement the application by following them. Keep the files updated as decisions change. Do not collapse this into a single README or prompt. ===== PRODUCT.md ===== # Grist · product brief ## Problem The core loop is buildable, but a dependable replacement becomes a real weekend or multi-day project. For Grist, combine spreadsheet formulas with relational records in a small self-hosted workspace. The hard boundary is mature formula engine, access rules, collaboration, integrations, and hosting, plus data model flexibility, collaboration, and integrations. ## Product outcome A table tool where the same data can be shared with different people seeing different rows and columns, enforced in the query rather than the interface, with formulas that survive reordering and an API that cannot bypass the rules. ## Target user A builder who needs a maintainable product foundation, not a one-off demo. ## Required capabilities - Docker - PostgreSQL - self-hosted server - optional object storage for attachments ## Explicit non-goals for v1 - A mature general formula engine. - Real-time multiplayer editing. - Automation and integration breadth. - Managed hosting and operations. - mature formula engine, access rules, collaboration, integrations, and hosting - real-time multiplayer - automation and integration breadth - scale and formula depth - enterprise permissions ## Success criteria - A restricted API response is confirmed to contain only permitted rows - A hidden column is confirmed absent from the raw response - Reordering rows is confirmed not to change formula results - A view is confirmed unable to widen access - The rule report matches a hand check for one principal ===== BRIEF.md ===== # Build brief · Grist The one-shot brief this plan expands. `BUILD_PLAN.md` (or `MILESTONES.md`) is the same sequence broken into steps and checks; where the two disagree, the plan wins. Build me a spreadsheet whose rows are records, with formulas and access rules that work per row to replace Grist. Build it in phases, in the order below. Do not write the whole thing in one pass. Finish a phase, run its "Done when" check, fix what fails, and only then start the next phase. ### Rule zero The distinguishing feature here is not formulas, it is access control that operates on rows and columns rather than on whole documents. Sharing a table where each person sees only their own rows, and where one column is hidden from most of them, is the thing a spreadsheet cannot do and the reason to build this rather than use one. That has a consequence for the architecture: filtering must happen on the server before data is sent, never in the client. A client-side filter over a full dataset is not access control, it is a display preference, and anyone can read past it. ### Stack (fixed, do not substitute) - Runtime: Python 3.12 with PostgreSQL · records are relational and access rules become query conditions, which is what a database is for - Formulas: A small expression language over records, with a written grammar · referencing records rather than cell addresses is what makes formulas survive row reordering - Access: Rules compiled into query conditions, applied server side · the whole premise; a client-side filter is not access control - Views: Grid, card and filtered views as queries over one table · copies diverge, so a view is never a copy - API: The same rule layer as the interface · an API that bypasses access rules is the gap nobody notices until it matters ### Data model (create this before Phase 1) - docs(id, name, owner_id), tables(id, doc_id, name) - columns(id, table_id, name, kind, ordinal, formula, reference_table_id) · kind includes reference, which is what makes it relational - records(id, table_id, ordinal), cells(record_id, column_id, value_json, computed_json) - views(id, table_id, kind, filter_json, sort_json, visible_columns_json) - access_rules(id, table_id, principal, row_condition_json, column_visibility_json, permission) · compiled into query conditions - deps(from_column_id, to_column_id) · the formula dependency graph - An access rule is data compiled into a query condition. If it is ever evaluated after fetching, it has stopped being access control. ### Phase 1 · Typed columns and real imports Build: Import a spreadsheet, infer column types, and let the user correct them. Handle the values that break naive inference: dates in several formats, numbers with separators, and a column that is mostly numbers with three text entries. Trap: A column that is ninety-nine percent numbers and one percent text will be inferred as numeric and the text rows silently dropped or zeroed, which loses data quietly. Report mixed columns, show the offending values, and make the user decide rather than guessing. Done when: A real spreadsheet imports with types inferred and correctable. A mixed column is reported with its non-conforming values shown rather than coerced. Dates in two different formats within one column are flagged rather than parsed inconsistently. Re-importing the same file reports no changes. Do not build yet: No formulas, views or access rules. ### Phase 2 · Formulas over records Build: Write the expression grammar first: literals, column references, references across tables, arithmetic, comparison and a closed function list. Implement the parser and evaluator against it. Build a dependency graph and recompute only what changed. Trap: Cell address references are what make spreadsheet formulas break when rows move, and reproducing them here throws away the advantage of having records. Reference columns and related records, and reordering becomes a display concern rather than a correctness one. Done when: A formula referencing a column produces correct values and reordering rows changes nothing. A reference across tables resolves correctly. Editing one cell recomputes only its dependents, verified by counting evaluations. A cyclic formula is refused at save with the cycle named. Do not build yet: No views or access rules yet. ### Phase 3 · Access rules that run on the server Build: Rules as data: a row condition and a column visibility set per principal. Compile them into query conditions applied before data leaves the server. Test by requesting through the API with a restricted principal rather than through the interface. Trap: Filtering in the client means the full dataset was sent and anyone can read it in the network inspector, which is the difference between hiding and protecting. Compile rules into the query, and prove it by checking what the server actually returned rather than what the interface displayed. Done when: A restricted principal's API request returns only permitted rows, verified by inspecting the raw response rather than the interface. A hidden column is absent from the response entirely rather than blanked. A rule change takes effect immediately for an active session. An unrestricted principal is unaffected. Do not build yet: No views or API surface beyond this test. ### Phase 4 · Views over one table Build: Views as queries with filters, sorts and visible columns. Editing through a view edits the table. A record that stops matching a view's filter leaves it rather than becoming a stale copy. Views compose with access rules rather than bypassing them. Trap: A view that materialises its own rows makes editing ambiguous and diverges within a day, and worse, a view defined with a filter is not access control even though it looks like one. Views are convenience; rules are protection, and conflating them is how data leaks. Done when: Editing through a filtered view updates the table. A record edited so it no longer matches leaves the view and stays in the table. A view cannot show a record the access rules exclude, verified with a restricted principal. Two views of one table reflect an edit made in either. Do not build yet: No API or export yet. ### Phase 5 · An API with the same rules Build: A documented API for reading and writing records, using the same rule layer as the interface. Tokens scoped to a document and a principal. Rate limit writes. Document the rule behaviour so an integrator is not surprised. Trap: An API that bypasses access rules because it is for internal use is the gap that gets found later, usually by whoever integrated with it and did not realise. One rule layer, applied to both surfaces, tested with a restricted token. Done when: An API read with a restricted token returns the same rows the interface would show that principal. An API write to a forbidden row is refused. Tokens are scoped to a document and a principal rather than being global. The documented behaviour matches the tested behaviour. Do not build yet: Backups and operations come last. ### Phase 6 · Export, backups and rule review Build: Export a document as spreadsheet files plus a schema description including formulas. Back up the database and restore it. Then produce a rule report showing, for each principal, exactly which rows and columns they can see. Trap: Access rules accumulate and interact, and nobody can hold the combined effect in their head after the fifth rule. A report showing what each principal can actually see is the only way to review them, and it is worth building before the rules get complicated rather than after. Done when: A document exports as spreadsheets plus a readable schema including formulas. A restore reproduces the document and its rules, verified by a restricted principal seeing the same rows. The rule report shows per principal what they can see, and matches a hand check for one principal. Rules can be exported for review. Do not build yet: Nothing after this. ### Out of scope (and why) - The mature formula engine of the commercial product, which covers far more than a closed function list and has been debugged against real use. - Real-time multiplayer editing, which is a substantial project on top of everything here. - The automation and integration breadth that makes the hosted product part of a workflow. - Hosting and operations, which are now yours along with the backups and the upgrades. ### README must contain - That access rules are compiled into queries and applied server side, and how to verify that - That formulas reference columns and records rather than cell addresses - That views are queries and never widen access - That the API uses the same rule layer as the interface - That the rule report is how combined rule effects are reviewed ===== ARCHITECTURE.md ===== # Architecture · Grist ## Stack | Part | Choice | Why | | --- | --- | --- | | Runtime | Python 3.12 with PostgreSQL | records are relational and access rules become query conditions, which is what a database is for | | Formulas | A small expression language over records, with a written grammar | referencing records rather than cell addresses is what makes formulas survive row reordering | | Access | Rules compiled into query conditions, applied server side | the whole premise; a client-side filter is not access control | | Views | Grid, card and filtered views as queries over one table | copies diverge, so a view is never a copy | | API | The same rule layer as the interface | an API that bypasses access rules is the gap nobody notices until it matters | ## Modules Each module has one owner concern and a documented way to replace it. | Module | Owns | How to replace it | | --- | --- | --- | | Types and import | Getting real data in without losing it. | Any inference, provided mixed columns are reported rather than coerced. | | Formulas | Grammar, evaluation and dependencies. | Replaceable, provided the replacement is also specified first and also references records. | | Access rules | Row and column restriction. | Not swappable for client-side filtering under any circumstances. It is the product. | | Views | Presentation over one table. | Must remain queries and must never widen access. | | API | Programmatic access under the same rules. | Any transport. One rule layer is the invariant. | ## Configuration Every runtime setting is an environment variable documented in `.env.example`, validated at startup, with a safe local default wherever one exists. - `DATABASE_URL` · required, secret · Your PostgreSQL connection string. - `MAX_FORMULA_DEPTH` · required · Maximum dependency chain length, catching pathological documents before they hang. - `API_WRITE_RATE_LIMIT` · required · Writes per minute per token. - `IMPORT_MAX_ROWS` · required · Ceiling on an import, so a mistaken file does not consume the database. - `RULE_REPORT_PATH` · required · Where rule reports are written for review. ## Production baseline - Security: least privilege, input validation at every boundary, secret redaction in logs, rate limits on abuse-prone paths, no invented security primitives. - Data: explicit schema and migrations, transactional writes where integrity matters, backup and restore procedures that have been exercised. - Integrations: adapters around third-party providers, idempotent webhook or job processing, bounded retries, timeouts. - Observability: structured logs with request or operation ids, an error-tracking hook, and health and readiness checks where a server exists. - Quality: unit tests for domain rules, integration tests at module boundaries, one end-to-end test of the critical path. ## Decision records For each dependency in the stack table, keep a short note: why it was chosen, its failure mode, and how it is replaced. Do not add infrastructure until a requirement in `PRODUCT.md` justifies it. ===== AGENTS.md ===== # Agent instructions · Grist product build - Read `PRODUCT.md` and `ARCHITECTURE.md` before changing code. The stack is fixed: Python 3.12 with PostgreSQL, A small expression language over records, with a written grammar, Rules compiled into query conditions, applied server side, Grid, card and filtered views as queries over one table, The same rule layer as the interface. - Implement milestone by milestone from `MILESTONES.md`; keep each change reviewable and leave the application runnable at every commit. - Treat authentication, payments, encryption, imports, webhooks and destructive actions as high-risk boundaries when present. - Never invent cryptography or silently weaken a requirement to make a check pass. - Put every external service behind an interface with a deterministic fake for tests. - Add migrations and rollback or recovery notes for every persistent data change. - Log useful operational context without credentials, tokens, passwords or personal data. - Update documentation and run every check before completing a milestone. ## Known traps - A column that is ninety-nine percent numbers and one percent text will be inferred as numeric and the text rows silently dropped or zeroed, which loses data quietly. Report mixed columns, show the offending values, and make the user decide rather than guessing. - Cell address references are what make spreadsheet formulas break when rows move, and reproducing them here throws away the advantage of having records. Reference columns and related records, and reordering becomes a display concern rather than a correctness one. - Filtering in the client means the full dataset was sent and anyone can read it in the network inspector, which is the difference between hiding and protecting. Compile rules into the query, and prove it by checking what the server actually returned rather than what the interface displayed. - A view that materialises its own rows makes editing ambiguous and diverges within a day, and worse, a view defined with a filter is not access control even though it looks like one. Views are convenience; rules are protection, and conflating them is how data leaks. - An API that bypasses access rules because it is for internal use is the gap that gets found later, usually by whoever integrated with it and did not realise. One rule layer, applied to both surfaces, tested with a restricted token. - Access rules accumulate and interact, and nobody can hold the combined effect in their head after the fifth rule. A report showing what each principal can actually see is the only way to review them, and it is worth building before the rules get complicated rather than after. ===== MILESTONES.md ===== # Delivery milestones · Grist Estimated effort: **multi-day** for the indie phases; the production-only milestones add the trust and operability layer. ## M1 · Typed columns and real imports A real spreadsheet imports with its types recognised and its awkward values handled. ### Steps 1. Infer types and let them be corrected Inference is a suggestion. The correction interface is what makes it safe. Files: `src/import.py` 2. Report mixed columns With the values that do not conform. Silent coercion loses data nobody notices. Files: `src/types.py` 3. Test with a real messy file Your own spreadsheet, not a generated one. Real files have all three problems. ### Done when - [ ] A real spreadsheet imports with correctable types - [ ] A mixed column is reported with its outliers shown - [ ] Inconsistent date formats are flagged rather than parsed - [ ] Re-importing reports no changes ### Watch out - A column that is ninety-nine percent numbers and one percent text will be inferred as numeric and the text rows silently dropped or zeroed, which loses data quietly. Report mixed columns, show the offending values, and make the user decide rather than guessing. ## M2 · Formulas over records A formula references a column rather than a cell address, so reordering rows changes nothing. ### Steps 1. Write the grammar before the parser Closed function list, column and reference syntax, precedence. In a file, first. Files: `docs/formula-grammar.md` 2. Reference columns, not addresses Row order becomes presentation. This is the whole advantage over a spreadsheet. Files: `src/formula.py` 3. Build the dependency graph Recompute dependents only, with a cycle check at save time. Files: `src/deps.py` ### Done when - [ ] Reordering rows does not change formula results - [ ] Cross-table references resolve - [ ] One edit recomputes only dependents, proved by count - [ ] A cyclic formula is refused with the cycle named ### Watch out - Cell address references are what make spreadsheet formulas break when rows move, and reproducing them here throws away the advantage of having records. Reference columns and related records, and reordering becomes a display concern rather than a correctness one. ## M3 · Access rules that run on the server Someone with restricted access cannot read rows they should not, even through the API. ### Steps 1. Model rules as data Row condition and column visibility per principal. Readable and reviewable. Files: `src/rules.py` 2. Compile into query conditions Applied in the database query. Never after fetching. Files: `src/compile.py` 3. Verify at the response level Raw API response with a restricted token. The interface is not the test. Files: `tests/test_access.py` ### Done when - [ ] A restricted API response contains only permitted rows - [ ] A hidden column is absent rather than blanked - [ ] A rule change applies immediately - [ ] An unrestricted principal is unaffected ### Watch out - Filtering in the client means the full dataset was sent and anyone can read it in the network inspector, which is the difference between hiding and protecting. Compile rules into the query, and prove it by checking what the server actually returned rather than what the interface displayed. ## M4 · Views over one table Grid, card and filtered views all read the same records and never diverge. ### Steps 1. Evaluate views as queries Filter, sort and column selection at read time. Files: `src/views.py` 2. Compose with rules Rules apply first, then the view filter. A view can never widen access. Files: `src/views.py` 3. Test the leaving case A record edited out of a filter. This is where a materialised view reveals itself. Files: `tests/test_views.py` ### Done when - [ ] Editing through a view updates the table - [ ] A record leaving a filter stays in the table - [ ] A view cannot show a rule-excluded record - [ ] Two views reflect an edit made in either ### Watch out - A view that materialises its own rows makes editing ambiguous and diverges within a day, and worse, a view defined with a filter is not access control even though it looks like one. Views are convenience; rules are protection, and conflating them is how data leaks. ## M5 · An API with the same rules Everything the interface can do is available programmatically, under identical restrictions. ### Steps 1. Reuse the rule layer The same code path as the interface. Two implementations diverge. Files: `src/api.py` 2. Scope tokens narrowly Document and principal, revocable individually. Files: `src/tokens.py` 3. Test with a restricted token Read and write. Both must be refused where the rules say so. Files: `tests/test_api.py` ### Done when - [ ] A restricted API read matches the interface for that principal - [ ] A forbidden API write is refused - [ ] Tokens are scoped and individually revocable - [ ] Documented and tested behaviour agree ### Watch out - An API that bypasses access rules because it is for internal use is the gap that gets found later, usually by whoever integrated with it and did not realise. One rule layer, applied to both surfaces, tested with a restricted token. ## M6 · Export, backups and rule review (production only) Data is portable, recoverable, and the rules are reviewed rather than assumed. ### Steps 1. Export data and schema Spreadsheets for the values, a readable schema for types and formulas. Files: `src/export.py` 2. Back up and restore Including rules. A restore where rules are missing is worse than no restore. Files: `scripts/backup.sh`, `README.md` 3. Build the rule report Per principal, what they can see. Verify one against a hand check. Files: `src/rule_report.py` ### Done when - [ ] Export produces spreadsheets and a readable schema - [ ] A restore reproduces rules, verified with a restricted principal - [ ] The rule report matches a hand check for one principal - [ ] Rules can be exported for review ### Watch out - Access rules accumulate and interact, and nobody can hold the combined effect in their head after the fifth rule. A report showing what each principal can actually see is the only way to review them, and it is worth building before the rules get complicated rather than after. ===== OPERATIONS.md ===== # Operations · Grist ## Backup The database holds data, formulas and rules together. A restore missing rules would silently widen access, which makes rules the most important part of the backup rather than an afterthought. ## Restore Restore and verify with a restricted principal that they still see only their rows. That check, rather than the document opening, is what proves the restore. Do a restore drill before the first real user, and write the date here when it passes. ## Monitoring Rule evaluation errors and API refusals by principal. A spike in refusals usually means a rule changed and someone lost access they needed, which is better caught by you than reported by them. ## Incident checklist If someone saw data they should not, check whether a view was mistaken for a rule, since that is the most likely cause. The rule report is how you establish what each principal could actually see at the time. 1. Contain the issue without destroying evidence or user data. 2. Record the timeline and affected scope. 3. Rotate exposed secrets and revoke compromised sessions or credentials. 4. Restore from a verified backup when needed. 5. Document the root cause, the remediation and the regression test. ## Release gate - [ ] A restricted API response is confirmed to contain only permitted rows - [ ] A hidden column is confirmed absent from the raw response - [ ] Reordering rows is confirmed not to change formula results - [ ] A view is confirmed unable to widen access - [ ] The rule report matches a hand check for one principal ## Launch constraint Do not market omitted Grist capabilities as implemented. The non-goals in `PRODUCT.md` remain user-visible limitations until they are deliberately delivered. ===== .env.example ===== # Copy to .env and fill in. Never commit .env; this file documents it. # Required · secret. Your PostgreSQL connection string. DATABASE_URL=postgres://user:pass@localhost:5432/tables # Required. Maximum dependency chain length, catching pathological documents before they hang. MAX_FORMULA_DEPTH=20 # Required. Writes per minute per token. API_WRITE_RATE_LIMIT=60 # Required. Ceiling on an import, so a mistaken file does not consume the database. IMPORT_MAX_ROWS=200000 # Required. Where rule reports are written for review. RULE_REPORT_PATH=./reports
README.md
# Grist · indie build Tables with typed columns and formulas that reference records rather than cell addresses, several views over one table, an access rule layer that filters rows and hides columns on the server, and an API that respects the same rules as the interface. Estimated effort: **multi-day**. Work `BUILD_PLAN.md` top to bottom · every phase ends in a check that has to pass before the next one starts. ## Stack | Part | Choice | Why | | --- | --- | --- | | Runtime | Python 3.12 with PostgreSQL | records are relational and access rules become query conditions, which is what a database is for | | Formulas | A small expression language over records, with a written grammar | referencing records rather than cell addresses is what makes formulas survive row reordering | | Access | Rules compiled into query conditions, applied server side | the whole premise; a client-side filter is not access control | | Views | Grid, card and filtered views as queries over one table | copies diverge, so a view is never a copy | | API | The same rule layer as the interface | an API that bypasses access rules is the gap nobody notices until it matters | ## Before you start Have every one of these ready. The plan assumes them from step one. - [ ] **Python 3.12 or newer** · free - Why: Runs the application and the formula evaluator. - Get it: python.org or your package manager, then a virtual environment. - Verify: python3 --version prints 3.12 or higher - [ ] **PostgreSQL 16 or newer** · free - Why: Access rules become query conditions, which is exactly what a relational database is good at. - Get it: Install locally or run it in Docker. - Verify: psql connects and reports 16 or higher - [ ] **A real spreadsheet with messy columns** · free - Why: Phase 1's type handling is only meaningful against real data, which has mixed columns, inconsistent dates and numbers stored as text. - Get it: Use one you actually maintain. The messier the better for this purpose. - Verify: It imports and at least one mixed column is reported - [ ] **A real sharing situation with different visibility** · free - Why: Access rules are the reason to build this, and they can only be designed against a real case: who sees which rows, and which column must stay hidden from most people. - Get it: Write down the actual case: the table, the principals and what each may see. If there is no such case, a spreadsheet is the better tool and this build has no purpose. - Verify: The case is written down before Phase 3 - [ ] **Willingness to write the formula grammar first** · free - Why: A formula language grown by accretion becomes unpredictable, and unpredictable formulas over shared data is worse than no formulas. - Get it: Half a page before Phase 2: literals, references, precedence and a closed function list. - Verify: The grammar file exists and predates the parser ## Quick start Follow Phase 1 in `BUILD_PLAN.md`; it creates the project. Then copy `.env.example` to `.env` and fill in the values it documents. ## Honest limits This build deliberately does not replace: - A mature general formula engine. - Real-time multiplayer editing. - Automation and integration breadth. - Managed hosting and operations. - mature formula engine, access rules, collaboration, integrations, and hosting - real-time multiplayer - automation and integration breadth - scale and formula depth - enterprise permissions If one of those is essential to you, that is the reason to keep paying for Grist, and the README should say so rather than pretend.
BRIEF.md
# Build brief · Grist The one-shot brief this plan expands. `BUILD_PLAN.md` (or `MILESTONES.md`) is the same sequence broken into steps and checks; where the two disagree, the plan wins. Build me a spreadsheet whose rows are records, with formulas and access rules that work per row to replace Grist. Build it in phases, in the order below. Do not write the whole thing in one pass. Finish a phase, run its "Done when" check, fix what fails, and only then start the next phase. ### Rule zero The distinguishing feature here is not formulas, it is access control that operates on rows and columns rather than on whole documents. Sharing a table where each person sees only their own rows, and where one column is hidden from most of them, is the thing a spreadsheet cannot do and the reason to build this rather than use one. That has a consequence for the architecture: filtering must happen on the server before data is sent, never in the client. A client-side filter over a full dataset is not access control, it is a display preference, and anyone can read past it. ### Stack (fixed, do not substitute) - Runtime: Python 3.12 with PostgreSQL · records are relational and access rules become query conditions, which is what a database is for - Formulas: A small expression language over records, with a written grammar · referencing records rather than cell addresses is what makes formulas survive row reordering - Access: Rules compiled into query conditions, applied server side · the whole premise; a client-side filter is not access control - Views: Grid, card and filtered views as queries over one table · copies diverge, so a view is never a copy - API: The same rule layer as the interface · an API that bypasses access rules is the gap nobody notices until it matters ### Data model (create this before Phase 1) - docs(id, name, owner_id), tables(id, doc_id, name) - columns(id, table_id, name, kind, ordinal, formula, reference_table_id) · kind includes reference, which is what makes it relational - records(id, table_id, ordinal), cells(record_id, column_id, value_json, computed_json) - views(id, table_id, kind, filter_json, sort_json, visible_columns_json) - access_rules(id, table_id, principal, row_condition_json, column_visibility_json, permission) · compiled into query conditions - deps(from_column_id, to_column_id) · the formula dependency graph - An access rule is data compiled into a query condition. If it is ever evaluated after fetching, it has stopped being access control. ### Phase 1 · Typed columns and real imports Build: Import a spreadsheet, infer column types, and let the user correct them. Handle the values that break naive inference: dates in several formats, numbers with separators, and a column that is mostly numbers with three text entries. Trap: A column that is ninety-nine percent numbers and one percent text will be inferred as numeric and the text rows silently dropped or zeroed, which loses data quietly. Report mixed columns, show the offending values, and make the user decide rather than guessing. Done when: A real spreadsheet imports with types inferred and correctable. A mixed column is reported with its non-conforming values shown rather than coerced. Dates in two different formats within one column are flagged rather than parsed inconsistently. Re-importing the same file reports no changes. Do not build yet: No formulas, views or access rules. ### Phase 2 · Formulas over records Build: Write the expression grammar first: literals, column references, references across tables, arithmetic, comparison and a closed function list. Implement the parser and evaluator against it. Build a dependency graph and recompute only what changed. Trap: Cell address references are what make spreadsheet formulas break when rows move, and reproducing them here throws away the advantage of having records. Reference columns and related records, and reordering becomes a display concern rather than a correctness one. Done when: A formula referencing a column produces correct values and reordering rows changes nothing. A reference across tables resolves correctly. Editing one cell recomputes only its dependents, verified by counting evaluations. A cyclic formula is refused at save with the cycle named. Do not build yet: No views or access rules yet. ### Phase 3 · Access rules that run on the server Build: Rules as data: a row condition and a column visibility set per principal. Compile them into query conditions applied before data leaves the server. Test by requesting through the API with a restricted principal rather than through the interface. Trap: Filtering in the client means the full dataset was sent and anyone can read it in the network inspector, which is the difference between hiding and protecting. Compile rules into the query, and prove it by checking what the server actually returned rather than what the interface displayed. Done when: A restricted principal's API request returns only permitted rows, verified by inspecting the raw response rather than the interface. A hidden column is absent from the response entirely rather than blanked. A rule change takes effect immediately for an active session. An unrestricted principal is unaffected. Do not build yet: No views or API surface beyond this test. ### Phase 4 · Views over one table Build: Views as queries with filters, sorts and visible columns. Editing through a view edits the table. A record that stops matching a view's filter leaves it rather than becoming a stale copy. Views compose with access rules rather than bypassing them. Trap: A view that materialises its own rows makes editing ambiguous and diverges within a day, and worse, a view defined with a filter is not access control even though it looks like one. Views are convenience; rules are protection, and conflating them is how data leaks. Done when: Editing through a filtered view updates the table. A record edited so it no longer matches leaves the view and stays in the table. A view cannot show a record the access rules exclude, verified with a restricted principal. Two views of one table reflect an edit made in either. Do not build yet: No API or export yet. ### Phase 5 · An API with the same rules Build: A documented API for reading and writing records, using the same rule layer as the interface. Tokens scoped to a document and a principal. Rate limit writes. Document the rule behaviour so an integrator is not surprised. Trap: An API that bypasses access rules because it is for internal use is the gap that gets found later, usually by whoever integrated with it and did not realise. One rule layer, applied to both surfaces, tested with a restricted token. Done when: An API read with a restricted token returns the same rows the interface would show that principal. An API write to a forbidden row is refused. Tokens are scoped to a document and a principal rather than being global. The documented behaviour matches the tested behaviour. Do not build yet: Backups and operations come last. ### Phase 6 · Export, backups and rule review Build: Export a document as spreadsheet files plus a schema description including formulas. Back up the database and restore it. Then produce a rule report showing, for each principal, exactly which rows and columns they can see. Trap: Access rules accumulate and interact, and nobody can hold the combined effect in their head after the fifth rule. A report showing what each principal can actually see is the only way to review them, and it is worth building before the rules get complicated rather than after. Done when: A document exports as spreadsheets plus a readable schema including formulas. A restore reproduces the document and its rules, verified by a restricted principal seeing the same rows. The rule report shows per principal what they can see, and matches a hand check for one principal. Rules can be exported for review. Do not build yet: Nothing after this. ### Out of scope (and why) - The mature formula engine of the commercial product, which covers far more than a closed function list and has been debugged against real use. - Real-time multiplayer editing, which is a substantial project on top of everything here. - The automation and integration breadth that makes the hosted product part of a workflow. - Hosting and operations, which are now yours along with the backups and the upgrades. ### README must contain - That access rules are compiled into queries and applied server side, and how to verify that - That formulas reference columns and records rather than cell addresses - That views are queries and never widen access - That the API uses the same rule layer as the interface - That the rule report is how combined rule effects are reviewed
AGENTS.md
# Agent instructions · Grist indie build - Read `README.md` and `BUILD_PLAN.md` before writing code. The stack is fixed: Python 3.12 with PostgreSQL, A small expression language over records, with a written grammar, Rules compiled into query conditions, applied server side, Grid, card and filtered views as queries over one table, The same rule layer as the interface. Do not substitute. - Work one phase at a time, in order. Do not start a phase until every "Done when" item of the previous one passes. - Prefer the fewest moving parts that satisfy the step. No frameworks, services or dependencies the plan does not name. - Secrets live in `.env`, never in source or logs. Keep `.env.example` current when a variable is introduced. - Do not invent cryptography, security guarantees, APIs or compliance claims. - Add a focused test for every destructive, security-sensitive or data-loss path the plan names. - Run the project checks before declaring a phase complete, and record any deliberate shortcut in the README under "Tradeoffs". ## Known traps - A column that is ninety-nine percent numbers and one percent text will be inferred as numeric and the text rows silently dropped or zeroed, which loses data quietly. Report mixed columns, show the offending values, and make the user decide rather than guessing. - Cell address references are what make spreadsheet formulas break when rows move, and reproducing them here throws away the advantage of having records. Reference columns and related records, and reordering becomes a display concern rather than a correctness one. - Filtering in the client means the full dataset was sent and anyone can read it in the network inspector, which is the difference between hiding and protecting. Compile rules into the query, and prove it by checking what the server actually returned rather than what the interface displayed. - A view that materialises its own rows makes editing ambiguous and diverges within a day, and worse, a view defined with a filter is not access control even though it looks like one. Views are convenience; rules are protection, and conflating them is how data leaks. - An API that bypasses access rules because it is for internal use is the gap that gets found later, usually by whoever integrated with it and did not realise. One rule layer, applied to both surfaces, tested with a restricted token.
BUILD_PLAN.md
# Build plan · Grist Tables with typed columns and formulas that reference records rather than cell addresses, several views over one table, an access rule layer that filters rows and hides columns on the server, and an API that respects the same rules as the interface. Phases are in dependency order. Each ends in a "Done when" list; treat an unticked item as a blocker, not a note. ## Phase 1 · Typed columns and real imports A real spreadsheet imports with its types recognised and its awkward values handled. ### Steps 1. Infer types and let them be corrected Inference is a suggestion. The correction interface is what makes it safe. Files: `src/import.py` 2. Report mixed columns With the values that do not conform. Silent coercion loses data nobody notices. Files: `src/types.py` 3. Test with a real messy file Your own spreadsheet, not a generated one. Real files have all three problems. ### Done when - [ ] A real spreadsheet imports with correctable types - [ ] A mixed column is reported with its outliers shown - [ ] Inconsistent date formats are flagged rather than parsed - [ ] Re-importing reports no changes ### Watch out - A column that is ninety-nine percent numbers and one percent text will be inferred as numeric and the text rows silently dropped or zeroed, which loses data quietly. Report mixed columns, show the offending values, and make the user decide rather than guessing. ## Phase 2 · Formulas over records A formula references a column rather than a cell address, so reordering rows changes nothing. ### Steps 1. Write the grammar before the parser Closed function list, column and reference syntax, precedence. In a file, first. Files: `docs/formula-grammar.md` 2. Reference columns, not addresses Row order becomes presentation. This is the whole advantage over a spreadsheet. Files: `src/formula.py` 3. Build the dependency graph Recompute dependents only, with a cycle check at save time. Files: `src/deps.py` ### Done when - [ ] Reordering rows does not change formula results - [ ] Cross-table references resolve - [ ] One edit recomputes only dependents, proved by count - [ ] A cyclic formula is refused with the cycle named ### Watch out - Cell address references are what make spreadsheet formulas break when rows move, and reproducing them here throws away the advantage of having records. Reference columns and related records, and reordering becomes a display concern rather than a correctness one. ## Phase 3 · Access rules that run on the server Someone with restricted access cannot read rows they should not, even through the API. ### Steps 1. Model rules as data Row condition and column visibility per principal. Readable and reviewable. Files: `src/rules.py` 2. Compile into query conditions Applied in the database query. Never after fetching. Files: `src/compile.py` 3. Verify at the response level Raw API response with a restricted token. The interface is not the test. Files: `tests/test_access.py` ### Done when - [ ] A restricted API response contains only permitted rows - [ ] A hidden column is absent rather than blanked - [ ] A rule change applies immediately - [ ] An unrestricted principal is unaffected ### Watch out - Filtering in the client means the full dataset was sent and anyone can read it in the network inspector, which is the difference between hiding and protecting. Compile rules into the query, and prove it by checking what the server actually returned rather than what the interface displayed. ## Phase 4 · Views over one table Grid, card and filtered views all read the same records and never diverge. ### Steps 1. Evaluate views as queries Filter, sort and column selection at read time. Files: `src/views.py` 2. Compose with rules Rules apply first, then the view filter. A view can never widen access. Files: `src/views.py` 3. Test the leaving case A record edited out of a filter. This is where a materialised view reveals itself. Files: `tests/test_views.py` ### Done when - [ ] Editing through a view updates the table - [ ] A record leaving a filter stays in the table - [ ] A view cannot show a rule-excluded record - [ ] Two views reflect an edit made in either ### Watch out - A view that materialises its own rows makes editing ambiguous and diverges within a day, and worse, a view defined with a filter is not access control even though it looks like one. Views are convenience; rules are protection, and conflating them is how data leaks. ## Phase 5 · An API with the same rules Everything the interface can do is available programmatically, under identical restrictions. ### Steps 1. Reuse the rule layer The same code path as the interface. Two implementations diverge. Files: `src/api.py` 2. Scope tokens narrowly Document and principal, revocable individually. Files: `src/tokens.py` 3. Test with a restricted token Read and write. Both must be refused where the rules say so. Files: `tests/test_api.py` ### Done when - [ ] A restricted API read matches the interface for that principal - [ ] A forbidden API write is refused - [ ] Tokens are scoped and individually revocable - [ ] Documented and tested behaviour agree ### Watch out - An API that bypasses access rules because it is for internal use is the gap that gets found later, usually by whoever integrated with it and did not realise. One rule layer, applied to both surfaces, tested with a restricted token. ## Not in this build - A mature general formula engine. - Real-time multiplayer editing. - Automation and integration breadth. - Managed hosting and operations. ## After v1, if you want it - Column-level formulas that reference aggregates across a related table - A rule simulator that answers what a given principal would see before you save a rule - Change history per record, which shared data always eventually needs
.env.example
# Copy to .env and fill in. Never commit .env; this file documents it. # Required · secret. Your PostgreSQL connection string. DATABASE_URL=postgres://user:pass@localhost:5432/tables # Required. Maximum dependency chain length, catching pathological documents before they hang. MAX_FORMULA_DEPTH=20 # Required. Writes per minute per token. API_WRITE_RATE_LIMIT=60 # Required. Ceiling on an import, so a mistaken file does not consume the database. IMPORT_MAX_ROWS=200000 # Required. Where rule reports are written for review. RULE_REPORT_PATH=./reports
PRODUCT.md
# Grist · product brief ## Problem The core loop is buildable, but a dependable replacement becomes a real weekend or multi-day project. For Grist, combine spreadsheet formulas with relational records in a small self-hosted workspace. The hard boundary is mature formula engine, access rules, collaboration, integrations, and hosting, plus data model flexibility, collaboration, and integrations. ## Product outcome A table tool where the same data can be shared with different people seeing different rows and columns, enforced in the query rather than the interface, with formulas that survive reordering and an API that cannot bypass the rules. ## Target user A builder who needs a maintainable product foundation, not a one-off demo. ## Required capabilities - Docker - PostgreSQL - self-hosted server - optional object storage for attachments ## Explicit non-goals for v1 - A mature general formula engine. - Real-time multiplayer editing. - Automation and integration breadth. - Managed hosting and operations. - mature formula engine, access rules, collaboration, integrations, and hosting - real-time multiplayer - automation and integration breadth - scale and formula depth - enterprise permissions ## Success criteria - A restricted API response is confirmed to contain only permitted rows - A hidden column is confirmed absent from the raw response - Reordering rows is confirmed not to change formula results - A view is confirmed unable to widen access - The rule report matches a hand check for one principal
BRIEF.md
# Build brief · Grist The one-shot brief this plan expands. `BUILD_PLAN.md` (or `MILESTONES.md`) is the same sequence broken into steps and checks; where the two disagree, the plan wins. Build me a spreadsheet whose rows are records, with formulas and access rules that work per row to replace Grist. Build it in phases, in the order below. Do not write the whole thing in one pass. Finish a phase, run its "Done when" check, fix what fails, and only then start the next phase. ### Rule zero The distinguishing feature here is not formulas, it is access control that operates on rows and columns rather than on whole documents. Sharing a table where each person sees only their own rows, and where one column is hidden from most of them, is the thing a spreadsheet cannot do and the reason to build this rather than use one. That has a consequence for the architecture: filtering must happen on the server before data is sent, never in the client. A client-side filter over a full dataset is not access control, it is a display preference, and anyone can read past it. ### Stack (fixed, do not substitute) - Runtime: Python 3.12 with PostgreSQL · records are relational and access rules become query conditions, which is what a database is for - Formulas: A small expression language over records, with a written grammar · referencing records rather than cell addresses is what makes formulas survive row reordering - Access: Rules compiled into query conditions, applied server side · the whole premise; a client-side filter is not access control - Views: Grid, card and filtered views as queries over one table · copies diverge, so a view is never a copy - API: The same rule layer as the interface · an API that bypasses access rules is the gap nobody notices until it matters ### Data model (create this before Phase 1) - docs(id, name, owner_id), tables(id, doc_id, name) - columns(id, table_id, name, kind, ordinal, formula, reference_table_id) · kind includes reference, which is what makes it relational - records(id, table_id, ordinal), cells(record_id, column_id, value_json, computed_json) - views(id, table_id, kind, filter_json, sort_json, visible_columns_json) - access_rules(id, table_id, principal, row_condition_json, column_visibility_json, permission) · compiled into query conditions - deps(from_column_id, to_column_id) · the formula dependency graph - An access rule is data compiled into a query condition. If it is ever evaluated after fetching, it has stopped being access control. ### Phase 1 · Typed columns and real imports Build: Import a spreadsheet, infer column types, and let the user correct them. Handle the values that break naive inference: dates in several formats, numbers with separators, and a column that is mostly numbers with three text entries. Trap: A column that is ninety-nine percent numbers and one percent text will be inferred as numeric and the text rows silently dropped or zeroed, which loses data quietly. Report mixed columns, show the offending values, and make the user decide rather than guessing. Done when: A real spreadsheet imports with types inferred and correctable. A mixed column is reported with its non-conforming values shown rather than coerced. Dates in two different formats within one column are flagged rather than parsed inconsistently. Re-importing the same file reports no changes. Do not build yet: No formulas, views or access rules. ### Phase 2 · Formulas over records Build: Write the expression grammar first: literals, column references, references across tables, arithmetic, comparison and a closed function list. Implement the parser and evaluator against it. Build a dependency graph and recompute only what changed. Trap: Cell address references are what make spreadsheet formulas break when rows move, and reproducing them here throws away the advantage of having records. Reference columns and related records, and reordering becomes a display concern rather than a correctness one. Done when: A formula referencing a column produces correct values and reordering rows changes nothing. A reference across tables resolves correctly. Editing one cell recomputes only its dependents, verified by counting evaluations. A cyclic formula is refused at save with the cycle named. Do not build yet: No views or access rules yet. ### Phase 3 · Access rules that run on the server Build: Rules as data: a row condition and a column visibility set per principal. Compile them into query conditions applied before data leaves the server. Test by requesting through the API with a restricted principal rather than through the interface. Trap: Filtering in the client means the full dataset was sent and anyone can read it in the network inspector, which is the difference between hiding and protecting. Compile rules into the query, and prove it by checking what the server actually returned rather than what the interface displayed. Done when: A restricted principal's API request returns only permitted rows, verified by inspecting the raw response rather than the interface. A hidden column is absent from the response entirely rather than blanked. A rule change takes effect immediately for an active session. An unrestricted principal is unaffected. Do not build yet: No views or API surface beyond this test. ### Phase 4 · Views over one table Build: Views as queries with filters, sorts and visible columns. Editing through a view edits the table. A record that stops matching a view's filter leaves it rather than becoming a stale copy. Views compose with access rules rather than bypassing them. Trap: A view that materialises its own rows makes editing ambiguous and diverges within a day, and worse, a view defined with a filter is not access control even though it looks like one. Views are convenience; rules are protection, and conflating them is how data leaks. Done when: Editing through a filtered view updates the table. A record edited so it no longer matches leaves the view and stays in the table. A view cannot show a record the access rules exclude, verified with a restricted principal. Two views of one table reflect an edit made in either. Do not build yet: No API or export yet. ### Phase 5 · An API with the same rules Build: A documented API for reading and writing records, using the same rule layer as the interface. Tokens scoped to a document and a principal. Rate limit writes. Document the rule behaviour so an integrator is not surprised. Trap: An API that bypasses access rules because it is for internal use is the gap that gets found later, usually by whoever integrated with it and did not realise. One rule layer, applied to both surfaces, tested with a restricted token. Done when: An API read with a restricted token returns the same rows the interface would show that principal. An API write to a forbidden row is refused. Tokens are scoped to a document and a principal rather than being global. The documented behaviour matches the tested behaviour. Do not build yet: Backups and operations come last. ### Phase 6 · Export, backups and rule review Build: Export a document as spreadsheet files plus a schema description including formulas. Back up the database and restore it. Then produce a rule report showing, for each principal, exactly which rows and columns they can see. Trap: Access rules accumulate and interact, and nobody can hold the combined effect in their head after the fifth rule. A report showing what each principal can actually see is the only way to review them, and it is worth building before the rules get complicated rather than after. Done when: A document exports as spreadsheets plus a readable schema including formulas. A restore reproduces the document and its rules, verified by a restricted principal seeing the same rows. The rule report shows per principal what they can see, and matches a hand check for one principal. Rules can be exported for review. Do not build yet: Nothing after this. ### Out of scope (and why) - The mature formula engine of the commercial product, which covers far more than a closed function list and has been debugged against real use. - Real-time multiplayer editing, which is a substantial project on top of everything here. - The automation and integration breadth that makes the hosted product part of a workflow. - Hosting and operations, which are now yours along with the backups and the upgrades. ### README must contain - That access rules are compiled into queries and applied server side, and how to verify that - That formulas reference columns and records rather than cell addresses - That views are queries and never widen access - That the API uses the same rule layer as the interface - That the rule report is how combined rule effects are reviewed
ARCHITECTURE.md
# Architecture · Grist ## Stack | Part | Choice | Why | | --- | --- | --- | | Runtime | Python 3.12 with PostgreSQL | records are relational and access rules become query conditions, which is what a database is for | | Formulas | A small expression language over records, with a written grammar | referencing records rather than cell addresses is what makes formulas survive row reordering | | Access | Rules compiled into query conditions, applied server side | the whole premise; a client-side filter is not access control | | Views | Grid, card and filtered views as queries over one table | copies diverge, so a view is never a copy | | API | The same rule layer as the interface | an API that bypasses access rules is the gap nobody notices until it matters | ## Modules Each module has one owner concern and a documented way to replace it. | Module | Owns | How to replace it | | --- | --- | --- | | Types and import | Getting real data in without losing it. | Any inference, provided mixed columns are reported rather than coerced. | | Formulas | Grammar, evaluation and dependencies. | Replaceable, provided the replacement is also specified first and also references records. | | Access rules | Row and column restriction. | Not swappable for client-side filtering under any circumstances. It is the product. | | Views | Presentation over one table. | Must remain queries and must never widen access. | | API | Programmatic access under the same rules. | Any transport. One rule layer is the invariant. | ## Configuration Every runtime setting is an environment variable documented in `.env.example`, validated at startup, with a safe local default wherever one exists. - `DATABASE_URL` · required, secret · Your PostgreSQL connection string. - `MAX_FORMULA_DEPTH` · required · Maximum dependency chain length, catching pathological documents before they hang. - `API_WRITE_RATE_LIMIT` · required · Writes per minute per token. - `IMPORT_MAX_ROWS` · required · Ceiling on an import, so a mistaken file does not consume the database. - `RULE_REPORT_PATH` · required · Where rule reports are written for review. ## Production baseline - Security: least privilege, input validation at every boundary, secret redaction in logs, rate limits on abuse-prone paths, no invented security primitives. - Data: explicit schema and migrations, transactional writes where integrity matters, backup and restore procedures that have been exercised. - Integrations: adapters around third-party providers, idempotent webhook or job processing, bounded retries, timeouts. - Observability: structured logs with request or operation ids, an error-tracking hook, and health and readiness checks where a server exists. - Quality: unit tests for domain rules, integration tests at module boundaries, one end-to-end test of the critical path. ## Decision records For each dependency in the stack table, keep a short note: why it was chosen, its failure mode, and how it is replaced. Do not add infrastructure until a requirement in `PRODUCT.md` justifies it.
AGENTS.md
# Agent instructions · Grist product build - Read `PRODUCT.md` and `ARCHITECTURE.md` before changing code. The stack is fixed: Python 3.12 with PostgreSQL, A small expression language over records, with a written grammar, Rules compiled into query conditions, applied server side, Grid, card and filtered views as queries over one table, The same rule layer as the interface. - Implement milestone by milestone from `MILESTONES.md`; keep each change reviewable and leave the application runnable at every commit. - Treat authentication, payments, encryption, imports, webhooks and destructive actions as high-risk boundaries when present. - Never invent cryptography or silently weaken a requirement to make a check pass. - Put every external service behind an interface with a deterministic fake for tests. - Add migrations and rollback or recovery notes for every persistent data change. - Log useful operational context without credentials, tokens, passwords or personal data. - Update documentation and run every check before completing a milestone. ## Known traps - A column that is ninety-nine percent numbers and one percent text will be inferred as numeric and the text rows silently dropped or zeroed, which loses data quietly. Report mixed columns, show the offending values, and make the user decide rather than guessing. - Cell address references are what make spreadsheet formulas break when rows move, and reproducing them here throws away the advantage of having records. Reference columns and related records, and reordering becomes a display concern rather than a correctness one. - Filtering in the client means the full dataset was sent and anyone can read it in the network inspector, which is the difference between hiding and protecting. Compile rules into the query, and prove it by checking what the server actually returned rather than what the interface displayed. - A view that materialises its own rows makes editing ambiguous and diverges within a day, and worse, a view defined with a filter is not access control even though it looks like one. Views are convenience; rules are protection, and conflating them is how data leaks. - An API that bypasses access rules because it is for internal use is the gap that gets found later, usually by whoever integrated with it and did not realise. One rule layer, applied to both surfaces, tested with a restricted token. - Access rules accumulate and interact, and nobody can hold the combined effect in their head after the fifth rule. A report showing what each principal can actually see is the only way to review them, and it is worth building before the rules get complicated rather than after.
MILESTONES.md
# Delivery milestones · Grist Estimated effort: **multi-day** for the indie phases; the production-only milestones add the trust and operability layer. ## M1 · Typed columns and real imports A real spreadsheet imports with its types recognised and its awkward values handled. ### Steps 1. Infer types and let them be corrected Inference is a suggestion. The correction interface is what makes it safe. Files: `src/import.py` 2. Report mixed columns With the values that do not conform. Silent coercion loses data nobody notices. Files: `src/types.py` 3. Test with a real messy file Your own spreadsheet, not a generated one. Real files have all three problems. ### Done when - [ ] A real spreadsheet imports with correctable types - [ ] A mixed column is reported with its outliers shown - [ ] Inconsistent date formats are flagged rather than parsed - [ ] Re-importing reports no changes ### Watch out - A column that is ninety-nine percent numbers and one percent text will be inferred as numeric and the text rows silently dropped or zeroed, which loses data quietly. Report mixed columns, show the offending values, and make the user decide rather than guessing. ## M2 · Formulas over records A formula references a column rather than a cell address, so reordering rows changes nothing. ### Steps 1. Write the grammar before the parser Closed function list, column and reference syntax, precedence. In a file, first. Files: `docs/formula-grammar.md` 2. Reference columns, not addresses Row order becomes presentation. This is the whole advantage over a spreadsheet. Files: `src/formula.py` 3. Build the dependency graph Recompute dependents only, with a cycle check at save time. Files: `src/deps.py` ### Done when - [ ] Reordering rows does not change formula results - [ ] Cross-table references resolve - [ ] One edit recomputes only dependents, proved by count - [ ] A cyclic formula is refused with the cycle named ### Watch out - Cell address references are what make spreadsheet formulas break when rows move, and reproducing them here throws away the advantage of having records. Reference columns and related records, and reordering becomes a display concern rather than a correctness one. ## M3 · Access rules that run on the server Someone with restricted access cannot read rows they should not, even through the API. ### Steps 1. Model rules as data Row condition and column visibility per principal. Readable and reviewable. Files: `src/rules.py` 2. Compile into query conditions Applied in the database query. Never after fetching. Files: `src/compile.py` 3. Verify at the response level Raw API response with a restricted token. The interface is not the test. Files: `tests/test_access.py` ### Done when - [ ] A restricted API response contains only permitted rows - [ ] A hidden column is absent rather than blanked - [ ] A rule change applies immediately - [ ] An unrestricted principal is unaffected ### Watch out - Filtering in the client means the full dataset was sent and anyone can read it in the network inspector, which is the difference between hiding and protecting. Compile rules into the query, and prove it by checking what the server actually returned rather than what the interface displayed. ## M4 · Views over one table Grid, card and filtered views all read the same records and never diverge. ### Steps 1. Evaluate views as queries Filter, sort and column selection at read time. Files: `src/views.py` 2. Compose with rules Rules apply first, then the view filter. A view can never widen access. Files: `src/views.py` 3. Test the leaving case A record edited out of a filter. This is where a materialised view reveals itself. Files: `tests/test_views.py` ### Done when - [ ] Editing through a view updates the table - [ ] A record leaving a filter stays in the table - [ ] A view cannot show a rule-excluded record - [ ] Two views reflect an edit made in either ### Watch out - A view that materialises its own rows makes editing ambiguous and diverges within a day, and worse, a view defined with a filter is not access control even though it looks like one. Views are convenience; rules are protection, and conflating them is how data leaks. ## M5 · An API with the same rules Everything the interface can do is available programmatically, under identical restrictions. ### Steps 1. Reuse the rule layer The same code path as the interface. Two implementations diverge. Files: `src/api.py` 2. Scope tokens narrowly Document and principal, revocable individually. Files: `src/tokens.py` 3. Test with a restricted token Read and write. Both must be refused where the rules say so. Files: `tests/test_api.py` ### Done when - [ ] A restricted API read matches the interface for that principal - [ ] A forbidden API write is refused - [ ] Tokens are scoped and individually revocable - [ ] Documented and tested behaviour agree ### Watch out - An API that bypasses access rules because it is for internal use is the gap that gets found later, usually by whoever integrated with it and did not realise. One rule layer, applied to both surfaces, tested with a restricted token. ## M6 · Export, backups and rule review (production only) Data is portable, recoverable, and the rules are reviewed rather than assumed. ### Steps 1. Export data and schema Spreadsheets for the values, a readable schema for types and formulas. Files: `src/export.py` 2. Back up and restore Including rules. A restore where rules are missing is worse than no restore. Files: `scripts/backup.sh`, `README.md` 3. Build the rule report Per principal, what they can see. Verify one against a hand check. Files: `src/rule_report.py` ### Done when - [ ] Export produces spreadsheets and a readable schema - [ ] A restore reproduces rules, verified with a restricted principal - [ ] The rule report matches a hand check for one principal - [ ] Rules can be exported for review ### Watch out - Access rules accumulate and interact, and nobody can hold the combined effect in their head after the fifth rule. A report showing what each principal can actually see is the only way to review them, and it is worth building before the rules get complicated rather than after.
OPERATIONS.md
# Operations · Grist ## Backup The database holds data, formulas and rules together. A restore missing rules would silently widen access, which makes rules the most important part of the backup rather than an afterthought. ## Restore Restore and verify with a restricted principal that they still see only their rows. That check, rather than the document opening, is what proves the restore. Do a restore drill before the first real user, and write the date here when it passes. ## Monitoring Rule evaluation errors and API refusals by principal. A spike in refusals usually means a rule changed and someone lost access they needed, which is better caught by you than reported by them. ## Incident checklist If someone saw data they should not, check whether a view was mistaken for a rule, since that is the most likely cause. The rule report is how you establish what each principal could actually see at the time. 1. Contain the issue without destroying evidence or user data. 2. Record the timeline and affected scope. 3. Rotate exposed secrets and revoke compromised sessions or credentials. 4. Restore from a verified backup when needed. 5. Document the root cause, the remediation and the regression test. ## Release gate - [ ] A restricted API response is confirmed to contain only permitted rows - [ ] A hidden column is confirmed absent from the raw response - [ ] Reordering rows is confirmed not to change formula results - [ ] A view is confirmed unable to widen access - [ ] The rule report matches a hand check for one principal ## Launch constraint Do not market omitted Grist capabilities as implemented. The non-goals in `PRODUCT.md` remain user-visible limitations until they are deliberately delivered.
.env.example
# Copy to .env and fill in. Never commit .env; this file documents it. # Required · secret. Your PostgreSQL connection string. DATABASE_URL=postgres://user:pass@localhost:5432/tables # Required. Maximum dependency chain length, catching pathological documents before they hang. MAX_FORMULA_DEPTH=20 # Required. Writes per minute per token. API_WRITE_RATE_LIMIT=60 # Required. Ceiling on an import, so a mistaken file does not consume the database. IMPORT_MAX_ROWS=200000 # Required. Where rule reports are written for review. RULE_REPORT_PATH=./reports
$ choose a build depth, inspect the files, then open the complete pack in your agent
People still pay for Grist because teams pay when a flexible database becomes a shared operational system with integrations and years of accumulated business logic. The recurring cost buys schema changes, formulas, permissions, imports, attachments, views, APIs, concurrency, backups, and migrations, not just the visible interface.
xmature formula engine, access rules, collaboration, integrations, and hosting
xreal-time multiplayer
xautomation and integration breadth
xscale and formula depth
xenterprise permissions
share on X ↗"I just replaced Grist ($8/mo) with one prompt"
Don't feel like building it? These folks already made it free.
Teable CommunityRelational tables, formula fields, forms, and views in a tidy self-hosted grid.open source↗
GristThe actual product, self-hosted: formulas, relational records, forms, and views in one small workspace.open source↗
Baserow CommunityA friendlier database grid with relations and formulas; less spreadsheet-like, more app-builder-like.open source↗all 3 free alternatives to Grist →· no votes, no pay-to-list · just what's real
Grist pricing
| plan | monthly | annual (per mo) | what you get |
|---|---|---|---|
| cloud free | $0/workspace | $0/workspace | Unlimited documents; 5,000 records/document; 30-day snapshots; 5,000 API calls/document/day; 1 GB attachments/document; 2 free guests/document; 100 AI credits lifetime. |
| cloud pro | $10/user | $8/user | 100,000 records/document; 3-year history; 40,000 API calls/document/day; 3 GB attachments/document; 100 AI credits/month. |
| cloud business | $30/user | $24/user | 150,000 records/document; 5-year history; 60,000 API calls/document/day; 20 GB attachments/document; 2,000 AI credits/month. |
| cloud enterprise | custom | — | Custom record, API, storage and support limits; enterprise controls; pricing page displays a 50-user minimum. |
| self-hosted community | $0/workspace | $0/workspace | Open-source Community edition on the customer's own infrastructure. |
| self-hosted grist | $30/user | $24/user | Paid self-hosted edition with advanced administration, AI, automations and audit features. |
| self-hosted enterprise | custom | — | Enterprise self-hosted controls and support; 50-user minimum. |
| hosting services | custom | — | Managed hosting and services; public page gives no numeric capacity or price. |
free tierCloud Free: unlimited documents, 5,000 records/document, 30-day snapshots, 5,000 API calls/document/day, 1 GB attachments/document, 2 guests/document and 100 AI credits lifetime; self-hosted Community is also $0 on your infrastructure
billingcloud and paid self-hosted plans offer monthly + annual billing; annual prices shown are 20% lower; Enterprise and hosting are quote-based
hidden costsCloud Business has a 5-user minimum; Enterprise offerings show a 50-user minimum; only 2 guests/document are free on Cloud; AI add-ons cost $10/month for 500 credits or $29/month for 2,000; self-hosting infrastructure is separate.
verified 2026-08-11 · source ↗
Vibecode Grist
Kinda. The core of Grist is buildable in a weekend with the prompt on this page, but there are real gaps: mature formula engine, access rules, collaboration, integrations, and hosting, real-time multiplayer. Read the honest list above before committing.
How much does Grist cost?
Grist costs about $8/month (Pro, checked 2026-07-31), which is $96 per year.
What do I lose by replacing Grist?
Honestly: mature formula engine, access rules, collaboration, integrations, and hosting; real-time multiplayer; automation and integration breadth; scale and formula depth; enterprise permissions. If any of those are load-bearing for you, keep paying.
Is there an open-source alternative to Grist?
Yes: Teable Community (Relational tables, formula fields, forms, and views in a tidy self-hosted grid.) Grist (The actual product, self-hosted: formulas, relational records, forms, and views in one small workspace.) Baserow Community (A friendlier database grid with relations and formulas; less spreadsheet-like, more app-builder-like.) All 3 curated free alternatives are at vibecodeit.com/grist/alternatives. The prompt is for when you want it exactly your way.