docs(plan): correct DB roadmap — Postgres trigger is M8-Py (FastAPI), not M5-Py (Temporal)
Earlier v4 r1 wording implied Postgres would re-enter "with Temporal." That was a false equivalence: Temporal worker (M5-Py) runs against its own backing store (`temporal` namespace) and does not touch `my-deepagent`'s `runs` / `run_phases` / `llm_calls` ORM tables, so M5-Py does not force a DB migration. The actual trigger for Postgres is a *second concurrent writer* on the my-deepagent DB, which first appears with FastAPI in M8-Py (and the later web GUI). SQLite WAL allows only one concurrent writer. Changes: - §1.3 Database: replaced "Postgres parked indefinitely" with explicit migration-trigger table (CLI=1 writer → SQLite; Temporal worker=still 1 writer → SQLite; FastAPI=2 writers → Postgres required). Sequencing: v0.2 PR #1 (Postgres baseline regen) lands ahead of M8-Py for a clean cut. - §22 Decision Log: added DR-2 documenting this correction. - §23 Kickoff Order: inserted "v0.2 PR #1 — Postgres migration" between Step-0-purge and M5-Py; annotated M5-Py and M8-Py with their DB implications. Also clarifies that `temporalio` is listed in plan-v4-draft.md but is not yet pulled into `my-deepagent/pyproject.toml`; install happens with M5-Py. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
39
docs/plan.md
39
docs/plan.md
@@ -61,13 +61,36 @@
|
||||
|
||||
### 1.3 Database
|
||||
|
||||
- **SQLite 3 (WAL mode)** via **aiosqlite**, ORM: **SQLAlchemy 2.0 async**.
|
||||
- **v0.1.0: SQLite 3 (WAL mode)** via **aiosqlite**, ORM: **SQLAlchemy 2.0 async**.
|
||||
- Migrations: **Alembic** (baseline + per-feature revisions).
|
||||
- WAL + `busy_timeout=5000` + `PRAGMA foreign_keys=ON` enforced at connect.
|
||||
- Postgres (the v3 default) is parked: single-machine + single-user removes the
|
||||
multi-process concurrency justification, and aiosqlite + the
|
||||
`ux_active_run_repo_base` partial unique index covers the active-run
|
||||
uniqueness invariant. Postgres can be reinstated for multi-tenant later.
|
||||
|
||||
**Why SQLite for v0.1.0** — single CLI process is the only writer. No Docker
|
||||
prerequisite for the `mydeepagent` install path, no service boot ordering, no
|
||||
network. WAL handles concurrent reads from the same process fine. The
|
||||
`ux_active_run_repo_base` partial unique index covers the active-run
|
||||
uniqueness invariant.
|
||||
|
||||
**Migration trigger → Postgres 16 (planned for v0.2 PR #1, ahead of M8-Py).**
|
||||
The bound is when *a second process* starts writing to `my-deepagent`'s
|
||||
`runs` / `run_phases` / `llm_calls` tables. That happens with FastAPI
|
||||
(M8-Py) or the web GUI port — not with Temporal. Temporal worker (M5-Py)
|
||||
runs against its own backing store (`temporal` namespace, separate Postgres
|
||||
DB or sqlite-cluster) and does not touch `my-deepagent` ORM tables, so M5-Py
|
||||
does *not* force a DB switch by itself. Concretely the cut point is:
|
||||
|
||||
| Milestone | Writer count to my-deepagent DB | Verdict |
|
||||
|-----------|--------------------------------|---------|
|
||||
| v0.1.0 (CLI + REPL) | 1 (the CLI process) | SQLite OK |
|
||||
| M5-Py (Temporal worker) | still 1 (worker = "CLI" continuation; Temporal DB is separate) | SQLite still OK |
|
||||
| M8-Py (FastAPI HTTP server) | 2 (CLI + API server) | **Postgres required** — SQLite WAL allows only one concurrent writer |
|
||||
| Web GUI / multi-tenant | 2+ | Postgres |
|
||||
|
||||
When the switch lands, the migration plan is: stop writers → `alembic
|
||||
downgrade base` against sqlite → re-generate the baseline against
|
||||
Postgres (SQLAlchemy stays the same; only dialect-specific bits — JSON
|
||||
columns, partial unique index syntax, UPSERT — need review) →
|
||||
`pg_isready` doctor check.
|
||||
|
||||
### 1.4 Logging
|
||||
|
||||
@@ -1900,6 +1923,7 @@ M5+:
|
||||
| ID | Decision | Rationale | Impact |
|
||||
|----|----------|-----------|--------|
|
||||
| DR-1 | **v3 → v4 major bump: delete TS monorepo, rewrite in Python on LangChain `deepagents`.** | (1) Claude/Anthropic direct API cost is prohibitive for a single-user toolchain. (2) OpenRouter cost-tuned models (DeepSeek, etc.) require a multi-turn, tool-using agent harness; `deepagents` is Python-only with no 1:1 TS port. (3) Switching languages is shorter than reimplementing the harness. | Step 0 (commit `0e61b2d`) deleted `apps/`, `packages/`, `tests/`, `scripts/`, pnpm/tsconfig metadata. The Python rewrite lives at `my-deepagent/` and reached Step 15 (real OpenRouter E2E PASS, ~$0.05/run) before the v3 codebase was removed. CC-39's separate `OpenRouterAdapter` is replaced by `my_deepagent.session.build_agent` (deepagents 0.6.1 with LocalShellBackend + SafetyShellMiddleware). v3 CC counters frozen; v4 begins its own series. Recovery: `git checkout pre-python-rewrite -- <path>`. |
|
||||
| DR-2 | **SQLite for v0.1.0; Postgres migration scheduled as v0.2 PR #1, ahead of M8-Py FastAPI.** | The v4 r1 first draft suggested Postgres should re-enter "with Temporal." That was wrong: Temporal (M5-Py) does not write to the `my-deepagent` ORM tables — it has its own backing store. The real trigger for Postgres is *a second writer on `runs` / `run_phases` / `llm_calls`*, which first appears with FastAPI (M8-Py) and the eventual web GUI. Until then, SQLite WAL handles single-process concurrent reads fine and saves new users a Docker prerequisite at install time. | Migration sequencing: v0.2 PR #1 = "stop writers → `alembic downgrade base` against SQLite → regenerate baseline against Postgres → adjust JSON column types / partial-unique-index syntax / UPSERT for the Postgres dialect → add `pg_isready` doctor check." M5-Py (Temporal) can be implemented on either SQLite or Postgres my-deepagent DB; the order (v0.2-PR-1 → M5-Py → M8-Py) is chosen for stack consistency, not necessity. Supersedes the "Postgres parked indefinitely" wording from earlier v4 r1 drafts. |
|
||||
|
||||
### Future Open Questions
|
||||
|
||||
@@ -1946,5 +1970,6 @@ v4 r1 order (Python, status as of v0.1.0):
|
||||
| Step 14 | TUI recovery (M7-Py) | DEFERRED — not in v0.1.0 |
|
||||
| Step 15 | End-to-end real OpenRouter integration test | DONE (`733c9be`) |
|
||||
| Step 0-purge | Delete v3 TS monorepo per DR-1 | DONE (`0e61b2d`) |
|
||||
| M5-Py | Temporal worker (`apps/worker`) | NEXT |
|
||||
| M8-Py | FastAPI + SSE (`apps/api`) | NEXT |
|
||||
| v0.2 PR #1 | **Postgres migration** — Alembic baseline regen against Postgres 16; SQLite removed. Triggered by upcoming M8-Py multi-process writes, sequenced *before* M8-Py for a clean cut. Adds `pg_isready` doctor check; `mydeepagent doctor` no longer offers SQLite fast-path. | PLANNED (next) |
|
||||
| M5-Py | Temporal worker (`apps/worker`). Temporal server uses its own backing DB (separate Postgres `temporal` namespace) and does not touch `my-deepagent`'s ORM tables, so M5-Py works on either SQLite or Postgres my-deepagent DB. Targeted post-v0.2-PR-1 for stack consistency. | PLANNED |
|
||||
| M8-Py | FastAPI + SSE (`apps/api`). Requires Postgres (see §1.3 trigger table). | PLANNED |
|
||||
|
||||
Reference in New Issue
Block a user