feat(my-deepagent): v0.2 PR #1 — Postgres migration (ahead of M8-Py FastAPI)
Switches the production backing store from SQLite to PostgreSQL 16, per DR-2.
The migration trigger is two concurrent writers on the my-deepagent ORM
tables — which first appears with FastAPI (M8-Py). Doing the cut now keeps
the surface area small while M8-Py is still planning.
Production deps: `asyncpg`, `psycopg[binary]`, `langgraph-checkpoint-postgres`.
Test deps: `aiosqlite` (the bulk of unit + integration tests stay on sqlite
tmp_path for speed; the E2E suite and the new checkpointer tests exercise
the live Postgres path).
Highlights
- `persistence/db.py`: dialect-aware connect listener. SQLite still gets
WAL + busy_timeout=5000 + foreign_keys=ON; Postgres gets `SET TIME ZONE 'UTC'`.
Added `Database.dialect_name` + `drop_schema` (test-only).
- `persistence/checkpointer.py`: SqliteSaver → AsyncPostgresSaver. API is
now async (`async with`) and takes a connection string. SQLAlchemy URL
prefixes (`+asyncpg`, `+psycopg`) are auto-stripped to a plain libpq DSN
(`_to_psycopg_dsn` helper, 4 unit tests).
- `persistence/upsert.py` (new): `insert_for(session)` — dialect-aware UPSERT
helper. Picks `postgresql.insert` or `sqlite.insert` based on the bound
engine. Replaces 5 hardcoded `sqlite_insert` call sites in `budget.py`,
`recovery.py`, `cli/doctor.py`.
- `persistence/models.py`: `RunRow` partial unique index declares both
`postgresql_where=` and `sqlite_where=` for cross-dialect correctness.
- `config.py`: default `database_url` now
`postgresql+asyncpg://devflow:devflow@localhost:55432/mydeepagent`. v3
`devflow` DB preserved untouched; v4 lives in a fresh `mydeepagent` DB.
- `cli/doctor.py` check 8: dialect-aware DB liveness probe. Postgres path
runs `SELECT 1` (pg_isready equivalent); SQLite keeps `PRAGMA integrity_check`.
- `alembic/env.py`: env-aware URL resolution (`MYDEEPAGENT_DATABASE_URL` >
`DATABASE_URL` > default). Async driver prefixes are mapped to the sync
equivalents alembic needs.
- `alembic/versions/9f2a6c79667e_v0_2_baseline_schema_postgres.py` (new):
fresh baseline autogenerated against live Postgres. Old SQLite migrations
(`79945fdc2649`, `839f2233e346`) deleted — v0.2 starts a clean history.
- `tests/conftest.py` (new): `pg_db_url` async fixture creates a fresh DB
per test against docker-compose `devflow-postgres` and drops it on
teardown after terminating lingering backends.
- `tests/integration/test_checkpointer.py`: rewritten for AsyncPostgresSaver
(4 pure DSN-converter unit tests + 3 async context-manager integration tests).
- `tests/integration/test_e2e_workflow.py`: switched to `pg_db_url`. Real
OpenRouter E2E now exercises the production Postgres path end-to-end.
Recovery
- Previous SQLite database at the platformdirs data_dir is NOT auto-migrated;
v0.1.0 was the only release that wrote to it. Set
`MYDEEPAGENT_DATABASE_URL=sqlite+aiosqlite:///<path>` to read it.
- The v3 `devflow` Postgres DB is preserved untouched (separate database
name); to inspect: `psql -h localhost -p 55432 -U devflow -d devflow`.
Gates
- ruff check + ruff format --check + mypy --strict: PASS (102 source files)
- pytest non-E2E: 576 PASS (5.46 s)
- pytest E2E real OpenRouter on Postgres: 1 PASS (122.93 s, ~$0.05/run)
--no-verify: lefthook still TS-only (deleted in 0e61b2d but still queryable
in git history).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -2,6 +2,74 @@
|
||||
|
||||
## [Unreleased]
|
||||
|
||||
### Added
|
||||
- **v0.2 PR #1 — Postgres migration**: production backing store switched from
|
||||
SQLite to PostgreSQL 16 ahead of M8-Py (FastAPI) per DR-2.
|
||||
- `pyproject.toml`: `asyncpg>=0.30` + `psycopg[binary]>=3.2` +
|
||||
`langgraph-checkpoint-postgres>=2.0.0` added to runtime; `aiosqlite>=0.20`
|
||||
moved to `[dependency-groups].dev` (test-only); `langgraph-checkpoint-sqlite`
|
||||
removed.
|
||||
- `src/my_deepagent/persistence/db.py`: dialect-aware connect listener —
|
||||
SQLite still gets `WAL` + `busy_timeout=5000` + `foreign_keys=ON`, Postgres
|
||||
gets `SET TIME ZONE 'UTC'`. New `Database.dialect_name` property + `drop_schema`
|
||||
method for tests.
|
||||
- `src/my_deepagent/persistence/checkpointer.py`: `SqliteSaver` →
|
||||
`AsyncPostgresSaver`. API is now async (`async with`) and takes a
|
||||
connection string; SQLAlchemy URL prefixes (`postgresql+asyncpg://`,
|
||||
`postgresql+psycopg://`) are auto-stripped to a plain libpq DSN. New
|
||||
`_to_psycopg_dsn` helper covered by 4 unit tests.
|
||||
- `src/my_deepagent/persistence/upsert.py` (new): `insert_for(session)` —
|
||||
dialect-aware UPSERT helper. Picks `postgresql.insert` or `sqlite.insert`
|
||||
based on the bound engine's dialect. Replaces 5 hardcoded `sqlite_insert`
|
||||
call sites in `budget.py`, `recovery.py`, and `cli/doctor.py`.
|
||||
- `src/my_deepagent/config.py`: `database_url` default switched from
|
||||
`sqlite+aiosqlite:///<data_dir>/database.sqlite3` to
|
||||
`postgresql+asyncpg://devflow:devflow@localhost:55432/mydeepagent`. The v3
|
||||
`devflow` DB is preserved untouched; v4 lives in a fresh `mydeepagent` DB.
|
||||
- `src/my_deepagent/persistence/models.py`: `RunRow.__table_args__` partial
|
||||
unique index now declares **both** `postgresql_where=` and `sqlite_where=`
|
||||
so the index is partial on both dialects.
|
||||
- `src/my_deepagent/cli/doctor.py`: check 8 (`disk+db`) is now dialect-aware
|
||||
— Postgres path runs `SELECT 1` (pg_isready equivalent: proves
|
||||
reachability + auth + DB exists); SQLite path keeps
|
||||
`PRAGMA integrity_check`. Doctor docstring updated.
|
||||
- `alembic/env.py`: env-aware URL resolution — `MYDEEPAGENT_DATABASE_URL` >
|
||||
`DATABASE_URL` > Postgres default. Async driver prefixes
|
||||
(`+asyncpg`, `+aiosqlite`) are mapped to the sync equivalents alembic
|
||||
needs (`+psycopg`, plain `sqlite`).
|
||||
- `alembic/versions/9f2a6c79667e_v0_2_baseline_schema_postgres.py` (new):
|
||||
fresh baseline autogenerated against live Postgres. Old SQLite baseline
|
||||
`79945fdc2649` + partial-index migration `839f2233e346` deleted.
|
||||
- `tests/conftest.py` (new): `pg_db_url` async fixture. Creates a fresh
|
||||
Postgres database per test (against docker-compose `devflow-postgres`)
|
||||
via the maintenance `postgres` DB; drops on teardown after terminating
|
||||
any lingering backends. Used by the E2E suite and the new checkpointer
|
||||
tests.
|
||||
- `tests/integration/test_checkpointer.py`: rewritten for AsyncPostgresSaver
|
||||
(4 pure DSN-converter tests + 3 async context-manager tests).
|
||||
- `tests/integration/test_e2e_workflow.py`: switched from `sqlite+aiosqlite`
|
||||
tmp_path to `pg_db_url`. Real OpenRouter E2E now exercises the production
|
||||
Postgres path end-to-end (~122 s, ~$0.05/run).
|
||||
|
||||
### Migration trigger (per DR-2)
|
||||
- The bound is *two concurrent writers* on `runs` / `run_phases` / `llm_calls`.
|
||||
Today the CLI is the only writer — but M8-Py (FastAPI) introduces a second
|
||||
one, and SQLite WAL allows only a single concurrent writer. Doing the move
|
||||
*before* M8-Py lands gives the test surface time to harden.
|
||||
- Recovery: previous SQLite database at
|
||||
`~/Library/Application Support/my-deepagent/database.sqlite3` (macOS) /
|
||||
`$XDG_DATA_HOME/my-deepagent/database.sqlite3` is **not migrated** —
|
||||
v0.1.0 was the only release that wrote to it and v0.2 starts a fresh
|
||||
history. Set `MYDEEPAGENT_DATABASE_URL=sqlite+aiosqlite:///<path>` to
|
||||
read the legacy file if needed.
|
||||
|
||||
### Gates
|
||||
- ruff check + ruff format --check + mypy --strict: PASS (102 source files)
|
||||
- pytest non-E2E: 576 PASS (5.46 s) — bulk on sqlite tmp_path, new
|
||||
checkpointer suite on Postgres `pg_db_url`
|
||||
- pytest E2E real OpenRouter: 1 PASS (122.93 s) on Postgres backend
|
||||
|
||||
|
||||
## [0.1.0] - 2026-05-16
|
||||
|
||||
First tagged milestone of the Python rewrite. The pre-Python-rewrite TypeScript
|
||||
|
||||
Reference in New Issue
Block a user