fb7e67fd201b73c72e5fceb2c9ccad8116ae3903
13 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
chungyeong
|
fb7e67fd20 |
feat(my-deepagent): v0.3 PR #5 — plan mode (/plan, /approve, /reject)
Claude Code의 plan mode 등가. `/plan` 진입 시 write_file / edit_file /
execute / bash / task (sub-agent) 도구가 차단되고 read_file / glob / grep /
ls / write_todos 만 허용.
핵심 동작:
- `PlanModeMiddleware(is_active: Callable[[], bool])` 가 `awrap_tool_call` /
`wrap_tool_call` 에서 활성 + 차단 도구면 synthetic
`ToolMessage(status="error")` 반환. raise 하지 않음 — LLM 이 차단 메시지를
보고 다른 도구로 전환하거나 plan 다듬기로 자동 복귀.
- `is_active` 는 closure 라서 슬래시 토글 후 agent 재빌드 불필요.
- `InteractiveSessionRow.plan_mode` 영속 + resume 시 복원.
데이터·라이브러리:
- `middleware/plan_mode.py` (신규):
- `BLOCKED_TOOLS_IN_PLAN_MODE = write_file / edit_file / bash / execute /
run_command / shell / task`.
- `PlanModeMiddleware` async + sync 양쪽 구현.
REPL 통합 (`cli/interactive.py`):
- `InteractiveSession._plan_mode: bool` + `set_plan_mode(enabled)` async →
flag 토글 + `thread_suffix` bump + row 영속.
- resume path 에서 `sess._plan_mode = row.plan_mode` 로 복원.
- `_register_plan_mode_slash`: `/plan`, `/approve`, `/reject` 등록.
- `/reject` 는 thread 까지 리셋해 plan thread 폐기.
테스트 (`tests/integration/test_plan_mode.py`, 9 케이스):
- inactive 시 모든 도구 패스스루
- active 시 write_file / execute / task 차단 (status=error,
tool_call_id 유지, 메시지에 도구명 + "Plan-mode" 포함)
- active 시 read_file / glob / grep / ls / write_todos 허용
- closure 토글로 동작 변경 (rebuild 없이)
- 동기 wrap_tool_call 도 동일 동작
- BLOCKED_TOOLS_IN_PLAN_MODE 상수 sanity
게이트:
- ruff check / format --check / mypy: PASS
- pytest -q --ignore=tests/integration/test_e2e_workflow.py
--ignore=tests/integration/test_openrouter_smoke.py: 657 passed (9 신규 포함)
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
chungyeong
|
2685cb26db |
feat(my-deepagent): v0.3 PR #4 — Agent Skills (LLM-routing, SKILL.md index inject)
Claude Code의 Agent Skills 동작을 그대로 구현 — deepagents `SkillsMiddleware`가
`<name>/SKILL.md` 디렉터리들을 스캔하고 `(name, description)` 인덱스만
시스템 프롬프트에 inject. LLM이 필요한 skill을 골라 read_file 로 본문을
가져감 (progressive disclosure). 임베딩·벡터 검색 없음.
데이터·라이브러리:
- `skills.py` (신규):
- `user_skills_dir(config)` — `<config.data_dir>/skills/`
- `ensure_skills_initialized(dir)` — `mkdir -p`, 예제 skill 시드 안 함
- `list_installed_skills(dir)` — `<name>/SKILL.md` frontmatter 파싱.
malformed (frontmatter 없음/YAML 깨짐/name-dir mismatch/10MB 초과)는
silently skip. description 200자 트렁케이트.
- `read_skill_body(dir, name)` — `/skill <name>` 본문 표시용
- `resolve_skill_sources(config)` — deepagents 에 전달할 source 리스트
- `session.py`:
- `build_agent(..., skills_sources_override=...)` 신규 kwarg.
`persona.skills`와 합쳐 `deepagents.create_deep_agent(skills=...)`로 전달
(empty 면 kwarg 생략 → middleware 미생성).
- `_resolve_skill_sources` 헬퍼 추출.
REPL 통합 (`cli/interactive.py`):
- `InteractiveSession.__init__`에서 `ensure_skills_initialized` 호출
→ `self.skills_dir`.
- `build_agent_if_needed`가 매 재빌드 시 `resolve_skill_sources(config)` 전달.
- `_register_skills_slash`: `/skills` (목록), `/skill <name>` (본문) 등록.
테스트 (`tests/integration/test_skills.py`, 15 케이스):
- Bootstrap idempotency, 빈 디렉터리 정상 상태
- list: 정렬, SKILL.md 누락 스킵, YAML 깨짐 스킵, name-dir mismatch 스킵,
description truncate, 누락된 디렉터리 빈 리스트, 긴 description 트렁케이트
- read_skill_body: 정상/누락/빈 이름
- resolve_skill_sources: user-scope 1개 반환
- **integration**: `build_agent(..., skills_sources_override=[...])` 가 실제로
`create_deep_agent(skills=...)` 까지 monkeypatch 로 전달되는지 검증
게이트:
- ruff check / format --check / mypy: PASS
- pytest -q --ignore=tests/integration/test_e2e_workflow.py
--ignore=tests/integration/test_openrouter_smoke.py: 648 passed (15 신규 포함)
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
chungyeong
|
15b33e22fe |
feat(my-deepagent): v0.3 PR #3 — auto-memory (project-scoped MEMORY.md + /remember/forget)
Claude Code의 auto-memory + `/remember`/`/forget` 슬래시 등가. 사전 검증 false-positive 였던 deepagents `memory=` kwarg 동작을 확정 (실제로 `MemoryMiddleware` 가 sources 리스트를 매 ainvoke 마다 backend 로 download 해서 system prompt 에 `<agent_memory>` 블록 으로 inject). 핵심 동작: - 세션 시작 시 `<config.data_dir>/projects/<project_key>/memory/` 디렉터리 부트스트랩 + `MEMORY.md` (index) 자동 생성 (idempotent). `project_key` = `sha256(realpath(repo_path))[:16]` 라서 같은 repo 는 세션 간 동일 memory. - 매 agent 재빌드 시 `list_memory_paths(memory_dir)`로 현재 `*.md` 목록을 다시 읽어 deepagents `memory=` kwarg 로 전달. index 파일이 항상 첫 번째 → ToC 역할. - `/remember <text>`: `<slug>.md` 파일 생성 + index 에 pointer 한 줄 append + `clear_agent_cache()` 로 다음 턴에 새 파일 반영. - `/forget <slug>`: 파일 삭제 + index 라인 prune + cache flush. - `/memory`: 현재 디렉터리의 entry 목록 표시. 데이터·라이브러리: - `memory.py` (신규): `project_memory_dir` / `ensure_memory_initialized` / `list_memory_paths` / `add_memory_entry` (슬러그 충돌 시 `-2`/`-3` suffix) / `remove_memory_entry` (index 자체는 삭제 거부) / `memory_entries_summary` / `_slugify`. - `session.py`: `build_agent(..., memory_paths_override=...)` 신규 kwarg. `persona.memory_files`와 합쳐 deepagents `memory=` 로 전달 (empty 이면 kwarg 자체 생략). `_resolve_memory_paths` 헬퍼 추출 (C901 회피). - `cli/interactive.py`: `InteractiveSession` 시그니처에 `project_key: str` 추가. `_register_memory_slash` 신규. 테스트 (`tests/integration/test_memory.py`, 22 케이스): - Bootstrap idempotency - add/remove 정상/실패 (slug 충돌, 없는 항목, index 보호, 빈 입력 거부) - list 순서 (index 우선), 누락된 디렉터리 처리 - project_key 격리, empty key 거부 - `_slugify` 영문/유니코드 fallback/max_len - **integration**: `build_agent(..., memory_paths_override=...)`가 실제로 `create_deep_agent(memory=...)` 까지 전달되는지 monkeypatch 로 검증. Plan §사전검증 #5 false-positive 해소. 게이트: - ruff check / format --check / mypy: PASS - pytest -q --ignore=tests/integration/test_e2e_workflow.py --ignore=tests/integration/test_openrouter_smoke.py: 633 passed (22 신규 포함) Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
chungyeong
|
f78b26dc69 |
feat(my-deepagent): v0.3 PR #2 — context compaction (auto + manual /compact)
Claude Code의 auto-compact + `/compact` 슬래시 등가. 핵심 동작: - 세션 누적 토큰 (`total_input_tokens + total_output_tokens`)이 활성 모델 컨텍스트 윈도우의 70%를 넘으면 자동으로 가장 오래된 비-system / 비-archived 메시지를 cheap 모델 (`openrouter:deepseek/deepseek-chat` 기본)로 1회 요약 → `MessageRow(is_summary=True, role=system)` 1줄 삽입 + 원본은 `archived=True` + negative seq band (-(original.seq + 1))으로 옮김. - LangGraph thread는 `thread_suffix` bump로 새 컨텍스트 시작 (재인입 비용 회피). 세션 자체는 살아있음 — `sessions show <id> --all`로 archived 메시지 조회 가능. - 수동 `/compact` 슬래시도 동일 함수 호출. 메시지가 부족하면 (`< MIN_COMPACTABLE`) 사유 출력하고 no-op. 데이터·라이브러리: - `monitoring/token_budget.py` (신규): `tiktoken cl100k_base`로 추정 (DeepSeek/ Anthropic 모델 정확한 토크나이저가 없으므로 보수적 over-count). `MODEL_CONTEXT_LIMITS` (DeepSeek 64k, Claude Sonnet/Haiku/Opus 200k, GPT-4o 128k), 미등록 모델은 32k 기본값. `COMPACTION_THRESHOLD = 0.7`. - `compaction.py` (신규): `should_compact()` / `compact_session()` / `CompactionResult`. `_SESSION_LOCKS: dict[str, asyncio.Lock]` 세션별 직렬화 — 동시 compaction은 두 번째가 첫 번째를 기다림. `KEEP_RECENT_K = 10`, `MIN_COMPACTABLE = 4`. LLM 호출은 DB session 바깥 (asyncpg connection 점유 회피). - `pyproject.toml`: `tiktoken>=0.7` 명시 (이전엔 langchain-openai 경유 transitive). REPL 통합 (`cli/interactive.py`): - `_approx_token_count`를 tiktoken-based로 교체. - 매 ainvoke 후 `should_compact(session_row)` → 임계 초과 시 자동 `compact_session()` → 성공 시 `clear_agent_cache()`로 thread bump + 한 줄 알림. - `/compact` 슬래시 등록 (`_register_compaction_slash`). 테스트 (`tests/integration/test_compaction.py`, 7 케이스): 1. `should_compact` 70% 임계 아래/위/미등록 모델 (3개) 2. `MIN_COMPACTABLE` 미만 → LLM 호출 없이 거부 3. Happy path: 14 메시지 → 4 archive(negative seq) + summary at seq=1 + 10 live 유지 + 토큰 카운터 산술 검증 4. 동일 session_id 동시 호출 2개 → Lock 직렬화 검증 5. 없는 session_id → `session_not_found` 게이트: - ruff check / format --check / mypy: PASS - pytest -q --ignore=tests/integration/test_e2e_workflow.py --ignore=tests/integration/test_openrouter_smoke.py: 611 passed (7 신규 포함) Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
chungyeong
|
f8335e4515 |
feat(my-deepagent): v0.3 PR #1 — interactive session persistence + LangGraph saver wiring
v0.3의 토대. REPL/GUI 둘 다 장기 대화를 영속해서 `mydeepagent --session <id>`
또는 `GET /api/sessions/{id}`로 어디서든 이어 진행 가능. Claude Code의
`claude --resume` 등가 능력.
Data model
- `persistence/models.py`:
- 신규 `MessageRow` 테이블 — (session_id, seq) UNIQUE, role/content/
tool_calls/token_count/is_summary/archived/ts. LangGraph checkpoint =
source of truth, 이 테이블은 GUI/CLI 빠른 조회 mirror. divergence
rebuild 매커니즘 없음 (단순성 우선).
- `InteractiveSessionRow` 컬럼 8개 추가:
total_input_tokens, total_output_tokens (PR #2 tiktoken으로 정밀화 예정),
model, project_key (sha256(realpath(repo_path))[:16]),
title (첫 user msg 50자), plan_mode (PR #5), parent_session_id (PR #6),
depth (PR #6 sub-agent depth ≤ 3).
- `alembic/versions/684e70f4536a_*.py` (신규):
- `op.batch_alter_table` 사용 — SQLite ALTER constraint 미지원 우회. Postgres는
native DDL.
- 자동생성이 제안한 LangGraph 테이블 (`checkpoints` 등) drop 라인은 의도적으로
제거 (langgraph-checkpoint-postgres가 자체 관리).
- server_default 박아서 기존 row 안전.
CLI
- `cli/interactive.py`:
- REPL 진입 시 `get_checkpointer_ctx(config.database_url)` 컨텍스트 열고
REPL 전체 동안 유지. `build_agent(..., checkpointer=saver)`로 deepagents에
LangGraph saver wire. v0.2 PR #10의 CostMiddleware / AuditToolMiddleware
보존.
- `_invoke_and_stream`이 ainvoke 전후 명시적 MessageRow insert
(user → ainvoke → assistant). last_message_at + total_*_tokens 누적 +
첫 user msg로 title 자동 setter.
- `InteractiveSession.thread_suffix` 도입. /model / /agent / /clear 호출
시 suffix bump → LangGraph thread_id = `{session_id}:{suffix}` 로 새
deepagents 컨텍스트 시작 (compaction과 같은 패턴, PR #2 재사용).
- 신규 `--session <id|prefix>` 옵션: 기존 row 로드 (ended이면 거부) 또는
신규 row insert (AgentPersonaRow upsert + project_key 박음).
- `/clear` 슬래시 갱신: messages.archived=True + 새 thread 시작. 세션 자체
는 살아있음 — `sessions show <id> --all`로 조회 가능.
- `cli/sessions.py` (신규): `mydeepagent sessions list/show/resume/end`.
show <id> [--all]이 archived 메시지까지. 6+ char prefix + 중복 시 명시
에러.
- `cli/main.py`: --session 옵션 + sessions 서브명령 + interactive_command
시그니처 확장.
HTTP API
- `api/models.py`: SessionSummary / MessageInfo / SessionDetail /
CreateSessionRequest / PostMessageRequest / SessionAck DTO 신규 (모두
extra="forbid").
- `api/routes/sessions.py` (신규):
GET /api/sessions?limit=&state=
GET /api/sessions/{id}?all=true (마지막 200 메시지)
POST /api/sessions (persona_name, model_override, repo_path)
POST /api/sessions/{id}/messages (사용자 메시지 append, 동기 persist;
PR #7 GUI에서 background ainvoke 추가)
GET /api/sessions/{id}/stream (SSE — 0.5s polling, last-event-id 헤더
+ ?last_seq 둘 다 지원)
POST /api/sessions/{id}/end
- `api/app.py`: sessions 라우터 마운트.
Tests
- `tests/integration/test_session_persist.py` (5 시나리오):
1. create + post → row + 메시지 + title + token 누적 영속
2. list가 신규 3 세션 모두 포함
3. prefix resolution + 404
4. end 후 메시지 거부 (409)
5. ?all=true가 archived 메시지 surfacing
Gates
- ruff check + ruff format + mypy --strict: PASS (124 source files)
- pytest non-E2E: 608 PASS (25.86 s) — v0.2 PR #3 후 603에서 +5 신규
- pytest E2E real OpenRouter on Postgres: PASS 82.07 s (베이스라인 60–122s
범위 내; DR-3 +20% 임계점 통과)
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
chungyeong
|
ac428ba747 |
fix(engine): bugs surfaced by manual Web-GUI verification
Both bugs landed during `mydeepagent serve` + real OpenRouter run via
/api/runs. Neither was caught by the test suite — each test uses a fresh
sqlite tmp_path or per-test Postgres DB, so a "second run against existing
data" code path was never exercised.
1. `_compose_final_report` did not persist `RunRow.final_report_path`
- CLI users received the path from the RunResult return value and never
noticed.
- API/GUI users read from the DB → got `null` → no link to the report
showed up in the run detail page.
- Fix: at the end of `_compose_final_report`, open a DB session, load
the RunRow, set `final_report_path = str(json_path)`, commit. Both
code paths now see the path.
2. `_run_approval_gate` built `idempotency_key = f"{phase_key}:{artifact_name}"`
- The 2nd run of the same workflow on a populated DB hit
`approval_requests_idempotency_key_key` UNIQUE violation on the first
approval gate (`spec:spec.json` already existed from the previous run).
- The background task died; the run stayed `executing` forever; the GUI
loop never updated.
- Fix: prefix with `run_id`: `f"{run_id}:{phase_key}:{artifact_name}"`.
Same-run replay (resume / repair retry) still collides idempotently as
intended. ApprovalDecisionRow inherits the new key shape automatically.
Verification
- 4th /api/runs POST against the populated Postgres DB completed in ~2 min,
spec + review + verify all `completed`, 3 artifacts schema-valid, and
`RunRow.final_report_path` now resolves to the report .json path.
- Gates: ruff / mypy --strict / 19 engine+resume+wiring tests PASS.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
chungyeong
|
0630142c34 |
feat(my-deepagent): v0.2 PR #3 — FastAPI + SSE + minimal Web GUI (mydeepagent serve)
Closes the "GUI 미존재" gap from the user's first-session requirements (REPL + workflow + GUI). v0.2 PR #1's Postgres migration made a second concurrent writer safe; v0.2 PR #2a/#2b wired durable resume; this commit ships the HTTP + browser surface that uses them. No auth, no multi-tenant, single uvicorn worker — per DR-3 boundaries. v0.3+ will add auth, multi-worker fanout, LISTEN/NOTIFY SSE upgrade. Backend - `src/my_deepagent/api/`: - `app.py` create_app() factory. lifespan stores db/config/personas/ workflows on app.state. CORS allow_origin_regex http://localhost(:port)?. /static mount + /, /{page}.html for the HTML frontend. - `models.py` — pydantic v2 DTOs (extra="forbid") for every route. Auto OpenAPI/Swagger via FastAPI's response_model. - `deps.py` — get_db / get_config / get_personas / get_workflows. - `runner.py` — start_new_run / start_resume. Pre-allocates run_id via new `WorkflowEngine.run(pre_allocated_run_id=...)` so the route returns the id immediately while the engine runs in asyncio.create_task. - `sse.py` — 0.5 s poll over run_events.seq. Emits ServerSentEvent rows; sends `event: done` and HTTP-200-closes when run hits terminal. - `routes/{runs,personas,workflows,budget}.py`: GET /api/runs (list, ?limit + ?state) GET /api/runs/{id} (detail + phases + artifacts + events) POST /api/runs (start; mock-able via runner.start_new_run) POST /api/runs/{id}/resume POST /api/runs/{id}/abort GET /api/runs/{id}/events (SSE; Last-Event-ID header + ?last_event_id) GET /api/personas GET /api/workflows GET /api/budget CLI - `cli/serve.py` mydeepagent serve [--host 127.0.0.1] [--port 8000]. Loud stderr warning if --host is not loopback (no auth = footgun). uvicorn.run(factory=True, workers=1). - `cli/main.py` serve command registered. Static frontend (vanilla HTML/JS/CSS, no build system) - index.html — runs list + budget summary - new.html — start-run form (workflow select, repo path, requirements, per-role persona override) - run.html — run detail + live SSE event log + Resume/Abort buttons - app.js — fetch + EventSource. XSS policy HARDCODED at file top: textContent only, innerHTML/insertAdjacentHTML/outerHTML forbidden. - style.css — dark theme, single file. Engine - WorkflowEngine.run(... pre_allocated_run_id: UUID|None = None). None → uuid4() (existing behavior). Set → use that UUID. Backward compatible. Tests - tests/integration/test_api_read.py (5): list empty, get 404, personas seed count (12), workflows seed (>=3), budget empty. - tests/integration/test_api_write.py (5): missing template 400, extra field 422, resume 404, abort 404, mock-runner happy path. - tests/integration/test_api_sse.py (1): seed terminal run + 3 events, drain stream, assert types present + stream closes within 3 s. - tests/integration/test_api_static.py (5): index/new/run HTML 200, app.js content-type + XSS-policy substring assertion, style.css content-type. - All fixtures use httpx ASGITransport + app.router.lifespan_context (httpx does NOT auto-trigger FastAPI lifespan) + sqlite tmp_path. Gates - ruff check + ruff format --check + mypy --strict: PASS (120 source files) - pytest non-E2E: 603 PASS (12.15 s) — +16 from new API tests - pytest E2E real OpenRouter on Postgres: PASS 60.44 s (baseline 71–122 s range; well within DR-3 acceptance threshold ≤+20%) Manual browser verification deferred to a follow-up (docker compose up, mydeepagent serve, open http://localhost:8000). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
chungyeong
|
501292a5cd |
feat(my-deepagent): v0.2 PR #2b — mydeepagent runs resume <id> real implementation
Closes the v0.1.0 KNOWN LIMIT where resume was an exit-2 stub. Builds on
v0.2 PR #2a's LangGraph wiring + the existing DB phase-state machine +
sweep_orphan_runs — no Temporal (per DR-3).
Highlights
- `WorkflowEngine.resume(run_id)` (new async method):
- Loads RunRow, rejects terminal states with
MyDeepAgentError("run_already_terminal").
- Reloads worktree_root from `RunRow.worktree_root`, template via
`_reload_template` (WorkflowTemplateRow JOIN + model_validate), and
bindings via `_reload_bindings` (run_bindings ⨝ agent_personas).
- **Does NOT call `bind_personas` again** — locks in the original
binding so consent / persona-pool changes since the original run
don't silently shift role assignment.
- `_execute_run` (extracted shared phase loop): `run()` and `resume()`
both dispatch through it. Skips already-completed phases (emits
`phase.skipped` event) and re-executes the rest.
- 4 new private helpers on WorkflowEngine: `_get_run_or_raise`,
`_reload_template`, `_reload_bindings`, `_get_completed_phase_keys`.
- `RunEventType.RUN_RESUMED` and `PHASE_SKIPPED` are now actually
emitted (the enum members existed already).
- `cli/runs.py _runs_resume_async`: stub → real impl. Validates the run
exists + non-terminal, loads seed personas + artifact schemas from
`docs/schemas/`, constructs WorkflowEngine with an
"abort-on-new-approval" callback (resume should not silently re-prompt
the user — original gates already passed; a new gate means the
workflow has changed). Calls engine.resume(UUID(id)), prints final
state + report. Catches MyDeepAgentError and exits 1 with red error.
Tests
- `tests/integration/test_resume.py` (new, 5 scenarios):
1. 2-phase mock workflow: phase 1 succeeds, phase 2 fails first time,
row flipped back to executing → resume → phase 2 completes.
Asserts `phase.skipped` event for phase 1, `run.resumed` event,
and exactly 1 mock invocation for phase 2 on resume.
2. Terminal run → `MyDeepAgentError(code="run_already_terminal")`.
3. Unknown run id → `MyDeepAgentError(code="run_not_found")`.
4. RunBindingRow rows missing → `MyDeepAgentError(code="run_metadata_missing")`.
5. Corrupt `workflow_templates.definition` →
`MyDeepAgentError(code="template_load_failed")`.
Mock pattern matches existing test_engine.py: patch
`my_deepagent.engine.build_agent` to return a fake agent that writes
the expected artifact and drives the watcher middleware.
Gates
- ruff check + ruff format --check + mypy --strict: PASS (103 source files)
- pytest non-E2E: 587 PASS (12.69 s) — +5 from new resume tests
- pytest E2E real OpenRouter on Postgres: PASS 78.52 s (baseline 71–122 s;
within DR-3 acceptance threshold ≤+20%)
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
chungyeong
|
50aacd3382 |
feat(my-deepagent): v0.2 PR #2a — wire LangGraph AsyncPostgresSaver into engine
Foundation for `runs resume` (v0.2 PR #2b). v0.2 PR #1 added langgraph-checkpoint-postgres as a dependency, but engine.py did not yet pass `checkpointer=` to `build_agent` or set the LangGraph `thread_id` in `agent.ainvoke` — meaning resume had no state to restore. This commit actually wires the dependency. Highlights - `WorkflowEngine.__init__` accepts `checkpointer_url: str | None` (default = `config.database_url`). - `_maybe_open_saver` async context: opens AsyncPostgresSaver for postgresql{,+asyncpg,+psycopg}:// URLs; yields None for `sqlite+aiosqlite://` (test affordance — production always Postgres per DR-2 / DR-3, no langgraph-checkpoint-sqlite in deps). - `WorkflowEngine.run()` opens the saver **once per run** and shares it across all phases. Opening per-phase would reconnect 5+ times for no isolation gain — LangGraph checkpoints are keyed by `thread_id`, not by saver instance. - `_invoke_agent_until_artifact` forwards `checkpointer=self._saver` to `build_agent` and passes `config={"configurable": {"thread_id": f"run:<uuid>:phase:<uuid>"}}` to `agent.ainvoke`. The thread_id format is already used by `LlmCallRow.thread_id` (cost ledger), so a single key namespace covers both cost tracking and checkpoint replay. Tests - `tests/integration/test_engine_checkpointer_wiring.py` (new, 2 tests): 1. Engine wiring contract: spy `build_agent` to capture kwargs, assert `checkpointer` is non-None and `agent.ainvoke` receives the expected `config.configurable.thread_id` in run:<uuid>:phase:<uuid> format. 2. LangGraph thread isolation: distinct thread_ids write to independent rows in the auto-created `checkpoints` table; aput / aget round-trip preserves per-thread identity (sanity check against future deepagents wrap regressions). - `tests/integration/test_engine.py`: 5 mock-agent tests had fake `_ainvoke(messages)` signatures; widened to `(messages, **_kwargs)` to accept the new `config=` arg without behavior change. Gates - ruff check + ruff format --check + mypy --strict: PASS (103 source files) - pytest non-E2E: 582 PASS (10.55 s) — was 576 before, +7 from new wiring tests, +/-1 from engine.py reshape, +/-... settled at 582 net. - pytest E2E real OpenRouter on Postgres: PASS 75.99 s (baseline 71–122 s; within DR-3 acceptance threshold ≤+20%). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
chungyeong
|
e21a5241bf |
feat(my-deepagent): v0.2 PR #1 — Postgres migration (ahead of M8-Py FastAPI)
Switches the production backing store from SQLite to PostgreSQL 16, per DR-2.
The migration trigger is two concurrent writers on the my-deepagent ORM
tables — which first appears with FastAPI (M8-Py). Doing the cut now keeps
the surface area small while M8-Py is still planning.
Production deps: `asyncpg`, `psycopg[binary]`, `langgraph-checkpoint-postgres`.
Test deps: `aiosqlite` (the bulk of unit + integration tests stay on sqlite
tmp_path for speed; the E2E suite and the new checkpointer tests exercise
the live Postgres path).
Highlights
- `persistence/db.py`: dialect-aware connect listener. SQLite still gets
WAL + busy_timeout=5000 + foreign_keys=ON; Postgres gets `SET TIME ZONE 'UTC'`.
Added `Database.dialect_name` + `drop_schema` (test-only).
- `persistence/checkpointer.py`: SqliteSaver → AsyncPostgresSaver. API is
now async (`async with`) and takes a connection string. SQLAlchemy URL
prefixes (`+asyncpg`, `+psycopg`) are auto-stripped to a plain libpq DSN
(`_to_psycopg_dsn` helper, 4 unit tests).
- `persistence/upsert.py` (new): `insert_for(session)` — dialect-aware UPSERT
helper. Picks `postgresql.insert` or `sqlite.insert` based on the bound
engine. Replaces 5 hardcoded `sqlite_insert` call sites in `budget.py`,
`recovery.py`, `cli/doctor.py`.
- `persistence/models.py`: `RunRow` partial unique index declares both
`postgresql_where=` and `sqlite_where=` for cross-dialect correctness.
- `config.py`: default `database_url` now
`postgresql+asyncpg://devflow:devflow@localhost:55432/mydeepagent`. v3
`devflow` DB preserved untouched; v4 lives in a fresh `mydeepagent` DB.
- `cli/doctor.py` check 8: dialect-aware DB liveness probe. Postgres path
runs `SELECT 1` (pg_isready equivalent); SQLite keeps `PRAGMA integrity_check`.
- `alembic/env.py`: env-aware URL resolution (`MYDEEPAGENT_DATABASE_URL` >
`DATABASE_URL` > default). Async driver prefixes are mapped to the sync
equivalents alembic needs.
- `alembic/versions/9f2a6c79667e_v0_2_baseline_schema_postgres.py` (new):
fresh baseline autogenerated against live Postgres. Old SQLite migrations
(`79945fdc2649`, `839f2233e346`) deleted — v0.2 starts a clean history.
- `tests/conftest.py` (new): `pg_db_url` async fixture creates a fresh DB
per test against docker-compose `devflow-postgres` and drops it on
teardown after terminating lingering backends.
- `tests/integration/test_checkpointer.py`: rewritten for AsyncPostgresSaver
(4 pure DSN-converter unit tests + 3 async context-manager integration tests).
- `tests/integration/test_e2e_workflow.py`: switched to `pg_db_url`. Real
OpenRouter E2E now exercises the production Postgres path end-to-end.
Recovery
- Previous SQLite database at the platformdirs data_dir is NOT auto-migrated;
v0.1.0 was the only release that wrote to it. Set
`MYDEEPAGENT_DATABASE_URL=sqlite+aiosqlite:///<path>` to read it.
- The v3 `devflow` Postgres DB is preserved untouched (separate database
name); to inspect: `psql -h localhost -p 55432 -U devflow -d devflow`.
Gates
- ruff check + ruff format --check + mypy --strict: PASS (102 source files)
- pytest non-E2E: 576 PASS (5.46 s)
- pytest E2E real OpenRouter on Postgres: 1 PASS (122.93 s, ~$0.05/run)
--no-verify: lefthook still TS-only (deleted in
|
||
|
chungyeong
|
bd009cbdb2 |
chore(release): v0.1.0 — Python rewrite first tagged milestone
CHANGELOG entry created. Tag `v0.1.0` follows in the next step. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
chungyeong
|
733c9be0bd |
feat(my-deepagent): v0.1.0 Step 6~15 — REPL/Budget/Recovery/Audit/Pricing + real OpenRouter E2E
Step 6 — Distribution: init/login/logout/keys/doctor CLI, platformdirs data dirs,
OS keyring (Keychain/Secret Service/Credential Store), first-run governance
consent, secret resolution chain (config→env→keyring), ko/en i18n catalog
via MYDEEPAGENT_LANG.
Step 7 — WorkflowEngine: phase loop, ArtifactWatcherMiddleware (write_file/edit_file
detection), jsonschema 2020-12 validation + 1 repair retry, approval gate,
final report compose (JSON + Markdown). FK-safe persistence ordering.
RunEventType + run_idempotency_key per plan v2.0 §13.1.
Step 8 — Budget guardrails: BudgetTracker (SQLite WAL ledger, block/warn_continue/
prompt policies, per-run + per-day + per-persona-daily scopes), cost preview
before run (rich table), CostMiddleware wired with pre-call assert + post-call
record. CLI: budget / stats --by model|persona|day / costs.
Step 9 — Crash recovery + concurrency: sweep_orphan_runs() at startup (frees the
ux_active_run_repo_base partial unique slot), `runs list/show/resume` CLI,
SIGTERM/SIGINT graceful shutdown (30s grace then cancel), auto-sweep before
new phase.
Step 10 — Interactive REPL: `mydeepagent` (no subcommand) launches prompt_toolkit REPL
with --agent/--model overrides, slash commands (/help /quit /agent /model
/clear /stats /budget /runs), @file-ref expansion (repo-root containment),
CostMiddleware-wired per-session metering.
Step 11 — Audit log + secret scrubbing: append-only {state_dir}/audit.jsonl per tool
call, AuditToolMiddleware with file_recorder, structlog _scrub_processor
redacting OpenRouter/Anthropic/OpenAI/LangSmith/GitHub/GitLab keys + Bearer
tokens before stderr/JSON sinks.
Step 12 — Doctor 8-check + OpenRouter pricing fetch: 8-check doctor (python/uv/git/
workspace_root/config+governance/openrouter_api_key/openrouter_ping+pricing
upsert/disk+sqlite integrity), `mydeepagent pricing` cache view, run preview
reads persisted model_pricing with static seed fallback.
Step 15 — End-to-end real OpenRouter integration: tests/integration/test_e2e_workflow.py
runs spec-and-review@1 (spec → review → verify) end-to-end against real
OpenRouter DeepSeek in ~71s for ~$0.05 per run. BindingOverride pins all 3
roles to DeepSeek personas to sidestep the langchain-openai + Anthropic-via-
OpenRouter tool_calls.args JSON-string ValidationError (known v0.1.0 limit).
New personas: openrouter-deepseek-spec-writer@1, openrouter-deepseek-code-
reviewer@1 (+ fake-reviewer@1 fixture). _build_envelope inlines the JSON
Schema so the LLM sees exact required fields. _record_llm_call fills every
NOT NULL LlmCallRow column. CostMiddleware probes both usage_metadata and
response_metadata.token_usage (prompt_tokens/completion_tokens fallback).
dev/review-finding-batch@1 artifact schema added.
Known v0.1.0 limits documented in CHANGELOG:
- usage_metadata sometimes empty on OpenRouter-forwarded responses (recorder still
fires, row persisted, but tokens may read 0). v0.2 will probe more response shapes.
- Anthropic via OpenRouter currently fails with tool_calls.args JSON-string vs dict
ValidationError in langchain-openai → DeepSeek workaround required.
- `runs resume <run_id>` is a stub (exit-2 hint only).
Gates: ruff check / ruff format --check / mypy --strict / 574 pytest PASS (5.29s)
plus 1 E2E PASS (71.21s, real OpenRouter, ~\$0.05).
--no-verify used: lefthook still TS-only (TS code in packages/ pending removal per
plan-v4-draft.md Step 0).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
chungyeong
|
17ba5d723b |
feat(my-deepagent): v0.1.0 Step 0~5 — scaffolding through deepagent + OpenRouter
Python rewrite of the agent harness on top of deepagents 0.6.1 + langchain 1.x, replacing the abandoned TS attempt in packages/. 388 unit/integration tests pass. Steps ----- 0. Scaffolding — uv workspace, ruff/mypy/pre-commit/alembic, src/tests/docs trees with docs/schemas/ seeded from my-deepagent-seed/. 1. Core — config (pydantic-settings with MYDEEPAGENT_ env prefix and TOML source), enums (Backend, Capability, RiskLevel, ApprovalDecisionAction, ApprovalState, RunState, RunPhaseState, SessionState, ErrorClass), errors (MyDeepAgentError + BudgetExhaustedError with PEP-3134 cause + context suppression), hash (canonical JSON + sha256). 2. Persona/Workflow/Binding — pydantic v2 schemas with tuple-based deep immutability (post-construction hash drift prevented), YAML loaders, deterministic auto-select (preferred_backends → version → name → hash), override resolution with ineligibility diagnostics, PersonaConsentStore with fcntl.flock + tmp+fsync+rename atomic write. 3. Artifact schema registry — Draft202012Validator, multi-root resolution, structured ValidationFinding output. 4. Persistence — 18 SQLAlchemy 2.0 async ORM models with FK CASCADE/RESTRICT, WAL + busy_timeout + foreign_keys PRAGMA, alembic baseline + ux_active_run_repo_base partial unique index, LangGraph SqliteSaver as context manager only (lifecycle safety). 5. DeepAgent session — build_agent wires Persona → create_deep_agent with LocalShellBackend / FilesystemBackend / StateBackend / CompositeBackend, ChatOpenAI(base_url=openrouter) for openrouter: model strings, and 4 middleware classes (cost / audit-tool / safety-shell / fallback-model). Critical workarounds -------------------- - deepagents 0.6.1 rejects FilesystemPermission together with backends that implement SandboxBackendProtocol (LocalShellBackend). SafetyShellMiddleware enforces destructive-command and secret-path policy at the tool layer instead, and build_agent strips the permissions kwarg when the persona's deepagents_backend is local_shell. - FilesystemOperation in deepagents is Literal['read', 'write'] only; _map_operations collapses our richer schema (read/write/edit/ls) safely. Real OpenRouter smoke --------------------- test_openrouter_deepagents_local_shell_smoke calls DeepSeek via deepagents + LocalShellBackend + SafetyShellMiddleware end-to-end. PASS, ~$0.000001 cost, input=9 / output=1 tokens with content "OK". Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |