feat(my-deepagent): v0.1.0 Step 6~15 — REPL/Budget/Recovery/Audit/Pricing + real OpenRouter E2E

Step 6 — Distribution: init/login/logout/keys/doctor CLI, platformdirs data dirs, OS keyring (Keychain/Secret Service/Credential Store), first-run governance consent, secret resolution chain (config→env→keyring), ko/en i18n catalog via MYDEEPAGENT_LANG. Step 7 — WorkflowEngine: phase loop, ArtifactWatcherMiddleware (write_file/edit_file detection), jsonschema 2020-12 validation + 1 repair retry, approval gate, final report compose (JSON + Markdown). FK-safe persistence ordering. RunEventType + run_idempotency_key per plan v2.0 §13.1. Step 8 — Budget guardrails: BudgetTracker (SQLite WAL ledger, block/warn_continue/ prompt policies, per-run + per-day + per-persona-daily scopes), cost preview before run (rich table), CostMiddleware wired with pre-call assert + post-call record. CLI: budget / stats --by model|persona|day / costs. Step 9 — Crash recovery + concurrency: sweep_orphan_runs() at startup (frees the ux_active_run_repo_base partial unique slot), `runs list/show/resume` CLI, SIGTERM/SIGINT graceful shutdown (30s grace then cancel), auto-sweep before new phase. Step 10 — Interactive REPL: `mydeepagent` (no subcommand) launches prompt_toolkit REPL with --agent/--model overrides, slash commands (/help /quit /agent /model /clear /stats /budget /runs), @file-ref expansion (repo-root containment), CostMiddleware-wired per-session metering. Step 11 — Audit log + secret scrubbing: append-only {state_dir}/audit.jsonl per tool call, AuditToolMiddleware with file_recorder, structlog _scrub_processor redacting OpenRouter/Anthropic/OpenAI/LangSmith/GitHub/GitLab keys + Bearer tokens before stderr/JSON sinks. Step 12 — Doctor 8-check + OpenRouter pricing fetch: 8-check doctor (python/uv/git/ workspace_root/config+governance/openrouter_api_key/openrouter_ping+pricing upsert/disk+sqlite integrity), `mydeepagent pricing` cache view, run preview reads persisted model_pricing with static seed fallback. Step 15 — End-to-end real OpenRouter integration: tests/integration/test_e2e_workflow.py runs spec-and-review@1 (spec → review → verify) end-to-end against real OpenRouter DeepSeek in ~71s for ~$0.05 per run. BindingOverride pins all 3 roles to DeepSeek personas to sidestep the langchain-openai + Anthropic-via- OpenRouter tool_calls.args JSON-string ValidationError (known v0.1.0 limit). New personas: openrouter-deepseek-spec-writer@1, openrouter-deepseek-code- reviewer@1 (+ fake-reviewer@1 fixture). _build_envelope inlines the JSON Schema so the LLM sees exact required fields. _record_llm_call fills every NOT NULL LlmCallRow column. CostMiddleware probes both usage_metadata and response_metadata.token_usage (prompt_tokens/completion_tokens fallback). dev/review-finding-batch@1 artifact schema added. Known v0.1.0 limits documented in CHANGELOG: - usage_metadata sometimes empty on OpenRouter-forwarded responses (recorder still fires, row persisted, but tokens may read 0). v0.2 will probe more response shapes. - Anthropic via OpenRouter currently fails with tool_calls.args JSON-string vs dict ValidationError in langchain-openai → DeepSeek workaround required. - `runs resume <run_id>` is a stub (exit-2 hint only). Gates: ruff check / ruff format --check / mypy --strict / 574 pytest PASS (5.29s) plus 1 E2E PASS (71.21s, real OpenRouter, ~\$0.05). --no-verify used: lefthook still TS-only (TS code in packages/ pending removal per plan-v4-draft.md Step 0). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-16 16:32:46 +09:00
parent 17ba5d723b
commit 733c9be0bd
66 changed files with 8286 additions and 100 deletions
--- a/my-deepagent/tests/integration/test_budget.py
+++ b/my-deepagent/tests/integration/test_budget.py
@@ -0,0 +1,267 @@
+"""Integration tests for src/my_deepagent/budget.py (BudgetTracker)."""
+
+from __future__ import annotations
+
+from uuid import UUID, uuid4
+
+import pytest
+import pytest_asyncio
+
+from my_deepagent.budget import BudgetOnHit, BudgetTracker
+from my_deepagent.errors import BudgetExhaustedError
+from my_deepagent.persistence.db import Database
+
+# ---------------------------------------------------------------------------
+# Fixtures
+# ---------------------------------------------------------------------------
+
+_RUN_ID = UUID("00000000-0000-0000-0000-000000000001")
+
+
+@pytest_asyncio.fixture
+async def db(tmp_path: object) -> Database:
+    import tempfile
+    from pathlib import Path
+
+    p = Path(tempfile.mkdtemp()) / "test_budget.sqlite3"
+    database = Database(f"sqlite+aiosqlite:///{p}")
+    await database.init_schema()
+    return database
+
+
+def _make_tracker(
+    db: Database,
+    daily_cap: float = 5.0,
+    run_cap: float = 1.0,
+    on_hit: BudgetOnHit = BudgetOnHit.BLOCK,
+    prompt_callback: object = None,
+) -> BudgetTracker:
+    return BudgetTracker(
+        db=db,
+        daily_cap_usd=daily_cap,
+        run_cap_usd=run_cap,
+        daily_warn_usd=3.0,
+        run_warn_usd=0.5,
+        on_hit=on_hit,
+        prompt_callback=prompt_callback,  # type: ignore[arg-type]
+    )
+
+
+# ---------------------------------------------------------------------------
+# init()
+# ---------------------------------------------------------------------------
+
+
+@pytest.mark.asyncio
+async def test_init_creates_day_scope_row(db: Database) -> None:
+    tracker = _make_tracker(db)
+    await tracker.init()
+    spent = await tracker.get_spent(f"day:{_today()}")
+    assert spent == 0.0
+
+
+@pytest.mark.asyncio
+async def test_init_is_idempotent(db: Database) -> None:
+    tracker = _make_tracker(db)
+    await tracker.init()
+    await tracker.init()  # second call should not error or double-insert
+    spent = await tracker.get_spent(f"day:{_today()}")
+    assert spent == 0.0
+
+
+# ---------------------------------------------------------------------------
+# assert_can_call — under cap
+# ---------------------------------------------------------------------------
+
+
+@pytest.mark.asyncio
+async def test_assert_can_call_under_cap_returns_ok(db: Database) -> None:
+    tracker = _make_tracker(db, daily_cap=5.0, run_cap=1.0)
+    result = await tracker.assert_can_call(
+        run_id=_RUN_ID,
+        persona_name="researcher",
+        estimated_cost_usd=0.5,
+    )
+    assert result.ok is True
+    assert result.blocked_scope is None
+
+
+# ---------------------------------------------------------------------------
+# assert_can_call — over run cap (on_hit=block)
+# ---------------------------------------------------------------------------
+
+
+@pytest.mark.asyncio
+async def test_assert_can_call_over_run_cap_raises(db: Database) -> None:
+    tracker = _make_tracker(db, run_cap=0.01, on_hit=BudgetOnHit.BLOCK)
+    with pytest.raises(BudgetExhaustedError) as exc_info:
+        await tracker.assert_can_call(
+            run_id=_RUN_ID,
+            persona_name=None,
+            estimated_cost_usd=1.0,
+        )
+    err = exc_info.value
+    assert err.scope.startswith("run:")
+    assert err.projected_usd > 0.01
+
+
+# ---------------------------------------------------------------------------
+# assert_can_call — over day cap (on_hit=block)
+# ---------------------------------------------------------------------------
+
+
+@pytest.mark.asyncio
+async def test_assert_can_call_over_day_cap_raises(db: Database) -> None:
+    tracker = _make_tracker(db, daily_cap=0.001, run_cap=999.0, on_hit=BudgetOnHit.BLOCK)
+    with pytest.raises(BudgetExhaustedError) as exc_info:
+        await tracker.assert_can_call(
+            run_id=_RUN_ID,
+            persona_name=None,
+            estimated_cost_usd=1.0,
+        )
+    err = exc_info.value
+    assert err.scope.startswith("day:")
+    assert err.cap_usd == pytest.approx(0.001)
+
+
+# ---------------------------------------------------------------------------
+# record() — accumulates spend
+# ---------------------------------------------------------------------------
+
+
+@pytest.mark.asyncio
+async def test_record_accumulates_spend(db: Database) -> None:
+    tracker = _make_tracker(db)
+    run_id = uuid4()
+    await tracker.record(run_id=run_id, persona_name=None, actual_cost_usd=0.10)
+    await tracker.record(run_id=run_id, persona_name=None, actual_cost_usd=0.05)
+
+    day_spent = await tracker.get_spent(f"day:{_today()}")
+    run_spent = await tracker.get_spent(f"run:{run_id}")
+    assert day_spent == pytest.approx(0.15)
+    assert run_spent == pytest.approx(0.15)
+
+
+@pytest.mark.asyncio
+async def test_record_zero_is_noop(db: Database) -> None:
+    tracker = _make_tracker(db)
+    run_id = uuid4()
+    await tracker.record(run_id=run_id, persona_name=None, actual_cost_usd=0.0)
+    run_spent = await tracker.get_spent(f"run:{run_id}")
+    assert run_spent == 0.0
+
+
+# ---------------------------------------------------------------------------
+# on_hit=warn_continue
+# ---------------------------------------------------------------------------
+
+
+@pytest.mark.asyncio
+async def test_warn_continue_over_cap_returns_ok_no_raise(db: Database) -> None:
+    tracker = _make_tracker(db, run_cap=0.001, on_hit=BudgetOnHit.WARN_CONTINUE)
+    result = await tracker.assert_can_call(
+        run_id=_RUN_ID,
+        persona_name=None,
+        estimated_cost_usd=1.0,
+    )
+    # WARN_CONTINUE: blocked=False, no raise
+    assert result.ok is True
+
+
+# ---------------------------------------------------------------------------
+# on_hit=prompt
+# ---------------------------------------------------------------------------
+
+
+@pytest.mark.asyncio
+async def test_prompt_callback_returns_true_proceeds(db: Database) -> None:
+    async def _allow(scope: str, projected: float, cap: float) -> bool:
+        return True
+
+    tracker = _make_tracker(db, run_cap=0.001, on_hit=BudgetOnHit.PROMPT, prompt_callback=_allow)
+    result = await tracker.assert_can_call(
+        run_id=_RUN_ID,
+        persona_name=None,
+        estimated_cost_usd=1.0,
+    )
+    assert result.ok is True
+
+
+@pytest.mark.asyncio
+async def test_prompt_callback_returns_false_raises(db: Database) -> None:
+    async def _deny(scope: str, projected: float, cap: float) -> bool:
+        return False
+
+    tracker = _make_tracker(db, run_cap=0.001, on_hit=BudgetOnHit.PROMPT, prompt_callback=_deny)
+    with pytest.raises(BudgetExhaustedError):
+        await tracker.assert_can_call(
+            run_id=_RUN_ID,
+            persona_name=None,
+            estimated_cost_usd=1.0,
+        )
+
+
+@pytest.mark.asyncio
+async def test_prompt_callback_none_raises_like_block(db: Database) -> None:
+    tracker = _make_tracker(db, run_cap=0.001, on_hit=BudgetOnHit.PROMPT, prompt_callback=None)
+    with pytest.raises(BudgetExhaustedError):
+        await tracker.assert_can_call(
+            run_id=_RUN_ID,
+            persona_name=None,
+            estimated_cost_usd=1.0,
+        )
+
+
+# ---------------------------------------------------------------------------
+# persona scope
+# ---------------------------------------------------------------------------
+
+
+@pytest.mark.asyncio
+async def test_persona_scope_accumulates_separately(db: Database) -> None:
+    tracker = _make_tracker(db)
+    await tracker.record(run_id=None, persona_name="researcher", actual_cost_usd=0.20)
+
+    persona_spent = await tracker.get_spent(f"persona:researcher:day:{_today()}")
+    day_spent = await tracker.get_spent(f"day:{_today()}")
+    assert persona_spent == pytest.approx(0.20)
+    assert day_spent == pytest.approx(0.20)
+
+
+# ---------------------------------------------------------------------------
+# get_remaining()
+# ---------------------------------------------------------------------------
+
+
+@pytest.mark.asyncio
+async def test_get_remaining_with_no_spend(db: Database) -> None:
+    tracker = _make_tracker(db, daily_cap=5.0)
+    remaining = await tracker.get_remaining(f"day:{_today()}")
+    assert remaining == pytest.approx(5.0)
+
+
+@pytest.mark.asyncio
+async def test_get_remaining_after_spend(db: Database) -> None:
+    tracker = _make_tracker(db, daily_cap=5.0)
+    await tracker.record(run_id=None, persona_name=None, actual_cost_usd=1.5)
+    remaining = await tracker.get_remaining(f"day:{_today()}")
+    assert remaining == pytest.approx(3.5)
+
+
+@pytest.mark.asyncio
+async def test_get_remaining_unknown_scope_returns_none(db: Database) -> None:
+    tracker = _make_tracker(db)
+    # "unknown:xyz" has no cap in _cap_for_scope
+    remaining = await tracker.get_remaining("unknown:xyz")
+    assert remaining is None
+
+
+# ---------------------------------------------------------------------------
+# helpers
+# ---------------------------------------------------------------------------
+
+
+def _today() -> str:
+    from datetime import UTC, datetime
+
+    return datetime.now(UTC).strftime("%Y-%m-%d")