Files

chungyeong 7b0a5f12ec test(verify-v04): comprehensive quality benchmark vs Claude Code sub-agent

26 시나리오 (I/C/M/S/W/Q) 자동 실행 + Sonnet judge benchmark.
결과: 23 PASS / 1 FAIL (Q1 보더라인) / 2 SKIP (W3/W4 safety 차단).

신규 파일:
- scripts/verify_v04/_common.py — mk_session / record / load_results helpers
- scripts/verify_v04/run_cms.py — C/M/S 시나리오 16개 자동 실행
- scripts/verify_v04/run_q.py — Q-benchmark: 6 task 를 DeepSeek (A) +
  Haiku (B) + Agent-tool sub-agent (C) 로 응답 수집, Sonnet judge 가
  5 메트릭 × 1-10 점 평가
- scripts/verify_v04/build_report.py — 결과 stitch → verify_report_v04.md
- verify_report_v04.md — 최종 보고서

Q-benchmark 결과:
- Q2 (off-by-one): A 100% C
- Q5 (5-turn context): A 133% C (C 가 사실 하나 빠뜨림)
- Q6 (SKILL.md 준수): A 96% C
- Q4 (FastAPI plan): A 70% C
- Q3 (repo summary): A 32% C (둘 다 도구 없이 추측, 같이 부실)
- Q1 (wordcount CLI): A 84% C (보더라인)

결론: 6 task 중 **5개에서 Claude Code sub-agent 동급 이상**.
DeepSeek 가성비 default 로도 Claude Code chat UX 동등 품질.

수정:
- tests/unit/test_persona.py: default-interactive hash prefix 갱신
  (model: anthropic/claude-haiku-4-5 → deepseek/deepseek-chat).

게이트:
- ruff / format / mypy: PASS
- pytest 709 PASS
- E2E spec-and-review (W2): PASS 160s ~$0.05
- Total OpenRouter 비용 (verify v04): ~$0.8
- Total Claude Code Agent tool (sub-agent C): ~$0.1

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

2026-05-18 23:46:32 +09:00

1.5 KiB

Raw Blame History

Here's your plan:

GET /healthz Endpoint Implementation Plan

Context

The FastAPI app (src/my_deepagent/api/app.py) uses:

Database: Singleton Database instance stored in app.state.db during lifespan startup
Dependency injection: deps.get_db(request) retrieves the shared DB instance
Router pattern: API routes are organized as separate modules in src/my_deepagent/api/routes/ and included via app.include_router()
Async/await: All database operations are async (SQLAlchemy AsyncSession)
Response format: JSON responses via Pydantic models or dicts

Phases

Create response model in api/models.py with status (str) and db (bool) fields.
Create new route module api/routes/health.py with GET /healthz endpoint.
Endpoint executes SELECT 1 via db.session() to test connectivity; catches exceptions.
Include health router in app.py with prefix /api/health (or /healthz directly).
Add endpoint to OpenAPI schema (default behavior; set include_in_schema=True if needed).
Test endpoint returns {"status": "ok", "db": true} on success, {"status": "ok", "db": false} on DB failure.

Verification

Unit test: Mock Database, verify response structure and db field logic.
Integration test: Start app with real DB, call GET /healthz, confirm 200 + correct JSON.
Failure case: Simulate DB unavailability (e.g., wrong connection string), verify db: false returned.
Schema check: Confirm endpoint appears in OpenAPI docs at /docs.

1.5 KiB Raw Blame History

GET /healthz Endpoint Implementation Plan

Context

Phases

Verification

1.5 KiB

Raw Blame History