dev-puppeteer/docs/plan.md

Devflow Implementation Plan v4 r1

0. Document Status

• v4 r1: language migration TS → Python. Major version bump; the TypeScript monorepo (apps/, packages/, tests/, scripts/, pnpm/tsconfig metadata) was deleted in 0e61b2d after being re-implemented under my-deepagent/. v3 CC counters are preserved as historical context; v4 begins its own CC series (DR-1 below; CC-Py-1 onward as new change clarifications land).
• This document supersedes v2 and all earlier v3 drafts where conflicting.
• Single-user, single-machine assumption. No auth, no retention policy, no observability dashboards, no multi-tenancy.
• Target OS: macOS 13+ / Linux. No Windows.
• All paths are Unix-style. All times are stored UTC.
• Decisions in this document are locked unless explicitly marked (provisional). Override requires updating this document, not only code.
• §1 Stack Decisions, §2 Directory Layout, §3 doctor checklist, §22 Decision Log have been rewritten for v4 r1. §4~§17 (DB schema, enums, hashing, template/ persona/binding, session runtime, prompt envelope, artifact registry, run events, fake adapter, state machines, errors, write set, SSE contract) are language-neutral domain spec and remain valid for the Python implementation.
• v3 CC history (informational):
• r1 applied CC-1 through CC-5.
• r2 applied CC-6 through CC-10.
• r3 applied CC-11 through CC-15.
• r4 applies CC-16 through CC-18.
• r5 applies CC-19.
• r6 applies CC-20.
• r7 applies CC-21 through CC-23.
• r8 applies CC-24 through CC-26.
• r9 applies CC-27 through CC-28.
• r10 applies CC-29 through CC-31.
• r11 applies CC-32.
• r12 applies CC-33 through CC-35.
• r13 applied CC-39 (final v3 revision; superseded by v4 r1).

1. Stack Decisions

1.1 Workspace

• Python 3.12+, managed by uv workspaces (uv sync, uv add, uv run).
• Pinned via .python-version. No Node, no pnpm, no tsc.
• pyproject.toml at repo root + per-package pyproject.toml under packages/<name>/ (uv workspace members).
• Imports are absolute. No from . import *.

1.2 Tooling

Concern	Choice	Notes
Lint / format	ruff	One root ruff.toml. ruff check . + ruff format --check ..
Type check	mypy --strict	mypy.ini enables strict mode; tests relax disallow_untyped_defs.
Test	pytest + pytest-asyncio + pytest-httpx + respx	pytest -q.
Pre-commit	pre-commit (.pre-commit-config.yaml)	Runs ruff + mypy + pytest --collect-only.
Schema validation	pydantic v2 + pydantic-settings	Replaces zod.
YAML	PyYAML	Persona/template YAML loaders.
JSON Schema	jsonschema (2020-12)	Artifact registry.
HTTP client	httpx (async)	OpenRouter / pricing fetch.
Logging	structlog + rich	Replaces pino. _scrub_processor redacts secrets before stderr / JSON sinks.
CLI	typer + prompt_toolkit	Replaces commander; prompt_toolkit drives the interactive REPL.
OS dirs	platformdirs	XDG data / state / config dirs.
Secrets	keyring	macOS Keychain / Linux Secret Service / Windows Credential Store.

1.3 Database

• SQLite 3 (WAL mode) via aiosqlite, ORM: SQLAlchemy 2.0 async.
• Migrations: Alembic (baseline + per-feature revisions).
• WAL + busy_timeout=5000 + PRAGMA foreign_keys=ON enforced at connect.
• Postgres (the v3 default) is parked: single-machine + single-user removes the multi-process concurrency justification, and aiosqlite + the ux_active_run_repo_base partial unique index covers the active-run uniqueness invariant. Postgres can be reinstated for multi-tenant later.

1.4 Logging

• structlog, JSON sink to stderr by default, rich pretty sink when stdout is a TTY.
• Standard fields: time, level, module, run_id?, phase_id?, role?, event_id?, interactive_session_id?.
• _scrub_processor redacts OpenRouter / Anthropic / OpenAI / LangSmith / GitHub / GitLab API keys and generic Bearer … tokens before emission.
• Levels: same semantics as v3 (trace/debug/info/warn/error).

1.5 Config

• Single pydantic-settings BaseSettings in my_deepagent.config.Config with MYDEEPAGENT_ env prefix and optional TOML source.
• Source precedence (high → low): explicit overrides → os.environ (with MYDEEPAGENT_ prefix) → .env → config.toml → schema defaults.
• Config is loaded once at process start, validated, frozen, and re-exported as an immutable typed Config.
• Validation failure is fatal (exit code 2).
• Required keys at v0.1.0:
  • MYDEEPAGENT_DATABASE_URL (default sqlite+aiosqlite:///<state_dir>/db.sqlite3)
  • MYDEEPAGENT_WORKSPACE_ROOT
  • MYDEEPAGENT_LOG_LEVEL
  • MYDEEPAGENT_OPENROUTER_API_KEY when the OpenRouter backend is enabled (resolution order: config → env → OS keyring → error).
• Path canonicalization: workspace_root is resolved via Path.resolve() at config load. Any path entering the system is canonicalized before storage or hashing.

Backend registration (deepagents-flavored):

class BackendConfig(BaseModel, frozen=True):
    id: Backend                 # openrouter | anthropic | openai | google | fake
    enabled: bool
    api_base_url: str | None = None  # openrouter default https://openrouter.ai/api/v1
    api_key_env: str | None = None   # default MYDEEPAGENT_OPENROUTER_API_KEY

• fake is always available.
• openrouter is available only when enabled and the resolved key is present.
• Doctor warns on misconfig; binding fails fast at run start with human_required:backend_unavailable.

1.6 HTTP / SSE

• FastAPI + uvicorn + sse-starlette for the M8-Py REST + SSE surface (v3 r13 §17 contract unchanged: same event types, same headers, same data: <json>\n\n framing).
• Body validation via the same pydantic v2 models used elsewhere.
• WebSocket remains deferred unless SSE fails under transcript volume.

2. Directory Layout

v4 r1 collapses the v3 multi-package monorepo into a single my-deepagent/ project. The TS apps/, packages/, tests/, scripts/ trees were deleted in 0e61b2d; v3 §4~§17 module-by-module spec still applies but each module now lives under my_deepagent/<name>.py instead of packages/<name>/src/<name>.ts.

<repo-root>/
├── docs/
│   ├── plan.md                          # this document
│   ├── plan-v4-draft.md                 # v4 r1 design memo (informational)
│   └── schemas/
│       ├── artifacts/                   # JSON Schema 2020-12 (language-neutral)
│       ├── personas/                    # YAML persona seed (language-neutral)
│       └── templates/                   # YAML workflow templates
├── docker-compose.yml                   # Postgres + Temporal (still relevant for M5-Py)
├── .env.example
├── .gitignore
├── my-deepagent-seed/                   # v0.1.0 bootstrap kit (historical, may be pruned)
└── my-deepagent/
    ├── pyproject.toml                   # uv workspace root
    ├── uv.lock
    ├── ruff.toml
    ├── mypy.ini
    ├── alembic.ini
    ├── .pre-commit-config.yaml
    ├── CHANGELOG.md
    ├── alembic/
    │   ├── env.py
    │   └── versions/                    # baseline + per-feature migrations
    ├── docs/schemas/                    # mirror of repo-root docs/schemas for loader convenience
    ├── src/my_deepagent/
    │   ├── config.py                    # pydantic-settings Config (replaces §1.5 zod schema)
    │   ├── enums.py                     # closed-set enums (§5)
    │   ├── errors.py                    # error taxonomy (§18)
    │   ├── hash.py                      # content-addressed hashing (§6)
    │   ├── persona.py                   # Persona + loader (§7.2)
    │   ├── workflow.py                  # WorkflowTemplate + loader (§7.1)
    │   ├── binding.py                   # autoSelect / override / consent store (§7.4)
    │   ├── artifact_schema.py           # JSON Schema 2020-12 registry (§10)
    │   ├── run_event.py                 # event types + idempotency keys (§11, §13.1)
    │   ├── prompt_envelope.py           # envelope builder (§9)
    │   ├── budget.py                    # BudgetTracker (v4-new)
    │   ├── secrets.py                   # config → env → keyring resolution chain
    │   ├── keys.py                      # OS keyring wrapper
    │   ├── audit.py                     # append-only JSONL audit log (v4-new)
    │   ├── logging.py                   # structlog + secret scrubber (§1.4)
    │   ├── governance.py                # first-run consent (v4-new)
    │   ├── i18n/                        # ko / en catalog
    │   ├── recovery.py                  # sweep_orphan_runs (§19)
    │   ├── session.py                   # deepagents adapter (§8.5, v4-new)
    │   ├── engine.py                    # WorkflowEngine — phase loop (§15)
    │   ├── persistence/
    │   │   ├── db.py                    # SQLAlchemy 2 async engine
    │   │   ├── models.py                # ORM models (§4)
    │   │   └── checkpointer.py          # LangGraph SqliteSaver context
    │   ├── middleware/
    │   │   ├── cost.py                  # CostMiddleware (v4-new)
    │   │   ├── budget.py                # BudgetMiddleware (v4-new)
    │   │   ├── audit.py                 # AuditToolMiddleware
    │   │   ├── safety.py                # SafetyShellMiddleware (deny-path / destructive command)
    │   │   └── artifact_watcher.py      # ArtifactWatcherMiddleware
    │   ├── monitoring/
    │   │   ├── pricing.py               # OpenRouter pricing cache
    │   │   └── cost_estimator.py        # pre-run preview
    │   ├── cli/                         # typer-driven CLI
    │   │   ├── main.py                  # entry (interactive REPL when no subcommand)
    │   │   ├── doctor.py                # §3 doctor checks (Python/uv version)
    │   │   ├── init.py
    │   │   ├── keys_cmd.py
    │   │   ├── run.py
    │   │   ├── runs.py
    │   │   ├── stats.py
    │   │   └── interactive.py           # prompt_toolkit REPL
    │   ├── tui/
    │   │   └── approval.py              # tri-state approval prompt
    │   └── slash.py                     # REPL slash commands
    └── tests/
        ├── unit/                        # pure-Python unit tests
        └── integration/                 # async + persistence + real OpenRouter (gated)

Future trees deferred:

• apps/api/, apps/worker/ (M5-Py / M8-Py): FastAPI app and temporalio worker. v4 r1 keeps them out until M5 lands.
• apps/web/: Web GUI port is out of scope for v0.1.0 (separate milestone).

3. mydeepagent doctor

Exit codes:

• 0: all green.
• 1: one or more red checks.
• 2: internal or unknown error.

Each check emits:

• name
• status: pass | fail | warn
• detail
• remediation

Closed check list (v4 r1, 8 checks — Node/pnpm/Docker/Drizzle dropped):

1. python: python --version satisfies >=3.12,<3.14.
2. uv: uv --version resolves (any).
3. git: git --version >=2.40.
4. workspace_root: MYDEEPAGENT_WORKSPACE_ROOT exists, is a directory, and is writable.
5. config+governance: Config loads from env + .env + config.toml without ValidationError; first-run governance consent file exists (or is created interactively on first run only).
6. openrouter_api_key: resolution chain (config → env → OS keyring) yields a non-empty value. Warn-only when the OpenRouter backend is not enabled.
7. openrouter_ping + pricing upsert: GET https://openrouter.ai/api/v1/models with the bearer key.
   • 200 → pass; pricing rows are upserted into model_pricing for use by the mydeepagent run cost preview.
   • 401 → fail.
   • any other non-200 / network error → warn.
8. disk+sqlite integrity:
   • Free disk on the workspace_root partition: warn under 10 GB, fail under 2 GB, green target ≥ 5 GB.
   • SQLite DB file (if present) opens and PRAGMA integrity_check returns ok.

Output:

• Rich human table by default.
• --json for machine-readable output.
• --quiet prints only nonzero results.

Notes:

• tmux / Docker / Postgres / pg_isready / drizzle migration checks from v3 §3 are dropped in v4 r1 — the v0.1.0 runtime is SQLite-only and tmux is out of scope for the deepagents-driven session model.
• --list-orphans and friends are owned by mydeepagent runs list/show (§19).

4. Database Schema

First migration prelude:

CREATE EXTENSION IF NOT EXISTS pgcrypto;

All tables use gen_random_uuid() primary keys unless noted. All times are timestamptz. Mutable rows include updated_at. JSON columns use jsonb.

4.1 workflow_templates

• id uuid primary key default gen_random_uuid()
• name text not null
• version int not null
• hash text not null unique
• definition jsonb not null
• created_at timestamptz not null default now()
• unique (name, version)

4.2 agent_personas

• id uuid primary key default gen_random_uuid()
• name text not null
• version int not null
• hash text not null unique
• definition jsonb not null
• created_at timestamptz not null default now()
• unique (name, version)

4.3 runs

• id uuid primary key default gen_random_uuid()
• template_id uuid not null references workflow_templates(id)
• template_hash text not null
• state text not null
• repo_path text not null
  • canonical absolute path
  • resolved through fs.realpathSync before insert
• base_branch text not null
• worktree_root text not null
  • canonical absolute path under WORKSPACE_ROOT/<runId>/
• current_phase_id uuid references run_phases(id) nullable and deferrable
• started_at timestamptz
• ended_at timestamptz
• final_report_path text
• paused_from_state text
  • set when transitioning to paused
  • cleared on resume
  • null when state is not paused
• created_at timestamptz not null default now()
• updated_at timestamptz

Active-run uniqueness:

CREATE UNIQUE INDEX ux_active_run_repo_base
ON runs (repo_path, base_branch)
WHERE state NOT IN ('completed', 'failed', 'aborted');

4.4 run_inputs

• id uuid primary key default gen_random_uuid()
• run_id uuid not null unique references runs(id) on delete cascade
• requirements_md text not null
• objective jsonb
• extra jsonb
• input_hash text not null

input_hash is based on:

• requirements_md
• objective
• extra
• canonical repo_path
• base_branch

4.5 run_bindings

• id uuid primary key default gen_random_uuid()
• run_id uuid not null references runs(id) on delete cascade
• role_id text not null
• persona_id uuid not null references agent_personas(id)
• persona_hash text not null
• backend text not null
• binding_hash text not null
• unique (run_id, role_id)

4.6 run_phases

• id uuid primary key default gen_random_uuid()
• run_id uuid not null references runs(id) on delete cascade
• phase_key text not null
• seq int not null
• state text not null
• attempts int not null default 0
• started_at timestamptz
• ended_at timestamptz
• unique (run_id, phase_key)

4.7 run_events

Append-only.

• id bigserial primary key
• run_id uuid not null references runs(id) on delete cascade
• phase_id uuid references run_phases(id)
• seq bigint not null
• type text not null
• payload jsonb not null
• idempotency_key text not null
• ts timestamptz not null default now()
• unique (run_id, seq)
• unique (run_id, idempotency_key)
• index (run_id, ts)

Concurrency:

• All inserts go through RunEventRepository.append().
• Raw SQL inserts into run_events are forbidden.
• append() takes pg_advisory_xact_lock(hash64('devflow:run-events', run_id)).
• Inside that same transaction it assigns:

seq := COALESCE(MAX(seq), 0) + 1

4.8 approval_requests

• id uuid primary key default gen_random_uuid()
• run_id uuid not null references runs(id)
• phase_id uuid references run_phases(id)
• gate_key text not null
• state text not null
• idempotency_key text not null
• payload jsonb not null
• created_at timestamptz not null default now()
• resolved_at timestamptz
• unique (idempotency_key)

4.9 approval_decisions

Append-only and immutable.

• id uuid primary key default gen_random_uuid()
• approval_request_id uuid not null references approval_requests(id)
• action text not null
  • approve
  • reject
  • request_changes
  • abort
• comment text
• decided_at timestamptz not null default now()
• idempotency_key text not null unique

pause is not an approval decision.

4.10 tui_sessions

• id uuid primary key default gen_random_uuid()
• run_id uuid not null references runs(id) on delete cascade
• role_id text not null
• backend text not null
• cwd text not null
• expected_artifact_path text
• expected_schema text
• last_prompt_hash text
• last_prompt_at timestamptz
• last_capture_seq bigint not null default 0
• last_known_pane_pid int
• tmux_session text
• tmux_window text
• state text not null
• recovery_attempts int not null default 0
• unique (run_id, role_id)

4.11 tui_transcript_chunks

Append-only.

• id bigserial primary key
• session_id uuid not null references tui_sessions(id) on delete cascade
• seq bigint not null
• content text not null
• captured_at timestamptz not null default now()
• unique (session_id, seq)

4.12 artifacts

• id uuid primary key default gen_random_uuid()
• run_id uuid not null references runs(id) on delete cascade
• phase_id uuid references run_phases(id)
• path text not null
• schema_id text not null
• hash text not null
• valid boolean not null
• validation_error jsonb
• created_at timestamptz not null default now()
• unique (run_id, path, hash)

4.13 commands

• id uuid primary key default gen_random_uuid()
• run_id uuid not null references runs(id) on delete cascade
• phase_id uuid references run_phases(id)
• kind text not null
  • git
  • test
  • e2e
  • doctor
  • backtest
  • other
• argv text[] not null
• cwd text not null
• exit_code int
• stdout_path text
• stderr_path text
• started_at timestamptz
• ended_at timestamptz

4.14 review_findings

• id uuid primary key default gen_random_uuid()
• run_id uuid not null references runs(id) on delete cascade
• phase_id uuid references run_phases(id)
• reviewer_role text not null
• severity text not null
  • info
  • low
  • medium
  • high
  • critical
• category text not null
  • correctness
  • evidence
  • style
  • security
  • performance
  • other
• file_path text
• line int
• summary text not null
• evidence text
• verifier_status text not null default 'unverified'
  • unverified
  • confirmed
  • rejected
• created_at timestamptz not null default now()

4.15 Backtest Stub Tables

backtest_iterations and backtest_metrics are created at M1 as stub tables:

• id uuid primary key default gen_random_uuid()
• run_id uuid not null references runs(id) on delete cascade
• payload jsonb
• created_at timestamptz not null default now()

Full schema is deferred to M12.

5. Enums

All enums live in packages/core/src/enums.ts as TypeScript const objects and Zod enums.

5.1 Backend

• codex
• claude
• fake
• openrouter

openrouter is HTTP-based and has no tmux/PTY; see §8.5.

Future gemini support adds an enum entry and a BackendProfile; no design change.

5.2 Capability

• spec_write
• phase_planning
• task_dag_planning
• code_edit
• test_first_development
• code_review
• evidence_check
• command_execute
• backtest_run
• metric_extract
• failure_mining
• objective_eval
• final_report_compose

5.3 RiskLevel

• low
• medium
• high

Risk is declared per phase in the template. Persona has maxRiskLevel. Binding fails when phase.risk > persona.maxRiskLevel.

5.4 ApprovalDecisionAction

• approve
• reject
• request_changes
• abort

pause is a run-level control operation, not an approval decision.

5.5 ApprovalState

• pending
• approved
• rejected
• changes_requested
• aborted
• paused

paused is not an auto-decision.

5.6 RunState

• created
• bound
• planning
• awaiting_approval
• executing
• paused
• completed
• failed
• aborted

5.7 RunPhaseState

• pending
• running
• awaiting_artifact
• validating
• awaiting_approval
• completed
• failed
• skipped

5.8 SessionState

• CREATED
• BOOTSTRAPPING
• READY
• BUSY
• WAITING_FOR_APPROVAL
• ARTIFACT_TIMEOUT
• HUNG
• CRASHED
• RESUMING
• REBOOTSTRAPPED
• FAILED_NEEDS_HUMAN

6. Content-Addressed Hashing

6.1 Canonical JSON

• Object keys sorted lexicographically by UTF-16 code units.
• No insignificant whitespace.
• Strings use standard JSON escaping.
• No Unicode normalization.
• Numbers use shortest round-trippable representation.
• Integers have no decimal point.
• No leading zeros.
• Arrays preserve order.
• No trailing newline.

packages/core/src/hash.ts exports:

canonicalize(value: unknown): string
hash(value: unknown): string

hash() returns sha256hex(canonicalize(value)).

6.2 Hash Subjects

• Template hash:
  • { name, version, roles, phases, gates, capabilitiesRequired }
• Persona hash:
  • { name, version, capabilities, backend, maxRiskLevel, allowedRoles, promptConfig, modelConfig }
• Binding hash:
  • { runId, roleId, templateHash, personaHash, backend, override }
• Run input hash:
  • { templateHash, bindings: sorted[bindingHash], requirementsMd, objective, repoPath, baseBranch, extra }
• Prompt hash:
  • { runId, roleId, phaseKey, expectedArtifact, expectedSchema, instructions, attempt }
• Artifact hash:
  • SHA-256 of file bytes.

Prompt hash uses phaseKey, not phaseId, because PromptEnvelope carries phaseKey.

7. Template, Persona, Binding

7.1 Template Schema

const TemplatePhase = z.object({
  key: z.string(),
  title: z.string(),
  risk: RiskLevel,
  roles: z.array(z.string()),
  expectedArtifact: z
    .object({
      path: z.string(),
      schema: z.string(),
    })
    .optional(),
  gates: z.array(z.string()).default([]),
  timeoutMs: z.number().int().positive().optional(),
});

const TemplateRole = z.object({
  id: z.string(),
  requiredCapabilities: z.array(Capability),
  preferredBackends: z.array(Backend).default([]),
  count: z.number().int().min(1).default(1),
  diversity: z
    .object({
      requireDifferentBackends: z.boolean().default(false),
    })
    .optional(),
});

const Template = z.object({
  name: z.string(),
  version: z.number().int().positive(),
  roles: z.array(TemplateRole),
  phases: z.array(TemplatePhase),
  defaultGates: z.array(z.string()).default([]),
});

7.2 Persona Schema

const Persona = z.object({
  name: z.string(),
  version: z.number().int().positive(),
  backend: Backend,
  capabilities: z.array(Capability),
  maxRiskLevel: RiskLevel,
  allowedRoles: z.array(z.string()).optional(),
  promptConfig: z
    .object({
      systemPrompt: z.string().optional(),
      instructionsPrelude: z.string().optional(),
    })
    .default({}),
  modelConfig: z.record(z.string(), z.unknown()).default({}),
});

modelConfig conventions:

• Personas bound to openrouter MUST set modelConfig.model to a routable OpenRouter model id, e.g. anthropic/claude-sonnet-4-5, deepseek/deepseek-chat, meta-llama/llama-3.1-70b-instruct.
• Other supported keys: maxTokens, temperature, topP. All optional.
• For tmux-based backends (codex, claude, fake), modelConfig.model is informational only and MAY be omitted.
• Binding fails fast with human_required:model_unavailable when an openrouter persona has no modelConfig.model.

7.3 Override Semantics

• Override may swap persona for a role.
• Override may constrain backend to a specific allowed backend.
• Override cannot add capabilities.
• Override cannot raise risk above persona maxRiskLevel.
• Diversity rules apply after override.
• Lock-time validation runs the full binding algorithm.
• On first binding failure, the run does not start.

7.4 Binding Algorithm

For each role:

1. Select override persona if present; otherwise run autoSelect.
2. Assert backend is enabled in config.backends.
3. Assert non-fake backend binary resolved at process start.
4. Assert role id is in allowedRoles, unless allowedRoles is absent.
5. Assert required capabilities are a subset of persona capabilities.
6. Assert every phase using the role has risk <= persona maxRiskLevel.
7. Expand roles with count > 1 into roleId#0, roleId#1, etc.
8. Enforce diversity rules after expansion.
9. Compute and persist binding_hash per role instance.

autoSelect is deterministic. Sort candidates by:

1. role preferredBackends order.
2. persona.version desc.
3. persona.name asc.
4. persona.hash asc.

Personas whose backend is not in preferredBackends are eligible only if all preferred-backend personas fail capability or risk checks.

Binding fails with human_required:no_eligible_persona if no persona satisfies requirements.

7.5 Seeding

Personas:

• docs/schemas/personas/<name>@<version>.yaml
• filename encodes immutable identity.
• loader parses with Persona schema.
• loader computes personaHash.
• loader upserts keyed by (name, version).
• hash mismatch on an existing row is fatal.

Templates:

• docs/schemas/templates/<name>@<version>.yaml
• same immutable version rule.

Deleting a published file is allowed only when no run references that hash.

8. Session Runtime

8.1 SessionAdapter Interface

export interface SessionAdapter {
  start(input: StartInput): Promise<SessionHandle>;
  sendPrompt(handle: SessionHandle, envelope: PromptEnvelope): Promise<{ promptId: string }>;
  probe(handle: SessionHandle): Promise<ProbeResult>;
  resume(handle: SessionHandle): Promise<SessionHandle>;
  rebootstrap(handle: SessionHandle): Promise<SessionHandle>;
  capture(handle: SessionHandle, fromSeq: bigint): AsyncIterable<TranscriptChunk>;
  dispose(handle: SessionHandle): Promise<void>;
}

export interface StartInput {
  runId: string;
  roleId: string;
  backend: Backend;
  cwd: string;
  expectedArtifactPath?: string;
  expectedSchema?: string;
  envelopePrelude?: string;
}

export interface SessionHandle {
  sessionId: string;
  pid?: number;
  tmuxSession?: string;
  tmuxWindow?: string;
}

export interface ProbeResult {
  alive: boolean;
  paneActive: boolean;
  lastOutputAt?: Date;
  hint?: string;
}

export interface TranscriptChunk {
  seq: bigint;
  content: string;
  capturedAt: Date;
}

For HTTP backends (openrouter) the SessionHandle.pid, tmuxSession, and tmuxWindow fields are always undefined. See §8.5 for the HTTP adapter mapping.

8.2 Session State Machine

• CREATED -> BOOTSTRAPPING -> READY
• READY <-> BUSY
• BUSY -> WAITING_FOR_APPROVAL
• BUSY -> ARTIFACT_TIMEOUT
• BUSY -> HUNG
• BUSY -> CRASHED
• HUNG | CRASHED | ARTIFACT_TIMEOUT -> RESUMING -> READY
• RESUMING -> REBOOTSTRAPPED -> READY
• exhausted errors -> FAILED_NEEDS_HUMAN

8.3 Recovery Counters

• sendPrompt retry: 2.
  • Means one initial send plus two adapter-level retries, three physical send attempts max.
• resume retry: 2.
• rebootstrap retry: 1.
• artifact repair retry: 1.
• max hung time: configurable; default 20 minutes.

Exhaustion creates a human gate with recoveryHint.

8.4 SessionManager Singleton

• M4: hosted in apps/api.
• M5+: hosted in apps/worker.
• Only SessionManager may call mutating SessionAdapter methods.
• Holds in-memory Map<sessionId, SessionHandle>.
• Takes pg_advisory_lock(hash64('devflow:session-manager')).
• Second instance exits code 3.
• On start:
  • query non-terminal tui_sessions.
  • call adapter.resume(handle).
  • success: place handle in map.
  • failure: session -> FAILED_NEEDS_HUMAN, append session.failed, create recovery gate.
• On SIGTERM/SIGINT:
  • refuse new prompts.
  • allow in-flight artifact polling up to 30s.
  • persist last_capture_seq.
  • release advisory lock.

8.5 OpenRouter Adapter — v4 r1 deepagents rewrite

Supersedes the v3 marker-extraction HTTP adapter (CC-39). In v4 the OpenRouter integration is a multi-turn, tool-using agent driven by LangChain deepagents 0.6.1 — no single-shot completions, no <<<DEVFLOW_ARTIFACT_*>>> markers, no transcript replay reconstruction.

Construction — my_deepagent.session.build_agent(persona, run_id, …):

llm = ChatOpenAI(
    model=persona.model,                  # e.g. "openrouter:deepseek/deepseek-chat"
    base_url=config.openrouter_api_base,  # https://openrouter.ai/api/v1
    api_key=resolve_openrouter_api_key(),
    timeout=persona.model_params.timeout,
)
agent = deepagents.create_deep_agent(
    model=llm,
    tools=[],                             # base tools come from LocalShellBackend
    instructions=persona.system_prompt,
    subagents=[_subagent_to_dict(s) for s in persona.subagents],
    middleware=[
        SafetyShellMiddleware(...),       # destructive command + deny-path guard
        AuditToolMiddleware(...),         # append-only JSONL audit log
        ArtifactWatcherMiddleware(...),   # write_file/edit_file detection
        CostMiddleware(...),              # usage_metadata + budget ledger
    ],
    backend=LocalShellBackend(            # bash + read_file + write_file + edit_file + ls
        cwd=worktree_root,
        # `permissions` kwarg is intentionally omitted for local_shell backend
        # (deepagents 0.6.1 NotImplementedError workaround — enforcement moves
        # to SafetyShellMiddleware).
    ),
)

Method mapping (driven by WorkflowEngine rather than a v3-style adapter interface):

• Start: create_deep_agent returns a CompiledStateGraph per phase. No persistent session object is shared across phases — each phase is a fresh agent invocation parameterized by persona + envelope.
• Send prompt: await agent.ainvoke({"messages": [HumanMessage(envelope)]}) where envelope is built by WorkflowEngine._build_envelope (§9 with the artifact JSON Schema inlined so the model sees the exact required fields).
• Tool use: native read_file / write_file / edit_file / ls / bash calls are emitted by the model and dispatched through LocalShellBackend, recorded by AuditToolMiddleware, gated by SafetyShellMiddleware.
• Probe / resume / rebootstrap / dispose: not applicable — the agent is ephemeral per phase. Crash recovery operates at the run/phase level via sweep_orphan_runs (§19), not at a session-adapter level.

Artifact production:

• The model writes the artifact directly to expected_artifact_path via the write_file tool. ArtifactWatcherMiddleware observes the tool call and notifies the engine.
• The envelope inlines the artifact's JSON Schema definition so the LLM has the exact required fields.
• Schema validation is performed by ArtifactSchemaRegistry.validate on the written file (§10). On failure, the engine retries once with a repair prompt; second failure raises human_required:artifact_invalid_after_repair.

Error mapping (preserved from CC-39, applied per-call by the LangChain exception path):

• HTTP 401 → human_required:backend_auth_failed.
• HTTP 429 → recoverable:rate_limited (exponential backoff: 1 s, 2 s, 4 s, max 30 s, owned by langchain-openai retries).
• HTTP 5xx → recoverable:network_blip.
• HTTP 400 with model_not_found → human_required:model_unavailable.
• BudgetTracker pre-call rejection → human_required:token_budget_exceeded.
• SafetyShellMiddleware blocked tool call → human_required:tool_quota_exceeded.

Known v0.1.0 limitations:

• usage_metadata is sometimes empty on responses forwarded by OpenRouter (deepagents wraps the underlying ChatOpenAI response so token counts may not surface). The recorder still fires and LlmCallRow is persisted, but input_tokens / output_tokens may read 0. v0.2 will probe additional response shapes (raw chunks / callbacks).
• Anthropic models via OpenRouter currently fail with a tool_calls.args JSON-string vs dict ValidationError inside langchain-openai 1.2.1. Workaround: pin DeepSeek personas via BindingOverride. Tracking for v0.2.

9. Prompt Envelope

9.1 Wire Format

DEVFLOW_PROMPT_BEGIN <uuid>
Run: <run-id>
Role: <role-id>
Phase: <phase-key>
Attempt: <int>
Expected artifact: <absolute-path>
Expected schema: <schema-id>
Dedup-Key: <prompt-hash>
Instructions:
<freeform multi-line instructions>
DEVFLOW_PROMPT_END <uuid>

9.2 Schema

const PromptEnvelope = z.object({
  uuid: z.string().uuid(),
  runId: z.string().uuid(),
  roleId: z.string(),
  phaseKey: z.string(),
  attempt: z.number().int().nonnegative(),
  expectedArtifact: z.string(),
  expectedSchema: z.string(),
  dedupKey: z.string(),
  instructions: z.string(),
});

9.3 Rules

• Prompt identity is dedupKey.
• Adapter treats duplicate dedupKey for the same session within a run lifetime as idempotent success and does not reprocess the prompt.
• attempt increments only when the engine intentionally re-sends after timeout or repair.
• Adapter-level retry does not increment attempt.
• Completion is never inferred from transcript text.
• Completion requires a schema-valid artifact.

9.4 Backend Prelude

Sent once at session bootstrap before the first envelope.

Required structure:

1. Backend identity statement.
2. Persona instructionsPrelude.
3. Protocol declaration: completion is signaled only by writing expected artifact files.
4. Envelope marker contract.
5. Approval/probe contract: DEVFLOW_PROBE must respond with one line READY or BUSY <reason>.

Codex and Claude-specific addenda live in packages/session/src/profiles/{codex,claude}.ts and are populated at M10.

10. Artifact Schema Registry

10.1 Layout

JSON Schema 2020-12 documents live at:

docs/schemas/artifacts/<schema_id>.json

schema_id format:

<domain>/<name>@<version>

Examples:

• dev/spec@1
• dev/phase-plan@1
• dev/dag@1
• dev/review-finding-batch@1
• bt/objective@1
• bt/iteration-result@1
• common/final-report@1

10.2 Loader

packages/core/src/artifact-schema.ts exports:

function loadSchema(id: string): JsonSchema;
function validateArtifact(
  id: string,
  data: unknown
): { ok: true } | { ok: false; errors: ValidationError[] };

Unknown schema id is fatal.

10.3 Validation Flow

1. Engine waits for expectedArtifactPath to appear.
2. Debounce 500ms after last mtime change.
3. Read file.
4. Compute SHA-256.
5. Validate against expectedSchema.
6. Valid:
   • insert artifact row with valid=true.
   • append artifact.validated.
   • advance phase.
7. Invalid:
   • insert artifact row with valid=false.
   • append artifact.invalid.
   • trigger one repair prompt.
   • after repair exhaustion, create human gate.
8. Timeout:
   • append artifact.timeout.
   • probe session.
   • enter recovery flow.

10.4 Final Report

At terminal run state, write atomically:

• <WORKSPACE_ROOT>/<runId>/<runId>.report.md
• <WORKSPACE_ROOT>/<runId>/<runId>.report.json

Both are written even on failed or aborted, best-effort.

common/final-report@1 minimum fields:

• runId
• templateHash
• bindings[]
• inputs
• phases[]
• approvals[]
• findings[]
• commands[]
• artifacts[]
• events.tail
• unresolved[]
• endedAt
• status

10.5 Backtest Objective Stub

bt/objective@1:

{
  "targets": [
    { "metric": "sharpe", "op": "gte", "value": 1.5, "weight": 1.0 },
    { "metric": "mdd", "op": "lte", "value": 0.15, "weight": 1.0 }
  ],
  "stopWhen": "all"
}

• op: gte | lte | eq | gt | lt
• stopWhen: all | weighted
• weighted threshold is hardcoded at 0.8 at M12.
• Full DSL deferred to M12.

11. Run Events

Closed event types:

run.created
run.started
run.paused
run.resumed
run.completed
run.failed
run.aborted
phase.started
phase.completed
phase.failed
phase.skipped
prompt.sent
prompt.repaired
artifact.expected
artifact.validated
artifact.invalid
artifact.timeout
approval.requested
approval.resolved
session.created
session.ready
session.busy
session.idle
session.crashed
session.recovered
session.failed
command.started
command.completed
command.failed
review.batch_recorded
finding.verifier_resolved
backtest.iteration_started
backtest.iteration_completed
backtest.objective_evaluated

11.1 Idempotency Keys

Every event append requires deterministic idempotency_key.

Event family	Key formula
run.created, run.started, run.completed, run.failed, run.aborted	<type>:<run_id>
run.paused	run.paused:<run_id>:<cause>
run.resumed	run.resumed:<run_id>:<cause>
phase.started, phase.completed, phase.failed, phase.skipped	<type>:<phase_id>:<phase_attempt>
prompt.sent, prompt.repaired	<type>:<prompt_dedup_key>
artifact.expected, artifact.timeout	<type>:<phase_id>:<phase_attempt>:<expected_path>
artifact.validated, artifact.invalid	<type>:<phase_id>:<expected_path>:<artifact_hash>
approval.requested	approval.requested:<approval_idempotency_key>
approval.resolved	approval.resolved:<approval_request_id>:<action>
session.created, session.failed	<type>:<session_id>
session.busy, session.idle	<type>:<session_id>:<prompt_dedup_key>
session.ready, session.crashed, session.recovered	<type>:<session_id>:<recovery_attempts>
command.started, command.completed, command.failed	<type>:<command_id>
review.batch_recorded	review.batch_recorded:<phase_id>:<reviewer_role>:<phase_attempt>
finding.verifier_resolved	finding.verifier_resolved:<finding_id>
backtest.iteration_started, backtest.iteration_completed, backtest.objective_evaluated	<type>:<iteration_id>

Definitions:

• phase_attempt is incremented before event append.
• recovery_attempts is incremented before event append.
• prompt_dedup_key is the envelope dedup key.
• approval_idempotency_key is from approval_requests.
• Artifact expected/timeout events are per-attempt.
• Artifact validated/invalid events are content-keyed by path + hash.

12. Fake Session Adapter

12.1 Behavior

• Deterministic.
• In-process.
• No PTY.
• No tmux.
• Drives engine end-to-end without real backends.

12.2 Sentinel Triggers

On sendPrompt, inspect expectedSchema.

Fixture path:

tests/fixtures/fake-artifacts/<expectedSchema>/<scenarioName>.json

scenarioName comes from instruction header:

Scenario: <name>

Default scenario: ok.

Scenarios:

• ok: write fixture to expectedArtifactPath after 50ms by default.
• invalid: write deliberately schema-invalid payload.
• timeout: never write.
• crash: throw RecoverableError.

12.3 Transcript

Fake adapter emits chunks such as:

[fake] received prompt <uuid>; will write <path> in 50ms

13. State Machines

13.1 Run State

States:

• created
• bound
• planning
• awaiting_approval
• executing
• paused
• completed
• failed
• aborted

Transitions:

From	Trigger	To	Side effects
created	lockBindings ok	bound	persist bindings; emit run.started
created	lockBindings fail	failed	emit run.failed
bound	phase plan needed	planning	emit phase.started
planning	plan artifact valid	awaiting_approval	request approval
awaiting_approval	approve	executing	emit approval.resolved, run.resumed
awaiting_approval	reject	failed	emit run.failed
awaiting_approval	request_changes	planning	increment phase attempts
awaiting_approval	timeout	paused	set paused_from_state='awaiting_approval'
executing	phase ok, more phases	executing	next phase
executing	normal workflow approval gate	awaiting_approval	request gate
executing	all phases done	completed	emit run.completed, write final report
executing	unrecoverable error	failed	emit run.failed
executing	manual pauseRun	paused	set paused_from_state='executing'
planning	manual pauseRun	paused	set paused_from_state='planning'
paused	resume	paused_from_state	emit run.resumed, clear paused_from_state
any non-terminal state	abortRun	aborted	emit run.aborted, dispose sessions

Non-terminal states for abortRun:

• created
• bound
• planning
• awaiting_approval
• executing
• paused

13.2 Run Phase State

States:

• pending
• running
• awaiting_artifact
• validating
• awaiting_approval
• completed
• failed
• skipped

Transitions:

From	Trigger	To
pending	start	running
running	prompt sent, artifact expected	awaiting_artifact
awaiting_artifact	artifact appears	validating
awaiting_artifact	timeout	running after probe/repair, or failed after exhaustion
validating	valid	awaiting_approval if gate, else completed
validating	invalid	running after one repair, else failed
awaiting_approval	approve	completed
awaiting_approval	reject / abort	failed
awaiting_approval	request_changes	running, attempt + 1

Replay rules:

• phase.started.payload.repair === true marks that attempt as the single allowed repair attempt. Replaying that attempt MUST use repair instructions, prompt.repaired, and must not start a third attempt.
• Repair replay from running may reuse an existing READY / bootstrapped session even if last_prompt_hash still contains the previous attempt's prompt hash; current-attempt prompt send has not happened yet.
• If phase state is running, existing artifact files are never accepted unless the current prompt event (prompt.sent or prompt.repaired) for the current dedup key is already recorded. Replay without prompt proof treats existing files as stale.
• If phase state is running, session state is BUSY, and last_prompt_hash matches the current prompt but the matching prompt event is missing, replay waits for the artifact with the current file signature as the baseline. This preserves idempotency without validating a stale pre-existing artifact.
• Baseline-protected waits must not synthesize durable prompt proof before the wait finishes. If replay crashes or is cancelled before validation, the next replay must still treat the existing artifact as baseline/stale unless real prompt proof already exists.
• If phase state is validating and no artifact row exists yet, replay re-reads and validates the current expectedArtifactPath instead of treating the state as corruption.
• If phase state is validating and artifact rows already exist for the same phase/path/schema, replay may reuse only an artifact row created at or after the current session last_prompt_at; older rows are treated as stale previous-attempt outputs and the file is revalidated.
• Session bootstrap DB row/state changes and session.created / session.ready events are written in one DB transaction after adapter start succeeds.

14. Approval State

States:

• pending
• approved
• rejected
• changes_requested
• aborted
• paused

14.1 Transitions

From	Event	To	Side effects
pending	approve decision	approved	insert decision row
pending	reject decision	rejected	insert decision row; run -> failed
pending	request_changes decision	changes_requested	insert decision row; increment attempt
pending	abort decision	aborted	insert decision row; run -> aborted
pending	timeout	paused	run -> paused; no decision row
paused	unpause	pending	re-arm gate; no decision row
terminal states	any decision	unchanged	return 409

Rules:

• A pending request can transition to one non-pending state per pending epoch.
• Terminal approval states reject further decisions.
• paused may return to pending only through unpause.
• Manual pause is run-level pauseRun; it leaves approval gate in pending.
• Only approve, reject, request_changes, and abort create approval_decisions rows.
• Default timeout is null.
• Timeout never auto-approves or auto-rejects.

14.2 Decision Idempotency

• GUI:
  • UUIDv4 per click.
  • reused across automatic UI retries for the same logical action.
• CLI:
  • UUIDv4 per invocation.
  • --client-token=<uuid> override for scripted retry.
• API:
  • existing (approval_request_id, action, client_token) returns existing row with status 200.
  • new decision inserts row and returns 201.
  • same token with different action returns 409.
  • decision on non-pending request returns 409.

14.3 Destructive Command Enforcement

Devflow-direct commands have hard enforcement. TUI-agent commands have best-effort enforcement.

Hard-blocked Devflow-direct patterns:

• rm -rf
• git reset --hard
• git clean
• git push --force
• git push --force-with-lease
• git worktree remove --force
• git branch -D
• docker volume rm
• docker compose down -v
• DROP DATABASE
• DROP SCHEMA
• migration rollback
• reads/writes touching .env*, ~/.ssh/, ~/.aws/, ~/.config/gcloud/, ~/.kube/
• files matching *token*, *secret*, *credentials*, *.pem, *.key

TUI-agent command enforcement is best-effort:

1. Prelude prohibits destructive operations.
2. Backend permission mode is set to safest available mode.
3. Transcript audit captures post-hoc evidence.
4. Human intervention goes through devflow attach.
5. Worktrees and branches are preserved by default.

v1 does not claim real-time blocking of TUI-internal commands.

15. Run Engine and Temporal Contract

The M4 RunEngine contract is frozen before M5. M5 reimplements the same interface through Temporal.

15.1 Public API

interface RunEngine {
  startRun(input: RunStartInput): Promise<{ runId: string }>;
  signalApproval(
    runId: string,
    approvalRequestId: string,
    action: ApprovalDecisionAction,
    clientToken: string,
    comment?: string
  ): Promise<void>;
  pauseRun(runId: string): Promise<void>;
  resumeRun(runId: string): Promise<void>;
  abortRun(runId: string, reason: string): Promise<void>;
  getStatus(runId: string): Promise<RunStatus>;
}

15.2 Temporal Shape

• Namespace: devflow.
• Task queue: devflow-runs.
• Single worker process: apps/worker.
• Workflow: runWorkflow(input: RunStartInput).
• Signals:
  • approve
  • pause
  • resume
  • abort
  • unpause
• No Updates in M5.
• Status is read from DB.

Activities:

• M5 compatibility activity surface:
  • prepareRunActivity(input)
  • lockBindingsActivity(runId)
  • failRunActivity(runId, reason)
  • advanceRunActivity(runId)
  • signalApprovalActivity(runId, approvalRequestId, action, clientToken, comment?)
  • pauseRunActivity(runId)
  • resumeRunActivity(runId)
  • abortRunActivity(runId, reason)
  • getStatusActivity(runId)
  • isRunTerminalActivity(runId)
  • composeFinalReportActivity(runId)
• advanceRunActivity is the M5 parity wrapper over M4 phase advancement. It may internally perform prompt send, artifact wait/validation, event recording, and approval request creation through the same DB/idempotency contracts already locked in sections 8 through 14.
• The granular activity split (sendPromptToSession, waitForArtifact, validateArtifact, recordEvent, requestApproval, runCommand) is deferred to a later hardening ADR. It is not an M5 acceptance gate.
• Prompt/session mutation still occurs only inside worker-hosted activities through SessionManager. M5+ API code never mutates SessionAdapter directly.

Retry policy:

• Default: max attempts 3, exponential backoff start 1s, max 30s.
• composeFinalReportActivity: max attempts 1.
• Activity-level failures serialize DevflowError; non-recoverable Devflow errors are rethrown as non-retryable Temporal failures.
• advanceRunActivity is cancellation-aware and idempotent by DB state, event idempotency keys, prompt dedup keys, and artifact content keys.
• Already-applied approval signal replay repairs missing final reports for every terminal run state: completed, failed, and aborted, regardless of whether the replayed approval action was approve, request_changes, reject, or abort.
• API-side already-applied approval replay is report-repair only. It must not call SessionAdapter mutation methods; reject/abort session disposal belongs to the worker/session-manager path that originally applies the decision.
• If a workflow closes before the API observes an approval signal result, closed-workflow settlement must first verify the requested decision was applied, then replay approval side effects, then wait for the terminal report.

15.3 Hard Constraints

• Workflow code holds only serializable state.
• No tmux handles in workflow state.
• No PTY refs in workflow state.
• No DB clients in workflow state.
• M5+ session interaction happens through activities calling SessionManager in apps/worker.
• M5+ API never calls mutating SessionAdapter methods.
• SessionManager advisory lock prevents API/worker ownership conflict during M4 -> M5 transition.
• Workflow code uses deterministic clock/randomness only.

16. WriteSet and Worktree

16.1 WriteSet

• Each task declares writeSet: string[].
• Patterns are relative to repo root.
• Glob engine: fast-glob.
• Options:

{
  cwd: worktreeRoot,
  dot: true,
  followSymbolicLinks: false,
  onlyFiles: true,
  suppressErrors: false
}

Conflict detection:

1. Expand writeSets.
2. Forbidden globs cause conflict if matched by more than one task:
   • pnpm-lock.yaml
   • package-lock.json
   • **/migrations/**
   • **/*.generated.*
   • root tsconfig*.json
   • biome.json
   • lefthook.yml
   • .github/**
   • .gitlab-ci.yml
3. Pairwise file intersections must be empty.

Conflict creates parallel_dag_approved gate.

16.2 Worktree Lifecycle

• Worktree root:
  • WORKSPACE_ROOT/<runId>/<laneId>
  • non-parallel main lane: WORKSPACE_ROOT/<runId>/main
• Created via git worktree add.
• Branch name:

devflow/<runId>/<laneId>

• Terminal run state does not remove worktrees or branches.
• Output branches are deliverables.
• Disk growth is accepted.
• Cleanup is manual:

devflow cleanup <run-id> [--lane=<id>]

Cleanup:

• uses git worktree remove without --force by default.
• refuses dirty worktrees.
• --force requires an additional gate.
• git branch -D is destructive and gated.
• doctor --list-orphans lists only; it never removes.

17. SSE Contract

Endpoints:

• GET /sse/runs/:runId
• GET /sse/global

Heartbeat every 15 seconds.

Events:

Event	Scope
run.state_changed	both
run.event_appended	run
phase.state_changed	run
approval.created	both
approval.resolved	both
session.state_changed	run
transcript.chunk_appended	run
artifact.validated	run

Reconnect:

• Run-scoped /sse/runs/:runId:
  • Last-Event-ID is last run_events.seq for that run.
  • server replays run.event_appended for seq > lastSeq.
  • derived non-run.event_appended SSE types are not replayed for historical rows; state is re-derived by fetch.
• Global /sse/global:
  • Last-Event-ID is last global run_events.id, because run_events.seq is only monotonic within a run.
  • fresh connects start at the latest global event id and emit only new summary events.
  • reconnects replay rows with id > lastId.
  • global stream emits only scope=both events: run.state_changed, approval.created, approval.resolved.
  • global stream never emits run.event_appended.

18. Errors

v4: my_deepagent.errors.MyDeepAgentError (replaces v3 DevflowError 1:1):

class ErrorClass(StrEnum):
    RECOVERABLE = "recoverable"
    HUMAN_REQUIRED = "human_required"
    FATAL = "fatal"


class MyDeepAgentError(Exception):
    error_class: ErrorClass
    code: str
    run_id: UUID | None
    phase_id: UUID | None
    recovery_hint: str | None
    cause: BaseException | None

Recoverable:

• network_blip
• pane_briefly_unresponsive
• prompt_send_transient
• db_serialization_retry
• rate_limited

Human required:

• artifact_invalid_after_repair
• artifact_timeout_exhausted
• prompt_send_exhausted
• destructive_command_blocked
• secret_access_blocked
• backend_unavailable
• no_eligible_persona
• writeset_conflict
• merge_conflict
• objective_not_met
• review_dispute_unresolved
• backend_auth_failed
• model_unavailable
• token_budget_exceeded (v4 r1: BudgetTracker rejects a call whose estimated cost would breach the per-run, per-day, or per-persona-daily cap with on_hit=block.)
• tool_quota_exceeded (v4 r1: SafetyShellMiddleware blocked a tool call due to deny-path / destructive-command policy, or a per-phase tool-call cap was hit.)

Fatal:

• db_unreachable
• workspace_permissions
• internal_state_corruption
• template_load_failed
• artifact_schema_unknown
• artifact_schema_load_failed
• migration_pending
• config_invalid

Mapping:

• recoverable -> retry; exhausted -> human_required.
• human_required / recovery gate -> run paused and gate created. This is distinct from normal workflow approval gates in §13.1, which use awaiting_approval.
• fatal -> run failed, sessions disposed, final report best-effort.

19. Concurrent Runs and Crash Recovery

19.1 Active Run Uniqueness

• MAX_CONCURRENT_RUNS, default 4.
• DB partial unique index is the source of truth:
  • one active run per (repo_path, base_branch).
• repo_path is canonicalized before insert.
• Advisory lock is auxiliary only:

pg_try_advisory_xact_lock(hash64('devflow:start-run', repoPath, baseBranch))

• Unique-index violation returns:

{ "currentRunId": "...", "currentState": "..." }

with HTTP 409.

19.2 Crash Recovery

M4, no Temporal:

• On apps/api startup, sweep non-terminal runs.
• Mark them failed.
• final_report_path = null.
• Append synthesized run.failed with reason process_restart_unrecovered.
• Cascade associated tui_sessions to FAILED_NEEDS_HUMAN.
• Append session.failed.
• This frees active-run uniqueness slots.

M5+:

• No sweep.
• Temporal durability owns in-flight workflow recovery.
• SessionManager resumes tmux sessions.
• Active-run partial index blocks duplicate runs until completion or explicit abort.

20. Milestones

M1: Monorepo + Postgres + CLI Doctor

• Scaffold workspace.
• Add pnpm, tsconfig, biome, lefthook, Vitest.
• Add Docker Compose for Postgres.
• Add Drizzle and first migration.
• Add devflow doctor.
• Implement checks 1-9.
• Stub checks 10-12 as warn where needed.
• Add SSE compatibility smoke test:
  • minimal Fastify 5 server.
  • fastify-sse-v2 plugin.
  • 30-second integration test.
  • receive 3 events and reconnect.
  • if plugin fails, implement native reply.raw SSE helper before M1 is green.

M2: Core Schema + Registry + Binding

• Implement enums.
• Implement canonical hashing.
• Implement Template schema.
• Implement Persona schema.
• Implement seed loader.
• Implement binding algorithm.
• Implement artifact schema registry.
• Add first schemas:
  • dev/spec@1
  • dev/phase-plan@1
  • common/final-report@1
• Tests:
  • schema validation.
  • override semantics.
  • risk enforcement.
  • diversity enforcement.
  • deterministic auto-select.

M3: Fake Session Runtime

• Implement SessionAdapter.
• Implement FakeSessionAdapter.
• Implement prompt envelope.
• Implement event recorder.
• Implement fake sentinel scenarios.
• Persist transcript chunks.
• Tests:
  • prompt correlation.
  • artifact validation.
  • invalid artifact.
  • timeout.
  • fake crash.

M4: Minimal Run Engine

• Implement packages/run-engine.
• Used directly by apps/api.
• No Temporal.
• Supports:
  • start run.
  • lock bindings.
  • approval.
  • fake prompt.
  • artifact wait/validate.
  • final report.
• Freeze the RunEngine contract.
• Full fake development@1 minus reviewers.

M5: Temporal Integration

• Reimplement RunEngine through Temporal.
• Preserve M4 behavior.
• Add parity tests using the same M4 scenarios.
• M5+ SessionManager lives in apps/worker.

M6: Real tmux SessionManager

• Implement TmuxSessionAdapter.
• Decoupled from M5.
• May begin after M3 is stable.
• Pre-M5 real tmux is opt-in smoke only.
• Production run path remains fake until both M5 and M6 are green.

M7: TUI Recovery State Machine

• Implement session state transitions.
• Implement recovery counters.
• Implement escalation to human gates.

M8: API + GUI Minimum

• Implement Fastify routes.
• Implement SSE.
• Implement GUI screens:
  • Dashboard.
  • Templates.
  • Personas.
  • New Run.
  • Run Detail.
  • Approvals.
  • TUI Sessions.

M9: development@1 Fake-Agent Full Run

• Add curated development@1.
• Add review consensus.
• Add verifier flow with fake reviewers.
• Add coverage gate >=70% lines for core/session/run-engine.

M10: Codex/Claude Opt-In Real Run

• Implement profiles:
  • packages/session/src/profiles/codex.ts
  • packages/session/src/profiles/claude.ts
• Real backends become production-default only after both M5 and M6 are green.
• Until then real tmux/Codex/Claude are developer-flagged opt-in smoke only.

M11: Parallel Lanes

• Add task DAG scheduler.
• Add writeSet detection.
• Add per-lane worktrees.
• Add merge coordinator.
• Add conflict gates.

M12: Backtest Workflow

• Add backtest-strategy@1.
• Add objective evaluator.
• Add metric parser extension points.
• Add failure mining artifacts.
• Add Backtest Lab GUI.

M13: Template Factory

• Generate draft template from natural language and repo discovery.
• Add harness design.
• Add template review.
• Add dry-run and promote flow.

21. Out of Scope

• Authentication.
• Authorization.
• Multi-user support.
• Data retention or archival policy.
• Observability dashboards.
• Remote template/persona registries.
• Multi-machine deployment.
• HA.
• Managed backups.
• Web ingress.
• TLS.
• Reverse proxy.

22. Decision Log

Open Questions Closed

#	Question	Resolution
OQ-1	Persona/template seeding format	Immutable YAML at docs/schemas/{personas,templates}/<name>@<version>.yaml
OQ-2	Approval timeout default	null; timeout freezes only
OQ-3	Final report format	Markdown and JSON
OQ-4	Temporal namespace/queue	namespace devflow, task queue devflow-runs
OQ-5	WriteSet glob engine	fast-glob
OQ-6	Backtest objective DSL	Stub in M12, full DSL deferred
OQ-7	Codex/Claude prompt prelude	Structure locked, exact text deferred to M10

Blocking Corrections Applied

#	Issue	Resolution
CC-1	Terminal state deleted worktrees/branches	Preserve by default; manual gated cleanup only
CC-2	SessionManager location conflict	M4 API, M5+ worker
CC-3	Event duplicates under retry	run_events.idempotency_key
CC-4	Destructive command enforcement overclaimed	Devflow-direct hard, TUI best-effort
CC-5	UUID extension missing	CREATE EXTENSION IF NOT EXISTS pgcrypto
CC-6	Advisory lock not enough for active-run uniqueness	partial unique index
CC-7	Undefined transition sequence in event keys	cause-based keys
CC-8	Approval paused transition missing	explicit approval transition table
CC-9	AutoSelect order nondeterministic	deterministic sort
CC-10	SSE plugin compatibility assumed	M1 smoke + native fallback
CC-11	ApprovalAction included pause	split ApprovalDecisionAction; pauseRun is run-level
CC-12	Artifact hash key collision	include phase id and path
CC-13	Resume previous state not stored	runs.paused_from_state
CC-14	repo path aliasing	canonical realpath storage
CC-15	M4 sweep left tmux sessions ambiguous	cascade session state to FAILED_NEEDS_HUMAN; real tmux production-default only after M5+M6
CC-16	Prompt hash used phaseId but envelope uses phaseKey	prompt hash uses phaseKey
CC-17	abortRun transition too narrow	abort from any non-terminal run state
CC-18	approval pending transition wording conflicted with pause epoch	pending can transition once per pending epoch; paused may unpause to pending
CC-19	tsc -b --noEmit is brittle with TypeScript 5.6 project references on clean worktrees	build still uses tsc -b; no-emit verification uses root tsconfig.typecheck.json
CC-20	sendPrompt retry count was ambiguous against Temporal activity attempts	§8.3 now states retry budget means initial attempt plus retries; §15.2 remains Temporal-level attempts only
CC-21	Duplicate prompt dedup handling conflicted with adapter retry idempotency	duplicate dedupKey returns idempotent success without reprocessing
CC-22	Normal workflow approval gates and human-required recovery gates were easy to conflate	§13.1 names normal workflow gates; §18 keeps human_required recovery gates paused
CC-23	Phase start and event append could diverge under retry/error	phase start and phase.started append occur in one DB transaction
CC-24	Repair attempt replay lost repair prompt identity and one-repair budget	repair attempts are derived from phase.started.payload.repair, replay uses repair instructions and prompt.repaired, and cannot start attempt 3
CC-25	validating replay failed if crash happened before artifact row insert	replay revalidates the expected artifact file when state is validating but no artifact row exists
CC-26	Session bootstrap state/events could diverge	session row/state and session.created / session.ready events are committed in one DB transaction
CC-27	validating replay could reuse stale previous-attempt artifact rows	artifact-row replay requires artifact.created_at >= tui_sessions.last_prompt_at; otherwise the file is revalidated
CC-28	repair running replay rejected existing READY sessions with previous attempt prompt hash	current-attempt repair prompt is considered unsent, so replay may reuse the session and send prompt.repaired
CC-29	API Temporal approval replay omitted M4 approval side-effect repair	API approval signal reader now wires replayAppliedApprovalSideEffects, so already-applied terminal approval replays can repair missing final reports
CC-30	running replay could validate stale artifacts without prompt proof	running replay requires matching prompt event proof; BUSY replay without prompt event uses current artifact signature as baseline and ignores stale files
CC-31	M5 activity list over-specified granular activities not implemented by the M4 parity adapter	M5 locks the compatibility activity wrapper surface; granular activity split is deferred to a later hardening ADR
CC-32	Already-applied approve / request_changes replay repaired missing reports for completed / failed but missed aborted	approval replay side-effect repair now composes missing final reports for all terminal states
CC-33	API-side already-applied reject / abort replay tried to dispose sessions through DB-only replay validation runtime	API replay side effects are report-repair only; worker-side decision application owns session disposal
CC-34	Closed-workflow approval settlement waited for reports but did not replay approval side effects	settlement now verifies the requested decision, replays side effects, then waits for the terminal report
CC-35	Baseline-protected BUSY replay recorded synthetic prompt proof before the baseline wait was durable	baseline replay no longer records synthetic prompt events; replay without real prompt proof keeps treating existing files as stale
CC-36	SSE reconnect wording used per-run seq for global stream even though seq is not globally monotonic	/sse/runs/:runId uses per-run seq; /sse/global uses global run_events.id and emits only scope=both summary events
CC-37	Run SSE replay could emit historical derived events after the first page	run SSE drains historical rows up to a high-water seq with only run.event_appended, then switches to live derived events
CC-38	Normal phase start changed run state to planning / executing without a summary event source	phase.started payload includes runState; SSE derives run.state_changed from that live event
CC-39	No OpenRouter HTTP backend; users cannot pick cost-tuned per-persona models	add openrouter to Backend enum; HTTP OpenRouterAdapter in §8.5; persona modelConfig.model requirement; doctor check 13; new error codes rate_limited, backend_auth_failed, model_unavailable (final v3 entry — v4 reinterprets the OpenRouter integration as the deepagents-driven session adapter; the standalone HTTP OpenRouterAdapter from CC-39 is superseded by DR-1)

Decision Records (v4)

ID	Decision	Rationale	Impact
DR-1	v3 → v4 major bump: delete TS monorepo, rewrite in Python on LangChain deepagents.	(1) Claude/Anthropic direct API cost is prohibitive for a single-user toolchain. (2) OpenRouter cost-tuned models (DeepSeek, etc.) require a multi-turn, tool-using agent harness; deepagents is Python-only with no 1:1 TS port. (3) Switching languages is shorter than reimplementing the harness.	Step 0 (commit 0e61b2d) deleted apps/, packages/, tests/, scripts/, pnpm/tsconfig metadata. The Python rewrite lives at my-deepagent/ and reached Step 15 (real OpenRouter E2E PASS, ~$0.05/run) before the v3 codebase was removed. CC-39's separate OpenRouterAdapter is replaced by my_deepagent.session.build_agent (deepagents 0.6.1 with LocalShellBackend + SafetyShellMiddleware). v3 CC counters frozen; v4 begins its own series. Recovery: git checkout pre-python-rewrite -- <path>.

Future Open Questions

• FOQ-1, M12: full backtest objective DSL.
• FOQ-2, M13: template factory generation prompts.
• FOQ-3, post-M10: optional third backend such as Gemini.
• FOQ-4, post-M8: WebSocket vs SSE if transcript pressure requires it.

23. Kickoff Order

v3 historical order (TS, completed up to M8 before the v4 pivot):

1. M1.1: repo + pnpm + tsconfig + biome + lefthook + vitest workspace.
2. M1.2: docker-compose + Postgres healthcheck + drizzle-kit + first migration.
3. M1.3: apps/cli skeleton + devflow doctor.
4. M1.4: packages/core skeleton with config, enums, errors, hash, prompt-envelope, run-event types.
5. M2.1: Zod schemas for Template/Persona, persona YAML loader, hashing.
6. M2.2: Binding algorithm + tests.
7. M2.3: Artifact schema registry + first three schemas.
8. M3.1: SessionAdapter interface + FakeSessionAdapter.
9. M3.2: Transcript chunk capture + DB persistence.
10. M3.3: engine-shaped harness running a single fake phase end-to-end.
11. M4: assemble run engine; lock contract; full fake development@1 minus reviewers.
12. M5 in parallel with M6 once M4 is green.

v4 r1 order (Python, status as of v0.1.0):

Step	Scope	Status
Step 0	Scaffold my-deepagent/ (uv workspace, ruff, mypy, alembic, .pre-commit)	DONE (17ba5d7)
Step 1	devflow_core → my_deepagent.{config,enums,errors,hash,persona,prompt_envelope,run_event}	DONE
Step 2	devflow_db → my_deepagent.persistence.{db,models,checkpointer} + Alembic baseline	DONE
Step 3	mydeepagent doctor (typer)	DONE
Step 4	Persona / workflow seeding + binding (my_deepagent.{persona,workflow,binding})	DONE
Step 5	Artifact schema registry (my_deepagent.artifact_schema)	DONE
Step 6	Distribution: init/login/logout/keys, governance consent, i18n (ko/en)	DONE
Step 7	WorkflowEngine + ArtifactWatcherMiddleware (replaces v3 §15 in-process engine)	DONE
Step 8	Budget guardrails (my_deepagent.budget + cost preview + CostMiddleware)	DONE
Step 9	Crash recovery + concurrency (my_deepagent.recovery + mydeepagent runs …)	DONE
Step 10	Interactive REPL (mydeepagent no-subcommand + slash commands)	DONE
Step 11	Audit log + structlog secret scrubbing	DONE
Step 12	Doctor 8-check + OpenRouter pricing fetch + mydeepagent pricing	DONE
Step 13	Tmux adapter (M6-Py)	DEFERRED — not in v0.1.0
Step 14	TUI recovery (M7-Py)	DEFERRED — not in v0.1.0
Step 15	End-to-end real OpenRouter integration test	DONE (733c9be)
Step 0-purge	Delete v3 TS monorepo per DR-1	DONE (0e61b2d)
M5-Py	Temporal worker (apps/worker)	NEXT
M8-Py	FastAPI + SSE (apps/api)	NEXT