Complete batch import v2 runtime and host capability recovery

This commit is contained in:
phamnazage-jpg
2026-05-23 09:18:02 +08:00
parent e50c292c7f
commit cfa1eaa904
60 changed files with 3718 additions and 530 deletions

View File

@@ -0,0 +1,784 @@
# Batch Auto-Import V2 Implementation Plan
> **For Claude:** REQUIRED SUB-SKILL: Use superpowers:executing-plans to implement this plan task-by-task.
**Goal:** 实现 V2 的 URL + key 批量导入能力,覆盖模型发现、同模型别名归并、重复导入复用、异步确认、最终 gateway 验证、结果 API 与结果页所需状态投影。
**Architecture:** 采用 `BatchImportService + ConfirmationWorker + ValidationService + RunStateStore + ResultProjection` 分层架构。V2 只以 `import_runs / import_run_items / import_run_item_events` 作为运行态真相,旧 `import_batches/*` 仅保留 legacy linkage。重复导入决策基于 `provider_id + api_key_fingerprint + canonical_model_family`,最终可用性只认宿主真实 `/v1/chat/completions`
**Tech Stack:** Go 1.22.2、`database/sql` + SQLite、Chi、OpenAPI 3.1、Go `testing``httptest`、现有 `internal/host/sub2api` 适配层与 `tests/integration` 集成测试套件。
---
## 0. 实施约束
- 只通过宿主 HTTP API 工作,不直写宿主数据库。
- 所有状态枚举、字段名、API 路由必须遵循当前 canonical contract。
- 每个任务都先写失败测试,再做最小实现,再跑验证。
- 每个任务独立提交,避免大而混杂的 commit。
- 任何 UI/API 展示都只能读 V2 canonical state不得回退到日志拼接。
## 1. 任务总览
```text
T1 Canonical types and enums
T2 Probe models + alias normalization + canonical family
T3 Capability profile + smoke completion routing
T4 Provider ID + reuse policy
T5 Run/item/event state store repositories
T6 BatchImportService: Stage 0~2
T7 ConfirmationWorker + retry + lease
T8 ValidationService + access status
T9 ResultProjection
T10 HTTP API: runs/items
T11 CLI: batch-import
T12 Integration + contract verification
T13 Design restoration audit
```
## 2. 设计还原验证矩阵
### 2.1 目标覆盖矩阵
| 设计目标 | 对应任务 | 验证方式 |
|---|---|---|
| URL + key 自动发现模型 | T2, T6, T12 | `/v1/models` 拉取、集成测试 |
| 模型纠错与别名归一化 | T2, T4, T9, T12 | unit + item detail projection |
| 同模型跨中转快速识别 | T2, T4, T12 | `canonical_model_family` 测试 |
| 重复导入自动复用 | T4, T6, T9, T12 | reuse decision + projection |
| 已启用重复账号直接复用 | T4, T6, T9, T12 | `matched_account_state=active` |
| 已停用/已弃用账号快速启用 | T4, T6, T7, T9, T12 | `account_resolution=reactivated` |
| transport + model capability profile | T3, T9, T10, T12 | profile persistence + API schema |
| channel/account 演化 | T6, T12 | patch contract + host stub |
| 异步确认与重试 | T7, T12 | lease/retry/event trail |
| gateway completion 最终判定 | T8, T12 | `access_status` 唯一写入 |
| 结果 API 与结果页数据源 | T5, T9, T10, T12 | run/item/event projection |
| 单一状态源 | T5, T7, T8, T9 | 只读 `import_runs/*` |
### 2.2 契约覆盖矩阵
| 契约 | 对应任务 |
|---|---|
| `run_id / item_id / provider_id` | T1, T4, T5 |
| `run.state` | T1, T5, T9 |
| `current_stage / confirmation_status / access_status` | T1, T5, T7, T8 |
| `matched_account_state / account_resolution` | T4, T5, T6, T9, T10 |
| `api_key_fingerprint` | T4, T5, T6 |
| `canonical_model_families` | T2, T4, T5, T9, T10 |
| `provision_reused / reused_from_*` | T4, T5, T6, T9, T10 |
| `/api/batch-import/runs*` | T10, T12 |
如果 T1~T12 全部完成并通过验证T13 必须能证明上述矩阵全部为“已覆盖”,否则不得宣称 V2 可按设计实现。
## 3. 实施任务
### Task 1: Canonical Types And Enums
**Files:**
- Create: `internal/batch/types.go`
- Test: `internal/batch/types_test.go`
- Reference: `docs/2026-05-21-BATCH_AUTO_IMPORT_SPEC.md`
**Step 1: Write the failing test**
为以下枚举写失败测试:
- `RunState`
- `ItemStage`
- `ConfirmationStatus`
- `AccessStatus`
- `MatchedAccountState`
- `AccountResolution`
至少覆盖:
- 常量值是否与文档一致
- 非法字符串是否会在后续解析层被拒绝
**Step 2: Run test to verify it fails**
Run:
```bash
go test ./internal/batch -run 'TestRunStateConstants|TestItemStateConstants' -count=1
```
Expected: FAIL提示类型或常量不存在。
**Step 3: Write minimal implementation**
`internal/batch/types.go` 中定义上述类型与常量,不提前引入不需要的 helper。
**Step 4: Run test to verify it passes**
Run:
```bash
go test ./internal/batch -run 'TestRunStateConstants|TestItemStateConstants' -count=1
```
Expected: PASS
**Step 5: Commit**
```bash
git add internal/batch/types.go internal/batch/types_test.go
git commit -m "feat(batch): add canonical v2 state enums"
```
### Task 2: Probe Models, Alias Normalization, Canonical Family
**Files:**
- Create: `internal/probe/models.go`
- Create: `internal/probe/aliases.go`
- Test: `internal/probe/models_test.go`
- Test: `internal/probe/aliases_test.go`
- Reference: `docs/2026-05-21-BATCH_AUTO_IMPORT_TDD_PLAN.md`
**Step 1: Write the failing test**
覆盖:
- `/v1/models` OpenAI 格式解析
- 空模型列表
- 鉴权失败
- `kimi 2.6 / kimi-2.6 / kimi-k2.6` 归并到同一 `canonical_model_family`
- `deepseek-ai/DeepSeek-V4-Pro` vendor 前缀归一化
**Step 2: Run test to verify it fails**
Run:
```bash
go test ./internal/probe -run 'TestProviderModels|TestCanonicalModelFamily' -count=1
```
Expected: FAIL
**Step 3: Write minimal implementation**
实现:
- `ProviderModels`
- `NormalizeModelID`
- `CanonicalModelID`
- `CanonicalModelFamily`
- `BuildAliasTable`
- `ResolveRequestedModel`
- `RecommendModels`
**Step 4: Run test to verify it passes**
Run:
```bash
go test ./internal/probe -run 'TestProviderModels|TestCanonicalModelFamily' -count=1
```
Expected: PASS
**Step 5: Commit**
```bash
git add internal/probe/models.go internal/probe/aliases.go internal/probe/models_test.go internal/probe/aliases_test.go
git commit -m "feat(probe): add model discovery and canonical family normalization"
```
### Task 3: Capability Profile And Smoke Completion Routing
**Files:**
- Create: `internal/probe/capability.go`
- Create: `internal/probe/completion.go`
- Test: `internal/probe/capability_test.go`
- Test: `internal/probe/completion_test.go`
- Reference: `docs/2026-05-22-BATCH_AUTO_IMPORT_V2_ARCHITECTURE.md`
**Step 1: Write the failing test**
覆盖:
- `responses` 不支持但 `chat/completions` 可用
- transport profile 的 advisory 记录
- per-model profile 记录
- `ResolveSmokeModel` 基于别名与能力选择 smoke model
**Step 2: Run test to verify it fails**
Run:
```bash
go test ./internal/probe -run 'TestProbeCapabilities|TestResolveSmokeModel|TestSmokeCompletion' -count=1
```
Expected: FAIL
**Step 3: Write minimal implementation**
实现:
- `TransportProfile`
- `ModelCapabilityProfile`
- `CapabilityProfile`
- `ProbeCapabilities`
- `CompletionResult`
- `ResolveSmokeModel`
- `SmokeCompletion`
**Step 4: Run test to verify it passes**
Run:
```bash
go test ./internal/probe -run 'TestProbeCapabilities|TestResolveSmokeModel|TestSmokeCompletion' -count=1
```
Expected: PASS
**Step 5: Commit**
```bash
git add internal/probe/capability.go internal/probe/completion.go internal/probe/capability_test.go internal/probe/completion_test.go
git commit -m "feat(probe): add capability profile and smoke completion routing"
```
### Task 4: Provider ID And Reuse Policy
**Files:**
- Create: `internal/batch/provider_id.go`
- Create: `internal/batch/reuse_policy.go`
- Test: `internal/batch/provider_id_test.go`
- Test: `internal/batch/reuse_policy_test.go`
- Reference: `docs/2026-05-21-BATCH_AUTO_IMPORT_SPEC.md:336`
**Step 1: Write the failing test**
覆盖:
- 同 host 不同 path 生成不同 `provider_id`
- 已存在 active provider 且 family 已覆盖 -> `reused`
- 已存在 active account -> `matched_account_state=active`, `account_resolution=reused`
- `disabled/deprecated` 账号 -> `reactivated`
- `broken` provider/account -> `replace`
- 同 family 不同 alias -> 视为已覆盖
**Step 2: Run test to verify it fails**
Run:
```bash
go test ./internal/batch -run 'TestNormalizeProviderID|TestDecideReuse' -count=1
```
Expected: FAIL
**Step 3: Write minimal implementation**
实现:
- `NormalizeProviderID`
- `ReuseDecision`
- `DecideReuse`
不要在这一步直接改 service。
**Step 4: Run test to verify it passes**
Run:
```bash
go test ./internal/batch -run 'TestNormalizeProviderID|TestDecideReuse' -count=1
```
Expected: PASS
**Step 5: Commit**
```bash
git add internal/batch/provider_id.go internal/batch/reuse_policy.go internal/batch/provider_id_test.go internal/batch/reuse_policy_test.go
git commit -m "feat(batch): add provider id and reuse policy"
```
### Task 5: Run/Item/Event State Store Repositories
**Files:**
- Modify: `internal/store/migrations/0007_batch_import_runs.sql`
- Modify: `internal/store/migrations/0008_batch_import_run_events.sql`
- Modify: `internal/store/sqlite/import_runs_repo.go`
- Create: `internal/store/sqlite/import_run_items_repo.go`
- Create: `internal/store/sqlite/import_run_item_events_repo.go`
- Modify: `internal/store/sqlite/db.go`
- Test: `internal/store/sqlite/import_runs_repo_test.go`
- Test: `tests/integration/store_init_test.go`
**Step 1: Write the failing test**
覆盖:
- run 创建/更新
- item upsert 持久化 `api_key_fingerprint / canonical_model_families / matched_account_state / account_resolution / provision_reused`
- event append/list
- lease 字段持久化
**Step 2: Run test to verify it fails**
Run:
```bash
go test ./internal/store/sqlite/... ./tests/integration/... -run 'TestRunStateStore|TestStoreAppliesLatestMigration' -count=1
```
Expected: FAIL
**Step 3: Write minimal implementation**
补足 repo 与 migration确保 schema 与文档完全一致。
**Step 4: Run test to verify it passes**
Run:
```bash
go test ./internal/store/sqlite/... ./tests/integration/... -run 'TestRunStateStore|TestStoreAppliesLatestMigration' -count=1
```
Expected: PASS
**Step 5: Commit**
```bash
git add internal/store/migrations/0007_batch_import_runs.sql internal/store/migrations/0008_batch_import_run_events.sql internal/store/sqlite/import_runs_repo.go internal/store/sqlite/import_run_items_repo.go internal/store/sqlite/import_run_item_events_repo.go internal/store/sqlite/db.go internal/store/sqlite/import_runs_repo_test.go tests/integration/store_init_test.go
git commit -m "feat(store): complete v2 runtime state repositories"
```
### Task 6: BatchImportService Stage 0~2
**Files:**
- Create: `internal/batch/service.go`
- Create: `internal/batch/capability_profile.go`
- Create: `internal/batch/channel_evolution.go`
- Test: `internal/batch/service_test.go`
- Test: `internal/batch/channel_evolution_test.go`
- Reference: `internal/provision/import_service.go`
**Step 1: Write the failing test**
覆盖:
- 创建 run + items
- reuse preflight 跳过重复 provision
- active 账号重复导入 -> reused
- deprecated 账号重复导入 -> reactivated
- patch-only 新 alias
- legacy batch/provider link 回写
**Step 2: Run test to verify it fails**
Run:
```bash
go test ./internal/batch -run 'TestBatchImport_StartRun|TestModelMappingDelta' -count=1
```
Expected: FAIL
**Step 3: Write minimal implementation**
实现:
- `BatchImportService.StartRun`
- `ImportRoutingStrategy`
- `BuildImportRoutingStrategy`
- `ChannelPatchContract`
- `ModelMappingDelta`
先接现有 `provision.ImportService`,不要提前扩展 UI/API。
**Step 4: Run test to verify it passes**
Run:
```bash
go test ./internal/batch -run 'TestBatchImport_StartRun|TestModelMappingDelta' -count=1
```
Expected: PASS
**Step 5: Commit**
```bash
git add internal/batch/service.go internal/batch/capability_profile.go internal/batch/channel_evolution.go internal/batch/service_test.go internal/batch/channel_evolution_test.go
git commit -m "feat(batch): implement v2 run setup and provision stages"
```
### Task 7: ConfirmationWorker, Lease And Retry
**Files:**
- Create: `internal/batch/confirmation.go`
- Test: `internal/batch/confirmation_test.go`
- Reference: `docs/2026-05-22-BATCH_AUTO_IMPORT_V2_ARCHITECTURE.md:398`
**Step 1: Write the failing test**
覆盖:
- 只捞 `confirm + pending + retry_due + lease_expired`
- `403` probe race -> advisory
- 初次 `503 no available accounts` -> retry -> success
- 多 worker lease 互斥
- `disabled/deprecated` 命中后 reactivated 投影正确
**Step 2: Run test to verify it fails**
Run:
```bash
go test ./internal/batch -run 'TestConfirmationWorker' -count=1
```
Expected: FAIL
**Step 3: Write minimal implementation**
实现:
- `ConfirmationWorker.Tick`
- `ConfirmationWorker.ConfirmItem`
- retry 计划
- lease 生命周期
- advisory event 写入
**Step 4: Run test to verify it passes**
Run:
```bash
go test ./internal/batch -run 'TestConfirmationWorker' -count=1
```
Expected: PASS
**Step 5: Commit**
```bash
git add internal/batch/confirmation.go internal/batch/confirmation_test.go
git commit -m "feat(batch): add confirmation worker and retry handling"
```
### Task 8: ValidationService And Final Access Status
**Files:**
- Create: `internal/batch/validation.go`
- Test: `internal/batch/validation_test.go`
- Reference: `internal/access/closure.go`
**Step 1: Write the failing test**
覆盖:
- `confirmed/advisory + chat 200 -> active`
- exhausted transient -> `degraded`
- definitive invalid path -> `broken`
- 只有 ValidationService 可以写 `access_status`
**Step 2: Run test to verify it fails**
Run:
```bash
go test ./internal/batch -run 'TestValidationService' -count=1
```
Expected: FAIL
**Step 3: Write minimal implementation**
实现:
- `ValidationService.ValidateItem`
- `access_status` 映射
- 对 run summary 的最小更新
**Step 4: Run test to verify it passes**
Run:
```bash
go test ./internal/batch -run 'TestValidationService' -count=1
```
Expected: PASS
**Step 5: Commit**
```bash
git add internal/batch/validation.go internal/batch/validation_test.go
git commit -m "feat(batch): add validation service for final access status"
```
### Task 9: ResultProjection
**Files:**
- Create: `internal/batch/status_projection.go`
- Test: `internal/batch/status_projection_test.go`
- Reference: `docs/2026-05-22-BATCH_AUTO_IMPORT_V2_API_SCHEMAS.md`
**Step 1: Write the failing test**
覆盖:
- run summary 聚合
- item summary/detail projection
- warning 文案模板
- `provision_reused` badge
- `matched_account_state / account_resolution` 文案与 badge
**Step 2: Run test to verify it fails**
Run:
```bash
go test ./internal/batch -run 'TestStatusProjection' -count=1
```
Expected: FAIL
**Step 3: Write minimal implementation**
实现:
- run list projection
- item list projection
- item detail projection
- warning/badge mapping
**Step 4: Run test to verify it passes**
Run:
```bash
go test ./internal/batch -run 'TestStatusProjection' -count=1
```
Expected: PASS
**Step 5: Commit**
```bash
git add internal/batch/status_projection.go internal/batch/status_projection_test.go
git commit -m "feat(batch): add result projection for v2 runs and items"
```
### Task 10: HTTP API For Runs And Items
**Files:**
- Create: `internal/app/http_batch_import.go`
- Create: `internal/app/http_batch_runs.go`
- Modify: `internal/app/http_api.go`
- Test: `internal/app/http_batch_import_test.go`
- Test: `internal/app/http_batch_runs_test.go`
- Reference: `docs/openapi.yaml`
**Step 1: Write the failing test**
覆盖:
- `POST /api/batch-import/runs`
- `GET /api/batch-import/runs`
- `GET /api/batch-import/runs/{run_id}`
- `GET /api/batch-import/runs/{run_id}/items`
- `GET /api/batch-import/runs/{run_id}/items/{item_id}`
- `subscription/self_service` 条件必填
- 列表过滤 `matched_account_state / account_resolution`
**Step 2: Run test to verify it fails**
Run:
```bash
go test ./internal/app -run 'TestBatchImportHTTP|TestBatchRunsHTTP' -count=1
```
Expected: FAIL
**Step 3: Write minimal implementation**
按 OpenAPI 只输出 projection不泄漏 legacy 表结构。
**Step 4: Run test to verify it passes**
Run:
```bash
go test ./internal/app -run 'TestBatchImportHTTP|TestBatchRunsHTTP' -count=1
```
Expected: PASS
**Step 5: Commit**
```bash
git add internal/app/http_batch_import.go internal/app/http_batch_runs.go internal/app/http_api.go internal/app/http_batch_import_test.go internal/app/http_batch_runs_test.go
git commit -m "feat(api): add batch import v2 endpoints"
```
### Task 11: CLI Entry For Batch Import
**Files:**
- Modify: `cmd/cli/main.go`
- Create: `cmd/cli/batch_import.go`
- Test: `cmd/cli/batch_import_test.go`
**Step 1: Write the failing test**
覆盖:
- 参数解析
- `subscription` 必填订阅参数
- `self_service` 必填 `probe_api_key`
- `--confirm-timeout`
- 结果输出 `run_id/result_page`
**Step 2: Run test to verify it fails**
Run:
```bash
go test ./cmd/cli -run 'TestBatchImportCLI' -count=1
```
Expected: FAIL
**Step 3: Write minimal implementation**
实现 CLI 到 V2 API/service 的入口,不在 CLI 层重复业务逻辑。
**Step 4: Run test to verify it passes**
Run:
```bash
go test ./cmd/cli -run 'TestBatchImportCLI' -count=1
```
Expected: PASS
**Step 5: Commit**
```bash
git add cmd/cli/main.go cmd/cli/batch_import.go cmd/cli/batch_import_test.go
git commit -m "feat(cli): add v2 batch import command"
```
### Task 12: Integration And End-To-End Verification
**Files:**
- Create: `tests/integration/batch_import_v2_test.go`
- Modify: `tests/integration/host_stub_test.go`(如需 stub 扩展)
**Step 1: Write the failing test**
至少覆盖 6 条真实业务链:
- 发现模型并归一化
- 重复导入 active 账号 -> reused
- deprecated 账号 -> reactivated
- 同 family 不同 alias -> patch_only
- probe race + warmup retry -> advisory + active
- run/item/event 详情可从 V2 新表完全读出
**Step 2: Run test to verify it fails**
Run:
```bash
go test ./tests/integration/... -run 'TestBatchImportV2' -count=1
```
Expected: FAIL
**Step 3: Write minimal implementation**
补齐 host stub、fake adapter、seed data确保每条链路都可复现。
**Step 4: Run test to verify it passes**
Run:
```bash
go test ./tests/integration/... -run 'TestBatchImportV2' -count=1
```
Expected: PASS
**Step 5: Commit**
```bash
git add tests/integration/batch_import_v2_test.go tests/integration/host_stub_test.go
git commit -m "test(integration): cover batch import v2 flows"
```
### Task 13: Design Restoration Audit
**Files:**
- Create: `docs/2026-05-22-BATCH_AUTO_IMPORT_V2_RESTORATION_CHECKLIST.md`
- Modify: `docs/EXECUTION_BOARD.md`
**Step 1: Write the failing audit checklist**
列出必须逐项勾选的设计恢复项:
- 8 项 Objective
- canonical contract
- 结果 API
- migration
- worker/retry/lease
- reuse/reactivation
**Step 2: Run verification to identify gaps**
Run:
```bash
go test ./... -count=1
go test ./tests/integration/... -count=1
go test -cover ./internal/... -count=1
go vet ./...
gofmt -l .
```
Expected: 在实现完成前,这一步用来发现剩余设计缺口;在最终完成时必须全绿。
**Step 3: Write the audit artifact**
将每一项设计要求映射到:
- 代码文件
- 测试文件
- API 路由
- 状态字段
**Step 4: Update board with true gate**
在执行板中明确:
- 哪些任务完成
- 哪些设计要求已还原
- 是否可宣称“V2 设计已被完整实现”
**Step 5: Commit**
```bash
git add docs/2026-05-22-BATCH_AUTO_IMPORT_V2_RESTORATION_CHECKLIST.md docs/EXECUTION_BOARD.md
git commit -m "docs(v2): add restoration checklist and completion gate"
```
## 4. 全局验证门禁
完成 T1~T13 后,必须一次性通过:
```bash
gofmt -l .
go vet ./...
go test ./... -count=1
go test ./tests/integration/... -count=1
go test -cover ./internal/... -count=1
```
额外检查:
- `docs/openapi.yaml` 与 handler 响应字段一致
- `import_runs/*` 足以支撑结果页,不依赖 legacy 表拼接
- `matched_account_state / account_resolution / provision_reused` 能在 item detail 里直接读到
- `canonical_model_family` 能把同模型别名判定为同一族
## 5. 计划完整性结论
这份计划只有在满足以下条件时,才算“任务可以完全还原规划设计”:
1. T1~T12 实现完成并全部通过验证
2. T13 的还原清单中不存在未映射设计项
3. 任一 Objective 都能指向至少一条:
- 实现任务
- 自动化测试
- API 或状态字段证据
4. 结果页/API 不需要额外新增未规划字段才能解释最终状态
如果 T13 审核时发现任何一项设计要求无法映射到任务或测试,这份计划必须回退修改,不能直接进入实现。
## 6. 推荐提交顺序
建议按以下小步提交:
1. `feat(batch): add canonical v2 state enums`
2. `feat(probe): add model discovery and canonical family normalization`
3. `feat(probe): add capability profile and smoke completion routing`
4. `feat(batch): add provider id and reuse policy`
5. `feat(store): complete v2 runtime state repositories`
6. `feat(batch): implement v2 run setup and provision stages`
7. `feat(batch): add confirmation worker and retry handling`
8. `feat(batch): add validation service for final access status`
9. `feat(batch): add result projection for v2 runs and items`
10. `feat(api): add batch import v2 endpoints`
11. `feat(cli): add v2 batch import command`
12. `test(integration): cover batch import v2 flows`
13. `docs(v2): add restoration checklist and completion gate`
Plan complete and saved to `docs/plans/2026-05-22-batch-auto-import-v2-implementation-plan.md`. Two execution options:
**1. Subagent-Driven (this session)** - I dispatch fresh subagent per task, review between tasks, fast iteration
**2. Parallel Session (separate)** - Open new session with executing-plans, batch execution with checkpoints
Which approach?