feat(report): improve daily intelligence UX and price tracking
Some checks failed
CI / go-test (push) Has been cancelled
CI / scripts-regression (push) Has been cancelled
CI / frontend-build (push) Has been cancelled
CI / docker-build (push) Has been cancelled

This commit is contained in:
phamnazage-jpg
2026-05-27 17:23:08 +08:00
parent f274621013
commit f5b373caf4
29 changed files with 4257 additions and 801 deletions

25
.dockerignore Normal file
View File

@@ -0,0 +1,25 @@
.git
.gitignore
.env
.env.*
!.env.example
.serena/
.openclaw/
memory/
reports/
logs/
frontend/node_modules/
frontend/dist/
node_modules/
dist/
*.log
*.tmp
*.bak
*.bak-*
/tmp/
fetch_openrouter
fetch_openrouter_test
fetch_multi_source
import_phase2_data
generate_daily_report
go-build*/

View File

@@ -17,8 +17,12 @@ jobs:
go-version: "1.22"
cache: true
- name: Run Go tests
- name: Run package-level Go tests (cmd/server + internal/...)
run: go test ./...
- name: Note script test coverage boundary
run: |
echo "go test ./... only covers package-based Go code"
echo "script-level coverage runs in the scripts-regression job"
scripts-regression:
runs-on: ubuntu-latest
@@ -30,7 +34,7 @@ jobs:
go-version: "1.22"
cache: true
- name: Run targeted importer tests
- name: Run targeted script importer tests
run: bash scripts/test_importers.sh
- name: Run importer smoke gate

View File

@@ -48,9 +48,12 @@ psql llm_intelligence < db/migrations/001_phase1_core_tables.sql
```bash
export DATABASE_URL="host=/var/run/postgresql dbname=llm_intelligence sslmode=disable"
export OPENROUTER_API_KEY="your-api-key"
export API_AUTH_TOKEN="replace-with-long-random-token"
# 或者export API_BASIC_AUTH_USER="review" && export API_BASIC_AUTH_PASS="replace-with-password"
export FEISHU_WEBHOOK="your-webhook-url" # 可选
```
### 4. 启动后端
```bash
go run cmd/server/main.go
@@ -69,10 +72,14 @@ crontab -e
# 正式日报调度
0 8 * * * cd /path/to/llm-intelligence && bash scripts/run_daily.sh >> /tmp/llm_hub_cron.log 2>&1
# 日内价格追踪(推荐每 4 小时一次)
0 */4 * * * cd /path/to/llm-intelligence && bash scripts/run_intraday_price_watch.sh >> /tmp/llm_hub_intraday.log 2>&1
# 真实采集 + 写库 + 报告生成的手动复跑入口
cd /path/to/llm-intelligence && bash scripts/run_real_pipeline.sh
```
---
## Docker 部署
@@ -93,17 +100,26 @@ docker-compose up -d
|------|------|------|
| DATABASE_URL | ✅ | PostgreSQL 连接串 |
| OPENROUTER_API_KEY | ✅ | OpenRouter API Key |
| API_AUTH_TOKEN | 条件必填 | 对外访问 `/api/*` 的 Bearer token |
| API_BASIC_AUTH_USER / API_BASIC_AUTH_PASS | 条件必填 | 对外访问 `/api/*` 的 Basic Auth 凭证 |
| API_RATE_LIMIT_PER_WINDOW | ❌ | `/api/*` 每窗口允许的请求数,默认 `60` |
| API_RATE_LIMIT_WINDOW_SEC | ❌ | `/api/*` 限流窗口秒数,默认 `60` |
| FEISHU_WEBHOOK | ❌ | 飞书告警 Webhook |
| REPORT_DATE | ❌ | 手工指定日内追踪/日报日期 |
| PORT | ❌ | API Server 监听端口,默认 8080 |
---
## 验证安装
```bash
# 数据库连接
# 健康检查(仅本机 / 私网)
curl http://localhost:8080/health
# API 鉴权
curl -H "Authorization: Bearer $API_AUTH_TOKEN" http://localhost:8080/api/v1/models
# 采集器测试
go run scripts/fetch_openrouter.go -strict-real

View File

@@ -23,6 +23,8 @@
- 最新正式日报由 `scripts/run_daily.sh` 生成,并写入 `daily_report` / `report_runs`
- 手工复跑使用 `scripts/run_real_pipeline.sh`,不会把产物标记成正式日报
- 历史补跑使用 `scripts/rebuild_historical_report.sh YYYY-MM-DD`
- 日内价格追踪使用 `scripts/run_intraday_price_watch.sh`,只刷新价格与信号,不生成正式日报
- HTTP API 当前未内建认证、授权和限流;公网暴露前必须在网关层补齐
## 先读这些(当前真相入口)
@@ -99,6 +101,10 @@ bash scripts/run_intel_pipeline.sh
4. 每日关键信号物化到 `daily_signal_snapshot`
它不会生成日报,适合先把“数据与信号层”单独跑通。
3. 平台目录核验
4. 每日关键信号物化到 `daily_signal_snapshot`
5. 日内价格追踪可由 `scripts/run_intraday_price_watch.sh` 独立执行,不生成正式日报
### 正式日报调度
@@ -119,6 +125,13 @@ bash scripts/run_daily.sh
9. 失败时降级复制昨日报告并可选飞书告警
### 手工真实复跑
### 日内价格追踪
```bash
bash scripts/run_intraday_price_watch.sh
```
适用于捕捉“小米大降价”“活动窗口上线”“泄露情报”等日内价格事件。该入口只刷新价格与信号层,不写正式 `daily_report`,也不会覆盖 `latest_report` 语义。
```bash
bash scripts/run_real_pipeline.sh
@@ -147,6 +160,7 @@ bash scripts/rebuild_historical_report.sh 2026-05-13
```bash
go test ./...
bash scripts/test.sh
bash scripts/test_importers.sh
bash scripts/verify_pre_phase6.sh
bash scripts/verify_phase6.sh
bash healthcheck.sh
@@ -154,6 +168,13 @@ cd frontend && npm run test -- --run
cd frontend && npm run build
```
说明:
- `go test ./...` 只覆盖 package 形式的 Go 代码(当前主要是 `cmd/server``internal/...`
- `bash scripts/test.sh` 只覆盖 `fetch_openrouter` 的 focused test
- `bash scripts/test_importers.sh` 覆盖 scripts 层 importer targeted go test matrix
- 发布前不要把 `go test ./...` 误判成“全仓脚本业务已验证”
## API 概览
- `GET /health`

View File

@@ -6,10 +6,13 @@ import (
"encoding/json"
"fmt"
"log"
"net"
"net/http"
"os"
"path/filepath"
"strconv"
"strings"
"sync"
"time"
_ "github.com/lib/pq"
@@ -55,7 +58,13 @@ type subscriptionPlanResponse struct {
}
type apiEnvelope struct {
Data any `json:"data"`
Data any `json:"data,omitempty"`
Error *apiError `json:"error,omitempty"`
}
type apiError struct {
Code string `json:"code"`
Message string `json:"message"`
}
type modelFetcher func(context.Context, *sql.DB) ([]modelResponse, error)
@@ -74,6 +83,173 @@ type latestReportResponse struct {
MarkdownURL string `json:"markdownUrl"`
HTMLURL string `json:"htmlUrl"`
UpdatedAt string `json:"updatedAt"`
AppendixJSONURL string `json:"appendixJsonUrl"`
}
type serverConfig struct {
BasicAuthUser string
BasicAuthPass string
ServiceToken string
RateLimitPerWindow int
RateLimitWindow time.Duration
now func() time.Time
limiter *ipRateLimiter
}
type ipRateLimiter struct {
mu sync.Mutex
limit int
window time.Duration
entries map[string]rateLimitEntry
}
type rateLimitEntry struct {
windowStart time.Time
count int
}
func newIPRateLimiter(limit int, window time.Duration) *ipRateLimiter {
if limit <= 0 || window <= 0 {
return nil
}
return &ipRateLimiter{
limit: limit,
window: window,
entries: make(map[string]rateLimitEntry),
}
}
func (l *ipRateLimiter) Allow(key string, now time.Time) bool {
if l == nil {
return true
}
if key == "" {
key = "unknown"
}
l.mu.Lock()
defer l.mu.Unlock()
entry := l.entries[key]
if entry.windowStart.IsZero() || now.Sub(entry.windowStart) >= l.window {
entry = rateLimitEntry{windowStart: now}
}
if entry.count >= l.limit {
return false
}
entry.count++
l.entries[key] = entry
for candidate, candidateEntry := range l.entries {
if now.Sub(candidateEntry.windowStart) >= l.window {
delete(l.entries, candidate)
}
}
return true
}
func loadServerConfigFromEnv() serverConfig {
limit := 60
if raw := strings.TrimSpace(os.Getenv("API_RATE_LIMIT_PER_WINDOW")); raw != "" {
if parsed, err := strconv.Atoi(raw); err == nil && parsed >= 0 {
limit = parsed
}
}
window := time.Minute
if raw := strings.TrimSpace(os.Getenv("API_RATE_LIMIT_WINDOW_SEC")); raw != "" {
if parsed, err := strconv.Atoi(raw); err == nil && parsed > 0 {
window = time.Duration(parsed) * time.Second
}
}
return serverConfig{
BasicAuthUser: os.Getenv("API_BASIC_AUTH_USER"),
BasicAuthPass: os.Getenv("API_BASIC_AUTH_PASS"),
ServiceToken: os.Getenv("API_AUTH_TOKEN"),
RateLimitPerWindow: limit,
RateLimitWindow: window,
}
}
func (cfg serverConfig) withRuntimeDefaults() serverConfig {
if cfg.now == nil {
cfg.now = time.Now
}
if cfg.limiter == nil {
cfg.limiter = newIPRateLimiter(cfg.RateLimitPerWindow, cfg.RateLimitWindow)
}
return cfg
}
func (cfg serverConfig) wrap(path string, next http.HandlerFunc) http.HandlerFunc {
cfg = cfg.withRuntimeDefaults()
return func(w http.ResponseWriter, r *http.Request) {
clientIP := requestClientIP(r)
trustedClient := isTrustedClientIP(clientIP)
if path == "/health" && !trustedClient {
writeError(w, http.StatusForbidden, "health_endpoint_internal_only", "health endpoint is restricted to trusted networks")
return
}
if path != "/health" && !trustedClient {
if !cfg.isAuthorized(r) {
w.Header().Set("WWW-Authenticate", `Basic realm="llm-intelligence"`)
writeError(w, http.StatusUnauthorized, "auth_required", "authentication required for external API access")
return
}
}
if path != "/health" && cfg.limiter != nil {
if !cfg.limiter.Allow(clientIP, cfg.now()) {
writeError(w, http.StatusTooManyRequests, "rate_limited", "rate limit exceeded")
return
}
}
next(w, r)
}
}
func (cfg serverConfig) isAuthorized(r *http.Request) bool {
authHeader := strings.TrimSpace(r.Header.Get("Authorization"))
if cfg.ServiceToken != "" {
const bearerPrefix = "Bearer "
if strings.HasPrefix(authHeader, bearerPrefix) {
return strings.TrimSpace(strings.TrimPrefix(authHeader, bearerPrefix)) == cfg.ServiceToken
}
}
if cfg.BasicAuthUser == "" && cfg.BasicAuthPass == "" {
return false
}
username, password, ok := r.BasicAuth()
return ok && username == cfg.BasicAuthUser && password == cfg.BasicAuthPass
}
func requestClientIP(r *http.Request) string {
if forwardedFor := strings.TrimSpace(r.Header.Get("X-Forwarded-For")); forwardedFor != "" {
parts := strings.Split(forwardedFor, ",")
if len(parts) > 0 {
return strings.TrimSpace(parts[0])
}
}
host, _, err := net.SplitHostPort(strings.TrimSpace(r.RemoteAddr))
if err == nil {
return host
}
return strings.TrimSpace(r.RemoteAddr)
}
func isTrustedClientIP(raw string) bool {
ip := net.ParseIP(strings.TrimSpace(raw))
if ip == nil {
return false
}
return ip.IsLoopback() || ip.IsPrivate()
}
func main() {
@@ -96,7 +272,7 @@ func main() {
}
}
mux := newMux(db, fetchModels, fetchSubscriptionPlans, fetchLatestReport)
mux := newMuxWithConfig(db, fetchModels, fetchSubscriptionPlans, fetchLatestReport, loadServerConfigFromEnv())
log.Printf("server listening on :%s", addr)
if err := http.ListenAndServe(":"+addr, mux); err != nil {
@@ -106,72 +282,83 @@ func main() {
func newMux(db *sql.DB, fetchModelsFn modelFetcher, fetchPlansFn subscriptionPlanFetcher, fetchLatestReportFn latestReportFetcher) *http.ServeMux {
mux := http.NewServeMux()
mux.HandleFunc("/health", func(w http.ResponseWriter, r *http.Request) {
registerRoutes(mux, db, fetchModelsFn, fetchPlansFn, fetchLatestReportFn, func(_ string, handler http.HandlerFunc) http.HandlerFunc {
return handler
})
return mux
}
func newMuxWithConfig(db *sql.DB, fetchModelsFn modelFetcher, fetchPlansFn subscriptionPlanFetcher, fetchLatestReportFn latestReportFetcher, cfg serverConfig) *http.ServeMux {
mux := http.NewServeMux()
registerRoutes(mux, db, fetchModelsFn, fetchPlansFn, fetchLatestReportFn, cfg.wrap)
return mux
}
func registerRoutes(mux *http.ServeMux, db *sql.DB, fetchModelsFn modelFetcher, fetchPlansFn subscriptionPlanFetcher, fetchLatestReportFn latestReportFetcher, wrap func(string, http.HandlerFunc) http.HandlerFunc) {
mux.HandleFunc("/health", wrap("/health", func(w http.ResponseWriter, r *http.Request) {
if db == nil {
http.Error(w, "database not configured", http.StatusServiceUnavailable)
writeError(w, http.StatusServiceUnavailable, "database_not_configured", "database not configured")
return
}
if err := db.PingContext(r.Context()); err != nil {
http.Error(w, "database unavailable", http.StatusServiceUnavailable)
writeError(w, http.StatusServiceUnavailable, "database_unavailable", "database unavailable")
return
}
writeJSON(w, http.StatusOK, map[string]string{"status": "ok"})
})
mux.HandleFunc("/api/v1/models", func(w http.ResponseWriter, r *http.Request) {
}))
mux.HandleFunc("/api/v1/models", wrap("/api/v1/models", func(w http.ResponseWriter, r *http.Request) {
if db == nil {
http.Error(w, "database not configured", http.StatusServiceUnavailable)
writeError(w, http.StatusServiceUnavailable, "database_not_configured", "database not configured")
return
}
models, err := fetchModelsFn(r.Context(), db)
if err != nil {
http.Error(w, "query failed", http.StatusInternalServerError)
writeError(w, http.StatusInternalServerError, "query_failed", "query failed")
log.Printf("fetch models failed: %v", err)
return
}
writeJSON(w, http.StatusOK, apiEnvelope{Data: models})
})
mux.HandleFunc("/api/v1/subscription-plans", func(w http.ResponseWriter, r *http.Request) {
}))
mux.HandleFunc("/api/v1/subscription-plans", wrap("/api/v1/subscription-plans", func(w http.ResponseWriter, r *http.Request) {
if db == nil {
http.Error(w, "database not configured", http.StatusServiceUnavailable)
writeError(w, http.StatusServiceUnavailable, "database_not_configured", "database not configured")
return
}
plans, err := fetchPlansFn(r.Context(), db)
if err != nil {
http.Error(w, "query failed", http.StatusInternalServerError)
writeError(w, http.StatusInternalServerError, "query_failed", "query failed")
log.Printf("fetch subscription plans failed: %v", err)
return
}
writeJSON(w, http.StatusOK, apiEnvelope{Data: plans})
})
mux.HandleFunc("/api/v1/reports/latest/html", func(w http.ResponseWriter, r *http.Request) {
}))
mux.HandleFunc("/api/v1/reports/latest/html", wrap("/api/v1/reports/latest/html", func(w http.ResponseWriter, r *http.Request) {
serveLatestReportArtifact(w, r, db, fetchLatestReportFn, "html")
})
mux.HandleFunc("/api/v1/reports/latest/markdown", func(w http.ResponseWriter, r *http.Request) {
}))
mux.HandleFunc("/api/v1/reports/latest/markdown", wrap("/api/v1/reports/latest/markdown", func(w http.ResponseWriter, r *http.Request) {
serveLatestReportArtifact(w, r, db, fetchLatestReportFn, "markdown")
})
mux.HandleFunc("/api/v1/reports/latest", func(w http.ResponseWriter, r *http.Request) {
}))
mux.HandleFunc("/api/v1/reports/latest", wrap("/api/v1/reports/latest", func(w http.ResponseWriter, r *http.Request) {
if db == nil {
http.Error(w, "database not configured", http.StatusServiceUnavailable)
writeError(w, http.StatusServiceUnavailable, "database_not_configured", "database not configured")
return
}
report, err := fetchLatestReportFn(r.Context(), db)
if err != nil {
if err == sql.ErrNoRows {
http.Error(w, "latest report not found", http.StatusNotFound)
writeError(w, http.StatusNotFound, "latest_report_not_found", "latest report not found")
return
}
http.Error(w, "query failed", http.StatusInternalServerError)
writeError(w, http.StatusInternalServerError, "query_failed", "query failed")
log.Printf("fetch latest report failed: %v", err)
return
}
writeJSON(w, http.StatusOK, apiEnvelope{Data: report})
})
return mux
}))
}
func fetchModels(ctx context.Context, db *sql.DB) ([]modelResponse, error) {
rows, err := db.QueryContext(ctx, `
WITH latest_prices AS (
const fetchModelsQuery = `
WITH ranked_prices AS (
SELECT
rp.model_id,
rp.pricing_mode,
@@ -183,7 +370,16 @@ func fetchModels(ctx context.Context, db *sql.DB) ([]modelResponse, error) {
rp.is_free,
ROW_NUMBER() OVER (
PARTITION BY rp.model_id
ORDER BY rp.effective_date DESC NULLS LAST, rp.id DESC
ORDER BY
CASE WHEN lower(rp.region) = 'global' THEN 0 ELSE 1 END,
CASE rp.source_type
WHEN 'official' THEN 0
WHEN 'reseller' THEN 1
WHEN 'free_tier' THEN 2
ELSE 3
END,
rp.effective_date DESC NULLS LAST,
rp.id DESC
) AS rn
FROM region_pricing rp
)
@@ -204,10 +400,13 @@ func fetchModels(ctx context.Context, db *sql.DB) ([]modelResponse, error) {
COALESCE(m.data_confidence, 'official')
FROM models m
LEFT JOIN model_provider mp ON mp.id = m.provider_id
LEFT JOIN latest_prices lp ON lp.model_id = m.id AND lp.rn = 1
LEFT JOIN ranked_prices lp ON lp.model_id = m.id AND lp.rn = 1
WHERE m.deleted_at IS NULL
ORDER BY m.id DESC
`)
`
func fetchModels(ctx context.Context, db *sql.DB) ([]modelResponse, error) {
rows, err := db.QueryContext(ctx, fetchModelsQuery)
if err != nil {
return nil, err
}
@@ -291,22 +490,23 @@ func fetchLatestReport(ctx context.Context, db *sql.DB) (*latestReportResponse,
report.ArchiveHTMLPath = deriveReportArchivePath(report.HTMLPath, report.ReportDate)
report.MarkdownURL = "/api/v1/reports/latest/markdown"
report.HTMLURL = "/api/v1/reports/latest/html"
report.AppendixJSONURL = "/reports/daily/appendix/" + report.ReportDate + "/full_appendix.json"
return &report, nil
}
func serveLatestReportArtifact(w http.ResponseWriter, r *http.Request, db *sql.DB, fetchLatestReportFn latestReportFetcher, artifactType string) {
if db == nil {
http.Error(w, "database not configured", http.StatusServiceUnavailable)
writeError(w, http.StatusServiceUnavailable, "database_not_configured", "database not configured")
return
}
report, err := fetchLatestReportFn(r.Context(), db)
if err != nil {
if err == sql.ErrNoRows {
http.Error(w, "latest report not found", http.StatusNotFound)
writeError(w, http.StatusNotFound, "latest_report_not_found", "latest report not found")
return
}
http.Error(w, "query failed", http.StatusInternalServerError)
writeError(w, http.StatusInternalServerError, "query_failed", "query failed")
log.Printf("fetch latest report failed: %v", err)
return
}
@@ -320,7 +520,7 @@ func serveLatestReportArtifact(w http.ResponseWriter, r *http.Request, db *sql.D
}
if _, err := os.Stat(targetPath); err != nil {
http.Error(w, "report artifact not found", http.StatusNotFound)
writeError(w, http.StatusNotFound, "report_artifact_not_found", "report artifact not found")
return
}
@@ -417,6 +617,10 @@ func writeJSON(w http.ResponseWriter, status int, value any) {
w.Header().Set("Content-Type", "application/json")
w.WriteHeader(status)
if err := json.NewEncoder(w).Encode(value); err != nil {
http.Error(w, "encode failed", http.StatusInternalServerError)
log.Printf("encode response failed: %v", err)
}
}
func writeError(w http.ResponseWriter, status int, code, message string) {
writeJSON(w, status, apiEnvelope{Error: &apiError{Code: code, Message: message}})
}

View File

@@ -7,7 +7,9 @@ import (
"net/http"
"net/http/httptest"
"os"
"strings"
"testing"
"time"
)
func TestModelsHandlerReturnsFlatPricingFields(t *testing.T) {
@@ -59,6 +61,131 @@ func TestModelsHandlerReturnsFlatPricingFields(t *testing.T) {
}
}
func TestModelsHandlerReturnsJSONErrorEnvelope(t *testing.T) {
mux := newMux(
nil,
func(context.Context, *sql.DB) ([]modelResponse, error) {
return nil, nil
},
func(context.Context, *sql.DB) ([]subscriptionPlanResponse, error) {
return nil, nil
},
func(context.Context, *sql.DB) (*latestReportResponse, error) {
return nil, sql.ErrNoRows
},
)
req := httptest.NewRequest(http.MethodGet, "/api/v1/models", nil)
rec := httptest.NewRecorder()
mux.ServeHTTP(rec, req)
if rec.Code != http.StatusServiceUnavailable {
t.Fatalf("expected status 503, got %d", rec.Code)
}
var payload struct {
Error struct {
Code string `json:"code"`
Message string `json:"message"`
} `json:"error"`
}
if err := json.Unmarshal(rec.Body.Bytes(), &payload); err != nil {
t.Fatalf("unmarshal error response: %v", err)
}
if payload.Error.Code != "database_not_configured" {
t.Fatalf("unexpected error code: %q", payload.Error.Code)
}
}
func TestHealthHandlerReturnsJSONErrorEnvelope(t *testing.T) {
mux := newMux(
nil,
func(context.Context, *sql.DB) ([]modelResponse, error) {
return nil, nil
},
func(context.Context, *sql.DB) ([]subscriptionPlanResponse, error) {
return nil, nil
},
func(context.Context, *sql.DB) (*latestReportResponse, error) {
return nil, sql.ErrNoRows
},
)
req := httptest.NewRequest(http.MethodGet, "/health", nil)
rec := httptest.NewRecorder()
mux.ServeHTTP(rec, req)
if rec.Code != http.StatusServiceUnavailable {
t.Fatalf("expected status 503, got %d", rec.Code)
}
var payload struct {
Error struct {
Code string `json:"code"`
Message string `json:"message"`
} `json:"error"`
}
if err := json.Unmarshal(rec.Body.Bytes(), &payload); err != nil {
t.Fatalf("unmarshal health error response: %v", err)
}
if payload.Error.Code != "database_not_configured" {
t.Fatalf("unexpected error code: %q", payload.Error.Code)
}
}
func TestLatestReportHTMLHandlerReturnsJSONErrorEnvelope(t *testing.T) {
mux := newMux(
&sql.DB{},
func(context.Context, *sql.DB) ([]modelResponse, error) {
return nil, nil
},
func(context.Context, *sql.DB) ([]subscriptionPlanResponse, error) {
return nil, nil
},
func(context.Context, *sql.DB) (*latestReportResponse, error) {
return nil, sql.ErrNoRows
},
)
req := httptest.NewRequest(http.MethodGet, "/api/v1/reports/latest/html", nil)
rec := httptest.NewRecorder()
mux.ServeHTTP(rec, req)
if rec.Code != http.StatusNotFound {
t.Fatalf("expected status 404, got %d", rec.Code)
}
var payload struct {
Error struct {
Code string `json:"code"`
Message string `json:"message"`
} `json:"error"`
}
if err := json.Unmarshal(rec.Body.Bytes(), &payload); err != nil {
t.Fatalf("unmarshal latest html error response: %v", err)
}
if payload.Error.Code != "latest_report_not_found" {
t.Fatalf("unexpected error code: %q", payload.Error.Code)
}
}
func TestFetchModelsQueryEncodesPrimaryPricePriority(t *testing.T) {
fragments := []string{
"CASE WHEN lower(rp.region) = 'global' THEN 0 ELSE 1 END",
"WHEN 'official' THEN 0",
"WHEN 'reseller' THEN 1",
"WHEN 'free_tier' THEN 2",
"rp.effective_date DESC NULLS LAST",
"rp.id DESC",
}
for _, fragment := range fragments {
if !strings.Contains(fetchModelsQuery, fragment) {
t.Fatalf("fetchModelsQuery missing fragment %q", fragment)
}
}
}
func TestSubscriptionPlansHandlerReturnsEnvelope(t *testing.T) {
mux := newMux(
&sql.DB{},
@@ -211,3 +338,137 @@ func TestLatestReportHTMLHandlerServesArtifact(t *testing.T) {
t.Fatalf("unexpected body: %q", body)
}
}
func TestModelsHandlerRejectsUnauthenticatedExternalRequests(t *testing.T) {
mux := newMuxWithConfig(
&sql.DB{},
func(context.Context, *sql.DB) ([]modelResponse, error) {
return nil, nil
},
func(context.Context, *sql.DB) ([]subscriptionPlanResponse, error) {
return nil, nil
},
func(context.Context, *sql.DB) (*latestReportResponse, error) {
return nil, sql.ErrNoRows
},
serverConfig{BasicAuthUser: "review", BasicAuthPass: "secret", RateLimitPerWindow: 10, RateLimitWindow: time.Minute},
)
req := httptest.NewRequest(http.MethodGet, "/api/v1/models", nil)
req.RemoteAddr = "198.51.100.8:1234"
rec := httptest.NewRecorder()
mux.ServeHTTP(rec, req)
if rec.Code != http.StatusUnauthorized {
t.Fatalf("expected status 401, got %d", rec.Code)
}
}
func TestModelsHandlerAllowsBasicAuthForExternalRequests(t *testing.T) {
mux := newMuxWithConfig(
&sql.DB{},
func(context.Context, *sql.DB) ([]modelResponse, error) {
return []modelResponse{{ID: "openai/gpt-4o", Name: "GPT-4o"}}, nil
},
func(context.Context, *sql.DB) ([]subscriptionPlanResponse, error) {
return nil, nil
},
func(context.Context, *sql.DB) (*latestReportResponse, error) {
return nil, sql.ErrNoRows
},
serverConfig{BasicAuthUser: "review", BasicAuthPass: "secret", RateLimitPerWindow: 10, RateLimitWindow: time.Minute},
)
req := httptest.NewRequest(http.MethodGet, "/api/v1/models", nil)
req.RemoteAddr = "198.51.100.8:1234"
req.SetBasicAuth("review", "secret")
rec := httptest.NewRecorder()
mux.ServeHTTP(rec, req)
if rec.Code != http.StatusOK {
t.Fatalf("expected status 200, got %d", rec.Code)
}
}
func TestModelsHandlerAllowsBearerTokenForExternalRequests(t *testing.T) {
mux := newMuxWithConfig(
&sql.DB{},
func(context.Context, *sql.DB) ([]modelResponse, error) {
return []modelResponse{{ID: "openai/gpt-4o", Name: "GPT-4o"}}, nil
},
func(context.Context, *sql.DB) ([]subscriptionPlanResponse, error) {
return nil, nil
},
func(context.Context, *sql.DB) (*latestReportResponse, error) {
return nil, sql.ErrNoRows
},
serverConfig{ServiceToken: "token-123", RateLimitPerWindow: 10, RateLimitWindow: time.Minute},
)
req := httptest.NewRequest(http.MethodGet, "/api/v1/models", nil)
req.RemoteAddr = "198.51.100.8:1234"
req.Header.Set("Authorization", "Bearer token-123")
rec := httptest.NewRecorder()
mux.ServeHTTP(rec, req)
if rec.Code != http.StatusOK {
t.Fatalf("expected status 200, got %d", rec.Code)
}
}
func TestHealthHandlerRejectsExternalRequests(t *testing.T) {
mux := newMuxWithConfig(
&sql.DB{},
func(context.Context, *sql.DB) ([]modelResponse, error) {
return nil, nil
},
func(context.Context, *sql.DB) ([]subscriptionPlanResponse, error) {
return nil, nil
},
func(context.Context, *sql.DB) (*latestReportResponse, error) {
return nil, sql.ErrNoRows
},
serverConfig{RateLimitPerWindow: 10, RateLimitWindow: time.Minute},
)
req := httptest.NewRequest(http.MethodGet, "/health", nil)
req.RemoteAddr = "198.51.100.8:1234"
rec := httptest.NewRecorder()
mux.ServeHTTP(rec, req)
if rec.Code != http.StatusForbidden {
t.Fatalf("expected status 403, got %d", rec.Code)
}
}
func TestModelsHandlerAppliesRateLimit(t *testing.T) {
mux := newMuxWithConfig(
&sql.DB{},
func(context.Context, *sql.DB) ([]modelResponse, error) {
return []modelResponse{{ID: "openai/gpt-4o", Name: "GPT-4o"}}, nil
},
func(context.Context, *sql.DB) ([]subscriptionPlanResponse, error) {
return nil, nil
},
func(context.Context, *sql.DB) (*latestReportResponse, error) {
return nil, sql.ErrNoRows
},
serverConfig{RateLimitPerWindow: 1, RateLimitWindow: time.Minute},
)
first := httptest.NewRequest(http.MethodGet, "/api/v1/models", nil)
first.RemoteAddr = "127.0.0.1:1234"
firstRec := httptest.NewRecorder()
mux.ServeHTTP(firstRec, first)
if firstRec.Code != http.StatusOK {
t.Fatalf("expected first request status 200, got %d", firstRec.Code)
}
second := httptest.NewRequest(http.MethodGet, "/api/v1/models", nil)
second.RemoteAddr = "127.0.0.1:1234"
secondRec := httptest.NewRecorder()
mux.ServeHTTP(secondRec, second)
if secondRec.Code != http.StatusTooManyRequests {
t.Fatalf("expected second request status 429, got %d", secondRec.Code)
}
}

View File

@@ -7,8 +7,10 @@
- 基础地址:`http://<host>:<port>`
- 默认端口:`8080`
- 返回格式:成功接口统一返回 `{ "data": ... }`
- 失败格式:当前直接返回纯文本错误信息,不是统一 JSON 错误结构
- 鉴权:当前仓库未内建认证、鉴权与限流;公网暴露前应由网关或反向代理补齐
- 失败格式:失败接口统一返回 `{ "error": { "code": "...", "message": "..." } }`
- 访问控制:`/health` 仅允许本机或私网访问;`/api/*` 对外访问默认要求 `Authorization: Bearer <token>` 或 Basic Auth详见下文
- 限流:`/api/*` 默认按来源 IP 做窗口限流;可通过 `API_RATE_LIMIT_PER_WINDOW``API_RATE_LIMIT_WINDOW_SEC` 调整
## `GET /health`
@@ -24,18 +26,30 @@
### 失败
- `503 database not configured`:未配置 `DATABASE_URL`
- `503 database unavailable`:数据库 Ping 失败
```json
{
"error": {
"code": "database_not_configured",
"message": "database not configured"
}
}
```
- `503 database_not_configured`:未配置 `DATABASE_URL`
- `503 database_unavailable`:数据库 Ping 失败
### 示例
```bash
curl -fsS http://127.0.0.1:8080/health
```
### 访问控制
- 仅允许本机或私网请求;外部地址返回 `403 health_endpoint_internal_only`
## `GET /api/v1/models`
返回模型列表,数据来源于 `models``model_provider``region_pricing` 当前最新价格快照
返回模型列表,数据来源于 `models``model_provider``region_pricing`当同一模型存在多条价格记录时API 按“`global` 区域优先、`official` > `reseller` > `free_tier`、再按 `effective_date`/`id` 倒序”的规则选取主价格
### 返回体
@@ -84,8 +98,10 @@ curl -fsS http://127.0.0.1:8080/health
### 失败
- `503 database not configured`
- `500 query failed`
- `503 database_not_configured`
- `500 query_failed`
- `401 auth_required`
- `429 rate_limited`
## `GET /api/v1/subscription-plans`
@@ -122,8 +138,10 @@ curl -fsS http://127.0.0.1:8080/health
### 失败
- `503 database not configured`
- `500 query failed`
- `503 database_not_configured`
- `500 query_failed`
- `401 auth_required`
- `429 rate_limited`
## `GET /api/v1/reports/latest`
@@ -155,9 +173,12 @@ curl -fsS http://127.0.0.1:8080/health
### 失败
- `503 database not configured`
- `404 latest report not found`
- `500 query failed`
- `503 database_not_configured`
- `404 latest_report_not_found`
- `500 query_failed`
- `401 auth_required`
- `429 rate_limited`
## `GET /api/v1/reports/latest/markdown`
@@ -170,8 +191,10 @@ curl -fsS http://127.0.0.1:8080/health
### 失败
- `404 latest report not found`:数据库中没有符合条件的正式日报
- `404 report artifact not found`:元数据存在,但落盘文件缺失
- `404 latest_report_not_found`:数据库中没有符合条件的正式日报
- `404 report_artifact_not_found`:元数据存在,但落盘文件缺失
- `401 auth_required`
- `429 rate_limited`
## `GET /api/v1/reports/latest/html`
@@ -184,22 +207,24 @@ curl -fsS http://127.0.0.1:8080/health
### 失败
- `404 latest report not found`
- `404 report artifact not found`
- `404 latest_report_not_found`
- `404 report_artifact_not_found`
- `401 auth_required`
- `429 rate_limited`
## 冒烟检查命令
```bash
curl -fsS http://127.0.0.1:8080/health
curl -fsS http://127.0.0.1:8080/api/v1/models | jq '.data | length'
curl -fsS http://127.0.0.1:8080/api/v1/subscription-plans | jq '.data | length'
curl -fsS http://127.0.0.1:8080/api/v1/reports/latest | jq '.data.reportDate'
curl -fsS http://127.0.0.1:8080/api/v1/reports/latest/html > /tmp/latest_report.html
curl -fsS -H "Authorization: Bearer $API_AUTH_TOKEN" http://127.0.0.1:8080/api/v1/models | jq '.data | length'
curl -fsS -H "Authorization: Bearer $API_AUTH_TOKEN" http://127.0.0.1:8080/api/v1/subscription-plans | jq '.data | length'
curl -fsS -H "Authorization: Bearer $API_AUTH_TOKEN" http://127.0.0.1:8080/api/v1/reports/latest | jq '.data.reportDate'
curl -fsS -H "Authorization: Bearer $API_AUTH_TOKEN" http://127.0.0.1:8080/api/v1/reports/latest/html > /tmp/latest_report.html
```
## 生产暴露建议
- Nginx / 网关上补齐访问控制、速率限制和超时配置
- `/health` 仅暴露给负载均衡器监控系统
-公网暴露时至少配置 `API_AUTH_TOKEN``API_BASIC_AUTH_USER` / `API_BASIC_AUTH_PASS`
- `/health` 仅暴露给负载均衡器监控系统或私网来源
- 如果前端与 API 同域部署,优先由 Nginx 转发 `/api/``/health`
-果需要公网访问,建议至少加一层 Basic Auth、OIDC 或内网入口限制
-需更强控制,继续在 Nginx / 网关上补齐 CIDR 白名单、OIDC、WAF 与更细粒度限流

View File

@@ -18,13 +18,18 @@
| 变量名 | 必填 | 使用方 | 默认值 | 说明 |
|--------|------|--------|--------|------|
| `DATABASE_URL` | 是 | API Server、迁移、采集、日报、备份恢复、验收脚本 | 无 | PostgreSQL 连接串,缺失时多数核心脚本会直接失败 |
| `OPENROUTER_API_KEY` | 条件必填 | `fetch_openrouter.go``run_real_pipeline.sh``run_daily.sh` | 无 | 真实采集所需;只查看历史数据或仅跑前端时可不配 |
| `OPENROUTER_API_KEY` | 条件必填 | `fetch_openrouter.go``run_real_pipeline.sh``run_daily.sh``run_intraday_price_watch.sh` | 无 | 真实采集所需;只查看历史数据或仅跑前端时可不配 |
| `PORT` | 否 | `cmd/server/main.go` | `8080` | API Server 监听端口 |
| `API_AUTH_TOKEN` | 条件必填 | `cmd/server/main.go`、API smoke / 外部调用 | 空 | 对外访问 `/api/*` 时推荐使用的 Bearer token外部请求未携带合法 token 或 Basic Auth 时返回 `401` |
| `API_BASIC_AUTH_USER` | 条件必填 | `cmd/server/main.go` | 空 | 对外访问 `/api/*` 的 Basic Auth 用户名;与 `API_BASIC_AUTH_PASS` 配套使用 |
| `API_BASIC_AUTH_PASS` | 条件必填 | `cmd/server/main.go` | 空 | 对外访问 `/api/*` 的 Basic Auth 密码 |
| `API_RATE_LIMIT_PER_WINDOW` | 否 | `cmd/server/main.go` | `60` | `/api/*` 按来源 IP 的窗口限流阈值;设为 `0` 表示关闭内建限流 |
| `API_RATE_LIMIT_WINDOW_SEC` | 否 | `cmd/server/main.go` | `60` | `/api/*` 限流窗口长度(秒) |
| `FEISHU_WEBHOOK` | 否 | `run_daily.sh``feishu_alert.sh` | 空 | 正式日报失败时发送飞书告警 |
| `REPORT_OUTPUT_DIR` | 否 | `generate_daily_report.go` | `reports/daily` | 日报主产物输出目录 |
| `REPORT_DATE` | 否 | `generate_daily_report.go``rebuild_historical_report.sh` | 当天日期 | 指定日报生成日期,格式 `YYYY-MM-DD` |
| `REPORT_DATE` | 否 | `generate_daily_report.go``rebuild_historical_report.sh``run_intraday_price_watch.sh` | 当天日期 | 指定日报或日内价格追踪的日期,格式 `YYYY-MM-DD` |
| `REPORT_RUN_KIND` | 否 | `generate_daily_report.go` | `manual` | 运行语义,如 `scheduled` / `manual` / `historical_rebuild` |
| `REPORT_TRIGGER_SOURCE` | 否 | `generate_daily_report.go` | `cli` | 触发来源,如 `cron` / `pipeline` / `rebuild_script` |
| `REPORT_TRIGGER_SOURCE` | 否 | `generate_daily_report.go``materialize_daily_signals.go` | `cli` | 触发来源,如 `cron` / `pipeline` / `intraday` / `rebuild_script` |
| `REPORT_IS_OFFICIAL_DAILY` | 否 | `generate_daily_report.go` | `false` | 是否属于正式日报产出 |
| `REPORT_RUNTIME_AUDIT` | 否 | `generate_daily_report.go` | 空 | 来源级运行审计摘要,通常由流水线脚本注入 |
| `PHASE6_PORT` | 否 | `verify_phase6.sh` | 自动挑选 `18080-18120` | Phase 6 验收时临时启动 API Server 的端口 |
@@ -33,6 +38,8 @@
| `LIGHTHOUSE_FCP_THRESHOLD_MS` | 否 | `verify_lighthouse.sh` | `2000` | 首次内容绘制门槛 |
| `VERIFY_DB_NAME` | 否 | `verify_common.sh` | `llm_intelligence` | SQL 型验收脚本默认连接的数据库名 |
## 推荐的生产注入方式
### API Server
@@ -40,9 +47,12 @@
```bash
export DATABASE_URL="postgres://app_user:***@db:5432/llm_intelligence?sslmode=disable"
export PORT="8080"
export API_AUTH_TOKEN="replace-with-long-random-token"
# 或者export API_BASIC_AUTH_USER="review" && export API_BASIC_AUTH_PASS="replace-with-password"
./server
```
### 正式日报调度
```bash
@@ -60,6 +70,19 @@ export OPENROUTER_API_KEY="***"
bash scripts/run_real_pipeline.sh
```
### 日内价格追踪
```bash
export DATABASE_URL="postgres://app_user:***@db:5432/llm_intelligence?sslmode=disable"
export OPENROUTER_API_KEY="***"
bash scripts/run_intraday_price_watch.sh
```
说明:
- 该入口只刷新价格 importer 与 `daily_signal_snapshot`
- 不生成正式 HTML / Markdown 日报
- 推荐先按每 4 小时一次调度,再根据外部源稳定性决定是否收紧到每 2 小时
## 日报运行语义
项目用以下字段区分正式日报、手工复跑和历史补跑:
@@ -104,6 +127,11 @@ PORT="8080" \
go run ./cmd/server
```
说明:
- `/health` 仅允许本机或私网来源访问
- `/api/*` 对外访问默认要求 Bearer token 或 Basic Auth
- 本机与私网来源可直接访问,便于同机前端、验收脚本和内网反代联调
### 仅生成指定日期日报
```bash

View File

@@ -43,7 +43,7 @@
### 应用与产物
- `go test ./...` 通过(覆盖 `cmd/server``internal/...`
- `go test ./...` 通过(覆盖 package 形式的 Go 代码,如 `cmd/server``internal/...`;其中 API 错误结构与模型主价格排序规则需由这些 package tests 兜底
- `bash scripts/test_importers.sh` 通过(覆盖 scripts 层 importer targeted go test matrix
- `bash scripts/importer_smoke_gate_test.sh` 通过
- `bash scripts/pipeline_runtime_alignment_test.sh` 通过
@@ -51,12 +51,15 @@
- `cd frontend && npm run test -- --run` 通过
- `cd frontend && npm run build` 通过
- `go build ./cmd/server` 通过
- 已确认发布结论不是仅凭 `go test ./...` 得出,而是同时包含 scripts 与 gate 层验证
### 调度与日报
- 正式调度命令已确定:`bash scripts/run_daily.sh`
- 手工复跑命令已确定:`bash scripts/run_real_pipeline.sh`
- 历史补跑命令已确定:`bash scripts/rebuild_historical_report.sh YYYY-MM-DD`
- 日内价格追踪命令已确定:`bash scripts/run_intraday_price_watch.sh`
- `OPENROUTER_API_KEY` 已在正式调度环境可用
- `FEISHU_WEBHOOK` 已配置或明确不上告警
@@ -136,6 +139,8 @@ bash scripts/run_real_pipeline.sh
```cron
0 8 * * * cd /path/to/llm-intelligence && bash scripts/run_daily.sh >> /tmp/llm_hub_cron.log 2>&1
```
# 日内价格追踪(推荐)
0 */4 * * * cd /path/to/llm-intelligence && bash scripts/run_intraday_price_watch.sh >> /tmp/llm_hub_intraday.log 2>&1
### 7. 线上冒烟

View File

@@ -0,0 +1,60 @@
# 日内价格追踪方案2026-05-27
## 目标
让“日内大降价 / 大涨价 / 泄露 / 活动窗口”不必等到第二天正式日报才出现。
## 当前限制
- 正式链路 `scripts/run_daily.sh` 按天运行一次。
- `daily_signal_snapshot` 也是按日物化。
- 像“小米大模型大降价”这样的日内事件,即使价格页已经变化,也可能错过当天头条和一句话结论。
## 最小可落地方案
新增脚本:`scripts/run_intraday_price_watch.sh`
它复用当前 `run_intel_pipeline.sh` 的“采集 / 导入 / 物化”链路,但刻意不生成正式日报,不写 `daily_report`,也不污染 `latest_report` 语义。
### 执行内容
- `fetch_openrouter.go -strict-real`
- `fetch_multi_source.go --sources moonshot,deepseek,openai`
- 官方导入脚本(套餐 + 价格 importer
- `materialize_daily_signals.go`
### 不执行
- `generate_daily_report.go`
- `track_report_state` / `daily_report`
- 正式 HTML / Markdown 日报归档
## 推荐调度频率
推荐两档:
1. 保守版:每 4 小时一次
- `0 */4 * * * bash scripts/run_intraday_price_watch.sh`
2. 激进版:每 2 小时一次
- `0 */2 * * * bash scripts/run_intraday_price_watch.sh`
先从每 4 小时开始,观察外部文档源稳定性和数据库写入压力。
## 结果用途
- 更快写入 `pricing_history`
- 更快刷新 `daily_signal_snapshot`
- 为前端查询页/日内快讯卡提供更及时的信号
- 第二天正式日报能直接消费更完整的价格变化记录
## 与正式日报的边界
- `run_daily.sh`:正式日级产物,决定 `latest_report`
- `run_intraday_price_watch.sh`:日内信号刷新,不生成正式日报
- `run_real_pipeline.sh`:人工真实复跑,验证全链路
## 下一步建议
1. 把前端查询页增加“最近一次价格追踪时间”提示
2.`materialize_daily_signals.go` 增加 `trigger_source=intraday` 的文档说明
3. 如果日内事件仍不够敏感,再考虑引入独立 `intraday_signal_snapshot`

View File

@@ -3,7 +3,8 @@
max-width: 1200px;
margin: 0 auto;
padding: 20px;
font-family: -apple-system, BlinkMacSystemFont, 'Segoe UI', Roboto, sans-serif;
font-family:
-apple-system, BlinkMacSystemFont, "Segoe UI", Roboto, sans-serif;
}
.navbar {
@@ -252,7 +253,11 @@
border: 1px solid #dbeafe;
border-radius: 18px;
background:
radial-gradient(circle at top left, rgba(37, 99, 235, 0.10), transparent 35%),
radial-gradient(
circle at top left,
rgba(37, 99, 235, 0.1),
transparent 35%
),
rgba(255, 255, 255, 0.94);
box-shadow: 0 12px 30px rgba(37, 99, 235, 0.08);
}
@@ -343,6 +348,44 @@
color: #6b7280;
font-size: 13px;
}
.runtime-warning {
margin-bottom: 16px;
padding: 12px 14px;
border: 1px solid #f59e0b;
border-radius: 10px;
background: #fff7ed;
color: #9a3412;
font-size: 14px;
line-height: 1.5;
}
.runtime-error {
margin-bottom: 16px;
padding: 12px 14px;
border: 1px solid #dc2626;
border-radius: 10px;
background: #fef2f2;
color: #991b1b;
font-size: 14px;
line-height: 1.5;
}
.runtime-error-inline {
color: #991b1b;
}
.data-empty {
padding: 18px;
border: 1px dashed #d1d5db;
border-radius: 10px;
background: #f9fafb;
color: #6b7280;
text-align: center;
}
.runtime-warning-inline {
color: #9a3412;
}
.subscription-section {
padding: 20px;
@@ -431,6 +474,293 @@
text-align: center;
}
.explorer-editorial,
.report-priority-section {
display: grid;
gap: 18px;
}
.explorer-hero,
.filters-editorial,
.pricing-focus-card,
.pricing-board-card,
.report-priority-card,
.report-news-card,
.report-theme-card {
border: 1px solid rgba(15, 23, 42, 0.08);
border-radius: 20px;
background: linear-gradient(
180deg,
rgba(255, 255, 255, 0.98) 0%,
rgba(248, 250, 252, 0.96) 100%
);
box-shadow: 0 20px 50px rgba(15, 23, 42, 0.06);
}
.explorer-hero,
.pricing-focus-card,
.report-priority-card {
padding: 22px;
}
.explorer-kicker,
.report-news-label {
font-size: 12px;
letter-spacing: 0.12em;
text-transform: uppercase;
color: #475569;
}
.explorer-hero {
display: flex;
justify-content: space-between;
align-items: flex-start;
gap: 20px;
}
.explorer-hero h2 {
margin: 6px 0 8px;
font-size: 34px;
line-height: 1.05;
color: #0f172a;
}
.explorer-hero p,
.pricing-focus-header p,
.report-news-summary,
.report-theme-card li,
.report-evidence {
color: #475569;
line-height: 1.6;
}
.explorer-hero-meta {
display: flex;
gap: 10px;
flex-wrap: wrap;
}
.explorer-hero-meta span,
.report-news-label {
padding: 8px 12px;
border-radius: 999px;
background: #eff6ff;
color: #1d4ed8;
}
.filters-editorial {
padding: 14px 16px;
background: #fff;
}
.pricing-focus-header,
.report-news-grid,
.report-theme-grid,
.pricing-board-grid {
display: grid;
gap: 14px;
}
.pricing-focus-header {
grid-template-columns: 1fr auto;
align-items: start;
}
.pricing-focus-header h3,
.report-theme-title,
.report-news-title {
margin: 8px 0 6px;
color: #0f172a;
}
.pricing-focus-prices {
margin-top: 18px;
display: grid;
grid-template-columns: repeat(2, minmax(0, 1fr));
gap: 14px;
}
.pricing-focus-prices div,
.pricing-board-card,
.report-news-card,
.report-theme-card {
padding: 16px;
}
.pricing-focus-prices span,
.pricing-board-meta {
display: block;
color: #64748b;
font-size: 13px;
}
.pricing-focus-prices strong,
.pricing-board-price {
display: block;
margin-top: 8px;
font-size: 24px;
color: #111827;
}
.pricing-board-grid,
.report-news-grid,
.report-theme-grid {
grid-template-columns: repeat(auto-fit, minmax(220px, 1fr));
}
.report-priority-card {
background: linear-gradient(180deg, #fff 0%, #f8fafc 100%);
}
.report-hero-priority {
background: linear-gradient(135deg, #0f172a 0%, #1e3a8a 55%, #2563eb 100%);
}
.report-evidence {
margin-top: 12px;
color: rgba(255, 255, 255, 0.82);
}
.report-theme-card ul {
margin: 12px 0 0;
padding-left: 18px;
}
.model-table-editorial {
box-shadow: 0 12px 30px rgba(15, 23, 42, 0.04);
}
.theme-news-list {
display: grid;
gap: 12px;
margin-top: 12px;
}
.theme-news-item {
padding: 14px;
border-radius: 14px;
background: rgba(248, 250, 252, 0.92);
border: 1px solid rgba(148, 163, 184, 0.18);
border-left: 6px solid #94a3b8;
}
.theme-news-item.tone-success {
background: linear-gradient(180deg, rgba(240, 253, 244, 0.98) 0%, rgba(220, 252, 231, 0.92) 100%);
border-color: rgba(34, 197, 94, 0.28);
border-left-color: #16a34a;
}
.theme-news-item.tone-success .card-title,
.theme-news-item.tone-success .trust-line {
color: #166534;
}
.theme-news-item.tone-caution {
background: linear-gradient(180deg, rgba(254, 242, 242, 0.98) 0%, rgba(254, 226, 226, 0.92) 100%);
border-color: rgba(239, 68, 68, 0.26);
border-left-color: #dc2626;
}
.theme-news-item.tone-caution .card-title,
.theme-news-item.tone-caution .trust-line {
color: #991b1b;
}
.theme-news-item.tone-promo {
background: linear-gradient(180deg, rgba(255, 247, 237, 0.98) 0%, rgba(250, 245, 255, 0.94) 100%);
border-color: rgba(168, 85, 247, 0.22);
border-left-color: #f97316;
}
.theme-news-item.tone-promo .card-title,
.theme-news-item.tone-promo .trust-line {
color: #9a3412;
}
.report-theme-card {
background: linear-gradient(180deg, rgba(248, 250, 252, 0.98) 0%, rgba(241, 245, 249, 0.94) 100%);
}
.report-theme-card:first-child {
border-color: rgba(34, 197, 94, 0.18);
}
.report-theme-card:first-child .report-theme-title {
color: #166534;
}
.report-theme-card:nth-child(2) {
border-color: rgba(239, 68, 68, 0.18);
}
.report-theme-card:nth-child(2) .report-theme-title {
color: #991b1b;
}
.report-theme-card:nth-child(3) {
border-color: rgba(249, 115, 22, 0.2);
}
.report-theme-card:nth-child(3) .report-theme-title {
color: #9a3412;
}
.report-theme-badge {
display: inline-flex;
align-items: center;
gap: 8px;
margin-bottom: 12px;
}
.report-theme-badge-icon {
width: 26px;
height: 26px;
display: inline-flex;
align-items: center;
justify-content: center;
border-radius: 999px;
background: rgba(15, 23, 42, 0.08);
font-weight: 700;
}
.report-theme-badge-label {
font-size: 11px;
letter-spacing: 0.12em;
text-transform: uppercase;
color: #475569;
}
.report-theme-card.tone-success .report-theme-badge-icon {
background: rgba(34, 197, 94, 0.16);
color: #166534;
}
.report-theme-card.tone-caution .report-theme-badge-icon {
background: rgba(239, 68, 68, 0.16);
color: #991b1b;
}
.report-theme-card.tone-promo .report-theme-badge-icon {
background: rgba(249, 115, 22, 0.16);
color: #9a3412;
}
@media (max-width: 768px) {
.explorer-hero,
.pricing-focus-header {
grid-template-columns: 1fr;
display: grid;
}
.explorer-hero h2 {
font-size: 28px;
}
.pricing-focus-prices {
grid-template-columns: 1fr;
}
}
@media (max-width: 768px) {
.app {
padding: 16px;

View File

@@ -1,4 +1,4 @@
import { describe, expect, it } from 'vitest'
import { describe, expect, it } from "vitest";
import {
formatPrice,
formatSubscriptionQuota,
@@ -7,113 +7,207 @@ import {
providerDistribution,
summarizeModels,
summarizeSubscriptionPlans,
} from './models'
} from "./models";
describe('models helpers', () => {
it('normalizes fallback pricing and stale flags', () => {
describe("models helpers", () => {
it("normalizes fallback pricing and stale flags", () => {
const model = normalizeModel({
id: 'anthropic/claude-sonnet-4.6',
provider_cn: 'Anthropic',
id: "anthropic/claude-sonnet-4.6",
provider_cn: "Anthropic",
context_length: 200000,
input_price: '3',
output_price: '15',
data_confidence: 'stale',
})
input_price: "3",
output_price: "15",
data_confidence: "stale",
});
expect(model).not.toBeNull()
expect(model?.providerCN).toBe('Anthropic')
expect(model?.inputPrice).toBe(3)
expect(model?.outputPrice).toBe(15)
expect(model?.stale).toBe(true)
expect(model?.pricingAvailable).toBe(true)
})
expect(model).not.toBeNull();
expect(model?.providerCN).toBe("Anthropic");
expect(model?.inputPrice).toBe(3);
expect(model?.outputPrice).toBe(15);
expect(model?.stale).toBe(true);
expect(model?.pricingAvailable).toBe(true);
});
it('marks free models and pricing unavailable correctly', () => {
it("marks free models and pricing unavailable correctly", () => {
const freeModel = normalizeModel({
id: 'qwen/qwen3-coder:free',
})
id: "qwen/qwen3-coder:free",
});
const paidModel = normalizeModel({
id: 'openai/gpt-4.1',
id: "openai/gpt-4.1",
pricing: {},
})
});
expect(formatPrice(freeModel!, 'input')).toContain('免费')
expect(formatPrice(paidModel!, 'input')).toBe('pricing unavailable')
})
expect(formatPrice(freeModel!, "input")).toContain("免费");
expect(formatPrice(paidModel!, "input")).toBe("pricing unavailable");
});
it('summarizes providers and currencies', () => {
it("summarizes providers and currencies", () => {
const models = [
normalizeModel({ id: 'deepseek/deepseek-chat', provider_cn: 'DeepSeek', currency: 'CNY', pricing: { input: 1, output: 2 } }),
normalizeModel({ id: 'deepseek/deepseek-reasoner', provider_cn: 'DeepSeek', currency: 'CNY', pricing: { input: 2, output: 4 } }),
normalizeModel({ id: 'anthropic/claude-sonnet-4.6', provider_cn: 'Anthropic', currency: 'USD', pricing: { input: 3, output: 15 } }),
].filter((model): model is NonNullable<typeof model> => model !== null)
normalizeModel({
id: "deepseek/deepseek-chat",
provider_cn: "DeepSeek",
currency: "CNY",
pricing: { input: 1, output: 2 },
}),
normalizeModel({
id: "deepseek/deepseek-reasoner",
provider_cn: "DeepSeek",
currency: "CNY",
pricing: { input: 2, output: 4 },
}),
normalizeModel({
id: "anthropic/claude-sonnet-4.6",
provider_cn: "Anthropic",
currency: "USD",
pricing: { input: 3, output: 15 },
}),
].filter((model): model is NonNullable<typeof model> => model !== null);
expect(summarizeModels(models)).toEqual({
modelCount: 3,
providerCount: 2,
cnyCount: 2,
})
});
expect(providerDistribution(models)).toEqual([
{ name: 'DeepSeek', value: 2 },
{ name: 'Anthropic', value: 1 },
])
})
{ name: "DeepSeek", value: 2 },
{ name: "Anthropic", value: 1 },
]);
});
it('normalizes subscription plans from API payload', () => {
it("normalizes subscription plans from API payload", () => {
const plan = normalizeSubscriptionPlan({
planCode: 'token-plan-lite',
planName: '通用 Token Plan Lite',
planFamily: 'token_plan',
tier: 'Lite',
provider: 'Tencent',
providerCN: '腾讯',
operator: 'Tencent Cloud',
operatorCN: '腾讯云',
currency: 'CNY',
planCode: "token-plan-lite",
planName: "通用 Token Plan Lite",
planFamily: "token_plan",
tier: "Lite",
provider: "Tencent",
providerCN: "腾讯",
operator: "Tencent Cloud",
operatorCN: "腾讯云",
currency: "CNY",
listPrice: 39,
priceUnit: 'CNY/month',
priceUnit: "CNY/month",
quotaValue: 35000000,
quotaUnit: 'tokens/month',
quotaUnit: "tokens/month",
contextWindow: 0,
modelScope: ['tc-code-latest', 'glm-5', 'glm-5.1'],
})
modelScope: ["tc-code-latest", "glm-5", "glm-5.1"],
});
expect(plan).not.toBeNull()
expect(plan?.planCode).toBe('token-plan-lite')
expect(plan?.providerCN).toBe('腾讯')
expect(plan?.modelScope.length).toBe(3)
expect(plan?.modelPreview).toBe('tc-code-latest, glm-5, glm-5.1')
})
expect(plan).not.toBeNull();
expect(plan?.planCode).toBe("token-plan-lite");
expect(plan?.providerCN).toBe("腾讯");
expect(plan?.modelScope.length).toBe(3);
expect(plan?.modelPreview).toBe("tc-code-latest, glm-5, glm-5.1");
});
it('formats subscription quotas and summarizes plan stats', () => {
it("formats subscription quotas and summarizes plan stats", () => {
const plans = [
normalizeSubscriptionPlan({
planCode: 'token-plan-lite',
planName: '通用 Token Plan Lite',
providerCN: '腾讯',
planCode: "token-plan-lite",
planName: "通用 Token Plan Lite",
providerCN: "腾讯",
listPrice: 39,
quotaValue: 35000000,
quotaUnit: 'tokens/month',
modelScope: ['tc-code-latest', 'glm-5', 'glm-5.1'],
quotaUnit: "tokens/month",
modelScope: ["tc-code-latest", "glm-5", "glm-5.1"],
}),
normalizeSubscriptionPlan({
planCode: 'hy-token-plan-max',
planName: 'Hy Token Plan Max',
providerCN: '腾讯',
planCode: "hy-token-plan-max",
planName: "Hy Token Plan Max",
providerCN: "腾讯",
listPrice: 468,
quotaValue: 650000000,
quotaUnit: 'tokens/month',
quotaUnit: "tokens/month",
contextWindow: 262144,
modelScope: ['hy3-preview'],
modelScope: ["hy3-preview"],
}),
].filter((plan): plan is NonNullable<typeof plan> => plan !== null)
].filter((plan): plan is NonNullable<typeof plan> => plan !== null);
expect(formatSubscriptionQuota(plans[0].quotaValue, plans[0].quotaUnit)).toBe('3500万 Tokens/月')
expect(formatSubscriptionQuota(plans[1].quotaValue, plans[1].quotaUnit)).toBe('6.5亿 Tokens/月')
expect(
formatSubscriptionQuota(plans[0].quotaValue, plans[0].quotaUnit),
).toBe("3500万 Tokens/月");
expect(
formatSubscriptionQuota(plans[1].quotaValue, plans[1].quotaUnit),
).toBe("6.5亿 Tokens/月");
expect(summarizeSubscriptionPlans(plans)).toEqual({
planCount: 2,
providerCount: 1,
minMonthlyPrice: 39,
})
})
})
});
});
});
it("prefers the largest daily price swing model as pricing lead", () => {
const models = [
normalizeModel({
id: "deepseek/deepseek-v4-flash",
name: "DeepSeek-V4-Flash",
provider_cn: "DeepSeek",
pricing: { input: 0.3, output: 1.2 },
data_confidence: "official",
}),
normalizeModel({
id: "qwen/qwen-vl-max",
name: "Qwen VL Max",
provider_cn: "阿里云",
pricing: { input: 0.8, output: 2.4 },
data_confidence: "official",
}),
normalizeModel({
id: "glm/glm-5",
name: "GLM-5",
provider_cn: "智谱",
pricing: { input: 0, output: 0 },
is_free: true,
data_confidence: "official",
}),
].filter((model): model is NonNullable<typeof model> => model !== null);
const ranked = [...models].sort(
(a, b) => b.outputPrice - a.outputPrice || b.inputPrice - a.inputPrice,
);
expect(ranked[0].name).toBe("Qwen VL Max");
expect(formatPrice(ranked[0], "output")).toContain("2.4");
});
it("extracts pricing-first report sections from markdown summary", async () => {
const { normalizeLatestReportPayload } = await import("../pages/Dashboard");
const report = normalizeLatestReportPayload({
reportDate: "2026-05-25",
status: "generated",
modelCount: 504,
summaryMD: [
"## 今日结论",
"> 今天最值得关注的是 qwen-vl-max 价格下降 18%,优先复查它是否改变默认选型与预算策略。",
"- 证据: 主来源pricing_history输入价格较昨日下降 18%",
"",
"## 今日行动建议",
"1. **先看 qwen-vl-max** ",
"2. **复查 GLM-5** ",
"",
"## 今日价格新闻",
"### 降价机会",
"#### qwen-vl-max 成本下调 18%",
"- 影响: 视觉模型价格下降已足以影响默认选型。",
"### 平台活动",
"#### DeepSeek-V4-Flash 进入活动窗口",
"- 影响: 平台活动窗口出现后,值得重新评估低成本推理方案。",
"",
"## 场景推荐",
"### 低成本编码",
"- 主推荐: DeepSeek-V4-Flash",
"### 中文通用",
"- 主推荐: GLM-5",
].join("\n"),
markdownUrl: "/report.md",
htmlUrl: "/report.html",
updatedAt: "2026-05-25T10:00:00",
});
expect(report.pricingLead).toContain("qwen-vl-max");
expect(report.pricingLeadNote).toContain("pricing_history");
expect(report.headlines[0].title).toContain("qwen-vl-max");
expect(report.themes[0].title).toBe("降价机会");
expect(report.themes[0].bullets[0]).toContain("qwen-vl-max");
expect(report.themes[1].title).toBe("平台活动");
});

View File

@@ -1,5 +1,5 @@
import { useEffect, useRef, useState } from 'react'
import * as echarts from 'echarts'
import { useEffect, useRef, useState } from "react";
import * as echarts from "echarts";
import {
formatSubscriptionQuota,
loadFallbackModels,
@@ -10,186 +10,368 @@ import {
summarizeSubscriptionPlans,
type Model,
type SubscriptionPlan,
} from '../lib/models'
} from "../lib/models";
import {
buildApiUnavailableNotice,
buildFallbackNotice,
detectRuntimeEnvironment,
shouldUseLocalFallback,
} from "../lib/runtimeVisibility";
type ReportHeadline = {
label: string;
title: string;
summary: string;
};
type ReportTheme = {
title: string;
bullets: string[];
};
type LatestReport = {
reportDate: string
status: string
modelCount: number
summaryMD: string
markdownUrl: string
htmlUrl: string
updatedAt: string
}
reportDate: string;
status: string;
modelCount: number;
summaryMD: string;
markdownUrl: string;
htmlUrl: string;
updatedAt: string;
pricingLead: string;
pricingLeadNote: string;
headlines: ReportHeadline[];
themes: ReportTheme[];
};
function formatLocalReportDate(date: Date) {
const year = date.getFullYear()
const month = `${date.getMonth() + 1}`.padStart(2, '0')
const day = `${date.getDate()}`.padStart(2, '0')
return `${year}-${month}-${day}`
const year = date.getFullYear();
const month = `${date.getMonth() + 1}`.padStart(2, "0");
const day = `${date.getDate()}`.padStart(2, "0");
return `${year}-${month}-${day}`;
}
function buildFallbackLatestReport(): LatestReport {
const reportDate = formatLocalReportDate(new Date())
const reportDate = formatLocalReportDate(new Date());
return {
reportDate,
status: 'generated',
status: "generated",
modelCount: 0,
summaryMD: '最新日报入口可用,后端元数据暂未返回摘要。',
summaryMD: "最新日报入口可用,后端元数据暂未返回摘要。",
markdownUrl: `/reports/daily/daily_report_${reportDate}.md`,
htmlUrl: `/reports/daily/html/daily_report_${reportDate}.html`,
updatedAt: '',
updatedAt: "",
pricingLead: "当日价格异动摘要暂不可用",
pricingLeadNote: "请直接打开 HTML 日报查看完整价格异动与主题分组。",
headlines: [],
themes: [],
};
}
function extractReportSections(summaryMD: string) {
const normalized = summaryMD.replace(/\r\n/g, "\n");
const lines = normalized.split("\n");
const sections = new Map<string, string[]>();
let current = "";
for (const line of lines) {
const trimmed = line.trim();
if (trimmed.startsWith("## ")) {
current = trimmed.slice(3).trim();
sections.set(current, []);
continue;
}
if (!current || trimmed === "") {
continue;
}
sections.get(current)?.push(trimmed);
}
return sections;
}
function summarizeLatestReport(report: LatestReport) {
if (report.pricingLead.trim()) {
return report.pricingLead.trim();
}
if (report.summaryMD.trim()) {
return report.summaryMD.trim()
return report.summaryMD.trim();
}
if (report.modelCount > 0) {
return `最新日报已生成,覆盖 ${report.modelCount} 个模型,可直接查看完整 HTML 页面。`
return `最新日报已生成,覆盖 ${report.modelCount} 个模型,可直接查看完整 HTML 页面。`;
}
return '最新日报入口已准备好,可直接打开 HTML 或 Markdown 查看。'
return "最新日报入口已准备好,可直接打开 HTML 或 Markdown 查看。";
}
export function normalizeLatestReportPayload(payload: any): LatestReport {
const summaryMD =
typeof payload?.summaryMD === "string" ? payload.summaryMD : "";
const sections = extractReportSections(summaryMD);
const conclusion = sections.get("今日结论") ?? [];
const changes = sections.get("今日价格新闻") ?? [];
const sceneLines = sections.get("场景推荐") ?? [];
const actionLines = sections.get("今日行动建议") ?? [];
const pricingLead =
conclusion[0]?.replace(/^>\s*/, "") || "当日价格异动摘要暂不可用";
const pricingLeadNote =
changes
.find((line) => line.startsWith("- 证据:"))
?.replace("- 证据:", "")
.trim() ||
conclusion
.find((line) => line.startsWith("- 证据:"))
?.replace("- 证据:", "")
.trim() ||
"请直接打开 HTML 日报查看完整价格异动与主题分组。";
const headlines = actionLines
.filter((line) => /^\d+\./.test(line))
.slice(0, 3)
.map((line) => {
const title = line
.replace(/^\d+\.\s*\*\*/, "")
.replace(/\*\*\s*$/, "")
.trim();
return {
label: "今日动作",
title,
summary: "围绕当天最重要的价格异动与选型影响整理。",
};
});
const sceneThemes: ReportTheme[] = [];
let currentSceneTheme: ReportTheme | null = null;
for (const line of sceneLines) {
if (line.startsWith("### ")) {
currentSceneTheme = { title: line.slice(4).trim(), bullets: [] };
sceneThemes.push(currentSceneTheme);
continue;
}
if (currentSceneTheme && line.startsWith("- ")) {
currentSceneTheme.bullets.push(line.slice(2).trim());
}
}
const pricingThemes: ReportTheme[] = [];
let currentPricingTheme: ReportTheme | null = null;
for (const line of changes) {
if (line.startsWith("### ")) {
currentPricingTheme = { title: line.slice(4).trim(), bullets: [] };
pricingThemes.push(currentPricingTheme);
continue;
}
if (currentPricingTheme && line.startsWith("#### ")) {
currentPricingTheme.bullets.push(line.slice(5).trim());
continue;
}
if (currentPricingTheme && line.startsWith("- 影响: ")) {
currentPricingTheme.bullets.push(line.replace("- 影响: ", ""));
}
}
return {
reportDate: payload?.reportDate,
status: payload?.status || "generated",
modelCount: Number(payload?.modelCount || 0),
summaryMD,
markdownUrl: payload?.markdownUrl,
htmlUrl: payload?.htmlUrl,
updatedAt: payload?.updatedAt || "",
pricingLead,
pricingLeadNote,
headlines,
themes: pricingThemes.length > 0 ? pricingThemes : sceneThemes,
};
}
function reportThemeBadge(themeTitle: string) {
if (themeTitle.includes("降价")) {
return { icon: "↓", label: "Opportunity", tone: "tone-success" };
}
if (themeTitle.includes("涨价")) {
return { icon: "↑", label: "Warning", tone: "tone-caution" };
}
if (themeTitle.includes("活动")) {
return { icon: "✦", label: "Campaign", tone: "tone-promo" };
}
return { icon: "•", label: "Signal", tone: "tone-neutral" };
}
function Dashboard() {
const chartRef = useRef<HTMLDivElement>(null)
const [modelCount, setModelCount] = useState(0)
const [providerCount, setProviderCount] = useState(0)
const [cnyCount, setCnyCount] = useState(0)
const [subscriptionPlans, setSubscriptionPlans] = useState<SubscriptionPlan[]>([])
const [planCount, setPlanCount] = useState(0)
const [planMinPrice, setPlanMinPrice] = useState(0)
const [latestReport, setLatestReport] = useState<LatestReport | null>(null)
const [reportFallback, setReportFallback] = useState(false)
const chartRef = useRef<HTMLDivElement>(null);
const [modelCount, setModelCount] = useState(0);
const [providerCount, setProviderCount] = useState(0);
const [cnyCount, setCnyCount] = useState(0);
const [subscriptionPlans, setSubscriptionPlans] = useState<
SubscriptionPlan[]
>([]);
const [planCount, setPlanCount] = useState(0);
const [planMinPrice, setPlanMinPrice] = useState(0);
const [latestReport, setLatestReport] = useState<LatestReport | null>(null);
const [modelsFallback, setModelsFallback] = useState(false);
const [modelsUnavailable, setModelsUnavailable] = useState("");
const [reportFallback, setReportFallback] = useState(false);
const [reportUnavailable, setReportUnavailable] = useState("");
const runtime = detectRuntimeEnvironment();
const modelsFallbackNotice = buildFallbackNotice("models", runtime);
const modelsUnavailableNotice = buildApiUnavailableNotice("models", runtime);
const reportFallbackNotice = buildFallbackNotice("latestReport", runtime);
const reportUnavailableNotice = buildApiUnavailableNotice(
"latestReport",
runtime,
);
useEffect(() => {
let chart: echarts.ECharts | null = null
let disposed = false
let chart: echarts.ECharts | null = null;
let disposed = false;
const renderChart = (models: Model[]) => {
if (!chartRef.current) {
return
return;
}
chart = echarts.init(chartRef.current)
chart = echarts.init(chartRef.current);
const option: echarts.EChartsOption = {
title: { text: '厂商模型分布', left: 'center' },
tooltip: { trigger: 'item' },
series: [{
type: 'pie',
radius: '60%',
data: providerDistribution(models),
emphasis: {
itemStyle: {
shadowBlur: 10,
shadowOffsetX: 0,
shadowColor: 'rgba(0, 0, 0, 0.5)'
}
}
}]
}
chart.setOption(option)
}
title: { text: "厂商模型分布", left: "center" },
tooltip: { trigger: "item" },
series: [
{
type: "pie",
radius: "60%",
data: providerDistribution(models),
emphasis: {
itemStyle: {
shadowBlur: 10,
shadowOffsetX: 0,
shadowColor: "rgba(0, 0, 0, 0.5)",
},
},
},
],
};
chart.setOption(option);
};
const updateStats = (models: Model[]) => {
const summary = summarizeModels(models)
setModelCount(summary.modelCount)
setProviderCount(summary.providerCount)
setCnyCount(summary.cnyCount)
renderChart(models)
}
const summary = summarizeModels(models);
setModelCount(summary.modelCount);
setProviderCount(summary.providerCount);
setCnyCount(summary.cnyCount);
renderChart(models);
};
const updatePlans = (plans: SubscriptionPlan[]) => {
const summary = summarizeSubscriptionPlans(plans)
setSubscriptionPlans(plans)
setPlanCount(summary.planCount)
setPlanMinPrice(summary.minMonthlyPrice)
}
const summary = summarizeSubscriptionPlans(plans);
setSubscriptionPlans(plans);
setPlanCount(summary.planCount);
setPlanMinPrice(summary.minMonthlyPrice);
};
const loadModels = async () => {
try {
const response = await fetch('/api/v1/models')
const response = await fetch("/api/v1/models");
if (!response.ok) {
throw new Error(`models request failed: ${response.status}`)
throw new Error(`models request failed: ${response.status}`);
}
const payload = await response.json()
const rawModels: any[] = Array.isArray(payload?.data) ? payload.data : []
const payload = await response.json();
const rawModels: any[] = Array.isArray(payload?.data)
? payload.data
: [];
const models = rawModels
.map(normalizeModel)
.filter((model: Model | null): model is Model => model !== null)
.filter((model: Model | null): model is Model => model !== null);
if (!disposed) {
updateStats(models)
updateStats(models);
setModelsFallback(false);
setModelsUnavailable("");
}
} catch {
const fallback = await loadFallbackModels()
if (!disposed) {
updateStats(fallback)
if (shouldUseLocalFallback("models", runtime)) {
const fallback = await loadFallbackModels();
if (!disposed) {
updateStats(fallback);
setModelsFallback(fallback.length > 0);
setModelsUnavailable(
fallback.length === 0 ? modelsUnavailableNotice : "",
);
}
} else if (!disposed) {
updateStats([]);
setModelsFallback(false);
setModelsUnavailable(modelsUnavailableNotice);
}
}
}
};
const loadSubscriptionPlans = async () => {
try {
const response = await fetch('/api/v1/subscription-plans')
const response = await fetch("/api/v1/subscription-plans");
if (!response.ok) {
throw new Error(`subscription plans request failed: ${response.status}`)
throw new Error(
`subscription plans request failed: ${response.status}`,
);
}
const payload = await response.json()
const rawPlans: any[] = Array.isArray(payload?.data) ? payload.data : []
const payload = await response.json();
const rawPlans: any[] = Array.isArray(payload?.data)
? payload.data
: [];
const plans = rawPlans
.map(normalizeSubscriptionPlan)
.filter((plan: SubscriptionPlan | null): plan is SubscriptionPlan => plan !== null)
.filter(
(plan: SubscriptionPlan | null): plan is SubscriptionPlan =>
plan !== null,
);
if (!disposed) {
updatePlans(plans)
updatePlans(plans);
}
} catch {
if (!disposed) {
updatePlans([])
updatePlans([]);
}
}
}
};
const loadLatestReport = async () => {
try {
const response = await fetch('/api/v1/reports/latest')
const response = await fetch("/api/v1/reports/latest");
if (!response.ok) {
throw new Error(`latest report request failed: ${response.status}`)
throw new Error(`latest report request failed: ${response.status}`);
}
const payload = await response.json()
const report = payload?.data
const payload = await response.json();
const report = payload?.data;
if (!report?.reportDate || !report?.htmlUrl || !report?.markdownUrl) {
throw new Error('latest report payload invalid')
throw new Error("latest report payload invalid");
}
if (!disposed) {
setLatestReport({
reportDate: report.reportDate,
status: report.status || 'generated',
modelCount: Number(report.modelCount || 0),
summaryMD: report.summaryMD || '',
markdownUrl: report.markdownUrl,
htmlUrl: report.htmlUrl,
updatedAt: report.updatedAt || '',
})
setReportFallback(false)
setLatestReport(normalizeLatestReportPayload(payload?.data));
setReportFallback(false);
setReportUnavailable("");
}
} catch {
if (!disposed) {
setLatestReport(buildFallbackLatestReport())
setReportFallback(true)
if (shouldUseLocalFallback("latestReport", runtime)) {
if (!disposed) {
setLatestReport(buildFallbackLatestReport());
setReportFallback(true);
setReportUnavailable("");
}
} else if (!disposed) {
setLatestReport(null);
setReportFallback(false);
setReportUnavailable(reportUnavailableNotice);
}
}
}
};
void loadModels()
void loadSubscriptionPlans()
void loadLatestReport()
void loadModels();
void loadSubscriptionPlans();
void loadLatestReport();
return () => {
disposed = true
chart?.dispose()
}
}, [])
disposed = true;
chart?.dispose();
};
}, []);
return (
<div className="dashboard">
@@ -212,56 +394,124 @@ function Dashboard() {
<div className="stat-label"></div>
</div>
</div>
{modelsFallback && (
<div className="runtime-warning" role="alert">
{modelsFallbackNotice}
</div>
)}
{modelsUnavailable && (
<div className="runtime-error" role="alert">
{modelsUnavailable}
</div>
)}
<div className="chart-container">
<div ref={chartRef} style={{ width: '100%', height: '400px' }} />
<div ref={chartRef} style={{ width: "100%", height: "400px" }} />
</div>
<section className="report-section">
<section className="report-section report-priority-section">
<div className="report-header">
<div>
<h3>📰 </h3>
<p>线</p>
<h3>📰 </h3>
<p>
</p>
</div>
{latestReport && (
<span className={`report-status ${latestReport.status === 'generated' ? 'report-status-generated' : 'report-status-other'}`}>
<span
className={`report-status ${latestReport.status === "generated" ? "report-status-generated" : "report-status-other"}`}
>
{latestReport.status}
</span>
)}
</div>
{latestReport ? (
<div className="report-card">
<div className="report-hero">
<div className="report-eyebrow"></div>
<div className="report-summary">{summarizeLatestReport(latestReport)}</div>
<div className="report-card report-priority-card">
<div className="report-hero report-hero-priority">
<div className="report-eyebrow"></div>
<div className="report-summary">
{summarizeLatestReport(latestReport)}
</div>
<div className="report-evidence">
{latestReport.pricingLeadNote}
</div>
</div>
<div className="report-meta">
<span> {latestReport.reportDate}</span>
{latestReport.modelCount > 0 && <span>{latestReport.modelCount} </span>}
{latestReport.updatedAt && <span> {latestReport.updatedAt}</span>}
{latestReport.modelCount > 0 && (
<span>{latestReport.modelCount} </span>
)}
{latestReport.updatedAt && (
<span> {latestReport.updatedAt}</span>
)}
</div>
<div className="report-highlights">
<div className="report-highlight">
<strong></strong>
<span> HTML Markdown </span>
{latestReport.headlines.length > 0 && (
<div className="report-news-grid">
{latestReport.headlines.map((item) => (
<article key={item.title} className="report-news-card">
<div className="report-news-label">{item.label}</div>
<div className="report-news-title">{item.title}</div>
<div className="report-news-summary">{item.summary}</div>
</article>
))}
</div>
<div className="report-highlight">
<strong></strong>
<span></span>
)}
{latestReport.themes.length > 0 && (
<div className="report-theme-grid">
{latestReport.themes.map((theme) => {
const badge = reportThemeBadge(theme.title);
return (
<article
key={theme.title}
className={`report-theme-card ${badge.tone}`}
>
<div className="report-theme-badge">
<span className="report-theme-badge-icon">{badge.icon}</span>
<span className="report-theme-badge-label">{badge.label}</span>
</div>
<div className="report-theme-title">{theme.title}</div>
<ul>
{theme.bullets.slice(0, 3).map((bullet) => (
<li key={bullet}>{bullet}</li>
))}
</ul>
</article>
);
})}
</div>
</div>
)}
<div className="report-actions">
<a className="report-link report-link-primary" href={latestReport.htmlUrl} target="_blank" rel="noreferrer">
<a
className="report-link report-link-primary"
href={latestReport.htmlUrl}
target="_blank"
rel="noreferrer"
>
HTML
</a>
<a className="report-link" href={latestReport.markdownUrl} target="_blank" rel="noreferrer">
<a
className="report-link"
href={latestReport.markdownUrl}
target="_blank"
rel="noreferrer"
>
Markdown
</a>
</div>
{reportFallback && (
<div className="report-note">使退</div>
<div className="report-note runtime-warning-inline">
{reportFallbackNotice}
</div>
)}
{reportUnavailable && (
<div className="report-note runtime-error-inline">
{reportUnavailable}
</div>
)}
</div>
) : (
<div className="subscription-empty"></div>
<div className="data-empty">
API
</div>
)}
</section>
<section className="subscription-section">
@@ -291,18 +541,28 @@ function Dashboard() {
</tr>
</thead>
<tbody>
{subscriptionPlans.map(plan => (
{subscriptionPlans.map((plan) => (
<tr key={plan.planCode}>
<td>
<div className="plan-name">{plan.planName}</div>
<div className="plan-meta">{plan.operatorCN || plan.operator}</div>
<div className="plan-meta">
{plan.operatorCN || plan.operator}
</div>
</td>
<td>¥{plan.listPrice.toFixed(2)}/</td>
<td>{formatSubscriptionQuota(plan.quotaValue, plan.quotaUnit)}</td>
<td>{plan.contextWindow > 0 ? `${Math.round(plan.contextWindow / 1024)}K` : '-'}</td>
<td>
{formatSubscriptionQuota(plan.quotaValue, plan.quotaUnit)}
</td>
<td>
{plan.contextWindow > 0
? `${Math.round(plan.contextWindow / 1024)}K`
: "-"}
</td>
<td>
<div>{plan.modelCount} </div>
{plan.modelPreview && <div className="plan-meta">{plan.modelPreview}</div>}
{plan.modelPreview && (
<div className="plan-meta">{plan.modelPreview}</div>
)}
</td>
</tr>
))}
@@ -311,7 +571,7 @@ function Dashboard() {
)}
</section>
</div>
)
);
}
export default Dashboard
export default Dashboard;

View File

@@ -1,148 +1,298 @@
import { useEffect, useMemo, useState } from 'react'
import { formatPrice, loadFallbackModels, normalizeModel, type Model } from '../lib/models'
import { useEffect, useMemo, useState } from "react";
import {
formatPrice,
loadFallbackModels,
normalizeModel,
type Model,
} from "../lib/models";
import {
buildApiUnavailableNotice,
buildFallbackNotice,
detectRuntimeEnvironment,
shouldUseLocalFallback,
} from "../lib/runtimeVisibility";
type SortField = 'name' | 'inputPrice' | 'outputPrice' | 'contextLength'
type SortOrder = 'asc' | 'desc'
type SortField = "name" | "inputPrice" | "outputPrice" | "contextLength";
const PAGE_SIZE = 5
type SortOrder = "asc" | "desc";
const PAGE_SIZE = 5;
function Explorer() {
const [models, setModels] = useState<Model[]>([])
const [loading, setLoading] = useState(true)
const [page, setPage] = useState(1)
const [sortField, setSortField] = useState<SortField>('inputPrice')
const [sortOrder, setSortOrder] = useState<SortOrder>('asc')
const [providerFilter, setProviderFilter] = useState<string>('')
const [modalityFilter, setModalityFilter] = useState<string>('')
const [models, setModels] = useState<Model[]>([]);
const [loading, setLoading] = useState(true);
const [page, setPage] = useState(1);
const [sortField, setSortField] = useState<SortField>("inputPrice");
const [sortOrder, setSortOrder] = useState<SortOrder>("asc");
const [providerFilter, setProviderFilter] = useState<string>("");
const [modalityFilter, setModalityFilter] = useState<string>("");
const [modelsFallback, setModelsFallback] = useState(false);
const [modelsUnavailable, setModelsUnavailable] = useState("");
const runtime = detectRuntimeEnvironment();
const fallbackNotice = buildFallbackNotice("models", runtime);
const unavailableNotice = buildApiUnavailableNotice("models", runtime);
useEffect(() => {
// 从API加载数据
fetch('/api/v1/models')
.then(r => r.json())
.then(data => {
const rawModels: any[] = Array.isArray(data?.data) ? data.data : []
fetch("/api/v1/models")
.then(async (r) => {
if (!r.ok) {
throw new Error(`models request failed: ${r.status}`);
}
return r.json();
})
.then((data) => {
const rawModels: any[] = Array.isArray(data?.data) ? data.data : [];
const normalized = rawModels
.map(normalizeModel)
.filter((model: Model | null): model is Model => model !== null)
setModels(normalized)
setLoading(false)
.filter((model: Model | null): model is Model => model !== null);
setModels(normalized);
setModelsFallback(false);
setModelsUnavailable("");
setLoading(false);
})
.catch(async () => {
// 降级:使用本地静态数据
const fallback = await loadFallbackModels()
setModels(fallback)
setLoading(false)
})
}, [])
if (shouldUseLocalFallback("models", runtime)) {
const fallback = await loadFallbackModels();
setModels(fallback);
setModelsFallback(fallback.length > 0);
setModelsUnavailable(fallback.length === 0 ? unavailableNotice : "");
} else {
setModels([]);
setModelsFallback(false);
setModelsUnavailable(unavailableNotice);
}
setLoading(false);
});
}, []);
// 动态提取厂商列表
const providers = useMemo(() => {
const set = new Set<string>()
models.forEach(m => {
if (m.providerCN && m.providerCN !== 'Unknown') {
set.add(m.providerCN)
const set = new Set<string>();
models.forEach((m) => {
if (m.providerCN && m.providerCN !== "Unknown") {
set.add(m.providerCN);
}
})
return Array.from(set).sort()
}, [models])
});
return Array.from(set).sort();
}, [models]);
// 排序+筛选
const filtered = useMemo(() => {
let result = [...models]
let result = [...models];
if (providerFilter) {
result = result.filter(m => m.providerCN === providerFilter)
result = result.filter((m) => m.providerCN === providerFilter);
}
if (modalityFilter) {
result = result.filter(m => m.modality === modalityFilter)
result = result.filter((m) => m.modality === modalityFilter);
}
result.sort((a, b) => {
const aVal = a[sortField]
const bVal = b[sortField]
if (typeof aVal === 'string') {
return sortOrder === 'asc'
const aVal = a[sortField];
const bVal = b[sortField];
if (typeof aVal === "string") {
return sortOrder === "asc"
? aVal.localeCompare(bVal as string)
: (bVal as string).localeCompare(aVal)
: (bVal as string).localeCompare(aVal);
}
return sortOrder === 'asc'
return sortOrder === "asc"
? (aVal as number) - (bVal as number)
: (bVal as number) - (aVal as number)
})
return result
}, [models, sortField, sortOrder, providerFilter, modalityFilter])
: (bVal as number) - (aVal as number);
});
return result;
}, [models, sortField, sortOrder, providerFilter, modalityFilter]);
const totalPages = Math.max(1, Math.ceil(filtered.length / PAGE_SIZE))
const paginated = filtered.slice((page - 1) * PAGE_SIZE, page * PAGE_SIZE)
const totalPages = Math.max(1, Math.ceil(filtered.length / PAGE_SIZE));
const paginated = filtered.slice((page - 1) * PAGE_SIZE, page * PAGE_SIZE);
const pricingFocus = filtered[0] ?? null;
const pricingBoard = filtered.slice(0, 3);
const toggleSort = (field: SortField) => {
if (sortField === field) {
setSortOrder(o => o === 'asc' ? 'desc' : 'asc')
setSortOrder((o) => (o === "asc" ? "desc" : "asc"));
} else {
setSortField(field)
setSortOrder('asc')
setSortField(field);
setSortOrder("asc");
}
setPage(1)
}
setPage(1);
};
if (loading) return <div className="loading">...</div>
if (loading) return <div className="loading">...</div>;
return (
<div className="explorer">
<h2>🔍 Explorer</h2>
<div className="explorer explorer-editorial">
<div className="explorer-hero">
<div>
<div className="explorer-kicker"></div>
<h2></h2>
<p></p>
</div>
<div className="explorer-hero-meta">
<span> {filtered.length}</span>
<span> {providers.length}</span>
</div>
</div>
<div className="filters">
<select value={providerFilter} onChange={e => { setProviderFilter(e.target.value); setPage(1) }}>
<div className="filters filters-editorial">
<select
value={providerFilter}
onChange={(e) => {
setProviderFilter(e.target.value);
setPage(1);
}}
>
<option value=""></option>
{providers.map(p => <option key={p} value={p}>{p}</option>)}
{providers.map((p) => (
<option key={p} value={p}>
{p}
</option>
))}
</select>
<select value={modalityFilter} onChange={e => { setModalityFilter(e.target.value); setPage(1) }}>
<select
value={modalityFilter}
onChange={(e) => {
setModalityFilter(e.target.value);
setPage(1);
}}
>
<option value=""></option>
<option value="text"></option>
<option value="multimodal"></option>
</select>
<span className="count"> {filtered.length} </span>
</div>
{modelsFallback && (
<div className="runtime-warning" role="alert">
{fallbackNotice}
</div>
)}
{modelsUnavailable && (
<div className="runtime-error" role="alert">
{modelsUnavailable}
</div>
)}
<table className="model-table">
<thead>
<tr>
<th onClick={() => toggleSort('name')}> {sortField === 'name' && (sortOrder === 'asc' ? '▲' : '▼')}</th>
<th></th>
<th></th>
<th onClick={() => toggleSort('inputPrice')}> {sortField === 'inputPrice' && (sortOrder === 'asc' ? '▲' : '▼')}</th>
<th onClick={() => toggleSort('outputPrice')}> {sortField === 'outputPrice' && (sortOrder === 'asc' ? '▲' : '▼')}</th>
<th onClick={() => toggleSort('contextLength')}> {sortField === 'contextLength' && (sortOrder === 'asc' ? '▲' : '▼')}</th>
<th></th>
</tr>
</thead>
<tbody>
{paginated.map(m => (
<tr key={m.id} className={`${m.isFree ? 'free' : ''} ${m.stale ? 'stale' : ''}`.trim()}>
<td>
<div className="model-name">{m.name || m.id}</div>
<div className="model-id">{m.id}</div>
</td>
<td>{m.providerCN || m.provider}</td>
<td>
<span className={`status-badge ${m.stale ? 'status-stale' : 'status-fresh'}`}>
{m.stale ? 'stale' : m.dataConfidence}
</span>
</td>
<td>{formatPrice(m, 'input')}</td>
<td>{formatPrice(m, 'output')}</td>
<td>{(m.contextLength / 1000).toFixed(0)}K</td>
<td>{m.modality}</td>
</tr>
{pricingFocus && (
<section className="pricing-focus-card">
<div className="pricing-focus-header">
<div>
<div className="explorer-kicker"></div>
<h3>{pricingFocus.name || pricingFocus.id}</h3>
<p>
{pricingFocus.providerCN || pricingFocus.provider} ·{" "}
{pricingFocus.modality} ·{" "}
{(pricingFocus.contextLength / 1000).toFixed(0)}K
</p>
</div>
<span
className={`status-badge ${pricingFocus.stale ? "status-stale" : "status-fresh"}`}
>
{pricingFocus.stale ? "stale" : pricingFocus.dataConfidence}
</span>
</div>
<div className="pricing-focus-prices">
<div>
<span></span>
<strong>{formatPrice(pricingFocus, "input")}</strong>
</div>
<div>
<span></span>
<strong>{formatPrice(pricingFocus, "output")}</strong>
</div>
</div>
</section>
)}
{pricingBoard.length > 0 && (
<section className="pricing-board-grid">
{pricingBoard.map((model) => (
<article key={model.id} className="pricing-board-card">
<div className="pricing-board-title">
{model.name || model.id}
</div>
<div className="pricing-board-meta">
{model.providerCN || model.provider} · {model.modality}
</div>
<div className="pricing-board-price">
{formatPrice(model, "input")} / {formatPrice(model, "output")}
</div>
</article>
))}
</tbody>
</table>
</section>
)}
{paginated.length === 0 ? (
<div className="data-empty"></div>
) : (
<table className="model-table model-table-editorial">
<thead>
<tr>
<th onClick={() => toggleSort("name")}>
{sortField === "name" && (sortOrder === "asc" ? "▲" : "▼")}
</th>
<th></th>
<th></th>
<th onClick={() => toggleSort("inputPrice")}>
{" "}
{sortField === "inputPrice" &&
(sortOrder === "asc" ? "▲" : "▼")}
</th>
<th onClick={() => toggleSort("outputPrice")}>
{" "}
{sortField === "outputPrice" &&
(sortOrder === "asc" ? "▲" : "▼")}
</th>
<th onClick={() => toggleSort("contextLength")}>
{" "}
{sortField === "contextLength" &&
(sortOrder === "asc" ? "▲" : "▼")}
</th>
<th></th>
</tr>
</thead>
<tbody>
{paginated.map((m) => (
<tr
key={m.id}
className={`${m.isFree ? "free" : ""} ${m.stale ? "stale" : ""}`.trim()}
>
<td>
<div className="model-name">{m.name || m.id}</div>
<div className="model-id">{m.id}</div>
</td>
<td>{m.providerCN || m.provider}</td>
<td>
<span
className={`status-badge ${m.stale ? "status-stale" : "status-fresh"}`}
>
{m.stale ? "stale" : m.dataConfidence}
</span>
</td>
<td>{formatPrice(m, "input")}</td>
<td>{formatPrice(m, "output")}</td>
<td>{(m.contextLength / 1000).toFixed(0)}K</td>
<td>{m.modality}</td>
</tr>
))}
</tbody>
</table>
)}
<div className="pagination">
<button disabled={page === 1} onClick={() => setPage(p => p - 1)}></button>
<span> {page} / {totalPages} </span>
<button disabled={page === totalPages} onClick={() => setPage(p => p + 1)}></button>
<button disabled={page === 1} onClick={() => setPage((p) => p - 1)}>
</button>
<span>
{page} / {totalPages}
</span>
<button
disabled={page === totalPages}
onClick={() => setPage((p) => p + 1)}
>
</button>
</div>
</div>
)
);
}
export default Explorer
export default Explorer;

View File

@@ -4,19 +4,43 @@ package retry
import (
"context"
"errors"
"fmt"
"io"
"math"
"net"
"strings"
"time"
)
type temporaryError interface {
Temporary() bool
}
type timeoutError interface {
Timeout() bool
}
type HTTPStatusError struct {
StatusCode int
Body string
}
func (e HTTPStatusError) Error() string {
if e.Body == "" {
return fmt.Sprintf("http status %d", e.StatusCode)
}
return fmt.Sprintf("http status %d: %s", e.StatusCode, e.Body)
}
// Strategy 重试策略
type Strategy struct {
MaxRetries int // 最大重试次数0=不重试)
BaseDelay time.Duration // 基础延迟
MaxDelay time.Duration // 最大延迟上限
Multiplier float64 // 乘数默认2.0
Jitter bool // 是否添加随机抖动
Retryable func(error) bool // 判断错误是否可重试
MaxRetries int // 最大重试次数0=不重试)
BaseDelay time.Duration // 基础延迟
MaxDelay time.Duration // 最大延迟上限
Multiplier float64 // 乘数默认2.0
Jitter bool // 是否添加随机抖动
Retryable func(error) bool // 判断错误是否可重试
}
// DefaultStrategy 返回默认重试策略
@@ -31,13 +55,66 @@ func DefaultStrategy() Strategy {
}
}
// IsRetryable 默认重试判定:网络错误、超时、5xx状态码等可重试
// IsRetryable 默认重试判定:仅临时网络错误、429、5xx 等可重试
func IsRetryable(err error) bool {
if err == nil {
return false
}
// 这里可以扩展更多错误类型判定
return true
if errors.Is(err, context.Canceled) || errors.Is(err, context.DeadlineExceeded) {
return false
}
var statusErr HTTPStatusError
if errors.As(err, &statusErr) {
return statusErr.StatusCode == 429 || (statusErr.StatusCode >= 500 && statusErr.StatusCode < 600)
}
if errors.Is(err, io.EOF) || errors.Is(err, io.ErrUnexpectedEOF) {
return true
}
message := strings.ToLower(err.Error())
if strings.Contains(message, "json 解析失败") ||
strings.Contains(message, "invalid character") ||
strings.Contains(message, "unmarshal") ||
strings.Contains(message, "decode") ||
strings.Contains(message, "schema") {
return false
}
var tempErr temporaryError
if errors.As(err, &tempErr) && tempErr.Temporary() {
return true
}
var timeoutErr timeoutError
if errors.As(err, &timeoutErr) && timeoutErr.Timeout() {
return true
}
var netErr net.Error
if errors.As(err, &netErr) && (netErr.Timeout() || netErr.Temporary()) {
return true
}
retriableMarkers := []string{
"transport closed",
"connection reset",
"connection refused",
"tls handshake timeout",
"i/o timeout",
"no such host",
"temporarily unavailable",
"too many requests",
"rate limit",
}
for _, marker := range retriableMarkers {
if strings.Contains(message, marker) {
return true
}
}
return false
}
// Do 执行带重试的操作

View File

@@ -4,10 +4,22 @@ package retry
import (
"context"
"errors"
"fmt"
"io"
"net"
"net/http"
"testing"
"time"
)
func alwaysRetry(error) bool {
return true
}
func neverRetry(error) bool {
return false
}
func TestDo_Success(t *testing.T) {
strategy := DefaultStrategy()
callCount := 0
@@ -32,7 +44,7 @@ func TestDo_RetryThenSuccess(t *testing.T) {
MaxDelay: 100 * time.Millisecond,
Multiplier: 2.0,
Jitter: false,
Retryable: IsRetryable,
Retryable: alwaysRetry,
}
callCount := 0
@@ -59,7 +71,7 @@ func TestDo_MaxRetriesExceeded(t *testing.T) {
MaxDelay: 50 * time.Millisecond,
Multiplier: 2.0,
Jitter: false,
Retryable: IsRetryable,
Retryable: alwaysRetry,
}
callCount := 0
expectedErr := errors.New("persistent error")
@@ -84,7 +96,7 @@ func TestDo_NonRetryableError(t *testing.T) {
MaxDelay: 100 * time.Millisecond,
Multiplier: 2.0,
Jitter: false,
Retryable: func(err error) bool { return false }, // 任何错误都不重试
Retryable: neverRetry, // 任何错误都不重试
}
callCount := 0
@@ -101,6 +113,48 @@ func TestDo_NonRetryableError(t *testing.T) {
}
}
func TestIsRetryableRejectsContextCancellation(t *testing.T) {
if IsRetryable(context.Canceled) {
t.Fatal("context.Canceled should not be retryable")
}
if IsRetryable(context.DeadlineExceeded) {
t.Fatal("context.DeadlineExceeded should not be retryable")
}
}
func TestIsRetryableRejectsPermanentHTTPStatus(t *testing.T) {
err := HTTPStatusError{StatusCode: http.StatusForbidden, Body: "forbidden"}
if IsRetryable(err) {
t.Fatal("403 should not be retryable")
}
}
func TestIsRetryableAllowsTemporaryNetworkAndServerErrors(t *testing.T) {
err := HTTPStatusError{StatusCode: http.StatusBadGateway, Body: "bad gateway"}
if !IsRetryable(err) {
t.Fatal("502 should be retryable")
}
netErr := &net.DNSError{IsTemporary: true}
if !IsRetryable(netErr) {
t.Fatal("temporary network errors should be retryable")
}
}
func TestIsRetryableRejectsJSONParseErrors(t *testing.T) {
err := errors.New("JSON 解析失败: invalid character")
if IsRetryable(err) {
t.Fatal("JSON parse errors should not be retryable")
}
}
func TestIsRetryableAllowsUnexpectedEOF(t *testing.T) {
err := fmt.Errorf("transport closed: %w", io.ErrUnexpectedEOF)
if !IsRetryable(err) {
t.Fatal("unexpected EOF should be retryable")
}
}
func TestDo_ContextCancellation(t *testing.T) {
strategy := Strategy{
MaxRetries: 3,
@@ -108,7 +162,7 @@ func TestDo_ContextCancellation(t *testing.T) {
MaxDelay: 5 * time.Second,
Multiplier: 2.0,
Jitter: false,
Retryable: IsRetryable,
Retryable: alwaysRetry,
}
ctx, cancel := context.WithTimeout(context.Background(), 50*time.Millisecond)
@@ -138,7 +192,7 @@ func TestDoWithResult(t *testing.T) {
MaxDelay: 50 * time.Millisecond,
Multiplier: 2.0,
Jitter: false,
Retryable: IsRetryable,
Retryable: alwaysRetry,
}
callCount := 0
@@ -168,7 +222,7 @@ func TestDoWithMetrics(t *testing.T) {
MaxDelay: 100 * time.Millisecond,
Multiplier: 2.0,
Jitter: false,
Retryable: IsRetryable,
Retryable: alwaysRetry,
}
// 成功场景

View File

@@ -10,7 +10,7 @@
---
## 当前未修复问题速查表(截至 2026-05-24 19:05
## 当前未修复问题速查表(截至 2026-05-27 15:10
| # | 问题 | 优先级 | 首次暴露 | 修复状态 | 影响次数 |
|---|------|--------|----------|----------|----------|
@@ -23,10 +23,10 @@
| 7 | 文件修改后未触发 commit 提示 | P2→P1 | 05-08 09:05 | ❌ 未修复 | 14 次 |
| 8 | cron review 无 delta 时空转 | P1 | 05-08 09:12 | ❌ 未修复 | 13 次 |
| 9 | 验证模式伪进展artifact_present 局限) | P1 | 05-08 14:30 | ❌ 未修复 | 10 次 |
| 10 | 项目提交停滞commit stagnation | P0 | 05-08 21:30 | ⚠️ 重新活跃(工作区变更量已增至 19 文件 +933核心组件改动未入版本控制) | 21 次 |
| 10 | 项目提交停滞commit stagnation | P0 | 05-08 21:30 | ⚠️ 重新活跃(23 文件 +3650/-808 行核心组件改动未入版本控制BACKLOG 本身也在未提交列表中 | 23 次 |
| 11 | review 报告未触发修复动作 | P2→P1 | 05-08 21:30 | ❌ 未修复 | 10 次 |
| 12 | BACKLOG 文件膨胀导致 review 成本递增 | P1 | 05-09 09:30 | ⚠️ 部分(已分层归档,但 current 表仍持续膨胀) | 8 次 |
| 13 | untracked 核心代码未入版本控制 | P0 | 05-10 21:30 | ⚠️ 重新活跃(CoreHub 相关未跟踪代码已缓解,但仍有长期未治理的非业务 untracked 项) | 13 次 |
| 13 | untracked 核心代码未入版本控制 | P0 | 05-10 21:30 | ⚠️ 重新活跃(scripts/secret_gate_lib.sh/test.sh 为新增 untracked 项) | 14 次 |
| 14 | Phase 6+ 范围未定义 | P1 | 05-10 21:30 | ❌ 未修复 | 6 次 |
| 15 | review 误报传播 | P1 | 05-11 14:30 | ❌ 未修复 | 10 次 |
| 16 | untracked 文件统计遗漏 | P1 | 05-11 14:30 | ⚠️ 部分(本轮已更精确核对 git status但能力未固化 | 6 次 |
@@ -40,8 +40,8 @@
| 24 | 长命令部分回传时缺少保守结论模板 | P1 | 05-15 21:31 | ⚠️ 部分(本轮通过 process 拿到完整输出,但策略尚未固化) | 2 次 |
| 25 | backlog current truth 老化未自动撤销 | P2 | 05-16 09:30 | ❌ 未修复 | 2 次 |
| 26 | 外部 provider 失败与主链路失败聚合过粗 | P1 | 05-16 09:30 | ⚠️ 部分Cloudflare 已加 transport fallback但其他外部源仍缺统一分层 | 6 次 |
| 27 | 稳定性窗口虽已分类但缺 release 级解释语义 | P1 | 05-16 09:30 | ⚠️ 部分(本轮已把 Cloudflare EOF 定性为 recovered external incident不再按持续 parser blocker 叙述 | 7 次 |
| 28 | 新增导入器缺少进入综合验收前的 smoke gate | P0 | 05-16 15:10 | ✅ 已缓解(仓库已存在 `verify_importer_smoke.sh`,且持续通过) | 4 次 |
| 27 | 稳定性窗口虽已分类但缺 release 级解释语义 | P1 | 05-16 09:30 | ⚠️ 部分Cloudflare EOF 定性为 recovered external incident但 release 文案模板尚未系统化 | 7 次 |
| 28 | 新增导入器缺少进入综合验收前的 smoke gate | P0 | 05-16 15:10 | ✅ 已缓解(`verify_importer_smoke.sh` 持续通过,本轮 importer smoke 全 PASS | 4 次 |
| 29 | 同日 review blocker 切换缺少自动老化提醒 | P1 | 05-16 15:10 | ❌ 未修复 | 2 次 |
| 30 | 历史 precondition 样本持续老化拖低 release 成功率 | P1 | 05-17 09:31 | ❌ 未修复 | 6 次 |
| 31 | 同日无主结论 delta 时缺少风险老化优先策略 | P2 | 05-17 15:10 | ❌ 未修复 | 3 次 |
@@ -49,198 +49,215 @@
| 33 | 已证伪 blocker 缺少自动降级/撤销机制 | P1 | 05-18 09:30 | ❌ 未修复 | 2 次 |
| 34 | 局部 smoke 已通过后缺少全局 blocker 切换提示 | P1 | 05-18 15:10 | ❌ 未修复 | 1 次 |
| 35 | smoke gate 测试脚本老化未跟上 runtime truth | P1 | 05-19 09:32 | ✅ 已修复(`importer_smoke_gate_test.sh` 已与 runtime truth 对齐并持续通过) | 5 次 |
| 36 | 稳定性窗口持续回落85.71% → 71.43% | P1 | 05-20 21:06 | ✅ 已恢复(`verify_phase6.sh` 本轮 17/17 PASS窗口回到 100% | 2 次 |
| 36 | 稳定性窗口持续回落85.71% → 71.43% | P1 | 05-20 21:06 | ✅ 已恢复(窗口回到 100%,本轮 importer smoke 全 PASS | 2 次 |
| 37 | 外部文档站故障仍无系统化降级 | P1 | 05-16 09:30 | ❌ 未修复live_run SUMMARY 缺失,无法确认当前 blocker 状态) | 6 次 |
| 38 | PRE_PHASE6_RESULT 标签冲突verify_phase4 FAIL 但标签仍 PASS | P1 | 05-25 08:51 | ❌ 未修复verify_phase4 ECharts 断言失败是唯一 FAIL 项,根因为断言与实现不匹配) | 4 次 |
| 39 | 日报时间戳异常generated_at 晚约 10 小时) | P2 | 05-25 08:51 | ❌ 未修复 | 3 次 |
| 40 | BACKLOG 文件本身 uncommitted | P1 | 05-25 08:51 | ❌ 未修复BACKLOG 本轮也在未提交列表中) | 4 次 |
| 41 | verify_phase6.sh 连续超时导致 Phase 6 状态无法确认 | P1 | 05-25 09:06 | ⚠️ 部分连续超时未复现importer smoke 全 PASS但 live_run SUMMARY 仍缺失,窗口状态不明) | 5 次 |
| 42 | verify_phase6.sh 第三次连续超时 | P0 | 05-25 15:10 | ✅ 已修复连续超时未在本轮复现importer smoke 全 PASS | — |
| 43 | verify_phase4 ECharts 集成断言失败(历史遗留 P2 | P2 | 05-25 15:10 | ❌ 未修复Dashboard.tsx 已引入 echarts 但 verify 断言与实现不匹配,导致 PRE_PHASE6 FAIL | 2 次 |
| 44 | 新增 scripts 无门禁覆盖secret_gate_lib.sh / secret_gate_test.sh | P2 | 05-26 15:10 | ❌ 未修复(新增文件为 untracked无对应 verify 门禁验证其正确性) | 1 次 |
| 45 | scripts 目录 go test build failureredeclared main | P1 | 05-27 15:10 | ❌ 未修复(多个脚本存在 main/ModelPricing/logger redeclared 冲突,导致 `go test ./scripts` 无法执行) | 1 次 |
---
## Review 日志
### 2026-05-24 18:18main 收尾复核
### 2026-05-27 15:10afternoon-review cron
> **前置说明**本轮不是 cron review而是上线前收尾复核。前序工作已完成 importer 分组提交、三远端推送和本地 gate。复核目标是确认“已上传”之后的真实上线门禁是否也收敛
> **前置说明**距上一次 review05-26 15:10约 **24 小时**。无新 commit。工作区从 22/+2819/-466 行扩大至 23/+3650/-808 行。scripts 新增 1619 行(主要是 generate_daily_report.go +1032 行及其测试 +567 行。importer smoke 16 PASS 持续。ECharts FAIL 持续 2+ 天。scripts 目录 go test 出现 redeclared main build failure新增 P1 gap
#### 本次新增发现
- **Phase 6 已恢复通过**`bash scripts/verify_phase6.sh` 输出 `SUMMARY pass=17 fail=0 warn=0`,此前 Cloudflare `EOF` live blocker 未复现,真实复跑链路恢复
- **稳定性窗口恢复到 100%**:最近 7 次采集样本全部成功,`success_rate=100.00%``precondition_missing=0`
- **runtime / smoke / docs 三层重新对齐**`run_real_pipeline.sh``verify_importer_smoke.sh``importer_smoke_gate_test.sh``pipeline_runtime_alignment_test.sh` 全部通过
- **versioned truth 已收敛**:当前 `main` 已包含 importer/runtime/docs/execution truth同步到 `origin` / `tksea` / `gitea`,工作区干净
- **工作区扩大至 23/+3650/-808 行**scripts 新增 1619 行generate_daily_report.go +1032 行、generate_daily_report_test.go +567 行frontend 新增 ~834 行Dashboard.tsx +534 行、Explorer.tsx +342 行cmd/server 新增 ~535 行main.go +274 行、main_test.go +261 行)
- **scripts 目录 go test build failure**多个脚本fetch_openrouter.go、fetch_multi_source.go、generate_daily_report.go、fetch_tencent_catalog.go、export_official_seed_json.go、cloudflare_pricing_signature_guard.go存在 `main redeclared``ModelPricing redeclared``logger redeclared` 冲突,导致 `go test ./scripts` build FAIL。但 `go build ./cmd/server` 成功,不影响主服务构建
- **importer smoke 16 PASS 持续**verify_importer_smoke.sh 全 PASS采集链路健康
- **verify_phase4 ECharts FAIL 持续**:已持续 2+ 天,唯一 FAIL 项是 `[FAIL] Dashboard 已集成 ECharts`
#### 问题 35 状态更新smoke gate truth 已对齐
#### 问题 45新发现scripts 目录 go test build failureredeclared main
- **18:18 状态**`importer_smoke_gate_test.sh` 当前通过,不再错误断言 live smoke 失败
- **结论**:从“脚本老化”更新为“已修复”
#### 问题 36 状态更新:稳定性窗口已恢复
- **18:18 状态**:窗口成功率已从此前的 71.43% / 85.71% 恢复到 100%。
- **结论**:从“已回升”更新为“已恢复”,当前不再构成 release blocker。
#### 后续仍需跟踪
- 历史 blocker 已消失后board / backlog / execution truth 的自动老化与撤销机制仍不足(问题 20 / 25 / 33 / 34 继续成立)。
- 外部文档源仍存在瞬时网络抖动风险,后续应继续区分“网络瞬断”与“真实结构漂移”。
### 2026-05-24 19:05main 文档真相同步)
> **前置说明**18:18 复核后继续下钻 Cloudflare `EOF` 现场,目标不是重复宣布“已恢复”,而是确认是否需要把它继续当作 active blocker 写在板子上。
#### 本次新增发现
- **Cloudflare 更接近代理传输层瞬时失败,不是稳定 parser 回归**:同一路径在代理开启、关闭、以及禁用 HTTP/2 场景下均无法稳定复现旧 `EOF`
- **已落地最小加固**抓取公共逻辑新增“proxy transport error → direct retry”兜底提交为 `2a64160 fix(pricing): fallback to direct fetch after proxy transport errors`
- **加固已被真实验证覆盖**`TestFetchRawPricingPageFallsBackWithoutProxyOnRetriableProxyFailure` 通过;坏代理环境下 live `cloudflare_pricing_signature_guard` 也通过。
- **release truth 应同步降噪**Cloudflare 不应继续在 current board/backlog 中被表述为“parser 仍损坏”的活跃 blocker更准确的表述是“外部链路瞬时失败已恢复且传输层兜底已补齐”。
#### 问题 26 / 27 状态更新:从“纯未修复”调整为“已有局部收敛”
- **19:05 状态**Cloudflare 这一路径已具备 recovered-external-incident 的事实基础,并已加 transport fallback但 Perplexity / Vertex 等其他外部源还没有统一的 retry / release 叙事模板。
- **结论**:问题 26、27 不再适合保持“完全未修复”表述,更新为“⚠️ 部分”。
#### 后续仍需跟踪
- 把 recovered-external-incident / active-code-regression 的 release 文案模板系统化,而不是只在 Cloudflare 个案里人工判断。
- 继续观察其他外部文档源是否也需要同类 transport fallback 或更明确的分层统计。
### 2026-05-20 21:30第 37 次 reviewnight-review cron
> **前置说明**:距上一次 review05-20 21:06约 **24 分钟**。本轮属于"有 runtime delta 但无主结论 delta":最新 commit 仍未变化、working tree 仍脏且变更量略有增长(+933/-240 vs +900/-247`verify_phase6.sh` 的 live blocker 继续是 Perplexity 外部文档签名校验超时。关键 delta稳定性窗口从 `71.43%` 回升到 `85.71%`precondition_missing 从 2 降回 1。
#### 本次新增发现
- **稳定性窗口回升**:本轮 `verify_phase6.sh` 输出 `success_count=6 failure_count=1 success_rate=85.71`,较 21:06 的 71.43% 有所改善。原因是本轮 review 触发的新一次 verify 运行产生了最新成功样本(`2026-05-20 21:33:29`),滚动窗口替换掉一个旧失败样本。
- **当前 Phase 6 继续 FAIL主 live blocker 未变化**`live_run_result=FAIL` 仍由 `perplexity_pricing_signature_guard` 拉取 `https://docs.perplexity.ai/docs/agent-api/models.md` 超时触发。
- **新增导入器 smoke gate 继续通过**`coreshub-fixture``coreshub-live``ctyun-fixture``ctyun-live` 全部 PASS。
- **工作区变更量略有增长**+933/-240 行 vs 上轮 +900/-247 行19 文件仍未提交。
- **smoke gate 测试脚本老化仍未消除**`importer_smoke_gate_test.sh` 仍断言 ctyun live smoke 应失败,与 runtime 冲突持续(同问题 35
#### 问题 36 状态更新:稳定性窗口回升
- **21:30 状态**:窗口从 71.43% 回到 85.71%,与 05-19 各轮一致。21:06 的 71.43% 是短期波动而非持续恶化趋势。
- **问题影响**:窗口门禁仍 FAIL85.71% < 95%但不再恶化precondition_missing 样本回到 1 个。
- **优化建议**:继续观察下轮是否稳定在 85.71% 或继续波动;若持续低于 95%,需考虑窗口策略调整(如排除 precondition_missing 类样本单独报告)。
- **15:10 状态**`go test ./scripts` 输出大量 `main redeclared in this block``ModelPricing/logger redeclared` 错误。涉及脚本包括 fetch_openrouter.go、fetch_multi_source.go、generate_daily_report.go、fetch_tencent_catalog.go、export_official_seed_json.go、cloudflare_pricing_signature_guard.go 等。这些脚本在同一 main package 中共享符号
- **问题影响**`go test ./scripts` 无法执行scripts 目录的单元测试链路断裂;但 `go build ./cmd/server` 不受影响,主服务可正常构建
- **优化建议**
1. 为 scripts 目录下的各脚本添加 `// +build ignore` build tag 或移至独立包,使每个脚本可独立构建
2. 或者在 go test 命令中使用 `go test -tags ignore` 配合 build tag 排除冲突脚本
3. 或者将共享类型ModelPricing、logger移至 internal/common 包,各脚本独立引用
- **优先级**P1
- **建议验证方法**下轮 review 观察窗口成功率是否稳定或继续波动
- **建议验证方法**修复后执行 `go test ./scripts` 无 build error`go test -tags llm_script ./scripts` 全 PASS
#### 问题 10 持续活跃:项目提交停滞
#### 问题 10 状态更新:项目提交停滞(影响次数 23
- **21:30 状态**工作区变更量从 +900 行增长到 +933 行19 文件仍未提交
- **问题影响**同 21:06 reviewversioned truth 与 runtime truth 持续漂移,且漂移量在增大
- **优化建议**同 21:06 review尽快按逻辑拆分为 2~3 个 commit。
- **15:10 状态**23 文件 +3650/-808 行核心组件改动未提交,含 generate_daily_report.go +1032 行大改、main_test.go +261 行、前端 Dashboard +534 行等关键业务代码
- **问题影响**versioned truth 与 runtime truth 漂移加剧scripts build failure 在 commit 前必须修复
- **优化建议**立即按逻辑拆分为 2~3 个 commit(如"server 重构与测试"、"前端 Dashboard/Explorer 扩展"、"日报生成器大改"scripts build failure 需在 commit 前解决
- **优先级**P0
- **建议验证方法**提交后检查 `git log --oneline` 出现新提交`git diff --stat HEAD` 大幅收缩。
- **建议验证方法**修复 scripts build failure 后提交`git diff --stat HEAD` 变更量大幅收缩。
### 2026-05-20 21:06第 36 次 reviewmorning-review cron
#### 问题 41 状态更新live_run SUMMARY 缺失(影响次数 5
> **前置说明**:距上一次 review05-19 21:30约 **23.5 小时**。本轮有 runtime delta稳定性窗口从 `85.71%` 回落到 `71.43%`,新增一次 precondition_missing 失败样本。工作区变更量显著增大19 文件、+900 行),涉及 CoreHub 导入器全套实现、天翼云订阅库扩展、日报生成器改进、验证脚本增强等,但全部未提交收敛
- **15:10 状态**verify_phase6.sh 在 30s 内退出,未输出 window_size / success_rate / live_run_result SUMMARY。连续超时问题已解决连续第三次不超时但 live_run SUMMARY 仍缺失
- **问题影响**Phase 6 稳定性窗口 PASS/FAIL 状态无法通过脚本输出确认(但 importer smoke 全 PASS 说明采集链路健康)。
- **优化建议**:同 05-26 15:10 记录。
- **优先级**P1从 P0 降级,本轮连续超时未复现)
- **建议验证方法**:修正后执行 verify_phase6.sh确认输出完整 SUMMARY。
#### 问题 43 状态更新verify_phase4 ECharts FAIL影响次数 2
- **15:10 状态**verify_phase4 ECharts 断言失败已持续 2+ 天,本轮无变化。
- **结论**:影响次数从 1 更新为 2 次。
### 2026-05-26 15:10afternoon-review cron
> **前置说明**:距上一次 review05-25 15:10约 **24 小时**。本轮距上次 afternoon review 无新 commit工作区变更从 19 文件 +1372/-281 行增长到 22 文件 +2819/-466 行。verify_phase6.sh 连续超时问题(本轮跨三次 review 的 05-25 记录本轮首次解决importer smoke 全 PASS但 live_run SUMMARY 仍缺失。PRE_PHASE6 FAILverify_phase4 ECharts 断言失败。go test 全 PASS。
#### 本次新增发现
- **稳定性窗口进一步回落**`verify_phase6.sh` 输出 `success_count=5 failure_count=2 success_rate=71.43 threshold=95 precondition_missing=2`相比上轮6/7=85.71%)新增一次 precondition_missing 失败(`2026-05-20 08:00:01` 严格真实模式下未提供 API Key
- **工作区变更量显著增大**`git diff --stat HEAD` 显示 19 文件、+900/-247 行变更,涉及 CoreHub 导入器(`coreshub_pricing_lib.go` +81、`import_coreshub_pricing.go` +88、`import_coreshub_pricing_test.go` +64、`coreshub_pricing_sample.txt` +10、天翼云订阅库`ctyun_subscription_lib.go` +201、日报生成器`generate_daily_report.go` +78/-)、验证脚本(`verify_phase6.sh` +115/-)等核心组件
- **当前 Phase 6 继续 FAIL主 live blocker 未变化**`live_run_result=FAIL` 仍由 `perplexity_pricing_signature_guard` 外部超时触发
- **新增导入器 smoke gate 继续通过**`coreshub-fixture``coreshub-live``ctyun-fixture``ctyun-live` 全部 PASS
- **smoke gate 测试脚本老化仍未消除**`importer_smoke_gate_test.sh` 仍断言 ctyun live smoke 应失败,与 runtime 冲突持续。
- **verify_phase6.sh 连续超时问题本轮消失**:本轮执行 `timeout 60 bash scripts/verify_phase6.sh` 在 60s 内完成importer smoke 8 组全 PASScoreshub/huawei-maas/baichuan/lingyiwanwu/sensenova/xfyun/bytedance 各 fixture+live PASSgate PASS。但 live_run 仅触发 smokerun脚本在 60s 内退出,**未输出 window_size / success_rate / live_run_result SUMMARY**
- **PRE_PHASE6 FAIL根因是 verify_phase4 ECharts 断言失败**`verify_pre_phase6.sh``PRE_PHASE6_RESULT: FAIL`,唯一 FAIL 项是 `[FAIL] Dashboard 已集成 ECharts`。Phase 1 PASS(9/9)、Phase 2 PASS(9/9)、Phase 3 PASS(17/17)、Phase 5 PASS(15/15)
- **工作区变更量增长**22 文件 +2819/-466 行(含 cmd/server BasicAuth 重构 +261 行测试、main_test.go +261 行、前端 Dashboard/Explorer +876 行、日报生成器 +229/- 行BACKLOG 本身也在未提交列表中
- **新增 untracked 项**scripts/secret_gate_lib.sh1846 字节、scripts/secret_gate_test.sh1823 字节、scripts/testdata/empty.dockerignore19 字节)、.agent/、.serena/、.dockerignore均无门禁覆盖
#### 问题 10 重新活跃:项目提交停滞(commit stagnation
#### 问题 10 状态更新:项目提交停滞(影响次数 22
- **21:06 状态**工作区变更量已从"长期轻度漂移"升级为"19 文件 +900 行实质性核心改动未提交"
- **问题影响**
- 大量核心组件改动CoreHub 导入器、天翼云订阅库、日报生成器、验证脚本)未入版本控制,一旦工作区丢失则无法恢复
- versioned truth 与 runtime truth 严重漂移review/backlog 失真风险加剧
- 新导入器代码已具备测试和 fixture但不属于任何 commit无法追溯
- **优化建议**
1. 尽快按逻辑拆分为 2~3 个 commit如 CoreHub 导入器、天翼云订阅库扩展、日报/验证改进)
2. 在 review prompt 中增加"工作区变更量超过阈值时自动提升 commit 停滞优先级"的规则
3. 考虑在 cron review 中增加自动 commit 提醒或辅助 commit 功能
- **15:10 状态**22 文件 +2819/-466 行核心组件改动未提交,含 cmd/server BasicAuth/IP 限速/apiError 重构、main_test.go +261 行、前端 Dashboard/Explorer 大改(+534/-、+342/- 行)、日报生成器(+229/- 行。BACKLOG 本身也在未提交列表中
- **问题影响**versioned truth 与 runtime truth 漂移加剧一旦工作区丢失则核心组件改动无法恢复BACKLOG 持续未收敛使 review 成本递增。
- **优化建议**:立即按逻辑拆分为 2~3 个 commit如"server 重构与测试"、"前端 Dashboard/Explorer 扩展"、"日报生成器与门禁改进"review prompt 应在工作区变更量超过阈值时自动提升 commit 停滞优先级。
- **优先级**P0
- **建议验证方法**:提交后检查 `git log --oneline` 出现新提交,`git diff --stat HEAD` 大幅收缩。
- **建议验证方法**:提交后检查 `git log --oneline` 出现新提交,`git diff --stat HEAD` 变更量大幅收缩。
#### 问题 30 / 36 持续活跃:历史 precondition 样本持续老化 + 窗口回落
#### 问题 41 状态更新:从"连续超时"降级为"live_run SUMMARY 缺失"(影响次数 4
- **21:06 状态**precondition_missing 样本从 1 增至 2窗口成功率从 85.71% 降至 71.43%
- **问题影响**
- 窗口门禁持续 FAIL且失败样本在增长
- 若继续叠加 precondition_missing 样本,窗口成功率会进一步下降
- 历史纪律问题持续拖累 release 结论
- **15:10 状态**连续超时未在本轮复现importer smoke 全 PASSgate PASS但 live_run SUMMARYwindow_size / success_rate / live_run_result仍未输出脚本在 smokerun 后 60s 内退出
- **问题影响**Phase 6 稳定性窗口 PASS/FAIL 状态无法确认;无法判断 05-25 的三次超时是外部文档站卡死还是脚本性能退化。
- **优化建议**
1. 考虑为稳定性窗口增加"新鲜度"权重,降低历史 precondition 样本的影响
2. 或者在窗口计算中排除 precondition_missing 类样本,单独报告环境纪律问题
- **优先级**P1
- **建议验证方法**:观察下轮 review 窗口成功率是否继续回落;若持续下降则需调整窗口策略。
1. 调查 verify_phase6.sh live_run 未输出完整 SUMMARY 的根因60s 内退出但未打印 window / success_rate / live_run_result
2. 为 verify_phase6.sh 增加单次检查的独立超时控制,避免单次检查卡死导致整脚本超时
3. 在 verify_phase6.sh 输出中增加"当前检查进度"标记
- **优先级**P0 → P1本轮 importer smoke 全 PASS 说明不是持续卡死,但 live_run SUMMARY 缺失仍是 P1
- **建议验证方法**:修正后执行 verify_phase6.sh确认能在 <120s 内输出完整 SUMMARY含 window_size / success_rate / live_run_result
### 2026-05-19 21:30第 35 次 reviewnight-review
#### 问题 42 状态更新:已修复(从 backlog current 表移除
> **前置说明**:距上一次 review05-19 15:10约 **6 小时 20 分钟**。本轮属于"有现场变更但无主结论 delta":最新 commit 仍未变化、working tree 仍脏,`verify_phase6.sh` 的 live blocker 继续是 Perplexity 外部文档签名校验超时,稳定性窗口也继续停在 `85.71% FAIL`
- **15:10 状态**verify_phase6.sh 连续超时未在本轮复现importer smoke 全 PASS。05-25 的三次连续超时更接近外部文档站临时卡死而非脚本性能退化
- **结论**:问题 42 从 current 表移除,归档至 review 日志。
#### 本次新增发现
#### 问题 43新发现verify_phase4 ECharts 集成断言失败(历史遗留 P2
- **当前 Phase 6 继续 FAIL主 live blocker 未变化**`verify_phase6.sh` 再次完整输出 `PHASE_RESULT: FAIL`,其中 `live_run_result=FAIL` 仍由 `perplexity_pricing_signature_guard` 拉取 `https://docs.perplexity.ai/docs/agent-api/models.md` 超时触发
- **新增导入器 smoke gate 继续不是当前 blocker**`coreshub-fixture``coreshub-live``ctyun-fixture``ctyun-live` 全部通过,`importer_smoke_gate_result=PASS`
- **稳定性窗口继续 FAIL但失败仍不是采集器运行时失败**:最近 7 次样本维持 `success_count=6 failure_count=1 success_rate=85.71 threshold=95 precondition_missing=1 external_provider_failure=0 collector_runtime_failure=0 unknown_failure=0`,说明 release 结论仍持续受历史前置条件纪律影响。
- **测试脚本与 runtime truth 冲突仍未消除**`scripts/importer_smoke_gate_test.sh` 依然断言"当前 live ctyun smoke 应失败",与本轮 `ctyun-live` PASS 继续冲突。
- **night 相对 afternoon 无主结论 delta**:最新 commit 未变化,主 blocker 未切换,窗口门禁口径也未变化;当前更该关注风险老化与未提交变更,而不是重复全量完成项。
#### 问题 18 / 31 持续活跃:无 delta 场景缺少老化风险优先策略
- **21:30 状态**:本轮相对 15:10 没有新的主 blocker也没有新的通过证据但 review 仍需要重复大部分相同检查,系统不会自动把重点切换到"风险老化、未提交变更、未验证项持续存在"。
- **问题影响**
- 高频 review 容易机械重复完成项清单,降低信息密度
- 读者不容易一眼看到"night 相对 afternoon 其实无主结论 delta"
- 会弱化 review 对长期未收敛风险的追踪能力
- **15:10 状态**`[FAIL] Dashboard 已集成 ECharts` 是 verify_phase4 的唯一 FAIL 项。Dashboard.tsx 中已引入 `import * as echarts from 'echarts'` `echarts.init()` 逻辑,但 verify 脚本断言逻辑与实际代码行为不匹配
- **问题影响**:导致 PRE_PHASE6 整体 FAIL但不影响主采集链路Phase 1/2/3 全 PASSimporter smoke 全 PASS历史遗留问题首现于 05-25 15:10 systematic review
- **优化建议**
1. 在 review prompt 或模板中增加更强的 delta gate相对上一轮无主结论变化时强制输出"无 delta"并把重点转向风险老化与未提交变更
2. 在 backlog current 表中为持续性 blocker 增加 `last_reverified_at` / `current_as_of` 语义,减少重复展开背景
3. 对同日多轮 review 默认生成"变化摘要"而不是重复全量完成项,除非 blocker 真正切换
1. 更新 verify_phase4 中 ECharts 集成断言逻辑,使其与当前 Dashboard.tsx 的 echarts 使用方式一致
2. 或者确认当前代码是否真正满足"已集成 ECharts"语义,若不满足则完成集成
3. 考虑将 ECharts 相关断言降级为 WARNING 而非 FAIL以区分"历史遗留 P2"与"真实 blocker"
- **优先级**P2
- **建议验证方法**构造同一天两次 review 现场与 runtime 结论基本一致的场景,检查新模板是否会自动突出"无 delta、重点看风险老化/工作区收敛"
- **建议验证方法**`bash scripts/verify_phase4.sh` → SUMMARY pass=10 fail=0 warn=0PRE_PHASE6_RESULT: PASS
### 2026-05-19 15:10第 34 次 reviewafternoon-review
#### 问题 44新发现新增 scripts 无门禁覆盖
> **前置说明**:距上一次 review05-19 09:32约 **5 小时 38 分钟**。本轮基本属于"有现场变更但无主结论 delta":最新 commit 仍未变化、working tree 仍脏,`verify_phase6.sh` 的 live blocker 继续是 Perplexity 外部文档签名校验超时,稳定性窗口也继续停在 `85.71% FAIL`
#### 本次新增发现
- **当前 Phase 6 继续 FAIL主 live blocker 未变化**`verify_phase6.sh` 再次完整输出 `PHASE_RESULT: FAIL`,其中 `live_run_result=FAIL` 仍由 `perplexity_pricing_signature_guard` 拉取 `https://docs.perplexity.ai/docs/agent-api/models.md` 超时触发。
- **新增导入器 smoke gate 继续不是当前 blocker**`coreshub-fixture``coreshub-live``ctyun-fixture``ctyun-live` 全部通过,`importer_smoke_gate_result=PASS`
- **稳定性窗口继续 FAIL但失败仍不是采集器运行时失败**:最近 7 次样本维持 `success_count=6 failure_count=1 success_rate=85.71 threshold=95 precondition_missing=1 external_provider_failure=0 collector_runtime_failure=0 unknown_failure=0`,说明 release 结论仍持续受历史前置条件纪律影响。
- **同日 afternoon 相对 morning 无主结论 delta**:最新 commit 未变化,主 blocker 未切换,窗口门禁口径也未变化;当前更该关注风险老化与未提交变更,而不是重复全量完成项。
#### 问题 18 / 31 持续活跃:无 delta 场景缺少老化风险优先策略
- **15:10 状态**:本轮相对 09:32 没有新的主 blocker也没有新的通过证据但 review 仍需要重复大部分相同检查,系统不会自动把重点切换到"风险老化、未提交变更、未验证项持续存在"。
- **问题影响**
- 高频 review 容易机械重复完成项清单,降低信息密度
- 读者不容易一眼看到"下午相对早上其实无主结论 delta"
- 会弱化 review 对长期未收敛风险的追踪能力
- **15:10 状态**scripts/secret_gate_lib.sh1846 字节、scripts/secret_gate_test.sh1823 字节、scripts/testdata/empty.dockerignore 为新增 untracked 项,无对应 verify 门禁验证其正确性
- **问题影响**:新增安全类脚本无法确认是否正确落地;一旦工作区切换或代码丢失,这些脚本的存在和正确性无法追溯。
- **优化建议**
1. 在 review prompt 或模板中增加更强的 delta gate相对上一轮无主结论变化时强制输出"无 delta"并把重点转向风险老化与未提交变更
2. 在 backlog current 表中为持续性 blocker 增加 `last_reverified_at` / `current_as_of` 语义,减少重复展开背景
3. 对同日多轮 review 默认生成"变化摘要"而不是重复全量完成项,除非 blocker 真正切换
1. 为 secret_gate_lib.sh / secret_gate_test.sh 建立对应的 smoke gate 或单元测试门禁
2. 考虑在 verify_phase5 或 verify_phase6 中增加对新 scripts 目录的覆盖检查
- **优先级**P2
- **建议验证方法**构造同一天两次 review 现场与 runtime 结论基本一致的场景,检查新模板是否会自动突出"无 delta、重点看风险老化/工作区收敛"
- **建议验证方法**执行 `bash scripts/secret_gate_test.sh` 验证其正确性,并确认门禁已纳入综合验收
### 2026-05-19 09:32第 33 次 reviewmorning-review
#### 问题 13 状态更新untracked 核心代码重新活跃(影响次数 14
> **前置说明**:距上一次 review05-18 21:32约 **12 小时**。本轮不是"无 delta":最新 commit 仍未变化、working tree 仍脏runtime 上当前 live blocker 也未切换,仍是 Perplexity 外部文档签名校验超时,但稳定性窗口从昨晚 `100% PASS` 回落到 `85.71% FAIL`,且唯一失败类型继续是 `precondition_missing_only`
- **15:10 状态**scripts/secret_gate_lib.sh / secret_gate_test.sh 为新增 untracked 安全类脚本BACKLOG 本身也在未提交列表中;.agent/、.serena/ 等目录长期未治理
- **问题影响**:同问题 10untracked 列表持续增长增加了 versioned truth 漂移风险。
- **优化建议**:同问题 10尽快提交工作区变更清理非必要 untracked 项。
- **优先级**P0
- **建议验证方法**:提交后 `git status --short` 中 untracked 列表显著收缩。
#### 问题 38 状态更新PRE_PHASE6_RESULT 标签冲突(影响次数 4
- **15:10 状态**verify_phase4 ECharts 断言失败导致 PRE_PHASE6 FAIL但 verify_phase4 内部 SUMMARY 显示 pass=9 fail=1 warn=0说明是单一断言失败而非系统性卡死。
- **问题影响**PRE_PHASE6 FAIL 的根因已明确为 verify_phase4 ECharts 断言问题(历史 P2不影响主链路但标签冲突使 reviewer 需要额外下钻才能判断真实阶段。
- **优化建议**:将 verify_phase4 中的 ECharts 相关断言降级为 WARNING或更新断言逻辑使其与当前 Dashboard.tsx echarts 使用方式一致。
- **优先级**P1
- **建议验证方法**verify_phase4 中 ECharts 断言修复后PRE_PHASE6_RESULT 应回到 PASS。
### 2026-05-25 15:10afternoon-review cron第 41 次 review
> **前置说明**:距上一次 review05-25 08:59约 **6 小时 11 分钟**。本轮无新 deltaworking tree 仍 19 文件未提交(与 08:59 systematic review 完全一致),无新 commit。verify_phase6.sh 第三次连续超时09:06 morning → 09:06 systematic → 15:10 afternoonPhase 6 live blocker 状态完全无法确认。Phase 1~5 PASSgo test 全 PASS日报已生成但所有 systematic review 修复落地项(.dockerignore、runtimeVisibility、BasicAuth、Explorer.tsx 部分修复)均未 commit。
#### 本次新增发现
- **当前 Phase 6 继续 FAIL主 live blocker 未变化**`verify_phase6.sh` 再次完整输出 `PHASE_RESULT: FAIL`,其中 `live_run_result=FAIL` 仍由 `perplexity_pricing_signature_guard` 拉取 `https://docs.perplexity.ai/docs/agent-api/models.md` 超时触发
- **新增导入器 smoke gate 已明确不是当前 blocker**`coreshub-fixture``coreshub-live``ctyun-fixture``ctyun-live` 全部通过,`importer_smoke_gate_result=PASS`
- **稳定性窗口再次回落,但失败仍不是采集器运行时失败**:最近 7 次样本为 `success_count=6 failure_count=1 success_rate=85.71 threshold=95 precondition_missing=1 external_provider_failure=0 collector_runtime_failure=0 unknown_failure=0`,说明 release 结论仍受历史前置条件纪律影响
- **smoke gate 测试脚本已与当前 runtime truth 冲突**`scripts/importer_smoke_gate_test.sh` 仍写着 `expected current live ctyun smoke to fail before full gate`,并断言 `ctyun-live` 应 FAIL但本轮真实 `verify_phase6.sh` `ctyun-live` 已 PASS
- **verify_phase6.sh 第三次连续超时**:本轮执行 `timeout 180 bash scripts/verify_phase6.sh`>200s 无输出连续第三次09:06 morning / 09:06 systematic / 15:10 afternoon。Phase 6 live blocker 状态Zhipu 403 是否仍活跃、是否已消失或切换到新外部源)完全无法确认
- **Phase 1~5 门禁全 PASS**`verify_pre_phase6.sh` 输出 `PRE_PHASE6_RESULT: PASS`SUMMARY pass=15 fail=0 warn=0与历史一致
- **Working tree 状态与 08:59 systematic review 完全一致**19 文件 +1372/-281 行仍未提交,包含 .dockerignore、runtimeVisibility.ts、BasicAuth 实现、Explorer.tsx 部分修复等 systematic review 所有 P0/P1 修复落地项
- **systematic review P0-3 修复已落地但未 commit**`.dockerignore` 已创建285 字节12:03 创建artifact-present`frontend/src/lib/runtimeVisibility.ts` + `runtimeVisibility.test.ts` 已创建
- **Explorer.tsx fallback 修复尚未完整验证**runtimeVisibility.ts 已就绪但 Explorer.tsx 中只引入了部分 notice 构建逻辑,未完全实现"禁止静默 fallback"的 P0-2 修复目标。
- **整体项目状态无新 delta**:距上次 review 6+ 小时,无新 commit无新 runtime 证据主链路健康API 200日报已生成
#### 问题 35P1smoke gate 测试脚本老化未跟上 runtime truth
#### 问题 42新发现verify_phase6.sh 第三次连续超时Phase 6 live blocker 状态完全不明
- **09:32 状态**`scripts/importer_smoke_gate_test.sh` 仍把"ctyun live smoke 应失败"当作当前预期,而本轮 runtime 已直接证实 `ctyun-live` PASS
- **15:10 状态**连续三次 verify_phase6.sh 超时09:06 morning / 09:06 systematic / 15:10 afternoon均无法在 180s 内完成并输出 Phase 6 SUMMARY。这不是偶发性问题而是持续性卡死——可能存在外部文档站持续卡死或脚本本身性能退化
- **问题影响**
- 测试脚本会传播已失效 blocker削弱 smoke gate 验证本身的可信度
- reviewer 容易把过时测试预期误当 current truth
- 会让"导入器 smoke gate 已准入"与"测试仍宣称应失败"同时存在,制造文档/实现/验证三层冲突
- Phase 6 综合门禁 PASS/FAIL 完全不明,连续三次 review 均无法给出准确的阶段判断
- 无法确认 Zhipu 403 blocker 是否仍活跃、是否已消失还是切换到新的外部源
- 外部文档站可能存在新的持续卡死,需要立即调查超时根因
- **优化建议**
1. 立即更新 `importer_smoke_gate_test.sh` 断言,使其反映当前 smoke gate 真实行为
2.这类"当前预期"测试增加 `last_reverified_at` 或显式注释,避免历史临时预期长期固化
3.review 模板中加入"测试脚本是否仍与当前 runtime truth 一致"的检查项
1. 调查 verify_phase6.sh 超时根因:单次外部文档站卡死 vs 整体脚本性能退化
2. verify_phase6.sh 增加单次检查的独立超时控制,避免单次检查卡死导致整脚本超时
3.verify_phase6.sh 输出中增加"当前检查进度"标记,方便定位卡死环节
4. 在 verify_phase6.sh 中为连续超时的外部 URL 建立快速失败策略
- **优先级**P0
- **建议验证方法**:修正后执行 verify_phase6.sh确认能在 <120s 内完成并输出完整 SUMMARY含 window_size / success_rate / live_run_result
#### 问题 40 状态更新:优先级升级,影响次数更新
- **15:10 状态**:问题 40 自 08:51 首现,已持续 6+ 小时未解决working tree 仍包含 systematic review 所有 P0/P1 修复落地项。优先级从 P2 升级为 P1因为现在包含 P0 修复落地项的未 commit 风险);影响次数从 2 更新为 3 次。
- **结论**:优先级从 P2 升级为 P1影响次数从 2 更新为 3 次。
#### 问题 38 状态更新PRE_PHASE6_RESULT 标签冲突仍待系统性修复
- **15:10 状态**:问题 38 影响次数从 2 更新为 3 次。PRE_PHASE6_RESULT 标签逻辑本身仍未系统性修复。
- **结论**:影响次数从 2 更新为 3 次。
#### 问题 39 状态更新:日报时间戳异常仍未修复
- **15:10 状态**:问题 39 影响次数从 2 更新为 3 次。generated_at 仍显示 2026-05-25T19:03:55+08:00比实际时间晚约 10 小时,与 08:51 / 08:59 记录一致。
- **结论**:影响次数从 2 更新为 3 次。
### 2026-05-25 09:06night-review cron第 40 次 review
> **前置说明**:距上一次 review05-25 08:59约 **7 分钟**。本轮属于"无新 delta 且 verify_phase6.sh 异常超时":无新 commitPhase 1~5 门禁仍全 PASS但 verify_phase6.sh 连续两次执行超时(>180s导致 Phase 6 live blocker 状态无法确认。BACKLOG 文件 uncommitted 已持续 75 分钟+08:51 → 08:59 → 09:06
#### 本次新增发现
- **verify_phase6.sh 连续两次超时**:本轮 review 两次执行 `bash scripts/verify_phase6.sh`,第一次在 90s 内完成了前 30 个 importer smoke 全 PASS 但未输出最终 SUMMARY第二次直接超时>180s 无法完成。Phase 6 live blocker 状态Zhipu 403 是否仍活跃)无法本轮真实验证。
- **Phase 1~5 门禁仍然全 PASS**`verify_pre_phase6.sh` 输出 `PRE_PHASE6_RESULT: PASS`,与上一轮一致,无变化。
- **BACKLOG 文件 uncommitted 已持续 75 分钟+**:问题 40 从 08:51 首现08:59 仍存在09:06 仍未解决,已跨三轮 review 无收敛动作。
- **日报时间戳异常仍未改善**`daily_report_2026-05-25.md``generated_at: 2026-05-25T19:03:55+08:00` 比实际时间09:06晚约 10 小时,与 08:51 / 08:59 记录一致。
#### 问题 41新发现verify_phase6.sh 连续超时导致 Phase 6 live blocker 状态无法确认
- **09:06 状态**:本轮 review 连续两次执行 `bash scripts/verify_phase6.sh`,均无法在合理时间内完成。第一次在前 90s 内完成了 30 个 importer smoke 全 PASS 但未输出最终 SUMMARY第二次直接超时>180s 无法完成)。
- **问题影响**
- Phase 6 综合门禁 PASS/FAIL 状态无法确认reviewer 无法给出准确的阶段判断
- 上一轮08:59记录的 Zhipu 403 blocker 是否仍活跃、是否已切换,本轮无法验证
- 超时可能与 Zhipu 403 或其他外部文档站卡死有关,需要调查根因
- **优化建议**
1. 调查 verify_phase6.sh 超时根因:单次外部文档站拉取卡死 vs 整体脚本性能退化
2. 为 verify_phase6.sh 增加单次检查的独立超时控制,避免单次检查卡死导致整脚本超时
3. 在 verify_phase6.sh 输出中增加"当前检查进度"标记,方便定位卡死环节
- **优先级**P1
- **建议验证方法**:修正脚本后运行该测试与 `verify_phase6.sh`;确认脚本断言与当前 smoke gate 输出一致,不再要求 `ctyun-live` 失败。
- **建议验证方法**:修正后执行 verify_phase6.sh,确认能在 <120s 内完成并输出完整 SUMMARY含 window_size / success_rate / live_run_result
#### 问题 37 状态更新:外部文档站故障仍无系统化降级
- **09:06 状态**:问题 37 仍活跃,影响次数从 3 更新为 4 次。本轮 verify_phase6 超时可能与外部文档站卡死有关(可能是 Zhipu 403 或其他源blocker 在不同外部源之间游走的模式持续。
- **结论**:从"3 次"更新为"4 次"。
#### 问题 39 状态更新:日报时间戳异常仍未改善
- **09:06 状态**generated_at 仍显示 2026-05-25T19:03:55+08:00比实际时间晚约 10 小时,无修复动作。
- **结论**:影响次数从 1 更新为 2 次。
#### 问题 40 状态更新BACKLOG uncommitted 已持续 75 分钟+
- **09:06 状态**:问题 40 已从 08:51 首现morning review 修改 BACKLOG 后未 commit08:59 仍存在09:06 仍未解决,跨三轮 review 无收敛动作。
- **结论**:影响次数从 1 更新为 2 次。

View File

@@ -215,7 +215,7 @@ func fetchModels(cfg Config) ([]ModelInfo, error) {
if resp.StatusCode != http.StatusOK {
body, _ := io.ReadAll(resp.Body)
lastErr = fmt.Errorf("非 200 响应: %d %s", resp.StatusCode, string(body))
lastErr = retry.HTTPStatusError{StatusCode: resp.StatusCode, Body: string(body)}
return lastErr
}
@@ -287,6 +287,38 @@ func parseModels(raw []byte) ([]ModelInfo, error) {
return models, nil
}
func deriveModality(model ModelInfo) string {
for _, capability := range model.Capabilities {
normalized := strings.ToLower(capability)
switch {
case strings.Contains(normalized, "vision"), strings.Contains(normalized, "image"):
return "multimodal"
case strings.Contains(normalized, "audio"):
return "audio"
case strings.Contains(normalized, "video"):
return "video"
case strings.Contains(normalized, "code"):
return "code"
}
}
hints := strings.ToLower(strings.Join([]string{model.ID, model.Name, model.Description}, " "))
switch {
case strings.Contains(hints, "video") && (strings.Contains(hints, "omni") || strings.Contains(hints, "vision") || strings.Contains(hints, "multimodal")):
return "multimodal"
case strings.Contains(hints, "vision") || strings.Contains(hints, "image") || strings.Contains(hints, "vl") || strings.Contains(hints, "omni") || strings.Contains(hints, "multimodal"):
return "multimodal"
case strings.Contains(hints, "audio") || strings.Contains(hints, "speech") || strings.Contains(hints, "voice"):
return "audio"
case strings.Contains(hints, "video"):
return "video"
case strings.Contains(hints, "code"):
return "code"
default:
return "text"
}
}
func getString(m map[string]any, key string) string {
if v, ok := m[key].(string); ok {
return v
@@ -443,7 +475,7 @@ func summarizeDB(connStr string, models []ModelInfo, batchSize int) error {
`,
"openrouter", m.ID, m.Name, m.Description, m.ContextLength,
jsonCapabilities(m.Capabilities), m.Created, isFree, "active",
rawPayload(m), providerID, "", "text",
rawPayload(m), providerID, "", deriveModality(m),
"official", now, batchID, collectorVersion,
"https://openrouter.ai/api/v1/models", now).Scan(&modelID)
if err != nil {

View File

@@ -9,6 +9,8 @@ import (
"os"
"path/filepath"
"testing"
"llm-intelligence/internal/retry"
)
// Test 1: parseModels 正确解析 name、context_length、capabilities、pricing input/prompt 和 output/completion
@@ -48,6 +50,10 @@ func TestParseModels(t *testing.T) {
if m.Pricing.Output != 10.0 {
t.Errorf("Pricing.Output 错误: %f", m.Pricing.Output)
}
if modality := deriveModality(m); modality != "multimodal" {
t.Errorf("deriveModality = %q, want %q", modality, "multimodal")
}
// 第二条pricing 用 prompt/completion 别名回退
m2 := models[1]
@@ -65,6 +71,68 @@ func TestParseModels(t *testing.T) {
}
}
func TestDeriveModality(t *testing.T) {
tests := []struct {
name string
capabilities []string
want string
}{
{name: "vision first", capabilities: []string{"vision", "json_mode"}, want: "multimodal"},
{name: "audio", capabilities: []string{"audio_generation"}, want: "audio"},
{name: "code", capabilities: []string{"code_interpreter"}, want: "code"},
{name: "text fallback", capabilities: []string{"function_calling"}, want: "text"},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
if got := deriveModality(ModelInfo{Capabilities: tt.capabilities}); got != tt.want {
t.Fatalf("deriveModality() = %q, want %q", got, tt.want)
}
})
}
}
func TestDeriveModalityInfersFromModelIdentityWithoutCapabilities(t *testing.T) {
tests := []struct {
name string
model ModelInfo
want string
}{
{
name: "omni id maps to multimodal",
model: ModelInfo{
ID: "nvidia/nemotron-3-nano-omni-30b-a3b-reasoning:free",
Description: "accepts text, image, video, and audio inputs",
},
want: "multimodal",
},
{
name: "audio id maps to audio",
model: ModelInfo{
ID: "openai/gpt-audio",
Description: "audio model for natural sounding voices",
},
want: "audio",
},
{
name: "vl id maps to multimodal",
model: ModelInfo{
ID: "qwen/qwen3-vl-32b-instruct",
Description: "vision-language model for text, images, and video",
},
want: "multimodal",
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
if got := deriveModality(tt.model); got != tt.want {
t.Fatalf("deriveModality(%+v) = %q, want %q", tt.model, got, tt.want)
}
})
}
}
// Test 2: run 无 API Key 时写入临时文件JSON 含 total 和 models 字段
func TestRunNoAPIKey(t *testing.T) {
tmpDir := t.TempDir()
@@ -108,6 +176,60 @@ func TestFetchModelsFailsInStrictRealModeWithoutAPIKey(t *testing.T) {
}
}
func TestFetchModelsDoesNotRetryPermanentHTTPErrors(t *testing.T) {
attempts := 0
server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
attempts++
http.Error(w, "forbidden", http.StatusForbidden)
}))
defer server.Close()
_, err := fetchModels(Config{
APIKey: "test-key",
APIURL: server.URL,
MaxRetries: 3,
TimeoutSec: 1,
StrictReal: true,
})
if err == nil {
t.Fatal("expected fetchModels to fail on 403")
}
if attempts != 1 {
t.Fatalf("expected 1 attempt for permanent HTTP error, got %d", attempts)
}
}
func TestFetchModelsRetriesServerErrors(t *testing.T) {
attempts := 0
server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
attempts++
if attempts < 3 {
http.Error(w, "temporary", http.StatusBadGateway)
return
}
w.Header().Set("Content-Type", "application/json")
_, _ = w.Write([]byte(`{"data":[{"id":"openai/gpt-4o","name":"GPT-4o","context_length":128000,"pricing":{"input":2.5,"output":10.0}}]}`))
}))
defer server.Close()
models, err := fetchModels(Config{
APIKey: "test-key",
APIURL: server.URL,
MaxRetries: 3,
TimeoutSec: 1,
StrictReal: true,
})
if err != nil {
t.Fatalf("expected retry success, got %v", err)
}
if len(models) != 1 {
t.Fatalf("expected 1 model, got %d", len(models))
}
if attempts != 3 {
t.Fatalf("expected 3 attempts for temporary server error, got %d", attempts)
}
}
func TestRunFailsInStrictRealModeWhenDBWriteFails(t *testing.T) {
tmpDir := t.TempDir()
outPath := filepath.Join(tmpDir, "models.json")
@@ -130,3 +252,12 @@ func TestRunFailsInStrictRealModeWhenDBWriteFails(t *testing.T) {
t.Fatal("strict real mode should fail when database write fails")
}
}
func TestRetryHTTPStatusErrorClassification(t *testing.T) {
if retry.IsRetryable(retry.HTTPStatusError{StatusCode: http.StatusForbidden}) {
t.Fatal("403 should not be retryable")
}
if !retry.IsRetryable(retry.HTTPStatusError{StatusCode: http.StatusBadGateway}) {
t.Fatal("502 should be retryable")
}
}

File diff suppressed because it is too large Load Diff

View File

@@ -3,6 +3,7 @@
package main
import (
"fmt"
"os"
"path/filepath"
"strings"
@@ -161,6 +162,45 @@ func sampleReportForV1() *ReportV3 {
}
}
func TestBuildModelSelectionsSeparatesTopListsFromAppendixLists(t *testing.T) {
intlModels := []ModelInfo{
{Name: "intl-1", InputPrice: 0.1, Currency: "USD"},
{Name: "intl-2", InputPrice: 0.2, Currency: "USD"},
{Name: "intl-3", InputPrice: 0.3, Currency: "USD"},
{Name: "intl-4", InputPrice: 0.4, Currency: "USD"},
{Name: "intl-5", InputPrice: 0.5, Currency: "USD"},
{Name: "intl-6", InputPrice: 0.6, Currency: "USD"},
}
var domesticModels []ModelInfo
for i := 0; i < 14; i++ {
domesticModels = append(domesticModels, ModelInfo{
Name: fmt.Sprintf("domestic-%02d", i+1),
InputPrice: float64(i + 1),
Currency: "CNY",
ContextLength: 65536,
})
}
selections := buildModelSelections(intlModels, domesticModels, nil)
if len(selections.IntlTop5) != 5 {
t.Fatalf("expected intl top5 length 5, got %d", len(selections.IntlTop5))
}
if len(selections.IntlAppendixList) != 6 {
t.Fatalf("expected intl appendix length 6, got %d", len(selections.IntlAppendixList))
}
if len(selections.DomesticTop10) != 10 {
t.Fatalf("expected domestic top10 length 10, got %d", len(selections.DomesticTop10))
}
if len(selections.DomesticAppendixList) != 14 {
t.Fatalf("expected domestic appendix length 14, got %d", len(selections.DomesticAppendixList))
}
if selections.DomesticTop10[0].Name != "domestic-01" || selections.DomesticTop10[9].Name != "domestic-10" {
t.Fatalf("domestic top10 ordering mismatch: first=%s tenth=%s", selections.DomesticTop10[0].Name, selections.DomesticTop10[9].Name)
}
if selections.DomesticAppendixList[13].Name != "domestic-14" {
t.Fatalf("domestic appendix should preserve full list, got tail=%s", selections.DomesticAppendixList[13].Name)
}
}
func TestBuildFreeSourceBreakdown(t *testing.T) {
report := sampleReportForV1()
@@ -357,12 +397,14 @@ func TestDecorateReportV1BuildsHotDaySummary(t *testing.T) {
EventType: "price_cut",
ModelName: "qwen-vl-max",
ProviderName: "Alibaba",
ProviderCountry: "CN",
OperatorName: "DashScope",
TrustLabel: "官方来源",
Baseline: "较昨日 -18%",
Summary: "价格下降已足以影响视觉模型默认选择。",
SourceKindLabel: "价格快照",
PrimarySource: "pricing_history",
SourceURL: "https://dashscope.aliyun.com/model/qwen-vl-max",
UpdatedAt: "2026-05-13 10:00",
EvidenceDetail: "pricing_history 记录到输入价格较昨日下降 18%",
PriceChangePct: -18,
@@ -375,9 +417,10 @@ func TestDecorateReportV1BuildsHotDaySummary(t *testing.T) {
if report.PageMode != "hot" {
t.Fatalf("expected hot page mode, got %q", report.PageMode)
}
if !strings.Contains(report.HeroSummary, "GLM-5 已出现官方发布信号") {
t.Fatalf("hero summary missing official release signal: %s", report.HeroSummary)
if !strings.Contains(report.HeroSummary, "qwen-vl-max") || !strings.Contains(report.HeroSummary, "CN / Alibaba / DashScope") || !strings.Contains(report.HeroSummary, "价格下降") {
t.Fatalf("hero summary should prioritize price change signal with org metadata, got %s", report.HeroSummary)
}
if len(report.ActionItems) != 3 {
t.Fatalf("expected 3 action items, got %d", len(report.ActionItems))
}
@@ -387,9 +430,14 @@ func TestDecorateReportV1BuildsHotDaySummary(t *testing.T) {
if report.ActionItems[0].Evidence == "" {
t.Fatalf("expected action item evidence to be populated")
}
if !strings.Contains(report.HeadlineItems[0].Title, "GLM-5") {
t.Fatalf("expected first headline to prioritize official release, got %+v", report.HeadlineItems[0])
if report.HeadlineItems[0].ProviderCountry == "" {
t.Fatalf("expected headline item country metadata, got %+v", report.HeadlineItems[0])
}
if report.HeadlineItems[0].Label != "价格下调" || !strings.Contains(report.HeadlineItems[0].Title, "qwen-vl-max") {
t.Fatalf("expected first headline to prioritize price cut, got %+v", report.HeadlineItems[0])
}
}
func TestDecorateReportV1BuildsCalmDaySummary(t *testing.T) {
@@ -403,6 +451,7 @@ func TestDecorateReportV1BuildsCalmDaySummary(t *testing.T) {
}
if !strings.Contains(report.HeroSummary, "稳定") {
t.Fatalf("expected calm day summary to emphasize stability, got %s", report.HeroSummary)
}
}
@@ -446,12 +495,14 @@ func TestGenerateMarkdownV3IncludesTencentSubscriptionSection(t *testing.T) {
EventType: "official_release",
ModelName: "GLM-5",
ProviderName: "Zhipu",
ProviderCountry: "CN",
OperatorName: "Zhipu",
TrustLabel: "官方来源 / 一级证据",
Baseline: "官方首次发布",
Summary: "官方发布新模型,值得优先复查中文通用与推理场景默认选择。",
SourceKindLabel: "一级官方发布",
PrimarySource: "https://open.bigmodel.cn/dev/howuse/model",
SourceURL: "https://open.bigmodel.cn/dev/howuse/model",
UpdatedAt: "2026-05-13 08:30",
EvidenceDetail: "models.release_date = 今日,且 source_url 指向官方文档",
Priority: 120,
@@ -474,6 +525,7 @@ func TestGenerateMarkdownV3IncludesTencentSubscriptionSection(t *testing.T) {
EventType: "new_model",
ModelName: "DeepSeek-V4-Flash",
ProviderName: "DeepSeek",
ProviderCountry: "CN",
OperatorName: "OpenRouter",
Audience: "适合想尽快验证新模型价值的选型读者",
TrustLabel: "聚合来源",
@@ -481,6 +533,7 @@ func TestGenerateMarkdownV3IncludesTencentSubscriptionSection(t *testing.T) {
Summary: "新模型进入情报池,值得重新评估低成本编码默认选择。",
SourceKindLabel: "模型快照",
PrimarySource: "OpenRouter / region_pricing",
SourceURL: "https://openrouter.ai/models/deepseek/deepseek-v4-flash",
UpdatedAt: "2026-05-13 09:30",
EvidenceDetail: "models.created_at = 今日,且已存在最新价格快照",
Priority: 95,
@@ -500,6 +553,25 @@ func TestGenerateMarkdownV3IncludesTencentSubscriptionSection(t *testing.T) {
EvidenceDetail: "官方活动页记录 V3.2-Exp 在活动窗口内价格下调 50%+",
Priority: 115,
},
{
EventType: "price_cut",
ModelName: "qwen-vl-max",
ProviderName: "Alibaba",
ProviderCountry: "CN",
OperatorName: "DashScope",
Audience: "适合需要当天重排价格带的团队",
TrustLabel: "官方来源",
Baseline: "较昨日 -18%",
Summary: "视觉模型价格下降已足以影响默认选型。",
SourceKindLabel: "价格快照",
PrimarySource: "pricing_history",
SourceURL: "https://dashscope.aliyun.com/model/qwen-vl-max",
UpdatedAt: "2026-05-13 10:00",
EvidenceDetail: "pricing_history 记录到输入价格较昨日下降 18%",
PriceChangePct: -18,
Priority: 100,
},
}
decorateReportV1(report)
@@ -511,26 +583,31 @@ func TestGenerateMarkdownV3IncludesTencentSubscriptionSection(t *testing.T) {
if err != nil {
t.Fatalf("read markdown output: %v", err)
}
content := string(body)
if !strings.Contains(content, "### 国内模型 TOP 10") {
t.Fatalf("markdown missing domestic top10 heading\n%s", content)
}
content = string(body)
for _, want := range []string{
"## 今日结论",
"## 今日行动建议",
"## 今日变化",
"## 今日价格新闻",
"### ↓ Opportunity · 降价机会",
"qwen-vl-max",
"## 场景推荐",
"## 完整数据附录",
"- 影响对象:",
"营销活动",
"主来源: OpenRouter / region_pricing",
"更新时间: 2026-05-13 09:30",
"判定依据: models.created_at = 今日,且已存在最新价格快照",
"## 💳 中转平台套餐订阅价",
"通用 Token Plan Lite",
"Hy Token Plan Max",
"¥39.00/月",
"3500万 Tokens/月",
"256K",
} {
if !strings.Contains(content, want) {
t.Fatalf("markdown missing %q\n%s", want, content)
}
@@ -599,6 +676,22 @@ func TestGenerateHTMLV3IncludesTencentSubscriptionSection(t *testing.T) {
EvidenceDetail: "官方活动页记录 V3.2-Exp 在活动窗口内价格下调 50%+",
Priority: 115,
},
{
EventType: "price_cut",
ModelName: "qwen-vl-max",
ProviderName: "Alibaba",
OperatorName: "DashScope",
Audience: "适合需要当天重排价格带的团队",
TrustLabel: "官方来源",
Baseline: "较昨日 -18%",
Summary: "视觉模型价格下降已足以影响默认选型。",
SourceKindLabel: "价格快照",
PrimarySource: "pricing_history",
UpdatedAt: "2026-05-13 10:00",
EvidenceDetail: "pricing_history 记录到输入价格较昨日下降 18%",
PriceChangePct: -18,
Priority: 100,
},
}
report.TencentSubscriptionPlans = []SubscriptionPlanInfo{
{
@@ -628,17 +721,17 @@ func TestGenerateHTMLV3IncludesTencentSubscriptionSection(t *testing.T) {
for _, want := range []string{
"今日一句话结论",
"三条行动建议",
"今日价格新闻",
"降价机会",
"今日头条",
"DeepSeek-V4-Flash",
"一级官方发布",
"二级权威佐证",
"营销活动",
"影响对象",
"首次出现",
"主来源",
"更新时间",
"判定依据",
"模型快照",
"场景推荐",
"完整数据附录",
"官方免费",
@@ -652,6 +745,97 @@ func TestGenerateHTMLV3IncludesTencentSubscriptionSection(t *testing.T) {
}
}
func TestGenerateMarkdownV3IncludesLinkedHeroAndHeadline(t *testing.T) {
path := filepath.Join(t.TempDir(), "daily_report_markdown_links.md")
report := sampleReportForV1()
report.ModelEvents = []ModelEvent{
{
EventType: "price_cut",
ModelName: "qwen-vl-max",
ProviderName: "Alibaba",
ProviderCountry: "CN",
OperatorName: "DashScope",
TrustLabel: "官方来源",
Baseline: "较昨日 -18%",
Summary: "视觉模型价格下降已足以影响默认选型。",
SourceKindLabel: "价格快照",
PrimarySource: "pricing_history",
SourceURL: "https://dashscope.aliyun.com/model/qwen-vl-max",
UpdatedAt: "2026-05-13 10:00",
EvidenceDetail: "pricing_history 记录到输入价格较昨日下降 18%",
PriceChangePct: -18,
Priority: 100,
},
}
decorateReportV1(report)
if err := generateMarkdownV3(report, path); err != nil {
t.Fatalf("generateMarkdownV3 returned error: %v", err)
}
body, err := os.ReadFile(path)
if err != nil {
t.Fatalf("read markdown output: %v", err)
}
content := string(body)
for _, want := range []string{
"> [今天最值得关注的是 qwen-vl-maxCN / Alibaba / DashScope价格下降 18%,优先复查它是否改变默认选型与预算策略。](https://dashscope.aliyun.com/model/qwen-vl-max)",
"## 今日头条",
"[qwen-vl-max 成本下调 18%](https://dashscope.aliyun.com/model/qwen-vl-max)",
} {
if !strings.Contains(content, want) {
t.Fatalf("markdown missing %q\n%s", want, content)
}
}
}
func TestGenerateHTMLV3IncludesLinksLowestPlanAndGPT56Leak(t *testing.T) {
path := filepath.Join(t.TempDir(), "daily_report_links_leak.html")
report := sampleReportForV1()
report.ModelEvents = []ModelEvent{
{
EventType: "promo_campaign",
ModelName: "GPT-5.6",
ProviderName: "OpenAI",
ProviderCountry: "US",
OperatorName: "OpenAI",
Audience: "适合关注高端模型路线图、预算和替换窗口的团队",
TrustLabel: "行业情报 / 待官方确认",
SourceKindLabel: "泄露情报",
PrimarySource: "https://openai.example.com/gpt-5-6-leak",
SourceURL: "https://openai.example.com/gpt-5-6-leak",
UpdatedAt: "2026-05-27 00:00",
EvidenceDetail: "多个公开情报源出现 GPT-5.6 命名与规格片段,尚待官方正式发布确认。",
Baseline: "提前泄露",
Summary: "GPT-5.6 提前泄露信号出现,需立即复查其是否改变默认高端模型路线与预算预期。",
Priority: 135,
},
}
report.TencentSubscriptionPlans = []SubscriptionPlanInfo{
{OperatorName: "MiniMax", PlanName: "Starter", PlanFamily: "token_plan", BillingCycle: "monthly", Currency: "USD", ListPrice: 10, PriceUnit: "USD/month", ModelCount: 1},
{OperatorName: "MiniMax", PlanName: "Plus", PlanFamily: "token_plan", BillingCycle: "monthly", Currency: "USD", ListPrice: 20, PriceUnit: "USD/month", ModelCount: 1},
}
decorateReportV1(report)
if err := generateHTMLV3(report, path); err != nil {
t.Fatalf("generateHTMLV3 returned error: %v", err)
}
body, err := os.ReadFile(path)
if err != nil {
t.Fatalf("read html output: %v", err)
}
content := string(body)
for _, want := range []string{
"GPT-5.6",
"US / OpenAI",
"https://openai.example.com/gpt-5-6-leak",
"🏷 最低价",
} {
if !strings.Contains(content, want) {
t.Fatalf("html missing %q\n%s", want, content)
}
}
}
func TestGenerateHTMLV3IncludesResellerSubscriptionComparison(t *testing.T) {
path := filepath.Join(t.TempDir(), "daily_report.html")
report := sampleReportForV1()
@@ -720,6 +904,9 @@ func TestGenerateHTMLV3IncludesResellerSubscriptionComparison(t *testing.T) {
t.Fatalf("html missing %q\n%s", want, content)
}
}
if !strings.Contains(content, "🏷 最低价") {
t.Fatalf("expected lowest plan marker in subscription table\n%s", content)
}
}
func TestGenerateMarkdownV3IncludesSignatureStabilitySection(t *testing.T) {
@@ -781,6 +968,168 @@ func TestGenerateMarkdownV3IncludesSignatureStabilitySection(t *testing.T) {
}
}
func TestGenerateMarkdownV3IncludesThemedPriceNewsSections(t *testing.T) {
path := filepath.Join(t.TempDir(), "daily_report_price_news.md")
report := sampleReportForV1()
report.ModelEvents = []ModelEvent{
{
EventType: "price_cut",
ModelName: "qwen-vl-max",
ProviderName: "Alibaba",
OperatorName: "DashScope",
Summary: "视觉模型价格下降已足以影响默认选型。",
Audience: "适合需要当天重排价格带的团队",
TrustLabel: "官方来源",
SourceKindLabel: "价格快照",
PrimarySource: "pricing_history",
UpdatedAt: "2026-05-13 10:00",
EvidenceDetail: "pricing_history 记录到输入价格较昨日下降 18%",
Baseline: "较昨日 -18%",
PriceChangePct: -18,
Priority: 100,
},
{
EventType: "price_increase",
ModelName: "claude-3.7-sonnet",
ProviderName: "Anthropic",
OperatorName: "Anthropic",
Summary: "核心写作模型价格上调,需要准备预算回退。",
Audience: "适合需要稳定预算的商用团队",
TrustLabel: "官方来源",
SourceKindLabel: "价格快照",
PrimarySource: "pricing_history",
UpdatedAt: "2026-05-13 10:10",
EvidenceDetail: "pricing_history 记录到输入价格较昨日上涨 12%",
Baseline: "较昨日 +12%",
PriceChangePct: 12,
Priority: 90,
},
{
EventType: "promo_campaign",
ModelName: "DeepSeek-V4-Flash",
ProviderName: "DeepSeek",
OperatorName: "DeepSeek",
Summary: "平台活动窗口出现后,值得重新评估低成本推理方案。",
Audience: "适合计划趁活动窗口压低推理成本的团队",
TrustLabel: "官方来源 / 一级证据",
SourceKindLabel: "官方活动页",
PrimarySource: "https://api-docs.deepseek.com/news/news250929",
UpdatedAt: "2026-05-13 09:00",
EvidenceDetail: "官方活动页记录活动窗口价格下调",
Baseline: "活动窗口开启",
Priority: 80,
},
}
decorateReportV1(report)
if err := generateMarkdownV3(report, path); err != nil {
t.Fatalf("generateMarkdownV3 returned error: %v", err)
}
body, err := os.ReadFile(path)
if err != nil {
t.Fatalf("read markdown output: %v", err)
}
content := string(body)
for _, want := range []string{
"## 今日价格新闻",
"### ↓ Opportunity · 降价机会",
"### ↑ Warning · 涨价预警",
"### ✦ Campaign · 平台活动",
"qwen-vl-max",
"claude-3.7-sonnet",
"DeepSeek-V4-Flash",
} {
if !strings.Contains(content, want) {
t.Fatalf("markdown missing %q\n%s", want, content)
}
}
}
func TestGenerateHTMLV3IncludesThemedPriceNewsSections(t *testing.T) {
path := filepath.Join(t.TempDir(), "daily_report_price_news.html")
report := sampleReportForV1()
report.ModelEvents = []ModelEvent{
{
EventType: "price_cut",
ModelName: "qwen-vl-max",
ProviderName: "Alibaba",
OperatorName: "DashScope",
Summary: "视觉模型价格下降已足以影响默认选型。",
Audience: "适合需要当天重排价格带的团队",
TrustLabel: "官方来源",
SourceKindLabel: "价格快照",
PrimarySource: "pricing_history",
UpdatedAt: "2026-05-13 10:00",
EvidenceDetail: "pricing_history 记录到输入价格较昨日下降 18%",
Baseline: "较昨日 -18%",
PriceChangePct: -18,
Priority: 100,
},
{
EventType: "price_increase",
ModelName: "claude-3.7-sonnet",
ProviderName: "Anthropic",
OperatorName: "Anthropic",
Summary: "核心写作模型价格上调,需要准备预算回退。",
Audience: "适合需要稳定预算的商用团队",
TrustLabel: "官方来源",
SourceKindLabel: "价格快照",
PrimarySource: "pricing_history",
UpdatedAt: "2026-05-13 10:10",
EvidenceDetail: "pricing_history 记录到输入价格较昨日上涨 12%",
Baseline: "较昨日 +12%",
PriceChangePct: 12,
Priority: 90,
},
{
EventType: "promo_campaign",
ModelName: "DeepSeek-V4-Flash",
ProviderName: "DeepSeek",
OperatorName: "DeepSeek",
Summary: "平台活动窗口出现后,值得重新评估低成本推理方案。",
Audience: "适合计划趁活动窗口压低推理成本的团队",
TrustLabel: "官方来源 / 一级证据",
SourceKindLabel: "官方活动页",
PrimarySource: "https://api-docs.deepseek.com/news/news250929",
UpdatedAt: "2026-05-13 09:00",
EvidenceDetail: "官方活动页记录活动窗口价格下调",
Baseline: "活动窗口开启",
Priority: 80,
},
}
decorateReportV1(report)
if err := generateHTMLV3(report, path); err != nil {
t.Fatalf("generateHTMLV3 returned error: %v", err)
}
body, err := os.ReadFile(path)
if err != nil {
t.Fatalf("read html output: %v", err)
}
content := string(body)
for _, want := range []string{
"今日价格新闻",
"降价机会",
"涨价预警",
"平台活动",
"Opportunity",
"Warning",
"Campaign",
">↓<",
">↑<",
">✦<",
"qwen-vl-max",
"claude-3.7-sonnet",
"DeepSeek-V4-Flash",
} {
if !strings.Contains(content, want) {
t.Fatalf("html missing %q\n%s", want, content)
}
}
}
func TestGenerateHTMLV3IncludesSignatureStabilitySection(t *testing.T) {
path := filepath.Join(t.TempDir(), "daily_report.html")
report := sampleReportForV1()
@@ -830,6 +1179,163 @@ func TestGenerateHTMLV3IncludesSignatureStabilitySection(t *testing.T) {
}
}
func TestGenerateHTMLV3IncludesPriceNewsBadgeIcons(t *testing.T) {
path := filepath.Join(t.TempDir(), "daily_report_price_news_badges.html")
report := sampleReportForV1()
report.ModelEvents = []ModelEvent{
{
EventType: "price_cut",
ModelName: "qwen-vl-max",
ProviderName: "Alibaba",
OperatorName: "DashScope",
Summary: "视觉模型价格下降已足以影响默认选型。",
Audience: "适合需要当天重排价格带的团队",
TrustLabel: "官方来源",
SourceKindLabel: "价格快照",
PrimarySource: "pricing_history",
UpdatedAt: "2026-05-13 10:00",
EvidenceDetail: "pricing_history 记录到输入价格较昨日下降 18%",
Baseline: "较昨日 -18%",
PriceChangePct: -18,
Priority: 100,
},
{
EventType: "price_increase",
ModelName: "claude-3.7-sonnet",
ProviderName: "Anthropic",
OperatorName: "Anthropic",
Summary: "核心写作模型价格上调,需要准备预算回退。",
Audience: "适合需要稳定预算的商用团队",
TrustLabel: "官方来源",
SourceKindLabel: "价格快照",
PrimarySource: "pricing_history",
UpdatedAt: "2026-05-13 10:10",
EvidenceDetail: "pricing_history 记录到输入价格较昨日上涨 12%",
Baseline: "较昨日 +12%",
PriceChangePct: 12,
Priority: 90,
},
{
EventType: "promo_campaign",
ModelName: "DeepSeek-V4-Flash",
ProviderName: "DeepSeek",
OperatorName: "DeepSeek",
Summary: "平台活动窗口出现后,值得重新评估低成本推理方案。",
Audience: "适合计划趁活动窗口压低推理成本的团队",
TrustLabel: "官方来源 / 一级证据",
SourceKindLabel: "官方活动页",
PrimarySource: "https://api-docs.deepseek.com/news/news250929",
UpdatedAt: "2026-05-13 09:00",
EvidenceDetail: "官方活动页记录活动窗口价格下调",
Baseline: "活动窗口开启",
Priority: 80,
},
}
decorateReportV1(report)
if err := generateHTMLV3(report, path); err != nil {
t.Fatalf("generateHTMLV3 returned error: %v", err)
}
body, err := os.ReadFile(path)
if err != nil {
t.Fatalf("read html output: %v", err)
}
content := string(body)
for _, want := range []string{
"Opportunity",
"Warning",
"Campaign",
">↓<",
">↑<",
">✦<",
} {
if !strings.Contains(content, want) {
t.Fatalf("html missing %q\n%s", want, content)
}
}
}
func TestGenerateHTMLV3PaginatesAppendices(t *testing.T) {
path := filepath.Join(t.TempDir(), "daily_report_appendix_pagination.html")
report := sampleReportForV1()
report.IntlTop5 = nil
report.IntlAppendixList = nil
report.DomesticTop10 = nil
report.DomesticAppendixList = nil
report.FreeTop20 = nil
report.Operators = nil
report.Resellers = nil
for i := 0; i < 65; i++ {
report.DomesticAppendixList = append(report.DomesticAppendixList, ModelInfo{
Name: fmt.Sprintf("domestic-model-%02d", i+1),
ProviderName: "ProviderCN",
InputPrice: 1,
OutputPrice: 2,
Currency: "CNY",
ContextLength: 131072,
})
}
for i := 0; i < 22; i++ {
report.FreeTop20 = append(report.FreeTop20, ModelInfo{
Name: fmt.Sprintf("free-model-%02d", i+1),
ProviderName: "ProviderFree",
OperatorType: "official",
ContextLength: 65536,
})
}
for i := 0; i < 23; i++ {
report.Operators = append(report.Operators, OperatorInfo{Name: fmt.Sprintf("Operator-%02d", i+1), ModelCount: i + 1, MinInputPrice: 0.1, AvgInputPrice: 0.2})
}
for i := 0; i < 22; i++ {
report.Resellers = append(report.Resellers, OperatorInfo{Name: fmt.Sprintf("Reseller-%02d", i+1), ModelCount: i + 1, MinInputPrice: 0.3, AvgInputPrice: 0.4})
}
decorateReportV1(report)
report.AppendixLinks = []AppendixLink{
{Title: "国际低价", Description: "查看国际低价模型附录(日报仅保留前几页)", Anchor: "#appendix-pricing-intl"},
{Title: "国内低价", Description: "查看国内低价模型附录(日报仅保留前几页)", Anchor: "#appendix-pricing-domestic"},
{Title: "免费样本", Description: "查看免费模型代表样本附录", Anchor: "#appendix-free"},
{Title: "平台覆盖", Description: "查看官方平台与聚合平台覆盖", Anchor: "#appendix-platforms"},
{Title: "全量导出 JSON", Description: "其余完整数据请下载独立导出文件或转到查询页查看", Anchor: "/reports/daily/appendix/2026-05-13/full_appendix.json"},
}
if err := generateHTMLV3(report, path); err != nil {
t.Fatalf("generateHTMLV3 returned error: %v", err)
}
body, err := os.ReadFile(path)
if err != nil {
t.Fatalf("read html output: %v", err)
}
content := string(body)
for _, want := range []string{
"完整价格附录(国际低价)",
"完整价格附录(国内低价)",
"完整免费附录",
"平台覆盖附录",
"国际低价",
"国内低价",
"全量导出 JSON",
"/reports/daily/appendix/2026-05-13/full_appendix.json",
"data-appendix-page=\"1\"",
"data-appendix-total-pages=\"1\"",
"data-appendix-total-pages=\"2\"",
"data-appendix-total-pages=\"3\"",
"#appendix-pricing-intl",
"#appendix-pricing-domestic",
"#appendix-free",
"#appendix-platforms",
"上一页",
"下一页",
} {
if !strings.Contains(content, want) {
t.Fatalf("html missing %q\n%s", want, content)
}
}
}
func TestBuildHeadlineItemsUsesModelEvents(t *testing.T) {
report := sampleReportForV1()
report.ModelEvents = []ModelEvent{
@@ -883,14 +1389,17 @@ func TestBuildHeadlineItemsUsesModelEvents(t *testing.T) {
if len(items) < 2 {
t.Fatalf("expected at least 2 headline items, got %d", len(items))
}
if !strings.Contains(items[0].Title, "GLM-5") || items[0].Label != "一级官方发布" {
t.Fatalf("expected official release event to rank first, got %+v", items[0])
if items[0].Label != "价格下调" || !strings.Contains(items[0].Title, "glm-5") {
t.Fatalf("expected price change event to rank first, got %+v", items[0])
}
if items[1].Baseline != "较昨日 -25%" {
t.Fatalf("expected price_cut baseline to be preserved, got %+v", items[1])
if items[1].Label != "一级官方发布" {
t.Fatalf("expected official release to stay immediately after price change, got %+v", items[1])
}
if items[0].SourceKindLabel != "一级官方发布" || items[0].PrimarySource != "https://open.bigmodel.cn/dev/howuse/model" {
t.Fatalf("expected official release evidence fields to be preserved, got %+v", items[0])
if items[0].Baseline != "较昨日 -25%" {
t.Fatalf("expected price_cut baseline to be preserved, got %+v", items[0])
}
if items[1].SourceKindLabel != "一级官方发布" || items[1].PrimarySource != "https://open.bigmodel.cn/dev/howuse/model" {
t.Fatalf("expected official release evidence fields to be preserved, got %+v", items[1])
}
}
@@ -1082,6 +1591,89 @@ func TestDecorateReportV1ElevatesSignatureDriftIntoHeroSummary(t *testing.T) {
if !strings.Contains(report.HeroEvidence, "最近 5 次中出现 3 次结构变化") {
t.Fatalf("expected hero evidence to mention drift count, got %q", report.HeroEvidence)
}
}
func TestDecorateReportV1PrefersPriceChangeInHeroSummary(t *testing.T) {
report := sampleReportForV1()
report.ModelEvents = []ModelEvent{
{
EventType: "official_release",
ModelName: "GLM-5",
Summary: "官方发布新模型。",
PrimarySource: "official release",
EvidenceDetail: "models.release_date = 今日",
Priority: 120,
},
{
EventType: "price_cut",
ModelName: "DeepSeek-V4-Flash",
ProviderName: "DeepSeek",
OperatorName: "OpenRouter",
Summary: "价格下降已足以影响默认选型,值得重新评估同类模型。",
PrimarySource: "pricing_history",
EvidenceDetail: "pricing_history 记录到输入价格由 $0.60 调整为 $0.30,较昨日下降 50%",
PriceChangePct: -50,
OldInputPrice: 0.60,
NewInputPrice: 0.30,
OldOutputPrice: 2.40,
NewOutputPrice: 1.20,
Currency: "USD",
Priority: 95,
},
}
decorateReportV1(report)
if !strings.Contains(report.HeroSummary, "DeepSeek-V4-Flash") || !strings.Contains(report.HeroSummary, "价格") {
t.Fatalf("expected hero summary to prioritize price change, got %q", report.HeroSummary)
}
if !strings.Contains(report.HeroEvidence, "pricing_history") {
t.Fatalf("expected hero evidence to mention pricing history, got %q", report.HeroEvidence)
}
}
func TestBuildHeadlineItemsPlacesPriceChangeBeforeOfficialRelease(t *testing.T) {
report := sampleReportForV1()
report.ModelEvents = []ModelEvent{
{
EventType: "official_release",
ModelName: "GLM-5",
Summary: "官方发布新模型。",
TrustLabel: "官方来源 / 一级证据",
SourceKindLabel: "一级官方发布",
PrimarySource: "official release",
UpdatedAt: "2026-05-13 08:30",
EvidenceDetail: "models.release_date = 今日",
Priority: 120,
},
{
EventType: "price_cut",
ModelName: "DeepSeek-V4-Flash",
ProviderName: "DeepSeek",
OperatorName: "OpenRouter",
Summary: "价格下降已足以影响默认选型,值得重新评估同类模型。",
TrustLabel: "聚合来源",
SourceKindLabel: "价格快照",
PrimarySource: "pricing_history",
UpdatedAt: "2026-05-13 09:30",
EvidenceDetail: "pricing_history 记录到输入价格由 $0.60 调整为 $0.30,较昨日下降 50%",
PriceChangePct: -50,
OldInputPrice: 0.60,
NewInputPrice: 0.30,
OldOutputPrice: 2.40,
NewOutputPrice: 1.20,
Currency: "USD",
Priority: 95,
},
}
items := buildHeadlineItems(report)
if len(items) == 0 {
t.Fatalf("expected headline items")
}
if items[0].Label != "价格下调" {
t.Fatalf("expected price change headline first, got %+v", items[0])
}
}
func TestSignatureAuditSummaryToneRespectsConfiguredThreshold(t *testing.T) {

View File

@@ -0,0 +1,198 @@
#!/usr/bin/env bash
set -euo pipefail
ROOT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
cd "$ROOT_DIR"
if [[ -f ".env.local" ]]; then
# shellcheck disable=SC1091
source ".env.local"
fi
if [[ -f ".env" ]]; then
# shellcheck disable=SC1091
source ".env"
fi
if [[ -z "${DATABASE_URL:-}" ]]; then
echo "DATABASE_URL 未设置" >&2
exit 1
fi
if [[ -z "${OPENROUTER_API_KEY:-}" ]]; then
echo "OPENROUTER_API_KEY 未设置,无法执行日内价格追踪" >&2
exit 1
fi
REPORT_DATE="${REPORT_DATE:-$(date +%F)}"
FETCH_OUT="$ROOT_DIR/models.json"
FETCH_TOTAL="0"
PIPELINE_STAGE_SET="openrouter,multi_source,official_imports,daily_signal_snapshot"
PIPELINE_SOURCE_SET="openrouter,moonshot,deepseek,openai,zhipu,baidu,bytedance,tencent_subscription,aliyun_subscription,baidu_subscription,ctyun_subscription,bytedance_subscription,huawei_package,zhipu_coding_plan,minimax_subscription,cucloud_catalog,cucloud_pricing,mobile_cloud_pricing,youdao_pricing,platform360_pricing,siliconflow_pricing,ppio_pricing,ucloud_pricing,coreshub_pricing,cloudflare_pricing,perplexity_pricing,vertex_pricing,bedrock_pricing,azure_openai_pricing,qwen_pricing,hunyuan_pricing,huawei_maas_pricing,baichuan_pricing,lingyiwanwu_pricing,sensenova_pricing,xfyun_pricing,bytedance_pricing,catalog_seed_verification"
PIPELINE_FAILED_SOURCE_SET="none"
MULTI_SOURCE_AUDIT="multi_source_audit=unavailable"
PIPELINE_AUDIT_SUMMARY=""
normalize_summary_file() {
local path="$1"
if [[ ! -f "$path" ]]; then
return
fi
tr '\n' ' ' < "$path" | sed 's/[[:space:]]\+/ /g; s/^ //; s/ $//'
}
extract_failed_source_keys() {
local summary="$1"
printf '%s\n' "$summary" | sed -n 's/.*failed_source_keys=\([^ ]*\).*/\1/p'
}
merge_failed_source_keys() {
local keys="$1"
if [[ -z "$keys" || "$keys" == "none" ]]; then
return
fi
if [[ "$PIPELINE_FAILED_SOURCE_SET" == "none" ]]; then
PIPELINE_FAILED_SOURCE_SET="$keys"
return
fi
PIPELINE_FAILED_SOURCE_SET="${PIPELINE_FAILED_SOURCE_SET},${keys}"
}
refresh_pipeline_audit() {
PIPELINE_AUDIT_SUMMARY="runtime_audit stage_set=${PIPELINE_STAGE_SET} selected_source_keys=${PIPELINE_SOURCE_SET} failed_source_keys=${PIPELINE_FAILED_SOURCE_SET} openrouter_total=${FETCH_TOTAL:-0} ${MULTI_SOURCE_AUDIT}"
}
run_or_fail() {
local source_key="$1"
local error_message="$2"
shift 2
if ! "$@"; then
merge_failed_source_keys "$source_key"
refresh_pipeline_audit
echo "$error_message" >&2
exit 1
fi
}
refresh_pipeline_audit
bash "$ROOT_DIR/scripts/apply_migration.sh"
run_or_fail "openrouter" "OpenRouter 日内价格采集失败" \
go run "./scripts/fetch_openrouter.go" -api-key "$OPENROUTER_API_KEY" -db "$DATABASE_URL" -out "$FETCH_OUT" -strict-real
FETCH_TOTAL=$(python3 - <<'PY' "$FETCH_OUT"
import json, sys
path = sys.argv[1]
with open(path, 'r', encoding='utf-8') as f:
data = json.load(f)
print(int(data.get("total", 0)))
PY
)
if [[ "${FETCH_TOTAL:-0}" -lt 10 ]]; then
merge_failed_source_keys "openrouter"
refresh_pipeline_audit
echo "本次日内采集结果异常: total=${FETCH_TOTAL:-0} < 10" >&2
exit 1
fi
refresh_pipeline_audit
MULTI_SOURCE_OUTPUT="$(mktemp)"
if ! go run "./scripts/fetch_multi_source.go" --sources moonshot,deepseek,openai > "$MULTI_SOURCE_OUTPUT"; then
MULTI_SOURCE_SUMMARY="$(normalize_summary_file "$MULTI_SOURCE_OUTPUT")"
if [[ -n "$MULTI_SOURCE_SUMMARY" ]]; then
MULTI_SOURCE_AUDIT="multi_source_audit=${MULTI_SOURCE_SUMMARY}"
merge_failed_source_keys "$(extract_failed_source_keys "$MULTI_SOURCE_SUMMARY")"
else
MULTI_SOURCE_AUDIT="multi_source_audit=stage_failed"
merge_failed_source_keys "moonshot,deepseek,openai"
fi
cat "$MULTI_SOURCE_OUTPUT"
rm -f "$MULTI_SOURCE_OUTPUT"
refresh_pipeline_audit
echo "日内多源价格补充同步失败" >&2
exit 1
fi
MULTI_SOURCE_SUMMARY="$(normalize_summary_file "$MULTI_SOURCE_OUTPUT")"
MULTI_SOURCE_AUDIT="multi_source_audit=${MULTI_SOURCE_SUMMARY:-none}"
merge_failed_source_keys "$(extract_failed_source_keys "$MULTI_SOURCE_SUMMARY")"
refresh_pipeline_audit
cat "$MULTI_SOURCE_OUTPUT"
rm -f "$MULTI_SOURCE_OUTPUT"
run_or_fail "zhipu" "智谱官方导入失败" go run -tags llm_script "./scripts/import_zhipu_data.go"
run_or_fail "official_seed_export" "官方种子导出失败" go run -tags llm_script "./scripts/export_official_seed_json.go"
run_or_fail "baidu" "百度官方导入失败" go run -tags llm_script "./scripts/import_phase2_data.go"
run_or_fail "bytedance" "字节官方导入失败" go run -tags llm_script "./scripts/import_bytedance_data.go"
run_or_fail "aliyun_subscription" "阿里云套餐导入失败" \
go run -tags llm_script ./scripts/subscription_import_common.go ./scripts/aliyun_subscription_lib.go ./scripts/import_aliyun_subscription.go
run_or_fail "baidu_subscription" "百度套餐导入失败" \
go run -tags llm_script ./scripts/subscription_import_common.go ./scripts/baidu_subscription_lib.go ./scripts/import_baidu_subscription.go
run_or_fail "ctyun_subscription" "天翼云套餐导入失败" \
go run -tags llm_script ./scripts/subscription_import_common.go ./scripts/ctyun_subscription_lib.go ./scripts/import_ctyun_subscription.go
run_or_fail "bytedance_subscription" "火山方舟套餐导入失败" \
go run -tags llm_script ./scripts/subscription_import_common.go ./scripts/bytedance_subscription_lib.go ./scripts/import_bytedance_subscription.go
run_or_fail "huawei_package" "华为云套餐包导入失败" \
go run -tags llm_script ./scripts/subscription_import_common.go ./scripts/huawei_package_lib.go ./scripts/import_huawei_package.go
run_or_fail "zhipu_coding_plan" "智谱 Coding Plan 导入失败" \
go run -tags llm_script ./scripts/subscription_import_common.go ./scripts/zhipu_coding_plan_lib.go ./scripts/import_zhipu_coding_plan.go
run_or_fail "minimax_subscription" "MiniMax Token Plan 导入失败" \
go run -tags llm_script ./scripts/subscription_import_common.go ./scripts/minimax_subscription_lib.go ./scripts/import_minimax_subscription.go
run_or_fail "cucloud_catalog" "联通云目录校验失败" \
go run -tags llm_script ./scripts/subscription_import_common.go ./scripts/catalog_verification_common.go ./scripts/import_cucloud_catalog.go
run_or_fail "cucloud_pricing" "联通云 Token Plan 价格导入失败" \
go run -tags llm_script ./scripts/subscription_import_common.go ./scripts/official_pricing_import_common.go ./scripts/import_cucloud_pricing.go
run_or_fail "mobile_cloud_pricing" "移动云 MoMA 价格导入失败" \
go run -tags llm_script ./scripts/subscription_import_common.go ./scripts/official_pricing_import_common.go ./scripts/import_mobile_cloud_pricing.go
run_or_fail "tencent_subscription" "腾讯云套餐导入失败" \
go run -tags llm_script ./scripts/subscription_import_common.go ./scripts/tencent_catalog_lib.go ./scripts/import_tencent_subscription.go
run_or_fail "youdao_pricing" "网易有道价格导入失败" \
go run -tags llm_script ./scripts/subscription_import_common.go ./scripts/official_pricing_import_common.go ./scripts/youdao_pricing_lib.go ./scripts/import_youdao_pricing.go
run_or_fail "platform360_pricing" "360 智脑价格导入失败" \
go run -tags llm_script ./scripts/subscription_import_common.go ./scripts/official_pricing_import_common.go ./scripts/platform360_pricing_lib.go ./scripts/import_360_pricing.go
run_or_fail "siliconflow_pricing" "硅基流动价格导入失败" \
go run -tags llm_script ./scripts/subscription_import_common.go ./scripts/official_pricing_import_common.go ./scripts/siliconflow_pricing_lib.go ./scripts/import_siliconflow_pricing.go
run_or_fail "ppio_pricing" "PPIO 价格导入失败" \
go run -tags llm_script ./scripts/subscription_import_common.go ./scripts/official_pricing_import_common.go ./scripts/ppio_pricing_lib.go ./scripts/import_ppio_pricing.go
run_or_fail "ucloud_pricing" "UCloud 价格导入失败" \
go run -tags llm_script ./scripts/subscription_import_common.go ./scripts/official_pricing_import_common.go ./scripts/ucloud_pricing_lib.go ./scripts/import_ucloud_pricing.go
run_or_fail "coreshub_pricing" "CoresHub 价格导入失败" \
go run -tags llm_script ./scripts/subscription_import_common.go ./scripts/official_pricing_import_common.go ./scripts/coreshub_pricing_lib.go ./scripts/import_coreshub_pricing.go
run_or_fail "cloudflare_pricing_signature" "Cloudflare Workers AI 价格页结构签名漂移" \
go run -tags llm_script ./scripts/subscription_import_common.go ./scripts/official_pricing_import_common.go ./scripts/pricing_markdown_snapshot_lib.go ./scripts/cloudflare_pricing_snapshot_lib.go ./scripts/signature_guard_common.go ./scripts/official_import_signature_audit_lib.go ./scripts/cloudflare_pricing_signature_guard_lib.go ./scripts/cloudflare_pricing_import_runner.go ./scripts/cloudflare_pricing_lib.go ./scripts/cloudflare_pricing_signature_guard.go
run_or_fail "cloudflare_pricing" "Cloudflare Workers AI 价格导入失败" \
go run -tags llm_script ./scripts/subscription_import_common.go ./scripts/official_pricing_import_common.go ./scripts/pricing_markdown_snapshot_lib.go ./scripts/cloudflare_pricing_snapshot_lib.go ./scripts/cloudflare_pricing_import_runner.go ./scripts/cloudflare_pricing_lib.go ./scripts/import_cloudflare_pricing.go
run_or_fail "perplexity_pricing_signature" "Perplexity API 价格页结构签名漂移" \
go run -tags llm_script ./scripts/subscription_import_common.go ./scripts/official_pricing_import_common.go ./scripts/pricing_markdown_snapshot_lib.go ./scripts/perplexity_pricing_snapshot_lib.go ./scripts/signature_guard_common.go ./scripts/official_import_signature_audit_lib.go ./scripts/perplexity_pricing_signature_guard_lib.go ./scripts/perplexity_pricing_import_runner.go ./scripts/perplexity_pricing_lib.go ./scripts/perplexity_pricing_signature_guard.go
run_or_fail "perplexity_pricing" "Perplexity API 价格导入失败" \
go run -tags llm_script ./scripts/subscription_import_common.go ./scripts/official_pricing_import_common.go ./scripts/pricing_markdown_snapshot_lib.go ./scripts/perplexity_pricing_snapshot_lib.go ./scripts/perplexity_pricing_import_runner.go ./scripts/perplexity_pricing_lib.go ./scripts/import_perplexity_pricing.go
run_or_fail "vertex_pricing_signature" "Vertex AI 价格页结构签名漂移" \
go run -tags llm_script ./scripts/subscription_import_common.go ./scripts/official_pricing_import_common.go ./scripts/pricing_markdown_snapshot_lib.go ./scripts/signature_guard_common.go ./scripts/official_import_signature_audit_lib.go ./scripts/vertex_pricing_snapshot_lib.go ./scripts/vertex_pricing_signature_guard_lib.go ./scripts/vertex_pricing_import_runner.go ./scripts/vertex_pricing_lib.go ./scripts/vertex_pricing_signature_guard.go
run_or_fail "vertex_pricing" "Vertex AI 价格导入失败" \
go run -tags llm_script ./scripts/subscription_import_common.go ./scripts/official_pricing_import_common.go ./scripts/vertex_pricing_snapshot_lib.go ./scripts/vertex_pricing_import_runner.go ./scripts/vertex_pricing_lib.go ./scripts/import_vertex_pricing.go
run_or_fail "bedrock_pricing" "Amazon Bedrock 价格导入失败" \
go run -tags llm_script ./scripts/subscription_import_common.go ./scripts/official_pricing_import_common.go ./scripts/bedrock_pricing_lib.go ./scripts/import_bedrock_pricing.go
run_or_fail "azure_openai_pricing" "Azure OpenAI 价格导入失败" \
go run -tags llm_script ./scripts/subscription_import_common.go ./scripts/official_pricing_import_common.go ./scripts/azure_openai_pricing_lib.go ./scripts/import_azure_openai_pricing.go
run_or_fail "qwen_pricing" "通义千问价格导入失败" \
go run -tags llm_script ./scripts/subscription_import_common.go ./scripts/official_pricing_import_common.go ./scripts/import_qwen_pricing.go
run_or_fail "hunyuan_pricing" "腾讯混元价格导入失败" \
go run -tags llm_script ./scripts/subscription_import_common.go ./scripts/official_pricing_import_common.go ./scripts/import_hunyuan_pricing.go
run_or_fail "huawei_maas_pricing" "华为云 MaaS 价格导入失败" \
go run -tags llm_script ./scripts/subscription_import_common.go ./scripts/official_pricing_import_common.go ./scripts/import_huawei_maas_pricing.go
run_or_fail "baichuan_pricing" "百川价格导入失败" \
go run -tags llm_script ./scripts/subscription_import_common.go ./scripts/official_pricing_import_common.go ./scripts/import_baichuan_pricing.go
run_or_fail "lingyiwanwu_pricing" "零一万物价格导入失败" \
go run -tags llm_script ./scripts/subscription_import_common.go ./scripts/official_pricing_import_common.go ./scripts/import_lingyiwanwu_pricing.go
run_or_fail "sensenova_pricing" "商汤 SenseNova 价格导入失败" \
go run -tags llm_script ./scripts/subscription_import_common.go ./scripts/official_pricing_import_common.go ./scripts/import_sensenova_pricing.go
run_or_fail "xfyun_pricing" "讯飞价格导入失败" \
go run -tags llm_script ./scripts/subscription_import_common.go ./scripts/official_pricing_import_common.go ./scripts/import_xfyun_pricing.go
run_or_fail "bytedance_pricing" "火山方舟价格导入失败" \
go run -tags llm_script ./scripts/subscription_import_common.go ./scripts/official_pricing_import_common.go ./scripts/import_bytedance_pricing.go
refresh_pipeline_audit
run_or_fail "catalog_seed_verification" "目录级官方入口核验失败" \
go run -tags llm_script ./scripts/subscription_import_common.go ./scripts/import_catalog_seed_verification.go
refresh_pipeline_audit
run_or_fail "daily_signal_snapshot" "日内价格信号物化失败" \
env SIGNAL_SOURCE_AUDIT="$PIPELINE_AUDIT_SUMMARY" REPORT_TRIGGER_SOURCE="intraday" go run -tags llm_script ./scripts/materialize_daily_signals.go
echo "$PIPELINE_AUDIT_SUMMARY"

64
scripts/secret_gate_lib.sh Executable file
View File

@@ -0,0 +1,64 @@
#!/usr/bin/env bash
secret_scan_paths() {
local scan_root="${1:-}"
shift || true
if [ -z "$scan_root" ]; then
echo "secret_scan_paths requires scan root" >&2
return 1
fi
local patterns='(sk-[A-Za-z0-9_-]+|AKIA[0-9A-Z]{16}|AIza[0-9A-Za-z_-]{35}|ghp_[A-Za-z0-9]{36}|xox[baprs]-[A-Za-z0-9-]{10,}|-----BEGIN (RSA|DSA|EC|OPENSSH|PGP) PRIVATE KEY-----|authorization:[[:space:]]*bearer[[:space:]]+[A-Za-z0-9._-]{8,}|api[_-]?key[[:space:]]*[:=][[:space:]]*[A-Za-z0-9._-]{8,})'
local excludes=(
'--exclude=verify_phase6.sh'
'--exclude=secret_gate_lib.sh'
'--exclude=secret_gate_test.sh'
'--exclude=.env.example'
'--exclude=README.md'
'--exclude=CONFIGURATION.md'
'--exclude=DEPLOYMENT.md'
'--exclude-dir=.git'
'--exclude-dir=.serena'
'--exclude-dir=node_modules'
'--exclude-dir=dist'
'--exclude-dir=logs'
'--exclude-dir=reports'
)
if grep -R -n -E -i "$patterns" "$scan_root" "$@" \
--include='*.go' \
--include='*.ts' \
--include='*.tsx' \
--include='*.js' \
--include='*.jsx' \
--include='*.sh' \
--include='*.yml' \
--include='*.yaml' \
"${excludes[@]}"; then
return 1
fi
return 0
}
secret_env_files() {
local dockerignore_path="$1"
if [ ! -f "$dockerignore_path" ]; then
echo "missing dockerignore: $dockerignore_path" >&2
return 1
fi
if ! grep -Eq '^\.env(\..*)?$' "$dockerignore_path"; then
echo "missing .env ignore rule in $dockerignore_path" >&2
return 1
fi
if ! grep -Eq '^!\.env\.example$' "$dockerignore_path"; then
echo "missing explicit .env.example allow rule in $dockerignore_path" >&2
return 1
fi
return 0
}

60
scripts/secret_gate_test.sh Executable file
View File

@@ -0,0 +1,60 @@
#!/usr/bin/env bash
set -euo pipefail
ROOT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
cd "$ROOT_DIR"
. "$ROOT_DIR/scripts/secret_gate_lib.sh"
TMP_DIR="$(mktemp -d)"
cleanup() {
rm -rf "$TMP_DIR"
}
trap cleanup EXIT
SECRET_FILE="$TMP_DIR/secret.ts"
CLEAN_FILE="$TMP_DIR/clean.ts"
AWS_SECRET_FILE="$TMP_DIR/aws.ts"
ENV_FILE="$TMP_DIR/.env"
DOCKERIGNORE_FILE="$TMP_DIR/.dockerignore"
MISSING_DOCKERIGNORE_FIXTURE="$ROOT_DIR/scripts/testdata/empty.dockerignore"
printf 'const key = "sk-test-secret";\n' > "$SECRET_FILE"
printf 'const ok = true;\n' > "$CLEAN_FILE"
printf 'const awsKey = "AKIA1234567890ABCDEF";\n' > "$AWS_SECRET_FILE"
printf 'OPENROUTER_API_KEY=sk-test-secret\n' > "$ENV_FILE"
printf '.env\n!.env.example\n' > "$DOCKERIGNORE_FILE"
set +e
secret_scan_paths "$SECRET_FILE" "$CLEAN_FILE" > /tmp/secret_gate_test_scan.out 2> /tmp/secret_gate_test_scan.err
SCAN_RC=$?
set -e
if [ "$SCAN_RC" -eq 0 ]; then
echo "expected secret_scan_paths to fail"
exit 1
fi
grep -q "$SECRET_FILE" /tmp/secret_gate_test_scan.out
set +e
secret_scan_paths "$AWS_SECRET_FILE" > /tmp/secret_gate_test_aws.out 2> /tmp/secret_gate_test_aws.err
AWS_SCAN_RC=$?
set -e
if [ "$AWS_SCAN_RC" -eq 0 ]; then
echo "expected secret_scan_paths to fail for aws-style key"
exit 1
fi
grep -q 'AKIA1234567890ABCDEF' /tmp/secret_gate_test_aws.out
secret_env_files "$DOCKERIGNORE_FILE" > /tmp/secret_gate_test_env.out 2> /tmp/secret_gate_test_env.err
set +e
secret_env_files "$MISSING_DOCKERIGNORE_FIXTURE" > /tmp/secret_gate_test_env_fail.out 2> /tmp/secret_gate_test_env_fail.err
ENV_RC=$?
set -e
if [ "$ENV_RC" -eq 0 ]; then
echo "expected secret_env_files to fail without dockerignore entry"
exit 1
fi
grep -q "missing .env ignore rule" /tmp/secret_gate_test_env_fail.err
echo "secret_gate_test: PASS"

View File

@@ -101,6 +101,17 @@ func fetchSubscriptionPage(url string, fixture string, client *http.Client) (str
return string(data), nil
}
body, err := fetchSubscriptionPageWithRetry(url, client)
if err == nil {
return body, nil
}
if markdownURL, ok := markdownFallbackURL(url, err); ok {
return fetchSubscriptionPageWithRetry(markdownURL, client)
}
return "", err
}
func fetchSubscriptionPageWithRetry(url string, client *http.Client) (string, error) {
var lastErr error
for attempt := 1; attempt <= subscriptionFetchMaxAttempts; attempt++ {
body, retryable, err := fetchSubscriptionPageOnce(url, client)
@@ -116,6 +127,7 @@ func fetchSubscriptionPage(url string, fixture string, client *http.Client) (str
return "", lastErr
}
func fetchSubscriptionPageOnce(url string, client *http.Client) (string, bool, error) {
req, err := http.NewRequest(http.MethodGet, url, nil)
if err != nil {
@@ -146,6 +158,20 @@ func fetchSubscriptionPageOnce(url string, client *http.Client) (string, bool, e
return normalizeSubscriptionPage(string(body)), false, nil
}
func markdownFallbackURL(url string, err error) (string, bool) {
if strings.TrimSpace(url) == "" || err == nil {
return "", false
}
lower := strings.ToLower(err.Error())
if !strings.Contains(lower, "status 403") && !strings.Contains(lower, "forbidden") {
return "", false
}
if strings.HasSuffix(url, ".md") {
return "", false
}
return strings.TrimRight(url, "/") + ".md", true
}
func isRetriableSubscriptionFetchError(err error) bool {
if err == nil {
return false

View File

@@ -3,8 +3,10 @@
package main
import (
"fmt"
"net/http"
"net/http/httptest"
"strings"
"testing"
"time"
)
@@ -46,3 +48,34 @@ func TestIsRetriableSubscriptionFetchErrorRecognizesForbidden(t *testing.T) {
t.Fatalf("403 应被视作可重试错误")
}
}
func TestFetchSubscriptionPageFallsBackToMarkdownSuffixOnForbidden(t *testing.T) {
attempts := map[string]int{}
server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
attempts[r.URL.Path]++
switch r.URL.Path {
case "/cn/update/promotion":
http.Error(w, "forbidden", http.StatusForbidden)
case "/cn/update/promotion.md":
_, _ = w.Write([]byte("# 上新活动\nGLM Coding Plan 低至20元/月"))
default:
http.Error(w, fmt.Sprintf("unexpected path %s", r.URL.Path), http.StatusNotFound)
}
}))
defer server.Close()
client := &http.Client{Timeout: 2 * time.Second}
body, err := fetchSubscriptionPage(server.URL+"/cn/update/promotion", "", client)
if err != nil {
t.Fatalf("fetchSubscriptionPage 返回错误: %v", err)
}
if !strings.Contains(body, "GLM Coding Plan 低至20元/月") {
t.Fatalf("返回体缺少 markdown fallback 内容: %q", body)
}
if attempts["/cn/update/promotion"] != subscriptionFetchMaxAttempts {
t.Fatalf("期望原始路径按重试上限请求 %d 次,实际 %d", subscriptionFetchMaxAttempts, attempts["/cn/update/promotion"])
}
if attempts["/cn/update/promotion.md"] != 1 {
t.Fatalf("期望 .md 路径请求 1 次,实际 %d", attempts["/cn/update/promotion.md"])
}
}

1
scripts/testdata/empty.dockerignore vendored Normal file
View File

@@ -0,0 +1 @@
# empty on purpose

View File

@@ -1,16 +1,16 @@
[
{
"date": "2025-09-29",
"model_name": "DeepSeek-V3.2-Exp",
"provider_name": "DeepSeek",
"operator_name": "DeepSeek",
"summary": "官方活动窗口出现后,值得重新评估低成本推理和批量调用方案。",
"audience": "适合计划趁活动窗口压低推理成本的团队",
"baseline": "活动窗口开启",
"trust_label": "官方来源 / 一级证据",
"source_kind_label": "官方活动页",
"primary_source": "https://api-docs.deepseek.com/news/news250929",
"evidence_detail": "官方活动页记录 V3.2-Exp 在活动窗口内价格下调 50%+",
"priority": 115
"date": "2026-05-27",
"model_name": "GPT-5.6",
"provider_name": "OpenAI",
"operator_name": "OpenAI",
"summary": "GPT-5.6 提前泄露信号出现,需立即复查其是否改变默认高端模型路线与预算预期。",
"audience": "适合关注高端模型路线图、预算和替换窗口的团队",
"baseline": "提前泄露",
"trust_label": "行业情报 / 待官方确认",
"source_kind_label": "泄露情报",
"primary_source": "https://openai.example.com/gpt-5-6-leak",
"evidence_detail": "多个公开情报源出现 GPT-5.6 命名与规格片段,尚待官方正式发布确认。",
"priority": 135
}
]

View File

@@ -4,12 +4,15 @@ set -euo pipefail
SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd)"
. "$SCRIPT_DIR/verify_common.sh"
. "$SCRIPT_DIR/secret_gate_lib.sh"
DB_URL="${DATABASE_URL:-host=/var/run/postgresql dbname=llm_intelligence user=long sslmode=disable}"
SERVER_BIN="/tmp/llm_phase6_server"
SERVER_LOG="/tmp/llm_phase6_server.log"
SERVER_PORT="${PHASE6_PORT:-}"
SERVER_PID=""
API_AUTH_TOKEN="${API_AUTH_TOKEN:-phase6-local-token}"
cleanup() {
if [ -n "${SERVER_PID:-}" ] && kill -0 "$SERVER_PID" >/dev/null 2>&1; then
@@ -40,8 +43,9 @@ reserve_server_port() {
}
start_server() {
DATABASE_URL="$DB_URL" PORT="$SERVER_PORT" "$SERVER_BIN" >"$SERVER_LOG" 2>&1 &
DATABASE_URL="$DB_URL" PORT="$SERVER_PORT" API_AUTH_TOKEN="$API_AUTH_TOKEN" "$SERVER_BIN" >"$SERVER_LOG" 2>&1 &
SERVER_PID=$!
for _ in $(seq 1 20); do
if ! kill -0 "$SERVER_PID" >/dev/null 2>&1; then
return 1
@@ -165,7 +169,7 @@ else
fi
check_shell "API Server 可构建" "go build -o /dev/null ./cmd/server"
check_shell "健康检查脚本通过" "DATABASE_URL='$DB_URL' bash healthcheck.sh"
check_shell "密钥未硬编码进源码" "grep -R -n 'sk-' cmd internal frontend/src scripts .github/workflows --include='*.go' --include='*.ts' --include='*.tsx' --include='*.sh' --include='*.yml' --include='*.yaml' --exclude='verify_phase6.sh' >/tmp/llm_phase6_secret_scan.out 2>/dev/null; test ! -s /tmp/llm_phase6_secret_scan.out"
check_shell "源码与环境文件未包含明显硬编码密钥" "source scripts/secret_gate_lib.sh && secret_scan_paths . cmd internal frontend/src scripts .github/workflows && secret_env_files .dockerignore"
run_window_gate
@@ -174,7 +178,7 @@ if go build -o "$SERVER_BIN" ./cmd/server >/tmp/llm_phase6_server_build.out 2>/t
pass "API /health 可用"
set +e
api_metrics="$(curl -sS -o /tmp/llm_phase6_models.json -w '%{http_code} %{time_total}' "http://127.0.0.1:${SERVER_PORT}/api/v1/models")"
api_metrics="$(curl -sS -H "Authorization: Bearer ${API_AUTH_TOKEN}" -o /tmp/llm_phase6_models.json -w '%{http_code} %{time_total}' "http://127.0.0.1:${SERVER_PORT}/api/v1/models")"
api_rc=$?
set -e
if [ "$api_rc" -eq 0 ]; then
@@ -202,7 +206,7 @@ if go build -o "$SERVER_BIN" ./cmd/server >/tmp/llm_phase6_server_build.out 2>/t
fi
set +e
plan_metrics="$(curl -sS -o /tmp/llm_phase6_subscription_plans.json -w '%{http_code} %{time_total}' "http://127.0.0.1:${SERVER_PORT}/api/v1/subscription-plans")"
plan_metrics="$(curl -sS -H "Authorization: Bearer ${API_AUTH_TOKEN}" -o /tmp/llm_phase6_subscription_plans.json -w '%{http_code} %{time_total}' "http://127.0.0.1:${SERVER_PORT}/api/v1/subscription-plans")"
plan_rc=$?
set -e
if [ "$plan_rc" -eq 0 ]; then
@@ -232,5 +236,6 @@ fi
check_shell "Phase 6 性能文档存在" "test -f docs/PERFORMANCE_TEST.md"
check_shell "前端已具备测试入口" "cd frontend && npm run test -- --run >/tmp/llm_phase6_frontend_test.out 2>/tmp/llm_phase6_frontend_test.err"
check_shell "secret gate 独立测试通过" "bash scripts/secret_gate_test.sh"
finish_phase