pham/tokens-reef
2
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
2,771 Commits 3 Branches 0 Tags
fa0aacc5592e7a7ded1e1059bef5e25c3bc3bc6c
Commit Graph

519 Commits

Author SHA1 Message Date
pham
fa0aacc559 fix startup bootstrap recovery and local verification 2026-04-23 10:27:13 +08:00
pham
32b2c23a04 chore: ignore local artifacts and remove dead api key trend impl
Some checks failed
CI / test (push) Has been cancelled
Details
CI / golangci-lint (push) Has been cancelled
Details
Security Scan / backend-security (push) Has been cancelled
Details
Security Scan / frontend-security (push) Has been cancelled
Details
2026-04-22 11:58:17 +08:00
User
fded346295 refactor(goroutine): replace bare goroutines with managed executors
Some checks failed
CI / test (push) Has been cancelled
Details
CI / golangci-lint (push) Has been cancelled
Details
Security Scan / backend-security (push) Has been cancelled
Details
Security Scan / frontend-security (push) Has been cancelled
Details 
- Add internal/pkg/safego/ utility for safe goroutine launching
  with panic recovery, structured logging (slog + zap), and optional
  error callback support

- Replace 8 fire-and-forget goroutines in service layer:
  user_service.go (1), subscription_service.go (4), redeem_service.go (2),
  promo_service.go (1) — all now use safego.Go()

- Add inline panic recovery to 6 long-running goroutines:
  pricing_service.go scheduler loop
  Redis PubSub subscribers: tls_fingerprint/error_passthrough/api_key caches
  OAuth session cleanup: openai/oauth/geminicli
  S3 stream upload in sora_s3_storage.go

- All changes verified: go build PASS, service tests ALL PASS,
  repository tests ALL PASS
 2026-04-18 10:43:00 +08:00
User
60d15d2ba4 feat(ops): add usage_logs partition status to ops dashboard 
Add partition management integration to the smart ops system:
- Backend: Add GetUsageLogsPartitionStatus endpoint in OpsHandler
- Backend: Add partition query methods in OpsRepository
- Backend: Add UsageLogsPartitionStatus type in OpsService
- Frontend: Add OpsPartitionStatusCard component
- Frontend: Add partition status display in OpsDashboard
- i18n: Add Chinese and English translations

The partition status card shows:
- Whether usage_logs is partitioned
- Current row count vs threshold (100K)
- Partition count (if partitioned)
- Warning message when partitioning is recommended

This allows administrators to monitor partition status directly
from the ops dashboard without checking server logs.
 2026-04-16 23:16:17 +08:00
User
eb5adbbae5 feat(ops): add usage_logs partitioning check at deployment startup
Some checks failed
CI / test (push) Has been cancelled
Details
CI / golangci-lint (push) Has been cancelled
Details
Security Scan / backend-security (push) Has been cancelled
Details
Security Scan / frontend-security (push) Has been cancelled
Details 
Add CheckUsageLogsPartitioning function that:
- Checks if usage_logs table is partitioned
- Warns with prominent banner if not partitioned and rows > 100K
- Provides actionable guidance for manual partition migration

This helps operators identify performance risks early and take
appropriate action before data volume causes issues.
 2026-04-16 22:11:15 +08:00
User
ebe2d250f3 feat(security): enhance JWT secret auto-generation warning 
Add prominent warning messages when JWT secret is auto-generated:
- Use multi-line banner format for better visibility
- Include actionable guidance for production deployments
- Update both setup.go and security_secret_bootstrap.go

This helps operators notice the security concern and take
appropriate action before deploying to production.
 2026-04-16 21:37:03 +08:00
User
d96a9f384a feat: merge sub2apipro features and add Chinese model pricing
Some checks failed
CI / test (push) Has been cancelled
Details
CI / golangci-lint (push) Has been cancelled
Details
Security Scan / backend-security (push) Has been cancelled
Details
Security Scan / frontend-security (push) Has been cancelled
Details 
## Merged Features from sub2apipro
- Sora video generation integration (OpenAI Sora API)
- Group management enhancements
- Usage log improvements
- Security headers middleware

## Chinese Model Pricing Updates
- GLM-5, GLM-5-Turbo, GLM-5.1, GLM-4.7, GLM-4.5-Air
- Baichuan4, Baichuan4-Turbo, Baichuan4-Air, Baichuan-M3-Plus
- DeepSeek-V3, DeepSeek-V3.2, DeepSeek-R1
- Qwen3-8B (free), Qwen2.5-72B-Instruct

## URL Whitelist Additions
- api.baichuan-ai.com (百川智能)
- api.siliconflow.cn (硅基流动)
- api.z.ai (智谱国际)
- api.groq.com (Groq加速推理)

## Documentation
- Added merge guide (docs/MERGE_GUIDE.md)
- Added quick reference (docs/MERGE_QUICKREF.md)
- Added review reports (docs/reviews/)
 2026-04-15 12:02:07 +08:00
IanShaw027
7dc7ff22d2 fix: preserve messages dispatch config in repository hydration 2026-04-10 18:13:18 +08:00
IanShaw027
269c7a065c fix(test): restore integration expectations 2026-04-09 22:08:42 +08:00
IanShaw027
b6946e78a2 fix(lint): restore repository sort helpers 2026-04-09 21:49:10 +08:00
IanShaw027
2b70d1d332 merge upstream main into fix/bug-cleanup-main 2026-04-09 21:35:48 +08:00
Wesley Liddick
00c08c574e Merge pull request #1539 from warksu/fix/loadfactor-not-synced-to-scheduler-cache 
fix: 同步 LoadFactor 到调度快照缓存
 2026-04-09 21:15:40 +08:00
IanShaw027
62962c05f1 fix(lint): 修复 CI 中的 ineffassign 和 unused 代码告警，修正 group 排序集成测试兼容性 2026-04-09 19:25:08 +08:00
swark2006
118ff85fbf fix: 同步 LoadFactor 到调度快照缓存 
LoadFactor 字段在构建调度快照缓存时缺失，导致调度算法
EffectiveLoadFactor() 无法使用配置的负载因子，回退到 Concurrency。
这会影响账号的负载率计算，进而影响调度优先级。

修复：在 buildSchedulerMetadataAccount 中添加 LoadFactor 字段。
 2026-04-09 18:50:11 +08:00
IanShaw027
5f8e60a1b7 feat(table): 表格排序与搜索改为后端处理 2026-04-09 18:14:28 +08:00
IanShaw027
23c4d592f8 feat(group): 增加messages调度模型映射配置 2026-04-09 12:29:28 +08:00
ius
265687b56d fix: 优化调度快照缓存以避免 Redis 大 MGET 2026-04-08 10:39:15 -07:00
Wesley Liddick
bf2b590273 Merge pull request #1397 from weak-fox/fix/active-filter-excludes-rate-limited 
解决账号管理中“正常”筛选包含限流中账号
 2026-04-07 08:26:51 +08:00
erio
19655a15f1 fix: gofmt formatting 2026-04-05 18:56:40 +08:00
erio
f345b0f595 fix: use upstream versions of shared files and remove only Sora code 
Restore gateway_service.go, setting_handler.go, routes/admin.go,
dto/settings.go, group_repo.go, api_key_repo.go, wire_gen.go to
upstream/main versions and surgically remove only Sora references.

This preserves upstream-only features (RequireOauthOnly, RequirePrivacySet,
GroupResolution, etc.) that were missing when using release branch versions.
 2026-04-05 18:48:41 +08:00
erio
c6089ccb33 fix: remove Sora DI from wire_gen.go and clean remaining upstream Sora references 2026-04-05 17:50:01 +08:00
erio
1b1c08f7fb fix: remove media_type from usage_log SQL queries and test stubs 
- Remove media_type column from all INSERT/SELECT/SCAN in usage_log_repo
- Remove media_type mock arg from request_type and integration tests
- Adjust scan stub value arrays from 47 to 46 elements
 2026-04-05 17:22:22 +08:00
erio
0c72be0403 fix: gofmt alignment and remove media_type from usage_log repo queries 2026-04-05 17:22:22 +08:00
erio
b4bd89b96b fix: remove media_type from usage_log SQL queries + gofmt + inline test helpers 2026-04-05 17:22:22 +08:00
erio
5bb8b2add6 fix: resolve CI failures — gofmt, unused functions, test parameter mismatches 
- gofmt: user.go, config_test.go, group_handler.go, smart_retry_test.go
- Remove unused: mergeGroupIDs, resolveProxyURL, "time" import
- Fix api_contract_test.go: remove extra Sora args from NewAdminService,
  NewSettingHandler, NewAccountHandler; remove Sora field expectations
- Fix account_test_service_openai_test.go: restore test helpers
 2026-04-05 17:22:22 +08:00
erio
62e80c602d revert: completely remove all Sora functionality 2026-04-05 17:11:01 +08:00
erio
feb6999d9a fix: channel cache fail-close, group conflict check across pages, status toggle stale data 
- GetGroupPlatforms failure now stores error-TTL cache and returns error (fail-close)
- Frontend group-to-channel conflict map loads all channels instead of current page only
- Toggle channel status reloads list when active filter would hide the changed item
 2026-04-04 11:25:01 +08:00
erio
3851628a43 fix: resolve golangci-lint issues 
- Fix errcheck: defer rows.Close() with nolint
- Fix errcheck: type assertion with ok check in channel cache
- Fix staticcheck ST1005: lowercase error string
- Fix staticcheck SA5011: nil check cost before use in openai gateway
- Fix gofmt: format chatcompletions_to_responses.go
 2026-04-04 11:17:24 +08:00
erio
d72ac92694 feat: image output token billing, channel-mapped billing source, credits balance precheck 
- Parse candidatesTokensDetails from Gemini API to separate image/text output tokens
- Add image_output_tokens and image_output_cost to usage_log (migration 089)
- Support per-image-token pricing via output_cost_per_image_token from model pricing data
- Channel pricing ImageOutputPrice override works in token billing mode
- Auto-fill image_output_price in channel pricing form from model defaults
- Add "channel_mapped" billing model source as new default (migration 088)
- Bills by model name after channel mapping, before account mapping
- Fix channel cache error TTL sign error (115s → 5s)
- Fix Update channel only invalidating new groups, not removed groups
- Fix frontend model_mapping clearing sending undefined instead of {}
- Credits balance precheck via shared AccountUsageService cache before injection
- Skip credits injection for accounts with insufficient balance
- Don't mark credits exhausted for "exhausted your capacity on this model" 429s
 2026-04-04 11:15:59 +08:00
erio
2555951be4 feat(channel): 渠道管理全链路集成 — 模型映射、定价、限制、用量统计 
- 渠道模型映射：支持精确匹配和通配符映射，按平台隔离
- 渠道模型定价：支持 token/按次/图片三种计费模式，区间分层定价
- 模型限制：渠道可限制仅允许定价列表中的模型
- 计费模型来源：支持 requested/upstream 两种计费模型选择
- 用量统计：usage_logs 新增 channel_id/model_mapping_chain/billing_tier/billing_mode 字段
- Dashboard 支持 model_source 维度（requested/upstream/mapping）查看模型统计
- 全部 gateway handler 统一接入 ResolveChannelMappingAndRestrict
- 修复测试：同步 SoraGenerationRepository 接口、SQL INSERT 参数、scan 字段
 2026-04-04 11:13:58 +08:00
erio
a51e0047b7 feat(usage): 使用记录增加计费模式字段 — 记录/展示/筛选 token/按次/图片 
- DB: usage_logs 表新增 billing_mode VARCHAR(20) 列
- 后端: RecordUsage 写入时根据 image_count 判定计费模式
- 前端: 使用记录表格新增计费模式 badge 列 + 筛选下拉
 2026-04-04 11:11:06 +08:00
erio
0b1ce6be8f feat(channel): 缓存扁平化 + 网关映射集成 + 计费模式统一 + 模型限制 
- 缓存按 (groupID, platform, model) 三维 key 扁平化，避免跨平台同名模型冲突
- buildCache 批量查询 group platform，按平台过滤展开定价和映射
- model_mapping 改为嵌套格式 {platform: {src: dst}}
- channel_model_pricing 新增 platform 列
- 前端按平台维度重构：每个平台独立配置分组/映射/定价
- 迁移 086: platform 列 + model_mapping 嵌套格式迁移
 2026-04-04 11:09:28 +08:00
erio
ebac0dc628 feat(channel): 缓存扁平化 + 网关映射集成 + 计费模式统一 + 模型限制 
- 缓存重构为 O(1) 哈希结构 (pricingByGroupModel, mappingByGroupModel)
- 渠道模型映射接入网关流程 (Forward 前应用, a→b→c 映射链)
- 新增 billing_model_source 配置 (请求模型/最终模型计费)
- usage_logs 新增 channel_id, model_mapping_chain, billing_tier 字段
- 每种计费模式统一支持默认价格 + 区间定价
- 渠道模型限制开关 (restrict_models)
- 分组按平台分类展示 + 彩色图标
- 必填字段红色星号 + 模型映射 UI
- 去除模型通配符支持
 2026-04-04 11:09:01 +08:00
erio
29d58f2414 feat(channel): 模型映射 + 分组搜索 + 卡片折叠 + 冲突校验 
- 渠道模型映射：新增 model_mapping JSONB 字段，在账号映射之前执行
- 分组选择：添加搜索过滤 + 平台图标
- 定价卡片：支持折叠/展开，已有数据默认折叠
- 模型冲突校验：前后端均禁止同一渠道内重复模型
- 迁移 083: channels 表添加 model_mapping 列
 2026-04-04 11:06:36 +08:00
erio
983fe58959 fix: CI lint/test fixes — gofmt, errcheck, handler test args 2026-04-04 11:01:22 +08:00
erio
91c9b8d062 feat(channel): 渠道管理系统 — 多模式定价 + 统一计费解析 
Cherry-picked from release/custom-0.1.106: a9117600
 2026-04-04 11:00:55 +08:00
QTom
aeed2eb9ad feat(group-filter): 分组账号过滤控制 — require_oauth_only + require_privacy_set 
为 OpenAI/Antigravity/Anthropic/Gemini 分组新增两个布尔控制字段：
- require_oauth_only: 创建/更新账号绑定分组时拒绝 apikey 类型加入
- require_privacy_set: 调度选号时跳过 privacy 未成功设置的账号并标记 error

后端：Ent schema 新增字段 + 迁移、Group CRUD 全链路透传、
      gateway_service 与 openai_account_scheduler 两套调度路径过滤
前端：创建/编辑表单 toggle 开关（OpenAI/Antigravity/Anthropic/Gemini 平台可见）

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-03-31 13:04:55 +08:00
weak-fox
a61d58716f fix(admin): exclude rate-limited accounts from active filter 2026-03-31 00:00:46 +08:00
Wesley Liddick
6a2cf09ee0 Merge pull request #1349 from touwaeriol/feat/antigravity-internal500-penalty 
feat(antigravity): progressive penalty for consecutive INTERNAL 500 errors
 2026-03-30 15:54:04 +08:00
erio
3ee6f085db refactor: extract internal500 penalty logic to dedicated file 
Move constants, detection, and penalty functions from
antigravity_gateway_service.go to antigravity_internal500_penalty.go.
Fix gofmt alignment and replace hardcoded duration strings with
constant references.
 2026-03-27 20:11:24 +08:00
erio
093a5a260e feat(antigravity): progressive penalty for consecutive INTERNAL 500 errors 
When an antigravity account returns 500 "Internal error encountered."
on all 3 retry attempts, increment a Redis counter and apply escalating
penalties:
- 1st round: temp unschedulable 10 minutes
- 2nd round: temp unschedulable 10 hours
- 3rd round: permanently mark as error

Counter resets on any successful response (< 400).
 2026-03-27 20:11:24 +08:00
YilinMacAir
1f39bf8a78 fix:修复由于数据库唯一键导致软删除apikey后key没有被释放后续无法再自定义相同的key 2026-03-27 16:37:10 +08:00
shaw
1854050df3 feat(tls-fingerprint): 新增 TLS 指纹 Profile 数据库管理及代码质量优化 
新增功能：
- 新增 TLS 指纹 Profile CRUD 管理（Ent schema + 迁移 + Admin API + 前端管理界面）
- 支持账号绑定数据库中的自定义 TLS Profile，或随机选择（profile_id=-1）
- HTTPUpstream.DoWithTLS 接口从 bool 改为 *tlsfingerprint.Profile，支持按账号指定 Profile
- AccountUsageService 注入 TLSFingerprintProfileService，统一 usage 场景与网关的 Profile 解析逻辑

代码优化：
- 删除已被 TLSFingerprintProfileService 完全取代的 registry.go 死代码（418 行）
- 提取 3 个 dialer 的重复 TLS 握手逻辑为 performTLSHandshake() 共用函数
- 修复 GetTLSFingerprintProfileID 缺少 json.Number 处理的 bug
- gateway_service.Forward 中 ResolveTLSProfile 从重试循环内重复调用改为预解析局部变量
- 删除冗余的 buildClientHelloSpec() 单行 wrapper 和 int64(e.ID) 无效转换
- tls_fingerprint_profile_cache.go 日志从 log.Printf 改为 slog 结构化日志
- dialer_capture_test.go 添加 //go:build integration 标签，防止 CI 失败
- 去重 TestProfileExpectation 类型至共享 test_types_test.go
- 修复 9 个测试文件缺少 tlsfingerprint import 的编译错误
- 修复 error_policy_integration_test.go 中 handleError 回调签名被错误替换的问题
 2026-03-27 14:33:05 +08:00
shaw
b20e142249 feat: 网关请求头 wire casing 保持、转发行为开关、调试日志增强及 accept-encoding 恢复 
- 新增 header_util.go，通过 setHeaderRaw/getHeaderRaw/addHeaderRaw 绕过
  Go 的 canonical-case 规范化，保持真实 Claude CLI 抓包的请求头大小写
  （如 "x-app" 而非 "X-App"，"X-Stainless-OS" 而非 "X-Stainless-Os"）
- 新增管理后台开关：指纹统一化（默认开启）和 metadata 透传（默认关闭），
  使用 atomic.Value + singleflight 缓存模式，60s TTL
- 调试日志从控制台 body 打印升级为文件级完整快照
  （按真实 wire 顺序输出 headers + 格式化 JSON body + 上下文元数据）
- 恢复 accept-encoding 到白名单，在 http_upstream.go 新增 decompressResponseBody
  处理 gzip/brotli/deflate 解压（Go 显式设置 Accept-Encoding 时不会自动解压）
- OAuth 服务 axios UA 从 1.8.4 更新至 1.13.6
- 测试断言改用 getHeaderRaw 适配 raw header 存储方式
 2026-03-26 11:17:25 +08:00
Wesley Liddick
c7137dffa8 Merge pull request #1218 from LvyuanW/openai-runtime-recheck 
fix(openai): prevent rescheduling rate-limited accounts
 2026-03-24 15:21:18 +08:00
Wesley Liddick
68d7ec9155 Merge pull request #1220 from weak-fox/feat/account-privacy-mode-filter 
feat: 管理员账号列表支持按 Privacy 状态筛选
 2026-03-24 15:18:30 +08:00
weak-fox
4838ab74b3 feat(admin): add account privacy mode filter 2026-03-23 10:16:52 +08:00
Wang Lvyuan
ad7c10727a fix(account): preserve runtime state during credentials-only updates 2026-03-23 03:49:28 +08:00
Ethan0x0000
a2418c6040 feat(ops): adapt repository INSERT/SELECT + add setOpsEndpointContext in error logger middleware 2026-03-21 23:38:00 +08:00
Ethan0x0000
9259dcb6f5 test(repo): cover requested model repository semantics 
Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-openagent)

Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>
 2026-03-21 01:23:20 +08:00