refactor: clean up project structure
- Remove old review reports (keep latest only) - Move docs/ to deploy/docs-backup/ - Move performance-testing/ to deploy/ - Clean up test output files - Organize root directory
This commit is contained in:
192
deploy/performance-testing/config/database-optimization.md
Normal file
192
deploy/performance-testing/config/database-optimization.md
Normal file
@@ -0,0 +1,192 @@
|
||||
# Sub2API 数据库连接池优化配置
|
||||
|
||||
## 📊 当前配置分析
|
||||
|
||||
根据 `backend/internal/repository/db_pool.go` 和 `backend/internal/config/config.go` 的分析,当前数据库连接池配置支持以下参数:
|
||||
|
||||
```go
|
||||
type dbPoolSettings struct {
|
||||
MaxOpenConns int // 最大打开连接数
|
||||
MaxIdleConns int // 最大空闲连接数
|
||||
ConnMaxLifetime time.Duration // 连接最大生命周期
|
||||
ConnMaxIdleTime time.Duration // 空闲连接最大存活时间
|
||||
}
|
||||
```
|
||||
|
||||
## 🎯 推荐配置
|
||||
|
||||
### 小规模部署(< 100 QPS)
|
||||
|
||||
```yaml
|
||||
# config.yaml
|
||||
database:
|
||||
max_open_conns: 25
|
||||
max_idle_conns: 10
|
||||
conn_max_lifetime_minutes: 30
|
||||
conn_max_idle_time_minutes: 5
|
||||
```
|
||||
|
||||
### 中等规模(100-500 QPS)
|
||||
|
||||
```yaml
|
||||
# config.yaml
|
||||
database:
|
||||
max_open_conns: 50
|
||||
max_idle_conns: 20
|
||||
conn_max_lifetime_minutes: 30
|
||||
conn_max_idle_time_minutes: 5
|
||||
```
|
||||
|
||||
### 大规模部署(500-2000 QPS)
|
||||
|
||||
```yaml
|
||||
# config.yaml
|
||||
database:
|
||||
max_open_conns: 100
|
||||
max_idle_conns: 30
|
||||
conn_max_lifetime_minutes: 15
|
||||
conn_max_idle_time_minutes: 3
|
||||
```
|
||||
|
||||
### 超大规模(> 2000 QPS)
|
||||
|
||||
```yaml
|
||||
# config.yaml
|
||||
database:
|
||||
max_open_conns: 200
|
||||
max_idle_conns: 50
|
||||
conn_max_lifetime_minutes: 10
|
||||
conn_max_idle_time_minutes: 2
|
||||
```
|
||||
|
||||
## 🔧 配置参数详解
|
||||
|
||||
### MaxOpenConns
|
||||
|
||||
| 场景 | 推荐值 | 说明 |
|
||||
|------|--------|------|
|
||||
| 小规模 | 25-50 | 避免连接数过多占用资源 |
|
||||
| 中等 | 50-100 | 平衡并发和资源消耗 |
|
||||
| 大规模 | 100-200 | 需要配合应用水平扩展 |
|
||||
|
||||
**计算公式**:
|
||||
```
|
||||
MaxOpenConns = 预期并发请求数 × (1 + 峰值系数) / 单请求平均连接时间
|
||||
```
|
||||
|
||||
### MaxIdleConns
|
||||
|
||||
| 场景 | 推荐值 | 说明 |
|
||||
|------|--------|------|
|
||||
| 小规模 | 5-10 | 保持基础连接预热 |
|
||||
| 中等 | 15-25 | 覆盖正常并发 |
|
||||
| 大规模 | 30-50 | 减少连接建立开销 |
|
||||
|
||||
**原则**:`MaxIdleConns <= MaxOpenConns * 0.5`
|
||||
|
||||
### ConnMaxLifetime
|
||||
|
||||
| 场景 | 推荐值 | 说明 |
|
||||
|------|--------|------|
|
||||
| 开发/测试 | 1小时 | 减少连接重建 |
|
||||
| 生产 | 15-30分钟 | 平衡连接重建开销和资源 |
|
||||
|
||||
**原则**:应小于 PostgreSQL 的 `idle_in_transaction_session_timeout`
|
||||
|
||||
### ConnMaxIdleTime
|
||||
|
||||
| 场景 | 推荐值 | 说明 |
|
||||
|------|--------|------|
|
||||
| 高频 | 1-3分钟 | 快速回收空闲连接 |
|
||||
| 正常 | 5-10分钟 | 平衡连接复用和资源 |
|
||||
|
||||
## 📈 性能调优步骤
|
||||
|
||||
### 1. 基准测试
|
||||
|
||||
```bash
|
||||
# 使用 pgbench 进行基准测试
|
||||
pgbench -h localhost -U postgres -d sub2api -c 10 -j 4 -T 60
|
||||
|
||||
# 测试不同连接池配置
|
||||
for conn in 10 25 50 100; do
|
||||
echo "Testing MaxOpenConns=$conn"
|
||||
# 调整配置后重新测试
|
||||
done
|
||||
```
|
||||
|
||||
### 2. 监控关键指标
|
||||
|
||||
通过 Prometheus 查询:
|
||||
|
||||
```promql
|
||||
# 数据库连接使用率
|
||||
sub2api_db_connections{state="active"} / sub2api_db_connections{state="max"} * 100
|
||||
|
||||
# 等待连接的请求数
|
||||
pg_stat_activity_waiting
|
||||
|
||||
# 连接等待时间
|
||||
pg_stat_activity.max_wait_time
|
||||
```
|
||||
|
||||
### 3. 优化建议
|
||||
|
||||
**问题:高连接等待**
|
||||
- 增加 `MaxOpenConns`
|
||||
- 检查慢查询
|
||||
- 优化索引
|
||||
|
||||
**问题:频繁连接重建**
|
||||
- 增加 `MaxIdleConns`
|
||||
- 增加 `ConnMaxLifetime`
|
||||
|
||||
**问题:内存持续增长**
|
||||
- 减少 `MaxIdleConns`
|
||||
- 减少 `ConnMaxIdleTime`
|
||||
|
||||
## 🚀 PostgreSQL 服务端优化
|
||||
|
||||
除了应用层配置,还需要在 PostgreSQL 服务端进行优化:
|
||||
|
||||
```sql
|
||||
-- postgresql.conf 优化
|
||||
|
||||
-- 连接池相关
|
||||
max_connections = 200
|
||||
|
||||
-- 内存相关
|
||||
shared_buffers = 256MB
|
||||
effective_cache_size = 1GB
|
||||
work_mem = 16MB
|
||||
maintenance_work_mem = 128MB
|
||||
|
||||
-- 查询优化
|
||||
random_page_cost = 1.1
|
||||
effective_io_concurrency = 200
|
||||
|
||||
-- 写入优化
|
||||
wal_buffers = 16MB
|
||||
checkpoint_completion_target = 0.9
|
||||
|
||||
-- 连接优化
|
||||
tcp_keepalives_idle = 60
|
||||
tcp_keepalives_interval = 10
|
||||
tcp_keepalives_count = 6
|
||||
```
|
||||
|
||||
## 📊 性能基线参考
|
||||
|
||||
| 连接池配置 | 10 VU | 50 VU | 100 VU | 200 VU |
|
||||
|-----------|-------|-------|--------|--------|
|
||||
| 25/10 | 200ms | 500ms | 1000ms | 2000ms |
|
||||
| 50/20 | 150ms | 300ms | 600ms | 1200ms |
|
||||
| 100/30 | 100ms | 200ms | 400ms | 800ms |
|
||||
| 200/50 | 80ms | 150ms | 300ms | 600ms |
|
||||
|
||||
## ⚠️ 注意事项
|
||||
|
||||
1. **不要盲目增大连接池**:PostgreSQL 单实例推荐 100-200 连接
|
||||
2. **监控实际使用**:使用 `pg_stat_activity` 观察连接使用情况
|
||||
3. **考虑使用 PgBouncer**:高并发场景推荐使用连接池中间件
|
||||
4. **测试峰值场景**:确保峰值负载下连接池不会成为瓶颈
|
||||
Reference in New Issue
Block a user