91a0b77f7a
- 修改 shouldVerifyCacheManager_withMaximumIntegerTtl 为 shouldVerifyCacheManager_withMaximumAllowedTtl - 使用正确的最大TTL值（10080分钟，7天）而不是 Integer.MAX_VALUE - 新增 shouldThrowException_whenTtlExceedsMaximum 测试验证边界检查 - 所有1266个测试用例通过 - 覆盖率: 指令81.89%, 行88.48%, 分支51.55% docs: 添加项目状态报告 - 生成 PROJECT_STATUS_REPORT.md 详细记录项目当前状态 - 包含质量指标、已完成功能、待办事项和技术债务
569 lines
15 KiB
Markdown
569 lines
15 KiB
Markdown
# AI测试常见问题速查表
|
||
|
||
基于蚊子项目1210+测试经验
|
||
快速识别和修复AI生成测试的常见问题
|
||
|
||
---
|
||
|
||
## 🚨 第一类：虚假测试（立即修复）
|
||
|
||
### 1.1 测试框架本身
|
||
|
||
❌ **错误示例**（AI常生成）
|
||
```java
|
||
@Test
|
||
void shouldReturnName_whenGetName() {
|
||
User user = new User("John");
|
||
assertEquals("John", user.getName()); // 测试了Lombok
|
||
}
|
||
|
||
@Test
|
||
void shouldSetAge_whenSetAge() {
|
||
User user = new User();
|
||
user.setAge(25);
|
||
assertEquals(25, user.getAge()); // 测试了setter
|
||
}
|
||
```
|
||
|
||
✅ **正确做法**
|
||
```java
|
||
// 不测试getter/setter，除非有自定义逻辑
|
||
// 删除或合并为参数化测试
|
||
|
||
@ParameterizedTest
|
||
@CsvSource({
|
||
"John, 25, active",
|
||
"Jane, 30, inactive"
|
||
})
|
||
void shouldCreateUser_withValidData(String name, int age, String status) {
|
||
User user = User.builder()
|
||
.name(name)
|
||
.age(age)
|
||
.status(status)
|
||
.build();
|
||
|
||
assertThat(user.getName()).isEqualTo(name);
|
||
assertThat(user.getAge()).isEqualTo(age);
|
||
assertThat(user.getStatus()).isEqualTo(status);
|
||
}
|
||
```
|
||
|
||
**检查方法**
|
||
```bash
|
||
# 统计getter/setter测试数量
|
||
grep -r "void should.*when[Gg]et\|void should.*when[Ss]et" src/test/java | wc -l
|
||
|
||
# 如果>20个，需要清理
|
||
```
|
||
|
||
---
|
||
|
||
### 1.2 虚假断言
|
||
|
||
❌ **错误示例**（无意义断言）
|
||
```java
|
||
@Test
|
||
void shouldProcessOrder() {
|
||
Order result = orderService.process(orderRequest);
|
||
|
||
assertNotNull(result); // 太弱
|
||
assertNotNull(result.getId()); // 还是太弱
|
||
assertTrue(result.getTotal() > 0); // 不够具体
|
||
// 缺少：验证具体金额、状态、库存扣减
|
||
}
|
||
|
||
@Test
|
||
void shouldValidateInput() {
|
||
// 没有实际验证
|
||
assertTrue(true); // 总是通过！
|
||
}
|
||
```
|
||
|
||
✅ **正确做法**
|
||
```java
|
||
@Test
|
||
void shouldCalculateTotalCorrectly_whenProcessingOrder() {
|
||
// Given
|
||
OrderRequest request = createOrderRequest(BigDecimal.valueOf(100), 2);
|
||
|
||
// When
|
||
Order result = orderService.process(request);
|
||
|
||
// Then - 验证具体业务结果
|
||
assertThat(result.getTotal())
|
||
.isEqualByComparingTo(BigDecimal.valueOf(200)); // 精确验证
|
||
assertThat(result.getStatus()).isEqualTo("PAID");
|
||
assertThat(result.getItems()).hasSize(2);
|
||
|
||
// 验证副作用
|
||
verify(inventoryService).deductStock("SKU001", 2);
|
||
verify(paymentService).charge(eq("USER001"), eq(BigDecimal.valueOf(200)));
|
||
|
||
// 验证数据库状态
|
||
Order saved = orderRepository.findById(result.getId()).orElseThrow();
|
||
assertThat(saved.getCreatedAt()).isNotNull();
|
||
}
|
||
```
|
||
|
||
**识别脚本**
|
||
```bash
|
||
# 查找虚假断言
|
||
find src/test/java -name "*Test.java" -exec grep -l "assertTrue(true)\|assertNotNull(new" {} \;
|
||
|
||
# 统计assertNotNull使用率
|
||
find src/test/java -name "*Test.java" -exec grep -c "assertNotNull" {} \; | sort -rn | head -20
|
||
```
|
||
|
||
---
|
||
|
||
## 🎯 第二类：边界条件缺失（必须补充）
|
||
|
||
### 2.1 系统化边界矩阵
|
||
|
||
| 类型 | 必须测试的值 | AI常遗漏 |
|
||
|------|------------|---------|
|
||
| **数值** | MIN_VALUE, -1, 0, 1, MAX_VALUE | 极大/极小值 |
|
||
| **字符串** | null, "", "a", "最大长度", "特殊字符🔑" | 超长、Unicode |
|
||
| **集合** | null, empty, 1 element, max size | null、空列表 |
|
||
| **时间** | MIN, epoch, now, MAX, null | 边界时间 |
|
||
| **并发** | 1 thread, 10 threads, race condition | 并发测试 |
|
||
|
||
❌ **错误示例**（缺少边界）
|
||
```java
|
||
@Test
|
||
void shouldCalculateDiscount() {
|
||
BigDecimal price = BigDecimal.valueOf(100);
|
||
String userType = "VIP";
|
||
|
||
BigDecimal discount = calculator.calculate(price, userType);
|
||
|
||
assertEquals(BigDecimal.valueOf(80), discount); // 只测试正常值
|
||
}
|
||
```
|
||
|
||
✅ **正确做法**（参数化边界测试）
|
||
```java
|
||
@ParameterizedTest
|
||
@CsvSource({
|
||
"100, VIP, 80", // 正常VIP
|
||
"100, NORMAL, 100", // 正常普通用户
|
||
"0, VIP, 0", // 边界：0价格
|
||
"-10, VIP, -8", // 边界：负数价格
|
||
"100, null, 100", // 边界：null用户类型
|
||
"100, UNKNOWN, 100", // 边界：未知类型
|
||
"999999999, VIP, 799999999" // 边界：极大值
|
||
})
|
||
void shouldCalculateDiscount_withBoundaryValues(
|
||
BigDecimal price, String userType, BigDecimal expected) {
|
||
|
||
BigDecimal discount = calculator.calculate(price, userType);
|
||
|
||
assertThat(discount).isEqualByComparingTo(expected);
|
||
}
|
||
```
|
||
|
||
---
|
||
|
||
### 2.2 自动生成边界测试的Prompt
|
||
|
||
```markdown
|
||
请为这个类生成边界条件测试，使用参数化测试覆盖：
|
||
|
||
1. 数值边界：MIN_VALUE, -1, 0, 1, MAX_VALUE
|
||
2. 字符串边界：null, "", "a", 最大长度, 特殊字符🔑
|
||
3. 集合边界：null, empty, 1 element, max size
|
||
4. 时间边界：MIN, epoch, now, MAX
|
||
|
||
要求：
|
||
- 使用JUnit 5 @ParameterizedTest
|
||
- 每个边界值一个测试用例
|
||
- 验证异常处理
|
||
- 命名：shouldHandleXxx_whenYxxIsBoundaryValue
|
||
|
||
类代码：
|
||
[粘贴代码]
|
||
```
|
||
|
||
---
|
||
|
||
## 🔧 第三类：Mock使用不当（重构）
|
||
|
||
### 3.1 Mock决策树
|
||
|
||
```
|
||
需要Mock吗？
|
||
├─ 是第三方外部服务？（支付/短信/邮件）
|
||
│ └─ 是 → 可以Mock
|
||
│
|
||
├─ 是基础设施？（数据库/缓存/消息队列）
|
||
│ └─ 是 → 用Testcontainers，不要Mock
|
||
│
|
||
├─ 是核心业务逻辑？（Service/Controller）
|
||
│ └─ 是 → 禁止Mock，用真实实现
|
||
│
|
||
└─ 否 → 不Mock
|
||
```
|
||
|
||
❌ **错误示例**（过度Mock）
|
||
```java
|
||
@SpringBootTest
|
||
class OrderServiceTest {
|
||
|
||
@MockBean
|
||
private OrderRepository orderRepository; // ❌ 不应该Mock Repository
|
||
|
||
@MockBean
|
||
private InventoryService inventoryService; // ⚠️ 可以Spy
|
||
|
||
@MockBean
|
||
private PaymentService paymentService; // ✅ 外部服务可以Mock
|
||
|
||
@Test
|
||
void shouldCreateOrder() {
|
||
when(orderRepository.save(any())).thenReturn(mockOrder); // 测试了Mock
|
||
when(inventoryService.checkStock(any())).thenReturn(true);
|
||
when(paymentService.charge(any(), any())).thenReturn(success);
|
||
|
||
Order result = orderService.create(request);
|
||
|
||
// 这个测试其实什么都没验证！
|
||
assertNotNull(result);
|
||
}
|
||
}
|
||
```
|
||
|
||
✅ **正确做法**
|
||
```java
|
||
@SpringBootTest
|
||
@Testcontainers
|
||
class OrderServiceTest {
|
||
|
||
@Container
|
||
static PostgreSQLContainer<?> postgres = new PostgreSQLContainer<>();
|
||
|
||
@Autowired
|
||
private OrderRepository orderRepository; // ✅ 真实Repository
|
||
|
||
@SpyBean
|
||
private InventoryService inventoryService; // ✅ Spy部分方法
|
||
|
||
@MockBean
|
||
private PaymentService paymentService; // ✅ 外部服务Mock
|
||
|
||
@Test
|
||
void shouldCreateOrder_andSaveToDatabase() {
|
||
// Given
|
||
OrderRequest request = createOrderRequest();
|
||
when(paymentService.charge(any(), any())).thenReturn(success);
|
||
|
||
// When
|
||
Order result = orderService.create(request);
|
||
|
||
// Then
|
||
// 验证真实数据库写入
|
||
Order saved = orderRepository.findById(result.getId()).orElseThrow();
|
||
assertThat(saved.getStatus()).isEqualTo("PAID");
|
||
assertThat(saved.getTotal()).isEqualByComparingTo(request.getTotal());
|
||
|
||
// 验证真实调用外部服务
|
||
verify(paymentService).charge(
|
||
eq(request.getUserId()),
|
||
eq(request.getTotal())
|
||
);
|
||
}
|
||
}
|
||
```
|
||
|
||
**Mock审计脚本**
|
||
```bash
|
||
#!/bin/bash
|
||
# mock-audit.sh
|
||
|
||
echo "=== Mock审计报告 ==="
|
||
|
||
# 1. 统计MockBean数量
|
||
echo "1. MockBean统计："
|
||
grep -r "@MockBean" src/test/java --include="*.java" | wc -l
|
||
|
||
# 2. 检查Repository Mock
|
||
echo "2. Repository Mock（应该为0）："
|
||
grep -r "@MockBean.*Repository" src/test/java --include="*.java" | wc -l
|
||
|
||
# 3. 检查Service Mock
|
||
echo "3. Service Mock（应该<5）："
|
||
grep -r "@MockBean.*Service" src/test/java --include="*.java" | wc -l
|
||
|
||
# 4. 计算Mock比例
|
||
total_beans=$(grep -r "@MockBean\|@Autowired\|@SpyBean" src/test/java --include="*.java" | wc -l)
|
||
mock_beans=$(grep -r "@MockBean" src/test/java --include="*.java" | wc -l)
|
||
ratio=$((mock_beans * 100 / total_beans))
|
||
echo "4. Mock比例：${ratio}%"
|
||
|
||
if [ $ratio -gt 50 ]; then
|
||
echo "⚠️ Warning: Mock比例过高，建议重构"
|
||
fi
|
||
```
|
||
|
||
---
|
||
|
||
## 📊 第四类：性能测试缺失（补充）
|
||
|
||
### 4.1 AI常忽视的性能测试
|
||
|
||
❌ **错误**（没有性能测试）
|
||
```java
|
||
@Test
|
||
void shouldProcessLargeFile() {
|
||
byte[] largeFile = new byte[10 * 1024 * 1024]; // 10MB
|
||
// 只测试功能，不测试性能
|
||
}
|
||
```
|
||
|
||
✅ **正确做法**
|
||
```java
|
||
@Test
|
||
void shouldProcessLargeFile_withinTimeLimit() {
|
||
byte[] largeFile = createLargeFile(10 * 1024 * 1024); // 10MB
|
||
|
||
long start = System.currentTimeMillis();
|
||
|
||
ProcessResult result = fileProcessor.process(largeFile);
|
||
|
||
long duration = System.currentTimeMillis() - start;
|
||
|
||
// 性能断言
|
||
assertThat(duration).isLessThan(1000); // 必须在1秒内完成
|
||
assertThat(result).isNotNull();
|
||
assertThat(result.getProcessedBytes()).isEqualTo(largeFile.length);
|
||
}
|
||
|
||
@Test
|
||
void shouldHandleConcurrentRequests() throws InterruptedException {
|
||
int threadCount = 10;
|
||
CountDownLatch latch = new CountDownLatch(threadCount);
|
||
AtomicInteger successCount = new AtomicInteger(0);
|
||
|
||
for (int i = 0; i < threadCount; i++) {
|
||
new Thread(() -> {
|
||
try {
|
||
service.process(createRequest());
|
||
successCount.incrementAndGet();
|
||
} finally {
|
||
latch.countDown();
|
||
}
|
||
}).start();
|
||
}
|
||
|
||
assertTrue(latch.await(5, TimeUnit.SECONDS));
|
||
assertThat(successCount.get()).isEqualTo(threadCount);
|
||
}
|
||
```
|
||
|
||
---
|
||
|
||
## 📝 第五类：可读性/可维护性差（重构）
|
||
|
||
### 5.1 测试命名反模式
|
||
|
||
❌ **AI常生成**
|
||
```java
|
||
@Test // ❌ 无法看出测试什么
|
||
void test1() { }
|
||
|
||
@Test // ❌ 不清晰
|
||
void shouldWork() { }
|
||
|
||
@Test // ❌ 没有Given/When
|
||
void createUserTest() { }
|
||
```
|
||
|
||
✅ **正确命名**
|
||
```java
|
||
// 格式：should[预期行为]_when[条件/场景]
|
||
|
||
@Test
|
||
void shouldReturnUserDetails_whenUserExists() { }
|
||
|
||
@Test
|
||
void shouldThrowNotFoundException_whenUserDoesNotExist() { }
|
||
|
||
@Test
|
||
void shouldSendWelcomeEmail_whenNewUserRegisters() { }
|
||
|
||
// 复杂场景
|
||
@Test
|
||
void shouldCalculateDiscountedPrice_whenUserIsVIP_andOrderExceeds1000() { }
|
||
```
|
||
|
||
### 5.2 Given-When-Then结构
|
||
|
||
✅ **标准结构**
|
||
```java
|
||
@Test
|
||
void shouldDeactivateAccount_whenUserRequestsDeletion() {
|
||
// Given - 准备数据和状态
|
||
User user = createActiveUser("user123");
|
||
given(userRepository.findById("user123")).willReturn(Optional.of(user));
|
||
|
||
// When - 执行操作
|
||
accountService.deactivateAccount("user123");
|
||
|
||
// Then - 验证结果和副作用
|
||
assertThat(user.isActive()).isFalse();
|
||
verify(userRepository).save(user);
|
||
verify(emailService).sendDeactivationConfirmation("user123");
|
||
verify(cacheManager).evict("user:user123");
|
||
}
|
||
```
|
||
|
||
---
|
||
|
||
## 🔍 自动化检查清单
|
||
|
||
### 创建检查脚本
|
||
```bash
|
||
#!/bin/bash
|
||
# testing-quality-check.sh
|
||
|
||
echo "🧪 测试质量检查报告"
|
||
echo "===================="
|
||
|
||
# 1. 虚假测试检查
|
||
echo "1. 检查虚假测试..."
|
||
fake_tests=$(grep -r "assertTrue(true)\|assertFalse(false)\|assertNotNull(new" src/test/java --include="*.java" | wc -l)
|
||
echo " 找到 $fake_tests 个虚假断言"
|
||
|
||
# 2. 边界测试检查
|
||
echo "2. 检查边界条件..."
|
||
boundary_tests=$(grep -r "MIN_VALUE\|MAX_VALUE\|null.*empty" src/test/java --include="*.java" | wc -l)
|
||
echo " 边界测试: $boundary_tests"
|
||
|
||
# 3. Mock比例检查
|
||
echo "3. 检查Mock比例..."
|
||
total=$(find src/test/java -name "*Test.java" | wc -l)
|
||
mocks=$(grep -r "@MockBean" src/test/java --include="*.java" | wc -l)
|
||
if [ $total -gt 0 ]; then
|
||
ratio=$((mocks * 100 / total))
|
||
echo " Mock比例: ${ratio}%"
|
||
fi
|
||
|
||
# 4. 参数化测试检查
|
||
echo "4. 检查参数化测试..."
|
||
param_tests=$(grep -r "@ParameterizedTest\|@CsvSource" src/test/java --include="*.java" | wc -l)
|
||
echo " 参数化测试: $param_tests"
|
||
|
||
# 5. 命名规范检查
|
||
echo "5. 检查测试命名..."
|
||
incorrect=$(grep -r "void test[0-9]*\|void shouldWork\|void .*Test()" src/test/java --include="*.java" | wc -l)
|
||
echo " 命名不规范: $incorrect"
|
||
|
||
echo ""
|
||
echo "===================="
|
||
echo "检查完成！"
|
||
```
|
||
|
||
---
|
||
|
||
## ✅ 检查清单模板
|
||
|
||
### 每次生成测试后检查
|
||
|
||
**功能性**
|
||
- [ ] 测试验证真实业务逻辑，不只是框架代码
|
||
- [ ] 每个测试至少2个有意义的断言
|
||
- [ ] 测试包含边界条件（null/空/极值）
|
||
- [ ] 测试包含异常场景
|
||
|
||
**质量**
|
||
- [ ] Mock比例 < 50%
|
||
- [ ] Service/Controller未Mock
|
||
- [ ] Repository使用Testcontainers
|
||
- [ ] 分支覆盖率 > 60%
|
||
|
||
**可维护性**
|
||
- [ ] 使用should_when命名格式
|
||
- [ ] 使用Given-When-Then结构
|
||
- [ ] 没有硬编码值
|
||
- [ ] 参数化测试替代重复测试
|
||
|
||
**性能/并发**
|
||
- [ ] 大对象处理有性能断言
|
||
- [ ] 并发场景有测试
|
||
- [ ] 没有资源泄漏
|
||
|
||
---
|
||
|
||
## 🎯 AI生成测试Prompt优化
|
||
|
||
### 原Prompt（容易生成虚假测试）
|
||
```
|
||
为UserService生成单元测试
|
||
```
|
||
|
||
### 优化后Prompt（生成高质量测试）
|
||
```
|
||
为UserService生成生产级单元测试，要求：
|
||
|
||
1. 使用真实数据库（Testcontainers），禁止Mock Repository
|
||
2. 每个测试至少2个断言：验证返回值和副作用
|
||
3. 使用参数化测试覆盖边界条件：null, 空值, 极值, 并发
|
||
4. 测试名使用should_when格式
|
||
5. 包含Given-When-Then结构
|
||
6. 验证异常场景和错误处理
|
||
7. 包含性能断言（执行时间<100ms）
|
||
8. 不要测试getter/setter/构造函数
|
||
|
||
代码：
|
||
[粘贴代码]
|
||
|
||
生成测试时请思考：
|
||
- 这个测试在生产环境有用吗？
|
||
- 能发现真实bug吗？
|
||
- 维护成本高吗？
|
||
```
|
||
|
||
---
|
||
|
||
## 📈 质量评分标准
|
||
|
||
| 维度 | 权重 | 优秀 | 良好 | 需改进 |
|
||
|------|------|------|------|--------|
|
||
| 功能性 | 30% | 无虚假测试，验证业务逻辑 | 少量框架测试 | 大量getter/setter测试 |
|
||
| 边界覆盖 | 25% | 系统化边界测试 | 常见边界测试 | 缺少边界测试 |
|
||
| Mock使用 | 20% | 比例<30%，核心真实 | 比例<50% | 比例>70% |
|
||
| 可维护性 | 15% | 参数化，无重复 | 有重复但清晰 | 大量重复代码 |
|
||
| 性能/并发 | 10% | 有性能+并发测试 | 有性能测试 | 无性能测试 |
|
||
|
||
**评分**
|
||
- A级(90-100)：生产就绪
|
||
- B级(80-89)：优秀
|
||
- C级(70-79)：良好，可优化
|
||
- D级(60-69)：及格，需改进
|
||
- F级(<60)：不合格，需重写
|
||
|
||
---
|
||
|
||
## 🆘 快速修复指南
|
||
|
||
### 问题1：太多getter/setter测试
|
||
```bash
|
||
# 自动识别并标记
|
||
find src/test/java -name "*Test.java" -exec grep -l "shouldReturn.*whenGet\|shouldSet.*whenSet" {} \; | xargs -I {} echo "Review: {}"
|
||
```
|
||
|
||
### 问题2：Mock比例过高
|
||
```bash
|
||
# 识别过度Mock的测试
|
||
find src/test/java -name "*Test.java" -exec sh -c 'count=$(grep -c "@MockBean" "$1"); if [ "$count" -gt 5 ]; then echo "$1: $count mocks"; fi' _ {} \;
|
||
```
|
||
|
||
### 问题3：缺少边界测试
|
||
```bash
|
||
# 识别缺少边界测试的类
|
||
find src/test/java -name "*Test.java" -exec sh -c 'if ! grep -q "ParameterizedTest\|MIN_VALUE\|MAX_VALUE\|null" "$1"; then echo "Missing boundary: $1"; fi' _ {} \;
|
||
```
|
||
|
||
---
|
||
|
||
**立即可用，快速提升AI生成测试质量！** 🚀
|