Repository navigation
[test] Coverage/mutation run: tests that pass without verifying behaviour (dispatcher, wire bytes, checksums, parse_packet, cpm) #243
Copy link
Copy link
Closed
Description
Environment
- CP version: v0.9.4 (
f925e5f6); measured with gcov (C unit + integration suites) andgo test -coverprofile; mutation testing with a source-level C mutator and gremlins (Go)
Description
A coverage + mutation run over the current test suites found areas where tests pass but do not actually verify the behaviour. Tracking them here so they can be picked up individually:
- The control-socket dispatcher is never executed.
unix-manager.c::unix_command_executehas 0% coverage.tests/unit/unix_control.ccalls the ping/info handlers directly, and the reload tests use SIGHUP, so handshake parsing, unknown commands and malformed JSON are untested over the real socket. - Wire bytes are not verified. Every GRE/VXLAN/ZMQ integration case (
testdata/cases/*/verify.json) usespacket_count_only/packet_count_range. No test checks the VXLAN header/VNI, the GRE protocol field, or the ZMQ batch/record/packet-header layout indocs/ZMQ-WIRE-FORMAT.md§3–§6 (only the heartbeat record is byte-checked). - Checksum tests are round-trips. The TCP/UDP (IPv4 + IPv6) checksum tests compute a checksum and then verify it with the same code, so a wrong pseudo-header sum still passes: 26 surviving mutants in
calculate_pseudo_header_sum/cksum_accumulate. Known-answer vectors are needed (only the IPv4 header has the RFC 1071 example). parse_packetboundaries. 34 surviving mutants, e.g.caplen < offset + ip_hdr_len→<=is undetected (no frame with caplen exactly equal to the header length), and there are no IPv6 extension-header cases.- Untested features with no test at all:
req_pattern_judge_pkt_directionon real packets (only the expression grammar is tested),rotating_fileoutput, ZMQ batch flush rules (§8), corrupt/truncated pcap input topcap_file, numeric config range checks (see [cpworker] Out-of-range numeric config silently clamped by cJSON (snaplen/buffer_size_mb/timeout_ms/ring_size) #234/[cpworker] zmq.hwm=0 means an unbounded queue and is not validated #235), and live libpcap capture fidelity (reload tests drive libpcap onlobut only assert log lines). - Go: gremlins test efficacy is 53% for
cpdaemon/pkg/cpm(63 surviving mutants, mostlysyncer.go,worker_mgr.go,synclog.go) and 74% forcpgolib/cpworker(handshake recv-error and invalid-JSON branches,argumentsomitted when empty).
Expected
Each area has at least one test derived from its spec (RFC / docs/*.md) that fails when the behaviour is wrong.
中文原文
测试环境
- CP 版本:v0.9.4(
f925e5f6);用 gcov(C 单测 + 集成测试)和go test -coverprofile测量;mutation 测试使用源码级 C 变异器与 gremlins(Go)
问题描述
对现有测试做了一次覆盖率 + mutation 测量,发现一些"测试通过但并未真正验证行为"的地方,在此汇总跟踪,可逐项认领:
- 控制 socket 分发器从未被执行:
unix-manager.c::unix_command_execute覆盖率 0%。单测直接调用 ping/info handler,reload 测试走 SIGHUP;经真实 socket 的握手解析、未知命令、畸形 JSON 均无测试。 - 线上字节未被验证:GRE/VXLAN/ZMQ 集成用例全部只用
packet_count_only/packet_count_range;没有测试检查 VXLAN 头/VNI、GRE 协议字段,或docs/ZMQ-WIRE-FORMAT.md§3–§6 的 batch/record/包头布局(只有心跳记录做了字节校验)。 - 校验和测试是 round-trip:TCP/UDP(IPv4 + IPv6)校验和测试用同一段代码计算再验证,伪首部求和写错也能通过(
calculate_pseudo_header_sum/cksum_accumulate共 26 个变异体存活)。需要已知答案向量(目前只有 IPv4 头用了 RFC 1071 示例)。 parse_packet边界:34 个变异体存活,例如caplen < offset + ip_hdr_len改成<=不被发现(没有 caplen 恰好等于头长的帧),也没有 IPv6 扩展头用例。- 完全没有测试的功能:真实报文上的
req_pattern_judge_pkt_direction(只测了表达式语法)、rotating_file输出、ZMQ batch flush 规则(§8)、pcap_file读入损坏/截断的 pcap、数值配置范围校验(见 [cpworker] Out-of-range numeric config silently clamped by cJSON (snaplen/buffer_size_mb/timeout_ms/ring_size) #234/[cpworker] zmq.hwm=0 means an unbounded queue and is not validated #235)、libpcap 实时抓包保真度(reload 测试在lo上跑 libpcap 但只断言日志)。 - Go:gremlins 测试有效率
cpdaemon/pkg/cpm53%(63 个存活,主要在syncer.go、worker_mgr.go、synclog.go),cpgolib/cpworker74%(握手接收错误与非法 JSON 分支、参数为空时省略arguments)。
期望
每一项至少有一个从其规范(RFC / docs/*.md)推导出的测试,并且在行为出错时会失败。
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
No labels
Activity
fix(cpworker): report pcap_file read errors instead of end of file (#278
fix(cpworker): validate numeric config ranges instead of silently tru…
test(cpworker): test checksums, parse_packet bounds, zmq flush and co…
test(cpworker): check gre, vxlan and zmq wire bytes against the wire-…
test(cpdaemon,cpgolib): check cpm config mapping, buffer sizing, sync…
timmy21 commentedon Oct 13, 2026
Addressed in #314 (1e2b82e), #315 (c4e1a8e) and #316 (0ace1c5):