Skip to content

fix(cpworker): count every output packet once, with its sliced length - #310

Merged
timmy21 merged 2 commits into
0.9.xfrom
fix/308-output-counters
Oct 9, 2026
Merged

timmy21 merged 2 commits into
0.9.xfrom
fix/308-output-counters

Conversation

@timmy21

@timmy21 timmy21 commented Oct 9, 2026 •

Copy link
Copy Markdown
Contributor

Problem

The output counters (fwd_*, direction_drop_*, ratelimit_drop_*, error_drop_*) had no definition, and each output counted them differently (#308):

  • gre/vxlan added the GRE/VXLAN header to fwd, error_drop and ratelimit_drop bytes, but not to direction_drop bytes.
  • zmq counted fwd/error_drop bytes as whole batches, including the batch header, record headers and MPLS labels. Its direction_drop/ratelimit_drop bytes included the MPLS label.
  • A gre/vxlan partial send counted the packet in fwd_packets and split its bytes across fwd_bytes and error_drop_bytes.
  • A VXLAN-split packet counted once per fragment. A failed build_fragment returned success without counting the packet.
  • rate_limit_mbps charged a different length per output, so fwd_bytes could not be compared with the limit.

Change

Each packet handed to an output is counted once, as 1 packet and L bytes, in exactly one bucket. L is the packet's caplen after slice, without any encapsulation.

Output Change
gre All buckets and the rate limiter use L (no GRE header). A partial send counts as error_drop only
vxlan do_send_packet only sends; vxlan_send_packet counts once after the whole packet, all fragments included, is sent or fails. A partial send, a failed fragment or a failed fragment build counts as error_drop. All buckets and the rate limiter use L
zmq The batch tracks data_bytes, the frame bytes of its captured packets (replacing heartbeat_bytes); a flush adds it to fwd_bytes or error_drop_bytes. Direction/ratelimit drops and the rate limiter no longer add the MPLS label
null, file, rotating_file Already followed the definition; unchanged

Naming (second commit): length/caplen meant a different length in each output. output_frame_len() in output.h computes the counted length (caplen after slice, capped per output: GRE_MAX_FRAME_LEN, VXLAN_MAX_FRAME_LEN, ZMQ_MAX_FRAME_LEN), and gre, vxlan, null and zmq call it frame_len. header->caplen always means the captured length; vxlan uses payload_len (frame + capture_time), zmq record_len (frame + MPLS label). send_whole_or_split takes only frame_len, so a split packet is parsed with the same length it is sent with (they differed when caplen > 65535).

Docs: the definition is in docs/REFACTOR-CPCTL.md §6.2. ZMQ-WIRE-FORMAT §8 and the rate_limit_mbps row in USAGE-CPWORKER refer to it (English and zh-Hans).

Behaviour changes

  • gre, vxlan and zmq report lower fwd_bytes/drop bytes than before: encapsulation headers are no longer counted. These values are uploaded to CPM as fwdBytes.
  • vxlan with split reports one fwd_packets per original packet instead of one per fragment.
  • rate_limit_mbps for gre, vxlan and zmq is slightly looser, since the 4–8 byte headers are no longer charged. While the limit is active, fwd_bytes grows by at most rate_limit_mbps.

Tests

  • New tests/unit/output_vxlan.c: sliced length without VXLAN header or capture_time, split counted once, send error (EMSGSIZE) counted as one error_drop, rate limit charging L. A GRE rate-limit/direction test runs when a raw socket is available and is ignored otherwise.
  • tests/unit/output_zmq.c: two assertions now expect frame bytes; new tests for sliced length plus conservation, and for the rate limit charging L.
  • The new tests fail against 0.9.x (vxlan 5/5, zmq 4).
  • pktm VM, on both commits: make test 12/12 passed (GRE test also run under sudo); run_test.sh all passed.

Fixes #308

timmy.yuan added 2 commits October 9, 2026 10:05
The output counters had no definition, and each output counted bytes
differently: gre/vxlan added their encapsulation header to fwd, error
and ratelimit bytes but not to direction drops, zmq counted whole
batches including batch and record headers, a partial send split one
packet across fwd_bytes and error_drop_bytes, and a VXLAN-split packet
counted once per fragment.

Each packet handed to an output is now counted once, in exactly one of
fwd, direction_drop, ratelimit_drop or error_drop, with its caplen after
slice and no encapsulation. A partial send, a failed split fragment or a
failed fragment build counts the whole packet as error_drop. zmq batches
track the frame bytes of their captured packets. rate_limit_mbps charges
the same length, so fwd_bytes stays within the limit.

The definition is written into REFACTOR-CPCTL.md §6.2, and the zmq wire
format and usage docs refer to it.
…output

length and caplen meant different things in each output: the sliced
frame, the frame capped at 65535, the VXLAN payload with capture_time,
or the zmq record length with the MPLS label. send_whole_or_split took
both caplen and length, and parsed the split with the uncapped one.

output_frame_len() in output.h now computes the length an output
carries and counts (caplen after slice, capped per output), and gre,
vxlan, null and zmq use it as frame_len. header->caplen always means the
captured length. vxlan uses payload_len for the frame plus capture_time,
and zmq record_len for the frame plus the MPLS label. vxlan parses and
sends a split packet with the same frame_len.
@timmy21
timmy21 merged commit e12a3c2 into 0.9.x Oct 9, 2026
1 check passed
@timmy21
timmy21 deleted the fix/308-output-counters branch October 9, 2026 03:06
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

cpworker: output counters have no definition; byte and packet counts differ per output and do not add up

1 participant