Repository navigation
fix(cpworker): count every output packet once, with its sliced length - #310
Merged
Merged
Conversation
added 2 commits
October 9, 2026 10:05
The output counters had no definition, and each output counted bytes differently: gre/vxlan added their encapsulation header to fwd, error and ratelimit bytes but not to direction drops, zmq counted whole batches including batch and record headers, a partial send split one packet across fwd_bytes and error_drop_bytes, and a VXLAN-split packet counted once per fragment. Each packet handed to an output is now counted once, in exactly one of fwd, direction_drop, ratelimit_drop or error_drop, with its caplen after slice and no encapsulation. A partial send, a failed split fragment or a failed fragment build counts the whole packet as error_drop. zmq batches track the frame bytes of their captured packets. rate_limit_mbps charges the same length, so fwd_bytes stays within the limit. The definition is written into REFACTOR-CPCTL.md §6.2, and the zmq wire format and usage docs refer to it.
…output length and caplen meant different things in each output: the sliced frame, the frame capped at 65535, the VXLAN payload with capture_time, or the zmq record length with the MPLS label. send_whole_or_split took both caplen and length, and parsed the split with the uncapped one. output_frame_len() in output.h now computes the length an output carries and counts (caplen after slice, capped per output), and gre, vxlan, null and zmq use it as frame_len. header->caplen always means the captured length. vxlan uses payload_len for the frame plus capture_time, and zmq record_len for the frame plus the MPLS label. vxlan parses and sends a split packet with the same frame_len.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
The output counters (
fwd_*,direction_drop_*,ratelimit_drop_*,error_drop_*) had no definition, and each output counted them differently (#308):fwd_packetsand split its bytes acrossfwd_bytesanderror_drop_bytes.build_fragmentreturned success without counting the packet.rate_limit_mbpscharged a different length per output, sofwd_bytescould not be compared with the limit.Change
Each packet handed to an output is counted once, as 1 packet and
Lbytes, in exactly one bucket.Lis the packet'scaplenafterslice, without any encapsulation.L(no GRE header). A partial send counts aserror_droponlydo_send_packetonly sends;vxlan_send_packetcounts once after the whole packet, all fragments included, is sent or fails. A partial send, a failed fragment or a failed fragment build counts aserror_drop. All buckets and the rate limiter useLdata_bytes, the frame bytes of its captured packets (replacingheartbeat_bytes); a flush adds it tofwd_bytesorerror_drop_bytes. Direction/ratelimit drops and the rate limiter no longer add the MPLS labelNaming (second commit):
length/caplenmeant a different length in each output.output_frame_len()inoutput.hcomputes the counted length (caplen afterslice, capped per output:GRE_MAX_FRAME_LEN,VXLAN_MAX_FRAME_LEN,ZMQ_MAX_FRAME_LEN), and gre, vxlan, null and zmq call itframe_len.header->caplenalways means the captured length; vxlan usespayload_len(frame +capture_time), zmqrecord_len(frame + MPLS label).send_whole_or_splittakes onlyframe_len, so a split packet is parsed with the same length it is sent with (they differed when caplen > 65535).Docs: the definition is in
docs/REFACTOR-CPCTL.md§6.2.ZMQ-WIRE-FORMAT§8 and therate_limit_mbpsrow inUSAGE-CPWORKERrefer to it (English and zh-Hans).Behaviour changes
fwd_bytes/drop bytes than before: encapsulation headers are no longer counted. These values are uploaded to CPM asfwdBytes.splitreports onefwd_packetsper original packet instead of one per fragment.rate_limit_mbpsfor gre, vxlan and zmq is slightly looser, since the 4–8 byte headers are no longer charged. While the limit is active,fwd_bytesgrows by at mostrate_limit_mbps.Tests
tests/unit/output_vxlan.c: sliced length without VXLAN header orcapture_time, split counted once, send error (EMSGSIZE) counted as oneerror_drop, rate limit chargingL. A GRE rate-limit/direction test runs when a raw socket is available and is ignored otherwise.tests/unit/output_zmq.c: two assertions now expect frame bytes; new tests for sliced length plus conservation, and for the rate limit chargingL.0.9.x(vxlan 5/5, zmq 4).make test12/12 passed (GRE test also run under sudo);run_test.sh allpassed.Fixes #308