Repository navigation
[workspace] bazel/tap: Add tap2pcap, to convert Envoy socket tap traces to PCAP - #5627
Conversation
…envoyproxy#5622) A pure Python replacement for the `tap2pcap` tool of the Envoy API (`@envoy_api//tools:tap2pcap`), which shells out to `od` and to Wireshark's `text2pcap`, and is tested by comparing `tshark` output to a golden file. Neither tool is in the Envoy CI image, so that test only passes while its result is in the remote cache. `//tap:tap2pcap` writes the PCAP itself: a classic PCAP file with one TCP segment (dummy Ethernet and IPv4/IPv6 headers, real checksums, per-direction sequence numbers) for each read or write of a socket buffered trace. Payloads over what an IP packet can carry are split. The `tap2pcap` macro (`//tap:defs.bzl`) makes the binary from the caller's trace wrapper message, so this module does not depend on the Envoy API (which depends on this module): tap2pcap( name = "tap2pcap", message = "envoy.data.tap.v3.wrapper_pb2.TraceWrapper", proto = "//envoy/data/tap/v3:pkg_py_proto", ) Tested on the PCAP bytes (decoded back, with every length and checksum verified), and end to end on a binary made for a test proto with the fields of Envoy's, for text and binary traces. Signed-off-by: Ryan Northey <ryan@synca.io> Backport note (workspace): picked as is, for the Envoy release branches that build with WORKSPACE (v1.37-v1.39), which have the same `tap2pcap_test` and the same CI image as `main` had. (cherry picked from commit 5f5e48b)
There was a problem hiding this comment.
🟡 Changes recommended
Unconditional protobuf loads reference a repository absent from WORKSPACE, causing CI build and test commands to fail during package loading before the new tests can run.
1 open issue
What changed in this PR
Backports tap2pcap to toolshed’s WORKSPACE line for Envoy release branches, enabling socket tap trace conversion without external Wireshark utilities.
Changes:
- Adds pure-Python PCAP generation with IPv4/IPv6 support, checksums, timestamps, and payload segmentation.
- Exposes a Bazel macro that uses the caller’s protobuf message, avoiding a dependency on Envoy’s API.
- Adds unit and binary integration tests for conversion and error handling.
| File | Description |
|---|---|
bazel/tap/tests/trace.proto |
Defines a minimal Envoy-compatible test schema. |
bazel/tap/tests/tap2pcap_test.py |
Tests packet generation, checksums, segmentation, and errors. |
bazel/tap/tests/tap2pcap_binary_test.py |
Tests text and binary conversion through the generated executable. |
bazel/tap/tests/socket.pb_text |
Supplies socket events with byte and string payloads. |
bazel/tap/tests/pcap.py |
Decodes generated packets and validates lengths and checksums. |
bazel/tap/tests/http.pb_text |
Supplies an unsupported trace for rejection testing. |
bazel/tap/tests/BUILD |
Defines test libraries, protobuf targets, and the test executable. |
bazel/tap/tap2pcap.py |
Implements trace parsing and PCAP generation. |
bazel/tap/defs.bzl |
Provides the caller-configurable tap2pcap macro. |
bazel/tap/BUILD |
Exposes the conversion library and macro definitions. |
Review effort: Balanced
| load("@com_google_protobuf//bazel:proto_library.bzl", "proto_library") | ||
| load("@com_google_protobuf//bazel:py_proto_library.bzl", "py_proto_library") |
There was a problem hiding this comment.
This does not happen: the workspace CI of this PR loads and runs the package. test (workspace) runs bazel test --config=ci --config=rbe --config=bes //... //sysroot:sysroots with --noenable_bzlmod --enable_workspace (the branch's .bazelrc), and both //tap/tests:tap2pcap_test and //tap/tests:tap2pcap_binary_test pass there (job), as do the workspace build, gcc, macos and cross jobs. So @com_google_protobuf resolves in this branch's WORKSPACE setup; the note in toolchains/test/BUILD is out of date on that point. (bot assisted)

Backports #5622 (
bazel/tap:tap2pcap) to theworkspaceline, for the Envoy release branches that build with WORKSPACE (v1.37 - v1.39).Those branches have the same
@envoy_api//tools:tap2pcap_testand the same CI image asmainhad: the test shells out to Wireshark'stext2pcap, which the image does not have, so it only passes while its result is in the remote cache (the API job onrelease/v1.39reports "Executed 0 out of 5 tests"). envoyproxy/envoy#49392 fixed that onmainwith@envoy_toolshed//tap:defs.bzl, which this line does not have.Picked as is: the
tap2pcapmacro, the Python implementation (tap2pcap.py, unchanged frommain) and its tests. With this in a0.4.13.post6, envoyproxy/envoy#49392 can be picked to those branches.