Skip to content

Proposal: observe ingress traffic and correlate it with egress per pod #250

Description

@JimBugwadia

Summary

kyverno-runtime currently observes egress traffic and DNS queries and answers, but it doesn't observe general ingress traffic, and it can't correlate an inbound connection with the outbound connections it might have triggered. This issue proposes two additive changes to close both gaps.

Current state

  • Egress: pkg/bpf/egressfilter attaches a cgroup_skb/egress program per pod and enforces or observes IPv4 destination allow/deny rules.
  • DNS: pkg/bpf/dnsquery observes outbound DNS queries. pkg/bpf/egressfilter/_cprog/dns.c snoops DNS answers on a cgroup_skb/ingress hook, but only to populate the IP-to-domain map the egress filter uses — it's not general ingress visibility.
  • Ingress: no general-purpose ingress observation exists. docs/architecture.html states this is deliberate: ingress control belongs to admission control, RBAC, and NetworkPolicy, not this runtime agent.
  • Correlation: pkg/runtimeevent.NetFacts has no direction field and no per-connection identifier, so nothing can tell that two events belong to the same connection, let alone that an inbound and an outbound connection are related.

Proposal

1. Add ingress observation (observe-only)

  • Add a cgroup_skb/ingress program that records ip_events for every ingress packet, not just DNS answers, mirroring the existing cgroup_egress program.
  • Add a Direction field to NetFacts and rename DestIP to PeerIP, since "destination" is misleading for an ingress event.
  • Add a direction field to the network behavior in RuntimePolicySpec, defaulting to egress for backward compatibility with existing policies.
  • Reject mode: enforce for a direction: ingress network behavior, mirroring the existing rule that rejects enforce for a dns behavior. Ingress stays observation-only; enforcement of inbound traffic remains a NetworkPolicy or CNI concern.

2. Add connection-level correlation

Two additive pieces, so a pod's ingress and egress connections can be shown together with a stated confidence level, without claiming request-level causality:

  • Per-connection identity. Add kprobe or tracepoint hooks in syscall context (tcp_v4_connect and tcp_v6_connect for egress, inet_csk_accept or the sock:inet_sock_set_state tracepoint for ingress) that emit {socketCookie, pid, tgid, comm, 4-tuple, direction, timestamp} through a ring buffer. This gives an exact per-connection key (bpf_get_socket_cookie) and syscall-accurate timestamps, which cgroup_skb packet events don't provide.
  • Mesh-layer identity enrichment. When a service mesh is present, use its mTLS peer identity (SPIFFE ID) or access logs to attribute a connection's peer to a real Kubernetes identity instead of a raw IP. This needs no new eBPF, and it directly answers "which pod or service is on the other end" for ingress.

Present correlated pairs with an explicit confidence tier, rather than one undifferentiated "correlated" signal:

  • Confirmed: the connections share a distributed-tracing header, such as traceparent, when the workload propagates one.
  • Likely: same PID or process, overlapping connection lifetime.
  • Concurrent: same pod, same time window, no stronger signal.

Non-goals

  • Request-level (L7) causality proof. That needs application-level trace-context propagation, which this proposal can surface when present but can't create.
  • Ingress enforcement. Blocking inbound traffic stays a NetworkPolicy or CNI responsibility.
  • IPv6 support. Egress is currently IPv4-only; that gap is tracked separately.

Open questions

  • Where does the per-pod merged timeline surface: a new CLI command, a new CRD status field, or a separate view built on the event stream?
  • Do the confidence-tier labels above match how policy authors and SREs would want to reason about this data?

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions