Skip to content

fix(fc): slot-driven guest IPv6; strip addresses by default to kill Happy Eyeballs (keeps bind(::)) - #3587

Open
AdaAibaby wants to merge 1 commit into
e2b-dev:mainfrom
AdaAibaby:fix/disable-ipv6-kernel-args
Open

AdaAibaby wants to merge 1 commit into
e2b-dev:mainfrom
AdaAibaby:fix/disable-ipv6-kernel-args

Conversation

@AdaAibaby

@AdaAibaby AdaAibaby commented Aug 18, 2026 •

Copy link
Copy Markdown
Contributor

Fixes #3585.

Problem

kernel_args.go hardcoded ipv6.disable=0 and ipv6.autoconf=1, enabling IPv6 inside every Firecracker guest. But the host-side network setup is IPv4-only — no router advertisement is sent on the host tap — so SLAAC produces only an unroutable fe80:: link-local address.

With IPv6 enabled but unroutable, a runtime that runs its own Happy Eyeballs (RFC 8305) races AAAA first, hits the dead fe80:: path, and waits out a ~250 ms timeout before falling back to IPv4 on every dual-stack outbound connection. This affects Node.js (autoSelectFamily, default-on since Node 20), curl --happy-eyeballs, Go dual-stack dialing, and similar.

Scope correction vs. the original description: plain glibc getaddrinfo tools (pip, apt, most curl) are mostly spared — with only a link-local source address, RFC 6724 already sorts IPv4 ahead of IPv6, so they usually try IPv4 first. The penalty is real but concentrated in runtimes/libraries that do their own A/AAAA racing, not "every tool loses 5s".

Fix

Derive the guest's IPv6 state from whether the host slot actually has an IPv6 router, instead of hardcoding it:

  • Slot.HasIPv6Router() bool — single source of truth; returns false today.
  • ProcessOptions.IPv6RouterConfigured — carries the slot's answer into buildKernelArgs without coupling the kernel-args layer to the network package; set from Slot.HasIPv6Router() in Process.Create.
  • buildKernelArgs — no router (the default) → emit sysctl.net.ipv6.conf.{all,default}.disable_ipv6=1 on the kernel command line. Router present → keep ipv6.disable=0 / ipv6.autoconf=1.

Why address-stripping sysctls instead of ipv6.disable=1

ipv6.disable=1 turns off the entire AF_INET6 stack — socket(AF_INET6) returns EAFNOSUPPORT, so a guest service that binds :: (the dual-stack wildcard, the default for many web servers / Node / Python) fails to start. Sandboxes run arbitrary user code, so that is a real compatibility regression.

Instead, sysctl.net.ipv6.conf.{all,default}.disable_ipv6=1 (kernel cmdline, supported since Linux 5.8) strips IPv6 addresses from every interface — the kernel then has no IPv6 source address and RFC 6724 prefers IPv4, which removes the Happy Eyeballs penalty — while keeping the AF_INET6 socket API and bind(::) working. The only behaviour it drops is an explicit [::1] loopback bind, which has no routable counterpart without a v6 stack anyway.

Verification

Kernel-cmdline sysctl parsing is confirmed end-to-end on the actual guest kernel: booted vmlinux-6.1.158 (the guest Firecracker kernel this environment ships — alongside 6.1.102) under firecracker with a busybox initramfs, with vs. without the sysctl args.

boot /proc/cmdline has sysctl.* all.disable_ipv6 default.disable_ipv6 ip -6 addr (lo) ipv6 proc tree
with sysctl args yes 1 1 no inet6 address PRESENT
control (without) no 0 0 ::1/128 PRESENT

The guest 6.1 kernel applies the command-line sysctls (disable_ipv6 flips to 1), every IPv6 address is gone (even ::1), and the AF_INET6 stack stays compiled-in and enabled (the /proc/sys/net/ipv6 tree is present) — i.e. address-stripping, not a stack teardown. socket(AF_INET6) + bind(::) remaining usable under disable_ipv6=1 (while ipv6.disable=1 fails all of socket/bind(::)/bind(::1)) was separately confirmed via netns.

mode AF_INET6 socket bind(::) bind(::1) v6 addrs
current main yes yes yes fe80 + ::1
this PR (strip addrs) yes yes no none
ipv6.disable=1 no no no none

Changes

File What changed
network/slot.go HasIPv6Router() bool — documented extension point, returns false
fc/process.go ProcessOptions.IPv6RouterConfigured; set from slot.HasIPv6Router() before buildKernelArgs
fc/kernel_args.go IPv6 args derived from options.IPv6RouterConfigured instead of hardcoded
fc/kernel_args_test.go Updated pinned defaults; added ipv6 router configured case
docs/ARCHITECTURE.md Documents the IPv4-only guest default and the dual-stack upgrade path

Testing

go build / go vet clean for fc + network. go test ./pkg/sandbox/fc/ passes (pinned-cmdline + new dual-stack case). Kernel-cmdline sysctl parsing confirmed end-to-end on guest vmlinux-6.1.158 (see the Verification table above); the AF_INET6/bind(::) retention under disable_ipv6=1 confirmed via netns.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 926e6ed210

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

@AdaAibaby AdaAibaby changed the title fix(fc): disable IPv6 in guest kernel to eliminate Happy Eyeballs latency fix(fc): slot-driven guest IPv6 config; default to IPv4-only to eliminate Happy Eyeballs latency Aug 18, 2026
@blancadesal

Copy link
Copy Markdown

Tested this on a bare-metal server (Debian 13/trixie, kernel 6.12) while trying to run the DEV-LOCAL.md local-dev flow end-to-end.

Reproduced the bug on main: apt-get inside the guest tried an unroutable IPv6 address first (Cannot initiate the connection to deb.debian.org:80 (2a04:4e42:...)) before falling back to IPv4.

Cherry-picked this PR onto the same environment, rebuilt the orchestrator, and confirmed a clean build with no IPv6 attempts – apt-get went straight to IPv4. 🎉

…Happy Eyeballs

Fixes e2b-dev#3585.

The host tap has no IPv6 router (no RA, no routable prefix), yet the guest
kernel args hardcoded ipv6.disable=0 / ipv6.autoconf=1. SLAAC then produces only
an unroutable fe80:: link-local address. A runtime that runs its own Happy
Eyeballs (RFC 8305: Node's autoSelectFamily, curl, Go dual-stack dialing) races
AAAA first and waits out a ~250 ms timeout before falling back to IPv4 on every
outbound dual-stack connection. (glibc getaddrinfo tools are mostly spared: with
only a link-local source, RFC 6724 already sorts IPv4 first — so the impact is
real but narrower than "every tool".)

Derive the guest's IPv6 state from whether the host slot actually has an IPv6
router, instead of hardcoding it:

- Slot.HasIPv6Router() bool — single source of truth, returns false today.
- ProcessOptions.IPv6RouterConfigured — carries the slot's answer into
  buildKernelArgs without coupling kernel-args to the network package; set from
  Slot.HasIPv6Router() in Process.Create.
- buildKernelArgs — when no router (default), emit
  sysctl.net.ipv6.conf.{all,default}.disable_ipv6=1 on the kernel cmdline
  (supported since Linux 5.8; guest kernels are 5.10/6.1). This strips IPv6
  addresses from every interface so the kernel has no IPv6 source and prefers
  IPv4, WITHOUT disabling the AF_INET6 stack: unlike ipv6.disable=1, bind(::)
  still works, so a dual-stack wildcard server in the guest still binds. The one
  dropped behaviour is an explicit [::1] loopback bind, which has no routable
  counterpart without a v6 stack anyway. When a router is present, keep
  ipv6.disable=0 / ipv6.autoconf=1 for a routable SLAAC address.

Chose address-stripping sysctls over ipv6.disable=1 (the simpler, blunter switch)
specifically to preserve bind(::); kernel-cmdline sysctl semantics verified on
6.8 (flushes fe80/::1, keeps AF_INET6 + bind(::)).

- fc/kernel_args.go: IPv6 args derived from options.IPv6RouterConfigured
- fc/process.go: ProcessOptions.IPv6RouterConfigured, set from slot in Create
- network/slot.go: HasIPv6Router() extension point
- fc/kernel_args_test.go: pinned defaults updated; added "ipv6 router configured"
- docs/ARCHITECTURE.md: documents the IPv4-only default and dual-stack upgrade path
@AdaAibaby
AdaAibaby force-pushed the fix/disable-ipv6-kernel-args branch from 7653fa2 to 53d9838 Compare November 11, 2026 02:02
@AdaAibaby AdaAibaby changed the title fix(fc): slot-driven guest IPv6 config; default to IPv4-only to eliminate Happy Eyeballs latency fix(fc): slot-driven guest IPv6; strip addresses by default to kill Happy Eyeballs (keeps bind(::)) Nov 11, 2026

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

kernel: ipv6.disable=0 with no IPv6 routing causes Happy Eyeballs latency on all outbound sandbox connections

3 participants