Repository navigation
fix(fc): slot-driven guest IPv6; strip addresses by default to kill Happy Eyeballs (keeps bind(::)) - #3587
fix(fc): slot-driven guest IPv6; strip addresses by default to kill Happy Eyeballs (keeps bind(::))#3587AdaAibaby wants to merge 1 commit into
Conversation
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 926e6ed210
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
|
Tested this on a bare-metal server (Debian 13/trixie, kernel 6.12) while trying to run the Reproduced the bug on Cherry-picked this PR onto the same environment, rebuilt the orchestrator, and confirmed a clean build with no IPv6 attempts – |
…Happy Eyeballs Fixes e2b-dev#3585. The host tap has no IPv6 router (no RA, no routable prefix), yet the guest kernel args hardcoded ipv6.disable=0 / ipv6.autoconf=1. SLAAC then produces only an unroutable fe80:: link-local address. A runtime that runs its own Happy Eyeballs (RFC 8305: Node's autoSelectFamily, curl, Go dual-stack dialing) races AAAA first and waits out a ~250 ms timeout before falling back to IPv4 on every outbound dual-stack connection. (glibc getaddrinfo tools are mostly spared: with only a link-local source, RFC 6724 already sorts IPv4 first — so the impact is real but narrower than "every tool".) Derive the guest's IPv6 state from whether the host slot actually has an IPv6 router, instead of hardcoding it: - Slot.HasIPv6Router() bool — single source of truth, returns false today. - ProcessOptions.IPv6RouterConfigured — carries the slot's answer into buildKernelArgs without coupling kernel-args to the network package; set from Slot.HasIPv6Router() in Process.Create. - buildKernelArgs — when no router (default), emit sysctl.net.ipv6.conf.{all,default}.disable_ipv6=1 on the kernel cmdline (supported since Linux 5.8; guest kernels are 5.10/6.1). This strips IPv6 addresses from every interface so the kernel has no IPv6 source and prefers IPv4, WITHOUT disabling the AF_INET6 stack: unlike ipv6.disable=1, bind(::) still works, so a dual-stack wildcard server in the guest still binds. The one dropped behaviour is an explicit [::1] loopback bind, which has no routable counterpart without a v6 stack anyway. When a router is present, keep ipv6.disable=0 / ipv6.autoconf=1 for a routable SLAAC address. Chose address-stripping sysctls over ipv6.disable=1 (the simpler, blunter switch) specifically to preserve bind(::); kernel-cmdline sysctl semantics verified on 6.8 (flushes fe80/::1, keeps AF_INET6 + bind(::)). - fc/kernel_args.go: IPv6 args derived from options.IPv6RouterConfigured - fc/process.go: ProcessOptions.IPv6RouterConfigured, set from slot in Create - network/slot.go: HasIPv6Router() extension point - fc/kernel_args_test.go: pinned defaults updated; added "ipv6 router configured" - docs/ARCHITECTURE.md: documents the IPv4-only default and dual-stack upgrade path
7653fa2 to
53d9838
Compare
Fixes #3585.
Problem
kernel_args.gohardcodedipv6.disable=0andipv6.autoconf=1, enabling IPv6 inside every Firecracker guest. But the host-side network setup is IPv4-only — no router advertisement is sent on the host tap — so SLAAC produces only an unroutablefe80::link-local address.With IPv6 enabled but unroutable, a runtime that runs its own Happy Eyeballs (RFC 8305) races AAAA first, hits the dead
fe80::path, and waits out a ~250 ms timeout before falling back to IPv4 on every dual-stack outbound connection. This affects Node.js (autoSelectFamily, default-on since Node 20),curl --happy-eyeballs, Go dual-stack dialing, and similar.Fix
Derive the guest's IPv6 state from whether the host slot actually has an IPv6 router, instead of hardcoding it:
Slot.HasIPv6Router() bool— single source of truth; returnsfalsetoday.ProcessOptions.IPv6RouterConfigured— carries the slot's answer intobuildKernelArgswithout coupling the kernel-args layer to the network package; set fromSlot.HasIPv6Router()inProcess.Create.buildKernelArgs— no router (the default) → emitsysctl.net.ipv6.conf.{all,default}.disable_ipv6=1on the kernel command line. Router present → keepipv6.disable=0/ipv6.autoconf=1.Why address-stripping sysctls instead of
ipv6.disable=1ipv6.disable=1turns off the entireAF_INET6stack —socket(AF_INET6)returnsEAFNOSUPPORT, so a guest service that binds::(the dual-stack wildcard, the default for many web servers / Node / Python) fails to start. Sandboxes run arbitrary user code, so that is a real compatibility regression.Instead,
sysctl.net.ipv6.conf.{all,default}.disable_ipv6=1(kernel cmdline, supported since Linux 5.8) strips IPv6 addresses from every interface — the kernel then has no IPv6 source address and RFC 6724 prefers IPv4, which removes the Happy Eyeballs penalty — while keeping theAF_INET6socket API andbind(::)working. The only behaviour it drops is an explicit[::1]loopback bind, which has no routable counterpart without a v6 stack anyway.Verification
Kernel-cmdline sysctl parsing is confirmed end-to-end on the actual guest kernel: booted
vmlinux-6.1.158(the guest Firecracker kernel this environment ships — alongside6.1.102) under firecracker with a busybox initramfs, with vs. without the sysctl args./proc/cmdlinehas sysctl.*all.disable_ipv6default.disable_ipv6ip -6 addr(lo)::1/128The guest 6.1 kernel applies the command-line sysctls (
disable_ipv6flips to 1), every IPv6 address is gone (even::1), and theAF_INET6stack stays compiled-in and enabled (the/proc/sys/net/ipv6tree is present) — i.e. address-stripping, not a stack teardown.socket(AF_INET6)+bind(::)remaining usable underdisable_ipv6=1(whileipv6.disable=1fails all of socket/bind(::)/bind(::1)) was separately confirmed via netns.bind(::)bind(::1)ipv6.disable=1Changes
network/slot.goHasIPv6Router() bool— documented extension point, returnsfalsefc/process.goProcessOptions.IPv6RouterConfigured; set fromslot.HasIPv6Router()beforebuildKernelArgsfc/kernel_args.gooptions.IPv6RouterConfiguredinstead of hardcodedfc/kernel_args_test.goipv6 router configuredcasedocs/ARCHITECTURE.mdTesting
go build/go vetclean forfc+network.go test ./pkg/sandbox/fc/passes (pinned-cmdline + new dual-stack case). Kernel-cmdline sysctl parsing confirmed end-to-end on guestvmlinux-6.1.158(see the Verification table above); theAF_INET6/bind(::)retention underdisable_ipv6=1confirmed via netns.