http_get reports success (carry=0, HTTP 200) on a truncated body — silent data loss

[JC-000]

The defect is the false success, not the byte count. http_get returns carry=0 with http_status=200 after consuming a body tens of kilobytes short of the Content-Length it was given. A caller cannot distinguish a complete body from a 51% one, which makes HTTPS_BODY_TO_REU fetches untrustworthy rather than merely slow.

What this evidence can and cannot support. Every figure below is http_body_total, a 24-bit consumed count on the C64 side. None of them observes what the server sent, except in the offline runs where the listener's Content-Length is known by construction. They establish that the fetch is short and whether the shortfall is stable; they cannot by themselves distinguish "we stopped asking" from "we dropped what arrived".

Offline reproduction — local TLS listener, Content-Length, no chunking, no internet. PRG 1c828c4adbb7f9f58efa538045a170ebeb9b345d47eb4c4f01b2cd7247425807 (e4fda6d, BACKEND=uci USE_NISTCURVES_ONCHIP=1 HTTPS_SNI=www.foo.invalid), U64E @ 48 MHz:

served	http_body_total	delta	net_last_error	carry
40,000	40,000	0	$00	0
160,000	78,293	81,707	$86	0
160,000	not captured	—	$00	0
160,000	160,000	0	$00	0

It is intermittent, not size-dependent. The same 160,000 B body on the same PRG both failed and succeeded within minutes. An earlier reading of these numbers as a size threshold was wrong: the two runs differed in time as well as size, and time was the variable that mattered. A size threshold inferred from two runs at different sizes is unfalsifiable until one of them is repeated.

Intermittent silent truncation is worse than deterministic truncation, because no fixed-size test catches it reliably.

Live (wikipedia, chunked) — same-day curl anchor 125,703 B at 06:16:

tree	PRG	consumed
master 48657f5	012c7848…	117,192 B
e4fda6d	0712def5…	89,526 B
e4fda6d (n=2)	0712def5…	73,720 B

The anchor is a moving target — the article is user-editable — which is why it is quoted with its capture time. It does not weaken the finding: a 36 KB shortfall is not an edit, though a few-hundred-byte difference between runs may be. Two runs of the same PRG differ by 15,806 B, so these live counts characterise variance, not any difference between trees.

Both trees. The live wikipedia fetch truncated on master 48657f5 (PRG 012c7848…, 117,192 B against a same-day 125,703 B anchor) and on e4fda6d (PRG 0712def5…, 89,526 B and 73,720 B). The offline reproduction was run on e4fda6d only. Because the failure is intermittent, a single offline run on master would not have discriminated — a complete run there would not show absence — so it was deliberately not used as evidence. The supported statement is: the defect is observed on both trees, it is intermittent, and there is no evidence the bump introduced it.

Leads, recorded as leads — no cause is attributed:

• Chunked desync above 64 KB (CLAUDE.md, http_recv_response). Excluded for the offline case: the local listener sends Content-Length, so no chunking is involved. Negative evidence for the live case too — neither live plateau sits at the boundary (89,526 is 24,010 past 65,536; 117,192 is 51,656 past).
• The net_poll ring clamp that d9cd021 fixed. tcp_recv_overflow ($A248) is the one-byte discriminator: src/net/uci/net.s:387 sets it when the ring fills and then drops the delivery, commenting "The clamp at @do_poll makes this unreachable in practice … If it latches, the request/free-space arithmetic has regressed — that is the wikipedia-stall bug's signature." Not captured on any run above; instrumented now.
• net_last_error = $86 correlates with the one captured short run and $00 with every complete one, but $86 is documented as appearing on passing runs (CLAUDE.md, github.com 2026-08-28), and the one $00 run at 160,000 B is the run whose count was not captured. Candidate, not cause.

State at the stall on the short run: tcp_recv_head == tcp_recv_tail == $0ACB (ring empty), uci_read_hdr = $FFFF (no-data sentinel), uci_status = '02,NO DATA: 11', tls_state=$00, tls_recv_progress=$05, tls_recv_sub_progress=$02.

Why this survived: the only rig exercising this path could not go red on it — see the companion issue on rig_https_banner.py.

Two method traps for whoever picks this up:

1. Reading http_body_total after the rig returns is unreliable. A post-hoc read returned all zeros including http_status, on a run the rig itself had read as 200. Wrap the rig's _dump_diag so the read happens while the transport is live.
2. The listener's post-send time.sleep(1.0) is not the cause. Checked and cleared: TCP orders FIN after queued data and sendall blocks on the C64's window.

[JC-000]

tcp_recv_overflow result — the ring is excluded, and $86 is excluded as the cause

Three further 160,000 B offline repeats, same PRG 1c828c4adbb7f9f58efa538045a170ebeb9b345d47eb4c4f01b2cd7247425807 (e4fda6d), same local listener with Content-Length and no chunking, each capturing http_body_total, tcp_recv_overflow and net_last_error in one live-transport dump:

run	http_body_total	delta	tcp_recv_overflow	net_last_error	uci_read_hdr	head/tail
1	25,557 B	134,443	0	$86	$0016	$0E54 = $0E54
2	160,000 B	0	0	$00	$FFFF	$00B3 / $00CB
3	91,605 B	68,395	0	$00	$FFFF	$00B6 = $00B6

All three returned carry=0, http_status=200.

Lead 2 (the net_poll ring clamp) is excluded

tcp_recv_overflow reads 0 on both truncating runs. src/net/uci/net.s:387 sets that byte when the ring fills and then drops the rest of a delivery; it never latched. So the bytes were not dropped on arrival — we stopped asking. d9cd021's clamp has not regressed and the request/free-space arithmetic is not the fault. The comment's claim that the overflow path is "unreachable in practice" is upheld, not falsified.

Both leads recorded in the issue body are now dead: chunked desync was already excluded for the offline case (the listener sends Content-Length), and the ring is excluded here. The loss is upstream of the ring, in the poll / request / termination path.

$86 is excluded as the cause

Run 3 truncated by 68,395 B with net_last_error = $00. The earlier 3-for-3 correlation between $86 and short runs is broken by a counterexample, which is what it needed. $86 remains a real signal when present — it has exactly one setter in the FPGA — but it does not explain this defect.

One observation worth chasing, not a claim

Run 1 stopped at 25,557 B with uci_read_hdr = $0016 and uci_status = '00,OK' — a 22-byte read header and an OK status, not the $FFFF no-data sentinel seen on runs 2 and 3. At the moment of the dump the firmware had data available and an OK status, and the client had already returned success. That is consistent with the client deciding the body was complete while the transport still had bytes to give, but the dump is taken after http_get returns, so it is a lead and not a measurement of the stall state itself.

Range now observed

Offline at a constant 160,000 B: 25,557 / 78,293 / 91,605 / 160,000 B. The shortfall varies by more than 130 KB across runs of one image against one listener. That reinforces the intermittency finding and makes any fixed-threshold test unreliable as a detector.

[JC-000]

Correction to my previous comment, plus the tcp_recv_overflow result

Retracting a claim I made above: "$86 is excluded as the cause" was wrong. I based it on a run that truncated by 68,395 B carrying net_last_error = $00, and treated that as the counterexample the correlation needed. That run did not complete — it hung, and its sentinel was never set (TIMEOUT: sentinel not set after 900s, progress=0x03). Its carry=0 and its byte count are uninitialised values read after a timeout, not the results of a finished fetch. It is not a counterexample to anything.

Method note, which is the reusable part: carry and http_body_total read from a run whose sentinel never fired are stale and must not be quoted as a measurement. Every offline run below was audited for sentinel set versus TIMEOUT before being included.

The measurement

Offline, PRG 1c828c4adbb7f9f58efa538045a170ebeb9b345d47eb4c4f01b2cd7247425807 (e4fda6d), local TLS listener, 160,000 B Content-Length, no chunking, U64E @ 48 MHz:

run	consumed	delta	net_last_error	tcp_recv_overflow	outcome
1	25,557 B	134,443	$86	0	completed, carry=0, HTTP 200
2	160,000 B	0	$00	0	completed, carry=0
3	91,605 B	—	$00	0	HUNG, sentinel never set after 900 s

The d9cd021 ring-clamp lead is excluded

tcp_recv_overflow = 0 on a run short by 134 KB. src/net/uci/net.s:387 sets that byte when the ring fills and then drops the rest of a delivery. It never latched, so the bytes were not dropped on arrival — we stopped asking. The clamp has not regressed, and the comment there asserting the overflow path is "unreachable in practice" is upheld, not falsified.

Both leads recorded in the issue body are now excluded: chunked desync was already out for the offline case (the listener sends Content-Length), and the ring is out here. The loss is upstream of the ring, in the poll / request / termination path.

$86 survived a test that could have killed it

Among completed runs only, the correlation is 5/5:

• short: 78,293 B ($86), 25,557 B ($86)
• exact: 40,000 B ($00), 160,000 B ($00), 160,000 B ($00)

Still recorded as a candidate, not a cause — CLAUDE.md documents $86 appearing on passing runs (github.com, 2026-08-28), so a known counterexample class exists. But my earlier exclusion of it was based on a run that had not completed, and I withdraw it.

A second failure mode, not described in the issue body

Run 3 consumed 91,605 B and never returned within 900 s. That is distinct from the silent-success truncation this issue describes: http_get can also hang outright rather than return a short body with carry=0. Both are reachable on the same image against the same listener at the same body size.

Range now observed

At a constant 160,000 B offline: 25,557 / 78,293 / 91,605 (hung) / 160,000. The shortfall varies by over 130 KB across runs of one image against one listener, which is why no fixed-size or fixed-threshold test is a reliable detector.