Repository navigation
Data<V> bare V branch admits undefined and shadows readonly V[] #1
Description
environment
- @metreeca/pipe: 0.9.20 (
feature/crawl) - typescript: 5.9.3
- node.js: 24.19.0
- operating system: macOS (Darwin 24.6.0)
steps to reproduce
// 1. `V` does not exclude `undefined`
let calls = 0;
const stream = iterate<undefined | number>(() =>
++calls <= 3 ? (calls === 2 ? undefined : calls) : undefined
);
await stream(toArray()); // [1], after 2 generator calls
// 2. bare `V` shadows `readonly V[]` when `V` is itself an array
const arrays = items<number[]>([1, 2]); // typed `Pipe<number[]>`
await arrays(toArray()); // typed `number[][]`, actually `[1, 2]`expected behaviour
-
Data<V>should be built over aVexcludingundefined, so theundefined | Data<V>signatures of
iterate(),flatMap()andcrawl()keep their marker distinct from stream values. Streams never carry
undefined(dropped both as values enter a pipe and as tasks hand them downstream), so admitting it intoV
describes a state that cannot exist. -
Vandreadonly V[]should stay distinguishable whenVis itself an array type, so a stream of arrays is
either expressible or rejected at compile time rather than silently mistyped.
actual behaviour
-
The bare
Vbranch swallowsundefined: forV = undefined | numberthe typeundefined | Data<V>collapses
toData<V>, making the value returned by aniterate()generator simultaneously a legal contribution (nothing,
per theDatadocumentation) and the end-of-stream marker. The runtime resolves the ambiguity in favour of
termination (src/feeds/iterate.ts:52), so the probe above stops after the second call and drops item3.
flatMap()(src/tasks/flatMap.ts:46) andcrawl()(src/tasks/crawl.ts:62,:105) carry the same signature
and the same collapse. Related: theDatadocumentation listsundefinedas an accepted shape contributing
nothing, but the union has noundefinedmember; it is reachable only whenVitself admits it. -
items<number[]>([1, 2])type-checks against the bareVbranch and declaresPipe<number[]>, butflatten()
(src/index.core.ts:36) treats every iterable as a sequence, so the array is expanded and the pipe actually
carries1and2;toArray()is typednumber[][]while returning[1, 2]. Conversely, inference resolves
items([1, 2, 3])toPipe<number>, so a stream of arrays cannot be opened from an array literal at all.
workarounds
For (1), keep undefined out of the item type and signal "no items" with an empty array. For (2), wrap array items
in a single-element array (items<number[]>([[1, 2]])), which is sound but at odds with the bare V branch.
notes
Both defects trace to the bare V branch of the union at src/index.ts:67.
Activity
knoan commentedon Aug 24, 2026
Proposal: drop
undefinedfromData-returning signaturesRather than reconstructing
Data<V>over aVthat excludesundefined, removeundefinedfrom the signaturesbuilt over it:
iterate()(src/feeds/iterate.ts:50):() => Awaitable<Data<V>>flatMap()(src/tasks/flatMap.ts:46):(item: V) => Awaitable<Data<R>>crawl()(src/tasks/crawl.ts:62,:109,:110): visitor and traverser likewiseStreams never carry
undefined, so it has no place either among theDatashapes or in the signatures returningthem. The
Datadocumentation entry listingundefinedas a shape contributing nothing (src/index.ts:41) goeswith them: "contributes nothing" is already expressible by every other shape as an empty batch (
[], an emptyiterable, an empty pipe).
The termination role is likewise redundant.
iterate()currently keys the end of the stream on theundefinedmarker (
src/feeds/iterate.ts:52), but an empty batch already terminates it too, one line further down:flatten([])yields a done iterator and the
first.donebranch returns (:60). Keying termination solely on the empty batchcollapses the two markers into one and removes the ambiguity reported above at its source, with no need for a
V-excluding-undefinedconstruction.In
flatMap()andcrawl(), where no termination is involved,undefinedcarries only the "no items" meaning andis plainly superseded by the empty batch.
This addresses defect (1) only; defect (2), the
readonly V[]shadowing, remains open.fix: rebuild the data shapes around feeds carrying defined items
knoan commentedon Aug 25, 2026
Solution
Both defects traced to the bare
Vbranch of theData<V>union, now dropped: data shapes are batches only(
readonly (undefined | V)[],Iterable<undefined | V>,AsyncIterable<undefined | V>,Feed<V>), and every shapeaccepts optional entries, dropped as they enter the feed.
Changes
Data<V>no longer admits bare values:feed<number[]>([1, 2])is rejected at compile time, whilefeed<number[]>([[1, 2]])anditems<number[]>([1, 2])carry arrays wholetask or sink ever observes
undefined;mapandflatMapno longer filter themselvesundefinedstays a distinct marker where signatures use it:flatMaptakesData<R>rather thanundefined | Data<R>, with an empty batch contributing nothingPipecontract toFeed,datafeed tofeed;chainandmergeaccept feeds onlyKnown limitation
Vis not constrained to excludeundefined: an explicititerate<undefined | number>(…)still collapses theend-of-feed marker with a legal item type. "Feeds never carry
undefined" is a documented contract rather than anenforced one, and the case is marginal enough not to warrant a
V extends {}-style constraint.Related
release/0.9