Concurrent loads() calls silently corrupt each other's values #327

Summary

serialize() declares context=SerializationContext() as a default argument.
Python evaluates a default once, at import, so every rule in the process shares
one mutable SerializationContext — and the expression, function and indexing
rules mutate it in place.

A thread serializing a function call therefore changes what other threads see
mid-serialization. A tuple comes back as the string '[1, 2, 3]' instead of a
list, an object as '{a = 1}' instead of a dict. No exception is raised; a
caller doing schema validation downstream reports it as a type error in the
user's configuration.

Reproduction

from concurrent.futures import ThreadPoolExecutor
import hcl2

PLAIN = "x = [1, 2, 3]\ny = {a = 1}\n"
TOGGLES = "z = f([1, 2, 3], {a = 1})\n"   # serializing this sets the shared flag
EXPECT = {"x": [1, 2, 3], "y": {"a": 1}}

def work(i):
    if i % 2:
        hcl2.loads(TOGGLES)
        return None
    return hcl2.loads(PLAIN)

with ThreadPoolExecutor(max_workers=8) as pool:
    results = [r for r in pool.map(work, range(800)) if r is not None]

bad = [r for r in results if r != EXPECT]
print(f"{len(bad)} of {len(results)} corrupted")
print("example:", bad[0] if bad else None)

Output on 8.1.3:

400 of 400 corrupted
example: {'x': '[1, 2, 3]', 'y': '{a = 1}'}

The threads have to overlap for it to appear, so it is workload-dependent:

Cause

hcl2/rules/base.py:42 and the other 49 serialize() signatures:

def serialize(self, options=SerializationOptions(), context=SerializationContext()) -> Any:

SerializationContext is mutated in place through
with context.modify(inside_dollar_string=True) in rules/expressions.py
(lines 155, 269, 306), rules/functions.py:95 and rules/indexing.py (94,
151).

Nothing supplies a context at any public entry point — api.serialize calls
tree.serialize(), and query/_base.py:95 calls
self._node.serialize(options=options) — so both land on the shared object.
Eight call sites inside rules/base.py also drop the context they were given
and re-default to it; the rules below them already thread it correctly.

Suggested shape of the fix

Have the structural rules in base.py build a fresh context when called without
one, and thread the context they were given into every child call:

def serialize(self, options=SerializationOptions(), context=None) -> Any:
    context = context if context is not None else SerializationContext()
    ...
    return self.body.serialize(options, context)

That is enough to make the shared default unreachable in practice: with it, the
reproduction above reports 0 of 400. The remaining serialize() signatures
keep the mutable default, which is no longer reachable but is still a trap for a
future call site — worth removing separately, since it is 50 signatures.

options=SerializationOptions() has the same shape but is only ever read, so it
is benign unless a caller mutates the object it passes in.

Happy to open a PR. The branch I have includes a regression test that
interleaves 800 parses across 8 threads; the count is high because the effect
needs roughly 400 interleaved parses to show at all, and a smaller test passes
against the unfixed code.

The investigation behind this report was produced with AI assistance, working on my behalf. Every reproduction, version comparison and measurement cited was executed rather than inferred; I reviewed it before filing.

This issue, and the investigation behind it, were produced by an AI assistant (Claude) working on behalf of the author. Please review with that provenance in mind.