Repository navigation
A heredoc whose interpolation spans lines cannot be flattened to valid HCL #347
Description
Summary
preserve_heredocs=False returns a heredoc as quoted-string source. That form
cannot represent a body whose interpolation spans lines: the newlines inside
${...} are expression source, where neither a raw newline nor an escaped one
is valid.
Reproduction
locals {
a = <<EOT
${
1 + 2
}
EOT
}OpenTofu v1.12.5 evaluates that to "3\n".
Flattening it produces "${\n 1 + 2\n}\n" with the newlines escaped, and
OpenTofu rejects that source with
This character is not used within the language.
Leaving them raw instead gives a quoted string spanning three physical lines,
which it rejects as an invalid multi-line string. Neither form is valid, so the
flattened output cannot be read back by Terraform whichever way it is written.
Why it is filed separately
The escapes-and-spans work in #339 makes the surrounding cases correct -- a
single-line interpolation now survives flattening and restoring intact -- but
it cannot fix this one, because there is no valid quoted spelling to produce.
The options are all judgement calls rather than corrections:
- collapse the interpolation onto one line, which is semantically identical for
most expressions but not for one containing a#comment or a nested
heredoc; - leave the heredoc unflattened, which contradicts what the option promises;
- or raise, which turns a silent corruption into a loud failure but breaks
callers who are flattening documents that contain one.
Worth deciding deliberately rather than by whichever fix lands first.
This issue, and the investigation behind it, were produced by an AI assistant (Claude) working on behalf of the author. Please review with that provenance in mind.
Activity
livingstaccato commentedon Sep 2, 2026
A suggestion, since all three options here trade something different away.
My preference is to refuse the conversion and leave the heredoc alone, because it is the only option that never changes what a document means.
preserve_heredocs=Falsewould then have one documented case where it declines, in the same shape as the two refusalsstrings_to_heredocsalready makes: a value that does not end in a newline stays quoted, and one carrying a lone carriage return stays quoted, both because the target form cannot express them. A multi-line interpolation is the same kind of thing — the quoted form cannot express it — so declining is consistent rather than a special case.Collapsing the interpolation onto one line is tempting and mostly works, but "mostly" is doing real work there: a
#comment inside the expression swallows everything after it on the joined line, and a nested heredoc cannot be collapsed at all. Both are rare, and both fail silently by producing a valid file that means something else, which is the worst failure mode of the three.Raising is defensible, and better than the silent corruption in place today, but it breaks callers who are flattening documents that happen to contain one — including anyone using
hcl2tojsonin a pipeline, where the failure would be a stack trace rather than a diagnostic.If declining is acceptable, the check is cheap: the span scanner added in #346 already identifies interpolation spans, so it is a test for a newline inside one.
livingstaccato commentedon Sep 2, 2026
Superseded by #354, which takes the first of the three options above.
Updating this since the issue text says the choice is open and it no longer is, at least as a proposal: the PR declines the conversion and hands the heredoc back in the form
preserve_heredocs=Trueproduces, so the document round-trips unchanged. The two alternatives are still described above and still available — if either is preferred, the PR is small enough to redirect.Worth restating what the current behaviour is, since the issue frames this as a choice between designs and one of them is "keep emitting output that cannot be read":
preserve_heredocs=Falsewrites"${\n 1 + 2\n}\n"today, andhcl2.loadsof that raisesUnexpectedToken. Whichever option is chosen, that part is a defect rather than a preference.fix: decline to flatten a heredoc whose interpolation spans lines (am…
fix: decline to flatten a heredoc whose interpolation spans lines (am…