Skip to content

fix(lua): convert strings to numbers as lua_stringtonumber does - #92

Merged
wolfy-j merged 1 commit into
mainfrom
fix/string-to-number
Nov 15, 2026
Merged

wolfy-j merged 1 commit into
mainfrom
fix/string-to-number

Conversation

@wolfy-j

@wolfy-j wolfy-j commented Nov 15, 2026

Copy link
Copy Markdown
Contributor

Defect

String-to-number conversion does not follow Lua 5.4 (lua_stringtonumber):

Lua Before Lua 5.4
tonumber("10"), "10" + 0, -"10" 10.0 (float) 10 (integer)
tonumber("1e5") nil 100000.0
tonumber("1e400") nil inf
tonumber("0x1p4"), tonumber("0x1.8") nil 16.0, 1.5
tonumber("0xffffffffffffffffff") nil -1 (hex integers wrap)
tonumber("\t\v\f\r 5 \n"), tonumber("+0x5") nil 5
tonumber(7) 7.0 7
tonumber("ff", 16) 255.0 255
tonumber("ffffffffffffffff", 16) nil -1
tonumber("10", 99), tonumber(10, 16) nil, 10.0 argument errors
" 7 " // 2 error 3
0xffffffffffffffff, 1e400 (literals) NaN -1, inf
0x1p4, 0xA.8p0, 0x.8 (literals) syntax error 16.0, 10.5, 0.5
return 3x, return 1.2.3 various malformed number

Cause

There were five separate conversions, and none matched the reference:

  • parseNumber (used by arithmetic coercion, unary minus, CheckNumber, ToInt, LVAsNumber and math.tointeger) always returned a float, and used Go's strconv syntax.
  • parseNumberValue (literals, bitwise coercion) kept the subtype but used strconv, which rejects wrapping hex integers and hex floats without p.
  • tonumber had its own parser: it switched to float only on ., so exponents failed, and it always returned floats.
  • The lexer stopped hexadecimal numerals at . and p. The compiler turned any numeral it could not parse into NaN instead of reporting an error.
  • The parser and the type checker (numparse) each had another strconv-based parser.

Fix

numparse.Parse implements l_str2int and l_str2d from Lua 5.4 lobject.c:

  • C isspace trimming and an optional sign.
  • Decimal integers that switch to float on overflow.
  • Hex integers that wrap modulo 2^64.
  • strtod float syntax, including hex mantissas with an optional fraction and p exponent.
  • Out-of-range floats become infinities. inf/nan are rejected.

numparse.ParseBase implements tonumber's base conversion. The other conversions are removed, and every caller uses these two:

  • parseNumberValue wraps numparse.Parse. parseNumber is removed.
  • Arithmetic coercion (objectArith, //, unary minus) keeps the integer subtype of numeric strings: + - * % on two integers give integers, the same as the VM fast paths.
  • tonumber follows luaB_tonumber. Numbers keep their subtype, strings use Parse, and with a base the first argument must be a string, the base must be in 2..36, and the result is an integer.
  • CheckNumber, ToInt/ToInt64, LVAsNumber and LString.Format convert through parseNumberValue. math.tointeger is exactInteger, so math.tointeger(2.0^63) is nil (it used to overflow).
  • The lexer reads numerals as read_numeral does (hex digits, ., exponent marks with a sign, and one touching letter) and rejects malformed ones with malformed number. The compiler raises a compile error instead of emitting NaN, and the constant folder treats an unparsable numeral as non-constant.
  • The parser's literal-type numbers and numparse's literal helpers (ParseIntegerLiteral, ParseFloatLiteral, ParseNumberLiteral, ParseIntegralLiteral) are built on Parse. parser.go is regenerated with goyacc. Apart from the prologue, only the //line comments change.

Tests

  • types/numparse: TestParse (65 cases) and TestParseBase (21 cases). They cover integers, overflow to float, wrapping hex, decimal and hex floats, infinities, denormals, whitespace, signs and the rejected forms.
  • string_to_number_test.go: tonumber with and without a base, string coercion for every arithmetic operator, bitwise operators and unary minus, math.tointeger, literals, and the error cases. 43 of these fail on main.
  • compiler/parse/lexer_number_test.go: numeral tokens, where numerals stop before operators, and malformed numerals.

Expected values come from the Lua 5.4 reference interpreter.

Not in this PR

  • CheckInt/OptInt (the luaL_checkinteger equivalents) still reject numeric strings and truncate floats, so string.rep("x", "3") raises.
  • OP_IDIV does not dispatch __idiv metamethods.

Verification

go vet ./..., golangci-lint v2.8.0 with the CI flags, go test ./... (full fixture corpus), the coverage collector run, and the race "other" shard (go test -race on all non-fixture tests and the non-regression fixture categories) pass. The regression-fixture race shards are left to CI.

The runtime picks this up through a go-lua dependency bump after merge.

String-to-number conversion had five independent implementations (the
runtime's parseNumber and parseNumberValue, tonumber's own parser, the
parser's literal parser and numparse for the type checker), and none
followed Lua: parseNumber always produced floats, so tonumber("10"),
"10" + 0 and -"10" were floats; tonumber rejected exponents ("1e5"),
hex floats, hex integers that wrap, numerals outside the double range and
whitespace other than space, tab and newline; tonumber with a base
returned floats and accepted out-of-range bases and numbers; the lexer
stopped hexadecimal numerals at '.' and 'p', and a numeral the compiler
could not parse became NaN instead of a compile error.

numparse.Parse now implements l_str2int and l_str2d from Lua 5.4 and
numparse.ParseBase implements tonumber's base conversion. Every
conversion uses them: tonumber, arithmetic and bitwise coercion of string
operands (including // and unary minus, which keep the integer subtype),
CheckNumber, ToInt, LVAsNumber, math.tointeger, the lexer (which now reads
numerals as read_numeral does and rejects malformed ones), literal
compilation and the type checker's literal helpers.
@wolfy-j
wolfy-j requested a review from skhaz November 15, 2026 18:42
@wolfy-j
wolfy-j requested a review from skhaz November 15, 2026 18:48
@wolfy-j
wolfy-j merged commit 5b50a56 into main Nov 15, 2026
12 checks passed
@wolfy-j
wolfy-j deleted the fix/string-to-number branch November 15, 2026 20:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants