Repository navigation
fix(utf8): advance multibyte iterators and bound offset positions - #91
Merged
Merged
Conversation
skhaz
approved these changes
Nov 15, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
utf8.codestreated its 1-based control value as the next byte index without skipping continuation bytes. IteratingAä€😀returned the first multibyte codepoint, then raisedinvalid UTF-8 codeon its continuation byte. Advance to the next leading byte while preserving the byte positions Lua returns.The adjacent
utf8.offsetimplementation indexed beyond the string for zero-offset queries at the end, includingutf8.offset("a", 0, 2). It also used the wrong negative-index adjustment and the wrong default start for backwards traversal. Bounds now match Lua 5.4: invalid positions and continuation-byte starting positions raise Lua errors; backwards traversal starts at the end; offset zero accepts the end sentinel safely.Regression scripts fail on the previous implementation, pass after the fix, and also pass against reference Lua 5.4. Coverage includes 1–4-byte characters, empty input, truncated UTF-8, the valid replacement character, backwards traversal, negative positions, the end sentinel and out-of-range positions. Focused UTF-8 tests pass with the race detector.
This is the dependency correction needed before exposing the UTF-8 library through runtime #1086. No interpreter refactor is included.