numfmt: don't panic on a multi-byte suffix under a multi-byte-separat…

[nagendramohan]

…or locale

Under a locale whose decimal separator is multi-byte (e.g. ar_SA.UTF-8, where it is the Arabic '٫', U+066B, 2 bytes), an input like '1٫€K' aborted with a char-boundary panic in find_valid_number_with_suffix.

The numeric part was iterated with s.chars().skip(numeric_part.len()) — using the byte length as a character count — which skipped past the multi-byte separator incorrectly, and the result was then sliced with &s[..=numeric_part.len()], landing inside a multi-byte suffix character.

Iterate the remainder from the numeric part's byte offset (a valid char boundary, since it is a prefix of the input) and slice using each suffix character's actual UTF-8 length. Malformed input is now rejected as GNU does ('invalid suffix in input'), and valid multi-byte-separator input such as '1٫5K' still parses.

Adds a regression test (verified failing before the fix). Closes #13937.

Closes #13937.

Under a locale whose decimal separator is multi-byte (e.g. ar_SA.UTF-8, where it's the Arabic ٫, U+066B, 2 bytes), an input like 1٫€K aborted with a char-boundary panic:

$ LC_ALL=ar_SA.UTF-8 numfmt --from=si '1٫€K'
thread 'main' panicked at src/uu/numfmt/src/format.rs:70:20:
byte index 4 is not a char boundary; it is inside '€' (bytes 3..6) of 1٫€K

Root cause: the numeric part was iterated with s.chars().skip(numeric_part.len()) — using its byte length as a character count — so a multi-byte separator was mis-skipped, and the result was then sliced with &s[..=numeric_part.len()], landing inside a multi-byte suffix char.

Fix: iterate the remainder from the numeric part's byte offset (a valid char boundary) and slice using each suffix char's actual len_utf8(). Malformed input is now rejected like GNU (invalid suffix in input), and valid multi-byte-separator input like 1٫5K still parses.

Adds a regression test (verified failing before the fix). cargo fmt, cargo clippy, and the full numfmt test suite pass.

[oech3]

Duplicated PR. Please close this.

[github-actions]

GNU testsuite comparison:

Skip an intermittent issue tests/cut/bounded-memory (fails in this run but passes in the 'main' branch)
Skip an intermittent issue tests/tail/symlink (fails in this run but passes in the 'main' branch)
Skipping an intermittent issue tests/tail/follow-name (passes in this run but fails in the 'main' branch)
Note: The gnu test tests/tail/pipe-f is now being skipped but was previously passing.
Congrats! The gnu test tests/cut/cut-huge-range is now passing!
Congrats! The gnu test tests/tail/tail-n0f is now passing!
Skip an intermittent issue tests/pr/bounded-memory (was skipped on 'main', now failing)

[codspeed-hq]

Merging this PR will improve performance by 5.04%

⚠️ Different runtime environments detected

Some benchmarks with significant performance changes were compared across different runtime environments,
which may affect the accuracy of the results.

Open the report in CodSpeed to investigate

⚡ 1 improved benchmark
✅ 356 untouched benchmarks
⏩ 50 skipped benchmarks¹

Performance Changes

	Mode	Benchmark	BASE	HEAD	Efficiency
⚡	Simulation	numfmt_stream_to_si_precision	348.8 ms	332.1 ms	+5.04%

Tip

Curious why this is faster? Comment @codspeedbot explain why this is faster on this PR, or directly use the CodSpeed MCP with your agent.

---

_{Comparing nagendramohan:fix/numfmt-multibyte-suffix-panic-13937 (fdb2e42) with main (df30282)}

Footnotes

1. 50 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports. ↩