Decode line ranges using the requested text encoding

[Gonghan-Princess]

Describe your changes

Fix encoding-aware line ranges in read_file_lines without loading the whole file into memory.

The default whole-file path decodes text before splitting lines, but the range path splits raw bytes at 0x0a and decodes each chunk independently. For UTF-16/UTF-32 this can split a code unit, produce garbage, or raise UnicodeDecodeError. The binary line count also gives incorrect negative indexes for these encodings.

• Decode incrementally with the requested encoding for both the range scan and negative-index line count.
• Preserve existing LF-delimited behavior, including raw CRLF and embedded bare CR when strip_white=False, by setting newline="\n".
• Add regressions for UTF-8, UTF-8 with BOM, UTF-16/UTF-32 (including big-endian variants), positive/negative ranges, trailing newlines, out-of-range indexes, empty files, and unstripped line endings.
• Add an Unreleased changelog entry. The README already documents the encoding argument and requires no API update.

Minimal reproduction on the previous implementation:

from pathlib import Path
import fsutil

path = Path("encoded.txt")
path.write_bytes("甲\n乙\n丙\n".encode("utf-16"))
fsutil.read_file_lines(path, line_start=1, line_end=1, encoding="utf-16")
# Expected: ["乙"]. Previously returned ["\u5900\u0a4e"] on Windows/Python 3.12.

Related issue

No issue filed; found while testing encoded line-range reads.

Validation

Windows, Python 3.12.10, using the project's existing pinned test environment:

• Regression RED before the source fix: 28 failed, 65 passed.
• python -X utf8 -m pytest tests --cov=fsutil --cov-report=term-missing --cov-fail-under=90: 258 passed, 3 existing Windows-specific skips, 98.36% coverage.
• python -X utf8 -m pre_commit run --all-files --show-diff-on-failure: all configured hooks passed.
• python -X utf8 -m mypy --install-types --non-interactive: passed, 13 source files.
• git diff --check: passed.

The upstream multi-platform/Python-version CI matrix has not been run locally.

Checklist before requesting a review

• I have performed a self-review of my code.
• I have added tests for the proposed changes.
• I have run the tests and there are not errors.

AI assistance

AI tools assisted with reproducing the bug, implementing the minimal fix, writing tests, and preparing this description. No human review is claimed. No third-party code or assets were copied into this change.

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 98.48%. Comparing base (68a343a) to head (0291353).
⚠️ Report is 2 commits behind head on main.

Additional details and impacted files

@@           Coverage Diff           @@
##             main     #195   +/-   ##
=======================================
  Coverage   98.48%   98.48%           
=======================================
  Files          13       13           
  Lines         793      794    +1     
=======================================
+ Hits          781      782    +1     
  Misses         12       12           

Flag	Coverage Δ
unittests	98.48% <100.00%> (+<0.01%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.