Skip to content

fix: use explicit utf-8 encoding and LF newlines for file I/O - #2308

Closed
sravankumarkunadi wants to merge 1 commit into
elementary-data:masterfrom
sravankumarkunadi:fix/explicit-utf8-encoding-file-io
Closed

fix: use explicit utf-8 encoding and LF newlines for file I/O#2308
sravankumarkunadi wants to merge 1 commit into
elementary-data:masterfrom
sravankumarkunadi:fix/explicit-utf8-encoding-file-io

Conversation

@sravankumarkunadi

@sravankumarkunadi sravankumarkunadi commented Jul 27, 2026

Copy link
Copy Markdown

What

Pass encoding="utf-8" explicitly to every Path.read_text() / Path.write_text() call in scripts/generate_report_from_json.py and tests/unit/messages/utils.py, and newline="\n" to the fixture writes.

Why

read_text()/write_text() fall back to locale.getpreferredencoding() and platform newline translation when no encoding is given. That's UTF-8/LF on Linux and macOS, but cp1252/CRLF on Windows.

Before

  • scripts/generate_report_from_json.py raises UnicodeDecodeError on Windows when the report template (elementary/monitor/data_monitoring/report/index.html) or the JSON output contains any non-cp1252 character.
  • tests/unit/messages/utils.py run with OVERRIDE=true on Windows rewrites the expected JSON/text fixtures with CRLF line endings, producing a diff on every fixture file that touches the repo.

After

  • Both files are read and written as UTF-8 on every platform.
  • Regenerated fixtures keep LF line endings regardless of the platform they were generated on.

Notes

No behavior change on Linux/macOS — this only makes the already-assumed defaults explicit. Path.write_text(newline=...) requires Python 3.10, which matches this project's python = ">=3.10,<3.14".

Summary by CodeRabbit

  • Bug Fixes
    • Improved report generation consistency across platforms by standardizing UTF-8 file encoding.
    • Normalized line endings in generated reports and expected test files for more reliable output.

Path.read_text()/write_text() default to the platform's preferred
encoding and newline translation. On Windows this means cp1252 and
CRLF, which breaks reading the report template and JSON output
(UnicodeDecodeError) and rewrites the message test fixtures with CRLF
line endings when running with OVERRIDE=true.

Pass encoding="utf-8" explicitly on all reads and writes, and
newline="\n" on the fixture writes so generated expected files stay
LF regardless of platform.
@github-actions

Copy link
Copy Markdown
Contributor

👋 @sravankumarkunadi
Thank you for raising your pull request.
Please make sure to add tests and document all user-facing changes.
You can do this by editing the docs files in this pull request.

@coderabbitai

coderabbitai Bot commented Jul 27, 2026

Copy link
Copy Markdown

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: 50757995-9dc1-4db9-840b-f1aec7ab7a22

📥 Commits

Reviewing files that changed from the base of the PR and between 594b89d and 7179c48.

📒 Files selected for processing (2)
  • scripts/generate_report_from_json.py
  • tests/unit/messages/utils.py

📝 Walkthrough

Walkthrough

Report generation and message fixture helpers now use explicit UTF-8 encoding. Fixture writes also normalize newlines while preserving trailing newline behavior for expected text.

Changes

Encoding Consistency

Layer / File(s) Summary
UTF-8 file I/O updates
scripts/generate_report_from_json.py, tests/unit/messages/utils.py
Report inputs and templates are read as UTF-8. Expected JSON and text fixtures use UTF-8 with normalized newline handling for creation, reading, and overrides.

Estimated code review effort: 1 (Trivial) | ~5 minutes

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title accurately summarizes the main change: explicit UTF-8 encoding and LF newlines for file I/O.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@sravankumarkunadi

Copy link
Copy Markdown
Author

Closing this — opened in error, unrelated to the work I'm actually doing on this repo. Sorry for the noise.

@sravankumarkunadi
sravankumarkunadi deleted the fix/explicit-utf8-encoding-file-io branch July 27, 2026 03:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant