fix(out): reconfigure stdout to UTF-8 on any non-UTF-8 terminal

[bearomorphism]

Description

Closes #956.

Why

Commitizen emits Unicode characters that fall outside the Latin-1 repertoire — most visibly the rocket emoji (\U0001f680) in out.success("Configuration complete 🚀") and the typographic apostrophe (\u2019) scattered throughout the conventional-commits info text. When Python's print() encodes those characters through sys.stdout, it uses whatever encoding the operating system assigned to the terminal. On a system whose LANG is set to de_CH.ISO8859-1 (or any other non-UTF-8 locale), that encoding is Latin-1, which cannot represent either character, so Python raises UnicodeEncodeError and crashes the command mid-output.

The original guard in commitizen/out.py:7-9 only reconfigures sys.stdout to UTF-8 when sys.platform == "win32". Linux and macOS users on legacy ISO-8859-1 locales — the reporter ran AlmaLinux 8.9 and macOS Ventura 13.6.3, both with LANG=de_CH.ISO8859-1 — received no such reconfiguration, so every print() call that contained an out-of-range character exploded. Maintainer @Lee-W acknowledged the reproduction in the issue thread; a verification comment on the triage audit (#1964) confirmed the code path is unchanged in master (v4.15.1).

The fix extracts the reconfiguration into a small _ensure_utf8_stdout() helper that is encoding-agnostic and platform-agnostic. It is called once at module-import time so that every subsequent print() in every command — write, line, success, info, warn — is already operating on a UTF-8 stream before the first byte is written.

What changed

File	Change
commitizen/out.py	Drop the sys.platform == "win32" guard (lines 7–9); add _ensure_utf8_stdout() helper; call it unconditionally on sys.stdout at module import
tests/test_out.py	New file — 6 unit tests: UTF-8 no-op, dashless-alias no-op (UTF8), ISO-8859-1 reconfigure, cp1252 reconfigure, non-TextIOWrapper skip, and end-to-end emoji write-through

How it works

• Encoding normalisation before comparison. The helper computes (stream.encoding or "").lower().replace("-", "").replace("_", "") before comparing against "utf8". This collapses the alias zoo — UTF-8, utf_8, UTF8 — into a single canonical string, so the no-op path is taken for all valid UTF-8 spelling variants without maintaining an allowlist.

• errors="replace" as a safety net. Even after reconfiguration, a terminal that physically can't render a byte sequence (e.g., a very old xterm with a broken locale) will receive a ? placeholder rather than raising a second UnicodeEncodeError. Crashing mid-output after a user has already answered several cz init prompts is a worse experience than a one-character substitution.

• Module-level call, not per-print wrapping. _ensure_utf8_stdout(sys.stdout) executes once when commitizen.out is first imported. Because every CLI command imports out before writing anything, the reconfiguration is guaranteed to be in place for the full lifetime of the process — no per-function guard is needed.

• isinstance(stream, io.TextIOWrapper) guard. Pipes, io.StringIO instances, and pytest's stdout-capture objects are not TextIOWrapper subclasses and do not expose reconfigure(). Checking the type before calling keeps the helper safe in test environments and in any context where sys.stdout has been replaced by a non-standard object.

• AttributeError / ValueError are swallowed silently. A stream whose underlying buffer is already closed raises ValueError; a stream backed by a read-only file descriptor raises AttributeError. Neither case should crash the import. The # pragma: no cover annotation marks these as genuine safety-net branches rather than untested logic.

Backward compatibility

• Windows behaviour is unchanged: the new helper fires on any platform when encoding is not UTF-8, including cp1252 (the historical Windows case), so the reconfiguration that previously lived behind the win32 guard still happens.
• Callers that already have a UTF-8 sys.stdout (the common case on modern Linux/macOS) hit the early-return branch — no reconfigure() call, no observable difference.
• All existing tests pass; tests/test_out.py is a new file, so there are no test regressions.
• No CLI flags, exit codes, or configuration keys are added or removed.

Checklist

• I have read the contributing guidelines

Was generative AI tooling used to co-author this PR?

• Yes (please specify the tool below)

Generated-by: Claude following the guidelines

Code Changes

• Add test cases to all the changes you introduce
• Run uv run poe all locally to ensure this change passes linter check and tests
• Manually test the changes (see "Steps to Test" below)
• Update the documentation for the changes

Expected Behavior

Scenario	Outcome
LANG=de_CH.ISO8859-1 cz info on Linux/macOS	Prints full info text without UnicodeEncodeError; \u2019 (typographic apostrophe) passes through or is replaced by ?
LANG=de_CH.ISO8859-1 cz init completes successfully	Prints Configuration complete 🚀 without crashing on \U0001f680
LANG=en_US.UTF-8 cz info (UTF-8 terminal)	Behaviour unchanged — _ensure_utf8_stdout hits the early-return branch and calls no reconfigure()
Non-TextIOWrapper stdout (e.g. piped, pytest capture)	_ensure_utf8_stdout returns immediately; no AttributeError

Steps to Test This Pull Request

git fetch fork fix/956-cross-platform-stdout-utf8
git checkout fork/fix/956-cross-platform-stdout-utf8

# 1. Targeted regression test.
uv run pytest tests/test_out.py -v

# 2. Reproduce-the-bug-then-verify-the-fix sequence.
# Simulate a Latin-1 stdout without needing to change the OS locale:
python - << 'EOF'
import io, sys

# Replace stdout with a Latin-1 TextIOWrapper — exactly what the OS gives
# you when LANG=de_CH.ISO8859-1.
sys.stdout = io.TextIOWrapper(sys.stdout.buffer, encoding="latin-1")

# Re-import out so _ensure_utf8_stdout() runs against our stub stream.
import importlib, commitizen.out as _out
importlib.reload(_out)

# Before the fix this line crashed with UnicodeEncodeError.
_out.write("Conventional commits info: \u2019")
_out.success("Configuration complete \U0001f680")
print("OK — no UnicodeEncodeError", file=sys.__stdout__)
EOF

# On a real Linux/macOS machine with a non-UTF-8 locale:
# LANG=de_CH.ISO8859-1 cz info       # previously crashed
# LANG=de_CH.ISO8859-1 cz init       # previously crashed at success message

Additional Context

This fix emerged from the cross-cutting audit in #1964, which confirmed the Windows-only guard in commitizen/out.py:7-9 was the sole cause and that the code path is unchanged in master (v4.15.1). The _ensure_utf8_stdout helper was designed to be testable without monkey-patching the real sys.stdout and reimporting the module, which is why it accepts an arbitrary stream argument rather than hard-coding sys.stdout inside the function body.

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 98.24%. Comparing base (4b93a50) to head (4606a51).
⚠️ Report is 3 commits behind head on master.

Additional details and impacted files

@@           Coverage Diff           @@
##           master    #1975   +/-   ##
=======================================
  Coverage   98.23%   98.24%           
=======================================
  Files          61       61           
  Lines        2779     2785    +6     
=======================================
+ Hits         2730     2736    +6     
  Misses         49       49           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[Copilot]

Pull request overview

This PR addresses crashes caused by UnicodeEncodeError when Commitizen prints Unicode characters to a non‑UTF‑8 sys.stdout (e.g., ISO‑8859‑1 locales), by normalizing stdout to UTF‑8 early and adding targeted regression tests.

Changes:

• Replace the Windows-only stdout reconfiguration with a platform-agnostic _ensure_utf8_stdout() helper.
• Invoke _ensure_utf8_stdout(sys.stdout) at commitizen.out import time so all subsequent output uses the normalized stream.
• Add a new tests/test_out.py suite covering UTF‑8 no-op behavior and reconfiguration for common non‑UTF‑8 encodings.

Reviewed changes

Copilot reviewed 2 out of 7 changed files in this pull request and generated 2 comments.

File	Description
commitizen/out.py	Introduces _ensure_utf8_stdout() and calls it at import time to avoid UnicodeEncodeError on non‑UTF‑8 stdout.
tests/test_out.py	Adds regression tests validating stdout reconfiguration logic across encodings and stream types.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.