[Easy] sanitizer.py: add utf-8 encoding and log invalid regex in .termstoryignore (fixes #297)

[Mubashir78]

Summary

This PR fixes two robustness gaps in .termstoryignore loading within termstory/sanitizer.py: it pins file encoding to UTF-8 for consistent cross-platform decoding of non-ASCII patterns, and replaces silent error swallowing with explicit logging so invalid regex patterns and file-level I/O failures are now visible to operators. These changes improve debuggability and prevent silent misconfiguration of custom redaction rules.

Changes

• Sanitizer: Added import logging and initialized module-level logger in termstory/sanitizer.py
• File I/O: Updated open(path, 'r') to open(path, 'r', encoding='utf-8') in load_custom_ignore_rules() to ensure consistent UTF-8 decoding across platforms
• Error Handling: Replaced bare except re.error: pass with except re.error as e: logger.warning("Invalid regex in %s line %r: %s", path, line, e) to log invalid regex patterns
• Error Handling: Replaced bare except Exception: pass with except Exception as e: logger.warning("Failed to load ignore rules from %s: %s", path, e) to log file-level failures

Fixes

• Fixes [Easy] sanitizer.py: .termstoryignore loaded without utf-8; invalid regex silently dropped #297 — adds UTF-8 encoding and logging to .termstoryignore loading

Testing

Run the existing sanitizer test suite to verify custom ignore rule loading still functions correctly:

python -m pytest tests/test_sanitizer.py::test_custom_termstoryignore -v

To verify logging behavior, create a malformed .termstoryignore file and check that warnings are emitted:

echo "[invalid_regex" > ~/.termstoryignore
python -c "import termstory.sanitizer"  # Should log warning about invalid regex

Notes

The load_custom_ignore_rules() function executes once at module import time, so changes to .termstoryignore still require a TermStory restart to take effect . This is a documented limitation in SECURITY.md.

[git-miku]

Tip

👋 Hey @Mubashir78 — Miku's on it. Here are the most useful commands for this PR:

💬 Command	What it does
:miku /review	🔍 Line-by-line review
:miku /review-sc	✏️ Inline suggested changes
:miku /security	🛡️ Security scan
:miku /perf	⚡ Performance notes
:miku /test	🧪 Test suggestions
:miku /summarize	📝 TL;DR

Note

🎓 This repo is part of ECSOC26. Run :miku /ecsoc to add the ECSOC26 label to this PR.

Note

⭐ Star this repo to unlock all commands. Say :miku /help for the full list.

📖 All commands

🤖 AI-powered — 14 commands

Command	Description	Works on
:miku /ask <question>	Answer a question about the codebase	Issues & PRs
:miku /review	Full code review (auto-fetches diff)	PRs only
:miku /review-sc	Post inline suggested-change comments	PRs only
:miku /security	OWASP security review (injection, auth, crypto…)	PRs only
:miku /perf	Performance & efficiency analysis	PRs only
:miku /docs	Generate docstrings for changed code	PRs only
:miku /test	Suggest unit tests for changed code	PRs only
:miku /roast	Humorous but constructive code roast 🔥	PRs only
:miku /checklist	Smart diff-aware PR checklist	PRs only
:miku /summarize	TL;DR of this issue or PR	Issues & PRs
:miku /explain <target>	Explain a file, function, or concept	Issues & PRs
:miku /changelog	Draft a CHANGELOG entry	PRs only
:miku /estimate	Story-point effort estimate	Issues only
:miku /chat <message>	General conversation	Issues & PRs

🔧 Issue & PR management — 18 commands

Command	Description	Works on
:miku /assign [@user]	Assign to yourself or another user	Issues & PRs
:miku /unassign [@user]	Remove yourself or another user	Issues & PRs
:miku /label <name>	Add a label	Issues & PRs
:miku /remove-label <name>	Remove a label	Issues & PRs
:miku /close [reason]	Close this issue or PR	Issues & PRs
:miku /reopen	Reopen a closed issue or PR	Issues & PRs
:miku /approve [message]	Submit an approving review on a PR	PRs only
:miku /retitle <new title>	Change the title	Issues & PRs
:miku /duplicate #N	Mark as duplicate of #N and close	Issues & PRs
:miku /milestone [name]	Set or list open milestones	Issues & PRs
:miku /ship [version]	Mark PR as shipped / deployed	PRs only
:miku /lock [reason]	Lock conversation (resolved / off-topic / spam)	Issues & PRs
:miku /unlock	Unlock conversation	Issues & PRs
:miku /cc @user1 [@user2]	Request reviews	PRs only
:miku /lgtm [cancel]	Signal "Looks Good To Me"	Issues & PRs
:miku /hold [cancel]	Block merge; :miku /hold cancel releases it	Issues & PRs
:miku /wip	Toggle WIP (converts PR to draft)	Issues & PRs
:miku /merge	Merge this PR (CI + approval required)	PRs only

🎉 Community & utility — 9 commands

Command	Description	Works on
:miku /vote	Open a 👍 👎 🚀 reaction poll	Issues & PRs
:miku /kudos @user [msg]	Give public kudos to a contributor	Issues & PRs
:miku /stats	Repo stars, forks, top contributors	Issues & PRs
:miku /remind <dur> <msg>	Schedule a reminder (e.g. 2h, 30m, 1d)	Issues & PRs
:miku /size	Show PR size breakdown	PRs only
:miku /ping [@user ...]	Ping contributors with an AI message	Issues & PRs
:miku /joke	Tell a programming joke	Issues & PRs
:miku /shrug	Add ¯_(ツ)_/¯ label	Issues & PRs
:miku /help	Show this message	Issues & PRs

[greptile-apps]

Greptile Summary

This PR improves robustness of .termstoryignore loading in termstory/sanitizer.py by pinning file I/O to UTF-8 encoding and introducing a module-level logger — but the logger is never wired into the two except blocks it was meant to replace.

• import logging and logger = logging.getLogger(__name__) are added, but both except re.error: pass and except Exception: pass are left unchanged, so invalid regex patterns and I/O failures are still silently swallowed.
• The encoding='utf-8' fix on open() is correctly applied and does address the cross-platform decoding inconsistency.

Confidence Score: 4/5

Safe to merge only after wiring the logger into both except blocks; the utf-8 encoding change is correct and harmless.

The logger is defined but never called, so the primary goal of the PR — surfacing invalid regex and I/O errors — is not achieved. Misconfigured .termstoryignore patterns continue to be silently dropped.

termstory/sanitizer.py — both except blocks need to call logger.warning()

Important Files Changed

Filename	Overview
termstory/sanitizer.py	Adds import logging + logger init and utf-8 encoding fix, but both except blocks remain bare pass — the logger is dead code and the core logging objective of the PR is unimplemented

Sequence Diagram

%%{init: {'theme': 'neutral'}}%%
sequenceDiagram
    participant M as Module Import
    participant L as load_custom_ignore_rules()
    participant F as File System
    participant R as re.compile()
    participant Log as logger (termstory.sanitizer)

    M->>L: called at module load time
    L->>F: "open(path, 'r', encoding='utf-8')"
    alt File readable
        F-->>L: lines
        loop each non-comment line
            L->>R: re.compile(line, IGNORECASE)
            alt Valid regex
                R-->>L: compiled pattern appended
            else re.error
                Note over L,Log: except re.error: pass — logger.warning NOT called
            end
        end
    else Exception (I/O error etc.)
        Note over L,Log: except Exception: pass — logger.warning NOT called
    end
    L-->>M: tuple(local_patterns)

Loading

%%{init: {'theme': 'base', 'themeVariables': {"darkMode": true, "background": "#0d1117", "primaryColor": "#21262d", "primaryTextColor": "#e6edf3", "primaryBorderColor": "#8b949e", "lineColor": "#8b949e", "textColor": "#e6edf3", "edgeLabelBackground": "#161b22", "actorBkg": "#21262d", "actorBorder": "#8b949e", "actorTextColor": "#e6edf3", "actorLineColor": "#8b949e", "signalColor": "#8b949e", "signalTextColor": "#e6edf3", "noteBkgColor": "#373320", "noteBorderColor": "#d4a72c", "noteTextColor": "#f0e6c0", "labelBoxBkgColor": "#21262d", "labelBoxBorderColor": "#8b949e", "labelTextColor": "#e6edf3", "loopTextColor": "#e6edf3", "activationBkgColor": "#30363d", "activationBorderColor": "#8b949e"}}}%%
sequenceDiagram
    participant M as Module Import
    participant L as load_custom_ignore_rules()
    participant F as File System
    participant R as re.compile()
    participant Log as logger (termstory.sanitizer)

    M->>L: called at module load time
    L->>F: "open(path, 'r', encoding='utf-8')"
    alt File readable
        F-->>L: lines
        loop each non-comment line
            L->>R: re.compile(line, IGNORECASE)
            alt Valid regex
                R-->>L: compiled pattern appended
            else re.error
                Note over L,Log: except re.error: pass — logger.warning NOT called
            end
        end
    else Exception (I/O error etc.)
        Note over L,Log: except Exception: pass — logger.warning NOT called
    end
    L-->>M: tuple(local_patterns)

Loading

Comments Outside Diff (2)

1. termstory/sanitizer.py, line 28-31 (link)

   Logger initialized but never called

   The logger object is defined at module level but neither except block was updated to use it — both still silently pass. A .termstoryignore file with an invalid regex like [unclosed will compile no pattern and give no warning, which is the exact problem this PR aimed to fix. The same applies to the outer except Exception: pass on file I/O errors.

2. termstory/sanitizer.py, line 26-31 (link)

   Both except blocks still use bare pass, so the logger is never called. Wire up the warning calls that the PR description describes.

_{Reviews (2): Last reviewed commit: "Merge branch 'main' into fix/297-sanitiz..." | Re-trigger Greptile}

[bitflicker64]

/rewrite

[git-miku]

miku /rewrite

✅ PR description rewritten! Check the PR body above.

Preview

Summary

This PR fixes two robustness gaps in .termstoryignore loading within termstory/sanitizer.py: it pins file encoding to UTF-8 for consistent cross-platform decoding of non-ASCII patterns, and replaces silent error swallowing with explicit logging so invalid regex patterns and file-level I/O failures are now visible to operators. These changes improve debuggability and prevent silent misconfiguration of custom redaction rules.

Changes

• Sanitizer: Added import logging and initialized module-level logger in termstory/sanitizer.py
• File I/O: Updated open(path, 'r') to open(path, 'r', encoding='utf-8') in load_custom_ignore_rules() to ensure consistent UTF-8 decoding across platforms
• Error Handling: Replaced bare except re.error: pass with except re.error as e: logger.warning("Invalid regex in %s line %r: %s", path, line, e) to log invalid regex patterns
• Error Handling: Replaced bare except Exception: pass with except Exception as e: logger.warning("Failed to load ignore rules from %s: %s", path, e) to log file-level failures

Fixes

• Fixes [Easy] sanitizer.py: .termstoryignore loaded without utf-8; invalid regex silently dropped #297 — adds UTF-8 encoding and logging to .termstoryignore loading

Testing

Run the existing sanitizer test suite to verify custom ignore rule loading still functions correctly:

python -m pytest tests/test_sanitizer.py::test_custom_termstoryignore -v

To verify logging behavior, create a malformed .termstoryignore file and check that warnings are emitted:

echo "[invalid_regex" > ~/.termstoryignore
python -c "import termstory.sanitizer"  # Should log warning about invalid regex

Notes

The load_custom_ignore_rules() function executes once at module import time, so changes to .termstoryignore still require a TermStory restart to take effect . This is a documented limitation in SECURITY.md.

[bitflicker64]

Can you resolve conflicts? Thanks.

[Mubashir78]

Can you resolve conflicts? Thanks.

Uh what conflict? The greplite says 5/5 and can be merged safely.

[Mubashir78]

:miku /review-sc

[git-miku]

miku /review-sc

✅ No mechanical fixes found. Reviewed 1 file(s).

[bitflicker64]

Dismissed because a newer commit was pushed; Greptile will re-review the current head.

[bitflicker64]

/ecsoc /triage /approve /lgtm

[git-miku]

Approved via Miku.

[git-miku]

miku /ecsoc

🏷️ Added ECSOC26.

---

miku /triage

✅ Marked as triaged (triage/triaged applied, needs-triage removed).

---

miku /approve

✅ Approved! 🎉

---

miku /lgtm

✅ LGTM! Label added by @bitflicker64.

Use :miku /lgtm cancel to retract.