tpu-rs

Encoding-aware text processing utilities for Rust, the command line, and AI coding agents.

tpu-rs is a Cargo workspace that ships two cooperating tools focused on one problem: reading and writing text files without corrupting their encoding or line endings. The common offenders — PowerShell Get-Content / Set-Content, shell redirection through the active code page, and naive editors — silently mangle UTF-8, UTF-16, smart quotes, em-dashes, and box-drawing characters. tpu detects the actual on-disk encoding (UTF-8, UTF-16LE/BE, Windows-1252, Shift-JIS, …) and CRLF/LF/CR convention and round-trips them faithfully with atomic writes. The CLI keeps the prior contents at <file>.bak; the MCP server cleans up successful .bak backups after verification.

Crates in this workspace

Crate	What it is	Published
tpu	Encoding-aware file I/O CLI: read, write, replace, edit, find, doctor, validate, …	crates.io/crates/tpu
tpu-mcp	Model Context Protocol server exposing tpu's primitives as tools for GitHub Copilot Chat and other MCP clients	crates.io/crates/tpu-mcp

The VS Code extension that bundles tpu-mcp and auto-registers it as an MCP server is published to the Visual Studio Marketplace:

• Marketplace: MikeGrierTools.tpu-mcp

Install

tpu CLI — from crates.io

cargo install tpu

tpu-mcp server — from crates.io

cargo install tpu-mcp

tpu-mcp spawns tpu as a subprocess, so install both into the same location (or ensure tpu is on PATH).

VS Code extension — from the Marketplace

code --install-extension MikeGrierTools.tpu-mcp

The extension carries its own bundled tpu-mcp binary and registers an MCP server with VS Code on startup; no separate cargo install is required for the Copilot Chat integration (currently published for Windows: win32-x64 and win32-arm64).

Why use it

• No silent mojibake. Mutating tools refuse to introduce new mojibake patterns, and tpu doctor (or tpu_doctor over MCP) diagnoses and optionally repairs existing damage.
• Round-trips real-world encodings. UTF-8, UTF-16LE/BE, Windows-1252, Shift-JIS, and CRLF/LF/CR line endings are detected and preserved.
• Automatic Git worktree policy. Within a repository, .gitattributes working-tree-encoding, text, and eol policy (plus core.autocrlf / core.eol) is discovered automatically. Reads decode the declared worktree encoding, writes preserve it and normalise to a definite Git EOL convention, and tpu doctor --fix=eol repairs existing mismatches, including UTF-16. Discovery and open handles are cached for each operation, so a glob rooted above multiple repositories opens each worktree at most once while later operations still notice repository or configuration changes. Multi-file operations resolve policy as each path is encountered; they do not snapshot or lock .gitattributes for the entire operation, so concurrent attribute edits can cause earlier and later paths to use different policies.
• Safer than shelling out. Atomic writes, automatic .bak backups, pre-flight validate selectors, and walk tools that warn on inaccessible entries instead of aborting mid-tree.
• First-class agent integration. Every CLI primitive is also an MCP tool, so Copilot Chat can edit files without falling back to PowerShell.

Repository layout

crates/
  tpu/        # encoding-aware CLI + library
  tpu-mcp/    # MCP server + VS Code extension
tools/        # repo-local helpers (e.g. check-encoding.ps1)

See crates/tpu-mcp/README.md for command reference, configuration, and integration notes, and run tpu --help for the full CLI command reference.

License

MIT — see LICENSE.