Skip to content

kiku-jw/DocStripper

Folders and files

NameName
Last commit message
Last commit date

Latest commit

Β 

History

11 Commits
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 

Repository files navigation

DocStripper Logo

🧹 DocStripper

AI-powered batch document cleaner β€” Remove noise from text documents automatically

Python Version License Product Hunt

DocStripper automatically removes noise from text documents. Remove page numbers, headers/footers, duplicate lines, and empty lines from .txt, .docx, and .pdf files. Choose between Fast Clean (instant) or Smart Clean (AI-powered). Works entirely in your browser - 100% private, no uploads, no sign-ups.

🌐 Try it online β†’ β€” No installation needed!

πŸ“¦ Latest Release: v2.1.0 β€” UX enhancements & distribution ready


✨ Features

  • ⚑ Fast Clean β€” Instant rule-based cleaning
  • πŸ€– Smart Clean (Beta) β€” AI-powered cleaning with on-device LLM
  • 🎚️ 4 Cleaning Temperaments β€” Gentle (safe), Moderate, Thorough, Aggressive
  • βš™οΈ WebWorker Processing β€” Large files processed in background (no UI freezing)
  • πŸ”„ Side-by-Side Preview β€” Compare Original | Cleaned
  • πŸ’Ύ Settings Persistence β€” Your preferences are saved automatically
  • πŸ”’ 100% Private β€” All processing happens in your browser, works completely offline
  • πŸ“‘ Works Offline Badge β€” Visual indicator that everything stays on your device
  • πŸ“Š Real-time Statistics β€” See exactly what was removed
  • πŸ“₯ Batch Download (ZIP) β€” Download multiple cleaned files at once
  • 🎨 Dark Theme β€” Toggle between light and dark themes
  • πŸ“± Mobile Responsive β€” Works great on mobile devices

🎯 Quick Start

Web App (Recommended)

  1. Visit https://kiku-jw.github.io/DocStripper/
  2. Upload your files
  3. Choose Fast Clean (instant) or Smart Clean (AI-powered)
  4. Adjust Cleaning Temperament slider: Gentle (recommended), Moderate, Thorough, or Aggressive
  5. Click "Start Cleaning"
  6. Download or copy the cleaned results

CLI Tool

Installation Options

Option 1: PyPI (Recommended)

pip install docstripper
docstripper document.txt

Option 2: Homebrew (macOS)

brew tap kiku-jw/docstripper
brew install docstripper
docstripper document.txt

Option 3: Manual Installation

git clone https://github.com/kiku-jw/DocStripper.git
cd DocStripper
python tool.py document.txt

See INSTALL.md for detailed installation instructions.

Usage

# Clean a file
python tool.py document.txt

# Clean multiple files
python tool.py file1.txt file2.txt file3.docx

# Preview changes (dry-run)
python tool.py --dry-run document.txt

# Undo last operation
python tool.py --undo
 
# Pipe stdin to stdout (no file writes)
cat input.pdf | python tool.py - --stdout > output.txt

# Keep headers/footers if needed
python tool.py --keep-headers input.pdf --stdout

πŸ“– Example

Before:

Page 1 of 10
Confidential - Internal Use Only
Executive Summary
This is auto-
matic text processing.
Important content here.
Important content here.

1
2
3

After (Gentle Mode):

Executive Summary
This is automatic text processing.
Important content here.
More content.

Key Changes:

  • βœ… Page numbers removed
  • βœ… Headers/footers removed
  • βœ… Repeating headers removed
  • βœ… Duplicates collapsed
  • βœ… Hyphenation fixed
  • βœ… Empty lines removed

🎨 What Gets Removed?

Cleaning Temperaments

Gentle (Recommended - Default)

  • βœ… Page numbers (1, 2, 3...)
  • βœ… Headers/footers ("Page X of Y", "Confidential", etc.)
  • βœ… Repeating headers/footers across pages
  • βœ… Duplicate lines
  • βœ… Empty lines
  • βœ… Punctuation-only lines (---, ***, ===)
  • βœ… Hyphenation fixed (auto-\nmatic β†’ automatic)
  • βœ… Preserves paragraph spacing
  • ❌ Line merging disabled (preserves formatting)
  • ❌ Whitespace normalization disabled
  • ❌ Unicode normalization disabled

Moderate

  • All Gentle features plus:
  • βœ… Merges broken lines (protects lists and tables)
  • βœ… Preserves paragraph spacing

Thorough

  • All Moderate features plus:
  • βœ… Normalizes whitespace (protects tables)
  • βœ… Normalizes Unicode punctuation (smart quotes, dashes β†’ ASCII)
  • βœ… Preserves paragraph spacing (better readability)

Aggressive

  • All Thorough features plus:
  • βœ… Normalizes Unicode punctuation
  • ❌ Removes paragraph spacing (more compact output)

CLI Flags (defaults ON)

  • --no-merge-lines β€” disable merging broken lines
  • --no-dehyphenate β€” disable de-hyphenation across line breaks
  • --no-normalize-ws β€” disable whitespace normalization
  • --no-normalize-unicode β€” disable Unicode punctuation normalization
  • --keep-headers β€” keep headers/footers/page numbers
  • --stdout β€” write cleaned text to stdout instead of modifying files (supports - for stdin)

Protection Features:

  • βœ… Lists are never merged or broken
  • βœ… Tables preserve spacing
  • βœ… Content headers never removed

πŸ› οΈ Supported Formats

Format Status Notes
.txt βœ… Full UTF-8, Latin-1
.docx βœ… Basic Text extraction only (Web + CLI)
.pdf βœ… Basic Text extraction only (Web + CLI). Web uses PDF.js automatically. CLI requires pdftotext (poppler-utils)

PDF Support:

  • macOS: brew install poppler
  • Ubuntu/Debian: sudo apt-get install poppler-utils
  • Windows: Download from poppler-windows releases

πŸ”§ Requirements

Web App

  • Modern web browser (Chrome, Firefox, Safari, Edge)
  • No installation or dependencies required
  • Works completely offline after first load

CLI Tool

  • Python 3.9+ (for CLI tool)
  • PDF support (optional): pdftotext from poppler-utils

πŸ“ Changelog

See GitHub Releases for release notes and changelog.


πŸ“ License

MIT License β€” see LICENSE.txt for details.


🀝 Contributing

Contributions are welcome! See Contributing Guide for guidelines.


Made with ❀️ for clean documents

⭐ Star this repo | 🌐 Try online | πŸš€ Product Hunt | πŸ› Report Bug


πŸ’ Support

Support this project and help keep it free:

β˜• Support on Gumroad | β˜• Buy Me a Coffee | πŸ™ Thanks.dev | πŸ’š Ko-fi

πŸ”— Connect

About

CLI utility for cleaning copied or extracted text documents.

Topics

Resources

License

Code of conduct

Security policy

Stars

3 stars

Watchers

1 watching

Forks

Packages

 
 
 

Contributors