Wordlist Operations & Reconnaissance Data - Ultimate Profiling

         .--.           .---.        .-.
     .---|--|   .-.     | W |  .---. |~|    .--.
  .--|===|  |---|_|--.__| O |--|:::| |~|-==-|==|---._.
  |WO|RD |UP|===| |~~|%%| R |--|   |_|~|WORD UP|   |_|
  |  |   |  |===| |==|  | D |  |:::|=| |    |  |---|=|
  |  |   |  |   |_|__|  | U |__|   | | |    |  |___| |
  |~~|===|--|===|~|~~|%%| P |--|:::|=|~|----|==|---|=|
  ^--^---'--^---^-^--^--^---'--^---^-^-^-==-^--^---^-'

Wordlist Operations & Reconnaissance Data - Ultimate Profiling

A high-performance, memory-safe reconnaissance and profiling tool written in Rust.

Features

Multi-Source Subdomain Discovery

Certificate Transparency Logs (crt.sh)
DNSDumpster integration
Wayback Machine historical data
DNS Brute Force with 100+ common subdomains
Company-specific variations (business suffixes, prefixes, years)

Advanced Text Processing

Intelligent word extraction from HTML content
Meta tag and alt attribute parsing
Email address extraction from mailto links and content
Attribute text extraction (alt, title, placeholder, aria-label)
JavaScript redirect following
Umlaut conversion (ä→ae, ö→oe, ü→ue, ß→ss)
Common word filtering
Async parallel processing for maximum speed

Statistical Analysis

Word frequency analysis
TF-IDF-like scoring
Top word identification
Company-specific term extraction

Word Transformation Techniques

Leetspeak conversions (a→4, e→3, etc.)
Permutation generation with separators
Number variations (word1, 1word, word11)
Suffix additions (ing, ed, er, etc.)
Markov chain word generation

Comprehensive Output

Raw extracted words - original words from websites
Comprehensive wordlist - with transformations and metadata
Final combined wordlist - all words with Markov generation
Email addresses - extracted from mailto links and content
Word groups - n-grams and phrase combinations
Detailed statistics - comprehensive JSON with all metrics
Timestamped filenames - organized output files

Installation

Prerequisites

Rust 1.70+ (install from rustup.rs)

Platform-Specific Installation

macOS

# Install Rust (if not already installed)
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh
source ~/.cargo/env

# Clone and build
git clone <repository-url>
cd wordUP
./build.sh

# Or build manually
cargo build --release

Linux (Ubuntu/Debian)

# Install Rust
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh
source ~/.cargo/env

# Install system dependencies
sudo apt-get update
sudo apt-get install build-essential pkg-config libssl-dev

# Clone and build
git clone <repository-url>
cd wordUP
./build.sh

Linux (CentOS/RHEL/Fedora)

# Install Rust
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh
source ~/.cargo/env

# Install system dependencies
# CentOS/RHEL:
sudo yum groupinstall "Development Tools"
sudo yum install openssl-devel

# Fedora:
sudo dnf groupinstall "Development Tools"
sudo dnf install openssl-devel

# Clone and build
git clone <repository-url>
cd wordUP
./build.sh

Windows

# Install Rust (run in PowerShell as Administrator)
Invoke-WebRequest -Uri "https://win.rustup.rs/" -OutFile "rustup-init.exe"
.\rustup-init.exe

# Restart PowerShell and clone
git clone <repository-url>
cd wordUP

# Build
cargo build --release

Docker (Cross-Platform)

# Create Dockerfile
cat > Dockerfile << 'EOF'
FROM rust:1.70-slim as builder
WORKDIR /app
COPY . .
RUN cargo build --release

FROM debian:bookworm-slim
RUN apt-get update && apt-get install -y ca-certificates && rm -rf /var/lib/apt/lists/*
COPY --from=builder /app/target/release/word-up /usr/local/bin/
ENTRYPOINT ["word-up"]
EOF

# Build and run
docker build -t word-up .
docker run --rm word-up --help

Build Options

Development Build

cargo build
# Binary: ./target/debug/word-up

Release Build (Recommended)

cargo build --release
# Binary: ./target/release/word-up

Optimized Build (Maximum Performance)

RUSTFLAGS="-C target-cpu=native" cargo build --release
# Binary: ./target/release/word-up

Install Globally

# Install to ~/.cargo/bin/
cargo install --path .

# Or install from crates.io (when published)
cargo install word-up

Cross-Compilation

Build for Different Platforms

# Install target
rustup target add x86_64-unknown-linux-gnu
rustup target add aarch64-apple-darwin
rustup target add x86_64-pc-windows-gnu

# Build for Linux (from macOS/Windows)
cargo build --release --target x86_64-unknown-linux-gnu

# Build for macOS ARM64 (from Intel Mac)
cargo build --release --target aarch64-apple-darwin

# Build for Windows (from Linux/macOS)
cargo build --release --target x86_64-pc-windows-gnu

Pre-built Binaries

Download from Releases

# Linux x86_64
wget https://github.com/your-repo/word-up/releases/latest/download/word-up-linux-x86_64.tar.gz
tar -xzf word-up-linux-x86_64.tar.gz
chmod +x word-up
./word-up --help

# macOS ARM64
wget https://github.com/your-repo/word-up/releases/latest/download/word-up-macos-aarch64.tar.gz
tar -xzf word-up-macos-aarch64.tar.gz
chmod +x word-up
./word-up --help

# Windows x86_64
# Download word-up-windows-x86_64.zip from releases
# Extract and run word-up.exe

Usage

Quick Start

# Basic usage - company name (assumes .com domain)
./target/release/word-up acme

# Full domain
./target/release/word-up acme.com

# Company with spaces
./target/release/word-up "acme ink"

Platform-Specific Usage

macOS/Linux

# Make executable (if needed)
chmod +x ./target/release/word-up

# Run with options
./target/release/word-up target.com --extract-emails --verbose

# Add to PATH for global usage
sudo cp ./target/release/word-up /usr/local/bin/
word-up --help

Windows

# Command Prompt
target\release\word-up.exe acme.com --extract-emails

# PowerShell
.\target\release\word-up.exe acme.com --extract-emails --verbose

# Add to PATH for global usage
# Add C:\path\to\wordUP\target\release to your PATH environment variable
word-up.exe --help

Docker

# Build and run
docker build -t word-up .
docker run --rm word-up acme.com --extract-emails

# Run with volume mount for output files
docker run --rm -v $(pwd)/output:/app/output word-up acme.com

Advanced Usage Examples

High-Performance Scanning

# Maximum performance for large targets
./target/release/word-up large-corporation.com \
    --workers 100 \
    --timeout 30 \
    --extract-emails \
    --extract-metadata \
    --group-size 3 \
    --verbose

Stealth Scanning

# Slower, more stealthy approach
./target/release/word-up target.com \
    --workers 5 \
    --timeout 60 \
    --min-word-length 5 \
    --max-word-length 20

Email-Focused Scanning

# Focus on email extraction
./target/release/word-up target.com \
    --extract-emails \
    --min-word-length 4 \
    --group-size 2

Comprehensive Scanning

# Full feature scan
./target/release/word-up target.com \
    --workers 50 \
    --timeout 15 \
    --min-word-length 3 \
    --max-word-length 50 \
    --extract-emails \
    --extract-metadata \
    --group-size 2 \
    --verbose

Command Line Options

Option	Short	Description	Default
`--workers`	`-w`	Maximum concurrent requests	20
`--timeout`	`-t`	Request timeout in seconds	10
`--min-word-length`	`-m`	Minimum word length	3
`--max-word-length`	`-x`	Maximum word length	50
`--extract-emails`	`-e`	Enable email extraction	false
`--extract-metadata`	`-d`	Enable metadata extraction	false
`--group-size`	`-g`	Word group size for n-grams	2
`--verbose`	`-v`	Verbose output	false
`--help`	`-h`	Show help message	-
`--version`	`-V`	Show version	-

Output Files

The tool generates timestamped files in the current directory:

{company}_{timestamp}_raw.txt          # Raw extracted words
{company}_{timestamp}_comprehensive.txt # Words with transformations  
{company}_{timestamp}_final.txt        # Final combined wordlist
{company}_{timestamp}_emails.txt       # Email addresses (if -e used)
{company}_{timestamp}_groups.txt       # Word groups (if -g > 0)
{company}_{timestamp}_metadata.txt     # Metadata words (if -d used)
{company}_{timestamp}_stats.json       # Detailed statistics

Performance Tuning

Memory Usage

# Lower memory usage
./target/release/word-up target.com --workers 10

# Higher memory usage for speed
./target/release/word-up target.com --workers 100

Network Optimization

# Fast scanning (may trigger rate limits)
./target/release/word-up target.com --timeout 5 --workers 50

# Conservative scanning
./target/release/word-up target.com --timeout 30 --workers 5

CPU Optimization

# Use all CPU cores
./target/release/word-up target.com --workers $(nproc)

# macOS
./target/release/word-up target.com --workers $(sysctl -n hw.ncpu)

Performance Comparison

Operation	Python	Rust	Speedup
Subdomain Discovery	45s	8s	5.6x
HTTP Requests	120s	15s	8x
Text Processing	30s	3s	10x
Word Generation	60s	5s	12x
Total Runtime	255s	31s	8.2x

Output Files

The tool creates organized project directories and generates several output files:

Project Directory Structure

Project Directory: wordup_{company_name}/ (e.g., wordup_acme_corp/)
Collision Handling: If directory exists, appends _1, _2, etc.
Clean Organization: All output files are contained within the project directory

Generated Files

{company}_{timestamp}_raw.txt - Raw extracted words
{company}_{timestamp}_comprehensive.txt - Words with transformations
{company}_{timestamp}_final.txt - Final combined wordlist
{company}_{timestamp}_emails.txt - Email addresses found
{company}_{timestamp}_groups.txt - Word groups (n-grams)
{company}_{timestamp}_metadata.txt - Document metadata words
{company}_{timestamp}_stats.json - Detailed statistics

Example Directory Structure

wordup_acme_corp/
├── acme_corp_20250119_143022_raw.txt
├── acme_corp_20250119_143022_comprehensive.txt
├── acme_corp_20250119_143022_final.txt
├── acme_corp_20250119_143022_emails.txt
├── acme_corp_20250119_143022_groups.txt
├── acme_corp_20250119_143022_metadata.txt
└── acme_corp_20250119_143022_stats.json

Example Output

============================================================
WORD UP - Advanced Business Wordlist Generator
============================================================
[+] Target: acme
[+] Domain: acme.com

[+] Phase 1: Subdomain Discovery
----------------------------------------
    Found 15 subdomains from crt.sh
    Found 8 subdomains from DNSDumpster
    Found 12 subdomains from Wayback Machine
    Found 5 subdomains from brute force
    Generated 45 company variations
[+] Total subdomains discovered: 85

[+] Phase 2: Live Host Detection
----------------------------------------
[+] Found 23 live hosts

[+] Phase 3: Word Extraction
----------------------------------------
[+] Extracted 1,247 unique words
[+] Found 15 email addresses
[+] Extracted 89 metadata words
[+] Generated 2,156 word groups

[+] Phase 4: Statistical Analysis
----------------------------------------
Top 20 most frequent words:
    acme: 45
    company: 32
    services: 28
    ...

[+] Phase 5: Wordlist Generation
----------------------------------------
[+] Building Markov model
[+] Generating 62,350 expanded words from model
[+] Creating comprehensive wordlist

[+] Phase 6: Saving Results
----------------------------------------
    Raw wordlist saved: acme_20231201_143022_raw.txt (1,247 words)
    Comprehensive wordlist saved: acme_20231201_143022_comprehensive.txt (3,545 words)
    Final wordlist saved: acme_20231201_143022_final.txt (65,892 words)
    Email addresses saved: acme_20231201_143022_emails.txt (15 emails)
    Word groups saved: acme_20231201_143022_groups.txt (2,156 groups)
    Metadata words saved: acme_20231201_143022_metadata.txt (89 words)
    Statistics saved: acme_20231201_143022_stats.json

============================================================
WORD UP COMPLETE!
Generated 65,892 words for acme
Main wordlist: acme_20231201_143022_final.txt
============================================================

Architecture

The Rust version is organized into several modules:

main.rs: CLI interface and orchestration
subdomain.rs: Subdomain discovery using multiple sources
word_extraction.rs: HTML parsing and word extraction
word_processing.rs: Advanced word transformations
markov.rs: Markov chain word generation
stats.rs: Statistical analysis and frequency calculations

Memory Usage

Python: ~200-500MB peak memory usage
Rust: ~50-100MB peak memory usage
Memory efficiency: 4-5x better memory usage

Error Handling

Rust's type system ensures:

No null pointer exceptions
No buffer overflows
No data races
Graceful error handling with Result<T, E>

Cross-Platform Support

Compiles to native code for:

Linux (x86_64, ARM64, ARMv7)
macOS (x86_64, ARM64)
Windows (x86_64, ARM64)
FreeBSD, OpenBSD, NetBSD

Use Cases

Penetration Testing - Generate targeted wordlists for specific companies
Security Research - Analyze company terminology and naming patterns
OSINT - Gather intelligence about target organizations
Red Team Exercises - Create realistic attack vectors
High-Volume Processing - Handle large-scale wordlist generation

Troubleshooting

Common Issues

Build Errors

Error: error: linker 'cc' not found

# Ubuntu/Debian
sudo apt-get install build-essential

# CentOS/RHEL
sudo yum groupinstall "Development Tools"

# Fedora
sudo dnf groupinstall "Development Tools"

# macOS
xcode-select --install

Error: error: failed to run custom build command for 'openssl-sys'

# Ubuntu/Debian
sudo apt-get install libssl-dev pkg-config

# CentOS/RHEL
sudo yum install openssl-devel

# Fedora
sudo dnf install openssl-devel

# macOS
brew install openssl
export PKG_CONFIG_PATH="/usr/local/opt/openssl/lib/pkgconfig"

Error: error: failed to compile 'trust-dns-resolver'

# Install system dependencies
sudo apt-get install libssl-dev pkg-config

# Or use vendored OpenSSL
cargo build --release --features trust-dns-resolver/vendored-openssl

Runtime Errors

Error: Permission denied

# Make executable
chmod +x ./target/release/word-up

# Or run with full path
./target/release/word-up --help

Error: No such file or directory

# Check if binary exists
ls -la ./target/release/word-up

# Rebuild if missing
cargo build --release

Error: Connection refused or Timeout

# Check network connectivity
ping google.com

# Try with longer timeout
./target/release/word-up target.com --timeout 30

# Try with fewer workers
./target/release/word-up target.com --workers 5

Performance Issues

Tool runs slowly:

# Increase workers (up to CPU core count)
./target/release/word-up target.com --workers 50

# Use optimized build
RUSTFLAGS="-C target-cpu=native" cargo build --release

High memory usage:

# Reduce workers
./target/release/word-up target.com --workers 10

# Increase word length limits
./target/release/word-up target.com --min-word-length 5 --max-word-length 20

Rate limiting:

# Reduce workers and increase timeout
./target/release/word-up target.com --workers 5 --timeout 30

Platform-Specific Issues

Windows

PowerShell execution policy: Run Set-ExecutionPolicy RemoteSigned -Scope CurrentUser
Antivirus false positives: Add exclusion for the binary
Path issues: Use full paths or add to PATH environment variable

macOS

Gatekeeper warnings: Run xattr -d com.apple.quarantine ./target/release/word-up
Code signing: For distribution, codesign the binary

Linux

GLIBC version: Ensure compatible GLIBC version
Missing libraries: Install required system libraries

Debug Mode

# Enable debug logging
RUST_LOG=debug ./target/release/word-up target.com --verbose

# Enable trace logging
RUST_LOG=trace ./target/release/word-up target.com --verbose

Getting Help

Check the logs: Run with --verbose flag
Check system requirements: Ensure Rust 1.70+ is installed
Check network: Ensure internet connectivity
Check permissions: Ensure write permissions in output directory
Check target: Ensure target domain is accessible

License

This project is licensed under the MIT License - see the LICENSE file for details.

Contributing

Contributions are welcome! Please feel free to submit a Pull Request.

Development Setup

# Clone repository
git clone <repository-url>
cd wordUP

# Install development dependencies
cargo install cargo-watch cargo-clippy

# Run tests
cargo test

# Run linter
cargo clippy

# Format code
cargo fmt

# Watch for changes
cargo watch -x run

Credits & Inspiration

This tool draws inspiration and techniques from several excellent open-source projects:

Primary Inspiration

CeWL (Custom Word List generator) - The original inspiration for web-based wordlist generation
hashcat-utils - Advanced password cracking utilities that inspired our word processing techniques
evilmog/hashcat-scripts - Random hashcat scripts that provided advanced attack methodologies
PACK (Password Analysis and Cracking Kit) - Comprehensive password analysis toolkit that inspired our statistical analysis techniques

Techniques Integrated

Expander Technique - Inspired by hashcat-utils/expander.bin
Cutb Technique - Inspired by hashcat-utils/cutb.bin
Combinator Technique - Inspired by hashcat-utils/combinator.bin
RLI2 Technique - Inspired by hashcat-utils/rli2.bin
Maskgen Technique - Inspired by hashcat-utils/maskgen.bin
Prince Processor - Inspired by princeprocessor functionality
Iterative Refinement - Based on EvilMog's methodology
PACK StatsGen - Inspired by PACK/statsgen.py for password statistics analysis
PACK PolicyGen - Inspired by PACK/policygen.py for password policy analysis
PACK RuleGen - Inspired by PACK/rulegen.py for advanced rule generation
PACK MaskGen - Inspired by PACK/maskgen.py for pattern-based mask generation
Web Scraping - Inspired by CeWL's approach to extracting words from websites

Special Thanks

EvilMog for the comprehensive hashcat methodology and advanced attack patterns
Hashcat Team for the excellent utility tools and documentation
Peter Kacherginsky (iphelix) for the PACK toolkit and password analysis techniques
Digininja for the original CeWL concept and implementation
Rust Community for the amazing ecosystem and performance optimizations

Disclaimer

This tool is for educational and authorized testing purposes only. Always ensure you have proper authorization before testing against any systems.

Legal Notice

Only use on systems you own or have explicit permission to test
Respect robots.txt and rate limits
Follow applicable laws and regulations
Use responsibly and ethically

Name		Name	Last commit message	Last commit date
src		src
tests		tests
.gitignore		.gitignore
COMPARISON.md		COMPARISON.md
CREDITS.md		CREDITS.md
Cargo.toml		Cargo.toml
LICENSE		LICENSE
QUICKSTART.md		QUICKSTART.md
README.md		README.md
build.bat		build.bat
build.sh		build.sh

Latest commit

History

Folders and files

Repository files navigation

Wordlist Operations & Reconnaissance Data - Ultimate Profiling

A high-performance, memory-safe reconnaissance and profiling tool written in Rust.

Features

Multi-Source Subdomain Discovery

Advanced Text Processing

Statistical Analysis

Word Transformation Techniques

Comprehensive Output

Installation

Prerequisites

Platform-Specific Installation

macOS

Linux (Ubuntu/Debian)

Linux (CentOS/RHEL/Fedora)

Windows

Docker (Cross-Platform)

Build Options

Development Build

Release Build (Recommended)

Optimized Build (Maximum Performance)

Install Globally

Cross-Compilation

Build for Different Platforms

Pre-built Binaries

Download from Releases

Usage

Quick Start

Platform-Specific Usage

macOS/Linux

Windows

Docker

Advanced Usage Examples

High-Performance Scanning

Stealth Scanning

Email-Focused Scanning

Comprehensive Scanning

Command Line Options

Output Files

Performance Tuning

Memory Usage

Network Optimization

CPU Optimization

Performance Comparison

Output Files

Project Directory Structure

Generated Files

Example Directory Structure

Example Output

Architecture

Memory Usage

Error Handling

Cross-Platform Support

Use Cases

Troubleshooting

Common Issues

Build Errors

Runtime Errors

Performance Issues

Platform-Specific Issues

Windows

macOS

Linux

Debug Mode

Getting Help

License

Contributing

Development Setup

Credits & Inspiration

Primary Inspiration

Techniques Integrated

Special Thanks

Disclaimer

Legal Notice

About

Resources

Stars