CheckSysAsm

A Python tool for Gentoo Linux that verifies all system binaries comply with specified CPU instruction set constraints.

Why This Tool Exists

I created this tool after needing to move my Gentoo server installation to an older machine. The original system was compiled with -march=native in make.conf, optimized for the newer CPU. When I changed the march setting to match the older hardware and attempted a world rebuild, it didn't complete successfully the first time—as these things rarely do.

Rather than starting another full world rebuild from scratch (which can take days and lacks a reliable keep-going or resume option), I needed a way to identify exactly which binaries were incompatible. This tool provides a surgical approach: scan the system, find the problematic binaries, and rebuild only the affected packages. Much more efficient than rebuilding everything.

Purpose

When compiling Gentoo with specific CPU flags (e.g., -march=x86-64-v2), you want to ensure that all installed binaries are compatible with your target CPU. This tool:

1. Takes GCC compilation flags as input (like -march=x86-64-v2)
2. Translates those flags into allowed CPU instruction sets
3. Scans all executable code in your system (executables, shared libraries, kernel modules)
4. Identifies binaries that use forbidden CPU instructions
5. Maps violating binaries to their Gentoo packages using equery
6. Generates a ready-to-run emerge command to rebuild all affected packages

The Key Output: One-Command Rebuild

The most important feature is the package report with emerge command. After scanning, you get a file containing:

emerge -av1 \
  =sys-devel/gcc-13.2.1_p20240210 \
  =kde-apps/kolf-25.08.3 \
  =media-video/ffmpeg-7.1 \
  ...

Just copy and paste this command to rebuild all incompatible packages - no manual package hunting required!

Installation

Prerequisites

• Python 3.8 or later
• GCC (for architecture detection with -march=native)
• capstone disassembly library
• pyelftools for ELF parsing
• gentoolkit (for Gentoo package mapping with equery)

Install from source

# Clone the repository
cd /path/to/CheckSysAsm

# Create and activate virtual environment
python3 -m venv venv
source venv/bin/activate

# Install in development mode
pip install -e .

Install dependencies on Gentoo

# Enable Python bindings for capstone
echo "dev-libs/capstone python" >> /etc/portage/package.use/checksysasm

# Install system packages
emerge dev-libs/capstone dev-python/pyelftools gentoolkit

# Or use pip
pip install capstone pyelftools

Usage

Basic System Scan

Check all binaries against x86-64-v2 baseline:

checksysasm -m x86-64-v2

Auto-detect Native CPU

Use your CPU's native instruction set as the baseline:

checksysasm -m native

Check Specific Binary

Check a single binary instead of scanning the entire system:

checksysasm -m x86-64-v2 --check-binary /usr/bin/python3

Save Results to File

Save violations to a file:

checksysasm -m x86-64-v2 -o violations.txt

Output Formats

Different output formats are supported:

# Text format (default, human-readable)
checksysasm -m x86-64-v2 -o report.txt --format txt

# JSON format (machine-readable)
checksysasm -m x86-64-v2 -o report.json --format json

# CSV format (spreadsheet-compatible)
checksysasm -m x86-64-v2 -o report.csv --format csv

# Simple list (just file paths)
checksysasm -m x86-64-v2 -o files.list --format list

Gentoo Package Report

Generate a report mapping violating binaries to their packages:

checksysasm -m x86-64-v2 -o violations.txt --package-report packages.txt

This requires gentoolkit to be installed (emerge gentoolkit).

Scan Additional Directories

Scan custom directories in addition to standard system paths:

checksysasm -m x86-64-v2 --scan-path /opt --scan-path /usr/local

Parallel Processing

Adjust the number of parallel workers (default: 4):

checksysasm -m x86-64-v2 -j 8

Skip Kernel Modules

Exclude kernel modules from scanning:

checksysasm -m x86-64-v2 --no-kernel-modules

Skip Debug Files

Skip *.debug files (separate debug symbols), which are often very large and can cause memory issues:

checksysasm -m x86-64-v2 --skip-debug-files

Important: Files skipped with --skip-debug-files are never treated as violations, even if you use --exclude-as-violation. This is different from --exclude patterns, which respect the --exclude-as-violation flag.

# Debug files are ALWAYS skipped (never violations)
checksysasm -m x86-64-v2 --skip-debug-files --exclude-as-violation

# But excluded patterns CAN be violations
checksysasm -m x86-64-v2 --exclude "*/firefox/*" --exclude-as-violation

Verbose Output

Use verbose mode to see more details during scanning:

# Verbose level 1: Show violations as they're found
checksysasm -m x86-64-v2 -v

# Verbose level 2: Show ALL visited files (useful for debugging memory issues or crashes)
checksysasm -m x86-64-v2 -vv

The -vv option prints every file as it's being processed, which helps identify which file causes the program to crash if you're experiencing memory issues.

Exclude Files from Scanning

Exclude specific files or patterns from the scan:

# Exclude all files matching a pattern
checksysasm -m x86-64-v2 --exclude "*/firefox/*"

# Exclude multiple patterns
checksysasm -m x86-64-v2 --exclude "*/firefox/*" --exclude "*/chrome/*"

# Read exclusion patterns from a file (one pattern per line)
checksysasm -m x86-64-v2 --exclude-file excludes.txt

# Combine file and command-line patterns
checksysasm -m x86-64-v2 --exclude-file excludes.txt --exclude "*/temp/*"

# Treat excluded files as violations (for compliance reporting)
checksysasm -m x86-64-v2 --exclude "*/problematic/*" --exclude-as-violation

Note: The --exclude-as-violation flag only affects files matched by --exclude and --exclude-file. Files skipped with --skip-debug-files are never treated as violations.

Example exclusion file (excludes.txt):

# Browsers (comment lines start with #)
*/firefox/*
*/chrome/*
*/chromium/*

# CUDA libraries
*/cuda/*
/opt/nvidia/*

# Custom compiled software
/usr/local/custom/*

This is useful for:

• Avoiding files that block the process or cause crashes
• Skipping known problematic binaries
• Creating custom compliance policies
• Managing large lists of exclusions

Resumable Scans

Use session files to resume interrupted scans:

# Start a scan with session file (auto-saves every 100 binaries)
checksysasm -m x86-64-v2 --session-file scan.session

# If interrupted (Ctrl+C), resume with:
checksysasm -m x86-64-v2 --session-file scan.session

# You can add --skip-debug-files when resuming even if the original scan didn't use it
# Debug files from the session will be filtered out from final reports
checksysasm -m x86-64-v2 --session-file scan.session --skip-debug-files

Note: When resuming with --skip-debug-files, any *.debug files that were marked as violations in the session file will be automatically filtered out from the final report and package report. This allows you to retroactively exclude debug files without restarting the scan.

Supported CPU Architectures

x86-64 Microarchitecture Levels

• x86-64 - Baseline (SSE2)
• x86-64-v2 - SSE4.2, POPCNT
• x86-64-v3 - AVX2, BMI1, BMI2, F16C, FMA, LZCNT, MOVBE
• x86-64-v4 - AVX-512 (F, DQ, CD, BW, VL)

Intel Microarchitectures

• core2 - Core 2 (SSSE3)
• nehalem - Nehalem (SSE4.2)
• sandybridge - Sandy Bridge (AVX)
• ivybridge - Ivy Bridge (AVX, F16C)
• haswell - Haswell (AVX2)
• broadwell - Broadwell (AVX2)
• skylake - Skylake (AVX2)
• skylake-avx512 - Skylake-X (AVX-512)
• cascadelake - Cascade Lake (AVX-512, VNNI)
• icelake-client, icelake-server - Ice Lake (AVX-512)
• tigerlake - Tiger Lake (AVX-512)
• alderlake - Alder Lake (AVX2, no AVX-512)
• sapphirerapids - Sapphire Rapids (AVX-512)

AMD Microarchitectures

• k8 - K8 (SSE2)
• barcelona - Barcelona (SSE4a)
• bdver1-4 - Bulldozer family (AVX, FMA)
• btver1-2 - Bobcat/Jaguar (SSSE3/AVX)
• znver1-4 - Zen family (AVX2/AVX-512)

Individual Feature Flags

You can also specify individual instruction set extensions:

checksysasm -m haswell -f -mavx2 -f -mfma -f -mbmi2

How It Works

1. Parse GCC flags: The tool translates GCC compilation flags into a set of allowed CPU instruction extensions (SSE2, SSE3, AVX, AVX2, AVX-512, etc.)

2. Scan filesystem: It recursively scans standard system directories for ELF binaries:

   • /bin, /sbin, /usr/bin, /usr/sbin
   • /lib, /lib64, /usr/lib, /usr/lib64
   • /lib/modules (kernel modules)
   • Any additional paths you specify

3. Disassemble binaries: Using the Capstone disassembler, it analyzes each binary's executable sections to identify CPU instructions used

4. Check compliance: Each instruction is checked against the allowed instruction set. Any instruction that requires an extension not in the allowed set is flagged as a violation

5. Map to packages (Gentoo): Using equery belongs, violating binaries are mapped back to the Gentoo packages that installed them

Output Format

Text Format (Default)

================================================================================
SYSTEM BINARY INSTRUCTION SET COMPLIANCE REPORT
Generated: 2025-12-08 10:30:45
================================================================================

Total non-compliant binaries: 15
Forbidden extensions found: avx2, avx512f

================================================================================

[1] /usr/bin/example-binary
    Extensions: avx2
    Instructions:
      [avx2] vbroadcasti128, vextracti128, vpbroadcastb, vperm2i128, vpsllvd

[2] /usr/lib64/libexample.so
    Extensions: avx512f
    Instructions:
      [avx512f] vaddpd, valignd, vmovdqa32 ... and 12 more
...

JSON Format

{
  "generated": "2025-12-08T10:30:45",
  "total_violations": 15,
  "violations": [
    {
      "path": "/usr/bin/example",
      "compliant": false,
      "forbidden_instructions": [
        ["vbroadcasti128", "avx2"],
        ["vperm2i128", "avx2"]
      ],
      "forbidden_extensions": ["avx2"]
    }
  ]
}

Package Report (Gentoo)

================================================================================
GENTOO PACKAGE COMPLIANCE REPORT
Generated: 2025-12-08 10:30:45
================================================================================

Total packages with non-compliant binaries: 8

Package: sys-devel/gcc-13.2.1_p20240210
  Non-compliant files: 5
    /usr/bin/gcc
    /usr/bin/g++
    /usr/libexec/gcc/x86_64-pc-linux-gnu/13/cc1
    ...

Package: kde-apps/kolf-25.08.3
  Non-compliant files: 1
    /usr/bin/kolf

...

================================================================================
REBUILD COMMAND
================================================================================

emerge -av1 \
  =sys-devel/gcc-13.2.1_p20240210 \
  =kde-apps/kolf-25.08.3 \
  ...

================================================================================

This emerge command at the end is the most useful output - it's ready to copy and paste to rebuild all affected packages!

Use Cases

Validating Gentoo Build

After compiling your Gentoo system with specific CFLAGS, verify everything is compliant:

# Your make.conf has: CFLAGS="-march=x86-64-v2 -O2 -pipe"
checksysasm -m x86-64-v2 --package-report packages.txt

Preparing for CPU Downgrade

Before moving a system to older hardware:

# Check what would break on older CPU
checksysasm -m core2 -o issues.txt

Finding Packages to Rebuild

Identify which packages need rebuilding after changing CPU flags:

checksysasm -m x86-64-v2 --package-report rebuild.txt

# Check the end of rebuild.txt for the emerge command
tail -20 rebuild.txt

# Copy and run the emerge command to rebuild all affected packages
emerge -av1 =pkg1 =pkg2 =pkg3 ...

Regenerating Package Report

If you want to regenerate the package report without re-scanning (e.g., after packages are installed/updated):

# Use the regen-package-report script with your session file
regen-package-report ~/scan.json

This rebuilds the package mapping from your existing scan session, which is much faster than re-scanning the entire system.

If you only kept the generated packages.txt file, you can still print the rebuild command without re-scanning:

checksysasm --package-report-input ~/packages.txt

Performance

Scanning an entire Gentoo system typically takes:

• Small system (~2000 binaries): 2-5 minutes
• Large system (~10000 binaries): 10-20 minutes

Performance factors:

• Number of binaries
• Binary sizes
• CPU speed
• Number of parallel workers (-j flag)

Limitations

1. x86-64 only: Currently only supports x86-64 architecture
2. Static analysis: Only checks instructions present in binaries, not runtime code generation
3. False positives: Some instructions may be in code paths that never execute
4. Kernel modules: May require root access to scan some kernel module directories

Contributing

Contributions welcome! Areas for improvement:

• Support for ARM architectures
• More accurate instruction set detection
• Performance optimizations
• Better handling of kernel modules
• Support for other Linux distributions

License

GPL-3.0-or-later

Author

System Binary Checker project

See Also

• GCC x86 Options
• x86-64 psABI
• Gentoo Handbook
• Capstone Disassembler