Skip to content

Nagusamenotame/civilized-HyperTensor

 
 

Repository files navigation

IRIS-MD (3)

DOI License: CC BY 4.0

This repository is the civilized fork of HyperTensor: a fully reorganized, implementation-focused clone that keeps the runnable engineering surface, algorithmic scripts, examples, and build infrastructure from the original research.

Upstream & Provenance

What Is Civilized-HyperTensor?

This is a pragmatically curated, implementation-focused fork of the original HyperTensor repository. It retains the execution surfaces that matter while cutting archival research artifacts:

Core (Kept Intact):

  • Complete self-contained core: hypercore library, configurations, examples, verification scripts
  • All algorithmic scripts: 412 Python scripts implementing the research findings
  • Rich example collection: 150 example implementations (2 original + 43 domain applications + 105 core techniques)
  • Core benchmark data: Essential datasets (gsm8k, wikitext2, decision matrices)
  • Build infrastructure: Dockerfile, CMakeLists.txt, setup.py for modern deployment

Artifacts Removed (for Maintainability):

  • ❌ Paper submission archives and generated publication artifacts
  • ❌ Generated documentation trees and HTML/PDF outputs
  • ❌ TeX build cruft — (.bbl, .bcf, .run.xml, .out files)
  • ❌ Large auxiliary data — (odlyzko zeros, mmlu, lmfdb — available separately if needed)
  • ❌ EC2 orchestration — (cloud job coordination scripts)
  • ❌ Secondary tools — (hypersort, modelfiles — can be separate repos)

Result: a focused, maintainable checkout that keeps the runnable engineering and research implementation surfaces intact.

Repository Structure

civilized-HyperTensor/
├── runtime/                          # Core execution engine (96+ modules)
├── tests/                            # Test suite (13+ test files)
├── hypercore/                        # HyperTensor algorithmic core (14 modules)
├── lib/                              # Utility library
├── configs/                          # Configuration files (8+)
├── repro/                            # Reproduction guides and checklists
├── examples/                         # 150 example implementations
│   ├── original/                     # Original 2 examples from upstream
│   ├── domains/          # 43 new domain-specific applications (physics, finance, biotech, etc.)
│   └── core_techniques/              # 105 algorithmic technique demonstrations
├── scripts/                          # 412 research algorithm implementations
│   ├── gtc/                          # GTC (Global Transform Cache) suite
│   ├── ott/                          # OTT optimization suite
│   ├── analysis/                     # Analysis & statistical tools
│   └── [70+ more subdirectories]
├── data/                             # 74 benchmark datasets
├── benchmarks/                       # Benchmark architectures & configs
├── hypersort/                        # Sorting algorithm implementations
├── modelfiles/                       # Model architectures (11 files)
├── CMakeLists.txt                    # C++ build configuration
├── Dockerfile                        # Docker containerization
├── setup.py                          # Python package setup
├── pyproject.toml                    # Project metadata
└── LICENSE                           # MIT License

Repository Statistics

Category Count Notes
Example Python Files 150 2 original + 43 domains + 105 techniques
Algorithm/Script Files 412 Core implementations (orchestration removed)
Core Engineering Modules 96+ Runtime + hypercore
Test Files 13 Full test suite
Core Data Files 5 Active benchmarks
Configuration Files 8 Templates for all scenarios

Quick Start

Running Examples

# Run a specific example
python examples/core_techniques/1_jury_gate.py

# Run all examples in a domain
python examples/domains/domain_finance_portfolio_risk.py

# Batch test all examples (with timeout)
cd examples
for file in original/*.py core_techniques/*.py domains/*.py; do
  echo "Testing $file..."
  timeout 15 python "$file" > /dev/null 2>&1 && echo "" || echo ""
done

Using in Docker

# Build image
docker build -t hypertensor:latest .

# Run container
docker run -it hypertensor:latest

Installing Locally

# Install development mode
pip install -e .

# Or from setup.py
python setup.py install

CI/CD Pipeline

The repository includes automated testing via GitHub Actions:

What CI Tests:

  • C/C++ library build (Ubuntu 24.04): CMake configuration and compilation of libhypercore
  • macOS cross-platform build (macOS 14 arm64): Ensures portability across architectures
  • Python reproducibility: PyTorch-backed verification scripts, jury pipelines, and audit scripts run in CI
  • Python code health: Syntax validation and cross-platform path verification
  • Docker build: Ensures containerization works end-to-end

Build Boundary:

  • The default build path targets the self-contained hypercore library and reproducibility scripts
  • The broader runtime tree is retained as source reference and optional future work, not as a default build dependency

What Is Intentionally Excluded:

  • Generated paper PDFs and HTML artifacts are not stored in the repo
  • TeX build cruft stays out of version control
  • Legacy shell wrappers and orchestration scripts are removed
  • These outputs can still be regenerated from source where needed

For Production Use: The repo now keeps the execution path reproducible end-to-end without carrying around generated publication artifacts. That gives you a lean checkout with a full runnable research pipeline.

Key Features

Algorithms Implemented

  • GTC (Global Transform Cache): Adaptive caching, quantization, routing strategies
  • OTT (Optimized Tensor Transforms): Manifold optimization, geodesic interpolation
  • Jury Ensemble Methods: Bayesian aggregation, trajectory gates, domain specialists
  • Quantization & Compression: NF4, FP4, AWQ, calibrated quantization
  • Attention Mechanisms: RoPE variations, sliding windows, speculative decoding
  • Decomposition Methods: Tensor trains, SVD analysis, Grassmann interpolation
  • Online Learning: Adaptive rank, Oja's algorithm, natural gradients
  • Domain Applications: 43+ specialized implementations (agriculture, finance, aerospace, biomedical, etc.)

Example Domains Covered

Physics, Transportation, Supply Chain, Telecommunications, Manufacturing, Space Exploration, Materials Science, Quantum Computing, Real Estate, Recommendation Systems, Robotics, Social Science, Sports Analytics, Urban Planning, Insurance, Pharmacogenomics, Public Policy, Neuroscience, Medicine, Climate, Geophysics, Hydrology, Oceanography, Environmental Monitoring, Music/Audio, Biophysics, Biomedical Imaging, Cybersecurity, Drug Discovery, Education, Economics, Energy, Finance, Food Science, Geospatial, Law, Epidemiology, and more.

What Was Filtered Out

These items were deliberately removed to create a focused, usable fork rather than a complete archive:

  • ❌ Generated publication trees and HTML/PDF outputs
  • ❌ TeX build artifacts (*.bbl, *.bcf, *.run.xml, *.out)

Development Artifacts (not needed for reproduction):

  • ❌ Paper submission archives and peer-review working material
  • ❌ Archive metadata and generated publication bundles

Secondary Tools (can be separate repositories):

  • ❌ hypersort/ — Specialized sorting implementations
  • ❌ modelfiles/ — Ollama model configuration files

Large Auxiliary Data (available separately if needed):

  • ❌ data/odlyzko/zeros* — Large mathematical datasets
  • ❌ data/mmlu/ — Standard benchmark (can download)
  • ❌ data/lmfdb/ — External reference (can fetch)

Cloud Orchestration (environment-specific):

  • ❌ scripts/ec2_*/ — EC2 job coordination
  • ❌ scripts/campaign/ — Multi-GPU experiment runners
  • ❌ scripts/conversion_farm/ — Model format conversion automation

What Remains:

  • All core algorithms — 412 runnable Python implementations
  • All examples — 150 demonstrations across all domains
  • All runtime — 96+ tensor operation modules
  • Reproduction guides — Step-by-step reconstruction instructions

License

All code is licensed under MIT. See LICENSE for details.

Citation

If you use this repository in your research, please cite both the fork and the original:

@software{hypertensor_civilized,
  title = {civilized-HyperTensor: Reorganized Research-Grade Implementation},
  author = {Nagusamenotame},
  url = {https://github.com/Nagusamenotame/civilized-HyperTensor},
  year = {2026}
}

@software{hypertensor_original,
  title = {HyperTensor: Research Platform for Advanced AI Optimization},
  author = {Nagusamenotame},
  url = {https://github.com/Nagusamenotame/HyperTensor},
  doi = {10.5281/zenodo.20077378}
}

Validation & Testing

All core engineering surfaces have been validated:

  • ✅ Runtime modules: Functional parity with upstream
  • ✅ Examples: 147/150 runnable (3 torch-only examples require PyTorch)
  • ✅ Scripts: All core algorithm implementations present (some require optional dependencies)
  • ✅ Reproducibility: Core execution paths and benchmark data remain intact

Support & Contributing

This is an archival, implementation-focused repository. For upstream research questions, refer to the original HyperTensor repository. For bugs or improvements to this specific reorganization, open issues on this repository.

About

This repository is the civilized fork of HyperTensor: a fully reorganized, implementation-focused clone that keeps the runnable engineering surface, algorithmic scripts, examples, and build infrastructure from the original research.

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

 
 
 

Contributors

Languages

  • Python 67.0%
  • C 31.6%
  • Cuda 1.3%
  • Other 0.1%