Multi-language support for build, test, scan, and lint commands

[miohansen]

Feature Request

Extend build, test, scan, and lint commands to support multiple programming languages beyond Go.

Current State

The EAC tooling currently provides strong support for Go-based projects:

• Build: Go module compilation
• Test: Godog (BDD), Go test
• Scan: Go security scanning (gosec)
• Lint: golangci-lint

Limited support exists for:

• Markdown: markdownlint-cli2
• JavaScript: eslint (mentioned but not fully integrated)
• GitHub Actions: actionlint

Motivation

Use Cases

1. Polyglot Repositories: Projects using multiple languages (e.g., Go backend + TypeScript frontend)
2. Extension Ecosystem: Extensions written in various languages (Python, Node.js, Rust)
3. Data Projects: Python-based data pipelines, Jupyter notebooks
4. Infrastructure: Terraform, Ansible, Kubernetes YAML
5. Documentation: MDX, AsciiDoc, reStructuredText

Benefits

• Consistent tooling across all project languages
• Unified CD Model application regardless of language
• Single command interface: r2r eac build, r2r eac test, etc.
• Standardized output formats and evidence collection
• Multi-language monorepo support

Proposed Language Support

Priority 1: Core Languages

Python

• Build: pip install, poetry build, setup.py
• Test: pytest, unittest, behave (BDD)
• Lint: pylint, flake8, black, mypy
• Scan: bandit, safety, pip-audit
• Package: wheel, sdist

JavaScript/TypeScript

• Build: npm run build, tsc, webpack, vite
• Test: jest, mocha, vitest, playwright
• Lint: eslint, prettier, tslint
• Scan: npm audit, snyk, retire.js
• Package: npm pack, yarn pack

Java

• Build: maven, gradle
• Test: junit, testng, cucumber-jvm (BDD)
• Lint: checkstyle, pmd, spotbugs
• Scan: dependency-check, spotbugs, snyk
• Package: jar, war

C#/.NET

• Build: dotnet build, MSBuild
• Test: dotnet test, NUnit, xUnit, SpecFlow (BDD)
• Lint: StyleCop, dotnet format, Roslynator
• Scan: dotnet list package --vulnerable, security-scan
• Package: dotnet pack

Priority 2: Systems Languages

Rust

• Build: cargo build
• Test: cargo test, cargo bench
• Lint: cargo clippy, rustfmt
• Scan: cargo audit, cargo-deny
• Package: cargo package

C/C++

• Build: cmake, make, bazel
• Test: googletest, catch2, ctest
• Lint: clang-tidy, cpplint, cppcheck
• Scan: cppcheck, flawfinder, clang-analyzer
• Package: Platform-specific (deb, rpm, etc.)

Priority 3: Infrastructure & Config

Terraform

• Build: terraform plan
• Test: terratest, terraform validate
• Lint: tflint, terraform fmt
• Scan: tfsec, checkov, terrascan

Docker

• Build: docker build
• Test: Container structure tests
• Lint: hadolint (Dockerfile)
• Scan: trivy, grype, snyk container

Kubernetes

• Build: kustomize build, helm package
• Test: kubeval, conftest
• Lint: yamllint, kube-linter
• Scan: kubesec, checkov, polaris

Proposed Implementation

Phase 1: Language Detection & Configuration

Auto-detection based on files:

# .r2r/eac/repository.yml
modules:
  - moniker: backend-api
    type: service
    languages:
      - go: 
          version: ">=1.21"
      - python:
          version: ">=3.10"
          manager: poetry
      - typescript:
          version: ">=5.0"
          manager: npm

File-based detection:

• Python: requirements.txt, pyproject.toml, setup.py
• JavaScript/TypeScript: package.json, tsconfig.json
• Java: pom.xml, build.gradle
• C#: *.csproj, *.sln
• Rust: Cargo.toml

Phase 2: Unified Command Interface

# Build all languages in module
r2r eac build my-module

# Test all languages in module
r2r eac test my-module

# Lint all languages in module
r2r eac lint my-module

# Scan all languages in module
r2r eac scan my-module

# Language-specific execution
r2r eac build my-module --language python
r2r eac test my-module --language typescript

Phase 3: Tooling Configuration

Per-language tool configuration:

# .r2r/eac/lint-config.yml
go:
  tool: golangci-lint
  config: .golangci.yml
  
python:
  tools:
    - pylint:
        config: .pylintrc
    - black:
        config: pyproject.toml
    - mypy:
        config: mypy.ini

typescript:
  tools:
    - eslint:
        config: .eslintrc.json
    - prettier:
        config: .prettierrc

Phase 4: Output Standardization

Unified output format:

┌✓ Build ─────────────────────────────────────────────────┐
│  ✓ backend-api:go (1.2s)
│  ✓ backend-api:python (3.4s)
│  ✓ frontend:typescript (8.7s)
└─────────────────────────────────────────────────────────┘

Evidence collection:

• Test results: JUnit XML (all languages)
• Scan results: SARIF format (all scanners)
• Lint results: Checkstyle XML or JSON
• Build artifacts: Language-specific paths

Phase 5: Container Strategy

Option A: Native tools

• Install tools locally (current approach)
• Fast execution, no Docker overhead
• Requires dependency management

Option B: Container-based

• Run each language toolchain in Docker
• No local installation required
• Consistent across environments
• Slower due to container overhead

Hybrid approach:

• Default: Use local tools if available
• Fallback: Use containerized tools
• CI/CD: Always use containers for reproducibility

Configuration Examples

Multi-Language Module

# .r2r/eac/repository.yml
modules:
  - moniker: full-stack-app
    name: Full Stack Application
    type: service
    languages:
      - typescript:
          root: frontend/
          build: npm run build
          test: npm test
          lint: npm run lint
      - go:
          root: backend/
          build: go build
          test: go test ./...
          lint: golangci-lint run
      - python:
          root: ml-service/
          build: poetry build
          test: pytest
          lint: pylint src/
    files:
      source:
        - "frontend/**/*.{ts,tsx}"
        - "backend/**/*.go"
        - "ml-service/**/*.py"

Data Project (Python-focused)

modules:
  - moniker: data-pipeline
    name: Data Processing Pipeline
    type: databricks-pipeline
    languages:
      - python:
          version: ">=3.10"
          manager: poetry
          test_framework: pytest
          bdd_framework: behave
          notebooks: notebooks/**/*.ipynb
    files:
      source:
        - "src/**/*.py"
        - "notebooks/**/*.ipynb"
      tests:
        - "tests/**/*.py"
        - "tests/**/*.feature"

Testing Strategy

Test Suite Tags (Multi-Language)

Current: @L0, @L1, @L2, @L3, @L4

Additional language tags:

@L1 @go @unit
Scenario: Go unit test

@L1 @python @unit
Scenario: Python unit test

@L2 @typescript @integration @deps:docker
Scenario: TypeScript Docker integration test

@L3 @java @acceptance @deps:kubernetes
Scenario: Java acceptance test in Kubernetes

Security Scanning Strategy

Language-Specific Scanners

Dependency scanning:

• Go: govulncheck, nancy
• Python: safety, pip-audit, bandit
• JavaScript: npm audit, snyk, retire.js
• Java: dependency-check, snyk
• C#: dotnet list package --vulnerable
• Rust: cargo audit

SAST (Static Analysis):

• Go: gosec
• Python: bandit, semgrep
• JavaScript: eslint-plugin-security, semgrep
• Java: spotbugs, semgrep
• C#: Security analyzers, security-scan

Unified SARIF output:
All scanners convert to SARIF format for evidence collection and reporting.

Documentation Requirements

New Documentation

1. Multi-Language Guide: docs/how-to-guides/eac/commands/multi-language.md
2. Language Configuration: docs/reference/eac/configuration/languages.md
3. Tool Installation: docs/reference/eac/tools/language-tools.md
4. Examples: docs/tutorials/multi-language-project.md

Updated Documentation

• Build command: Add language-specific examples
• Test command: Show multi-language test execution
• Lint command: Document language-specific linters
• Scan command: Document language-specific scanners

Success Criteria

• Support Python, TypeScript, Java, C# (Priority 1)
• Auto-detect languages in modules
• Unified command interface for all languages
• Standardized output formats (JUnit XML, SARIF)
• Evidence collection works for all languages
• Container-based fallback for missing tools
• Documentation and examples for each language
• Migration guide for existing projects
• CI/CD pipeline examples for multi-language projects

Open Questions

1. Tool Installation: Auto-install missing tools or require manual setup?
2. Container Strategy: Default to native or containerized execution?
3. Configuration: Single config file or per-language configs?
4. Versioning: How to handle language/tool version conflicts in multi-language modules?
5. Performance: Parallel execution of language-specific tasks?
6. CI/CD: Pre-built container images with all language tools?

Related Issues

• Lint command fails due to missing linter dependencies #103 - Lint command missing linter dependencies (addresses Go/Markdown only)
• Test EAC commands for extension development workflow #98 - Test EAC commands in Docker (container-based execution)
• Complete ext-env-check extension as showcase for extension development #97 - ext-env-check extension completion (extension in different languages)

Impact

• Scope: Large - affects build, test, lint, scan commands
• Effort: ~3-6 months depending on language coverage
• Benefit: Opens EAC tooling to entire language ecosystem
• Risk: Complexity in managing multiple toolchains