Add Server-Side Repository Cloning Feature for Semaphore Pro

[Copilot]

This PR introduces comprehensive documentation and specifications for Server-Side Repository Cloning in Semaphore Pro, addressing environments where runners cannot directly access git servers due to network restrictions, corporate firewalls, or air-gapped deployments.

Problem Statement

In many enterprise environments, runners are deployed in restricted networks that cannot establish direct connections to external git servers. This prevents runners from cloning repositories, making CI/CD workflows impossible in these environments. The solution is to enable repository cloning through the Semaphore Server, which acts as a proxy with external access.

Solution Overview

The server-side repository cloning feature allows:

# Simple configuration to enable the feature
server_side_cloning:
  enabled: true
  cache_duration: "1h"
  compression: "gzip"

# Runners configured to use server-side cloning
export SEMAPHORE_RUNNER_CLONING_MODE=server_side
export SEMAPHORE_RUNNER_FALLBACK_DIRECT=true

Key Benefits

• 🔒 Network Isolation: Runners don't need direct internet access
• 🎯 Centralized Control: All repository access monitored through server
• ⚡ Performance: Intelligent caching reduces bandwidth usage
• 📋 Compliance: Enhanced audit trail and security controls
• 🔄 Flexibility: Automatic fallback to direct cloning when needed

Implementation Details

API Design

• RESTful API with OpenAPI 3.0 specification
• Asynchronous cloning with real-time status monitoring
• Secure token-based authentication
• Comprehensive error handling and retry mechanisms

Security Architecture

• End-to-end encryption (AES-256-GCM)
• Role-based access control (RBAC)
• Comprehensive audit logging
• Network isolation and firewall integration
• SOC 2 and ISO 27001 compliance ready

Deployment Support

• Docker Compose configurations for development
• Kubernetes manifests for production
• Enterprise security hardening examples
• Cloud provider specific configurations (AWS, Azure, GCP)

Files Added

• FEATURE_SERVER_SIDE_CLONING.md - Complete technical specification and architecture
• api-server-side-cloning.yaml - OpenAPI 3.0 specification for developers
• SECURITY.md - Security architecture, compliance, and threat modeling
• UI_MOCKUPS.md - User interface design and user experience workflows
• configurations/ - Ready-to-use deployment configurations
• examples/ - Practical implementation examples in Bash and Python

Use Cases

Corporate Networks

# Firewall rules: only server has outbound access
services:
  semaphore-server:
    environment:
      SEMAPHORE_SERVER_SIDE_CLONING: "true"
      SEMAPHORE_CLONE_ALLOWED_HOSTS: "github.com,gitlab.internal.corp"

Air-Gapped Environments

# Complete network isolation for runners
runner:
  environment:
    SEMAPHORE_RUNNER_CLONING_MODE: "server_side"
    SEMAPHORE_RUNNER_FALLBACK_DIRECT: "false"  # No external access

Performance Optimization

# Python client with intelligent caching
client = SemaphoreRepositoryClient(server_url, token)
result = client.clone_and_download(
    repository_url="https://github.com/large-repo.git",
    options=CloneOptions(cache_duration="2h", shallow=True)
)

This feature enables Semaphore Pro to serve enterprise customers with strict network security requirements while maintaining the ease of use and performance that makes Semaphore powerful.

Fixes #20.

---

💡 You can make Copilot smarter by setting up custom instructions, customizing its development environment and configuring Model Context Protocol (MCP) servers. Learn more Copilot coding agent tips in the docs.

[Copilot]

@fiftin 👋 This repository doesn't have Copilot instructions. With Copilot instructions, I can understand the repository better, work faster and produce higher quality PRs.

I can generate a .github/copilot-instructions.md file for you automatically. Click here to open a pre-filled issue and assign it to me. I'll write the instructions, and then tag you for review.