Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ Thank you for helping keep ReconScript safe, scoped, and respectful of consent.

- Only contribute features that preserve read-only reconnaissance. Never add intrusive or destructive functionality.
- All scans must remain single-target by default. CIDR support is opt-in via `ALLOW_CIDR=true`.
- Non-local targets require a signed scope manifest and consent verification. Never ship production keys; use the dev keys in `keys/` for local testing only.
- Non-local targets require a signed scope manifest and consent verification. Never ship production keys in the repository.
- Evidence level defaults to `low`. `high` evidence must remain explicitly gated behind signed consent.
- Do not expose the Flask UI publicly unless `ENABLE_PUBLIC_UI=true` **and** `ENABLE_RBAC=true` are set. The UI warns when public mode is active.

Expand Down
7 changes: 1 addition & 6 deletions Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -7,8 +7,6 @@ ENV PYTHONDONTWRITEBYTECODE=1 \

WORKDIR /app

ARG INCLUDE_DEV_KEYS=false

COPY requirements.txt ./

RUN python -m venv /opt/venv \
Expand All @@ -23,8 +21,6 @@ ENV PYTHONDONTWRITEBYTECODE=1 \

WORKDIR /app

ARG INCLUDE_DEV_KEYS=false

RUN apt-get update \
&& apt-get install --no-install-recommends -y \
libcairo2 \
Expand All @@ -42,8 +38,7 @@ RUN apt-get update \
COPY --from=builder /opt/venv /opt/venv
COPY . .

RUN if [ "$INCLUDE_DEV_KEYS" != "true" ]; then rm -f keys/dev_*; fi \
&& chown -R reconscript:reconscript /app
RUN chown -R reconscript:reconscript /app

USER reconscript

Expand Down
6 changes: 3 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -42,14 +42,14 @@ python -m pip install -r requirements-dev.txt
Before starting the Flask UI, generate deployment-specific secrets and point the application at them:

```bash
export FLASK_SECRET_KEY_FILE=/secure/path/flask_secret.key
export FLASK_SECRET_KEY="$(openssl rand -hex 32)"
export ADMIN_USER=security-admin
export ADMIN_PASSWORD='replace-with-strong-passphrase'
export CONSENT_PUBLIC_KEY_PATH=/secure/path/consent_ed25519.pub
export REPORT_SIGNING_KEY_PATH=/secure/path/report_ed25519.priv
python start.py
```
The launcher checks dependencies, loads environment variables from `.env` if present, and starts the Flask server on <http://127.0.0.1:5000>. Use `start.sh`, `start.bat`, or `start.ps1` for platform-specific wrappers. Set `ALLOW_DEV_SECRETS=true` only for local demos that intentionally reuse the sample keys in `keys/`.
The launcher checks dependencies, loads environment variables from `.env` if present, and starts the Flask server on <http://127.0.0.1:5000>. Use `start.sh`, `start.bat`, or `start.ps1` for platform-specific wrappers.

### Run a CLI Scan
```bash
Expand All @@ -64,7 +64,7 @@ A Docker Compose definition is provided for isolated demonstrations:
docker compose up --build
```

Mount the `results/` directory when running containers so generated artefacts persist outside the container lifecycle. Override the required secrets via environment variables or secrets managers at runtime; the container image omits the developer keys unless built with `--build-arg INCLUDE_DEV_KEYS=true`.
Mount the `results/` directory when running containers so generated artefacts persist outside the container lifecycle. Override the required secrets via environment variables or secrets managers at runtime.

### Observability
ReconScript exposes Prometheus-compatible metrics at `/metrics` and a readiness probe at `/healthz`. Scrape the metrics endpoint to monitor scan durations, completion counts, and open-port histograms.
Expand Down
12 changes: 5 additions & 7 deletions docs/AUDIT_REMEDIATION.md
Original file line number Diff line number Diff line change
Expand Up @@ -30,11 +30,10 @@ Each finding from the previous engineering audit is addressed below with the imp
- **CI/Docs/Infra:** README references the matrix workflow; caching drastically shortens rerun latency.
- **Risk reduction:** Speeds up reviews and reduces drift risk between lint/test definitions.

## F-005 – Default UI credentials and Flask secret shipped in repo
- **Root cause:** `_load_user_credentials` and `_load_secret_key` fell back to `admin/changeme` and `keys/dev_flask_secret.key`.
- **Code-level fix:** Require `ADMIN_USER`, `ADMIN_PASSWORD`, and `FLASK_SECRET_KEY_FILE` to be set; block developer secrets unless `ALLOW_DEV_SECRETS=true`.
- **Root cause:** `_load_user_credentials` and `_load_secret_key` fell back to `admin/changeme` and the bundled developer Flask secret.
- **Code-level fix:** Require `ADMIN_USER`, `ADMIN_PASSWORD`, and `FLASK_SECRET_KEY` to be provided; reject developer secrets entirely.
- **Tests added:** `tests/conftest.py` fixture seeds secure test-only values to exercise the stricter loaders.
- **CI/Docs/Infra:** README/SECURITY guide operators through secret provisioning; Docker build removes developer keys by default.
- **CI/Docs/Infra:** README/SECURITY guide operators through secret provisioning; developer secrets are no longer shipped.
- **Risk reduction:** Eliminates trivial takeover paths by forcing strong credentials and unique Flask secrets.

## F-006 – Consent/report signing keys defaulted to bundled dev keys
Expand Down Expand Up @@ -79,9 +78,8 @@ Each finding from the previous engineering audit is addressed below with the imp
- **CI/Docs/Infra:** Accessibility improvements captured in roadmap milestones.
- **Risk reduction:** Improves UX for keyboard and screen-reader users, aligning with enterprise accessibility standards.

## F-012 – Container image bundled secrets and lacked healthcheck
- **Root cause:** Dockerfile copied `keys/` unconditionally and provided no runtime health signal.
- **Code-level fix:** Added `INCLUDE_DEV_KEYS` build arg to strip sample keys by default and configured a Python-based `HEALTHCHECK` hitting `/healthz`.
- **Root cause:** Dockerfile copied developer keys unconditionally and provided no runtime health signal.
- **Code-level fix:** Removed developer keys from the repository entirely and configured a Python-based `HEALTHCHECK` hitting `/healthz`.
- **Tests added:** Covered indirectly via CLI dry-run tests; integration tests planned for metrics and health endpoints.
- **CI/Docs/Infra:** README instructs operators to pass secrets at runtime; Dockerfile change improves readiness reporting.
- **Risk reduction:** Prevents accidental secret leakage in images and enables orchestrators to detect unhealthy containers.
Expand Down
2 changes: 1 addition & 1 deletion docs/HELP.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@

## Configuration Tips
- Copy `.env.example` to `.env` to override defaults.
- Review the `keys/` directory and replace development keys before production usage.
- Provision secrets using environment variables (see README) before production usage.
- Use `requirements.txt` for runtime installs and `requirements-dev.txt` for local development tooling.

## Troubleshooting
Expand Down
5 changes: 2 additions & 3 deletions docs/SECURITY.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,12 +9,11 @@ This project is designed for safe, educational, and authorized use only.
Report findings respectfully by opening a private issue or contacting the maintainer listed in the README.

## Hardening Checklist
- Generate production-only keys and set the following environment variables before starting any service:
- `FLASK_SECRET_KEY_FILE` → path to a randomly generated 32-byte secret.
- Generate production-only secrets and set the following environment variables before starting any service:
- `FLASK_SECRET_KEY` → randomly generated 32-byte secret string.
- `ADMIN_USER` / `ADMIN_PASSWORD` → non-default credentials (password ≥ 12 characters).
- `CONSENT_PUBLIC_KEY_PATH` → consent verification key.
- `REPORT_SIGNING_KEY_PATH` → report signing key used by `--sign-report` or UI downloads.
- Leave `ALLOW_DEV_SECRETS` unset (default) in production; setting it to `true` is only for ephemeral demos that explicitly use the sample keys shipped under `keys/`.
- Run scans only against systems where you have explicit written permission.
- Keep dependencies updated using `requirements.lock` or `pyproject.toml` and run `pip-audit`/`bandit` regularly.
- Enable HTTPS termination, reverse-proxy authentication, and network segmentation before exposing the UI publicly.
Expand Down
3 changes: 2 additions & 1 deletion install_dependencies.py
Original file line number Diff line number Diff line change
Expand Up @@ -8,8 +8,9 @@
import importlib
import subprocess
import sys
from collections.abc import Iterable
from pathlib import Path
from typing import Dict, Iterable
from typing import Dict

ROOT = Path(__file__).resolve().parent
REQUIREMENTS_FILE = ROOT / "requirements.txt"
Expand Down
1 change: 0 additions & 1 deletion keys/dev_ed25519.priv

This file was deleted.

2 changes: 0 additions & 2 deletions keys/dev_ed25519.pub

This file was deleted.

1 change: 0 additions & 1 deletion keys/dev_flask_secret.key

This file was deleted.

5 changes: 2 additions & 3 deletions nacl/__init__.py
Original file line number Diff line number Diff line change
@@ -1,14 +1,13 @@
"""Minimal ed25519-compatible subset of the PyNaCl API used for tests."""

from .exceptions import BadSignatureError, CryptoError
from .signing import SignedMessage, SigningKey, VerifyKey

# Expose the ``exceptions`` module at the package level to match the
# ``pynacl`` import style used by the application code (``from nacl import
# exceptions``). Our lightweight shim previously only re-exported the
# classes, which meant importing the module itself failed with
# ``ImportError`` when the real dependency was not installed.
from . import exceptions as exceptions # noqa: E402 (re-export for compatibility)
from .exceptions import BadSignatureError, CryptoError
from .signing import SignedMessage, SigningKey, VerifyKey

__all__ = [
"BadSignatureError",
Expand Down
1 change: 0 additions & 1 deletion recon_script.py
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,5 @@

from reconscript.cli import main


if __name__ == "__main__": # pragma: no cover - CLI bootstrap
raise SystemExit(main())
1 change: 0 additions & 1 deletion reconscript/__main__.py
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,5 @@

from .cli import main


if __name__ == "__main__": # pragma: no cover - CLI bootstrap
raise SystemExit(main())
3 changes: 2 additions & 1 deletion reconscript/cli.py
Original file line number Diff line number Diff line change
Expand Up @@ -5,8 +5,9 @@
import argparse
import logging
import sys
from collections.abc import Iterable
from pathlib import Path
from typing import Iterable, Optional
from typing import Optional

from . import __version__
from .consent import ConsentError, ConsentManifest, load_manifest, validate_manifest
Expand Down
8 changes: 6 additions & 2 deletions reconscript/consent.py
Original file line number Diff line number Diff line change
Expand Up @@ -72,7 +72,9 @@ def _allow_dev_secrets() -> bool:
def _guard_key_path(path: Path, *, env_var: str) -> Path:
resolved = path.expanduser().resolve()
if not resolved.exists():
raise ConsentError(f"{env_var} must point to an existing file (got {resolved}).")
raise ConsentError(
f"{env_var} must point to an existing file (got {resolved})."
)
try:
if DEV_KEYS_DIR in resolved.parents and not _allow_dev_secrets():
raise ConsentError(
Expand Down Expand Up @@ -108,7 +110,9 @@ def _resolve_key_path(provided: Optional[Path], env_var: str) -> Path:
return _guard_key_path(provided, env_var=env_var)
env_value = os.environ.get(env_var)
if not env_value:
raise ConsentError(f"{env_var} must be set to the path of the authorised signing key.")
raise ConsentError(
f"{env_var} must be set to the path of the authorised signing key."
)
return _guard_key_path(Path(env_value), env_var=env_var)


Expand Down
79 changes: 14 additions & 65 deletions reconscript/core.py
Original file line number Diff line number Diff line change
Expand Up @@ -3,10 +3,9 @@
from __future__ import annotations

import logging
import os
import time
from collections.abc import Callable, Iterable, Sequence
from datetime import datetime, timezone
from typing import Iterable, Optional, Sequence

from . import __version__
from .consent import ConsentManifest
Expand All @@ -16,7 +15,6 @@
record_scan_started,
)
from .report import embed_runtime_metadata
from .scope import ScopeValidation, ScopeError, ensure_within_allowlist, validate_target
from .scanner import (
DEFAULT_PORTS,
REDACTION_KEYS,
Expand All @@ -33,6 +31,7 @@
validate_port_list,
)
from .scanner.throttle import TokenBucket
from .scope import ScopeError, ScopeValidation, ensure_within_allowlist, validate_target

LOGGER = logging.getLogger(__name__)
REPORT_LOGGER = logging.getLogger("reconscript.report")
Expand All @@ -44,7 +43,7 @@ class ReconError(RuntimeError):
"""Raised when a scan cannot proceed."""


def _determine_redactions(extra: Optional[Iterable[str]]) -> set[str]:
def _determine_redactions(extra: Iterable[str] | None) -> set[str]:
redactions = set(REDACTION_KEYS)
for item in extra or []:
redactions.add(str(item).lower())
Expand Down Expand Up @@ -75,7 +74,7 @@ def _validate_consent(
return manifest


def _hostname_for_requests(scope: ScopeValidation, override: Optional[str]) -> str:
def _hostname_for_requests(scope: ScopeValidation, override: str | None) -> str:
if override:
return override
if scope.kind == "hostname":
Expand All @@ -86,15 +85,15 @@ def _hostname_for_requests(scope: ScopeValidation, override: Optional[str]) -> s
def run_recon(
*,
target: str,
hostname: Optional[str] = None,
ports: Optional[Sequence[int]] = None,
expected_ip: Optional[str] = None,
hostname: str | None = None,
ports: Sequence[int] | None = None,
expected_ip: str | None = None,
enable_ipv6: bool = False,
dry_run: bool = False,
evidence_level: str = "low",
consent_manifest: ConsentManifest | None = None,
extra_redactions: Optional[Iterable[str]] = None,
progress_callback: Optional[callable] = None,
extra_redactions: Iterable[str] | None = None,
progress_callback: Callable[[str, float], None] | None = None,
) -> dict[str, object]:
"""Execute the ReconScript workflow with safety controls."""

Expand Down Expand Up @@ -157,7 +156,9 @@ def run_recon(
"findings": [],
}
)
embed_runtime_metadata(report, started_at, completed_at=started_at, duration=0.0)
embed_runtime_metadata(
report, started_at, completed_at=started_at, duration=0.0
)
record_scan_completed(scope.target, 0.0, 0)
REPORT_LOGGER.info(serialize_results(report))
return report
Expand All @@ -170,7 +171,6 @@ def run_recon(
evidence_level=evidence_level,
redaction_keys=redactions,
)
<<<<<<< HEAD

http_host = _hostname_for_requests(scope, hostname)
started_clock = time.perf_counter()
Expand Down Expand Up @@ -225,13 +225,10 @@ def run_recon(

completed_at = datetime.now(timezone.utc)
duration = time.perf_counter() - started_clock
embed_runtime_metadata(report, started_at, completed_at=completed_at, duration=duration)
record_scan_completed(scope.target, duration, len(open_ports))
=======
embed_runtime_metadata(
report, started_at, completed_at=started_at, duration=0.0
report, started_at, completed_at=completed_at, duration=duration
)
>>>>>>> 74be2f0 (style: fix reporters.py syntax and apply Black formatting)
record_scan_completed(scope.target, duration, len(open_ports))
REPORT_LOGGER.info(serialize_results(report))

if progress_callback:
Expand All @@ -252,54 +249,6 @@ def run_recon(
record_scan_failed(target, failure_reason)


<<<<<<< HEAD
=======
http_host = _hostname_for_requests(scope, hostname)

started_clock = time.perf_counter()

if progress_callback:
progress_callback("Starting TCP connect scan", 0.1)
open_ports = tcp_connect_scan(config, bucket)
report["open_ports"] = open_ports

http_results: dict[int, dict[str, object]] = {}
if open_ports:
if progress_callback:
progress_callback("Collecting HTTP metadata", 0.4)
http_results = http_probe_services(config, http_host, open_ports)
report["http_checks"] = http_results

tls_details = None
if any(port in (443, 8443) for port in open_ports):
if progress_callback:
progress_callback("Retrieving TLS certificates", 0.6)
tls_port = 443 if 443 in open_ports else 8443
tls_details = fetch_tls_certificate(config, tls_port)
report["tls_cert"] = tls_details

if progress_callback:
progress_callback("Fetching robots.txt", 0.75)
report["robots"] = fetch_robots(config, http_host)

if progress_callback:
progress_callback("Generating findings", 0.9)
report["findings"] = generate_findings(http_results)

completed_at = datetime.now(timezone.utc)
duration = time.perf_counter() - started_clock
embed_runtime_metadata(
report, started_at, completed_at=completed_at, duration=duration
)
REPORT_LOGGER.info(serialize_results(report))

if progress_callback:
progress_callback("Reconnaissance complete", 1.0)

return report
>>>>>>> 74be2f0 (style: fix reporters.py syntax and apply Black formatting)


__all__ = [
"run_recon",
"ReconError",
Expand Down
3 changes: 2 additions & 1 deletion reconscript/logging.py
Original file line number Diff line number Diff line change
Expand Up @@ -5,9 +5,10 @@
import json
import logging
import os
from collections.abc import Iterable
from logging.handlers import RotatingFileHandler
from pathlib import Path
from typing import Iterable, Optional
from typing import Optional

DEFAULT_MAX_BYTES = int(os.environ.get("LOG_MAX_BYTES", str(10 * 1024 * 1024)))
DEFAULT_BACKUP_COUNT = int(os.environ.get("LOG_BACKUP_COUNT", "5"))
Expand Down
Loading
Loading