Add environment control package by Am1n3e · Pull Request #11 · ServiceNow/webarena-verified

Am1n3e · 2026-02-02T11:50:26Z

Adds environment control package for managing WebArena Docker containers.

Added

environment_control package (packages/environment_control/)
- CLI interface (env-ctrl) for status checks, reset, and server operations
- REST API server with web dashboard for remote environment control
- Site-specific operations for gitlab, reddit, shopping, shopping_admin, and wikipedia
Python client library (webarena_verified.environments.env_ctrl_client)
- HttpClient for HTTP API access
- EnvCtrlDockerClient for docker exec access
Documentation (docs/environments/)
- New Environments section with overview
- Docker images documentation (moved from getting_started)
- Environment control documentation with architecture diagram
Integration tests

Dashboard

The env-ctrl server includes a web dashboard for manual environment management:

Usage

CLI (via docker exec)

docker exec <container> env-ctrl status
docker exec <container> env-ctrl serve --port 8877

HTTP API

curl http://localhost:8877/status
curl -X POST http://localhost:8877/reset

Python Client

from webarena_verified.environments.env_ctrl_client import HttpClient

client = HttpClient(base_url="http://localhost:8877")
result = client.status()

Introduces a new package for managing WebArena environment containers with: - CLI interface (env-ctrl) for status checks, reset, and server operations - REST API server with dashboard for remote environment control - Site-specific operations for gitlab, reddit, shopping, shopping_admin, and wikipedia - Docker and HTTP client implementations for the main webarena_verified package - Integration tests for CLI, server, and dashboard functionality Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

Am1n3e · 2026-02-02T11:52:32Z

+version = "0.1.0"
+description = "Environment control for WebArena Docker containers"
+readme = "README.md"
+requires-python = ">=3.9"


Add comment on why 3.9 (lowest supported version on the site docker files)

Add documentation explaining why the package exists (version mismatch, limited container access), how it works (CLI and HTTP interfaces), and include a mermaid diagram showing the architecture.

Copilot

Pull request overview

Adds a new environment_control package plus client bindings and CLI hooks in webarena_verified to control/inspect Docker-based environments via a lightweight REST server and dashboard.

Changes:

Introduces packages/environment_control (CLI, REST server + HTML dashboard, ops implementations for multiple sites).
Adds webarena_verified.environments.env_ctrl_client (stdlib HTTP client + docker-exec client) and exposes a new webarena-verified env ... CLI group.
Adds Docker-based integration tests and updates dependency/lock files (pytest-playwright, rerunfailures, local editable package).

Reviewed changes

Copilot reviewed 37 out of 40 changed files in this pull request and generated 13 comments.

Show a summary per file

File	Description
uv.lock	Adds editable local `environment-control` package and new dev test dependencies.
pyproject.toml	Moves dev tooling deps into `dependency-groups`, adds local uv source for `environment-control`, adds ruff per-file ignores for packages.
src/webarena_verified/types/environment.py	Adds `EnvCtrlResult` model for env-ctrl command results.
src/webarena_verified/types/init.py	Re-exports `EnvCtrlResult`.
src/webarena_verified/environments/env_ctrl_client/http_client.py	Adds stdlib-only HTTP client for env-ctrl REST API.
src/webarena_verified/environments/env_ctrl_client/docker_client.py	Adds docker-exec client for running env-ctrl commands inside containers.
src/webarena_verified/environments/env_ctrl_client/base.py	Adds protocol interface intended to unify env control clients.
src/webarena_verified/environments/env_ctrl_client/init.py	Exposes client APIs and keeps `EnvCtrlClient` alias.
src/webarena_verified/main.py	Adds `webarena-verified env {status,start,stop,init}` subcommands.
tests/integration/environment_control/conftest.py	Adds Docker container fixture spinning up the env-ctrl server for integration tests.
tests/integration/environment_control/test_server.py	Adds integration tests for REST endpoints and client utilities.
tests/integration/environment_control/test_dashboard.py	Adds HTML + Playwright-based dashboard integration tests.
tests/integration/environment_control/test_cli.py	Adds integration tests for env-ctrl CLI behavior.
packages/environment_control/uv.lock	Adds lockfile for the new package.
packages/environment_control/pyproject.toml	Defines the new package build, scripts, and ruff config.
packages/environment_control/README.md	Documents package usage.
packages/environment_control/environment_control/init.py	Exposes package public API and version.
packages/environment_control/environment_control/main.py	Enables `python -m environment_control ...`.
packages/environment_control/environment_control/cli.py	Implements env-ctrl CLI commands.
packages/environment_control/environment_control/env-ctrl	Adds a local dev wrapper script for env-ctrl.
packages/environment_control/environment_control/py.typed	Marks package as typed (PEP 561).
packages/environment_control/environment_control/_internal/init.py	Exposes internal config/logging utilities.
packages/environment_control/environment_control/_internal/config.py	Adds env-based server configuration.
packages/environment_control/environment_control/_internal/logging.py	Adds package-specific logging setup.
packages/environment_control/environment_control/server/init.py	Exposes server entrypoint.
packages/environment_control/environment_control/server/app.py	Implements REST API server + dashboard rendering.
packages/environment_control/environment_control/server/assets/dashboard.html	Adds HTML dashboard UI and JS actions.
packages/environment_control/environment_control/ops/init.py	Adds ops registry and exports ops surface.
packages/environment_control/environment_control/ops/base.py	Implements base ops API (init/start/stop/restart/health/cleanup).
packages/environment_control/environment_control/ops/types.py	Defines core types (Result/Health/ExecLog/OpsConfig).
packages/environment_control/environment_control/ops/mixins/init.py	Exposes mixins for ops implementations.
packages/environment_control/environment_control/ops/mixins/supervisor.py	Adds supervisord operations (status/start/stop, parsing, wait).
packages/environment_control/environment_control/ops/sites/init.py	Exports site-specific ops implementations.
packages/environment_control/environment_control/ops/sites/dummy.py	Adds dummy ops for tests/dev.
packages/environment_control/environment_control/ops/sites/gitlab.py	Adds GitLab ops implementation.
packages/environment_control/environment_control/ops/sites/map.py	Adds OpenStreetMap ops implementation.
packages/environment_control/environment_control/ops/sites/reddit.py	Adds Reddit/Postmill ops implementation.
packages/environment_control/environment_control/ops/sites/shopping.py	Adds Shopping ops implementation.
packages/environment_control/environment_control/ops/sites/shopping_admin.py	Adds Shopping Admin ops implementation.
packages/environment_control/environment_control/ops/sites/wikipedia.py	Adds Wikipedia ops implementation.

Comments suppressed due to low confidence (1)

src/webarena_verified/main.py:1207

env_command is annotated as int | None, but all code paths return an int. Consider tightening the return type to int to avoid implying None is a valid return (especially since the result is passed to sys.exit(...)).

        return 0
    except Exception as e:
        logger.error(f"Failed to trim network logs: {e}")

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

- Create dedicated docs/environments/ section with overview page - Move Docker images docs from getting_started/environments.md - Add environment_control documentation with architecture diagram - Add Python client usage examples and API reference - Enable mermaid diagram support in mkdocs - Add redirect from old environments URL

- Add Dashboard section explaining the web UI - Include screenshot showing status, action buttons, and console

- Fix Result.to_dict() mutating self.value dict - Fix HTML injection by escaping env_name and site_url in dashboard - Add /restart endpoint to server and CLI - Add /is_ready as alias for /status endpoint - Fix dashboard JS to use data.details.value schema - Add restart() method to HttpClient and DockerClient - Fix stage_files() resource leak and tar exit code check - Simplify EnvCtrlClientProtocol to common methods only - Add Docker availability check in test fixtures - Remove low-value dashboard HTML content tests - Remove unused env-ctrl wrapper script - Add Python 3.9 version comment in pyproject.toml - Fix README HTTP API examples to use correct endpoints

- Add docker pytest marker for Docker-dependent tests - Use dynamic port allocation via env_ctrl_server_port fixture - Use pytest rootpath instead of manual path calculation - Return EnvControlContainer dataclass with name, base_url, port - Update tests to use fixture base_url instead of hardcoded URLs - Fix type annotation for generator fixture - Apply ruff formatting

- Consolidate test_dashboard_interactive.py into test_dashboard.py - Reduce granular tests into fewer comprehensive tests - Use env_control_container.base_url instead of hardcoded URL

Run env-ctrl integration tests only when relevant files change: - packages/environment_control/** - tests/integration/environment_control/** - src/webarena_verified/environments/env_ctrl_client/** Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

Am1n3e requested a review from Copilot February 2, 2026 11:50

Copilot started reviewing on behalf of Am1n3e February 2, 2026 11:51 View session

Am1n3e commented Feb 2, 2026

View reviewed changes

Comment thread packages/environment_control/environment_control/env-ctrl Outdated

Am1n3e commented Feb 2, 2026

View reviewed changes

Comment thread packages/environment_control/README.md

Move contributing deps to dev dependency group

a7fdce8

Am1n3e force-pushed the pr/3-env-control branch from 8091e9e to a7fdce8 Compare February 2, 2026 11:53

Expand environment_control README with motivation and architecture

44c6878

Add documentation explaining why the package exists (version mismatch, limited container access), how it works (CLI and HTTP interfaces), and include a mermaid diagram showing the architecture.

Copilot AI reviewed Feb 2, 2026

View reviewed changes

Am1n3e added 4 commits February 2, 2026 13:24

Add dashboard documentation with screenshot

d87e36f

- Add Dashboard section explaining the web UI - Include screenshot showing status, action buttons, and console

Refactor environment control tests and add Playwright CI support

339436f

Fix ruff lint errors in gitlab.py and __main__.py

4e0c232

Am1n3e commented Feb 2, 2026

View reviewed changes

Comment thread tests/integration/environment_control/test_dashboard.py Outdated

Am1n3e and others added 8 commits February 2, 2026 13:45

Remove /is_ready endpoint alias, keep only /status

b4502b3

Rename invoke task collection from contrib to dev

a57853f

Merge Playwright interactive tests into test_dashboard

0a72e39

- Consolidate test_dashboard_interactive.py into test_dashboard.py - Reduce granular tests into fewer comprehensive tests - Use env_control_container.base_url instead of hardcoded URL

Remove running state from DummyOps for test order independence

34c8300

Add Playwright install to env-ctrl and copilot-setup workflows

b2bbf27

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

Am1n3e merged commit 605173a into main Feb 2, 2026
3 checks passed

Am1n3e deleted the pr/3-env-control branch February 2, 2026 13:21

Navigation Menu

Repository navigation

Add environment control package - #11

Add environment control package#11
Am1n3e merged 15 commits into
mainfrom
pr/3-env-control

Am1n3e commented Feb 2, 2026 •

edited

Loading

Uh oh!

Uh oh!

Am1n3e Feb 2, 2026

Uh oh!

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

Am1n3e commented Feb 2, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Added

Dashboard

Usage

CLI (via docker exec)

HTTP API

Python Client

Uh oh!

Uh oh!

Am1n3e Feb 2, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Am1n3e commented Feb 2, 2026 •

edited

Loading