diff --git a/documentation/architecture.md b/documentation/architecture.md new file mode 100644 index 00000000..ff680ff4 --- /dev/null +++ b/documentation/architecture.md @@ -0,0 +1,181 @@ + + +# Architecture + +This document describes how ERM is structured and how its layers +interact at runtime. + +--- + +## Entry Points + +- `main.py` — Thin entrypoint. Calls `run()` from `erm.py`. +- `erm.py` — Defines the `Bot` class, initializes all database + collections, loads cogs, and starts the task scheduler. + +--- + +## Directory Structure + +``` +erm/ +├── cogs/ # Discord command groups (slash commands) +├── datamodels/ # MongoDB collection wrappers and typed data classes +├── events/ # Discord gateway event handlers +├── tasks/ # Background loops (discord.ext.tasks) +├── ui/ # Discord UI components (Views, Selects, Modals) +├── utils/ # Shared utilities, API clients, and helpers +├── menus.py # Large collection of interactive menus and flows +├── helpers.py # Test helpers and mock infrastructure +├── erm.py # Bot class and startup logic +└── main.py # Entrypoint +``` + +--- + +## Cogs + +Cogs live in `cogs/` and contain the bot's slash command groups. +They are loaded automatically on startup via `pkgutil.iter_modules`. + +| Cog | Responsibility | +|-----|----------------| +| `Actions` | Automated server actions triggered by conditions | +| `ActivityMonitoring` | Staff activity monitoring and reporting | +| `ActivityNotices` | Leave of absence and reduced activity requests | +| `Configuration` | Server setup and settings management | +| `CustomCommands` | User-defined slash commands per server | +| `ERLC` | ER:LC in-game integration (bans, kicks, player lookup) | +| `GameLogging` | In-game event log forwarding to Discord channels | +| `Infractions` | Staff infraction tracking and management | +| `Jishaku` | Developer/debug REPL (conditionally loaded) | +| `MC` | MapleCounty game integration | +| `OAuth2` | Discord OAuth2 flow handling | +| `Privacy` | User data and consent management | +| `Punishments` | Player punishment logging (warns, bans, kicks, BOLOs) | +| `Reminders` | Scheduled reminder creation and delivery | +| `Search` | Cross-guild search for punishments and shift records | +| `ShiftLogging` | Staff shift start, end, break, and quota tracking | +| `StaffConduct` | Staff conduct reviews and documentation | +| `Utility` | General-purpose utility commands | + +--- + +## Events + +Event handlers live in `events/` and are named after the Discord +gateway event they handle. Each file is a standalone module registered +on the bot at startup. + +Notable events: + +- `on_message` — Handles custom command dispatch and message-based triggers +- `on_shift_start` / `on_shift_end` / `on_shift_edit` / `on_shift_void` — Custom + events dispatched internally when shift state changes +- `on_punishment` / `on_punishment_delete` — Custom events for punishment lifecycle +- `on_infraction_create` / `on_infraction_revoke` — Custom events for infraction lifecycle +- `on_loa_accept` / `on_loa_deny` — Custom events for LOA request resolution +- `on_member_remove` — Cleans up shift state when a member leaves + +--- + +## Tasks + +Background tasks live in `tasks/` and use `discord.ext.tasks.loop`. +They are registered in `utils/task_loader.py` and started with a +staggered 30-second delay between each to avoid startup load spikes. + +| Task | Interval | Purpose | +|------|----------|---------| +| `check_reminders` | Periodic | Delivers due reminders to users | +| `check_loa` | Periodic | Expires LOA requests past their end date | +| `iterate_ics` | Periodic | Processes integration command storage | +| `iterate_prc_logs` | Periodic | Polls PRC API and forwards logs to Discord | +| `statistics_check` | Periodic | Updates analytics records | +| `tempban_checks` | Periodic | Lifts expired temporary bans | +| `check_whitelisted_car` | Periodic | Enforces whitelisted vehicle rules in-game | +| `change_status` | Periodic | Rotates the bot's Discord presence status | +| `process_scheduled_pms` | Periodic | Delivers scheduled direct messages | +| `sync_weather` | Periodic | Syncs real-world weather to ERLC in-game weather | +| `iterate_conditions` | Periodic | Evaluates server conditions and triggers actions | +| `check_infractions` | Hourly | Reverts expired temporary role changes from infractions | +| `prc_automations` | Periodic | Runs configured PRC automation rules | +| `mc_discord_checks` | Periodic | Runs MapleCounty Discord integration checks | + +`check_reminders` and `iterate_conditions` are controlled by +`REMINDERS_ENABLED` and `ACTIONS_ENABLED` respectively and will not +start if those are set to `FALSE`. + +`change_status` does not run when `ENVIRONMENT=CUSTOM`. + +--- + +## Data Layer + +### `datamodels/` + +Each file in `datamodels/` wraps a MongoDB collection as a typed +class. They inherit from `utils/mongo.py`'s `Document` class, which +provides standard async CRUD operations (`find_by_id`, `insert`, +`update_by_id`, `delete_by_id`, etc.). + +All datamodel instances are attached to the `Bot` object at startup +and accessed via `bot.db.` throughout the codebase. + +### `utils/mongo.py` + +Provides the `Document` base class used by all datamodels. Wraps +`pymongo`'s async client with convenience methods. + +--- + +## Utils + +| Module | Purpose | +|--------|---------| +| `api.py` | FastAPI application exposing internal HTTP endpoints | +| `prc_api.py` | Client for the ER:LC PRC API with typed response models | +| `mc_api.py` | Client for the MapleCounty API | +| `utils.py` | General shared helpers used across cogs and tasks | +| `autocompletes.py` | Discord slash command autocomplete handlers | +| `conditions.py` | Condition evaluation logic used by the Actions system | +| `constants.py` | Base server config schema, colour constants, condition | +| | and weather code mappings | +| `emojis.py` | `EmojiController` for resolving custom emoji by name | +| `paginators.py` | Reusable paginated embed components | +| `bloxlink.py` | Bloxlink API client for Roblox username resolution | +| `accounts.py` | Staff account lookup helpers | +| `AI.py` | AI API client wrapper | +| `hot_reload.py` | Development cog reloading utility | +| `log_tracker.py` | `LogTracker` for structured internal logging | +| `timestamp.py` | Time delta formatting helpers | +| `username_check.py` | `UsernameChecker` for flagging suspicious Roblox usernames | +| `task_loader.py` | Registers and starts all background tasks | +| `basedataclass.py` | `BaseDataClass` used by PRC and MC API response models | + +--- + +## Internal API + +`utils/api.py` runs a FastAPI application alongside the bot using +`uvicorn`. It exposes HTTP endpoints used by the ERM website and +panel. It is not required for basic bot operation and can be ignored +for most development work. + +--- + +## Menus + + + + +`menus.py` is a large single file containing the majority of the +bot's interactive Discord UI flows — multi-step modals, confirmation +prompts, paginated views, and context-sensitive menus. When adding +new interactive flows, check `menus.py` first for existing patterns +to follow or reuse. + +Do not use this file for any new views, instead, please create a new file +in the `ui` folder and reference that. `menus.py` will be removed in a later +version of ERM. + diff --git a/documentation/coding-assistants.md b/documentation/coding-assistants.md new file mode 100644 index 00000000..bb0c8942 --- /dev/null +++ b/documentation/coding-assistants.md @@ -0,0 +1,45 @@ + + +# AI Coding Assistants + +This document provides guidance for AI tools and developers using AI +assistance when contributing to ERM. + +AI tools helping with ERM development should follow the standard +development process. +--- + +## Licensing and Legal Requirements + +All contributions must comply with ERM's (Emergency Reponse Management) licensing requirements: + +- All code must be compatible with the Attribution-NonCommercial-ShareAlike (CC BY-NC-SA) license +- Use appropriate SPDX license identifiers + +--- + +## Signed-off-by and Developer Certificate of Origin + +AI agents **MUST NOT** add Signed-off-by tags. Only humans can legally +certify the Developer Certificate of Origin (DCO). The human submitter +is responsible for: + +- Reviewing all AI-generated code +- Ensuring compliance with licensing requirements +- Adding their own Signed-off-by tag to certify the DCO (Developer Certificate of Origin) +- Taking full responsibility for the contribution + +--- + +## Attribution + +When AI tools contribute to ERM development, proper attribution +helps track the evolving role of AI in the development process. +Contributions should include an Assisted-by tag in the following format: + + +*Assisted-by: AGENT_NAME:MODEL_VERSION* + +Example: *Assisted-by: Claude Haiku 4.5* + +This document was taken as a example from https://github.com/torvalds/linux diff --git a/documentation/contributing.md b/documentation/contributing.md new file mode 100644 index 00000000..12ef9bb8 --- /dev/null +++ b/documentation/contributing.md @@ -0,0 +1,98 @@ + + +# Contributing + +This document covers the process for contributing to ERM. + +--- + +## Before You Start + +- Read the [license](../LICENSE). All contributions must be compatible + with the Attribution-NonCommercial-ShareAlike (CC BY-NC-SA) license. +- If you are using an AI coding assistant, read + [coding-assistants.md](coding-assistants.md) before submitting. +- Set up your local environment using [setup.md](setup.md). + +--- + +## Workflow + +1. Fork the repository and create a branch from `Development`. +2. Make your changes. +3. Run the test suite locally with `pytest` before pushing. +4. Open a pull request against `Development` with a clear description of + what changed and why. + +--- + +## Branch Naming + +Use a short, descriptive name prefixed with the type of change: + +- `fix/` — Bug fixes +- `feat/` — New features +- `refactor/` — Code restructuring without behavior changes +- `docs/` — Documentation changes only +- `chore/` — Dependency updates, CI changes, and similar maintenance + +Example: `fix/shift-end-break-overlap` + +--- + +## Commit Messages + +Write commits in the imperative mood and keep the subject line +under 72 characters. + +``` +Add quota enforcement to shift types +Fix break duration calculation on overnight shifts +Remove unused LOA reminder fallback +``` + +Do not use `Signed-off-by` tags unless you are a human contributor +certifying the Developer Certificate of Origin (DCO). See +[coding-assistants.md](coding-assistants.md) for details on AI attribution. + +--- + +## Code Style + +- Python 3.12. Match the style of the surrounding code. +- All database calls must be `async`. Use `await` throughout — do not + use blocking pymongo calls. +- Ensure the use of the `Document` class whenever accessing different collections. +- Do not create Documents in commands, define them on the instance method `self` in the setup hook. +- Use `discord.ext.tasks.loop` for new background tasks. +- New interactive Discord flows must have their own UI file in `ui/`. Do not add new enties to `menus.py`. +- Use `utils/constants.py` for shared colour values (`BLANK_COLOR`, + `GREEN_COLOR`, `RED_COLOR`) rather than hardcoding hex values. +- Use `decouple.config()` for all environment variable access. +- Do not use inline ENV comments + +--- + +## Tests + +The test suite lives in `test_erm.py`. `helpers.py` provides mock +infrastructure for the `Bot` class and Discord objects. + +Run all tests: + +```bash +pytest +``` + +CI runs `pytest` and `flake8` (with `--exit-zero`) on every push +via the GitHub Actions workflow in `.github/workflows/app.yaml`. + +--- + +## Pull Request Checklist + +- [ ] `pytest` passes locally +- [ ] No new blocking `flake8` errors introduced +- [ ] New environment variables added to `.env.template` with a comment +- [ ] New background tasks registered in `utils/task_loader.py` +- [ ] DCO certified with a `Signed-off-by` tag (human contributors only) diff --git a/documentation/database-schema.md b/documentation/database-schema.md new file mode 100644 index 00000000..b900d44b --- /dev/null +++ b/documentation/database-schema.md @@ -0,0 +1,234 @@ + + +# Database Schema + +This document describes the MongoDB collections used by ERM and +their primary fields. All collections are accessed via their +corresponding datamodel class in `datamodels/`. + +The default database name is `erm`. This can be overridden with +the `DB_NAME` environment variable. + +--- + +## Collections + +### `settings` + +Per-guild bot configuration. One document per Discord server. + +Key fields: + +- `_id` — Discord guild ID +- `antiping` — Antiping configuration (enabled, protected roles, bypass roles) +- `staff_management` — Staff roles, management roles, LOA role, RA role, channel +- `punishments` — Punishment log channels (general, kick, ban, BOLO) +- `shift_management` — Shift role, quota, nickname prefix, maximum staff, role quotas +- `shift_types` — List of custom shift type definitions +- `game_security` — Game security webhook and channel configuration +- `game_logging` — Channels for message logs, STS logs, and priority logs +- `ERLC` — ER:LC integration settings (player log channel, kill log channel, + RDM channel, automatic shifts, elevation requirements) +- `customisation` — Custom prefix + +The base schema is defined in `utils/constants.py` as `base_configuration`. + +--- + +### `shifts` + +Individual staff shift records. + +Key fields: + +- `_id` — Shift document ID +- `UserID` — Discord user ID +- `Guild` — Discord guild ID +- `Type` — Shift type name +- `StartEpoch` — Unix timestamp of shift start +- `EndEpoch` — Unix timestamp of shift end (absent if shift is active) +- `Breaks` — List of break objects, each with `StartEpoch` and `EndEpoch` +- `Moderations` — List of moderation actions taken during the shift +- `AddedTime` / `RemovedTime` — Manual time adjustments in seconds + +--- + +### `warnings` + +Player punishment records (warnings, kicks, bans, BOLOs). + +Key fields: + +- `_id` — Document ID +- `UserID` — Roblox user ID of the punished player +- `Username` — Roblox username at time of punishment +- `Type` — Punishment type name +- `Reason` — Reason string +- `Moderator` — Discord user ID of the issuing moderator +- `Guild` — Discord guild ID +- `Epoch` — Unix timestamp of the punishment +- `ID` — Short human-readable punishment ID + +--- + +### `infractions` + +Staff member infraction records. + +Key fields: + +- `_id` — Document ID +- `user_id` — Discord user ID of the staff member +- `guild_id` — Discord guild ID +- `type` — Infraction type name +- `reason` — Reason string +- `moderator_id` — Discord user ID of the issuing moderator +- `epoch` — Unix timestamp +- `temp_roles_expire_at` — Unix timestamp for temporary role expiry (optional) + +--- + +### `activity_notices` + +Leave of absence and reduced activity requests. + +Key fields: + +- `_id` — Document ID +- `UserID` — Discord user ID +- `Guild` — Discord guild ID +- `Type` — `"LOA"` or `"RA"` +- `StartEpoch` / `EndEpoch` — Duration of the notice +- `Reason` — Reason string +- `Status` — `"pending"`, `"accepted"`, or `"denied"` +- `ReviewedBy` — Discord user ID of the reviewer + +--- + +### `custom_commands` + +User-defined slash commands per guild. + +Key fields: + +- `_id` — Document ID +- `Guild` — Discord guild ID +- `Name` — Command name +- `Response` — Response content +- `Roles` — Role restrictions for command use + +--- + +### `reminders` + +Scheduled reminder records. + +Key fields: + +- `_id` — Document ID +- `UserID` — Discord user ID +- `Guild` — Discord guild ID +- `Message` — Reminder content +- `Epoch` — Unix timestamp at which to deliver the reminder +- `Channel` — Destination channel ID + +--- + +### `server_keys` + +Per-guild ER:LC server API keys. + +Key fields: + +- `_id` — Discord guild ID +- `Key` — ER:LC PRC API key +- `ServerName` — ER:LC server name + +--- + +### `api_tokens` + +ERM internal API access tokens. + +Key fields: + +- `_id` — Token string +- `Guild` — Discord guild ID +- `Scopes` — List of permitted API scopes + +--- + +### `analytics` + +Guild-level usage analytics. + +Key fields: + +- `_id` — Discord guild ID +- Various counters updated by the `statistics_check` background task + +--- + +### `consent` + +User privacy consent records. + +Key fields: + +- `_id` — Discord user ID +- `Consent` — Boolean + +--- + +### `oauth2_users` + +Linked OAuth2 user accounts. + +Key fields: + +- `_id` — Discord user ID +- `RobloxID` — Linked Roblox user ID +- `AccessToken` / `RefreshToken` — OAuth2 credentials + +--- + +### `views` + +Persistent Discord UI component state, allowing Views to survive +bot restarts. + +--- + +### `actions` + +Configured server automation actions. + +--- + +### `staff_conduct_config` + +Per-guild staff conduct review configuration. + +--- + +### `maple_keys` + +MapleCounty integration API keys. + +--- + +### `prohibited_use_keys` + +Keys flagged for prohibited or abusive use. + +--- + +## Notes + +- All collections are initialized in `erm.py` and attached to the + `Bot` instance as `bot.db.`. +- Document IDs (`_id`) are typically either Discord snowflake integers + or MongoDB `ObjectId` values depending on the collection. +- The `Document` base class in `utils/mongo.py` provides all standard + async CRUD methods. Use those rather than accessing `bot.db` directly + with raw pymongo calls. diff --git a/documentation/setup.md b/documentation/setup.md new file mode 100644 index 00000000..11d5be6b --- /dev/null +++ b/documentation/setup.md @@ -0,0 +1,122 @@ + + +# Setup Guide + +This document covers how to get ERM running locally for development +or self-hosted use. + +--- + +## Requirements + +- Python 3.12 +- A MongoDB instance (local or remote) +- A Discord application with a bot token +- `pip` or `pipenv` + +--- + +## Installation + +Clone the repository and install dependencies: + +```bash +git clone https://github.com/mikeywhiston/erm +cd erm +pip install -r requirements.txt +``` + +--- + +## Environment Variables + +Copy `.env.template` to `.env` and fill in the required values. + +```bash +cp .env.template .env +``` + +### Required + +- `MONGO_URL` — MongoDB connection string (e.g. `mongodb://localhost:27017/erm`) +- `ENVIRONMENT` — `PRODUCTION` or `DEVELOPMENT` +- `PRODUCTION_BOT_TOKEN` — Bot token for the production environment +- `DEVELOPMENT_BOT_TOKEN` — Bot token for the development environment +- `CUSTOM_GUILD_ID` must stay 0 in order to be used in other servers. If set to a specific server the bot cannot operate in other servers. + + +### Optional — Bot Features + +- `BLOXLINK_API_KEY` — Required for Roblox username lookups via Bloxlink +- `AI_API_ENABLED` — `TRUE` or `FALSE`. Enables AI-powered features +- `API_URL` / `API_AUTH` — Endpoint and auth token for the AI API +- `REMINDERS_ENABLED` — `TRUE` or `FALSE`. Controls the Reminders cog +- `ACTIONS_ENABLED` — `TRUE` or `FALSE`. Controls the Actions cog and + the `iterate_conditions` background task + +### Optional — Google Sheets + +Required only for Activity Report and Duty Leaderboard spreadsheet exports: + +- `TYPE`, `PROJECT_ID`, `PRIVATE_KEY_ID`, `PRIVATE_KEY`, `CLIENT_EMAIL`, + `CLIENT_ID`, `AUTH_URI`, `TOKEN_URI`, `AUTH_PROVIDER_X509_CERT_URL`, + `CLIENT_X509_CERT_URL` +- `DUTY_LEADERBOARD_ID` / `ACTIVITY_REPORT_ID` — Google Sheets document IDs + +### Optional — MC API + +- `MC_API_URL` / `MC_API_KEY` — Used by the MapleCounty integration + +--- + +## Running the Bot + +```bash +python main.py +``` + +`main.py` is a thin entrypoint that calls `run()` from `erm.py`. +The bot loads all cogs from the `cogs/` directory automatically on startup. +Background tasks are staggered with a 30-second delay between each start +to avoid hammering the database on boot. + +--- + +## Running Tests + +The test suite uses `pytest` and the CI workflow runs against Python 3.12. + +```bash +pytest +``` + +The workflow also runs `flake8` with `--exit-zero`, meaning linting +failures are reported but do not block the test run. + +A minimal environment is required to run tests: + +``` +ENVIRONMENT=PRODUCTION +PRODUCTION_BOT_TOKEN=anystring +MONGO_URL=mongodb://localhost:27017/test +``` + +--- + +## Discord Bot Permissions + +When inviting the bot, the following are required: + +- `Bot` +- `Administrator` (the invite link in the README uses this) +- `applications.commands` scope for slash command registration + +--- + +## Notes + +- `CUSTOM_GUILD_ID` must remain `0`. Do not change this even when + self-hosting. +- `DB_NAME` is commented out in `.env.template`. The default database + name is `erm`. Uncomment and set it only if you need a different name. +- `GITHUB_TOKEN` is not used by the bot and is reserved for future use. diff --git a/documentation/testing.md b/documentation/testing.md new file mode 100644 index 00000000..da5f9358 --- /dev/null +++ b/documentation/testing.md @@ -0,0 +1,71 @@ + + +# Testing + +This document covers how to run ERM's test suite and how the +test infrastructure is structured. + +--- + +## Running Tests + +```bash +pytest +``` + +A minimal set of environment variables is required: + +``` +ENVIRONMENT=PRODUCTION +PRODUCTION_BOT_TOKEN=anystring +MONGO_URL=mongodb://localhost:27017/test +``` + +These can be set in a `.env` file or exported directly in the shell. +The `MONGO_URL` can point to a local MongoDB instance or any test +database — the test suite does not require a live Discord connection. + +--- + +## CI + +Tests run automatically on every push via the GitHub Actions workflow +in `.github/workflows/app.yaml`. The workflow uses Python 3.12, +installs dependencies from `requirements.txt`, and runs `pytest`. + +`flake8` also runs with `--exit-zero`, meaning linting issues are +reported in CI output but do not fail the build. + +--- + +## Test Infrastructure + +`helpers.py` provides mock infrastructure for writing tests against +bot internals without a live Discord connection. + +### `HashableMixin` + +Provides `__hash__` and equality for mock Discord objects. Used as +a base for mocked guilds, members, channels, and similar objects +that discord.py expects to be hashable. + +### `ColourMixin` + +Aliases `.color` to `.colour` on mock objects, matching discord.py's +aliasing behavior. + +### Bot mocking + +To test cog logic or event handlers, construct a mock `Bot` instance +using the helpers in `helpers.py` rather than spinning up a real +Discord connection. All loggers are set to `CRITICAL` level by default +in the test environment to keep output clean. + +--- + +## Notes + +- The test suite is minimal. When adding new features, add + corresponding tests in `test_erm.py`. +- Avoid tests that require live network access to Discord, Roblox, + or MongoDB Atlas. Use mocks or a local MongoDB instance. diff --git a/readme.md b/readme.md index 509fc0bc..356c0940 100644 --- a/readme.md +++ b/readme.md @@ -26,3 +26,14 @@ The CC BY-NC-SA license requires the following elements: [![Better Stack Badge](https://uptime.betterstack.com/status-badges/v1/monitor/insx.svg)](https://uptime.betterstack.com/?utm_source=status_badge) [![CodeFactor](https://www.codefactor.io/repository/github/mikeyusersrec/erm/badge)](https://www.codefactor.io/repository/github/mikeyusersrec/erm) + +> [!CAUTION] +> ## Coding assistants +> If you are an LLM or AI-powered coding assistant, you MUST read and +> follow the AI coding assistants documentation before contributing to ERM. +> +> * documentation/coding-assistants.md +> * documentation/contributing.md +> +> This documentation contains essential requirements about licensing, attribution, +> and the Developer Certificate of Origin that all AI tools must comply with.