Neural Network Bot Runner

[tgrunnagle]

Summary

This PR implements the foundational neural network bot inference system for TowerFall, enabling trained ML models to play in end-user games. It introduces NeuralNetBot and NeuralNetBotRunner classes that integrate trained ActorCriticNetwork models with the game client infrastructure, following the existing RuleBasedBot pattern.

Closes: #96

Changes

New Files

• bot2/src/bot/bots/neural_net_bot.py (359 lines)

  • NeuralNetBot: Core bot class that uses neural network inference for decision-making
  • NeuralNetBotRunner: Runner class that integrates the bot with GameClient
  • NeuralNetBotConfig: Configuration dataclass for bot behavior

• bot2/tests/unit/test_neural_net_bot.py (715 lines)

  • Comprehensive test suite with 60 tests covering all functionality
  • Tests for initialization, action translation, state deduplication, edge cases, and runner integration

Modified Files

• bot2/src/bot/bots/__init__.py
  • Added exports for NeuralNetBot, NeuralNetBotConfig, and NeuralNetBotRunner

Key Features

1. Neural Network Inference

• Uses ActorCriticNetwork with deterministic action selection
• Converts game state to observation vectors via ObservationBuilder
• Properly handles PyTorch tensor operations with device support (CPU/CUDA)
• Network set to evaluation mode with gradient computation disabled

2. Action Translation

• Translates 27 discrete actions to game inputs:
  • Movement actions (0-7): Keyboard press/release for A, D, W, S keys
  • Aim actions (8-23): 16-bucket discretization of aim directions (0-2π radians)
  • Shooting actions (24-25): Mouse button press/release
  • No-op (26): Empty action list
• Handles aim direction defaults for shooting without prior aim (uses 0.0 radians/right)

3. Efficient Input Deduplication

• Keyboard inputs: Only sends when key state changes
• Mouse aim position: Tracks and sends only when position changes
• Mouse button state: Tracks and sends only when press/release state changes
• Prevents redundant network traffic and improves performance

4. State Management

• Tracks previous movement key states, aim direction, and shooting state
• reset() methods on both bot and runner for new game/episode
• Graceful handling of dead player and missing player states

5. Pattern Consistency

• Follows existing RuleBasedBot and RuleBasedBotRunner architecture
• Uses same BotAction types and return patterns
• Implements BaseBot interface correctly
• Compatible with existing GameClient infrastructure

Technical Implementation

Neural Network Integration

# Observation building
observation = self.observation_builder.build(self.current_state, self.player_id)

# Tensor conversion and inference
obs_tensor = torch.from_numpy(observation).float().unsqueeze(0).to(self.device)
with torch.no_grad():
    action_idx, _, _, _ = self.network.get_action_and_value(
        obs_tensor, deterministic=True
    )

Smart Deduplication

• Bot level: Tracks state and only returns changed actions
• Runner level: Double-checks and only sends changed inputs to client
• Aim position tracking: Both position and button state tracked separately

Sentinel Value Pattern

• Uses -1.0 as sentinel for uninitialized aim direction
• Allows proper deduplication of first aim action
• Defaults to 0.0 (right) for shooting without prior aim

Testing

Test Coverage

• 60 comprehensive tests covering:
  • Configuration (default and custom)
  • Initialization and network setup
  • Action decision logic
  • Action translation (all 27 action types)
  • State deduplication (movement, aim, shooting)
  • Reset functionality
  • Runner integration with GameClient
  • Edge cases (missing player, dead player, no enemies)
  • Module imports and inheritance

Test Results

• ✅ All 1,096 tests pass (including 60 new tests)
• ✅ Linting passes
• ✅ Type checking passes
• ✅ Code formatting passes

Code Quality

Adherence to Project Standards

• ✅ Uses native Python typing (list, str | None)
• ✅ Follows Pydantic v2 patterns where applicable
• ✅ Module imports use bot package name
• ✅ Comprehensive docstrings for all public methods
• ✅ Consistent with existing codebase patterns

Performance Considerations

• Network in evaluation mode (no gradient computation)
• Efficient state tracking with minimal overhead
• Smart deduplication reduces network traffic
• Reuses observation builder across calls

Review Feedback Addressed

This PR includes fixes for issues identified during code review:

1. Mouse input deduplication: Now tracks both button state AND aim position, allowing aim updates while holding the shoot button
2. Default aim handling: Uses sentinel value (-1.0) for initialization and defaults to 0.0 (right) for shooting without prior aim
3. Test coverage: Added test for shooting without prior aim to verify correct default behavior

Integration Points

This implementation is ready for integration with:

• TASK-002: WebSocket Bot Client (will use NeuralNetBotRunner to connect bots via WebSocket)
• TASK-003: Bot Manager (will manage lifecycle and model loading)
• TASK-004: REST API (will expose bot management endpoints)

Files Changed

bot2/src/bot/bots/__init__.py          |   4 +
bot2/src/bot/bots/neural_net_bot.py    | 359 ++++++++++++++++
bot2/tests/unit/test_neural_net_bot.py | 715 ++++++++++++++++++++++++++++++
3 files changed, 1078 insertions(+)

Acceptance Criteria

All acceptance criteria from issue #96 have been met:

• ✅ NeuralNetBot extends BaseBot and implements decide_actions()
• ✅ Uses ObservationBuilder to convert GameState to observation vectors
• ✅ Runs inference through ActorCriticNetwork.get_action_and_value() with deterministic=True
• ✅ Translates discrete Action enum values to BotAction tuples
• ✅ NeuralNetBotRunner follows RuleBasedBotRunner pattern
• ✅ Runner tracks previous action state to send only changed inputs
• ✅ reset() methods implemented for both bot and runner
• ✅ NeuralNetBotConfig dataclass for configuration
• ✅ Module exports added to bot/bots/__init__.py
• ✅ All unit tests pass (task bot2:test:unit)
• ✅ Code passes linting and type checking (task bot2:check)

Future Enhancements

Out of scope for this PR (addressed in subsequent tasks):

• WebSocket connection management (TASK-002)
• Bot lifecycle management (TASK-003)
• Model loading from registry (TASK-003)
• REST API endpoints (TASK-004)
• Integration with Go backend (TASK-005)