Internal Node Id Dependent on Order of Action Execution

[forrestmckee]

The order of vulnerability execution defines the order that __discovered_nodes are added internally. There is functionality in place that uses this integer value for downstream tasks. (1, 2, 3, 4, etc)

This also means that the action masking is dependent on the order of vulnerability execution (repro)

Doesn't that mean that during training, the state-action value approximations are based on an integer encoding that changes according to the order of actions at every reset? Checkpointing or transfer learning would also suffer.

For ports, local/remote vulnerabilities, etc you use model.Environment.identifiers when retrieving via integer encoding, so they're fixed.

Let me know if you need any clarification, or if I'm missing something here.

Thanks!

[blumu]

Yes, the agent is essentially forced to refer to nodes by order of their discovery (relative) instead of unique IDs (absolute). This was meant to be more realistic and prevent an agent from just learning and remembering absolute node IDs from a particular network instance. It was meant precisely to assess agents on transfer learning tasks (e.g., learning from absolute node IDs would not transfer well to other instances/topologies). An alternative approach could be to shuffle the node IDs on every episode reset.