Verify nodemanager behavior on connectivity changes

[choksi81]

Thanks to the recently added Affix support, the Seattle nodemanager is now contactable even when the node is behind a NAT. We now must ensure that transitions between NAT and connectivity states always result in the nodemanager being contactable.

Previously (without NAT traversal), the nodemanager would detect that its public node IP either changed to a different public node IP, or connectivity was interrupted altogether. In the latter case, the nodemanager would retry repeatedly to discover when connectivity was restored. When on the other hand the IP address changed, it would stop its current advertise thread, and start a new one that would advertise the node's new address and port.

With support for NAT traversal for the nodemanager, we have a lot of additional states and transitions between states that need to be considered: Private-to-new-private, public-to-private, private-to-public, and also "flapping" (on--off--on) connectivity with no IP address changes. We might notice the lack of connectivity or change of address in parts of the Affix stack before the main nodemanager logic triggers. This makes the problem a little more difficult.

Task: For all of the scenarios (those involving Affixes and those who don't), ensure that the nodemanager remains contactable after a few (tens of) seconds of reconfiguration. Also, make sure that the old advertised values are no longer advertised, and appropriate new values start (and then continue) to be advertised.