CI incorrectly runs build tool tests on output of test tool

[rotu]

When a test writes output that mimics build output, like "[ERROR] []: Original error msg: custom exception raised in configure callback" (as seen in ros2/rclcpp#777), it falsely flags it as a MSBuild error. The text output of colcon test-result should never cause a CI failure.

This might be a bug in the deprecated Warnings plugin currently in use or might be a misconfiguration of CI rules.

[dirk-thomas]

When a test writes output that mimics build output

Since the Jenkins plugin parses the output to find known patterns I don't see how we could avoid this. Do you have a specific idea how that could be achieved? If not, I am inclined to close this as wontfix since it is outside our hands.

[rotu]

@dirk-thomas Yes, both Warnings and Warnings-NG plugins can (should) be configured to analyze a file other than consoleText. By default, colcon emits build and test output to log files already.