Travis CI Tests Inconsistently pass/fail code changes

[connor-wool]

During the process of getting my pull request ready (#402) I've submitted a number of different commits, each of which triggers the Travis CI system to rebuild the solution, then run the linter and the test system.

When I submitted commit b41dba6 Travis CI ran the tests and a portion of the control system failed, which led me to believe that I had caused an issue with the operation of the control system that I was going to have to fix.

Later though, I reverted commit dfdc296 which modified only the .gitignore for the repository. This reversion, commit 20979c4 then passed the Travis CI build and tests, without any change to the underlying code that the CI system was testing.

This leads me to believe there is some undiagnosed issue with one of the following:

• How the tests for the Control System are written
• How the CI system runs the tests for the code as a whole

I don't have any actionable solutions, I just wanted to open this up so we had a starting point to work on it.

[bkallaher]

Based on the failure you reference it seems to be a problem with boost not being able to get a mutex lock. This could just be how ROS Test kills a subprocess on timeout. The other case is that the lock failure causes the test to hang until the timeout is triggered by ROS Test. The only way to test which one it is right now is to watch the output of tests manually until a failure is seen.

There is this open issue in the Travis repo that would make it so someone could just look at a timestamp but no progress seems to be made: travis-ci/travis-ci#4968

[connor-wool]

Ok, that's what I was wondering, I've never worked with boost before so there's a lot of it that's still confusing.

I re-wrote part of the PR that I made because I thought the issue might be with how boost wanted to handle the copy of a data member (ie. the mutex lock error was caused by how I was trying to access and copy data). Seems like it might actually be two separate issues, but oh well, the code is written at this point.