Use connextdds 7.3.0 for Rolling

[lobolanja]

This PR aims to update the CI to enable testing of rmw_connext once the rti_connext_dds-7.3.0 packages are available in the upstream ROS 2 repositories.

On Linux, Humble and Jazzy will continue to use Connext 6. On Windows, much of the plumbing to support multiple connext versions was lost in the Conda transition since there was only one version to support.

We've made the decision to merge as-is, which means that Humble and Jazzy CI on Windows will build with Connext 7.3 until changes are made. There's an initial draft in #812.

Resolves: #804

[nuclearsandwich]

Nothing wrong with this PR, but the Windows agents have not yet been updated to supply connext 7.3.0 yet so even once this is updated to conditionally use the target version of connext we'll need infra changes before this can merge.

[lobolanja]

Nothing wrong with this PR, but the Windows agents have not yet been updated to supply connext 7.3.0 yet so even once this is updated to conditionally use the target version of connext we'll need infra changes before this can merge.

@​nuclearsandwich Does that mean we don’t need to change the logic to support Humble and Jazzy, as mentioned in this comment?
#811 (comment)

[nuclearsandwich]

Does that mean we don’t need to change the logic to support Humble and Jazzy, as mentioned in this comment?

Sorry for making that ambiguity. I should have said that my comment does not reflect changes needed in this PR but changes to the infra to support it's work. We do still need to install 6.0.1 for Humble and Jazzy.

[lobolanja]

@​nuclearsandwich I’m making the necessary changes to install 6.0.1 for Humble and Jazzy.

[nuclearsandwich]

@lobolanja if you cherry-pick 85140fc from the windows-connext-7.3.0 branch onto this PR it includes the private submodule update that makes all 7.3.0 files available on Windows. You can use the same pattern to include them in windows_docker_resources/Dockerfile that the existing packages use (the files are split and need to be recombined when copying).

[nuclearsandwich]

Another random note while I'm thinking of it. We historically have not had RTI connext working on arm64 and it's still forcibly disabled in the ci_launcher:

ci/job_templates/ci_launcher_job.xml.em

Lines 101 to 110 in cab2da7

	@[ if os_name in ['linux-aarch64']]@
	<hudson.plugins.parameterizedtrigger.BooleanParameters>
	<configs>
	<hudson.plugins.parameterizedtrigger.BooleanParameterConfig>
	<name>CI_USE_CONNEXTDDS</name>
	<value>false</value>
	</hudson.plugins.parameterizedtrigger.BooleanParameterConfig>
	</configs>
	</hudson.plugins.parameterizedtrigger.BooleanParameters>
	@[ end if]@

I actually think we should leave that as-is for this PR because even though RTI is available it's not necessarily supported / tested by ROS 2 yet.

[nuclearsandwich]

It took ruthlessly commenting out the re-assembly of the sideloaded RTI connext packages (I wonder if, in generally, we should not be installing those when using the debs. May be we do skip installing those downstream I can't remember)

But there's now a job running beyond package installation.

Just to be safe after bringing back the other changes.

[nuclearsandwich]

Windows with the relevant cherry-pick

Made a mistake with the configuration and amended it:

[nuclearsandwich]

And one more, Linux CI with the RTI binaries instead of debs:

Once more un-commenting the stuff I commented out before it worked:

Until the blood runs clear

[nuclearsandwich]

RHEL build

Fingers still remember the old package name:

[nuclearsandwich]

@lobolanja I made a few more changes on the main fork's copy of this branch. The easiest thing for you to do may be to pull those changes and force-push them to this branch since it is included in my rebase.

I haven't addressed the version differences in the entry_point.sh as you point out still needs doing, but I did get at least one build up to rmw_connextdds on each platform but ARM (which I think we should punt on as mentioned above).

The next thing to do, in addition to the entry_point.sh script is run a larger CI build that actually exercises the communication tests. This will be much more time consuming so I'll start it off in the morning (I want to scale CI down for the weekend) and I don't want to trigger big jobs right before the nightlies.

[nuclearsandwich]

Full CI

• Linux
• Linux-aarch64
• Linux-rhel
• Windows

Updated binaries are now available.

[lobolanja]

@nuclearsandwich @clalancette
I’ve updated entry_point.sh as well as the Ubuntu and RHEL Dockerfiles to support Humble and Jazzy.

⚠️ Could you run the Ubuntu validation (using the RTI web installers) to confirm I haven’t broken anything?

Also, it’s still unclear to me whether the Windows Dockerfile (windows_docker_resources/Dockerfile) is expected to support Humble and Jazzy as well.

I’m currently working on the Windows changes, but it’s been a bit of a pain — Windows Dockerfiles are pretty tricky to deal with.

[lobolanja]

Hi @nuclearsandwich
I’ve spent the whole day trying to get this working on Windows to support Humble and Jazzy, but I haven’t been able to make it work yet.

Would you mind if we go ahead and merge this as-is, and continue working on adding Humble and Jazzy support later this week while we stabilize the rest?
Supporting this on the Windows dockerfile will likely require using some complementary .bat file to handle the installation logic, since we’re quite limited with just the Windows Dockerfile syntax.

Let me know your thoughts!

[nuclearsandwich]

Retriggering CI to make sure the current configuration builds with 7.3.0 at least.

• Linux
• Linux-aarch64
• Linux-rhel
• Windows

Now with requeues

• Linux
• Linux-aarch64
• Linux-RHEL

[mjcarroll]

Looks like both Linux and Linux-aarch64 got bounced.

[nuclearsandwich]

Merging with incomplete final CI based on earlier results.

[Blast545]

👨‍🌾 @lobolanja Is there planned work to get this into the build.ros2.org farm as well?
It seems that after ros2/rmw_connextdds#181 was merged, the Rci__nightly-connext_ubuntu_noble_amd64 jobs have been failing with:

03:21:50 -- Found rmw_implementation_cmake: 7.8.1 (/tmp/ws/install_isolated/rmw_implementation_cmake/share/rmw_implementation_cmake/cmake)
03:21:50 CMake Error at /tmp/ws/install_isolated/rmw_implementation_cmake/share/rmw_implementation_cmake/cmake/get_default_rmw_implementation.cmake:60 (message):
03:21:50   Could not find ROS middleware implementation 'NOTFOUND'.  Choose one of the
03:21:50   following:
03:21:50 Call Stack (most recent call first):
03:21:50   CMakeLists.txt:24 (get_default_rmw_implementation)
03:21:50 
03:21:50 
03:21:50 -- Configuring incomplete, errors occurred!

I imagine it's trying to find the previous rti_connext_dds packages. Reference: Rci__nightly-connext_ubuntu_noble_amd64/363

[lobolanja]

Thanks for flagging this @Blast545!

Just a heads-up—it’s a holiday in Spain until Monday, so we’ll be taking a look at this first thing Monday morning (Spain time).

cc: @fgallegosalido

[fgallegosalido]

@Blast545 it seems that the CI script is still setting the environment variable CONNEXTDDS_DIR=/opt/rti.com/rti_connext_dds-6.0.1, which in rolling is wrong as it downloads Connext 7.3.0.

[Blast545]

This was fixed with: ros2/ros_buildfarm_config#324, @fgallegosalido thanks for taking a look!