colcon-core: Cross-compiling ROS 2 workspaces - please do not promote building packages in underlays to an error and suggestion

After updating colcon to latest releases I observed the following:

colcon build --merge-install
[1.454s] WARNING:colcon.colcon_core.verb:Some selected packages are already built in one or more underlay workspaces:
	'ament_cppcheck' is in: /opt/ros/rolling
	'ament_lint' is in: /opt/ros/rolling
	'ament_lint_auto' is in: /opt/ros/rolling
	'ament_uncrustify' is in: /opt/ros/rolling
	'image_geometry' is in: /opt/ros/rolling
	'ament_flake8' is in: /opt/ros/rolling
	'ament_pep257' is in: /opt/ros/rolling
	'ament_copyright' is in: /opt/ros/rolling
	'ament_cpplint' is in: /opt/ros/rolling
	'ament_lint_cmake' is in: /opt/ros/rolling
	'ament_xmllint' is in: /opt/ros/rolling
	'launch' is in: /opt/ros/rolling
	'ament_cmake_lint_cmake' is in: /opt/ros/rolling
	'launch_testing' is in: /opt/ros/rolling
	'launch_xml' is in: /opt/ros/rolling
	'launch_yaml' is in: /opt/ros/rolling
	'ament_cmake_copyright' is in: /opt/ros/rolling
	'ament_cmake_cppcheck' is in: /opt/ros/rolling
	'ament_cmake_cpplint' is in: /opt/ros/rolling
	'ament_cmake_flake8' is in: /opt/ros/rolling
	'ament_cmake_pep257' is in: /opt/ros/rolling
	'ament_cmake_uncrustify' is in: /opt/ros/rolling
	'ament_cmake_xmllint' is in: /opt/ros/rolling
	'launch_testing_ament_cmake' is in: /opt/ros/rolling
	'ament_lint_common' is in: /opt/ros/rolling
	'cv_bridge' is in: /opt/ros/rolling
	'image_transport' is in: /opt/ros/rolling
If a package in a merged underlay workspace is overridden and it installs headers, then all packages in the overlay must sort their include directories by workspace order. Failure to do so may result in build failures or undefined behavior at run time.
If the overridden package is used by another package in any underlay, then the overriding package in the overlay must be API and ABI compatible or undefined behavior at run time may occur.

If you understand the risks and want to override a package anyways, add the following to the command line:
	--allow-overriding ament_cmake_copyright ament_cmake_cppcheck ament_cmake_cpplint ament_cmake_flake8 ament_cmake_lint_cmake ament_cmake_pep257 ament_cmake_uncrustify ament_cmake_xmllint ament_copyright ament_cppcheck ament_cpplint ament_flake8 ament_lint ament_lint_auto ament_lint_cmake ament_lint_common ament_pep257 ament_uncrustify ament_xmllint cv_bridge image_geometry image_transport launch launch_testing launch_testing_ament_cmake launch_xml launch_yaml

This may be promoted to an error in a future release of colcon-core.

This is the exact behavior we’re currently using in the Hardware Acceleration Working Group (HAWG) for cross-compilation. The use case for us is clear, we’re cross-compiling packages that exist in underlays to other (compute) architectures. Seing this warning is already a bit annoying but turning it into an error would break things completely for us so please do not.

Currently, the options available to mitigate this are:

  --allow-overriding PKG_NAME [PKG_NAME ...]
                        Allow building packages that exist in underlay workspaces

Would you be open to a new flag --allow-overriding-all that directly ignores and allows building all packages regardless of whether they’re already present in underlays?

If you confirm this is of interest, I’m happy put a few cycles to code it. I’d love to have this for Humble. With this new flag, we’d be able to create mixins that integrate it directly to further simplify user’s flow.

About this issue

  • Original URL
  • State: closed
  • Created 5 years ago
  • Reactions: 5
  • Comments: 15 (4 by maintainers)

Most upvoted comments

Ideally, this should get implemented using an environmental variable which allows the python logic to pick it every time to set it off completely and across builds. E.g.:

I think this should be possible too with the COLCON_EXTENSION_BLOCKLIST environment variable

 COLCON_EXTENSION_BLOCKLIST
                        Block extensions which should not be used

See this test for examples: https://github.com/colcon/colcon-core/blob/c11ce9cb27482dac85f05b50b3cd2c1b459ae6b3/test/test_entry_point.py#L108-L131

Experiencing the same problem here, same as the OP. Current warning is annoying, but the foreshadowed error would be very disruptive.

For me, happens even when not merge-installing. Simply replicated by creating a new package and building more than once

~/temp_ws/src$ ros2 pkg create temp_pkg --dependencies rclcpp
going to create a new package
package name: temp_pkg
destination directory: /home/morten/temp_ws/src
package format: 3
version: 0.0.0
description: TODO: Package description
maintainer: ['morten <email>']
licenses: ['TODO: License declaration']
build type: ament_cmake
dependencies: ['rclcpp']
creating folder ./temp_pkg
creating ./temp_pkg/package.xml
creating source and include folder
creating folder ./temp_pkg/src
creating folder ./temp_pkg/include/temp_pkg
creating ./temp_pkg/CMakeLists.txt
~/temp_ws/src$ cd ..
~/temp_ws$ colcon build --packages-select temp_pkg
Starting >>> temp_pkg
Finished <<< temp_pkg [1.84s]                  

Summary: 1 package finished [1.94s]
~/temp_ws$ colcon build --packages-select temp_pkg
[0.187s] WARNING:colcon.colcon_core.verb:Some selected packages are already built in one or more underlay workspaces:
	'temp_pkg' is in: /home/morten/temp_ws/install/temp_pkg
If a package in a merged underlay workspace is overridden and it installs headers, then all packages in the overlay must sort their include directories by workspace order. Failure to do so may result in build failures or undefined behavior at run time.
If the overridden package is used by another package in any underlay, then the overriding package in the overlay must be API and ABI compatible or undefined behavior at run time may occur.

If you understand the risks and want to override a package anyways, add the following to the command line:
	--allow-overriding temp_pkg

This may be promoted to an error in a future release of colcon-core.
Starting >>> temp_pkg
Finished <<< temp_pkg [0.14s]                  

Summary: 1 package finished [0.24s]

With the following defaults:

$ cat ~/.colcon/defaults.yaml 
{
    "build": {
        "symlink-install": true,
        "cmake-args": ["-DCMAKE_BUILD_TYPE=Release"],
    }
}