colcon build Slow? What Overlays, COLCON_IGNORE and –symlink-install Actually Save, Measured on ROS 2 Jazzy

한국어로 읽기

I set up four tiny ament_cmake packages in a ROS 2 Jazzy container, built them once, and then ran colcon build again without touching a single file. It still took about a second. Building only one of the four with --packages-select took a bit over half a second, not a quarter of that. Hiding two packages with COLCON_IGNORE saved roughly 0.3 seconds per build. Then I added --cmake-args -DBUILD_TESTING=0, the tip the official colcon tutorial gives for skipping tests. That next build took more than twice as long as a normal no-change rebuild, because colcon re-ran CMake configure in all four packages. Removing the flag cost about the same again, and the tests still didn’t come back.

So in this setup, switching a flag between builds cost more than leaving packages out saved. If colcon build feels slow, first stop changing the arguments from one build to the next. After that, cut down how many packages each build touches. This article explains why, using what colcon does on every run, five measured runs in a container, and the commands that produced them.

The question comes up often. A Robotics Stack Exchange question from November 2023 asks how to speed up a colcon build that feels much slower than catkin or plain CMake, and how to find the cause. It had no accepted answer when I checked on 2026-10-04. This article doesn’t try to explain colcon-versus-catkin differences on that poster’s machine. It covers the part you can control: what each common speed-up option actually removes from a build. You should know how to source a setup file, run colcon build, and tell an ament_cmake package from an ament_python one. Nothing here moves a robot or runs a simulator. It’s purely about the build tool.

What colcon build repeats on every run

When you run colcon build, colcon looks for packages under src, applies any package selection arguments to choose this run’s set, and hands each selected package to an executor. The executor runs them one at a time (sequential) or concurrently (parallel). What happens to each package depends on its build type.

For a CMake-based package, colcon-cmake runs the same steps every time. It checks whether CMake needs to reconfigure, runs cmake --build, runs the install step if there is an install target, and writes the environment hooks (colcon-cmake build.py, L87-L113). If nothing changed, the compiler has almost no work. But the build tool still checks that every target is up to date, and the install step still runs. That’s why a build with no changes isn’t free, and why it grows with the number of selected packages.

Reconfiguring means CMake reads CMakeLists.txt again and regenerates the build files. colcon-cmake does this only when one of these is true: you pass --cmake-force-configure, there is no CMakeCache.txt, the build files are missing, or the CMake arguments differ from the previous run (L121-L140). That last condition is the one that showed up in the measurements.

There’s a second source-level detail. When you pass --symlink-install, colcon-ros adds -DAMENT_CMAKE_SYMLINK_INSTALL=1 to the CMake arguments of every ament_cmake package (colcon-ros ament_cmake/build.py, L50-L53). It looks like an install option, but for CMake packages it also changes the CMake arguments. So adding or dropping it between builds should cause the same reconfigure as changing --cmake-args. That was my reading of the code before running anything. The experiment below checks it.

Four common fixes, three different kinds of savings

The Jazzy colcon tutorial mentions overlays and --symlink-install in the main text, and COLCON_IGNORE and -DBUILD_TESTING=0 in its tips at the end. Read separately, all four look like “make the build faster” switches. They don’t all save the same kind of time.

Fewer packages per build: overlays, COLCON_IGNORE, package selection

Overlay. Put the few packages you edit all the time in their own workspace, and build it on top of an underlay, meaning an existing install you source first. The underlay can be the binary ROS install or another colcon workspace. The tutorial recommends this when you plan to iterate on a small number of packages (Jazzy tutorial source, L161-L162). Build the underlay first, then source it in a new terminal, then build the overlay. Overlay packages can depend on underlay packages, but not the other way round. Afterwards you only need to source the last workspace in the chain (colcon: Using multiple workspaces). The same page warns that overriding an underlay package with a different version in the overlay doesn’t work in every case.

COLCON_IGNORE. An empty file with this name in a directory stops colcon from indexing that path. In colcon-core, the ignore check runs at priority 1000, ahead of the other package identification extensions, and skips any path that contains the file (colcon-core ignore.py, L11-L28). It stays in effect for every colcon command until you delete the file. If you commit it, it applies to your teammates too. If other packages in the workspace depend on an ignored package, that package has to come from an underlay instead.

Selection arguments change the set for a single run only. --packages-select builds just the named packages and assumes their dependencies are already built. --packages-up-to also builds their dependencies recursively. --packages-above also builds everything that depends on them, which is what you want after changing a package other packages use (colcon: How to, package selection arguments).

No rebuild at all for some edits: –symlink-install

With --symlink-install, colcon uses symbolic links instead of copying files from the source and build directories into install where possible (colcon build verb). The tutorial’s examples are Python files and other non-compiled resources: if you edit them in src, the installed version changes too (tutorial source, L172-L174). For Python packages, colcon-core uses setup.py develop --editable when setuptools supports it, and falls back to a normal install otherwise (colcon-core python/build.py, L104-L119). For ament_cmake, ament_cmake_core replaces CMake’s install() with a version that creates symlinks (symlink_install/install.cmake, L16-L37).

This doesn’t change C++ sources: compiled code still has to be compiled. I’d also expect a rebuild after adding a new file or changing the entry_points or data_files in setup.py, because the link targets are worked out at build time. That’s my reading of the colcon-core code, not something the docs state, and I didn’t test it. On Windows, symlinks need administrator rights or Developer Mode.

Less work inside CMake packages: -DBUILD_TESTING=0

BUILD_TESTING is a CMake option that defaults to ON. Test targets are normally wrapped in if(BUILD_TESTING) (CMake CTest module), and ament_cmake_test declares the same option with default ON (ament_cmake_test-extras.cmake, L19). The ros2 pkg create template for ament_cmake puts the lint tests inside that block (ros2cli template, L79-L88). Turning it off only saves the test portion of CMake packages. How much that is depends on how many test targets your packages have. It does nothing for ament_python packages. And because you pass it through --cmake-args, it falls under the “arguments changed, reconfigure” rule above.

How I measured it

The script creates three throwaway workspaces under /tmp/colcon_lab inside the container:

• ws_cmake: four independent ament_cmake packages, cm_1 to cm_4. Each has a one-line C++ executable, one installed YAML file, and a test-only executable target inside if(BUILD_TESTING).
• ws_py_copy and ws_py_symlink: the same ament_python package py_demo, whose module py_demo/core.py contains VALUE = 'v1'. I kept copy installs and symlink installs in separate workspaces so they wouldn’t mix.

Each colcon build is timed by wall clock using time.monotonic(). All builds use --executor sequential, except one comparison that uses the default executor. Builds with no changes take about a second, and in a trial run two back-to-back runs of the same build differed a lot. So the script times each of those three times and records the median.

Whether CMake reconfigured is read from the log colcon keeps for each package, log/latest_build/<package>/command.log. A configure step shows up there as a direct /usr/bin/cmake call on the package source directory. I didn’t start that way. My first version compared the modification time of CMakeCache.txt before and after each build. It reported “no reconfigure” for a build that was clearly slower. If CMake reconfigures but no cached value changes, the cache file can keep its old timestamp. If you check this in your own workspace, look at command.log, not the cache file’s timestamp. My second attempt also failed: I searched the log for " cmake " with spaces on both sides, but the real lines read CMAKE_PREFIX_PATH=... /usr/bin/cmake <src> .... I fixed the match to bin/cmake . A third fix: checking whether the test executable file existed always returned true, because an earlier build had left it on disk. The script now checks whether the generated Makefile still defines the test target.

The predictions were written in plan.md before the measured runs. The script is experiment.py, and the raw output of all five runs is in results.json. All three are pinned to commit 0c285fb. The whole experiment directory is at github.com/sungpyo9053/blog/tree/main/experiments/colcon-build-slow-overlay-colcon-ignore-symlink-install-order.

To reproduce it on Linux with Docker, run these three lines in an empty directory. The second line checks that the downloaded script matches the hash recorded in results.json. If it prints FAILED instead of experiment.py: OK, stop: you have a different file than the one measured here.

curl -fsSLO https://raw.githubusercontent.com/sungpyo9053/blog/0c285fb1b57bacb758f786e15b2bf7e12da69faa/experiments/colcon-build-slow-overlay-colcon-ignore-symlink-install-order/experiment.py
echo "2989285c5d908c4828f8f3b64a395c8bdaf0adced5cd6fe4442c5958acb4b29e  experiment.py" | sha256sum -c -
docker run --rm --network none --cpus 1 --memory 1g -v "$PWD":/work ros@sha256:066420e07f60aa18262f2479981def87ebcfcec42eefb0c0c57c4a46098348ca bash -c "source /opt/ros/jazzy/setup.bash && python3 /work/experiment.py"

A good run prints one JSON object per line, ends with a summary line whose failures list is empty, and exits with code 0. If any colcon command fails, the script prints an error line and exits with code 1. The measurement harness also isolated the container further: a read-only /work mount, /tmp on tmpfs, a PID limit, and HOME and ROS_LOG_DIR pointed at /tmp. Those extra settings aren’t needed for the script to work.

Results from five runs

Environment (as recorded in results.json): image ros:jazzy-ros-base, digest ros@sha256:066420e07f60aa18262f2479981def87ebcfcec42eefb0c0c57c4a46098348ca; ROS distro jazzy; Python 3.12.3; rclpy 7.1.12-1noble.20260902.053513; RMW rmw_fastrtps_cpp (default); colcon-core 0.21.3+upstream-1; no network; limits 1 CPU, 1 GiB. Inside the container os.cpu_count() reported 2 and the scheduler affinity was 2. All 5 runs exited with code 0 and an empty failures list. Each full run took between 33.67 and 34.71 seconds.

Ranges are the lowest and highest of the five runs. For no-change rebuilds, each run’s value is the median of 3 timings.

No-change rebuild of ws_cmake	Packages built	Prediction (plan.md)	Measured, 5 runs	Packages reconfigured
All packages, --executor sequential	4	Above zero, longest of the three, roughly proportional to package count	1.019–1.137 s	0
COLCON_IGNORE in cm_3 and cm_4	2	Between the other two	0.709–0.718 s	0
--packages-select cm_1	1	Above zero, shortest	0.562–0.573 s	0
All packages, default executor (colcon build)	4	No prediction	1.088–1.104 s	0

For comparison, the first clean build of the same four packages took 4.54–4.587 s. In every no-change rebuild, cm_1‘s command.log showed two steps, build and install, and no configure. With the two COLCON_IGNORE files in place, colcon list --names-only found only cm_1 and cm_2 in all five runs. After the files were deleted, it found all 4 again.

The next builds changed only the arguments. No source files changed. Each was timed once per run, in this order:

Arguments compared with the previous build	Measured, 5 runs	Packages reconfigured	BUILD_TESTING in cm_1 cache	Test target in generated Makefile
Added --cmake-args -DBUILD_TESTING=0	2.329–2.455 s	4	0	no
Same arguments again	0.844–0.868 s	0	0	no
Dropped the argument (plain command)	2.326–2.357 s	4	0	no
Added --symlink-install	3.058–4.3 s	4	0	no
--symlink-install again	0.912–0.933 s	0	0	no
colcon_defaults.yaml with symlink-install: true, no arguments on the command line	0.907–0.934 s	0	0	no

The clean build had BUILD_TESTING ON with the test target present. The 4.3 s value for “added --symlink-install” came from run 5. The other four runs fell between 3.058 and 3.108 s. From that build on, AMENT_CMAKE_SYMLINK_INSTALL in cm_1‘s cache was 1, and the installed params.yaml was a symlink pointing into src in all five runs. The build that dropped -DBUILD_TESTING=0 invoked configure in 4 packages, but cmake_cache_rewritten was 0. The CMakeCache.txt timestamps didn’t change.

py_demo (ament_python)	Clean build	Value read after editing core.py to v2, no rebuild	After rebuilding
Copy install	1.624–1.833 s	v1 (not reflected)	rebuild took 1.615–1.623 s, then v2
--symlink-install	2.008–2.176 s	v2 (reflected)	not needed

Selected raw lines from run 1:

{"step": "environment", "ros_distro": "jazzy", "colcon_core": "0.21.3+upstream-1", "os_cpu_count": 2, "sched_affinity": 2, "python": "3.12.3"}
{"step": "nochange_rebuild_4pkgs", "args": "colcon build --executor sequential", "samples_seconds": [1.035, 1.03, 1.025], "median_seconds": 1.03, "packages_finished": 4, "cmake_configure_invoked": 0, "cm_1_commands": ["build", "install"]}
{"step": "nochange_select_1pkg", "args": "colcon build --executor sequential --packages-select cm_1", "samples_seconds": [0.567, 0.602, 0.565], "median_seconds": 0.567, "packages_finished": 1, "cmake_configure_invoked": 0, "cm_1_commands": ["build", "install"]}
{"step": "nochange_args_changed_BUILD_TESTING_0", "args": "colcon build --executor sequential --cmake-args -DBUILD_TESTING=0", "seconds": 2.341, "packages_finished": 4, "cmake_configure_invoked": 4, "cmake_cache_rewritten": 4, "cm_1_BUILD_TESTING_in_cache": "0", "cm_1_AMENT_CMAKE_SYMLINK_INSTALL_in_cache": "OFF", "cm_1_selftest_in_generated_makefile": false, "cm_1_commands": ["configure", "build", "install"]}
{"step": "nochange_args_dropped_back_to_default", "args": "colcon build --executor sequential", "seconds": 2.347, "packages_finished": 4, "cmake_configure_invoked": 4, "cmake_cache_rewritten": 0, "cm_1_BUILD_TESTING_in_cache": "0", "cm_1_AMENT_CMAKE_SYMLINK_INSTALL_in_cache": "OFF", "cm_1_selftest_in_generated_makefile": false, "cm_1_commands": ["configure", "build", "install"]}
{"step": "py_edit_without_rebuild_copy", "before_edit": "v1", "after_edit_no_rebuild": "v1", "reflected": false}
{"step": "py_edit_without_rebuild_symlink", "before_edit": "v1", "after_edit_no_rebuild": "v2", "reflected": true}

Against the predictions in plan.md: these matched in all 5 runs. The no-change build order was 4 packages > 2 > 1. COLCON_IGNORE removed exactly the ignored packages from discovery. Changing the arguments reconfigured every CMake package. Repeating the same arguments, or pinning them in colcon_defaults.yaml, reconfigured none. A Python edit showed up without a rebuild only with the symlink install. The proportionality prediction did not hold. plan.md expected no-change rebuild time to grow roughly in proportion to package count, but building 1 package took 0.562–0.573 s against 1.019–1.137 s for all 4. plan.md made no prediction about what happens to BUILD_TESTING after -DBUILD_TESTING=0 is dropped. In all five runs the cached value stayed 0 and the test target didn’t return.

What the numbers say

Everything below is my interpretation of the tables above. Where I give a single figure, it’s the median of the five per-run values. I calculated it myself; it isn’t in the output.

Package count matters, but each build also has a fixed cost. The no-change medians were about 0.565 s for 1 package, 0.716 s for 2, and 1.03 s for 4. A straight line through the 1-package and 4-package points gives about 0.155 s per extra package. Starting from about 0.41 s, that line predicts about 0.72 s for 2 packages, which is close to what I measured. So with these tiny packages, roughly 0.4 s of every build doesn’t depend on how many packages you build. That’s probably colcon starting up, discovering packages, and similar shared work. I didn’t time those steps separately, so that part is a guess. With real packages, the per-package share will be larger and the fixed share will matter less. These numbers can’t tell you by how much.

Changing flags cost more than leaving packages out saved. Adding -DBUILD_TESTING=0 took about 2.34 s, against about 0.86 s for repeating the same arguments. That’s about 1.5 s extra, or roughly 0.37 s per reconfigured package. Adding --symlink-install cost about 2.2 s extra. In comparison, ignoring two packages saved about 0.31 s per no-change build. With these packages, one flag switch used up about five builds’ worth of that saving. A habit like “add -DBUILD_TESTING=0 today because I’m in a hurry” makes things slower here. The colcon_defaults.yaml build didn’t reconfigure even though its command line differed from the previous build. So colcon seems to compare the final arguments, including those from the config file, with the previous run, not the literal command line.

Dropping -DBUILD_TESTING=0 doesn’t turn tests back on. That build paid the full reconfigure cost and still left BUILD_TESTING at 0. This matches how CMake caches work: a value stored in CMakeCache.txt stays there until something overwrites it. Leaving the argument out doesn’t overwrite it. Passing -DBUILD_TESTING=ON explicitly, or clearing the cache, should restore it, but I didn’t measure either. If you ever build with tests off and later want to run colcon test, check that the tests really came back.

The parallel executor didn’t help here, for a reason specific to this container. colcon’s default number of parallel workers is os.cpu_count() (executor arguments). That reported 2 inside the container, which had a 1-CPU quota. So two workers were most likely sharing one CPU. On a multi-core workstation the result could look very different, and this experiment says nothing about that.

The symlink install costs a little up front and pays off on every edit after that. The symlink clean build of py_demo took about 0.4 s longer than the copy build. After that, Python edits needed no rebuild, while the copy install needed about 1.6 s per edit. The YAML file installed by an ament_cmake package was also a symlink into src. So editing an existing launch or config file that’s already installed should work the same way. I only checked the symlink itself, though, not a reload after an edit.

Limits of this experiment

• The C++ packages are one-line programs. The timings show colcon’s and CMake’s per-package overhead, not the compile times of a real workspace.
• I didn’t build an actual underlay/overlay pair. Overlays help by reducing how many packages each build touches, and I measured that effect with --packages-select and COLCON_IGNORE.
• I didn’t measure recompiling after a C++ change, adding a new Python file, or changing setup.py under --symlink-install.
• The argument-switch builds ran once per run, in a fixed order. The no-change build with tests off (0.844–0.868 s) was faster than the earlier one with tests on (1.019–1.137 s). But the experiment wasn’t designed to measure the effect of BUILD_TESTING itself, so don’t read that gap as the effect of turning testing off.
• All numbers come from one host and one 1-CPU, 1 GiB container. Before running, I checked the package versions in the image with dpkg-query: colcon-cmake 0.2.30, colcon-ros 0.5.0, colcon-parallel-executor 0.3.0, cmake 3.28.3. The source links above point to the current upstream commits, which could differ in detail from those Jazzy packages. The measured behavior is from the Jazzy packages.
• I didn’t compare against catkin, and I didn’t test arm64 boards like Jetson.

A practical order for your own workspace

1. Pin your flags first. Put the options you always want in colcon_defaults.yaml at the workspace root (colcon configuration), so every build uses the same arguments. This is the form I tested:

   build:
     symlink-install: true

   The configuration docs say other build options, such as cmake-args, can go in the same file. I only tested symlink-install.

2. Find out where the time goes. Build once with the defaults and once with --executor sequential, then compare per-package times. Each build leaves a log/build_<timestamp>/ directory with a command.log and stdout_stderr.log per package, and log/latest_build points to the newest one (colcon log files). The console_start_end event handler prints when each package starts and finishes (event handler arguments). Use sequential mode to diagnose, not to speed things up. The tutorial suggests it for machines with little CPU or RAM, where a parallel build can freeze the system. If a build is slow when you didn’t change anything, check that package’s command.log for a direct cmake call on the source directory. That’s a configure step, and it usually means the arguments changed.
3. Take packages you rarely change out of the everyday build. For a long-term setup, move them to an underlay. To leave a package out for a while, use COLCON_IGNORE. For a single build, use --packages-select or --packages-above.
4. Match the remaining fixes to package type. For ament_python packages, the pinned --symlink-install removes the rebuild after Python edits. For ament_cmake packages in a workspace where you don’t run colcon test, consider -DBUILD_TESTING=0, also pinned in the config file and not added only now and then.

This order isn’t right for everyone. If you run tests in the same workspace often, turning BUILD_TESTING off and back on may cost more than it saves. Before ignoring a package other packages depend on, make sure an underlay provides it. And the size of each saving in your workspace will be different from these toy numbers. Step 2 is how you find out what they actually are.