Gazebo Shows Real Time Factor 1.0. Did Your ROS 2 Control Loop Meet Its Deadline? Measured on Jazzy

한국어로 읽기

I ran a 100 Hz rclpy timer (period 10 ms, deadline = period) inside a ROS 2 Jazzy container. Its node used use_sim_time, and a separate OS process published /clock paced to a target real time factor of 1.0, the way a simulator would. Every 50th callback slept for 22 ms on purpose. Across a 5-second phase in runs 2 to 5 (out of 5), the real time factor measured at the /clock publisher was between 0.9995 and 1.0. In those same 5 seconds the control callback missed its 10 ms deadline 20 times in every run, and the timer skipped 10 periods. The mean interval between callbacks was 10.205 to 10.211 ms. The longest was 22.903 to 23.103 ms.

Run 1 was worse. The container hit its CPU quota during that phase (5 throttled periods, 49643 µs in total), the publisher-side RTF fell to 0.9678 (0.737 in its worst 100 ms window), and the longest callback interval reached 41.491 ms.

The simulator clock kept up, but the control loop didn’t finish on time. Those are two separate facts, and RTF only reports the first one. A real time factor of 1.0 tells you that simulated time advanced as fast as wall-clock time. It doesn’t tell you whether any particular callback met its deadline. To know that, you have to record the worst interval, count deadline misses and count skipped timer periods yourself.

This article is for ROS 2 developers who see “Real time factor: 100 %” in Gazebo and wonder whether that means their controller is running “in real time”. It starts with why the ROS 2 documentation uses “real time” in two unrelated ways. Then it covers how RTF is actually computed and why a ROS 2 timer’s average interval hides late callbacks. After the full measurements, it ends with the few lines of code you need to check your own node. You should already know what a node, a timer and use_sim_time are. Nothing here runs on a real robot or a real-time kernel.

“Real time” means two different things in ROS 2

The confusion is old. A Robotics Stack Exchange question from June 2021 (number 22252, “Why does ROS2 claim to be ‘real time’?”) asked how ROS 2 could call itself real-time on a general-purpose OS. The accepted answer said that ROS usually means soft real-time. The official design documents are more careful, and they actually separate two meanings.

Real time as the rate at which time passes. The ROS 2 design article “Clock and Time” says: “In this article the term ‘real time’ is used to express the true rate of progression of time. This is not connected to ‘real-time’ computing with deterministic deadlines.” (130_ros_time.md, L35-L38; rendered at design.ros2.org). Simulated time, ROS time and the real time factor all belong to this meaning. The same article uses the phrase “real time factor” for how much faster simulated time advances than real time (L75-L76).

Real-time computing as a guarantee about deadlines. The design article “Introduction to Real-time Systems” defines a deadline as “the finite window of time in which a certain task must be completed” (120_realtime_background.md, L40). It then says a real-time system “is not defined by low latency, but by a deterministic schedule”. Latency must be measurable, a maximum allowable latency must be set, and you need both a real-time OS and deterministic user code to get real-time performance (L60-L64; rendered at design.ros2.org). The same article sorts systems by what a missed deadline means (L47-L55):

  • Hard real-time: missing a deadline is a system failure. The examples are airplane sensors and autopilots, spacecraft and planetary rovers.
  • Soft real-time: the system tries to meet deadlines, doesn’t fail when it misses one, and may lower its quality of service. The example is entertainment audio and video.
  • Firm real-time: results that arrive after the deadline are treated as invalid, but the system doesn’t fail.
Meaning Question it answers Unit
Rate of time (RTF, sim time) Is the simulated clock running faster or slower than the wall clock? A ratio with no unit, such as 1.0 or 0.5
Real-time computing (deadlines) Does this task finish by its deadline every time? Time (ms, µs), worst-case latency, number of misses

Nothing in the official ROS 2 docs says that ROS 2 makes your code real-time by itself. The design background says its core components “must not interfere with the requirements of real-time computing” (L26-L31). Whether a given loop meets its deadlines depends on the OS, the executor, your code and the hardware, and you find out by measuring.

What the real time factor actually measures

SDFormat defines <real_time_factor> as a “target simulation speedup factor, defined by ratio of simulation time to real-time”, with a default of 1.0 (physics.sdf, L23-L25). It’s a target, and it’s a ratio. If 10 s of simulated time pass in 10 s of wall-clock time, RTF is 1.0. If they take 20 s, RTF is 0.5. One warning: a comment in the Gazebo message definition says “0.5 means that 1 second in real time takes 2 seconds in simulation” (world_stats.proto, L57-L60). That reads backwards compared with the SDF definition and the server code. With RTF 0.5, one simulated second takes two wall-clock seconds.

The number you see on screen has been smoothed twice. At the commits I checked:

  • The Gazebo Sim server updates realTimeFactor = simTimeDiff / realTimeDiff on every iteration, comparing the simulated time added in one step with the steady-clock time that step took (SimulationRunner.cc, L437-L467).
  • It publishes world statistics at no more than 10 messages per second (L856-L862). With a 1 ms step, that’s about 10 samples out of roughly 1000 iterations each second.
  • The gz-gui WorldStats plugin applies an exponential filter with kAlpha = 0.9 to the simulated and real times before dividing, then shows the result as a percentage (WorldStats.cc, L229-L258).
  • Isaac Sim computes RTF per frame as simulated_elapsed_time / real_elapsed_time and can publish it as a std_msgs/msg/Float32 (Isaac Sim 5.1.0 RTF tutorial).

In every tool, RTF describes how fast the simulator moved its own clock. When your controller’s callbacks ran isn’t part of that calculation. Your node only sees simulated time through /clock. With use_sim_time set, ROS time returns the latest value received on /clock, and the design leaves the publishing rate and granularity of /clock to the application (L98, L123). If the simulator publishes /clock from its own process and your node runs late in another process, RTF can stay at 1.0 while your callback misses its deadline. That’s the claim the experiment tests.

The arithmetic: RTF 1.0 with a 15 ms overrun

Before running anything, it helps to see that both numbers fit in one small calculation. These are illustrative values chosen to compare the two quantities. They aren’t measurements or recommended limits.

  • A 100 Hz loop has a period of 1000 / 100 = 10 ms. Treat the period as the deadline.
  • 1000 cycles over 10000 ms of wall-clock time give a mean interval of 10000 / 1000 = 10 ms.
  • Those cycles covered 1000 × 10 ms of simulated time, so RTF = 10000 / 10000 = 1.0.
  • If the longest interval was 25 ms, that cycle overran its deadline by 25 − 10 = 15 ms.
  • If 3 of the 1000 cycles missed, the miss rate is 3 / 1000 × 100 = 0.3 %.

The mean and the RTF look perfect, and the deadline was still missed. It works the other way too. At RTF 0.5, a node with use_sim_time that runs one step per 10 ms of simulated time gets 10 / 0.5 = 20 ms of wall-clock time per step. If your wall-clock budget per step is 25 ms, you have 25 − 20 = 5 ms to spare. So RTF can be exactly 1.0 while deadlines are missed, and it can be far from 1.0 while every step stays within budget. These nine calculations are in a small script, example.py, with expected and actual values recorded in verification.json. That’s arithmetic only, not a ROS 2 run.

The model has one unrealistic assumption. It keeps all 1000 cycles after a 25 ms gap, and a real ROS 2 timer doesn’t do that.

Why the average hides a late callback: the rcl timer grid

In rcl_timer_call (Jazzy branch), the next call time always moves forward by exactly one period from the previous scheduled time, not from “now”. If that new time has already passed, meaning the timer missed at least one cycle, rcl jumps ahead by as many whole periods as needed (timer.c, L298-L319). The comment in the source says the point is “to avoid extending each cycle by the time between the timer being ready and the callback being triggered”.

Here’s what that does with a 10 ms period when the callback at t = 0 runs for 22 ms. The callback scheduled for t = 10 can’t start until about t = 22, so it starts about 12 ms late. By then t = 20 has already passed, so the next scheduled time jumps to t = 30 and the t = 20 callback never runs. The intervals between callback starts look like 22 ms followed by 8 ms.

Two things follow from the source:

  1. A short interval follows every long one, so over a long run the mean interval drifts back toward the period.
  2. Missed periods are skipped, not replayed, so you get fewer callbacks than the elapsed time would suggest.

If you log only the average interval, a late callback disappears. It shows up only in the maximum interval, the number of deadline misses and the number of skipped periods. That’s a reading of the source. The experiment below checks it.

The official ROS 2 tutorial “Understanding real-time programming” makes the same point with its own example output. It assumes a 1 kHz (1 ms) loop and a maximum allowable latency of 5 % of the period, which is 50 µs. The printed statistics show Mean 19871.8 ns and Max 2752187 ns, and the tutorial says: “our average latency was really good in this run, but the maximum latency was unacceptable because it actually exceeded our update loop!” (Real-Time-Programming.rst, L164-L186; rendered at docs.ros.org). That’s roughly 19.9 µs on average, which passes, and 2.75 ms at worst, which fails. These are the tutorial’s example numbers, not something I measured, and the tutorial notes that its demo currently supports only ConnextDDS.

The experiment: one script, five phases, five runs

The script runs a 100 Hz timer for 5 seconds in each of five phases. Every callback records its start and end on a monotonic clock (time.monotonic_ns(), which never jumps backwards) and on the timer’s own clock. Callbacks run on a SingleThreadedExecutor. In the simulated-time phases, the script starts a copy of itself as a separate OS process that publishes /clock. That process advances simulated time by 1 ms per message and paces itself to the target RTF. If it falls behind, it doesn’t burst to catch up, so a real slowdown shows up as a lower RTF. It also samples (monotonic time, simulated time) about every 100 ms, which is roughly the resolution Gazebo Sim’s statistics topic gives you.

Phase Timer clock /clock Injected delay
wall_baseline steady none none
wall_injected steady none 22 ms sleep in every 50th callback
sim_rtf1 ROS time (use_sim_time) separate process, target RTF 1.0 none
sim_rtf1_injected ROS time separate process, target RTF 1.0 22 ms sleep in every 50th callback
sim_rtf05 ROS time separate process, target RTF 0.5 none

The 22 ms sleep is a deliberate control, not a fault that happened by itself. A callback counts as a deadline miss when its response time (start delay against the scheduled grid plus execution time) exceeds 10 ms. Skipped periods are counted by replaying the same rule rcl uses on the recorded start times.

Three trial runs changed what the script records. First, counting “intervals longer than 10 ms” was useless on its own, because about half of all intervals ran a hair over 10 ms even with no injected delay. So the script also counts intervals longer than the period plus the 5 % jitter allowance the official tutorial assumes. Second, one trial showed tens-of-milliseconds gaps in the baseline phase with nothing injected. So the script now records when the longest interval happened and the container’s cgroup CPU counters (cpu.stat), which show whether the CPU quota ran out. Third, the trials showed something I hadn’t predicted. Measured with the node’s ROS time, the injected 22 ms callback seemed to take almost no time at all. That’s why both clocks are recorded.

Before the five recorded runs, the plan wrote down predictions from the rcl source and the arithmetic above. The table below puts each prediction next to what happened.

Environment

  • Image: ros:jazzy-ros-base, digest ros@sha256:066420e07f60aa18262f2479981def87ebcfcec42eefb0c0c57c4a46098348ca
  • ROS distro: jazzy; Python 3.12.3; rclpy 7.1.12-1noble.20260902.053513
  • RMW: rmw_fastrtps_cpp (default); ROS_AUTOMATIC_DISCOVERY_RANGE: SUBNET (image default)
  • Container limits: 1 CPU, 1 GiB, network none. CPUs visible inside the container: 2
  • Host kernel: 7.0.0-1009-aws, SMP PREEMPT (not PREEMPT_RT)
  • Runs: 5, all with exit code 0, each about 28 s. Script sha256 eefd43cdc31aeef1a5ea597987df4ba6448c85a0191719c8ae4d1d3f72480e89

Prediction vs measurement

Ranges are the minimum to maximum across the 5 runs. Intervals are wall-clock (monotonic) unless stated otherwise.

Phase Prediction (plan.md) Measured, 5 runs (results.json)
wall_baseline Mean interval ≈ 10 ms. Max and misses not predicted. Mean 10.0 ms in all runs. Max 10.079–11.148 ms. Deadline misses 0 in all runs. Skipped periods 0. Intervals over 10 ms: 242–252 of 498. Intervals over 10.5 ms: 0 in four runs, 5 in run 4.
wall_injected Mean ≈ 10.2 ms. Max ≈ 22 ms, next interval ≈ 8 ms. Skipped ≈ 10. Misses ≈ 20. Mean 10.205 ms. Max 22.517–22.581 ms. Min 7.404–7.467 ms. Skipped 10, misses 20 in all runs. Callbacks 489 over 498 grid periods. Max start lateness 12.52–12.619 ms.
sim_rtf1 RTF ≈ 1.0. Mean interval ≈ 10 ms. RTF 1.0 in all runs. Mean 10.0 ms. Max 10.209–11.685 ms. Misses 0 in all runs.
sim_rtf1_injected Publisher RTF ≈ 1.0. Max ≈ 22 ms. Skipped ≈ 10. Misses ≈ 20. Runs 2–5: publisher RTF 0.9995–1.0, smallest 100 ms window 0.975–0.9992. Mean 10.205–10.211 ms. Max 22.903–23.103 ms. Skipped 10, misses 20. No CPU throttling.
Run 1: publisher RTF 0.9678, smallest 100 ms window 0.737. Mean 10.505 ms. Max 41.491 ms. Skipped 8, misses 18. Throttled 5 times, 49643 µs.
sim_rtf05 RTF ≈ 0.5. Wall mean ≈ 20 ms, sim-time mean ≈ 10 ms. No interval over the 25 ms budget. Sim-time misses 0. Runs 1, 2, 4: RTF 0.5, mean 20.0 ms, max 20.172–20.263 ms, over 25 ms budget 0.
Runs 3 and 5: publisher RTF 0.4907 and 0.494, smallest 100 ms window 0.3441 and 0.2735, max 38.849 and 62.528 ms, over budget 8 and 3, no CPU throttling recorded.
All runs: sim-time mean interval 10.0 ms, deadline misses 0.

Callback execution time of the injected callback in sim_rtf1_injected

Measured with the node’s ROS time: 0.0 ms in all 5 runs. Measured with the monotonic clock: 22.102–22.217 ms (longest per run).

Raw output

Three lines copied from the stdout in results.json: the environment line from run 2, sim_rtf1_injected from run 2, and sim_rtf1_injected from run 1.

{"phase": "environment", "kernel": "7.0.0-1009-aws", "kernel_version": "#9~24.04.1-Ubuntu SMP PREEMPT Mon Jul 13 22:43:41 UTC 2026", "python": "3.12.3", "cpus_visible": 2, "rmw": "default", "discovery_range": "SUBNET"}
{"phase": "sim_rtf1_injected", "timer_clock": "ros_time(use_sim_time)", "target_rtf": 1.0, "inject_ms": 22.0, "inject_every": 50, "period_ms": 10.0, "callbacks": 489, "grid_periods_spanned": 498, "skipped_periods": 10, "injected": 10, "interval_wall_ms": {"mean": 10.205, "min": 6.945, "p99": 22.804, "max": 23.087}, "max_wall_interval_at_s": 3.313, "interval_timer_clock_ms": {"mean": 10.205, "max": 22.0}, "start_lateness_ms": {"mean": 0.247, "max": 12.0}, "response_ms_max": 22.108, "deadline_ms": 10.0, "deadline_misses": 20, "wall_intervals_over_period": 230, "wall_intervals_over_period_plus_jitter": 12, "jitter_tolerance_ms": 0.5, "max_wall_interval_minus_deadline_ms": 13.087, "wall_intervals_over_budget": 0, "wall_budget_ms": 25.0, "exec_timer_clock_ms_max_injected": 0.0, "exec_wall_ms_max_injected": 22.108, "cgroup_cpu_delta": {"nr_periods": 50, "nr_throttled": 0, "throttled_usec": 0, "usage_usec": 2416551}, "first_clock_wait_s": 0.318, "rtf_seen_by_control_node": 1.0, "rtf_clock_publisher": 1.0, "rtf_clock_publisher_100ms_min": 0.9992, "rtf_clock_publisher_100ms_max": 1.001, "rtf_windows": 49}
{"phase": "sim_rtf1_injected", "timer_clock": "ros_time(use_sim_time)", "target_rtf": 1.0, "inject_ms": 22.0, "inject_every": 50, "period_ms": 10.0, "callbacks": 474, "grid_periods_spanned": 481, "skipped_periods": 8, "injected": 10, "interval_wall_ms": {"mean": 10.505, "min": 1.034, "p99": 22.883, "max": 41.491}, "max_wall_interval_at_s": 2.804, "interval_timer_clock_ms": {"mean": 10.169, "max": 23.0}, "start_lateness_ms": {"mean": 0.376, "max": 14.0}, "response_ms_max": 23.217, "deadline_ms": 10.0, "deadline_misses": 18, "wall_intervals_over_period": 250, "wall_intervals_over_period_plus_jitter": 69, "jitter_tolerance_ms": 0.5, "max_wall_interval_minus_deadline_ms": 31.491, "wall_intervals_over_budget": 2, "wall_budget_ms": 25.0, "exec_timer_clock_ms_max_injected": 0.0, "exec_wall_ms_max_injected": 22.217, "cgroup_cpu_delta": {"nr_periods": 50, "nr_throttled": 5, "throttled_usec": 49643, "usage_usec": 3117128}, "first_clock_wait_s": 0.317, "rtf_seen_by_control_node": 0.968, "rtf_clock_publisher": 0.9678, "rtf_clock_publisher_100ms_min": 0.737, "rtf_clock_publisher_100ms_max": 1.0175, "rtf_windows": 49}

What the numbers say, and what they don’t

The timer behaved as the source predicted. In wall_injected, every run matched the prediction. The mean interval stayed within about a quarter of a millisecond of the period, the longest interval was the injected 22 ms plus a little, and the shortest was about 7.4 ms, the short gap that follows a late callback. Each injection removed one period. The 20 misses are what the rcl rule predicts: 10 injected callbacks plus the 10 callbacks right after them, which each started about 12.5 ms late. If you logged only the mean, you’d see none of those 20 misses.

RTF 1.0 and 20 deadline misses happened in the same 5 seconds. Runs 2 to 5 of sim_rtf1_injected are the direct answer to the question in the title. The /clock process kept its rate, and no 100 ms window fell more than a few percent below 1.0. Only the control node, running in its own process, was late. A dashboard that showed only this RTF would have read about 100 % the whole time.

Run 1 didn’t match the prediction. RTF fell about 3 % below 1.0, and one 100 ms window fell to about three quarters. The same phase is the only sim_rtf1_injected run with CPU throttling recorded. The most likely explanation is that the container used up its CPU quota and the /clock process and the control node were paused together. I didn’t line up the throttling times with the time of the 41.491 ms interval, though, so this is an inference. Even here, RTF moved by a few percent while the worst interval overran the deadline by more than three periods. The size of the RTF change tells you nothing about the size of the overrun.

The RTF 0.5 arithmetic held on average but not at the worst point. In runs 3 and 5 of sim_rtf05, the phase RTF was within about 2 % of 0.5, but the longest wall-clock interval went well past the 25 ms budget, and some 100 ms windows fell to roughly a third or a quarter of real time. No CPU throttling was recorded in those phases, so this data can’t say what caused the gaps. The 10 / 0.5 = 20 ms calculation describes an average step. To know whether each step stayed within a wall-clock budget, you still need the maximum interval.

An interval over the period isn’t the same as a missed deadline. In the baseline, about half the intervals were slightly over 10 ms and there were zero deadline misses. If you judge by intervals, decide your jitter allowance first and write it down next to the result.

Don’t time callbacks with sim time. While the injected callback held the single-threaded executor, the node couldn’t process any /clock messages, so its ROS time stood still. Measured with get_clock().now(), a 22 ms callback took zero time. If your node uses use_sim_time, measure callback execution with a monotonic clock as well.

How to check your own node

Start by recording a monotonic timestamp when each callback starts. This is the recorder from experiment.py:

def __call__(self):
    wall_start = time.monotonic_ns()
    timer_start = self.clock.now().nanoseconds
    injected = self.inject and (len(self.rows) + 1) % INJECT_EVERY == INJECT_EVERY // 2
    if injected:
        time.sleep(INJECT_MS / 1000)
    self.rows.append((wall_start, timer_start, time.monotonic_ns(),
                      self.clock.now().nanoseconds, injected))

In Jazzy, rclpy’s Timer doesn’t pass the expected and actual call times to your callback, so you have to record them yourself like this. To count skipped periods, replay the rcl rule over the recorded start times:

nxt = expected + period_ns
if nxt <= ts:
    ahead = 1 + (ts - nxt) // period_ns
    skipped += ahead
    nxt += ahead * period_ns
expected = nxt

Then report four things next to RTF, never instead of it:

  1. The maximum interval (or maximum response time) between callbacks.
  2. The number of deadline misses, and the rule you used: interval or response time, and how much jitter you allowed.
  3. The number of skipped periods, which is scheduled grid periods minus callbacks that actually ran.
  4. Which clock you measured with: monotonic (wall) or the node’s ROS time.

If you want ROS 2 to tell you about missed deadlines while the system runs, use the deadline QoS policy. The docs define its duration as “the expected maximum amount of time between subsequent messages being published to a topic” (QoS docs, L53-L55). When a subscriber doesn’t receive a message within that time, you get a “Requested deadline missed” event (L308-L310). In rclpy you can register a handler with SubscriptionEventCallbacks(deadline=...) (event_handler.py, L152-L201). This detects misses. It doesn’t prevent them.

If your measurements push you toward a real-time kernel or scheduler priorities, read the official tutorial’s warning first. It says not to set the real-time priority limit (rtprio) to 99, because that can interfere with important system processes such as watchdogs (Understanding real-time programming).

Reproduce it

The script, the plan with its predictions and the raw output of all five runs are in the experiment directory. The numbers in this article come from these files at a fixed commit: experiment.py, plan.md and results.json.

mkdir rtf-vs-deadline && cd rtf-vs-deadline
curl -fsSLO https://raw.githubusercontent.com/sungpyo9053/blog/d3bd73224cf16594e730a7c8de4adbf1e2fc82da/experiments/ros2-real-time-factor-vs-real-time-computing-deadline/experiment.py
sha256sum experiment.py
docker run --rm --network none --cpus 1 --memory 1g -v "$PWD":/work \
  ros@sha256:066420e07f60aa18262f2479981def87ebcfcec42eefb0c0c57c4a46098348ca \
  bash -c "source /opt/ros/jazzy/setup.bash && python3 /work/experiment.py"

If sha256sum prints eefd43cdc31aeef1a5ea597987df4ba6448c85a0191719c8ae4d1d3f72480e89, you have the same script. A normal run exits with code 0 and prints seven JSON lines: one environment line, one per phase and a final done line. If it stops with no /clock received within 15 s, the simulated-time phases never started and there’s nothing to compare. The harness also mounted the directory read-only and used a private /tmp, but the CPU, memory and network limits were the same as above. Note that --cpus 1 is a CPU-time quota, not a dedicated core, which is why the container saw two CPUs. Your numbers will differ on another host. Compare the shape first: skipped periods and deadline misses in wall_injected, and RTF against misses in sim_rtf1_injected.

Limits

  • This is one quota-limited container on a kernel that isn’t PREEMPT_RT, running Python (rclpy), the default RMW and a single-threaded executor, with five runs of 5 seconds per phase. It says nothing about C++ nodes, multi-threaded executors, real-time kernels or real robot hardware.
  • The 22 ms delay was injected on purpose. Natural delays showed up in different runs and phases each time, including a dirty baseline during the trial runs. Five clean baseline runs don’t prove the baseline is always clean in this environment.
  • The /clock publisher here is a stand-in for a simulator. It isn’t Gazebo or Isaac Sim, and their RTF displays smooth and sample the ratio differently, as described above.
  • In the simulated-time phases, response time adds a start delay in sim time to an execution time in wall time. That’s only meaningful when execution is short. Sim-time values are also quantized to the 1 ms /clock step.
  • Nothing here says what miss rate is acceptable for your application, and a deadline check is not a safety certification. Before you move a control loop onto real hardware, confirm that hardware’s own stop mechanisms separately.

Read RTF 1.0 as “the simulation ran at the speed I asked for.” To say the control loop met its deadlines, you need the maximum interval, the miss count, the skipped periods and the clock you used next to it. If those are missing from your log, don’t write “it ran in real time”, even when RTF says 1.0.