Gymnasium check_env “share an object” error in ROS 2 robot envs

한국어로 읽기

What happened when I ran it

Here’s the bug. A custom robot environment keeps its joint angles in one NumPy array, self._q. It updates that array in place (self._q += 0.25) and returns the same array from every step(). The training loop stores each observation with buffer.append(obs). I wanted to know what ends up in the buffer, which tests catch it, and whether the same thing happens when a ROS 2 node reuses one sensor_msgs/JointState message. So I ran a small harness in a ROS 2 Jazzy container. The joint moves 0.25 rad per step and steps are 0.125 s apart. These are made-up numbers picked to avoid float rounding, not a real robot spec.

Environment: image ros:jazzy-ros-base (ros@sha256:066420e07f60aa18262f2479981def87ebcfcec42eefb0c0c57c4a46098348ca), Python 3.12.3, numpy 1.26.4, x86_64, rclpy 7.1.12-1noble.20260902.053513, RMW rmw_fastrtps_cpp (default). No network, 1 CPU, 1 GiB. The same script ran 5 times. All 5 exited with code 0 and took 1.52 s to 1.54 s each. Every buffer, test and ROS 2 value was identical across the 5 runs. Only the timing lines differed.

The container has no Gymnasium installed, so the environment is a stand-in that copies Gymnasium’s reset() -> (obs, info) and step() -> (obs, reward, terminated, truncated, info) return shapes. I tried 4 ways of returning the observation: reuse = self._q, view = self._q[:], asarray = np.asarray(self._q) and copy = self._q.copy(). Each one went into 3 buffers: a list filled for 4 steps, a deque(maxlen=4) filled for 6 steps, and copy_on_store, which stores np.array(obs) for 4 steps the way Stable-Baselines3’s ReplayBuffer.add() does. The true value was recorded separately as a float right after each step.

Check Prediction written before the run (plan.md) Measured (same in all 5 runs)
reuse + list, stored values all four slots 1.0 rad [1.0, 1.0, 1.0, 1.0]
Stored mean / true mean 1.0 / 0.625 rad 1.0 / 0.625
Mean error 0.375 rad 0.375
Oldest-slot error 0.75 rad 0.75
Newest-slot error 0 0.0
Finite-difference velocity from the first pair (stored / true) 0 / 2 rad/s 0.0 / 2.0
view and asarray + list same as reuse stored values and errors identical to reuse
Distinct Python objects in the buffer no prediction reuse 1, view 4, asarray 1, copy 4
np.shares_memory(oldest, newest) no prediction reuse, view, asarray: true; copy: false
reuse + deque(maxlen=4), 6 steps all four 1.5, mean error 0.375, oldest error 0.75 [1.5, 1.5, 1.5, 1.5], 0.375, 0.75
copy + deque(maxlen=4), 6 steps 0.75, 1.0, 1.25, 1.5, mean 1.125 [0.75, 1.0, 1.25, 1.5], 1.125
copy_on_store, all 4 return styles correct values error 0.0, velocity 2.0
“newest slot equals truth” test passes for all 4 styles (misses the bug) passed for all 4
“oldest slot equals truth” test passes only for copy passed only for copy

With reuse + list, I changed the number of steps N:

N Oldest-slot error (rad) Mean error (rad) Newest-slot error (rad)
2 0.25 0.125 0.0
4 0.75 0.375 0.0
8 1.75 0.875 0.0
16 3.75 1.875 0.0
64 15.75 7.875 0.0

ROS 2 JointState: one node had a publisher and a subscription on topic joint_obs (RELIABLE, KEEP_LAST 16). It created one JointState message, added 0.25 rad to position[0] each time and published it 4 times. After each publish, the node appended that same message object to a local list. The subscription callback appended whatever it received to a second list. The prediction was that the local list would show the last value four times and the subscriber would get 0.25, 0.5, 0.75, 1.0.

Where it was recorded Stored values (rad) Distinct objects Oldest-slot error
Publisher’s own list [1.0, 1.0, 1.0, 1.0] 1 0.75
Messages received by the subscription [0.25, 0.5, 0.75, 1.0] 4 0.0

Cost of copying a 7-joint float64 array (20000 calls × 7 samples, median per call). The prediction was less than a microsecond. Across the 5 runs, the per-call medians were 378.8 to 386.1 ns for ndarray.copy, 417.2 to 428.0 ns for np.array, and 55.8 to 57.2 ns for just returning the reference (the call-overhead baseline).

Raw output from run 1, showing only the lines that matter:

{"m": "env", "python": "3.12.3", "numpy": "1.26.4", "machine": "x86_64", "step_rad": 0.25, "dt_s": 0.125, "n_steps": 4}
{"m": "buffer", "env": "reuse", "buffer": "list", "steps": 4, "stored_rad": [1.0, 1.0, 1.0, 1.0], "expected_rad": [0.25, 0.5, 0.75, 1.0], "mean_stored": 1.0, "mean_expected": 0.625, "mean_err": 0.375, "oldest_err": 0.75, "newest_err": 0.0, "fd_vel_stored": 0.0, "fd_vel_expected": 2.0, "distinct_ids": 1, "oldest_shares_mem_newest": true}
{"m": "buffer", "env": "view", "buffer": "list", "steps": 4, "stored_rad": [1.0, 1.0, 1.0, 1.0], "expected_rad": [0.25, 0.5, 0.75, 1.0], "mean_stored": 1.0, "mean_expected": 0.625, "mean_err": 0.375, "oldest_err": 0.75, "newest_err": 0.0, "fd_vel_stored": 0.0, "fd_vel_expected": 2.0, "distinct_ids": 4, "oldest_shares_mem_newest": true}
{"m": "buffer", "env": "copy", "buffer": "list", "steps": 4, "stored_rad": [0.25, 0.5, 0.75, 1.0], "expected_rad": [0.25, 0.5, 0.75, 1.0], "mean_stored": 0.625, "mean_expected": 0.625, "mean_err": 0.0, "oldest_err": 0.0, "newest_err": 0.0, "fd_vel_stored": 2.0, "fd_vel_expected": 2.0, "distinct_ids": 4, "oldest_shares_mem_newest": false}
{"m": "unit_test", "env": "reuse", "last_only_pass": true, "oldest_pass": false, "all_pass": false}
{"m": "unit_test", "env": "copy", "last_only_pass": true, "oldest_pass": true, "all_pass": true}
{"m": "ros2_jointstate", "position_type": "array", "matched": 1, "expected_rad": [0.25, 0.5, 0.75, 1.0], "pub_local_rad": [1.0, 1.0, 1.0, 1.0], "pub_local_ids": 1, "pub_local_oldest_err": 0.75, "sub_count": 4, "sub_rad": [0.25, 0.5, 0.75, 1.0], "sub_ids": 4, "sub_oldest_err": 0.0, "sub_matches": true}

What the numbers mean

Every prediction held. If the environment hands back the same array every step, a buffer that stores references has nothing but the latest value in every slot. In the 4-step case, the joint actually moved through 0.25, 0.5, 0.75 and 1.0 rad. The buffer says it sat at 1.0 rad the whole time. The velocity you’d compute from two stored observations drops from 2 rad/s to 0. If your policy or your analysis estimates joint velocity from consecutive observations, that information is gone.

The error isn’t spread evenly. It’s largest in the oldest slot and zero in the newest. That’s why a test that checks only the latest observation passes no matter how the env returns data. In the N table, the oldest-slot error follows 0.25 × (N − 1) and the mean error follows 0.125 × (N − 1). These formulas are my fit to the measured rows, not something the script printed. So in a plain list, the error grows with buffer length. A fixed window like deque(maxlen=4) doesn’t build up error over time. Instead, the whole window always shows one single moment. Frame stacks have exactly that shape.

The view row is the one that trips people up. self._q[:] creates a new Python object each call, so the buffer holds 4 distinct objects and an id()-based check says everything is fine. But all 4 share memory with the env’s array, and the stored values are just as wrong as with reuse. Counting objects can’t detect this. You have to check memory sharing or values.

Copying is cheap at this size. Subtracting the reference baseline from the ndarray.copy median gives about 323 to 329 ns per call in each run. That’s a microbenchmark on one shared container CPU. Compare it against your own step time rather than treating it as a general figure.

Why every slot ends up with the last value

A Python list slot doesn’t hold a value. It points to an object. Neither y = x nor list.append(obs) copies anything (Python FAQ: “Why did changing list ‘y’ also change list ‘x’?”). self._q += 0.25 changes the existing array instead of making a new one. Put the two together and all four slots point at one array, which holds 1.0 after the fourth step.

The two common “fixes” don’t copy either:

  • self._q[:] is basic indexing, and basic indexing always creates views. A view shares the original’s data buffer.
  • np.asarray(a): “No copy is performed if the input is already an ndarray with matching dtype and order” (numpy.asarray). A float64 array comes straight back as the same object.

self._q.copy() or np.array(self._q) allocate a new data buffer. To check whether two arrays overlap, use np.shares_memory(a, b). Gymnasium’s own checker uses that function.

Gymnasium documents this requirement in Env.step(): “Users commonly keep the data from every timestep, most obviously in a replay or rollout buffer, so each call must return new observation and info objects rather than modifying and returning the same buffer.” (Gymnasium Env API, core.py L117–L120 at v1.4.0). reset() has the same rule (core.py L165–L167).

Where this shows up in ROS 2 code

If you wrap a ROS 2 robot in a Gymnasium environment, the easy design is a JointState callback that writes into a preallocated array, plus a step() that returns that array. This sketch shows the pattern. I didn’t run it. It follows the same structure as the measured reuse case:

def _on_joint_state(self, msg):
    self._q[:] = msg.position      # overwrites the same array in place

def step(self, action):
    self._send_command(action)
    self._wait_for_next_state()
    return self._q, reward, terminated, truncated, {}   # bug: same object every step
    # fix: return self._q.copy(), reward, terminated, truncated, {}

The JointState measurement covers the other common case: reusing one message object and also logging it. On the publisher side, the local list ended up with the same object four times and the oldest entry was off by 0.75 rad. In rclpy, position came back as an array (Python’s array.array), which is mutable. Changing position[0] changes every logged reference. The subscriber, though, got four separate messages with the correct values. That fits the idea that the value is captured when you call publish(). But I didn’t measure when serialization happens. I tested only a single rclpy process with the default RMW. Intra-process communication in C++ and other RMWs weren’t tested, so don’t read this as “ROS 2 messages are always safe”. The rule is the same as for the Gymnasium env: if you keep your own history of an object you keep mutating, copy before you store.

The Gymnasium check_env “share an object” error

The container had no Gymnasium, so the checker results in this section come from a separate check. I made fresh virtualenvs on Ubuntu with Python 3.12.3 and numpy 2.5.3, then installed gymnasium==1.4.0 in one and gymnasium==1.3.0 in the other. I ran the same one-joint env through check_env(env, skip_render_check=True). These are pass/fail results, not timed measurements.

On Gymnasium 1.4.0, check_env() raised this AssertionError for reuse, view and asarray. Only copy passed:

AssertionError: The observations returned by `env.reset() (call 1)` and `env.step() (call 2)` share an object, therefore, modifying one will modify the other. Environments must return new observation data on every call as users keep the data returned to them.

Five things to know about this check:

  1. It only exists from Gymnasium 1.4.0. The docs say “gymnasium.utils.env_checker.check_env() checks this”, but the check (check_returned_data_not_reused) came in with PR #1683 and first shipped in v1.4.0, released on PyPI on 2026-10-05. On 1.3.0, all four return styles passed check_env(), including reuse. If you’re on 1.3.0 or older, passing check_env() tells you nothing about this bug.
  2. It compares identity, not values. It calls reset, up to two steps, then reset again, and checks every pair with np.shares_memory for arrays, matching keys for dicts, and element by element for tuples (env_checker.py L273–L307, passive_env_checker.py L179–L218). The docstring says: “an array or dictionary that an environment intends to be constant must also be copied”. That’s why [:] and np.asarray fail.
  3. A failure doesn’t always mean corrupted data. If you return a constant array, such as a fixed goal position, without copying it, nothing is wrong with the values, but the check still fails. The official Create a Custom Environment tutorial’s GridWorldEnv puts its internal target array straight into the Dict observation. When I pasted the page’s code and ran it, 1.4.0 raised the same “share an object” error, while 1.3.0 printed “Environment passes all checks!”. For Dict observations, copy every array, constants included.
  4. gym.make() only warns. The passive checker that gym.make() adds compares just the first reset→step and step→step pairs, once each, and logs a warning (wrappers/common.py L284–L311, passive_env_checker.py L221–L247). On 1.4.0 I saw two warnings, but execution continued and the buffer was still corrupted. On 1.3.0, gym.make() gave no warning at all. Going by the source, an env that only starts reusing after those first pairs would get no warning. I didn’t test that case.
  5. The v1.4.0 release notes say this check found and fixed two reused info elements in Gymnasium’s own Humanoid and HumanoidStandup. Info dicts are checked too.

The fix, and a test that catches it on any Gymnasium version

In this order:

  1. Return a fresh array from reset() and step(): return self._q.copy(). For Dict observations, copy each array, including ones that never change. Don’t rely on [:] or np.asarray().
  2. Add a test that collects at least two steps without copying and compares the oldest entry with the first step’s real value. It doesn’t depend on Gymnasium’s version, and it catches what a newest-only test misses.
  3. If you’re on Gymnasium 1.4.0 or later, put check_env() in CI. Read a failure as “same object returned twice”, not as proof that the values are already wrong.

joint_buffer_check.py does steps 2 and 3 on a real gymnasium.Env. It ran in the separate virtualenv described above (Python 3.12.3, gymnasium 1.4.0, numpy 2.5.3), not in the container. Gymnasium 1.4.0 requires Python 3.10 or later.

python3 -m venv .venv
.venv/bin/python -m pip install "gymnasium==1.4.0"
import sys

import numpy as np
import gymnasium as gym
from gymnasium import spaces
from gymnasium.utils.env_checker import check_env


class OneJointEnv(gym.Env):
    """Toy env: one joint that moves 0.25 rad every step."""

    def __init__(self, copy_obs=True):
        self.copy_obs = copy_obs
        self.observation_space = spaces.Box(-10.0, 10.0, shape=(1,), dtype=np.float64)
        self.action_space = spaces.Box(-1.0, 1.0, shape=(1,), dtype=np.float32)
        self._q = np.zeros(1, dtype=np.float64)

    def _get_obs(self):
        return self._q.copy() if self.copy_obs else self._q

    def reset(self, *, seed=None, options=None):
        super().reset(seed=seed)
        self._q[:] = 0.0
        return self._get_obs(), {}

    def step(self, action):
        self._q += 0.25  # toy dynamics: +0.25 rad per step, ignores the action
        return self._get_obs(), 0.0, False, False, {}


def oldest_survives(env):
    env.reset(seed=0)
    buffer = []
    for _ in range(4):
        obs, *_ = env.step(env.action_space.sample())
        buffer.append(obs)  # naive buffer: stores references, no copy
    return float(buffer[0][0]) == 0.25


copy_obs = sys.argv[1:] != ["reuse"]
ok = oldest_survives(OneJointEnv(copy_obs))
print("oldest element test:", "PASS" if ok else "FAIL")
try:
    check_env(OneJointEnv(copy_obs), skip_render_check=True)
    print("check_env: PASS")
except AssertionError as e:
    ok = False
    print("check_env: FAIL -", str(e).split(",")[0])
sys.exit(0 if ok else 1)

.venv/bin/python joint_buffer_check.py printed PASS twice and exited 0. .venv/bin/python joint_buffer_check.py reuse exited 1 with:

oldest element test: FAIL
check_env: FAIL - The observations returned by `env.reset() (call 1)` and `env.step() (call 2)` share an object

On Gymnasium 1.3.0, the reuse run printed FAIL for the oldest-element test and PASS for check_env. The oldest-element test is the one that caught the bug on both versions. In your own env, replace 0.25 with the actual first observation, saved as a plain number.

Does your buffer copy for you?

Whether this bug hurts you depends on who stores the data.

  • Stable-Baselines3 2.9.0 ReplayBuffer.add() copies with np.array(obs) under the comment “Copy to avoid modification by reference” (buffers.py L265–L271). That’s the copy_on_store case above, which stayed correct for every return style. I only read the source and didn’t run SB3, so this doesn’t cover every SB3 code path. Your own logging lists and evaluation loops aren’t covered either.
  • Gymnasium’s FrameStackObservation appends incoming observations to a deque by reference (stateful_observation.py L415–L436). In the separate virtualenv check, wrapping the reuse env with stack_size=4 left all four stacked frames at the last value on both 1.3.0 and 1.4.0. Your policy loses the motion it was supposed to see. That’s not a wrapper bug. The wrapper assumes the env follows the new-object rule.
  • PyTorch: torch.tensor(obs) “Constructs a tensor … by copying data” (_torch_docs.py L9587). torch.from_numpy(obs): “The returned tensor and ndarray share the same memory” (L4606–L4607). A hand-written DQN replay memory that goes through torch.tensor is safe only because of that copy. Switch to from_numpy for speed and the protection is gone. Docstrings only. I didn’t run PyTorch.

Limits

  • The harness used a stand-in env with one joint, a float64 Box observation and made-up dynamics. No physics simulator or real robot was involved, and I didn’t measure any effect on training such as reward curves or convergence.
  • The check_env, gym.make(), GridWorld and FrameStackObservation results came from a separate virtualenv check with numpy 2.5.3, not from the container, which had numpy 1.26.4. View, asarray and copy behave the same way in both versions according to the NumPy docs.
  • The ROS 2 result covers one rclpy process with rmw_fastrtps_cpp only.
  • I didn’t check other RL libraries (RLlib, TorchRL, CleanRL, d3rlpy), Dict observations where only some keys are reused, or what reused info actually breaks.
  • Gymnasium 1.4.0 was three days old when I checked. Later patches may change the check or its message.
  • Corrupted joint logs can lead to wrong conclusions, like “the joint was stationary here”. These tests check software data integrity. They don’t replace velocity limits, workspace limits or a hardware E-stop on a real robot.

Reproduce it

The script, plan and raw output for all 5 runs are in the reproduction folder. The exact files cited here are pinned at commit 486ac73: experiment.py, plan.md, results.json. On Linux with Docker:

curl -fsSLO https://raw.githubusercontent.com/sungpyo9053/blog/486ac73c4af035cb084672fec88e5e90958c7ba5/experiments/gymnasium-step-observation-buffer-reuse-replay-aliasing/experiment.py
sha256sum experiment.py   # expect c9b2e2879c63a3d9ba1db30064eb2b3c866884cefbf631a807cb58db9a935e86
docker run --rm --network none --cpus 1 --memory 1g -v "$PWD":/work \
  ros@sha256:066420e07f60aa18262f2479981def87ebcfcec42eefb0c0c57c4a46098348ca \
  bash -c "source /opt/ros/jazzy/setup.bash && python3 /work/experiment.py"

You should get one JSON line per measurement and exit code 0. The first line shows the container’s Python and numpy versions.

Sources