베이지 배경 위 두 막대 가운데 위쪽에는 갈색 구슬 세 개와 투명 구슬 한 개가 네 개씩 묶여 있고 아래쪽에는 갈색 구슬만 세 개씩 묶여 있으며, 빠진 투명 구슬들이 앞쪽 녹색 접시에 모여 있는 개념 그림

Transform PointCloud2 frame in ROS 2 Python, keep intensity

한국어로 읽기

Predictions vs. measurements — measured results table
At a glance: the main measured table from this article. Full details below.

What happened when we ran it

The usual situation: a lidar or depth camera driver publishes a sensor_msgs/PointCloud2 in its own frame (laser here), and you want the same points in base_link. In an rclpy callback the obvious code is to pull xyz out with read_points_numpy, apply the rotation and translation, and pack the result back up with sensor_msgs_py.point_cloud2.create_cloud, passing the original msg.header. How read_points_numpy itself returns the points is covered in ROS 2 Python PointCloud2 to xyz NumPy array.

We ran that code, plus the official alternatives, on ROS 2 Jazzy in a container. The input was a synthetic cloud with x, y, z and intensity. The coordinates came out right. The message didn’t: intensity was gone, each point had shrunk, and header.frame_id still said laser. Nothing raised an error. The official tf2_sensor_msgs.do_transform_cloud kept every field and set the target frame, but it crashed on a cloud with padding bytes at the end of each point. The raw results come first, then what they mean.

Environment

  • Image: ros:jazzy-ros-base, pinned as ros@sha256:8eed0a4669bbf43ee331878a4c39bd8ebddea8e3312a58b4fae8edb14526b617
  • Python 3.12.3, NumPy 1.26.4, sensor_msgs_py 5.3.8, tf2_sensor_msgs 0.36.23, tf2_ros_py 0.36.23, rclpy 7.1.12-1noble.20260912.162354 (package versions read from package.xml at run time)
  • RMW: rmw_fastrtps_cpp (default)
  • No network, 1 CPU, 1 GiB memory
  • The harness ran the same script 5 times. All 5 exited with code 0 and took 1.87–1.91 s each. Apart from timing fields, every measured value was identical in all 5 runs.

Input cloud, packed by hand with a NumPy structured array (not with create_cloud, so the function under test didn’t build its own input): 1000 points, fields x, y, z, intensity, all float32 at offsets 0, 4, 8, 12, point_step 16, len(data) 16000, frame_id laser, stamp [1700000000, 123456789], is_dense=True, a different intensity value for every point, point 0 = (1.0, 0.5, 0.0). Transform: parent base_link, child laser, translation [0.25, 0.0, 0.125], rotation z = w = 0.7071067811865476 (+90° about z).

Predictions vs. measurements

The predictions were written in the plan from the Jazzy source before anything ran.

Case What was done Predicted Measured
A read_points_numpy xyz → R·p + t → create_cloud(cloud.header, xyz_fields, arr) fields x, y, z; point_step 12; len(data) 12000 (75% of original); frame_id laser; no exception fields x, y, z; point_step 12; len(data) 12000; ratio to original 0.75; frame_id laser; point 0 = (-0.25, 1.0, 0.125); no exception
A′ Read case A’s output at the old intensity offset 12 Every compared point returns the next point’s x 999 of 999 compared points returned the next point’s x. Old offset for point 2 = byte 36; point 3’s x starts at byte 36
A2 Set frame_id = 'base_link' on case A’s output Shared Header object, so the original changes too Output frame_id base_link; original message’s frame_id afterwards: base_link
B N×3 xyz array + the original 4 fields Exception (3 columns vs. 4 fields) ValueError: The length of the last dimension of arr must be equal to the number of fields in dtype
C read_points(cloud).copy(), replace x/y/z, new Header (same stamp, base_link), original fields, point_step=cloud.point_step 4 fields; point_step 16; len(data) 16000; intensity unchanged; frame_id base_link; original untouched 4 fields; point_step 16; len(data) 16000; intensity identical; frame_id base_link; stamp kept; original untouched
C0 Write to the read_points result without copying Original data changes read_points array writeable, shape [1000]; original data changed
D do_transform_cloud(cloud, transform_stamped) Same as C; original data and header untouched 4 fields; point_step 16; len(data) 16000; intensity identical; frame_id base_link; original untouched; not byte-identical to C; max xyz difference vs. C 5.960464477539063e-08 m
D2 do_transform_cloud(cloud, transform_stamped.transform) (plain Transform) frame_id stays laser frame_id laser; coordinates within tolerance
E Same 4 fields, padded to point_step 32, through do_transform_cloud AssertionError on 5.3.8 AssertionError: PointFields and structured NumPy array dtype do not match for all fields!
E2 Same padded cloud, rebuilt as in C Success; point_step 32; len(data) 32000 Success; point_step 32; len(data) 32000; intensity identical
E3 Same padded cloud, xyz-only rebuild as in A point_step 12; len(data) 12000 (37.5%); padding gone too point_step 12; len(data) 12000; ratio 0.375; frame_id laser
F Organized cloud, width 4 × height 3, through do_transform_cloud height 1, width 12, len(data) 192 unchanged height 1; width 12; len(data) 192, same as original
G import tf2_sensor_msgs, then tf2_ros.Buffer.transform(cloud, 'base_link') frame_id base_link; fields kept frame_id base_link; 4 fields; intensity identical
G0 Same call in a fresh python3 process that never imports tf2_sensor_msgs Exception: no PointCloud2 handler registered tf2_sensor_msgs not in sys.modules; TypeException: Type <class 'sensor_msgs.msg._point_cloud2.PointCloud2'> is not loaded or supported
H Over real topics: static transform on /tf_static, cloud on /cloud_laser, a node calls Buffer.transform and publishes /cloud_base, a checker node receives it Same as G after serialization 4 fields; point_step 16; len(data) 16000; frame_id base_link; intensity identical; transform available after 0.067 s, message received after 0.069–0.07 s

Coordinates were checked against the hand formula for +90° about z, (x, y) → (−y, x) plus t, with a tolerance of 1e-6 m. The maximum absolute error was 5.960464477539063e-08 m in A, C, D, D2, E2, G and H, and 6.984919309616089e-09 m in F. All passed. The input had is_dense=True, and every output message had is_dense false. Every output kept the stamp [1700000000, 123456789].

Raw output (run 1)

The environment line and cases A, E and G0, copied unchanged:

{"kind": "env", "python": "3.12.3", "numpy": "1.26.4", "sensor_msgs_py": "5.3.8", "tf2_sensor_msgs": "0.36.23", "tf2_ros_py": "0.36.23", "tf2_sensor_msgs_file": "/opt/ros/jazzy/lib/python3.12/site-packages/tf2_sensor_msgs/__init__.py", "byteorder": "little"}
{"kind": "case", "case": "A_xyz_only_original_header", "ok": true, "ms": 1.19, "fields": ["x", "y", "z"], "offsets": [0, 4, 8], "point_step": 12, "row_step": 12000, "height": 1, "width": 1000, "len_data": 12000, "w_h_step": 12000, "frame_id": "laser", "stamp": [1700000000, 123456789], "is_dense": false, "len_matches_w_h_step": true, "len_ratio_vs_original": 0.75, "same_field_names": false, "same_shape": true, "point0": [-0.25, 1.0, 0.125], "max_abs_err_vs_hand": 5.960464477539063e-08, "within_tol": true, "intensity_field_present": false, "header_is_same_object": true, "original_unchanged": true, "old_offset12_byte_for_point2": 36, "point3_x_byte": 36, "old_offset12_equals_next_x_count": 999, "old_offset12_compared": 999}
{"kind": "case", "case": "E_do_transform_cloud_padded32", "ok": false, "ms": 0.67, "exception": "AssertionError", "message": "PointFields and structured NumPy array dtype do not match for all fields!                     Check their field order, names and types."}
{"kind": "case", "case": "G0_buffer_transform_without_import", "ms": 265.77, "tf2_sensor_msgs_loaded": false, "ok": false, "exception": "TypeException", "message": "Type <class 'sensor_msgs.msg._point_cloud2.PointCloud2'> is not loaded or supported"}

Every case came out the way the source code predicted, so sensor_msgs_py 5.3.8 and tf2_sensor_msgs 0.36.23 behave as their code reads. In short:

  • Packing xyz back with only x/y/z fields drops intensity without an error. Each point goes from 16 to 12 bytes, so the message loses a quarter of its data, 4 bytes × 1000 points. Those 4000 bytes were the intensity.
  • create_cloud doesn’t touch the header. If you pass msg.header, the base_link numbers still carry the laser label. If you then fix frame_id on the output, you also change the input message, because the two share one Header object.
  • On Jazzy, Buffer.transform / do_transform_cloud are the short path, as long as you import tf2_sensor_msgs and each point has no padding after its last field.
  • If each point does have trailing padding, do_transform_cloud fails with PointFields and structured NumPy array dtype do not match for all fields! on sensor_msgs_py 5.3.8. Rebuilding with the original fields and point_step=cloud.point_step works.

Why intensity disappears: create_cloud only knows the fields you pass

A PointCloud2 stores all points one after another in a single byte array, data. A separate list, fields, says where each value sits inside a point. Each PointField has a name, an offset (bytes from the start of the point), a datatype and a count. point_step is the size of one point in bytes, row_step the size of one row, and data is row_step × height bytes (PointCloud2.msg, PointField.msg).

In the Jazzy source, create_cloud(header, fields, points, point_step=None) builds the message entirely from its arguments (point_cloud2.py L283-L304):

  • fields is copied in as given.
  • point_step becomes points.dtype.itemsize, and row_step is that times width.
  • A 1-D array gives height = 1, width = number of points. is_dense is always set to False.
  • header is the object you passed, not a copy.

Pass three float32 fields at offsets 0, 4 and 8 and the dtype ends at byte 8 + 4 = 12. That’s the new point_step. The function isn’t deleting intensity. It is building a message from a field list that never had intensity in it.

The quiet part is what happens downstream. Code that still reads intensity at offset 12 now reads byte 2 × 12 + 12 = 36 for point 2, and that is exactly where point 3’s x begins (3 × 12 = 36). Case A′ confirmed it for every point. You get plausible float values, not an exception.

Case B is the other common attempt: keep the N×3 array but pass the original four fields. NumPy can’t map 3 columns onto 4 named fields, so it raises a ValueError. At least that one fails loudly.

Why frame_id stays “laser”

header.frame_id tells every consumer which frame the numbers are in. Since create_cloud uses the header you give it, passing msg.header keeps laser. The ROS 2 tf2 docs recommend the transform<T>(target_frame, ...) methods because they “read the source frame_id from the datatype and write out the target frame_id in the resulting datatype” (About Tf2, Jazzy). When you do the math yourself, setting the target frame is your job.

A cloud whose numbers are in base_link but whose label says laser will probably get transformed a second time by RViz or the next node, so the points end up drawn in the wrong place. That comes from reading the code. The experiment didn’t render anything to check it.

Fixing the label after the fact has its own trap (case A2). out.header is msg.header, so out.header.frame_id = 'base_link' also relabels the input message. Build a new Header(stamp=msg.header.stamp, frame_id='base_link') instead. That’s what do_transform_cloud does, with a source comment saying it creates a new Header “so original header is not altered” (tf2_sensor_msgs.py L105-L113).

The short path: Buffer.transform with import tf2_sensor_msgs

tf2_ros.Buffer.transform(obj, target_frame) reads obj.header.frame_id and obj.header.stamp, calls lookup_transform(target_frame, source_frame, stamp), and hands the result to whatever do_transform function is registered for that message type (buffer_interface.py L74-L102). The PointCloud2 handler is registered as a side effect of importing tf2_sensor_msgs. Without that import you get case G0’s TypeException: ... PointCloud2 ... is not loaded or supported, which reads like a missing package rather than a missing import.

A trimmed version of what the transformer node in case H did (full code in experiment.py):

import tf2_sensor_msgs  # noqa: F401  registers the PointCloud2 handler with tf2_ros
from rclpy.duration import Duration
from tf2_ros import Buffer, TransformListener

buf = Buffer()
TransformListener(buf, node)
pub_base = node.create_publisher(PointCloud2, 'cloud_base', 10)

def on_laser(msg):
    pub_base.publish(buf.transform(msg, 'base_link', timeout=Duration(seconds=0)))

node.create_subscription(PointCloud2, 'cloud_laser', on_laser, 10)

If you already have the TransformStamped from lookup_transform('base_link', 'laser', t), call from tf2_sensor_msgs import do_transform_cloud and then do_transform_cloud(msg, ts). Pass the whole TransformStamped. The function only writes the new frame_id when it gets a stamped transform, because a plain Transform carries no frame name. Case D2 shows the result: correct coordinates, still labelled laser.

A few more things the official function does, all visible in the table:

  • It rotates and translates x, y and z only. Its docstring says “the rest of the cloud is kept untouched”. If your cloud has normals, rotate them yourself (rotation only). Viewpoint fields (vp_x, vp_y, vp_z) are positions, so they need rotation and translation. Our cloud had neither field, so this point comes from the source, not from a measurement.
  • Organized clouds come back flattened to height = 1 (case F), because read_points returns a 1-D array. The byte count stays the same, so a len(data) check alone won’t catch it. Anything that looks points up by pixel index will break.
  • is_dense is always False in the output.

One outdated answer is still around: an older Robotics Stack Exchange answer says you need Iron to import do_transform_cloud in Python. The NumPy implementation was also merged into the Humble branch in April 2025 (geometry2 PR #778). We didn’t check which Humble release first shipped it.

“PointFields and structured NumPy array dtype do not match” on Jazzy

Some drivers pad every point to an aligned size, so point_step is larger than the end of the last field. Case E used the same four float32 fields padded to 32 bytes per point. do_transform_cloud reads the cloud with read_points, which keeps the real 32-byte item size, and then calls create_cloud(new_header, cloud.fields, points) without point_step. Jazzy’s create_cloud then asserts that the array’s dtype equals dtype_from_fields(fields, None) (point_cloud2.py L268-L271). Without a point_step, that dtype stops at the end of the last field (16 bytes here, not 32), so the comparison fails.

  • Reported for Jazzy in common_interfaces issue #305. A comment there reports the same failure on Humble. The issue title blames read_points(). PR #324 explains that the actual cause is the comparison in create_cloud().
  • Fixed in Rolling by PR #295, which falls back to points.dtype.itemsize when no point_step is given.
  • The Jazzy backport, PR #326, was still open and unmerged when we checked on 2026-11-02. Once it ships, case E may start passing. Check the sensor_msgs_py version in your image.

The workaround is the point_step argument that create_cloud gained in PR #223, which was added because of exactly this kind of padded lidar layout. Case E2 shows that passing it keeps the padded 32-byte layout intact.

Rebuilding by hand, with checks before you publish

This function is the one cases C and E2 ran. xyz_target is an N×3 array already in the target frame. In the experiment it came from turning the TransformStamped quaternion into a rotation matrix and computing R·p + t (manual_xyz in experiment.py).

from std_msgs.msg import Header
from sensor_msgs_py import point_cloud2 as pc2

def rebuild_in_target(cloud, xyz_target, target_frame):
    pts = pc2.read_points(cloud).copy()          # read_points is a view: copy, or you edit the input
    pts['x'], pts['y'], pts['z'] = xyz_target.T  # replace xyz only; intensity etc. stay as they are
    header = Header(stamp=cloud.header.stamp, frame_id=target_frame)
    out = pc2.create_cloud(header, cloud.fields, pts, point_step=cloud.point_step)
    assert [f.name for f in out.fields] == [f.name for f in cloud.fields]
    assert out.point_step == cloud.point_step
    assert (out.height, out.width) == (1, cloud.width * cloud.height)  # read_points is 1-D
    assert len(out.data) == len(cloud.data) == out.width * out.height * out.point_step
    assert out.header.frame_id == target_frame
    return out

Each part of the function maps to a case:

  • .copy(): read_points returns a NumPy array laid directly over the message’s bytes. Writing to it without a copy changed the original message in case C0. do_transform_cloud copies for the same reason.
  • New Header: keeps the observation stamp, sets the target frame, and leaves the input’s header alone (case A2).
  • cloud.fields + point_step=cloud.point_step: keeps intensity and any end padding (cases C and E2).
  • The asserts are the pre-publish check: same field names, same point_step, the shape change made explicit, len(data) equal to the original and to width × height × point_step, and the target frame_id. Comparing field names catches case A. Comparing frame_id catches A and D2. The shape check is there because the byte count alone misses flattening.

D and C agreed on coordinates within tolerance but were not byte-for-byte identical. Our reading is that C used the script’s own rotation matrix and D used the one inside tf2_sensor_msgs, so the float rounding differs slightly. That’s why the comparison uses coordinate differences instead of byte equality.

Which one to use on Jazzy

  1. Compare point_step with the end of the last field (largest offset + its size). If they match, use Buffer.transform(msg, 'base_link') after import tf2_sensor_msgs, or do_transform_cloud(msg, transform_stamped).
  2. If point_step is larger (trailing padding) and you’re on sensor_msgs_py 5.3.8, the official function stops with the AssertionError. Rebuild with the original fields and point_step=cloud.point_step, as above.
  3. Whichever path you use, check fields, point_step, len(data), shape and frame_id against the input before publishing.

Reproduce it

The script, plan and the raw output of all 5 runs are in experiments/ros2-python-transform-pointcloud2-frame-keep-fields: experiment.py, plan.md (written in Korean) and results.json. In an empty directory with Docker:

curl -fsSLO https://raw.githubusercontent.com/sungpyo9053/blog/7a50adac5d6acc7780adf41264d6f100680813c2/experiments/ros2-python-transform-pointcloud2-frame-keep-fields/experiment.py
sha256sum experiment.py  # expect 32157cff61b951007b7912d459e4ba1afd175f473b587064eb4b5c3bae651474
docker run --rm --network none --cpus 1 --memory 1g -v "$PWD":/work \
  ros@sha256:8eed0a4669bbf43ee331878a4c39bd8ebddea8e3312a58b4fae8edb14526b617 \
  bash -c "source /opt/ros/jazzy/setup.bash && python3 /work/experiment.py"

A matching run exits with code 0. The case A line shows "point_step": 12 and "frame_id": "laser", and the case E line shows "ok": false with "exception": "AssertionError". The ms and other timing values will differ from run to run.

Limits

  • Synthetic messages in a container. Nothing here was checked against a real lidar, depth camera or robot.
  • One rotation (+90° about z), one float32 four-field layout, one padded size (32 bytes per point) and one organized size (4×3).
  • Not tested: row padding (row_step > width × point_step), where the len(cloud.data) assert in the function above would likely fail; NaN points; normal or viewpoint fields; looking up the transform at the message stamp when the transform is changing over time; C++ tf2::doTransform and pcl_ros; performance. The timings in the table come from one 1-CPU container and are not a benchmark.
  • The padding result applies to sensor_msgs_py 5.3.8 and tf2_sensor_msgs 0.36.23. It can change once the Jazzy backport is released.
  • A cloud with the wrong frame_id can put obstacles in the wrong place for perception or mapping nodes. Before connecting to a real robot, replay a rosbag or use a simulator and overlay the original and transformed clouds in RViz. The first time you run it on hardware, keep the robot stationary with the emergency stop within reach.