A camera frame and a joint reading that arrive together were not taken together. They travel separate paths with different delays, so pairing the newest of each puts them tens of milliseconds apart, and the gap is worst exactly when the robot is moving. Torq lines them up for you. Leader.observation() hands back a frame along with the joint positions from the instant that frame was taken.

​
How it works

1

The follower timestamps everything

Camera frames and joint readings are stamped as they are captured, against one clock on one machine.
2

It tells the leader where that clock started

Announced once with the descriptor, so the leader can place incoming video on the same timeline.
3

The leader holds each frame briefly

Long enough for joint readings from just after the frame to arrive.
4

Joints are calculated for the frame's own instant

Between the reading before it and the reading after, rather than taking whichever was newest.
Until the first frame arrives you get joints and no images. That is normal, and it is how you know video is not up yet.

​
Buffer

sync_buffer_ms is how many milliseconds a frame waits before you get it. Use 0 for a live view, where the freshest picture matters more than exact joint values. Leave the default when you are recording.

​
Alignment quality

Alignment is not always perfect. A frame can arrive with no reading after it, or the nearest reading can be far away, and every observation reports which happened. This is what lets you check a recording before training on it instead of trusting it. These add up across an episode and are written into the recording at end_episode(), so you can check a whole session at once rather than row by row.
A large interpolated_ms across most rows means the nearest reading was far from the frame, so those joint values are less trustworthy than they look.

Leader

Reading a synced observation and checking its quality.

Recording

What a synced observation looks like on disk.

Types

Every field on a synced observation.