OPS-04 · RobotOps: Running a Robot Fleet

Fleet Telemetry and Observability: What Every Robot Should Stream Before It Fails

Fleet telemetry and observability for robot fleets: the six signals every robot should stream, how to normalize mixed vendors, and how to catch drift early.

FLEET UPTIME · 30D4 UNITS ONLINEOPS-04 · FLEET CONSOLE

Every robot in a fleet should stream the same six signal families, on a fixed schema, whether it is a six-axis arm bolted to the floor or an autonomous mobile robot crossing an aisle: power and battery state (charge, voltage, cell temperature, health), drive and joint health (motor current, joint temperature, following error), task and mission state (what it is doing, progress, success or failure), localization confidence (how sure it is of its own pose), safety and near-miss events (protective stops, field violations, emergency stops, speed reductions), and structured error and warning codes. Everything else is detail. If those six arrive continuously — normalized to one schema, timestamped, and tagged with a unit ID — you can run a fleet. If any one is missing, you are flying that part of the fleet blind, and you will find out which part at the worst possible moment.

That is the whole thesis of fleet telemetry, and the rest is engineering: getting those signals off machines built by different vendors, into one shape, and onto a dashboard that shows the fleet rather than one robot at a time. This is a field briefing, not a tooling pitch. Where a claim rests on a standard, we name the standard; where the industry is selling a concept borrowed from software, we say so plainly. It is one layer of the broader discipline covered in our RobotOps explainer.

What exactly should every robot stream?

The point of fixing the list is that it makes the fleet comparable. When fifty robots all report the same six families in the same units, you can rank them, diff them, and spot the one that is drifting away from the pack. The table below is the canonical set, why each one earns its bandwidth, and — for mobile robots — where it already maps in a real, published standard.

Signal family What the robot reports Why it matters VDA5050 state-message analogue
Power / battery State of charge, voltage, cell temperature, battery health, charging flag Range, forced-charge scheduling, thermal risk powerSupply (charge, voltage, health, charging; renamed from batteryState)
Drive / joint health Motor current, joint temperature, following/position error, torque Earliest signature of mechanical wear vendor-specific (carried in info/error fields)
Task / mission state Current action, order progress, completed vs failed Throughput, stuck-task detection actionStates (SCHEDULED / RUNNING / FINISHED / FAILED)
Localization confidence Pose estimate plus a confidence score, map ID A robot unsure of where it is, is a safety problem agvPosition.localizationScore, positionInitialized
Safety / near-miss Protective stops, field violations, e-stop state, speed cuts Leading indicator of collisions and layout faults safetyState (eStop, fieldViolation)
Errors / warnings Coded errors and warnings with severity Triage, and cross-fleet pattern detection errors[] (empty array = healthy)

Two of these are chronically under-reported in the field. Localization confidence is the one operators most often forget to stream: a robot that is 60% sure where it is will still move, and it is far cheaper to catch that as a falling score than as a collision. Near-miss events are the other. A protective stop that never became an incident is exactly the leading indicator you want, and it is usually discarded because nothing “went wrong.”

How do you normalize telemetry from different vendors?

Mixed fleets do not agree on words. One vendor calls it batteryCharge, another stateOfCharge; one reports temperature in Celsius, another in raw ADC counts; an arm vendor buries joint data in a proprietary log while an AMR publishes clean JSON. Left alone, that heterogeneity means every dashboard, every alert, and every model has to special-case each supplier — which does not scale past a couple of vendors.

The fix is a canonical schema you own: define the six families once, in your units, with your field names and one clock, then map each vendor into it at the edge. Downstream, everything speaks one language. This is unglamorous integration work, and it is the single highest-leverage thing you can build, because it decouples your observability stack from any one supplier.

For mobile robots there is a real anchor to normalize toward. VDA5050 is the automotive-industry interface (JSON over MQTT) between mobile robots and a fleet master control, and its state message is the closest thing the AMR world has to a standardized telemetry envelope. It carries agvPosition with a localizationScore, powerSupply (renamed from batteryState at the 2.1 revision, August 2024), a safetyState with emergency-stop and field-violation flags, actionStates for task progress, and an errors array that is empty when the robot is healthy. That maps almost one-to-one onto the six families above. The caveat — and it is instructive — is that VDA5050 keeps renaming and restructuring fields across versions (the latest major revision, v3.0.0 from March 2026, reworked error severities and zone handling again), which is precisely why you map every vendor into your stable schema rather than adopting anyone else’s field names wholesale. Standards drift too. Mixed-vendor coordination is a running theme across running a robot fleet.

Which transports carry robot telemetry — DDS or MQTT?

Both, at different layers, and confusing them is a common design error.

Inside the robot, the real-time nervous system is usually DDS — the Object Management Group’s Data Distribution Service, a data-centric publish/subscribe standard for real-time and embedded systems, with fine-grained quality-of-service controls (reliability, deadlines, resource limits) and its own RTPS interoperability wire protocol. It is the default middleware under ROS 2: nodes publish and subscribe to named topics, and DDS moves the data. For local recording, rosbag2 captures those topics to disk — MCAP has been the default container since ROS 2 Iron (May 2023), with the older SQLite file (.db3) still supported as an alternative — plus a metadata sidecar, giving you the black-box recorder you will want when a robot does something inexplicable and you need to replay the exact topic stream frame by frame.

Off the robot, the fleet transport is usually MQTT — an OASIS standard, and ISO/IEC 20922 since 2016, a deliberately lightweight publish/subscribe protocol designed for constrained devices and thin, unreliable links. That bandwidth profile is exactly why it, not DDS, tends to carry robot-to-cloud telemetry, and it is the transport under VDA5050. The mental model: DDS is the high-rate bus within the machine; MQTT is the summarized stream that leaves it. You do not push kilohertz DDS traffic to the cloud, and you do not run a safety loop over MQTT.

What does observability mean once you move it from servers to robots?

Observability is a software-engineering idea before it is a robotics one, and it is worth importing precisely. In site-reliability practice it rests on three pillars: metrics, logs, and traces. Metrics are cheap numeric time series — battery percentage, cycle time, CPU load — that you aggregate and alert on. Logs are timestamped, mostly textual event records: this error fired, this action started. Traces follow a single unit of work across components; for a robot, a mission that crosses perception, planning, motor control, and the fleet manager. The distinction that matters: monitoring tells you whether a known thing is broken; observability is whether your telemetry is rich enough to answer a question you did not think to ask in advance — without shipping new code to the robot.

OpenTelemetry (OTel) is the CNCF’s vendor-neutral standard for exactly those three signals in distributed software: one instrumentation model, one wire protocol (OTLP), one collector, and you route to any backend — the project itself reached CNCF Graduated status in May 2026, the foundation’s highest maturity tier. Here is the honest framing. OTel was built for microservices, not motors. There is no official robotics profile of OpenTelemetry as of 2026, and a robot’s hard-real-time control loop is never going to emit OTel spans at kilohertz rates. But the model is portable, and it is increasingly the shape fleet-side telemetry takes once it has left the robot and been downsampled. Emit metrics, logs, and traces in a standard envelope so you are not welded to one vendor’s dashboard. Treat OTel as a concept to borrow, not a robot standard to comply with — anyone selling you “OpenTelemetry for robots” as a finished product is selling you the mapping, not a specification.

How do you go from one robot’s logs to a whole-fleet dashboard?

A single robot’s rosbag is a forensic tool: it answers “what did this machine do at 14:32?” A fleet dashboard answers a different question — “which of my two hundred machines is about to become a problem?” — and you cannot get there by scaling up log-tailing. The move is from records to aggregates.

Three things change at fleet scale. First, cardinality: two hundred robots times dozens of signals is a firehose, so you aggregate into per-unit and per-cohort rollups (fleet median cycle time, count of protective stops per robot per hour) rather than staring at raw streams. Second, comparison becomes the primary lens: the useful view is not one robot’s battery curve but all of them overlaid, because the outlier is the story. Third, the unit ID and timestamp carry the weight — without consistent tagging and one clock, you cannot line up an event on robot 47 with a shared charger or a specific aisle. A fleet view that cannot answer “show me every robot that took a protective stop in zone 3 this shift” is a collection of dashboards, not fleet observability.

How do you catch drift and anomalies before they become failures?

The signal you are hunting is almost never a single bad reading. It is drift — a slow change in a signal’s baseline that precedes a hard fault. A joint that draws a little more current each week. A localization score that trends down in one particular aisle as the environment or a sensor degrades. A cycle time creeping up on one cohort. None of these trips a fixed threshold until it is nearly too late, which is why threshold-only alerting is a weak net.

The fleet is what makes drift catchable. Because a fleet is many near-identical robots on near-identical duty cycles, you can build a baseline per cohort and treat the pack as its own control group: the robot pulling away from its cohort’s normal is flagged long before it hits an absolute limit. That is anomaly detection in its most defensible form — relative, not absolute. Be skeptical of vendor “AI anomaly detection” claims that promise to predict failures with tidy percentages; the honest version is that per-cohort trending catches the class of slow degradation reliably, and the specific hit rate is site-specific and rarely independently audited. The mechanism is sound. The marketing numbers around it usually are not.

Edge or cloud — what runs on the robot, what gets streamed?

This is a bandwidth-and-latency decision, not an ideology. Some computation must be local because the network cannot be in the loop; some must be central because no single robot can see the fleet.

Decision Runs on the robot (edge) Streamed to the fleet / cloud
Safety-critical loops (e-stop, collision avoidance) Yes — deterministic, no network in the loop Report the event, never run the loop remotely
High-frequency raw signals (kHz joint data, full sensor feeds) Recorded locally (rosbag2) for forensics Downsampled summaries and features only
Anomaly detection Fast unit-local checks (thresholds, light models) Fleet-wide, cross-unit, heavier models
Long retention and trend baselines Storage-bound, kept short Yes — this is the cloud’s job
What actually crosses MQTT Metrics, events, errors, and state at a manageable rate

The default that keeps operators out of trouble: compute safety and high-rate loops at the edge, stream summaries and events, and keep the full-rate recording local so you have a black box when something goes wrong. Streaming every raw signal to the cloud is a bill and a bottleneck; computing everything on the robot blinds the fleet. The line between them is drawn by what needs the whole fleet to be meaningful. Getting this right is also what makes the uptime numbers behind warehouse robotics ROI credible rather than aspirational.

How does fleet telemetry connect to predictive maintenance?

Directly — telemetry is the input, predictive maintenance is one of the outputs. The same drive-current and joint-temperature streams that let you rank robots on a dashboard are the raw material a condition-monitoring model reads to say a bearing is degrading. You cannot do predictive maintenance without fleet telemetry; the observability layer is the prerequisite, not a separate project — see our full treatment of predictive maintenance for robot fleets for how that model actually works. Telemetry also feeds task orchestration (route the low-battery robot to a charger), safety review (which zones generate the most near-misses), and fleet-wide software rollout (which cohort is healthy enough to update first). Predictive maintenance is the most-discussed payoff, but it is one consumer of the stream among several — all of them explored across running a robot fleet.

A closing caution, in keeping with the brief. “RobotOps” — the lens for this whole cluster — is a coined frame: DevOps and SRE thinking applied to robot fleets, not an ISO or VDA body of practice. The engineering underneath it is real and standardized: MQTT, DDS, ROS 2, OpenTelemetry, and VDA5050 are all published, checkable specifications. The discipline of pointing them at a fleet is what you are actually buying, and no dashboard substitutes for deciding, up front, exactly what every robot owes you. For the terms used here, see the glossary.

Frequently asked

What data should every robot in a fleet report?

Six signal families: power and battery state (charge, voltage, cell temperature, health); drive and joint health (motor current, temperature, following error); task and mission state (current action, progress, success or failure); localization confidence (how sure the robot is of its own pose); safety and near-miss events (protective stops, field violations, emergency stops, speed reductions); and structured, coded error and warning messages. Each one should be timestamped and tagged with a unit ID, on a fixed schema.

What is the difference between telemetry and observability for robots?

Telemetry is the raw stream a robot emits. Observability is whether that stream is rich enough to answer a question you did not anticipate — without shipping new code to the robot. Monitoring tells you a known thing broke; observability lets you ask why. In site-reliability practice it rests on three pillars: metrics, logs, and traces.

Can I use OpenTelemetry for robots?

OpenTelemetry is the CNCF vendor-neutral standard for metrics, logs, and traces in distributed software, with one wire protocol (OTLP) and one collector. It was built for microservices, not motors, and there is no official robotics profile as of 2026 — a hard-real-time control loop will not emit spans at kilohertz rates. But the three-signal model is portable and increasingly shapes fleet-side telemetry once it leaves the robot. Treat it as a concept to borrow, not a robot standard to comply with.

MQTT or DDS — which transport should carry robot telemetry?

Usually both, at different layers. DDS (an OMG standard) is the real-time publish and subscribe bus inside a robot and the default middleware under ROS 2. MQTT (an OASIS standard, also ISO/IEC 20922) is the lightweight transport for robot-to-fleet and robot-to-cloud links, and it is what VDA5050 rides on. DDS moves data within the machine; MQTT carries the summarized stream off it.

Does VDA5050 already standardize robot telemetry?

Partly, for mobile robots. Its state message — JSON over MQTT — carries agvPosition with a localizationScore, powerSupply (renamed from batteryState at the 2.1 revision), a safetyState with emergency-stop and field-violation flags, actionStates for task progress, and an errors array that is empty when the robot is healthy. It is a fleet-control interface, not a full observability specification, and field names have been reworked across versions, so map it into your own schema rather than adopting it wholesale.

What is telemetry drift, and how do you catch it before a failure?

Drift is a slow shift in a signal's normal baseline — a joint drawing a little more current each week, a localization score trending down in one aisle — that precedes a hard fault. You catch it with per-cohort baselines and trend or anomaly detection across near-identical robots, not with a single fixed threshold, because the value that matters is the change from normal, not any one reading.

Should anomaly detection run on the robot (edge) or in the cloud?

Split it. Safety-critical and high-frequency loops stay on the robot, where they must be deterministic and independent of the network. Fleet-wide trending, cross-robot comparison, heavier models, and long retention belong in the cloud. Stream downsampled summaries, events, and errors continuously; keep the full-rate recording locally for incident forensics.