Agentic data curation: describe a moment, and Agents find every example

Day 3 of Agent Week. Day 1 introduced Roboto Agents; Day 2 put them on triage and interactive analysis; this morning they traced failures to the code behind them.

“Find every stair traversal from the last six months.” Starting today, that sentence is a complete workflow.

Roboto Agents can now curate your data: describe a moment you care about in plain language, and Agents search your time-series, video, and logs for it, then gather every example into a Collection, a set you can review with your team, pull into a notebook, or export straight to ML-ready formats like LeRobot from Hugging Face.

Watch Roboto turn a stair-climb analysis into a reusable Agent that finds every example, builds a Collection, and exports a LeRobot dataset to Hugging Face.

Why we built it

Talk to enough robotics teams and you hear two versions of the same complaint.

Teams with robots in the field are drowning in data. As the fleet grows, reviewing every log by hand in a visualizer becomes impossible, so most logs never get opened. The rare events, edge cases, and near-misses worth studying sit untouched in storage.

Teams building learned policies have the opposite problem: they can’t get enough of the right data, and their training and evaluation sets still come together through manual labeling, one episode at a time.

Both problems come down to search: finding the events that matter in huge volumes of multimodal data. Our Agents do exactly that, working from plain natural language. Describe your failure modes and you get triage. Describe what you want to train on — recoveries, rare object interactions, unusual terrain — and you get curation.

Being able to sift automatically and curate the interesting events into Collections for review or training is what builds a data flywheel: what your fleet records today trains the models it deploys tomorrow. That matters more as the industry moves toward end-to-end models. When the policy is one learned model, the reason a robot failed often lives in its training data, not in a code path an engineer can step through. Curation becomes a debugging tool as much as a training one.

How to set up automatic curation

Curation runs on three primitives: Skills spot the moments you care about, Agents run them against your data, and Actions keep the loop running.

1. Set up curation Skills. Each one describes, in plain language, an event worth surfacing: a near-miss with a pedestrian, a stair traversal, a recovery after a slip. Skills can reason across modalities — camera frames, joint states, operator commands — so the scenario you describe doesn’t have to live in a single signal.

2. Create an Agent to run them. For every hit, the Agent creates an Event: a timestamped, labeled marker on the moment worth reviewing or training on. Events are the curation artifact and feed straight into Collections.

3. Run it on demand. Point the Agent at an existing dataset to take a first pass over your history, or to vet a new Skill before trusting it unattended.

4. Automate it with Actions. Wire the Agent to an Action and it runs the moment new data arrives. Events land in Collections on their own. No backlog, no manual sifting.

The user guide, Automatic Data Curation with Agents, walks through the full setup, from writing your first curation Skill to exporting a Collection.

Moon Surgical turns procedure runs into training data

Moon Surgical builds collaborative robots for soft-tissue surgery. Every procedure produces hours of video and telemetry, and the moments worth training on are scattered through them.

“

Reviewing procedure runs manually was never going to scale. Now we describe the moments we care about, and Roboto curates them into Collections we can evaluate and train on, accelerating how we develop and validate new Physical AI capabilities for our surgical platform.

Get started

Book a demo to see Agents running against your own data. Want to try it yourself first? Follow the curation guide and build your first Collection.

That’s a wrap on Agent Week. A week ago, a new log sat in storage until someone had a reason to open it. Today it gets a first look on arrival, answers when you ask, a root cause when it fails, and a place in a training set when it captures something worth learning from. Everything we launched is live now, and it’s all at Agent Week.