Can home robots learn how you want a task done?
A 312-task simulation benchmark separates finishing a household task from respecting a user's preference. Today's robot policies often completed the task while ignoring how the user asked it to be done.
The short version
3 key points
- 1.LIBERO-PeRM contains 312 simulated tasks and 40,800 rollout episodes spanning motion, skill and user-profile preferences.
- 2.In zero-shot tests, six robot policies frequently finished tasks while failing the preference-satisfaction measure.
- 3.Fine-tuning helped, but unseen layouts, preference composition and conflicts remained difficult; no real robot or household user was tested.
Finishing a task is not the same as respecting a person
A robot can put an object in the right place and still do the job the wrong way for its user—moving too close to something fragile, grasping an item by the wrong part or ignoring which object a person prefers. A new benchmark makes that distinction measurable. Across six vision-language-action policies tested without preference-specific training, task completion often remained high while preference satisfaction was much lower.
The LIBERO-PeRM paper, submitted on 15 October and listed for NeurIPS 2026, builds 312 simulated manipulation tasks from the existing LIBERO environment. The authors define three kinds of preference: motion choices, such as avoiding an object; skill choices, such as grasping by a handle; and user-profile choices, such as selecting the item suited to a named profile. More than 100 generated trajectories per task produced 40,800 rollout episodes.[1]
The sharpest result came before fine-tuning
The benchmark scores success and preference satisfaction separately. In one illustrative result, the strongest tested base policy completed 97.5% of motion-preference tasks and 100% of skill-preference tasks, but satisfied the stated preference only 24.38% and 25% of the time. That gap is the paper's central contribution: conventional success metrics can make a personalised robot look capable when it has ignored how a person asked the task to be performed.
Supervised fine-tuning made preferences more learnable, but the training recipe created trade-offs. Mixed training retained 91% success on tasks without preferences and reached 84.55% task success across preference tasks. Sequential adaptation retained only 52.33% success on no-preference tasks. The mixed approach also handled contextual prompts better than preference-only tuning, while weakening motion-preference learning. Results fell substantially on unseen layouts.[1]
The distinction that matters
Two scores for one household task
The benchmark separates the goal from the way the user asked for it to be achieved.
Task success
Did the robot complete the physical goal?
Object or scene reaches the target state
Preference satisfaction
Did it follow the requested motion, skill or profile?
Execution matches the user's stated constraint
Conflicts reveal the missing conversation
When two learned preferences had to be combined, evaluation success and satisfaction were both 25%. In a decomposition test—inferring one preference from a combined instruction and another known preference—task success was 37.5% but preference satisfaction was zero. In task-versus-preference conflicts, 37.5% of trials completed neither side. The benchmark does not test whether a robot notices the conflict, asks for clarification or explains why it cannot comply, all of which may matter more to a household user than silent execution.
The evidence remains deliberately narrow. Every task ran in simulation; preferences were predefined, expressible in language and automatically measurable. Fifteen survey participants rated demonstrations as plausible and human-like, but there was no real robot, home or longitudinal user study. Generated demonstrations and the evaluation metrics may reflect the pipeline's own biases, and the accessible paper gives no funding or competing-interests statement. The useful next step is a real-robot study in which diverse users set, revise and resolve preferences over time—with safety failures and requests for clarification recorded alongside task completion.[1]
Evidence check
The benchmark uses simulated, predefined and language-expressible preferences. Its 15-person survey is only initial validation; no real robot or longitudinal household use was studied.
Reporting basis: Source analysis. Original source date: 15 October 2026.
Explore the full source and evidence record
What this means for people
- People with accessibility, safety or household preferences need robots to respect how a task is performed, not merely its endpoint.
- A useful home robot should surface conflicts and ask for clarification instead of silently choosing between incompatible instructions.
Global context
The author team is based in China and the benchmark uses an international open robotics environment. Real homes, languages, bodies and cultural preferences are far more varied than the simulation.
What the evidence does not yet show
- The benchmark uses simulated, predefined and language-expressible preferences. Its 15-person survey is only initial validation; no real robot or longitudinal household use was studied.
What to watch next
- Independent real-robot studies testing evolving preferences, clarification behaviour and safe conflict resolution with diverse users.
Living evidence record
Impact record IAI-01D0586
Evidence stage
Studied
Confidence
Supported
Reporting basis
Source analysis
Independent or research support
Not yet
Record status
Monitoring
Last checked
16 October 2026
Source trail
1 direct source across 1 source type.
People impact
Documented in this record.
Uncertainty
Limits and next checks are explicit.
Stages describe the evidence available—not whether a technology is good or bad. See the public method.
Single-source reporting disclosure
This record analyses one direct source. It can establish what Li et al. (arXiv) published or reported, but it is not independent corroboration of every performance claim or predicted outcome. The confidence label will change only when broader evidence is added.
Evidence trail
Sources used for this report
Links checked 16 October 2026
Research question, topics and regions
Preference learning in simulated robotic manipulation
Topics: Robotics · Personalisation · Household technology · Vision-language-action models · Human preferences
Regions: China · Global
This report is labelled source analysis. We summarise and analyse source material in our own words; company statements remain attributed claims until independently supported. Translated summaries preserve the meaning of the original source and link back to it. Read our editorial standards.
Watch the background, then check the evidence
The Royal Society
YouTube loads only when you choose. Press play in the player to start. If playback is unavailable, use “Watch on YouTube” below.
AI basics · Background explainer
Published 2018-11-28 · 2 min 31 sec
What is artificial intelligence? | The Royal Society
A short introduction to artificial intelligence from the Royal Society. Start here for basic concepts, then use current reporting to explore what has changed.
Evidence check: Published in 2018, before today’s generative AI tools. Useful for basic concepts, rather than an overview of current capabilities.
Source checked 14 October 2026. Check available captions on the publisher’s video page.
The Impact Brief · Free
Keep following the stories that matter to you.
Get our weekday briefing on what changed and why it matters. Choose your topics below.
Continue the story
There’s more to this story
Technology
Cloud offloading could change the performance and battery trade-off for robots
Microsoft Research reports that moving some physical-AI inference from a robot's onboard processor to edge or cloud GPUs can improve task success, efficiency and the complexity of workloads the machine can handle.
2 min · 1 source
Technology
Gemini Robotics ER 2 pushes embodied reasoning into developer tools
Google DeepMind released Gemini Robotics ER 2 for building physical agents, reporting gains in safety-instruction following, human-proximity behaviour and reasoning about real environments.
2 min · 2 sources
Technology
Can sparse federated learning cut IoT communication without losing accuracy?
A peer-reviewed benchmark reports communication reductions of up to 477-fold for a hierarchical federated-learning method while remaining near strong accuracy baselines. The tests used stored sensor datasets and simulated client hierarchies—not a live device fleet or a privacy audit.
8 min · 2 sources
Reader commentary
What does this mean where you are?
Tell us what you have seen at work, in your community or in everyday life. A question is welcome too. No account is needed; we review notes before publishing them.
Explore commentary across the portal →Published reader notes
0No published reader notes yet. You can start the evidence-led discussion above.
Prefer a private correction or response? Contact the newsroom.