Do uncertainty displays help nurses use AI scheduling advice?
Not automatically. In a 16-person German hospital study, prediction intervals were often misread and lowered confidence; brief explanations attracted attention but did not change advice acceptance.
The short version
3 key points
- 1.Sixteen staff at one German hospital—12 ward managers or shift planners and four registered nurses—completed within-subject tasks using five prototype variants.
- 2.Confidence was significantly higher without an uncertainty band than with it, while explanations did not significantly change accepted advice items or confidence.
- 3.The sample was small, the interface used mock data and a rule-based heuristic, and no live staffing, patient outcome or long-term automation-bias effect was tested.
The answer: showing uncertainty did not make it usable
Sixteen nurses and shift planners could identify days needing action from a scheduling forecast, but many misread the prediction interval intended to show uncertainty. They looked at where the band peaked or dipped rather than at its width. A short explanatory note did not reliably correct that interpretation.
Decision confidence averaged 5.00 on a seven-point scale without an uncertainty band, 3.94 with the band and 4.44 when historical performance was added. Confidence was significantly higher without the band than with it. That does not mean uncertainty should be hidden; it means presentation can create confusion without showing what action uncertainty should change.
Brief explanations of scheduling advice were welcomed and drew more visual attention, but participants accepted almost the same number of suggestions with and without them. Confidence also barely changed. Transparency that feels useful is not necessarily transparency that changes a decision.[1]
Who took part and what they used
The on-site study recruited 16 staff at one German hospital, aged 28 to 64 with a mean age of 40.1. Twelve were assistant ward managers or ward managers who frequently planned shifts; four were registered nurses directly affected by scheduling. Researchers balanced gender but withheld exact counts to reduce traceability.
Participants used a prototype built around forecasts of care demand and staff absence, a simplified plan for 22 fictitiously named nurses and advice from a rule-based optimisation heuristic. The heuristic proposed one-shift changes to reduce understaffing, overstaffing or violations such as too many consecutive workdays.
The schedule was adapted from a real ward but reduced and standardised. Forecasts and shifts were internally consistent, yet some ward sizes and rules did not match participants' work. This was a usability and decision-support experiment, not a live deployment that allocated staff or affected patients.[1]
Five interface variants tested two questions
A within-subject design exposed every participant to three forecast views: no uncertainty visualisation, a prediction interval, and the interval plus historical forecast performance. The study asked whether users understood forecasts, could identify higher or lower uncertainty and felt confident acting.
A second block compared scheduling advice with and without short natural-language explanations such as 'reduction of overstaffing'. Advice was highlighted in the shift plan, and participants could accept or decline each item. Existing rule violations were also shown.
Researchers combined task answers, confidence ratings, questionnaires, eye tracking and think-aloud interviews. Repeated-measures models and paired tests compared conditions. This mixed design captures behaviour and perception, although sixteen participants provide limited statistical power and many comparisons remain exploratory.[1]
Basic forecasts were clearer than the uncertainty layer
Participants performed well when asked which days required a scheduling change. F1 scores were 0.96 without uncertainty and 0.94 in both uncertainty conditions. They also identified low care demand or high staff absence reasonably well, with no significant differences across views.
Performance fell when questions asked where uncertainty was highest or lowest. For one high-uncertainty question, mean F1 was 0.38 with the band and 0.28 when history was added. For the low-uncertainty question, adding history improved recall, but most other differences were not significant.
Think-aloud comments exposed an actionability gap. Even participants who grasped the graphic asked where uncertainty came from and what to do differently. A wider interval may justify contingency planning, but the interface did not translate that concept into a staffing option, escalation rule or operational cost.[1]
Confidence fell without a matching fall in measured performance
A mixed-effects model found a design effect on confidence, with the no-uncertainty view significantly above the interval-only view after correction. The historical-performance view sat between them and did not differ significantly from either. Overall user-experience scores did not differ, suggesting frustration was specific to uncertainty.
Objective task performance was not significantly associated with confidence. A planner can therefore feel less sure without being less accurate, or feel sure without performing better. Confidence is not a sufficient proxy for calibrated reliance.
The ethical response is not to maximise confidence. A scheduling tool should help users understand when a forecast is fragile, what alternatives remain viable and which decisions require review. Hiding uncertainty may make an interface feel easier while encouraging unjustified certainty.[1]
Explanations changed attention more than decisions
Participants accepted an average of 4.44 advice items without explanations and 4.50 with explanations. Confidence averaged 5.69 and 5.75. Neither difference was statistically significant, and questionnaire measures did not show a significant improvement.
Eye tracking showed a behavioural shift: explanations increased fixation time on the advice area and reduced attention to the underlying shift plan. Some interviewees said explanations improved willingness to use advice, but extra salience can also displace inspection of the schedule that a human is meant to oversee.
A vendor can show that people read or like an explanation without proving better choices, fewer staffing violations or fairer workloads. Explanation quality must be evaluated against a defined decision and outcome.[1]
Human oversight meant more than a final approval button
Participants described AI as a starting point under professional supervision. Eight supported an AI-assisted workflow if staff preferences and fairness constraints could be incorporated. They wanted the system to account for individual requests, workload equity and social context that a penalty function did not capture.
The strongest design implications follow from that gap: distinguish suggestions from decisions, make uncertainty actionable, show the basis and limits of advice, let planners edit and reorder proposals, and allow them to record why a recommendation was accepted or rejected.
A schedule shapes fatigue, family life, continuity of care and perceived fairness. An optimisation that satisfies staffing ratios can still distribute nights, weekends or difficult assignments inequitably. Those values need to be inputs and review criteria, not after-the-fact repairs.[1]
Limits and what would change the assessment
The single-site sample and mock data make this an informative prototype study, not evidence that one interface works across hospitals. Larger multi-site studies should compare prediction intervals with frequencies, scenarios and concrete contingency actions, and preregister a small set of decision outcomes.
A longitudinal trial should measure how confidence, reliance and skill change after weeks of use. It should test automation bias, scheduling time, rule violations, workload distribution, staff satisfaction and patient-safety indicators. Staff representatives should help define fairness constraints before deployment.
Until then, the useful conclusion is modest: adding an uncertainty band or explanation is not enough. The interface must teach the concept, link it to a choice and preserve attention to the underlying schedule. Good human oversight is designed into the workflow rather than asserted in a label.[1]
What this means for people
- Nurses may bear the consequences of schedules that satisfy formal rules but overlook workload equity, preferences and social constraints.
- Shift planners need uncertainty presented as an actionable choice, not only a statistical band.
- Hospital leaders should evaluate explanations against scheduling quality and staff outcomes, not attention or confidence alone.
Global context
The study was conducted in one German hospital under local staffing rules and a Bavarian-funded project. Hospitals elsewhere use different ratios, contracts, unions and scheduling cultures, so replication must be locally grounded.
What the evidence does not yet show
- Only 16 participants from one hospital took part, limiting power and generalisability.
- The study used mock forecasts and a simplified 22-person schedule, not live workforce decisions.
- Only prediction intervals represented uncertainty; other formats may perform differently.
- Advice came from a rule-based heuristic, so results do not generalise to every scheduler.
- No staffing efficiency, fairness, patient outcome or long-term automation-bias measure was tested.
What to watch next
- Multi-site trials using live but safely sandboxed scheduling decisions
- Alternative uncertainty formats linked to concrete contingency actions
- Measures of workload equity, staff satisfaction and patient safety
- Longitudinal evidence on trust calibration and automation bias
Living evidence record
Impact record IAI-0KXHS1Y
Evidence stage
Studied
Confidence
Supported
Reporting basis
Source analysis
Independent or research support
Not yet
Record status
Monitoring
Last checked
14 October 2026
Source trail
1 direct source across 1 source type.
People impact
Documented in this record.
Uncertainty
Limits and next checks are explicit.
Stages describe the evidence available—not whether a technology is good or bad. See the public method.
Single-source reporting disclosure
This record analyses one direct source. It can establish what arXiv published or reported, but it is not independent corroboration of every performance claim or predicted outcome. The confidence label will change only when broader evidence is added.
Evidence trail
Sources used for this report
Links checked 14 October 2026
Research question, topics and regions
How nurses and shift planners understand uncertainty and explanations in an AI-supported scheduling prototype
Topics: Nursing · Scheduling · Explainable AI · Uncertainty · Workforce planning · Human oversight
Regions: Germany · Europe
This report is labelled source analysis. We summarise and analyse source material in our own words; company statements remain attributed claims until independently supported. Translated summaries preserve the meaning of the original source and link back to it. Read our editorial standards.
Watch the background, then check the evidence
UNESCO
AI-generated illustration · not a video frame
YouTube loads only when you choose. Press play in the player to start. If playback is unavailable, use “Watch on YouTube” below.
AI, jobs & human choice · Background explainer
Published 2024-10-14 · 11 min 27 sec
UNESCO Talks: Interview with 2024 Nobel laureate Professor Daron Acemoglu
Economist Daron Acemoglu discusses how choices about AI affect workers, inequality and society. Pair the interview with our comparison of task exposure, hiring and skills evidence.
Evidence check: An expert interview from 2024, offering a perspective on policy and work rather than a measured forecast of job losses.
Source checked 14 October 2026. Check available captions on the publisher’s video page.
The Impact Brief · Free
Keep following the stories that matter to you.
Get our weekday briefing on what changed and why it matters. Choose your topics below.
Continue the story
There’s more to this story
Work & Skills
Does supportive AI improve work outcomes?
A survey of 695 German AI users linked perceived support to better self-reported work outcomes mainly through proactive job redesign. Because every measure came from one cross-sectional questionnaire, it cannot show that AI—or job crafting—caused higher productivity or wellbeing.
8 min · 1 source
Work & Skills
Does algorithmic scheduling drive hotel-worker burnout?
A peer-reviewed survey of 635 hotel workers in Shandong links greater exposure to algorithm-influenced schedules with less control over time, more work–family conflict and higher burnout. The pathway remained after adjustment but weakened substantially, and the one-time self-report design cannot prove that scheduling software caused the outcomes.
8 min · 1 source
Work & Skills
OECD interviews show how early AI agents are being kept within human checkpoints
Practitioners in 25 organisations across 11 countries describe agents moving into real workflows, with autonomy limited around consequential actions. The interview study is a window into practice, not a global adoption rate.
5 min · 2 sources
Reader commentary
What does this mean where you are?
Tell us what you have seen at work, in your community or in everyday life. A question is welcome too. No account is needed; we review notes before publishing them.
Explore commentary across the portal →Published reader notes
0No published reader notes yet. You can start the evidence-led discussion above.
Prefer a private correction or response? Contact the newsroom.