As organizations increasingly rely on artificial intelligence, automation, and Decision Support Systems (DSS), an important question emerges: How much should a human trust a computerized recommendation?
From a Management Information Systems (MIS) perspective, maximizing trust is not necessarily desirable. A highly trusted system can create risk when users rely on it beyond its actual capabilities, while a reliable system provides limited value if users consistently ignore its recommendations. The more appropriate objective is calibrated trust: aligning the user’s level of trust with the actual capabilities, reliability, limitations, and operating conditions of the information system.
What Is Calibrated Trust?
Trust in automation generally refers to a user’s willingness to rely on an automated system under conditions involving uncertainty or vulnerability. Lee and See (2004), in their influential work on trust in automation, emphasized the importance of appropriate reliance rather than simply increasing trust. Calibrated trust can therefore be understood as a state in which the user’s trust corresponds reasonably well to the system’s actual trustworthiness.
Three situations can be distinguished:
Under-trust → Calibrated trust → Over-trust
- With under-trust, users place less confidence in a system than its demonstrated capabilities justify and may ignore useful recommendations.
- With over-trust, users assume that the system is more capable or reliable than it actually is and may accept its recommendations without sufficient verification.
- With calibrated trust, users understand both the strengths and limitations of the system and adjust their reliance accordingly.
The objective is therefore not simply to develop information systems that users trust, but systems that users trust appropriately.
Calibrated Trust and Decision Support Systems
The relationship between calibration and Decision Support Systems has deep roots in Information Systems research. Kasper (1996), for example, developed a theory of DSS design centered on user calibration, emphasizing the relationship between a decision maker’s confidence and the actual quality of decisions made with computerized assistance.
A modern decision-support process can be simplified as:
Data → Information System → Analysis → Recommendation → Human Evaluation → Decision
The critical distinction is that a recommendation should not automatically become a decision.
Modern DSS increasingly use AI and machine learning to detect patterns, estimate probabilities, identify anomalies, predict outcomes, and recommend alternatives. Consequently, the relationship between technological capability and human judgment becomes increasingly important.
Traditional MIS research often asked:
“Will users adopt and use the system?”
For AI-enabled DSS, an additional question becomes necessary:
“Will users rely on the system appropriately?”
This represents an important shift from technology acceptance toward appropriate technology reliance.
Recommended: Decision Support Systems for Airport Parking Position Allocation
The Risks of Too Much or Too Little Trust
Parasuraman and Riley’s influential research on automation distinguishes between problems such as misuse and disuse. Excessive trust can encourage inappropriate reliance on automation, while insufficient trust can cause users to reject useful technological assistance.
This relationship can be simplified as:
- Over-trust → Excessive reliance → Potential misuse
- Under-trust → Insufficient reliance → Potential disuse
Neither represents successful information-system implementation.
Related research on automation bias has also shown that computerized decision aids can sometimes lead users to follow incorrect recommendations or fail to detect information that the automated system has overlooked.
A useful question for MIS designers is therefore not merely:
“Do users trust the system?”
but rather:
“Do users know when they should and should not trust the system?”
Why Accuracy Alone Is Not Enough
Suppose an AI-based DSS has an overall historical accuracy of 95%. That figure alone does not mean that users should rely on it equally in every situation. The system might perform extremely well under normal operating conditions but considerably worse when data are incomplete, environmental conditions change, sensors degrade, or the system encounters circumstances poorly represented in its training data.
This leads to an important principle: Calibrated trust should be context-sensitive, not merely system-sensitive.
Users need to understand not only how reliable a system generally is, but also when its reliability may increase or decrease.
For this reason, transparency and explainability can contribute to trust calibration. Where appropriate, a DSS can communicate what it recommends, why it recommends it, which information it used, what information may be missing, and how uncertain the assessment is.
A Simple MIS Example
Consider a logistics company using an AI-based DSS to recommend whether a shipment should be rerouted. The system analyzes weather, port congestion, vessel position, historical delays, terminal capacity, and transportation costs and produces:
Recommendation: Reroute via Port B
Probability of significant delay at Port A: 82%
Confidence: High
Primary factors: congestion and forecast weather deterioration.
A manager who automatically follows every recommendation demonstrates excessive reliance. A manager who routinely rejects algorithmic recommendations demonstrates insufficient reliance. A manager with calibrated trust might instead give substantial weight to the recommendation because the relevant data are current and the model is reliable in this domain, while independently checking contractual or operational factors that are outside the model. The information system therefore supports managerial judgment rather than replacing it.
Calibrated Trust in Aviation
Aviation provides an especially useful environment for understanding calibrated trust because pilots routinely interact with sophisticated automated and decision-support systems.
A familiar example is the Traffic Alert and Collision Avoidance System (TCAS). TCAS processes surveillance information to identify potential traffic conflicts. TCAS II can issue Resolution Advisories (RAs) recommending vertical maneuvers or restrictions intended to maintain or increase separation.
This illustrates an important principle: pilots should understand not only that the system provides information, but also the meaning and operational significance of its different outputs.
Appropriate reliance therefore depends on understanding what the automation is doing, its operational authority, and the conditions under which its guidance should be followed.
NASA’s Emergency Landing Planner
A more direct example of intelligent decision support comes from NASA research on an Emergency Landing Planner (ELP).
The system was designed to assist pilots during emergency diversion decisions. Experimental research involving commercial pilots examined whether different levels of automation transparency influenced pilot trust in the planner.
This is particularly relevant to calibrated trust because an emergency landing recommendation should not simply appear as:
DIVERT TO AIRPORT B
A better decision-support interface can provide the reasoning and relevant information behind that recommendation, allowing pilots to evaluate whether reliance is appropriate.
NASA has also explored an AI Flight Advisor, a research concept involving AI-based analysis of aircraft states and aviation information to help identify unusual situations and provide pilots with advice regarding possible corrective actions.
Such concepts illustrate the progression:
Information System → Decision Support System → Intelligent DSS → Human–AI Teaming
A Realistic Aviation Scenario
Imagine an intelligent flight-decision-support system recommending diversion to Airport A during an abnormal aircraft situation:
Recommended Airport: A
Suitability: 87%
Weather information: updated 3 minutes ago
Runway condition information: updated 42 minutes ago
Current braking-action information: unavailable.
Airport B receives a slightly lower suitability assessment but has much more recent runway-condition information. The crew now receives something more valuable than a simple ranking. It receives information that helps evaluate the reliability of the recommendation itself. Depending on aircraft condition, fuel, weather, ATC information, company procedures, and other operational factors, the pilots may still select Airport A—or determine that Airport B is more appropriate. The key principle is that the system has supported the decision without silently replacing human judgment.
Designing Systems for Calibrated Trust
From an MIS perspective, calibrated trust is therefore not merely a psychological characteristic of the user. It is also a system-design and governance issue. A well-designed intelligent DSS should help users distinguish between:
- Recommendation ≠ Decision
- Confidence ≠ Certainty
- Historical accuracy ≠ Reliability in every situation
- Automation ≠ Authority
Where operationally appropriate, systems can support calibrated trust by communicating uncertainty, highlighting missing or degraded data, explaining major factors behind recommendations, presenting meaningful alternatives, and clearly defining operational limitations. Research by McGuirl and Sarter, for example, investigated whether presenting dynamic system-confidence information could support trust calibration in an aviation decision aid designed to help pilots manage in-flight icing situations. This reinforces a central principle for modern MIS: users may need information not only about the operational problem, but also about the quality and limitations of the information system’s assessment of that problem.
From Technology Acceptance to Appropriate Reliance
The increasing use of AI creates an important evolution for Management Information Systems. Traditional IS research frequently focused on whether users would accept and use technology. Intelligent DSS introduce a second challenge: ensuring that users rely on technology appropriately. The progression can therefore be conceptualized as:
Technology Acceptance → Technology Reliance → Appropriate Reliance → Calibrated Human–AI Collaboration
High system usage does not necessarily indicate successful implementation if users routinely accept unreliable recommendations. Similarly, questioning or rejecting an automated recommendation does not necessarily indicate technological resistance when the user has correctly recognized the system’s limitations.
Conclusion
As information systems increasingly predict, diagnose, recommend, and prioritize, trust becomes an important component of organizational decision-making. But more trust is not automatically better. Too little trust can cause users to reject valuable technological assistance. Too much trust can transform decision support into uncritical dependence. The more desirable objective is calibrated trust, in which human reliance reflects the actual capabilities, limitations, reliability, and situational performance of the information system. This is particularly important for Decision Support Systems because they increasingly occupy the space between data and human action. The future of intelligent information systems should therefore not simply aim to make humans trust AI more. It should enable humans to understand when automation deserves reliance, when it deserves scrutiny, and when human judgment should prevail.
That is the essence of calibrated trust.
References
- Kasper, G. M. (1996). A Theory of Decision Support System Design for User Calibration. Information Systems Research, 7(2), 215–232.
- Lee, J. D., & See, K. A. (2004). Trust in Automation: Designing for Appropriate Reliance. Human Factors, 46(1), 50–80.
- McGuirl, J. M., & Sarter, N. B. (2006). Supporting Trust Calibration and the Effective Use of Decision Aids by Presenting Dynamic System Confidence Information. Human Factors, 48(4).
- Parasuraman, R., & Riley, V. (1997). Humans and Automation: Use, Misuse, Disuse, Abuse. Human Factors, 39(2).
- Skitka, L. J., Mosier, K. L., & Burdick, M. (1999). Does Automation Bias Decision-Making? International Journal of Human–Computer Studies, 51(5).
- Federal Aviation Administration. Airborne Collision Avoidance System (ACAS) / Traffic Alert and Collision Avoidance System (TCAS).
- NASA. An Experimental Study of the Effect of Transparency on Pilot Trust in the Emergency Landing Planner.
- NASA. Artificial Intelligence Flight Advisor.