Advancements in artificial intelligence (AI) have led to the development of intelligent decision-support systems (IDSSs), which can significantly influence human ethical decision-making. These systems, ranging from those guiding moral deliberation to providing concrete moral advice, raise questions about the extent to which humans are outsourcing their ethical decision-making to technology. While some argue that IDSSs can enhance informed and rational decision-making by considering moral factors, others caution against blindly following decisions inconsistent with personal values. To better understand these implications, researchers conducted a systematic review published in the Technological Forecasting and Social Change journal to explore how IDSSs can either hinder or facilitate ethical decision-making and the mechanisms behind these effects. The review aims to identify sources and operations of IDSSs and their outcomes on both individual and societal levels. Ultimately, the findings seek to inform stakeholders, including users, developers, policymakers, and academics, about the potential impact of IDSSs on ethical decision-making and how to address associated challenges.
Societal Implications of AI Advances for Decision Support Systems
The evolution of decision-support systems (DSS) into intelligent decision-support systems (IDSS) is driven by advancements in artificial intelligence (AI). These systems aim to assist humans in various decision scenarios, including ethical ones. While traditionally DSS provided rule-based expert advice, IDSS now offer sophisticated assistance, ranging from military targeting decisions to personal moral advice. This advancement has led to the concept of moral enhancement through AI, aiming to improve moral capacities and decision-making. However, the purely positive implications of IDSS on ethical decision-making are viewed sceptically. Scholars are categorising AI systems to understand their impact on ethical decision-making, distinguishing between those supporting moral deliberation, providing advice, or replacing ethical decision-making entirely. Yet, a systematic analysis of how IDSS shape ethical decision-making and their long-term societal repercussions is lacking. The article aims to synthesise existing knowledge on the sources and features influencing IDSS operations and their effects on ethical decision-making to understand their societal implications better.
Systematic Literature Review Scope & Methodology
The systematic literature review examined publications focusing on intelligent decision-support systems (IDSSs) designed to assist or take over ethical decision-making. The review aims to analyse past research, facilitate theory development, and advance knowledge on this topic. The methodology involves a three-stage iterative process, including identification of relevant literature, structural and in-depth content analysis, and integration of the literature. The literature search, not limited by date, includes peer-reviewed academic journal publications, edited books, book chapters, conference proceedings, and practitioner-oriented articles written in English until December 31, 2023. Publications discussing the influence of IDSSs on individuals' ethical decision-making processes are considered, specifically those employing AI techniques to assist or automate decision-making. The search yielded 1,731 hits, resulting in 34 relevant journal articles, book chapters, and conference proceedings after analysis. Forward- and backward searches identified an additional 11 relevant contributions, totaling 45 publications for further analysis.
The Impact of Intelligent Decision-Support Systems on Ethical Decision-Making
There are two main sources of influence: the human creators of the system and the system itself. These influences can manifest in two methods within IDSSs: process-oriented navigation and outcome-oriented navigation. Process-oriented navigation guides users through ethical decision-making without imposing predetermined values, while outcome-oriented navigation directs users towards predetermined ethical outcomes. The review suggests that process-oriented navigation enhances deliberation but outcome-oriented navigation may lead to moral deskilling, reducing users' ability to make ethical decisions independently. Moreover, autonomy enhancement is associated with process-oriented navigation, which helps users clarify their values, while outcome-oriented navigation may limit autonomy by nudging users towards predetermined decisions. The review also discusses motivation enhancement, which is less dependent on navigation type and more on system properties that appeal to users' emotions. Lastly, whether an ethical decision is executed, action enhancement is not distinctly tied to a navigation type, but both process-oriented and outcome-oriented navigation may influence it. However, blindly following recommendations from outcome-oriented navigation may not lead to ethical decisions, potentially hindering moral progress.
Theoretical contributions include vocabulary, classifications, and an integrated framework for understanding IDSS impacts. Additionally, the study holds practical implications for technology companies, offering insights into the sources of influence within IDSSs, different operational features, and potential ramifications for ethical decision-making at both individual and societal levels.
Decoding the Impact of IDSS: Nine Key Takeaways
The nine established takeaways from this study offer propositions for empirical testing in future research, serving as tools to facilitate discussions and investigations into the influence of IDSSs on ethical decision-making.
- Dual sources of influence: The impact of IDSSs on ethical decision-making stems from both their human creators and the technological system itself, with their influences reciprocally shaping each other.
- Two methods of influence: IDSSs influence ethical decision-making through either process-oriented navigation, guiding users through their decision-making process without imposing predetermined values, or outcome-oriented navigation, directing users towards predetermined decision outcomes based on embedded values.
- Deliberation enhancement: IDSSs that adopt process-oriented navigation facilitate users' ethical decision-making by promoting deliberation and moral reasoning, leading to a deeper understanding of ethical principles and improved decision-making.
- Moral deskilling: Outcome-oriented navigation in IDSSs may lead to moral deskilling, as users rely on the system's recommendations without engaging in ethical deliberation, potentially undermining their ability to make ethical decisions independently.
- Motivation enhancement: The impact of IDSSs on motivation enhancement is influenced more by inherent system properties, such as immersiveness, than the navigation type, as emotional appeal plays a significant role in decision-making.
- Autonomy enhancement: Process-oriented navigation in IDSSs enhances users' autonomy by allowing them to clarify their values and make decisions consistent with their principles, whereas outcome-oriented navigation may limit autonomy by imposing predetermined values.
- Moral responsibility gap: IDSSs adopting outcome-oriented navigation may widen the moral responsibility gap by reducing users' involvement in decision-making, leading to diminished accountability for ethical decisions.
- Action enhancement: Both process-oriented and outcome-oriented navigation may influence action enhancement in ethical decision-making, but blindly following IDSS recommendations may not necessarily lead to ethical actions, potentially undermining moral integrity.
- Moral progress: Process-oriented navigation in IDSSs promotes moral progress by allowing for individual and collective discovery and application of new values, while outcome-oriented navigation may restrict moral progress by limiting users' exposure to diverse ethical perspectives.
The study emphasises the importance of a human-centered approach to the development and governance of IDSSs, calling for further empirical research to validate the proposed interrelations. However, the research has limitations. It focused on specific types of literature, excluding insights from other technologies like games or virtual environments designed for ethical decision-making education. Contextual and individual factors that may moderate the impact of IDSSs were not fully considered. Additionally, the lack of empirical evidence supporting the findings and propositions is noted. Companies developing IDSSs should prioritise systems that promote process-oriented navigation, but if outcome-oriented navigation is pursued, measures must be taken to mitigate negative impacts and ensure transparency and user accountability.
Source: Technological Forecasting and Social Change
Image Credit: iStock
