Advances in artificial intelligence have led to decision support systems that can outperform human experts in specific tasks and complement human judgment, particularly under uncertainty or time pressure. While combining human and AI assessments is expected to reduce errors, the integration of AI into routine decision-making remains limited, partly due to concerns about reliability and trust.

A key challenge is that most AI systems provide only point predictions or internal confidence scores, which do not fully capture true uncertainty. Reliable decision-making requires transparent and well-calibrated uncertainty information. Conformal Prediction offers a model-agnostic approach to uncertainty quantification by generating prediction sets with guaranteed coverage and communicating how reliable an AI recommendation is. However, it remains unclear how users interpret and incorporate such uncertainty information into their decisions.

This project studies whether AI systems equipped with Conformal Prediction improve human decision-making. We conduct an incentivized experiment in which participants answer quiz questions with monetary stakes and can consult an AI assistant based on a large language model. Depending on the treatment, the AI provides either no assistance, a point prediction, or a prediction set reflecting model uncertainty. By observing confidence judgments, opt-out behavior, and decision quality, we examine whether uncertainty-aware AI leads to better decisions or increases cognitive load and reduces performance. The project contributes to the literature on human–AI interaction by providing evidence on how uncertainty communication shapes trust, confidence, and decision quality.

Principal Investigator: Prof. Dr. Oliver Hinz

Research Assistants: Frederik Hering

The financial advisory landscape is undergoing a fundamental transformation toward automated, digital-first solutions. However, while current robo-advisors have facilitated access to financial advice and LLM-based chatbots offer more coherent responses, large-scale empirical analyses reveal critical flaws. Current systems offer remarkably limited individualization, often confirming investor preconceptions with generic portfolios rather than providing truly optimal, personalized guidance. This limits their economic benefit and fails to build lasting trust.

This project investigates the design, feasibility, and user acceptance of hyper-personalized financial co-pilots: proactive AI agents designed to serve as genuinely trusted financial advisors. Moving beyond simple chatbots, the approach leverages advanced context engineering to maintain a secure, persistent memory of the client's financial goals, risk tolerance, portfolio, past conversations, and stated values. This stateful, evolving understanding enables demonstrably competent advice, which is the primary driver of sustainable user trust. Even basic personalization can significantly increase client investment volumes, highlighting the immense potential of true hyper-personalization.

Through qualitative research, prototype development, and a controlled user study, this project will provide actionable insights into how agentic AI can enhance user trust, improve financial decision-making, and offer financial institutions a scalable model for premium, personalized advisory services.

Principal Investigator: Prof. Dr. Oliver Hinz

Artificial intelligence (AI) is nowadays widely used by businesses and organizations. However, AI is known to yield disparate outcomes. Disparate outcomes systematically deviate from statistical, moral, or regulatory standards, especially in ways that disadvantage groups of people from certain sociodemographics (race, gender, or other attributes deemed sensitive). Importantly, it is not sufficient to simply omit these sensitive attributes from the data because other attributes may serve as proxies. This has been shown, e.g., in the field of credit scoring. A prominent example is the Apple credit card, which does not consider the gender of users in its analysis, yet women have been systematically granted lower credit limits than men.

Disparate outcomes can be defined in different ways and depend on the context and legal framework applicable. Two of the most common definitions are statistical parity and equality of opportunity. Statistical parity is common in many legal frameworks in the US and measures disparate outcomes as the difference in the distribution of a predicted label of a group vs. the rest of the population. Notably, automatically locating disparate outcomes in AI applications is non-trivial. First, the sensitive attributes inducing disparate outcomes may be numerous, high-dimensional, and both categorical (e.g., race) and continuous (e.g., age). Consequently, a simple brute-force search is often computationally intractable. Second, disparate outcomes can be defined in various ways that partly contradict each other. The choice of the definition must be carefully tailored to the context of the application.

This project aims to develop a tool that, given a set of sensitive attributes, automatically locates the group of individuals affected by disparate outcomes. The human auditor would simply choose a set of sensitive attributes (e.g., age, gender, race, disability), a definition of disparate outcomes (e.g., equality of opportunity) and apply the tool on the labeled data. The tool outputs the groups that are subject to statistically significant disparate outcomes. Henceforth, we refer to this tool as ALD (Automatic Location of Disparate outcomes). With ALD, our project aims to provide a data-driven tool that handles all types of sensitive attributes in low- and high-dimensional settings and can be applied across different use cases. Thereby, the project advances the body of research on algorithmic bias by providing a benchmark for algorithmic audits that is broadly applicable. Importantly, ALD shall audit AI applications deployed in practice. This helps businesses and organizations to mitigate reputational and legal risks and ultimately contributes to protecting end users from algorithmic discrimination.

Principal Investigator: Prof. Dr. Oliver Hinz

Research Assistants: Moritz von Zahn

Contemporary artificial intelligence (AI) systems' high predictive performance frequently comes at the expense of users' understanding of why systems produce a certain output. This can create considerable downsides so researchers developed explainable AI (XAI). The most popular XAI methods are feature-based explanations (e.g., SHAP values and LIME), which explain the behavior of an AI system by showing the contribution of individual features to the prediction. Importantly, feature-based explanations provide either local or global explanations. On the one hand, local explanations “zoom into” individual AI predictions and show how each input feature has contributed to the ultimate prediction. On the other hand, global explanations holistically explain the AI predictions at hand by showing the model-wide distribution of feature contributions. While both local and global explanations are widespread in research and practice, a systematic comparison of the two is lacking. We aim to address this research gap by systematically comparing local vs. global explanations with regard to user trust (in the system and its predictions), decision performance (economical consequences of user decisions), and human learning (users learning from XAI). To achieve this, we will conduct an incentivized, preregistered online experiment building upon a real-world use case.

Principal Investigator: Prof. Dr. Oliver Hinz

Research Assistants: Moritz von Zahn