I2SC Lecture Series

Balancing performance and engagement in AI augmentation: empirical evidence from an entrepreneurial pitch writing task

Augmenting human work by AI, i.e., supporting rather than replacing humans in their work, promises performance benefits while also enabling organizational goals of individual learning or intellectual diversity. There is an inherent tension to the concept of augmentation, though: AI-based systems become increasingly reliable and able to generate work-relevant output that does not require any further human input. Converging empirical evidence suggests that human-AI interaction can boost performance in decision-making and open-ended, creative tasks. At the same time, interaction challenges, such as overreliance on algorithmic advice, point towards a lack of human effort and engagement when interacting with AI. This raises the central design question of how we can maximise conflicting outcomes of performance and engagement for AI augmentation to manifest.

As part of four online lab and field experiments, we study how alternative designs of algorithmic advice might improve the interrelated outcomes of performance and engagement in the context of an entrepreneurial pitch writing task. Using an LLM to manipulate our treatment of algorithmic advice, we explore the potential of (i) elaborated algorithmic advice providing critique and reflection-focused feedback compared to (ii) solution-focused algorithmic advice improving users' pitch without any feedback. 

Analysing 976 business pitches and 404 users, we leverage both cognitive outcomes and behavioural interaction logs to capture what human engagement denotes in augmentation scenarios. We show that both designs of algorithmic advice lead to comparable improvements in business pitch quality. However, humans exposed to elaborated advice iterate on their writing more significantly and develop their business pitches more independently from the algorithmic advice. Together, these results indicate that there are different paths that enable performance gains through human-AI interaction. Our results also help explain why 'default' designs of solution-focused algorithmic advice might induce users to fixate on algorithmic advice. If effortful engagement is key to learning, skill maintenance and supervision of AI, our study illustrates how algorithmic advice may serve as a critical stimulator rather than attenuating human agency.

The Critical Role of Consent in Digital Systems

Traditional permission models tend to be system and outcome-centric—they are based on engineers' or lawyers' conception of privacy, where ease of programming or fidelity to some policy is the primary goal. Such approaches do not center users' nuanced needs around privacy and safety. For example, the forms of information exchange that people manage are complex due to factors like social identities, such as race and gender, as well as lived experiences. Yet, existing privacy controls do not allow users to indicate such dimensions as perimeters when managing flows of information across different individuals. Another example is how platforms automatically opt users into AI development without meaningful ways to specify what kinds of information, if any, they are comfortable contributing to model training. 

Instead, my research shows that consent is a better framework for considering a range of privacy, permissions, and related issues because it is user-centric and process-centric—a consent approach would not be satisfied until users can adjust who they interact with, clearly, easily, flexibly, and at any time. 

In this talk, I first briefly introduce the theoretical framework of affirmative consent to reimagine social computing systems. Drawing from feminist literature, which emphasizes the moral importance of respecting the agency of parties with less power, I defined that affirmative consent is voluntary, informed, revertible, specific, and unburdensome. These principles serve as guidelines that lead to concrete new design ideas for consentful systems. 

Then, I present examples of consentful systems. First, I will discuss developing nuanced consent-granting interfaces to address privacy issues around platforms' data collection and algorithmic inferences. Second, I will introduce Moa, a social platform I built for enabling the sharing of sensitive information. Moa's most novel element is an audience selection process that uses what I call "consent boundaries", which allow users to flexibly define each post or comment's audience based on factors such as common social identity or lived experience, all while preserving anonymity—neither senders nor recipients learn each other's identities, even as the post reaches the right audience. I will conclude the talk with my vision on consentful systems—especially on embedding consent into generative AI development to make it more just and sustainable. 

A realistic theory of prediction market accuracy

Interest in prediction markets is currently experiencing a major resurgence. However, despite their growing prominence, we still know relatively little about when and why prediction markets are accurate. Existing literature tends either to argue for their general accuracy or to highlight cases in which they underperform relative to alternative forecasting methods. Yet growing empirical evidence suggests that prediction markets vary widely in their forecasting performance. We approach this problem by developing a unified and realistic theory of prediction market accuracy that explicitly incorporates the diverse ways in which prediction markets are designed and implemented. Our research design synthesizes insights from existing empirical studies and categorizes prediction markets based on key institutional and informational features. We hypothesize that there are two ideal types of prediction markets that are highly dissimilar, yet both have the potential to generate highly accurate forecasts. This insight has broader implications for both theory and practice: it provides a foundation for more systematic evaluations of prediction markets and offers guidance for designing markets that achieve high forecasting accuracy.

Can AI help us stay resilient to propaganda? Designing for critical news consumption
This talk presents a research program on AI-supported systems for propaganda detection and critical thinking. Drawing on dual-system theory, inoculation theory, and digital nudging, the work explores how Large Language Models (LLMs) can help users recognize propaganda techniques, engage in more reflective System 2 processing, and develop greater propaganda awareness during online news consumption. Across multiple design iterations and controlled experiments, the research investigates how interventions such as explanations, contextual information, and inoculation strategies influence critical thinking and resistance to manipulative content. The findings show that these interventions can effectively increase critical engagement and propaganda awareness during news consumption. At the same time, the results show limitations of one-shot inoculation approaches, as some of these effects diminish once the systems are no longer available. Building on these findings, the talk discusses future research directions that move beyond static detection interfaces toward multimodal and conversational AI systems, including voice-based Socratic dialogue agents. The broader goal is to design interactive AI systems that promote sustained reflective reasoning and strengthen long-term resilience against manipulative and AI-generated media.