I2SC Lecture Series

Policy Impact: From AI Regulation to Academic Inclusion

This talk examines how policies affect people across multiple domains. I analyze surveillance loopholes in the EU AI Act introduced by German regulators that potentially compromise civil liberties. The discussion then critiques academic name-change policies that create unnecessary barriers for transgender and queer scholars. Finally, I evaluate academic publishing frameworks that exclude underrepresented communities and propose practical reforms to create more equitable spaces. Throughout, I demonstrate how seemingly neutral policies can cause harm when developed without input from marginalized communities. 

Beyond AI Hype: Building Machine Learning Systems that Serve Communities

Technological advancement, particularly Artificial Intelligence, is often accompanied by significant hype and grand promises. But who do these technologies serve? I will motivate this talk by conducting a reality check of current technologies, such as chatbots, social media platforms, search engines, and knowledge bases for languages spoken by millions in the Horn of Africa. Drawing from hands-on experience, I will then explore the development of machine learning systems for underrepresented languages, focusing on Machine Translation and Automatic Speech Recognition for Amharic and Tigrinya. These case studies offer insight into the technical and social challenges involved, as well as the broader implications for the communities these technologies are intended to serve. I will conclude by discussing how we evaluate machine learning systems. Evaluation is central to defining scientific progress in the field, with benchmarks playing a critical role. However, benchmarks can also be strategically manipulated or misused. I will highlight how these issues manifest in practice and what they reveal about the incentives and priorities driving current AI research.

AI Safety Benchmarks Do Not Benchmark Safety

As artificial intelligence systems become increasingly integrated into critical aspects of society, the need for robust safety evaluation is crucial. However, this talk contends that current AI safety benchmarks, while offering insights into specific model behaviors, fundamentally fall short of measuring genuine safety. We argue this from multiple perspectives: First, these benchmarks typically assess only a narrow, predefined set of known risks, providing limited coverage of the vast landscape of potential failures, including complex edge cases, subtle biases, and  unknown unknowns. Second, we posit that safety is not merely the absence of quantifiable risk or a property inherent to an isolated component; rather, it is an emergent property of the entire socio-technical system operating within its real-world context. Finally, unlike mature engineering disciplines that incorporate robust methods for handling uncertainty and rely on continuous monitoring and adaptation, AI benchmarks offer static evaluations that cannot account for the dynamic nature of risk or the intersubjective and societal dimensions of what constitutes acceptable safety. By highlighting these limitations, this talk aims to demonstrate that a sole reliance on current benchmarks provides an incomplete and potentially misleading assessment of AI safety, underscoring the urgent need for more comprehensive, systemic, and context-aware evaluation paradigms.

Delving into ChatGPT usage in academic writing through excess vocabulary

Recent large language models (LLMs) can generate and revise text with human-level performance and have been widely commercialized in systems like ChatGPT. These models come with clear limitations: they can produce inaccurate information, reinforce existing biases, and be easily misused. Yet, many scientists have been using them to assist their scholarly writing. How widespread is LLM usage in the academic literature currently? To answer this question, we use an unbiased, large-scale approach, free from any assumptions about academic LLM usage. We study vocabulary changes in 14 million PubMed abstracts from 2010-2024 and show how the appearance of LLMs led to an abrupt increase in the frequency of certain style words. Our analysis based on excess word usage suggests that at least 10% of 2024 abstracts were processed with LLMs. This lower bound differed across disciplines, countries, and journals, and was as high as 30% for some PubMed sub-corpora. We show that the appearance of LLM-based writing assistants has had an unprecedented impact on the scientific literature, surpassing the effect of major world events such as the COVID-19 pandemic.

Evaluating Political Bias: Insights into Robustness and Multilinguality

Due to the widespread use of large language models (LLMs), it is crucial to understand whether they embed a specific "worldview" and what these views reflect. In this talk, I will discuss the challenges in performing a robust evaluation of political bias in LLMs both from a monolingual and a multilingual perspective. I will present the results of two studies: the first involves developing a framework for systematically evaluating political bias in LLMs, with a detailed look at biases in specific policy issues and the importance of a robust evaluation when probing LLMs. The second study examines to what extent political biases are carried over across languages. Results suggest that biases are similar across European languages. I will conclude by discussing relevant open research questions regarding political biases in LLMs.