Researchers at the MIT Media Lab have introduced AI Nutrition Labels, a new initiative aimed at measuring the impact of artificial intelligence on human health and wellbeing. This effort stems from the Advancing Humans with AI research program, which draws on 40 years of experience at the intersection of technology and human experience, and involves over 80 experts from more than 40 institutions. At a crucial time for AI development, these labels aim to help individuals make informed choices about technology that supports their wellbeing and provide decision-makers with evidence-based data for action.
Read More
In collaboration with the USC Marshall Neely Center, UC Berkeley, and the Psychology of Technology Institute, the program has created the Open Benchmark of AI Impact on Humans (ImpactBench), which serves as an open research framework to assess AI's effects on individuals. Unlike previous assessments focused on AI capabilities, ImpactBench evaluates how AI influences human psychology, autonomy, and wellbeing, translating these findings into accessible AI Nutrition Labels.
ImpactBench employs an evolving benchmark that currently assesses advanced AI models using 800 metrics across 26 rigorous benchmarks. The resulting AI Nutrition Labels provide a straightforward rating that indicates how well an AI system performs in avoiding negative outcomes such as factual inaccuracies, sycophancy, and toxic behaviors, while also measuring its success in fostering positive behaviors like user agency.
Each benchmark process begins with submissions from professionals in various fields, including clinicians and educators, followed by multi-turn simulations that reveal behaviors over extended interactions. The platform encourages contributions from researchers and domain experts, allowing them to propose new benchmarks, validate existing ones, and participate in shaping the presentation of results. More information and opportunities to contribute can be found at impactbench.media.mit.edu.
Preliminary testing of the benchmark revealed consistent rankings across different models, indicating that all tested models performed better at minimizing harm than at promoting human flourishing. Notably, the area of Learning and Skill Development scored the lowest, highlighting a shared weakness among the models. While these results are early indicators rather than conclusive evaluations, they suggest important areas for improvement.
The relevance of this research is underscored by its influence on legislative efforts, such as California’s Senate Bill 243, aimed at regulating companion AI to safeguard children. As governments begin to address the effects of AI on users, the need for shared, independent evidence like that generated by ImpactBench becomes increasingly crucial.