Hey Everyone,
With OpenAI raising even more funds and Nvidia’s profits growing at as astounding rate, it’s never been more important to understand and evaluate AI risks. That’s why I think the AI Risk Repository is so important.
The AI Risk Repository is a comprehensive database developed by the FutureTech Group at MIT’s Computer Science & Artificial Intelligence Laboratory (CSAIL).
It aims to provide an accessible and updatable overview of the risks associated with advanced AI systems. This repository documents over 700 potential risks, making it one of the most extensive sources of information on AI risks available today.
OpenAI is seeking to raise funds at a valuation a of more than $100 billion. Thrive will reportedly throw in around $1 billion for its part. Microsoft is also expected to participate, according to the WSJ. Meanwhile Nvidia’s earnings were great, but not at the same pace as previous quarters. OpenAI has already burned though $8.5 billion on AI training and staffing, and promises AGI and a revolution. At Klarna, that revolution is already near (via ). Meanwhile with their Earnings, Nvidia announced a $50 Billion stock buyback. No alarm bells for you right?
Nvidia reported 122% revenue growth on surging demand for data center chips. In short, the AI bubble hasn’t even begun to pop.
Humanity isn’t Ready
But as a society and civilization we are talking about AI risks a lot less. Why is that? The way American and China shape AI regulation around AI risks will shape humanity’s relationship with the technology. I asked of Futuristic Lawyer Newsletter for his take on this.
AI Risk Repository – August 29th, 2024.
The AI Risk Database captures 700+ risks extracted from 43 existing frameworks, with quotes and page numbers.
Website: https://airisk.mit.edu/
Preprint: Read the paper.
Database: Make a Google Sheets copy.
Feedback: Give Detailed Feedback.
The repository includes 777 risks extracted from 43 different AI risk classifications, frameworks, and taxonomies.
51% of the risks extracted were attributed to AI systems, while 34% were attributed to humans.
Six times more risks were presented as occurring after (65%) than before deployment (10%). What can we understand big picture from this project?
According to the repository, some of the top AI risks include:
Misalignment with Human Values: AI systems may act in ways that are misaligned with human values and ethical standards, leading to unintended negative consequences.
Autonomous Weapons: The development and deployment of autonomous weapons pose risks of misuse, escalation of conflicts, and unintended consequences in warfare.
Privacy Invasion: AI technologies can be used to collect, analyze, and exploit personal data in ways that infringe on individual privacy rights.
Bias and Discrimination: AI systems can perpetuate and even exacerbate existing biases and discrimination in decision-making processes, affecting areas like hiring, law enforcement, and lending.
Economic Displacement: The automation of jobs through AI can lead to significant economic displacement, affecting livelihoods and creating social inequality.
Loss of Human Control: As AI systems become more autonomous, there is a risk of losing effective human oversight and control, potentially leading to harmful outcomes.
Security Threats: AI systems can be vulnerable to attacks and manipulations, such as adversarial attacks, which can undermine their reliability and safety.
Unintended Consequences: AI systems may produce unintended and unforeseen consequences due to their complexity and the difficulty of predicting all possible outcomes.
Most people don’t even begin to realize the risks posed by this technology. I’m not even sure if the benefits outweigh the costs here or if the profit motive can regulate something this potentially adversarial to human well-being and the best of our human values.
These risks underscore the importance of careful design, oversight, and regulation of AI technologies to ensure they are developed and deployed responsibly.
Understanding the Actual Risks of AI
By , August, 2024. Tobias is a lawyer and writes out of Denmark.
The structure and key findings of the AI risk repository + comparison with risk frameworks applied by BigTech.
Introduction
On August 14, MIT FutureTech and its partners released the AI risk repository – the most comprehensive database of AI risks assembled to date. The stated goal is to create a common frame of reference for understanding and addressing the risks from AI.
You can explore the database for yourself and read the preprint via the website here.
In this post, we will look closer into how the new database can be used in combination with the “domain taxonomy” and the “casual taxonomy” and how it differs from similar risk frameworks that are used by BigTech.
The AI risk repository was released by Peter Slattery, Neil Thompson, Alexander Saeri and Emily Grundy from MIT FutureTech in collaboration with Michael Noetel and Jess Graham from the University of Queensland, James Dao and Soroush Pour from Harmony Intelligence, Stephen Casper from MIT CSAIL, and Risto Uuk from Future of Life Institute and KU Leuven.
How the AI Repository Can Be Used & the Domain Taxonomy
As I see it, the project has two important use cases: It provides practitioners with a clear overview of the full risk landscape, and it helps to reveal research gaps in the current literature on AI risks.
As for the first point, the literature on AI risks is rich but fragmented.
There are several good risk frameworks that can be used by organizations that want to (or are legally obligated to) analyze and mitigate potential risks when deploying an AI model. However, up until now, there haven’t been any coordinated efforts to create a database like the AI risk repository which combines insights from many different frameworks. As a result, organizations, risk evaluators, security professionals, and policymakers do not have a clear overview of the full risk landscape, and risk mitigation plans and policies may be incomplete and insufficient.
Secondly, the AI risk repository reveals research gaps in the current literature on AI risks.
Building on risk classification systems from 43 peer-reviewed articles, preprints, conference papers, and other reports – that were carefully selected after screening through more than 17.000 documents – the authors identify 777 AI risks in total. Each of these risks is listed in the AI risk database with a short description from the relevant paper and a reference citation.
The risks are also grouped into 7 domains and 23 sub-domains. The authors refer to this categorization as the “domain taxonomy”.
The 7 risk domains with percentage points from highest to lowest according to how often they were cited across the 43 risk frameworks look as follows:
AI system safety, failures, and limitations (76%)
Socioeconomic and environmental harms (73%)
Discrimination and toxicity (71%)
Privacy and security (68%)
Malicious actors and misuse (68%)
Misinformation (44%)
Human-Computer Interaction (41%)
And for the 23 sub-domains:
Unfair discrimination and misrepresentation (63%)
Compromise of privacy by obtaining, leaking or correctly inferring sensitive information (61%)
Lack of capability or robustness (59%)
Cyberattacks, weapon development or use, and mass harm (54%)
AI pursuing its own goals in conflict with human goals or values (46%)
Disinformation, surveillance, and influence at scale (41%)
False or misleading information (39%)
Power centralization and unfair distribution of benefits (37%)
Exposure to toxic content (34%)
Increased inequality and decline in employment quality (34%)
Fraud, scams, and targeted manipulation (34%)
AI system security vulnerabilities and attacks (32%)
Economic and cultural devaluation of human effort (32%)
Governance failure (32%)
Environmental harm (32%)
Loss of human agency and autonomy (27%)
Lack of transparency or interpretability (27%)
Overreliance and unsafe use (24%)
Unequal performance across groups (20%)
AI possessing dangerous capabilities (20%)
Pollution of information ecosystem and loss of consensus reality (12%)
Competitive dynamics (12%)
AI welfare and rights (2%)
Of all these sub-domains, I am generally most concerned about a combination of “pollution of information ecosystem and loss of consensus reality” and “overreliance and unsafe use”. The personalized infinite scrolling loops on social media platforms like Facebook, Instagram, TikTok, and X, are both addictive and polluting and causing mental and spiritual harm, especially to the young users that are being targeted. As we can see, these two sub-domains appear to be underemphasized across the AI risk frameworks relative to other sub-domains
Casual Taxonomy
The AI risk repository does not only group the identified risks according to domain categories but also by “casual factors”.
The risks are sorted by “entity”, whether the risk is caused by an AI or a human, “intent” whether the risk occurs due to an intentional or unintentional outcome from pursuing a goal, and “timing” whether the risk occurs pre-deployment or post-deployment. Specifically, it looks like this:
Looking at the first two categories, I am reminded of how Mark Zuckerberg argued in a blog post following Llama 3.1s release, “Open Source AI Is the Path Forward” that AI should be developed open-source (according to Meta’s home-cooked version of “open source”) for safety reasons:
“My framework for understanding safety is that we need to protect against two categories of harm: unintentional and intentional. Unintentional harm is when an AI system may cause harm even when it was not the intent of those running it to do so. For example, modern AI models may inadvertently give bad health advice. Or, in more futuristic scenarios, some worry that models may unintentionally self-replicate or hyper-optimize goals to the detriment of humanity. Intentional harm is when a bad actor uses an AI model with the goal of causing harm.
It’s worth noting that unintentional harm covers the majority of concerns people have around AI – ranging from what influence AI systems will have on the billions of people who will use them to most of the truly catastrophic science fiction scenarios for humanity. On this front, open source should be significantly safer since the systems are more transparent and can be widely scrutinized. Historically, open source software has been more secure for this reason”
Considering that the majority of identified risks in the AI risk repository comes from the AI system itself rather than human decision-making and that more risks are “unintentional” than “intentional”, Zuckerberg seems to raise a valid point.
Open code is a better protection against unintentional harms that are caused by system failures because users of the model have access to do thorough security testing and report and fix mistakes.
At the same time, it’s impossible for the developer of an open foundation model to control how the model is used since it can be installed and run on a local hard drive. Guardrails can be installed by the developer but anyone with a strong interest in bypassing these guardrails is typically able to do so. The same cannot be said for closed AI models where the developer has much more control over its application.
An example of intentional harm humans can do with open-access AI is to make AI-generated nudes. San Francisco’s city attorney, David Chiu recently brought a case to shut down 16 of the most popular websites and apps that use AI to “undress” or “nudify” women and girls. Quoting from the complaint:
“By exploiting open-source AI image generation models, these websites and apps manipulate images of real women and girls without their consent to create photorealistic images showing these women and girls with AI-generated nude bodies and intimate body parts (..)
Nonconsensual intimate images (“NCII”) generated from Defendants’ websites—and other similar websites—are used to bully, threaten, and humiliate women and girls (..)
Victims have little to no recourse, as they face significant obstacles to remove these images once they have been disseminated. They are left with profound psychological, emotional, economic, and reputational harms, and without control and autonomy over their bodies and images.”
By treating the code to AI models as a trade secret, the chances of intentional harms would be significantly reduced. On the other hand, BigTech’s standing practice of refusing to reveal any technical details about how their models work raises another host of security concerns.
The open AI vs. closed AI discussion is not at all straightforward but it currently seems like Meta’s open approach is the winning strategy, and the premise that most AI harms are unintentionally caused by the systems seems to be in line with the statistics from the AI risk repository.
Comparison with OpenAI’s Preparedness Framework and Google DeepMind’s Safety Work
In comparison to the comprehensive overview in the AI risk repository, OpenAI managed to identify a grand total of four risks in its “preparedness framework” which covers the company’s “processes to track, evaluate, forecast, and protect against catastrophic risks posed by increasingly powerful models.”
Read: OpenAI Is a Leader in AI Unsafety
OpenAI used the preparedness framework to describe, evaluate, and mitigate risks before releasing GPT-4o, according to GPT-4o System Card. OpenAI assessed that GPT-4o scored low on three of the four risk categories in the framework, “Cybersecurity”, “Biological Threats”, and “Model Autonomy”, while scoring medium on the fourth category, “Persuasion” due to the model’s capability to generate convincing text about politically sensitive topics.
Generally, OpenAI tends to be fixated on concerns that would make a good plot line for a science-fiction movie. The preparedness framework seems to be built around the imaginative danger of a superintelligent AI that develops an agency of its own, engages in warfare with never-before-seen bio- and cybersecurity weapons, and manipulates people with superb persuasion skills in the pursuit of its uncanny mission.
Present risks related to AI are of course much more mundane, not “catastrophic” in the same sense as a nuclear war but nonetheless, the risks are, you know, real.
If OpenAI abandoned its simplistic framework in favor of the AI risks repository, GPT-5 and onwards could be evaluated based on relevant risk categories such as “energy consumption”, “fairness”, “robustness”, “privacy leakage”, “toxicity generation”, “sociocultural and political harms”, “inequality of wealth”, “AGIs with poor ethics, morals and values”, “bias”, “incompetence”, “competing for jobs”, “overreliance”, degradation”, among many others. Thorough risk assessments in these categories would be very interesting and useful to read.
If OpenAI was indeed very concerned about the dangers of its future models, it could commission a thorough risk assessment for example from non-partisan members of academia. The report should span +1000 pages and deeply consider the consequential impact of generative AI on society. If AI is the most transformative general-purpose technology in human history as claimed by prophets and profiteers, an all-encompassing risk report would be in order.
Actually, Google has done work in this direction. Google is behind five of the top 20 most cited papers on AI risk classification that are all included as resources in the AI risk repository. Google DeepMind recently released another report, The Ethics of Advanced AI Assistants (2024), which fills 274 pages and is included in the AI risk repository as well.
The granularity of the AI risk repository makes it a very good starting point to understand the actual risks of AI and brace ourselves for the negative impacts AI may have now and in the future. I hope the database will contribute to raising awareness and the general standards for AI risk assessments, although I wouldn’t expect it to be adopted by BigTech.
Credits on the AI Risk Repository: Thanks to everyone involved: Alexander Saeri, Jess Graham, Emily Grundy, Michael Noetel, Risto Uuk, Soroush J. Pour, James Dao, Stephen Casper, and Neil Thompson. Follow them on LinkedIn for more insights on AI risks.
writes the EU AI Act Newsletter here on Substack.
Read More in AI Supremacy
