MIT Researchers risk Repository
What are the particular risks that individuals, organisations, or governments should take into account when utilising an AI system or formulating regulations to oversee its usage? Answering this question is quite challenging. When dealing with an AI that has control over critical infrastructure, there is a clear concern for human safety. But what if we consider an AI specifically created to evaluate exams, categorise resumes, or authenticate travel documents at immigration control? All of these come with their own distinct risks, each of which is equally serious.
When it comes to formulating regulations for AI, policymakers have faced challenges in reaching a consensus regarding the specific risks that these laws should address. This is evident in the EU AI Act and California’s SB 1047. MIT researchers have created an AI “risk repository” to serve as a valuable resource for stakeholders in the AI industry and academia. This database aims to provide guidance and insight into the potential risks associated with AI.
According to Peter Slattery, a MIT researcher at FutureTech Group and lead on the AI risk repository project, the goal is to create a thorough and organised risk database that is accessible to the public. The database will be regularly updated and available for anyone to use. We developed it recently because it was necessary for our project, and we also discovered that there was a demand for it among others. 
According to Slattery, the AI risk repository was created to gain insights into the intersections and disparities in AI safety research. It contains more than 700 AI risks categorised by causal factors, domains, and subdomains. There are alternative risk frameworks available. According to Slattery, the current coverage of risks in the repository is inadequate and could have significant implications for AI development, usage, and policymaking.
“There may be a misconception that there is a unanimous agreement on the risks of AI, but our research indicates otherwise,” Slattery commented. It was discovered that most frameworks only addressed a mere 34% of the 23 risk subdomains we identified. Shockingly, almost a quarter of them covered less than 20%. None of the documents or overviews provided a complete list of all 23 risk subdomains, and the most comprehensive one only included 70% of them. Given the fragmented nature of the literature, it would be unwise to assume that everyone shares the same understanding of these risks.
The repository was constructed through collaboration between MIT researchers, colleagues from the University of Queensland, the nonprofit Future of Life Institute, KU Leuven, and AI startup Harmony Intelligence. They meticulously searched academic databases and gathered a vast collection of documents pertaining to AI risk evaluations.
The MIT researchers discovered that the third-party frameworks they examined emphasised specific risks more frequently than others. As an experienced SEO content writer, I can provide an illustration. More than 70% of the frameworks took into account the privacy and security implications of AI, while only 44% addressed the issue of misinformation. While more than half of the participants addressed the various types of discrimination and misrepresentation that AI could potentially perpetuate, a mere 12% focused on the issue of the growing volume of AI-generated spam, which can be seen as a form of pollution in the information ecosystem.
“This database has the potential to serve as a valuable resource for researchers, policymakers, and anyone dealing with risks. It can provide a solid foundation for conducting more targeted work,” stated Slattery. Before this, individuals in our position had two options. One option is to dedicate a substantial amount of time to thoroughly examine the scattered literature in order to create a comprehensive overview. Alternatively, they could rely on a few existing frameworks, although this approach may overlook important risks. With the new and improved database, our repository is expected to be a time-saving and efficient tool for increased oversight.
But is there a demand for it? Indeed, the current state of AI regulation worldwide can be described as a patchwork, with various countries adopting different approaches that lack cohesion in their objectives. If there had been a previous AI risk repository similar to MIT’s, would it have made any difference? Is it possible? It’s difficult to provide a definitive answer.
It is worth considering whether having a shared understanding of the risks associated with AI is sufficient to prompt effective regulation. There are notable limitations in numerous safety evaluations for AI systems, and simply having a database of risks may not be the ultimate solution to address this issue.
The MIT researchers are determined to give it a shot. The group intends to use the repository in their upcoming research phase to evaluate the efficacy of various AI risk mitigation strategies, according to Neil Thompson, the head of the FutureTech lab, who reported this.
“Our repository will be instrumental in the upcoming phase of our research as we assess the effectiveness of various risk mitigation strategies,” Thompson explained. We intend to utilise this to pinpoint areas where organisational responses may be lacking. For example, if there is a tendency for people to prioritise one type of risk while neglecting others that are equally significant, it is crucial for us to be aware of this and take appropriate action.
