Managing extreme AI risks amid rapid progress
Artificial Intelligence (AI) is progressing rapidly, and companies are shifting their focus to developing generalist AI systems that can autonomously act and pursue goals. Increases in capabilities and autonomy may soon massively amplify AI's impact, with risks that include large-scale social h...
| Автори: | , , , , , , , , , , , , , , , , , , , , , , , , |
|---|---|
| Формат: | Internet publication |
| Мова: | English |
| Опубліковано: |
2023
|
| _version_ | 1826317763077996544 |
|---|---|
| author | Bengio, Y Hinton, G Yao, A Song, D Abbeel, P Darrell, T Harari, YN Zhang, Y-Q Xue, L Shalev-Shwartz, S Hadfield, G Clune, J Maharaj, T Hutter, F Baydin, AG McIlraith, S Gao, Q Acharya, A Krueger, D Dragan, A Torr, P Russell, S Kahneman, D Brauner, J Mindermann, S |
| author_facet | Bengio, Y Hinton, G Yao, A Song, D Abbeel, P Darrell, T Harari, YN Zhang, Y-Q Xue, L Shalev-Shwartz, S Hadfield, G Clune, J Maharaj, T Hutter, F Baydin, AG McIlraith, S Gao, Q Acharya, A Krueger, D Dragan, A Torr, P Russell, S Kahneman, D Brauner, J Mindermann, S |
| author_sort | Bengio, Y |
| collection | OXFORD |
| description | Artificial Intelligence (AI) is progressing rapidly, and companies are shifting their focus to developing generalist AI systems that can autonomously act and pursue goals. Increases in capabilities and autonomy may soon massively amplify AI's impact, with risks that include large-scale social harms, malicious uses, and an irreversible loss of human control over autonomous AI systems. Although researchers have warned of extreme risks from AI, there is a lack of consensus about how exactly such risks arise, and how to manage them. Society's response, despite promising first steps, is incommensurate with the possibility of rapid, transformative progress that is expected by many experts. AI safety research is lagging. Present governance initiatives lack the mechanisms and institutions to prevent misuse and recklessness, and barely address autonomous systems. In this short consensus paper, we describe extreme risks from upcoming, advanced AI systems. Drawing on lessons learned from other safety-critical technologies, we then outline a comprehensive plan combining technical research and development with proactive, adaptive governance mechanisms for a more commensurate preparation.
|
| first_indexed | 2025-03-11T16:59:04Z |
| format | Internet publication |
| id | oxford-uuid:79b0f23a-9dbc-43d1-88b6-f90b1318e361 |
| institution | University of Oxford |
| language | English |
| last_indexed | 2025-03-11T16:59:04Z |
| publishDate | 2023 |
| record_format | dspace |
| spelling | oxford-uuid:79b0f23a-9dbc-43d1-88b6-f90b1318e3612025-03-06T16:21:24ZManaging extreme AI risks amid rapid progressInternet publicationhttp://purl.org/coar/resource_type/c_7ad9uuid:79b0f23a-9dbc-43d1-88b6-f90b1318e361EnglishSymplectic Elements2023Bengio, YHinton, GYao, ASong, DAbbeel, PDarrell, THarari, YNZhang, Y-QXue, LShalev-Shwartz, SHadfield, GClune, JMaharaj, THutter, FBaydin, AGMcIlraith, SGao, QAcharya, AKrueger, DDragan, ATorr, PRussell, SKahneman, DBrauner, JMindermann, SArtificial Intelligence (AI) is progressing rapidly, and companies are shifting their focus to developing generalist AI systems that can autonomously act and pursue goals. Increases in capabilities and autonomy may soon massively amplify AI's impact, with risks that include large-scale social harms, malicious uses, and an irreversible loss of human control over autonomous AI systems. Although researchers have warned of extreme risks from AI, there is a lack of consensus about how exactly such risks arise, and how to manage them. Society's response, despite promising first steps, is incommensurate with the possibility of rapid, transformative progress that is expected by many experts. AI safety research is lagging. Present governance initiatives lack the mechanisms and institutions to prevent misuse and recklessness, and barely address autonomous systems. In this short consensus paper, we describe extreme risks from upcoming, advanced AI systems. Drawing on lessons learned from other safety-critical technologies, we then outline a comprehensive plan combining technical research and development with proactive, adaptive governance mechanisms for a more commensurate preparation. |
| spellingShingle | Bengio, Y Hinton, G Yao, A Song, D Abbeel, P Darrell, T Harari, YN Zhang, Y-Q Xue, L Shalev-Shwartz, S Hadfield, G Clune, J Maharaj, T Hutter, F Baydin, AG McIlraith, S Gao, Q Acharya, A Krueger, D Dragan, A Torr, P Russell, S Kahneman, D Brauner, J Mindermann, S Managing extreme AI risks amid rapid progress |
| title | Managing extreme AI risks amid rapid progress |
| title_full | Managing extreme AI risks amid rapid progress |
| title_fullStr | Managing extreme AI risks amid rapid progress |
| title_full_unstemmed | Managing extreme AI risks amid rapid progress |
| title_short | Managing extreme AI risks amid rapid progress |
| title_sort | managing extreme ai risks amid rapid progress |
| work_keys_str_mv | AT bengioy managingextremeairisksamidrapidprogress AT hintong managingextremeairisksamidrapidprogress AT yaoa managingextremeairisksamidrapidprogress AT songd managingextremeairisksamidrapidprogress AT abbeelp managingextremeairisksamidrapidprogress AT darrellt managingextremeairisksamidrapidprogress AT harariyn managingextremeairisksamidrapidprogress AT zhangyq managingextremeairisksamidrapidprogress AT xuel managingextremeairisksamidrapidprogress AT shalevshwartzs managingextremeairisksamidrapidprogress AT hadfieldg managingextremeairisksamidrapidprogress AT clunej managingextremeairisksamidrapidprogress AT maharajt managingextremeairisksamidrapidprogress AT hutterf managingextremeairisksamidrapidprogress AT baydinag managingextremeairisksamidrapidprogress AT mcilraiths managingextremeairisksamidrapidprogress AT gaoq managingextremeairisksamidrapidprogress AT acharyaa managingextremeairisksamidrapidprogress AT kruegerd managingextremeairisksamidrapidprogress AT dragana managingextremeairisksamidrapidprogress AT torrp managingextremeairisksamidrapidprogress AT russells managingextremeairisksamidrapidprogress AT kahnemand managingextremeairisksamidrapidprogress AT braunerj managingextremeairisksamidrapidprogress AT mindermanns managingextremeairisksamidrapidprogress |