Mapping global dynamics of benchmark creation and saturation in artificial intelligence
Recent studies raised concerns over the state of AI benchmarking, reporting issues such as benchmark overfitting, benchmark saturation and increasing centralization of benchmark dataset creation. To facilitate monitoring of the health of the AI benchmarking ecosystem, the authors introduce methodolo...
Main Authors: | Simon Ott, Adriano Barbosa-Silva, Kathrin Blagec, Jan Brauner, Matthias Samwald |
---|---|
Format: | Article |
Language: | English |
Published: |
Nature Portfolio
2022-11-01
|
Series: | Nature Communications |
Online Access: | https://doi.org/10.1038/s41467-022-34591-0 |
Similar Items
-
A curated, ontology-based, large-scale knowledge graph of artificial intelligence tasks and benchmarks
by: Kathrin Blagec, et al.
Published: (2022-06-01) -
Examining perceptions of the usefulness and usability of a mobile-based system for pharmacogenomics clinical decision support: a mixed methods study
by: Kathrin Blagec, et al.
Published: (2016-02-01) -
An ontology-based, mobile-optimized system for pharmacogenomic decision support at the point-of-care.
by: Jose Antonio Miñarro-Giménez, et al.
Published: (2014-01-01) -
Universal artificial intelligence: evaluation and benchmarks
by: Mishra, Pallavi
Published: (2017) -
THE IMPACT OF ARTIFICIAL INTELLIGENCE ON THE CREATION OF MULTIMEDIA CONTENT
by: Srečo Zakrajšek
Published: (2024-03-01)