A BenchCouncil view on benchmarking emerging and future computing
The measurable properties of the artifacts or objects in the computer, management, or finance disciplines are extrinsic, not inherent — dependent on their problem definitions and solution instantiations. The processes of problem definition, solution instantiation, and measurement are entangled. Only...
| Main Author: | |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
KeAi Communications Co. Ltd.
2022-04-01
|
| Series: | BenchCouncil Transactions on Benchmarks, Standards and Evaluations |
| Subjects: | |
| Online Access: | http://www.sciencedirect.com/science/article/pii/S2772485922000515 |
| _version_ | 1827986907636695040 |
|---|---|
| author | Jianfeng Zhan |
| author_facet | Jianfeng Zhan |
| author_sort | Jianfeng Zhan |
| collection | DOAJ |
| description | The measurable properties of the artifacts or objects in the computer, management, or finance disciplines are extrinsic, not inherent — dependent on their problem definitions and solution instantiations. The processes of problem definition, solution instantiation, and measurement are entangled. Only after the instantiation can the solutions to the problem be measured. Definition, instantiation, and measurement have complex mutual influences. Meanwhile, the technology inertia brings instantiation bias — trapped into a subspace or even a point at a high-dimension solution space. These daunting challenges, which emerging computing aggravates, make metrology cannot work for benchmark communities. It is pressing to establish independent benchmark science and engineering.This article presents a unifying benchmark definition, a conceptual framework, and a traceable and supervised learning-based benchmarking methodology, laying the foundation for benchmark science and engineering. I also discuss BenchCouncil’s plans for emerging and future computing. The ongoing projects include defining the challenges of intelligence, instinct, quantum computers, Metaverse, planet-scale computers, and reformulating data centers, artificial intelligence for science, and CPU benchmark suites. Also, BenchCouncil will collaborate with ComputerCouncil on open-source computer systems for planet-scale computing, AI for science systems, and Metaverse. |
| first_indexed | 2024-04-09T23:38:48Z |
| format | Article |
| id | doaj.art-d5175f98e39b4bd58f6925457b3a9b24 |
| institution | Directory Open Access Journal |
| issn | 2772-4859 |
| language | English |
| last_indexed | 2024-04-09T23:38:48Z |
| publishDate | 2022-04-01 |
| publisher | KeAi Communications Co. Ltd. |
| record_format | Article |
| series | BenchCouncil Transactions on Benchmarks, Standards and Evaluations |
| spelling | doaj.art-d5175f98e39b4bd58f6925457b3a9b242023-03-20T04:05:16ZengKeAi Communications Co. Ltd.BenchCouncil Transactions on Benchmarks, Standards and Evaluations2772-48592022-04-0122100064A BenchCouncil view on benchmarking emerging and future computingJianfeng Zhan0Research Center for Advanced Computer Systems, Institute of Computing Technology, Chinese Academy of Sciences, ChinaThe measurable properties of the artifacts or objects in the computer, management, or finance disciplines are extrinsic, not inherent — dependent on their problem definitions and solution instantiations. The processes of problem definition, solution instantiation, and measurement are entangled. Only after the instantiation can the solutions to the problem be measured. Definition, instantiation, and measurement have complex mutual influences. Meanwhile, the technology inertia brings instantiation bias — trapped into a subspace or even a point at a high-dimension solution space. These daunting challenges, which emerging computing aggravates, make metrology cannot work for benchmark communities. It is pressing to establish independent benchmark science and engineering.This article presents a unifying benchmark definition, a conceptual framework, and a traceable and supervised learning-based benchmarking methodology, laying the foundation for benchmark science and engineering. I also discuss BenchCouncil’s plans for emerging and future computing. The ongoing projects include defining the challenges of intelligence, instinct, quantum computers, Metaverse, planet-scale computers, and reformulating data centers, artificial intelligence for science, and CPU benchmark suites. Also, BenchCouncil will collaborate with ComputerCouncil on open-source computer systems for planet-scale computing, AI for science systems, and Metaverse.http://www.sciencedirect.com/science/article/pii/S2772485922000515Benchmark science and engineeringBenchmarking challengesExtrinsic propertyProcess entanglementInstantiation biasUnified benchmark definition |
| spellingShingle | Jianfeng Zhan A BenchCouncil view on benchmarking emerging and future computing BenchCouncil Transactions on Benchmarks, Standards and Evaluations Benchmark science and engineering Benchmarking challenges Extrinsic property Process entanglement Instantiation bias Unified benchmark definition |
| title | A BenchCouncil view on benchmarking emerging and future computing |
| title_full | A BenchCouncil view on benchmarking emerging and future computing |
| title_fullStr | A BenchCouncil view on benchmarking emerging and future computing |
| title_full_unstemmed | A BenchCouncil view on benchmarking emerging and future computing |
| title_short | A BenchCouncil view on benchmarking emerging and future computing |
| title_sort | benchcouncil view on benchmarking emerging and future computing |
| topic | Benchmark science and engineering Benchmarking challenges Extrinsic property Process entanglement Instantiation bias Unified benchmark definition |
| url | http://www.sciencedirect.com/science/article/pii/S2772485922000515 |
| work_keys_str_mv | AT jianfengzhan abenchcouncilviewonbenchmarkingemergingandfuturecomputing AT jianfengzhan benchcouncilviewonbenchmarkingemergingandfuturecomputing |