Abstract
Benchmarking is how the performance of a computing system is determined. Surprisingly, even for classical computers this is not a straightforward process. One must choose the appropriate benchmark and metrics to extract meaningful results. Different benchmarks test the system in different ways, and each individual metric may or may not be of interest. Choosing the appropriate approach is tricky. The situation is even more open ended for quantum computers, where there is a wider range of hardware, fewer established guidelines, and additional complicating factors. Notably, quantum noise significantly impacts performance and is difficult to model accurately. Here, we discuss benchmarking of quantum computers from a computer architecture perspective and provide numerical simulations highlighting challenges that suggest caution.
| Original language | English (US) |
|---|---|
| Article number | 142 |
| Journal | ACM Computing Surveys |
| Volume | 54 |
| Issue number | 7 |
| DOIs | |
| State | Published - Sep 2022 |
Bibliographical note
Funding Information:Supported in part by NSF CCF-1553042. Authors’ addresses: S. Resch and U. R. Karpuzcu, University of Minnesota, 200 Union St SE, Minneapolis, Minnesota, 55455; emails: {resc0059, ukarpuzc}@umn.edu. Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from permissions@acm.org. © 2021 Association for Computing Machinery. 0360-0300/2021/06-ART142 $15.00 https://doi.org/10.1145/3464420
Publisher Copyright:
© 2021 ACM.
Keywords
- Benchmarking quantum computers
- quantum noise