Benchmarking the stack

Gambetta said that one of the things that IBM did recently is rewrite the Qiskit stack in Rust, rather than its original Python. The expectation was that compiled code would lead to better performance overall, which likely motivated the decision to develop a cross-platform benchmarking suite. The suite involves generating quantum circuits from abstract representations of algorithms, and manipulating and optimizing the ensuing circuit. The resulting circuits were then examined for how many gate operations they required (fewer is generally better) and how quickly they could be transformed.

Unsurprisingly, IBM feels that Qiskit did extremely well on its tests, but the details provide a window into where things stand in the world of quantum-computing software stacks, as seven different ones were included in the tests. One detail is that building quantum circuits isn't a major time sink; most tasks completed in around a second or less, and even the worst result for one software package took only a bit more than a minute. That said, a number of the software packages failed to complete a subset of the tests.

There was also clearly a tradeoff. For example, a toolkit called Staq tended to complete circuit building faster than Qiskit, but the resulting algorithm would require more operations to complete on quantum hardware. Meanwhile, software called Tket took longer than Qiskit, but tended to produce quantum algorithms that required fewer operations to execute, especially when targeting non-IBM hardware. Fewer operations can be critical, as each operation is an opportunity for an error to ruin a calculation.

This is all valuable because many of these toolkits are open source, so it should be possible to use the code from one to inform the performance of some of the others.

Beyond Qiskit

In a somewhat related announcement, IBM is also opening up its software stack so that users don't have to be entirely dependent on its tools. Right now, the company is supporting six third-party Qiskit functions that break down into two categories.

The first can be used as stand-alone applications and are focused on providing solutions to problems for users who have no expertise programming quantum computers. One calculates the ground-state energy of molecules, and the second performs optimizations.

But the remainder are focused on letting users get more out of existing quantum hardware, which tends to be error prone. But some errors occur more often than others. These errors can be due to specific quirks of individual hardware qubits or simply because some specific operations are more error prone than others. These can be handled in two different ways. One is to design the circuit being executed to avoid the situations that are most likely to produce an error. The second is to examine the final state of the algorithm to assess whether errors likely occurred and adjust to compensate for any. And third parties are providing software that can handle both of these.

One of those third parties is Q-CTRL, and we talked to its CEO, Michael Biercuk. "We build software that is really focused on everything from the lowest level of hardware manipulation, something that we call quantum firmware, up through compilation and strategies that help users map their problem onto what has to be executed on hardware," he told Ars. (Q-CTRL is also providing the optimization tool that's part of this Qiskit update.)

"We're focused on suppressing errors everywhere that they can occur inside the processor," he continued. "That means the individual gate or logic operations, but it also means the execution of the circuit. There are some errors that only occur in the whole execution of a circuit as opposed to manipulating an individual quantum device." Biercuk said Q-CTRL's techniques are hardware agnostic and have been demonstrated on machines that use very different types of qubits, like trapped ions. While the sources of error on the different hardware may be distinct, the manifestations of those problems are often quite similar, making it easier for Q-CTRL's approach to work around the problems.

Those work-arounds include things like altering the properties of the microwave pulses that perform operations on IBM's hardware, and replacing the portion of Qiskit that converts an algorithm to a series of gate operations. The software will also perform operations that suppress errors that can occur when qubits are left idle during the circuit execution.

As a result of all these differences, he claimed that using Q-CTRL's software allows the execution of more complex algorithms than are possible via Qiskit's default compilation and execution. "We've shown, for instance, optimization with all 156 qubits on [an IBM] system, and importantly—I want to emphasize this word—successful optimization," Biercuk told Ars. "What it means is you run it and you get the right answer, as opposed to I ran it and I kind of got close."

Compensating for errors

Asif Sinay, the CEO of QEDMA, says one of the things his software does is quite similar. "When someone sends us an algorithm, we are just looking at the specific gates that he's going to run, and then we analyze the device as a function of what he sent to us," Sinay said. "Sometimes there are some tricks that you can compress the algorithm, you can do some changes inside your algorithm."

But QEDMA's software also handles errors after they occur. If a user is willing to run the algorithm multiple times, it's possible to profile the errors that occur during its execution, which will often differ between runs. "If you don't know what the noise looks like, you cannot fix the errors," Sinay said. With a sufficient profile of the noise, you can recognize when it's likely to have occurred and can compensate for that. (IBM has also demonstrated a form of this, although it's computationally expensive.) Like Biercuk, Sinay was confident that this approach would start producing useful calculations on quantum computers before we are able to perform those calculations with error-corrected quantum computers. "The middleware is the bridge to close the gap between the hardware and the application, because without the middleware, without reducing these errors, these devices will not be useful," Sinay told Ars.

As for IBM, Gambetta said that he's excited about what might be possible when algorithm developers no longer have to worry as much about reducing noise enough to get a calculation to run to completion. So far, quantum algorithms have been defined through rigorous mathematical proofs, rather than by testing different approaches on actual hardware. As the hardware and software stacks continue to mature, that may start to change. "Most of what people have been doing is working out how to get rid of the effects of noise on our quantum circuits," Gambetta said. "And so the abstraction of the circuit function is the first time algorithm developers are not going to need to worry."