The JDN questioned two of the main French experts in the field. The combination of the two technologies suggests a certain potential.
Is it possible to train a language model of several hundred billion parameters (or large Model Language for LLM) on a quantum computer? “If we stick to the current state of research on the subject, the answer is no. Existing quantum computers are not powerful enough to train and infer this type of model,” replied Xavier Vasques, vice-president, CTO and director of R&D within IBM France.
At IBM, we test modest -size neural networks on quantum computers with up to 5,000 doors, a door being the basic unit of quantum calculation. “We aim to reach 100 million doors in 2029, then 1 billion in 2033. This will make it possible to execute significantly larger neural networks. But it is still premature to assert that we will be able to execute LLM on its future configurations”, relativizes Xavier Vasques.
A beam of indications
“Via quantum computer science, it is proven that classic statistical algorithms, such as SVMs (For support-vector machine, editor’s note), can benefit from an exponential performance gain, in order to benefit from a supercomputer of several million qubits (or quantum bit, editor’s note) “, specifies Cyrille Allouche, responsible for quantum R&D within Eviden, a company of the ATOS group specializing in digital, cloud, big data and cybersecurity.” Similar in the generative AI which, let us recall, corresponds to matrix-vector connections. “
One of the main difficulties would be in particular in data storage. “If I want to load a volume of data in size N, I need a number of door of 3 power N. Quantum IT is therefore not compatible with the massive data loads necessary for learning models”, specifies Cyrille Allouche.
However, there is a bundle of clues that demonstrate that the marriage between generative and quantum computer science will indeed happen. Generative AI bases, large language models are none other than neural networks. However, a founding article published in June 2021 in the scientific journal Nature Computational Science shows that the training of quantum neural networks is faster than their conventional equivalents (cf. graphic below). “The models used as part of this study are certainly of modest size”, recognizes Xavier Vasques. “This result nevertheless suggests significant potential with networks that would have a much larger number of parameters.”
Another element: the quantum computer lends itself particularly well to optimization problems via approximate quantum optimization algorithms. “We will be able to use this technology to adjust the parameters of the neurons network and ensure that its predictions converge as close as possible to the expected results,” explains Xavier Vasques. Behind the scenes, the optimization algorithm will make it possible to finely adjust the weights of each neuron. Again, this result suggests significant potential on the generative AI front.
“The gradient descent optimization methods highlight a quadratic advantage that does not allow exponential gains,” said Cyrille Allouche d’Eviden. “At Atos, we have carried out research on this field, especially in terms of Reinforcement Learning, which showed that it was possible to reach a slightly better learning gain compared to the traditional method, but the difference is not obvious.”
“Via quantum IT, methods have been discovered to create very high quality synthetic data”
What about computer vision? On this point, a study by the European space agency was able to demonstrate a substantial gain. Compared to an image recognition rate of 85% via a classic network of neurons with two million parameters, ESA obtained a recognition rate of 96% from the same image base, namely satellite photos aimed at detecting eruption volcanoes. And this, via a network of quantum neurons of only 40,000 parameters. “Or a much less energy and data consumer model,” says Xavier Vasques.
An article published in Nature Communications in 2024 drives the point home. He draws up a benchmark comparing six classic machine learning models with a quantum model. Conclusion: The quantum model reaches such a precise result with 10 times less data (see the graph below).
Another area pointed out by Xavier Vasques: that of the opposing networks networks that allow you to generate synthetic data, created virtually, and useful for model training. “Via quantum IT, we discovered methods allowing to create very high quality synthetic data”, Pointe Xavier Vasques, referring to an article published in the journal Nature in 2019. Here, the advantage would therefore be indirect.
Patterns detection
And the CTO of IBM France to continue: “In the field of complex data management, we also discovered that quantum IT was more efficient than conventional IT to detect patterns.” This suggests new applications in terms of data modeling in chemistry or even materials physics, these parts allowing to feed the LLM in high quality training data.
“Finally, the quantum increases precision via smaller models that require less data. We can in parallel improve optimization which makes it possible to obtain a reliable result more quickly”, summarizes Xavier Vasques. What about the longer term? The colossal calculation capacity of quantum computers could make it possible to create much more complex activation functions within neurons. “This would open a new field of application for major language models. But this perspective remains at the level of theory,” notes Xavier Vasques. And Cyrille allocates to recall: “For the moment, it has not been proven that quantum IT allow the networks of neurons to reach exponential performance gains as is the case for example in molecular or materials simulations.”
