95% of generative AI projects fail: to create value, you have to go beyond fantasies, limit the use of AI to the necessary, test rigorously and aim for industrialization.
The release of the “State of Ai in Business” study had the merit of shaking the spirits in the right direction: 95% of generative AI projects create no value for the companies that carry them. If this observation surprised most of the actors in this world, it was greeted by the “old” of this field as obvious. The “proof of concept” swarm today, but the successful industrializations, allowing to go into production, remain very rare.
Beyond the analyzes proposed by the authors of the MIT, the field of Deep Learning has suffered from its renaissance in 2012 from fundamental weaknesses in 2012 which allow to better understand the situation. Among these weaknesses, an original sin: that of creating tools for which we have no fundamental mathematical understanding. This blindness has hovered over the academic domain since its creation, and without this theoretical compass, researchers have evolved by practicing excessive empiricism. We could cite these complex publications in temporal series from 2021 to 2022 (1) totally invalidated by an almost simplistic approach, as we could evoke many beliefs relating to the large Language Models (among others: the emergence of new capacities (2), or the self-correction of models (3)) then completely invalidated.
This lack of understanding could be seen as a non-professional. Many technical applications are based on a partial understanding of physical phenomena. But to this blindness was added another problem: the fact that these neural networks obtain results all the more interesting as they are large. The bulimia of models has worsened our understanding deficit, until producing the large Language Models: generic tools, certainly fascinating, but which we do not know how to control in terms of security, quality or interpretability.
The large Language Models (LLMS) crystallize today the debate, and are an excellent example of the Dichotomy of the AI. On the one hand, these tools are fascinating and revolutionary. On the other hand, assessing their quality in order to transform them into professional and controlled tools can lead the best experts to despair. LLMS are a double trap. On the one hand, these are stochastic, random tools, trained overall on a data supposed to represent a problem (and after ten years in this area, I can recognize that representing a problem by data can sometimes give nostalgia for the V. briefly, at least). But above all, these models take the outset and out of language, opening the door to an easy anthropomorphism that will spoil any chance of correctly considering and therefore manipulating these tools. Some researchers have even attempted to study “human generalization function” (4), where why someone will decide that this or that problem should be able to be resolved by an LLM.
The challenge today is not to abandon the AI generally, but to get rid of fantasies to find a minimum of common sense. AI is a catalog of tools. Any tool requires correct use and conditions of use. A methodological element which, typically, to hope to transform an AI prototype into a product is to minimize AI by using it only where it is absolutely necessary, in order to minimize the risks and maximize the explanability of the product. Furthermore, testing these tools is certainly less trivial but is not impossible either and here is an absolute necessity. Finally, asking the question of the user and its relation to the product is all the more vital as fantasies are the norm about these models.
The AI will not disappear tomorrow, but the warnings like the MIT report impose on us hindsight and sobriety if we want to succeed.
{References}
(1) Are effective transformers for time series Forecasting?, Zeng et al
(2) Are Emerging Abilities of Large Language Models A Mirage? Schaeffer et al
(3) Large Language Models Cannot Self-Correct Reasoning Yet, Huang et al
(4) Do Large Language Models Perform the Way People Expect? Measuring the Human Generalization Function, Vafa et al
