Interview with Joëlle Pineau, the director of Fair, the Meta IA research laboratory, which returns for JDN on robotics advances, the impact of open source models and trajectory towards an advanced machine intelligence.
JDN. In February, Fair, the Meta AI Research Laboratory that you direct, announced that it has managed to decode brain language with an 80%precision. Do you plan the future integration of this technology into augmented reality headsets ?
Joëlle Pineau. Such technology could actually replace the keyboard for people with disabilities. However, its use requires wearing a large helmet equipped with several sensors, which constitutes an important barrier to an adoption by the general public. This is a notable technological advance, made possible thanks to the progress of generative AI in recent years. Thanks to these new models, we can decode the formation of sentences from the neural signals captured by these devices.
Fair recently published a benchmark called Partnr for PLanning and Reasoning Tasks in Human-Robot Collaboration, In order to promote research in collaborative robotics. What are your recent progress in this area ?
Benchmark’s objective is to assess and improve collaboration between humans and robots in daily tasks. This large -scale benchmark includes 100,000 tasks in natural language, designed to assess robot performance in simulated environments. At the start of the year, we demonstrated the progress of our planning models by integrating them into the Boston Dynamics Spot robot. These algorithms allow robots to sequence actions, that is to say to use the information available to define a sequence of actions and perform a task, such as the entry of an object. In October 2024, we had already presented our tactile sensor named Digit 360, developed in partnership with the company Gelsight. This sensor makes it possible to collect detailed touch data and, ultimately, build better representations.
Are we far from seeing robots attending everyday tasks, such as cleaning or cooking ? Could Meta market domestic robots ?
It is obvious that everyone is looking forward to this type of robots. If I had to do a prediction, I would say that it will take more than 5 to 10 years. Regarding Meta, it is too early to make announcements, but it is obvious that robotics is a subject that interests the company and that it monitors closely.
What look at the ACT AC, the European regulation framing the AI ? As a researcher, do you find this legislation too restrictive ?
It is not my role as advising the EU on this point, but it seems important to me to find a balance between protection and speed of innovation. Today, we cannot guarantee that all the models that we develop can be available in Europe, due to the legislation. What is certain is that regulatory fragmentation on the scale of different European countries, which each have their own legislation or interpretation, does not facilitate things and can create uncertainty among companies.
The development of models like Deepseek has shown that major language models are doomed to become a convenience according to you ?
It is indeed the trajectory that is currently observed. However, we have little information on the training of the model, especially since we know that Deepseek has been drawn by distillation of other models, a technique that allows a transfer of knowledge from one model to another. This thus raises the question of responsibility and transparency. However, even if these great models of language become amenities, this does not reduce, in my eyes, the interest of causing wider and rich models.
Developing Act is the ultimate objective of Openai. Share this goal at Fair ?
Fair is an entity dedicated to basic research in artificial intelligence. Our mission is not to develop a product of artificial general intelligence (AG), but to provide the essential components to achieve this. It is comparable to a car team that provides the parts necessary to design a Formula 1, without delivering the full vehicle. To reach AGE, we focus on our mission to an advanced machine intelligence (friend), in collaboration with the Genai teams at Meta, now one of the five main divisions of the company alongside WhatsApp, Messenger, Facebook and Instagram.
What is the difference between friend and act?
General artificial intelligence (AGA) aims to develop systems capable of carrying out any intellectual task that a human could accomplish. Our Llama model can be considered our vehicle towards Act, while Meta I act as a product interface for our users. Advanced machine intelligence (AMI), on the other hand, encompasses the fundamental principles to improve the representation of language and images, as well as planning and reasoning capacities in the real world. This is a long-term objective aimed at developing models capable of manipulating abstract knowledge, thus going beyond simple language models.
To achieve this, Fair is betting on World Models. Why are these models the logical continuation of LLM according to you ?
Language models predict tokens (words or symbols) which limits their ability to achieve truly generalized intelligence. We can thus consider language models as a subset of World Models, which must be able to predict the results of actions, the rest of an image, a video or sound.
Currently, we use an architecture based on transformers, well suited to the prediction and generation of sequences. However, to promote learning the World Models, it will be necessary to adopt new architectures allowing, for example, to understand cause and effect relationships. It is therefore a natural evolution of LLM to World Models, essential to allow a more multimodal prediction.
Yann Lecun, Chief Ai Scientist at Fair, has been working on Jepa architecture since 2022. Is this one of the tracks followed by your researchers to achieve this goal?
Indeed, the Jepa project (Joint-Embedding Predictive Architecture) is one of the hypotheses that could promote the development of World Models. Our researchers based in Paris, Montreal and New York are currently working on this subject. The question is therefore not to know if we are heading towards World Models, because it seems acquired, but rather to determine which architecture will allow their development on a large scale. This remains an open question, and Jepa is one of the main tracks we explore.
The Parisian pole of Fair celebrated its 10th anniversary this year. How, innovations from your laboratories benefit commercially in Meta ?
First, continuously improving the performance of our Llama model. Then, some of our work find specific applications in the various META services, in collaboration with the product teams. For example, the progress made on image and videos interpretation algorithms benefit directly from the connected glasses. Content moderation is another example while Mark Zuckerberg recently announced the development of a community notes system, intended to replace the verification program with third parties in the United States. Several research projects aim to design a community notes system capable of representing the diversity of opinions and offering a balanced reflection from different points of view.
How do you decide whether or not to publish your work in open source ?
Meta generates its income mainly through its products and therefore does not need to market its AI models. Our research contributes to the improvement of Meta AI, whose interface is integrated into several products of the company. At Fair, our goal is to develop the best general AI models, which benefits Meta. For example, Llama is a model used internally and adaptable as required. This is precisely the interest of open source. Take the example of the Dino model, published in 2021: Meta was used by Meta, but also by other companies for various applications, whether research against cancer or reforestation. In a way, we come back to the origins of the company, since part of the software used to develop Facebook was made up of open source protocols
Joëlle Pineau is vice-president of AI research and director general of Fair, the META group entity dedicated to basic research in artificial intelligence. She is also an associate professor at the IT school at McGill University, where she co -edit the reasoning and learning laboratory. She holds a master’s degree in science and a doctorate in robotics from Carnegie-Mellon University.
