Team behind popular Falcon AI models unveils new startup with $20 million in funding aimed at helping companies tailor LLMs for business

Adaptive, a startup founded by the team that built the open source large language model Falcon and that then worked together at open source AI company Hugging Face, has emerged from stealth with $20 million in initial venture capital round.

The company is working on technology that makes it easier for businesses to train large language models (LLMs) that are tailored to their specific needs.

The seed investment is being led by Index Ventures with participation from ICONIQ Capital, Motier Ventures, Databricks Ventures, HuggingFund by Factorial, and some individual angel investors. The company’s valuation was not disclosed, although tech publication The Information previously reported that the funding round valued the startup at $100 million.

Adaptive is working on a way to improve on a process that is known as reinforcement learning from human feedback, or RLHF. This process has been a key to taking LLMs, which are initially trained from a huge amount of text to predict the next word in a sentence, and making them more useful as the engines that power chatbots, such as OpenAI’s ChatGPT.

RLHF involves gathering feedback from human evaluators on the quality of an LLM’s responses. The LLM is then further trained to provide answers that are more like the ones that the evaluators rate highly. But RLHF has typically involved hiring contractors to evaluate a model, often using a simple thumbs or thumbs down to grade its answers. This method is expensive—the cost of data annotation contracts make up a good portion of the training costs of LLM-based chatbots, for example— and the quality of the feedback is sometimes too crude to produce good results for many business use cases of LLMs.

“It is hard to get the model to do what you want,” Julien Launay, Adaptive’s cofounder and CEO, said.

Adaptive wants to allow LLMs to learn on a regular and on-going basis from how a company’s own employees or customers actually interact with the software. The next actions and responses that a user makes in response to the LLM’s output is a much richer training signal in many cases than a thumbs or thumbs down given by a paid evaluator.

Launay said that Adaptive plans to offer a package of solutions that capture the way people interact with an LLM’s responses and then allows the model to be trained and fine-tuned from this data. Adaptive also provides a platform for running the reinforcement learning algorithms that tailors the model, as this process is difficult for many non-expert teams to implement. It also lets a business pick exactly what data it wants to gather, what objective it wants the model to achieve, and which reinforcement learning algorithm it wants to use to do this training. This control helps businesses gain a better handle on the trade-offs between cost and performance, Launay said.

The platform will also help companies run a process called reinforcement learning from AI feedback (RLAIF), where a separate AI model critiques the responses of the AI model that is being trained. This can lower the cost of training and result in a better range of training data than using human evaluators.

Adaptive will be entering a marketplace that is getting increasingly crowded. Platforms for RLHF training are also being offered by some of the big data labelling companies that traditionally provided human evaluators. These include Appen and Scale AI. Similar tools are also offered by Surge AI, CarperAI, and Encord. But most of these RLHF tools aren’t designed to capture preference data from model users once a model is deployed.

The technology Adaptive is building will work on top of any open source LLM model or any model that a business has built itself. Open source models are proving increasingly popular with companies that are seeking more control over both the output of generative AI models and ways to reduce the cost of generative AI applications. The startup’s technology will not, however, allow business to fine tune third-party proprietary models, such as those available from OpenAI, Google, Anthropic, and Cohere. “We need access to the model weights,” Launay said.

Adaptive’s platform is designed to help customers test the performance of different LLMs against one another and help them monitor how these models perform once deployed. Adaptive is developing dashboards and metrics that can relate LLM outputs to key business metrics, such as whether a customer’s query was resolved successfully.

He said that Adaptive already has some customers using its platform, although he declined to name them. The company, which currently has only nine employees, said it is planning to use the new venture capital funding to expand its teams in both Paris, where it is based, and New York, with an emphasis on “go to market” and sales teams.

Launay had previously worked at an AI hardware startup in Amsterdam with Adaptive cofounder Daniel Hesslow, now the startup’s chief research scientist. The two later wound up working with cofounder Baptiste Pannier, now Adaptive’s chief technology officer, as part of the team that built the Falcon LLM family of open source models at the Technology Innovation Institute in Abu Dhabi. The Falcon models impressed people with their performance for their size and the innovative training techniques its developers had used. The Falcon models have regularly topped the leaderboards that Hugging Face maintains for model performance and popularity.

The team then went to work at Hugging Face, which both builds its own open source AI models and offers a popular repository of other open source models.

Bryan Offutt, the Index Ventures partner who led the investment into Adaptive, says he was impressed by the combination technical expertise and understanding of business needs exhibited by the company’s founding team as well as its energy, which he described as “infectious.”  He said that the problem the team is trying to crack—how to tune a generative AI model for user preferences—is a technical challenge with which many companies are struggling.

He said that a challenge for Adaptive going forward will be to work with customers to find ways to incentivize the people using LLMs to provide the feedback that will be most valuable for training. If a person gives a full explanation for why they find a model’s response helpful or unhelpful, that is extremely valuable data for refining the model. But having to provide this kind of detailed feedback for every model response is time-consuming and would likely annoy users. So Adaptive will need to find ways to work with its customers to balance the need for feedback with the burden this places on LLM users, Offutt said.

This story was originally featured on Fortune.com