Google is excited to release TxGemma, a collection of open models designed to improve the efficiency of therapeutic development by leveraging the power of large language models. Developing a new therapeutic is risky, notoriously slow, and can cost billions of dollars. 90% of drug candidates fail beyond phase 1 trials.
Building on Google DeepMind's Gemma, a family of lightweight, state-of-the-art open models, TxGemma is specifically trained to understand and predict the properties of therapeutic entities throughout the entire discovery process, from identifying promising targets to helping predict clinical trial outcomes. This can potentially shorten the time from lab to bedside, and reduce the costs associated with traditional methods.
TxGemma can be used for making predictions relating to various therapeutic tasks drawn from TDC datasets. These predictions include classification (multiple choice), regression (numeric value), or generation. Examples include:
Classification
Given a drug SMILES string, the model can:
• Predict the drug's toxicity.
• Predict whether the drug can cross the blood-brain barrier.
• Predict whether the drug is active against a certain protein e.g., a choline transporter.
• Predict whether the drug is a carcinogen.
Last October, Google introduced Tx-LLM, a language model trained for a variety of therapeutic tasks related to drug development. After huge interest to use and fine-tune this model for therapeutic applications, we have developed its open successor at a practical scale: TxGemma, which they are releasing for developers to adapt to their own therapeutic data and tasks.
TxGemma models, fine-tuned from Gemma 2 using 7 million training examples, are open models designed for prediction and conversational therapeutic data analysis. These models are available in three sizes: 2B, 9B and 27B. Each size includes a ‘predict’ version, specifically tailored for narrow tasks drawn from Therapeutic Data Commons, for example predicting if a molecule is toxic.
The largest TxGemma model (27B predict version) delivers strong performance. It's not only better than, or roughly equal to, our previous state-of-the-art generalist model (Tx-LLM) on almost every task, but it also rivals or beats many models that are specifically designed for single tasks.
Conversational AI for deeper insights
TxGemma also includes 9B and 27B ‘chat’ versions. These models have general instruction tuning data added to their training, enabling them to explain their reasoning, answer complex questions, and engage in multi-turn discussions. For example, a researcher could ask TxGemma-Chat why it predicted a particular molecule to be toxic and receive an explanation based on the molecule's structure. This conversational capability comes at a small cost to the raw performance on therapeutic tasks compared to TxGemma-Predict.
Extending TxGemma's capabilities through fine-tuning
As part of the release, we’re including a fine-tuning example Colab notebook that demonstrates how developers can adapt TxGemma to their own therapeutic data and tasks. This notebook uses the TrialBench dataset to show how to fine-tune TxGemma for predicting adverse events in clinical trials. Fine-tuning allows researchers to leverage their proprietary data to create models tailored to their unique research needs, possibly leading to even more accurate predictions that help researchers assess how safe or or effective a potential new therapy might be.
Orchestrating workflows for advanced therapeutic discovery with Agentic-Tx
Beyond single-step predictions, Google is demonstrating how TxGemma can be integrated into agentic systems to tackle more complex research problems. Standard language models often struggle with tasks requiring up-to-date external knowledge or multi-step reasoning. To address this, it've developed Agentic-Tx, a therapeutics-focused agentic system powered by Gemini 2.0 Pro. Agentic-Tx is equipped with 18 tools, including:
• TxGemma as a tool for multi-step reasoning
• General search tools from PubMed, Wikipedia and the web
• Specific molecular tools
• Gene and protein tools
.png){kind=small}
