Repurposing Protein Folding Models for Generation with Latent Diffusion

Overview

PLAID is a multimodal generative model that produces a protein's 1D sequence and 3D structure at the same time by learning the latent space of protein folding models. It accepts compositional function and organism prompts and can be trained on sequence databases, which are 2 to 4 orders of magnitude larger than structure databases. The method trains a diffusion model over the latent space of a folding model and, at inference, decodes both sequence and structure from a sampled embedding using frozen folding-model weights.

Key Takeaways

PLAID simultaneously generates discrete protein sequence and continuous all-atom structural coordinates, addressing the multimodal co-generation problem.
It learns to sample from the latent space of protein folding models to generate new proteins.
It accepts compositional function and organism prompts for control over what is generated.
It can be trained using sequence-only data, which is far more abundant than experimental structure data.
During inference, frozen weights from the protein folding model decode structure from the sampled embedding.
The work uses ESMFold, a successor to AlphaFold2 that replaces a retrieval step with a protein language model.

Stats & Key Facts

#Sequence databases are 2 to 4 orders of magnitude larger than structure databases.
#The 2024 Nobel Prize was awarded to AlphaFold2.

Repurposing Protein Folding Models for Generation with Latent Diffusion

What PLAID does

PLAID extends generative modeling beyond structure prediction.

›It is a multimodal generative model that generates protein 1D sequence and 3D structure simultaneously.
›It learns the latent space of protein folding models to generate new proteins.
›It accepts compositional function and organism prompts.

The authors frame PLAID as a step beyond protein folding, noting that the 2024 Nobel Prize awarded to AlphaFold2 marked recognition of AI's role in biology.

From structure prediction to drug design

Previous generative models had limitations for real-world use.

›Many existing models only produce backbone atoms, while all-atom structure requires knowing the sequence to place sidechain atoms.
›Protein biologics for human use need to be humanized to avoid being destroyed by the human immune system, which motivates organism specificity.
›Drug discovery involves complex constraints, such as a preference for tablets over vials adding a solubility requirement.

Producing the all-atom structure creates a multimodal generation problem requiring simultaneous generation of discrete and continuous modalities, which PLAID targets directly.

Generating useful proteins through control

Control specification is central to making generation useful.

›Simply generating proteins is less useful than controlling generation to get useful proteins.
›PLAID mirrors the idea of compositional textual prompts used in image generation.
›As a proof of concept, it considers compositional constraints along two axes: function and organism.

The authors report that PLAID learns the tetrahedral cysteine-Fe2+/Fe3+ coordination pattern often found in metalloproteins while maintaining high sequence-level diversity.

Training with sequence-only data

PLAID's training requirement is a key advantage.

›The generative model requires only sequences to train.
›Sequence databases are considerably larger than structural ones because sequences are cheaper to obtain.
›This lets the model learn from a larger and broader database than structure-based methods.

How the method works

PLAID trains a diffusion model over a folding model's latent space.

›A diffusion model is trained over the latent space of a protein folding model.
›During training, only sequences are needed to obtain the embedding.
›During inference, after sampling from the latent space, frozen folding-model weights decode both sequence and structure.

The work uses ESMFold, a successor to AlphaFold2 that replaces a retrieval step with a protein language model, allowing the structural understanding in the pretrained weights to be reused.

Frequently Asked Questions

What is PLAID?

PLAID is a multimodal generative model that simultaneously generates a protein's 1D sequence and 3D structure by learning the latent space of protein folding models.

Why does PLAID generate sequence and structure together?

Producing an all-atom structure requires the sequence to place sidechain atoms, which creates a multimodal generation problem that needs simultaneous generation of discrete sequence and continuous structural coordinates.

Why can PLAID train on sequences alone?

It learns a diffusion model over the latent space of a folding model, so only sequences are needed for the embedding during training, and frozen folding-model weights decode structure at inference.

What advantage does sequence-only training provide?

Sequence databases are 2 to 4 orders of magnitude larger than structure databases because sequences are much cheaper to obtain than experimental structures, giving the model a larger and broader training set.

Which folding model does PLAID use?

It uses ESMFold, a successor to AlphaFold2 that replaces a retrieval step with a protein language model.

PLAID repurposes the latent space of protein folding models to generate new proteins with both sequence and structure under function and organism prompts.