Fast activation maximization for molecular sequence design
© 2021. The Author(s)..
BACKGROUND: Optimization of DNA and protein sequences based on Machine Learning models is becoming a powerful tool for molecular design. Activation maximization offers a simple design strategy for differentiable models: one-hot coded sequences are first approximated by a continuous representation, which is then iteratively optimized with respect to the predictor oracle by gradient ascent. While elegant, the current version of the method suffers from vanishing gradients and may cause predictor pathologies leading to poor convergence.
RESULTS: Here, we introduce Fast SeqProp, an improved activation maximization method that combines straight-through approximation with normalization across the parameters of the input sequence distribution. Fast SeqProp overcomes bottlenecks in earlier methods arising from input parameters becoming skewed during optimization. Compared to prior methods, Fast SeqProp results in up to 100-fold faster convergence while also finding improved fitness optima for many applications. We demonstrate Fast SeqProp's capabilities by designing DNA and protein sequences for six deep learning predictors, including a protein structure predictor.
CONCLUSIONS: Fast SeqProp offers a reliable and efficient method for general-purpose sequence optimization through a differentiable fitness predictor. As demonstrated on a variety of deep learning models, the method is widely applicable, and can incorporate various regularization techniques to maintain confidence in the sequence designs. As a design tool, Fast SeqProp may aid in the development of novel molecules, drug therapies and vaccines.
| Medienart: |
E-Artikel |
|---|
| Erscheinungsjahr: |
2021 |
|---|---|
| Erschienen: |
2021 |
| Enthalten in: |
Zur Gesamtaufnahme - volume:22 |
|---|---|
| Enthalten in: |
BMC bioinformatics - 22(2021), 1 vom: 20. Okt., Seite 510 |
| Sprache: |
Englisch |
|---|
| Beteiligte Personen: |
Linder, Johannes [VerfasserIn] |
|---|
| Links: |
|---|
| Themen: |
Activation maximization |
|---|
| Anmerkungen: |
Date Completed 22.10.2021 Date Revised 03.04.2024 published: Electronic Citation Status MEDLINE |
|---|
| doi: |
10.1186/s12859-021-04437-5 |
|---|
| funding: |
|
|---|---|
| Förderinstitution / Projekttitel: |
|
| PPN (Katalog-ID): |
NLM332107124 |
|---|
| LEADER | 01000caa a22002652 4500 | ||
|---|---|---|---|
| 001 | NLM332107124 | ||
| 003 | DE-627 | ||
| 005 | 20240403234337.0 | ||
| 007 | cr uuu---uuuuu | ||
| 008 | 231225s2021 xx |||||o 00| ||eng c | ||
| 024 | 7 | |a 10.1186/s12859-021-04437-5 |2 doi | |
| 028 | 5 | 2 | |a pubmed24n1362.xml |
| 035 | |a (DE-627)NLM332107124 | ||
| 035 | |a (NLM)34670493 | ||
| 040 | |a DE-627 |b ger |c DE-627 |e rakwb | ||
| 041 | |a eng | ||
| 100 | 1 | |a Linder, Johannes |e verfasserin |4 aut | |
| 245 | 1 | 0 | |a Fast activation maximization for molecular sequence design |
| 264 | 1 | |c 2021 | |
| 336 | |a Text |b txt |2 rdacontent | ||
| 337 | |a ƒaComputermedien |b c |2 rdamedia | ||
| 338 | |a ƒa Online-Ressource |b cr |2 rdacarrier | ||
| 500 | |a Date Completed 22.10.2021 | ||
| 500 | |a Date Revised 03.04.2024 | ||
| 500 | |a published: Electronic | ||
| 500 | |a Citation Status MEDLINE | ||
| 520 | |a © 2021. The Author(s). | ||
| 520 | |a BACKGROUND: Optimization of DNA and protein sequences based on Machine Learning models is becoming a powerful tool for molecular design. Activation maximization offers a simple design strategy for differentiable models: one-hot coded sequences are first approximated by a continuous representation, which is then iteratively optimized with respect to the predictor oracle by gradient ascent. While elegant, the current version of the method suffers from vanishing gradients and may cause predictor pathologies leading to poor convergence | ||
| 520 | |a RESULTS: Here, we introduce Fast SeqProp, an improved activation maximization method that combines straight-through approximation with normalization across the parameters of the input sequence distribution. Fast SeqProp overcomes bottlenecks in earlier methods arising from input parameters becoming skewed during optimization. Compared to prior methods, Fast SeqProp results in up to 100-fold faster convergence while also finding improved fitness optima for many applications. We demonstrate Fast SeqProp's capabilities by designing DNA and protein sequences for six deep learning predictors, including a protein structure predictor | ||
| 520 | |a CONCLUSIONS: Fast SeqProp offers a reliable and efficient method for general-purpose sequence optimization through a differentiable fitness predictor. As demonstrated on a variety of deep learning models, the method is widely applicable, and can incorporate various regularization techniques to maintain confidence in the sequence designs. As a design tool, Fast SeqProp may aid in the development of novel molecules, drug therapies and vaccines | ||
| 650 | 4 | |a Journal Article | |
| 650 | 4 | |a Activation maximization | |
| 650 | 4 | |a DNA | |
| 650 | 4 | |a Deep learning | |
| 650 | 4 | |a Design | |
| 650 | 4 | |a Gradient ascent | |
| 650 | 4 | |a Neural network | |
| 650 | 4 | |a Optimization | |
| 650 | 4 | |a Protein | |
| 650 | 4 | |a RNA | |
| 650 | 4 | |a Sequence design | |
| 700 | 1 | |a Seelig, Georg |e verfasserin |4 aut | |
| 773 | 0 | 8 | |i Enthalten in |t BMC bioinformatics |d 2000 |g 22(2021), 1 vom: 20. Okt., Seite 510 |w (DE-627)NLM109215982 |x 1471-2105 |7 nnns |
| 773 | 1 | 8 | |g volume:22 |g year:2021 |g number:1 |g day:20 |g month:10 |g pages:510 |
| 856 | 4 | 0 | |u http://dx.doi.org/10.1186/s12859-021-04437-5 |3 Volltext |
| 912 | |a GBV_USEFLAG_A | ||
| 912 | |a GBV_NLM | ||
| 951 | |a AR | ||
| 952 | |d 22 |j 2021 |e 1 |b 20 |c 10 |h 510 | ||