Computation or Weight Adaptation? Rethinking the Role of Plasticity in Learning
Abstract The human brain is an adaptive learning system that can generalize to new tasks and unfamiliar environments. The traditional view is that such adaptive behavior requires a structural change of the learning system (e.g., via neural plasticity). In this work, we use artificial neural networks, specifically large language models (LLMs), to challenge the traditional view about the role of plasticity in learning and suggest that such an adaptive behavior can be achieved solely through computation if the learning system is sufficiently trained. We focus on statistical learning paradigms. These require identifying underlying regularities in seemingly arbitrary word sequences and are largely considered to require neural plasticity. LLMs can capture arbitrary structureswithoutweight adaptation despite the divergence from their natural language training data. Our work provides novel insights into the role of plasticity in learning, showing that sufficiently trained learning systems are highly flexible, adapting to new tasks and environments solely through computation, much more than previously acknowledged. Furthermore, our work opens the door for future research to use deep learning models to conjure hypotheses about the brain..
| Medienart: |
|
|---|
Preprint| Erscheinungsjahr: |
2024 |
|---|---|
| Erschienen: |
2024 |
| Enthalten in: |
bioRxiv.org - (2024) vom: 11. März Zur Gesamtaufnahme - year:2024 |
|---|
| Sprache: |
Englisch |
|---|
| Beteiligte Personen: |
Lior, Gili [VerfasserIn] |
|---|
| Links: |
Volltext [kostenfrei] |
|---|
| Themen: |
|---|
| doi: |
10.1101/2024.03.07.583890 |
|---|
| funding: |
|
|---|---|
| Förderinstitution / Projekttitel: |
|
| PPN (Katalog-ID): |
XBI042838975 |
|---|
| LEADER | 01000caa a22002652 4500 | ||
|---|---|---|---|
| 001 | XBI042838975 | ||
| 003 | DE-627 | ||
| 005 | 20240312090627.0 | ||
| 007 | cr uuu---uuuuu | ||
| 008 | 240309s2024 xx |||||o 00| ||eng c | ||
| 024 | 7 | |a 10.1101/2024.03.07.583890 |2 doi | |
| 035 | |a (DE-627)XBI042838975 | ||
| 035 | |a (biorXiv)10.1101/2024.03.07.583890 | ||
| 040 | |a DE-627 |b ger |c DE-627 |e rakwb | ||
| 041 | |a eng | ||
| 100 | 1 | |a Lior, Gili |e verfasserin |4 aut | |
| 245 | 1 | 0 | |a Computation or Weight Adaptation? Rethinking the Role of Plasticity in Learning |
| 264 | 1 | |c 2024 | |
| 336 | |a Text |b txt |2 rdacontent | ||
| 337 | |a Computermedien |b c |2 rdamedia | ||
| 338 | |a Online-Ressource |b cr |2 rdacarrier | ||
| 520 | |a Abstract The human brain is an adaptive learning system that can generalize to new tasks and unfamiliar environments. The traditional view is that such adaptive behavior requires a structural change of the learning system (e.g., via neural plasticity). In this work, we use artificial neural networks, specifically large language models (LLMs), to challenge the traditional view about the role of plasticity in learning and suggest that such an adaptive behavior can be achieved solely through computation if the learning system is sufficiently trained. We focus on statistical learning paradigms. These require identifying underlying regularities in seemingly arbitrary word sequences and are largely considered to require neural plasticity. LLMs can capture arbitrary structureswithoutweight adaptation despite the divergence from their natural language training data. Our work provides novel insights into the role of plasticity in learning, showing that sufficiently trained learning systems are highly flexible, adapting to new tasks and environments solely through computation, much more than previously acknowledged. Furthermore, our work opens the door for future research to use deep learning models to conjure hypotheses about the brain. | ||
| 650 | 4 | |a Biology |7 (dpeaa)DE-84 | |
| 650 | 4 | |a 570 |7 (dpeaa)DE-84 | |
| 700 | 1 | |a Shalev, Yuval |4 aut | |
| 700 | 1 | |a Stanovsky, Gabriel |4 aut | |
| 700 | 1 | |a Goldstein, Ariel |4 aut | |
| 773 | 0 | 8 | |i Enthalten in |t bioRxiv.org |g (2024) vom: 11. März |
| 773 | 1 | 8 | |g year:2024 |g day:11 |g month:03 |
| 856 | 4 | 0 | |u http://dx.doi.org/10.1101/2024.03.07.583890 |z kostenfrei |3 Volltext |
| 912 | |a GBV_XBI | ||
| 951 | |a AR | ||
| 952 | |j 2024 |b 11 |c 03 | ||