Statistical Physics of Unsupervised Learning with Prior Knowledge in Neural Networks
Integrating sensory inputs with prior beliefs from past experiences in unsupervised learning is a common and fundamental characteristic of brain or artificial neural computation. However, a quantitative role of prior knowledge in unsupervised learning remains unclear, prohibiting a scientific understanding of unsupervised learning. Here, we propose a statistical physics model of unsupervised learning with prior knowledge, revealing that the sensory inputs drive a series of continuous phase transitions related to spontaneous intrinsic-symmetry breaking. The intrinsic symmetry includes both reverse symmetry and permutation symmetry, commonly observed in most artificial neural networks. Compared to the prior-free scenario, the prior reduces more strongly the minimal data size triggering the reverse-symmetry breaking transition, and moreover, the prior merges, rather than separates, permutation-symmetry breaking phases. We claim that the prior can be learned from data samples, which in physics corresponds to a two-parameter Nishimori constraint. This Letter thus reveals mechanisms about the influence of the prior on unsupervised learning.
Medienart: |
E-Artikel |
---|
Erscheinungsjahr: |
2020 |
---|---|
Erschienen: |
2020 |
Enthalten in: |
Zur Gesamtaufnahme - volume:124 |
---|---|
Enthalten in: |
Physical review letters - 124(2020), 24 vom: 19. Juni, Seite 248302 |
Sprache: |
Englisch |
---|
Beteiligte Personen: |
Hou, Tianqi [VerfasserIn] |
---|
Links: |
---|
Themen: |
---|
Anmerkungen: |
Date Completed 09.07.2020 Date Revised 09.07.2020 published: Print Citation Status PubMed-not-MEDLINE |
---|
doi: |
10.1103/PhysRevLett.124.248302 |
---|
funding: |
|
---|---|
Förderinstitution / Projekttitel: |
|
PPN (Katalog-ID): |
NLM312171366 |
---|
LEADER | 01000naa a22002652 4500 | ||
---|---|---|---|
001 | NLM312171366 | ||
003 | DE-627 | ||
005 | 20231225143834.0 | ||
007 | cr uuu---uuuuu | ||
008 | 231225s2020 xx |||||o 00| ||eng c | ||
024 | 7 | |a 10.1103/PhysRevLett.124.248302 |2 doi | |
028 | 5 | 2 | |a pubmed24n1040.xml |
035 | |a (DE-627)NLM312171366 | ||
035 | |a (NLM)32639836 | ||
040 | |a DE-627 |b ger |c DE-627 |e rakwb | ||
041 | |a eng | ||
100 | 1 | |a Hou, Tianqi |e verfasserin |4 aut | |
245 | 1 | 0 | |a Statistical Physics of Unsupervised Learning with Prior Knowledge in Neural Networks |
264 | 1 | |c 2020 | |
336 | |a Text |b txt |2 rdacontent | ||
337 | |a ƒaComputermedien |b c |2 rdamedia | ||
338 | |a ƒa Online-Ressource |b cr |2 rdacarrier | ||
500 | |a Date Completed 09.07.2020 | ||
500 | |a Date Revised 09.07.2020 | ||
500 | |a published: Print | ||
500 | |a Citation Status PubMed-not-MEDLINE | ||
520 | |a Integrating sensory inputs with prior beliefs from past experiences in unsupervised learning is a common and fundamental characteristic of brain or artificial neural computation. However, a quantitative role of prior knowledge in unsupervised learning remains unclear, prohibiting a scientific understanding of unsupervised learning. Here, we propose a statistical physics model of unsupervised learning with prior knowledge, revealing that the sensory inputs drive a series of continuous phase transitions related to spontaneous intrinsic-symmetry breaking. The intrinsic symmetry includes both reverse symmetry and permutation symmetry, commonly observed in most artificial neural networks. Compared to the prior-free scenario, the prior reduces more strongly the minimal data size triggering the reverse-symmetry breaking transition, and moreover, the prior merges, rather than separates, permutation-symmetry breaking phases. We claim that the prior can be learned from data samples, which in physics corresponds to a two-parameter Nishimori constraint. This Letter thus reveals mechanisms about the influence of the prior on unsupervised learning | ||
650 | 4 | |a Journal Article | |
700 | 1 | |a Huang, Haiping |e verfasserin |4 aut | |
773 | 0 | 8 | |i Enthalten in |t Physical review letters |d 1985 |g 124(2020), 24 vom: 19. Juni, Seite 248302 |w (DE-627)NLM099656183 |x 1079-7114 |7 nnns |
773 | 1 | 8 | |g volume:124 |g year:2020 |g number:24 |g day:19 |g month:06 |g pages:248302 |
856 | 4 | 0 | |u http://dx.doi.org/10.1103/PhysRevLett.124.248302 |3 Volltext |
912 | |a GBV_USEFLAG_A | ||
912 | |a GBV_NLM | ||
951 | |a AR | ||
952 | |d 124 |j 2020 |e 24 |b 19 |c 06 |h 248302 |