Mitigating Biases with Diverse Ensembles and Diffusion Models
Spurious correlations in the data, where multiple cues are predictive of the target labels, often lead to a phenomenon known as shortcut bias, where a model relies on erroneous, easy-to-learn cues while ignoring reliable ones. In this work, we propose an ensemble diversification framework exploiting Diffusion Probabilistic Models (DPMs) for shortcut bias mitigation. We show that at particular training intervals, DPMs can generate images with novel feature combinations, even when trained on samples displaying correlated input features. We leverage this crucial property to generate synthetic counterfactuals to increase model diversity via ensemble disagreement. We show that DPM-guided diversification is sufficient to remove dependence on primary shortcut cues, without a need for additional supervised signals. We further empirically quantify its efficacy on several diversification objectives, and finally show improved generalization and diversification performance on par with prior work that relies on auxiliary data collection..
| Medienart: |
|
|---|
Preprint| Erscheinungsjahr: |
2023 |
|---|---|
| Erschienen: |
2023 |
| Enthalten in: |
arXiv.org - (2023) vom: 23. Nov. Zur Gesamtaufnahme - year:2023 |
|---|
| Sprache: |
Englisch |
|---|
| Beteiligte Personen: |
Scimeca, Luca [VerfasserIn] |
|---|
| Links: |
Volltext [kostenfrei] |
|---|
| Themen: |
000 |
|---|
| Förderinstitution / Projekttitel: |
|
|---|
| PPN (Katalog-ID): |
XCH042760747 |
|---|
| LEADER | 01000naa a22002652 4500 | ||
|---|---|---|---|
| 001 | XCH042760747 | ||
| 003 | DE-627 | ||
| 005 | 20240306114444.0 | ||
| 007 | cr uuu---uuuuu | ||
| 008 | 240306s2023 xx |||||o 00| ||eng c | ||
| 035 | |a (DE-627)XCH042760747 | ||
| 035 | |a (chemrXiv)2311.16176 | ||
| 040 | |a DE-627 |b ger |c DE-627 |e rakwb | ||
| 041 | |a eng | ||
| 100 | 1 | |a Scimeca, Luca |e verfasserin |4 aut | |
| 245 | 1 | 0 | |a Mitigating Biases with Diverse Ensembles and Diffusion Models |
| 264 | 1 | |c 2023 | |
| 336 | |a Text |b txt |2 rdacontent | ||
| 337 | |a Computermedien |b c |2 rdamedia | ||
| 338 | |a Online-Ressource |b cr |2 rdacarrier | ||
| 520 | |a Spurious correlations in the data, where multiple cues are predictive of the target labels, often lead to a phenomenon known as shortcut bias, where a model relies on erroneous, easy-to-learn cues while ignoring reliable ones. In this work, we propose an ensemble diversification framework exploiting Diffusion Probabilistic Models (DPMs) for shortcut bias mitigation. We show that at particular training intervals, DPMs can generate images with novel feature combinations, even when trained on samples displaying correlated input features. We leverage this crucial property to generate synthetic counterfactuals to increase model diversity via ensemble disagreement. We show that DPM-guided diversification is sufficient to remove dependence on primary shortcut cues, without a need for additional supervised signals. We further empirically quantify its efficacy on several diversification objectives, and finally show improved generalization and diversification performance on par with prior work that relies on auxiliary data collection. | ||
| 650 | 4 | |a Computer Science - Machine Learning |7 (dpeaa)DE-84 | |
| 650 | 4 | |a Computer Science - Artificial Intelligence |7 (dpeaa)DE-84 | |
| 650 | 4 | |a Computer Science - Computer Vision and Pattern Recognition |7 (dpeaa)DE-84 | |
| 650 | 4 | |a 000 |7 (dpeaa)DE-84 | |
| 700 | 1 | |a Rubinstein, Alexander |4 aut | |
| 700 | 1 | |a Teney, Damien |4 aut | |
| 700 | 1 | |a Oh, Seong Joon |4 aut | |
| 700 | 1 | |a Nicolicioiu, Armand Mihai |4 aut | |
| 700 | 1 | |a Bengio, Yoshua |4 aut | |
| 773 | 0 | 8 | |i Enthalten in |t arXiv.org |g (2023) vom: 23. Nov. |
| 773 | 1 | 8 | |g year:2023 |g day:23 |g month:11 |
| 856 | 4 | 0 | |u https://arxiv.org/abs/2311.16176 |z kostenfrei |3 Volltext |
| 912 | |a GBV_XCH | ||
| 951 | |a AR | ||
| 952 | |j 2023 |b 23 |c 11 | ||