Exposing omitted moderators : Explaining why effect sizes differ in the social sciences
Policymakers increasingly rely on behavioral science in response to global challenges, such as climate change or global health crises. But applications of behavioral science face an important problem: Interventions often exert substantially different effects across contexts and individuals. We examine this heterogeneity for different paradigms that underlie many behavioral interventions. We study the paradigms in a series of five preregistered studies across one in-person and 10 online panels, with over 11,000 respondents in total. We find substantial heterogeneity across settings and paradigms, apply techniques for modeling the heterogeneity, and introduce a framework that measures typically omitted moderators. The framework's factors (Fluid Intelligence, Attentiveness, Crystallized Intelligence, and Experience) affect the effectiveness of many text-based interventions, producing different observed effect sizes and explaining variations across samples. Moderators are associated with effect sizes through two paths, with the intensity of the manipulation and with the effect of the manipulation directly. Our results motivate observing these moderators and provide a theoretical and empirical framework for understanding and predicting varying effect sizes in the social sciences.
| Medienart: |
E-Artikel |
|---|
| Erscheinungsjahr: |
2024 |
|---|---|
| Erschienen: |
2024 |
| Enthalten in: |
Zur Gesamtaufnahme - volume:121 |
|---|---|
| Enthalten in: |
Proceedings of the National Academy of Sciences of the United States of America - 121(2024), 12 vom: 19. März, Seite e2306281121 |
| Sprache: |
Englisch |
|---|
| Beteiligte Personen: |
Krefeld-Schwalb, Antonia [VerfasserIn] |
|---|
| Links: |
|---|
| Themen: |
Choice architecture |
|---|
| Anmerkungen: |
Date Completed 13.03.2024 Date Revised 27.03.2024 published: Print-Electronic Citation Status MEDLINE |
|---|
| doi: |
10.1073/pnas.2306281121 |
|---|
| funding: |
|
|---|---|
| Förderinstitution / Projekttitel: |
|
| PPN (Katalog-ID): |
NLM369565614 |
|---|
| LEADER | 01000caa a22002652 4500 | ||
|---|---|---|---|
| 001 | NLM369565614 | ||
| 003 | DE-627 | ||
| 005 | 20240328000116.0 | ||
| 007 | cr uuu---uuuuu | ||
| 008 | 240312s2024 xx |||||o 00| ||eng c | ||
| 024 | 7 | |a 10.1073/pnas.2306281121 |2 doi | |
| 028 | 5 | 2 | |a pubmed24n1351.xml |
| 035 | |a (DE-627)NLM369565614 | ||
| 035 | |a (NLM)38466835 | ||
| 040 | |a DE-627 |b ger |c DE-627 |e rakwb | ||
| 041 | |a eng | ||
| 100 | 1 | |a Krefeld-Schwalb, Antonia |e verfasserin |4 aut | |
| 245 | 1 | 0 | |a Exposing omitted moderators |b Explaining why effect sizes differ in the social sciences |
| 264 | 1 | |c 2024 | |
| 336 | |a Text |b txt |2 rdacontent | ||
| 337 | |a ƒaComputermedien |b c |2 rdamedia | ||
| 338 | |a ƒa Online-Ressource |b cr |2 rdacarrier | ||
| 500 | |a Date Completed 13.03.2024 | ||
| 500 | |a Date Revised 27.03.2024 | ||
| 500 | |a published: Print-Electronic | ||
| 500 | |a Citation Status MEDLINE | ||
| 520 | |a Policymakers increasingly rely on behavioral science in response to global challenges, such as climate change or global health crises. But applications of behavioral science face an important problem: Interventions often exert substantially different effects across contexts and individuals. We examine this heterogeneity for different paradigms that underlie many behavioral interventions. We study the paradigms in a series of five preregistered studies across one in-person and 10 online panels, with over 11,000 respondents in total. We find substantial heterogeneity across settings and paradigms, apply techniques for modeling the heterogeneity, and introduce a framework that measures typically omitted moderators. The framework's factors (Fluid Intelligence, Attentiveness, Crystallized Intelligence, and Experience) affect the effectiveness of many text-based interventions, producing different observed effect sizes and explaining variations across samples. Moderators are associated with effect sizes through two paths, with the intensity of the manipulation and with the effect of the manipulation directly. Our results motivate observing these moderators and provide a theoretical and empirical framework for understanding and predicting varying effect sizes in the social sciences | ||
| 650 | 4 | |a Journal Article | |
| 650 | 4 | |a choice architecture | |
| 650 | 4 | |a moderators | |
| 650 | 4 | |a online data collection | |
| 700 | 1 | |a Sugerman, Eli Rosen |e verfasserin |4 aut | |
| 700 | 1 | |a Johnson, Eric J |e verfasserin |4 aut | |
| 773 | 0 | 8 | |i Enthalten in |t Proceedings of the National Academy of Sciences of the United States of America |d 1915 |g 121(2024), 12 vom: 19. März, Seite e2306281121 |w (DE-627)NLM000008982 |x 1091-6490 |7 nnns |
| 773 | 1 | 8 | |g volume:121 |g year:2024 |g number:12 |g day:19 |g month:03 |g pages:e2306281121 |
| 856 | 4 | 0 | |u http://dx.doi.org/10.1073/pnas.2306281121 |3 Volltext |
| 912 | |a GBV_USEFLAG_A | ||
| 912 | |a GBV_NLM | ||
| 951 | |a AR | ||
| 952 | |d 121 |j 2024 |e 12 |b 19 |c 03 |h e2306281121 | ||