Multimodal affect analysis of psychodynamic play therapy

Objective: We explore state of the art machine learning based tools for automatic facial and linguistic affect analysis to allow easier, faster, and more precise quantification and annotation of children's verbal and non-verbal affective expressions in psychodynamic child psychotherapy. Method: The sample included 53 Turkish children: 41 with internalizing, externalizing and comorbid problems; 12 in the non-clinical range. We collected audio and video recordings of 148 sessions, which were manually transcribed. Independent raters coded children's expressions of pleasure, anger, sadness and anxiety using the Children's Play Therapy Instrument (CPTI). Automatic facial and linguistic affect analysis modalities were adapted, developed, and combined in a system that predicts affect. Statistical regression methods (linear and polynomial regression) and machine learning techniques (deep learning, support vector regression and extreme learning machine) were used for predicting CPTI affect dimensions. Results: Experimental results show significant associations between automated affect predictions and CPTI affect dimensions with small to medium effect sizes. Fusion of facial and linguistic features work best for pleasure predictions; however, for other affect predictions linguistic analyses outperform facial analyses. External validity analyses partially support anger and pleasure predictions. Discussion: The system enables retrieving affective expressions of children, but needs improvement for precision.

Medienart:

E-Artikel

Erscheinungsjahr:

2021

Erschienen:

2021

Enthalten in:

Zur Gesamtaufnahme - volume:31

Enthalten in:

Psychotherapy research : journal of the Society for Psychotherapy Research - 31(2021), 3 vom: 20. März, Seite 402-417

Sprache:

Englisch

Beteiligte Personen:

Halfon, Sibel [VerfasserIn]
Doyran, Metehan [VerfasserIn]
Türkmen, Batıkan [VerfasserIn]
Oktay, Eda Aydın [VerfasserIn]
Salah, Ali Albert [VerfasserIn]

Links:

Volltext

Themen:

Face analysis
Journal Article
Multimodal affect analysis
Psychodynamic play therapy
Research Support, Non-U.S. Gov't
Text analysis

Anmerkungen:

Date Completed 21.05.2021

Date Revised 21.05.2021

published: Print-Electronic

Citation Status MEDLINE

doi:

10.1080/10503307.2020.1839141

funding:

Förderinstitution / Projekttitel:

PPN (Katalog-ID):

NLM317163272