Algorithms for the adaptive assessment of procedural knowledge and skills
© 2022. The Author(s)..
Procedural knowledge space theory (PKST) was recently proposed by Stefanutti (British Journal of Mathematical and Statistical Psychology, 72(2) 185-218, 2019) for the assessment of human problem-solving skills. In PKST, the problem space formally represents how a family of problems can be solved and the knowledge space represents the skills required for solving those problems. The Markov solution process model (MSPM) by Stefanutti et al. (Journal of Mathematical Psychology, 103, 102552, 2021) provides a probabilistic framework for modeling the solution process of a task, via PKST. In this article, three adaptive procedures for the assessment of problem-solving skills are proposed that are based on the MSPM. Beside execution correctness, they also consider the sequence of moves observed in the solution of a problem with the aim of increasing efficiency and accuracy of assessments. The three procedures differ from one another in the assumption underlying the solution process, named pre-planning, interim-planning, and mixed-planning. In two simulation studies, the three adaptive procedures have been compared to one another and to the continuous Markov procedure (CMP) by Doignon and Falmagne (1988a). The last one accounts for dichotomous correct/wrong answers only. Results show that all the MSP-based adaptive procedures outperform the CMP in both accuracy and efficiency. These results have been obtained in the framework of the Tower of London test but the procedures can also be applied to all psychological and neuropsychological tests that have a problem space. Thus, the adaptive procedures presented in this paper pave the way to the adaptive assessment in the area of neuropsychological tests.
| Medienart: |
E-Artikel |
|---|
| Erscheinungsjahr: |
2023 |
|---|---|
| Erschienen: |
2023 |
| Enthalten in: |
Zur Gesamtaufnahme - volume:55 |
|---|---|
| Enthalten in: |
Behavior research methods - 55(2023), 7 vom: 16. Okt., Seite 3929-3951 |
| Sprache: |
Englisch |
|---|
| Beteiligte Personen: |
Brancaccio, Andrea [VerfasserIn] |
|---|
| Links: |
|---|
| Themen: |
Adaptive assessment |
|---|
| Anmerkungen: |
Date Completed 01.11.2023 Date Revised 04.11.2023 published: Print-Electronic Citation Status MEDLINE |
|---|
| doi: |
10.3758/s13428-022-01998-y |
|---|
| funding: |
|
|---|---|
| Förderinstitution / Projekttitel: |
|
| PPN (Katalog-ID): |
NLM350387303 |
|---|
| LEADER | 01000naa a22002652 4500 | ||
|---|---|---|---|
| 001 | NLM350387303 | ||
| 003 | DE-627 | ||
| 005 | 20231226044856.0 | ||
| 007 | cr uuu---uuuuu | ||
| 008 | 231226s2023 xx |||||o 00| ||eng c | ||
| 024 | 7 | |a 10.3758/s13428-022-01998-y |2 doi | |
| 028 | 5 | 2 | |a pubmed24n1167.xml |
| 035 | |a (DE-627)NLM350387303 | ||
| 035 | |a (NLM)36526887 | ||
| 040 | |a DE-627 |b ger |c DE-627 |e rakwb | ||
| 041 | |a eng | ||
| 100 | 1 | |a Brancaccio, Andrea |e verfasserin |4 aut | |
| 245 | 1 | 0 | |a Algorithms for the adaptive assessment of procedural knowledge and skills |
| 264 | 1 | |c 2023 | |
| 336 | |a Text |b txt |2 rdacontent | ||
| 337 | |a ƒaComputermedien |b c |2 rdamedia | ||
| 338 | |a ƒa Online-Ressource |b cr |2 rdacarrier | ||
| 500 | |a Date Completed 01.11.2023 | ||
| 500 | |a Date Revised 04.11.2023 | ||
| 500 | |a published: Print-Electronic | ||
| 500 | |a Citation Status MEDLINE | ||
| 520 | |a © 2022. The Author(s). | ||
| 520 | |a Procedural knowledge space theory (PKST) was recently proposed by Stefanutti (British Journal of Mathematical and Statistical Psychology, 72(2) 185-218, 2019) for the assessment of human problem-solving skills. In PKST, the problem space formally represents how a family of problems can be solved and the knowledge space represents the skills required for solving those problems. The Markov solution process model (MSPM) by Stefanutti et al. (Journal of Mathematical Psychology, 103, 102552, 2021) provides a probabilistic framework for modeling the solution process of a task, via PKST. In this article, three adaptive procedures for the assessment of problem-solving skills are proposed that are based on the MSPM. Beside execution correctness, they also consider the sequence of moves observed in the solution of a problem with the aim of increasing efficiency and accuracy of assessments. The three procedures differ from one another in the assumption underlying the solution process, named pre-planning, interim-planning, and mixed-planning. In two simulation studies, the three adaptive procedures have been compared to one another and to the continuous Markov procedure (CMP) by Doignon and Falmagne (1988a). The last one accounts for dichotomous correct/wrong answers only. Results show that all the MSP-based adaptive procedures outperform the CMP in both accuracy and efficiency. These results have been obtained in the framework of the Tower of London test but the procedures can also be applied to all psychological and neuropsychological tests that have a problem space. Thus, the adaptive procedures presented in this paper pave the way to the adaptive assessment in the area of neuropsychological tests | ||
| 650 | 4 | |a Journal Article | |
| 650 | 4 | |a Research Support, Non-U.S. Gov't | |
| 650 | 4 | |a Adaptive assessment | |
| 650 | 4 | |a Markov models | |
| 650 | 4 | |a Problem space | |
| 650 | 4 | |a Procedural knowledge space theory | |
| 650 | 4 | |a Tower of London test | |
| 700 | 1 | |a de Chiusole, Debora |e verfasserin |4 aut | |
| 700 | 1 | |a Stefanutti, Luca |e verfasserin |4 aut | |
| 773 | 0 | 8 | |i Enthalten in |t Behavior research methods |d 2005 |g 55(2023), 7 vom: 16. Okt., Seite 3929-3951 |w (DE-627)NLM157164268 |x 1554-3528 |7 nnns |
| 773 | 1 | 8 | |g volume:55 |g year:2023 |g number:7 |g day:16 |g month:10 |g pages:3929-3951 |
| 856 | 4 | 0 | |u http://dx.doi.org/10.3758/s13428-022-01998-y |3 Volltext |
| 912 | |a GBV_USEFLAG_A | ||
| 912 | |a GBV_NLM | ||
| 951 | |a AR | ||
| 952 | |d 55 |j 2023 |e 7 |b 16 |c 10 |h 3929-3951 | ||