Optimizing sepsis treatment strategies via a reinforcement learning model
Purpose The existing sepsis treatment lacks effective reference and relies too much on the experience of clinicians. Therefore, we used the reinforcement learning model to build an assisted model for the sepsis medication treatment. Methods Using the latest Sepsis 3.0 diagnostic criteria, 19,582 sepsis patients were screened from the Medical Intensive Care Information III database for treatment strategy research, and forty-six features were used in modeling. The study object of the medication strategy is the dosage of vasopressor drugs and intravenous infusion. Dueling DDQN is proposed to predict the patient’s medication strategy (vasopressor and intravenous infusion dosage) through the relationship between the patient’s state, reward function, and medication action. We also constructed protection against the possible high-risk behaviors of Dueling DDQN, especially sudden dose changes of vasopressors can lead to harmful clinical effects. In order to improve the guiding effect of clinically effective medication strategies on the model, we proposed a hybrid model (safe-dueling DDQN + expert strategies) to optimize medication strategies. Results The Dueling DDQN medication model for sepsis patients is superior to clinical strategies and other models in terms of off-policy evaluation values and mortality, and reduced the mortality of clinical strategies from 16.8 to 13.8%. Safe-Dueling DDQN we proposed, compared with Dueling DDQN, has an overall reduction in actions involving vasopressors and reduces large dose fluctuations. The hybrid model we proposed can switch between expert strategies and safe dueling DDQN strategies based on the current state of patients. Conclusions The reinforcement learning model we proposed for sepsis medication treatment, has practical clinical value and can improve the survival rate of patients to a certain extent while ensuring the balance and safety of medication..
| Medienart: |
E-Artikel |
|---|
| Erscheinungsjahr: |
2024 |
|---|---|
| Erschienen: |
2024 |
| Enthalten in: |
Zur Gesamtaufnahme - volume:14 |
|---|---|
| Enthalten in: |
Biomedical Engineering Letters - 14(2024), 2 vom: 04. Jan., Seite 279-289 |
| Sprache: |
Englisch |
|---|
| Beteiligte Personen: |
Zhang, Tianyi [VerfasserIn] |
|---|
| Links: |
Volltext [lizenzpflichtig] |
|---|
| Themen: |
|---|
| Anmerkungen: |
© Korean Society of Medical and Biological Engineering 2024. Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law. |
|---|
| doi: |
10.1007/s13534-023-00343-2 |
|---|
| funding: |
|
|---|---|
| Förderinstitution / Projekttitel: |
|
| PPN (Katalog-ID): |
SPR054810485 |
|---|
| LEADER | 01000naa a22002652 4500 | ||
|---|---|---|---|
| 001 | SPR054810485 | ||
| 003 | DE-627 | ||
| 005 | 20240218064619.0 | ||
| 007 | cr uuu---uuuuu | ||
| 008 | 240218s2024 xx |||||o 00| ||eng c | ||
| 024 | 7 | |a 10.1007/s13534-023-00343-2 |2 doi | |
| 035 | |a (DE-627)SPR054810485 | ||
| 035 | |a (SPR)s13534-023-00343-2-e | ||
| 040 | |a DE-627 |b ger |c DE-627 |e rakwb | ||
| 041 | |a eng | ||
| 100 | 1 | |a Zhang, Tianyi |e verfasserin |4 aut | |
| 245 | 1 | 0 | |a Optimizing sepsis treatment strategies via a reinforcement learning model |
| 264 | 1 | |c 2024 | |
| 336 | |a Text |b txt |2 rdacontent | ||
| 337 | |a Computermedien |b c |2 rdamedia | ||
| 338 | |a Online-Ressource |b cr |2 rdacarrier | ||
| 500 | |a © Korean Society of Medical and Biological Engineering 2024. Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law. | ||
| 520 | |a Purpose The existing sepsis treatment lacks effective reference and relies too much on the experience of clinicians. Therefore, we used the reinforcement learning model to build an assisted model for the sepsis medication treatment. Methods Using the latest Sepsis 3.0 diagnostic criteria, 19,582 sepsis patients were screened from the Medical Intensive Care Information III database for treatment strategy research, and forty-six features were used in modeling. The study object of the medication strategy is the dosage of vasopressor drugs and intravenous infusion. Dueling DDQN is proposed to predict the patient’s medication strategy (vasopressor and intravenous infusion dosage) through the relationship between the patient’s state, reward function, and medication action. We also constructed protection against the possible high-risk behaviors of Dueling DDQN, especially sudden dose changes of vasopressors can lead to harmful clinical effects. In order to improve the guiding effect of clinically effective medication strategies on the model, we proposed a hybrid model (safe-dueling DDQN + expert strategies) to optimize medication strategies. Results The Dueling DDQN medication model for sepsis patients is superior to clinical strategies and other models in terms of off-policy evaluation values and mortality, and reduced the mortality of clinical strategies from 16.8 to 13.8%. Safe-Dueling DDQN we proposed, compared with Dueling DDQN, has an overall reduction in actions involving vasopressors and reduces large dose fluctuations. The hybrid model we proposed can switch between expert strategies and safe dueling DDQN strategies based on the current state of patients. Conclusions The reinforcement learning model we proposed for sepsis medication treatment, has practical clinical value and can improve the survival rate of patients to a certain extent while ensuring the balance and safety of medication. | ||
| 650 | 4 | |a Sepsis |7 (dpeaa)DE-He213 | |
| 650 | 4 | |a Medication |7 (dpeaa)DE-He213 | |
| 650 | 4 | |a Dueling DDQN |7 (dpeaa)DE-He213 | |
| 650 | 4 | |a Off-policy evaluation |7 (dpeaa)DE-He213 | |
| 650 | 4 | |a Mortality |7 (dpeaa)DE-He213 | |
| 700 | 1 | |a Qu, Yimeng |4 aut | |
| 700 | 1 | |a wang, Deyong |4 aut | |
| 700 | 1 | |a Zhong, Ming |4 aut | |
| 700 | 1 | |a Cheng, Yunzhang |4 aut | |
| 700 | 1 | |a Zhang, Mingwei |0 (orcid)0000-0002-4388-2838 |4 aut | |
| 773 | 0 | 8 | |i Enthalten in |t Biomedical Engineering Letters |d Korean Society of Medical and Biological Engineering, 2011 |g 14(2024), 2 vom: 04. Jan., Seite 279-289 |w (DE-627)SPR03173507X |7 nnns |
| 773 | 1 | 8 | |g volume:14 |g year:2024 |g number:2 |g day:04 |g month:01 |g pages:279-289 |
| 856 | 4 | 0 | |u https://dx.doi.org/10.1007/s13534-023-00343-2 |z lizenzpflichtig |3 Volltext |
| 912 | |a GBV_USEFLAG_A | ||
| 912 | |a GBV_SPRINGER | ||
| 951 | |a AR | ||
| 952 | |d 14 |j 2024 |e 2 |b 04 |c 01 |h 279-289 | ||