Nested ensemble selection : An effective hybrid feature selection method
© 2023 The Author(s)..
It has been shown that while feature selection algorithms are able to distinguish between relevant and irrelevant features, they fail to differentiate between relevant and redundant and correlated features. To address this issue, we propose a highly effective approach, called Nested Ensemble Selection (NES), that is based on a combination of filter and wrapper methods. The proposed feature selection algorithm differs from the existing filter-wrapper hybrid methods in its simplicity and efficiency as well as precision. The new algorithm is able to separate the relevant variables from the irrelevant as well as the redundant and correlated features. Furthermore, we provide a robust heuristic for identifying the optimal number of selected features which remains one of the greatest challenges in feature selection. Numerical experiments on synthetic and real-life data demonstrate the effectiveness of the proposed method. The NES algorithm achieves perfect precision on the synthetic data and near optimal accuracy on the real-life data. The proposed method is compared against several popular algorithms including mRMR, Boruta, genetic, recursive feature elimination, Lasso, and Elastic Net. The results show that NES significantly outperforms the benchmarks algorithms especially on multi-class datasets.
| Medienart: |
E-Artikel |
|---|
| Erscheinungsjahr: |
2023 |
|---|---|
| Erschienen: |
2023 |
| Enthalten in: |
Zur Gesamtaufnahme - volume:9 |
|---|---|
| Enthalten in: |
Heliyon - 9(2023), 9 vom: 28. Sept., Seite e19686 |
| Sprache: |
Englisch |
|---|
| Beteiligte Personen: |
Kamalov, Firuz [VerfasserIn] |
|---|
| Links: |
|---|
| Themen: |
Ensemble selection |
|---|
| Anmerkungen: |
Date Revised 18.10.2023 published: Electronic-eCollection Citation Status PubMed-not-MEDLINE |
|---|
| doi: |
10.1016/j.heliyon.2023.e19686 |
|---|
| funding: |
|
|---|---|
| Förderinstitution / Projekttitel: |
|
| PPN (Katalog-ID): |
NLM363030239 |
|---|
| LEADER | 01000naa a22002652 4500 | ||
|---|---|---|---|
| 001 | NLM363030239 | ||
| 003 | DE-627 | ||
| 005 | 20231226092503.0 | ||
| 007 | cr uuu---uuuuu | ||
| 008 | 231226s2023 xx |||||o 00| ||eng c | ||
| 024 | 7 | |a 10.1016/j.heliyon.2023.e19686 |2 doi | |
| 028 | 5 | 2 | |a pubmed24n1210.xml |
| 035 | |a (DE-627)NLM363030239 | ||
| 035 | |a (NLM)37809839 | ||
| 040 | |a DE-627 |b ger |c DE-627 |e rakwb | ||
| 041 | |a eng | ||
| 100 | 1 | |a Kamalov, Firuz |e verfasserin |4 aut | |
| 245 | 1 | 0 | |a Nested ensemble selection |b An effective hybrid feature selection method |
| 264 | 1 | |c 2023 | |
| 336 | |a Text |b txt |2 rdacontent | ||
| 337 | |a ƒaComputermedien |b c |2 rdamedia | ||
| 338 | |a ƒa Online-Ressource |b cr |2 rdacarrier | ||
| 500 | |a Date Revised 18.10.2023 | ||
| 500 | |a published: Electronic-eCollection | ||
| 500 | |a Citation Status PubMed-not-MEDLINE | ||
| 520 | |a © 2023 The Author(s). | ||
| 520 | |a It has been shown that while feature selection algorithms are able to distinguish between relevant and irrelevant features, they fail to differentiate between relevant and redundant and correlated features. To address this issue, we propose a highly effective approach, called Nested Ensemble Selection (NES), that is based on a combination of filter and wrapper methods. The proposed feature selection algorithm differs from the existing filter-wrapper hybrid methods in its simplicity and efficiency as well as precision. The new algorithm is able to separate the relevant variables from the irrelevant as well as the redundant and correlated features. Furthermore, we provide a robust heuristic for identifying the optimal number of selected features which remains one of the greatest challenges in feature selection. Numerical experiments on synthetic and real-life data demonstrate the effectiveness of the proposed method. The NES algorithm achieves perfect precision on the synthetic data and near optimal accuracy on the real-life data. The proposed method is compared against several popular algorithms including mRMR, Boruta, genetic, recursive feature elimination, Lasso, and Elastic Net. The results show that NES significantly outperforms the benchmarks algorithms especially on multi-class datasets | ||
| 650 | 4 | |a Journal Article | |
| 650 | 4 | |a Ensemble selection | |
| 650 | 4 | |a Feature selection | |
| 650 | 4 | |a Filter method | |
| 650 | 4 | |a Machine learning | |
| 650 | 4 | |a Random forest | |
| 650 | 4 | |a Synthetic data | |
| 650 | 4 | |a Wrapper method | |
| 700 | 1 | |a Sulieman, Hana |e verfasserin |4 aut | |
| 700 | 1 | |a Moussa, Sherif |e verfasserin |4 aut | |
| 700 | 1 | |a Reyes, Jorge Avante |e verfasserin |4 aut | |
| 700 | 1 | |a Safaraliev, Murodbek |e verfasserin |4 aut | |
| 773 | 0 | 8 | |i Enthalten in |t Heliyon |d 2015 |g 9(2023), 9 vom: 28. Sept., Seite e19686 |w (DE-627)NLM255913095 |x 2405-8440 |7 nnns |
| 773 | 1 | 8 | |g volume:9 |g year:2023 |g number:9 |g day:28 |g month:09 |g pages:e19686 |
| 856 | 4 | 0 | |u http://dx.doi.org/10.1016/j.heliyon.2023.e19686 |3 Volltext |
| 912 | |a GBV_USEFLAG_A | ||
| 912 | |a GBV_NLM | ||
| 951 | |a AR | ||
| 952 | |d 9 |j 2023 |e 9 |b 28 |c 09 |h e19686 | ||