Main Article Content
Abstract
The classification of disease diagnoses using the International Classification of Diseases (ICD-10) standard is essential for supporting clinical decision-making and administrative processes in healthcare systems. This study evaluated the performance of three machine learning algorithms, namely decision tree, random forest, and support vector machine (SVM), for ICD-10 diagnosis classification using 3,730 textual medical record entries collected from the Klinik Pratama UIN Sunan Kalijaga, Yogyakarta, Indonesia. The dataset exhibited significant class imbalance, which was addressed using the synthetic minority oversampling technique (SMOTE). The preprocessing procedures included text normalization and Term frequency-inverse document frequency (TF-IDF) vectorization, followed by model development with hyperparameter tuning through grid search cross validation. Model performance was assessed using accuracy, precision, recall, F1-score, confusion matrix, and five-fold cross validation. Random forest achieved the highest mean accuracy at 93.65%, followed by decision tree at 92.25% and SVM at 87.91%. These results indicate that ensemble-based approaches provide more reliable classification outcomes for imbalanced textual medical data. The findings are expected to support the development of semi-automated ICD-10 coding systems and improve the efficiency and accuracy of medical coding workflows.
Keywords
Article Details
Authors who publish with this journal agree to the following terms:
- Authors retain copyright and grant the journal right of first publication with the work simultaneously licensed under a Creative Commons Attribution License that allows others to share the work with an acknowledgment of the work's authorship and initial publication in this journal.
- Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgment of its initial publication in this journal.
- Authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website) prior to and during the submission process, as it can lead to productive exchanges, as well as earlier and greater citation of published work (See The Effect of Open Access).
References
- X. Zhan, M. Humbert-Droz, P. Mukherjee, and O. Gevaert, “Structuring clinical text with AI: Old versus new natural language processing techniques evaluated on eight common cardiovascular diseases,” Patterns, vol. 2, no. 7, Jul. 2021, doi: 10.1016/j.patter.2021.100289.
- World Health Organization, “International Statistical Classification of Diseases and Related Health Problems 10th Revision.” 2015. [Online]. Available: https://icd.who.int/browse10/2015/en
- Z.A. Gafurov, “Classification, clinic and diagnosis of orbital factures,” Frontline Med. Sci. Pharm. J., vol. 02, no. 03, pp. 19–34, Mar. 2022, doi: 10.37547/medical-fmspj-02-03-03.
- R. Verma, A. Jain, and D. Ladsaria, “Automated extraction of ICD-10 diagnosis codes from clinical notes,” 2021. [Online]. Available: https://api.semanticscholar.org/CorpusID:269313878.
- A. Wibowo, Indarti, and D. Laraswati, “Komparasi algoritma decision tree, random forest dan SVM untuk prognosis COVID-19,” IMTechno J. Ind. Manag. Technol., vol. 5, no. 2, pp. 10–15, Jul. 2024, doi: 10.31294/imtechno.v5i2.2868.
- A.F. Fadhlullah and T. Widiyaningtyas, “Comparative analysis of decision tree and random forest algorithms for diabetes prediction,” J. Teori Aplikasi Mat., vol. 8, no. 4, pp. 1121–1132, Oct. 2024, doi: 10.31764/jtam.v8i4.24388.
- S. Yulianty and M.K. Najib, “Comparing the accuracy of k-nearest neighbor (KNN), random forest, and decision tree methods in predicting diabetes,” Al-Aqlu J. Mat. Tek. Sains., vol. 3, no. 2, pp. 144–151, Jul. 2025. [Online]. Available: https://jurnal.yalamqa.com/index.php/aqlu
- A. Rajkomar, J. Dean, and I. Kohane, “Machine learning in medicine,” New Engl. J. Med., vol. 380, no. 14, pp. 1347–1358, Apr. 2019, doi: 10.1056/nejmra1814259.
- S. Qaiser and R. Ali, “Text mining: Use of TF-IDF to examine the relevance of words to documents,” Int. J. Comput. Appl., vol. 181, no. 1, pp. 25–29, Jul. 2018, doi: 10.5120/ijca2018917395.
- B. Biswas, T.-H. Pham, and P. Zhang, “TransICD: Transformer based code-wise attention model for explainable ICD coding,” 2021, arXiv:2104.10652.
- L. Rokach and O. Maimon, “Decision trees,” in Data Mining and Knowledge Discovery Handbook, O. Maimon and L. Rokach, Eds., Boston, MA, USA: Springer, 2005, pp. 165–192.
- R. Khan, N. Ahmad, J. Ali, and I. Maqsood, “Random forests and decision trees,” Int. J. Comput. Sci., vol. 9, no. 5, pp. 272–278, Sep. 2012.
- Adeen and P. Sondhi, “Random forest based heart disease prediction,” Int. J. Sci. Res. (IJSR), vol. 10, no. 2, pp. 1669–1672, Feb. 2021, doi: 10.21275/sr21225214148.
- G. Louppe, “Understanding random forests: From theory to practice,” Ph.D. dissertation, Dept. Electr. Eng. Comput. Sci, University of Liège, Liège, Belgium, 2014.
- K. Dharmarajan, K. Balasree, A.S. Arunachalam, and K. Abirmai, “Thyroid disease classification using decision tree and SVM,” Indian J. Public Health Res. Develop., vol. 11, no. 3, pp. 224–229, Mar. 2020, doi: 10.37506/IJPHRD.V11I3.822.
- M. Mohammadagha, “Hyperparameter optimization strategies for tree-based machine learning models prediction: A comparative study of AdaBoost, decision trees, and random forest,” 2025. [Online]. Available: https://dx.doi.org/10.2139/ssrn.5226457.
- T. Kavzoglu, F. Bilucan, and A. Teke, “Comparison of support vector machines, random forest, and decision tree methods for classification of Sentinel-2A image using different band combinations,” in 41st Asian Conf. Remote Sens., Deqing, China, 2020, pp. 2145–2152.
- P.W.S. Aji, Suprianto, and R. Dijaya, “Stroke disease prediction using random forest method,” KESATRIA J. Penerapan Sist. Inf. (Komp. Manaj.), vol. 4, no. 4, pp. 916–924, Oct. 2023, doi: 10.30645/kesatria.v4i4.242.g240.
- A. Zollanvari, “Model evaluation and selection,” in Machine Learning with Python, A. Zollanvari, Ed., Switzerland: Springer Cham, 2023, pp. 237–281.
- T. Barwahwala, A. Mahajan, S. Mittal, and O. Reich, “Is Model Accuracy Enough? A Field Evaluation of a Machine Learning Model to Catch Bogus Firms,” 2024. [Online]. Available: http://www.nber.org/papers/w32705
References
X. Zhan, M. Humbert-Droz, P. Mukherjee, and O. Gevaert, “Structuring clinical text with AI: Old versus new natural language processing techniques evaluated on eight common cardiovascular diseases,” Patterns, vol. 2, no. 7, Jul. 2021, doi: 10.1016/j.patter.2021.100289.
World Health Organization, “International Statistical Classification of Diseases and Related Health Problems 10th Revision.” 2015. [Online]. Available: https://icd.who.int/browse10/2015/en
Z.A. Gafurov, “Classification, clinic and diagnosis of orbital factures,” Frontline Med. Sci. Pharm. J., vol. 02, no. 03, pp. 19–34, Mar. 2022, doi: 10.37547/medical-fmspj-02-03-03.
R. Verma, A. Jain, and D. Ladsaria, “Automated extraction of ICD-10 diagnosis codes from clinical notes,” 2021. [Online]. Available: https://api.semanticscholar.org/CorpusID:269313878.
A. Wibowo, Indarti, and D. Laraswati, “Komparasi algoritma decision tree, random forest dan SVM untuk prognosis COVID-19,” IMTechno J. Ind. Manag. Technol., vol. 5, no. 2, pp. 10–15, Jul. 2024, doi: 10.31294/imtechno.v5i2.2868.
A.F. Fadhlullah and T. Widiyaningtyas, “Comparative analysis of decision tree and random forest algorithms for diabetes prediction,” J. Teori Aplikasi Mat., vol. 8, no. 4, pp. 1121–1132, Oct. 2024, doi: 10.31764/jtam.v8i4.24388.
S. Yulianty and M.K. Najib, “Comparing the accuracy of k-nearest neighbor (KNN), random forest, and decision tree methods in predicting diabetes,” Al-Aqlu J. Mat. Tek. Sains., vol. 3, no. 2, pp. 144–151, Jul. 2025. [Online]. Available: https://jurnal.yalamqa.com/index.php/aqlu
A. Rajkomar, J. Dean, and I. Kohane, “Machine learning in medicine,” New Engl. J. Med., vol. 380, no. 14, pp. 1347–1358, Apr. 2019, doi: 10.1056/nejmra1814259.
S. Qaiser and R. Ali, “Text mining: Use of TF-IDF to examine the relevance of words to documents,” Int. J. Comput. Appl., vol. 181, no. 1, pp. 25–29, Jul. 2018, doi: 10.5120/ijca2018917395.
B. Biswas, T.-H. Pham, and P. Zhang, “TransICD: Transformer based code-wise attention model for explainable ICD coding,” 2021, arXiv:2104.10652.
L. Rokach and O. Maimon, “Decision trees,” in Data Mining and Knowledge Discovery Handbook, O. Maimon and L. Rokach, Eds., Boston, MA, USA: Springer, 2005, pp. 165–192.
R. Khan, N. Ahmad, J. Ali, and I. Maqsood, “Random forests and decision trees,” Int. J. Comput. Sci., vol. 9, no. 5, pp. 272–278, Sep. 2012.
Adeen and P. Sondhi, “Random forest based heart disease prediction,” Int. J. Sci. Res. (IJSR), vol. 10, no. 2, pp. 1669–1672, Feb. 2021, doi: 10.21275/sr21225214148.
G. Louppe, “Understanding random forests: From theory to practice,” Ph.D. dissertation, Dept. Electr. Eng. Comput. Sci, University of Liège, Liège, Belgium, 2014.
K. Dharmarajan, K. Balasree, A.S. Arunachalam, and K. Abirmai, “Thyroid disease classification using decision tree and SVM,” Indian J. Public Health Res. Develop., vol. 11, no. 3, pp. 224–229, Mar. 2020, doi: 10.37506/IJPHRD.V11I3.822.
M. Mohammadagha, “Hyperparameter optimization strategies for tree-based machine learning models prediction: A comparative study of AdaBoost, decision trees, and random forest,” 2025. [Online]. Available: https://dx.doi.org/10.2139/ssrn.5226457.
T. Kavzoglu, F. Bilucan, and A. Teke, “Comparison of support vector machines, random forest, and decision tree methods for classification of Sentinel-2A image using different band combinations,” in 41st Asian Conf. Remote Sens., Deqing, China, 2020, pp. 2145–2152.
P.W.S. Aji, Suprianto, and R. Dijaya, “Stroke disease prediction using random forest method,” KESATRIA J. Penerapan Sist. Inf. (Komp. Manaj.), vol. 4, no. 4, pp. 916–924, Oct. 2023, doi: 10.30645/kesatria.v4i4.242.g240.
A. Zollanvari, “Model evaluation and selection,” in Machine Learning with Python, A. Zollanvari, Ed., Switzerland: Springer Cham, 2023, pp. 237–281.
T. Barwahwala, A. Mahajan, S. Mittal, and O. Reich, “Is Model Accuracy Enough? A Field Evaluation of a Machine Learning Model to Catch Bogus Firms,” 2024. [Online]. Available: http://www.nber.org/papers/w32705