A SHAP-Based Comparative Analysis of Machine Learning Model Interpretability in Financial Classification Tasks

Article DOI: https://doi.org/10.57017/jaes.v20.3(89).03

Author(s):

Chia-Pang CHAN Department of Information Management, Cheng Shiu University, Taiwan
Chiung-Hui TSAI Department of Computer Science and Information Engineering, Da-Yeh University, Taiwan
Fang-Kai TANG Department of E-Sports Technology, Cheng Shiu University, Taiwan
Jun-He YANG Department of Mass Communication, The Open University of Kaohsiung, Taiwan

Abstract:

As artificial intelligence technologies become increasingly prevalent across the financial sector, the interpretability of machine learning models has become a critical concern for regulatory authorities and financial institutions. This study employs SHAP (SHapley Additive exPlanations) to systematically compare the predictive performance and interpretability of five mainstream machine learning models in financial classification tasks. Using a real financial dataset containing 24 financial indicators to train logistic regression, five machine learning models - logistic regression, random forest, XGBoost, LightGBM, and support vector machine - are trained on this dataset. SHAP is then applied to analyse the feature importance patterns across models. Empirical results demonstrate that LightGBM achieves the best predictive performance (accuracy 95.90%, Area Under the Curve (AUC) 99.18%), while XGBoost shows advantages in terms of interpretability. SHAP analysis identifies those prior earnings per share is the most critical feature, and the Top-K overlap analysis reveals a high degree of consistency among tree-based models in feature importance recognition. This study provides scientific basis for financial institutions to select appropriate explainable AI models, and holds significant importance for enhancing transparency and trustworthiness in financial AI applications.

Copyright© 2025 The Author(s). This article is distributed under the terms of the license CC-BY 4.0., which permits any further distribution in any medium, provided the original work is properly cited.

Article’s history: Received 20th of June, 2025; Revised 18th of July, 2025; Accepted 15th of August, 2025; Available online: 30th of September, 2025. Published as article in the Volume XX, Fall, Issue 3(89), 2025.

How to cite:

Chan, C.-P., Tsai, C.-H., Tang, F.-K. & Yang, J.-H. (2025) A SHAP-Based Comparative Analysis of Machine Learning Model Interpretability in Financial Classification Tasks. Journal of Applied Economic Sciences, Volume XX, Fall, 3(89), 385 - 400. https://doi.org/10.57017/jaes.v20.3(89).03

Acknowledgments/Funding: The authors received no financial or material support that could have influenced the results or their interpretation.

Conflict of Interest Statement:The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Data Availability Statement:The data supporting the findings of this study are available from the corresponding author upon reasonable request.

References:

Alghamdi, H., & Alqithami, S. (2025). A robust machine learning framework for stock market classification. Expert Systems with Applications, 241, 128573. https://doi.org/10.1016/j.eswa.2025.128573

Ali, S., Abuhmed, T., El-Sappagh, S., Muhammad, K., Alonso-Moral, J. M., Confalonieri, R., Guidotti, R., Del Ser, J., Díaz-Rodríguez, N., & Herrera, F. (2023). Explainable artificial intelligence (XAI): what we know and what is left to attain trustworthy artificial intelligence. Information Fusion, 99, 101805. https://doi.org/10.1016/j.inffus.2023.101805

Altman, E. I., Sabato, G., & Wilson, N. (2010). The value of non-financial information in SME risk management. Journal of Credit Risk, 6(2), 95-127. https://doi.org/10.21314/JCR.2010.110

Angelov, P. P., Soares, E. A., Jiang, R., Arnold, N. I., & Atkinson, P. M. (2021). Explainable artificial intelligence: an analytical review. Wiley Interdisciplinary Reviews: Data Mining and Knowledge Discovery, 11(6), e1424. https://doi.org/10.1002/widm.1424

Arrieta, A. B., Díaz-Rodríguez, N., Del Ser, J., Bennetot, A., Tabik, S., Barbado, A., Garcia, S., Gil-Lopez, S., Molina, D., Benjamins, R., Chatila, R. & Herrera, F. (2020). Explainable artificial intelligence (XAI): concepts, taxonomies, opportunities and challenges toward responsible AI. Information Fusion, 58, 82-115. https://doi.org/10.1016/j.inffus.2019.12.012

Babaei, G., Giudici, P., & Raffinetti, E. (2023). Explainable FinTech lending. Journal of Economic Business, 125-126, 106126. https://doi.org/10.1016/j.jeconbus.2023.106126

Becker, G. S. (1968). Crime and punishment: An economic approach. Journal of Political Economy, 76(2), 169-217. https://doi.org/10.1086/259394

Benoumechiara, N., & Elie-Dit-Cosaque, K. (2019). Shapley effects for sensitivity analysis with dependent inputs: bootstrap and kriging-based algorithms, ESAIM: Proceedings and Surveys, Volume 65, 266–293. https://doi.org/10.1051/proc/201965266

Bhattacharya, S. (1979). Imperfect information, dividend policy, and "the bird in the hand" fallacy. The Bell Journal of Economics, 10(1), 259-270. https://doi.org/10.2307/3003330

Casey, B., Farhangi, A., & Vogl, R. (2019). Rethinking explainable machines: The GDPR's "right to explanation" debate and the rise of algorithmic audits in enterprise. Berkeley Technology Law Journal, 34(1), 143-188. https://doi.org/10.15779/Z38M32N986

Chen, H., Janizek, J. D., Lundberg, S., & Lee, S. I. (2020). True to the model or true to the data? arXiv Preprint. https://doi.org/10.48550/arXiv.2006.16234

Covert, I., & Lee, S. I. (2021). Improving KernelSHAP: Practical Shapley value estimation using linear regression. Proceedings of the 24th International Conference on Artificial Intelligence and Statistics, 3457-3465. Available from https://proceedings.mlr.press/v130/covert21a.html

Černevičienė, J., & Kabašinskas, A. (2024). Explainable artificial intelligence (XAI) in finance: a systematic literature review. Artificial Intelligence Review, 57(8), 216. https://doi.org/10.1007/s10462-024-10854-8

Ding, S., Cui, T., Bellotti, A. G., Abedin, M. Z., & Lucey, B. (2023). The role of feature importance in predicting corporate financial distress in pre and post COVID periods: Evidence from China. International Review of Financial Analysis, 90, 102851. https://doi.org/10.1016/j.irfa.2023.102851

Doshi-Velez, F., & Kim, B. (2017). Towards a rigorous science of interpretable machine learning. arXiv Preprint. https://doi.org/10.48550/arXiv.1702.08608

Fatma, N., & Hidayat, W. (2020). Earnings persistence, earnings power, and equity valuation in consumer goods firms. Asian Journal of Accounting Research, 5(1), 3-13. https://doi.org/10.1108/ajar-05-2019-0041

Khan, F. S., Mazhar, S. S., Mazhar, K., AlSaleh, D. A., & Mazhar, A. (2025). Model-agnostic explainable artificial intelligence methods in finance: a systematic review, recent developments, limitations, challenges and future directions. Artificial Intelligence Review, 58(232), 1-45. https://doi.org/10.1007/s10462-025-11215-9

Kumar, I. E., Venkatasubramanian, S., Scheidegger, C., & Friedler, S. (2020). Problems with Shapley-value-based explanations as feature importance measures. In International Conference on Machine Learning, pp. 5491-5500. https://doi.org/10.48550/arXiv.2002.11097

Lang, L.H.P., & Stulz, R. (1992). Contagion and competitive intra-industry effects of bankruptcy announcements: An empirical analysis. Journal of Financial Economics, 32(1), 45-60. https://doi.org/10.1016/0304-405X(92)90024-R

Lundberg, S. M., Erion, G., Chen, H., DeGrave, A., Prutkin, J. M., Nair, B., Katz, R., Himmelfarb, J., Bansal, N., & Lee, S. I. (2020). From local explanations to global understanding with explainable AI for trees. Nature Machine Intelligence, 2(1), 56-67. https://doi.org/10.1038/s42256-019-0138-9

Lundberg, S. M., & Lee, S. I. (2017). A unified approach to interpreting model predictions. Advances in Neural Information Processing Systems, 30. https://doi.org/10.48550/arXiv.1705.07874

Misheva, B.H., Osterrieder, J., Kulkarni, O., & Lin, S.F. (2021). Explainable AI in credit risk management. arXiv Preprint. https://doi.org/10.48550/arXiv.2103.00949

Nallakaruppan, M. K., Chaturvedi, H., Grover, V., Balusamy, B., Jaraut, P., Bahadur, J., Meena, V. P., & Hameed, I. A. (2024). Credit Risk Assessment and Financial Decision Support Using Explainable Artificial Intelligence. Risks, 12(10), 164. https://doi.org/10.3390/risks12100164

Nguyen, N., & Ngo, D. (2025). Comparative analysis of boosting algorithms for predicting personal default. Cogent Economics & Finance, 13(1), 2465971. https://doi.org/10.1080/23322039.2025.2465971

Ribeiro, M. T., Singh, S., & Guestrin, C. (2016). "Why should I trust you?" Explaining the predictions of any classifier. In Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 1135-1144. https://doi.org/10.1145/2939672.2939778

Saarela, M., & Podgorelec, V. (2024). Recent applications of explainable AI (XAI): A systematic literature review. Applied Sciences, 14(19), 8884. https://doi.org/10.3390/app14198884

Soliman, M. T. (2008). The use of DuPont analysis by market participants. The Accounting Review, 83(3), 823-853. https://doi.org/10.2308/accr.2008.83.3.823

Stiglitz, J. E., & Weiss, A. (1981). Credit Rationing in Markets with Imperfect Information. The American Economic Review, 71(3), 393–410.http://www.jstor.org/stable/1802787

Thanathamathee P., Sawangarreerak S., Chantamunee S., & Nizam D. N. (2024). SHAP-instance weighted and anchor explainable AI: enhancing XGBoost for financial fraud detection. Emerging Science Journal, 8(6), 2404–2430. https://doi.org/10.28991/ESJ-2024-08-06-016

Tyagi, S. (2022). Analysing machine learning models for credit scoring and investment decisions: Interpretability matters. arXiv Preprint.https://doi.org/10.48550/arXiv.2209.09362

Warner, J. B. (1977). Bankruptcy costs: Some evidence. The Journal of Finance, 32(2), 337-347. https://doi.org/10.1111/j.1540-6261.1977.tb03274.x

Yeo, W. J., van der Heever, W., Mao, R., Cambria, E., Satapathy, R., & Mengaldo, G. (2025). A comprehensive review on financial explainable AI. Artificial Intelligence Review, 58, 189. https://doi.org/10.1007/s10462-024-11077-7

Zhou, Y., Li, H., Xiao, Z., & Qiu, J. (2023). A user-centered explainable artificial intelligence approach for financial fraud detection. Finance Research Letters, 58, 104309. https://doi.org/10.1016/j.frl.2023.104309