• Journal of Internet Computing and Services
    ISSN 2287 - 1136 (Online) / ISSN 1598 - 0170 (Print)
    https://jics.or.kr/

An Interpretable Log Anomaly System Using Bayesian Probability and Closed Sequence Pattern Mining


Jiyoung Yun, Gun-Yoon Shin, Dong-Wook Kim, Sang-Soo Kim, Myung-Mook Han, Journal of Internet Computing and Services, Vol. 22, No. 2, pp. 77-87, Apr. 2021
10.7472/jksii.2021.22.2.77, Full Text:
Keywords: Explainable AI, Log anomaly detection, Bayesian Probability, Rule Extraction

Abstract

With the development of the Internet and personal computers, various and complex attacks begin to emerge. As the attacks become more complex, signature-based detection become difficult. It leads to the research on behavior-based log anomaly detection. Recent work utilizes deep learning to learn the order and it shows good performance. Despite its good performance, it does not provide any explanation for prediction. The lack of explanation can occur difficulty of finding contamination of data or the vulnerability of the model itself. As a result, the users lose their reliability of the model. To address this problem, this work proposes an explainable log anomaly detection system. In this study, log parsing is the first to proceed. Afterward, sequential rules are extracted by Bayesian posterior probability. As a result, the "If condition then results, post-probability" type rule set is extracted. If the sample is matched to the ruleset, it is normal, otherwise, it is an anomaly. We utilize HDFS datasets for the experiment, resulting in F1score 92.7% in test dataset.


Statistics
Show / Hide Statistics

Statistics (Cumulative Counts from November 1st, 2017)
Multiple requests among the same browser session are counted as one view.
If you mouse over a chart, the values of data points will be shown.


Cite this article
[APA Style]
Yun, J., Shin, G., Kim, D., Kim, S., & Han, M. (2021). An Interpretable Log Anomaly System Using Bayesian Probability and Closed Sequence Pattern Mining. Journal of Internet Computing and Services, 22(2), 77-87. DOI: 10.7472/jksii.2021.22.2.77.

[IEEE Style]
J. Yun, G. Shin, D. Kim, S. Kim, M. Han, "An Interpretable Log Anomaly System Using Bayesian Probability and Closed Sequence Pattern Mining," Journal of Internet Computing and Services, vol. 22, no. 2, pp. 77-87, 2021. DOI: 10.7472/jksii.2021.22.2.77.

[ACM Style]
Jiyoung Yun, Gun-Yoon Shin, Dong-Wook Kim, Sang-Soo Kim, and Myung-Mook Han. 2021. An Interpretable Log Anomaly System Using Bayesian Probability and Closed Sequence Pattern Mining. Journal of Internet Computing and Services, 22, 2, (2021), 77-87. DOI: 10.7472/jksii.2021.22.2.77.