ANOMALY DETECTION - REVIEW OF METHODS, TOOLS AND ALGORITHMS
DOI:
https://doi.org/10.17770/etr2023vol2.7283Keywords:
Algorithms, Anomaly detection, Novelty detection, Outlier detectionAbstract
This paper contains review of algorithms, methods and tools nowadays used for anomaly detection.
Anomaly detection is used in many domains of science and industry, some authors classify anomaly detection as data mining and data science tool, others state it is decision support tool under artificial intelligence domain and indeed the use cases of anomaly detection are very different.
The article describes the main algorithms used for anomaly detection from perspective of theory of computer science and practical use cases of anomaly detection in different domains of industry. Several paragraphs are dedicated to the frameworks used by one of the most popular and powerful anomaly detection tools available in the market - Microsoft Anomaly detector.
Downloads
References
V. Chandola, A. Banerjee, V. Kumar, “Anomaly detection: A survey,” ACM Comput. Surv., 2009, pp. 41, 1–72.
P. Kattamuri, “How to build a serverless real-time credit card fraud detection solution,” March, 2021. [Online] Available: https://cloud.google.com/blog/products/data-analytics/how-to-build-a-fraud-detection-solution [Accessed: Feb. 18, 2023].
HSBC Holdings plc, “HSBC to launch AML system with google cloud,” February, 2023. [Online] Available: https://www.pymnts.com/google/2023/google-pay-ditches-the-cvv-with-virtual-card-numbers-for-amex-holders/ [Accessed: Feb. 18, 2023].
Google LLC, “Visual inspection ai,” [Online] Available: https://cloud.google.com/solutions/visual-inspection-ai [Accessed: Feb. 18, 2023].
M. Narang, “Anomaly detection in machine learning,” March, 2023. [Online] Available: https://www.shiksha.com/online-courses/articles/anomaly-detection/ [Accessed: Feb. 23, 2023].
A. Kargwal, “Anomaly detection in machine learning,” August, 2022. [Online] Available: https://nimblebox.ai/blog/anomaly-detection-machine-learning [Accessed: Feb. 18, 2023].
S. Kumar, “5 anomaly detection algorithms every data scientist should know,” December, 2021. [Online] Available: https://towardsdatascience.com/5-anomaly-detection-algorithms-every-data-scientist-should-know-b36c3605ea16 [Accessed: Feb. 15, 2023].
Y. Zheng, J. Guo, D. Ghent, K. Tansey, X. Hu, J. Nie, & S. Chen, “Land surface temperature retrieval from sentinel-3 a sea and land surface temperature radiometer, using a split-window algorithm. Remote Sensing, ” 2019, pp. 11, 650. https://doi.org/10.3390/rs11060650
T. Sushir, “Anomaly detection: Guide to prevent network intrusions,” January, 2023. [Online] Available: https://geekflare.com/anomaly-detection/ [Accessed: Feb. 26, 2023].
The MathWorks, Inc, “Identify unexpected events and departures from normal behavior,” [Online] Available: https://la.mathworks.com/discovery/anomaly-detection.html [Accessed: Feb. 23, 2023].
Wikipedia, “Isolation forest,” [Online] Available: https://en.wikipedia.org/wiki/Isolation_forest [Accessed: Feb. 22, 2023].
Wikipedia, “Anomaly detection,” [Online] Available: https://en.wikipedia.org/wiki/Anomaly_detection [Accessed: Feb. 15, 2023].
Wikipedia, “Mahalanobis distance,” [Online] Available: https://en.wikipedia.org/wiki/Mahalanobis_distance [Accessed: Feb. 22, 2023].
Dataconomy Media GmbH, “Anomaly detection in machine learning”, [Online] Available:
https://dataconomy.com/2022/10/machine-learning-anomaly-detection/ [Accessed: Feb. 20, 2023].
Wikipedia, “Local outlier factor,” [Online] Available: https://en.wikipedia.org/wiki/Local_outlier_factor [Accessed: Feb. 22, 2023].
J. Yoon, S. O. Arik. “Unsupervised and semi-supervised anomaly detection with data-centric ML,” February, 2023. https://ai.googleblog.com/2023/02/unsupervised-and-semi-supervised.html [Accessed: Feb. 18, 2023].
Wikipedia, “Anomaly detection,” [Online] Available: https://en.wikipedia.org/wiki/Anomaly_detection [Accessed: Feb. 15, 2023].
Microsoft Corporation, “Azure cognitive services – overview,” [Online] Available: https://azure.microsoft.com/en-us/products/cognitive-services/#overview [Accessed: Feb. 22, 2023].
Wikipedia, “Time series,” [Online] Available: https://en.wikipedia.org/wiki/Time_series [Accessed: Feb. 22, 2023].
T. Xing, “Introducing azure anomaly detector api,” April, 2019. [Online] Available: https://techcommunity.microsoft.com/t5/ai-customer-engineering-team/introducing-azure-anomaly-detector-api/ba-p/490162 [Accessed: Feb. 22, 2023].
Microsoft Corporation, “Azure cognitive services – overview,” [Online] Available: https://azure.microsoft.com/en-us/explore/global-infrastructure/products-by-region/?products=cognitive-services®ions=all [Accessed: Mar. 5, 2023].
T. Xing. “Introducing multivariate anomaly detection,” April, 2021. [Online] Available: https://techcommunity.microsoft.com/t5/ai-cognitive-services-blog/introducing-multivariate-anomaly-detection/ba-p/2260679 [Accessed: Feb. 22, 2023].
P. Veličković, G. Cucurull, A. Casanova, A. Romero, P. Liò, Y. Bengio, “Graph attention networks,” February, 2018. [Online] Available: https://arxiv.org/abs/1710.10903 [Accessed: Mar. 27, 2023].
Iconiq Inc., “Anomaly detection using machine learning in python,” February, 2023. [Online] Available: https://www.projectpro.io/article/anomaly-detection-using-machine-learning-in-python-with-example/555 [Accessed: Feb. 23, 2023].
A. Graß, C. Beecks, J. A. C. Soto, “Unsupervised anomaly detection in production lines,” In J. Beyerer, C. Kühnert, O. Niggemann (Eds.), “Machine learning for cyber physical systems”, Springer Berlin Heidelberg, 2019, pp. 18–25.
J. A. Knott, G. C. Liknes, C. L. Giebink, S. Oh, G. M. Domke, R. E. McRoberts, V. F. Quirino, B. F. Walters, “Effects of outliers on remote sensing-assisted forest biomass estimation: A case study from the United States national forest inventory,” March, 2023. [Online] Available: https://besjournals.onlinelibrary.wiley.com/doi/10.1111/2041-210X.14084 [Accessed: Feb. 23, 2023].
A. Naib, “Anomaly Detection on Google Stock Data 2014 – 2022,” February, 2023. [Online] Available: https://www.analyticsvidhya.com/blog/2023/02/anomaly-detection-on-google-stock-data-2014-2022/ [Accessed: Mar. 5, 2023].
A. Naib, “Complete guide on How to learn Scikit-Learn for Data Science,” August, 2021. [Online] Available: https://www.analyticsvidhya.com/blog/2021/08/complete-guide-on-how-to-learn-scikit-learn-for-data-science/ [Accessed: Mar. 5, 2023].
