Generic placeholder image

Recent Patents on Engineering

Author(s): Richa Singh, Nidhi Srivastava and Ashwani Kumar*

DOI: 10.2174/0118722121242429230922070752

DownloadDownload PDF Flyer Cite As
Network Anomaly Detection using Autoencoder on Various Datasets: A Comprehensive Review

Article ID: e061023221861 Pages: 15

  • * (Excluding Mailing and Handling)

Abstract

The scientific community is currently very concerned about information and communication technology security because any assault or network anomaly can have a remarkable collision on a number of areas, including national security, the storage of private data, social welfare, economic concerns, and more. As a result, many strategies and approaches for this goal have been developed over time, making the anomaly detection domain a large research subject. The primary concern of this patent study is to review the most crucial elements relating to anomaly detection, including an overview of background analysis and a core study on the most important approaches, procedures, and systems in the field. To make the structure of this survey easier to understand, the domain of anomaly detection was examined along with five dimensions: Detection methods in network traffic, objectives of the patent paper, various datasets used, accuracy, and open issues/gaps. The gap which has been identified after the survey can be extended as a future scope might be helpful for the researcher.

Graphical Abstract

[1]
N.T. Van, L.T. Sach, and T.N. Thinh, "Temporal features learning using autoencoder for anomaly detection in network traffic", In: International conference on green technology and sustainable development, Springer: Cham, 2020, pp. 15-26.
[2]
V. Dutta, M. Pawlicki, R. Kozik, and M. Choraś, "Unsupervised network traffic anomaly detection with deep autoencoders", Log. J. IGPL., vol. 30, no. 6, pp. 912-925, 2022.
[http://dx.doi.org/10.1093/jigpal/jzac002]
[3]
P. Liu, X. Sun, Y. Han, Z. He, W. Zhang, and C. Wu, "Arrhythmia classification of LSTM autoencoder based on time series anomaly detection", Biomed. Signal Process. Control, vol. 71, p. 103228, 2022.
[http://dx.doi.org/10.1016/j.bspc.2021.103228]
[4]
Y. Wei, J. Jang-Jaccard, W. Xu, F. Sabrina, S. Camtepe, and M. Boulic, "LSTM-Autoencoder based anomaly detection for indoor air quality time series data", arXiv:2204.06701., 2022.
[5]
K. Patra, R.N. Sethi, and D.K. Behera, "Anomaly detection in rotating machinery using autoencoders based on bidirectional LSTM and GRU neural networks", Turk. J. Electr. Eng. Comput. Sci., vol. 30, no. 4, pp. 1637-1653, 2022.
[http://dx.doi.org/10.55730/1300-0632.3870]
[6]
X. Du, D. Liu, S. Ding, Z. Liu, X. Yuan, T. Li, and H. Deng, "Design of an autoencoder -based anomaly detection for the doh traffic system", In In 2022 IEEE 25th International Conference on Computer Supported Cooperative Work in Design, 04-06 May, Hangzhou, China, 2022, pp. 763-768
[7]
W. Xu, J. Jang-Jaccard, A. Singh, Y. Wei, and F. Sabrina, "Improving performance of autoencoder-based network anomaly detection on nsl-kdd dataset", IEEE Access, vol. 9, pp. 140136-140146, 2021.
[http://dx.doi.org/10.1109/ACCESS.2021.3116612]
[8]
D. Yang, and M. Hwang, "Unsupervised and ensemble-based anomaly detection method for network security", In 14th International Conference on Knowledge and Smart Technology (KST), 26-29 January, Chon buri, Thailand, 2022, pp. 75-79
[http://dx.doi.org/10.1109/KST53302.2022.9729061]
[9]
Y. Song, S. Hyun, and Y.G. Cheong, "Analysis of autoencoders for network intrusion detection", Sensors, vol. 21, no. 13, p. 4294, 2021.
[http://dx.doi.org/10.3390/s21134294] [PMID: 34201798]
[10]
J.E.D. Albuquerque Filho, L.C.P. Brandão, B.J.T. Fernandes, and A.M.A. Maciel, "A review of neural networks for anomaly detection", IEEE Access, vol. 10, pp. 112342-112367, 2022.
[http://dx.doi.org/10.1109/ACCESS.2022.3216007]
[11]
V. Chandola, A. Banerjee, and V. Kumar, "Anomaly detection: Asurvey", ACM Comput. Surv., vol. 41, no. 3, pp. 1-58, 2009.
[12]
N. Görnitz, M. Kloft, K. Rieck, and U. Brefeld, "Toward supervised anomaly detection", J. Artif. Intell. Res., vol. 46, p. 235262, 2013.
[13]
O. Chapelle, B. Scholkopf, and A. Zien, Semisupervised learning (Chapelle, O. et al., Eds.; 2006) [Book reviews]. IEEE Trans. Neural Netw, vol. 20, no. 3, pp. 542-542, year. 2009.
[14]
T. Schlegl, P. Seeböck, S.M. Waldstein, U. SchmidtErfurth, and G. Langs, "Unsupervised anomaly detection with generative adversarial networks to guide marker discovery", In: Proc. Int. Conf. Inf. Process. Med. Imag., Cham, Switzerland, Springer, 2017, p. 146157.
[15]
L. Yang, Y. Song, S. Gao, A. Hu, and B. Xiao, "Griffin: Real-time network intrusion detection system via ensemble of autoencoder in SDN", IEEE Trans. Netw. Serv. Manag., vol. 19, no. 3, pp. 2269-2281, 2022.
[http://dx.doi.org/10.1109/TNSM.2022.3175710]
[16]
Q.P. Nguyen, K.W. Lim, D.M. Divakaran, K.H. Low, and M.C. Chan, "GEE: A Gradient-based Explainable Variational Autoencoder for Network Anomaly Detection", In 2019 IEEE Conference on Communications and Network Security (CNS), 10-12 June, Washington, DC, USA, 2019, pp. 91-99
[http://dx.doi.org/10.1109/CNS.2019.8802833]
[17]
M. Said Elsayed, N.A. Le-Khac, S. Dev, and A.D. Jurcut, "Network anomaly detection using LSTM based autoencoder", Proceedings of the 16th ACM Symposium on QoS and Security for Wireless and Mobile Networks, pp. 37-45, 2020.
[http://dx.doi.org/10.1145/3416013.3426457]
[18]
B. Min, J. Yoo, S. Kim, D. Shin, and D. Shin, "Network anomaly detection using memory-augmented deep autoencoder", IEEE Access, vol. 9, pp. 104695-104706, 2021.
[http://dx.doi.org/10.1109/ACCESS.2021.3100087]
[19]
W.T. Lunardi, M.A. Lopez, and J.P. Giacalone, "ARCADE: Adversarially regularized convolutional autoencoder for network anomaly detection", arXiv:2205.01432., 2022.
[20]
A. Chohra, P. Shirani, E.B. Karbab, and M. Debbabi, "Chameleon: Optimized feature selection using particle swarm optimization and ensemble methods for network anomaly detection", Comput. Secur., vol. 117, p. 102684, 2022.
[http://dx.doi.org/10.1016/j.cose.2022.102684]
[21]
W. Ullah, T. Hussain, Z.A. Khan, U. Haroon, and S.W. Baik, "Intelligent dual stream CNN and echo state network for anomaly detection", Knowl. Base. Syst., vol. 253, p. 109456, 2022.
[http://dx.doi.org/10.1016/j.knosys.2022.109456]
[22]
Y. Yin, J. Jang-Jaccard, F. Sabrina, and J. Kwak, "Improving Multilayer-Perceptron (MLP)-based Network Anomaly Detection with Birch Clustering on CICIDS-2017 Dataset", arXiv:2208.09711., 2022.
[23]
A. Singh, and J. Jang-Jaccard, "Autoencoderbased unsupervised intrusion detection using multiscale convolutional recurrent networks", arXiv: 2204.03779, 2022.
[24]
X. Liang, Y. Gao, and S. Xu, "ASE: Anomaly scoring based ensemble learning for imbalanced datasets", arXiv:2203.10769., 2022.
[25]
X. Gao, C. Shan, C. Hu, Z. Niu, and Z. Liu, "An adaptive ensemble machine learning model for intrusion detection", IEEE Access, vol. 7, pp. 82512-82521, 2019.
[http://dx.doi.org/10.1109/ACCESS.2019.2923640]
[26]
M. Rashid, J. Kamruzzaman, T. Imam, S. Wibowo, and S. Gordon, "A tree-based stacking ensemble technique with feature selection for network intrusion detection", Appl. Intell., vol. 52, no. 9, pp. 9768-9781, 2022.
[http://dx.doi.org/10.1007/s10489-021-02968-1]
[27]
J.H. Moon, J.H. Yu, and K.A. Sohn, "An ensemble approach to anomaly detection using high- and low-variance principal components", Comput. Electr. Eng., vol. 99, p. 107773, 2022.
[http://dx.doi.org/10.1016/j.compeleceng.2022.107773]
[28]
S.Y. Wang, "Ensemble2: Anomaly Detection via EVT-Ensemble Framework for Seasonal KPIs in Communication Network", arXiv:2205.14305., 2022.
[29]
M. Sabih, and D.K. Vishwakarma, "A novel framework for detection of motion and appearance-based Anomaly using ensemble learning and LSTMs", Expert Syst. Appl., vol. 192, p. 116394, 2022.
[http://dx.doi.org/10.1016/j.eswa.2021.116394]
[30]
I.S. Thaseen, A.K. Chitturi, F. Al-Turjman, A. Shankar, M.R. Ghalib, and K. Abhishek, "An intelligent ensemble of long ‐short ‐term memory with genetic algorithm for network anomaly identification", Trans. Emerg. Telecommun. Technol., vol. 33, no. 10, p. e4149, 2022.
[http://dx.doi.org/10.1002/ett.4149]
[31]
R. Singh, A. Singh, and P. Bhattacharya, "A machine learning approach for anomaly detection to secure smart grid systems", In: Research Anthology on Smart Grid and Microgrid Development., IGI global, 2022, pp. 911-923.
[http://dx.doi.org/10.4018/978-1-6684-3666-0.ch040]
[32]
R. Singh, N. Srivastava, and A. Kumar, "Machine Learning Techniques for Anomaly Detection in Network Traffic", In 2021 Sixth International Conference on Image Information Processing (ICIIP), 26-28 November, Shimla, India, 2021, pp. 261-266
[http://dx.doi.org/10.1109/ICIIP53038.2021.9702647]
[33]
S. Roy, J. Li, B.J. Choi, and Y. Bai, "A lightweight supervised intrusion detection mechanism for IoT networks", Future Gener. Comput. Syst., vol. 127, pp. 276-285, 2022.
[http://dx.doi.org/10.1016/j.future.2021.09.027]
[34]
A. Jones, Z. Kong, and C. Belta, "Anomaly detection in cyber-physical systems: A formal methods approach", In 53rd IEEE Conference on Decision and Control, 15-17 December, Los Angeles, CA, USA, 2014, pp. 848-853
[http://dx.doi.org/10.1109/CDC.2014.7039487]
[35]
B. Wang, Q. Hua, H. Zhang, X. Tan, Y. Nan, R. Chen, and X. Shu, "Research on anomaly detection and real-time reliability evaluation with the log of cloud platform", Alex. Eng. J., vol. 61, no. 9, pp. 7183-7193, 2022.
[http://dx.doi.org/10.1016/j.aej.2021.12.061]
[36]
J.R. Jiang, and Y.T. Chen, "Industrial control system anomaly detection and classification based on network traffic", IEEE Access, vol. 10, pp. 41874-41888, 2022.
[http://dx.doi.org/10.1109/ACCESS.2022.3167814]
[37]
J. Kong, W. Kowalczyk, S. Menzel, and T. Bäck, "Improving imbalanced classification by anomaly detection", In 16th International Conference, PPSN 2020, September 5-9, Leiden, The Netherlands, Cham, 2020, pp. 512-523
[38]
M. Ghurab, G. Gaphari, F. Alshami, R. Alshamy, and S. Othman, "A detailed analysis of benchmark datasets for network intrusion detection system", Asian. J. Res. Comp. Sci., vol. 7, no. 4, pp. 14-33, 2021.
[http://dx.doi.org/10.9734/ajrcos/2021/v7i430185]
[39]
A. Özgür, and H. Erdem, "“A review of KDD99 dataset usage in intrusion detection and machine learning between 2010 and 2015”, PeerJ", Preprints, vol. 4, 1954.
[40]
N. Chandolikar, and V. Nandavadekar, "Efficient algorithm for intrusion attack classification by analyzing KDD Cup 99", In 2012 Ninth International Conference on Wireless and Optical Communications Networks (WOCN), 20-22 September, Indore, India, 2012, pp. 1-5
[http://dx.doi.org/10.1109/WOCN.2012.6335546]
[41]
P. Kushwaha, H. Buckchash, and B. Raman, "Anomaly based intrusion detection using filter based feature selection on KDDCUP99", In TENCON 2017-2017 IEEE Region 10 Conference., 05-08 November, Penang, Malaysia, 2017, pp. 839-844
[http://dx.doi.org/10.1109/TENCON.2017.8227975]
[42]
J. McHugh, "Testing Intrusion detection systems", ACM Trans. Inf. Syst. Secur., vol. 3, no. 4, pp. 262-294, 2000.
[http://dx.doi.org/10.1145/382912.382923]
[43]
P. Verma, A. Shayan, K. Shadab, and M. Sunil B, "Network intrusion detection using clustering and gradient boosting", In 2018 9th International Conference on Computing, Communication and Networking Technologies (ICCCNT), 2018 10-12 July, Bengaluru, India, 2018, pp. 1-7
[http://dx.doi.org/10.1109/ICCCNT.2018.8494186]
[44]
S. Revathi, and A. Malathi, "A detailed analysis on NSL-KDD dataset using various machine learning techniques for intrusion detection", Int. J. Eng. Res. Technol., vol. 2, no. 12, p. 18481853, 2013.
[45]
S. Singh Panwar, Y. Raiwani, and L.S. Panwar, "Evaluation of network intrusion detection with features selection and machine learning algorithms on CICIDS-2017 dataset", In International Conference on Advances in Engineering Science Management & Technology (ICAESMT), Uttaranchal University, Dehradun, India, 2019.
[http://dx.doi.org/10.2139/ssrn.3394103]
[46]
A. Roy, and K.J. Singh, "Multi-classification of UNSWNB15 dataset for network anomaly detection system", In: Proceedings of International Conference on Communication and Computational Technologies, Springer., 2021, pp. 429-451.
[47]
A. Shiravi, Toward developing a systematic approach to generate benchmark datasets for intrusion detection computers & security., vol. 31, no. 3, pp. 357-374, 2012.
[48]
L. Nicholas, S.Y. Ooi, Y.H. Pang, S.O. Hwang, and S-Y. Tan, "Study of long short-term memory in flow-based network intrusion detection system", J. Intell. Fuzzy Syst., vol. 35, no. 6, pp. 5947-5957, 2018.
[http://dx.doi.org/10.3233/JIFS-169836]
[49]
R. Singh, "Performance Optimization of Autoencoder Neural Network Based Model for Anomaly Detection in Network Traffic", In 2nd International Conference on Advance Computing and Innovative Technologies in Engineering (ICACITE), 2022 28-29 April, Noida, India, 2022, pp. 598-602
[http://dx.doi.org/10.1109/ICACITE53722.2022.9823785]
[50]
R. Singh, and G. Srivastav, "Novel framework for anomaly detection using machine learning technique on cic-ids2017 dataset", In 2021 International Conference on Technological Advancements and Innovations (ICTAI), 10-12 November,Tashkent, Uzbekistan, 2021, pp. 632-636
[http://dx.doi.org/10.1109/ICTAI53825.2021.9673238]
[51]
"Network Intrusion Detection, ISG group @UNSW Canberra", Available From: https://www.kaggle.com/datasets/dhoogla/unswnb15
[52]
R. Singh, "Performance Optimization of Autoencoder Neural Network Based Model for Anomaly Detection in Network Traffic", In International Conference on Technological, 28-29 April,Noida, India, 2022, pp. 598-602
[http://dx.doi.org/10.1109/ICACITE53722.2022.9823785]
[53]
R. Singh, and G. Srivastav, "Novel Framework for Anomaly Detection Using Machine Learning Technique on CIC-IDS2017 Dataset", In In 2021 International Conference on Technological Advancements and Innovations. 10-12 Nov., 2021.