Comprehensive Assessment of Networks and Datasets Utilized in IDS/IPS Evaluation.
DOI:
https://doi.org/10.33003/fjs-2026-1014-5147Keywords:
Intrusion Detection System (IDS), Intrusion Prevention System (IPS), network datasets, cyber security, network topologies, traffic analysisAbstract
Most modern networks depend on Intrusion Detection Systems (IDS) and Intrusion Prevention Systems (IPS) as core defense system against an increasingly expanding number of cyber threats. This is the reason why research effort over the years has concentrated on improving these systems, while attention is now been directed at the network settings in which they are tested. In this paper, we systematically review the IDS/IPS literature and discuss the evaluation environments and benchmark datasets used to evaluate detection performance, e.g., KDD Cup 99, NSL-KDD, UNSW-NB15, CICIDS2017, and Bot-IoT. The analysis shows that the popular datasets like the KDD Cup 99 are still in widespread use despite their known limitations, such as duplicate records and attack profiles that do not reflect modern threats. Nevertheless, the adoption of other recent datasets such as CICIDS2017 and Bot-IoT that capture more realistic traffic, and include IoT-specific scenarios, is still modest compared to their older counterparts. The review also shows that experiments are mostly based on artificial and not operational traffic and are mostly limited to laboratory and not production settings, thus limiting the external validity of reported results. Taken together, these observations establish a persistent discrepancy between benchmark settings and the realities of operational networks. The paper ends with practical recommendations to assist researchers and practitioners to choose evaluation environments to enhance the realism, dependability and transferability of IDS/IPS solutions.
References
Boyd, D. M., & Ellison, N. B. (2007). Social network sites: Definition, history, and scholarship. Journal of Computer-Mediated Communication, 13(1), 210–230. https://doi.org/10.1111/j.1083-6101.2007.00393.x
Cantone, M., Marrocco, C., & Bria, A. (2024). On the cross-dataset generalization of machine learning for network intrusion detection. IEEE Access, 12, 144489–144508. https://doi.org/10.1109/ACCESS.2024.3472907
Dahlman, E., Parkvall, S., & Sköld, J. (2021). 5G NR: The next generation wireless access technology (2nd ed.). Academic Press.
Edegbe, G. N., & Acheme, S. (2024). A systematic review of centralized and decentralized machine learning models: Security concerns, defenses and future directions. NIPES Journal of Science and Technology Research, 6(4), 161–175. https://doi.org/10.5281/zenodo.14681449
Forouzan, B. A. (2013). Data communications and networking (5th ed.). McGraw-Hill Education.
Goldschmidt, P., & Chudá, D. (2025). Network intrusion datasets: A survey, limitations, and recommendations. Computers & Security, 156, 104510. https://doi.org/10.1016/j.cose.2025.104510
Kaplan, A. M., & Haenlein, M. (2010). Users of the world, unite! The challenges and opportunities of social media. Business Horizons, 53(1), 59–68. https://doi.org/10.1016/j.bushor.2009.09.003
Khraisat, A., Gondal, I., Vamplew, P., & Kamruzzaman, J. (2019). Survey of intrusion detection systems: Techniques, datasets and challenges. Cybersecurity, 2, Article 20. https://doi.org/10.1186/s42400-019-0038-7
Koroniotis, N., Moustafa, N., Sitnikova, E., & Turnbull, B. (2019). Towards the development of realistic botnet dataset in the Internet of Things for network forensic analytics: Bot-IoT dataset. Future Generation Computer Systems, 100, 779–796. https://doi.org/10.1016/j.future.2019.05.041
Kurose, J. F., & Ross, K. W. (2021). Computer networking: A top-down approach (8th ed.). Pearson.
Martins, et al. (2025). Adaptive online learning approaches for detecting unknown attacks in network traffic: A systematic literature review. Security and Privacy.
Moustafa, N., & Slay, J. (2015). UNSW-NB15: A comprehensive data set for network intrusion detection systems (UNSW-NB15 network data set). In 2015 Military Communications and Information Systems Conference (MilCIS) (pp. 1–6). IEEE. https://doi.org/10.1109/MilCIS.2015.7348942
Pinto, D., Amorim, I., Maia, E., & Praça, I. (2025). A review on intrusion detection datasets: Tools, processes, and features. Computer Networks, 262, 111177.
Rahman Tori, A., & Hasan, K. F. (2026). An evaluation framework for network IDS/IPS datasets: Leveraging MITRE ATT&CK and industry relevance metrics. Computers & Security, 161, 104777.
Sharafaldin, I., Lashkari, A. H., & Ghorbani, A. A. (2018). Toward generating a new intrusion detection dataset and intrusion traffic characterization. In Proceedings of the 4th International Conference on Information Systems Security and Privacy (ICISSP 2018) (pp. 108–116). SciTePress. https://doi.org/10.5220/0006639801080116
Stallings, W. (2022). Data and computer communications (11th ed.). Pearson.
Tanenbaum, A. S., & Wetherall, D. J. (2011). Computer networks (5th ed.). Pearson.
Tavallaee, M., Bagheri, E., Lu, W., & Ghorbani, A. A. (2009). A detailed analysis of the KDD CUP 99 data set. In 2009 IEEE Symposium on Computational Intelligence for Security and Defense Applications (pp. 1–6). IEEE. https://doi.org/10.1109/CISDA.2009.5356528
Thirumaran, V., Joseph, N., & Srikanth, G. (2024). Intrusion detection system in cloud computing using VTR-HLSTM-based deep learning. Indonesian Journal of Electrical Engineering and Computer Science, 33(3), 1829–1842. https://doi.org/10.11591/ijeecs.v33.i3.pp1829-1842
Usman, O. L., & Kareem, M. A. (2026). An intrusion detection system based on hybridized firefly and artificial bee colony optimization algorithms. FUDMA Journal of Sciences, 10(ANB-K Special Issue), 40–47. https://doi.org/10.33003/fjs-2026-10(ANB-K)-5306
Downloads
Published
Issue
Section
Categories
License
Copyright (c) 2026 Glory Nosawaru Edegbe, Esosa Enoyoze, Ali Abubakar Musah
This work is licensed under a Creative Commons Attribution 4.0 International License.
