Authors

Helan Sunomitha M

Department of Electronics and Communication, Don Bosco Institute of Technology, Bangalore, India

Abstract

This research introduces a new hybrid ensemble framework that combines an Isolation Forest for swift outlier scoring, an LSTM autoencoder for reconstructing temporal–frequency sequences, and an ADWIN-based drift detector for adaptive windowing. Real-time anomaly detection is crucial for quickly recognizing unexpected patterns in high-velocity data streams across various fields such as cybersecurity, industrial monitoring, and finance. A Kafka-based ingestion pipeline mimics streaming conditions on benchmark datasets—including the Numenta Anomaly Benchmark (NAB) and KDD Cup 1999 network flows—allowing for evaluation under different anomaly rates and feature dimensionalities. Statistical, temporal, and frequency-domain features are extracted for each time window and projected into a lower-dimensional subspace using incremental PCA. The ensemble merges submodel scores with dynamically adaptive weights to ensure high precision and recall in the presence of concept drift. Experimental results show that the proposed method achieves an average F₁-score of 93.0%—an improvement of about 7% and 6% over static Isolation Forest and LSTM baselines, respectively—while maintaining sub-second detection delays on both server (28.4 ms) and edge (112.1 ms) platforms. Resource profiling reveals efficient CPU usage (

Keywords

real-time anomaly detection; streaming data; ensemble learning; concept drift adaptation; edge deployment; Isolation Forest; LSTM autoencoder; ADWIN

Citation of this Article

Helan Sunomitha M. (2025). An Ensemble Framework that is Adaptive for Detecting Anomalies in Real-Time Streaming Data. Current Journal of Engineering and Science Research. 2(12), 8-14. Article DOI: https://doi.org/10.47001/CJESR/2025.212002

Licence Copyright (c) 2026 Current Journal of Engineering and Science Research. This work is licensed under a Creative Commons Attribution Non Commercial 4.0 International Licence.

References

  1. Gao, J., Saligrama, V., & Vishwanathan, S. (2020). TadGAN: Time-series anomaly detection using generative adversarial networks. In Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining (pp. 1507–1517). ACM. https://doi.org/10.1145/3394486.3403146
  2. Xu, H., Gao, J., Zhao, L., & Li, J. (2018). Real-time anomaly detection in IoT data streams. IEEE Internet of Things Journal, 5(5), 3863–3875. https://doi.org/10.1109/JIOT.2018.2833884
  3. Zimek, A., & Filzmoser, P. (2020). There and back again: Outlier research and practice, past–present–future. ACM Computing Surveys, 53(3), Article 58. https://doi.org/10.1145/3371166
  4. You, J., Yan, K., & Kim, H. (2021). Adaptive windowing for real-time anomaly detection. Information Sciences, 546, 524–539. https://doi.org/10.1016/j.ins.2020.11.019
  5. Chandola, V., Banerjee, A., & Kumar, V. (2009). Anomaly detection: A survey. ACM Computing Surveys, 41(3), 1–58. https://doi.org/10.1145/1541880.1541882
  6. Ahmed, M., Mahmood, A. N., & Hu, J. (2016). A survey of network anomaly detection techniques. Journal of Network and Computer Applications, 60, 19–31. https://doi.org/10.1016/j.jnca.2015.11.016
  7. Lakhina, A., Crovella, M., & Diot, C. (2004). Diagnosing network-wide traffic anomalies. ACM SIGCOMM Computer Communication Review, 34(4), 219–230. https://doi.org/10.1145/1030194.1015487
  8. Liu, F. T., Ting, K. M., & Zhou, Z.-H. (2008). Isolation forest. In Proceedings of the 2008 Eighth IEEE International Conference on Data Mining (pp. 413–422). IEEE. https://doi.org/10.1109/ICDM.2008.17
  9. Ringberg, H., Soule, A., Rexford, J., & Diot, C. (2007). Sensitivity of PCA for traffic anomaly detection. In Proceedings of the 2007 ACM SIGMETRICS International Conference on Measurement and Modeling of Computer Systems (pp. 109–120). ACM. https://doi.org/10.1145/1254912.1254925
  10. Hundman, K., Constantinou, V., Laporte, C., Colwell, I., & Soderstrom, T. (2018). Detecting spacecraft anomalies using LSTM networks. In Proceedings of the 2018 International Conference on Space Mission Challenges for Information Technology (pp. 1–9). IEEE. https://doi.org/10.1109/SMC-IT.2018.00007
  11. Aggarwal, C. C. (2015). Outlier Analysis (2nd ed.). Springer. https://doi.org/10.1007/978-3-319-47578-3
  12. Erfani, S. M., Rajasegarar, S., Karunasekera, S., & Leckie, C. (2016). High-dimensional and large-scale anomaly detection using a linear one-class SVM with deep learning. Pattern Recognition, 58, 121–134. https://doi.org/10.1016/j.patcog.2016.03.028
  13. Guha, S., Mishra, N., Motwani, R., & O’Callaghan, L. (2016). Robust real-time change detection in data streams. IEEE Transactions on Knowledge and Data Engineering, 28(2), 397–409. https://doi.org/10.1109/TKDE.2015.2462243
  14. Ren, J., Xu, S., Liao, X., Hao, S., & Liu, Z. (2019). Real-time anomaly detection for streaming data using predictive models. IEEE Access, 7, 104495–104507. https://doi.org/10.1109/ACCESS.2019.293753