Fire Detection Based on Extraction of Spatio-Temporal Features by Convolutional Neural Networks and Fractal Analysis

Subject Areas : Renewable energy

Monir  Torabian ¹ (Department of Electrical Engineering, Najafabad Branch, Islamic Azad University, Najafabad, Iran)
Hossein  Pourghassem ² (Digital Processing and Machine Vision Research Center- Najafabad Branch, Islamic Azad University, Najafabad, Iran)
Homayoun  Mahdavi-Nasab ³ (Smart Microgrid Research Center- Najafabad Branch, Islamic Azad University, Najafabad, Iran)
Payam  Sanaee ⁴ (Department of Electrical Engineering, Najafabad Branch, Islamic Azad University, Najafabad, Iran)

Received: 2022-05-13 Accepted : 2022-10-16 Published : 2025-02-19

Keywords: Fractal, Fire Detection, Multiscale Convolutional Neural Network, YOLO Network, Blanket,

Abstract :

Fire is one of the dangers that can endanger human health in a short time and if it is not controlled in time, it will cause a lot of damage. Therefore, timely and accurate identification of the location of the fire can prevent the consequences of its expansion. In this research, a new method for fire detection is proposed based on the extraction of its temporal-spatial features in video frames. In the proposed algorithm, a multiscale convolutional neural network along with a YOLO (you only look once) network is used to extract spatial features and identify fire candidate regions. Then, fractal analysis based on the temporal blanket method is then used to remove non-moving textures similar to fire and to examine the temporal features of the candidate region. Finally, the fire region is separated from the other parts of the image by fusion the results of the two steps. The evaluation results of the proposed method on three data sets show that the accuracy of fire detection is about 96.1%, while the precision and recall values are 92% and 96.9%, respectively. Experimental results show that the proposed method performs better than existing algorithms and thus confirms the ability of this method for efficient use in the real world.

References:

[1] W.S. Qureshi, M. Ekpanyapong, M.N. Dailey, S. Rinsurongkawong, A. Malenichev, O.Krasotkina, "QuickBlaze: Early fire detection using combined video processing approach", Fire Technology, vol. 52, no. 5, pp. 1293–1317, Sept. 2016 (doi: 10.1007/s10694-015-0489-7).

[2] Z. Teng, J.H. Kim, D.J. Kang, "Fire detection base on hidden markov models", International Journal of Control, Automation and Systems, vol. 8, no. 4, pp. 822–830, Aug. 2010 (doi:10.1007/s12555-010-0414-2).

[3] Th. H. Chen, P. H. Wu, Y. Ch. Chiou, "An early fire-detection method based on image processing", Proceeding of the IEEE/ICIP, vol. 3, pp.1707-1710, Singapore, Oct. 2004 (doi: 10.1109/ICIP.2004.1421401). 

[4] Y. H. Habiboǧlu, O. Günay, A. E. Çetin, "Covariance matrix-based fire and flame detection method in video", Machine Vision and Application, vol. 23, no. 6, pp. 1103–1113, Nov. 2012 (doi:10.1007/s00138-011-0369-1).

[5] S.G. Kong, D. Jin, Sh. Li, H. Kim, "Fast fire flame detection in surveillance video using logistic regression and temporal smoothing", Fire Safety Journal, vol. 79, pp. 37-43, Jan. 2016 (doi: 10.1016/j.firesaf.2015.11.015).

[6] J. Seo, M. Kang, Ch. H. Kim, J. M. Kim, "An optimal many-core model-based supercomputing for accelerating video-equipped fire detection", The Journal of Supercomputing, vol. 71, no. 6, pp. 2275–2308, June 2015 (doi: 10.1007/s11227-015-1382-3).

[7] P. Foggia, A. Saggese, M. Vento, "Real-time Fire Detection for Video Surveillance Applications using a Combination of Experts based on Color, Shape and Motion", IEEE Trans. on Circuits and Systems for Video Technology, vol. 25, no. 9, pp. 1545 – 1556, Sept. 2015 (doi: 10.1109/TCSVT.2015.2392531).

[8] W. B. Horng, J. W. Peng, Ch. Y. Chen, "A new image-based real-time flame detection method using color analysis", Proceedings of the IEEE/ICNSC, USA, March 2005 (doi: 10.1109/ICNSC.2005.1461169).

[9] B. Töreyin, Y. Dedeoǧlu, U. Güdükbay and A. E. Çetin, "Computer vision based method for real-time fire and flame detection", Pattern Recognition Letters, vol. 27, no.1, pp. 49-58, Jan. 2006 (doi: 10.1016/j.patrec.2005.06.015).

[10] Y.R. Pu, Y.J. Chen, S.H. Lee, "Fire recognition based on correlation of segmentations by image processing techniques", Machine Vision and Application, vol. 26, no. 7–8, pp. 849–856, Nov. 2015 (doi: 10.1007/s00138-015-0698-6).

[11] S. Verstockt, S. V. Hoecke, T. Beji, B. Merci, B. Gouverneur, A. E. Cetin, P. DePotter, R. V. Walle, "A multi-modal video analysis approach for car park fire detection", Fire Safety Journal, vol. 57, pp. 44-57, Apr. 2013 (doi:10.1016/j.firesaf.2012.07.005).

[12] T. X. Truong, J. M. Kim, "Fire flame detection in video sequences using multi-stage pattern recognition techniques", Engineering Application of Artificial Intelligence, vol. 25, no. 7, pp. 1365-1372, Oct. 2012 (doi: 10.1016/j.engappai.2012.05.007).

[13] F. Yuan, "An integrated fire detection and suppression system based on widely available video surveillance", Machine Vision and Application, vol. 21, no. 6, pp. 941–948, Oct. 2010 (doi: 10.1007/s00138-010-0276-x).

[14] M. Torabian, H. Pourghassem, "Dynamic-based fire detection using fusion of markov random field and PCA optical flow", Proceeding of the IEEE/ICEE, Iran, Apr. 2019 (doi: 10.1109/IranianCEE.2019.8786704).

[15] F. Sthevanie, H. Nugroho, F.A. Yulianto, "Visual-based fire detection using local binary pattern-three orthogonal planes", Proceeding of the IEEE/CYBERNETICSCOM, pp. 155-159, Indonesia, Dec. 2013 (doi: 10.1109/CyberneticsCom.2013.6865801).

[16] K. Muhammad, J. Ahmad, Z. Lv, P. Bellavista, P. Yang, S. W. Baik, "Efficient deep CNN-based fire detection and localization in video surveillance applications", IEEE Trans. on Systems, Man, and Cybernetics: Systems, vol. 49, no. 7, pp. 1419–1434, July 2019 (doi: 10.1109/TSMC.2018.2830099).

[17] K. Muhammad, J. Ahmad, I. Mehmood, S. Rho, B.S. Wook, "Convolutional neural networks based fire detection in surveillance videos", IEEE Access, vol. 6, pp.18174-18183. March 2018 (doi: 10.1109/AC­CESS.2­018.2812835).

[18] K. Muhammad, J. Ahmad, S.W. Baik, "Early fire detection using convolutional neural networks during surveillance for effective disaster management", Neurocomputing, vol. 288, pp. 30–42, May 2018 (doi: 10.1016/j.neucom.2017.04.083).

[19] Y. Xie, J. Zhu, Y. Cao, Y. Zhang, D. Feng, Y. Zhang, M. Chen, "Efficient video fire detection exploiting motion-flicker-based dynamic features and deep static features", IEEE Access, vol. 8, pp. 81904–81917,  April 2020 (doi: 10.1109/ACCESS.2020.2991338).

[20] C. Chaoxia, W. Shang, F. Zhang, "Information-guided flame detection based on faster R-CNN", IEEE Access, vol. 8, pp. 58923-58932, April 2020 (doi: 10.1109/ACCESS.2020.2982994).

[21] S. Saponara, A. Elhanashi, A. Gagliardi, "Real‑time video fire/smoke detection based on CNN in antifire surveillance systems", Journal of Real-Time Image Processing, vol. 18, pp. 889–900, Nov. 2021 (doi: 10.1007/s11554-020-01044-0).

[22] Y. Xu, H. Ji, C. Fermüller, "Viewpoint invariant texture description using fractal analysis", International of  Journal Computer Vision, vol. 83, no. 1, pp. 85-100, June 2009 (doi: 10.1007/s11263-009-0220-6).

[23] S. Peleg, J. Naor, R. Hartley, D. Avnir, "Multiple resolution texture analysis and classification", IEEE Trans. on Pattern Analysis and Machine Intelligence, vol. 6, no. 4 ,pp. 518-523, July 1984 (doi: 10.1109/TPAMI.1984.4767557).

[24] B. Ch. Ko, K. H. Cheong, J. Y. Nam, "Early fire detection algorithm based on irregular patterns of flames and hierarchical Bayesian Networks", Fire Safety Journal, vol. 45, no. 4, pp. 262-270, June 2010 (doi: 10.1016/j.fir­esaf.2010.04.001).

[25] P. Li, W. Zhao, "Image fire detection algorithms based on convolutional neural networks", Case Studies in Thermal Engineernig, vol. 19, Article Number: 100625, June 2020 (doi: 10.1016/j.csite.2020.100625).

[26] K. Dimitropoulos, P. Barmpoutis, N. Grammalidis, "Spatio-temporal flame modeling and dynamic texture analysis for automatic video-based fire detection", IEEE Trans. on Circuits and Systems for Video Technology, vol. 25, no. 2, pp. 339–351, Feb. 2015 (doi: 10.1109/TCSVT.2014.2339592).

[27] K. Muhammad, M. Sajjad, M. Y. Lee, S. W. Baik, "Efficient visual attention driven framework for key frames extraction from hysteroscopy videos", Biomedical Signal Processing and Control, vol. 33, pp. 161-168, Mar. 2017 (doi: 10.1016/j.bspc.2016.11.011).

[28] T. Çelik, H. Demirel , "Fire detection in video sequence using a generic color model", Fire Safety Journal, vol. 44, no. 2, pp. 147-158, Feb. 2009 (doi: 10.1016/j.firesaf.2008.05.005).

[29] R. D. Lascio, A. Greco, A. Saggese, M. Vento, "Improving fire detection reliability by a combination of video analytics", Peoceeding of the ICIAR, vol. 8814, pp. 477-484, Oct. 2014 (doi: 10.1007/978-3-319-11758-4_52).

[30] Y. Wua, Y. He, P. Shivakumara, Z. Li, H. Guo, Tong Lu, "Channel-wise attention model-based fire and rating level detection in video", CAAI Transaction on Intelligence Technology, vol. 4, no. 2, pp. 1-9, April 2019 (doi: 10.1049/trit.2019.0022).

[31] A. Ra_ee, R. Dianat, M. Jamshidi, R. Tavakoli, S. Abbaspour, "Fire and smoke detection using wavelet analysis and disorder characteristics", Proceeding of the IEEE/ICCRD, pp. 262_265, China, Mar. 2011 (doi: 10.1109/ICCRD.2011.5764295).

[32] S. Rudz, K. Chetehouna, A. Hafiane, H. Laurent, O. Séro-Guillaume, "Investigation of a novel image segmentation method dedicated to forest fire applications", Measurement Science and Technology,  vol. 24, no. 7, Jun. 2013 (doi: 10.1088/0957-0233/24/7/075403).

[33] S. Li, Q. Yan, P. Liu, "An efficient fire detection method based on multiscale feature extraction, implicit deep supervision and channel attention mechanism", IEEE Trans. on Image Processing, vol. 29, pp. 8467-8475, Aug. 2020 (doi: 10.1109/TIP.2020.3016431).

[34] B. Kim, J. Lee, "A video-based fire detection using deep learning models", Applied Sciences, vol. 9, no. 14, July 2019 (doi: 10.3390/app9142862).

[35] J. Zhang, H. Zhu, P. Wang, X. Linga, "Lightweight network for forest fire detection and recognition", IEEE Access, vol. 9, pp. 10858 – 10870, Jan. 2021 (doi: 10.1109/ACCESS.2021.3050628).