Generic placeholder image

Recent Patents on Engineering

Author(s): Xu Liu, Hanhao Zhu*, Weihua Song, Jiahui Wang, Zhigang Chai and Shaohua Hong

DOI: 10.2174/0118722121257145230927041949

DownloadDownload PDF Flyer Cite As
Review of Object Detection Algorithms for Sonar Images based on Deep Learning

Article ID: e101023221955 Pages: 11

  • * (Excluding Mailing and Handling)

Explore Articles
About Journal
For Authors
For Editors
For Reviewers

Abstract

Background: Deep learning object detection algorithm is widely used in the field of image classification and has become an indispensable part. With the improvement of image classification accuracy, sonar image target detection algorithm based on deep learning has gradually become the focus of more and more people's research.

Objective: This article aims to provide a summary and analysis of deep learning-based sonar image object detection algorithms, with the hope of offering insights for future research in the field of sonar target detection technology.

Method: This patent paper systematically summarizes sonar image target detection algorithms based on deep learning. According to the method principle, the existing deep learning target detection algorithms are divided into four categories: Target detection algorithm based on candidate region, deep target detection method based on regression, Anchor Free deep learning target detection algorithm, and search-based target detection and recognition algorithm. Then, the performance of algorithms based on COCO data sets is compared, and the standard sonar data sets and formats are introduced.

Results: The sonar image object detection algorithm based on deep learning has made significant progress. The combination of deep learning and object detection methods has been applied to sonar images, resulting in the emergence of excellent performing algorithms. However, most algorithms are still in the developmental stage and face challenges in practical applications. Subsequently, several invention patents have been developed based on the aforementioned algorithms, including a feature extraction method for side-scan sonar images based on fully convolutional neural networks, an underwater sonar image target detection method based on improved YOLOv3-tiny, and more.

Conclusion: Sonar image object detection technology based on deep learning has a wide range of application needs but also faces many difficulties and challenges, we still need to continue to learn and explore in future research, and we believe that we can make greater breakthroughs in the future.

Keywords: Sonar images, object detection, deep learning, object detection algorithm, Sonar image dataset, artificial intelligence.

Graphical Abstract

[1]
G.J. Dobeck, "Algorithm fusion for the detection and classification of sea mines in the very shallow water region using side-scan sonar imagery", Int. Society for Optics & Photonics, 2000.
[http://dx.doi.org/10.1117/12.396262]
[2]
P.K. Lehardy, and C. Moore, Deep ocean search for Malaysia airlines flight 370.Proceedings of 2014 Ocean St.John’s., IEEE: St. John’s, NL, Canada, 2014, pp. 1-4.
[http://dx.doi.org/10.1109/OCEANS.2014.7003292]
[3]
G.E. Hinton, and R.R. Salakhutdinov, "Reducing the dimensionality of data with neural networks", Science, vol. 313, no. 5786, pp. 504-507, 2006.
[http://dx.doi.org/10.1126/science.1127647]
[4]
J. Ngiam, A. Khosla, M. Kim, J. Nam, H. Lee, and A.Y. Ng, Multimodal deep learning Proceedings of the 28th International Conference on Machine Learning (CML-11), 2011, pp. 689-696.
[5]
C. Tan, "Mathematical model construction of teaching evaluation in colleges and universities based on convolutional neural network under the background of big data", J. Funct. Spaces, vol. 2022, pp. 1-8, 2022.
[http://dx.doi.org/10.1155/2022/7064287]
[6]
M.Z. Ma, Research on underwater target recognition technology., Harbin Engineering University, 2007.
[7]
G.Y. Liu, Research on object recognition technology based on sonar image., Harbin Engineering University, 2009.
[8]
N. Hurtos, N. Palomeras, S. Nagappa, and J. Salvi, "Automatic detection ofunderwater chain links using a forwardlooking sonar", In: OCEANS-Bergen, 2013 MTS/ IEEE., 2013.
[9]
V. Myers, and J. Fawcett, "A template matching procedure for automatic target recognition in synthetic aperture sonar imagery", IEEE Signal Process. Lett., vol. 17, no. 7, pp. 683-686, 2021.
[10]
Q. Chen, Research on underwater target recognition technology., Harbin Engineering University, 2013.
[11]
E. Dura, Y. Zhang, X. Liao, G.J. Dobeck, and L. Carin, "Active leaming for detection of mine-like objects in side-scan sonar imagery", IEEE J. Oceanic Eng., vol. 30, no. 2, pp. 360-371, 2005.
[http://dx.doi.org/10.1109/JOE.2005.850931]
[12]
J. Tian, Target recognition and ship radiation noise recognition in hydroacoustic imaging., Institute of Acoustics, Chinese Academy of Sciences, 2004.
[13]
D.P. Williams, "Fast target detection in synthetic aperture sonar imagery: A new algorithm and large-scale performance analysis", IEEE J. Oceanic Eng., vol. 40, no. 1, pp. 71-92, 2015.
[http://dx.doi.org/10.1109/JOE.2013.2294532]
[14]
M. Gao, Research on feature extraction technology of underwater acoustic images., Harbin Engineering University, 2009.
[15]
J. Groena, E. Coirasa, and D. Williamsa, "Detection rate statistics in synthetic aperture sonarimages", 3rd International Conference & Exhibition on "Underwater Acoustic Measurements: Technologies & Results, 2009.
[16]
D. Liu, Object detection and tracking based on multi-resolution processing of sonar images., Harbin Engineering University, 2011.
[17]
M. Valdenegro-Toro, Objectness Scoring and Detection Proposals in Forward-Looking Sonar Images with Convolutional Neural Networks.Artificial Neural Networks in Pattern Recognition. ANNPR 2016. Lecture Notes in Computer Science, vol. 9896. Springer: Cham, 2016.
[http://dx.doi.org/10.1007/978-3-319-46182-3_18]
[18]
M. Valdenegro-Toro, "End-to-end object detection and recognition in forward-looking sonar images with convolutional neural networks", In: Autonomous Underwater Vehicles, 2016.
[http://dx.doi.org/10.1109/AUV.2016.7778662]
[19]
M. Valdenegro-Toro, "Best practices in convolutional networks for forward-looking sonar image recognition", In: OCEANS 2017 - Aberdeen., 2017.
[http://dx.doi.org/10.1109/OCEANSE.2017.8084987]
[20]
J. Kim, H. Cho, J. Pyo, and B. Kim, "The convolution neural network based agentvehicle detection using forwardlooking sonar mage", In: OCEANS 2016 MTS/IEEE Monterey. 2016, 2016, pp. 1-5.
[21]
J. Kim, and S.C. Yu, Convolutional neural network-based real-time ROV detection using forward-looking sonar image.In: 2016 IEEE/OES Autonomous Underwater Vehicles., AUV, 2016, pp. 396-400.
[http://dx.doi.org/10.1109/AUV.2016.7778702]
[22]
W. Hongjian, G. Na, C. Tao, X. Yao, R. Li, and L Benyin, A feature extraction method for sidescan sonar images based on fully convolutional neural networksHeilongjiang Province: CN110781924B February 14, .
[23]
W. Xingmei, J. Jia, S. Boxuan, W. Guoqiang, and L. Anhua, Adaptive Weight Convolutional Neural Network-based Classification Method for Underwater Sonar Images Using Deep LearningHeilongjiang: CN108427958A, August 21, .
[24]
Q. Ye, H. Huang, and C. Zhang, "Image enhancement using stochastic resonance [sonar image processing applications", In 2004 International Conference on Image Processing, 2004, pp. 263-266
[25]
R. Girshick, J. Donahue, T. Darrell, and J. Malik, "Rich feature hierarchies for accurate object detection and semantic segmentation", Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 580-587, 2014.
[http://dx.doi.org/10.1109/CVPR.2014.81]
[26]
J.R.R. Uijlings, K.E.A. van de Sande, T. Gevers, and A.W.M. Smeulders, "Selective search for object recognition", Int. J. Comput. Vis., vol. 104, no. 2, pp. 154-171, 2013.
[http://dx.doi.org/10.1007/s11263-013-0620-5]
[27]
N. Bodla, B. Singh, R. Chellappa, and L.S. Davis, "Soft-NMS--improving object detection with one line of code", Proceedings of the IEEE international conference on computer vision, pp. 5561-5569, 2017.
[http://dx.doi.org/10.1109/ICCV.2017.593]
[28]
R. Girshick, "Fast R-CNN", IEEE Conference on Computer Vision and Pattern Recognition, IEEE, pp. 1440-1448, 2015.
[29]
J. Sun, K. He, R. Girshick, and S. Ren, "Faster r-cnn: Towards realtime object detection with region proposal networks", Adv. Neural Inf. Process. Syst., pp. 91-99, 2015.
[30]
K. He, G. Gkioxari, P. Dollár, and R. Girshick, "Mask R-CNN", In IEEE International Conference on Computer Vision, 2017, pp. 2980-2988
[31]
T.Y. Lin, P. Dollar, R. Girshick, K. He, B. Hariharan, and S. Belongie, "Feature pyramid networks for object detection", Proceedings of the IEEE conference on computer vision and pattern recognition, 2017, pp. 2117-2125.
[32]
Z. Cai, and N. Vasconcelos, "Cascade r-cnn: Delving into high quality object detection", Proceedings of the IEEE conference on computer vision and pattern recognition, 2018, pp. 6154-6162.
[http://dx.doi.org/10.1109/CVPR.2018.00644]
[33]
Y. Li, Y.N. Chen, N. Wang, and Z. Zhang, "Scale-aware trident networks for object detection", Proceedings of the IEEE international conference on computer wision, 2019, pp. 6054-6063.
[34]
J. Redmon, S. Diwala, R. Girshick, and A. Farhadi, "You only look once: Unified, real-time object detection", Proceedings of the IEEE conference on computer vision and pattern recognition, 2016, pp. 779-788.
[http://dx.doi.org/10.1109/CVPR.2016.91]
[35]
J. Redmon, and A. Farhadi, "YOL09000: Better, faster, stronger", In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 7263-7271
[36]
J. Redmon, and A. Farhadi, "YOLOv3: An incremental improvement", arXiv, 2018.
[37]
Y. Huizhen, Z. Yujia, and L. Yuan, "Underwater Sonar Image Object Detection Method based on Improved YOLOv3-tiny", Shaanxi Province: CN112861919A, May 5.
[38]
A. Bochkovskiy, C.Y. Wang, and H.Y.M. Liao, "YOLOv4: Optimal speed and accuracy of object detection", arXiv, 2020.
[39]
" W. Liu, D. Anguelov, D. Erhan, C. Szegedy, S. Reed, C.Y. Fu, and A.C. Berg, "SSD: Single shot multibox detector", European conference on computer vision., pp. 21-37, 2016.Springer, Cham",
[http://dx.doi.org/10.1007/978-3-319-46448-0_2]
[40]
C.Y. Fu, W. Liu, A. Ranga, A. Tyagi, and A.C. Berg, "Dssd: Deconvolutional single shot detector", arXiv, 2017.
[41]
Z. Shen, Z. Liu, J. Li, Y.G. Jiang, and X. Xue, "Dsod: Learning deeply supervised object detectors from scratch", Proceedings of the IEEE international conference on computer vision, 2017, pp. 1919-1927.
[http://dx.doi.org/10.1109/ICCV.2017.212]
[42]
Z. Li, and F. Zhou, "FSSD: Feature fusion single shot multibox detector", arXiv , 2017.
[43]
J. Jeong, H. Park, and N. Kwak, "Enhancement of SSD by concatenating feature maps for object detection", arXiv, 2017.
[http://dx.doi.org/10.5244/C.31.76]
[44]
S. Zhang, L. Wen, X. Bian, Z. Lei, and S.Z. Li, "Single-shot refinement neural network for object detection", Proceedings of the IEEE conference on computer vision and pattern recognition, 2018, pp. 4203-4212.
[http://dx.doi.org/10.1109/CVPR.2018.00442]
[45]
L. Huang, Y. Yang, Y. Deng, and Y. Yu, "Densebox: Unifying landmark localization with end to end object detection", arXiv, 2015.
[46]
J. Yu, Y. Jiang, Z. Wang, Z. Cao, and T. Huang, "Unitbox: An advanced object detection network", Proceedings of the 24th ACM international conference on Multimedia, 2016, pp. 516-520.
[http://dx.doi.org/10.1145/2964284.2967274]
[47]
H. Law, and J. Deng, "Cornernet: Detecting objects as paired keypoints", Proceedings of the European Conference on Computer Vision (ECCV), 2018, pp. 734-750.
[48]
X. Zhou, D. Wang, and P. Krhenbühl, "Objects as points", arXiv , 2019.
[49]
X. Zhou, J. Zhuo, and P. Krahenbuhl, "Bottom-up object detection by grouping extreme and center points", Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2019, pp. 850-859.
[http://dx.doi.org/10.1109/CVPR.2019.00094]
[50]
C. Zhu, Y. He, and M. Savvides, "Feature selective anchor-free module for single-shot object detection", Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2019, pp. 840-849.
[http://dx.doi.org/10.1109/CVPR.2019.00093]
[51]
Z. Tian, C. Shen, H. Chen, and T. He, "Fcos: Fully convolutional one-stage object detection", Proceedings of the IEEE international conference on computer vision, 2019, pp. 9627-9636.
[52]
T. Kong, F. Sun, H. Liu, Y. Jiang, and J. Shi, "Foveabox: Beyound anchor-based object detection", IEEE Trans. Image Process., pp. 7389-7398, 2022.
[53]
N. Carion, F. Massa, G. Synnaeve, N. Usunier, A. Kirillov, and S. Zagoruyko, "End-to-End Object Detection with Transformers", arXiv .
[http://dx.doi.org/10.1007/978-3-030-58452-8_13]
[54]
D. Yoo, S. Park, J.Y. Lee, A.S. Paek, and I.S. Kweon, AttentionNet: Aggregating Weak Directions for Accurate Object Detection., IEEE, 2016.
[55]
J.W. Li, C.W. Qu, J.Q. Shao, and S.J. Peng, "Deep Learning-based ship detection data set and performance analysis of SAR images", In Proceedings of the Fifth Annual Symposium on High Resolution Earth Observation, 2018.
[56]
P. Xiang, W.W. Guo, Z.H. Zhang, W.X. Yu, P. Xiang, W.W. Guo, Z.H. Zhang, and W.X. Yu, "Opensar data sharing platform for sar interpretation", Inf. Tecnol., no. September, pp. 1-4, 2016.
[57]
X. Sun, Z.R. Wang, Y.R. Sun, W.H. Diao, Y. Zhang, and K. Fu, "Air-sarship-1.0: High-Resolution sarship detection Data set", J. Radar, no. August, pp. 852-862, 2019.
[58]
Y. Zhou, S.C. Chen, K. Wu, M.Q. Ning, H.K. Chen, and P. Zhang, "SCTD1.0: Sonar common target detection data set., vol. 48", Comput. Sci., 2021.
[59]
K. Xie, J. Yang, and K. Qiu, "A dataset with multibeam forward-looking sonar for underwater object detection", Sci. Data, vol. 9, no. 1, p. 739, 2022.
[http://dx.doi.org/10.1038/s41597-022-01854-w] [PMID: 36456623]
[60]
"Triton Imaging Inc, eXtended Triton Format (XTF)", Rev., vol. 31, pp. 9-45, 2011.
[61]
"Edge Tech, ", Inc.QMIPS File Format-Sonar User's Manual, vol. 2, pp. 9-3, 1999.