Generic placeholder image

Recent Advances in Computer Science and Communications

Author(s): Souvik Datta and Subbulekshmi Durairaj*

DOI: 10.2174/0126662558275210231121044758

DownloadDownload PDF Flyer Cite As
Review of Deep Learning Algorithms for Urban Remote Sensing Using Unmanned Aerial Vehicles (UAVs)

Article ID: e081223224285 Pages: 12

  • * (Excluding Mailing and Handling)

Abstract

This study conducts a comprehensive review of Deep Learning-based approaches for accurate object segmentation and detection in high-resolution imagery captured by Unmanned Aerial Vehicles (UAVs). The methodology employs three different existing algorithms tailored to detect roads, buildings, trees, and water bodies. These algorithms include Res-UNet for roads and buildings, DeepForest for trees, and WaterDetect for water bodies. To evaluate the effectiveness of this approach, the performance of each algorithm is compared with state-of-the-art (SOTA) models for each class. The results of the study demonstrate that the methodology outperforms SOTA models in all three classes, achieving an accuracy of 93% for roads and buildings using Res-U-Net, 95% for trees using DeepForest, and an impressive 98% for water bodies using Water Detect. The paper utilizes a Deep Learning-based approach for accurate object segmentation and detection in high-resolution UAV imagery, achieving superior performance to SOTA models, with reduced overfitting and faster training by employing three smaller models for each task.

Graphical Abstract

[1]
S. Fennell, P. Kaur, A. Jhunjhunwala, D. Narayanan, C. Loyola, J. Bedi, and Y. Singh, "Examining linkages between Smart Villages and Smart Cities: Learning from rural youth accessing the internet in India", Telecomm. Policy, vol. 42, no. 10, pp. 810-823, 2018.
[http://dx.doi.org/10.1016/j.telpol.2018.06.002]
[2]
B. Bansod, R. Singh, R. Thakur, and G. Singhal, "A comparision between satellite based and drone based remote sensing technology to achieve sustainable development: A review", J. Agric. Environ. Int. Dev., vol. 111, no. 2, pp. 383-407, 2017.
[3]
T. Hoeser, and C. Kuenzer, "Object detection and image segmentation with deep learning on earth observation data: A review-part I: Evolution and recent trends", Remote Sens. (Basel), vol. 12, no. 10, p. 1667, 2020.
[http://dx.doi.org/10.3390/rs12101667]
[4]
F. Isikdogan, A.C. Bovik, and P. Passalacqua, "Surface water mapping by Deep Learning", IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens., vol. 10, no. 11, pp. 4909-4918, 2017.
[http://dx.doi.org/10.1109/JSTARS.2017.2735443]
[5]
M. Salahuddin, "Dubai International Financial Center: 5+Design", Dubai International Financial Center: 5+Design. Available from: https://www.5plusdesign.com/master-planning/dubai-internationalfinancial-centre(Accessed on: Jul. 29, 2023).
[6]
W. Boonpook, Y. Tan, and B. Xu, "Deep learning-based multi-feature semantic segmentation in building extraction from images of UAV photogrammetry", Int. J. Remote Sens., vol. 42, no. 1, pp. 1-19, 2021.
[http://dx.doi.org/10.1080/01431161.2020.1788742]
[7]
Y. Dai, J. Gong, Y. Li, and Q. Feng, "Building segmentation and outline extraction from UAV image-derived point clouds by a line growing algorithm", Int. J. Digit. Earth, vol. 10, no. 11, pp. 1077-1097, 2017.
[http://dx.doi.org/10.1080/17538947.2016.1269841]
[8]
J. Anand, India has to invest $55 billion P.A. in urban infra to meet needs of growing population: World Bank Report. Available from: https://www.thehindu.com/news/national/india-needs-toinvest-55-billion-pa-in-urban-infra-to-effectively-meet-needs-of-fast-growing-urban-population-wb-report/article66135032.ece
[9]
R. Jha, "The Bengaluru floods: The rising challenge of urban floods in India", Available from: https://www.orfonline.org/expert-speak/the-bengaluru-floods/
[10]
" "India has to invest $55 billion p.a. in urban infra to meet needs of growing population: World Bank report", Available from:", https://timesofindia.indiatimes.com/mumbai-mangroves-are-stablebut-creeks-are-shrinking/articleshow/90330846.cms (Accessed on: Jul. 29, 2023).
[11]
M. Haigh, and J.S. Rawat, Landslide disasters: Seeking causes – a case study from Uttarakhand, India.Management of Mountain Watersheds., Springer, 2012, pp. 218-253.
[http://dx.doi.org/10.1007/978-94-007-2476-1_18]
[12]
""UAS Traffic Management",", Available from: https://onboard.thalesgroup.com/uas-traffic-management/ (Accessed on: Jul. 29, 2023).
[13]
L.F. Isikdogan, A. Bovik, and P. Passalacqua, "Seeing through the clouds with deepwatermap", IEEE Geosci. Remote Sens. Lett., vol. 17, no. 10, pp. 1662-1666, 2020.
[http://dx.doi.org/10.1109/LGRS.2019.2953261]
[14]
S. Takemoto, Moving towards climate-smart flood management in Bangkok and Tokyo, 2011.
[15]
""Review of the civil defence emergency management response to the 22 February christchurch earthquake",", Available from: https://www.civildefence.govt.nz/resources/review-of-the-civil-defence-emergency-management-response-to-the-22-february-christchurch-earthquake/(Accessed on: Jul. 29, 2023).
[16]
A. Krizhevsky, I. Sutskever, and E.H. Geoffrey, "Imagenet classification with deep convolutional neural networks", Adv. Neural Inf. Process. Syst., vol. 25, pp. 1097-1105, 2012.
[17]
K. Simonyan, and A. Zisserman, "Very deep convolutional networks for large-scale image recognition", arXiv:1409.1556, 2014.,
[18]
R. Girshick, J. Donahue, T. Darrell, and J. Malk, "Rich feature hierarchies for accurate object detection and semantic segmentation", In 2014 IEEE Conference on Computer Vision and Pattern Recognition 2014 23-28 June 2014, Columbus, OH, USA,
[http://dx.doi.org/10.1109/CVPR.2014.81]
[19]
R. Girshick, "Fast R-CNN Proc. IEEE Int. Conf. Comput. Vis., pp. 1440-1448, 2015",
[20]
S. Ren, K. He, R. Girshick, and J. Sun, "Faster R-CNN: Towards real-time object detection with region proposal networks", Adv. Neural Inf. Process. Syst., pp. 91-99, 2015.
[21]
J. Long, E. Shelhamer, and T. Darrell, "Fully convolutional networks for semantic segmentation", 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2015.07-12 June 2015, Boston, MA, USA,
[http://dx.doi.org/10.1109/CVPR.2015.7298965]
[22]
F. Yu, and V. Koltun, "Multi-scale context aggregation by dilated convolutions", arXiv:1511.07122, 2015.,
[23]
"Mumbai: Mangroves are stable-but creeks are shrinking: Mumbai News - Times of India",", Available from: https://timesofindia.indiatimes.com/mumbai-mangroves-are-stablebut-creeks-are-shrinking/articleshow/90330846.cms(Accessed on: Jul. 29, 2023).
[24]
K. He, G. Gkioxari, P. Dollár, and R. Girshick, "Mask R-CNN", In 2017 IEEE International Conference on Computer Vision (ICCV) 2017, pp. 2980-2988, Venice, Italy,
[http://dx.doi.org/10.1109/ICCV.2017.322]
[25]
Y. Hou, Z. Liu, T. Zhang, and Y. Li, "C-UNet: Complement UNet for remote sensing road extraction", Sensors, vol. 21, no. 6, p. 2153, 2021.
[http://dx.doi.org/10.3390/s21062153] [PMID: 33808588]
[26]
A. Yavariabdi, H. Kusetogullari, and H. Cicek, "UAV detection in airborne optic videos using dilated convolutions", J. Opt., vol. 50, no. 4, pp. 569-582, 2021.
[http://dx.doi.org/10.1007/s12596-021-00770-3]
[27]
A. Yavariabdi, H. Kusetogullari, T. Celik, and H. Cicek, "FASTUAV-net: A Multi-UAV detection algorithm for embedded platforms", Electronics, vol. 10, no. 6, p. 724, 2021.
[http://dx.doi.org/10.3390/electronics10060724]
[28]
X. Wu, W. Li, D. Hong, R. Tao, and Q. Du, "Deep learning for unmanned aerial vehicle-based object detection and tracking: A survey", IEEE Geosci. Remote Sens. Mag., vol. 10, no. 1, pp. 91-124, 2022.
[http://dx.doi.org/10.1109/MGRS.2021.3115137]
[29]
H. Yao, R. Qin, and X. Chen, "Unmanned aerial vehicle for Remote Sensing Applications—a review", Remote Sens., vol. 11, no. 12, p. 1443, 2019.
[http://dx.doi.org/10.3390/rs11121443]
[30]
A. Ramachandran, and A.K. Sangaiah, "A review on object detection in unmanned aerial vehicle surveillance", Int. J. Cogn. Comput. Eng., vol. 2, pp. 215-228, 2021.
[http://dx.doi.org/10.1016/j.ijcce.2021.11.005]
[31]
E.V. Butilă, and R.G. Boboc, "Urban traffic monitoring and analysis using unmanned aerial vehicles (uavs): A systematic literature review" Remote Sens., vol. 14, no. 3, p. 620, 2022.
[http://dx.doi.org/10.3390/rs14030620]
[32]
B. Peng, Y. Li, L. He, K. Fan, and L. Tong, "Road segmentation of UAV RS image using adversarial network with multi-scale context aggregation IGARSS 2018 - 2018 IEEE International Geoscience and Remote Sensing Symposium, 2018 22-27 July 2018, Valencia, Spain",
[http://dx.doi.org/10.1109/IGARSS.2018.8517641]
[33]
S. Hartling, V. Sagan, and M. Maimaitijiang, "Urban tree species classification using UAV-based multi-sensor data fusion and machine learning", GIsci. Remote Sens., vol. 58, no. 8, pp. 1250-1275, 2021.
[http://dx.doi.org/10.1080/15481603.2021.1974275]
[34]
D. Hernández, J.M. Cecilia, J.C. Cano, and C.T. Calafate, "Flood detection using real-time image segmentation from unmanned aerial vehicles on edge-computing platform", Remote Sens., vol. 14, no. 1, p. 223, 2022.
[http://dx.doi.org/10.3390/rs14010223]
[35]
V. Mnih, Machine Learning for Aerial Image Labeling., Library and Archives Canada: Ottawa, 2014.
[36]
B. Ashwath, Massachusetts roads dataset. Available from:https://www.kaggle.com/datasets/balraj98/massachusetts-roads-dataset
[37]
B.G. Weinstein, S. Marconi, S. Bohlman, A. Zare, and E. White, "Individual tree-crown detection in RGB imagery using semi-supervised Deep Learning Neural Networks", Remote Sens., vol. 11, no. 11, p. 1309, 2019.
[http://dx.doi.org/10.3390/rs11111309]
[38]
Deep Forest, A python package for RGB deep learning- NSF public access. Available from:https://par.nsf.gov/servlets/purl/10293184 Accessed on: Sep. 12, 2023).
[39]
""Sentinel-2 Surface Reflectance"", Available from:https://www.theia-land.fr/en/product/sentinel-2-surface-reflectance/ (Accessed on: Sep. 12, 2023).
[40]
""Open access hub"", Available from: https://scihub.copernicus.eu/ (Accessed on: Sep. 12, 2023).
[41]
O. Hagolle, M. Huc, D.V. Pascual, and G. Dedieu, "A multi-temporal method for cloud detection, applied to FORMOSAT-2, VENµS, LANDSAT and SENTINEL-2 images", Remote Sens. Environ., vol. 114, no. 8, pp. 1747-1755, 2010.
[http://dx.doi.org/10.1016/j.rse.2010.03.002]
[42]
M.C.R. Cordeiro, J.M. Martinez, and S. Peña-Luque, "Automatic water detection from multidimensional hierarchical clustering for Sentinel-2 images and a comparison with Level 2A processors", Remote Sens. Environ., vol. 253, p. 112209, 2021.
[http://dx.doi.org/10.1016/j.rse.2020.112209]
[43]
F.I. Diakogiannis, F. Waldner, P. Caccetta, and C. Wu, "ResUNet-a: A deep learning framework for semantic segmentation of remotely sensed data", ISPRS J. Photogramm. Remote Sens., vol. 162, pp. 94-114, 2020.
[http://dx.doi.org/10.1016/j.isprsjprs.2020.01.013]
[44]
Akash-Ramjyothi Semantic segmentation of remote sensing imagery. Available from: https://github.com/Akash-Ramjyothi/Satellite-Imagery-Road-Extraction(Accessed on: Sep. 12, 2023).
[45]
""Weecology/Deep Forest: Python package for tree crown detection in airborne RGB imagery"", Available from: https://github.com/weecology/DeepForest (Accessed on: Sep. 12, 2023).
[46]
"Cordmaur/WaterDetect: Water detect algorithm",", Available from: https://github.com/cordmaur/WaterDetect (Accessed on: Sep. 12, 2023).
[47]
P. Dialani, Why machine learning models should be smaller in size?. Available from: https://www.analyticsinsight.net/why-machine-learning-models-should-be-smaller-in-size/ (Accessed on: Jul. 29, 2023).
[48]
S. Candiago, F. Remondino, M. De Giglio, M. Dubbini, and M. Gattelli, "Evaluating multispectral images and vegetation indices for precision farming applications from UAV images", Remote Sens., vol. 7, no. 4, pp. 4026-4047, 2015.
[http://dx.doi.org/10.3390/rs70404026]