Using convolutional neural networks to identify illegal roofs from unmanned aerial vehicle images

Illegal structures, which are structures illegally built on land or buildings, are common in Taiwan. Urban residents frequently inform authorities of illegal roofs, a type of illegal structure, because of the potential fire hazards they pose. However, the government does not conduct timely inspection on illegal roofs because this process requires additional human resources. Therefore, developing an efficient and correct method for inspecting and reporting illegal structures is necessary. In this study, unmanned aerial vehicles (UAVs) were used to rapidly capture images, which were then used to generate orthophotos, a 3D building model, a digital surface model (DSM), and a data set containing 400 images of illegal roofs and 400 images of legal roofs. The data set was then used in a convolutional neural network (CNN) to train and evaluate image classification. The results revealed an illegal roof classification accuracy of 96.0%, with a loss of 0.09. In addition, You Only Look Once v3 (YOLOv3) was used to detect illegal buildings, and DSMs higher than 9 m were overlaid to improve the accuracy of the illegal roof identification model. Overall, the study results can help inspectors build a comprehensive database of illegal roofs, which can serve as a reference for budgeting demolition costs and human resources.