A geo-computational workflow for automatically classifying urban land cover using deep learning and street view imagery

Abstract

Due to urbanization, there is a growing demand for knowledge on urban patterns and their dynamics at multiple spatial scales. Reliable information on urban land cover is increasingly important when dealing with this growing demand. The existing methods, such as analysis of remotely sensed imagery, do not help to accurately classify urban land cover. The aim of this research is to accurately classify urban land cover of the Netherlands. This is done by developing a novel geo-computational workflow that is able to automatically classify urban land cover on a large scale using high-resolution street view imagery and deep learning. In addition, this research looks at what the key parameters and pre-processing strategies are for training a deep learning model for classifying urban land cover. Furthermore, the performance difference between a pre-trained and newly trained deep learning model for classifying urban land cover were compared and analyzed. Finally, the effect of the image distance on the performance of urban land cover classification was investigated. For the development of the workflow, street view images of the Dutch city Bergen op Zoom were used. In order to achieve this, a script to automatically extract street view images of the image capture locations was developed. Next, the extracted street view images were cropped and an urban land cover label was assigned. The labelled images were used to train a convolutional neural network to recognize urban land cover. Subsequently, the best performing models were used to classify urban land cover in a real-world application. The urban land cover classifications as found in the Dutch land cover map Basisregistratie Grootschalige Topografie, were used as a reference source. The results of this research showed that the larger the images dataset, the better the performance of the deep learning model. The performance of the trained models was measured in terms of accuracy, recall, precision and F1-score. The application of the workflow resulted in an overall accuracy of 52%. After applying data augmentation techniques, this accuracy increased to 54%. In addition, comparing the predictions for different image distances resulted in the highest accuracy of 70% for images within a range of three-meters from an image capture location. The performances achieved by the developed geo-computational workflow appeared to be not sufficient enough for urban land cover classification in a real-world application. This can be explained by the fact that the model was only trained on land cover located on a three-meter distance. In conclusion, future research should focus on improving the model that is developed in this study by increasing the number of training images for all used distances. This improvement could result in a higher performance of urban land cover classification and the workflow could be applied to the whole of the Netherlands.