Object detection using optical and lidar data fusion with graph-cuts

Abstract

Object detection in remotely sensed data has been a popular problem and is commonly used in a wide range of applications in domains such as agriculture, navigation, environmental management, urban monitoring and mapping. However, using only one type of data source may not be sufficient to solve this problem. Fusion of aerial optical and LiDAR data has been a promising approach in remote sensing as they carry complementary information for object detection. We propose frameworks that partition the data in multiple levels and detect objects with minimal supervision in the partitioned data. Our methodology involves thresholding the data according to height, and dividing the data into smaller components to process it efficiently in the preprocessing step. For the classification task, we propose two graph cut based procedures that detect objects in each component using height information from LiDAR, spectral information from aerial data, and spatial information from adjacency maps. The first procedure provides a binary classification, whereas the second one performs a multi-class classification. We use the first framework to separate buildings from trees in the high pixels, and roads from grass areas in the low pixels. The second procedure is used to detect all of the classes in each component at once. The only supervision our proposed methodology requires consists of samples that are used to estimate the weights of the edges in the graph for the graph-cut procedures. Experiments using a benchmark data set show that the performance of the proposed methodology that uses small amount of supervision is compatible with the ones in the literature.