Weakly supervised instance attention for multisource fine-grained object recognition with an application to tree species classification

Simple item page
buir.contributor.authorAygüneş, Bulut
buir.contributor.authorAksoy, Selim
buir.contributor.orcidAygüneş, Bulut|0000-0001-7197-0977
dc.citation.epage274en_US
dc.citation.spage262en_US
dc.citation.volumeNumber176en_US
dc.contributor.authorAygüneş, Bulut
dc.contributor.authorCinbiş, R. G.
dc.contributor.authorAksoy, Selim
dc.date.accessioned2022-02-17T08:05:14Z
dc.date.available2022-02-17T08:05:14Z
dc.date.issued2021-06
dc.departmentDepartment of Computer Engineeringen_US
dc.description.abstractMultisource image analysis that leverages complementary spectral, spatial, and structural information benefits fine-grained object recognition that aims to classify an object into one of many similar subcategories. However, for multisource tasks that involve relatively small objects, even the smallest registration errors can introduce high uncertainty in the classification process. We approach this problem from a weakly supervised learning perspective in which the input images correspond to larger neighborhoods around the expected object locations where an object with a given class label is present in the neighborhood without any knowledge of its exact location. The proposed method uses a single-source deep instance attention model with parallel branches for joint localization and classification of objects, and extends this model into a multisource setting where a refer- ence source that is assumed to have no location uncertainty is used to aid the fusion of multiple sources in four different levels: probability level, logit level, feature level, and pixel level. We show that all levels of fusion provide higher accuracies compared to the state-of-the-art, with the best performing method of feature-level fusion resulting in 53% accuracy for the recognition of 40 different types of trees, corresponding to an improvement of 5.7% over the best performing baseline when RGB, multispectral, and LiDAR data are used. We also provide an in-depth comparison by evaluating each model at various parameter complexity settings, where the increased model capacity results in a further improvement of 6.3% over the default capacity setting.en_US
dc.embargo.release2023-06-30
dc.identifier.doi10.1016/j.isprsjprs.2021.03.021en_US
dc.identifier.issn0924-2716
dc.identifier.urihttp://hdl.handle.net/11693/77451
dc.language.isoEnglishen_US
dc.publisherElsevier BVen_US
dc.relation.isversionofhttps://doi.org/10.1016/j.isprsjprs.2021.03.021en_US
dc.source.titleISPRS Journal of Photogrammetry and Remote Sensingen_US
dc.subjectMultisource classificationen_US
dc.subjectFine-grained object recognitionen_US
dc.subjectWeakly supervised learningen_US
dc.subjectDeep learningen_US
dc.titleWeakly supervised instance attention for multisource fine-grained object recognition with an application to tree species classificationen_US
dc.typeArticleen_US
Files
Original bundle
Now showing 1 - 1 of 1
Thumbnail Image
Name:
Weakly_supervised_instance_attention_for_multisource_fine-grained_object_recognition_with_an_application_to_tree_species_classification.pdf
Size:
2.94 MB
Format:
Adobe Portable Document Format
Description:
Download
License bundle
Now showing 1 - 1 of 1
No Thumbnail Available
Name:
license.txt
Size:
1.69 KB
Format:
Item-specific license agreed upon to submission
Description:
Download
Collections
Department of Computer Engineering