Learning visual similarity for image retrieval with global descriptors and capsule networks

Abstract

Finding matching images across large and unstructured datasets plays an im-portant role in many computer vision applications. With the emergence of deep learning-based solutions, various visual tasks such as image retrieval have been successfully addressed. Learning visual similarity is crucial for image matching and retrieval tasks. An alternative deep learning architecture, named capsule networks, enables learning richer information that describes the object without losing the essential spatial relationship between the object and its parts. Besides, global descriptors are widely used for representing images. The proposed architecture combines the power of global descriptors and revised capsule networks to enhance image retrieval performance. It benefits from multi-ple views of object images and highlights the spatial relationship between objects and their parts. Spatial Grouping Enhance strategy, which enhances sub-features parallelly, and self-attention layers, which explore global dependencies within in-ternal representations of images, are utilized to empower the image representa-tions. The approach captures resemblances between similar images and di˙er-ences between the non-similar images using both triplet loss and cost-sensitive regularized cross-entropy loss instead of learning classification for individual im-ages. Based on the experiments, the results are superior to the state-of-the-art approaches for Stanford Online Products.