Comparing the performance of humans and 3D-convolutional neural networks in material perception using dynamic cues

Abstract

There are numerous studies on material perception in humans. Similarly, there are various deep neural network models that are trained to perform different visual tasks such as object recognition. However, the intersection of material perception in humans and deep neural network models has not been investigated to our knowledge. Especially, the importance of the ability of deep neural networks in categorizing materials and also comparing human performance with the performance of deep convolutional neural networks has not been appreciated enough. Here we have built, trained and tested a 3D-convolutional neural network model that is able to categorize the animations of simulated materials. We have compared the performance of the deep neural network with that of humans and concluded that the conventional training of deep neural networks is not necessarily giving the optimal state of the network to be compared to the performance of the humans. In the material categorization task, the similarity between the performance of humans and deep neural networks increases and reaches the maximum similarity and then decreases as we train the network further. Also, by training the 3D-CNN on regular, temporally consistent animations and also training it on the temporally inconsistent animations and comparing the results we found out that the 3D-CNN model can use spatial information in order to categorize the material animations. In other words, we found out that the temporal, and consistent motion information is not necessary for the deep neural networks in order to categorize the material animations.