Analysis of speech content and voice for deceit detection

Abstract

Deceptive behavior is part of daily life, often without being recognized, leading to severe repercussions. With the recent improvements in machine learning, more reliable detection of deceit appears to be possible. Although current visual and multimodal models can identify deception with adequate precision, the individual use of speech content or voice still performs poorly. Therefore, we systematically analyze such essential communication forms focusing on feature extraction and optimization for deceit detection. To this end, we assess the reliability of employing transformers, spatial and temporal architectures, state-of-the-art pre-trained models, and handcrafted representations to detect deceit patterns. Furthermore, we conduct a thorough analysis to comprehend the distinct properties and discriminative power of the evaluated methods. The results demonstrate that speech content (transcribed text) provides more information than vocal characteristics. In addition, transformer architectures are found to be effective in representation learning and modeling, providing insights into optimal model configurations for deceit detection.