Semantic and goal-oriented signal processing: semantic extraction

Abstract

Advances in machine learning technology have enabled real-time extraction of semantic information in signals, which has the potential to revolutionize signal processing techniques and drastically improve their performance for next-generation applications. A graph-based semantic language and a goal-oriented semantic signal processing framework are adopted for structured and universal representation and efficient processing of semantic information. In the adopted framework, preprocessing of input signals is followed by a semantic extractor which identifies components from a set of application-specific predefined classes where the states, actions, and relations among the identified components are described by another application-specific predefined set called predicates. For additional information, the resulting semantic graph is also embedded with a hierarchical set of attributes. In this thesis, we focus on the crucial semantic extractor block, and to illustrate the proposed framework’s applicability, we present a real-time computer vision application on video-stream data where we adopt a tracking by detection paradigm for the identification of semantic components. Next, we show that with the adopted semantic representation and goal-filtering, the semantic signal processing framework can achieve an extremely high reduction in data rates compared to traditional approaches. Finally, we demonstrate a way to identify points of significant innovation over extended periods of time by tracking the evolution of multi-level attributes and discussing future research directions.