Structural and metrical information in linear systems

Abstract

We present a systematic approach to understand the information-theoretic re- lationships in linear systems. Our main aim is to understand what kind of information the output of a linear system carries about the input of the system and how much of this information is preserved in the measurement process. We recognize structural and metrical information as two fundamental concepts for classifying the information content of signals. We base our understanding of the problem on information-theoretic concepts like entropy, mutual information and channel capacity. We present our results as trade-o®s between cost and perfor- mance, yielding insights about di®erent aspects of the information °ow in a linear system. We especially focus on building a framework which indicates how accu- rately, and how many measurements must be made and how the measurement locations should be selected.