Browsing by Subject "Information Theory"

Now showing 1 - 3 of 3

Open Access
Joint source channel coding using sequential decoding
(1997) Doğrusöz, Bekir Ahmet
In systems using conventional source encoding, source sequence is changed into a series of approximately independent equally likely binary digits. Performance of a code is bounded with the rate distortion function and improves as the redundancy of the encoder output is decreased. However decreasing the redundancy implies increasing the block length and hence the complexity. For the systems requiring low complexity at transmitter, joint source channel (JSC) coding can be successfully used for direct encoding of source into the channel for lossless recovery. In such a system, without any distortion, compression depends on the redundancy of the source, and is bounded by the Renyi entropy of the source. In this thesis we analyze transmission of English text with a JSC coding system. Written English is a good example for sources with natural redundancy. Since we are unable to calculate the Renyi entropy of written English, we obtain estimates and compare with the experimental results. We also work on an alternative source encoding method for accuracycompression trade-off in joint source channel coding systems. The proposed stochastic distortion encoder (SDE) is capable of achieving accuracycompression trade-off at any average distortion constraint with very low block lengths, and hence performs better than or as good as an equivalent rate distortion encoder. As block length approaches infinity the performance of stochastic distortion encoder approaches rate distortion function. Formulations for optimal SDE design and results for block lengths 1,2 and 3 are also given.
Open Access
Signaling games in networked systems
(2018-07) Sarıtaş, Serkan
We investigate decentralized quadratic cheap talk and signaling game problems when the decision makers (an encoder and a decoder) have misaligned objective functions. We first extend the classical results of Crawford and Sobel on cheap talk to multi-dimensional sources and noisy channel setups, as well as to dynamic (multi-stage) settings. Under each setup, we investigate the equilibria of both Nash (simultaneous-move) and Stackelberg (leader-follower) games. We show that for scalar cheap talk, the quantized nature of Nash equilibrium policies holds for arbitrary sources; whereas Nash equilibria may be of non-quantized nature, and even linear for multi-dimensional setups. All Stackelberg equilibria policies are fully informative, unlike the Nash setup. For noisy signaling games, a Gauss-Markov source is to be transmitted over a memoryless additive Gaussian channel. Here, conditions for the existence of a ne equilibria, as well as informative equilibria are presented, and a dynamic programming formulation is obtained for linear equilibria. For all setups, conditions under which equilibria are noninformative are derived through information theoretic bounds. We then provide a different construction for signaling games in view of the presence of inconsistent priors among multiple decision makers, where we focus on binary signaling problems. Here, equilibria are analyzed, a characterization on when informative equilibria exist, and robustness and continuity properties to misalignment are presented under Nash and Stackelberg criteria. Lastly, we provide an analysis on the number of bins at equilibria for the quadratic cheap talk problem under the Gaussian and exponential source assumptions. Our findings reveal drastic differences in signaling behavior under team and game setups and yield a comprehensive analysis on the value of information; i.e., for the decision makers, whether there is an incentive for information hiding, or not, which have practical consequences in networked control applications. Furthermore, we provide conditions on when a ne policies may be optimal in decentralized multi-criteria control problems and for the presence of active information transmission even in strategic environments. The results also highlight that even when the decision makers have the same objective, presence of inconsistent priors among the decision makers may lead to a lack of robustness in equilibrium behavior.
Open Access
Uncertain linear equations
(2010) Pilancı, Mert
In this thesis, new theoretical and practical results on linear equations with various types of uncertainties and their applications are presented. In the first part, the case in which there are more equations than unknowns (overdetermined case) is considered. A novel approach is proposed to provide robust and accurate estimates of the solution of the linear equations when both the measurement vector and the coefficient matrix are subject to uncertainty. A new analytic formulation is developed in terms of the gradient flow to analyze and provide estimates to the solution. The presented analysis enables us to study and compare existing methods in literature. We derive theoretical bounds for the performance of our estimator and show that if the signal-to-noise ratio is low than a treshold, a significant improvement is made compared to the conventional estimator. Numerical results in applications such as blind identification, multiple frequency estimation and deconvolution show that the proposed technique outperforms alternative methods in mean-squared error for a significant range of signal-to-noise ratio values. The second type of uncertainty analyzed in the overdetermined case is where uncertainty is sparse in some basis. We show that this type of uncertainty on the coefficient matrix can be recovered exactly for a large class of structures, if we have sufficiently many equations. We propose and solve an optimization criterion and its convex relaxation to recover the uncertainty and the solution to the linear system. We derive sufficiency conditions for exact and stable recovery. Then we demonstrate with numerical examples that the proposed method is able to recover unknowns exactly with high probability. The performance of the proposed technique is compared in estimation and tracking of sparse multipath wireless channels. The second part of the thesis deals with the case where there are more unknowns than equations (underdetermined case). We extend the theory of polarization of Arikan for random variables with continuous distributions. We show that the Hadamard Transform and the Discrete Fourier Transform, polarizes the information content of independent identically distributed copies of compressible random variables, where compressibility is measured by Shannon’s differential entropy. Using these results we show that, the solution of the linear system can be recovered even if there are more unknowns than equations if the number of equations is sufficient to capture the entropy of the uncertainty. This approach is applied to sampling compressible signals below the Nyquist rate and coined ”Polar Sampling”. This result generalizes and unifies the sparse recovery theory of Compressed Sensing by extending it to general low entropy signals with an information theoretical analysis. We demonstrate the effectiveness of Polar Sampling approach on a numerical sub-Nyquist sampling example.