Optimal stochastic approaches for signal detection and estimation under inequality constraints

Abstract

Fundamental to the study of signal detection and estimation is the design of optimal procedures that operate on the noisy observations of some random phenomenon. For detection problems, the aim is to decide among a number of statistical hypotheses, whereas estimating certain parameters of the statistical model is required in estimation problems. In both cases, the solution depends on some goodness criterion by which detection (or estimation) performance is measured. Despite being a well-established field, the advances over the last several decades in hardware and digital signal processing have fostered a renewed interest in designing optimal procedures that take more into account the practical considerations. For example, in the detection of binary-valued scalar signals corrupted with additive noise, an analysis on the convexity properties of the error probability with respect to the transmit signal power has suggested that the error performance cannot be improved via signal power randomization/sharing under an average transmit power constraint when the noise has a unimodal distribution (such as the Gaussian distribution). On the contrary, it is demonstrated that performance enhancement is possible in the case of multimodal noise distributions and even under Gaussian noise for three or higher dimensional signal constellations. Motivated by these results, in this dissertation we adopt a structured approach built on concepts called stochastic signaling and detector randomization, and devise optimal detection procedures for power constrained communications systems operating over channels with arbitrary noise distributions. First, we study the problem of jointly designing the transmitted signals, decision rules, and detector randomization factors for an M-ary communications system with multiple detectors at the receiver. For each detector employed at the receiver, it is assumed that the transmitter can randomize its signal constellation (i.e., transmitter can employ stochastic signaling) according to some probability density function (PDF) under an average transmit power constraint. We show that stochastic signaling without detector randomization cannot achieve a smaller average probability of error than detector randomization with deterministic signaling for the same average power constraint and noise statistics when optimal maximum a-posteriori probability (MAP) detectors are employed in both cases. Next, we prove that a randomization between at most two MAP detectors corresponding to two deterministic signal vectors results in the optimal performance. Sufficient conditions are also provided to conclude ahead of time whether the correct decision performance can or cannot be improved by detector randomization. In the literature, the discussions on the benefits of stochastic signaling and detector randomization are severely limited to the Bayesian criterion. Therefore, we study the convexity/concavity properties for the problem of detecting the presence of a signal emitted from a power constrained transmitter in the presence of additive Gaussian noise under the Neyman-Pearson (NP) framework. First, it is proved that the detection probability corresponding to the α−level likelihood ratio test (LRT) is either concave or has two inflection points such that the function is concave, convex and finally concave with respect to increasing values of the signal power. Based on this result, optimal and near-optimal power sharing/randomization strategies are proposed for average and/or peak power constrained transmitters. Using a similar approach, the convexity/concavity properties of the detection probability are also investigated with respect to the jammer power. The results indicate that a weak Gaussian jammer should employ on-off time sharing to degrade the detection performance. Next, the previous analysis for the NP criterion is generalized to channels with arbitrary noise PDFs. Specifically, we address the problem of jointly designing the signaling scheme and the decision rule so that the detection probability is maximized under constraints on the average false alarm probability and average transmit power. In the case of a single detector at the receiver, it is shown that the optimal solution can be obtained by employing randomization between at most two signal values for the on-signal and using the corresponding NP-type LRT at the receiver. When multiple detectors are available at the receiver, the optimal solution involves a randomization among no more than three NP decision rules corresponding to three deterministic signal vectors. Up to this point, we have focused on signal detection problems. In the following, the trade-offs between parameter estimation accuracy and measurement device cost are investigateed under the influence of noise. First, we seek to determine the most favorable allocation of the total cost to measurement devices so that the average Fisher information of the resulting measurements is maximized for arbitrary observation and measurement statistics. Based on a recently proposed measurement device cost model, we present a generic optimization problem without assuming any specific estimator structure. Closed form expressions are obtained in the case of Gaussian observations and measurement noise. Finally, a more elaborate analysis of the relationship between parameter estimation accuracy and measurement device cost is presented. More specifically, novel convex measurement cost minimization problems are proposed based on various estimation accuracy constraints assuming a linear system subject to additive Gaussian noise for the deterministic parameter estimation problem. Robust allocation of the total cost to measurement devices is also considered by assuming a specific uncertainty model on the system matrix. Closed form solutions are obtained in the case of an invertible system matrix for two estimation accuracy criteria. Through numerical examples, various aspects of the proposed optimization problems are compared. Lastly, the discussion is extended to the Bayesian framework assuming that the estimated parameter is Gaussian distributed.