
HighDimensional Regression with Binary Coefficients. Estimating Squared Error and a Phase Transition
We consider a sparse linear regression model Y=Xβ^*+W where X has a Gaus...
read it

Global testing under the sparse alternatives for single index models
For the single index model y=f(β^τx,ϵ) with Gaussian design, and β is a...
read it

SubGaussian Error Bounds for Hypothesis Testing
We interpret likelihoodbased test functions from a geometric perspectiv...
read it

On model misspecification and KL separation for Gaussian graphical models
We establish bounds on the KL divergence between two multivariate Gaussi...
read it

Testing correlation of unlabeled random graphs
We study the problem of detecting the edge correlation between two rando...
read it

Statistical limits of spiked tensor models
We study the statistical limits of both detecting and estimating a rank...
read it

Information theoretic limits of learning a sparse rule
We consider generalized linear models in regimes where the number of non...
read it
The AllorNothing Phenomenon in Sparse Linear Regression
We study the problem of recovering a hidden binary ksparse pdimensional vector β from n noisy linear observations Y=Xβ+W where X_ij are i.i.d. N(0,1) and W_i are i.i.d. N(0,σ^2). A closely related hypothesis testing problem is to distinguish the pair (X,Y) generated from this structured model from a corresponding null model where (X,Y) consist of purely independent Gaussian entries. In the low sparsity k=o(p) and high signal to noise ratio k/σ^2=Ω(1) regime, we establish an `AllorNothing' informationtheoretic phase transition at a critical sample size n^*=2 k(p/k) /(1+k/σ^2), resolving a conjecture of gamarnikzadik. Specifically, we show that if _p→∞ n/n^*>1, then the maximum likelihood estimator almost perfectly recovers the hidden vector with high probability and moreover the true hypothesis can be detected with a vanishing error probability. Conversely, if _p→∞ n/n^*<1, then it becomes informationtheoretically impossible even to recover an arbitrarily small but fixed fraction of the hidden vector support, or to test hypotheses strictly better than random guess. Our proof of the impossibility result builds upon two key techniques, which could be of independent interest. First, we use a conditional second moment method to upper bound the KullbackLeibler (KL) divergence between the structured and the null model. Second, inspired by the celebrated area theorem, we establish a lower bound to the minimum mean squared estimation error of the hidden vector in terms of the KL divergence between the two models.
READ FULL TEXT
Comments
There are no comments yet.