Gradient Projection for Sparse Reconstruction Mário Figueiredo,

GPSR

     Gradient Projection  for Sparse  Reconstruction

 

 

   Mário Figueiredo,                               Robert D. Nowak,                                                Stephen J. Wright

   Institutode Telecomunicações           Electricaland Computer Engineering                   ComputerSciences Dept.

   InstitutoSuperior Técnico                  Universityof Wisconsin-Madison                        Universityof Wisconsin-Madison

   Lisboa,PORTUGAL                          Madison,WI, USA                                               Madison,WI, USA

 

 

Many problems in signal processing andstatistical inference are based on finding a sparse solution to an undeterminedlinear system of equations. 

BasisPursuit, the Least Absolute Shrinkage and Selection Operator (LASSO), wavelet-baseddeconvolution, and CompressedSensing are just a few well-known examples.

 

Computationally, the problem can be formulatedin different ways, most of them being convex optimization problems. Weconsidered a formulation in which a penalty term involving the scaled l₁-normof the signal is added to a least-squares term, a problem that can bereformulated as a convex quadratic program with bound constraints. This problemhas a potentially extremely large number of variables (though only a smallfraction of them are away from their bounds at the solution) and the data thatdefines it can often not be stored explicitly. We found that a solver ofgradient projection type, using special line search and termination techniques,gave faster solutions on our test problems than other techniques that had beenproposed previously, including interior-point techniques.  A debiasing step based on theconjugate-gradient algorithm improves the results further.

 

 

Finalversion,  September 12, 2007.

To appear in the IEEE Journal of SelectedTopics in Signal Processing: Special Issue on Convex Optimization Methods forSignal Processing).

 

NEW!  Updated version (January 19, 2009) of theMATLAB code is available here: GPSR_6.0 

 

The figure below shows a test case of a signalwith 4096 elements only 160 of which are not zero, and which is beingreconstructed from projection on 1024 unit-norm random vectors in4096-dimensional space; this is, of course, a highly under-determinedproblem.  The true signal is shown at thetop, while the reconstructions obtained from the l₁-regularizedformulation are shown in the second and third plots. Note that the locations ofthe spikes are reconstructed with high accuracy; their magnitudes areattenuated, but these can be corrected by applying our conjugate-gradientdebiasing approach. The lower part of the figure shows the minimum normsolution, which is not sparse and which bears little relation to the truesignal.

 

 

 

 

The figures below shows a comparison, in termsof computational speed, of GPSR versus three state-of-the-art solvers for thesame problem:

 

·        thel1-magic code, available here (from CalTech);

·        theSparseLab code, available here (from Stanford);

·        thenew l1_ls code (March 2007),available here (fromStanford);

·        thebound-optimization method (or iterative shrinkage/thresholding – IST),originally developed for wavelet-based deconvolution, described here.

 

The plots shows that our GPSR method is fasterand scales more favorably

 (w.r.t.n, the length of the unknown signal) than the competing techniques.

See the paper for details about theexperiments.

 

 

 

 

Funding Acknowledgment:

 

·        Thiswork was partially supported by the USA National Science Foundation (NSF),under grants CCF-0430504  and  CNS-0540147.

 

·        Thiswork was partially supported by the Portuguese Fundação para a Ciência e Tecnologia (FCT), under projectPOSC/EEA-CPS/61271/2004.