【源码】采用高斯-索思韦尔准则实现比随机选择收敛更快的坐标下降法

从Nesterov的工作开始，最近关于随机坐标下降算法的理论和应用已经进行了大量工作，表明随机坐标选择准则能达到与高斯-索思韦尔选择准则相同的收敛速度。

There has been significant recent work on the theory and application ofrandomized coordinate descent algorithms, beginning with the work of Nesterov,who showed that a random-coordinate selection rule achieves the sameconvergence rate as the Gauss-Southwell selection rule.

研究结果表明，我们永远不应该使用高斯-索思韦尔准则，因为它通常比随机选择复杂得多。

This result suggests that we should never use the Gauss-Southwell rule,because it is typically much more expensive than random selection.

然而，这些算法的经验行为与此理论结果相抵触：在选择准则的计算成本可比拟的应用中，高斯-索思韦尔选择准则倾向于具有比随机坐标选择更好的性能。

However, the empirical behaviours of these algorithms contradict thistheoretical result: in applications where the computational costs of theselection rules are comparable, the Gauss-Southwell selection rule tends toperform substantially better than random coordinate selection.

我们对高斯-索思韦尔准则进行了简单的分析，结果表明，除了极端情况，它的收敛速度比随机坐标选择要更快一些。

We give a simple analysis of the Gauss-Southwell rule showingthat—except in extreme cases—its convergence rate is faster than choosingrandom coordinates.

原文及完整源码下载地址：

http://page2.dfpan.com/fs/elfcfj62f21112b9160/

更多精彩文章请关注微信号：