凸优化有关的数值线性代数知识五：作业题

最新推荐文章于 2024-01-11 13:57:24 发布

使君杭千秋

最新推荐文章于 2024-01-11 13:57:24 发布

阅读量291

点赞数 1

分类专栏：凸优化

原文链接：https://blog.youkuaiyun.com/wangchy29/article/details/87906162

版权

凸优化专栏收录该内容

62 篇文章

订阅专栏

矩阵结果与算法复杂性

Algorithm flop counts allow for very accurate and precise prediction of running time on a given computer.

About how long does a 1 Gflop computer take to solve a system of 100 linear equations with 100 variables?

About how long does a 1 Gflop computer take to solve 10 systems of of 100 linear equations with 100 variables, with the same coefficient matrix but 10 different right hand sides?

Since matrix multiplication is associative, the flop count for multiplying three or more matrices doesn't depend on the order in which you multiply them.

多个矩阵相乘，计算的顺序对浮点数运算次数有影响，比如 $A\in R^{m \times n},B \in R^{n \times p},C \in R^{p \times q}$

考虑计算ABC的两种顺序，

（1）先计算AB，再计算(AB)C，首先计算AB所需浮点计算次数为mp(2n-1)，再计算(AB)C所需浮点计算次数为mq(2p-1)，一共mp(2n-1)+mq(2p-1)=2mp(n+q)-m(p+q)

（2）先计算BC，再计算A(BC)，首先计算BC所需浮点计算次数为nq(2p-1)，再计算A(BC)所需浮点计算次数为mq(2n-1)，一共nq(2p-1)+mq(2n-1)=2qn(m+p)-q(m+n)

求解已经因式分解的矩阵的线性方程组

Suppose $A \in R^{n \times n}$ is lower triangular. The flop count for computing Ab is the same order as the flop count for computing $A^{-1}b$ .

下三角矩阵的逆矩阵也是下三角矩阵，所以计算Ab和计算 $A^{-1}b$ 所需要浮点运算次数一样。

LU、Cholesky和因式分解

Suppose $A \in R^{m \times n}$ , and we need to compute x that minimizes $\| Ax-b\| _2^2 + \rho / 2 \| x\| _2^2$ , where ρ>0.

What is the flop count order when m≥n? (Provide your answer in terms of m and n. For example if you think the solution is m⋅n enter m*n in the answer box).

我是这样理解的：

目标函数写成

$L=\begin{Vmatrix}Ax-b \end{Vmatrix}_2^2+\rho /2\begin{Vmatrix}x\end{Vmatrix}_2^2 \\ =(Ax-b)^T(Ax-b)+\rho/2x^Tx \\ =x^TA^TAx-2b^TAx+b^Tb+\rho /2x^Tx \\ =x^T(A^TA+\rho/2I)x-2b^TAx+b^Tb \\$

$\frac{\partial L}{ \partial x}=2(A^TA+\rho /2I)x-2A^Tb\\ \frac{\partial^2 L}{ \partial x^2}=2(A^TA+\rho /2 I)\geq 0$

可知目标函数是凸函数，故令其一阶导数为０，得到

$(A^TA+\rho/2 I)x=A^Tb$

求解x，即求解此次方程组，记 $B=A^TA+\rho /2I,c=A^Tb$ ，故方程组为 Bx=c

计算B所需要的浮点运算次数：（1）A^TA: n^2(2m-1) 次 (2) $\rho /2 I$ :0次（3） $A^TA+\rho /2 I$ :只在对角线元素有加法，故n次

计算c的浮点运算： n(2m-1)

求解方程组： n^3

由于 $m\geq n$ ，所以阶次是 mn^2

For m≤n, the flop count order is

这个答案我不理解。

Suppose m≥n, we have already solved the problem for one value of ρ, and now we want to compute the solution for k other values of ρ (this gives the so called regularization path). The flop count order for doing this is

对于这个答案我是这样理解的，首先 $B=A^TA+\rho /2 I$ 是对称正定矩阵，所以B是可以分解成 B=LL^T ，对一个 $\rho$ 已经计算完了c,B 并求得了x值，所以已经求得了B 的分解，然后对于不同的 $\rho$ 值，B只改变对角线上的元素，所以对于其他的 $\rho$ 值，B的分解应该不再需要 n^3 ，而是n，然后分解以后的计算量阶次是 n^2 ，k个不同的 $\rho$ 值，则需要 kn^2 阶次的计算量。

分块消元和Schur补

Suppose you wish to solve the following set of linear equations

$\left[ \begin{array}{ll} A_{11} & A_{12} \\ A_{21} & A_{22} \end{array} \right] \left[ \begin{array}{l} x_1 \\ x_2 \end{array} \right] = \left[ \begin{array}{l} b_1 \\ b_2 \end{array} \right]$

It is a good idea to eliminate x_1 to solve the equations if

What is the computational complexity (order) of computing the Schur complement of the block $A_{11}$ in the matrix

$\left[ \begin{array}{ll} A_{11} & A_{12} \\ A_{21} & A_{22} \end{array} \right]$

with $A_{11} \in \mathbf{R}^{n \times n}$ , $A_{12} \in \mathbf{R}^{n \times m}$ and $A_{21} \in \mathbf{R}^{m \times n}$ , given that the cost to factor a square matrix of size n is f and the cost to back solve a square matrix of size n is s?

Your expression should only involve the symbols f, s, n and m.

$A_{11}$ 的Schur补为： $A_{22}-A_{21}A_{11}^{-1}A_{12}$ ，首先对 $A_{11}$ 分解需要f，再计算 $A_{11}^{-1}A_{12}$ 对应 $n \times n,n \times m$ ，需要ms，在计算 $A_{21}(A_{11}^{-1}A_{12})$ ，对应 $m \times n,n \times m$ 。需要 m^2(2n-1) ，减法需要 m^2 ，一共 f+ms+m^2(2n-1)+m^2=f+ms+2m^2n ，复杂度为 f+ms+m^2n 。