Convex Optimization : some definitions

线性与凸集理论概览
最新推荐文章于 2025-10-10 18:20:20 发布
原创 最新推荐文章于 2025-10-10 18:20:20 发布 · 532 阅读 · 0 ·
CC 4.0 BY-SA版权
版权声明：本文为博主原创文章，遵循 CC 4.0 BY-SA 版权协议，转载请附上原文出处链接和本声明。

本文深入探讨了线性关系与仿射关系的概念，解析了仿射集与凸集的特性及其几何意义，包括如何通过线性方程组表达集合，以及凸集如何包含任意两点间的线段。此外，还介绍了凸组合、凸包、锥形组合与锥形包的基本概念，以及超平面、半空间、多面体和多项式的定义。文中还详细讨论了保持凸性的各种运算，如尺度变换、平移、投影等，并解释了广义不等式与正则锥的关系。

Linear relation: Ax

Affine relation: Ax+b

Affine Set

  1. contains the line through any two distinct points in the set: x=ax_1+(1-a)x_2
  2. can be expressed as the solution set of a system of linear equations

Convex Set

  1. If two points are in the set, then the line segment as well.
  2. Affine set with 0<a<1

Convex Combinations

  1. Consider any number of points, with their coefficient being positive and summing up to 1

Convex Hull

  1. All convex combinations of any points from the set.

Conic (non-negative) combination

  1. Like convex combinations but without the constraint of "summing up to 1"
  2. Formed with any points together with the origin.

Conic Hull

  1. All conic combinations of any points from the set.

Hyperplane and halfspace

  1. hyperplanes are affine and convex: 
  2. halfspaces are convex

Polyhedron/polytope

  1. solution space of a system of inequalities like in a linear program
  2. intersections of a finit set of hyperplanes and halfspaces

Operations that preserves covexity

  1. S is convex set -> f(S) too, f^-1(S) as well. with f an affine function
  2. scale, translation, projection...
  3. perspective (projection) function and its inverse: f:R^{n+1}\rightarrow R^{n}. x^*=[\vec{x},t]\in R^{n+1}, f(x^*)=\{\vec{x}/t|t>0\}.
  4. linear fractional function: a fraction of linear functions. f:R^{m}\rightarrow R^{n}. f(x)=\frac{Ax+b}{Cx+d},with the denominator >0.
  5. can be seen as a streching of vision, or the projection of a 3d object  onto a camera. x=[x1,x2]. If the object is convex then the result is convex too.

 

 

 

 

Generalized Inequalities

A proper cone is closed (contains boundary), solid (not empty not a ray) and pointed(not a line).

define x\preceq _Ky \Leftrightarrow y-x\in K , reads "x less than y with respect to proper cone K". Remember K somehow define a 'positive' space, consider an orthant. The coordinates y-x must still in the same orthant.

  1. componentwise inequality: K=R_+^n, x<y  == xi<yi
  2. matrix inequality:K=S_+^n, X\preceq Y \Leftrightarrow Y-X \in S_+^n,(w.r.t positive semidefinite)
  3. \preceq_K is\ a\ generalized\ \leq, but the latter defines also a "linear ordering", while the former doesn't, where comes the difference between Minimum (less than anyone else) and Minimal(less than anyone else that are comparable).

TO BE CONTINUED

 

 

 

凸优化第五章对偶 作业题
清风吹斜阳
02-12 2030
Lagrange对偶问题 The dual function provides a lower bound on the optimal value for convex optimization problems only. 对偶函数可以为任意优化问题的最优值提供下界。 The dual function of an optimization problem evaluated at ...
【2011-2016】 NIPS汇总 - Advances in Neural Information Processing Systems
Testing的博客
10-02 1万+
Advances in Neural Information Processing Systems 24 (NIPS 2011) The papers below appear in Advances in Neural Information Processing Systems 24 edited by J. Shawe-Taylor and R.S. Zemel and P.L.
convex optimization
05-17
convex optimization中文版和相关资料 Stephen Boyd, Stanford University, California, Lieven Vandenberghe, University of California, Los Angeles Convex optimization problems arise frequently in many different fields. This book provides a comprehensive introduction to the subject, and shows in detail how such problems can be solved numerically with great efficiency. The book begins with the basic elements of convex sets and functions, and then describes various classes of convex optimization problems.
凸优化（Convex Optimization）介绍
最新发布
weixin_42849849的博客
10-10 1131
凸优化（Convex Optimization）是数学优化中的一个重要分支，因其良好的理论性质和广泛的实际应用而备受关注。凸优化问题具有一些非常理想的特性，例如：任何局部最优解都是全局最优解，且存在高效的数值算法可以求解大规模问题。尽管凸优化问题在理论上可能是NP难的（如某些非光滑问题），但大多数常见类型（如线性规划、二次规划、半定规划、锥规划等）都可以用内点法、梯度法、ADMM等算法高效求解。凸优化问题通常具有强对偶性（即原问题与对偶问题的最优值相等），这在算法设计和理论分析中非常有用。
Convex Optimization（凸优化）
11-28
看起来是厚厚的一本大部头，读起来并不太费力。它给出的实例多而好用、覆盖面全，不需要太深刻的数学功底，对于复杂的定理性质等也不强调证明，而是着眼于几何意义和实际用途，直观易懂。 作者本身的工科背景使得这本书在工业问题和计算机等实用方面的优点更为突出，数学依据上覆盖面广而不强调深，非常值得想要了解非线性规划的人一读。
Convex optimization 1 --- convex set
降低学习速度，提高学习效率的地方
04-11 597
1 Introduction 凸优化在工程领域发挥着重要作用，这个系列根据standford EE364a 的convex optimization 课程，进行梳理总结。 2 优化问题 2.1 常见的优化问题 2.1.1 决策类的优化问题 设定目标，考虑限制，给出决策。 2.1.2 模型参数辨识 选定模型，带入数据，计算系数。 此时目标是降低模型的预测误差，参数必须在合理的区间，给出最小的参数。 2.2 数学抽象 构建工程的最优化问题，constraints一般比较容易构建，如何选定object 函数非
Convex Optimization(1)
F~C~H的博客
11-13 1498
Introduction Mathematical Optimization The basic form: minimize:f0(x)subject&amp;nbsp;to:fi(x)≤biminimize: f_{0}(x)\\ subject\ to: f_{i}(x)\le b_{i}minimize:f0​(x)subject&amp;nbsp;to:fi​(x)≤bi​ . x=(x1,...,x...
维基百科中凸函数的定义 Convex function
热门推荐
GarfieldEr007的专栏
05-27 1万+
From Wikipedia, the free encyclopedia Convex function on an interval. A function (in black) is convex if and only if the region above its graph (in green) is aconve
【Introduction】: Demystifying the Principles of Generative Adversarial Networks (GANs): Essential ...
# 1. Introduction to Generative Adversarial Networks (GANs): Essential Basics and Techniques for Beginners Generative Adversarial Networks (GANs) represent a significant breakthrough in the field of ...
IE598NH-lecture-10-11-CCP.pdf
11-16
- **Convexity and Non-convexity**: Chance constraints can lead to non-convex feasible regions, making the optimization problem more challenging to solve. - **Approximation Techniques**: Due to the ...
ConvexOptimization
07-04
凸优化学习，数学基础学习，深度学习者学习，机器学习
Convex Optimization
11-10
非常经典，我们教材就用的这个！该版本非常清晰，强烈推荐！ Preface xi 1 Introduction 1 1.1 Mathematical optimization . . . . . . . . . . . . . . . . . . . . . . . . 1 1.2 Least-squares and linear programming . . . . . . . . . . . . . . . . . . 4 1.3 Convex optimization . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7 1.4 Nonlinear optimization . . . . . . . . . . . . . . . . . . . . . . . . . . 9 1.5 Outline . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11 1.6 Notation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14 Bibliography . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16 I Theory 19 2 Convex sets 21 2.1 Affine and convex sets . . . . . . . . . . . . . . . . . . . . . . . . . . . 21 2.2 Some important examples . . . . . . . . . . . . . . . . . . . . . . . . . 27 2.3 Operations that preserve convexity . . . . . . . . . . . . . . . . . . . . 35 2.4 Generalized inequalities . . . . . . . . . . . . . . . . . . . . . . . . . . 43 2.5 Separating and supporting hyperplanes . . . . . . . . . . . . . . . . . . 46 2.6 Dual cones and generalized inequalities . . . . . . . . . . . . . . . . . . 51 Bibliography . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 59 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 60 3 Convex functions 67 3.1 Basic properties and examples . . . . . . . . . . . . . . . . . . . . . . 67 3.2 Operations that preserve convexity . . . . . . . . . . . . . . . . . . . . 79 3.3 The conjugate function . . . . . . . . . . . . . . . . . . . . . . . . . . 90 3.4 Quasiconvex functions . . . . . . . . . . . . . . . . . . . . . . . . . . . 95 3.5 Log-concave and log-convex functions . . . . . . . . . . . . . . . . . . 104 3.6 Convexity with respect to generalized inequalities . . . . . . . . . . . . 108 Bibliography . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 112 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 113 viii Contents 4 Convex optimization problems 127 4.1 Optimization problems . . . . . . . . . . . . . . . . . . . . . . . . . . 127 4.2 Convex optimization . . . . . . . . . . . . . . . . . . . . . . . . . . . . 136 4.3 Linear optimization problems . . . . . . . . . . . . . . . . . . . . . . . 146 4.4 Quadratic optimization problems . . . . . . . . . . . . . . . . . . . . . 152 4.5 Geometric programming . . . . . . . . . . . . . . . . . . . . . . . . . . 160 4.6 Generalized inequality constraints . . . . . . . . . . . . . . . . . . . . . 167 4.7 Vector optimization . . . . . . . . . . . . . . . . . . . . . . . . . . . . 174 Bibliography . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 188 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 189 5 Duality 215 5.1 The Lagrange dual function . . . . . . . . . . . . . . . . . . . . . . . . 215 5.2 The Lagrange dual problem . . . . . . . . . . . . . . . . . . . . . . . . 223 5.3 Geometric interpretation . . . . . . . . . . . . . . . . . . . . . . . . . 232 5.4 Saddle-point interpretation . . . . . . . . . . . . . . . . . . . . . . . . 237 5.5 Optimality conditions . . . . . . . . . . . . . . . . . . . . . . . . . . . 241 5.6 Perturbation and sensitivity analysis . . . . . . . . . . . . . . . . . . . 249 5.7 Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 253 5.8 Theorems of alternatives . . . . . . . . . . . . . . . . . . . . . . . . . 258 5.9 Generalized inequalities . . . . . . . . . . . . . . . . . . . . . . . . . . 264 Bibliography . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 272 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 273 II Applications 289 6 Approximation and fitting 291 6.1 Norm approximation . . . . . . . . . . . . . . . . . . . . . . . . . . . . 291 6.2 Least-norm problems . . . . . . . . . . . . . . . . . . . . . . . . . . . 302 6.3 Regularized approximation . . . . . . . . . . . . . . . . . . . . . . . . 305 6.4 Robust approximation . . . . . . . . . . . . . . . . . . . . . . . . . . . 318 6.5 Function fitting and interpolation . . . . . . . . . . . . . . . . . . . . . 324 Bibliography . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 343 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 344 7 Statistical estimation 351 7.1 Parametric distribution estimation . . . . . . . . . . . . . . . . . . . . 351 7.2 Nonparametric distribution estimation . . . . . . . . . . . . . . . . . . 359 7.3 Optimal detector design and hypothesis testing . . . . . . . . . . . . . 364 7.4 Chebyshev and Chernoff bounds . . . . . . . . . . . . . . . . . . . . . 374 7.5 Experiment design . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 384 Bibliography . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 392 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 393 Contents ix 8 Geometric problems 397 8.1 Projection on a set . . . . . . . . . . . . . . . . . . . . . . . . . . . . 397 8.2 Distance between sets . . . . . . . . . . . . . . . . . . . . . . . . . . . 402 8.3 Euclidean distance and angle problems . . . . . . . . . . . . . . . . . . 405 8.4 Extremal volume ellipsoids . . . . . . . . . . . . . . . . . . . . . . . . 410 8.5 Centering . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 416 8.6 Classification . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 422 8.7 Placement and location . . . . . . . . . . . . . . . . . . . . . . . . . . 432 8.8 Floor planning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 438 Bibliography . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 446 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 447 III Algorithms 455 9 Unconstrained minimization 457 9.1 Unconstrained minimization problems . . . . . . . . . . . . . . . . . . 457 9.2 Descent methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 463 9.3 Gradient descent method . . . . . . . . . . . . . . . . . . . . . . . . . 466 9.4 Steepest descent method . . . . . . . . . . . . . . . . . . . . . . . . . 475 9.5 Newton’s method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 484 9.6 Self-concordance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 496 9.7 Implementation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 508 Bibliography . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 513 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 514 10 Equality constrained minimization 521 10.1 Equality constrained minimization problems . . . . . . . . . . . . . . . 521 10.2 Newton’s method with equality constraints . . . . . . . . . . . . . . . . 525 10.3 Infeasible start Newton method . . . . . . . . . . . . . . . . . . . . . . 531 10.4 Implementation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 542 Bibliography . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 556 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 557 11 Interior-point methods 561 11.1 Inequality constrained minimization problems . . . . . . . . . . . . . . 561 11.2 Logarithmic barrier function and central path . . . . . . . . . . . . . . 562 11.3 The barrier method . . . . . . . . . . . . . . . . . . . . . . . . . . . . 568 11.4 Feasibility and phase I methods . . . . . . . . . . . . . . . . . . . . . . 579 11.5 Complexity analysis via self-concordance . . . . . . . . . . . . . . . . . 585 11.6 Problems with generalized inequalities . . . . . . . . . . . . . . . . . . 596 11.7 Primal-dual interior-point methods . . . . . . . . . . . . . . . . . . . . 609 11.8 Implementation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 615 Bibliography . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 621 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 623 x Contents Appendices 631 A Mathematical background 633 A.1 Norms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 633 A.2 Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 637 A.3 Functions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 639 A.4 Derivatives . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 640 A.5 Linear algebra . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 645 Bibliography . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 652 B Problems involving two quadratic functions 653 B.1 Single constraint quadratic optimization . . . . . . . . . . . . . . . . . 653 B.2 The S-procedure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 655 B.3 The field of values of two symmetric matrices . . . . . . . . . . . . . . 656 B.4 Proofs of the strong duality results . . . . . . . . . . . . . . . . . . . . 657 Bibliography . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 659 C Numerical linear algebra background 661 C.1 Matrix structure and algorithm complexity . . . . . . . . . . . . . . . . 661 C.2 Solving linear equations with factored matrices . . . . . . . . . . . . . . 664 C.3 LU, Cholesky, and LDLT factorization . . . . . . . . . . . . . . . . . . 668 C.4 Block elimination and Schur complements . . . . . . . . . . . . . . . . 672 C.5 Solving underdetermined linear equations . . . . . . . . . . . . . . . . . 681 Bibliography . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 684 References 685 Notation 697 Index 701
学术论文写作之引言(Introduction)怎么写
ChanZany的博客
04-17 1万+
如何写引言： 引言的组成 The background information 背景知识 The relevant literature 相关文献 The nature ,scope,purpose and significance of the problems investigated 研究的问题的重要性以及研究的意义 The results of the investigation 研究结...
Convex Optimization(Stephen Boyd)
12-13
Convex optimization problems arise frequently in many different fields. This book provides a comprehensive introduction to the subject, and shows in detail how such problems can be solved numerically with great efficiency. The book begins with the basic elements of convex sets and functions, and then describes various classes of convex optimization problems. Duality and approximation techniques are th en covered, as are statistical estimation techniques. Various geometrical problems are then presented, and there is detailed discussion of unconstrained and constrained minimization problems, and interior-point methods. The focus of the book is on recognizing convex optimization problems and then finding the most appropriate technique for solving them. It contains many worked examples and homework exercises and will appeal to students, researchers and practitioners in fields such as engineering, computer science, mathematics, statistics, finance and economics.
评论
成就一亿技术人!
拼手气红包6.0元
还能输入1000个字符
 
红包 添加红包
表情包 插入表情
表情包 代码片
 条评论被折叠 查看
添加红包

请填写红包祝福语或标题

个

红包个数最小为10个

元

红包金额最低5元

当前余额3.43元 前往充值 >
需支付：10.00元
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付元
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明：

1.余额是钱包充值的虚拟货币，按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载，可以购买VIP、付费专栏及课程。

余额充值