好文转载 机器学习强基计划8-5：图解局部线性嵌入LLE算法(附Python实现)

最新推荐文章于 2025-04-04 13:35:00 发布
最新推荐文章于 2025-04-04 13:35:00 发布
阅读量670 收藏 1
点赞数
CC 4.0 BY-SA版权
分类专栏： 好文推荐 文章标签： 机器学习 python 算法
原文链接：https://blog.youkuaiyun.com/FRIGIDWINTER/article/details/130467687

在这里插入图片描述

目录

0 写在前面

机器学习强基计划聚焦深度和广度，加深对机器学习模型的理解与应用。“深”在详细推导算法模型背后的数学原理；“广”在分析多个机器学习模型：决策树、支持向量机、贝叶斯与马尔科夫决策、强化学习等。强基计划实现从理论到实践的全面覆盖，由本人亲自从底层编写、测试与文章配套的各个经典算法，不依赖于现有库，可以大大加深对算法的理解。

🚀详情：机器学习强基计划(附几十种经典模型源码)

1 流形学习

在机器学习强基计划8-4：流形学习等度量映射Isomap算法(附Python实现)中，我们介绍了流形(manifolds)的概念，它是可以局部欧几里得空间化的一个拓扑空间，是具有拓扑结构的点集，是欧几里得空间中的曲线、曲面等概念的推广。对流形的物理意义不熟悉的同学可以回顾上一篇文章。

在这里插入图片描述

流形学习(manifold learning)是基于流形的机器学习算法，它假设样本数据分布在高维特征空间中的低维嵌入流形上，虽然整体十分复杂，但局部上仍具有欧氏空间的性质，因此可以在局部建立降维映射关系，然后再将局部映射关系推广到全局——局部线性构造全局非线性。流形学习旨在从观测样本中去寻找产生数据分布的内在本质规律，而非破坏结构性地降维。

2 局部线性嵌入算法

2.1 什么是局部线性嵌入？

局部线性嵌入(Locally Linear Embedding, LLE)限制样本在降维后的低维空间中的 

     k 
    
   
  
    k 
   
  
</span><span class="katex-html"><span class="base"><span class="strut" style="height: 0.6944em;"></span><span class="mord mathnormal" style="margin-right: 0.0315em;">k</span></span></span></span></span>近邻局部线性关系，等价于原始空间。</p>

例如，若高维样本满足 

      x 
     
    
      i 
     
    
   
     = 
    
    
    
      w 
     
     
     
       i 
      
     
       j 
      
     
    
    
    
      x 
     
    
      j 
     
    
   
     + 
    
    
    
      w 
     
     
     
       i 
      
     
       k 
      
     
    
    
    
      x 
     
    
      k 
     
    
   
     + 
    
    
    
      w 
     
     
     
       i 
      
     
       l 
      
     
    
    
    
      x 
     
    
      l 
     
    
   
  
    \boldsymbol{x}_i=w_{ij}\boldsymbol{x}_j+w_{ik}\boldsymbol{x}_k+w_{il}\boldsymbol{x}_l 
   
  
</span><span class="katex-html"><span class="base"><span class="strut" style="height: 0.5944em; vertical-align: -0.15em;"></span><span class="mord"><span class="mord"><span class="mord"><span class="mord boldsymbol">x</span></span></span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height: 0.3117em;"><span class="" style="top: -2.55em; margin-right: 0.05em;"><span class="pstrut" style="height: 2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mathnormal mtight">i</span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height: 0.15em;"><span class=""></span></span></span></span></span></span><span class="mspace" style="margin-right: 0.2778em;"></span><span class="mrel">=</span><span class="mspace" style="margin-right: 0.2778em;"></span></span><span class="base"><span class="strut" style="height: 0.8694em; vertical-align: -0.2861em;"></span><span class="mord"><span class="mord mathnormal" style="margin-right: 0.0269em;">w</span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height: 0.3117em;"><span class="" style="top: -2.55em; margin-left: -0.0269em; margin-right: 0.05em;"><span class="pstrut" style="height: 2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight"><span class="mord mathnormal mtight" style="margin-right: 0.0572em;">ij</span></span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height: 0.2861em;"><span class=""></span></span></span></span></span></span><span class="mord"><span class="mord"><span class="mord"><span class="mord boldsymbol">x</span></span></span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height: 0.3117em;"><span class="" style="top: -2.55em; margin-right: 0.05em;"><span class="pstrut" style="height: 2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mathnormal mtight" style="margin-right: 0.0572em;">j</span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height: 0.2861em;"><span class=""></span></span></span></span></span></span><span class="mspace" style="margin-right: 0.2222em;"></span><span class="mbin">+</span><span class="mspace" style="margin-right: 0.2222em;"></span></span><span class="base"><span class="strut" style="height: 0.7333em; vertical-align: -0.15em;"></span><span class="mord"><span class="mord mathnormal" style="margin-right: 0.0269em;">w</span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height: 0.3361em;"><span class="" style="top: -2.55em; margin-left: -0.0269em; margin-right: 0.05em;"><span class="pstrut" style="height: 2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight"><span class="mord mathnormal mtight" style="margin-right: 0.0315em;">ik</span></span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height: 0.15em;"><span class=""></span></span></span></span></span></span><span class="mord"><span class="mord"><span class="mord"><span class="mord boldsymbol">x</span></span></span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height: 0.3361em;"><span class="" style="top: -2.55em; margin-right: 0.05em;"><span class="pstrut" style="height: 2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mathnormal mtight" style="margin-right: 0.0315em;">k</span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height: 0.15em;"><span class=""></span></span></span></span></span></span><span class="mspace" style="margin-right: 0.2222em;"></span><span class="mbin">+</span><span class="mspace" style="margin-right: 0.2222em;"></span></span><span class="base"><span class="strut" style="height: 0.5944em; vertical-align: -0.15em;"></span><span class="mord"><span class="mord mathnormal" style="margin-right: 0.0269em;">w</span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height: 0.3361em;"><span class="" style="top: -2.55em; margin-left: -0.0269em; margin-right: 0.05em;"><span class="pstrut" style="height: 2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight"><span class="mord mathnormal mtight">i</span><span class="mord mathnormal mtight" style="margin-right: 0.0197em;">l</span></span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height: 0.15em;"><span class=""></span></span></span></span></span></span><span class="mord"><span class="mord"><span class="mord"><span class="mord boldsymbol">x</span></span></span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height: 0.3361em;"><span class="" style="top: -2.55em; margin-right: 0.05em;"><span class="pstrut" style="height: 2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mathnormal mtight" style="margin-right: 0.0197em;">l</span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height: 0.15em;"><span class=""></span></span></span></span></span></span></span></span></span></span>，则低维样本同样满足<span class="katex--inline"><span class="katex"><span class="katex-mathml"> 
 
  
   
    
    
      z 
     
    
      i 
     
    
   
     = 
    
    
    
      w 
     
     
     
       i 
      
     
       j 
      
     
    
    
    
      z 
     
    
      j 
     
    
   
     + 
    
    
    
      w 
     
     
     
       i 
      
     
       k 
      
     
    
    
    
      z 
     
    
      k 
     
    
   
     + 
    
    
    
      w 
     
     
     
       i 
      
     
       l 
      
     
    
    
    
      z 
     
    
      l 
     
    
   
  
    \boldsymbol{z}_i=w_{ij}\boldsymbol{z}_j+w_{ik}\boldsymbol{z}_k+w_{il}\boldsymbol{z}_l 
   
  
</span><span class="katex-html"><span class="base"><span class="strut" style="height: 0.5944em; vertical-align: -0.15em;"></span><span class="mord"><span class="mord"><span class="mord"><span class="mord boldsymbol" style="margin-right: 0.0421em;">z</span></span></span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height: 0.3117em;"><span class="" style="top: -2.55em; margin-right: 0.05em;"><span class="pstrut" style="height: 2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mathnormal mtight">i</span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height: 0.15em;"><span class=""></span></span></span></span></span></span><span class="mspace" style="margin-right: 0.2778em;"></span><span class="mrel">=</span><span class="mspace" style="margin-right: 0.2778em;"></span></span><span class="base"><span class="strut" style="height: 0.8694em; vertical-align: -0.2861em;"></span><span class="mord"><span class="mord mathnormal" style="margin-right: 0.0269em;">w</span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height: 0.3117em;"><span class="" style="top: -2.55em; margin-left: -0.0269em; margin-right: 0.05em;"><span class="pstrut" style="height: 2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight"><span class="mord mathnormal mtight" style="margin-right: 0.0572em;">ij</span></span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height: 0.2861em;"><span class=""></span></span></span></span></span></span><span class="mord"><span class="mord"><span class="mord"><span class="mord boldsymbol" style="margin-right: 0.0421em;">z</span></span></span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height: 0.3117em;"><span class="" style="top: -2.55em; margin-right: 0.05em;"><span class="pstrut" style="height: 2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mathnormal mtight" style="margin-right: 0.0572em;">j</span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height: 0.2861em;"><span class=""></span></span></span></span></span></span><span class="mspace" style="margin-right: 0.2222em;"></span><span class="mbin">+</span><span class="mspace" style="margin-right: 0.2222em;"></span></span><span class="base"><span class="strut" style="height: 0.7333em; vertical-align: -0.15em;"></span><span class="mord"><span class="mord mathnormal" style="margin-right: 0.0269em;">w</span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height: 0.3361em;"><span class="" style="top: -2.55em; margin-left: -0.0269em; margin-right: 0.05em;"><span class="pstrut" style="height: 2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight"><span class="mord mathnormal mtight" style="margin-right: 0.0315em;">ik</span></span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height: 0.15em;"><span class=""></span></span></span></span></span></span><span class="mord"><span class="mord"><span class="mord"><span class="mord boldsymbol" style="margin-right: 0.0421em;">z</span></span></span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height: 0.3361em;"><span class="" style="top: -2.55em; margin-right: 0.05em;"><span class="pstrut" style="height: 2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mathnormal mtight" style="margin-right: 0.0315em;">k</span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height: 0.15em;"><span class=""></span></span></span></span></span></span><span class="mspace" style="margin-right: 0.2222em;"></span><span class="mbin">+</span><span class="mspace" style="margin-right: 0.2222em;"></span></span><span class="base"><span class="strut" style="height: 0.5944em; vertical-align: -0.15em;"></span><span class="mord"><span class="mord mathnormal" style="margin-right: 0.0269em;">w</span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height: 0.3361em;"><span class="" style="top: -2.55em; margin-left: -0.0269em; margin-right: 0.05em;"><span class="pstrut" style="height: 2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight"><span class="mord mathnormal mtight">i</span><span class="mord mathnormal mtight" style="margin-right: 0.0197em;">l</span></span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height: 0.15em;"><span class=""></span></span></span></span></span></span><span class="mord"><span class="mord"><span class="mord"><span class="mord boldsymbol" style="margin-right: 0.0421em;">z</span></span></span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height: 0.3361em;"><span class="" style="top: -2.55em; margin-right: 0.05em;"><span class="pstrut" style="height: 2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mathnormal mtight" style="margin-right: 0.0197em;">l</span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height: 0.15em;"><span class=""></span></span></span></span></span></span></span></span></span></span>，如图所示，其中<span class="katex--inline"><span class="katex"><span class="katex-mathml"> 
 
  
   
    
    
      w 
     
    
      i 
     
    
   
  
    \boldsymbol{w}_i 
   
  
</span><span class="katex-html"><span class="base"><span class="strut" style="height: 0.5944em; vertical-align: -0.15em;"></span><span class="mord"><span class="mord"><span class="mord"><span class="mord boldsymbol" style="margin-right: 0.0278em;">w</span></span></span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height: 0.3117em;"><span class="" style="top: -2.55em; margin-right: 0.05em;"><span class="pstrut" style="height: 2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mathnormal mtight">i</span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height: 0.15em;"><span class=""></span></span></span></span></span></span></span></span></span></span>称为样本<span class="katex--inline"><span class="katex"><span class="katex-mathml"> 
 
  
   
    
    
      x 
     
    
      i 
     
    
   
  
    \boldsymbol{x}_i 
   
  
</span><span class="katex-html"><span class="base"><span class="strut" style="height: 0.5944em; vertical-align: -0.15em;"></span><span class="mord"><span class="mord"><span class="mord"><span class="mord boldsymbol">x</span></span></span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height: 0.3117em;"><span class="" style="top: -2.55em; margin-right: 0.05em;"><span class="pstrut" style="height: 2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mathnormal mtight">i</span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height: 0.15em;"><span class=""></span></span></span></span></span></span></span></span></span></span>的<font color="#4a86e8"><strong>重构系数</strong></font>。</p>

在这里插入图片描述

2.2 算法原理推导

设高维样本 

      x 
     
    
      i 
     
    
   
  
    \boldsymbol{x}_i 
   
  
</span><span class="katex-html"><span class="base"><span class="strut" style="height: 0.5944em; vertical-align: -0.15em;"></span><span class="mord"><span class="mord"><span class="mord"><span class="mord boldsymbol">x</span></span></span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height: 0.3117em;"><span class="" style="top: -2.55em; margin-right: 0.05em;"><span class="pstrut" style="height: 2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mathnormal mtight">i</span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height: 0.15em;"><span class=""></span></span></span></span></span></span></span></span></span></span>的<span class="katex--inline"><span class="katex"><span class="katex-mathml"> 
 
  
   
   
     k 
    
   
  
    k 
   
  
</span><span class="katex-html"><span class="base"><span class="strut" style="height: 0.6944em;"></span><span class="mord mathnormal" style="margin-right: 0.0315em;">k</span></span></span></span></span>近邻样本集合为<span class="katex--inline"><span class="katex"><span class="katex-mathml"> 
 
  
   
    
    
      Q 
     
    
      i 
     
    
      k 
     
    
   
  
    Q_{i}^{k} 
   
  
</span><span class="katex-html"><span class="base"><span class="strut" style="height: 1.1078em; vertical-align: -0.2587em;"></span><span class="mord"><span class="mord mathnormal">Q</span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height: 0.8491em;"><span class="" style="top: -2.4413em; margin-left: 0em; margin-right: 0.05em;"><span class="pstrut" style="height: 2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight"><span class="mord mathnormal mtight">i</span></span></span></span><span class="" style="top: -3.063em; margin-right: 0.05em;"><span class="pstrut" style="height: 2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight"><span class="mord mathnormal mtight" style="margin-right: 0.0315em;">k</span></span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height: 0.2587em;"><span class=""></span></span></span></span></span></span></span></span></span></span>，则重构系数应满足</p> 


         min 
        
       
         ⁡ 
        
       
       
        
        
          w 
         
        
          1 
         
        
       
         , 
        
        
        
          w 
         
        
          2 
         
        
       
         , 
        
       
         ⋯ 
         
       
         , 
        
        
        
          w 
         
        
          m 
         
        
       
      
     
     
     
       ∑ 
      
      
      
        i 
       
      
        = 
       
      
        1 
       
      
     
       m 
      
     
     
      
      
        ∥ 
       
       
       
         x 
        
       
         i 
        
       
      
        − 
       
       
       
         ∑ 
        
        
        
          j 
         
        
          ∈ 
         
         
         
           Q 
          
         
           i 
          
         
           k 
          
         
        
       
       
        
        
          w 
         
         
         
           i 
          
         
           j 
          
         
        
        
        
          x 
         
        
          j 
         
        
       
      
        ∥ 
       
      
     
       2 
      
     
       2 
      
       
    
      s 
     
    
      . 
     
    
      t 
     
    
      . 
     
     
     
       ∑ 
      
      
      
        j 
       
      
        ∈ 
       
       
       
         Q 
        
       
         i 
        
       
         k 
        
       
      
     
     
     
       w 
      
      
      
        i 
       
      
        j 
       
      
     
    
      = 
     
    
      1 
     
     
     
       ( 
      
     
       i 
      
     
       = 
      
     
       1 
      
     
       , 
      
     
       2 
      
     
       , 
      
     
       ⋯ 
       
     
       , 
      
     
       m 
      
     
       ) 
      
     
    
   
     \underset{\boldsymbol{w}_1,\boldsymbol{w}_2,\cdots ,\boldsymbol{w}_m}{\min}\sum_{i=1}^m{\left\| \boldsymbol{x}_i-\sum_{j\in Q_{i}^{k}}{w_{ij}\boldsymbol{x}_j} \right\| _{2}^{2}}\,\, \mathrm{s}.\mathrm{t}. \sum_{j\in Q_{i}^{k}}{w_{ij}}=1\left( i=1,2,\cdots ,m \right) 
    
   
 </span><span class="katex-html"><span class="base"><span class="strut" style="height: 3.9339em; vertical-align: -1.6799em;"></span><span class="mord"><span class="mop op-limits"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height: 0.6679em;"><span class="" style="top: -2.4em; margin-left: 0em;"><span class="pstrut" style="height: 3em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight"><span class="mord mtight"><span class="mord mtight"><span class="mord mtight"><span class="mord boldsymbol mtight" style="margin-right: 0.0278em;">w</span></span></span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height: 0.3173em;"><span class="" style="top: -2.357em; margin-right: 0.0714em;"><span class="pstrut" style="height: 2.5em;"></span><span class="sizing reset-size3 size1 mtight"><span class="mord mtight">1</span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height: 0.143em;"><span class=""></span></span></span></span></span></span><span class="mpunct mtight">,</span><span class="mord mtight"><span class="mord mtight"><span class="mord mtight"><span class="mord boldsymbol mtight" style="margin-right: 0.0278em;">w</span></span></span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height: 0.3173em;"><span class="" style="top: -2.357em; margin-right: 0.0714em;"><span class="pstrut" style="height: 2.5em;"></span><span class="sizing reset-size3 size1 mtight"><span class="mord mtight">2</span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height: 0.143em;"><span class=""></span></span></span></span></span></span><span class="mpunct mtight">,</span><span class="minner mtight">⋯</span><span class="mspace mtight" style="margin-right: 0.1952em;"></span><span class="mpunct mtight">,</span><span class="mord mtight"><span class="mord mtight"><span class="mord mtight"><span class="mord boldsymbol mtight" style="margin-right: 0.0278em;">w</span></span></span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height: 0.1645em;"><span class="" style="top: -2.357em; margin-right: 0.0714em;"><span class="pstrut" style="height: 2.5em;"></span><span class="sizing reset-size3 size1 mtight"><span class="mord mathnormal mtight">m</span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height: 0.143em;"><span class=""></span></span></span></span></span></span></span></span></span><span class="" style="top: -3em;"><span class="pstrut" style="height: 3em;"></span><span class=""><span class="mop"><span class="mop">min</span></span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height: 0.8361em;"><span class=""></span></span></span></span></span></span><span class="mspace" style="margin-right: 0.1667em;"></span><span class="mop op-limits"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height: 1.6514em;"><span class="" style="top: -1.8723em; margin-left: 0em;"><span class="pstrut" style="height: 3.05em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight"><span class="mord mathnormal mtight">i</span><span class="mrel mtight">=</span><span class="mord mtight">1</span></span></span></span><span class="" style="top: -3.05em;"><span class="pstrut" style="height: 3.05em;"></span><span class=""><span class="mop op-symbol large-op">∑</span></span></span><span class="" style="top: -4.3em; margin-left: 0em;"><span class="pstrut" style="height: 3.05em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mathnormal mtight">m</span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height: 1.2777em;"><span class=""></span></span></span></span></span><span class="mspace" style="margin-right: 0.1667em;"></span><span class="mord"><span class="minner"><span class="minner"><span class="mopen"><span class="delimsizing mult"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height: 2.05em;"><span class="" style="top: -4.05em;"><span class="pstrut" style="height: 5.6em;"></span><span class="" style="width: 0.556em; height: 3.6em;"> 
            <svg width="0.556em" height="3.600em" viewBox="0 0 556 3600"> 
             <path d="M145 15 v585 v2400 v585 c2.667,10,9.667,15,21,15

c10,0,16.667,-5,20,-15 v-585 v-2400 v-585 c-2.667,-10,-9.667,-15,-21,-15
c-10,0,-16.667,5,-20,15z M188 15 H145 v585 v2400 v585 h43z
M367 15 v585 v2400 v585 c2.667,10,9.667,15,21,15
c10,0,16.667,-5,20,-15 v-585 v-2400 v-585 c-2.667,-10,-9.667,-15,-21,-15
c-10,0,-16.667,5,-20,15z M410 15 H367 v585 v2400 v585 h43z">
​xi​−j∈Qik​∑​wij​xj​


​22​s.t.j∈Qik​∑​wij​=1(i=1,2,⋯,m)

为便于优化，将其改写为矩阵形式

         min 
        
       
         ⁡ 
        
       
       
        
        
          w 
         
        
          1 
         
        
       
         , 
        
        
        
          w 
         
        
          2 
         
        
       
         , 
        
       
         ⋯ 
         
       
         , 
        
        
        
          w 
         
        
          m 
         
        
       
      
     
     
     
       ∑ 
      
      
      
        i 
       
      
        = 
       
      
        1 
       
      
     
       m 
      
     
     
      
      
        w 
       
      
        i 
       
      
        T 
       
      
      
       
        
        
          X 
         
        
          ~ 
         
        
       
      
        i 
       
      
        T 
       
      
      
       
        
        
          X 
         
        
          ~ 
         
        
       
      
        i 
       
      
      
      
        w 
       
      
        i 
       
      
       
    
      s 
     
    
      . 
     
    
      t 
     
    
      . 
     
     
     
       w 
      
     
       i 
      
     
       T 
      
     
     
     
       1 
      
      
      
        k 
       
      
        × 
       
      
        1 
       
      
     
    
      = 
     
    
      1 
     
     
     
       ( 
      
     
       i 
      
     
       = 
      
     
       1 
      
     
       , 
      
     
       2 
      
     
       , 
      
     
       ⋯ 
       
     
       , 
      
     
       m 
      
     
       ) 
      
     
    
   
     \underset{\boldsymbol{w}_1,\boldsymbol{w}_2,\cdots ,\boldsymbol{w}_m}{\min}\sum_{i=1}^m{\boldsymbol{w}_{i}^{T}\boldsymbol{\tilde{X}}_{i}^{T}\boldsymbol{\tilde{X}}_i\boldsymbol{w}_i}\,\, \mathrm{s}.\mathrm{t}. \boldsymbol{w}_{i}^{T}\mathbf{1}_{k\times 1}=1\left( i=1,2,\cdots ,m \right) 
    
   
 </span><span class="katex-html"><span class="base"><span class="strut" style="height: 2.9291em; vertical-align: -1.2777em;"></span><span class="mord"><span class="mop op-limits"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height: 0.6679em;"><span class="" style="top: -2.4em; margin-left: 0em;"><span class="pstrut" style="height: 3em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight"><span class="mord mtight"><span class="mord mtight"><span class="mord mtight"><span class="mord boldsymbol mtight" style="margin-right: 0.0278em;">w</span></span></span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height: 0.3173em;"><span class="" style="top: -2.357em; margin-right: 0.0714em;"><span class="pstrut" style="height: 2.5em;"></span><span class="sizing reset-size3 size1 mtight"><span class="mord mtight">1</span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height: 0.143em;"><span class=""></span></span></span></span></span></span><span class="mpunct mtight">,</span><span class="mord mtight"><span class="mord mtight"><span class="mord mtight"><span class="mord boldsymbol mtight" style="margin-right: 0.0278em;">w</span></span></span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height: 0.3173em;"><span class="" style="top: -2.357em; margin-right: 0.0714em;"><span class="pstrut" style="height: 2.5em;"></span><span class="sizing reset-size3 size1 mtight"><span class="mord mtight">2</span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height: 0.143em;"><span class=""></span></span></span></span></span></span><span class="mpunct mtight">,</span><span class="minner mtight">⋯</span><span class="mspace mtight" style="margin-right: 0.1952em;"></span><span class="mpunct mtight">,</span><span class="mord mtight"><span class="mord mtight"><span class="mord mtight"><span class="mord boldsymbol mtight" style="margin-right: 0.0278em;">w</span></span></span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height: 0.1645em;"><span class="" style="top: -2.357em; margin-right: 0.0714em;"><span class="pstrut" style="height: 2.5em;"></span><span class="sizing reset-size3 size1 mtight"><span class="mord mathnormal mtight">m</span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height: 0.143em;"><span class=""></span></span></span></span></span></span></span></span></span><span class="" style="top: -3em;"><span class="pstrut" style="height: 3em;"></span><span class=""><span class="mop"><span class="mop">min</span></span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height: 0.8361em;"><span class=""></span></span></span></span></span></span><span class="mspace" style="margin-right: 0.1667em;"></span><span class="mop op-limits"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height: 1.6514em;"><span class="" style="top: -1.8723em; margin-left: 0em;"><span class="pstrut" style="height: 3.05em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight"><span class="mord mathnormal mtight">i</span><span class="mrel mtight">=</span><span class="mord mtight">1</span></span></span></span><span class="" style="top: -3.05em;"><span class="pstrut" style="height: 3.05em;"></span><span class=""><span class="mop op-symbol large-op">∑</span></span></span><span class="" style="top: -4.3em; margin-left: 0em;"><span class="pstrut" style="height: 3.05em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mathnormal mtight">m</span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height: 1.2777em;"><span class=""></span></span></span></span></span><span class="mspace" style="margin-right: 0.1667em;"></span><span class="mord"><span class="mord"><span class="mord"><span class="mord"><span class="mord boldsymbol" style="margin-right: 0.0278em;">w</span></span></span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height: 0.8913em;"><span class="" style="top: -2.453em; margin-right: 0.05em;"><span class="pstrut" style="height: 2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight"><span class="mord mathnormal mtight">i</span></span></span></span><span class="" style="top: -3.113em; margin-right: 0.05em;"><span class="pstrut" style="height: 2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight"><span class="mord mathnormal mtight" style="margin-right: 0.1389em;">T</span></span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height: 0.247em;"><span class=""></span></span></span></span></span></span><span class="mord"><span class="mord"><span class="mord"><span class="mord accent"><span class="vlist-t"><span class="vlist-r"><span class="vlist" style="height: 0.9495em;"><span class="" style="top: -3em;"><span class="pstrut" style="height: 3em;"></span><span class="mord boldsymbol" style="margin-right: 0.0778em;">X</span></span><span class="" style="top: -3.6051em;"><span class="pstrut" style="height: 3em;"></span><span class="accent-body" style="left: -0.2875em;"><span class="mord mathbf">~</span></span></span></span></span></span></span></span></span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height: 1.1808em;"><span class="" style="top: -2.453em; margin-right: 0.05em;"><span class="pstrut" style="height: 2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight"><span class="mord mathnormal mtight">i</span></span></span></span><span class="" style="top: -3.4024em; margin-right: 0.05em;"><span class="pstrut" style="height: 2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight"><span class="mord mathnormal mtight" style="margin-right: 0.1389em;">T</span></span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height: 0.247em;"><span class=""></span></span></span></span></span></span><span class="mord"><span class="mord"><span class="mord"><span class="mord accent"><span class="vlist-t"><span class="vlist-r"><span class="vlist" style="height: 0.9495em;"><span class="" style="top: -3em;"><span class="pstrut" style="height: 3em;"></span><span class="mord boldsymbol" style="margin-right: 0.0778em;">X</span></span><span class="" style="top: -3.6051em;"><span class="pstrut" style="height: 3em;"></span><span class="accent-body" style="left: -0.2875em;"><span class="mord mathbf">~</span></span></span></span></span></span></span></span></span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height: 0.3117em;"><span class="" style="top: -2.55em; margin-right: 0.05em;"><span class="pstrut" style="height: 2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mathnormal mtight">i</span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height: 0.15em;"><span class=""></span></span></span></span></span></span><span class="mord"><span class="mord"><span class="mord"><span class="mord boldsymbol" style="margin-right: 0.0278em;">w</span></span></span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height: 0.3117em;"><span class="" style="top: -2.55em; margin-right: 0.05em;"><span class="pstrut" style="height: 2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mathnormal mtight">i</span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height: 0.15em;"><span class=""></span></span></span></span></span></span></span><span class="mspace" style="margin-right: 0.1667em;"></span><span class="mspace" style="margin-right: 0.1667em;"></span><span class="mord mathrm">s</span><span class="mord">.</span><span class="mord mathrm">t</span><span class="mord">.</span><span class="mord"><span class="mord"><span class="mord"><span class="mord boldsymbol" style="margin-right: 0.0278em;">w</span></span></span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height: 0.8913em;"><span class="" style="top: -2.453em; margin-right: 0.05em;"><span class="pstrut" style="height: 2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight"><span class="mord mathnormal mtight">i</span></span></span></span><span class="" style="top: -3.113em; margin-right: 0.05em;"><span class="pstrut" style="height: 2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight"><span class="mord mathnormal mtight" style="margin-right: 0.1389em;">T</span></span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height: 0.247em;"><span class=""></span></span></span></span></span></span><span class="mord"><span class="mord mathbf">1</span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height: 0.3361em;"><span class="" style="top: -2.55em; margin-left: 0em; margin-right: 0.05em;"><span class="pstrut" style="height: 2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight"><span class="mord mathnormal mtight" style="margin-right: 0.0315em;">k</span><span class="mbin mtight">×</span><span class="mord mtight">1</span></span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height: 0.2083em;"><span class=""></span></span></span></span></span></span><span class="mspace" style="margin-right: 0.2778em;"></span><span class="mrel">=</span><span class="mspace" style="margin-right: 0.2778em;"></span></span><span class="base"><span class="strut" style="height: 1em; vertical-align: -0.25em;"></span><span class="mord">1</span><span class="mspace" style="margin-right: 0.1667em;"></span><span class="minner"><span class="mopen delimcenter" style="top: 0em;">(</span><span class="mord mathnormal">i</span><span class="mspace" style="margin-right: 0.2778em;"></span><span class="mrel">=</span><span class="mspace" style="margin-right: 0.2778em;"></span><span class="mord">1</span><span class="mpunct">,</span><span class="mspace" style="margin-right: 0.1667em;"></span><span class="mord">2</span><span class="mpunct">,</span><span class="mspace" style="margin-right: 0.1667em;"></span><span class="minner">⋯</span><span class="mspace" style="margin-right: 0.1667em;"></span><span class="mspace" style="margin-right: 0.1667em;"></span><span class="mpunct">,</span><span class="mspace" style="margin-right: 0.1667em;"></span><span class="mord mathnormal">m</span><span class="mclose delimcenter" style="top: 0em;">)</span></span></span></span></span></span></span></p>

其中 

      w 
     
    
      i 
     
    
   
     = 
    
    
     
     
       [ 
      
      
       
        
         
          
          
            w 
           
           
           
             i 
            
           
             1 
            
           
          
         
        
        
         
          
          
            w 
           
           
           
             i 
            
           
             2 
            
           
          
         
        
        
         
         
           ⋯ 
          
         
        
        
         
          
          
            w 
           
           
           
             i 
            
           
             k 
            
           
          
         
        
       
      
     
       ] 
      
     
    
      T 
     
    
   
  
    \boldsymbol{w}_i=\left[ <span class="MathJax_Preview" style="color: inherit;"><span class="MJXp-math MJXp-display" id="MJXp-Span-1"><span class="MJXp-mtable" id="MJXp-Span-2"><span><span class="MJXp-mtr" id="MJXp-Span-3" style="vertical-align: baseline;"><span class="MJXp-mtd" id="MJXp-Span-4" style="text-align: center;"><span class="MJXp-msubsup" id="MJXp-Span-5"><span class="MJXp-mi MJXp-italic" id="MJXp-Span-6" style="margin-right: 0.05em;">w</span><span class="MJXp-mrow MJXp-script" id="MJXp-Span-7" style="vertical-align: -0.4em;"><span class="MJXp-mi MJXp-italic" id="MJXp-Span-8">i</span><span class="MJXp-mn" id="MJXp-Span-9">1</span></span></span></span><span class="MJXp-mtd" id="MJXp-Span-10" style="padding-left: 1em; text-align: center;"><span class="MJXp-msubsup" id="MJXp-Span-11"><span class="MJXp-mi MJXp-italic" id="MJXp-Span-12" style="margin-right: 0.05em;">w</span><span class="MJXp-mrow MJXp-script" id="MJXp-Span-13" style="vertical-align: -0.4em;"><span class="MJXp-mi MJXp-italic" id="MJXp-Span-14">i</span><span class="MJXp-mn" id="MJXp-Span-15">2</span></span></span></span><span class="MJXp-mtd" id="MJXp-Span-16" style="padding-left: 1em; text-align: center;"><span class="MJXp-mo" id="MJXp-Span-17" style="margin-left: 0em; margin-right: 0em;">⋯</span></span><span class="MJXp-mtd" id="MJXp-Span-18" style="padding-left: 1em; text-align: center;"><span class="MJXp-msubsup" id="MJXp-Span-19"><span class="MJXp-mi MJXp-italic" id="MJXp-Span-20" style="margin-right: 0.05em;">w</span><span class="MJXp-mrow MJXp-script" id="MJXp-Span-21" style="vertical-align: -0.4em;"><span class="MJXp-mi MJXp-italic" id="MJXp-Span-22">i</span><span class="MJXp-mi MJXp-italic" id="MJXp-Span-23">k</span></span></span></span></span></span></span></span></span><script type="math/tex; mode=display" id="MathJax-Element-1">\begin{matrix} w_{i1}& w_{i2}& \cdots& w_{ik}\\\end{matrix}</script> \right] ^T 
   
  
</span><span class="katex-html"><span class="base"><span class="strut" style="height: 0.5944em; vertical-align: -0.15em;"></span><span class="mord"><span class="mord"><span class="mord"><span class="mord boldsymbol" style="margin-right: 0.0278em;">w</span></span></span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height: 0.3117em;"><span class="" style="top: -2.55em; margin-right: 0.05em;"><span class="pstrut" style="height: 2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mathnormal mtight">i</span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height: 0.15em;"><span class=""></span></span></span></span></span></span><span class="mspace" style="margin-right: 0.2778em;"></span><span class="mrel">=</span><span class="mspace" style="margin-right: 0.2778em;"></span></span><span class="base"><span class="strut" style="height: 1.4312em; vertical-align: -0.35em;"></span><span class="minner"><span class="minner"><span class="mopen delimcenter" style="top: 0em;"><span class="delimsizing size1">[</span></span><span class="mord"><span class="mtable"><span class="col-align-c"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height: 0.85em;"><span class="" style="top: -3.01em;"><span class="pstrut" style="height: 3em;"></span><span class="mord"><span class="mord"><span class="mord mathnormal" style="margin-right: 0.0269em;">w</span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height: 0.3117em;"><span class="" style="top: -2.55em; margin-left: -0.0269em; margin-right: 0.05em;"><span class="pstrut" style="height: 2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight"><span class="mord mathnormal mtight">i</span><span class="mord mtight">1</span></span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height: 0.15em;"><span class=""></span></span></span></span></span></span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height: 0.35em;"><span class=""></span></span></span></span></span><span class="arraycolsep" style="width: 0.5em;"></span><span class="arraycolsep" style="width: 0.5em;"></span><span class="col-align-c"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height: 0.85em;"><span class="" style="top: -3.01em;"><span class="pstrut" style="height: 3em;"></span><span class="mord"><span class="mord"><span class="mord mathnormal" style="margin-right: 0.0269em;">w</span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height: 0.3117em;"><span class="" style="top: -2.55em; margin-left: -0.0269em; margin-right: 0.05em;"><span class="pstrut" style="height: 2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight"><span class="mord mathnormal mtight">i</span><span class="mord mtight">2</span></span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height: 0.15em;"><span class=""></span></span></span></span></span></span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height: 0.35em;"><span class=""></span></span></span></span></span><span class="arraycolsep" style="width: 0.5em;"></span><span class="arraycolsep" style="width: 0.5em;"></span><span class="col-align-c"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height: 0.85em;"><span class="" style="top: -3.01em;"><span class="pstrut" style="height: 3em;"></span><span class="mord"><span class="minner">⋯</span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height: 0.35em;"><span class=""></span></span></span></span></span><span class="arraycolsep" style="width: 0.5em;"></span><span class="arraycolsep" style="width: 0.5em;"></span><span class="col-align-c"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height: 0.85em;"><span class="" style="top: -3.01em;"><span class="pstrut" style="height: 3em;"></span><span class="mord"><span class="mord"><span class="mord mathnormal" style="margin-right: 0.0269em;">w</span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height: 0.3361em;"><span class="" style="top: -2.55em; margin-left: -0.0269em; margin-right: 0.05em;"><span class="pstrut" style="height: 2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight"><span class="mord mathnormal mtight" style="margin-right: 0.0315em;">ik</span></span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height: 0.15em;"><span class=""></span></span></span></span></span></span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height: 0.35em;"><span class=""></span></span></span></span></span></span></span><span class="mclose delimcenter" style="top: 0em;"><span class="delimsizing size1">]</span></span></span><span class="msupsub"><span class="vlist-t"><span class="vlist-r"><span class="vlist" style="height: 1.0812em;"><span class="" style="top: -3.3029em; margin-right: 0.05em;"><span class="pstrut" style="height: 2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mathnormal mtight" style="margin-right: 0.1389em;">T</span></span></span></span></span></span></span></span></span></span></span></span>，<span class="katex--inline"><span class="katex"><span class="katex-mathml"> 
 
  
   
    
     
      
      
        X 
       
      
        ~ 
       
      
     
    
      i 
     
    
   
     = 
    
    
    
      [ 
     
     
      
       
        
         
          
          
            x 
           
          
            i 
           
          
         
           − 
          
          
          
            x 
           
           
           
             j 
            
           
             1 
            
           
          
         
        
       
       
        
         
          
          
            x 
           
          
            i 
           
          
         
           − 
          
          
          
            x 
           
           
           
             j 
            
           
             2 
            
           
          
         
        
       
       
        
        
          ⋯ 
         
        
       
       
        
         
          
          
            x 
           
          
            i 
           
          
         
           − 
          
          
          
            x 
           
           
           
             j 
            
           
             k 
            
           
          
         
        
       
      
     
    
      ] 
     
    
   
     , 
    
   
     j 
    
   
     ∈ 
    
    
    
      Q 
     
    
      i 
     
    
      k 
     
    
   
  
    \boldsymbol{\tilde{X}}_i=\left[ <script type="math/tex; mode=display">\begin{matrix} \boldsymbol{x}_i-\boldsymbol{x}_{j_1}& \boldsymbol{x}_i-\boldsymbol{x}_{j_2}& \cdots& \boldsymbol{x}_i-\boldsymbol{x}_{j_k}\\\end{matrix}</script> \right] , j\in Q_{i}^{k} 
   
  
</span><span class="katex-html"><span class="base"><span class="strut" style="height: 1.0995em; vertical-align: -0.15em;"></span><span class="mord"><span class="mord"><span class="mord"><span class="mord accent"><span class="vlist-t"><span class="vlist-r"><span class="vlist" style="height: 0.9495em;"><span class="" style="top: -3em;"><span class="pstrut" style="height: 3em;"></span><span class="mord boldsymbol" style="margin-right: 0.0778em;">X</span></span><span class="" style="top: -3.6051em;"><span class="pstrut" style="height: 3em;"></span><span class="accent-body" style="left: -0.2875em;"><span class="mord mathbf">~</span></span></span></span></span></span></span></span></span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height: 0.3117em;"><span class="" style="top: -2.55em; margin-right: 0.05em;"><span class="pstrut" style="height: 2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mathnormal mtight">i</span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height: 0.15em;"><span class=""></span></span></span></span></span></span><span class="mspace" style="margin-right: 0.2778em;"></span><span class="mrel">=</span><span class="mspace" style="margin-right: 0.2778em;"></span></span><span class="base"><span class="strut" style="height: 1.2em; vertical-align: -0.35em;"></span><span class="minner"><span class="mopen delimcenter" style="top: 0em;"><span class="delimsizing size1">[</span></span><span class="mord"><span class="mtable"><span class="col-align-c"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height: 0.85em;"><span class="" style="top: -3.01em;"><span class="pstrut" style="height: 3em;"></span><span class="mord"><span class="mord"><span class="mord"><span class="mord"><span class="mord boldsymbol">x</span></span></span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height: 0.3117em;"><span class="" style="top: -2.55em; margin-right: 0.05em;"><span class="pstrut" style="height: 2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mathnormal mtight">i</span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height: 0.15em;"><span class=""></span></span></span></span></span></span><span class="mspace" style="margin-right: 0.2222em;"></span><span class="mbin">−</span><span class="mspace" style="margin-right: 0.2222em;"></span><span class="mord"><span class="mord"><span class="mord"><span class="mord boldsymbol">x</span></span></span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height: 0.3117em;"><span class="" style="top: -2.55em; margin-right: 0.05em;"><span class="pstrut" style="height: 2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight"><span class="mord mtight"><span class="mord mathnormal mtight" style="margin-right: 0.0572em;">j</span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height: 0.3173em;"><span class="" style="top: -2.357em; margin-left: -0.0572em; margin-right: 0.0714em;"><span class="pstrut" style="height: 2.5em;"></span><span class="sizing reset-size3 size1 mtight"><span class="mord mtight">1</span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height: 0.143em;"><span class=""></span></span></span></span></span></span></span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height: 0.2861em;"><span class=""></span></span></span></span></span></span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height: 0.35em;"><span class=""></span></span></span></span></span><span class="arraycolsep" style="width: 0.5em;"></span><span class="arraycolsep" style="width: 0.5em;"></span><span class="col-align-c"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height: 0.85em;"><span class="" style="top: -3.01em;"><span class="pstrut" style="height: 3em;"></span><span class="mord"><span class="mord"><span class="mord"><span class="mord"><span class="mord boldsymbol">x</span></span></span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height: 0.3117em;"><span class="" style="top: -2.55em; margin-right: 0.05em;"><span class="pstrut" style="height: 2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mathnormal mtight">i</span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height: 0.15em;"><span class=""></span></span></span></span></span></span><span class="mspace" style="margin-right: 0.2222em;"></span><span class="mbin">−</span><span class="mspace" style="margin-right: 0.2222em;"></span><span class="mord"><span class="mord"><span class="mord"><span class="mord boldsymbol">x</span></span></span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height: 0.3117em;"><span class="" style="top: -2.55em; margin-right: 0.05em;"><span class="pstrut" style="height: 2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight"><span class="mord mtight"><span class="mord mathnormal mtight" style="margin-right: 0.0572em;">j</span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height: 0.3173em;"><span class="" style="top: -2.357em; margin-left: -0.0572em; margin-right: 0.0714em;"><span class="pstrut" style="height: 2.5em;"></span><span class="sizing reset-size3 size1 mtight"><span class="mord mtight">2</span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height: 0.143em;"><span class=""></span></span></span></span></span></span></span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height: 0.2861em;"><span class=""></span></span></span></span></span></span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height: 0.35em;"><span class=""></span></span></span></span></span><span class="arraycolsep" style="width: 0.5em;"></span><span class="arraycolsep" style="width: 0.5em;"></span><span class="col-align-c"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height: 0.85em;"><span class="" style="top: -3.01em;"><span class="pstrut" style="height: 3em;"></span><span class="mord"><span class="minner">⋯</span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height: 0.35em;"><span class=""></span></span></span></span></span><span class="arraycolsep" style="width: 0.5em;"></span><span class="arraycolsep" style="width: 0.5em;"></span><span class="col-align-c"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height: 0.85em;"><span class="" style="top: -3.01em;"><span class="pstrut" style="height: 3em;"></span><span class="mord"><span class="mord"><span class="mord"><span class="mord"><span class="mord boldsymbol">x</span></span></span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height: 0.3117em;"><span class="" style="top: -2.55em; margin-right: 0.05em;"><span class="pstrut" style="height: 2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mathnormal mtight">i</span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height: 0.15em;"><span class=""></span></span></span></span></span></span><span class="mspace" style="margin-right: 0.2222em;"></span><span class="mbin">−</span><span class="mspace" style="margin-right: 0.2222em;"></span><span class="mord"><span class="mord"><span class="mord"><span class="mord boldsymbol">x</span></span></span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height: 0.3117em;"><span class="" style="top: -2.55em; margin-right: 0.05em;"><span class="pstrut" style="height: 2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight"><span class="mord mtight"><span class="mord mathnormal mtight" style="margin-right: 0.0572em;">j</span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height: 0.3448em;"><span class="" style="top: -2.3488em; margin-left: -0.0572em; margin-right: 0.0714em;"><span class="pstrut" style="height: 2.5em;"></span><span class="sizing reset-size3 size1 mtight"><span class="mord mathnormal mtight" style="margin-right: 0.0315em;">k</span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height: 0.1512em;"><span class=""></span></span></span></span></span></span></span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height: 0.2861em;"><span class=""></span></span></span></span></span></span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height: 0.35em;"><span class=""></span></span></span></span></span></span></span><span class="mclose delimcenter" style="top: 0em;"><span class="delimsizing size1">]</span></span></span><span class="mspace" style="margin-right: 0.1667em;"></span><span class="mpunct">,</span><span class="mspace" style="margin-right: 0.1667em;"></span><span class="mord mathnormal" style="margin-right: 0.0572em;">j</span><span class="mspace" style="margin-right: 0.2778em;"></span><span class="mrel">∈</span><span class="mspace" style="margin-right: 0.2778em;"></span></span><span class="base"><span class="strut" style="height: 1.1078em; vertical-align: -0.2587em;"></span><span class="mord"><span class="mord mathnormal">Q</span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height: 0.8491em;"><span class="" style="top: -2.4413em; margin-left: 0em; margin-right: 0.05em;"><span class="pstrut" style="height: 2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight"><span class="mord mathnormal mtight">i</span></span></span></span><span class="" style="top: -3.063em; margin-right: 0.05em;"><span class="pstrut" style="height: 2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight"><span class="mord mathnormal mtight" style="margin-right: 0.0315em;">k</span></span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height: 0.2587em;"><span class=""></span></span></span></span></span></span></span></span></span></span>，应用拉格朗日乘子法可得</p> 


      L 
     
     
     
       ( 
      
      
      
        w 
       
      
        1 
       
      
     
       , 
      
      
      
        w 
       
      
        2 
       
      
     
       , 
      
     
       ⋯ 
       
     
       , 
      
      
      
        w 
       
      
        m 
       
      
     
       , 
      
     
       λ 
      
     
       ) 
      
     
    
      = 
     
     
     
       ∑ 
      
      
      
        i 
       
      
        = 
       
      
        1 
       
      
     
       m 
      
     
     
      
      
        w 
       
      
        i 
       
      
        T 
       
      
      
       
        
        
          X 
         
        
          ~ 
         
        
       
      
        i 
       
      
        T 
       
      
      
       
        
        
          X 
         
        
          ~ 
         
        
       
      
        i 
       
      
      
      
        w 
       
      
        i 
       
      
     
    
      + 
     
    
      λ 
     
     
     
       ( 
      
      
      
        w 
       
      
        i 
       
      
        T 
       
      
      
      
        1 
       
       
       
         k 
        
       
         × 
        
       
         1 
        
       
      
     
       − 
      
     
       1 
      
     
       ) 
      
     
    
   
     L\left( \boldsymbol{w}_1,\boldsymbol{w}_2,\cdots ,\boldsymbol{w}_m,\lambda \right) =\sum_{i=1}^m{\boldsymbol{w}_{i}^{T}\boldsymbol{\tilde{X}}_{i}^{T}\boldsymbol{\tilde{X}}_i\boldsymbol{w}_i}+\lambda \left( \boldsymbol{w}_{i}^{T}\mathbf{1}_{k\times 1}-1 \right) 
    
   
 </span><span class="katex-html"><span class="base"><span class="strut" style="height: 1em; vertical-align: -0.25em;"></span><span class="mord mathnormal">L</span><span class="mspace" style="margin-right: 0.1667em;"></span><span class="minner"><span class="mopen delimcenter" style="top: 0em;">(</span><span class="mord"><span class="mord"><span class="mord"><span class="mord boldsymbol" style="margin-right: 0.0278em;">w</span></span></span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height: 0.3011em;"><span class="" style="top: -2.55em; margin-right: 0.05em;"><span class="pstrut" style="height: 2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight">1</span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height: 0.15em;"><span class=""></span></span></span></span></span></span><span class="mpunct">,</span><span class="mspace" style="margin-right: 0.1667em;"></span><span class="mord"><span class="mord"><span class="mord"><span class="mord boldsymbol" style="margin-right: 0.0278em;">w</span></span></span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height: 0.3011em;"><span class="" style="top: -2.55em; margin-right: 0.05em;"><span class="pstrut" style="height: 2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight">2</span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height: 0.15em;"><span class=""></span></span></span></span></span></span><span class="mpunct">,</span><span class="mspace" style="margin-right: 0.1667em;"></span><span class="minner">⋯</span><span class="mspace" style="margin-right: 0.1667em;"></span><span class="mspace" style="margin-right: 0.1667em;"></span><span class="mpunct">,</span><span class="mspace" style="margin-right: 0.1667em;"></span><span class="mord"><span class="mord"><span class="mord"><span class="mord boldsymbol" style="margin-right: 0.0278em;">w</span></span></span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height: 0.1514em;"><span class="" style="top: -2.55em; margin-right: 0.05em;"><span class="pstrut" style="height: 2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mathnormal mtight">m</span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height: 0.15em;"><span class=""></span></span></span></span></span></span><span class="mpunct">,</span><span class="mspace" style="margin-right: 0.1667em;"></span><span class="mord mathnormal">λ</span><span class="mclose delimcenter" style="top: 0em;">)</span></span><span class="mspace" style="margin-right: 0.2778em;"></span><span class="mrel">=</span><span class="mspace" style="margin-right: 0.2778em;"></span></span><span class="base"><span class="strut" style="height: 2.9291em; vertical-align: -1.2777em;"></span><span class="mop op-limits"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height: 1.6514em;"><span class="" style="top: -1.8723em; margin-left: 0em;"><span class="pstrut" style="height: 3.05em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight"><span class="mord mathnormal mtight">i</span><span class="mrel mtight">=</span><span class="mord mtight">1</span></span></span></span><span class="" style="top: -3.05em;"><span class="pstrut" style="height: 3.05em;"></span><span class=""><span class="mop op-symbol large-op">∑</span></span></span><span class="" style="top: -4.3em; margin-left: 0em;"><span class="pstrut" style="height: 3.05em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mathnormal mtight">m</span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height: 1.2777em;"><span class=""></span></span></span></span></span><span class="mspace" style="margin-right: 0.1667em;"></span><span class="mord"><span class="mord"><span class="mord"><span class="mord"><span class="mord boldsymbol" style="margin-right: 0.0278em;">w</span></span></span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height: 0.8913em;"><span class="" style="top: -2.453em; margin-right: 0.05em;"><span class="pstrut" style="height: 2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight"><span class="mord mathnormal mtight">i</span></span></span></span><span class="" style="top: -3.113em; margin-right: 0.05em;"><span class="pstrut" style="height: 2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight"><span class="mord mathnormal mtight" style="margin-right: 0.1389em;">T</span></span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height: 0.247em;"><span class=""></span></span></span></span></span></span><span class="mord"><span class="mord"><span class="mord"><span class="mord accent"><span class="vlist-t"><span class="vlist-r"><span class="vlist" style="height: 0.9495em;"><span class="" style="top: -3em;"><span class="pstrut" style="height: 3em;"></span><span class="mord boldsymbol" style="margin-right: 0.0778em;">X</span></span><span class="" style="top: -3.6051em;"><span class="pstrut" style="height: 3em;"></span><span class="accent-body" style="left: -0.2875em;"><span class="mord mathbf">~</span></span></span></span></span></span></span></span></span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height: 1.1808em;"><span class="" style="top: -2.453em; margin-right: 0.05em;"><span class="pstrut" style="height: 2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight"><span class="mord mathnormal mtight">i</span></span></span></span><span class="" style="top: -3.4024em; margin-right: 0.05em;"><span class="pstrut" style="height: 2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight"><span class="mord mathnormal mtight" style="margin-right: 0.1389em;">T</span></span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height: 0.247em;"><span class=""></span></span></span></span></span></span><span class="mord"><span class="mord"><span class="mord"><span class="mord accent"><span class="vlist-t"><span class="vlist-r"><span class="vlist" style="height: 0.9495em;"><span class="" style="top: -3em;"><span class="pstrut" style="height: 3em;"></span><span class="mord boldsymbol" style="margin-right: 0.0778em;">X</span></span><span class="" style="top: -3.6051em;"><span class="pstrut" style="height: 3em;"></span><span class="accent-body" style="left: -0.2875em;"><span class="mord mathbf">~</span></span></span></span></span></span></span></span></span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height: 0.3117em;"><span class="" style="top: -2.55em; margin-right: 0.05em;"><span class="pstrut" style="height: 2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mathnormal mtight">i</span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height: 0.15em;"><span class=""></span></span></span></span></span></span><span class="mord"><span class="mord"><span class="mord"><span class="mord boldsymbol" style="margin-right: 0.0278em;">w</span></span></span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height: 0.3117em;"><span class="" style="top: -2.55em; margin-right: 0.05em;"><span class="pstrut" style="height: 2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mathnormal mtight">i</span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height: 0.15em;"><span class=""></span></span></span></span></span></span></span><span class="mspace" style="margin-right: 0.2222em;"></span><span class="mbin">+</span><span class="mspace" style="margin-right: 0.2222em;"></span></span><span class="base"><span class="strut" style="height: 1.2413em; vertical-align: -0.35em;"></span><span class="mord mathnormal">λ</span><span class="mspace" style="margin-right: 0.1667em;"></span><span class="minner"><span class="mopen delimcenter" style="top: 0em;"><span class="delimsizing size1">(</span></span><span class="mord"><span class="mord"><span class="mord"><span class="mord boldsymbol" style="margin-right: 0.0278em;">w</span></span></span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height: 0.8913em;"><span class="" style="top: -2.453em; margin-right: 0.05em;"><span class="pstrut" style="height: 2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight"><span class="mord mathnormal mtight">i</span></span></span></span><span class="" style="top: -3.113em; margin-right: 0.05em;"><span class="pstrut" style="height: 2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight"><span class="mord mathnormal mtight" style="margin-right: 0.1389em;">T</span></span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height: 0.247em;"><span class=""></span></span></span></span></span></span><span class="mord"><span class="mord mathbf">1</span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height: 0.3361em;"><span class="" style="top: -2.55em; margin-left: 0em; margin-right: 0.05em;"><span class="pstrut" style="height: 2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight"><span class="mord mathnormal mtight" style="margin-right: 0.0315em;">k</span><span class="mbin mtight">×</span><span class="mord mtight">1</span></span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height: 0.2083em;"><span class=""></span></span></span></span></span></span><span class="mspace" style="margin-right: 0.2222em;"></span><span class="mbin">−</span><span class="mspace" style="margin-right: 0.2222em;"></span><span class="mord">1</span><span class="mclose delimcenter" style="top: 0em;"><span class="delimsizing size1">)</span></span></span></span></span></span></span></span></p>

对 

      w 
     
    
      i 
     
    
   
  
    \boldsymbol{w}_i 
   
  
</span><span class="katex-html"><span class="base"><span class="strut" style="height: 0.5944em; vertical-align: -0.15em;"></span><span class="mord"><span class="mord"><span class="mord"><span class="mord boldsymbol" style="margin-right: 0.0278em;">w</span></span></span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height: 0.3117em;"><span class="" style="top: -2.55em; margin-right: 0.05em;"><span class="pstrut" style="height: 2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mathnormal mtight">i</span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height: 0.15em;"><span class=""></span></span></span></span></span></span></span></span></span></span>求梯度，并令之为0可得</p> 


           ∇ 
          
          
          
            w 
           
          
            i 
           
          
         
        
          L 
         
         
         
           ( 
          
          
          
            w 
           
          
            1 
           
          
         
           , 
          
          
          
            w 
           
          
            2 
           
          
         
           , 
          
         
           ⋯ 
           
         
           , 
          
          
          
            w 
           
          
            m 
           
          
         
           , 
          
         
           λ 
          
         
           ) 
          
         
        
       
      
      
       
        
         
        
          = 
         
         
         
           w 
          
         
           i 
          
         
           T 
          
         
         
          
           
           
             X 
            
           
             ~ 
            
           
          
         
           i 
          
         
           T 
          
         
         
          
           
           
             X 
            
           
             ~ 
            
           
          
         
           i 
          
         
         
         
           w 
          
         
           i 
          
         
        
          + 
         
        
          λ 
         
         
         
           ( 
          
          
          
            w 
           
          
            i 
           
          
            T 
           
          
          
          
            1 
           
           
           
             k 
            
           
             × 
            
           
             1 
            
           
          
         
           − 
          
         
           1 
          
         
           ) 
          
         
        
       
      
     
     
      
       
        
       
      
      
       
        
         
        
          = 
         
        
          2 
         
         
          
           
           
             X 
            
           
             ~ 
            
           
          
         
           i 
          
         
           T 
          
         
         
          
           
           
             X 
            
           
             ~ 
            
           
          
         
           i 
          
         
         
         
           w 
          
         
           i 
          
         
        
          + 
         
        
          λ 
         
         
         
           1 
          
          
          
            k 
           
          
            × 
           
          
            1 
           
          
         
        
       
      
     
     
      
       
        
       
      
      
       
        
         
        
          = 
         
        
          0 
         
        
       
      
     
     
      
       
        
        
          ⇒ 
           
         
         
           w 
          
         
           i 
          
         
        
       
      
      
       
        
         
        
          = 
         
        
          − 
         
         
         
           1 
          
         
           2 
          
         
        
          λ 
         
         
          
          
            ( 
           
           
            
             
             
               X 
              
             
               ~ 
              
             
            
           
             i 
            
           
             T 
            
           
           
            
             
             
               X 
              
             
               ~ 
              
             
            
           
             i 
            
           
          
            ) 
           
          
          
          
            − 
           
          
            1 
           
          
         
         
         
           1 
          
          
          
            k 
           
          
            × 
           
          
            1 
           
          
         
        
       
      
     
    
   
     <script type="math/tex; mode=display">\begin{aligned}\nabla _{\boldsymbol{w}_i}L\left( \boldsymbol{w}_1,\boldsymbol{w}_2,\cdots ,\boldsymbol{w}_m,\lambda \right) &=\boldsymbol{w}_{i}^{T}\boldsymbol{\tilde{X}}_{i}^{T}\boldsymbol{\tilde{X}}_i\boldsymbol{w}_i+\lambda \left( \boldsymbol{w}_{i}^{T}\mathbf{1}_{k\times 1}-1 \right) \\&=2\boldsymbol{\tilde{X}}_{i}^{T}\boldsymbol{\tilde{X}}_i\boldsymbol{w}_i+\lambda \mathbf{1}_{k\times 1}\\&=0\\\Rightarrow \,\, \boldsymbol{w}_i&=-\frac{1}{2}\lambda \left( \boldsymbol{\tilde{X}}_{i}^{T}\boldsymbol{\tilde{X}}_i \right) ^{-1}\mathbf{1}_{k\times 1}\end{aligned}</script> 
    
   
 </span><span class="katex-html"><span class="base"><span class="strut" style="height: 7.5524em; vertical-align: -3.5262em;"></span><span class="mord"><span class="mtable"><span class="col-align-r"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height: 4.0262em;"><span class="" style="top: -6.2302em;"><span class="pstrut" style="height: 3.3848em;"></span><span class="mord"><span class="mord"><span class="mord">∇</span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height: 0.1611em;"><span class="" style="top: -2.55em; margin-left: 0em; margin-right: 0.05em;"><span class="pstrut" style="height: 2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight"><span class="mord mtight"><span class="mord mtight"><span class="mord mtight"><span class="mord boldsymbol mtight" style="margin-right: 0.0278em;">w</span></span></span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height: 0.3281em;"><span class="" style="top: -2.357em; margin-right: 0.0714em;"><span class="pstrut" style="height: 2.5em;"></span><span class="sizing reset-size3 size1 mtight"><span class="mord mathnormal mtight">i</span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height: 0.143em;"><span class=""></span></span></span></span></span></span></span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height: 0.2501em;"><span class=""></span></span></span></span></span></span><span class="mord mathnormal">L</span><span class="mspace" style="margin-right: 0.1667em;"></span><span class="minner"><span class="mopen delimcenter" style="top: 0em;">(</span><span class="mord"><span class="mord"><span class="mord"><span class="mord boldsymbol" style="margin-right: 0.0278em;">w</span></span></span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height: 0.3011em;"><span class="" style="top: -2.55em; margin-right: 0.05em;"><span class="pstrut" style="height: 2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight">1</span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height: 0.15em;"><span class=""></span></span></span></span></span></span><span class="mpunct">,</span><span class="mspace" style="margin-right: 0.1667em;"></span><span class="mord"><span class="mord"><span class="mord"><span class="mord boldsymbol" style="margin-right: 0.0278em;">w</span></span></span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height: 0.3011em;"><span class="" style="top: -2.55em; margin-right: 0.05em;"><span class="pstrut" style="height: 2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight">2</span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height: 0.15em;"><span class=""></span></span></span></span></span></span><span class="mpunct">,</span><span class="mspace" style="margin-right: 0.1667em;"></span><span class="minner">⋯</span><span class="mspace" style="margin-right: 0.1667em;"></span><span class="mspace" style="margin-right: 0.1667em;"></span><span class="mpunct">,</span><span class="mspace" style="margin-right: 0.1667em;"></span><span class="mord"><span class="mord"><span class="mord"><span class="mord boldsymbol" style="margin-right: 0.0278em;">w</span></span></span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height: 0.1514em;"><span class="" style="top: -2.55em; margin-right: 0.05em;"><span class="pstrut" style="height: 2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mathnormal mtight">m</span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height: 0.15em;"><span class=""></span></span></span></span></span></span><span class="mpunct">,</span><span class="mspace" style="margin-right: 0.1667em;"></span><span class="mord mathnormal">λ</span><span class="mclose delimcenter" style="top: 0em;">)</span></span></span></span><span class="" style="top: -4.3894em;"><span class="pstrut" style="height: 3.3848em;"></span><span class="mord"></span></span><span class="" style="top: -2.8894em;"><span class="pstrut" style="height: 3.3848em;"></span><span class="mord"></span></span><span class="" style="top: -0.8446em;"><span class="pstrut" style="height: 3.3848em;"></span><span class="mord"><span class="mrel">⇒</span><span class="mspace" style="margin-right: 0.1667em;"></span><span class="mspace" style="margin-right: 0.1667em;"></span><span class="mspace" style="margin-right: 0.2778em;"></span><span class="mord"><span class="mord"><span class="mord"><span class="mord boldsymbol" style="margin-right: 0.0278em;">w</span></span></span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height: 0.3117em;"><span class="" style="top: -2.55em; margin-right: 0.05em;"><span class="pstrut" style="height: 2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mathnormal mtight">i</span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height: 0.15em;"><span class=""></span></span></span></span></span></span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height: 3.5262em;"><span class=""></span></span></span></span></span><span class="col-align-l"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height: 4.0262em;"><span class="" style="top: -6.2302em;"><span class="pstrut" style="height: 3.3848em;"></span><span class="mord"><span class="mord"></span><span class="mspace" style="margin-right: 0.2778em;"></span><span class="mrel">=</span><span class="mspace" style="margin-right: 0.2778em;"></span><span class="mord"><span class="mord"><span class="mord"><span class="mord boldsymbol" style="margin-right: 0.0278em;">w</span></span></span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height: 0.8913em;"><span class="" style="top: -2.453em; margin-right: 0.05em;"><span class="pstrut" style="height: 2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight"><span class="mord mathnormal mtight">i</span></span></span></span><span class="" style="top: -3.113em; margin-right: 0.05em;"><span class="pstrut" style="height: 2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight"><span class="mord mathnormal mtight" style="margin-right: 0.1389em;">T</span></span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height: 0.247em;"><span class=""></span></span></span></span></span></span><span class="mord"><span class="mord"><span class="mord"><span class="mord accent"><span class="vlist-t"><span class="vlist-r"><span class="vlist" style="height: 0.9495em;"><span class="" style="top: -3em;"><span class="pstrut" style="height: 3em;"></span><span class="mord boldsymbol" style="margin-right: 0.0778em;">X</span></span><span class="" style="top: -3.6051em;"><span class="pstrut" style="height: 3em;"></span><span class="accent-body" style="left: -0.2875em;"><span class="mord mathbf">~</span></span></span></span></span></span></span></span></span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height: 1.1808em;"><span class="" style="top: -2.453em; margin-right: 0.05em;"><span class="pstrut" style="height: 2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight"><span class="mord mathnormal mtight">i</span></span></span></span><span class="" style="top: -3.4024em; margin-right: 0.05em;"><span class="pstrut" style="height: 2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight"><span class="mord mathnormal mtight" style="margin-right: 0.1389em;">T</span></span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height: 0.247em;"><span class=""></span></span></span></span></span></span><span class="mord"><span class="mord"><span class="mord"><span class="mord accent"><span class="vlist-t"><span class="vlist-r"><span class="vlist" style="height: 0.9495em;"><span class="" style="top: -3em;"><span class="pstrut" style="height: 3em;"></span><span class="mord boldsymbol" style="margin-right: 0.0778em;">X</span></span><span class="" style="top: -3.6051em;"><span class="pstrut" style="height: 3em;"></span><span class="accent-body" style="left: -0.2875em;"><span class="mord mathbf">~</span></span></span></span></span></span></span></span></span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height: 0.3117em;"><span class="" style="top: -2.55em; margin-right: 0.05em;"><span class="pstrut" style="height: 2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mathnormal mtight">i</span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height: 0.15em;"><span class=""></span></span></span></span></span></span><span class="mord"><span class="mord"><span class="mord"><span class="mord boldsymbol" style="margin-right: 0.0278em;">w</span></span></span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height: 0.3117em;"><span class="" style="top: -2.55em; margin-right: 0.05em;"><span class="pstrut" style="height: 2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mathnormal mtight">i</span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height: 0.15em;"><span class=""></span></span></span></span></span></span><span class="mspace" style="margin-right: 0.2222em;"></span><span class="mbin">+</span><span class="mspace" style="margin-right: 0.2222em;"></span><span class="mord mathnormal">λ</span><span class="mspace" style="margin-right: 0.1667em;"></span><span class="minner"><span class="mopen delimcenter" style="top: 0em;"><span class="delimsizing size1">(</span></span><span class="mord"><span class="mord"><span class="mord"><span class="mord boldsymbol" style="margin-right: 0.0278em;">w</span></span></span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height: 0.8913em;"><span class="" style="top: -2.453em; margin-right: 0.05em;"><span class="pstrut" style="height: 2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight"><span class="mord mathnormal mtight">i</span></span></span></span><span class="" style="top: -3.113em; margin-right: 0.05em;"><span class="pstrut" style="height: 2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight"><span class="mord mathnormal mtight" style="margin-right: 0.1389em;">T</span></span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height: 0.247em;"><span class=""></span></span></span></span></span></span><span class="mord"><span class="mord mathbf">1</span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height: 0.3361em;"><span class="" style="top: -2.55em; margin-left: 0em; margin-right: 0.05em;"><span class="pstrut" style="height: 2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight"><span class="mord mathnormal mtight" style="margin-right: 0.0315em;">k</span><span class="mbin mtight">×</span><span class="mord mtight">1</span></span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height: 0.2083em;"><span class=""></span></span></span></span></span></span><span class="mspace" style="margin-right: 0.2222em;"></span><span class="mbin">−</span><span class="mspace" style="margin-right: 0.2222em;"></span><span class="mord">1</span><span class="mclose delimcenter" style="top: 0em;"><span class="delimsizing size1">)</span></span></span></span></span><span class="" style="top: -4.3894em;"><span class="pstrut" style="height: 3.3848em;"></span><span class="mord"><span class="mord"></span><span class="mspace" style="margin-right: 0.2778em;"></span><span class="mrel">=</span><span class="mspace" style="margin-right: 0.2778em;"></span><span class="mord">2</span><span class="mord"><span class="mord"><span class="mord"><span class="mord accent"><span class="vlist-t"><span class="vlist-r"><span class="vlist" style="height: 0.9495em;"><span class="" style="top: -3em;"><span class="pstrut" style="height: 3em;"></span><span class="mord boldsymbol" style="margin-right: 0.0778em;">X</span></span><span class="" style="top: -3.6051em;"><span class="pstrut" style="height: 3em;"></span><span class="accent-body" style="left: -0.2875em;"><span class="mord mathbf">~</span></span></span></span></span></span></span></span></span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height: 1.1808em;"><span class="" style="top: -2.453em; margin-right: 0.05em;"><span class="pstrut" style="height: 2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight"><span class="mord mathnormal mtight">i</span></span></span></span><span class="" style="top: -3.4024em; margin-right: 0.05em;"><span class="pstrut" style="height: 2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight"><span class="mord mathnormal mtight" style="margin-right: 0.1389em;">T</span></span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height: 0.247em;"><span class=""></span></span></span></span></span></span><span class="mord"><span class="mord"><span class="mord"><span class="mord accent"><span class="vlist-t"><span class="vlist-r"><span class="vlist" style="height: 0.9495em;"><span class="" style="top: -3em;"><span class="pstrut" style="height: 3em;"></span><span class="mord boldsymbol" style="margin-right: 0.0778em;">X</span></span><span class="" style="top: -3.6051em;"><span class="pstrut" style="height: 3em;"></span><span class="accent-body" style="left: -0.2875em;"><span class="mord mathbf">~</span></span></span></span></span></span></span></span></span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height: 0.3117em;"><span class="" style="top: -2.55em; margin-right: 0.05em;"><span class="pstrut" style="height: 2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mathnormal mtight">i</span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height: 0.15em;"><span class=""></span></span></span></span></span></span><span class="mord"><span class="mord"><span class="mord"><span class="mord boldsymbol" style="margin-right: 0.0278em;">w</span></span></span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height: 0.3117em;"><span class="" style="top: -2.55em; margin-right: 0.05em;"><span class="pstrut" style="height: 2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mathnormal mtight">i</span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height: 0.15em;"><span class=""></span></span></span></span></span></span><span class="mspace" style="margin-right: 0.2222em;"></span><span class="mbin">+</span><span class="mspace" style="margin-right: 0.2222em;"></span><span class="mord mathnormal">λ</span><span class="mord"><span class="mord mathbf">1</span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height: 0.3361em;"><span class="" style="top: -2.55em; margin-left: 0em; margin-right: 0.05em;"><span class="pstrut" style="height: 2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight"><span class="mord mathnormal mtight" style="margin-right: 0.0315em;">k</span><span class="mbin mtight">×</span><span class="mord mtight">1</span></span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height: 0.2083em;"><span class=""></span></span></span></span></span></span></span></span><span class="" style="top: -2.8894em;"><span class="pstrut" style="height: 3.3848em;"></span><span class="mord"><span class="mord"></span><span class="mspace" style="margin-right: 0.2778em;"></span><span class="mrel">=</span><span class="mspace" style="margin-right: 0.2778em;"></span><span class="mord">0</span></span></span><span class="" style="top: -0.8446em;"><span class="pstrut" style="height: 3.3848em;"></span><span class="mord"><span class="mord"></span><span class="mspace" style="margin-right: 0.2778em;"></span><span class="mrel">=</span><span class="mspace" style="margin-right: 0.2778em;"></span><span class="mord">−</span><span class="mord"><span class="mopen nulldelimiter"></span><span class="mfrac"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height: 1.3214em;"><span class="" style="top: -2.314em;"><span class="pstrut" style="height: 3em;"></span><span class="mord"><span class="mord">2</span></span></span><span class="" style="top: -3.23em;"><span class="pstrut" style="height: 3em;"></span><span class="frac-line" style="border-bottom-width: 0.04em;"></span></span><span class="" style="top: -3.677em;"><span class="pstrut" style="height: 3em;"></span><span class="mord"><span class="mord">1</span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height: 0.686em;"><span class=""></span></span></span></span></span><span class="mclose nulldelimiter"></span></span><span class="mord mathnormal">λ</span><span class="mspace" style="margin-right: 0.1667em;"></span><span class="minner"><span class="minner"><span class="mopen delimcenter" style="top: 0em;"><span class="delimsizing size2">(</span></span><span class="mord"><span class="mord"><span class="mord"><span class="mord accent"><span class="vlist-t"><span class="vlist-r"><span class="vlist" style="height: 0.9495em;"><span class="" style="top: -3em;"><span class="pstrut" style="height: 3em;"></span><span class="mord boldsymbol" style="margin-right: 0.0778em;">X</span></span><span class="" style="top: -3.6051em;"><span class="pstrut" style="height: 3em;"></span><span class="accent-body" style="left: -0.2875em;"><span class="mord mathbf">~</span></span></span></span></span></span></span></span></span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height: 1.1808em;"><span class="" style="top: -2.453em; margin-right: 0.05em;"><span class="pstrut" style="height: 2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight"><span class="mord mathnormal mtight">i</span></span></span></span><span class="" style="top: -3.4024em; margin-right: 0.05em;"><span class="pstrut" style="height: 2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight"><span class="mord mathnormal mtight" style="margin-right: 0.1389em;">T</span></span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height: 0.247em;"><span class=""></span></span></span></span></span></span><span class="mord"><span class="mord"><span class="mord"><span class="mord accent"><span class="vlist-t"><span class="vlist-r"><span class="vlist" style="height: 0.9495em;"><span class="" style="top: -3em;"><span class="pstrut" style="height: 3em;"></span><span class="mord boldsymbol" style="margin-right: 0.0778em;">X</span></span><span class="" style="top: -3.6051em;"><span class="pstrut" style="height: 3em;"></span><span class="accent-body" style="left: -0.2875em;"><span class="mord mathbf">~</span></span></span></span></span></span></span></span></span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height: 0.3117em;"><span class="" style="top: -2.55em; margin-right: 0.05em;"><span class="pstrut" style="height: 2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mathnormal mtight">i</span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height: 0.15em;"><span class=""></span></span></span></span></span></span><span class="mclose delimcenter" style="top: 0em;"><span class="delimsizing size2">)</span></span></span><span class="msupsub"><span class="vlist-t"><span class="vlist-r"><span class="vlist" style="height: 1.3848em;"><span class="" style="top: -3.6337em; margin-right: 0.05em;"><span class="pstrut" style="height: 2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight"><span class="mord mtight">−</span><span class="mord mtight">1</span></span></span></span></span></span></span></span></span><span class="mspace" style="margin-right: 0.1667em;"></span><span class="mord"><span class="mord mathbf">1</span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height: 0.3361em;"><span class="" style="top: -2.55em; margin-left: 0em; margin-right: 0.05em;"><span class="pstrut" style="height: 2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight"><span class="mord mathnormal mtight" style="margin-right: 0.0315em;">k</span><span class="mbin mtight">×</span><span class="mord mtight">1</span></span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height: 0.2083em;"><span class=""></span></span></span></span></span></span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height: 3.5262em;"><span class=""></span></span></span></span></span></span></span></span></span></span></span></span></p>

考虑到 

      w 
     
    
      i 
     
    
      T 
     
    
    
    
      1 
     
     
     
       k 
      
     
       × 
      
     
       1 
      
     
    
   
     = 
    
    
    
      1 
     
     
     
       k 
      
     
       × 
      
     
       1 
      
     
    
      T 
     
    
    
    
      w 
     
    
      i 
     
    
   
     = 
    
   
     1 
    
   
  
    \boldsymbol{w}_{i}^{T}\mathbf{1}_{k\times 1}=\mathbf{1}_{k\times 1}^{T}\boldsymbol{w}_i=1 
   
  
</span><span class="katex-html"><span class="base"><span class="strut" style="height: 1.1em; vertical-align: -0.2587em;"></span><span class="mord"><span class="mord"><span class="mord"><span class="mord boldsymbol" style="margin-right: 0.0278em;">w</span></span></span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height: 0.8413em;"><span class="" style="top: -2.4413em; margin-right: 0.05em;"><span class="pstrut" style="height: 2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight"><span class="mord mathnormal mtight">i</span></span></span></span><span class="" style="top: -3.063em; margin-right: 0.05em;"><span class="pstrut" style="height: 2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight"><span class="mord mathnormal mtight" style="margin-right: 0.1389em;">T</span></span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height: 0.2587em;"><span class=""></span></span></span></span></span></span><span class="mord"><span class="mord mathbf">1</span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height: 0.3361em;"><span class="" style="top: -2.55em; margin-left: 0em; margin-right: 0.05em;"><span class="pstrut" style="height: 2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight"><span class="mord mathnormal mtight" style="margin-right: 0.0315em;">k</span><span class="mbin mtight">×</span><span class="mord mtight">1</span></span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height: 0.2083em;"><span class=""></span></span></span></span></span></span><span class="mspace" style="margin-right: 0.2778em;"></span><span class="mrel">=</span><span class="mspace" style="margin-right: 0.2778em;"></span></span><span class="base"><span class="strut" style="height: 1.1828em; vertical-align: -0.3414em;"></span><span class="mord"><span class="mord mathbf">1</span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height: 0.8413em;"><span class="" style="top: -2.4169em; margin-left: 0em; margin-right: 0.05em;"><span class="pstrut" style="height: 2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight"><span class="mord mathnormal mtight" style="margin-right: 0.0315em;">k</span><span class="mbin mtight">×</span><span class="mord mtight">1</span></span></span></span><span class="" style="top: -3.063em; margin-right: 0.05em;"><span class="pstrut" style="height: 2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight"><span class="mord mathnormal mtight" style="margin-right: 0.1389em;">T</span></span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height: 0.3414em;"><span class=""></span></span></span></span></span></span><span class="mord"><span class="mord"><span class="mord"><span class="mord boldsymbol" style="margin-right: 0.0278em;">w</span></span></span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height: 0.3117em;"><span class="" style="top: -2.55em; margin-right: 0.05em;"><span class="pstrut" style="height: 2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mathnormal mtight">i</span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height: 0.15em;"><span class=""></span></span></span></span></span></span><span class="mspace" style="margin-right: 0.2778em;"></span><span class="mrel">=</span><span class="mspace" style="margin-right: 0.2778em;"></span></span><span class="base"><span class="strut" style="height: 0.6444em;"></span><span class="mord">1</span></span></span></span></span>，两边同时左乘<span class="katex--inline"><span class="katex"><span class="katex-mathml"> 
 
  
   
    
    
      1 
     
     
     
       k 
      
     
       × 
      
     
       1 
      
     
    
      T 
     
    
   
  
    \mathbf{1}_{k\times 1}^{T} 
   
  
</span><span class="katex-html"><span class="base"><span class="strut" style="height: 1.1828em; vertical-align: -0.3414em;"></span><span class="mord"><span class="mord mathbf">1</span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height: 0.8413em;"><span class="" style="top: -2.4169em; margin-left: 0em; margin-right: 0.05em;"><span class="pstrut" style="height: 2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight"><span class="mord mathnormal mtight" style="margin-right: 0.0315em;">k</span><span class="mbin mtight">×</span><span class="mord mtight">1</span></span></span></span><span class="" style="top: -3.063em; margin-right: 0.05em;"><span class="pstrut" style="height: 2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight"><span class="mord mathnormal mtight" style="margin-right: 0.1389em;">T</span></span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height: 0.3414em;"><span class=""></span></span></span></span></span></span></span></span></span></span>可得</p> 


      − 
     
     
     
       1 
      
     
       2 
      
     
    
      λ 
     
    
      = 
     
     
     
       1 
      
      
       
       
         1 
        
        
        
          k 
         
        
          × 
         
        
          1 
         
        
       
         T 
        
       
       
        
        
          ( 
         
         
          
           
           
             X 
            
           
             ~ 
            
           
          
         
           i 
          
         
           T 
          
         
         
          
           
           
             X 
            
           
             ~ 
            
           
          
         
           i 
          
         
        
          ) 
         
        
        
        
          − 
         
        
          1 
         
        
       
       
       
         1 
        
        
        
          k 
         
        
          × 
         
        
          1 
         
        
       
      
     
    
   
     -\frac{1}{2}\lambda =\frac{1}{\mathbf{1}_{k\times 1}^{T}\left( \boldsymbol{\tilde{X}}_{i}^{T}\boldsymbol{\tilde{X}}_i \right) ^{-1}\mathbf{1}_{k\times 1}} 
    
   
 </span><span class="katex-html"><span class="base"><span class="strut" style="height: 2.0074em; vertical-align: -0.686em;"></span><span class="mord">−</span><span class="mord"><span class="mopen nulldelimiter"></span><span class="mfrac"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height: 1.3214em;"><span class="" style="top: -2.314em;"><span class="pstrut" style="height: 3em;"></span><span class="mord"><span class="mord">2</span></span></span><span class="" style="top: -3.23em;"><span class="pstrut" style="height: 3em;"></span><span class="frac-line" style="border-bottom-width: 0.04em;"></span></span><span class="" style="top: -3.677em;"><span class="pstrut" style="height: 3em;"></span><span class="mord"><span class="mord">1</span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height: 0.686em;"><span class=""></span></span></span></span></span><span class="mclose nulldelimiter"></span></span><span class="mord mathnormal">λ</span><span class="mspace" style="margin-right: 0.2778em;"></span><span class="mrel">=</span><span class="mspace" style="margin-right: 0.2778em;"></span></span><span class="base"><span class="strut" style="height: 3.2462em; vertical-align: -1.9248em;"></span><span class="mord"><span class="mopen nulldelimiter"></span><span class="mfrac"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height: 1.3214em;"><span class="" style="top: -2.11em;"><span class="pstrut" style="height: 3.3848em;"></span><span class="mord"><span class="mord"><span class="mord mathbf">1</span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height: 0.8231em;"><span class="" style="top: -2.3987em; margin-left: 0em; margin-right: 0.05em;"><span class="pstrut" style="height: 2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight"><span class="mord mathnormal mtight" style="margin-right: 0.0315em;">k</span><span class="mbin mtight">×</span><span class="mord mtight">1</span></span></span></span><span class="" style="top: -3.0448em; margin-right: 0.05em;"><span class="pstrut" style="height: 2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight"><span class="mord mathnormal mtight" style="margin-right: 0.1389em;">T</span></span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height: 0.3596em;"><span class=""></span></span></span></span></span></span><span class="mspace" style="margin-right: 0.1667em;"></span><span class="minner"><span class="minner"><span class="mopen delimcenter" style="top: 0em;"><span class="delimsizing size2">(</span></span><span class="mord"><span class="mord"><span class="mord"><span class="mord accent"><span class="vlist-t"><span class="vlist-r"><span class="vlist" style="height: 0.9495em;"><span class="" style="top: -3em;"><span class="pstrut" style="height: 3em;"></span><span class="mord boldsymbol" style="margin-right: 0.0778em;">X</span></span><span class="" style="top: -3.6051em;"><span class="pstrut" style="height: 3em;"></span><span class="accent-body" style="left: -0.2875em;"><span class="mord mathbf">~</span></span></span></span></span></span></span></span></span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height: 1.1808em;"><span class="" style="top: -2.453em; margin-right: 0.05em;"><span class="pstrut" style="height: 2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight"><span class="mord mathnormal mtight">i</span></span></span></span><span class="" style="top: -3.4024em; margin-right: 0.05em;"><span class="pstrut" style="height: 2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight"><span class="mord mathnormal mtight" style="margin-right: 0.1389em;">T</span></span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height: 0.247em;"><span class=""></span></span></span></span></span></span><span class="mord"><span class="mord"><span class="mord"><span class="mord accent"><span class="vlist-t"><span class="vlist-r"><span class="vlist" style="height: 0.9495em;"><span class="" style="top: -3em;"><span class="pstrut" style="height: 3em;"></span><span class="mord boldsymbol" style="margin-right: 0.0778em;">X</span></span><span class="" style="top: -3.6051em;"><span class="pstrut" style="height: 3em;"></span><span class="accent-body" style="left: -0.2875em;"><span class="mord mathbf">~</span></span></span></span></span></span></span></span></span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height: 0.3117em;"><span class="" style="top: -2.55em; margin-right: 0.05em;"><span class="pstrut" style="height: 2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mathnormal mtight">i</span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height: 0.15em;"><span class=""></span></span></span></span></span></span><span class="mclose delimcenter" style="top: 0em;"><span class="delimsizing size2">)</span></span></span><span class="msupsub"><span class="vlist-t"><span class="vlist-r"><span class="vlist" style="height: 1.3848em;"><span class="" style="top: -3.6337em; margin-right: 0.05em;"><span class="pstrut" style="height: 2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight"><span class="mord mtight">−</span><span class="mord mtight">1</span></span></span></span></span></span></span></span></span><span class="mspace" style="margin-right: 0.1667em;"></span><span class="mord"><span class="mord mathbf">1</span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height: 0.3361em;"><span class="" style="top: -2.55em; margin-left: 0em; margin-right: 0.05em;"><span class="pstrut" style="height: 2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight"><span class="mord mathnormal mtight" style="margin-right: 0.0315em;">k</span><span class="mbin mtight">×</span><span class="mord mtight">1</span></span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height: 0.2083em;"><span class=""></span></span></span></span></span></span></span></span><span class="" style="top: -3.6148em;"><span class="pstrut" style="height: 3.3848em;"></span><span class="frac-line" style="border-bottom-width: 0.04em;"></span></span><span class="" style="top: -4.0618em;"><span class="pstrut" style="height: 3.3848em;"></span><span class="mord"><span class="mord">1</span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height: 1.9248em;"><span class=""></span></span></span></span></span><span class="mclose nulldelimiter"></span></span></span></span></span></span></span></p>

联立消去 

     λ 
    
   
  
    \lambda 
   
  
</span><span class="katex-html"><span class="base"><span class="strut" style="height: 0.6944em;"></span><span class="mord mathnormal">λ</span></span></span></span></span>可得</p> 


       w 
      
     
       i 
      
     
    
      = 
     
     
      
       
        
        
          ( 
         
         
          
           
           
             X 
            
           
             ~ 
            
           
          
         
           i 
          
         
           T 
          
         
         
          
           
           
             X 
            
           
             ~ 
            
           
          
         
           i 
          
         
        
          ) 
         
        
        
        
          − 
         
        
          1 
         
        
       
       
       
         1 
        
        
        
          k 
         
        
          × 
         
        
          1 
         
        
       
      
      
       
       
         1 
        
        
        
          k 
         
        
          × 
         
        
          1 
         
        
       
         T 
        
       
       
        
        
          ( 
         
         
          
           
           
             X 
            
           
             ~ 
            
           
          
         
           i 
          
         
           T 
          
         
         
          
           
           
             X 
            
           
             ~ 
            
           
          
         
           i 
          
         
        
          ) 
         
        
        
        
          − 
         
        
          1 
         
        
       
       
       
         1 
        
        
        
          k 
         
        
          × 
         
        
          1 
         
        
       
      
     
    
   
     \boldsymbol{w}_i=\frac{\left( \boldsymbol{\tilde{X}}_{i}^{T}\boldsymbol{\tilde{X}}_i \right) ^{-1}\mathbf{1}_{k\times 1}}{\mathbf{1}_{k\times 1}^{T}\left( \boldsymbol{\tilde{X}}_{i}^{T}\boldsymbol{\tilde{X}}_i \right) ^{-1}\mathbf{1}_{k\times 1}} 
    
   
 </span><span class="katex-html"><span class="base"><span class="strut" style="height: 0.5944em; vertical-align: -0.15em;"></span><span class="mord"><span class="mord"><span class="mord"><span class="mord boldsymbol" style="margin-right: 0.0278em;">w</span></span></span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height: 0.3117em;"><span class="" style="top: -2.55em; margin-right: 0.05em;"><span class="pstrut" style="height: 2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mathnormal mtight">i</span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height: 0.15em;"><span class=""></span></span></span></span></span></span><span class="mspace" style="margin-right: 0.2778em;"></span><span class="mrel">=</span><span class="mspace" style="margin-right: 0.2778em;"></span></span><span class="base"><span class="strut" style="height: 4.3496em; vertical-align: -1.9248em;"></span><span class="mord"><span class="mopen nulldelimiter"></span><span class="mfrac"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height: 2.4248em;"><span class="" style="top: -2.11em;"><span class="pstrut" style="height: 3.3848em;"></span><span class="mord"><span class="mord"><span class="mord mathbf">1</span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height: 0.8231em;"><span class="" style="top: -2.3987em; margin-left: 0em; margin-right: 0.05em;"><span class="pstrut" style="height: 2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight"><span class="mord mathnormal mtight" style="margin-right: 0.0315em;">k</span><span class="mbin mtight">×</span><span class="mord mtight">1</span></span></span></span><span class="" style="top: -3.0448em; margin-right: 0.05em;"><span class="pstrut" style="height: 2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight"><span class="mord mathnormal mtight" style="margin-right: 0.1389em;">T</span></span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height: 0.3596em;"><span class=""></span></span></span></span></span></span><span class="mspace" style="margin-right: 0.1667em;"></span><span class="minner"><span class="minner"><span class="mopen delimcenter" style="top: 0em;"><span class="delimsizing size2">(</span></span><span class="mord"><span class="mord"><span class="mord"><span class="mord accent"><span class="vlist-t"><span class="vlist-r"><span class="vlist" style="height: 0.9495em;"><span class="" style="top: -3em;"><span class="pstrut" style="height: 3em;"></span><span class="mord boldsymbol" style="margin-right: 0.0778em;">X</span></span><span class="" style="top: -3.6051em;"><span class="pstrut" style="height: 3em;"></span><span class="accent-body" style="left: -0.2875em;"><span class="mord mathbf">~</span></span></span></span></span></span></span></span></span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height: 1.1808em;"><span class="" style="top: -2.453em; margin-right: 0.05em;"><span class="pstrut" style="height: 2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight"><span class="mord mathnormal mtight">i</span></span></span></span><span class="" style="top: -3.4024em; margin-right: 0.05em;"><span class="pstrut" style="height: 2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight"><span class="mord mathnormal mtight" style="margin-right: 0.1389em;">T</span></span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height: 0.247em;"><span class=""></span></span></span></span></span></span><span class="mord"><span class="mord"><span class="mord"><span class="mord accent"><span class="vlist-t"><span class="vlist-r"><span class="vlist" style="height: 0.9495em;"><span class="" style="top: -3em;"><span class="pstrut" style="height: 3em;"></span><span class="mord boldsymbol" style="margin-right: 0.0778em;">X</span></span><span class="" style="top: -3.6051em;"><span class="pstrut" style="height: 3em;"></span><span class="accent-body" style="left: -0.2875em;"><span class="mord mathbf">~</span></span></span></span></span></span></span></span></span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height: 0.3117em;"><span class="" style="top: -2.55em; margin-right: 0.05em;"><span class="pstrut" style="height: 2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mathnormal mtight">i</span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height: 0.15em;"><span class=""></span></span></span></span></span></span><span class="mclose delimcenter" style="top: 0em;"><span class="delimsizing size2">)</span></span></span><span class="msupsub"><span class="vlist-t"><span class="vlist-r"><span class="vlist" style="height: 1.3848em;"><span class="" style="top: -3.6337em; margin-right: 0.05em;"><span class="pstrut" style="height: 2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight"><span class="mord mtight">−</span><span class="mord mtight">1</span></span></span></span></span></span></span></span></span><span class="mspace" style="margin-right: 0.1667em;"></span><span class="mord"><span class="mord mathbf">1</span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height: 0.3361em;"><span class="" style="top: -2.55em; margin-left: 0em; margin-right: 0.05em;"><span class="pstrut" style="height: 2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight"><span class="mord mathnormal mtight" style="margin-right: 0.0315em;">k</span><span class="mbin mtight">×</span><span class="mord mtight">1</span></span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height: 0.2083em;"><span class=""></span></span></span></span></span></span></span></span><span class="" style="top: -3.6148em;"><span class="pstrut" style="height: 3.3848em;"></span><span class="frac-line" style="border-bottom-width: 0.04em;"></span></span><span class="" style="top: -4.4248em;"><span class="pstrut" style="height: 3.3848em;"></span><span class="mord"><span class="minner"><span class="minner"><span class="mopen delimcenter" style="top: 0em;"><span class="delimsizing size2">(</span></span><span class="mord"><span class="mord"><span class="mord"><span class="mord accent"><span class="vlist-t"><span class="vlist-r"><span class="vlist" style="height: 0.9495em;"><span class="" style="top: -3em;"><span class="pstrut" style="height: 3em;"></span><span class="mord boldsymbol" style="margin-right: 0.0778em;">X</span></span><span class="" style="top: -3.6051em;"><span class="pstrut" style="height: 3em;"></span><span class="accent-body" style="left: -0.2875em;"><span class="mord mathbf">~</span></span></span></span></span></span></span></span></span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height: 1.1808em;"><span class="" style="top: -2.453em; margin-right: 0.05em;"><span class="pstrut" style="height: 2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight"><span class="mord mathnormal mtight">i</span></span></span></span><span class="" style="top: -3.4024em; margin-right: 0.05em;"><span class="pstrut" style="height: 2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight"><span class="mord mathnormal mtight" style="margin-right: 0.1389em;">T</span></span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height: 0.247em;"><span class=""></span></span></span></span></span></span><span class="mord"><span class="mord"><span class="mord"><span class="mord accent"><span class="vlist-t"><span class="vlist-r"><span class="vlist" style="height: 0.9495em;"><span class="" style="top: -3em;"><span class="pstrut" style="height: 3em;"></span><span class="mord boldsymbol" style="margin-right: 0.0778em;">X</span></span><span class="" style="top: -3.6051em;"><span class="pstrut" style="height: 3em;"></span><span class="accent-body" style="left: -0.2875em;"><span class="mord mathbf">~</span></span></span></span></span></span></span></span></span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height: 0.3117em;"><span class="" style="top: -2.55em; margin-right: 0.05em;"><span class="pstrut" style="height: 2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mathnormal mtight">i</span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height: 0.15em;"><span class=""></span></span></span></span></span></span><span class="mclose delimcenter" style="top: 0em;"><span class="delimsizing size2">)</span></span></span><span class="msupsub"><span class="vlist-t"><span class="vlist-r"><span class="vlist" style="height: 1.3848em;"><span class="" style="top: -3.6337em; margin-right: 0.05em;"><span class="pstrut" style="height: 2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight"><span class="mord mtight">−</span><span class="mord mtight">1</span></span></span></span></span></span></span></span></span><span class="mspace" style="margin-right: 0.1667em;"></span><span class="mord"><span class="mord mathbf">1</span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height: 0.3361em;"><span class="" style="top: -2.55em; margin-left: 0em; margin-right: 0.05em;"><span class="pstrut" style="height: 2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight"><span class="mord mathnormal mtight" style="margin-right: 0.0315em;">k</span><span class="mbin mtight">×</span><span class="mord mtight">1</span></span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height: 0.2083em;"><span class=""></span></span></span></span></span></span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height: 1.9248em;"><span class=""></span></span></span></span></span><span class="mclose nulldelimiter"></span></span></span></span></span></span></span></p>

在低维空间中需要维护这组 

      w 
     
    
      i 
     
    
    
    
      ( 
     
    
      i 
     
    
      = 
     
    
      1 
     
    
      , 
     
    
      2 
     
    
      , 
     
    
      ⋯ 
      
    
      , 
     
    
      m 
     
    
      ) 
     
    
   
     ∈ 
    
    
    
      R 
     
     
     
       k 
      
     
       × 
      
     
       1 
      
     
    
   
  
    \boldsymbol{w}_i\left( i=1,2,\cdots ,m \right) \in \mathbb{R} ^{k\times 1} 
   
  
</span><span class="katex-html"><span class="base"><span class="strut" style="height: 1em; vertical-align: -0.25em;"></span><span class="mord"><span class="mord"><span class="mord"><span class="mord boldsymbol" style="margin-right: 0.0278em;">w</span></span></span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height: 0.3117em;"><span class="" style="top: -2.55em; margin-right: 0.05em;"><span class="pstrut" style="height: 2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mathnormal mtight">i</span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height: 0.15em;"><span class=""></span></span></span></span></span></span><span class="mspace" style="margin-right: 0.1667em;"></span><span class="minner"><span class="mopen delimcenter" style="top: 0em;">(</span><span class="mord mathnormal">i</span><span class="mspace" style="margin-right: 0.2778em;"></span><span class="mrel">=</span><span class="mspace" style="margin-right: 0.2778em;"></span><span class="mord">1</span><span class="mpunct">,</span><span class="mspace" style="margin-right: 0.1667em;"></span><span class="mord">2</span><span class="mpunct">,</span><span class="mspace" style="margin-right: 0.1667em;"></span><span class="minner">⋯</span><span class="mspace" style="margin-right: 0.1667em;"></span><span class="mspace" style="margin-right: 0.1667em;"></span><span class="mpunct">,</span><span class="mspace" style="margin-right: 0.1667em;"></span><span class="mord mathnormal">m</span><span class="mclose delimcenter" style="top: 0em;">)</span></span><span class="mspace" style="margin-right: 0.2778em;"></span><span class="mrel">∈</span><span class="mspace" style="margin-right: 0.2778em;"></span></span><span class="base"><span class="strut" style="height: 0.8491em;"></span><span class="mord"><span class="mord mathbb">R</span><span class="msupsub"><span class="vlist-t"><span class="vlist-r"><span class="vlist" style="height: 0.8491em;"><span class="" style="top: -3.063em; margin-right: 0.05em;"><span class="pstrut" style="height: 2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight"><span class="mord mathnormal mtight" style="margin-right: 0.0315em;">k</span><span class="mbin mtight">×</span><span class="mord mtight">1</span></span></span></span></span></span></span></span></span></span></span></span></span>，因此优化目标为</p> 


         min 
        
       
         ⁡ 
        
       
       
        
        
          z 
         
        
          1 
         
        
       
         , 
        
        
        
          z 
         
        
          2 
         
        
       
         , 
        
       
         ⋯ 
         
       
         , 
        
        
        
          z 
         
        
          m 
         
        
       
      
     
     
     
       ∑ 
      
      
      
        i 
       
      
        = 
       
      
        1 
       
      
     
       m 
      
     
     
      
      
        ∥ 
       
       
       
         z 
        
       
         i 
        
       
      
        − 
       
       
       
         ∑ 
        
        
        
          j 
         
        
          ∈ 
         
         
         
           Q 
          
         
           i 
          
         
           k 
          
         
        
       
       
        
        
          w 
         
         
         
           i 
          
         
           j 
          
         
        
        
        
          z 
         
        
          j 
         
        
       
      
        ∥ 
       
      
     
       2 
      
     
       2 
      
       
    
      s 
     
    
      . 
     
    
      t 
     
    
      . 
     
     
     
       ∑ 
      
     
       i 
      
     
     
     
       z 
      
     
       i 
      
     
    
      = 
     
    
      0 
     
    
      , 
     
     
     
       1 
      
      
      
        m 
       
      
        − 
       
      
        1 
       
      
     
     
     
       ∑ 
      
     
       i 
      
     
     
     
       z 
      
     
       i 
      
     
     
     
       z 
      
     
       i 
      
     
       T 
      
     
    
      = 
     
    
      I 
     
    
   
     \underset{\boldsymbol{z}_1,\boldsymbol{z}_2,\cdots ,\boldsymbol{z}_m}{\min}\sum_{i=1}^m{\left\| \boldsymbol{z}_i-\sum_{j\in Q_{i}^{k}}{w_{ij}\boldsymbol{z}_j} \right\| _{2}^{2}}\,\, \mathrm{s}.\mathrm{t}. \sum_i{\boldsymbol{z}_i}=0, \frac{1}{m-1}\sum_i{\boldsymbol{z}_i}\boldsymbol{z}_{i}^{T}=\boldsymbol{I} 
    
   
 </span><span class="katex-html"><span class="base"><span class="strut" style="height: 3.9339em; vertical-align: -1.6799em;"></span><span class="mord"><span class="mop op-limits"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height: 0.6679em;"><span class="" style="top: -2.4em; margin-left: 0em;"><span class="pstrut" style="height: 3em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight"><span class="mord mtight"><span class="mord mtight"><span class="mord mtight"><span class="mord boldsymbol mtight" style="margin-right: 0.0421em;">z</span></span></span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height: 0.3173em;"><span class="" style="top: -2.357em; margin-right: 0.0714em;"><span class="pstrut" style="height: 2.5em;"></span><span class="sizing reset-size3 size1 mtight"><span class="mord mtight">1</span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height: 0.143em;"><span class=""></span></span></span></span></span></span><span class="mpunct mtight">,</span><span class="mord mtight"><span class="mord mtight"><span class="mord mtight"><span class="mord boldsymbol mtight" style="margin-right: 0.0421em;">z</span></span></span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height: 0.3173em;"><span class="" style="top: -2.357em; margin-right: 0.0714em;"><span class="pstrut" style="height: 2.5em;"></span><span class="sizing reset-size3 size1 mtight"><span class="mord mtight">2</span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height: 0.143em;"><span class=""></span></span></span></span></span></span><span class="mpunct mtight">,</span><span class="minner mtight">⋯</span><span class="mspace mtight" style="margin-right: 0.1952em;"></span><span class="mpunct mtight">,</span><span class="mord mtight"><span class="mord mtight"><span class="mord mtight"><span class="mord boldsymbol mtight" style="margin-right: 0.0421em;">z</span></span></span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height: 0.1645em;"><span class="" style="top: -2.357em; margin-right: 0.0714em;"><span class="pstrut" style="height: 2.5em;"></span><span class="sizing reset-size3 size1 mtight"><span class="mord mathnormal mtight">m</span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height: 0.143em;"><span class=""></span></span></span></span></span></span></span></span></span><span class="" style="top: -3em;"><span class="pstrut" style="height: 3em;"></span><span class=""><span class="mop"><span class="mop">min</span></span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height: 0.8361em;"><span class=""></span></span></span></span></span></span><span class="mspace" style="margin-right: 0.1667em;"></span><span class="mop op-limits"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height: 1.6514em;"><span class="" style="top: -1.8723em; margin-left: 0em;"><span class="pstrut" style="height: 3.05em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight"><span class="mord mathnormal mtight">i</span><span class="mrel mtight">=</span><span class="mord mtight">1</span></span></span></span><span class="" style="top: -3.05em;"><span class="pstrut" style="height: 3.05em;"></span><span class=""><span class="mop op-symbol large-op">∑</span></span></span><span class="" style="top: -4.3em; margin-left: 0em;"><span class="pstrut" style="height: 3.05em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mathnormal mtight">m</span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height: 1.2777em;"><span class=""></span></span></span></span></span><span class="mspace" style="margin-right: 0.1667em;"></span><span class="mord"><span class="minner"><span class="minner"><span class="mopen"><span class="delimsizing mult"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height: 2.05em;"><span class="" style="top: -4.05em;"><span class="pstrut" style="height: 5.6em;"></span><span class="" style="width: 0.556em; height: 3.6em;"> 
            <svg width="0.556em" height="3.600em" viewBox="0 0 556 3600"> 
             <path d="M145 15 v585 v2400 v585 c2.667,10,9.667,15,21,15

c10,0,16.667,-5,20,-15 v-585 v-2400 v-585 c-2.667,-10,-9.667,-15,-21,-15
c-10,0,-16.667,5,-20,15z M188 15 H145 v585 v2400 v585 h43z
M367 15 v585 v2400 v585 c2.667,10,9.667,15,21,15
c10,0,16.667,-5,20,-15 v-585 v-2400 v-585 c-2.667,-10,-9.667,-15,-21,-15
c-10,0,-16.667,5,-20,15z M410 15 H367 v585 v2400 v585 h43z">
​zi​−j∈Qik​∑​wij​zj​


​22​s.t.i∑​zi​=0,m−11​i∑​zi​ziT​=I

将重构向量增广为 

        w 
       
      
        ~ 
       
      
     
    
      i 
     
    
   
     ∈ 
    
    
    
      R 
     
     
     
       m 
      
     
       × 
      
     
       1 
      
     
    
   
  
    \boldsymbol{\tilde{w}}_i\in \mathbb{R} ^{m\times 1} 
   
  
</span><span class="katex-html"><span class="base"><span class="strut" style="height: 0.8579em; vertical-align: -0.15em;"></span><span class="mord"><span class="mord"><span class="mord"><span class="mord accent"><span class="vlist-t"><span class="vlist-r"><span class="vlist" style="height: 0.7079em;"><span class="" style="top: -3em;"><span class="pstrut" style="height: 3em;"></span><span class="mord boldsymbol" style="margin-right: 0.0278em;">w</span></span><span class="" style="top: -3.3634em;"><span class="pstrut" style="height: 3em;"></span><span class="accent-body" style="left: -0.2875em;"><span class="mord mathbf">~</span></span></span></span></span></span></span></span></span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height: 0.3117em;"><span class="" style="top: -2.55em; margin-right: 0.05em;"><span class="pstrut" style="height: 2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mathnormal mtight">i</span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height: 0.15em;"><span class=""></span></span></span></span></span></span><span class="mspace" style="margin-right: 0.2778em;"></span><span class="mrel">∈</span><span class="mspace" style="margin-right: 0.2778em;"></span></span><span class="base"><span class="strut" style="height: 0.8141em;"></span><span class="mord"><span class="mord mathbb">R</span><span class="msupsub"><span class="vlist-t"><span class="vlist-r"><span class="vlist" style="height: 0.8141em;"><span class="" style="top: -3.063em; margin-right: 0.05em;"><span class="pstrut" style="height: 2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight"><span class="mord mathnormal mtight">m</span><span class="mbin mtight">×</span><span class="mord mtight">1</span></span></span></span></span></span></span></span></span></span></span></span></span>，多余的位置为0；约束条件表示了降维样本的分布模式——<font color="#f00"><strong>样本中心化去除平移自由度、样本单位协方差去除缩放自由度</strong></font>。</p>

将优化目标改为矩阵形式

             min 
            
           
             ⁡ 
            
           
          
            Z 
           
          
         
         
         
           ∑ 
          
          
          
            i 
           
          
            = 
           
          
            1 
           
          
         
           m 
          
         
         
          
          
            ∥ 
           
           
            
             
             
               Z 
              
             
               i 
              
             
            
           
             i 
            
           
          
            − 
           
           
            
             
             
               Z 
              
              
              
                w 
               
              
                ~ 
               
              
             
            
           
             i 
            
           
          
            ∥ 
           
          
         
           2 
          
         
           2 
          
         
        
       
      
      
       
        
         
        
          = 
         
         
          
           
           
             min 
            
           
             ⁡ 
            
           
          
            Z 
           
          
         
         
         
           ∑ 
          
          
          
            i 
           
          
            = 
           
          
            1 
           
          
         
           m 
          
         
         
          
           
           
             ( 
            
           
             Z 
            
            
            
              ( 
             
             
             
               i 
              
             
               i 
              
             
            
              − 
             
             
              
               
               
                 w 
                
               
                 ~ 
                
               
              
             
               i 
              
             
            
              ) 
             
            
           
             ) 
            
           
          
            T 
           
          
         
           Z 
          
          
          
            ( 
           
           
           
             i 
            
           
             i 
            
           
          
            − 
           
           
            
             
             
               w 
              
             
               ~ 
              
             
            
           
             i 
            
           
          
            ) 
           
          
         
        
       
      
     
     
      
       
        
       
      
      
       
        
         
        
          = 
         
         
          
           
           
             min 
            
           
             ⁡ 
            
           
          
            Z 
           
          
           
         
         
           t 
          
         
           r 
          
         
         
         
           ( 
          
         
           Z 
          
          
          
            ( 
           
          
            I 
           
          
            − 
           
           
            
            
              W 
             
            
              ~ 
             
            
           
          
            ) 
           
          
          
           
           
             ( 
            
           
             Z 
            
            
            
              ( 
             
            
              I 
             
            
              − 
             
             
              
              
                W 
               
              
                ~ 
               
              
             
            
              ) 
             
            
           
             ) 
            
           
          
            T 
           
          
         
           ) 
          
         
        
       
      
     
     
      
       
        
       
      
      
       
        
         
        
          = 
         
         
          
           
           
             min 
            
           
             ⁡ 
            
           
          
            Z 
           
          
           
         
         
           t 
          
         
           r 
          
         
         
         
           ( 
          
          
           
            
            
              Z 
             
            
              M 
             
            
              Z 
             
            
           
          
            T 
           
          
         
           ) 
          
           
        
       
      
     
     
      
       
        
        
          s 
         
        
          . 
         
        
          t 
         
        
          . 
         
         
          
           
           
             Z 
            
           
             Z 
            
           
          
         
           T 
          
         
        
       
      
      
       
        
         
        
          = 
         
         
         
           ( 
          
         
           m 
          
         
           − 
          
         
           1 
          
         
           ) 
          
         
        
          I 
         
        
       
      
     
    
   
     <script type="math/tex; mode=display">\begin{aligned}\underset{\boldsymbol{Z}}{\min}\sum_{i=1}^m{\left\| \boldsymbol{Zi}_i-\boldsymbol{Z\tilde{w}}_i \right\| _{2}^{2}}&=\underset{\boldsymbol{Z}}{\min}\sum_{i=1}^m{\left( \boldsymbol{Z}\left( \boldsymbol{i}_i-\boldsymbol{\tilde{w}}_i \right) \right) ^T\boldsymbol{Z}\left( \boldsymbol{i}_i-\boldsymbol{\tilde{w}}_i \right)}\\&=\underset{\boldsymbol{Z}}{\min}\,\,\mathrm{tr}\left( \boldsymbol{Z}\left( \boldsymbol{I}-\boldsymbol{\tilde{W}} \right) \left( \boldsymbol{Z}\left( \boldsymbol{I}-\boldsymbol{\tilde{W}} \right) \right) ^T \right) \\&=\underset{\boldsymbol{Z}}{\min}\,\,\mathrm{tr}\left( \boldsymbol{ZMZ}^T \right) \,\, \\\mathrm{s}.\mathrm{t}. \boldsymbol{ZZ}^T&=\left( m-1 \right) I\end{aligned}</script> 
    
   
 </span><span class="katex-html"><span class="base"><span class="strut" style="height: 9.7027em; vertical-align: -4.6014em;"></span><span class="mord"><span class="mtable"><span class="col-align-r"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height: 5.1014em;"><span class="" style="top: -7.1014em;"><span class="pstrut" style="height: 3.6514em;"></span><span class="mord"><span class="mord"><span class="mop op-limits"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height: 0.6679em;"><span class="" style="top: -2.3537em; margin-left: 0em;"><span class="pstrut" style="height: 3em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight"><span class="mord mtight"><span class="mord mtight"><span class="mord boldsymbol mtight" style="margin-right: 0.0698em;">Z</span></span></span></span></span></span><span class="" style="top: -3em;"><span class="pstrut" style="height: 3em;"></span><span class=""><span class="mop"><span class="mop">min</span></span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height: 0.7463em;"><span class=""></span></span></span></span></span></span><span class="mspace" style="margin-right: 0.1667em;"></span><span class="mop op-limits"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height: 1.6514em;"><span class="" style="top: -1.8723em; margin-left: 0em;"><span class="pstrut" style="height: 3.05em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight"><span class="mord mathnormal mtight">i</span><span class="mrel mtight">=</span><span class="mord mtight">1</span></span></span></span><span class="" style="top: -3.05em;"><span class="pstrut" style="height: 3.05em;"></span><span class=""><span class="mop op-symbol large-op">∑</span></span></span><span class="" style="top: -4.3em; margin-left: 0em;"><span class="pstrut" style="height: 3.05em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mathnormal mtight">m</span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height: 1.2777em;"><span class=""></span></span></span></span></span><span class="mspace" style="margin-right: 0.1667em;"></span><span class="mord"><span class="minner"><span class="minner"><span class="mopen delimcenter" style="top: 0em;">∥</span><span class="mord"><span class="mord"><span class="mord"><span class="mord boldsymbol">Zi</span></span></span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height: 0.3117em;"><span class="" style="top: -2.55em; margin-right: 0.05em;"><span class="pstrut" style="height: 2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mathnormal mtight">i</span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height: 0.15em;"><span class=""></span></span></span></span></span></span><span class="mspace" style="margin-right: 0.2222em;"></span><span class="mbin">−</span><span class="mspace" style="margin-right: 0.2222em;"></span><span class="mord"><span class="mord"><span class="mord"><span class="mord boldsymbol" style="margin-right: 0.0698em;">Z</span><span class="mord accent"><span class="vlist-t"><span class="vlist-r"><span class="vlist" style="height: 0.7079em;"><span class="" style="top: -3em;"><span class="pstrut" style="height: 3em;"></span><span class="mord boldsymbol" style="margin-right: 0.0278em;">w</span></span><span class="" style="top: -3.3634em;"><span class="pstrut" style="height: 3em;"></span><span class="accent-body" style="left: -0.2875em;"><span class="mord mathbf">~</span></span></span></span></span></span></span></span></span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height: 0.3117em;"><span class="" style="top: -2.55em; margin-right: 0.05em;"><span class="pstrut" style="height: 2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mathnormal mtight">i</span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height: 0.15em;"><span class=""></span></span></span></span></span></span><span class="mclose delimcenter" style="top: 0em;">∥</span></span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height: 0.954em;"><span class="" style="top: -2.4003em; margin-right: 0.05em;"><span class="pstrut" style="height: 2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight"><span class="mord mtight">2</span></span></span></span><span class="" style="top: -3.2029em; margin-right: 0.05em;"><span class="pstrut" style="height: 2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight"><span class="mord mtight">2</span></span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height: 0.2997em;"><span class=""></span></span></span></span></span></span></span></span></span><span class="" style="top: -4.0737em;"><span class="pstrut" style="height: 3.6514em;"></span><span class="mord"></span></span><span class="" style="top: -1.6737em;"><span class="pstrut" style="height: 3.6514em;"></span><span class="mord"></span></span><span class="" style="top: 0.29em;"><span class="pstrut" style="height: 3.6514em;"></span><span class="mord"><span class="mord mathrm">s</span><span class="mord">.</span><span class="mord mathrm">t</span><span class="mord">.</span><span class="mord"><span class="mord"><span class="mord"><span class="mord boldsymbol" style="margin-right: 0.0698em;">ZZ</span></span></span><span class="msupsub"><span class="vlist-t"><span class="vlist-r"><span class="vlist" style="height: 0.9173em;"><span class="" style="top: -3.139em; margin-right: 0.05em;"><span class="pstrut" style="height: 2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mathnormal mtight" style="margin-right: 0.1389em;">T</span></span></span></span></span></span></span></span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height: 4.6014em;"><span class=""></span></span></span></span></span><span class="col-align-l"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height: 5.1014em;"><span class="" style="top: -7.1014em;"><span class="pstrut" style="height: 3.6514em;"></span><span class="mord"><span class="mord"></span><span class="mspace" style="margin-right: 0.2778em;"></span><span class="mrel">=</span><span class="mspace" style="margin-right: 0.2778em;"></span><span class="mord"><span class="mop op-limits"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height: 0.6679em;"><span class="" style="top: -2.3537em; margin-left: 0em;"><span class="pstrut" style="height: 3em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight"><span class="mord mtight"><span class="mord mtight"><span class="mord boldsymbol mtight" style="margin-right: 0.0698em;">Z</span></span></span></span></span></span><span class="" style="top: -3em;"><span class="pstrut" style="height: 3em;"></span><span class=""><span class="mop"><span class="mop">min</span></span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height: 0.7463em;"><span class=""></span></span></span></span></span></span><span class="mspace" style="margin-right: 0.1667em;"></span><span class="mop op-limits"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height: 1.6514em;"><span class="" style="top: -1.8723em; margin-left: 0em;"><span class="pstrut" style="height: 3.05em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight"><span class="mord mathnormal mtight">i</span><span class="mrel mtight">=</span><span class="mord mtight">1</span></span></span></span><span class="" style="top: -3.05em;"><span class="pstrut" style="height: 3.05em;"></span><span class=""><span class="mop op-symbol large-op">∑</span></span></span><span class="" style="top: -4.3em; margin-left: 0em;"><span class="pstrut" style="height: 3.05em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mathnormal mtight">m</span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height: 1.2777em;"><span class=""></span></span></span></span></span><span class="mspace" style="margin-right: 0.1667em;"></span><span class="mord"><span class="minner"><span class="minner"><span class="mopen delimcenter" style="top: 0em;">(</span><span class="mord"><span class="mord"><span class="mord boldsymbol" style="margin-right: 0.0698em;">Z</span></span></span><span class="mspace" style="margin-right: 0.1667em;"></span><span class="minner"><span class="mopen delimcenter" style="top: 0em;">(</span><span class="mord"><span class="mord"><span class="mord"><span class="mord boldsymbol">i</span></span></span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height: 0.3117em;"><span class="" style="top: -2.55em; margin-right: 0.05em;"><span class="pstrut" style="height: 2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mathnormal mtight">i</span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height: 0.15em;"><span class=""></span></span></span></span></span></span><span class="mspace" style="margin-right: 0.2222em;"></span><span class="mbin">−</span><span class="mspace" style="margin-right: 0.2222em;"></span><span class="mord"><span class="mord"><span class="mord"><span class="mord accent"><span class="vlist-t"><span class="vlist-r"><span class="vlist" style="height: 0.7079em;"><span class="" style="top: -3em;"><span class="pstrut" style="height: 3em;"></span><span class="mord boldsymbol" style="margin-right: 0.0278em;">w</span></span><span class="" style="top: -3.3634em;"><span class="pstrut" style="height: 3em;"></span><span class="accent-body" style="left: -0.2875em;"><span class="mord mathbf">~</span></span></span></span></span></span></span></span></span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height: 0.3117em;"><span class="" style="top: -2.55em; margin-right: 0.05em;"><span class="pstrut" style="height: 2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mathnormal mtight">i</span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height: 0.15em;"><span class=""></span></span></span></span></span></span><span class="mclose delimcenter" style="top: 0em;">)</span></span><span class="mclose delimcenter" style="top: 0em;">)</span></span><span class="msupsub"><span class="vlist-t"><span class="vlist-r"><span class="vlist" style="height: 0.9812em;"><span class="" style="top: -3.2029em; margin-right: 0.05em;"><span class="pstrut" style="height: 2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mathnormal mtight" style="margin-right: 0.1389em;">T</span></span></span></span></span></span></span></span><span class="mspace" style="margin-right: 0.1667em;"></span><span class="mord"><span class="mord"><span class="mord boldsymbol" style="margin-right: 0.0698em;">Z</span></span></span><span class="mspace" style="margin-right: 0.1667em;"></span><span class="minner"><span class="mopen delimcenter" style="top: 0em;">(</span><span class="mord"><span class="mord"><span class="mord"><span class="mord boldsymbol">i</span></span></span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height: 0.3117em;"><span class="" style="top: -2.55em; margin-right: 0.05em;"><span class="pstrut" style="height: 2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mathnormal mtight">i</span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height: 0.15em;"><span class=""></span></span></span></span></span></span><span class="mspace" style="margin-right: 0.2222em;"></span><span class="mbin">−</span><span class="mspace" style="margin-right: 0.2222em;"></span><span class="mord"><span class="mord"><span class="mord"><span class="mord accent"><span class="vlist-t"><span class="vlist-r"><span class="vlist" style="height: 0.7079em;"><span class="" style="top: -3em;"><span class="pstrut" style="height: 3em;"></span><span class="mord boldsymbol" style="margin-right: 0.0278em;">w</span></span><span class="" style="top: -3.3634em;"><span class="pstrut" style="height: 3em;"></span><span class="accent-body" style="left: -0.2875em;"><span class="mord mathbf">~</span></span></span></span></span></span></span></span></span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height: 0.3117em;"><span class="" style="top: -2.55em; margin-right: 0.05em;"><span class="pstrut" style="height: 2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mathnormal mtight">i</span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height: 0.15em;"><span class=""></span></span></span></span></span></span><span class="mclose delimcenter" style="top: 0em;">)</span></span></span></span></span><span class="" style="top: -4.0737em;"><span class="pstrut" style="height: 3.6514em;"></span><span class="mord"><span class="mord"></span><span class="mspace" style="margin-right: 0.2778em;"></span><span class="mrel">=</span><span class="mspace" style="margin-right: 0.2778em;"></span><span class="mord"><span class="mop op-limits"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height: 0.6679em;"><span class="" style="top: -2.3537em; margin-left: 0em;"><span class="pstrut" style="height: 3em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight"><span class="mord mtight"><span class="mord mtight"><span class="mord boldsymbol mtight" style="margin-right: 0.0698em;">Z</span></span></span></span></span></span><span class="" style="top: -3em;"><span class="pstrut" style="height: 3em;"></span><span class=""><span class="mop"><span class="mop">min</span></span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height: 0.7463em;"><span class=""></span></span></span></span></span></span><span class="mspace" style="margin-right: 0.1667em;"></span><span class="mspace" style="margin-right: 0.1667em;"></span><span class="mord"><span class="mord mathrm">tr</span></span><span class="mspace" style="margin-right: 0.1667em;"></span><span class="minner"><span class="mopen delimcenter" style="top: 0em;"><span class="delimsizing size3">(</span></span><span class="mord"><span class="mord"><span class="mord boldsymbol" style="margin-right: 0.0698em;">Z</span></span></span><span class="mspace" style="margin-right: 0.1667em;"></span><span class="minner"><span class="mopen delimcenter" style="top: 0em;"><span class="delimsizing size2">(</span></span><span class="mord"><span class="mord"><span class="mord boldsymbol" style="margin-right: 0.0778em;">I</span></span></span><span class="mspace" style="margin-right: 0.2222em;"></span><span class="mbin">−</span><span class="mspace" style="margin-right: 0.2222em;"></span><span class="mord"><span class="mord"><span class="mord accent"><span class="vlist-t"><span class="vlist-r"><span class="vlist" style="height: 0.9495em;"><span class="" style="top: -3em;"><span class="pstrut" style="height: 3em;"></span><span class="mord boldsymbol" style="margin-right: 0.1597em;">W</span></span><span class="" style="top: -3.6051em;"><span class="pstrut" style="height: 3em;"></span><span class="accent-body" style="left: -0.2875em;"><span class="mord mathbf">~</span></span></span></span></span></span></span></span></span><span class="mclose delimcenter" style="top: 0em;"><span class="delimsizing size2">)</span></span></span><span class="mspace" style="margin-right: 0.1667em;"></span><span class="minner"><span class="minner"><span class="mopen delimcenter" style="top: 0em;"><span class="delimsizing size2">(</span></span><span class="mord"><span class="mord"><span class="mord boldsymbol" style="margin-right: 0.0698em;">Z</span></span></span><span class="mspace" style="margin-right: 0.1667em;"></span><span class="minner"><span class="mopen delimcenter" style="top: 0em;"><span class="delimsizing size2">(</span></span><span class="mord"><span class="mord"><span class="mord boldsymbol" style="margin-right: 0.0778em;">I</span></span></span><span class="mspace" style="margin-right: 0.2222em;"></span><span class="mbin">−</span><span class="mspace" style="margin-right: 0.2222em;"></span><span class="mord"><span class="mord"><span class="mord accent"><span class="vlist-t"><span class="vlist-r"><span class="vlist" style="height: 0.9495em;"><span class="" style="top: -3em;"><span class="pstrut" style="height: 3em;"></span><span class="mord boldsymbol" style="margin-right: 0.1597em;">W</span></span><span class="" style="top: -3.6051em;"><span class="pstrut" style="height: 3em;"></span><span class="accent-body" style="left: -0.2875em;"><span class="mord mathbf">~</span></span></span></span></span></span></span></span></span><span class="mclose delimcenter" style="top: 0em;"><span class="delimsizing size2">)</span></span></span><span class="mclose delimcenter" style="top: 0em;"><span class="delimsizing size2">)</span></span></span><span class="msupsub"><span class="vlist-t"><span class="vlist-r"><span class="vlist" style="height: 1.3812em;"><span class="" style="top: -3.6029em; margin-right: 0.05em;"><span class="pstrut" style="height: 2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mathnormal mtight" style="margin-right: 0.1389em;">T</span></span></span></span></span></span></span></span><span class="mclose delimcenter" style="top: 0em;"><span class="delimsizing size3">)</span></span></span></span></span><span class="" style="top: -1.6737em;"><span class="pstrut" style="height: 3.6514em;"></span><span class="mord"><span class="mord"></span><span class="mspace" style="margin-right: 0.2778em;"></span><span class="mrel">=</span><span class="mspace" style="margin-right: 0.2778em;"></span><span class="mord"><span class="mop op-limits"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height: 0.6679em;"><span class="" style="top: -2.3537em; margin-left: 0em;"><span class="pstrut" style="height: 3em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight"><span class="mord mtight"><span class="mord mtight"><span class="mord boldsymbol mtight" style="margin-right: 0.0698em;">Z</span></span></span></span></span></span><span class="" style="top: -3em;"><span class="pstrut" style="height: 3em;"></span><span class=""><span class="mop"><span class="mop">min</span></span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height: 0.7463em;"><span class=""></span></span></span></span></span></span><span class="mspace" style="margin-right: 0.1667em;"></span><span class="mspace" style="margin-right: 0.1667em;"></span><span class="mord"><span class="mord mathrm">tr</span></span><span class="mspace" style="margin-right: 0.1667em;"></span><span class="minner"><span class="mopen delimcenter" style="top: 0em;"><span class="delimsizing size2">(</span></span><span class="mord"><span class="mord"><span class="mord"><span class="mord boldsymbol" style="margin-right: 0.0698em;">ZMZ</span></span></span><span class="msupsub"><span class="vlist-t"><span class="vlist-r"><span class="vlist" style="height: 0.9173em;"><span class="" style="top: -3.139em; margin-right: 0.05em;"><span class="pstrut" style="height: 2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mathnormal mtight" style="margin-right: 0.1389em;">T</span></span></span></span></span></span></span></span><span class="mclose delimcenter" style="top: 0em;"><span class="delimsizing size2">)</span></span></span><span class="mspace" style="margin-right: 0.1667em;"></span><span class="mspace" style="margin-right: 0.1667em;"></span></span></span><span class="" style="top: 0.29em;"><span class="pstrut" style="height: 3.6514em;"></span><span class="mord"><span class="mord"></span><span class="mspace" style="margin-right: 0.2778em;"></span><span class="mrel">=</span><span class="mspace" style="margin-right: 0.2778em;"></span><span class="minner"><span class="mopen delimcenter" style="top: 0em;">(</span><span class="mord mathnormal">m</span><span class="mspace" style="margin-right: 0.2222em;"></span><span class="mbin">−</span><span class="mspace" style="margin-right: 0.2222em;"></span><span class="mord">1</span><span class="mclose delimcenter" style="top: 0em;">)</span></span><span class="mspace" style="margin-right: 0.1667em;"></span><span class="mord mathnormal" style="margin-right: 0.0785em;">I</span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height: 4.6014em;"><span class=""></span></span></span></span></span></span></span></span></span></span></span></span></p>

采用拉格朗日乘子法

      L 
     
     
     
       ( 
      
     
       Z 
      
     
       , 
      
     
       Λ 
      
     
       ) 
      
     
    
      = 
     
     
     
       t 
      
     
       r 
      
     
     
     
       ( 
      
      
       
        
        
          Z 
         
        
          M 
         
        
          Z 
         
        
       
      
        T 
       
      
     
       ) 
      
     
    
      + 
     
     
     
       t 
      
     
       r 
      
     
     
     
       ( 
      
      
      
        Λ 
       
      
        T 
       
      
      
      
        ( 
       
       
        
         
         
           Z 
          
         
           Z 
          
         
        
       
         T 
        
       
      
        − 
       
       
       
         ( 
        
       
         m 
        
       
         − 
        
       
         1 
        
       
         ) 
        
       
      
        I 
       
      
        ) 
       
      
     
       ) 
      
     
    
   
     L\left( \boldsymbol{Z},\boldsymbol{\varLambda } \right) =\mathrm{tr}\left( \boldsymbol{ZMZ}^T \right) +\mathrm{tr}\left( \boldsymbol{\varLambda }^T\left( \boldsymbol{ZZ}^T-\left( m-1 \right) \boldsymbol{I} \right) \right) 
    
   
 </span><span class="katex-html"><span class="base"><span class="strut" style="height: 1em; vertical-align: -0.25em;"></span><span class="mord mathnormal">L</span><span class="mspace" style="margin-right: 0.1667em;"></span><span class="minner"><span class="mopen delimcenter" style="top: 0em;">(</span><span class="mord"><span class="mord"><span class="mord boldsymbol" style="margin-right: 0.0698em;">Z</span></span></span><span class="mpunct">,</span><span class="mspace" style="margin-right: 0.1667em;"></span><span class="mord"><span class="mord"><span class="mord mathit">Λ</span></span></span><span class="mclose delimcenter" style="top: 0em;">)</span></span><span class="mspace" style="margin-right: 0.2778em;"></span><span class="mrel">=</span><span class="mspace" style="margin-right: 0.2778em;"></span></span><span class="base"><span class="strut" style="height: 1.8em; vertical-align: -0.65em;"></span><span class="mord"><span class="mord mathrm">tr</span></span><span class="mspace" style="margin-right: 0.1667em;"></span><span class="minner"><span class="mopen delimcenter" style="top: 0em;"><span class="delimsizing size2">(</span></span><span class="mord"><span class="mord"><span class="mord"><span class="mord boldsymbol" style="margin-right: 0.0698em;">ZMZ</span></span></span><span class="msupsub"><span class="vlist-t"><span class="vlist-r"><span class="vlist" style="height: 0.9173em;"><span class="" style="top: -3.139em; margin-right: 0.05em;"><span class="pstrut" style="height: 2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mathnormal mtight" style="margin-right: 0.1389em;">T</span></span></span></span></span></span></span></span><span class="mclose delimcenter" style="top: 0em;"><span class="delimsizing size2">)</span></span></span><span class="mspace" style="margin-right: 0.2222em;"></span><span class="mbin">+</span><span class="mspace" style="margin-right: 0.2222em;"></span></span><span class="base"><span class="strut" style="height: 1.8em; vertical-align: -0.65em;"></span><span class="mord"><span class="mord mathrm">tr</span></span><span class="mspace" style="margin-right: 0.1667em;"></span><span class="minner"><span class="mopen delimcenter" style="top: 0em;"><span class="delimsizing size2">(</span></span><span class="mord"><span class="mord"><span class="mord"><span class="mord mathit">Λ</span></span></span><span class="msupsub"><span class="vlist-t"><span class="vlist-r"><span class="vlist" style="height: 0.9146em;"><span class="" style="top: -3.1362em; margin-right: 0.05em;"><span class="pstrut" style="height: 2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mathnormal mtight" style="margin-right: 0.1389em;">T</span></span></span></span></span></span></span></span><span class="mspace" style="margin-right: 0.1667em;"></span><span class="minner"><span class="mopen delimcenter" style="top: 0em;"><span class="delimsizing size2">(</span></span><span class="mord"><span class="mord"><span class="mord"><span class="mord boldsymbol" style="margin-right: 0.0698em;">ZZ</span></span></span><span class="msupsub"><span class="vlist-t"><span class="vlist-r"><span class="vlist" style="height: 0.9173em;"><span class="" style="top: -3.139em; margin-right: 0.05em;"><span class="pstrut" style="height: 2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mathnormal mtight" style="margin-right: 0.1389em;">T</span></span></span></span></span></span></span></span><span class="mspace" style="margin-right: 0.2222em;"></span><span class="mbin">−</span><span class="mspace" style="margin-right: 0.2222em;"></span><span class="minner"><span class="mopen delimcenter" style="top: 0em;">(</span><span class="mord mathnormal">m</span><span class="mspace" style="margin-right: 0.2222em;"></span><span class="mbin">−</span><span class="mspace" style="margin-right: 0.2222em;"></span><span class="mord">1</span><span class="mclose delimcenter" style="top: 0em;">)</span></span><span class="mspace" style="margin-right: 0.1667em;"></span><span class="mord"><span class="mord"><span class="mord boldsymbol" style="margin-right: 0.0778em;">I</span></span></span><span class="mclose delimcenter" style="top: 0em;"><span class="delimsizing size2">)</span></span></span><span class="mclose delimcenter" style="top: 0em;"><span class="delimsizing size2">)</span></span></span></span></span></span></span></span></p>

令 

       ∂ 
      
     
       L 
      
      
      
        ( 
       
      
        Z 
       
      
        , 
       
      
        Λ 
       
      
        ) 
       
      
     
    
      / 
     
     
     
       ∂ 
      
     
       Z 
      
     
    
   
     = 
    
   
     2 
    
    
     
     
       Z 
      
     
       M 
      
     
    
   
     + 
    
   
     2 
    
    
     
     
       Λ 
      
     
       Z 
      
     
    
   
     = 
    
   
     0 
    
   
  
    {<!-- -->{\partial L\left( \boldsymbol{Z},\boldsymbol{\varLambda } \right)}/{\partial \boldsymbol{Z}}}=2\boldsymbol{ZM}+2\boldsymbol{\varLambda Z}=0 
   
  
</span><span class="katex-html"><span class="base"><span class="strut" style="height: 1em; vertical-align: -0.25em;"></span><span class="mord"><span class="mord"><span class="mord" style="margin-right: 0.0556em;">∂</span><span class="mord mathnormal">L</span><span class="mspace" style="margin-right: 0.1667em;"></span><span class="minner"><span class="mopen delimcenter" style="top: 0em;">(</span><span class="mord"><span class="mord"><span class="mord boldsymbol" style="margin-right: 0.0698em;">Z</span></span></span><span class="mpunct">,</span><span class="mspace" style="margin-right: 0.1667em;"></span><span class="mord"><span class="mord"><span class="mord mathit">Λ</span></span></span><span class="mclose delimcenter" style="top: 0em;">)</span></span></span><span class="mord">/</span><span class="mord"><span class="mord" style="margin-right: 0.0556em;">∂</span><span class="mord"><span class="mord"><span class="mord boldsymbol" style="margin-right: 0.0698em;">Z</span></span></span></span></span><span class="mspace" style="margin-right: 0.2778em;"></span><span class="mrel">=</span><span class="mspace" style="margin-right: 0.2778em;"></span></span><span class="base"><span class="strut" style="height: 0.7694em; vertical-align: -0.0833em;"></span><span class="mord">2</span><span class="mord"><span class="mord"><span class="mord boldsymbol" style="margin-right: 0.1142em;">ZM</span></span></span><span class="mspace" style="margin-right: 0.2222em;"></span><span class="mbin">+</span><span class="mspace" style="margin-right: 0.2222em;"></span></span><span class="base"><span class="strut" style="height: 0.6861em;"></span><span class="mord">2</span><span class="mord"><span class="mord"><span class="mord mathit">Λ</span><span class="mord boldsymbol" style="margin-right: 0.0698em;">Z</span></span></span><span class="mspace" style="margin-right: 0.2778em;"></span><span class="mrel">=</span><span class="mspace" style="margin-right: 0.2778em;"></span></span><span class="base"><span class="strut" style="height: 0.6444em;"></span><span class="mord">0</span></span></span></span></span>，移项转置可得<span class="katex--inline"><span class="katex"><span class="katex-mathml"> 
 
  
   
    
     
      
      
        M 
       
      
        Z 
       
      
     
    
      T 
     
    
   
     = 
    
   
     − 
    
    
    
      Z 
     
    
      T 
     
    
   
     Λ 
    
   
  
    \boldsymbol{MZ}^T=-\boldsymbol{Z}^T\boldsymbol{\varLambda } 
   
  
</span><span class="katex-html"><span class="base"><span class="strut" style="height: 0.9173em;"></span><span class="mord"><span class="mord"><span class="mord"><span class="mord boldsymbol" style="margin-right: 0.0698em;">MZ</span></span></span><span class="msupsub"><span class="vlist-t"><span class="vlist-r"><span class="vlist" style="height: 0.9173em;"><span class="" style="top: -3.139em; margin-right: 0.05em;"><span class="pstrut" style="height: 2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mathnormal mtight" style="margin-right: 0.1389em;">T</span></span></span></span></span></span></span></span><span class="mspace" style="margin-right: 0.2778em;"></span><span class="mrel">=</span><span class="mspace" style="margin-right: 0.2778em;"></span></span><span class="base"><span class="strut" style="height: 1.0007em; vertical-align: -0.0833em;"></span><span class="mord">−</span><span class="mord"><span class="mord"><span class="mord"><span class="mord boldsymbol" style="margin-right: 0.0698em;">Z</span></span></span><span class="msupsub"><span class="vlist-t"><span class="vlist-r"><span class="vlist" style="height: 0.9173em;"><span class="" style="top: -3.139em; margin-right: 0.05em;"><span class="pstrut" style="height: 2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mathnormal mtight" style="margin-right: 0.1389em;">T</span></span></span></span></span></span></span></span><span class="mord"><span class="mord"><span class="mord mathit">Λ</span></span></span></span></span></span></span>。设<span class="katex--inline"><span class="katex"><span class="katex-mathml"> 
 
  
   
   
     Y 
    
   
     = 
    
    
    
      Z 
     
    
      T 
     
    
   
     ∈ 
    
    
    
      R 
     
     
     
       m 
      
     
       × 
      
     
       d 
      
     
    
   
  
    \boldsymbol{Y}=\boldsymbol{Z}^T\in \mathbb{R} ^{m\times d} 
   
  
</span><span class="katex-html"><span class="base"><span class="strut" style="height: 0.6861em;"></span><span class="mord"><span class="mord"><span class="mord boldsymbol" style="margin-right: 0.2555em;">Y</span></span></span><span class="mspace" style="margin-right: 0.2778em;"></span><span class="mrel">=</span><span class="mspace" style="margin-right: 0.2778em;"></span></span><span class="base"><span class="strut" style="height: 0.9564em; vertical-align: -0.0391em;"></span><span class="mord"><span class="mord"><span class="mord"><span class="mord boldsymbol" style="margin-right: 0.0698em;">Z</span></span></span><span class="msupsub"><span class="vlist-t"><span class="vlist-r"><span class="vlist" style="height: 0.9173em;"><span class="" style="top: -3.139em; margin-right: 0.05em;"><span class="pstrut" style="height: 2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mathnormal mtight" style="margin-right: 0.1389em;">T</span></span></span></span></span></span></span></span><span class="mspace" style="margin-right: 0.2778em;"></span><span class="mrel">∈</span><span class="mspace" style="margin-right: 0.2778em;"></span></span><span class="base"><span class="strut" style="height: 0.8491em;"></span><span class="mord"><span class="mord mathbb">R</span><span class="msupsub"><span class="vlist-t"><span class="vlist-r"><span class="vlist" style="height: 0.8491em;"><span class="" style="top: -3.063em; margin-right: 0.05em;"><span class="pstrut" style="height: 2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight"><span class="mord mathnormal mtight">m</span><span class="mbin mtight">×</span><span class="mord mathnormal mtight">d</span></span></span></span></span></span></span></span></span></span></span></span></span>，则有<span class="katex--inline"><span class="katex"><span class="katex-mathml"> 
 
  
   
    
     
     
       M 
      
     
       Y 
      
     
    
   
     = 
    
   
     − 
    
    
     
     
       Y 
      
     
       Λ 
      
     
    
   
  
    \boldsymbol{MY}=-\boldsymbol{Y\varLambda } 
   
  
</span><span class="katex-html"><span class="base"><span class="strut" style="height: 0.6861em;"></span><span class="mord"><span class="mord"><span class="mord boldsymbol" style="margin-right: 0.2555em;">MY</span></span></span><span class="mspace" style="margin-right: 0.2778em;"></span><span class="mrel">=</span><span class="mspace" style="margin-right: 0.2778em;"></span></span><span class="base"><span class="strut" style="height: 0.7694em; vertical-align: -0.0833em;"></span><span class="mord">−</span><span class="mord"><span class="mord"><span class="mord boldsymbol" style="margin-right: 0.2555em;">Y</span><span class="mord mathit">Λ</span></span></span></span></span></span></span>，这与<a href="https://mr-winter.blog.youkuaiyun.com/article/details/128699242">主成分分析</a>类似，只需对<span class="katex--inline"><span class="katex"><span class="katex-mathml"> 
 
  
   
   
     M 
    
   
  
    \boldsymbol{M} 
   
  
</span><span class="katex-html"><span class="base"><span class="strut" style="height: 0.6861em;"></span><span class="mord"><span class="mord"><span class="mord boldsymbol" style="margin-right: 0.1142em;">M</span></span></span></span></span></span></span>特征值分解，并取<span class="katex--inline"><span class="katex"><span class="katex-mathml"> 
 
  
   
    
    
      d 
     
    
      ′ 
     
    
   
     ≪ 
    
   
     d 
    
   
  
    d'\ll d 
   
  
</span><span class="katex-html"><span class="base"><span class="strut" style="height: 0.791em; vertical-align: -0.0391em;"></span><span class="mord"><span class="mord mathnormal">d</span><span class="msupsub"><span class="vlist-t"><span class="vlist-r"><span class="vlist" style="height: 0.7519em;"><span class="" style="top: -3.063em; margin-right: 0.05em;"><span class="pstrut" style="height: 2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight"><span class="mord mtight">′</span></span></span></span></span></span></span></span></span><span class="mspace" style="margin-right: 0.2778em;"></span><span class="mrel">≪</span><span class="mspace" style="margin-right: 0.2778em;"></span></span><span class="base"><span class="strut" style="height: 0.6944em;"></span><span class="mord mathnormal">d</span></span></span></span></span>个最小特征值对应的特征向量构成<span class="katex--inline"><span class="katex"><span class="katex-mathml"> 
 
  
   
    
    
      Y 
     
    
      ∗ 
     
    
   
     ∈ 
    
    
    
      R 
     
     
     
       m 
      
     
       × 
      
      
      
        d 
       
      
        ′ 
       
      
     
    
   
  
    \boldsymbol{Y}^*\in \mathbb{R} ^{m\times d'} 
   
  
</span><span class="katex-html"><span class="base"><span class="strut" style="height: 0.8038em; vertical-align: -0.0391em;"></span><span class="mord"><span class="mord"><span class="mord"><span class="mord boldsymbol" style="margin-right: 0.2555em;">Y</span></span></span><span class="msupsub"><span class="vlist-t"><span class="vlist-r"><span class="vlist" style="height: 0.7647em;"><span class="" style="top: -3.139em; margin-right: 0.05em;"><span class="pstrut" style="height: 2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mbin mtight">∗</span></span></span></span></span></span></span></span><span class="mspace" style="margin-right: 0.2778em;"></span><span class="mrel">∈</span><span class="mspace" style="margin-right: 0.2778em;"></span></span><span class="base"><span class="strut" style="height: 0.9425em;"></span><span class="mord"><span class="mord mathbb">R</span><span class="msupsub"><span class="vlist-t"><span class="vlist-r"><span class="vlist" style="height: 0.9425em;"><span class="" style="top: -3.063em; margin-right: 0.05em;"><span class="pstrut" style="height: 2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight"><span class="mord mathnormal mtight">m</span><span class="mbin mtight">×</span><span class="mord mtight"><span class="mord mathnormal mtight">d</span><span class="msupsub"><span class="vlist-t"><span class="vlist-r"><span class="vlist" style="height: 0.8278em;"><span class="" style="top: -2.931em; margin-right: 0.0714em;"><span class="pstrut" style="height: 2.5em;"></span><span class="sizing reset-size3 size1 mtight"><span class="mord mtight"><span class="mord mtight">′</span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span></span>，转置后即得低维样本<span class="katex--inline"><span class="katex"><span class="katex-mathml"> 
 
  
   
    
    
      Z 
     
    
      ∗ 
     
    
   
     ∈ 
    
    
    
      R 
     
     
      
      
        d 
       
      
        ′ 
       
      
     
       × 
      
     
       m 
      
     
    
   
  
    \boldsymbol{Z}^*\in \mathbb{R} ^{d'\times m} 
   
  
</span><span class="katex-html"><span class="base"><span class="strut" style="height: 0.8038em; vertical-align: -0.0391em;"></span><span class="mord"><span class="mord"><span class="mord"><span class="mord boldsymbol" style="margin-right: 0.0698em;">Z</span></span></span><span class="msupsub"><span class="vlist-t"><span class="vlist-r"><span class="vlist" style="height: 0.7647em;"><span class="" style="top: -3.139em; margin-right: 0.05em;"><span class="pstrut" style="height: 2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mbin mtight">∗</span></span></span></span></span></span></span></span><span class="mspace" style="margin-right: 0.2778em;"></span><span class="mrel">∈</span><span class="mspace" style="margin-right: 0.2778em;"></span></span><span class="base"><span class="strut" style="height: 0.9425em;"></span><span class="mord"><span class="mord mathbb">R</span><span class="msupsub"><span class="vlist-t"><span class="vlist-r"><span class="vlist" style="height: 0.9425em;"><span class="" style="top: -3.063em; margin-right: 0.05em;"><span class="pstrut" style="height: 2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight"><span class="mord mtight"><span class="mord mathnormal mtight">d</span><span class="msupsub"><span class="vlist-t"><span class="vlist-r"><span class="vlist" style="height: 0.8278em;"><span class="" style="top: -2.931em; margin-right: 0.0714em;"><span class="pstrut" style="height: 2.5em;"></span><span class="sizing reset-size3 size1 mtight"><span class="mord mtight"><span class="mord mtight">′</span></span></span></span></span></span></span></span></span><span class="mbin mtight">×</span><span class="mord mathnormal mtight">m</span></span></span></span></span></span></span></span></span></span></span></span></span></p>

必须指出， 

     M 
    
   
  
    \boldsymbol{M} 
   
  
</span><span class="katex-html"><span class="base"><span class="strut" style="height: 0.6861em;"></span><span class="mord"><span class="mord"><span class="mord boldsymbol" style="margin-right: 0.1142em;">M</span></span></span></span></span></span></span>最小的特征值往往接近0，原因是根据权重和为一<span class="katex--inline"><span class="katex"><span class="katex-mathml"> 
 
  
   
    
     
      
      
        W 
       
      
        ~ 
       
      
     
    
      T 
     
    
   
     1 
    
   
     = 
    
   
     1 
    
   
  
    \boldsymbol{\tilde{W}}^T\mathbf{1}=\mathbf{1} 
   
  
</span><span class="katex-html"><span class="base"><span class="strut" style="height: 1.1808em;"></span><span class="mord"><span class="mord"><span class="mord"><span class="mord accent"><span class="vlist-t"><span class="vlist-r"><span class="vlist" style="height: 0.9495em;"><span class="" style="top: -3em;"><span class="pstrut" style="height: 3em;"></span><span class="mord boldsymbol" style="margin-right: 0.1597em;">W</span></span><span class="" style="top: -3.6051em;"><span class="pstrut" style="height: 3em;"></span><span class="accent-body" style="left: -0.2875em;"><span class="mord mathbf">~</span></span></span></span></span></span></span></span></span><span class="msupsub"><span class="vlist-t"><span class="vlist-r"><span class="vlist" style="height: 1.1808em;"><span class="" style="top: -3.4024em; margin-right: 0.05em;"><span class="pstrut" style="height: 2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mathnormal mtight" style="margin-right: 0.1389em;">T</span></span></span></span></span></span></span></span><span class="mord mathbf">1</span><span class="mspace" style="margin-right: 0.2778em;"></span><span class="mrel">=</span><span class="mspace" style="margin-right: 0.2778em;"></span></span><span class="base"><span class="strut" style="height: 0.6444em;"></span><span class="mord mathbf">1</span></span></span></span></span>可得<span class="katex--inline"><span class="katex"><span class="katex-mathml"> 
 
  
   
    
    
      ( 
     
    
      I 
     
    
      − 
     
     
      
       
       
         W 
        
       
         ~ 
        
       
      
     
       T 
      
     
    
      ) 
     
    
   
     1 
    
   
     = 
    
    
     
     
       ( 
      
     
       I 
      
     
       − 
      
      
       
       
         W 
        
       
         ~ 
        
       
      
     
       ) 
      
     
    
      T 
     
    
   
     1 
    
   
     = 
    
   
     0 
    
   
  
    \left( \boldsymbol{I}-\boldsymbol{\tilde{W}}^T \right) \mathbf{1}=\left( \boldsymbol{I}-\boldsymbol{\tilde{W}} \right) ^T\mathbf{1}=\mathbf{0} 
   
  
</span><span class="katex-html"><span class="base"><span class="strut" style="height: 1.8308em; vertical-align: -0.65em;"></span><span class="minner"><span class="mopen delimcenter" style="top: 0em;"><span class="delimsizing size2">(</span></span><span class="mord"><span class="mord"><span class="mord boldsymbol" style="margin-right: 0.0778em;">I</span></span></span><span class="mspace" style="margin-right: 0.2222em;"></span><span class="mbin">−</span><span class="mspace" style="margin-right: 0.2222em;"></span><span class="mord"><span class="mord"><span class="mord"><span class="mord accent"><span class="vlist-t"><span class="vlist-r"><span class="vlist" style="height: 0.9495em;"><span class="" style="top: -3em;"><span class="pstrut" style="height: 3em;"></span><span class="mord boldsymbol" style="margin-right: 0.1597em;">W</span></span><span class="" style="top: -3.6051em;"><span class="pstrut" style="height: 3em;"></span><span class="accent-body" style="left: -0.2875em;"><span class="mord mathbf">~</span></span></span></span></span></span></span></span></span><span class="msupsub"><span class="vlist-t"><span class="vlist-r"><span class="vlist" style="height: 1.1808em;"><span class="" style="top: -3.4024em; margin-right: 0.05em;"><span class="pstrut" style="height: 2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mathnormal mtight" style="margin-right: 0.1389em;">T</span></span></span></span></span></span></span></span><span class="mclose delimcenter" style="top: 0em;"><span class="delimsizing size2">)</span></span></span><span class="mspace" style="margin-right: 0.1667em;"></span><span class="mord mathbf">1</span><span class="mspace" style="margin-right: 0.2778em;"></span><span class="mrel">=</span><span class="mspace" style="margin-right: 0.2778em;"></span></span><span class="base"><span class="strut" style="height: 2.0313em; vertical-align: -0.65em;"></span><span class="minner"><span class="minner"><span class="mopen delimcenter" style="top: 0em;"><span class="delimsizing size2">(</span></span><span class="mord"><span class="mord"><span class="mord boldsymbol" style="margin-right: 0.0778em;">I</span></span></span><span class="mspace" style="margin-right: 0.2222em;"></span><span class="mbin">−</span><span class="mspace" style="margin-right: 0.2222em;"></span><span class="mord"><span class="mord"><span class="mord accent"><span class="vlist-t"><span class="vlist-r"><span class="vlist" style="height: 0.9495em;"><span class="" style="top: -3em;"><span class="pstrut" style="height: 3em;"></span><span class="mord boldsymbol" style="margin-right: 0.1597em;">W</span></span><span class="" style="top: -3.6051em;"><span class="pstrut" style="height: 3em;"></span><span class="accent-body" style="left: -0.2875em;"><span class="mord mathbf">~</span></span></span></span></span></span></span></span></span><span class="mclose delimcenter" style="top: 0em;"><span class="delimsizing size2">)</span></span></span><span class="msupsub"><span class="vlist-t"><span class="vlist-r"><span class="vlist" style="height: 1.3812em;"><span class="" style="top: -3.6029em; margin-right: 0.05em;"><span class="pstrut" style="height: 2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mathnormal mtight" style="margin-right: 0.1389em;">T</span></span></span></span></span></span></span></span><span class="mspace" style="margin-right: 0.1667em;"></span><span class="mord mathbf">1</span><span class="mspace" style="margin-right: 0.2778em;"></span><span class="mrel">=</span><span class="mspace" style="margin-right: 0.2778em;"></span></span><span class="base"><span class="strut" style="height: 0.6444em;"></span><span class="mord mathbf">0</span></span></span></span></span>，两边同时左乘<span class="katex--inline"><span class="katex"><span class="katex-mathml"> 
 
  
   
   
     ( 
    
   
     I 
    
   
     − 
    
    
     
     
       W 
      
     
       ~ 
      
     
    
   
     ) 
    
   
  
    \left( \boldsymbol{I}-\boldsymbol{\tilde{W}} \right) 
   
  
</span><span class="katex-html"><span class="base"><span class="strut" style="height: 1.8em; vertical-align: -0.65em;"></span><span class="minner"><span class="mopen delimcenter" style="top: 0em;"><span class="delimsizing size2">(</span></span><span class="mord"><span class="mord"><span class="mord boldsymbol" style="margin-right: 0.0778em;">I</span></span></span><span class="mspace" style="margin-right: 0.2222em;"></span><span class="mbin">−</span><span class="mspace" style="margin-right: 0.2222em;"></span><span class="mord"><span class="mord"><span class="mord accent"><span class="vlist-t"><span class="vlist-r"><span class="vlist" style="height: 0.9495em;"><span class="" style="top: -3em;"><span class="pstrut" style="height: 3em;"></span><span class="mord boldsymbol" style="margin-right: 0.1597em;">W</span></span><span class="" style="top: -3.6051em;"><span class="pstrut" style="height: 3em;"></span><span class="accent-body" style="left: -0.2875em;"><span class="mord mathbf">~</span></span></span></span></span></span></span></span></span><span class="mclose delimcenter" style="top: 0em;"><span class="delimsizing size2">)</span></span></span></span></span></span></span>有<span class="katex--inline"><span class="katex"><span class="katex-mathml"> 
 
  
   
   
     M 
    
   
     1 
    
   
     = 
    
   
     0 
    
   
  
    \boldsymbol{M}\mathbf{1}=\mathbf{0} 
   
  
</span><span class="katex-html"><span class="base"><span class="strut" style="height: 0.6861em;"></span><span class="mord"><span class="mord"><span class="mord boldsymbol" style="margin-right: 0.1142em;">M</span></span></span><span class="mord mathbf">1</span><span class="mspace" style="margin-right: 0.2778em;"></span><span class="mrel">=</span><span class="mspace" style="margin-right: 0.2778em;"></span></span><span class="base"><span class="strut" style="height: 0.6444em;"></span><span class="mord mathbf">0</span></span></span></span></span>，即此时特征向量为全1向量，无法反应数据特征，所以实际应用中会选取第<span class="katex--inline"><span class="katex"><span class="katex-mathml"> 
 
  
   
   
     2 
    
   
     &nbsp; 
    
    
    
      d 
     
    
      ′ 
     
    
   
     + 
    
   
     1 
    
   
  
    2~d'+1 
   
  
</span><span class="katex-html"><span class="base"><span class="strut" style="height: 0.8352em; vertical-align: -0.0833em;"></span><span class="mord">2</span><span class="mspace nobreak">&nbsp;</span><span class="mord"><span class="mord mathnormal">d</span><span class="msupsub"><span class="vlist-t"><span class="vlist-r"><span class="vlist" style="height: 0.7519em;"><span class="" style="top: -3.063em; margin-right: 0.05em;"><span class="pstrut" style="height: 2.7em;"></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight"><span class="mord mtight">′</span></span></span></span></span></span></span></span></span><span class="mspace" style="margin-right: 0.2222em;"></span><span class="mbin">+</span><span class="mspace" style="margin-right: 0.2222em;"></span></span><span class="base"><span class="strut" style="height: 0.6444em;"></span><span class="mord">1</span></span></span></span></span>个最小特征值对应的特征向量。</p>

3 Python实现

3.1 算法流程

LLE算法流程如表所示，原理与第二节一一对应，不熟悉的同学可以往上翻

在这里插入图片描述

3.2 核心代码

LLE核心代码实现如下

def run(self, outDim):
     # 获得距离矩阵
     D = self.calDist()
     D[D < 0] = 0
     # 获得样本近邻索引矩阵
     DIndex = np.argsort(D, axis=1)[:, 1:self.k + 1]
     # 重构系数矩阵
     W = np.zeros([self.m, self.m])
     # 全1向量 kx1
     unitK = np.ones([self.k, 1])
     # 根据近邻样本确定重构参数
     for i in range(self.m):
         # Xi近邻距离矩阵
         Xi = self.X[:, i].reshape(self.d, 1) - self.X[:, DIndex[i, :]]
         # 求逆，附加项防止矩阵不稳定
         XiInv = np.linalg.pinv(np.dot(Xi.T, Xi) + np.eye(self.k) * self.tol)
         wi = np.dot(XiInv, unitK) / np.dot(np.dot(unitK.T , XiInv), unitK)
         for j in range(self.k):
             W[DIndex[i, j], i] = wi[j]
     # 计算M矩阵
     M = np.dot((np.eye(self.m) - W), (np.eye(self.m) - W).T)
     # 特征值分解
     eigVal, eigVec = np.linalg.eig(M)
     # 获取最小的d'个特征值对应的索引
     index = np.argsort(np.abs(eigVal))[1:outDim + 1]
     eigVec_ = eigVec[:, index]
     # 计算低维样本
     Z = eigVec_.T
     return Z

 
 
 
 

  
  
 
 

 
 
  • 1
  • 2
  • 3
  • 4
  • 5
  • 6
  • 7
  • 8
  • 9
  • 10
  • 11
  • 12
  • 13
  • 14
  • 15
  • 16
  • 17
  • 18
  • 19
  • 20
  • 21
  • 22
  • 23
  • 24
  • 25
  • 26
  • 27
  • 28
  • 29

3.3 可视化

在这里插入图片描述

在这里插入图片描述

本文完整工程代码请通过下方名片联系博主获取

机器学习强基计划8-5：图解局部线性嵌入LLE算法(附Python实现)
FRIGIDWINTER的博客
05-04 2451
局部线性嵌入(Locally Linear Embedding, LLE)限制样本在降维后的低维空间中的k近邻局部线性关系，等价于原始空间。本文详解LLE算法原理并给出Python实现
机器学习强基计划0-1：教程导读(附几十种经典模型源码合集)
FRIGIDWINTER的博客
07-13 8257
订阅专栏前，请先看本文章简介
python进行局部线性嵌入（LLE）LocallyLinearEmbedding
zhangzhechun的专栏
03-20 911
例如，在二维空间中，数据点可以位于一条曲线上，而在三维空间中，数据点可以位于一个曲面上。因此，我们可以通过对每个数据点在其邻域内进行局部线性逼近，然后构建一个局部线性关系矩阵，最后通过求解一个优化问题来获得每个数据点在低维空间中的表示。在数据分析中，三维欧几里得空间也经常被用来表示数据，例如，在可视化数据时，我们可以使用三维坐标来显示数据点，以便更好地理解数据之间的关系。需要注意的是，LLE 的计算量较大，且对于高维数据，需要选择较大的局部邻域和局部线性回归维度，否则可能会造成降维效果的下降。
机器学习降维算法三：LLE (Locally Linear Embedding) 局部线性嵌入
weixin_34066347的博客
07-09 1003
如引用请务必注明此文出自：http://www.cnblogs.com/xbinworld LLE Locally linear embedding（LLE）[1] 是一种非线性降维算法，它能够使降维后的数据较好地保持原有流形结构。LLE可以说是流形学习方法最经典的工作之一。很多后续的流形学习、降维方法都与LLE有密切联系。 见图1，使用LLE将三维数据（b）映射到二维（c）之后，...
降维算法之局部线性嵌入
最新发布
qq_45670407的博客
04-04 652
降维算法之局部线性嵌入
局部线性嵌入(LLE)原理总结
weixin_41643439的博客
03-21 1237
https://www.cnblogs.com/pinard/p/6266408.html 局部线性嵌入(Locally Linear Embedding，以下简称LLE)也是非常重要的降维方法。和传统的PCA，LDA等关注样本方差的降维方法相比，LLE关注于降维时保持样本局部的线性特征，由于LLE在降维时保持了样本的局部特征，它广泛的用于图像图像识别，高维数据可视化等领域。下面我们就对L...
LLE - 局部线性嵌入算法(locallylinearembedding,LLE)
10-10
LLE - 局部线性嵌入算法(locallylinearembedding,LLE)是机器学习领域常用的数据降维方法，资源为该方法的matlab实现
LLE局部线性嵌入算法
热门推荐
Eleven-Seven工作小空间
07-09 1万+
LLE局部线性嵌入算法 导语：很久没发博文了，今日抽个小空，整理下上个学期做过的东西，写成博文，供给初学者参考： 一、LLE算法概念 LLE算法，即局部线性嵌入算法，是一种非线性降维算法，它利用线性重构的局部对称性找出高维数据空间中的非线性结构，并在保持各数据点临近位置关系情况下，把高维空间数据点映射为低维空间对应的数据点，它能够使降维后的数据保持原有的流行结构。 二、LLE算法步骤 1、求每
机器学习强基计划8-5：图解局部线性嵌入LLE算法(附Python实现)_lle算法实现流程图
2401_84689346的博客
04-29 758
机器学习强基计划聚焦深度和广度，加深对机器学习模型的理解与应用。“深”在详细推导算法模型背后的数学原理；“广”在分析多个机器学习模型：决策树、支持向量机、贝叶斯与马尔科夫决策、强化学习等。强基计划实现从理论到实践的全面覆盖，由本人亲自从底层编写、测试与文章配套的各个经典算法，不依赖于现有库，可以大大加深对算法的理解。别在网上瞎学了，我最近也做了一些资源的更新，只要你是我的粉丝，这期福利你都可拿走。我先来介绍一下这些东西怎么用，文末抱走。（1）Python所有方向的学习路线（新版）
机器学习强基计划8-1：图解主成分分析PCA算法(附Python实现)_pca 强化学习
2401_84563254的博客
05-03 93
机器学习强基计划聚焦深度和广度，加深对机器学习模型的理解与应用。“深”在详细推导算法模型背后的数学原理；“广”在分析多个机器学习模型：决策树、支持向量机、贝叶斯与马尔科夫决策、强化学习等。强基计划实现从理论到实践的全面覆盖，由本人亲自从底层编写、测试与文章配套的各个经典算法，不依赖于现有库，可以大大加深对算法的理解。机器学习强基计划(附几十种经典模型源码)Python崛起并且风靡，因为优点多、应用领域广、被大牛们认可。
局部线性嵌入降维算法（LLE）
11-02
对局部线性嵌入降维算法讲解的清楚、易懂
局部线性嵌入LLE的matlab代码
10-06
局部线性嵌入LLE的matlab代码，有详细的注释，代入数据就可以运行
降维算法 ❉ 局部线性嵌入（Locally Linear Embedding，LLE）
weixin_43156294的博客
05-22 1033
局部线性嵌入（Locally Linear Embedding，LLE）是一种用于维数约减的非线性技术，由Sam Roweis和Lawrence Saul于2000年提出。LLE的目标是在低维空间中保持数据点之间的局部邻域结构，同时尽可能保持全局结构。
manifold.LocallyLinearEmbedding（LLE）流形学习之局部线性嵌入算法详解
漫步量化
05-06 3421
高维数据难于可视化，需先经过降维处理。 线性降维框架如Principal Component Analysis(PCA,主成分分析)、Independent Component Analysis（独立成分分析）、 Linear Discriminant Analysis（线性判别分析）等，但常会错失数据结构中的非线性项。 Manifold Learing可以看作一种生成类似PCA的线性框架，不同...
【流行学习】局部线性嵌入(Locally Linear Embedding)
ｋ的博客
02-12 4762
一、前言 局部线性嵌入（LLE）假设数据在较小的局部是线性的，也就是说，某一个样本可以由它最近邻的几个样本线性表示，离样本远的样本对局部的线性关系没有影响，因此相比等距映射算法，降维的时间复杂度和空间复杂度都有极大的降低。 比如有一个样本x1x_1x1​ ，我们在它的原始高维邻域里用KKK-近邻思想找到和它最近的三个样本x2,x3,x4x_2,x_3,x_4x2​,x3​,x4​，然后我们假设x1x_1x1​可以由x2,x3,x4x_2,x_3,x_4x2​,x3​,x4​线性表示，即： x1=w12x2+
局部线性嵌入(LLE)算法解释
alaclp的专栏
04-15 5531
局部线性嵌入(LLE)算法   给定N个输入向量{x1,x2,…,xN}，xRd，通过LLE算法，得到输出向量yi，i[1,N]，yi  Rm，m (1)寻找相对于所求点的欧氏距离最近的k个样本点规定为所求点的近邻点，k是一个预先给定的值； (2)由每个样本点的近邻点计算出该样本点的局部重建权值矩阵； (3)由该样本点的局部重建权值矩阵及其近邻点计算出该样本点的输出向量．定义一个误差函数
局部线性嵌入(LLE)算法的详细推导及Python实现
weixin_38053887的博客
03-16 5996
  局部线性嵌入(Local Linear Embedding, LLE)算法是一种经典的流形学习算法，不同于等度量映射(Isometric Mapping, Isomap)算法考虑结点的全局连接信息，LLE算法只考虑每个结点的k近邻信息，所以速度比Isomap快很多。LLE的思想如下图所示，LLE在低维空间保持了原始高维空间样本邻域内的线性关系，这也是局部线性的一个最直接的解释：高维数据在局部上...
局部线性嵌入LLE
煎饼加鸡丝
07-25 948
[1]https://www.cnblogs.com/pinard/p/6266408.html [2]Graph Embedding Techniques, Applications, and Performance: A Survey 主要参考和图片来源[1] 局部线性嵌入(Locally Linear Embedding,LLE)，一种重要降维方法，与PCA、LDA相比...
强基计划点评：自招转强基的教育公平与效率分析
资源摘要信息: "强基计划"是近年来中国教育领域的一个重大改革措施，旨在通过选拔机制的优化，促进教育公平并提高教育效率。强基计划的出台背景主要是为了改进原有的自主招生制度，通过更加科学和规范的方式来选拔和...
评论
添加红包

请填写红包祝福语或标题

个

红包个数最小为10个

元

红包金额最低5元

当前余额3.43元 前往充值 >
需支付：10.00元
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付元
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明：

1.余额是钱包充值的虚拟货币，按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载，可以购买VIP、付费专栏及课程。

余额充值