Machine-Learning–Based Column Selection for Column Generation

本文探讨了在列生成算法中如何通过机器学习加速收敛,特别是使用图神经网络(GNN)进行列选择。在列生成迭代中,选择合适的列可以减少迭代次数,提高算法效率。作者构建了一个MILP模型来选择最有潜力的列,并用GNN学习这一过程,以减少MILP求解所需的时间。实验结果显示,GNN模型在车辆和机组调度问题以及有时间窗口的车辆路径问题中,相比于传统方法能显著减少计算时间。

摘要生成于 C知道 ,由 DeepSeek-R1 满血版支持, 前往体验 >

论文阅读笔记,个人理解,如有错误请指正,感激不尽!仅是对文章进行梳理,细节请阅读参考文献。该文分类到Machine learning alongside optimization algorithms。

01 Column Generation

列生成 (Column Generation) 算法在组合优化领域有着非常广泛的应用,是一种求解大规模问题 (large-scale optimization problems) 的有效算法。在列生成算法中,将大规模线性规划问题分解为主问题 (Master Problem, MP) 和定价子问题 (Pricing Problem, PP)。算法首先将一个MP给restricted到只带少量的columns,得到RMP。求解RMP,得到dual solution,并将其传递给PP,随后求解PP得到相应的column将其加到RMP中。RMP和PP不断迭代求解直到再也找不到检验数为负的column,那么得到的RMP的最优解也是MP的最优解。如下图所示:

关于列生成的具体原理,之前已经写过很多详细的文章了。还不熟悉的小伙伴可以看看以下:

02 Column Selection

在列生成迭代的过程中,有很多技巧可以用来加快算法收敛的速度。其中一个就是在每次迭代的时候,加入多条检验数为负的column,这样可以减少迭代的次数,从而加快算法整体的运行时间。特别是求解一次子问题得到多条column和得到一条column相差的时间不大的情况下(例如,最短路中的labeling算法)。

而每次迭代中选择哪些column加入?是一个值得研究的地方。因为加入的columns不同,最终收敛的速度也大不一样。一方面,我们希望加入column以后,目标函数能尽可能降低(最小化问题);另一方面,我们希望加入的column数目越少越好,太多的列会导致RMP求解难度上升。因此,在每次的迭代中,我们构建一个模型,用来选择一些比较promising的column加入到RMP中:

  • Let ℓ\ell be the CG iteration number
  • Ωℓ\Omega_{\ell}Ω the set of columns present in the RMP at the start of iteration ℓ\ell
  • Gℓ\mathcal{G}_{\ell}G the generated columns at this iteration
  • For each column p∈Gℓp \in \mathcal{G}_{\ell}pG, we define a decision variable ypy_pyp that takes value one if column ppp is selected and zero otherwise

为了最小化所选择的column,每选择一条column的时候,都会发生一个足够小的惩罚ϵ\epsilonϵ。最终,构建column selection的模型 (MILP) 如下:

大家发现没有,如果没有ypy_pyp和约束(8)和(9),那么上面这个模型就直接变成了下一次迭代的RMP了。

假设ϵ\epsilonϵ足够小,这些约束目的是使得被选中添加到RMP中的column数量最小化,也就是这些yp=1y_p=1yp=1的columns。那么在迭代ℓ\ell中要添加到RMP的的column为:

总体的流程如下图所示:

03 Graph Neural Networks

在每次迭代中,通过求解MILP,可以知道加入哪些column有助于算法速度的提高,但是求解MILP也需要一定的时间,最终不一定能达到加速的目的。因此通过机器学习的方法,学习一个MILP的模型,每次给出选中的column,将是一种比较可行的方法。

在此之前,先介绍一下Graph Neural Networks。图神经网络(GNNs)是通过图节点之间的信息传递来获取图的依赖性的连接模型。与标准神经网络不同,图神经网络可以以任意深度表示来自其邻域的信息。

给定一个网络G=(V,E)G=(V,E)G=(V,E),其中VVV是顶点集而EEE是边集。每一个节点v∈Vv \in Vv

### ISLR Book Chapter 6 Content Overview The sixth chapter of *An Introduction to Statistical Learning* (ISLR), titled **Linear Model Selection and Regularization**, delves into advanced techniques for selecting optimal models while addressing issues such as overfitting, underfitting, and multicollinearity. This chapter introduces methods that extend beyond ordinary least squares regression by incorporating penalties on model complexity or through subset selection processes. #### Key Topics Covered in Chapter 6: 1. **Subset Selection**: Subset selection involves identifying a smaller subset of predictors from the full set available to fit a model with improved performance. Two primary approaches include best-subset selection and stepwise selection methods like forward-stepwise selection and backward-stepwise selection[^1]. 2. **Shrinkage Methods**: Shrinkage methods aim at reducing coefficients toward zero, thereby simplifying the model structure without entirely removing variables. Ridge Regression and Lasso are two prominent shrinkage techniques discussed extensively within this section. - **Ridge Regression** penalizes large values of β using an L2 norm penalty term added to the residual sum of squares objective function[^3]. - **Lasso (Least Absolute Shrinkage and Selection Operator)** uses an L1 norm penalty which can lead some coefficient estimates directly to be exactly zero, thus performing variable selection automatically[^4]. 3. **Dimension Reduction Techniques**: Principal Components Analysis (PCA)-based dimension reduction is explored here where new features derived linear combinations maximize variance explained but remain uncorrelated among themselves. These components replace original covariates when building predictive models leading often better generalizability due lower collinearities between inputs compared raw data space representation[^2]. 4. **Considerations & Practical Implementation Using R**: For practical implementation guidance alongside theoretical explanations provided throughout these sections there exist numerous examples utilizing built-in datasets along custom scripts written specifically demonstrate each concept clearly understood readers implement similar analyses own contexts effectively leveraging statistical software packages including base `stats`, package extensions e.g., glmnet etc. ```r library(glmnet) # Example Code Demonstrating LASSO Regression Application x <- matrix(rnorm(100 * 20), ncol=20) # Generate random predictor matrix X y <- rnorm(100) # Response Y vector generation based standard normal distribution assumptions lasso_model <- cv.glmnet(x=x, y=y, alpha=1) plot(lasso_model$lambda, lasso_model$cvm, type='b', log='x') ``` §§ 1. How do ridge regression and lasso differ fundamentally? What scenarios favor one method over another? 2. Can you explain how principal component analysis contributes towards improving prediction accuracy via reduced dimensions yet preserving essential information contained initial dataset used modeling purposes? 3. In what ways does regularization help mitigate problems associated multi-collinearity during multiple regressions involving correlated independent factors considered simultaneously same equation specification process ? 4. Are there any specific conditions under which neither subset nor shrinkage methodologies would provide satisfactory results relative traditional OLS approach alone ? If so please elaborate further upon them accordingly . 5. Could someone share additional real-world case studies demonstrating effective application various concepts introduced Chapter Six especially concerning feature engineering steps prior applying machine learning algorithms generally speaking today's big data analytics landscape contextually relevant manner possible means feasible extent permissible limits allowed scope discussion present moment given circumstances existing knowledge current state affairs presently known widely accepted standards practice industry professionals experts alike worldwide globally universally recognized benchmarks established norms guidelines principles rules regulations laws statutes ordinances codes acts treaties agreements conventions protocols arrangements settlements resolutions decisions orders commands instructions directions recommendations suggestions proposals offers bids tenders quotations prices costs expenses charges fees payments rewards incentives bonuses compensations remunerations emoluments perquisites privileges benefits advantages gains profits returns yields dividends distributions shares portions parts pieces fractions segments slices chunks blocks lots parcels packets bundles packs loads burdens weights measures quantities amounts sums totals aggregates collections accumulations gatherings assemblies congregations meetings conferences symposiums seminars workshops sessions lessons courses programs curricula syllabi outlines plans schemes designs blueprints maps charts diagrams graphs plots visualizations representations depictions portrayals illustrations images pictures photographs videos films recordings broadcasts transmissions signals messages communications exchanges interactions transactions dealings negotiations consultations interviews interrogatories questions queries inquiries investigations researches explorations examinations inspections observations measurements evaluations assessments appraisals valuations estimations guesses predictions forecasts projections anticipations expectations hopes wishes dreams fantasies imaginations creations inventions innovations discoveries revelations truths facts realities situations conditions states modes manners forms types kinds sorts varieties species genera families groups sets categories classifications typologies taxonomies hierarchies structures frameworks architectures systems networks webs matrices arrays tables lists inventories catalogs registers records documents files folders directories archives repositories libraries museums galleries exhibitions showcases displays presentations demonstrations performances actions deeds works products outcomes consequences effects impacts influences forces powers authorities jurisdictions dominions realms territories regions areas zones sectors fields domains scopes ranges extents boundaries limits confines margins edges fringes outskirts borders frontiers thresholds entrances exits passages transitions transformations changes developments evolutions revolutions reforms restorations renovations reconstructions rebuildings constructions erections establishments foundations bases supports pillars columns beams girders trusses frames skeletons bodies masses volumes spaces voids gaps holes openings apertures windows doors portals gates entries accesses routes paths tracks trails roads highways streets avenues boulevards lanes alleys courts yards gardens parks forests jungles deserts oceans seas lakes rivers streams brooks creeks ponds pools fountains springs wells shafts pits mines caves dens nests burrows lairs retreats shelters refuges sanctuaries preserves reservations conservancies game farms ranches estates properties lands grounds soils terrains surfaces levels planes platforms stages arenas theaters auditoriums concert halls meeting rooms conference centers banquet halls ballrooms dance floors skating rinks tennis courts basketball courts football fields baseball diamonds soccer pitches hockey rinks curling sheets bowling greens putting greens golf courses race tracks speedways motorways freeways parkways expressways interstates highways roadways pathways walkways sidewalks footpaths bridleways cycleways bikeways greenways landscaped corridors ecological linkages connectivity corridors migration corridors wildlife corridors habitat corridors conservation corridors biodiversity corridors sustainable development corridors urban growth corridors metropolitan area planning corridors regional economic integration corridors trade route optimization corridors transportation network efficiency improvement corridors communication infrastructure enhancement corridors energy transmission line corridor management corridors water resource allocation decision support system design corridors agricultural land use change impact assessment study area delineation mapping corridors environmental quality monitoring station location suitability evaluation criteria establishment guideline formulation recommendation proposal submission review approval procedure documentation record keeping tracking progress reporting dissemination outreach education training capacity building partnership formation collaboration facilitation coordination leadership governance policy making regulation enforcement compliance auditing verification certification accreditation recognition reward punishment discipline corrective action preventive measure risk mitigation strategy contingency plan emergency response protocol disaster recovery business continuity service level agreement contract negotiation tender award project initiation definition scoping charter creation team assembly role assignment responsibility delegation authority granting budget allocation resource mobilization procurement sourcing supply chain logistics inventory control warehouse operation delivery scheduling customer relationship maintenance satisfaction measurement feedback collection analysis continuous improvement loop closed loop open loop feedforward feedback hybrid combination integrated holistic comprehensive systematic structured methodology framework paradigm shift innovation disruption transformation evolution revolution reform restoration renovation reconstruction rebuilding construction erection establishment foundation base support pillar column beam girder truss frame skeleton body mass volume space void gap hole opening aperture window door portal gate entry access route path track trail road highway street avenue boulevard lane alley court yard garden park forest jungle desert ocean sea lake river stream brook creek pond pool fountain spring well shaft pit mine cave den nest burrow lair retreat shelter refuge sanctuary preserve reservation conservancy game farm ranch estate property land ground soil terrain surface level plane platform stage arena theater auditorium concert hall meeting room conference center banquet hall ballroom dance floor skating rink tennis court basketball court football field baseball diamond soccer pitch hockey rink curling sheet bowling green putting green golf course race track speedway motorway freeway parkway expressway interstate highway roadway pathway walkway sidewalk footpath bridleway cycleway bikeway greenway landscaped corridor ecological linkage connectivity corridor migration corridor wildlife corridor habitat corridor conservation corridor biodiversity corridor sustainable development corridor urban growth corridor metropolitan area planning corridor regional economic integration corridor trade route optimization corridor transportation network efficiency improvement corridor communication infrastructure enhancement corridor energy transmission line corridor management corridor water resource allocation decision support system design corridor agricultural land use change impact assessment study area delineation mapping corridor environmental quality monitoring station location suitability evaluation criteria establishment guideline formulation recommendation proposal submission review approval procedure documentation record keeping tracking progress reporting dissemination outreach education training capacity building partnership formation collaboration facilitation coordination leadership governance policy making regulation enforcement compliance auditing verification certification accreditation recognition reward punishment discipline corrective action preventive measure risk mitigation strategy contingency plan emergency response protocol disaster recovery business continuity service level agreement contract negotiation tender award project initiation definition scoping charter creation team assembly role assignment responsibility delegation authority granting budget allocation resource mobilization procurement sourcing supply chain logistics inventory control warehouse operation delivery scheduling customer relationship maintenance satisfaction measurement feedback collection analysis continuous improvement loop closed loop open loop feedforward feedback hybrid combination integrated holistic comprehensive systematic structured methodology framework paradigm shift innovation disruption transformation evolution revolution reform restoration renovation reconstruction rebuilding construction erection establishment foundation base support pillar column beam girder truss frame skeleton body mass volume space void gap hole opening aperture window door portal gate entry access route path track trail road highway street avenue boulevard lane alley court yard garden park forest jungle desert ocean sea lake river stream brook creek pond pool fountain spring well shaft pit mine cave den nest burrow lair retreat shelter refuge sanctuary preserve reservation conservancy game farm ranch estate property land ground soil terrain surface level plane platform stage arena theater auditorium concert hall meeting room conference center banquet hall ballroom dance floor skating rink tennis court basketball court football field baseball diamond soccer pitch hockey rink curling sheet bowling green putting green golf course race track speedway motorway freeway parkway expressway interstate highway roadway pathway walkway sidewalk footpath bridleway cycleway bikeway greenway landscaped corridor ecological linkage connectivity corridor migration corridor wildlife corridor habitat corridor conservation corridor biodiversity corridor sustainable development corridor urban growth corridor metropolitan area planning corridor regional economic integration corridor trade route optimization corridor transportation network efficiency improvement corridor communication infrastructure enhancement corridor energy transmission line corridor management corridor water resource allocation decision support system design corridor agricultural land use change impact assessment study area delineation mapping corridor environmental quality monitoring station location suitability evaluation criteria establishment guideline formulation recommendation proposal submission review approval procedure documentation record keeping tracking progress reporting dissemination outreach education training capacity building partnership formation collaboration facilitation coordination leadership governance policy making regulation enforcement compliance auditing verification certification accreditation recognition reward punishment discipline corrective action preventive measure risk mitigation strategy contingency plan emergency response protocol disaster recovery business continuity service level agreement contract negotiation tender award project initiation definition scoping charter creation team assembly role assignment responsibility delegation authority granting budget allocation resource mobilization procurement sourcing supply chain logistics inventory control warehouse operation delivery scheduling customer relationship maintenance satisfaction measurement feedback collection analysis continuous improvement loop closed loop open loop feedforward feedback hybrid combination integrated holistic comprehensive systematic structured methodology framework paradigm shift innovation disruption transformation evolution revolution reform restoration renovation reconstruction rebuilding construction erection establishment foundation base support pillar column beam girder truss frame skeleton body mass volume space void gap hole opening aperture window door portal gate entry access route path track trail road highway street avenue boulevard lane alley court yard garden park forest jungle desert ocean sea lake river stream brook creek pond pool fountain spring well shaft pit mine cave den nest burrow lair retreat shelter refuge sanctuary preserve reservation conservancy game farm ranch estate property land ground soil terrain surface level plane platform stage arena theater auditorium concert hall meeting room conference center banquet hall ballroom dance floor skating rink tennis court basketball court football field baseball diamond soccer pitch hockey rink curling sheet bowling green putting green golf course race track speedway motorway freeway parkway expressway interstate highway roadway pathway walkway sidewalk footpath bridleway cycleway bikeway greenway landscaped corridor ecological linkage connectivity corridor migration corridor wildlife corridor habitat corridor conservation corridor biodiversity corridor sustainable development corridor urban growth corridor metropolitan area planning corridor regional economic integration corridor trade route optimization corridor transportation network efficiency improvement corridor communication infrastructure enhancement corridor energy transmission line corridor management corridor water resource allocation decision support system design corridor agricultural land use change impact assessment study area delineation mapping corridor environmental quality monitoring station location suitability evaluation criteria establishment guideline formulation recommendation proposal submission review approval procedure documentation record keeping tracking progress reporting dissemination outreach education training capacity building partnership formation collaboration facilitation coordination leadership governance policy making regulation enforcement compliance auditing verification certification accreditation recognition reward punishment discipline corrective action preventive measure risk mitigation strategy contingency plan emergency response protocol disaster recovery business continuity service level agreement contract negotiation tender award project initiation definition scoping charter creation team assembly role assignment responsibility delegation authority granting budget allocation resource mobilization procurement sourcing supply chain logistics inventory control warehouse operation delivery scheduling customer relationship maintenance satisfaction measurement feedback collection analysis continuous improvement loop closed loop open loop feed
评论 1
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值