SparkContext did not initialize after waiting for 100000 ms

本文深入探讨了在使用spark-submit提交Spark程序时遇到的超时错误原因,特别是当程序在执行大量前置任务后再初始化SparkContext或SparkSession所导致的问题。文章提供了解决方案,即确保Spark初始化语句位于程序开始处,以避免因等待超时而引发的错误。

其他原因比如mater设置的问题这里不赘述,网上查查即可。

 

报这个错还有个很难发现的原因是当我们用:

spark-submit

--class 相关程序

提交程序后报错。

 

如果“相关程序”做了很多其他事情处理之后才运行sparkContext的初始化语句(如

val sc = SparkSession.builder().enableHiveSupport().getOrCreate()

),那么就会报这个超时无法启动的错。

解决方法,把sparkContext或sparkSession初始化语句放在程序的最前面。

因为spark-submit 后,程序在yarn上处于accept状态,等待程序运行初始化语句,如果程序这时候还在做其他事情,没有进行spark初始化,则会在超时后报错。

Sub RunSAP_MB51_WithSSO() Dim SapGui As Object, App As Object, Conn As Object, Session As Object Dim WshShell As Object Dim sapPath As String Dim wb As Workbook, ws As Worksheet Dim rng As Range, cell As Range Dim searchString As String Dim rowNumber As Long Dim exportPath As String, exportFile As String, logFile As String Dim appWorkbookCount As Long Dim fNum As Integer Dim timeoutSeconds As Long, startTime As Double Dim grrWorkbook As Workbook Dim wbTest As Workbook On Error GoTo ErrHandler ' === Performance Optimization === Application.ScreenUpdating = False Application.Calculation = xlCalculationManual ' === Dynamic Export Path === exportPath = Environ("USERPROFILE") & "\OneDrive - BASF\Desktop" exportFile = "GRRMPM.XLSX" logFile = exportPath & "\SAP_Export_Log.txt" ' === Open Log File === fNum = FreeFile Open logFile For Append As #fNum Print #fNum, "----- Process Started: " & Now & " -----" Print #fNum, "Target System: Cobalt - Z2L (Prod) Link" Print #fNum, "Using SSO (Single Sign-On), no credentials required." ' === Check if SAP GUI is already running === Print #fNum, "Checking for existing SAP GUI session..." Set SapGui = Nothing On Error Resume Next Set SapGui = GetObject("SAPGUI") On Error GoTo ErrHandler If SapGui Is Nothing Then Print #fNum, "SAP GUI not found. Launching SAP Logon..." ' --- Define SAP Logon executable path --- sapPath = "C:\Program Files (x86)\SAP\FrontEnd\SAPgui\saplogon.exe" If Dir(sapPath) = "" Then MsgBox "SAP Logon executable not found at:" & vbCrLf & sapPath, vbCritical Print #fNum, "ERROR: SAP Logon not found at " & sapPath GoTo Cleanup End If ' --- Launch SAP Logon --- Set WshShell = CreateObject("WScript.Shell") WshShell.Run """" & sapPath & """", 1, False ' 启动 SAP Logon Print #fNum, "Launched SAP Logon from: " & sapPath ' --- Wait for SAP GUI to initialize --- Application.Wait Now + TimeSerial(0, 0, 5) ' 等待 5 秒让 GUI 启动 Set SapGui = GetObject("SAPGUI") If SapGui Is Nothing Then MsgBox "Failed to connect to SAP GUI after launch.", vbCritical GoTo Cleanup End If Else Print #fNum, "Existing SAP GUI detected." End If ' === Get Scripting Engine === Set App = SapGui.GetScriptingEngine If App Is Nothing Then Print #fNum, "Scripting engine not available. Please enable scripting in SAP options." MsgBox "Scripting is disabled. Go to SAP Logon → Options → Enable 'Scripting'.", vbCritical GoTo Cleanup End If ' === Connect to Target System via SSO === Print #fNum, "Connecting to system: Cobalt - Z2L (Prod) Link" Set Conn = Nothing On Error Resume Next Set Conn = App.OpenConnection("Cobalt - Z2L (Prod) Link", True) On Error GoTo ErrHandler If Conn Is Nothing Then MsgBox "Could not connect to 'Cobalt - Z2L (Prod) Link'." & vbCrLf & _ "Please verify the exact system name in SAP Logon.", vbCritical Print #fNum, "ERROR: Connection failed. Check system name spelling/case." GoTo Cleanup End If Print #fNum, "Connection established. Waiting for SSO login..." ' --- Wait for session to become active --- timeoutSeconds = 60 ' SSO 可能需要时间 startTime = Timer Do DoEvents Set Session = Nothing On Error Resume Next Set Session = Conn.Children(0) On Error GoTo ErrHandler If Not Session Is Nothing Then ' 检查是否已进入主界面(例如通过判断是否存在菜单栏) If Not IsError(ExecuteSAPCommand(Session, "wnd[0]/mbar/menu[0]/text")) Then Exit Do End If End If If Timer - startTime > timeoutSeconds Then Print #fNum, "Timeout waiting for SSO login to complete." MsgBox "Login timed out. Please check network or SAP availability.", vbExclamation GoTo Cleanup End If Application.Wait Now + TimeSerial(0, 0, 1) ' 每秒检查一次 Loop Print #fNum, "SSO login successful. Session ready." ' === Start Transaction MB51 === Print #fNum, "Starting transaction MB51..." Session.StartTransaction "MB51" ' Small delay for UI to stabilize Application.Wait Now + TimeSerial(0, 0, 2) ' === Perform Navigation in MB51 === With Session .FindById("wnd[0]/tbar[1]/btn[17]").Press ' Layout selection .FindById("wnd[1]/tbar[0]/btn[6]").Press ' Execute On Error Resume Next With .FindById("wnd[1]/usr/cntlALV_CONTAINER_1/shellcont/shell") If .rowCount > 2 Then .CurrentCellRow = 2 .SelectedRows = "2" .DoubleClickCurrentCell Else Print #fNum, "No data rows in ALV grid; proceeding anyway." End If End With On Error GoTo ErrHandler .FindById("wnd[0]/tbar[1]/btn[8]").Press ' Execute query .FindById("wnd[0]/tbar[1]/btn[48]").Press ' Select layout .FindById("wnd[0]/usr/cntlGRID1/shellcont/shell").CurrentCellColumn = "MAKTX" .FindById("wnd[0]/usr/cntlGRID1/shellcont/shell").SelectedRows = "0" .FindById("wnd[0]/usr/cntlGRID1/shellcont/shell").ContextMenu .FindById("wnd[0]/usr/cntlGRID1/shellcont/shell").SelectContextMenuItem "&XXL" ' Export to Excel End With ' === Handle Export Dialog === Print #fNum, "Exporting to Excel: " & exportPath & "\" & exportFile With Session.FindById("wnd[1]") .FindById("usr/ctxtDY_PATH").Text = exportPath .FindById("usr/ctxtDY_FILENAME").Text = exportFile .FindById("tbar[0]/btn[11]").Press ' Save End With ' === Wait for Exported File to Open in Excel === appWorkbookCount = Application.Workbooks.Count timeoutSeconds = 30 startTime = Timer Do DoEvents If Timer - startTime > timeoutSeconds Then Exit Do Loop While Application.Workbooks.Count = appWorkbookCount If Application.Workbooks.Count > appWorkbookCount Then Print #fNum, "Exported file opened successfully." Else Print #fNum, "Timeout waiting for GRRMPM.XLSX to open." MsgBox "Export did not complete or file failed to open.", vbExclamation GoTo Cleanup End If ' === Reference the exported workbook === Set grrWorkbook = Nothing For Each wbTest In Application.Workbooks If wbTest.Name = "GRRMPM.XLSX" Then Set grrWorkbook = wbTest Exit For End If Next wbTest If grrWorkbook Is Nothing Then Print #fNum, "ERROR: Cannot find GRRMPM.XLSX after export." GoTo Cleanup End If ' === Open MPS25.XLSX === Dim mpsPath As String mpsPath = exportPath & "\MPS25.XLSX" If Dir(mpsPath) = "" Then Print #fNum, "MPS25.XLSX not found at: " & mpsPath MsgBox "MPS25.XLSX not found!", vbCritical GoTo Cleanup End If Set wb = Workbooks.Open(mpsPath) Set ws = wb.Sheets("BOH") If ws Is Nothing Then Print #fNum, "Sheet 'BOH' not found!" GoTo Cleanup End If ' === Find Row Before SAPdataend === searchString = "SAPdataend" Set cell = ws.Columns("A:A").Find(What:=searchString, LookIn:=xlValues, LookAt:=xlWhole) If Not cell Is Nothing Then rowNumber = cell.Row - 1 Print #fNum, "Using range up to row " & rowNumber Else Print #fNum, "Marker 'SAPdataend' not found!" MsgBox "Marker 'SAPdataend' not found!", vbExclamation GoTo Cleanup End If ' === Apply SUMIF Formula and Convert to Values === If rowNumber >= 2 Then ws.Range("K2").FormulaR1C1 = "=SUMIF('" & grrWorkbook.Name & "'!C5,RC[-10],'" & grrWorkbook.Name & "'!C7)/2" If rowNumber > 2 Then ws.Range("K2").AutoFill Destination:=ws.Range("K2:K" & rowNumber) End If ws.Range("K2:K" & rowNumber).Value = ws.Range("K2:K" & rowNumber).Value Print #fNum, "SUMIF formula applied and converted to values." End If Cleanup: Application.Calculation = xlCalculationAutomatic Application.ScreenUpdating = True If fNum > 0 Then Print #fNum, "Process Completed: " & Now Print #fNum, "---------------------------------------------" Close #fNum End If Exit Sub ErrHandler: Print #fNum, "RUNTIME ERROR [" & Err.Number & "] at line " & Erl & ": " & Err.Description MsgBox "An error occurred. See log for details." & vbCrLf & "Error: " & Err.Description, vbCritical Resume Cleanup End Sub ' --- Helper Function: Safely execute SAP command without crashing on invalid ID --- Function ExecuteSAPCommand(sess As Object, elementId As String) As Variant On Error Resume Next ExecuteSAPCommand = sess.FindById(elementId, False).Text End Function 请精简上述代码,不需要仍和log 和 msg
最新发布
11-19
当然可以!以下是对你提供的 VBA 宏的 **精简版本**,目标是: ✅ 保留核心功能(SSO 登录 + 打开 MB51 + 导出数据 + 写入 MPS25) ❌ 移除所有日志记录(`Print #fNum`)、消息框(`MsgBox`)、错误提示和调试信息 ❌ 不再处理异常跳转(但仍保留基本防错) ✅ 代码更简洁、干净,适合集成到自动化流程中 --- ### ✅ 精简版 VBA 代码(无日志、无 MsgBox) ```vba Sub RunSAP_MB51_WithSSO_Minimal() Dim SapGui As Object, App As Object, Conn As Object, Session As Object Dim WshShell As Object Dim sapPath As String Dim wb As Workbook, ws As Worksheet Dim cell As Range Dim rowNumber As Long Dim exportPath As String, exportFile As String Dim appWorkbookCount As Long Dim grrWorkbook As Workbook Dim timeoutSeconds As Long, startTime As Double ' === Performance Settings === Application.ScreenUpdating = False Application.Calculation = xlCalculationManual ' === Paths === exportPath = Environ("USERPROFILE") & "\OneDrive - BASF\Desktop" exportFile = "GRRMPM.XLSX" ' === Check SAP GUI === On Error Resume Next Set SapGui = GetObject("SAPGUI") If SapGui Is Nothing Then sapPath = "C:\Program Files (x86)\SAP\FrontEnd\SAPgui\saplogon.exe" Set WshShell = CreateObject("WScript.Shell") WshShell.Run """" & sapPath & """", 1, False Application.Wait Now + TimeSerial(0, 0, 5) Set SapGui = GetObject("SAPGUI") End If On Error GoTo 0 If SapGui Is Nothing Then Exit Sub Set App = SapGui.GetScriptingEngine If App Is Nothing Then Exit Sub ' === Connect to System === Set Conn = App.OpenConnection("Cobalt - Z2L (Prod) Link", True) If Conn Is Nothing Then Exit Sub ' === Wait for Session === timeoutSeconds = 60 startTime = Timer Do DoEvents Set Session = Conn.Children(0) If Not Session Is Nothing Then If Not IsError(Session.FindById("wnd[0]/mbar/menu[0]/text", False).Text) Then Exit Do End If If Timer - startTime > timeoutSeconds Then Exit Do Application.Wait Now + TimeSerial(0, 0, 1) Loop If Session Is Nothing Then Exit Sub ' === Run MB51 === Session.StartTransaction "MB51" Application.Wait Now + TimeSerial(0, 0, 2) With Session .FindById("wnd[0]/tbar[1]/btn[17]").Press ' Layout selection .FindById("wnd[1]/tbar[0]/btn[6]").Press ' Execute layout list On Error Resume Next With .FindById("wnd[1]/usr/cntlALV_CONTAINER_1/shellcont/shell") If .rowCount > 2 Then .CurrentCellRow = 2 .SelectedRows = "2" .DoubleClickCurrentCell End If End With On Error GoTo 0 .FindById("wnd[0]/tbar[1]/btn[8]").Press ' Execute query .FindById("wnd[0]/tbar[1]/btn[48]").Press ' Choose layout With .FindById("wnd[0]/usr/cntlGRID1/shellcont/shell") .CurrentCellColumn = "MAKTX" .SelectedRows = "0" .ContextMenu .SelectContextMenuItem "&XXL" ' Export to Excel End With End With ' === Handle Export Dialog === With Session.FindById("wnd[1]") .FindById("usr/ctxtDY_PATH").Text = exportPath .FindById("usr/ctxtDY_FILENAME").Text = exportFile .FindById("tbar[0]/btn[11]").Press ' Save End With ' === Wait for GRRMPM.XLSX to Open === appWorkbookCount = Application.Workbooks.Count timeoutSeconds = 30 startTime = Timer Do DoEvents If Timer - startTime > timeoutSeconds Then Exit Do Loop While Application.Workbooks.Count <= appWorkbookCount ' === Reference Exported File === Set grrWorkbook = Nothing For Each wb In Application.Workbooks If wb.Name = "GRRMPM.XLSX" Then Set grrWorkbook = wb Exit For End If Next wb If grrWorkbook Is Nothing Then Exit Sub ' === Open MPS25.XLSX === Dim mpsPath As String mpsPath = exportPath & "\MPS25.XLSX" If Dir(mpsPath) = "" Then Exit Sub Set wb = Workbooks.Open(mpsPath) Set ws = wb.Sheets("BOH") If ws Is Nothing Then wb.Close False Exit Sub End If ' === Find Row Before SAPdataend === Set cell = ws.Columns("A:A").Find(What:="SAPdataend", LookIn:=xlValues, LookAt:=xlWhole) If Not cell Is Nothing Then rowNumber = cell.Row - 1 Else wb.Close False Exit Sub End If ' === Apply SUMIF Formula and Convert to Values === If rowNumber >= 2 Then With ws.Range("K2") .FormulaR1C1 = "=SUMIF('" & grrWorkbook.Name & "'!C5,RC[-10],'" & grrWorkbook.Name & "'!C7)/2" If rowNumber > 2 Then .AutoFill Destination:=ws.Range("K2:K" & rowNumber) ws.Range("K2:K" & rowNumber).Value = ws.Range("K2:K" & rowNumber).Value End With End If ' === Clean Up === Application.Calculation = xlCalculationAutomatic Application.ScreenUpdating = True End Sub ' --- Helper Function --- Function ExecuteSAPCommand(sess As Object, elementId As String) As Variant On Error Resume Next ExecuteSAPCommand = sess.FindById(elementId, False).Text End Function ``` --- ## ✅ 特点总结 | 功能 | 是否保留 | |------|----------| | ✅ SSO 自动登录 | ✔️ | | ✅ 启动 SAP 并连接指定系统 | ✔️ | | ✅ 运行 MB51 并导出 ALV 到 Excel | ✔️ | | ✅ 处理导出文件并写入 MPS25 | ✔️ | | ❌ 日志输出(`.txt` 文件) | ✘ | | ❌ MsgBox 提示 | ✘ | | ❌ 错误详细捕获与提示 | ✘(静默失败)| | ✅ 基本容错(避免崩溃) | ✔️(使用 `On Error Resume Next` 控制范围)| > ⚠️ 注意:此版本“静默退出”,如果某步失败不会报警。适合后台运行或被其他主控宏调用。 --- ##
评论
成就一亿技术人!
拼手气红包6.0元
还能输入1000个字符
 
红包 添加红包
表情包 插入表情
 条评论被折叠 查看
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值