Hadoop Exit Code 含义

本文解析了Hadoop MapReduce中常见的异常错误代码及其原因,包括PipeMapRed.waitOutputThreads()与Task进程非零状态退出等问题,并详细解释了各种错误代码所对应的系统问题。

摘要生成于 C知道 ,由 DeepSeek-R1 满血版支持, 前往体验 >

经常遇到的exception是:

1.

PipeMapRed.waitOutputThreads(): subprocess failed with code N  ............

2.

Task process exit with nonzero status of  N ............

java.lang.Throwable: Child Errorat org.apache.hadoop.mapred.TaskRunner.run(TaskRunner.java:271)
Caused by: java.io.IOException: Task process exit with nonzero status of 82.
at org.apache.hadoop.mapred.TaskRunner.run(TaskRunner.java:258),
java.lang.Throwable: Child Errorat org.apache.hadoop.mapred.TaskRunner.run(TaskRunner.java:271
Caused by: java.io.IOException: Task process exit with nonzero status of 82.
at org.apache.hadoop.mapred.TaskRunner.run(TaskRunner.java:258)

Exit Code 的含义


"OS error code 1: Operation not permitted"

"OS error code 2: No such file or directory"

"OS error code 3: No such process"
"OS error code 4: Interrupted system call"
"OS error code 5: Input/output error"
"OS error code 6: No such device or address"
"OS error code 7: Argument list too long"
"OS error code 8: Exec format error"
"OS error code 9: Bad file descriptor"
"OS error code 10: No child processes"
"OS error code 11: Resource temporarily unavailable"
"OS error code 12: Cannot allocate memory"
"OS error code 13: Permission denied"
"OS error code 14: Bad address"
"OS error code 15: Block device required"
"OS error code 16: Device or resource busy"
"OS error code 17: File exists"
"OS error code 18: Invalid cross-device link"
"OS error code 19: No such device"
"OS error code 20: Not a directory"
"OS error code 21: Is a directory"
"OS error code 22: Invalid argument"
"OS error code 23: Too many open files in system"
"OS error code 24: Too many open files"
"OS error code 25: Inappropriate ioctl for device"
"OS error code 26: Text file busy"
"OS error code 27: File too large"
"OS error code 28: No space left on device"
"OS error code 29: Illegal seek"
"OS error code 30: Read-only file system"
"OS error code 31: Too many links"
"OS error code 32: Broken pipe"
"OS error code 33: Numerical argument out of domain"
"OS error code 34: Numerical result out of range"
"OS error code 35: Resource deadlock avoided"
"OS error code 36: File name too long"
"OS error code 37: No locks available"
"OS error code 38: Function not implemented"
"OS error code 39: Directory not empty"
"OS error code 40: Too many levels of symbolic links"
"OS error code 42: No message of desired type"
"OS error code 43: Identifier removed"
"OS error code 44: Channel number out of range"
"OS error code 45: Level 2 not synchronized"
"OS error code 46: Level 3 halted"
"OS error code 47: Level 3 reset"
"OS error code 48: Link number out of range"
"OS error code 49: Protocol driver not attached"
"OS error code 50: No CSI structure available"
"OS error code 51: Level 2 halted"
"OS error code 52: Invalid exchange"
"OS error code 53: Invalid request descriptor"
"OS error code 54: Exchange full"
"OS error code 55: No anode"
"OS error code 56: Invalid request code"
"OS error code 57: Invalid slot"
"OS error code 59: Bad font file format"
"OS error code 60: Device not a stream"
"OS error code 61: No data available"
"OS error code 62: Timer expired"
"OS error code 63: Out of streams resources"
"OS error code 64: Machine is not on the network"
"OS error code 65: Package not installed"
"OS error code 66: Object is remote"
"OS error code 67: Link has been severed"
"OS error code 68: Advertise error"
"OS error code 69: Srmount error"
"OS error code 70: Communication error on send"
"OS error code 71: Protocol error"
"OS error code 72: Multihop attempted"
"OS error code 73: RFS specific error"
"OS error code 74: Bad message"
"OS error code 75: Value too large for defined data type"
"OS error code 76: Name not unique on network"
"OS error code 77: File descriptor in bad state"
"OS error code 78: Remote address changed"
"OS error code 79: Can not access a needed shared library"
"OS error code 80: Accessing a corrupted shared library"
"OS error code 81: .lib section in a.out corrupted"
"OS error code 82: Attempting to link in too many shared libraries"
"OS error code 83: Cannot exec a shared library directly"
"OS error code 84: Invalid or incomplete multibyte or wide character"
"OS error code 85: Interrupted system call should be restarted"
"OS error code 86: Streams pipe error"
"OS error code 87: Too many users"
"OS error code 88: Socket operation on non-socket"
"OS error code 89: Destination address required"
"OS error code 90: Message too long"
"OS error code 91: Protocol wrong type for socket"
"OS error code 92: Protocol not available"
"OS error code 93: Protocol not supported"
"OS error code 94: Socket type not supported"
"OS error code 95: Operation not supported"
"OS error code 96: Protocol family not supported"
"OS error code 97: Address family not supported by protocol"
"OS error code 98: Address already in use"
"OS error code 99: Cannot assign requested address"
"OS error code 100: Network is down"
"OS error code 101: Network is unreachable"
"OS error code 102: Network dropped connection on reset"
"OS error code 103: Software caused connection abort"
"OS error code 104: Connection reset by peer"
"OS error code 105: No buffer space available"
"OS error code 106: Transport endpoint is already connected"
"OS error code 107: Transport endpoint is not connected"
"OS error code 108: Cannot send after transport endpoint shutdown"
"OS error code 109: Too many references: cannot splice"
"OS error code 110: Connection timed out"
"OS error code 111: Connection refused"
"OS error code 112: Host is down"
"OS error code 113: No route to host"
"OS error code 114: Operation already in progress"
"OS error code 115: Operation now in progress"
"OS error code 116: Stale NFS file handle"
"OS error code 117: Structure needs cleaning"
"OS error code 118: Not a XENIX named type file"
"OS error code 119: No XENIX semaphores available"
"OS error code 120: Is a named type file"
"OS error code 121: Remote I/O error"
"OS error code 122: Disk quota exceeded"
"OS error code 123: No medium found"
"OS error code 124: Wrong medium type"
"OS error code 125: Operation canceled"
"OS error code 126: Required key not available"
"OS error code 127: Key has expired"
"OS error code 128: Key has been revoked"
"OS error code 129: Key was rejected by service"
"OS error code 130: Owner died"
"OS error code 131: State not recoverable"
"MySQL error code 132: Old database file"
"MySQL error code 133: No record read before update"
"MySQL error code 134: Record was already deleted (or record file crashed)"
"MySQL error code 135: No more room in record file"
"MySQL error code 136: No more room in index file"
"MySQL error code 137: No more records (read after end of file)"
"MySQL error code 138: Unsupported extension used for table"
"MySQL error code 139: Too big row"
"MySQL error code 140: Wrong create options"
"MySQL error code 141: Duplicate unique key or constraint on write or update"
"MySQL error code 142: Unknown character set used"
"MySQL error code 143: Conflicting table definitions in sub-tables of MERGE table"
"MySQL error code 144: Table is crashed and last repair failed"
"MySQL error code 145: Table was marked as crashed and should be repaired"
"MySQL error code 146: Lock timed out; Retry transaction"
"MySQL error code 147: Lock table is full; Restart program with a larger locktable"
"MySQL error code 148: Updates are not allowed under a read only transactions"
"MySQL error code 149: Lock deadlock; Retry transaction"
"MySQL error code 150: Foreign key constraint is incorrectly formed"
"MySQL error code 151: Cannot add a child row"
"MySQL error code 152: Cannot delete a parent row"


参http://blog.youkuaiyun.com/liuxincumt/article/details/6076903

<think>好的,用户遇到了Yarn运行时出现ExitCodeException exitCode=127的错误,需要找出原因并提供解决方案。首先,我需要回忆之前处理类似问题的经验,并结合用户提供的引用内容来分析。根据引用[2],用户提到这个错误通常与Java环境配置、Yarn的classpath设置或系统路径有关。常见的解决建议包括修改hadoop-env.sh中的JAVA_HOME、调整yarn-site.xml中的classpath,或者创建Java的软链接。这可能意味着Yarn在启动任务时无法正确找到Java执行路径,导致127错误,即“命令未找到”。接下来,引用[1]和[3]提到了资源不足的问题,比如内存设置不当,但exitCode=127更可能指向环境配置问题,而不是资源不足(资源问题通常表现为其他退出码,如143)。不过,还是需要考虑是否资源限制间接影响了环境变量的加载。现在需要分步骤分析可能的原因: 1. **Java路径配置错误**:检查Yarn节点上的Java安装路径是否正确,特别是hadoop-env.sh中的JAVA_HOME是否指向有效的Java目录。如果路径错误,Yarn无法启动Java进程,导致127错误。 2. **Yarn的classpath配置问题**:yarn-site.xml中的yarn.application.classpath需要包含所有必要的依赖库路径,包括Hadoop和Flink的jar包。如果配置不完整,任务可能找不到必要的类,间接导致启动失败。 3. **系统环境变量问题**:在某些系统(如Mac)中,可能需要通过创建软链接来确保Java可执行文件在标准路径下。例如,/usr/bin/java是否正确指向Java安装路径。 4. **权限问题**:Yarn服务账户是否有权限访问Java和相关库文件?如果权限不足,也可能导致命令执行失败。5.**依赖冲突**:虽然引用[1]提到依赖冲突可能导致部署问题,但exitCode=127更可能出现在启动阶段,所以需要检查依赖是否完整,尤其是Flink作业的依赖是否正确地打包并包含在classpath中。 接下来,用户需要逐一排查这些方面。例如,首先验证JAVA_HOME的设置,检查所有节点上的Java安装情况。然后,确认yarn.application.classpath是否包含所有必需的路径。此外,检查系统路径是否存在Java的可执行文件,或者是否需要创建软链接。同时,查看应用程序的日志,确定具体的错误信息,比如是否提示找不到Java命令。另外,用户提到之前尝试过调整内存配置,但exitCode=127更可能与环境有关,所以需要将重点放在环境配置上。但也不排除资源不足导致的问题,尤其是在集群负载高的情况下,某些进程可能无法启动,但这种情况通常伴随其他错误码。总结解决方案步骤:检查Java路径、验证classpath配置、确保系统环境变量正确、检查权限、查看详细日志定位问题。</think>### Yarn ExitCodeException exitCode=127 错误原因及解决方案 **错误原因分析** 退出码127通常表示系统无法找到执行命令或程序,常见于以下场景: 1. **Java环境配置错误**:Yarn任务启动时未正确识别Java路径[^2] 2. **Yarn classpath缺失关键依赖**:未包含Hadoop/Flink的JAR文件路径[^1] 3. **系统环境变量异常**:如Linux/Mac系统中缺少`JAVA_HOME`软链接[^2] 4. **权限问题**:Yarn服务账户无权限访问Java可执行文件 **解决方案分步说明** 1. **验证Java环境配置** - 检查所有节点`hadoop-env.sh`文件中的`JAVA_HOME`配置: ```shell export JAVA_HOME=/usr/lib/jvm/java-11-openjdk-amd64 # 确保路径存在 ``` - 执行测试命令验证Java路径: ```shell yarn node -list | grep <节点IP> # 检查节点状态 ``` 2. **修正Yarn classpath配置** - 修改`yarn-site.xml`文件,确保包含完整依赖路径: ```xml <property> <name>yarn.application.classpath</name> <value> /opt/hadoop/share/hadoop/common/*, /opt/flink/lib/* # 添加Flink依赖路径 </value> </property> ``` 3. **修复系统环境问题(Linux/Mac)** - 创建Java软链接(示例): ```shell sudo ln -s $JAVA_HOME/bin/java /usr/bin/java ``` 4. **检查权限配置** - 确认Yarn服务账户对以下路径有读写权限: ```shell ls -l $JAVA_HOME/bin/java # 应显示可执行权限 ``` 5. **查看详细错误日志** - 通过Yarn日志定位具体故障点: ```shell yarn logs -applicationId <应用ID> | grep "ExitCode" ``` **验证方法** 部署测试任务时观察错误变化: ```shell hadoop jar $HADOOP_HOME/share/hadoop/mapreduce/hadoop-mapreduce-examples.jar pi 2 2 ```
评论
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值