master节点创建lzo索引失败：Retrying connect to server: 0.0.0.0/0.0.0.0:8032.

最新推荐文章于 2023-12-13 17:58:01 发布
原创 最新推荐文章于 2023-12-13 17:58:01 发布 · 484 阅读 · 0 ·
CC 4.0 BY-SA版权
版权声明：本文为博主原创文章，遵循 CC 4.0 BY-SA 版权协议，转载请附上原文出处链接和本声明。
文章标签：

#hadoop #yarn #lzo #索引 #java

博客指出问题原因是未开启yarn集群，还提及了star-yarn.sh。涉及Hadoop、Yarn等信息技术相关内容。
原因：没有开启yarn集群
star-yarn.sh
Tier_
关注
向YARN提交任务Retrying connect to server 0.0.0.08032异常
u012667450的博客
11-18 4619
Retrying connect to server 0.0.0.0:8032异常 问题描述 **case 1：**基于HDP集群二次开发的集群中，在Client执行spark 任务的时候，在客户端页面提示Connecting to ResourceManager at 0.0.0.0:8032异常。如下图所示： **case 2：**在Client解决上述问题的同时，回归到web页面，进行冒烟测试，在使用Azkaban进行任务调度的时候，该异常二次发生，咦，刚刚已经解决了，为何还会发生呢？此时，心情是崩
lzo创建索引命令
qq_55906442的博客
05-07 888
前提： 1.lzo建索引的前提是文件很大，如果文件很小是不会创建索引的； 2.hadoop正常运行。 命令： hadoop jar /training/hadoop-3.1.3/share/hadoop/common/hadoop-lzo-0.4.20.jar com.hadoop.compression.lzo.DistributedLzoIndexer /input/文件名 hadoop jar /training/hadoop-3.1.3/share/hadoop/common/...
flink yarn-session 启动失败retrying connect to server 0.0.0.0/0.0.0.0:8032
qq_44540985的博客
12-13 1448
flink yarn-session 启动失败retrying connect to server 0.0.0.0/0.0.0.0:8032
Retrying connect to server 0.0.0.08032异常.md
11-18
向yarn提交任务时Retrying connect to server 0.0.0.0:8032异常
INFO Client: Retrying connect to server: 0.0.0.0/0.0.0.0:8032. Already tried 1 time(s)...
m0_51066566的博客
11-02 551
INFO Client: Retrying connect to server: 0.0.0.0/0.0.0.0:8032. Already tried 1 time(s)...
hadoop2.7.7 Retrying connect to server: 0.0.0.0/0.0.0.0:8032. Already tried 0 time(s)
qq 117791303
09-03 2445
Retrying connect to server: 0.0.0.0/0.0.0.0:8032. Already tried 0 time(s)
error: lzo/lzoconf.h: 没有那个文件或目录
08-20
在编译或构建某些项目时，如果遇到 `error: lzo/lzoconf.h: No such file or directory` 的错误，通常意味着系统缺少 LZO 压缩库的开发文件。LZO 是一种广泛使用的数据压缩库，许多软件依赖其进行压缩和解压操作。 ...
<?xml version="1.0" encoding="UTF-8"?> <project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd"> <modelVersion>4.0.0</modelVersion> <groupId>com.sina.data.game</groupId> <artifactId>TelevisionProgramGame</artifactId> <version>1.0-SNAPSHOT</version> <packaging>jar</packaging> <name>TelevisionProgramGame</name> <url>http://maven.apache.org</url> <properties> <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding> </properties> <dependencies> <dependency> <groupId>junit</groupId> <artifactId>junit</artifactId> <version>3.8.1</version> <scope>test</scope> </dependency> <dependency> <groupId>com.google.code.gson</groupId> <artifactId>gson</artifactId> <version>2.8.9</version> </dependency> <dependency> <groupId>org.apache.htrace</groupId> <artifactId>htrace-core</artifactId> <version>3.1.0</version> <classifier>incubating</classifier> </dependency> <dependency> <groupId>org.apache.hbase</groupId> <artifactId>hbase-client</artifactId> <version>1.1.3</version> </dependency> <dependency> <groupId>org.apache.hbase</groupId> <artifactId>hbase-common</artifactId> <version>1.1.3</version> </dependency> <dependency> <groupId>org.apache.hbase</groupId> <artifactId>hbase-protocol</artifactId> <version>1.1.3</version> </dependency> <dependency> <groupId>org.apache.hbase</groupId> <artifactId>hbase-server</artifactId> <version>1.1.3</version> </dependency> <dependency> <groupId>org.apache.hbase</groupId> <artifactId>hbase-hadoop-compat</artifactId> <version>1.1.3</version> </dependency> <dependency> <groupId>commons-configuration</groupId> <artifactId>commons-configuration</artifactId> <version>1.8</version> </dependency> <dependency> <groupId>org.slf4j</groupId> <artifactId>slf4j-api</artifactId> <version>1.4.3</version> </dependency> <dependency> <groupId>commons-httpclient</groupId> <artifactId>commons-httpclient</artifactId> <version>3.1-rc1</version> </dependency> <dependency> <groupId>io.netty</groupId> <artifactId>netty-all</artifactId> <version>4.0.24.Final</version> </dependency> <dependency> <groupId>commons-codec</groupId> <artifactId>commons-codec</artifactId> <version>1.3</version> </dependency> <dependency> <groupId>org.apache.hadoop</groupId> <artifactId>hadoop-auth</artifactId> <version>2.2.0</version> </dependency> <dependency> <groupId>commons-cli</groupId> <artifactId>commons-cli</artifactId> <version>1.2</version> </dependency> <dependency> <groupId>net.sf.json-lib</groupId> <artifactId>json-lib</artifactId> <version>2.3</version> <classifier>jdk15</classifier> </dependency> <dependency> <groupId>net.sf.ezmorph</groupId> <artifactId>ezmorph</artifactId> <version>1.0.6</version> </dependency> <dependency> <groupId>commons-collections</groupId> <artifactId>commons-collections</artifactId> <version>3.2</version> </dependency> <dependency> <groupId>commons-beanutils</groupId> <artifactId>commons-beanutils</artifactId> <version>20030211.134440</version> </dependency> <dependency> <groupId>commons-logging</groupId> <artifactId>commons-logging</artifactId> <version>1.1.3</version> </dependency> <dependency> <groupId>commons-logging</groupId> <artifactId>commons-logging-api</artifactId> <version>1.1</version> </dependency> <dependency> <groupId>org.apache.hadoop</groupId> <artifactId>hadoop-common</artifactId> <version>2.4.0</version> </dependency> <dependency> <groupId>hadoop-lzo</groupId> <artifactId>hadoop-lzo-cdh4</artifactId> <version>0.4.15</version> <classifier>gplextras</classifier> </dependency> <dependency> <groupId>org.apache.hadoop</groupId> <artifactId>hadoop-client</artifactId> <version>2.4.0</version> </dependency> <dependency> <groupId>org.apache.hadoop</groupId> <artifactId>hadoop-mapreduce-client-common</artifactId> <version>2.4.0</version> </dependency> <dependency> <groupId>org.apache.hadoop</groupId> <artifactId>hadoop-mapreduce-client-core</artifactId> <version>2.4.0</version> </dependency> <dependency> <groupId>org.apache.hadoop</groupId> <artifactId>hadoop-mapreduce-client-hs</artifactId> <version>2.4.0</version> </dependency> <dependency> <groupId>org.apache.hadoop</groupId> <artifactId>hadoop-mapreduce-client-hs-plugins</artifactId> <version>2.4.0</version> </dependency> <dependency> <groupId>org.apache.hadoop</groupId> <artifactId>hadoop-mapreduce-client-jobclient</artifactId> <version>2.4.0</version> </dependency> <dependency> <groupId>org.apache.hadoop</groupId> <artifactId>hadoop-mapreduce-client-shuffle</artifactId> <version>2.4.0</version> </dependency> <dependency> <groupId>org.apache.hadoop</groupId> <artifactId>hadoop-mapreduce-examples</artifactId> <version>2.4.0</version> </dependency> <dependency> <groupId>org.apache.hadoop</groupId> <artifactId>hadoop-nfs</artifactId> <version>2.4.0</version> </dependency> <dependency> <groupId>com.sina.Armyknife</groupId> <artifactId>sina</artifactId> <version>0.5</version> <scope>provided</scope> </dependency> <dependency> <groupId>com.sina.hivexec</groupId> <artifactId>hive-exec</artifactId> <version>0.13.0</version> </dependency> <dependency> <groupId>redis.clients</groupId> <artifactId>jedis</artifactId> <version>2.7.3</version> </dependency> <dependency> <groupId>junit</groupId> <artifactId>junit</artifactId> <version>RELEASE</version> </dependency> </dependencies> <repositories> <repository> <id>thirdparty</id> <name>3rd party</name> <url> http://repo.hadoop.data.sina.com.cn:8081/nexus/content/repositories/thirdparty </url> <releases> <enabled>true</enabled> </releases> <snapshots> <enabled>false</enabled> </snapshots> </repository> <repository> <id>cdh.repo</id> <name>Cloudera Repositories</name> <url> http://repo.hadoop.data.sina.com.cn:8081/nexus/content/repositories/cdh.repo/ </url> </repository> <repository> <id>central</id> <name>Central</name> <url> http://repo.hadoop.data.sina.com.cn:8081/nexus/content/repositories/central </url> <releases> <enabled>true</enabled> </releases> <snapshots> <enabled>false</enabled> </snapshots> </repository> </repositories> <pluginRepositories> <pluginRepository> <releases> <updatePolicy>never</updatePolicy> </releases> <snapshots> <enabled>false</enabled> </snapshots> <id>public</id> <name>Public Repositories</name> <url>http://repo.hadoop.data.sina.com.cn:8081/nexus/content/groups/public/</url> </pluginRepository> <pluginRepository> <id>central</id> <name>Public Repositories</name> <url>http://repo.hadoop.data.sina.com.cn:8081/nexus/content/repositories/central</url> </pluginRepository> </pluginRepositories> <build> <plugins> <plugin> <groupId>org.apache.maven.plugins</groupId> <artifactId>maven-compiler-plugin</artifactId> <configuration> <source>1.8</source> <target>1.8</target> <encoding>UTF-8</encoding> </configuration> </plugin> <plugin> <groupId>org.apache.maven.plugins</groupId> <artifactId>maven-compiler-plugin</artifactId> <version>3.11.0</version> <configuration> <compilerArgs> <arg>--add-modules=jdk.compiler</arg> <!-- 替代旧版tools.jar --> </compilerArgs> </configuration> </plugin> <plugin> <groupId>org.apache.maven.plugins</groupId> <artifactId>maven-assembly-plugin</artifactId> <version>2.4</version> <configuration> <descriptors> <descriptor> src/main/assembly/assembly.xml </descriptor> </descriptors> </configuration> <executions> <execution> <id>make-assembly</id> <phase>package</phase> <goals> <goal>single</goal> </goals> </execution> </executions> </plugin> <plugin> <groupId>org.apache.maven.plugins</groupId> <artifactId>maven-surefire-plugin</artifactId> <version>2.10</version> <configuration> <parallel>false</parallel> </configuration> </plugin> <!-- add extra plugins here --> <plugin> <groupId>org.apache.maven.plugins</groupId> <artifactId>maven-jar-plugin</artifactId> <configuration> <archive> <manifest> <addClasspath>true</addClasspath> <classpathPrefix>lib/</classpathPrefix> </manifest> </archive> </configuration> </plugin> </plugins> <pluginManagement> <plugins> <plugin> <groupId>org.eclipse.m2e</groupId> <artifactId>lifecycle-mapping</artifactId> <version>1.0.0</version> <configuration> <lifecycleMappingMetadata> <pluginExecutions> <pluginExecution> <pluginExecutionFilter> <groupId>org.apache.maven.plugins</groupId> <artifactId>maven-dependency-plugin</artifactId> <versionRange>[1.0.0,)</versionRange> <goals> <goal>copy-dependencies</goal> </goals> </pluginExecutionFilter> <action> <ignore/> </action> </pluginExecution> </pluginExecutions> </lifecycleMappingMetadata> </configuration> </plugin> <plugin> <artifactId>maven-assembly-plugin</artifactId> <configuration> <archive> </archive> <descriptorRefs> <descriptorRef>jar-with-dependencies</descriptorRef> </descriptorRefs> </configuration> </plugin> </plugins> </pluginManagement> </build> </project> pom文件如上，打包报错[ERROR] Failed to execute goal on project TelevisionProgramGame: Could not collect dependencies for project com.sina.data.game:TelevisionProgramGame:jar:1.0-SNAPSHOT [ERROR] Failed to read artifact descriptor for org.apache.htrace:htrace-core:jar:incubating:3.1.0 [ERROR] Caused by: The following artifacts could not be resolved: org.apache.htrace:htrace-core:pom:3.1.0 (absent): Could not transfer artifact org.apache.htrace:htrace-core:pom:3.1.0 from/to maven-default-http-blocker (http://0.0.0.0/): Blocked mirror for repositories: [thirdparty (http://repo.hadoop.data.sina.com.cn:8081/nexus/content/repositories/thirdparty, default, releases), cdh.repo (http://repo.hadoop.data.sina.com.cn:8081/nexus/content/repositories/cdh.repo/, default, releases+snapshots), central (http://repo.hadoop.data.sina.com.cn:8081/nexus/content/repositories/central, default, releases)] [ERROR] Failed to read artifact descriptor for hadoop-lzo:hadoop-lzo-cdh4:jar:gplextras:0.4.15 [ERROR] Caused by: The following artifacts could not be resolved: hadoop-lzo:hadoop-lzo-cdh4:pom:0.4.15 (absent): Could not transfer artifact hadoop-lzo:hadoop-lzo-cdh4:pom:0.4.15 from/to maven-default-http-blocker (http://0.0.0.0/): Blocked mirror for repositories: [thirdparty (http://repo.hadoop.data.sina.com.cn:8081/nexus/content/repositories/thirdparty, default, releases), cdh.repo (http://repo.hadoop.data.sina.com.cn:8081/nexus/content/repositories/cdh.repo/, default, releases+snapshots), central (http://repo.hadoop.data.sina.com.cn:8081/nexus/content/repositories/central, default, releases)] [ERROR] Failed to read artifact descriptor for com.sina.Armyknife:sina:jar:0.5 [ERROR] Caused by: The following artifacts could not be resolved: com.sina.Armyknife:sina:pom:0.5 (absent): Could not transfer artifact com.sina.Armyknife:sina:pom:0.5 from/to maven-default-http-blocker (http://0.0.0.0/): Blocked mirror for repositories: [thirdparty (http://repo.hadoop.data.sina.com.cn:8081/nexus/content/repositories/thirdparty, default, releases), cdh.repo (http://repo.hadoop.data.sina.com.cn:8081/nexus/content/repositories/cdh.repo/, default, releases+snapshots), central (http://repo.hadoop.data.sina.com.cn:8081/nexus/content/repositories/central, default, releases)] [ERROR] Failed to read artifact descriptor for com.sina.hivexec:hive-exec:jar:0.13.0 [ERROR] Caused by: The following artifacts could not be resolved: com.sina.hivexec:hive-exec:pom:0.13.0 (absent): Could not transfer artifact com.sina.hivexec:hive-exec:pom:0.13.0 from/to maven-default-http-blocker (http://0.0.0.0/): Blocked mirror for repositories: [thirdparty (http://repo.hadoop.data.sina.com.cn:8081/nexus/content/repositories/thirdparty, default, releases), cdh.repo (http://repo.hadoop.data.sina.com.cn:8081/nexus/content/repositories/cdh.repo/, default, releases+snapshots), central (http://repo.hadoop.data.sina.com.cn:8081/nexus/content/repositories/central, default, releases)] [ERROR]
最新发布
09-18
<url>http://0.0.0.0/</url> <!-- 禁用默认拦截器 --> <mirrorOf>maven-default-http-blocker</mirrorOf> </mirror> ``` #### 2️⃣ 解决方法二：在项目 POM 中添加例外 ```xml <!-- pom.xml --> <id>...
编译内核出错：recipe for target ‘arch/arm/boot/compressed/piggy.lzo‘ failed
weixin_44498318的博客
04-28 2689
问题： /bin/sh: 1: lzop: not found arch/arm/boot/compressed/Makefile:180: recipe for target 'arch/arm/boot/compressed/piggy.lzo' failed make[2]: *** [arch/arm/boot/compressed/piggy.lzo] Error 1 arch/arm/boot/Makefile:52: recipe for target 'arch/arm/boot/compr
lzo-2.06.tar.gz/lzo-2.10.tar.gz
08-23
标题中的“lzo-2.06.tar.gz/lzo-2.10.tar.gz”指的是LZO压缩库的两个不同版本的源代码压缩包，分别对应版本2.06和2.10。这些文件是用tar.gz格式打包的，这是一种在Unix和类Unix系统中常见的归档和压缩方法。tar主要...
Hadoop HA 搭建发现新增namenode其他节点报Retrying connect to server 0.0.0.0:8032异常
qq_41018861的博客
09-23 2371
问题是在笔者配置集群高可用时出现，刚开始集群正常后来出现了新增的namenode服务起不起来后来修改该节点hostname替换为0.0.0.0后才得已解决，但是这个问题困扰了笔者在搭建ha开始到解决问题的日日夜夜，今天突然集群报错，造成集群瘫痪掉。
解决Retrying connect to server: 0.0.0.0/0.0.0.0:8032. Already tried 0 time(s); retry policy is...
热门推荐
panguoyuan的专栏
08-21 7万+
yarn.resourcemanager.address S1PA124:8032 yarn.resourcemanager.scheduler.address S1PA124:8030 yarn.resourcemanager.resource-tracker.address S1PA124:8031
Connecting to ResourceManager at /0.0.0.0:8032
weixin_45075077的博客
09-17 9905
今天在学习spark on yarn调优发现，从spark启动任务出现了 Connecting to ResourceManager at /0.0.0.0:8032的错误。 当前集群配置如下：单核2G内存 node002 namenode,datanode,zookeeper,zkFc,journalnode,resourcemanager,nodemanager,worker，historys...
（05）LZO创建索引
song_quan_的博客
06-24 261
1）创建 LZO 文件的索引，LZO 压缩文件的可切片特性依赖于其索引，故我们需要手动为 LZO 压缩文件创建索引。若无索引，则 LZO 文件的切片只有一个。 2）测试 （1）将 bigtable.lzo（200M）上传到集群的根目录 （2）执行 wordcount 程序 （3）对上传的 LZO 文件建索引 （4）注意：如果以上任务，在运行过程中报如下异常 解决办法：在 hadoop102 的/opt/module/hadoop-3.
spark Yarn模式运行报错：Connecting to ResourceManager at /0.0.0.0:8032
qq_44881930的博客
08-03 4715
1、报错信息如下 11 20:11:07,922 INFO [main] org.apache.hadoop.ipc.Client: Retrying connect to server: 0.0.0.0/0.0.0.0:8030. Already tried 3 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1 SECONDS) 2014-05-11 20:11:09,966 IN
lzo 压缩文件并且创建索引
Genebrother
06-07 717
lzo 压缩算法压缩比较好，默认的lzo文件是不支持切片操作的，需要手动创建索引文件 1、确保服务器内已经安装好lzop命令 lzop message.log
hadoop伪分布式下yarn搭建中遇到的问题
swift1224L的博客
05-08 1085
Retrying connect to server: 0.0.0.0/0.0.0.0:8032 在配置了yarn的情况下，如果不想启动 YARN，务必把配置文件 mapred-site.xml 重命名，改成 mapred-site.xml.template，需要用时改回来就行。否则在该配置文件存在，而未开启 YARN 的情况下，运行程序会提示 “Retrying connect to serve...
hadoop安装配置过程中出现的问题
u010049696的专栏
07-01 1093
配置方法参考网址：http://www.micmiu.com/bigdata/hadoop/hadoop2x-cluster-setup/ 配置过程中出现的问题： 1、Retrying connect to server: 0.0.0.0/0.0.0.0:8032. Already tried 0 time(s); retry policy is...  解决方法：配置yarn-site
dolphinscheduler环境搭建
TS_forever007的博客
02-11 1538
dolphinscheduler的安装部署过程
评论
成就一亿技术人!
拼手气红包6.0元
还能输入1000个字符
 
红包 添加红包
表情包 插入表情
表情包 代码片
 条评论被折叠 查看
添加红包

请填写红包祝福语或标题

个

红包个数最小为10个

元

红包金额最低5元

当前余额3.43元 前往充值 >
需支付：10.00元
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付元
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明：

1.余额是钱包充值的虚拟货币，按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载，可以购买VIP、付费专栏及课程。

余额充值