Create Local Cloudera Parcels Repo to Save Your ASS

在中国通过Cloudera Manager安装Hadoop集群时,由于网络问题,下载Cloudera Parcels非常慢。为了解决这个问题,你需要创建本地Parcel仓库。按照Cloudera的指导,下载相应操作系统的parcel文件,修改Cloudera Manager的Local Parcel Repository Path,将parcel文件和对应的manifest.json及SHA校验文件放入仓库中。完成这些步骤后,Cloudera Manager会检测到本地仓库并显示在Parcels页面上。

摘要生成于 C知道 ,由 DeepSeek-R1 满血版支持, 前往体验 >

If you want to install Hadoop cluster via Cloudera Manager in CHINA, the network to cloudera parels is toooooo slow. You must create your local parcel repo to save your ass, thanks to the DAMN GFW!!

 

Follow this page from cloudera:

  • Download parcels:    
            Link to CDH4: http://archive.cloudera.com/cdh4/parcels/                                                                                  
            Link to IMPALA: http://archive.cloudera.com/impala/parcels/

  • Verify the location of the local parcel repository on your Cloudera Manager server: Go to the Administration page -> Properties tab -> Parcels category. You can change the local repository path in the Local Parcel Repository Path property. By default it is /opt/cloudera/parcel-repo.
  • Download the .parcel file for your operating system: (el5 or el6 for Red Hat 5 or 6, lucid or precise for Ubuntu and so on) and place it into the local parcel repository on your Cloudera Manager server. In my case it's CDH-4.3.0-1.cdh4.3.0.p0.22-el6.parcel and IMPALA-1.0.1-1.p0.431-el6.parcel
  • Open the manifest.json file in the same directory as the .parcel file you just copied. Find the section of the manifest that corresponds to the parcel you downloaded: In my case, I'm running RHEL 6 and copied the parcel file CDH-4.2.0-1.cdh4.2.0.p0.10-el6.parcel, then look for the section:
      {
          "parcelName": "CDH-4.3.0-1.cdh4.3.0.p0.22-el6.parcel",
          "components": [
            { "name":     "flume-ng",
              "version":  "1.3.0-cdh4.3.0",
              "pkg_version":  "1.3.0+159" 
            }
            ,{ "name":     "hadoop-0.20-mapreduce",
              "version":  "2.0.0-cdh4.3.0",
              "pkg_version":  "2.0.0+1357" 
            }
            ,{ "name":     "hadoop-hdfs",
              "version":  "2.0.0-cdh4.3.0",
              "pkg_version":  "2.0.0+1357" 
            }
            ,{ "name":     "hadoop-httpfs",
              "version":  "2.0.0-cdh4.3.0",
              "pkg_version":  "2.0.0+1357" 
            }
            ,{ "name":     "hadoop-mapreduce",
              "version":  "2.0.0-cdh4.3.0",
              "pkg_version":  "2.0.0+1357" 
            }
            ,{ "name":     "hadoop-yarn",
              "version":  "2.0.0-cdh4.3.0",
              "pkg_version":  "2.0.0+1357" 
            }
            ,{ "name":     "hadoop",
              "version":  "2.0.0-cdh4.3.0",
              "pkg_version":  "2.0.0+1357" 
            }
            ,{ "name":     "hbase",
              "version":  "0.94.6-cdh4.3.0",
              "pkg_version":  "0.94.6+96" 
            }
            ,{ "name":     "hcatalog",
              "version":  "0.5.0-cdh4.3.0",
              "pkg_version":  "0.5.0+9" 
            }
            ,{ "name":     "hive",
              "version":  "0.10.0-cdh4.3.0",
              "pkg_version":  "0.10.0+121" 
            }
            ,{ "name":     "mahout",
              "version":  "0.7-cdh4.3.0",
              "pkg_version":  "0.7+16" 
            }
            ,{ "name":     "oozie",
              "version":  "3.3.2-cdh4.3.0",
              "pkg_version":  "3.3.2+49" 
            }
            ,{ "name":     "pig",
              "version":  "0.11.0-cdh4.3.0",
              "pkg_version":  "0.11.0+28" 
            }
            ,{ "name":     "sqoop",
              "version":  "1.4.3-cdh4.3.0",
              "pkg_version":  "1.4.3+34" 
            }
            ,{ "name":     "sqoop2",
              "version":  "1.99.1-cdh4.3.0",
              "pkg_version":  "1.99.1+115" 
            }
            ,{ "name":     "whirr",
              "version":  "0.8.2-cdh4.3.0",
              "pkg_version":  "0.8.2+10" 
            }
            ,{ "name":     "zookeeper",
              "version":  "3.4.5-cdh4.3.0",
              "pkg_version":  "3.4.5+19" 
            }
            ,{ "name":     "hue",
              "version":  "2.3.0-cdh4.3.0",
              "pkg_version":  "2.3.0+136" 
            }
          ],
          "hash": "df5cc61b2d257aaf625341f709a4f8e09754038a"
        }
     
      {
          "parcelName": "IMPALA-1.0.1-1.p0.431-el6.parcel",
          "components": [
            { "name":     "impala",
              "version":  "1.0.1-SNAPSHOT",
              "pkg_version":  "1.0.1"
            }
          ],
          "hash": "992467f2e54bd394cbdd3f4ed97b6e9bead60ff0"
        }
     
  • Create a text file whose name is <parcel file name> .sha (e.g. CDH-4.2.0-1.cdh4.2.0.p0.10-el6.parcel.sha) and copy the hash code into it. In my case, I created file CDH-4.3.0-1.cdh4.3.0.p0.22-el6.parcel.sha and IMPALA-1.0.1-1.p0.431-el6.parcel.sha, then I copied the hash value into them respectively:
    # cat > CDH-4.3.0-1.cdh4.3.0.p0.22-el6.parcel.sha
    df5cc61b2d257aaf625341f709a4f8e09754038a
    ^C
     
    # cat > IMPALA-1.0.1-1.p0.431-el6.parcel.sha
    992467f2e54bd394cbdd3f4ed97b6e9bead60ff0
    ^C
     Place both of files into your local parcel repository, /opt/cloudera/parcel-repo by default, depending you settings in the above step.
  • Once these files are in place, Cloudera Manager will pick up the parcel and it will appear on the Hosts > Parcels page. Note that how quickly this occurs depends on the Parcel Update Frequency setting, set by default to 1 hour. You can change this on the Administration page -> Properties tab -> Parcels category.
当您遇到 "Failed to start Cloudera CM Server Service" 错误时,有几个可能的原因和解决方法: 1. 检查日志:首先,您可以查看 Cloudera CM Server 的日志文件以获取更详细的错误信息。默认情况下,日志文件位于 "/var/log/cloudera-scm-server/" 目录下,文件名为 "cloudera-scm-server.log"。使用文本编辑器或命令行工具打开该文件,查找与启动失败相关的错误消息,以帮助诊断问题。 2. 检查配置文件:确保您的 Cloudera CM Server 配置文件正确设置,并且没有任何错误或不兼容的配置项。可以尝试重新检查和编辑配置文件,然后重新启动服务。 3. 确保依赖项已安装:Cloudera CM Server 可能依赖其他软件包或库。确保这些依赖项已经正确安装,并且版本与 Cloudera CM Server 的要求相匹配。 4. 检查端口占用:Cloudera CM Server 使用特定的端口进行通信。确保这些端口没有被其他进程占用。您可以使用命令 "netstat -tuln" 查看当前系统上打开的端口。 5. 重新安装或更新:如果问题仍然存在,您可以尝试重新安装 Cloudera CM Server 或更新到最新版本,以解决可能存在的软件缺陷或错误。 6. 寻求支持:如果上述方法都无法解决问题,建议您寻求 Cloudera 官方支持或社区支持,他们会提供更具体的指导和故障排除步骤。 请注意,具体的解决方法可能因您的环境和具体错误而有所不同。根据错误日志和具体情况,选择适合您的解决方法。
评论
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值