Create Local Cloudera Parcels Repo to Save Your ASS

在中国通过Cloudera Manager安装Hadoop集群时,由于网络问题,下载Cloudera Parcels非常慢。为了解决这个问题,你需要创建本地Parcel仓库。按照Cloudera的指导,下载相应操作系统的parcel文件,修改Cloudera Manager的Local Parcel Repository Path,将parcel文件和对应的manifest.json及SHA校验文件放入仓库中。完成这些步骤后,Cloudera Manager会检测到本地仓库并显示在Parcels页面上。

摘要生成于 C知道 ,由 DeepSeek-R1 满血版支持, 前往体验 >

If you want to install Hadoop cluster via Cloudera Manager in CHINA, the network to cloudera parels is toooooo slow. You must create your local parcel repo to save your ass, thanks to the DAMN GFW!!

 

Follow this page from cloudera:

  • Download parcels:    
            Link to CDH4: http://archive.cloudera.com/cdh4/parcels/                                                                                  
            Link to IMPALA: http://archive.cloudera.com/impala/parcels/

  • Verify the location of the local parcel repository on your Cloudera Manager server: Go to the Administration page -> Properties tab -> Parcels category. You can change the local repository path in the Local Parcel Repository Path property. By default it is /opt/cloudera/parcel-repo.
  • Download the .parcel file for your operating system: (el5 or el6 for Red Hat 5 or 6, lucid or precise for Ubuntu and so on) and place it into the local parcel repository on your Cloudera Manager server. In my case it's CDH-4.3.0-1.cdh4.3.0.p0.22-el6.parcel and IMPALA-1.0.1-1.p0.431-el6.parcel
  • Open the manifest.json file in the same directory as the .parcel file you just copied. Find the section of the manifest that corresponds to the parcel you downloaded: In my case, I'm running RHEL 6 and copied the parcel file CDH-4.2.0-1.cdh4.2.0.p0.10-el6.parcel, then look for the section:
      {
          "parcelName": "CDH-4.3.0-1.cdh4.3.0.p0.22-el6.parcel",
          "components": [
            { "name":     "flume-ng",
              "version":  "1.3.0-cdh4.3.0",
              "pkg_version":  "1.3.0+159" 
            }
            ,{ "name":     "hadoop-0.20-mapreduce",
              "version":  "2.0.0-cdh4.3.0",
              "pkg_version":  "2.0.0+1357" 
            }
            ,{ "name":     "hadoop-hdfs",
              "version":  "2.0.0-cdh4.3.0",
              "pkg_version":  "2.0.0+1357" 
            }
            ,{ "name":     "hadoop-httpfs",
              "version":  "2.0.0-cdh4.3.0",
              "pkg_version":  "2.0.0+1357" 
            }
            ,{ "name":     "hadoop-mapreduce",
              "version":  "2.0.0-cdh4.3.0",
              "pkg_version":  "2.0.0+1357" 
            }
            ,{ "name":     "hadoop-yarn",
              "version":  "2.0.0-cdh4.3.0",
              "pkg_version":  "2.0.0+1357" 
            }
            ,{ "name":     "hadoop",
              "version":  "2.0.0-cdh4.3.0",
              "pkg_version":  "2.0.0+1357" 
            }
            ,{ "name":     "hbase",
              "version":  "0.94.6-cdh4.3.0",
              "pkg_version":  "0.94.6+96" 
            }
            ,{ "name":     "hcatalog",
              "version":  "0.5.0-cdh4.3.0",
              "pkg_version":  "0.5.0+9" 
            }
            ,{ "name":     "hive",
              "version":  "0.10.0-cdh4.3.0",
              "pkg_version":  "0.10.0+121" 
            }
            ,{ "name":     "mahout",
              "version":  "0.7-cdh4.3.0",
              "pkg_version":  "0.7+16" 
            }
            ,{ "name":     "oozie",
              "version":  "3.3.2-cdh4.3.0",
              "pkg_version":  "3.3.2+49" 
            }
            ,{ "name":     "pig",
              "version":  "0.11.0-cdh4.3.0",
              "pkg_version":  "0.11.0+28" 
            }
            ,{ "name":     "sqoop",
              "version":  "1.4.3-cdh4.3.0",
              "pkg_version":  "1.4.3+34" 
            }
            ,{ "name":     "sqoop2",
              "version":  "1.99.1-cdh4.3.0",
              "pkg_version":  "1.99.1+115" 
            }
            ,{ "name":     "whirr",
              "version":  "0.8.2-cdh4.3.0",
              "pkg_version":  "0.8.2+10" 
            }
            ,{ "name":     "zookeeper",
              "version":  "3.4.5-cdh4.3.0",
              "pkg_version":  "3.4.5+19" 
            }
            ,{ "name":     "hue",
              "version":  "2.3.0-cdh4.3.0",
              "pkg_version":  "2.3.0+136" 
            }
          ],
          "hash": "df5cc61b2d257aaf625341f709a4f8e09754038a"
        }
     
      {
          "parcelName": "IMPALA-1.0.1-1.p0.431-el6.parcel",
          "components": [
            { "name":     "impala",
              "version":  "1.0.1-SNAPSHOT",
              "pkg_version":  "1.0.1"
            }
          ],
          "hash": "992467f2e54bd394cbdd3f4ed97b6e9bead60ff0"
        }
     
  • Create a text file whose name is <parcel file name> .sha (e.g. CDH-4.2.0-1.cdh4.2.0.p0.10-el6.parcel.sha) and copy the hash code into it. In my case, I created file CDH-4.3.0-1.cdh4.3.0.p0.22-el6.parcel.sha and IMPALA-1.0.1-1.p0.431-el6.parcel.sha, then I copied the hash value into them respectively:
    # cat > CDH-4.3.0-1.cdh4.3.0.p0.22-el6.parcel.sha
    df5cc61b2d257aaf625341f709a4f8e09754038a
    ^C
     
    # cat > IMPALA-1.0.1-1.p0.431-el6.parcel.sha
    992467f2e54bd394cbdd3f4ed97b6e9bead60ff0
    ^C
     Place both of files into your local parcel repository, /opt/cloudera/parcel-repo by default, depending you settings in the above step.
  • Once these files are in place, Cloudera Manager will pick up the parcel and it will appear on the Hosts > Parcels page. Note that how quickly this occurs depends on the Parcel Update Frequency setting, set by default to 1 hour. You can change this on the Administration page -> Properties tab -> Parcels category.
评论
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值