Windows下IDEA本地读写HDFS

resources文件夹下添加core-site.xml

主要属性：

<property>
  <name>fs.defaultFS</name>
  <value>hdfs://hadoop-master:8020</value>
</property>

注意：cdh环境hdfs端口是8020，hadoop环境hdfs端口是9000

      另外,注意本地hosts配置

 

1. Scala普通读取：

object TestHdfs {
  val prop = new Properties()
  def main(args: Array[String]): Unit = {
    val conf = new Configuration()
    conf.set("fs.defaultFS", "hdfs://192.168.100.41:8020")    //可以不设置

    val fs = FileSystem.get(conf)

    val path = "hdfs://192.168.100.41:8020/test/ideaReadHdfs.txt"

    //读
    val fis = fs.open(new Path(path))

    IOUtils.copyBytes(fis,