spark高级数据分析实战---随机森林实现

最新推荐文章于 2024-04-29 23:48:30 发布

原创

最新推荐文章于 2024-04-29 23:48:30 发布 · 4.9k 阅读

3 ·

CC 4.0 BY-SA版权

补充决策树那块没写完的，废话不多说，直接上代码，详解可以看注释内容

package mllib.tree

import org.apache.log4j.{Level, Logger}
import org.apache.spark.mllib.evaluation.MulticlassMetrics
import org.apache.spark.mllib.linalg.Vectors
import org.apache.spark.mllib.regression.LabeledPoint
import org.apache.spark.mllib.tree.{RandomForest, DecisionTree}
import org.apache.spark.mllib.tree.model.{RandomForestModel, DecisionTreeModel}
import org.apache.spark.rdd.RDD
import org.apache.spark.{SparkContext, SparkConf}


/**
  * Created by 汪本成 on 2016/7/18.
  */
object randomForest {
  //屏蔽不必要的日志显示在终端上
//  Logger.getLogger("org.apache.spark").setLevel(Level.WARN)
//  Logger.getLogger("org.apache.eclipse.jetty.server").setLevel(Level.OFF)

  var beg = System.currentTimeMillis()
  //创建入口对象
  val conf = new SparkConf().setAppName("rndomForest").setMaster("local")
  val sc= new SparkContext(conf)

  val HDFS_COVDATA_PATH = "hdfs://192.168.43.150:9000/user/spark/sparkLearning/mllib/covtype.data"
  val rawData = sc.textFile(HDFS_COVDATA_PATH)

  //设置LabeledPoint格式
  val data =