LeeWei

sparkStreaming接收kafka消息，写入数据库 package com.ruijie.spark.stream import org.apache.kafka.clients.consumer.ConsumerConfig import org.apache.spark.streaming.{Seconds, StreamingContext} import org.apache.spark.{SparkConf, SparkContext} import org.apache.kafka.c

Docker内如何使用crontab run.sh #!/bin/bash timestamp=`date +%Y/%m/%d-%H:%M:%S` echo "System path is $PATH at $timestamp" entrypoint.sh #!/bin/bash # Start the run once job. echo "Docker container has been started" declare -p | grep -Ev 'BASHOPTS|BASH_V

standalone模式下的资源、任务调度源码剖析 整体流程图 standalone模式简介 standalone模式，是spark自带的资源调度框架，该框架下主要有三个节点： client 客户端，将应用程序提交，记录着要运行业务逻辑 master 负责资源管理 worker 负责任务的执行 spark Application有个Driver驱动，即用户提交的程序，Driver进程可以运...