god__send-优快云博客

原创 CentOS7安装Cdh5

CentOS7 Mini版安装Cdh5步骤一、准备环境1.机器配置2.必要的软件一、准备环境1.机器配置至少三台物理机或者虚拟机安装CentOS7 mini版，要求可以互相通信我的是4台虚拟机，配置均为2G+20G192.168.65.101 n1192.168.65.102 n2192.168.65.103 d1192.168.65.104 d22.必要的软件...

2018-10-24 17:50:33 496

原创 Cannot overwrite a path that is also being read from.

sparksql问题解决遇到这个问题的解决方案1.设置 spark.sql.hive.convertMetastoreParquet=true或者spark.sql.hive.convertMetastoreOrc=true2.设置checkpiont 阻断血缘，设置目录并设置checkpoint，清理目录spark.cleaner.referenceTracking.cleanCheckpoints=true。3.先插入临时表，再移动到目标表。...

2020-10-09 11:43:45 5115

原创每日一题3---无重复最长子串

代码public int lengthOfLongestSubstring(String s) { int n = s.length(); int res = 0, begin = 0, end = 0; HashSet<Character> characters = new HashSet<>(); while (end < n && begin < n) { if

2020-06-19 14:53:47 147

原创每日一题2---两数相加

代码/** * Definition for singly-linked list. * public class ListNode { * int val; * ListNode next; * ListNode(int x) { val = x; } * } */class Solution { public ListNode addTwoNumbers(ListNode l1, ListNode l2) { ListNode dummyHead =

2020-06-19 09:04:36 173

原创每日一题1---两数之和

这里写自定义目录标题代码解析结果代码class Solution { public int[] twoSum(int[] nums, int target) { Map<Integer, Integer> map = new HashMap<>(); for (int i = 0; i <= nums.length; i++) { int otherPart = target - nums[i];

2020-06-17 18:04:57 157

原创使用PySpark操作kudu

1.从kudu中加载数据使用pyspark需要导入spark中的包，并且初始化SparkSession。import osimport sysos.environ['SPARK_HOME'] = '/usr/hdp/spark2'sys.path.append("/usr/hdp/spark2/python")from pyspark.sql import SparkSession...

2018-12-13 11:13:07 2716