<?xml version="1.0" encoding="utf-8" ?><rss version="2.0"><channel><title><![CDATA[jtpython666的博客]]></title><description><![CDATA[]]></description><link>https://blog.youkuaiyun.com/jtpython666</link><language>zh-cn</language><generator>https://blog.youkuaiyun.com/</generator><copyright><![CDATA[Copyright &copy; jtpython666]]></copyright><item><title><![CDATA[凸性与久期]]></title><link>https://blog.youkuaiyun.com/jtpython666/article/details/132495066</link><guid>https://blog.youkuaiyun.com/jtpython666/article/details/132495066</guid><author>jtpython666</author><pubDate>Fri, 25 Aug 2023 14:28:33 +0800</pubDate><description><![CDATA[凸性是指债券到期收益率发生变动而引起的债券价格变动幅度的变动程度。凸性是指债券价格对收益率的二阶导数。]]></description><category></category></item><item><title><![CDATA[linux命令记录]]></title><link>https://blog.youkuaiyun.com/jtpython666/article/details/128814019</link><guid>https://blog.youkuaiyun.com/jtpython666/article/details/128814019</guid><author>jtpython666</author><pubDate>Tue, 31 Jan 2023 11:09:58 +0800</pubDate><description><![CDATA[home/was/apache-tomcat-9.0.41, cd logs后 tail - f  catalina.out  日志输出路径。/home/was/apache-tomcat-0.0.41/logs  tomcat启动脚本路径。]]></description><category></category></item><item><title><![CDATA[查询数据库元数据]]></title><link>https://blog.youkuaiyun.com/jtpython666/article/details/128660475</link><guid>https://blog.youkuaiyun.com/jtpython666/article/details/128660475</guid><author>jtpython666</author><pubDate>Thu, 12 Jan 2023 15:25:02 +0800</pubDate><description><![CDATA[【代码】查询数据库元数据。]]></description><category></category></item><item><title><![CDATA[python将csv文件插入kudu表]]></title><link>https://blog.youkuaiyun.com/jtpython666/article/details/128348122</link><guid>https://blog.youkuaiyun.com/jtpython666/article/details/128348122</guid><author>jtpython666</author><pubDate>Fri, 16 Dec 2022 21:21:19 +0800</pubDate><description><![CDATA[【代码】python将csv文件插入kudu表。]]></description><category></category></item><item><title><![CDATA[银行黄金会计分录]]></title><link>https://blog.youkuaiyun.com/jtpython666/article/details/128222178</link><guid>https://blog.youkuaiyun.com/jtpython666/article/details/128222178</guid><author>jtpython666</author><pubDate>Wed, 07 Dec 2022 16:14:24 +0800</pubDate><description><![CDATA[银行黄金会计账目]]></description><category></category></item><item><title><![CDATA[季度时间转换]]></title><link>https://blog.youkuaiyun.com/jtpython666/article/details/127889225</link><guid>https://blog.youkuaiyun.com/jtpython666/article/details/127889225</guid><author>jtpython666</author><pubDate>Wed, 16 Nov 2022 17:14:15 +0800</pubDate><description><![CDATA[【代码】季度时间转换。]]></description><category></category></item><item><title><![CDATA[regexp_extract用法]]></title><link>https://blog.youkuaiyun.com/jtpython666/article/details/127818491</link><guid>https://blog.youkuaiyun.com/jtpython666/article/details/127818491</guid><author>jtpython666</author><pubDate>Sat, 12 Nov 2022 11:54:21 +0800</pubDate><description><![CDATA[版权声明：本文为优快云博主「weixin_43383676」的原创文章，遵循CC 4.0 BY-SA版权协议，转载请附上原文出处链接及本声明。原文链接：https://blog.youkuaiyun.com/weixin_43383676/article/details/101020406。函数：regexp_extract(str, regexp[, idx]) ----字符串正则表达式解析函数。idx是返回结果 取表达式的哪一部分 默认值为1。要注意的是idx的数字不能大于表达式中()的个数。]]></description><category></category></item><item><title><![CDATA[正则函数regexp_replace用法]]></title><link>https://blog.youkuaiyun.com/jtpython666/article/details/127147551</link><guid>https://blog.youkuaiyun.com/jtpython666/article/details/127147551</guid><author>jtpython666</author><pubDate>Sun, 02 Oct 2022 23:04:11 +0800</pubDate><description><![CDATA[正则regexp_replace用法，方括号内全部可以匹配[()（） ]]]></description><category></category></item><item><title><![CDATA[hive求一个日期是星期几]]></title><link>https://blog.youkuaiyun.com/jtpython666/article/details/125789554</link><guid>https://blog.youkuaiyun.com/jtpython666/article/details/125789554</guid><author>jtpython666</author><pubDate>Thu, 14 Jul 2022 17:43:09 +0800</pubDate><description><![CDATA[使用hive函数获取星期几。pmod(datediff(#date#, ‘1920-01-01’) - 3, 7)#date#表示给的日期。输出的结果为0-6的数，分别表示 日，一，二 … 六。2016-01-01 5
2016-01-02 6
2016-01-03 0如果想让周一到周六对应数字1-7只需要将查询出来的数据进行判断就行了，如下：IF(pmod(datediff(#date#, ‘1920-01-01’) - 3, 7)=‘0’, 7, pmod(datediff(#date#, ‘1920-]]></description><category></category></item><item><title><![CDATA[大数据复习--spark]]></title><link>https://blog.youkuaiyun.com/jtpython666/article/details/125707575</link><guid>https://blog.youkuaiyun.com/jtpython666/article/details/125707575</guid><author>jtpython666</author><pubDate>Tue, 12 Jul 2022 15:18:15 +0800</pubDate><description><![CDATA[spark与hadoop之间的根本差异是多个作业之间的数据通信问题，spark多个作业之间数据通信是基于内存，而hadoop是基于磁盘。
一、executor与core
sparkexecutor是集群中运行在工作节点（worker）中的一个Jvm进程，是整个集群中的专门用于计算的节点。在提交应用中，可以提供参数指定计算节点的个数，以及对应的资源。这里的资源一般指的是executor的内存大小和使用的虚拟cpu核（core）数量。
–num-executors   配置executor的数量
–execut]]></description><category></category></item><item><title><![CDATA[大数据复习-hive]]></title><link>https://blog.youkuaiyun.com/jtpython666/article/details/125639986</link><guid>https://blog.youkuaiyun.com/jtpython666/article/details/125639986</guid><author>jtpython666</author><pubDate>Sun, 10 Jul 2022 16:09:27 +0800</pubDate><description><![CDATA[一、hive建表优化
分区表：将表中的数据按照指定的规则份文件夹管理。
分桶表：对hive表分桶可以将表中记录按分桶键（字段）的哈希值分散进多个文件中，这些小文件称为桶。桶以文件为单位管理数据。分区针对的是数据的存储路径，分桶针对的是数据文件。如果查询的维度是分桶字段，查询的时候先确定数据所在的桶，提升查询效率。
进行join的时候，关联字段就是分桶字段，只有相同的桶之间的数据进行关联，从而提升关联效率。
设置下set hive.enforce.bucketing=true
set mapreduce.jo]]></description><category></category></item><item><title><![CDATA[大数据复习-生产调优]]></title><link>https://blog.youkuaiyun.com/jtpython666/article/details/125623186</link><guid>https://blog.youkuaiyun.com/jtpython666/article/details/125623186</guid><author>jtpython666</author><pubDate>Tue, 05 Jul 2022 22:28:48 +0800</pubDate><description><![CDATA[一、HDFS核心参数
修改namenode内存，比如设置namenode内存为3g，hadoop2.0参数为HADOOP_NAMENODE_OPTS=-Xmx3072m
namenode最小值1G，每增加1000000个block，增加1G内存。datenode最小值4G，block数，或者副本数升高，都应该调大datanode的值。一个datanode上的副本总数低于4000000，调为4G，超过4000000，每增加1000000，增加1G。
HADOOP_NAMENODE_OPTS=“INFO,RFA]]></description><category></category></item><item><title><![CDATA[大数据面试技术复习hadoop,hdfs,yarn]]></title><link>https://blog.youkuaiyun.com/jtpython666/article/details/125559315</link><guid>https://blog.youkuaiyun.com/jtpython666/article/details/125559315</guid><author>jtpython666</author><pubDate>Tue, 05 Jul 2022 16:26:47 +0800</pubDate><description><![CDATA[大数据复习笔记
一、hadoop
hadoop特性：高可靠，高拓展，高效，，高容错性。
面试考点：
1.hadoop1.0和2.0区别
**  hadoop1.0主要组成为mapreduce（计算+调度）,hdfs（存储）,common(辅助工具)，2.0中加入了yarn负责资源调度，mapreduce只负责计算。3.0中组成上没有区别。**
2.常用端口号
hadoop3.x
hdfs namenode 内部通常端口 ：8020/9000/9820
hdfs namenode 对用户的查询端口：9870
]]></description><category></category></item><item><title><![CDATA[删除外部表源数据]]></title><link>https://blog.youkuaiyun.com/jtpython666/article/details/125047321</link><guid>https://blog.youkuaiyun.com/jtpython666/article/details/125047321</guid><author>jtpython666</author><pubDate>Mon, 30 May 2022 15:23:30 +0800</pubDate><description><![CDATA[alter table dwd.portfolio_info set tblproperties(‘external’ = ‘false’);
truncate table dwd_farx.portfolio_info;
alter table dwd_farx.portfolio_info set tblproperties(‘external’=‘true’);

]]></description><category></category></item><item><title><![CDATA[查询字段个数]]></title><link>https://blog.youkuaiyun.com/jtpython666/article/details/125047260</link><guid>https://blog.youkuaiyun.com/jtpython666/article/details/125047260</guid><author>jtpython666</author><pubDate>Mon, 30 May 2022 15:20:15 +0800</pubDate><description><![CDATA[select count(column_name) from all_tab_columns a where a.table_name = 'SECURITIES'


]]></description><category></category></item><item><title><![CDATA[查询字段个数]]></title><link>https://blog.youkuaiyun.com/jtpython666/article/details/124819822</link><guid>https://blog.youkuaiyun.com/jtpython666/article/details/124819822</guid><author>jtpython666</author><pubDate>Tue, 17 May 2022 13:40:41 +0800</pubDate><description><![CDATA[select count(column_name) from all_tab_columns a where a.table_name = 'SECURITIES'


]]></description><category></category></item><item><title><![CDATA[回购shell脚本]]></title><link>https://blog.youkuaiyun.com/jtpython666/article/details/124750377</link><guid>https://blog.youkuaiyun.com/jtpython666/article/details/124750377</guid><author>jtpython666</author><pubDate>Fri, 13 May 2022 13:47:50 +0800</pubDate><description><![CDATA[#!/bin/bash
# FileName: alldateduringtwodays1.sh
# Description: Print all the date during the two days you inpute.
#Simple Usage:sh repo_asset_3days.sh 2017-04-01 2017-06-14 /user/etl/fa45/dwd/repo_asset.sql &amp;
 
echo "input param length:" $# 
if [[ $# ]]></description><category></category></item><item><title><![CDATA[oracle 抽到hive]]></title><link>https://blog.youkuaiyun.com/jtpython666/article/details/124729891</link><guid>https://blog.youkuaiyun.com/jtpython666/article/details/124729891</guid><author>jtpython666</author><pubDate>Thu, 12 May 2022 14:06:19 +0800</pubDate><description><![CDATA[oracle 到hive
{
  "job": {
    "jobId": "${jobId}",
    "jobVersionId": "${jobVersionId}",
    "executeId": "${executeId}",
    "serverPort": "${serverPort}",
    "msg": "${msg}",
    "setting": {
      "speed": {
        "channel": "1"
      },
      "erro]]></description><category></category></item><item><title><![CDATA[datax hive抽到oracle]]></title><link>https://blog.youkuaiyun.com/jtpython666/article/details/124729859</link><guid>https://blog.youkuaiyun.com/jtpython666/article/details/124729859</guid><author>jtpython666</author><pubDate>Thu, 12 May 2022 14:05:26 +0800</pubDate><description><![CDATA[datax  hive-oracle
{
  "job": {
    "jobId": "${jobId}",
    "jobVersionId": "${jobVersionId}",
    "executeId": "${executeId}",
    "serverPort": "${serverPort}",
    "msg": "${msg}",
    "setting": {
      "speed": {
        "channel": "1"
      },
     ]]></description><category></category></item><item><title><![CDATA[hadoop跑数命令]]></title><link>https://blog.youkuaiyun.com/jtpython666/article/details/123877253</link><guid>https://blog.youkuaiyun.com/jtpython666/article/details/123877253</guid><author>jtpython666</author><pubDate>Thu, 31 Mar 2022 16:45:14 +0800</pubDate><description><![CDATA[/home/ctmagent/etl_app/fahs/hql/dwd/trade/incre--hs�������ϴ�·��
/home/ctmagent/etl_app/fa45/hql/dwd/trade/incre--yss�������ϴ�·��
cd /home/ctmagent/etl_app/fahs/hql/dwd/trade/incre
cd /home/ctmagent/etl_app/fa45/hql/dwd/trade/incre
hadoop fs -put -f bond_tr]]></description><category></category></item></channel></rss>