Linux--19：Linux下查看大型代码的工具【MARK】

最新推荐文章于 2021-05-07 07:45:54 发布
原创 最新推荐文章于 2021-05-07 07:45:54 发布 · 1k 阅读 · 0 ·
CC 4.0 BY-SA版权
版权声明：本文为博主原创文章，遵循 CC 4.0 BY-SA 版权协议，转载请附上原文出处链接和本声明。
文章标签：

#Linux #类SourceInsight

本文分享了在 Debian Sid 中使用 Vim 结合 ctags、taglist、cscope、cppcomplete 和 global 的编程体验。重点介绍了这些工具如何帮助开发者提高效率，包括函数跳转、代码补全等功能。

我用下来的感觉是（在debian sid中）

(g)vim + ctags + taglist + cscope + cppcomplete + global

大型程序一定要global的参与，它的分析比较全面。

taglist 精干，需要ctags的支撑 ，直接可以在左边列出函数列表，全局参数列表。（可以排序）

cscope 比较强大，可以对函数以及部分类型定义进行跳转，但有些BUG，好像在某些条件下无法正确找到分析枚举的定义。

cppcomplete 需要ctags的支撑，可以补全类型或者函数名（可不是普通Ctrl+P/N的那种）

global 新版本可以嵌入vim使用，提供比较完整解析和类型索引，和cscope比，稍微差些的就是对类型引用的打印列表中没有标识这个引用在什么函数中进行的。

这些都在我平常的使用中，部分功能还在摸索中。
linux下阅读源代码的工具
xzjiang_365的博客
05-25 1222
目前的lxr是专门为Linux下面的Apache服务器设计的，通过运行perl脚本，检索源代码索引文件，将数据发送到网络客户端的Web浏览器上。，他为那些不熟悉vim或者是不习惯终端作业的人提供了图形界面的选择，界面是这一款软件的一个亮点，比 source insight还好看，从windows转过来做Linux开发的人可能感觉比较亲切。调用图的分析分析大致可分为“静态”和“动态”两种，所谓静态分析是指在不运行待分析的程序的前提下进行分析，那么动态分析自然就是记录程序实际运行时的函数调用情况了。
Linux 安装 MySQL8 ，MySQL版本 mysql-8.0.31-linux-glibc2.12-x86_64.tar.xz
SOFT2030的专栏
07-29 1007
[root@localhost tmp]# [root@localhost tmp]# wget https://dev.mysql.com/get/Downloads/MySQL-8.0/mysql-8.0.31-linux-glibc2.12-x86_64.tar.xz --2024-07-29 06:28:00-- https://dev.mysql.com/get/Downloads/MySQL-8.0/mysql-8.0.31-linux-glibc2.12-x86_64.tar.xz Res
Linux内核源码+查看工具
11-02
Linux内核源码+查看工具，Linux内核源码+查看工具Linux内核源码+查看工具
Linux下看代码的好工具Kscope
gbtux2008的专栏
01-29 1687
Kscope是Linux下的一款类似于SourceInsight工具，下面将其安装过程总结如下： 1.先下载kscopekscope最新安装版本kscope-1.6.2.tar.gzhttp://download.chinaunix.net/download/0006000/5469.shtml 按照说明#./configure            #make      
Linux下的代码阅读和查找工具
u010665051的专栏
08-04 1756
1.ctags        ctags命令配合vim编辑器可以帮助程序员很方便地查看源代码，使用方法：        （1）在源代码顶层目录下执行ctags -R * 递归建立源代码信息库；        （2）vim -t tagname 打开含有tagname的文件；        vim编辑器中        （3）:tag tagname：跳到含有tagname的文件中；  
2025-05-19 10:33:54 [JobThread.run-130] <br>----------- datax-web job execute start -----------<br>----------- Param: 2025-05-19 10:33:54 [BuildCommand.buildDataXParam-100] ------------------Command parameters:-p"-DlastTime='2025-05-19%08:00:00' -DcurrentTime='2025-05-19%10:33:54'" 2025-05-19 10:33:54 [ExecutorJobHandler.execute-57] ------------------DataX process id: 2669844 2025-05-19 10:33:54 [AnalysisStatistics.analysisStatisticsLog-53] 2025-05-19 10:33:54 [AnalysisStatistics.analysisStatisticsLog-53] DataX (DATAX-OPENSOURCE-3.0), From Alibaba ! 2025-05-19 10:33:54 [AnalysisStatistics.analysisStatisticsLog-53] Copyright (C) 2010-2017, Alibaba Group. All Rights Reserved. 2025-05-19 10:33:54 [AnalysisStatistics.analysisStatisticsLog-53] 2025-05-19 10:33:54 [AnalysisStatistics.analysisStatisticsLog-53] 2025-05-19 10:33:54 [ProcessCallbackThread.callbackLog-186] <br>----------- datax-web job callback finish. 2025-05-19 10:33:55 [AnalysisStatistics.analysisStatisticsLog-53] 2025-05-19 10:33:55.146 [main] INFO MessageSource - JVM TimeZone: GMT+08:00, Locale: zh_CN 2025-05-19 10:33:55 [AnalysisStatistics.analysisStatisticsLog-53] 2025-05-19 10:33:55.148 [main] INFO MessageSource - use Locale: zh_CN timeZone: sun.util.calendar.ZoneInfo[id="GMT+08:00",offset=28800000,dstSavings=0,useDaylight=false,transitions=0,lastRule=null] 2025-05-19 10:33:55 [AnalysisStatistics.analysisStatisticsLog-53] 2025-05-19 10:33:55.161 [main] INFO VMInfo - VMInfo# operatingSystem class => sun.management.OperatingSystemImpl 2025-05-19 10:33:55 [AnalysisStatistics.analysisStatisticsLog-53] 2025-05-19 10:33:55.167 [main] INFO Engine - the machine info => 2025-05-19 10:33:55 [AnalysisStatistics.analysisStatisticsLog-53] 2025-05-19 10:33:55 [AnalysisStatistics.analysisStatisticsLog-53] osInfo: Linux amd64 4.19.90-17.ky10.x86_64 2025-05-19 10:33:55 [AnalysisStatistics.analysisStatisticsLog-53] jvmInfo: Huawei Technologies Co., Ltd 1.8 25.242-b08 2025-05-19 10:33:55 [AnalysisStatistics.analysisStatisticsLog-53] cpu num: 32 2025-05-19 10:33:55 [AnalysisStatistics.analysisStatisticsLog-53] 2025-05-19 10:33:55 [AnalysisStatistics.analysisStatisticsLog-53] totalPhysicalMemory: -0.00G 2025-05-19 10:33:55 [AnalysisStatistics.analysisStatisticsLog-53] freePhysicalMemory: -0.00G 2025-05-19 10:33:55 [AnalysisStatistics.analysisStatisticsLog-53] maxFileDescriptorCount: -1 2025-05-19 10:33:55 [AnalysisStatistics.analysisStatisticsLog-53] currentOpenFileDescriptorCount: -1 2025-05-19 10:33:55 [AnalysisStatistics.analysisStatisticsLog-53] 2025-05-19 10:33:55 [AnalysisStatistics.analysisStatisticsLog-53] GC Names [PS MarkSweep, PS Scavenge] 2025-05-19 10:33:55 [AnalysisStatistics.analysisStatisticsLog-53] 2025-05-19 10:33:55 [AnalysisStatistics.analysisStatisticsLog-53] MEMORY_NAME | allocation_size | init_size 2025-05-19 10:33:55 [AnalysisStatistics.analysisStatisticsLog-53] PS Eden Space | 256.00MB | 256.00MB 2025-05-19 10:33:55 [AnalysisStatistics.analysisStatisticsLog-53] Code Cache | 240.00MB | 2.44MB 2025-05-19 10:33:55 [AnalysisStatistics.analysisStatisticsLog-53] Compressed Class Space | 1,024.00MB | 0.00MB 2025-05-19 10:33:55 [AnalysisStatistics.analysisStatisticsLog-53] PS Survivor Space | 42.50MB | 42.50MB 2025-05-19 10:33:55 [AnalysisStatistics.analysisStatisticsLog-53] PS Old Gen | 683.00MB | 683.00MB 2025-05-19 10:33:55 [AnalysisStatistics.analysisStatisticsLog-53] Metaspace | -0.00MB | 0.00MB 2025-05-19 10:33:55 [AnalysisStatistics.analysisStatisticsLog-53] 2025-05-19 10:33:55 [AnalysisStatistics.analysisStatisticsLog-53] 2025-05-19 10:33:55 [AnalysisStatistics.analysisStatisticsLog-53] 2025-05-19 10:33:55.181 [main] INFO Engine - 2025-05-19 10:33:55 [AnalysisStatistics.analysisStatisticsLog-53] { 2025-05-19 10:33:55 [AnalysisStatistics.analysisStatisticsLog-53] "content":[ 2025-05-19 10:33:55 [AnalysisStatistics.analysisStatisticsLog-53] { 2025-05-19 10:33:55 [AnalysisStatistics.analysisStatisticsLog-53] "reader":{ 2025-05-19 10:33:55 [AnalysisStatistics.analysisStatisticsLog-53] "parameter":{ 2025-05-19 10:33:55 [AnalysisStatistics.analysisStatisticsLog-53] "password":"**********", 2025-05-19 10:33:55 [AnalysisStatistics.analysisStatisticsLog-53] "connection":[ 2025-05-19 10:33:55 [AnalysisStatistics.analysisStatisticsLog-53] { 2025-05-19 10:33:55 [AnalysisStatistics.analysisStatisticsLog-53] "querySql":[ 2025-05-19 10:33:55 [AnalysisStatistics.analysisStatisticsLog-53] "select CONCAT(comm_code,id) as id, `uuid`, `record_time`, `open_mode`, `comm_code`, `open_object`, `name`, `phone_num`, `person_type`, `open_device`, `device_place`, `person_uuid`, `picture_url`, `open_video`, `created_by`, `created_time`, `updated_by`, `updated_time`, `del_flag`, `mczt`, `pass`, `in_out_flag`, `temp`, `temp_normal`, `extra`, `health_info`, `created_time` as GY_RKGXSJ, case when `del_flag`=2 then 1 else 0 end as GY_SFYSC from `door_open_record` where `record_time` >= '2025-05-19%08:00:00' and `record_time` < '2025-05-19%10:33:54'" 2025-05-19 10:33:55 [AnalysisStatistics.analysisStatisticsLog-53] ], 2025-05-19 10:33:55 [AnalysisStatistics.analysisStatisticsLog-53] "jdbcUrl":[ 2025-05-19 10:33:55 [AnalysisStatistics.analysisStatisticsLog-53] "jdbc:mysql://172.20.20.57:8066/wuye?serverTimezone=Asia/Shanghai&useLegacyDatetimeCode=false&useSSL=false&nullNamePatternMatchesAll=true&useUnicode=true&characterEncoding=UTF-8" 2025-05-19 10:33:55 [AnalysisStatistics.analysisStatisticsLog-53] ] 2025-05-19 10:33:55 [AnalysisStatistics.analysisStatisticsLog-53] } 2025-05-19 10:33:55 [AnalysisStatistics.analysisStatisticsLog-53] ], 2025-05-19 10:33:55 [AnalysisStatistics.analysisStatisticsLog-53] "splitPk":"", 2025-05-19 10:33:55 [AnalysisStatistics.analysisStatisticsLog-53] "username":"root" 2025-05-19 10:33:55 [AnalysisStatistics.analysisStatisticsLog-53] }, 2025-05-19 10:33:55 [AnalysisStatistics.analysisStatisticsLog-53] "name":"mysqlreader" 2025-05-19 10:33:55 [AnalysisStatistics.analysisStatisticsLog-53] }, 2025-05-19 10:33:55 [AnalysisStatistics.analysisStatisticsLog-53] "writer":{ 2025-05-19 10:33:55 [AnalysisStatistics.analysisStatisticsLog-53] "parameter":{ 2025-05-19 10:33:55 [AnalysisStatistics.analysisStatisticsLog-53] "password":"**********", 2025-05-19 10:33:55 [AnalysisStatistics.analysisStatisticsLog-53] "column":[ 2025-05-19 10:33:55 [AnalysisStatistics.analysisStatisticsLog-53] "`id`", 2025-05-19 10:33:55 [AnalysisStatistics.analysisStatisticsLog-53] "`uuid`", 2025-05-19 10:33:55 [AnalysisStatistics.analysisStatisticsLog-53] "`record_time`", 2025-05-19 10:33:55 [AnalysisStatistics.analysisStatisticsLog-53] "`open_mode`", 2025-05-19 10:33:55 [AnalysisStatistics.analysisStatisticsLog-53] "`comm_code`", 2025-05-19 10:33:55 [AnalysisStatistics.analysisStatisticsLog-53] "`open_object`", 2025-05-19 10:33:55 [AnalysisStatistics.analysisStatisticsLog-53] "`name`", 2025-05-19 10:33:55 [AnalysisStatistics.analysisStatisticsLog-53] "`phone_num`", 2025-05-19 10:33:55 [AnalysisStatistics.analysisStatisticsLog-53] "`person_type`", 2025-05-19 10:33:55 [AnalysisStatistics.analysisStatisticsLog-53] "`open_device`", 2025-05-19 10:33:55 [AnalysisStatistics.analysisStatisticsLog-53] "`device_place`", 2025-05-19 10:33:55 [AnalysisStatistics.analysisStatisticsLog-53] "`person_uuid`", 2025-05-19 10:33:55 [AnalysisStatistics.analysisStatisticsLog-53] "`picture_url`", 2025-05-19 10:33:55 [AnalysisStatistics.analysisStatisticsLog-53] "`open_video`", 2025-05-19 10:33:55 [AnalysisStatistics.analysisStatisticsLog-53] "`created_by`", 2025-05-19 10:33:55 [AnalysisStatistics.analysisStatisticsLog-53] "`created_time`", 2025-05-19 10:33:55 [AnalysisStatistics.analysisStatisticsLog-53] "`updated_by`", 2025-05-19 10:33:55 [AnalysisStatistics.analysisStatisticsLog-53] "`updated_time`", 2025-05-19 10:33:55 [AnalysisStatistics.analysisStatisticsLog-53] "`del_flag`", 2025-05-19 10:33:55 [AnalysisStatistics.analysisStatisticsLog-53] "`mczt`", 2025-05-19 10:33:55 [AnalysisStatistics.analysisStatisticsLog-53] "`pass`", 2025-05-19 10:33:55 [AnalysisStatistics.analysisStatisticsLog-53] "`in_out_flag`", 2025-05-19 10:33:55 [AnalysisStatistics.analysisStatisticsLog-53] "`temp`", 2025-05-19 10:33:55 [AnalysisStatistics.analysisStatisticsLog-53] "`temp_normal`", 2025-05-19 10:33:55 [AnalysisStatistics.analysisStatisticsLog-53] "`extra`", 2025-05-19 10:33:55 [AnalysisStatistics.analysisStatisticsLog-53] "`health_info`", 2025-05-19 10:33:55 [AnalysisStatistics.analysisStatisticsLog-53] "`GY_RKGXSJ`", 2025-05-19 10:33:55 [AnalysisStatistics.analysisStatisticsLog-53] "`GY_SFYSC`" 2025-05-19 10:33:55 [AnalysisStatistics.analysisStatisticsLog-53] ], 2025-05-19 10:33:55 [AnalysisStatistics.analysisStatisticsLog-53] "connection":[ 2025-05-19 10:33:55 [AnalysisStatistics.analysisStatisticsLog-53] { 2025-05-19 10:33:55 [AnalysisStatistics.analysisStatisticsLog-53] "jdbcUrl":"jdbc:mysql://172.20.20.67:3306/kitty?serverTimezone=Asia/Shanghai&useLegacyDatetimeCode=false&useSSL=false&nullNamePatternMatchesAll=true&useUnicode=true&characterEncoding=UTF-8", 2025-05-19 10:33:55 [AnalysisStatistics.analysisStatisticsLog-53] "table":[ 2025-05-19 10:33:55 [AnalysisStatistics.analysisStatisticsLog-53] "door_open_record" 2025-05-19 10:33:55 [AnalysisStatistics.analysisStatisticsLog-53] ] 2025-05-19 10:33:55 [AnalysisStatistics.analysisStatisticsLog-53] } 2025-05-19 10:33:55 [AnalysisStatistics.analysisStatisticsLog-53] ], 2025-05-19 10:33:55 [AnalysisStatistics.analysisStatisticsLog-53] "username":"root" 2025-05-19 10:33:55 [AnalysisStatistics.analysisStatisticsLog-53] }, 2025-05-19 10:33:55 [AnalysisStatistics.analysisStatisticsLog-53] "name":"mysqlwriter" 2025-05-19 10:33:55 [AnalysisStatistics.analysisStatisticsLog-53] } 2025-05-19 10:33:55 [AnalysisStatistics.analysisStatisticsLog-53] } 2025-05-19 10:33:55 [AnalysisStatistics.analysisStatisticsLog-53] ], 2025-05-19 10:33:55 [AnalysisStatistics.analysisStatisticsLog-53] "setting":{ 2025-05-19 10:33:55 [AnalysisStatistics.analysisStatisticsLog-53] "errorLimit":{ 2025-05-19 10:33:55 [AnalysisStatistics.analysisStatisticsLog-53] "record":100, 2025-05-19 10:33:55 [AnalysisStatistics.analysisStatisticsLog-53] "percentage":0.02 2025-05-19 10:33:55 [AnalysisStatistics.analysisStatisticsLog-53] }, 2025-05-19 10:33:55 [AnalysisStatistics.analysisStatisticsLog-53] "speed":{ 2025-05-19 10:33:55 [AnalysisStatistics.analysisStatisticsLog-53] "byte":1048576, 2025-05-19 10:33:55 [AnalysisStatistics.analysisStatisticsLog-53] "channel":3 2025-05-19 10:33:55 [AnalysisStatistics.analysisStatisticsLog-53] } 2025-05-19 10:33:55 [AnalysisStatistics.analysisStatisticsLog-53] } 2025-05-19 10:33:55 [AnalysisStatistics.analysisStatisticsLog-53] } 2025-05-19 10:33:55 [AnalysisStatistics.analysisStatisticsLog-53] 2025-05-19 10:33:55 [AnalysisStatistics.analysisStatisticsLog-53] 2025-05-19 10:33:55.200 [main] INFO PerfTrace - PerfTrace traceId=job_-1, isEnable=false 2025-05-19 10:33:55 [AnalysisStatistics.analysisStatisticsLog-53] 2025-05-19 10:33:55.200 [main] INFO JobContainer - DataX jobContainer starts job. 2025-05-19 10:33:55 [AnalysisStatistics.analysisStatisticsLog-53] 2025-05-19 10:33:55.201 [main] INFO JobContainer - Set jobId = 0 2025-05-19 10:33:55 [AnalysisStatistics.analysisStatisticsLog-53] 2025-05-19 10:33:55.533 [job-0] INFO OriginalConfPretreatmentUtil - Available jdbcUrl:jdbc:mysql://172.20.20.57:8066/wuye?serverTimezone=Asia/Shanghai&useLegacyDatetimeCode=false&useSSL=false&nullNamePatternMatchesAll=true&useUnicode=true&characterEncoding=UTF-8&yearIsDateType=false&zeroDateTimeBehavior=convertToNull&tinyInt1isBit=false&rewriteBatchedStatements=true. 2025-05-19 10:33:55 [AnalysisStatistics.analysisStatisticsLog-53] 2025-05-19 10:33:55.791 [job-0] INFO OriginalConfPretreatmentUtil - table:[door_open_record] all columns:[ 2025-05-19 10:33:55 [AnalysisStatistics.analysisStatisticsLog-53] id,uuid,record_time,open_mode,comm_code,open_object,name,phone_num,person_type,open_device,device_place,person_uuid,picture_url,open_video,created_by,created_time,updated_by,updated_time,del_flag,mczt,pass,in_out_flag,GY_RKGXSJ,GY_SFYSC,temp,temp_normal,extra,health_info 2025-05-19 10:33:55 [AnalysisStatistics.analysisStatisticsLog-53] ]. 2025-05-19 10:33:55 [AnalysisStatistics.analysisStatisticsLog-53] 2025-05-19 10:33:55.798 [job-0] INFO OriginalConfPretreatmentUtil - Write data [ 2025-05-19 10:33:55 [AnalysisStatistics.analysisStatisticsLog-53] INSERT INTO %s (`id`,`uuid`,`record_time`,`open_mode`,`comm_code`,`open_object`,`name`,`phone_num`,`person_type`,`open_device`,`device_place`,`person_uuid`,`picture_url`,`open_video`,`created_by`,`created_time`,`updated_by`,`updated_time`,`del_flag`,`mczt`,`pass`,`in_out_flag`,`temp`,`temp_normal`,`extra`,`health_info`,`GY_RKGXSJ`,`GY_SFYSC`) VALUES(?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?) 2025-05-19 10:33:55 [AnalysisStatistics.analysisStatisticsLog-53] ], which jdbcUrl like:[jdbc:mysql://172.20.20.67:3306/kitty?serverTimezone=Asia/Shanghai&useLegacyDatetimeCode=false&useSSL=false&nullNamePatternMatchesAll=true&useUnicode=true&characterEncoding=UTF-8&yearIsDateType=false&zeroDateTimeBehavior=convertToNull&rewriteBatchedStatements=true&tinyInt1isBit=false] 2025-05-19 10:33:55 [AnalysisStatistics.analysisStatisticsLog-53] 2025-05-19 10:33:55.798 [job-0] INFO JobContainer - jobContainer starts to do prepare ... 2025-05-19 10:33:55 [AnalysisStatistics.analysisStatisticsLog-53] 2025-05-19 10:33:55.799 [job-0] INFO JobContainer - DataX Reader.Job [mysqlreader] do prepare work . 2025-05-19 10:33:55 [AnalysisStatistics.analysisStatisticsLog-53] 2025-05-19 10:33:55.799 [job-0] INFO JobContainer - DataX Writer.Job [mysqlwriter] do prepare work . 2025-05-19 10:33:55 [AnalysisStatistics.analysisStatisticsLog-53] 2025-05-19 10:33:55.799 [job-0] INFO JobContainer - jobContainer starts to do split ... 2025-05-19 10:33:55 [AnalysisStatistics.analysisStatisticsLog-53] 2025-05-19 10:33:55.800 [job-0] INFO JobContainer - Job set Max-Byte-Speed to 1048576 bytes. 2025-05-19 10:33:55 [AnalysisStatistics.analysisStatisticsLog-53] 2025-05-19 10:33:55.801 [job-0] INFO JobContainer - DataX Reader.Job [mysqlreader] splits to [1] tasks. 2025-05-19 10:33:55 [AnalysisStatistics.analysisStatisticsLog-53] 2025-05-19 10:33:55.802 [job-0] INFO JobContainer - DataX Writer.Job [mysqlwriter] splits to [1] tasks. 2025-05-19 10:33:55 [AnalysisStatistics.analysisStatisticsLog-53] 2025-05-19 10:33:55.827 [job-0] INFO JobContainer - jobContainer starts to do schedule ... 2025-05-19 10:33:55 [AnalysisStatistics.analysisStatisticsLog-53] 2025-05-19 10:33:55.830 [job-0] INFO JobContainer - Scheduler starts [1] taskGroups. 2025-05-19 10:33:55 [AnalysisStatistics.analysisStatisticsLog-53] 2025-05-19 10:33:55.832 [job-0] INFO JobContainer - Running by standalone Mode. 2025-05-19 10:33:55 [AnalysisStatistics.analysisStatisticsLog-53] 2025-05-19 10:33:55.836 [taskGroup-0] INFO TaskGroupContainer - taskGroupId=[0] start [1] channels for [1] tasks. 2025-05-19 10:33:55 [AnalysisStatistics.analysisStatisticsLog-53] 2025-05-19 10:33:55.840 [taskGroup-0] INFO Channel - Channel set byte_speed_limit to 2000000. 2025-05-19 10:33:55 [AnalysisStatistics.analysisStatisticsLog-53] 2025-05-19 10:33:55.840 [taskGroup-0] INFO Channel - Channel set record_speed_limit to -1, No tps activated. 2025-05-19 10:33:55 [AnalysisStatistics.analysisStatisticsLog-53] 2025-05-19 10:33:55.847 [taskGroup-0] INFO TaskGroupContainer - taskGroup[0] taskId[0] attemptCount[1] is started 2025-05-19 10:33:55 [AnalysisStatistics.analysisStatisticsLog-53] 2025-05-19 10:33:55.850 [0-0-0-reader] INFO CommonRdbmsReader$Task - Begin to read record by Sql: [select CONCAT(comm_code,id) as id, `uuid`, `record_time`, `open_mode`, `comm_code`, `open_object`, `name`, `phone_num`, `person_type`, `open_device`, `device_place`, `person_uuid`, `picture_url`, `open_video`, `created_by`, `created_time`, `updated_by`, `updated_time`, `del_flag`, `mczt`, `pass`, `in_out_flag`, `temp`, `temp_normal`, `extra`, `health_info`, `created_time` as GY_RKGXSJ, case when `del_flag`=2 then 1 else 0 end as GY_SFYSC from `door_open_record` where `record_time` >= '2025-05-19%08:00:00' and `record_time` < '2025-05-19%10:33:54' 2025-05-19 10:33:55 [AnalysisStatistics.analysisStatisticsLog-53] ] jdbcUrl:[jdbc:mysql://172.20.20.57:8066/wuye?serverTimezone=Asia/Shanghai&useLegacyDatetimeCode=false&useSSL=false&nullNamePatternMatchesAll=true&useUnicode=true&characterEncoding=UTF-8&yearIsDateType=false&zeroDateTimeBehavior=convertToNull&tinyInt1isBit=false&rewriteBatchedStatements=true]. 2025-05-19 10:33:56 [AnalysisStatistics.analysisStatisticsLog-53] 2025-05-19 10:33:56.927 [0-0-0-reader] INFO CommonRdbmsReader$Task - Finished read record by Sql: [select CONCAT(comm_code,id) as id, `uuid`, `record_time`, `open_mode`, `comm_code`, `open_object`, `name`, `phone_num`, `person_type`, `open_device`, `device_place`, `person_uuid`, `picture_url`, `open_video`, `created_by`, `created_time`, `updated_by`, `updated_time`, `del_flag`, `mczt`, `pass`, `in_out_flag`, `temp`, `temp_normal`, `extra`, `health_info`, `created_time` as GY_RKGXSJ, case when `del_flag`=2 then 1 else 0 end as GY_SFYSC from `door_open_record` where `record_time` >= '2025-05-19%08:00:00' and `record_time` < '2025-05-19%10:33:54' 2025-05-19 10:33:56 [AnalysisStatistics.analysisStatisticsLog-53] ] jdbcUrl:[jdbc:mysql://172.20.20.57:8066/wuye?serverTimezone=Asia/Shanghai&useLegacyDatetimeCode=false&useSSL=false&nullNamePatternMatchesAll=true&useUnicode=true&characterEncoding=UTF-8&yearIsDateType=false&zeroDateTimeBehavior=convertToNull&tinyInt1isBit=false&rewriteBatchedStatements=true]. 2025-05-19 10:33:57 [AnalysisStatistics.analysisStatisticsLog-53] 2025-05-19 10:33:57.242 [0-0-0-writer] WARN CommonRdbmsWriter$Task - 回滚此次写入, 采用每次写入一行方式提交. 因为:Data truncation: Out of range value for column 'id' at row 1 2025-05-19 10:33:57 [AnalysisStatistics.analysisStatisticsLog-53] 2025-05-19 10:33:57.247 [0-0-0-writer] ERROR StdoutPluginCollector - 2025-05-19 10:33:57 [AnalysisStatistics.analysisStatisticsLog-53] com.mysql.jdbc.MysqlDataTruncation: Data truncation: Out of range value for column 'id' at row 1 2025-05-19 10:33:57 [AnalysisStatistics.analysisStatisticsLog-53] at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:3976) ~[mysql-connector-java-5.1.47.jar:5.1.47] 2025-05-19 10:33:57 [AnalysisStatistics.analysisStatisticsLog-53] at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:3914) ~[mysql-connector-java-5.1.47.jar:5.1.47] 2025-05-19 10:33:57 [AnalysisStatistics.analysisStatisticsLog-53] at com.mysql.jdbc.MysqlIO.sendCommand(MysqlIO.java:2530) ~[mysql-connector-java-5.1.47.jar:5.1.47] 2025-05-19 10:33:57 [AnalysisStatistics.analysisStatisticsLog-53] at com.mysql.jdbc.MysqlIO.sqlQueryDirect(MysqlIO.java:2683) ~[mysql-connector-java-5.1.47.jar:5.1.47] 2025-05-19 10:33:57 [AnalysisStatistics.analysisStatisticsLog-53] at com.mysql.jdbc.ConnectionImpl.execSQL(ConnectionImpl.java:2495) ~[mysql-connector-java-5.1.47.jar:5.1.47] 2025-05-19 10:33:57 [AnalysisStatistics.analysisStatisticsLog-53] at com.mysql.jdbc.PreparedStatement.executeInternal(PreparedStatement.java:1903) ~[mysql-connector-java-5.1.47.jar:5.1.47] 2025-05-19 10:33:57 [AnalysisStatistics.analysisStatisticsLog-53] at com.mysql.jdbc.PreparedStatement.execute(PreparedStatement.java:1242) ~[mysql-connector-java-5.1.47.jar:5.1.47] 2025-05-19 10:33:57 [AnalysisStatistics.analysisStatisticsLog-53] at com.alibaba.datax.plugin.rdbms.writer.CommonRdbmsWriter$Task.doOneInsert(CommonRdbmsWriter.java:382) [plugin-rdbms-util-0.0.1-SNAPSHOT.jar:na] 2025-05-19 10:33:57 [AnalysisStatistics.analysisStatisticsLog-53] at com.alibaba.datax.plugin.rdbms.writer.CommonRdbmsWriter$Task.doBatchInsert(CommonRdbmsWriter.java:362) [plugin-rdbms-util-0.0.1-SNAPSHOT.jar:na] 2025-05-19 10:33:57 [AnalysisStatistics.analysisStatisticsLog-53] at com.alibaba.datax.plugin.rdbms.writer.CommonRdbmsWriter$Task.startWriteWithConnection(CommonRdbmsWriter.java:297) [plugin-rdbms-util-0.0.1-SNAPSHOT.jar:na] 2025-05-19 10:33:57 [AnalysisStatistics.analysisStatisticsLog-53] at com.alibaba.datax.plugin.rdbms.writer.CommonRdbmsWriter$Task.startWrite(CommonRdbmsWriter.java:319) [plugin-rdbms-util-0.0.1-SNAPSHOT.jar:na] 2025-05-19 10:33:57 [AnalysisStatistics.analysisStatisticsLog-53] at com.alibaba.datax.plugin.writer.mysqlwriter.MysqlWriter$Task.startWrite(MysqlWriter.java:78) [mysqlwriter-0.0.1-SNAPSHOT.jar:na] 2025-05-19 10:33:57 [AnalysisStatistics.analysisStatisticsLog-53] at com.alibaba.datax.core.taskgroup.runner.WriterRunner.run(WriterRunner.java:56) [datax-core-0.0.1-SNAPSHOT.jar:na] 2025-05-19 10:33:57 [AnalysisStatistics.analysisStatisticsLog-53] at java.lang.Thread.run(Thread.java:748) [na:1.8.0_242] 2025-05-19 10:33:57 [AnalysisStatistics.analysisStatisticsLog-53] 2025-05-19 10:33:57.259 [0-0-0-writer] ERROR StdoutPluginCollector - 脏数据: 2025-05-19 10:33:57 [AnalysisStatistics.analysisStatisticsLog-53] {"exception":"Data truncation: Out of range value for column 'id' at row 1","record":[{"byteSize":22,"index":0,"rawData":"0591202111180387851990","type":5},{"byteSize":32,"index":1,"rawData":"4d0986595d40485aa9839e0cccec4044","type":5},{"byteSize":8,"index":2,"rawData":1747612856000,"type":7},{"byteSize":4,"index":3,"rawData":1004,"type":3},{"byteSize":16,"index":4,"rawData":"0591202111180387","type":5},{"byteSize":29,"index":5,"rawData":"05912021111803871658544258319","type":5},{"byteSize":2,"index":6,"rawData":"周云","type":5},{"byteSize":0,"index":7,"rawData":"","type":5},{"byteSize":1,"index":8,"rawData":1,"type":3},{"byteSize":28,"index":9,"rawData":"NYN0300010000020000S1F001436","type":5},{"byteSize":5,"index":10,"rawData":"西门M29","type":5},{"byteSize":32,"index":11,"rawData":"c68130c53b1441cc9e9205a0df155888","type":5},{"byteSize":83,"index":12,"rawData":"https://iot.yinxismart.top:8888/group1/M00/02/0C/rBQUOGgqdLiARP_wAADchi_tzQg504.jpg","type":5},{"byteSize":0,"index":13,"type":5},{"byteSize":0,"index":14,"type":5},{"byteSize":0,"index":15,"type":7},{"byteSize":0,"index":16,"type":5},{"byteSize":0,"index":17,"type":7},{"byteSize":1,"index":18,"rawData":1,"type":3},{"byteSize":1,"index":19,"rawData":1,"type":3},{"byteSize":1,"index":20,"rawData":1,"type":3},{"byteSize":1,"index":21,"rawData":0,"type":3},{"byteSize":0,"index":22,"type":5},{"byteSize":1,"index":23,"rawData":0,"type":3},{"byteSize":0,"index":24,"type":5},{"byteSize":0,"index":25,"type":5},{"byteSize":0,"index":26,"type":7},{"byteSize":1,"index":27,"rawData":0,"type":3}],"type":"writer"} 2025-05-19 10:33:57 [AnalysisStatistics.analysisStatisticsLog-53] 2025-05-19 10:33:57.260 [0-0-0-writer] ERROR StdoutPluginCollector - 脏数据: 2025-05-19 10:33:57 [AnalysisStatistics.analysisStatisticsLog-53] {"exception":"Data truncation: Out of range value for column 'id' at row 1","record":[{"byteSize":22,"index":0,"rawData":"0591202111180387851991","type":5},{"byteSize":32,"index":1,"rawData":"d52cf199b8b54901bad84a21f3fa25b1","type":5},{"byteSize":8,"index":2,"rawData":1747612880000,"type":7},{"byteSize":4,"index":3,"rawData":1004,"type":3},{"byteSize":16,"index":4,"rawData":"0591202111180387","type":5},{"byteSize":29,"index":5,"rawData":"05912021111803871742111902520","type":5},{"byteSize":3,"index":6,"rawData":"庄华琴","type":5},{"byteSize":0,"index":7,"rawData":"","type":5},{"byteSize":1,"index":8,"rawData":2,"type":3},{"byteSize":16,"index":9,"rawData":"0020220987YXJDSQ","type":5},{"byteSize":10,"index":10,"rawData":"中联天玺南出入口门禁","type":5},{"byteSize":32,"index":11,"rawData":"729da5882c6a41d9bfda0949931a1be1","type":5},{"byteSize":83,"index":12,"rawData":"https://iot.yinxismart.top:8888/group1/M00/02/0C/rBQUN2gqdNeASmFSAAJOMjYE8R8862.jpg","type":5},{"byteSize":0,"index":13,"type":5},{"byteSize":0,"index":14,"type":5},{"byteSize":0,"index":15,"type":7},{"byteSize":0,"index":16,"type":5},{"byteSize":0,"index":17,"type":7},{"byteSize":1,"index":18,"rawData":1,"type":3},{"byteSize":1,"index":19,"rawData":1,"type":3},{"byteSize":1,"index":20,"rawData":1,"type":3},{"byteSize":1,"index":21,"rawData":0,"type":3},{"byteSize":0,"index":22,"type":5},{"byteSize":1,"index":23,"rawData":0,"type":3},{"byteSize":0,"index":24,"type":5},{"byteSize":0,"index":25,"type":5},{"byteSize":0,"index":26,"type":7},{"byteSize":1,"index":27,"rawData":0,"type":3}],"type":"writer"} 2025-05-19 10:33:57 [AnalysisStatistics.analysisStatisticsLog-53] 2025-05-19 10:33:57.262 [0-0-0-writer] ERROR StdoutPluginCollector - 脏数据: 2025-05-19 10:33:57 [AnalysisStatistics.analysisStatisticsLog-53] {"exception":"Data truncation: Out of range value for column 'id' at row 1","record":[{"byteSize":22,"index":0,"rawData":"0591202111180387851992","type":5},{"byteSize":32,"index":1,"rawData":"4bd01f7d35f4437da3e276c6c31a5c5c","type":5},{"byteSize":8,"index":2,"rawData":1747612889000,"type":7},{"byteSize":4,"index":3,"rawData":1004,"type":3},{"byteSize":16,"index":4,"rawData":"0591202111180387","type":5},{"byteSize":29,"index":5,"rawData":"05912021111803871741681870967","type":5},{"byteSize":2,"index":6,"rawData":"严巧","type":5},{"byteSize":0,"index":7,"rawData":"","type":5},{"byteSize":1,"index":8,"rawData":1,"type":3},{"byteSize":28,"index":9,"rawData":"NYN0300010000020000S1F001436","type":5},{"byteSize":5,"index":10,"rawData":"西门M29","type":5},{"byteSize":32,"index":11,"rawData":"7f932dde89b64205a0d17d58871f9bad","type":5},{"byteSize":83,"index":12,"rawData":"https://iot.yinxismart.top:8888/group1/M00/02/0B/rBQUNmgqdNmAQ_S7AAC22NFh6HM442.jpg","type":5},{"byteSize":0,"index":13,"type":5},{"byteSize":0,"index":14,"type":5},{"byteSize":0,"index":15,"type":7},{"byteSize":0,"index":16,"type":5},{"byteSize":0,"index":17,"type":7},{"byteSize":1,"index":18,"rawData":1,"type":3},{"byteSize":1,"index":19,"rawData":1,"type":3},{"byteSize":1,"index":20,"rawData":1,"type":3},{"byteSize":1,"index":21,"rawData":0,"type":3},{"byteSize":0,"index":22,"type":5},{"byteSize":1,"index":23,"rawData":0,"type":3},{"byteSize":0,"index":24,"type":5},{"byteSize":0,"index":25,"type":5},{"byteSize":0,"index":26,"type":7},{"byteSize":1,"index":27,"rawData":0,"type":3}],"type":"writer"} 2025-05-19 10:33:57 [AnalysisStatistics.analysisStatisticsLog-53] 2025-05-19 10:33:57.263 [0-0-0-writer] ERROR StdoutPluginCollector - 脏数据: 2025-05-19 10:33:57 [AnalysisStatistics.analysisStatisticsLog-53] {"exception":"Data truncation: Out of range value for column 'id' at row 1","record":[{"byteSize":22,"index":0,"rawData":"0591202111180387851993","type":5},{"byteSize":32,"index":1,"rawData":"a766ed3cea29444181be6cedd1f816f5","type":5},{"byteSize":8,"index":2,"rawData":1747612931000,"type":7},{"byteSize":4,"index":3,"rawData":1004,"type":3},{"byteSize":16,"index":4,"rawData":"0591202111180387","type":5},{"byteSize":29,"index":5,"rawData":"05912021111803871659255470895","type":5},{"byteSize":3,"index":6,"rawData":"华明娟","type":5},{"byteSize":0,"index":7,"rawData":"","type":5},{"byteSize":1,"index":8,"rawData":1,"type":3},{"byteSize":28,"index":9,"rawData":"NYN0300010000020000S1F001436","type":5},{"byteSize":5,"index":10,"rawData":"西门M29","type":5},{"byteSize":32,"index":11,"rawData":"02fb5d620ac3460f8957de4b5b44336a","type":5},{"byteSize":83,"index":12,"rawData":"https://iot.yinxismart.top:8888/group1/M00/02/0D/rBQUNWgqdQOAeyFkAACw3Hvmovc046.jpg","type":5},{"byteSize":0,"index":13,"type":5},{"byteSize":0,"index":14,"type":5},{"byteSize":0,"index":15,"type":7},{"byteSize":0,"index":16,"type":5},{"byteSize":0,"index":17,"type":7},{"byteSize":1,"index":18,"rawData":1,"type":3},{"byteSize":1,"index":19,"rawData":1,"type":3},{"byteSize":1,"index":20,"rawData":1,"type":3},{"byteSize":1,"index":21,"rawData":0,"type":3},{"byteSize":0,"index":22,"type":5},{"byteSize":1,"index":23,"rawData":0,"type":3},{"byteSize":0,"index":24,"type":5},{"byteSize":0,"index":25,"type":5},{"byteSize":0,"index":26,"type":7},{"byteSize":1,"index":27,"rawData":0,"type":3}],"type":"writer"} 2025-05-19 10:33:57 [AnalysisStatistics.analysisStatisticsLog-53] 2025-05-19 10:33:57.264 [0-0-0-writer] ERROR StdoutPluginCollector - 脏数据: 2025-05-19 10:33:57 [AnalysisStatistics.analysisStatisticsLog-53] {"exception":"Data truncation: Out of range value for column 'id' at row 1","record":[{"byteSize":22,"index":0,"rawData":"0591202111180387851994","type":5},{"byteSize":32,"index":1,"rawData":"9a923175d25b41acbf8515f0d2b1d2ff","type":5},{"byteSize":8,"index":2,"rawData":1747613034000,"type":7},{"byteSize":4,"index":3,"rawData":1004,"type":3},{"byteSize":16,"index":4,"rawData":"0591202111180387","type":5},{"byteSize":29,"index":5,"rawData":"05912021111803871745223203681","type":5},{"byteSize":3,"index":6,"rawData":"林清唐","type":5},{"byteSize":0,"index":7,"rawData":"","type":5},{"byteSize":1,"index":8,"rawData":0,"type":3},{"byteSize":28,"index":9,"rawData":"NYN0300010000020000S1F001436","type":5},{"byteSize":5,"index":10,"rawData":"西门M29","type":5},{"byteSize":32,"index":11,"rawData":"d4d1b42ec6d540b9b459029dec96334a","type":5},{"byteSize":83,"index":12,"rawData":"https://iot.yinxismart.top:8888/group1/M00/02/0D/rBQUNWgqdWqAEVqIAAC_n9Jyueo433.jpg","type":5},{"byteSize":0,"index":13,"type":5},{"byteSize":0,"index":14,"type":5},{"byteSize":0,"index":15,"type":7},{"byteSize":0,"index":16,"type":5},{"byteSize":0,"index":17,"type":7},{"byteSize":1,"index":18,"rawData":1,"type":3},{"byteSize":1,"index":19,"rawData":1,"type":3},{"byteSize":1,"index":20,"rawData":1,"type":3},{"byteSize":1,"index":21,"rawData":0,"type":3},{"byteSize":0,"index":22,"type":5},{"byteSize":1,"index":23,"rawData":0,"type":3},{"byteSize":0,"index":24,"type":5},{"byteSize":0,"index":25,"type":5},{"byteSize":0,"index":26,"type":7},{"byteSize":1,"index":27,"rawData":0,"type":3}],"type":"writer"} 2025-05-19 10:33:57 [AnalysisStatistics.analysisStatisticsLog-53] 2025-05-19 10:33:57.265 [0-0-0-writer] ERROR StdoutPluginCollector - 脏数据: 2025-05-19 10:33:57 [AnalysisStatistics.analysisStatisticsLog-53] {"exception":"Data truncation: Out of range value for column 'id' at row 1","record":[{"byteSize":22,"index":0,"rawData":"0591202111180387851995","type":5},{"byteSize":32,"index":1,"rawData":"847f99174d4d4d26aaae669dd638e123","type":5},{"byteSize":8,"index":2,"rawData":1747613052000,"type":7},{"byteSize":4,"index":3,"rawData":1004,"type":3},{"byteSize":16,"index":4,"rawData":"0591202111180387","type":5},{"byteSize":29,"index":5,"rawData":"05912021111803871710205089306","type":5},{"byteSize":3,"index":6,"rawData":"陈惠钦","type":5},{"byteSize":0,"index":7,"rawData":"","type":5},{"byteSize":1,"index":8,"rawData":1,"type":3},{"byteSize":16,"index":9,"rawData":"0020220996YXJDSQ","type":5},{"byteSize":7,"index":10,"rawData":"北门消防门入口","type":5},{"byteSize":32,"index":11,"rawData":"2e6fdb84fdd44a94939c019a8e0d2b85","type":5},{"byteSize":83,"index":12,"rawData":"https://iot.yinxismart.top:8888/group1/M00/02/0C/rBQUOGgqdYWAEJkoAAHEBO1Px4w499.jpg","type":5},{"byteSize":0,"index":13,"type":5},{"byteSize":0,"index":14,"type":5},{"byteSize":0,"index":15,"type":7},{"byteSize":0,"index":16,"type":5},{"byteSize":0,"index":17,"type":7},{"byteSize":1,"index":18,"rawData":1,"type":3},{"byteSize":1,"index":19,"rawData":1,"type":3},{"byteSize":1,"index":20,"rawData":1,"type":3},{"byteSize":1,"index":21,"rawData":0,"type":3},{"byteSize":0,"index":22,"type":5},{"byteSize":1,"index":23,"rawData":0,"type":3},{"byteSize":0,"index":24,"type":5},{"byteSize":0,"index":25,"type":5},{"byteSize":0,"index":26,"type":7},{"byteSize":1,"index":27,"rawData":0,"type":3}],"type":"writer"} 2025-05-19 10:33:57 [AnalysisStatistics.analysisStatisticsLog-53] 2025-05-19 10:33:57.266 [0-0-0-writer] ERROR StdoutPluginCollector - 脏数据: 2025-05-19 10:33:57 [AnalysisStatistics.analysisStatisticsLog-53] {"exception":"Data truncation: Out of range value for column 'id' at row 1","record":[{"byteSize":22,"index":0,"rawData":"0591202111180387851996","type":5},{"byteSize":32,"index":1,"rawData":"6befb459b2da4ce1a3f9e7e9f3b9dcb4","type":5},{"byteSize":8,"index":2,"rawData":1747613064000,"type":7},{"byteSize":4,"index":3,"rawData":1004,"type":3},{"byteSize":16,"index":4,"rawData":"0591202111180387","type":5},{"byteSize":29,"index":5,"rawData":"05912021111803871741253388198","type":5},{"byteSize":3,"index":6,"rawData":"王枝发","type":5},{"byteSize":0,"index":7,"rawData":"","type":5},{"byteSize":1,"index":8,"rawData":1,"type":3},{"byteSize":28,"index":9,"rawData":"NYN0300010000020000S1F001436","type":5},{"byteSize":5,"index":10,"rawData":"西门M29","type":5},{"byteSize":32,"index":11,"rawData":"fe8e4331e4514d57bec126d6c767cc6c","type":5},{"byteSize":83,"index":12,"rawData":"https://iot.yinxismart.top:8888/group1/M00/02/0D/rBQUNWgqdYiAQR8YAAC_5wURlHI916.jpg","type":5},{"byteSize":0,"index":13,"type":5},{"byteSize":0,"index":14,"type":5},{"byteSize":0,"index":15,"type":7},{"byteSize":0,"index":16,"type":5},{"byteSize":0,"index":17,"type":7},{"byteSize":1,"index":18,"rawData":1,"type":3},{"byteSize":1,"index":19,"rawData":1,"type":3},{"byteSize":1,"index":20,"rawData":1,"type":3},{"byteSize":1,"index":21,"rawData":0,"type":3},{"byteSize":0,"index":22,"type":5},{"byteSize":1,"index":23,"rawData":0,"type":3},{"byteSize":0,"index":24,"type":5},{"byteSize":0,"index":25,"type":5},{"byteSize":0,"index":26,"type":7},{"byteSize":1,"index":27,"rawData":0,"type":3}],"type":"writer"} 2025-05-19 10:33:57 [AnalysisStatistics.analysisStatisticsLog-53] 2025-05-19 10:33:57.268 [0-0-0-writer] ERROR StdoutPluginCollector - 脏数据: 2025-05-19 10:33:57 [AnalysisStatistics.analysisStatisticsLog-53] {"exception":"Data truncation: Out of range value for column 'id' at row 1","record":[{"byteSize":22,"index":0,"rawData":"0591202111180387851997","type":5},{"byteSize":32,"index":1,"rawData":"bd4ff446847c483ea12dbc1a746ef9e0","type":5},{"byteSize":8,"index":2,"rawData":1747613074000,"type":7},{"byteSize":4,"index":3,"rawData":1004,"type":3},{"byteSize":16,"index":4,"rawData":"0591202111180387","type":5},{"byteSize":29,"index":5,"rawData":"05912021111803871666857417929","type":5},{"byteSize":3,"index":6,"rawData":"刘必梅","type":5},{"byteSize":0,"index":7,"rawData":"","type":5},{"byteSize":1,"index":8,"rawData":1,"type":3},{"byteSize":28,"index":9,"rawData":"NYN0300010000020000S1F001436","type":5},{"byteSize":5,"index":10,"rawData":"西门M29","type":5},{"byteSize":32,"index":11,"rawData":"09af342659144542a694fc75c8cd1c6f","type":5},{"byteSize":83,"index":12,"rawData":"https://iot.yinxismart.top:8888/group1/M00/02/0C/rBQUN2gqdZKAMtxUAADEYp0rtqk024.jpg","type":5},{"byteSize":0,"index":13,"type":5},{"byteSize":0,"index":14,"type":5},{"byteSize":0,"index":15,"type":7},{"byteSize":0,"index":16,"type":5},{"byteSize":0,"index":17,"type":7},{"byteSize":1,"index":18,"rawData":1,"type":3},{"byteSize":1,"index":19,"rawData":1,"type":3},{"byteSize":1,"index":20,"rawData":1,"type":3},{"byteSize":1,"index":21,"rawData":0,"type":3},{"byteSize":0,"index":22,"type":5},{"byteSize":1,"index":23,"rawData":0,"type":3},{"byteSize":0,"index":24,"type":5},{"byteSize":0,"index":25,"type":5},{"byteSize":0,"index":26,"type":7},{"byteSize":1,"index":27,"rawData":0,"type":3}],"type":"writer"} 2025-05-19 10:33:57 [AnalysisStatistics.analysisStatisticsLog-53] 2025-05-19 10:33:57.269 [0-0-0-writer] ERROR StdoutPluginCollector - 脏数据: 2025-05-19 10:33:57 [AnalysisStatistics.analysisStatisticsLog-53] {"exception":"Data truncation: Out of range value for column 'id' at row 1","record":[{"byteSize":22,"index":0,"rawData":"0591202111180387851998","type":5},{"byteSize":32,"index":1,"rawData":"f238007b8c634bb394c0425e9ffc2716","type":5},{"byteSize":8,"index":2,"rawData":1747613150000,"type":7},{"byteSize":4,"index":3,"rawData":1004,"type":3},{"byteSize":16,"index":4,"rawData":"0591202111180387","type":5},{"byteSize":29,"index":5,"rawData":"05912021111803871675924700007","type":5},{"byteSize":3,"index":6,"rawData":"俞玉琴","type":5},{"byteSize":0,"index":7,"rawData":"","type":5},{"byteSize":1,"index":8,"rawData":1,"type":3},{"byteSize":28,"index":9,"rawData":"NYN0300010000020000S1F001436","type":5},{"byteSize":5,"index":10,"rawData":"西门M29","type":5},{"byteSize":32,"index":11,"rawData":"198e3637945e4376a32310eae176b3ef","type":5},{"byteSize":83,"index":12,"rawData":"https://iot.yinxismart.top:8888/group1/M00/02/0C/rBQUN2gqdd6AAFFhAADRKbsR6uM999.jpg","type":5},{"byteSize":0,"index":13,"type":5},{"byteSize":0,"index":14,"type":5},{"byteSize":0,"index":15,"type":7},{"byteSize":0,"index":16,"type":5},{"byteSize":0,"index":17,"type":7},{"byteSize":1,"index":18,"rawData":1,"type":3},{"byteSize":1,"index":19,"rawData":1,"type":3},{"byteSize":1,"index":20,"rawData":1,"type":3},{"byteSize":1,"index":21,"rawData":0,"type":3},{"byteSize":0,"index":22,"type":5},{"byteSize":1,"index":23,"rawData":0,"type":3},{"byteSize":0,"index":24,"type":5},{"byteSize":0,"index":25,"type":5},{"byteSize":0,"index":26,"type":7},{"byteSize":1,"index":27,"rawData":0,"type":3}],"type":"writer"} 2025-05-19 10:33:57 [AnalysisStatistics.analysisStatisticsLog-53] 2025-05-19 10:33:57.475 [taskGroup-0] INFO TaskGroupContainer - taskGroup[0] taskId[0] is successed, used[1629]ms 2025-05-19 10:33:57 [AnalysisStatistics.analysisStatisticsLog-53] 2025-05-19 10:33:57.476 [taskGroup-0] INFO TaskGroupContainer - taskGroup[0] completed it's tasks. 2025-05-19 10:34:05 [AnalysisStatistics.analysisStatisticsLog-53] 2025-05-19 10:34:05.848 [job-0] INFO StandAloneJobContainerCommunicator - Total 361 records, 98167 bytes | Speed 9.59KB/s, 36 records/s | Error 361 records, 98167 bytes | All Task WaitWriterTime 0.001s | All Task WaitReaderTime 1.066s | Percentage 100.00% 2025-05-19 10:34:05 [AnalysisStatistics.analysisStatisticsLog-53] 2025-05-19 10:34:05.849 [job-0] ERROR JobContainer - 运行scheduler 模式[standalone]出错. 2025-05-19 10:34:05 [AnalysisStatistics.analysisStatisticsLog-53] 2025-05-19 10:34:05.849 [job-0] ERROR JobContainer - Exception when job run 2025-05-19 10:34:05 [AnalysisStatistics.analysisStatisticsLog-53] com.alibaba.datax.common.exception.DataXException: Code:[Framework-14], Description:[DataX传输脏数据超过用户预期，该错误通常是由于源端数据存在较多业务脏数据导致，请仔细检查DataX汇报的脏数据日志信息, 或者您可以适当调大脏数据阈值 .]. - 脏数据条数检查不通过，限制是[100]条，但实际上捕获了[361]条. 2025-05-19 10:34:05 [AnalysisStatistics.analysisStatisticsLog-53] at com.alibaba.datax.common.exception.DataXException.asDataXException(DataXException.java:30) ~[datax-common-0.0.1-SNAPSHOT.jar:na] 2025-05-19 10:34:05 [AnalysisStatistics.analysisStatisticsLog-53] at com.alibaba.datax.core.util.ErrorRecordChecker.checkRecordLimit(ErrorRecordChecker.java:58) ~[datax-core-0.0.1-SNAPSHOT.jar:na] 2025-05-19 10:34:05 [AnalysisStatistics.analysisStatisticsLog-53] at com.alibaba.datax.core.job.scheduler.AbstractScheduler.schedule(AbstractScheduler.java:89) ~[datax-core-0.0.1-SNAPSHOT.jar:na] 2025-05-19 10:34:05 [AnalysisStatistics.analysisStatisticsLog-53] at com.alibaba.datax.core.job.JobContainer.schedule(JobContainer.java:535) ~[datax-core-0.0.1-SNAPSHOT.jar:na] 2025-05-19 10:34:05 [AnalysisStatistics.analysisStatisticsLog-53] at com.alibaba.datax.core.job.JobContainer.start(JobContainer.java:119) ~[datax-core-0.0.1-SNAPSHOT.jar:na] 2025-05-19 10:34:05 [AnalysisStatistics.analysisStatisticsLog-53] at com.alibaba.datax.core.Engine.start(Engine.java:86) [datax-core-0.0.1-SNAPSHOT.jar:na] 2025-05-19 10:34:05 [AnalysisStatistics.analysisStatisticsLog-53] at com.alibaba.datax.core.Engine.entry(Engine.java:168) [datax-core-0.0.1-SNAPSHOT.jar:na] 2025-05-19 10:34:05 [AnalysisStatistics.analysisStatisticsLog-53] at com.alibaba.datax.core.Engine.main(Engine.java:201) [datax-core-0.0.1-SNAPSHOT.jar:na] 2025-05-19 10:34:05 [AnalysisStatistics.analysisStatisticsLog-53] 2025-05-19 10:34:05.850 [job-0] INFO StandAloneJobContainerCommunicator - Total 361 records, 98167 bytes | Speed 95.87KB/s, 361 records/s | Error 361 records, 98167 bytes | All Task WaitWriterTime 0.001s | All Task WaitReaderTime 1.066s | Percentage 100.00% 2025-05-19 10:34:05 [AnalysisStatistics.analysisStatisticsLog-53] 2025-05-19 10:34:05.850 [job-0] ERROR Engine - 2025-05-19 10:34:05 [AnalysisStatistics.analysisStatisticsLog-53] 2025-05-19 10:34:05 [AnalysisStatistics.analysisStatisticsLog-53] 经DataX智能分析,该任务最可能的错误原因是: 2025-05-19 10:34:05 [AnalysisStatistics.analysisStatisticsLog-53] com.alibaba.datax.common.exception.DataXException: Code:[Framework-14], Description:[DataX传输脏数据超过用户预期，该错误通常是由于源端数据存在较多业务脏数据导致，请仔细检查DataX汇报的脏数据日志信息, 或者您可以适当调大脏数据阈值 .]. - 脏数据条数检查不通过，限制是[100]条，但实际上捕获了[361]条. 2025-05-19 10:34:05 [AnalysisStatistics.analysisStatisticsLog-53] at com.alibaba.datax.common.exception.DataXException.asDataXException(DataXException.java:30) 2025-05-19 10:34:05 [AnalysisStatistics.analysisStatisticsLog-53] at com.alibaba.datax.core.util.ErrorRecordChecker.checkRecordLimit(ErrorRecordChecker.java:58) 2025-05-19 10:34:05 [AnalysisStatistics.analysisStatisticsLog-53] at com.alibaba.datax.core.job.scheduler.AbstractScheduler.schedule(AbstractScheduler.java:89) 2025-05-19 10:34:05 [AnalysisStatistics.analysisStatisticsLog-53] at com.alibaba.datax.core.job.JobContainer.schedule(JobContainer.java:535) 2025-05-19 10:34:05 [AnalysisStatistics.analysisStatisticsLog-53] at com.alibaba.datax.core.job.JobContainer.start(JobContainer.java:119) 2025-05-19 10:34:05 [AnalysisStatistics.analysisStatisticsLog-53] at com.alibaba.datax.core.Engine.start(Engine.java:86) 2025-05-19 10:34:05 [AnalysisStatistics.analysisStatisticsLog-53] at com.alibaba.datax.core.Engine.entry(Engine.java:168) 2025-05-19 10:34:05 [AnalysisStatistics.analysisStatisticsLog-53] at com.alibaba.datax.core.Engine.main(Engine.java:201) 2025-05-19 10:34:05 [AnalysisStatistics.analysisStatisticsLog-53] 2025-05-19 10:34:05 [JobThread.run-165] <br>----------- datax-web job execute end(finish) -----------<br>----------- ReturnT:ReturnT [code=500, msg=command exit value(1) is failed, content=null] 2025-05-19 10:34:05 [TriggerCallbackThread.callbackLog-186] <br>----------- datax-web job callback finish. 如何清空脏数据？
05-20
DataX 是阿里巴巴开源的一款异构数据源离线同步工具，支持多种数据源之间的高效数据传输。对于脏数据的处理，DataX 提供了灵活的配置选项来控制和管理这些异常数据。 #### 脏数据定义与处理机制 DataX 将在传输过程...
./datax.py ../job/job.json DataX (DATAX-OPENSOURCE-3.0), From Alibaba ! Copyright (C) 2010-2017, Alibaba Group. All Rights Reserved. 2025-11-11 11:16:32.262 [main] INFO MessageSource - JVM TimeZone: GMT+08:00, Locale: zh_CN 2025-11-11 11:16:32.265 [main] INFO MessageSource - use Locale: zh_CN timeZone: sun.util.calendar.ZoneInfo[id="GMT+08:00",offset=28800000,dstSavings=0,useDaylight=false,transitions=0,lastRule=null] 2025-11-11 11:16:32.304 [main] INFO VMInfo - VMInfo# operatingSystem class => sun.management.OperatingSystemImpl 2025-11-11 11:16:32.314 [main] INFO Engine - the machine info => osInfo: Linux amd64 3.10.0-1127.19.1.el7.x86_64 jvmInfo: Oracle Corporation 1.8 25.291-b10 cpu num: 16 totalPhysicalMemory: -0.00G freePhysicalMemory: -0.00G maxFileDescriptorCount: -1 currentOpenFileDescriptorCount: -1 GC Names [PS MarkSweep, PS Scavenge] MEMORY_NAME | allocation_size | init_size PS Eden Space | 256.00MB | 256.00MB Code Cache | 240.00MB | 2.44MB Compressed Class Space | 1,024.00MB | 0.00MB PS Survivor Space | 42.50MB | 42.50MB PS Old Gen | 683.00MB | 683.00MB Metaspace | -0.00MB | 0.00MB 2025-11-11 11:16:32.330 [main] INFO Engine - { "setting":{ "speed":{ "channel":1 }, "errorLimit":{ "record":0, "percentage":0.02 } }, "content":[ { "reader":{ "name":"streamreader", "parameter":{ "column":[ { "value":"DataX", "type":"string" }, { "value":20250101, "type":"long" }, { "value":"2025-01-01 00:00:00", "type":"date" }, { "value":true, "type":"bool" }, { "value":"test", "type":"bytes" } ], "sliceRecordCount":100000 } }, "writer":{ "name":"streamwriter", "parameter":{ "print":false, "encoding":"UTF-8" } } } ] } 2025-11-11 11:16:32.355 [main] INFO PerfTrace - PerfTrace traceId=job_-1, isEnable=false 2025-11-11 11:16:32.356 [main] INFO JobContainer - DataX jobContainer starts job. 2025-11-11 11:16:32.357 [main] INFO JobContainer - Set jobId = 0 2025-11-11 11:16:32.379 [job-0] INFO JobContainer - jobContainer starts to do prepare ... 2025-11-11 11:16:32.380 [job-0] INFO JobContainer - DataX Reader.Job [streamreader] do prepare work . 2025-11-11 11:16:32.380 [job-0] INFO JobContainer - DataX Writer.Job [streamwriter] do prepare work . 2025-11-11 11:16:32.380 [job-0] INFO JobContainer - jobContainer starts to do split ... 2025-11-11 11:16:32.381 [job-0] INFO JobContainer - Job set Channel-Number to 1 channels. 2025-11-11 11:16:32.381 [job-0] INFO JobContainer - DataX Reader.Job [streamreader] splits to [1] tasks. 2025-11-11 11:16:32.382 [job-0] INFO JobContainer - DataX Writer.Job [streamwriter] splits to [1] tasks. 2025-11-11 11:16:32.411 [job-0] INFO JobContainer - jobContainer starts to do schedule ... 2025-11-11 11:16:32.414 [job-0] INFO JobContainer - Scheduler starts [1] taskGroups. 2025-11-11 11:16:32.416 [job-0] INFO JobContainer - Running by standalone Mode. 2025-11-11 11:16:32.423 [taskGroup-0] INFO TaskGroupContainer - taskGroupId=[0] start [1] channels for [1] tasks. 2025-11-11 11:16:32.430 [taskGroup-0] INFO Channel - Channel set byte_speed_limit to -1, No bps activated. 2025-11-11 11:16:32.430 [taskGroup-0] INFO Channel - Channel set record_speed_limit to -1, No tps activated. 2025-11-11 11:16:32.449 [taskGroup-0] INFO TaskGroupContainer - taskGroup[0] taskId[0] attemptCount[1] is started 2025-11-11 11:16:32.550 [taskGroup-0] INFO TaskGroupContainer - taskGroup[0] taskId[0] is successed, used[102]ms 2025-11-11 11:16:32.551 [taskGroup-0] INFO TaskGroupContainer - taskGroup[0] completed it's tasks. 2025-11-11 11:16:42.453 [job-0] INFO StandAloneJobContainerCommunicator - Total 100000 records, 2600000 bytes | Speed 253.91KB/s, 10000 records/s | Error 0 records, 0 bytes | All Task WaitWriterTime 0.025s | All Task WaitReaderTime 0.059s | Percentage 100.00% 2025-11-11 11:16:42.454 [job-0] INFO AbstractScheduler - Scheduler accomplished all tasks. 2025-11-11 11:16:42.454 [job-0] INFO JobContainer - DataX Writer.Job [streamwriter] do post work. 2025-11-11 11:16:42.455 [job-0] INFO JobContainer - DataX Reader.Job [streamreader] do post work. 2025-11-11 11:16:42.455 [job-0] INFO JobContainer - DataX jobId [0] completed successfully. 2025-11-11 11:16:42.456 [job-0] INFO HookInvoker - No hook invoked, because base dir not exists or is a file: /data/datax/hook 2025-11-11 11:16:42.457 [job-0] INFO JobContainer - [total cpu info] => averageCpu | maxDeltaCpu | minDeltaCpu -1.00% | -1.00% | -1.00% [total gc info] => NAME | totalGCCount | maxDeltaGCCount | minDeltaGCCount | totalGCTime | maxDeltaGCTime | minDeltaGCTime PS MarkSweep | 0 | 0 | 0 | 0.000s | 0.000s | 0.000s PS Scavenge | 0 | 0 | 0 | 0.000s | 0.000s | 0.000s 2025-11-11 11:16:42.457 [job-0] INFO JobContainer - PerfTrace not enable! 2025-11-11 11:16:42.458 [job-0] INFO StandAloneJobContainerCommunicator - Total 100000 records, 2600000 bytes | Speed 253.91KB/s, 10000 records/s | Error 0 records, 0 bytes | All Task WaitWriterTime 0.025s | All Task WaitReaderTime 0.059s | Percentage 100.00% 2025-11-11 11:16:42.459 [job-0] INFO JobContainer - 任务启动时刻 : 2025-11-11 11:16:32 任务结束时刻 : 2025-11-11 11:16:42 任务总计耗时 : 10s 任务平均流量 : 253.91KB/s 记录写入速度 : 10000rec/s 读出记录总数 : 100000 读写失败总数 : 0 。执行示例json正常执行但是执行./datax.py ../job/data-water.json DataX (DATAX-OPENSOURCE-3.0), From Alibaba ! Copyright (C) 2010-2017, Alibaba Group. All Rights Reserved. 2025-11-11 11:18:45.975 [main] WARN ConfigParser - 插件[null,null]加载失败，1s后重试... Exception:Code:[Framework-12], Description:[DataX插件初始化错误, 该问题通常是由于DataX安装错误引起，请联系您的运维解决 .]. - 插件加载失败，未完成指定插件加载:[null] 2025-11-11 11:18:46.998 [main] ERROR Engine - 经DataX智能分析,该任务最可能的错误原因是: com.alibaba.datax.common.exception.DataXException: Code:[Framework-12], Description:[DataX插件初始化错误, 该问题通常是由于DataX安装错误引起，请联系您的运维解决 .]. - 插件加载失败，未完成指定插件加载:[null] at com.alibaba.datax.common.exception.DataXException.asDataXException(DataXException.java:30) at com.alibaba.datax.core.util.ConfigParser.parsePluginConfig(ConfigParser.java:142) at com.alibaba.datax.core.util.ConfigParser.parse(ConfigParser.java:63) at com.alibaba.datax.core.Engine.entry(Engine.java:131) at com.alibaba.datax.core.Engine.main(Engine.java:201)
11-12
- DataX 插件通常位于 `plugin/reader` 和 `plugin/writer` 目录下。 - 确保 `data-water.json` 中使用的插件在对应目录中存在。 - 例如：`mysqlreader` 应存在于 `plugin/reader/mysqlreader/`。 3. **检查 ...
翻译 SourceForge logo Home Browse Ethernet bridge tables Mailing Lists Thread: [Ebtables-user] [RELEASE] ebtables version 2.0.8-rc3 Brought to you by: bdschuym Summary Files Reviews Support Mailing Lists Tickets ▾ Code Mailing Lists Menu ▾ ebtables-user [Ebtables-user] [RELEASE] ebtables version 2.0.8-rc3 From: Bart De S. <bds...@pa...> - 2006-12-18 19:09:02 Hi all, I've just uploade release candidate 3 of ebtables v2.0.8. Changes: * fixed a few reported bugs * ebt_among --among-dst-file and --among-src-file: allow the list to be given in a file (circumvents command line max. line length) * ebt_nat --snat-arp: if it's an arp packet, also change the source address in the arp header * ebt_mark --mark-or, --mark-xor, --mark-and Cheers, Bart Re: [Ebtables-user] [RELEASE] ebtables version 2.0.8-rc3 From: Carl-Daniel H. <c-d...@gm...> - 2006-12-19 13:44:46 Hi Bart, the -fPIC problem on x86_64 is still there: # make [...] ld -shared -soname libebtc.so -o libebtc.so -lc getethertype.o communication.o libebtc.o useful_functions.o ebtables.o ld: ebtables.o: relocation R_X86_64_32 against `a local symbol' can not be used when making a shared object; recompile with -fPIC ebtables.o: could not read symbols: Bad value make: *** [libebtc] Error 1 If you need more information about the problem, please take a look at http://www.x86-64.org/lists/discuss/msg05760.html . Besides that, I get these warnings: ebtables.c: In function `list_em': ebtables.c:324: warning: long long unsigned int format, uint64_t arg (arg 2) ebtables.c:324: warning: long long unsigned int format, uint64_t arg (arg 3) ebtables.c:326: warning: long long unsigned int format, uint64_t arg (arg 2) ebtables.c:326: warning: long long unsigned int format, uint64_t arg (arg 3) extensions/ebt_among.c: In function `parse': extensions/ebt_among.c:348: warning: implicit declaration of function `close' This patch fixes them: --- ebtables-v2.0.8-rc3/ebtables.c 2006-12-17 22:27:17.000000000 +0100 +++ ebtables-v2.0.8-rc3-modified/ebtables.c 2006-12-19 13:24:41.000000000 +0100 @@ -321,9 +321,9 @@ uint64_t bcnt = hlp->cnt.bcnt; if (replace->flags & LIST_X) - printf("-c %llu %llu", pcnt, bcnt); + printf("-c %llu %llu", (unsigned long long)pcnt, (unsigned long long)bcnt); else - printf(", pcnt = %llu -- bcnt = %llu", pcnt, bcnt); + printf(", pcnt = %llu -- bcnt = %llu", (unsigned long long)pcnt, (unsigned long long)bcnt); } printf("\n"); hlp = hlp->next; diff -urN ebtables-v2.0.8-rc3/extensions/ebt_among.c ebtables-v2.0.8-rc3-modified/extensions/ebt_among.c --- ebtables-v2.0.8-rc3/extensions/ebt_among.c 2006-12-17 22:27:17.000000000 +0100 +++ ebtables-v2.0.8-rc3-modified/extensions/ebt_among.c 2006-12-19 13:08:15.000000000 +0100 @@ -7,6 +7,7 @@ */ #include <stdio.h> +#include <unistd.h> #include <string.h> #include <stdlib.h> #include <getopt.h> Explanation: uint64_t is unsigned long long on i386, but unsigned long on x86_64. The problems of creating rules on x86_64 is still there. However, it seems that Al Viro has fixed something in this area: http://www2.kernel.org/git/?p=linux/kernel/git/torvalds/linux-2.6.git;a=commitdiff;h=bb2ef25c2c62444b8fdb0346a23658a419803df9 Regards, Carl-Daniel -- http://www.hailfinger.org/ Re: [Ebtables-user] [RELEASE] ebtables version 2.0.8-rc3 From: Bart De S. <bds...@pa...> - 2006-12-22 18:37:32 Op di, 19-12-2006 te 14:44 +0100, schreef Carl-Daniel Hailfinger: > Hi Bart, > > the -fPIC problem on x86_64 is still there: <snip> > Explanation: uint64_t is unsigned long long on i386, but unsigned long > on x86_64. Should be fixed in CVS, thanks. > The problems of creating rules on x86_64 is still there. However, it seems > that Al Viro has fixed something in this area: > http://www2.kernel.org/git/?p=linux/kernel/git/torvalds/linux-2.6.git;a=commitdiff;h=bb2ef25c2c62444b8fdb0346a23658a419803df9 That's unrelated, it fixes the checking for maliciously constructed ebtables tables. cheers, Bart Re: [Ebtables-user] [RELEASE] ebtables version 2.0.8-rc3 From: Carl-Daniel H. <c-d...@gm...> - 2006-12-23 01:29:15 Bart De Schuymer wrote: > Op di, 19-12-2006 te 14:44 +0100, schreef Carl-Daniel Hailfinger: >> Hi Bart, >> >> the -fPIC problem on x86_64 is still there: > > <snip> >> Explanation: uint64_t is unsigned long long on i386, but unsigned long >> on x86_64. > > Should be fixed in CVS, thanks. Thanks! >> The problems of creating rules on x86_64 is still there. However, it seems >> that Al Viro has fixed something in this area: >> http://www2.kernel.org/git/?p=linux/kernel/git/torvalds/linux-2.6.git;a=commitdiff;h=bb2ef25c2c62444b8fdb0346a23658a419803df9 > > That's unrelated, it fixes the checking for maliciously constructed > ebtables tables. Then normal ebtables userspace on x86_64 definitely counts as malicious. I was referring to this part of the changelog: > While we are at it, don't subtract unrelated pointers... The problems with loading rules on x86_64 have disappeared after applying the patch series from Al Viro. However, the "among" match still reports problems: # ebtables -t nat -A ebtables-experiment --among-dst 2:3:4:5:6:7=192.168.0.1,2:3:4:5:6:8=192.168.0.3,2:3:4:5:6:9=192.168.0.2 -j ACCEPT old_size=56,new_size=1124, sizeof(ebt_entry_match)=40, msize=1084 wh_dst_ofs=16 The kernel doesn't support a certain ebtables extension, consider recompiling your kernel or insmod the extension. dmesg says (debugging added by me): ebtables: among: wrong size: 1084 against expected 1080, rounded to 1080, sizeof(ebt_among_info)=12, wh_size(dst)=1068, wh_size(src)=0, sizeof(ebt_entry_match)=40 kernel msg: ebtables bug: please report to author: match->check failed The source of the problem is easy to spot: In userspace, wh_dst_ofs==16 (because (**match).match_size==16 at line 357 in extensions/ebt_among.c), but in kernelspace we check wh_dst_ofs against sizeof(struct ebt_among_info)==12. That is obviously going to fail. Always. Regards, Carl-Daniel -- http://www.hailfinger.org/ Re: [Ebtables-user] [RELEASE] ebtables version 2.0.8-rc3 From: Bart De S. <bds...@pa...> - 2006-12-23 17:22:31 Op za, 23-12-2006 te 02:29 +0100, schreef Carl-Daniel Hailfinger: > The problems with loading rules on x86_64 have disappeared after > applying the patch series from Al Viro. Well, I'll be damned :) > However, the "among" match still reports problems: > The source of the problem is easy to spot: > In userspace, wh_dst_ofs==16 (because (**match).match_size==16 at > line 357 in extensions/ebt_among.c), but in kernelspace we check > wh_dst_ofs against sizeof(struct ebt_among_info)==12. Are you saying sizeof(struct ebt_among_info) differs between userspace and the kernel? Please double check this... cheers, Bart Re: [Ebtables-user] [RELEASE] ebtables version 2.0.8-rc3 From: Carl-Daniel H. <c-d...@gm...> - 2006-12-23 19:37:56 Bart De Schuymer wrote: > Op za, 23-12-2006 te 02:29 +0100, schreef Carl-Daniel Hailfinger: >> However, the "among" match still reports problems: >> The source of the problem is easy to spot: >> In userspace, wh_dst_ofs==16 (because (**match).match_size==16 at >> line 357 in extensions/ebt_among.c), but in kernelspace we check >> wh_dst_ofs against sizeof(struct ebt_among_info)==12. > > Are you saying sizeof(struct ebt_among_info) differs between userspace > and the kernel? Please double check this... No, I wanted to say that match_size is set in userspace to something which is != sizeof(struct ebt_among_info). More diagnostics: on x86-64: libebtc.c:1175 in ebt_register_match: m->m->match_size set to 8 aligned from 6 libebtc.c:1175 in ebt_register_match: m->m->match_size set to 56 aligned from 52 libebtc.c:1175 in ebt_register_match: m->m->match_size set to 32 aligned from 28 libebtc.c:1175 in ebt_register_match: m->m->match_size set to 8 aligned from 8 libebtc.c:1175 in ebt_register_match: m->m->match_size set to 24 aligned from 24 libebtc.c:1175 in ebt_register_match: m->m->match_size set to 8 aligned from 2 libebtc.c:1175 in ebt_register_match: m->m->match_size set to 72 aligned from 72 libebtc.c:1175 in ebt_register_match: m->m->match_size set to 16 aligned from 12 (among) libebtc.c:1175 in ebt_register_match: m->m->match_size set to 32 aligned from 32 on i386: libebtc.c:1175 in ebt_register_match: m->m->match_size set to 8 aligned from 6 libebtc.c:1175 in ebt_register_match: m->m->match_size set to 52 aligned from 52 libebtc.c:1175 in ebt_register_match: m->m->match_size set to 28 aligned from 28 libebtc.c:1175 in ebt_register_match: m->m->match_size set to 8 aligned from 8 libebtc.c:1175 in ebt_register_match: m->m->match_size set to 12 aligned from 12 libebtc.c:1175 in ebt_register_match: m->m->match_size set to 4 aligned from 2 libebtc.c:1175 in ebt_register_match: m->m->match_size set to 72 aligned from 72 libebtc.c:1175 in ebt_register_match: m->m->match_size set to 12 aligned from 12 (among) libebtc.c:1175 in ebt_register_match: m->m->match_size set to 24 aligned from 24 Userspace sets wh_dst_ofs=EBT_ALIGN(sizeof(struct ebt_among_info)), but the kernel expects wh_dst_ofs=sizeof(struct ebt_among_info). On i386, the EBT_ALIGN macro has no effect for the among match, but on x86_64 it does. That explains why the bug was never noticed on i386. So we either change the kernel or userspace. Changing the kernel would mean that suddenly all existing userspace works. Regards, Carl-Daniel -- http://www.hailfinger.org/ Re: [Ebtables-user] [RELEASE] ebtables version 2.0.8-rc3 From: Bart De S. <bds...@pa...> - 2007-01-10 19:08:19 Attachments: patch_among.diff Op za, 23-12-2006 te 20:37 +0100, schreef Carl-Daniel Hailfinger: > Userspace sets wh_dst_ofs=EBT_ALIGN(sizeof(struct ebt_among_info)), but the > > kernel expects wh_dst_ofs=sizeof(struct ebt_among_info). On i386, the EBT_ALIGN > macro has no effect for the among match, but on x86_64 it does. That explains > why the bug was never noticed on i386. > > So we either change the kernel or userspace. Changing the kernel would mean > that suddenly all existing userspace works. It's best not to force people into using a specific kernel, so adjusting userspace is the thing to do. Please test the attached userspace patch. Cheers, Bart Re: [Ebtables-user] [RELEASE] ebtables version 2.0.8-rc3 From: Carl-Daniel H. <c-d...@gm...> - 2008-01-28 17:13:26 On 10.01.2007 20:08, Bart De Schuymer wrote: > Op za, 23-12-2006 te 20:37 +0100, schreef Carl-Daniel Hailfinger: > > >> Userspace sets wh_dst_ofs=EBT_ALIGN(sizeof(struct ebt_among_info)), but the >> kernel expects wh_dst_ofs=sizeof(struct ebt_among_info). On i386, the EBT_ALIGN >> macro has no effect for the among match, but on x86_64 it does. That explains >> why the bug was never noticed on i386. >> >> So we either change the kernel or userspace. Changing the kernel would mean >> that suddenly all existing userspace works. >> > > It's best not to force people into using a specific kernel, so adjusting > userspace is the thing to do. > > Please test the attached userspace patch. > I'm very sorry, but I have been unable to take the machine in question out of production to test the patch. I found a commit in OpenVZ which touches the code in question and will probably be upstreamed sometime. http://git.openvz.org/?p=linux-2.6.24-openvz;a=commit;h=a3d28217a51b57b2980f5fc1203cf7b402ca9bb7 Regards, Carl-Daniel
最新发布
12-16
- `--mark-or`、`--mark-xor`、`--mark-and` 致谢， Bart --- **回复**： Carl-Daniel H. <c-d...@gm...> **日期**： 2006-12-19 13:44:46 Bart， x86_64 平台的 `-fPIC` 编译问题仍存在： ```bash # ...
[2025-11-11 10:01:06.632] INFO [30772] [CID:Bsp_Storage_defrag_0007] <Bsp_Storage_defrag_0007.py:59> 第121次执行接口，修改时间为24h之后便于多次触发 [2025-11-11 10:01:07.674] INFO [30772] [CID:Bsp_Storage_defrag_0007] <common.py:669> dt_modify:2025-11-16 [2025-11-11 10:01:07.674] INFO [30772] [CID:Bsp_Storage_defrag_0007] <common.py:670> modify_sys_time()-> from 2025-11-15 to 2025-11-16 [2025-11-11 10:01:08.737] INFO [30772] [CID:Bsp_Storage_defrag_0007] <common.py:674> output:2025-11-11 [2025-11-11 10:01:09.100] INFO [32156] [CID:Bsp_Storage_defrag_0007] <phone_base.py:1057> screenshot ok,use 0.32s,image name:script_frame_png@wrapper.TestModuleAwLogicIssue@3K6596001HD00000@1762826468775@Bsp_Storage_defrag_0007@null [2025-11-11 10:01:09.323] INFO [32156] [CID:Bsp_Storage_defrag_0007] <grpc_base.py:612> device_id:3K6596001HD00000, Apk android.camera.cts is installed already [2025-11-11 10:01:09.324] INFO [32156] [CID:Bsp_Storage_defrag_0007] <__init__.py:289> current uiautomator version 3.4.2 [2025-11-11 10:01:09.561] INFO [32156] [CID:Bsp_Storage_defrag_0007] <__init__.py:402> forward tcp:57802 tcp:9008 [2025-11-11 10:01:09.575] ERROR [32156] [CID:Bsp_Storage_defrag_0007] <__init__.py:261> sn:3K6596001HD00000 POST->url:http://127.0.0.1:forward:57802to9008/jsonrpc/0 param:{'data': '{"jsonrpc": "2.0", "id": "ping", "method": "ping"}', 'json': None, 'timeout': 3},request error: ('Connection aborted.', RemoteDisconnected('Remote end closed connection without response')) [2025-11-11 10:01:09.657] INFO [32156] [CID:Bsp_Storage_defrag_0007] <__init__.py:335> current uiautomator running info: [2025-11-11 10:01:10.739] ERROR [32156] [CID:Bsp_Storage_defrag_0007] <__init__.py:261> sn:3K6596001HD00000 POST->url:http://127.0.0.1:forward:57802to9008/jsonrpc/0 param:{'data': '{"jsonrpc": "2.0", "id": "ping", "method": "ping"}', 'json': None, 'timeout': 3},request error: ('Connection aborted.', RemoteDisconnected('Remote end closed connection without response')) [2025-11-11 10:01:10.753] INFO [32156] [CID:Bsp_Storage_defrag_0007] <__init__.py:393> forward tcp:57811 tcp:7920 [2025-11-11 10:01:10.769] ERROR [32156] [CID:Bsp_Storage_defrag_0007] <__init__.py:261> sn:3K6596001HD00000 GET->url:http://127.0.0.1:forward:57811to7920/uiautomator param:{'allow_redirects': True},request error: ('Connection aborted.', RemoteDisconnected('Remote end closed connection without response')) [2025-11-11 10:01:10.770] WARNING [32156] [CID:Bsp_Storage_defrag_0007] <__init__.py:264> atx-instrument has something wrong, auto recovering [2025-11-11 10:01:10.784] ERROR [32156] [CID:Bsp_Storage_defrag_0007] <__init__.py:261> sn:3K6596001HD00000 POST->url:http://127.0.0.1:forward:57802to9008/jsonrpc/0 param:{'data': '{"jsonrpc": "2.0", "id": "ping", "method": "ping"}', 'json': None, 'timeout': 3},request error: ('Connection aborted.', RemoteDisconnected('Remote end closed connection without response')) [2025-11-11 10:01:10.799] ERROR [32156] [CID:Bsp_Storage_defrag_0007] <__init__.py:261> sn:3K6596001HD00000 POST->url:http://127.0.0.1:forward:57802to9008/jsonrpc/0 param:{'data': '{"jsonrpc": "2.0", "id": "ping", "method": "ping"}', 'json': None, 'timeout': 3},request error: ('Connection aborted.', RemoteDisconnected('Remote end closed connection without response')) [2025-11-11 10:01:11.069] INFO [32156] [CID:Bsp_Storage_defrag_0007] <init.py:598> init Install [2025-11-11 10:01:11.171] INFO [32156] [CID:Bsp_Storage_defrag_0007] <init.py:654> No process matching atx-instrument found. [2025-11-11 10:01:11.310] INFO [32156] [CID:Bsp_Storage_defrag_0007] <phone_base.py:175> AtxAppQueryController query state: [2025-11-11 10:01:13.053] INFO [32156] [CID:Bsp_Storage_defrag_0007] <grpc_base.py:612> device_id:3K6596001HD00000, Apk android.camera.cts is installed already [2025-11-11 10:01:18.624] INFO [32156] [CID:Bsp_Storage_defrag_0007] <grpc_base.py:612> device_id:3K6596001HD00000, Apk com.oplus.autotest.olib.uiautomator is installed already [2025-11-11 10:01:18.624] INFO [32156] [CID:Bsp_Storage_defrag_0007] <phone_base.py:195> apk:OlibOplusAtx,package install com.oplus.autotest.olib.uiautomator install state:True [2025-11-11 10:01:19.042] INFO [32156] [CID:Bsp_Storage_defrag_0007] <init.py:585> Install atx-instrument 0.0.8 [2025-11-11 10:01:19.045] INFO [32156] [CID:Bsp_Storage_defrag_0007] <init.py:119> url:http://maven.scm.adc.com:8081/nexus/repository/olib-shares/other/bin/atx/0.0.8/atx-instrument_linux_arm64,path:D:\Users\18915215633\.uiautomator2\cache\atx-instrument_linux_arm64-da8d61b8f6\atx-instrument_linux_arm64 [2025-11-11 10:01:19.885] INFO [32156] [CID:Bsp_Storage_defrag_0007] <init.py:671> /data/local/tmp/atx-instrument server --nouia -d --atxPkg com.oplus.autotest.olib.uiautomator --class androidx.test.runner.AndroidJUnitRunner --testPkg com.oplus.autotest.olib.uiautomator runoutput:INFO[0000] run atx-instrument in background 0.0.8 INFO[0000] atx-instrument listening on :7920 [2025-11-11 10:01:19.901] ERROR [32156] [CID:Bsp_Storage_defrag_0007] <init.py:684> ('Connection aborted.', RemoteDisconnected('Remote end closed connection without response')) [2025-11-11 10:01:21.117] ERROR [32156] [CID:Bsp_Storage_defrag_0007] <__init__.py:261> sn:3K6596001HD00000 POST->url:http://127.0.0.1:forward:57802to9008/jsonrpc/0 param:{'data': '{"jsonrpc": "2.0", "id": "ping", "method": "ping"}', 'json': None, 'timeout': 3},request error: ('Connection aborted.', RemoteDisconnected('Remote end closed connection without response')) [2025-11-11 10:01:23.183] WARNING [32156] [CID:Bsp_Storage_defrag_0007] <__init__.py:455> _prepare_atx_instrument out lock, use time:12.383s [2025-11-11 10:01:24.054] ERROR [32144] [CID:Bsp_Storage_defrag_0007] <tc.py:1493> 测试模块aw/logic问题:Fail to modify system time for phone! [2025-11-11 10:01:24.055] ERROR [32144] [CID:Bsp_Storage_defrag_0007] <tc.py:1507> Traceback (most recent call last): File "E:\olib_docker_64\25111109323000_32288\Python37\site-packages\olib\tc.py", line 2041, in exe_case_body self.exe_main(case_name) File "E:\olib_docker_64\25111109323000_32288\Python37\site-packages\olib\tc.py", line 2023, in exe_main args=(self.mark_log_point_loader(getattr(self, case_name)),)) File "E:\olib_docker_64\25111109323000_32288\Python37\site-packages\olib\olib_timeout.py", line 101, in func_timeout raise_exception(exception) File "C:\olib\python37\lib\site-packages\func_timeout\py3_raise.py", line 7, in raise_exception raise exception[0] from None File "E:\olib_docker_64\25111109323000_32288\Python37\site-packages\olib\tc.py", line 136, in fix_timeout func(*args, **kwargs) File "E:\olib_docker_64\25111109323000_32288\Python37\site-packages\olib\tc.py", line 1073, in wrapper value = func(*args, **kwargs) File "E:\olib_docker_64\25111109323000_32288\Python37\site-packages\olib_aw\teams\bsp\storage\common.py", line 844, in wrapper func(instance, *args, **kwargs) File "E:\olib_docker_64\25111109323000_32288\Python37\site-packages\olib_scripts_oplus_bsp\scripts\bsp\storage\storage_defrag\Bsp_Storage_defrag_0007.py", line 60, in main modify_sys_time_by_hours(self.dut, hours=25) File "E:\olib_docker_64\25111109323000_32288\Python37\site-packages\olib_aw\teams\bsp\common.py", line 676, in modify_sys_time_by_hours raise TestModuleAwLogicIssue("Fail to modify system time for phone!") olib.olib_except.script_except.TestModuleAwLogicIssue: 测试模块aw/logic问题:Fail to modify system time for phone! [2025-11-11 10:01:24.090] INFO [22612] [CID:Bsp_Storage_defrag_0007] <phone_base.py:1219> device_id:3K6596001HD00000, reuse phone object. [2025-11-11 10:01:24.090] INFO [22612] [CID:Bsp_Storage_defrag_0007] <tc_utils.py:240> start device_point [2025-11-11 10:01:24.618] INFO [32664] [CID:Bsp_Storage_defrag_0007] <phone_base.py:1057> screenshot ok,use 0.22s,image name:script_step@wrapper@3K6596001HD00000@1762826484398@Bsp_Storage_defrag_0007@null [2025-11-11 10:01:24.743] INFO [14736] [CID:Bsp_Storage_defrag_0007] <tc.py:1058> Entering .Bsp_Storage_defrag_0007.cleanup_iteration [2025-11-11 10:01:24.743] INFO [14736] [CID:Bsp_Storage_defrag_0007] <tc.py:1075> Exiting .Bsp_Storage_defrag_0007.cleanup_iteration [2025-11-11 10:01:24.745] INFO [26248] [CID:Bsp_Storage_defrag_0007] <tc.py:1058> Entering .Bsp_Storage_defrag_0007.cleanup_testcase [2025-11-11 10:01:24.745] INFO [26248] [CID:Bsp_Storage_defrag_0007] <tc.py:1075> Exiting .Bsp_Storage_defrag_0007.cleanup_testcase [2025-11-11 10:01:24.745] INFO [32144] [CID:Bsp_Storage_defrag_0007] <tc.py:1733> 处理用例log [2025-11-11 10:01:24.746] INFO [32144] [CID:Bsp_Storage_defrag_0007] <tc.py:3343> current disk free space : 185G; [2025-11-11 10:01:24.819] INFO [32144] [CID:Bsp_Storage_defrag_0007] <tc.py:2490> start-copy c:\olib\logs\3656885_251111093211_11036\Bsp_Storage_defrag_0007_14997949_38955632 to D:\gtest-cli\execute-agent\home\tmp\script-38955632-424\OLIB-r0--0-0\0\Bsp_Storage_defrag_0007_14997949_38955632
11-12
- 查看 `atx-instrument` 和 `uiautomator2` 是否正常运行。 - 可尝试手动重启服务或重新安装依赖包。 4. **排查端口冲突**： - 使用 `netstat` 或 `lsof` 检查端口是否被占用。 - 尝试更换端口或重启测试框架...
linux查看源代码软件,linux下源码查看工具source-navigator安装
weixin_42481708的博客
05-07 1441
在Linux环境下不少人推荐使用Source-Navigator工具用来查看源代码。它是Linux开发的一个IDE，但是大部分人均用它来阅读，而不是开发，总而言之，不多废话，看看在10.04下，如何安装步骤如下：1. 下载最新的source navigator包，URL为：我下的最新的包为sourcenav-6.0.tar.gz ，大小：18.9MB.2.进入到文件目录3.解压之，命令：tar...
linux下 看代码的工具kscope
幸福大街28号
04-06 570
<br />很像windows下的source insight
linux服务器 阅读代码 工具,Linux 平台下阅读源码的工具
weixin_31366459的博客
04-29 774
前言看源代码是一个程序员必须经历的事情,也是可以提升能力的一个捷径.个人认为: 要完全掌握一个软件的方法只有阅读源码在Windows下有sourceinsight这个源码阅读软件(虽然我没用过,但是网上评价还不错),由于我是个Linuxer,并不喜欢用Windows,所以自然是选择在Linux下阅读源码的工具了.下面我将逐一介绍在Linux下阅读源码的工具.vim+ctags+cscope源码阅读...
linux源代码 及 查看工具、教程
eqxu的专栏
06-29 9460
  1、比较好的linux源代码  0.11版本 2.4版本  2.6版本下载地址：http://www.oldlinux.org/Linux.old/kernel/linux-0.11-pre.tar.Z 最好是看一下http://www.oldlinux.org/index_cn.html的linux源代码内核完全注释 可以免费下载 赵炯博士所著  相关视频：由尚德系列讲座之linux内核
windows下查看linux内核代码用什么软件比较方便,【crifan推荐开发工具】Windows下查看代码工具：Source Insight...
weixin_42476888的博客
04-30 1358
【背景】之前就写过Source Insight相关的帖子：但是都是解决问题的。没有去推荐。现在好好推荐一下。windows下查看代码的好软件：Source InsightSource Insight简介Source Insight，是个用于查看代码的软件。其只能在Windows平台上运行的软件。Source Insight是收费软件。对于用Source Insight去查看代码：有人会说，现在用于...
代码查看工具
qq_33671888的博客
07-14 457
1 understand 2 SonarQube 3 golang : go-callvis gometalinter gosec(检查代码安全)goreporter 4 sublimeText
Linux下源代码阅读工具
weixin_33845881的博客
06-28 250
转载链接：http://www.cnblogs.com/lidabo/p/4957908.htmlLinux源代码分析和阅读工具比较Windows下的源码阅读工具Souce Insight 凭 借着其易用性和多种编程语言的支持，无疑是这个领域的“带头大哥”。Linux/UNIX环境下呢？似乎仍然是处于百花齐放，各有千秋的春秋战国时代，实 际上，似乎其环境下的任何软件都是处于一...
如何将vim改造成强大的代码阅读器
simonGeek的专栏
12-14 7924
最近在接触C/C++编程，一般是在公司的远程服务器阅读和编写代码，这就会涉及C/C++的代码阅读器的问题。通过请教和调研，使用比较广泛的方式是在vim上添加插件，增强vim的功能。 经过比较和分析，使用比较广泛的插件是Ctags和Taglist两个插件。 简单说明一下这两个插件的关系。 插件1：Ctag：通过这个插件为程序中的变量、函数、类生成对应的tag标签。可以理解成Map的形式
评论
添加红包

请填写红包祝福语或标题

个

红包个数最小为10个

元

红包金额最低5元

当前余额3.43元 前往充值 >
需支付：10.00元
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付元
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明：

1.余额是钱包充值的虚拟货币，按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载，可以购买VIP、付费专栏及课程。

余额充值