Workflow工作经验之常见错误

错误一： Out of Memory

ERROR >>>

ERROR - OutOfMemoryError: unable to create new native thread

ERROR - DRException: Failed to start dataflow graph etl\.fa_master_gen\.fa_master - createView-phase0

ERROR - DROperatorException: etl\.fa_master_gen\.fa_master - createView

ERROR - DataViewException: Failed to run view etl.fa_master_gen.fa_master in gen/data/views/etl.fa_master_gen.fa_master

ERROR <<<

1. 去找到对应的environment.yaml 文件，修改workflows一栏，把java: maxHeapSize调大一点

2. 也可以去workflow/workflow_workflow.custom.yaml 修改cpu和memory的配置

3. 也可以删掉gen文件夹和workspace文件夹然后重新跑一遍程序

错误二：ConnectException:Connection refused

ERROR - ConnectException:Connection refused
ERROR - RMIException: An error occurred while communicating with the DataRushcluster. This usually indicates that the process encountered a runtime error onthe cluster. The job logs on the cluster will have more details.

去environment.yaml文件查看端口号是否和cluster匹配。登录172.20.9.42:1100，可以看到端口号。

错误三： IOException: No space left on device

ERROR >>>
ERROR - IOException: No space left on device
ERROR - RuntimeException: Unable to extract jar:file:/opt/datarush/work/QA-UNAR-S3/classcache/cache/4038313593/datarush-hadoop-cdh5-6.3.1-11.jar!/datarush-hadoop-cdh5/org.apache.hive/hive-metastore-0.11.0.jar
ERROR - DROperatorException:s01_ETL\.readRaw\.InputData_AccountOwner_DQM\.account_owner_dqm - createView

ERROR <<<

有可能是tmp folder满了，注意，可能是node也可能是主机的folder。叫IT check一下




错误四： OutOfMemoryError

ERROR >>>
ERROR   - OutOfMemoryError: unable to create new native thread
ERROR   - DRException: Failed to start dataflow graph s03_postETL\.cm_customer_agg\.cm_customer_agg - createView-phase0
ERROR   - DROperatorException: s03_postETL\.cm_customer_agg\.cm_customer_agg - createView
ERROR <<<

在local上跑的话，可以试一下

ulimit -u 10240

再去跑code