[GitHub] [dolphinscheduler] songxiaohu123 commented on issue #11433: 3.0.0-release稳定版本，数据质量问题

[GitBox <gi...@apache.org>]

songxiaohu123 commented on issue #11433:
URL: https://github.com/apache/dolphinscheduler/issues/11433#issuecomment-1211911158

   完整的内容如下
   /opt/module/spark-3.0.0/bin/spark-submit --master yarn --deploy-mode cluster --driver-cores 1 --driver-memory 512M --num-executors 2 --executor-cores 2 --executor-memory 2G --queue default --conf spark.yarn.maxAppAttempts=1 /home/jar/dolphinscheduler-data-quality-3.0.1-alpha-SNAPSHOT.jar "{\"name\":\"$t(null_check)\",\"env\":{\"type\":\"batch\",\"config\":null},\"readers\":[{\"type\":\"JDBC\",\"config\":{\"database\":\"data_gateway\",\"password\":\"rootmysql\",\"driver\":\"com.mysql.cj.jdbc.Driver\",\"user\":\"root\",\"output_table\":\"data_gateway_dwg_center_channel\",\"table\":\"dwg_center_channel\",\"url\":\"jdbc:mysql://192.168.101.29:3306/data_gateway?allowLoadLocalInfile=false&autoDeserialize=false&allowLocalInfile=false&allowUrlInLocalInfile=false\"} }],\"transformers\":[{\"type\":\"sql\",\"config\":{\"index\":1,\"output_table\":\"null_items\",\"sql\":\"SELECT * FROM data_gateway_dwg_center_channel WHERE (channel_name is null or channel_name = '') AND (center_name is not nu
 ll)\"} },{\"type\":\"sql\",\"config\":{\"index\":2,\"output_table\":\"null_count\",\"sql\":\"SELECT COUNT(*) AS nulls FROM null_items\"} }],\"writers\":[{\"type\":\"JDBC\",\"config\":{\"database\":\"dolphinscheduler3\",\"password\":\"rootmysql\",\"driver\":\"com.mysql.cj.jdbc.Driver\",\"user\":\"root\",\"table\":\"t_ds_dq_execute_result\",\"url\":\"jdbc:mysql://192.168.101.29:3306/dolphinscheduler3?useUnicode=true&characterEncoding=UTF-8&allowLoadLocalInfile=false&autoDeserialize=false&allowLocalInfile=false&allowUrlInLocalInfile=false\",\"sql\":\"select 0 as rule_type,'$t(null_check)' as rule_name,0 as process_definition_id,2 as process_instance_id,2 as task_instance_id,null_count.nulls AS statistics_value,${comparison_name} AS comparison_value,1 AS comparison_type,0 as check_type,10 as threshold,0 as operator,0 as failure_strategy,'hdfs://mycluster:8020/user/ys/data_quality_error_data/0_2_dqtest' as error_output_path,'2022-08-11 17:23:25' as create_time,'2022-08-11 17:23:25' as up
 date_time from null_count \"} },{\"type\":\"JDBC\",\"config\":{\"database\":\"dolphinscheduler3\",\"password\":\"rootmysql\",\"driver\":\"com.mysql.cj.jdbc.Driver\",\"user\":\"root\",\"table\":\"t_ds_dq_task_statistics_value\",\"url\":\"jdbc:mysql://192.168.101.29:3306/dolphinscheduler3?useUnicode=true&characterEncoding=UTF-8&allowLoadLocalInfile=false&autoDeserialize=false&allowLocalInfile=false&allowUrlInLocalInfile=false\",\"sql\":\"select 0 as process_definition_id,2 as task_instance_id,1 as rule_id,'Q9BFKCMZRAA4YYPPEUG8WESGNDASRAVNILHQAW1OJUI=' as unique_code,'null_count.nulls'AS statistics_name,null_count.nulls AS statistics_value,'2022-08-11 17:23:25' as data_time,'2022-08-11 17:23:25' as create_time,'2022-08-11 17:23:25' as update_time from null_count\"} },{\"type\":\"hdfs_file\",\"config\":{\"path\":\"hdfs://mycluster:8020/user/ys/data_quality_error_data/0_2_dqtest\",\"input_table\":\"null_items\"} }]}"


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org