You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@dolphinscheduler.apache.org by GitBox <gi...@apache.org> on 2022/10/07 16:02:24 UTC
[GitHub] [dolphinscheduler] zcw5116 opened a new issue, #12257: [Bug] [Data Quality] No main class set in JAR
zcw5116 opened a new issue, #12257:
URL: https://github.com/apache/dolphinscheduler/issues/12257
### Search before asking
- [X] I had searched in the [issues](https://github.com/apache/dolphinscheduler/issues?q=is%3Aissue) and found no similar issues.
### What happened
1. DS Version: 3.1.0
2. Data Quality , datasource clickhouse check null
3. Spark Submit Missing main class
the following logs:
[LOG-PATH]: /opt/dolphinscheduler/worker-server/logs/20221007/7133023937376_2-25-29.log, [HOST]: Host{address='192.168.66.190:1234', ip='192.168.66.190', port=1234}
[INFO] 2022-10-07 15:53:04.851 +0000 - Begin to pulling task
[INFO] 2022-10-07 15:53:04.855 +0000 - Begin to initialize task
[INFO] 2022-10-07 15:53:04.855 +0000 - Set task startTime: Fri Oct 07 15:53:04 UTC 2022
[INFO] 2022-10-07 15:53:04.856 +0000 - Set task envFile: /opt/dolphinscheduler/worker-server/conf/dolphinscheduler_env.sh
[INFO] 2022-10-07 15:53:04.856 +0000 - Set task appId: 25_29
[INFO] 2022-10-07 15:53:04.856 +0000 - End initialize task
[INFO] 2022-10-07 15:53:04.857 +0000 - Set task status to TaskExecutionStatus{code=1, desc='running'}
[INFO] 2022-10-07 15:53:04.858 +0000 - TenantCode:xichen check success
[INFO] 2022-10-07 15:53:04.859 +0000 - ProcessExecDir:/opt/ds_base_dir/exec/process/7120503728992/7133023937376_2/25/29 check success
[INFO] 2022-10-07 15:53:04.859 +0000 - Resources:{} check success
[INFO] 2022-10-07 15:53:04.860 +0000 - Task plugin: DATA_QUALITY create success
[INFO] 2022-10-07 15:53:04.860 +0000 - data quality task params {"localParams":[],"resourceList":[],"ruleId":6,"ruleInputParameter":{"check_type":"0","comparison_type":1,"comparison_name":"10","failure_strategy":"0","operator":"4","src_connector_type":4,"src_datasource_id":1,"src_field":"c1_name","src_table":"dim_app_content","threshold":"10"},"sparkParameters":{"deployMode":"local","driverCores":1,"driverMemory":"512M","executorCores":2,"executorMemory":"2G","numExecutors":2,"others":"--conf spark.yarn.maxAppAttempts=1"}}
[INFO] 2022-10-07 15:53:04.872 +0000 - Success initialized task plugin instance success
[INFO] 2022-10-07 15:53:04.872 +0000 - Success set taskVarPool: null
[INFO] 2022-10-07 15:53:04.873 +0000 - data quality task command: **${SPARK_HOME2}/bin/spark-submit --master local --driver-cores 1 --driver-memory 512M --num-executors 2 --executor-cores 2 --executor-memory 2G --conf spark.yarn.maxAppAttempts=1 /opt/dolphinscheduler/worker-server/libs/dolphinscheduler-data-quality-3.1.0.jar "{\"name\":\"$t(uniqueness_check)\",\"env\":{\"type\":\"batch\",\"config\":null},\"readers\":[{\"type\":\"JDBC\",\"config\**":{\"database\":\"tech\",\"password\":\"123456\",\"driver\":\"ru.yandex.clickhouse.ClickHouseDriver\",\"user\":\"default\",\"output_table\":\"tech_dim_app_content\",\"table\":\"dim_app_content\",\"url\":\"jdbc:clickhouse://ch001:8123/tech\"} }],\"transformers\":[{\"type\":\"sql\",\"config\":{\"index\":1,\"output_table\":\"duplicate_items\",\"sql\":\"SELECT c1_name FROM tech_dim_app_content group by c1_name having count(*) > 1\"} },{\"type\":\"sql\",\"config\":{\"index\":2,\"output_table\":\"duplicate_count\",\"sql\":\"SELECT COUNT(*) AS d
uplicates FROM duplicate_items\"} }],\"writers\":[{\"type\":\"JDBC\",\"config\":{\"database\":\"ds\",\"password\":\"123456\",\"driver\":\"com.mysql.cj.jdbc.Driver\",\"user\":\"root\",\"table\":\"t_ds_dq_execute_result\",\"url\":\"jdbc:mysql://127.0.0.1:3306/ds?useUnicode=true&characterEncoding=UTF-8&useSSL=false&allowPublicKeyRetrieval=true&allowLoadLocalInfile=false&autoDeserialize=false&allowLocalInfile=false&allowUrlInLocalInfile=false\",\"sql\":\"select 0 as rule_type,'$t(uniqueness_check)' as rule_name,0 as process_definition_id,25 as process_instance_id,29 as task_instance_id,duplicate_count.duplicates AS statistics_value,10 AS comparison_value,1 AS comparison_type,0 as check_type,10 as threshold,4 as operator,0 as failure_strategy,'hdfs://mycluster:8020/user/xichen/data_quality_error_data/0_25_ch' as error_output_path,'2022-10-07 15:53:04' as create_time,'2022-10-07 15:53:04' as update_time from duplicate_count \"} },{\"type\":\"JDBC\",\"config\":{\"database\":\"ds\",\"passwo
rd\":\"123456\",\"driver\":\"com.mysql.cj.jdbc.Driver\",\"user\":\"root\",\"table\":\"t_ds_dq_task_statistics_value\",\"url\":\"jdbc:mysql://127.0.0.1:3306/ds?useUnicode=true&characterEncoding=UTF-8&useSSL=false&allowPublicKeyRetrieval=true&allowLoadLocalInfile=false&autoDeserialize=false&allowLocalInfile=false&allowUrlInLocalInfile=false\",\"sql\":\"select 0 as process_definition_id,29 as task_instance_id,6 as rule_id,'1SYYJMCZZRKHUDNCCFM4ZRXJRQUM4DPPSQW5QMYGOLU=' as unique_code,'duplicate_count.duplicates'AS statistics_name,duplicate_count.duplicates AS statistics_value,'2022-10-07 15:53:04' as data_time,'2022-10-07 15:53:04' as create_time,'2022-10-07 15:53:04' as update_time from duplicate_count\"} },{\"type\":\"hdfs_file\",\"config\":{\"path\":\"hdfs://mycluster:8020/user/xichen/data_quality_error_data/0_25_ch\",\"input_table\":\"duplicate_items\"} }]}"
[INFO] 2022-10-07 15:53:04.873 +0000 - tenantCode user:xichen, task dir:25_29
[INFO] 2022-10-07 15:53:04.873 +0000 - create command file:/opt/ds_base_dir/exec/process/7120503728992/7133023937376_2/25/29/25_29.command
[INFO] 2022-10-07 15:53:04.874 +0000 - command : #!/bin/sh
BASEDIR=$(cd `dirname $0`; pwd)
cd $BASEDIR
source /opt/dolphinscheduler/worker-server/conf/dolphinscheduler_env.sh
${SPARK_HOME2}/bin/spark-submit --master local --driver-cores 1 --driver-memory 512M --num-executors 2 --executor-cores 2 --executor-memory 2G --conf spark.yarn.maxAppAttempts=1 /opt/dolphinscheduler/worker-server/libs/dolphinscheduler-data-quality-3.1.0.jar "{\"name\":\"$t(uniqueness_check)\",\"env\":{\"type\":\"batch\",\"config\":null},\"readers\":[{\"type\":\"JDBC\",\"config\":{\"database\":\"tech\",\"password\":\"123456\",\"driver\":\"ru.yandex.clickhouse.ClickHouseDriver\",\"user\":\"default\",\"output_table\":\"tech_dim_app_content\",\"table\":\"dim_app_content\",\"url\":\"jdbc:clickhouse://ch001:8123/tech\"} }],\"transformers\":[{\"type\":\"sql\",\"config\":{\"index\":1,\"output_table\":\"duplicate_items\",\"sql\":\"SELECT c1_name FROM tech_dim_app_content group by c1_name having count(*) > 1\"} },{\"type\":\"sql\",\"config\":{\"index\":2,\"output_table\":\"duplicate_count\",\"sql\":\"SELECT COUNT(*) AS duplicates FROM duplicate_items\"} }],\"writers\":[{\"type\":\"JDBC\",\
"config\":{\"database\":\"ds\",\"password\":\"123456\",\"driver\":\"com.mysql.cj.jdbc.Driver\",\"user\":\"root\",\"table\":\"t_ds_dq_execute_result\",\"url\":\"jdbc:mysql://127.0.0.1:3306/ds?useUnicode=true&characterEncoding=UTF-8&useSSL=false&allowPublicKeyRetrieval=true&allowLoadLocalInfile=false&autoDeserialize=false&allowLocalInfile=false&allowUrlInLocalInfile=false\",\"sql\":\"select 0 as rule_type,'$t(uniqueness_check)' as rule_name,0 as process_definition_id,25 as process_instance_id,29 as task_instance_id,duplicate_count.duplicates AS statistics_value,10 AS comparison_value,1 AS comparison_type,0 as check_type,10 as threshold,4 as operator,0 as failure_strategy,'hdfs://mycluster:8020/user/xichen/data_quality_error_data/0_25_ch' as error_output_path,'2022-10-07 15:53:04' as create_time,'2022-10-07 15:53:04' as update_time from duplicate_count \"} },{\"type\":\"JDBC\",\"config\":{\"database\":\"ds\",\"password\":\"123456\",\"driver\":\"com.mysql.cj.jdbc.Driver\",\"user\":\"roo
t\",\"table\":\"t_ds_dq_task_statistics_value\",\"url\":\"jdbc:mysql://127.0.0.1:3306/ds?useUnicode=true&characterEncoding=UTF-8&useSSL=false&allowPublicKeyRetrieval=true&allowLoadLocalInfile=false&autoDeserialize=false&allowLocalInfile=false&allowUrlInLocalInfile=false\",\"sql\":\"select 0 as process_definition_id,29 as task_instance_id,6 as rule_id,'1SYYJMCZZRKHUDNCCFM4ZRXJRQUM4DPPSQW5QMYGOLU=' as unique_code,'duplicate_count.duplicates'AS statistics_name,duplicate_count.duplicates AS statistics_value,'2022-10-07 15:53:04' as data_time,'2022-10-07 15:53:04' as create_time,'2022-10-07 15:53:04' as update_time from duplicate_count\"} },{\"type\":\"hdfs_file\",\"config\":{\"path\":\"hdfs://mycluster:8020/user/xichen/data_quality_error_data/0_25_ch\",\"input_table\":\"duplicate_items\"} }]}"
[INFO] 2022-10-07 15:53:04.878 +0000 - task run command: sudo -u xichen sh /opt/ds_base_dir/exec/process/7120503728992/7133023937376_2/25/29/25_29.command
[INFO] 2022-10-07 15:53:04.878 +0000 - process start, process id is: 26016
[INFO] 2022-10-07 15:53:07.238 +0000 - process has exited, execute path:/opt/ds_base_dir/exec/process/7120503728992/7133023937376_2/25/29, processId:26016 ,exitStatusCode:1 ,processWaitForStatus:true ,processExitValue:1
[INFO] 2022-10-07 15:53:07.241 +0000 - Send task execute result to master, the current task status: TaskExecutionStatus{code=6, desc='failure'}
[INFO] 2022-10-07 15:53:07.241 +0000 - Remove the current task execute context from worker cache
[INFO] 2022-10-07 15:53:07.241 +0000 - The current execute mode isn't develop mode, will clear the task execute file: /opt/ds_base_dir/exec/process/7120503728992/7133023937376_2/25/29
[INFO] 2022-10-07 15:53:07.242 +0000 - Success clear the task execute file: /opt/ds_base_dir/exec/process/7120503728992/7133023937376_2/25/29
[INFO] 2022-10-07 15:53:07.879 +0000 - -> 2022-10-07 23:53:06,923 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
Exception in thread "main" org.apache.spark.SparkException: No main class set in JAR; please specify one with --class.
at org.apache.spark.deploy.SparkSubmit.error(SparkSubmit.scala:972)
at org.apache.spark.deploy.SparkSubmit.prepareSubmitEnvironment(SparkSubmit.scala:492)
at org.apache.spark.deploy.SparkSubmit.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:898)
at org.apache.spark.deploy.SparkSubmit.doRunMain$1(SparkSubmit.scala:180)
at org.apache.spark.deploy.SparkSubmit.submit(SparkSubmit.scala:203)
at org.apache.spark.deploy.SparkSubmit.doSubmit(SparkSubmit.scala:90)
at org.apache.spark.deploy.SparkSubmit$$anon$2.doSubmit(SparkSubmit.scala:1043)
at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:1052)
at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
[INFO] 2022-10-07 15:53:07.881 +0000 - FINALIZE_SESSION
### What you expected to happen
[Data Quality] No main class set in JAR
### How to reproduce
[Data Quality] No main class set in JAR
### Anything else
_No response_
### Version
3.1.x
### Are you willing to submit PR?
- [ ] Yes I am willing to submit a PR!
### Code of Conduct
- [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] pandong2011 commented on issue #12257: [Bug] [Data Quality] No main class set in JAR
Posted by GitBox <gi...@apache.org>.
pandong2011 commented on issue #12257:
URL: https://github.com/apache/dolphinscheduler/issues/12257#issuecomment-1287660284
> > > > > >
> > > > >
> > > > >
> > > > > > There are still some serious problems，
> > > > > > such as hdfs namenode addresss, the default value is mycluser:8020, and cannot modified。
> > > > > > Another problem， the jars dependencies， are not packaged in the data quality jar
> > > > >
> > > > >
> > > > > There are still some serious problems， such as hdfs namenode addresss, the default value is mycluser:8020, and cannot modified。 Another problem， the jars dependencies， are not packaged in the data quality jar
> > > >
> > > >
> > > > the jars you should modify the config , refer https://dolphinscheduler.apache.org/zh-cn/docs/latest/user_doc/guide/data-quality.html. /conf/common.properties data-quality.jar.name=dolphinscheduler-data-quality-dev-SNAPSHOT.jar 
> > > > make sure your path in libs has this name's jar 
> > > > at the end , you will get this.
> > > > 
> > >
> > >
> > > But this jar is not a fat jar, such as clickhouse and mysql driver are not contained
> >
> >
> > why not you copy mysql driver to your spark_home
>
> mysql is ok, but clickhouse depends on more jar packages
You can make it a fat jar by modifying the configuration of source pom.xml
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] zcw5116 commented on issue #12257: [Bug] [Data Quality] No main class set in JAR
Posted by GitBox <gi...@apache.org>.
zcw5116 commented on issue #12257:
URL: https://github.com/apache/dolphinscheduler/issues/12257#issuecomment-1274959794
>
> There are still some serious problems，
> such as hdfs namenode addresss, the default value is mycluser:8020, and cannot modified。
> Another problem， the jars dependencies， are not packaged in the data quality jar
There are still some serious problems，
such as hdfs namenode addresss, the default value is mycluser:8020, and cannot modified。
Another problem， the jars dependencies， are not packaged in the data quality jar
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] zcw5116 commented on issue #12257: [Bug] [Data Quality] No main class set in JAR
Posted by GitBox <gi...@apache.org>.
zcw5116 commented on issue #12257:
URL: https://github.com/apache/dolphinscheduler/issues/12257#issuecomment-1274949361
> you should add param like this:
>
> --conf spark.yarn.maxAppAttempts=1 --class org.apache.dolphinscheduler.data.quality.DataQualityApplication
It's not ok, There are still some serious problems，
such as hdfs namenode addresss, the default value is mycluser:8020, and cannot modified。
Another problem， the jars dependencies， are not packaged in the data quality jar
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] tjudadc commented on issue #12257: [Bug] [Data Quality] No main class set in JAR
Posted by GitBox <gi...@apache.org>.
tjudadc commented on issue #12257:
URL: https://github.com/apache/dolphinscheduler/issues/12257#issuecomment-1274589376
--conf spark.yarn.maxAppAttempts=1 --class org.apache.dolphinscheduler.data.quality.DataQualityApplication
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] zcw5116 commented on issue #12257: [Bug] [Data Quality] No main class set in JAR
Posted by GitBox <gi...@apache.org>.
zcw5116 commented on issue #12257:
URL: https://github.com/apache/dolphinscheduler/issues/12257#issuecomment-1274958723
>
> There are still some serious problems，
> such as hdfs namenode addresss, the default value is mycluser:8020, and cannot modified。
> Another problem， the jars dependencies， are not packaged in the data quality jar
There are still some serious problems，
such as hdfs namenode addresss, the default value is mycluser:8020, and cannot modified。
Another problem， the jars dependencies， are not packaged in the data quality jar
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] tjudadc commented on issue #12257: [Bug] [Data Quality] No main class set in JAR
Posted by GitBox <gi...@apache.org>.
tjudadc commented on issue #12257:
URL: https://github.com/apache/dolphinscheduler/issues/12257#issuecomment-1275432451
> >
>
> > There are still some serious problems，
> > such as hdfs namenode addresss, the default value is mycluser:8020, and cannot modified。
> > Another problem， the jars dependencies， are not packaged in the data quality jar
>
> There are still some serious problems， such as hdfs namenode addresss, the default value is mycluser:8020, and cannot modified。 Another problem， the jars dependencies， are not packaged in the data quality jar
you should modify the hdfs namenode addresss. my dolphhinscheduler home is /home/dolphincheduler/dolphin,which is the installed path,not the download path.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] zhoujingmin commented on issue #12257: [Bug] [Data Quality] No main class set in JAR
Posted by GitBox <gi...@apache.org>.
zhoujingmin commented on issue #12257:
URL: https://github.com/apache/dolphinscheduler/issues/12257#issuecomment-1274029668
Exception in thread "main" org.apache.spark.SparkException: Cannot load main class from JAR file
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] zcw5116 commented on issue #12257: [Bug] [Data Quality] No main class set in JAR
Posted by GitBox <gi...@apache.org>.
zcw5116 commented on issue #12257:
URL: https://github.com/apache/dolphinscheduler/issues/12257#issuecomment-1275466183
> > >
> >
> >
> > > There are still some serious problems，
> > > such as hdfs namenode addresss, the default value is mycluser:8020, and cannot modified。
> > > Another problem， the jars dependencies， are not packaged in the data quality jar
> >
> >
> > There are still some serious problems， such as hdfs namenode addresss, the default value is mycluser:8020, and cannot modified。 Another problem， the jars dependencies， are not packaged in the data quality jar
>
> the jars you should modify the config , refer https://dolphinscheduler.apache.org/zh-cn/docs/latest/user_doc/guide/data-quality.html. /conf/common.properties data-quality.jar.name=dolphinscheduler-data-quality-dev-SNAPSHOT.jar 
>
> make sure your path in libs has this name's jar 
>
> at the end , you will get this.
>
> 
But this jar is not a fat jar, such as clickhouse and mysql driver are not contained
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] tjudadc commented on issue #12257: [Bug] [Data Quality] No main class set in JAR
Posted by GitBox <gi...@apache.org>.
tjudadc commented on issue #12257:
URL: https://github.com/apache/dolphinscheduler/issues/12257#issuecomment-1275436242
> >
>
> > There are still some serious problems，
> > such as hdfs namenode addresss, the default value is mycluser:8020, and cannot modified。
> > Another problem， the jars dependencies， are not packaged in the data quality jar
>
> There are still some serious problems， such as hdfs namenode addresss, the default value is mycluser:8020, and cannot modified。 Another problem， the jars dependencies， are not packaged in the data quality jar
the jars you should modify the config , refer https://dolphinscheduler.apache.org/zh-cn/docs/latest/user_doc/guide/data-quality.html.
<server-name>/conf/common.properties
data-quality.jar.name=dolphinscheduler-data-quality-dev-SNAPSHOT.jar
make sure your path in libs has this name's jar
at the end , you will get this.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] zcw5116 commented on issue #12257: [Bug] [Data Quality] No main class set in JAR
Posted by GitBox <gi...@apache.org>.
zcw5116 commented on issue #12257:
URL: https://github.com/apache/dolphinscheduler/issues/12257#issuecomment-1275487981
> > > > >
> > > >
> > > >
> > > > > There are still some serious problems，
> > > > > such as hdfs namenode addresss, the default value is mycluser:8020, and cannot modified。
> > > > > Another problem， the jars dependencies， are not packaged in the data quality jar
> > > >
> > > >
> > > > There are still some serious problems， such as hdfs namenode addresss, the default value is mycluser:8020, and cannot modified。 Another problem， the jars dependencies， are not packaged in the data quality jar
> > >
> > >
> > > the jars you should modify the config , refer https://dolphinscheduler.apache.org/zh-cn/docs/latest/user_doc/guide/data-quality.html. /conf/common.properties data-quality.jar.name=dolphinscheduler-data-quality-dev-SNAPSHOT.jar 
> > > make sure your path in libs has this name's jar 
> > > at the end , you will get this.
> > > 
> >
> >
> > But this jar is not a fat jar, such as clickhouse and mysql driver are not contained
>
> why not you copy mysql driver to your spark_home
mysql is ok, but clickhouse depends on more jar packages
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] github-actions[bot] commented on issue #12257: [Bug] [Data Quality] No main class set in JAR
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on issue #12257:
URL: https://github.com/apache/dolphinscheduler/issues/12257#issuecomment-1271778532
Thank you for your feedback, we have received your issue, Please wait patiently for a reply.
* In order for us to understand your request as soon as possible, please provide detailed information、version or pictures.
* If you haven't received a reply for a long time, you can [join our slack](https://s.apache.org/dolphinscheduler-slack) and send your question to channel `#troubleshooting`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] tjudadc commented on issue #12257: [Bug] [Data Quality] No main class set in JAR
Posted by GitBox <gi...@apache.org>.
tjudadc commented on issue #12257:
URL: https://github.com/apache/dolphinscheduler/issues/12257#issuecomment-1275486804
> > > >
> > >
> > >
> > > > There are still some serious problems，
> > > > such as hdfs namenode addresss, the default value is mycluser:8020, and cannot modified。
> > > > Another problem， the jars dependencies， are not packaged in the data quality jar
> > >
> > >
> > > There are still some serious problems， such as hdfs namenode addresss, the default value is mycluser:8020, and cannot modified。 Another problem， the jars dependencies， are not packaged in the data quality jar
> >
> >
> > the jars you should modify the config , refer https://dolphinscheduler.apache.org/zh-cn/docs/latest/user_doc/guide/data-quality.html. /conf/common.properties data-quality.jar.name=dolphinscheduler-data-quality-dev-SNAPSHOT.jar 
> > make sure your path in libs has this name's jar 
> > at the end , you will get this.
> > 
>
> But this jar is not a fat jar, such as clickhouse and mysql driver are not contained
why not you copy mysql driver to your spark_home
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] pandong2011 commented on issue #12257: [Bug] [Data Quality] No main class set in JAR
Posted by GitBox <gi...@apache.org>.
pandong2011 commented on issue #12257:
URL: https://github.com/apache/dolphinscheduler/issues/12257#issuecomment-1287659670
> > > > >
> > > >
> > > >
> > > > > There are still some serious problems，
> > > > > such as hdfs namenode addresss, the default value is mycluser:8020, and cannot modified。
> > > > > Another problem， the jars dependencies， are not packaged in the data quality jar
> > > >
> > > >
> > > > There are still some serious problems， such as hdfs namenode addresss, the default value is mycluser:8020, and cannot modified。 Another problem， the jars dependencies， are not packaged in the data quality jar
> > >
> > >
> > > the jars you should modify the config , refer https://dolphinscheduler.apache.org/zh-cn/docs/latest/user_doc/guide/data-quality.html. /conf/common.properties data-quality.jar.name=dolphinscheduler-data-quality-dev-SNAPSHOT.jar 
> > > make sure your path in libs has this name's jar 
> > > at the end , you will get this.
> > > 
> >
> >
> > But this jar is not a fat jar, such as clickhouse and mysql driver are not contained
>
> why not you copy mysql driver to your spark_home
Copying is not the solution because the jar is run with internal build commands and should be a fat jar
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] ruanwenjun closed issue #12257: [Bug] [Data Quality] No main class set in JAR
Posted by GitBox <gi...@apache.org>.
ruanwenjun closed issue #12257: [Bug] [Data Quality] No main class set in JAR
URL: https://github.com/apache/dolphinscheduler/issues/12257
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org