You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@dolphinscheduler.apache.org by GitBox <gi...@apache.org> on 2020/09/29 10:23:51 UTC
[GitHub] [incubator-dolphinscheduler] zosimer opened a new issue #3855: dolphinscheduler 连接spark-thriftserver执行sql时，动态分区没有成功，不提示任何错误，任务瞬间执行完成
zosimer opened a new issue #3855:
URL: https://github.com/apache/incubator-dolphinscheduler/issues/3855
以下是sql语句：
set hive.exec.dynamic.partition=true;
set hive.exec.dynamic.partition.mode=nonstrict;
set hive.exec.max.dynamic.partitions.pernode=600000;
set hive.exec.max.dynamic.partitions=6000000;
set hive.exec.max.created.files=6000000;
INSERT overwrite TABLE ods.ods_app_report_vo PARTITION(month,day) as
select a.*,substr(a.`date`,1,7) as month,substr(a.`date`,1,10) as day from ods.ods_app_report_vo_np as a where substr(a.`date`,1,10) = '2020-09-28'
下面是日志：
[INFO] 2020-09-29 18:13:13.171 - [taskAppId=TASK-9-26-189]:[176] - SQL title : 测试动态分区
[INFO] 2020-09-29 18:13:13.172 - [taskAppId=TASK-9-26-189]:[499] - after replace sql , preparing : set hive.exec.dynamic.partition=true;
set hive.exec.dynamic.partition.mode=nonstrict;
set hive.exec.max.dynamic.partitions.pernode=600000;
set hive.exec.max.dynamic.partitions=6000000;
set hive.exec.max.created.files=6000000;
INSERT overwrite TABLE ods.ods_app_report_vo PARTITION(month,day) as
select a.*,substr(a.`date`,1,7) as month,substr(a.`date`,1,10) as day from ods.ods_app_report_vo_np as a where substr(a.`date`,1,10) = '2020-09-28'
[INFO] 2020-09-29 18:13:13.172 - [taskAppId=TASK-9-26-189]:[504] - Sql Params are replaced sql , parameters:
[INFO] 2020-09-29 18:13:13.172 - [taskAppId=TASK-9-26-189]:[52] - can't find udf function resource
[INFO] 2020-09-29 18:13:13.236 - [taskAppId=TASK-9-26-189]:[418] - prepare statement replace sql : org.apache.hive.jdbc.HivePreparedStatement@30cc6d1e
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-dolphinscheduler] zosimer commented on issue #3855: When the dolphinscheduler connects with spark thriftserver to execute SQL, the dynamic partition is not successful, and the data does not enter the target partition table. No error is prompted. The task is completed in an instant and is shown to be successful
Posted by GitBox <gi...@apache.org>.
zosimer commented on issue #3855:
URL: https://github.com/apache/incubator-dolphinscheduler/issues/3855#issuecomment-701441788
The same is true of hivesql
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-dolphinscheduler] xingchun-chen commented on issue #3855: dolphinscheduler 连接spark-thriftserver执行sql时，动态分区没有成功，不提示任何错误，任务瞬间执行完成
Posted by GitBox <gi...@apache.org>.
xingchun-chen commented on issue #3855:
URL: https://github.com/apache/incubator-dolphinscheduler/issues/3855#issuecomment-701216050
please update the title in English
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-dolphinscheduler] zosimer commented on issue #3855: When the dolphinscheduler connects with spark thriftserver to execute SQL, the dynamic partition is not successful, and the data does not enter the target partition table. No error is prompted. The task is completed in an instant and is shown to be successful
Posted by GitBox <gi...@apache.org>.
zosimer commented on issue #3855:
URL: https://github.com/apache/incubator-dolphinscheduler/issues/3855#issuecomment-701418767
Using the spark SQL client to execute the SQL is successful
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-dolphinscheduler] zosimer commented on issue #3855: When the dolphinscheduler connects with spark thriftserver to execute SQL, the dynamic partition is not successful, and the data does not enter the target partition table. No error is prompted. The task is completed in an instant and is shown to be successful
Posted by GitBox <gi...@apache.org>.
zosimer commented on issue #3855:
URL: https://github.com/apache/incubator-dolphinscheduler/issues/3855#issuecomment-701432821
dolphinscheduler1.3.2
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-dolphinscheduler] geosmart commented on issue #3855: dolphinscheduler 连接spark-thriftserver执行sql时，动态分区没有成功，不提示任何错误，任务瞬间执行完成
Posted by GitBox <gi...@apache.org>.
geosmart commented on issue #3855:
URL: https://github.com/apache/incubator-dolphinscheduler/issues/3855#issuecomment-701108899
please check the worker log, you may find same error message
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org