You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@dolphinscheduler.apache.org by GitBox <gi...@apache.org> on 2022/01/17 13:16:24 UTC
[GitHub] [dolphinscheduler] seagle-yuan opened a new issue #8092: [Bug] [task-datax] cannot use Parameter for clickhouse datax task
seagle-yuan opened a new issue #8092:
URL: https://github.com/apache/dolphinscheduler/issues/8092
### Search before asking
- [X] I had searched in the [issues](https://github.com/apache/dolphinscheduler/issues?q=is%3Aissue) and found no similar issues.
### What happened
when I used Parameter for clickhouse datax task. it was always failed .
### What you expected to happen
[WARN] 2022-01-17 20:19:50.042 - [taskAppId=TASK-540354961620992_3-13328-48308]:[480] - database driver [CLICKHOUSE] is not support grammatical analysis sql
[INFO] 2022-01-17 20:19:50.042 - [taskAppId=TASK-540354961620992_3-13328-48308]:[457] - try to execute sql analysis query column name
.......
ru.yandex.clickhouse.except.ClickHouseException: ClickHouse exception, code: 41, host: 10.17.16.46, port: 8123; Code: 41, e.displayText() = DB::ParsingException: Cannot parse datetime: Cannot parse UInt32 from String: While processing (paymentType != 'yb') AND (datatime >= (toUnixTimestamp(concat('${dtdh}', ':00:00')) * 1000)) AND (datatime < (toUnixTimestamp(concat('2022-01-17 19', ':59:59')) * 1000)) (version 21.6.6.51 (official build))
at ru.yandex.clickhouse.except.ClickHouseExceptionSpecifier.specify(ClickHouseExceptionSpecifier.java:58)
at ru.yandex.clickhouse.except.ClickHouseExceptionSpecifier.specify(ClickHouseExceptionSpecifier.java:28)
at ru.yandex.clickhouse.ClickHouseStatementImpl.checkForErrorAndThrow(ClickHouseStatementImpl.java:815)
at ru.yandex.clickhouse.ClickHouseStatementImpl.getInputStream(ClickHouseStatementImpl.java:611)
at ru.yandex.clickhouse.ClickHouseStatementImpl.executeQuery(ClickHouseStatementImpl.java:114)
at ru.yandex.clickhouse.ClickHouseStatementImpl.executeQuery(ClickHouseStatementImpl.java:97)
at ru.yandex.clickhouse.ClickHouseStatementImpl.executeQuery(ClickHouseStatementImpl.java:92)
at ru.yandex.clickhouse.ClickHouseStatementImpl.executeQuery(ClickHouseStatementImpl.java:87)
at ru.yandex.clickhouse.ClickHousePreparedStatementImpl.executeQuery(ClickHousePreparedStatementImpl.java:128)
at org.apache.dolphinscheduler.plugin.task.datax.DataxTask.tryExecuteSqlResolveColumnNames(DataxTask.java:552)
at org.apache.dolphinscheduler.plugin.task.datax.DataxTask.parsingSqlColumnNames(DataxTask.java:458)
at org.apache.dolphinscheduler.plugin.task.datax.DataxTask.buildDataxJobContentJson(DataxTask.java:279)
at org.apache.dolphinscheduler.plugin.task.datax.DataxTask.buildDataxJsonFile(DataxTask.java:210)
at org.apache.dolphinscheduler.plugin.task.datax.DataxTask.handle(DataxTask.java:161)
at org.apache.dolphinscheduler.server.worker.runner.TaskExecuteThread.run(TaskExecuteThread.java:226)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.Throwable: Code: 41, e.displayText() = DB::ParsingException: Cannot parse datetime: Cannot parse UInt32 from String: While processing (paymentType != 'yb') AND (datatime >= (toUnixTimestamp(concat('${dtdh}', ':00:00')) * 1000)) AND (datatime < (toUnixTimestamp(concat('2022-01-17 19', ':59:59')) * 1000)) (version 21.6.6.51 (official build))
at ru.yandex.clickhouse.except.ClickHouseExceptionSpecifier.specify(ClickHouseExceptionSpecifier.java:53)
... 19 common frames omitted
### How to reproduce
create clickhouse datax task. AND use time Parameter In "WHERE " filter
### Anything else
I had fix this bug.
please edit DataxUtils.class and update getSqlStatementParser function.
>
public static SQLStatementParser getSqlStatementParser(DbType dbType, String sql) {
switch (dbType) {
case MYSQL:
return new MySqlStatementParser(sql);
case POSTGRESQL:
return new PGSQLStatementParser(sql);
case ORACLE:
return new OracleStatementParser(sql);
case SQLSERVER:
return new SQLServerStatementParser(sql);
case CLICKHOUSE:
return new ClickhouseStatementParser(sql);
default:
return null;
}
}
### Version
2.0.2
### Are you willing to submit PR?
- [X] Yes I am willing to submit a PR!
### Code of Conduct
- [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] github-actions[bot] commented on issue #8092: [Bug] [task-datax] cannot use Parameter for clickhouse datax task
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on issue #8092:
URL: https://github.com/apache/dolphinscheduler/issues/8092#issuecomment-1014525292
Hi:
* Thank you for your feedback, we have received your issue, Please wait patiently for a reply.
* In order for us to understand your request as soon as possible, please provide detailed information、version or pictures.
* If you haven't received a reply for a long time, you can subscribe to the developer's email，Mail subscription steps reference https://dolphinscheduler.apache.org/en-us/community/development/subscribe.html ,Then write the issue URL in the email content and send question to dev@dolphinscheduler.apache.org.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] zhuangchong closed issue #8092: [Bug] [task-datax] cannot use Parameter for clickhouse datax task
Posted by GitBox <gi...@apache.org>.
zhuangchong closed issue #8092:
URL: https://github.com/apache/dolphinscheduler/issues/8092
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org