You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@dolphinscheduler.apache.org by GitBox <gi...@apache.org> on 2022/11/09 08:22:35 UTC
[GitHub] [dolphinscheduler] lenian opened a new issue, #12837: [Bug] [SqlTask] sql parses global variables having single quotes
lenian opened a new issue, #12837:
URL: https://github.com/apache/dolphinscheduler/issues/12837
### Search before asking
- [X] I had searched in the [issues](https://github.com/apache/dolphinscheduler/issues?q=is%3Aissue) and found no similar issues.
### What happened
**global variable：**current_time : ${system.datetime}
**SqlTask definition：**ALTER TABLE cdc_test.lab_report RENAME cdc_test.lab_report_archive_${current_time};
**SqlTask parse result：**ALTER TABLE cdc_test.lab_report RENAME cdc_test.lab_report_archive_'20221109155439';
**operation screenshot ：**
**Run result log：**
[LOG-PATH]: E:\MananaCare\code\demo\apache-dolphin2\logs\20221109\7494408333056_1-4-7.log, [HOST]: Host{address='10.168.5.187:1234', ip='10.168.5.187', port=1234}
[INFO] 2022-11-09 15:54:39.277 +0800 - Begin to pulling task
[INFO] 2022-11-09 15:54:39.281 +0800 - Begin to initialize task
[INFO] 2022-11-09 15:54:39.281 +0800 - Set task startTime: 1667980479281
[INFO] 2022-11-09 15:54:39.304 +0800 - Set task envFile: /etc/profile
[INFO] 2022-11-09 15:54:39.304 +0800 - Set task appId: 4_7
[INFO] 2022-11-09 15:54:39.305 +0800 - End initialize task
[INFO] 2022-11-09 15:54:39.322 +0800 - Set task status to TaskExecutionStatus{code=1, desc='running'}
[INFO] 2022-11-09 15:54:42.618 +0800 - TenantCode:lenian check success
[INFO] 2022-11-09 15:54:42.622 +0800 - ProcessExecDir:/tmp/dolphinscheduler/exec/process/lenian/7486873665920/7494408333056_1/4/7 check success
[INFO] 2022-11-09 15:54:42.626 +0800 - Resources:{} check success
[INFO] 2022-11-09 15:54:42.638 +0800 - Task plugin: SQL create success
[INFO] 2022-11-09 15:54:42.638 +0800 - Success initialized task plugin instance success
[INFO] 2022-11-09 15:54:42.639 +0800 - Success set taskVarPool: null
[INFO] 2022-11-09 15:54:42.640 +0800 - Full sql parameters: SqlParameters{type='MYSQL', datasource=3, sql='ALTER TABLE cdc_test.lab_report RENAME cdc_test.lab_report_archive_${current_time};', sqlType=1, sendEmail=null, displayRows=10, limit=0, udfs='null', showType='null', connParams='null', groupId='0', title='null', preStatements=[], postStatements=[]}
[INFO] 2022-11-09 15:54:42.640 +0800 - sql type : MYSQL, datasource : 3, sql : ALTER TABLE cdc_test.lab_report RENAME cdc_test.lab_report_archive_${current_time}; , localParams : [],udfs : null,showType : null,connParams : null,varPool : [] ,query max result limit 0
[INFO] 2022-11-09 15:54:42.684 +0800 - setSqlParamsMap: Property with paramName: current_time put in sqlParamsMap of content ALTER TABLE cdc_test.lab_report RENAME cdc_test.lab_report_archive_${current_time}; successfully.
[INFO] 2022-11-09 15:54:42.684 +0800 - after replace sql , preparing : ALTER TABLE cdc_test.lab_report RENAME cdc_test.lab_report_archive_?;
[INFO] 2022-11-09 15:54:42.684 +0800 - Sql Params are replaced sql , parameters:20221109155439(VARCHAR)
[INFO] 2022-11-09 15:54:42.687 +0800 - can't find udf function resource
[INFO] 2022-11-09 15:54:42.816 +0800 - prepare statement replace sql : ALTER TABLE cdc_test.lab_report RENAME cdc_test.lab_report_archive_?;, sql parameters : {1=Property{prop='current_time', direct=IN, type=VARCHAR, value='20221109155439'}}
[ERROR] 2022-11-09 15:54:42.829 +0800 - execute sql error: You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ''20221109155439'' at line 1
[ERROR] 2022-11-09 15:54:42.830 +0800 - sql task error
java.sql.SQLSyntaxErrorException: You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ''20221109155439'' at line 1
at com.mysql.cj.jdbc.exceptions.SQLError.createSQLException(SQLError.java:120)
at com.mysql.cj.jdbc.exceptions.SQLError.createSQLException(SQLError.java:97)
at com.mysql.cj.jdbc.exceptions.SQLExceptionsMapping.translateException(SQLExceptionsMapping.java:122)
at com.mysql.cj.jdbc.ClientPreparedStatement.executeInternal(ClientPreparedStatement.java:955)
at com.mysql.cj.jdbc.ClientPreparedStatement.executeUpdateInternal(ClientPreparedStatement.java:1094)
at com.mysql.cj.jdbc.ClientPreparedStatement.executeUpdateInternal(ClientPreparedStatement.java:1042)
at com.mysql.cj.jdbc.ClientPreparedStatement.executeLargeUpdate(ClientPreparedStatement.java:1345)
at com.mysql.cj.jdbc.ClientPreparedStatement.executeUpdate(ClientPreparedStatement.java:1027)
at com.zaxxer.hikari.pool.ProxyPreparedStatement.executeUpdate(ProxyPreparedStatement.java:61)
at com.zaxxer.hikari.pool.HikariProxyPreparedStatement.executeUpdate(HikariProxyPreparedStatement.java)
at org.apache.dolphinscheduler.plugin.task.sql.SqlTask.executeUpdate(SqlTask.java:379)
at org.apache.dolphinscheduler.plugin.task.sql.SqlTask.executeFuncAndSql(SqlTask.java:250)
at org.apache.dolphinscheduler.plugin.task.sql.SqlTask.handle(SqlTask.java:175)
at org.apache.dolphinscheduler.server.worker.runner.DefaultWorkerDelayTaskExecuteRunnable.executeTask(DefaultWorkerDelayTaskExecuteRunnable.java:51)
at org.apache.dolphinscheduler.server.worker.runner.WorkerTaskExecuteRunnable.run(WorkerTaskExecuteRunnable.java:180)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at com.google.common.util.concurrent.TrustedListenableFutureTask$TrustedFutureInterruptibleTask.runInterruptibly(TrustedListenableFutureTask.java:131)
at com.google.common.util.concurrent.InterruptibleTask.run(InterruptibleTask.java:74)
at com.google.common.util.concurrent.TrustedListenableFutureTask.run(TrustedListenableFutureTask.java:82)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
[ERROR] 2022-11-09 15:54:42.831 +0800 - Task execute failed, due to meet an exception
org.apache.dolphinscheduler.plugin.task.api.TaskException: Execute sql task failed
at org.apache.dolphinscheduler.plugin.task.sql.SqlTask.handle(SqlTask.java:182)
at org.apache.dolphinscheduler.server.worker.runner.DefaultWorkerDelayTaskExecuteRunnable.executeTask(DefaultWorkerDelayTaskExecuteRunnable.java:51)
at org.apache.dolphinscheduler.server.worker.runner.WorkerTaskExecuteRunnable.run(WorkerTaskExecuteRunnable.java:180)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at com.google.common.util.concurrent.TrustedListenableFutureTask$TrustedFutureInterruptibleTask.runInterruptibly(TrustedListenableFutureTask.java:131)
at com.google.common.util.concurrent.InterruptibleTask.run(InterruptibleTask.java:74)
at com.google.common.util.concurrent.TrustedListenableFutureTask.run(TrustedListenableFutureTask.java:82)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.sql.SQLSyntaxErrorException: You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ''20221109155439'' at line 1
at com.mysql.cj.jdbc.exceptions.SQLError.createSQLException(SQLError.java:120)
at com.mysql.cj.jdbc.exceptions.SQLError.createSQLException(SQLError.java:97)
at com.mysql.cj.jdbc.exceptions.SQLExceptionsMapping.translateException(SQLExceptionsMapping.java:122)
at com.mysql.cj.jdbc.ClientPreparedStatement.executeInternal(ClientPreparedStatement.java:955)
at com.mysql.cj.jdbc.ClientPreparedStatement.executeUpdateInternal(ClientPreparedStatement.java:1094)
at com.mysql.cj.jdbc.ClientPreparedStatement.executeUpdateInternal(ClientPreparedStatement.java:1042)
at com.mysql.cj.jdbc.ClientPreparedStatement.executeLargeUpdate(ClientPreparedStatement.java:1345)
at com.mysql.cj.jdbc.ClientPreparedStatement.executeUpdate(ClientPreparedStatement.java:1027)
at com.zaxxer.hikari.pool.ProxyPreparedStatement.executeUpdate(ProxyPreparedStatement.java:61)
at com.zaxxer.hikari.pool.HikariProxyPreparedStatement.executeUpdate(HikariProxyPreparedStatement.java)
at org.apache.dolphinscheduler.plugin.task.sql.SqlTask.executeUpdate(SqlTask.java:379)
at org.apache.dolphinscheduler.plugin.task.sql.SqlTask.executeFuncAndSql(SqlTask.java:250)
at org.apache.dolphinscheduler.plugin.task.sql.SqlTask.handle(SqlTask.java:175)
... 9 common frames omitted
[INFO] 2022-11-09 15:54:42.834 +0800 - Get a exception when execute the task, will send the task execute result to master, the current task execute result is TaskExecutionStatus{code=6, desc='failure'}
### What you expected to happen
**global variable：**current_time : ${system.datetime}
**SqlTask definition：**ALTER TABLE cdc_test.lab_report RENAME cdc_test.lab_report_archive_${current_time};
**SqlTask parse result：**ALTER TABLE cdc_test.lab_report RENAME **cdc_test.lab_report_archive_20221109155439**;
### How to reproduce
**global variable：**current_time : ${system.datetime}
**SqlTask definition：**ALTER TABLE cdc_test.lab_report RENAME cdc_test.lab_report_archive_${current_time};
**SqlTask parse result：**ALTER TABLE cdc_test.lab_report RENAME cdc_test.lab_report_archive_'20221109155439';
### Anything else
_No response_
### Version
dev
### Are you willing to submit PR?
- [X] Yes I am willing to submit a PR!
### Code of Conduct
- [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] github-actions[bot] commented on issue #12837: [Bug] [SqlTask] sql parses global variables having single quotes
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on issue #12837:
URL: https://github.com/apache/dolphinscheduler/issues/12837#issuecomment-1364419121
This issue has been closed because it has not received response for too long time. You could reopen it if you encountered similar problems in the future.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] github-actions[bot] closed issue #12837: [Bug] [SqlTask] sql parses global variables having single quotes
Posted by GitBox <gi...@apache.org>.
github-actions[bot] closed issue #12837: [Bug] [SqlTask] sql parses global variables having single quotes
URL: https://github.com/apache/dolphinscheduler/issues/12837
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] github-actions[bot] commented on issue #12837: [Bug] [SqlTask] sql parses global variables having single quotes
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on issue #12837:
URL: https://github.com/apache/dolphinscheduler/issues/12837#issuecomment-1308381075
Thank you for your feedback, we have received your issue, Please wait patiently for a reply.
* In order for us to understand your request as soon as possible, please provide detailed information、version or pictures.
* If you haven't received a reply for a long time, you can [join our slack](https://s.apache.org/dolphinscheduler-slack) and send your question to channel `#troubleshooting`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] github-actions[bot] commented on issue #12837: [Bug] [SqlTask] sql parses global variables having single quotes
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on issue #12837:
URL: https://github.com/apache/dolphinscheduler/issues/12837#issuecomment-1353934436
This issue has been automatically marked as stale because it has not had recent activity for 30 days. It will be closed in next 7 days if no further activity occurs.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] Radeity commented on issue #12837: [Bug] [SqlTask] sql parses global variables having single quotes
Posted by GitBox <gi...@apache.org>.
Radeity commented on issue #12837:
URL: https://github.com/apache/dolphinscheduler/issues/12837#issuecomment-1315608947
@lenian Hi, the problem is in global parameter setting, the default type is `VARCHAR`, which causes the sql variable surrounded by single quotes. You can use local parameter in task node and set `LONG` type for the parameter.
If it helps, please close this issue. I'll open another issue to discuss about whether add type definition in global parameters.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org