You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@dolphinscheduler.apache.org by GitBox <gi...@apache.org> on 2021/10/06 23:20:16 UTC
[GitHub] [dolphinscheduler] sakinsun opened a new issue #6451: [Bug] [DataX] 从mysql读取数据写入到SqlServer 2012数据库 (DS版本：1.3.6)
sakinsun opened a new issue #6451:
URL: https://github.com/apache/dolphinscheduler/issues/6451
### Search before asking
- [X] I had searched in the [issues](https://github.com/apache/dolphinscheduler/issues?q=is%3Aissue) and found no similar issues.
### What happened
我在DS 1.3.6版本中，使用datax插件，数据源为mysql，目标源为sqlserver，使用的是 "非自定义模板"的方式，就是 mysql的地方写sql语句，目标sqlserver地方直接填写表名，然后运行这个节点。结果运行报错，报错内容如下：
”Description:[获取表字段相关信息失败.]. - 获取表:test_tmp_20211001 的字段的元信息时失败. 请联系 DBA 核查该库、表信息. - com.microsoft.sqlserver.jdbc.SQLServerException: “`”附近有语法错误。”
DataX在DS下运行后，日志中可以看到在获取sqlserver元数据字段的时候是这样的：
"writer":{
"name":"sqlserverwriter",
"parameter":{
"column":[
"`suppid`"
]
sqlserver表字段是用 ` 这个号引起来的，导致在sqlserver里面查询元数据报错，DS源码如下：
switch (dbType) {
case MYSQL:
return String.format("`%s`", column);
case POSTGRESQL:
return String.format("\"%s\"", column);
case ORACLE:
return String.format("\"%s\"", column);
case SQLSERVER:
return String.format("`%s`", column); --------- 就是这里！！！！
default:
return column;
}
}
源码所在github位置：dolphinscheduler-task-plugin/dolphinscheduler-task-datax/src/main/java/org/apache/dolphinscheduler/plugin/task/datax/DataxUtils.java
### What you expected to happen
如上述，在DS中同样使用datax插件 且使用"自定义模板"的方式，贴好配置信息及表字段，并且字段 无 ` 引号的情况下，mysql数据顺利写入sqlserver中，写入成功。
### How to reproduce
改用"自定义模板"，但是使用了这个，那界面图形化就失去了意义，而且通过贴配置信息很不方便，每次都要贴账号密码及链接。
### Anything else
_No response_
### Are you willing to submit PR?
- [X] Yes I am willing to submit a PR!
### Code of Conduct
- [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] github-actions[bot] commented on issue #6451: [Bug] [DataX] Read data from mysql and write to SqlServer 2012 database (DS version: 1.3.6)
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on issue #6451:
URL: https://github.com/apache/dolphinscheduler/issues/6451#issuecomment-937318758
Hi:
* Thank you for your feedback, we have received your issue, Please wait patiently for a reply.
* In order for us to understand your request as soon as possible, please provide detailed information、version or pictures.
* If you haven't received a reply for a long time, you can subscribe to the developer's email，Mail subscription steps reference https://dolphinscheduler.apache.org/zh-cn/community/development/subscribe.html ,Then write the issue URL in the email content and send question to dev@dolphinscheduler.apache.org.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] github-actions[bot] closed issue #6451: [Bug] [DataX] Read data from mysql and write to SqlServer 2012 database (DS version: 1.3.6)
Posted by GitBox <gi...@apache.org>.
github-actions[bot] closed issue #6451:
URL: https://github.com/apache/dolphinscheduler/issues/6451
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] sakinsun commented on issue #6451: [Bug] [DataX] Read data from mysql and write to SqlServer 2012 database (DS version: 1.3.6)
Posted by GitBox <gi...@apache.org>.
sakinsun commented on issue #6451:
URL: https://github.com/apache/dolphinscheduler/issues/6451#issuecomment-937336313
https://github.com/sakinsun/pic/blob/main/sqlserverwriter-column.png
github上的 ` 间隔号无法正常显示，所以我把ds上的日志截图了
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] sakinsun commented on issue #6451: [Bug] [DataX] Read data from mysql and write to SqlServer 2012 database (DS version: 1.3.6)
Posted by GitBox <gi...@apache.org>.
sakinsun commented on issue #6451:
URL: https://github.com/apache/dolphinscheduler/issues/6451#issuecomment-939376374
First,I am sorry because I am chinese,so my english is not very good.But I am very interested to DS.
My suggestion is to remove the symbol "`" in the code processing SQL server module from this code file :DataxUtils.java。
source code
case SQLSERVER:
return String.format("`%s`", column);
suggest code
case SQLSERVER:
return String.format("%s", column); -- no symbol `
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] dailidong commented on issue #6451: [Bug] [DataX] Read data from mysql and write to SqlServer 2012 database (DS version: 1.3.6)
Posted by GitBox <gi...@apache.org>.
dailidong commented on issue #6451:
URL: https://github.com/apache/dolphinscheduler/issues/6451#issuecomment-939291008
English is needed, please describe the question in English
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] sakinsun commented on issue #6451: [Bug] [DataX] Read data from mysql and write to SqlServer 2012 database (DS version: 1.3.6)
Posted by GitBox <gi...@apache.org>.
sakinsun commented on issue #6451:
URL: https://github.com/apache/dolphinscheduler/issues/6451#issuecomment-937336313
https://github.com/sakinsun/pic/blob/main/sqlserverwriter-column.png
github上的 ` 间隔号无法正常显示，所以我把ds上的日志截图了
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] github-actions[bot] commented on issue #6451: [Bug] [DataX] Read data from mysql and write to SqlServer 2012 database (DS version: 1.3.6)
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on issue #6451:
URL: https://github.com/apache/dolphinscheduler/issues/6451#issuecomment-937318662
### Search before asking
-[X] I had searched in the [issues](https://github.com/apache/dolphinscheduler/issues?q=is%3Aissue) and found no similar issues.
### What happened
In the DS 1.3.6 version, I use the datax plug-in, the data source is mysql, the target source is sqlserver, and the "non-custom template" method is used, that is, write SQL statements in mysql and fill in the table name directly in the target sqlserver. , And then run this node. As a result, an error was reported during operation, and the error content is as follows:
"Description: [Failed to obtain table field related information.].-Failed to obtain the meta information of the field of table: test_tmp_20211001. Please contact DBA to check the database and table information.-com.microsoft.sqlserver.jdbc.SQLServerException: "`" There is a grammatical error nearby."
After DataX is running under DS, it can be seen in the log that it is like this when obtaining the sqlserver metadata field:
"writer":{
"name":"sqlserverwriter",
"parameter":{
"column":[
"`suppid`"
]
The sqlserver table field is caused by `this number, which results in an error when querying metadata in sqlserver. The DS source code is as follows:
switch (dbType) {
case MYSQL:
return String.format("`%s`", column);
case POSTGRESQL:
return String.format("\"%s\"", column);
case ORACLE:
return String.format("\"%s\"", column);
case SQLSERVER:
return String.format("`%s`", column); --------- This is it! ! ! !
default:
return column;
}
}
The github location where the source code is located: dolphinscheduler-task-plugin/dolphinscheduler-task-datax/src/main/java/org/apache/dolphinscheduler/plugin/task/datax/DataxUtils.java
### What you expected to happen
As mentioned above, the datax plug-in is also used in DS and the "custom template" method is used to paste the configuration information and table fields, and the fields are without `quotation marks, the mysql data is successfully written to the sqlserver, and the writing is successful.
### How to reproduce
Use "custom template" instead, but using this, the graphical interface loses its meaning, and it is very inconvenient to paste configuration information. You have to paste account passwords and links every time.
### Anything else
_No response_
### Are you willing to submit PR?
-[X] Yes I am willing to submit a PR!
### Code of Conduct
-[X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] github-actions[bot] commented on issue #6451: [Bug] [DataX] Read data from mysql and write to SqlServer 2012 database (DS version: 1.3.6)
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on issue #6451:
URL: https://github.com/apache/dolphinscheduler/issues/6451#issuecomment-955819328
This issue has been closed because it has not received response for too long time. You could reopen it if you encountered similar problems in the future.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] github-actions[bot] commented on issue #6451: [Bug] [DataX] Read data from mysql and write to SqlServer 2012 database (DS version: 1.3.6)
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on issue #6451:
URL: https://github.com/apache/dolphinscheduler/issues/6451#issuecomment-954311474
This issue has been automatically marked as stale because it has not had recent activity for 14 days. It will be closed in next 3 days if no further activity occurs.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] dailidong commented on issue #6451: [Bug] [DataX] Read data from mysql and write to SqlServer 2012 database (DS version: 1.3.6)
Posted by GitBox <gi...@apache.org>.
dailidong commented on issue #6451:
URL: https://github.com/apache/dolphinscheduler/issues/6451#issuecomment-939290621
good job, please submit a PR
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] github-actions[bot] commented on issue #6451: [Bug] [DataX] Read data from mysql and write to SqlServer 2012 database (DS version: 1.3.6)
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on issue #6451:
URL: https://github.com/apache/dolphinscheduler/issues/6451#issuecomment-937318662
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] dailidong edited a comment on issue #6451: [Bug] [DataX] Read data from mysql and write to SqlServer 2012 database (DS version: 1.3.6)
Posted by GitBox <gi...@apache.org>.
dailidong edited a comment on issue #6451:
URL: https://github.com/apache/dolphinscheduler/issues/6451#issuecomment-939291008
by the way, as a global project, English is needed, please describe the question in English
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org