You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@seatunnel.apache.org by "QiMingChina (via GitHub)" <gi...@apache.org> on 2023/04/03 08:35:16 UTC
[GitHub] [incubator-seatunnel] QiMingChina opened a new issue, #4482: [Bug] [connector-clickhouse] “Distributed（'clusterw_7shard','default','user_info'）”Character segmentation problem about the local table named user_info
QiMingChina opened a new issue, #4482:
URL: https://github.com/apache/incubator-seatunnel/issues/4482
### Search before asking
- [X] I had searched in the [issues](https://github.com/apache/incubator-seatunnel/issues?q=is%3Aissue+label%3A%22bug%22) and found no similar issues.
### What happened
I use connector-clickhouse to write data to clickhouse,find error below
I checked the source code and found a string segmentation problem
<img width="769" alt="4b07390c5aae29966fa14794d87df76" src="https://user-images.githubusercontent.com/48780003/229455087-efe59d44-a8cb-4d57-a3f4-fd11cebcac61.png">
The replace method in the source code does not remove the brackets from the table name
Causes a problem with the subsequent spliced SQL statements
<img width="799" alt="0c5d73943db855361e898ba8466356f" src="https://user-images.githubusercontent.com/48780003/229455539-3c04b251-352b-4958-8be3-d974082fc031.png">
### SeaTunnel Version
Seatunnel 2.3.1
### SeaTunnel Config
```conf
env {
execution.parallelism = 4
job.mode = "STREAMING"
job.name = "test.kafka.2.clickhouse"
}
source {
Kafka {
result_table_name = "user_info_kafka"
schema = {
fields {
name = "string"
age = "int"
salary = "double"
}
}
format = json
topic = "kafka-test"
bootstrap.servers = "XXXX:9092"
consumer.group = "kafka-clickhouse-test01"
commit_on_checkpoint = false
kafka.config = {
auto.offset.reset = "latest"
enable.auto.commit = "true"
}
}
}
transform {
Sql {
source_table_name = "user_info_kafka"
result_table_name = "clus_user_info"
query = "select name, age, salary from user_info_kafka"
}
}
sink {
Clickhouse {
host = "XXXX:8123"
database = "default"
table = "clus_user_info"
username = "default"
password = "XXX"
split_mode = true
sharding_key = "name"
}
}
```
### Running Command
```shell
./bin/seatunnel.sh --config ./config/kafka-to-clickhouse-test01.conf -e local
```
### Error Exception
```log
Exception in thread "main" org.apache.seatunnel.core.starter.exception.CommandExecuteException: SeaTunnel job executed failed
at org.apache.seatunnel.core.starter.seatunnel.command.ClientExecuteCommand.execute(ClientExecuteCommand.java:181)
at org.apache.seatunnel.core.starter.SeaTunnel.run(SeaTunnel.java:40)
at org.apache.seatunnel.core.starter.seatunnel.SeaTunnelClient.main(SeaTunnelClient.java:34)
Caused by: org.apache.seatunnel.connectors.seatunnel.clickhouse.exception.ClickhouseConnectorException: ErrorCode:[API-05], ErrorDescription:[Table not existed] - Cannot get table from clickhouse, resultSet is empty
at org.apache.seatunnel.connectors.seatunnel.clickhouse.sink.client.ClickhouseProxy.getClickhouseDistributedTable(ClickhouseProxy.java:102)
at org.apache.seatunnel.connectors.seatunnel.clickhouse.sink.client.ClickhouseProxy.getClickhouseTable(ClickhouseProxy.java:230)
at org.apache.seatunnel.connectors.seatunnel.clickhouse.sink.client.ClickhouseSink.prepare(ClickhouseSink.java:145)
at org.apache.seatunnel.engine.core.parse.ConnectorInstanceLoader.loadSinkInstance(ConnectorInstanceLoader.java:90)
at org.apache.seatunnel.engine.core.parse.JobConfigParser.sampleAnalyze(JobConfigParser.java:414)
at org.apache.seatunnel.engine.core.parse.JobConfigParser.parse(JobConfigParser.java:132)
at org.apache.seatunnel.engine.core.parse.MultipleTableJobConfigParser.parse(MultipleTableJobConfigParser.java:112)
at org.apache.seatunnel.engine.client.job.JobExecutionEnvironment.getLogicalDag(JobExecutionEnvironment.java:155)
at org.apache.seatunnel.engine.client.job.JobExecutionEnvironment.execute(JobExecutionEnvironment.java:147)
at org.apache.seatunnel.core.starter.seatunnel.command.ClientExecuteCommand.execute(ClientExecuteCommand.java:140)
```
### Flink or Spark Version
_No response_
### Java or Scala Version
_No response_
### Screenshots
_No response_
### Are you willing to submit PR?
- [X] Yes I am willing to submit a PR!
### Code of Conduct
- [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-seatunnel] MonsterChenzhuo commented on issue #4482: [Bug] [connector-clickhouse] “Distributed（'clusterw_7shard','default','user_info'）”Character segmentation problem about the local table named user_info
Posted by "MonsterChenzhuo (via GitHub)" <gi...@apache.org>.
MonsterChenzhuo commented on issue #4482:
URL: https://github.com/apache/incubator-seatunnel/issues/4482#issuecomment-1500808570
please assign to me; thx。
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-seatunnel] EricJoy2048 closed issue #4482: [Bug] [connector-clickhouse] “Distributed（'clusterw_7shard','default','user_info'）”Character segmentation problem about the local table named user_info
Posted by "EricJoy2048 (via GitHub)" <gi...@apache.org>.
EricJoy2048 closed issue #4482: [Bug] [connector-clickhouse] “Distributed（'clusterw_7shard','default','user_info'）”Character segmentation problem about the local table named user_info
URL: https://github.com/apache/incubator-seatunnel/issues/4482
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-seatunnel] github-actions[bot] commented on issue #4482: [Bug] [connector-clickhouse] “Distributed（'clusterw_7shard','default','user_info'）”Character segmentation problem about the local table named user_info
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on issue #4482:
URL: https://github.com/apache/incubator-seatunnel/issues/4482#issuecomment-1539225343
This issue has been automatically marked as stale because it has not had recent activity for 30 days. It will be closed in next 7 days if no further activity occurs.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org