You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@seatunnel.apache.org by GitBox <gi...@apache.org> on 2022/07/31 12:21:05 UTC
[GitHub] [incubator-seatunnel] hanwei996 opened a new issue, #2313: [Bug] contains invalid config, error: please specify [topics] as non-empty
hanwei996 opened a new issue, #2313:
URL: https://github.com/apache/incubator-seatunnel/issues/2313
### Search before asking
- [X] I had searched in the [issues](https://github.com/apache/incubator-seatunnel/issues?q=is%3Aissue+label%3A%22bug%22) and found no similar issues.
### What happened
source kafka to sink kafka error : Plugin[org.apache.seatunnel.flink.kafka.sink.KafkaSink] contains invalid config, error: please specify [topics] as non-empty
-------------------
Sink Kafka has no topics parameter in the official documentation
<img width="788" alt="image" src="https://user-images.githubusercontent.com/43572770/182026073-627cf1da-3a13-4946-b2bc-77c9fdcdcee3.png">
### SeaTunnel Version
2.1.2
### SeaTunnel Config
```conf
config/example01.conf
`
env {
# You can set flink configuration here
execution.parallelism = 1
#execution.checkpoint.interval = 10000
#execution.checkpoint.data-uri = "hdfs://localhost:9000/checkpoint"
}
source {
KafkaTableStream {
consumer.bootstrap.servers = "hadoop01:9092"
consumer.group.id = "seatunnel-learn"
topics = test_csv
result_table_name = test
format.type = csv
schema = "[{\"field\":\"name\",\"type\":\"string\"},{\"field\":\"age\", \"type\": \"int\"}]"
format.field-delimiter = ";"
format.allow-comments = "true"
format.ignore-parse-errors = "true"
}
}
transform {
sql {
sql = "select name,age from test where age > '"${age}"'"
}
}
sink {
kafka {
topic = "test_sink"
producer.bootstrap.servers = "hadoop01:9092"
}
}`
```
### Running Command
```shell
bin/start-seatunnel-flink.sh --config config/example01.conf -i age=18
```
### Error Exception
```log
2022-07-31 19:29:36,070 ERROR org.apache.seatunnel.core.base.command.BaseTaskExecuteCommand [] - Plugin[org.apache.seatunnel.flink.kafka.sink.KafkaSink] contains invalid config, error: please specify [topics] as non-empty
```
### Flink or Spark Version
Flink 1.12.7
### Java or Scala Version
_No response_
### Screenshots
_No response_
### Are you willing to submit PR?
- [ ] Yes I am willing to submit a PR!
### Code of Conduct
- [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-seatunnel] TyrantLucifer commented on issue #2313: [Bug] contains invalid config, error: please specify [topics] as non-empty
Posted by GitBox <gi...@apache.org>.
TyrantLucifer commented on issue #2313:
URL: https://github.com/apache/incubator-seatunnel/issues/2313#issuecomment-1200583668
In 2.1.2 release, KafkaTableStream has some bug on parse `schema` information, you can change seatunnel release to 2.1.3 to solve this problem.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-seatunnel] hanwei996 commented on issue #2313: [Bug] contains invalid config, error: please specify [topics] as non-empty
Posted by GitBox <gi...@apache.org>.
hanwei996 commented on issue #2313:
URL: https://github.com/apache/incubator-seatunnel/issues/2313#issuecomment-1200536758
sink {
kafka {
topics = "test_sink"
producer.bootstrap.servers = "hadoop01:9092"
}
}
Change from topic to topics ， Serialization error
<img width="953" alt="image" src="https://user-images.githubusercontent.com/43572770/182051354-d230d14f-83f7-4391-b164-b0441e3051e8.png">
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-seatunnel] TyrantLucifer commented on issue #2313: [Bug] contains invalid config, error: please specify [topics] as non-empty
Posted by GitBox <gi...@apache.org>.
TyrantLucifer commented on issue #2313:
URL: https://github.com/apache/incubator-seatunnel/issues/2313#issuecomment-1200448610
You can change the `topic` config of kafka sink to solve this problem temporarily , I think there is a gap between the documentation and the code, in code this option be treated as `topics`, @CalvinKirs what we should do next? Change doc or change code, I think change code is a best solution because of kafka sink only can accept one topic, so `topics` is meaningless, what do you think about it?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-seatunnel] github-actions[bot] commented on issue #2313: [Bug] contains invalid config, error: please specify [topics] as non-empty
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on issue #2313:
URL: https://github.com/apache/incubator-seatunnel/issues/2313#issuecomment-1495165725
This issue has been automatically marked as stale because it has not had recent activity for 30 days. It will be closed in next 7 days if no further activity occurs.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org