You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@seatunnel.apache.org by GitBox <gi...@apache.org> on 2022/07/19 06:22:52 UTC
[GitHub] [incubator-seatunnel] gaaraG opened a new issue, #2206: [Bug] [seatunnel-api-flink] Connectors dependencies repeat additions
gaaraG opened a new issue, #2206:
URL: https://github.com/apache/incubator-seatunnel/issues/2206
### Search before asking
- [X] I had searched in the [issues](https://github.com/apache/incubator-seatunnel/issues?q=is%3Aissue+label%3A%22bug%22) and found no similar issues.
### What happened
I had a bug testing a task with non-1 parallelism.
And then I tested that the dependency repeats when parallelism is 1, but only when parallelism is not 1. I don't know if this is a Flink bug or if it's on this side, but my quest dependencies shouldn't be repeated.
I will submit a PR later.
### SeaTunnel Version
dev
### SeaTunnel Config
```conf
env {
# You can set flink configuration here
execution.parallelism = 2
}
source {
# This is a example input plugin **only for test and demonstrate the feature input plugin**
Clickhouse {
host = "localhost:8123"
database = "default"
sql = "select * from uk_price_paid"
username = "default"
password = "6hGIAsc7"
}
# If you would like to get more information about how to configure seatunnel and see full list of input plugins,
# please go to https://seatunnel.apache.org/docs/flink/configuration/source-plugins/Fake
}
transform {
# If you would like to get more information about how to configure seatunnel and see full list of filter plugins,
# please go to https://seatunnel.apache.org/docs/flink/configuration/transform-plugins/Sql
}
sink {
# choose stdout output plugin to output data to console
Clickhouse {
host = "localhost:8123"
table = "uk_price_paid1"
database = "default"
username = "default"
password = "6hGIAsc7"
}
# If you would like to get more information about how to configure seatunnel and see full list of output plugins,
# please go to https://seatunnel.apache.org/docs/flink/configuration/sink-plugins/Console
}
```
### Running Command
```shell
./bin/start-seatunnel-flink-new-connector.sh -c config/flink.batch.conf
```
### Error Exception
```log
Caused by: java.lang.IllegalStateException: The library registration references a different set of library BLOBs than previous registrations for this job:
old:[file:/home/gaara/software/apache-seatunnel-incubating-2.1.3-SNAPSHOT/connectors/seatunnel/connector-clickhouse-2.1.3-SNAPSHOT-2.11.12.jar]
new:[file:/home/gaara/software/apache-seatunnel-incubating-2.1.3-SNAPSHOT/connectors/seatunnel/connector-clickhouse-2.1.3-SNAPSHOT-2.11.12.jar, file:/home/gaara/software/apache-seatunnel-incubating-2.1.3-SNAPSHOT/connectors/seatunnel/connector-clickhouse-2.1.3-SNAPSHOT-2.11.12.jar]
at org.apache.flink.runtime.execution.librarycache.BlobLibraryCacheManager$ResolvedClassLoader.verifyClassLoader(BlobLibraryCacheManager.java:431)
at org.apache.flink.runtime.execution.librarycache.BlobLibraryCacheManager$ResolvedClassLoader.access$500(BlobLibraryCacheManager.java:356)
at org.apache.flink.runtime.execution.librarycache.BlobLibraryCacheManager$LibraryCacheEntry.getOrResolveClassLoader(BlobLibraryCacheManager.java:232)
at org.apache.flink.runtime.execution.librarycache.BlobLibraryCacheManager$LibraryCacheEntry.access$1100(BlobLibraryCacheManager.java:199)
at org.apache.flink.runtime.execution.librarycache.BlobLibraryCacheManager$DefaultClassLoaderLease.getOrResolveClassLoader(BlobLibraryCacheManager.java:333)
at org.apache.flink.runtime.taskmanager.Task.createUserCodeClassloader(Task.java:1024)
at org.apache.flink.runtime.taskmanager.Task.doRun(Task.java:628)
at org.apache.flink.runtime.taskmanager.Task.run(Task.java:566)
at java.lang.Thread.run(Thread.java:748)
```
### Flink or Spark Version
flink 1.13.6
### Java or Scala Version
java 1.8.0_212
### Screenshots
### Are you willing to submit PR?
- [X] Yes I am willing to submit a PR!
### Code of Conduct
- [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-seatunnel] Hisoka-X commented on issue #2206: [Bug] [seatunnel-api-flink] Connectors dependencies repeat additions
Posted by GitBox <gi...@apache.org>.
Hisoka-X commented on issue #2206:
URL: https://github.com/apache/incubator-seatunnel/issues/2206#issuecomment-1323377471
@hdd0921 Please update your seatunnel version. We already fix this bug
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-seatunnel] Hisoka-X commented on issue #2206: [Bug] [seatunnel-api-flink] Connectors dependencies repeat additions
Posted by GitBox <gi...@apache.org>.
Hisoka-X commented on issue #2206:
URL: https://github.com/apache/incubator-seatunnel/issues/2206#issuecomment-1188823926
Close by #2207
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-seatunnel] hdd0921 commented on issue #2206: [Bug] [seatunnel-api-flink] Connectors dependencies repeat additions
Posted by GitBox <gi...@apache.org>.
hdd0921 commented on issue #2206:
URL: https://github.com/apache/incubator-seatunnel/issues/2206#issuecomment-1323399286
> @hdd0921 Please update your seatunnel version. We already fix this bug
ok,I am trying,thanks very much!!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-seatunnel] hdd0921 commented on issue #2206: [Bug] [seatunnel-api-flink] Connectors dependencies repeat additions
Posted by GitBox <gi...@apache.org>.
hdd0921 commented on issue #2206:
URL: https://github.com/apache/incubator-seatunnel/issues/2206#issuecomment-1323367007
I have the same problem. But It's the first time to use it, I don't know how to operate. Could you please give me some suggestions or any guidance documents here？
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-seatunnel] Hisoka-X closed issue #2206: [Bug] [seatunnel-api-flink] Connectors dependencies repeat additions
Posted by GitBox <gi...@apache.org>.
Hisoka-X closed issue #2206: [Bug] [seatunnel-api-flink] Connectors dependencies repeat additions
URL: https://github.com/apache/incubator-seatunnel/issues/2206
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org