You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@seatunnel.apache.org by GitBox <gi...@apache.org> on 2022/07/28 11:48:02 UTC
[GitHub] [incubator-seatunnel] TyrantLucifer opened a new issue, #2289: [Bug] [connector-file] avro version conflicts
TyrantLucifer opened a new issue, #2289:
URL: https://github.com/apache/incubator-seatunnel/issues/2289
### Search before asking
- [X] I had searched in the [issues](https://github.com/apache/incubator-seatunnel/issues?q=is%3Aissue+label%3A%22bug%22) and found no similar issues.
### What happened
The avro version that contained in flink-avro is 1.10.0, but the avro version that contained in seatunnel main pom is 1.8.2.
When we try to use file connector writer parquet file in flink, it will be conflicted between this two different version and throw `NoSuchMethod` exception.
In order to solve this problem, I think we have two plans to unified avro version:
1. Exclude avro in seatunnel-flink-api and include avro 1.8.2 version in it
2. Update parquet-avro version from 1.10.0 to 1.12.0 in main pom, this will include avro 1.10.1 version, but it might be conflicted with spark-core, we also should exclude avro in spark-core
I think the first solution is best, what do you think about? @CalvinKirs @Hisoka-X
### SeaTunnel Version
dev
### SeaTunnel Config
```conf
none
```
### Running Command
```shell
none
```
### Error Exception
```log
NoSuchMethod Exception
```
### Flink or Spark Version
_No response_
### Java or Scala Version
_No response_
### Screenshots
_No response_
### Are you willing to submit PR?
- [X] Yes I am willing to submit a PR!
### Code of Conduct
- [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-seatunnel] Hisoka-X commented on issue #2289: [Bug] [connector-file] avro version conflicts
Posted by GitBox <gi...@apache.org>.
Hisoka-X commented on issue #2289:
URL: https://github.com/apache/incubator-seatunnel/issues/2289#issuecomment-1198104301
Sorry, I missed something.
> Exclude avro in seatunnel-flink-api and include avro 1.8.2 version in it
Agree with you, the first one better.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-seatunnel] TyrantLucifer commented on issue #2289: [Bug] [connector-file] avro version conflicts
Posted by GitBox <gi...@apache.org>.
TyrantLucifer commented on issue #2289:
URL: https://github.com/apache/incubator-seatunnel/issues/2289#issuecomment-1198367094
pr #2294 , @Hisoka-X ,please review
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-seatunnel] TyrantLucifer closed issue #2289: [Bug] [connector-file] avro version conflicts
Posted by GitBox <gi...@apache.org>.
TyrantLucifer closed issue #2289: [Bug] [connector-file] avro version conflicts
URL: https://github.com/apache/incubator-seatunnel/issues/2289
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-seatunnel] Hisoka-X commented on issue #2289: [Bug] [connector-file] avro version conflicts
Posted by GitBox <gi...@apache.org>.
Hisoka-X commented on issue #2289:
URL: https://github.com/apache/incubator-seatunnel/issues/2289#issuecomment-1198067406
I checked root pom. The version already are 1.10.0, any thing wrong with me? Or you should update local code.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org