You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by yanghua <gi...@git.apache.org> on 2018/05/09 12:29:42 UTC
[GitHub] flink pull request #5978: [FLINK-8554] Upgrade AWS SDK
GitHub user yanghua opened a pull request:
https://github.com/apache/flink/pull/5978
[FLINK-8554] Upgrade AWS SDK
## What is the purpose of the change
*This pull request upgrades AWS SDK*
## Brief change log
- *Upgrades AWS SDK*
## Verifying this change
This change is a trivial rework / code cleanup without any test coverage.
## Does this pull request potentially affect one of the following parts:
- Dependencies (does it add or upgrade a dependency): (**yes** / no)
- The public API, i.e., is any changed class annotated with `@Public(Evolving)`: (yes / **no**)
- The serializers: (yes / **no** / don't know)
- The runtime per-record code paths (performance sensitive): (yes / **no** / don't know)
- Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Yarn/Mesos, ZooKeeper: (yes / **no** / don't know)
- The S3 file system connector: (**yes** / no / don't know)
## Documentation
- Does this pull request introduce a new feature? (yes / **no**)
- If yes, how is the feature documented? (not applicable / docs / JavaDocs / **not documented**)
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/yanghua/flink FLINK-8554
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/flink/pull/5978.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #5978
----
commit aa45a83b9733715638137bdcad0f09dbf0f07ad4
Author: yanghua <ya...@...>
Date: 2018-05-09T12:25:22Z
[FLINK-8554] Upgrade AWS SDK
----
---
[GitHub] flink issue #5978: [FLINK-8554] Upgrade AWS SDK
Posted by yanghua <gi...@git.apache.org>.
Github user yanghua commented on the issue:
https://github.com/apache/flink/pull/5978
@StephanEwen close this or not?
---
[GitHub] flink issue #5978: [FLINK-8554] Upgrade AWS SDK
Posted by yanghua <gi...@git.apache.org>.
Github user yanghua commented on the issue:
https://github.com/apache/flink/pull/5978
version: 1.11.319:
```
[INFO] org.apache.flink:flink-connector-kinesis_2.11:jar:1.6-SNAPSHOT
[INFO] +- com.amazonaws:aws-java-sdk-kinesis:jar:1.11.319:compile
[INFO] | +- com.amazonaws:aws-java-sdk-core:jar:1.11.319:compile
[INFO] | \- com.amazonaws:jmespath-java:jar:1.11.319:compile
[INFO] +- com.amazonaws:amazon-kinesis-producer:jar:0.12.9:compile
[INFO] | \- (com.amazonaws:aws-java-sdk-core:jar:1.11.312:compile - omitted for conflict with 1.11.319)
[INFO] \- com.amazonaws:amazon-kinesis-client:jar:1.9.0:compile
[INFO] \- (com.amazonaws:aws-java-sdk-kinesis:jar:1.11.272:compile - omitted for conflict with 1.11.319)
```
version: 1.11.325
```
[INFO] org.apache.flink:flink-connector-kinesis_2.11:jar:1.6-SNAPSHOT
[INFO] +- com.amazonaws:aws-java-sdk-kinesis:jar:1.11.325:compile
[INFO] | +- com.amazonaws:aws-java-sdk-core:jar:1.11.325:compile
[INFO] | \- com.amazonaws:jmespath-java:jar:1.11.325:compile
[INFO] +- com.amazonaws:amazon-kinesis-producer:jar:0.12.9:compile
[INFO] | \- (com.amazonaws:aws-java-sdk-core:jar:1.11.312:compile - omitted for conflict with 1.11.325)
[INFO] \- com.amazonaws:amazon-kinesis-client:jar:1.9.0:compile
[INFO] \- (com.amazonaws:aws-java-sdk-kinesis:jar:1.11.272:compile - omitted for conflict with 1.11.325)
```
---
[GitHub] flink issue #5978: [FLINK-8554] Upgrade AWS SDK
Posted by yanghua <gi...@git.apache.org>.
Github user yanghua commented on the issue:
https://github.com/apache/flink/pull/5978
@StephanEwen here is test error in the [build job](https://travis-ci.org/apache/flink/jobs/380523257), and it seems the aws sdk is under license "Apache 2.0", so should I do the second item you asked?
---
[GitHub] flink issue #5978: [FLINK-8554] Upgrade AWS SDK
Posted by StephanEwen <gi...@git.apache.org>.
Github user StephanEwen commented on the issue:
https://github.com/apache/flink/pull/5978
Before merging this, we need to
- diff the dependency trees between the versions
- clear all licenses for the dependency changes. put changes into the NOTICE files
- check that no unshaded classes are added to the jar
---
[GitHub] flink issue #5978: [FLINK-8554] Upgrade AWS SDK
Posted by bowenli86 <gi...@git.apache.org>.
Github user bowenli86 commented on the issue:
https://github.com/apache/flink/pull/5978
Are there anything in `1.11.325` we desperately need?
If not, I would oppose upgrading AWS SDK too frequently. Highly likely that we don't need any of the new changes in `1.11.325`. As you can see, the current sdk version is `1.11.319` which is upgraded just a few days ago.
There're a few reasons we should discourage it:
- It doesn't add much value, and we don't really need it
- It costs lots of unnecessary work from both contributors and Flink community (committers, reviewers, etc)
- AWS releases their SDK very frequently, in a much faster pace than we can possibly catch up
---