You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@streampark.apache.org by "zhoulii (via GitHub)" <gi...@apache.org> on 2023/04/18 09:51:17 UTC
[GitHub] [incubator-streampark] zhoulii opened a new pull request, #2650: [Improve] Bump Hadoop Version to 3.3.4
zhoulii opened a new pull request, #2650:
URL: https://github.com/apache/incubator-streampark/pull/2650
## What changes were proposed in this pull request
there are lots of vulnerabilities in current hadoop dependency, we should bump hadoop to a higher version. this pr upgrades hadoop version to 3.3.4, and replace all hadoop dependencies with `hadoop-client-api` and `hadoop-client-runtime`, and with these changes, we can reduce the number of dependencies in $STREAMPARK/lib from 303 to 242.
this pr was tested on `hadoop-2.10.2` & `hadoop 3.3.5 (the newest version)` with the following operations:
- yarn-application
- submit application
- trigger savepoint
- cancel with savepoint
- yarn-session
- submit application
- trigger savepoint
- cancel with savepoint
## Does this pull request potentially affect one of the following parts
- Dependencies (does it add or upgrade a dependency): (**yes** / no)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@streampark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-streampark] zhoulii commented on pull request #2650: [Improve] Bump Hadoop Version to 3.3.4
Posted by "zhoulii (via GitHub)" <gi...@apache.org>.
zhoulii commented on PR #2650:
URL: https://github.com/apache/incubator-streampark/pull/2650#issuecomment-1513067891
>
@wolfboys Thanks for the review. For now Tested with `hadoop-2.10.2` and `hadoop 3.3.5` . We should do more tests, but there are too many versions between 2.x and 3.x（about 50+）, maybe we can pick up some typical versions, any suggestions ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@streampark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-streampark] wolfboys commented on pull request #2650: [Improve] Bump Hadoop Version to 3.3.4
Posted by "wolfboys (via GitHub)" <gi...@apache.org>.
wolfboys commented on PR #2650:
URL: https://github.com/apache/incubator-streampark/pull/2650#issuecomment-1512978196
Thanks for your contribution. Are other versions of Hadoop compatible?, For example, versions 2. x and 3. x and more
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@streampark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-streampark] wolfboys merged pull request #2650: [Improve] Bump Hadoop Version to 3.3.4
Posted by "wolfboys (via GitHub)" <gi...@apache.org>.
wolfboys merged PR #2650:
URL: https://github.com/apache/incubator-streampark/pull/2650
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@streampark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org