You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@bigtop.apache.org by GitBox <gi...@apache.org> on 2020/11/25 13:03:07 UTC
[GitHub] [bigtop] iwasakims opened a new pull request #704: BIGTOP-3280. Bump Hadoop to 3.2.1.
iwasakims opened a new pull request #704:
URL: https://github.com/apache/bigtop/pull/704
https://issues.apache.org/jira/browse/BIGTOP-3280
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [bigtop] iwasakims commented on pull request #704: BIGTOP-3280. Bump Hadoop to 3.2.1.
Posted by GitBox <gi...@apache.org>.
iwasakims commented on pull request #704:
URL: https://github.com/apache/bigtop/pull/704#issuecomment-748714544
> May I ask why we upgrade hadoop from 3.2.1 to 3.2.2 not directly to 3.3.3?
I'm assuming some downstream projects depending on hadoop are not ready for hadoop-3.3.0 which contains big changes in transitive dependencies. I think we can release the next bigtop (on hadoop-3.x) early by using 3.2.x. In addition, hadoop-3.2.2 should contain more bug fixes than 3.3.0 (released 9 months ago).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [bigtop] iwasakims commented on pull request #704: BIGTOP-3280. Bump Hadoop to 3.2.1.
Posted by GitBox <gi...@apache.org>.
iwasakims commented on pull request #704:
URL: https://github.com/apache/bigtop/pull/704#issuecomment-733705957
This PR addresses packaging issues only. Contents of bundled configs might be updated based on the result of smoke-tests.
I think we should merge this after 1.5.0 is published for the ease of release process.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [bigtop] iwasakims merged pull request #704: BIGTOP-3280. Bump Hadoop to 3.2.1.
Posted by GitBox <gi...@apache.org>.
iwasakims merged pull request #704:
URL: https://github.com/apache/bigtop/pull/704
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [bigtop] iwasakims commented on pull request #704: BIGTOP-3280. Bump Hadoop to 3.2.1.
Posted by GitBox <gi...@apache.org>.
iwasakims commented on pull request #704:
URL: https://github.com/apache/bigtop/pull/704#issuecomment-734226154
There is ongoing release process of Hadoop 3.2.2. We can remove patch0-HADOOP-16739-branch-3.2.diff if we can use hadoop-3.2.2. We need patch0-revert-HADOOP-16598.diff again instead..
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [bigtop] guyuqi commented on pull request #704: BIGTOP-3280. Bump Hadoop to 3.2.1.
Posted by GitBox <gi...@apache.org>.
guyuqi commented on pull request #704:
URL: https://github.com/apache/bigtop/pull/704#issuecomment-748711811
@iwasakims
As disscussed in [here](https://issues.apache.org/jira/browse/BIGTOP-3280?focusedCommentId=17238449&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-17238449), we plan to upgrade hadoop to 3.3.3.
May I ask why we upgrade hadoop from 3.2.1 to 3.2.2 not directly to 3.3.3?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [bigtop] iwasakims commented on pull request #704: BIGTOP-3280. Bump Hadoop to 3.2.1.
Posted by GitBox <gi...@apache.org>.
iwasakims commented on pull request #704:
URL: https://github.com/apache/bigtop/pull/704#issuecomment-734084032
We need protobuf built without `--disable-shared` for libhdfspp (C++ client library for HDFS). You need to reinstall protobuf before building hadoop with `-Pnative`.
```
sudo mv /usr/local/bin/protoc /tmp/
sudo mv /usr/src/protobuf-2.5.0 /tmp/
./gradelw toolchain
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [bigtop] iwasakims commented on pull request #704: BIGTOP-3280. Bump Hadoop to 3.2.1.
Posted by GitBox <gi...@apache.org>.
iwasakims commented on pull request #704:
URL: https://github.com/apache/bigtop/pull/704#issuecomment-734173503
Thanks @oflebbe and @guyuqi. I update the PR based on @guyuqi 's suggestion.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [bigtop] oflebbe commented on pull request #704: BIGTOP-3280. Bump Hadoop to 3.2.1.
Posted by GitBox <gi...@apache.org>.
oflebbe commented on pull request #704:
URL: https://github.com/apache/bigtop/pull/704#issuecomment-734117985
Hi @iwasakims , having a shared library in an system incompatible version installed system visible is asking for trouble, AFAIK.
Can we install the shared protobuf library in the libhdfspp RPATH as well ?
* For instance packaging the protobuf dependency with libhfspp
or
* install shared protobuf in an private directory and extending the RPATH of libhfspp to include protobuf shared library ?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [bigtop] oflebbe edited a comment on pull request #704: BIGTOP-3280. Bump Hadoop to 3.2.1.
Posted by GitBox <gi...@apache.org>.
oflebbe edited a comment on pull request #704:
URL: https://github.com/apache/bigtop/pull/704#issuecomment-734117985
Hi @iwasakims , having a shared library in an system incompatible version installed system visible is asking for trouble, AFAIK.
Can we install the shared protobuf library in the libhdfspp RPATH as well ?
* For instance packaging the protobuf dependency with libhfspp
or
* install shared protobuf in an private directory and extending the RPATH of libhfspp to include protobuf shared library ? And add an install dependency to libproto, of course.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [bigtop] oflebbe edited a comment on pull request #704: BIGTOP-3280. Bump Hadoop to 3.2.1.
Posted by GitBox <gi...@apache.org>.
oflebbe edited a comment on pull request #704:
URL: https://github.com/apache/bigtop/pull/704#issuecomment-734117985
Hi @iwasakims , having a shared library in an system incompatible version installed system visible is asking for trouble, AFAIK.
Can we install the shared protobuf library in the libhdfspp RPATH as well ?
* For instance packaging the protobuf dependency with libhfspp
or
* install shared protobuf in an private directory and extending the RPATH of libhfspp to include protobuf shared library ? And add an install dependency to libhfspp, of course.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [bigtop] guyuqi commented on pull request #704: BIGTOP-3280. Bump Hadoop to 3.2.1.
Posted by GitBox <gi...@apache.org>.
guyuqi commented on pull request #704:
URL: https://github.com/apache/bigtop/pull/704#issuecomment-734612788
+1
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [bigtop] iwasakims commented on pull request #704: BIGTOP-3280. Bump Hadoop to 3.2.1.
Posted by GitBox <gi...@apache.org>.
iwasakims commented on pull request #704:
URL: https://github.com/apache/bigtop/pull/704#issuecomment-748076562
Thanks, @guyuqi. I'm merging this since bigtop-1.5.0 was released. Let me post a patch to upgrade to hadoop-3.2.2 in follow-up JIRA. 3.2.2 will be released in a week.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org