You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@bigtop.apache.org by "guluo2016 (via GitHub)" <gi...@apache.org> on 2023/03/24 02:37:18 UTC
[GitHub] [bigtop] guluo2016 opened a new pull request, #1094: BIGTOP-3915:soft link error about jar of hadoop in {hbase_home}/lib
guluo2016 opened a new pull request, #1094:
URL: https://github.com/apache/bigtop/pull/1094
### Description of PR
Solve soft link error ,
Details see: [BIGTOP-3915](https://issues.apache.org/jira/browse/BIGTOP-3915)
### How was this patch tested?
building hbase rpm,
installing hbase,
check soft link about hadoop in {hbase_home}/lib
### For code changes:
rpm: hbase.spec
deb: hbase/rules
Please review @iwasakims thanks
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@bigtop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [bigtop] iwasakims merged pull request #1094: BIGTOP-3915:soft link error about jar of hadoop in {hbase_home}/lib
Posted by "iwasakims (via GitHub)" <gi...@apache.org>.
iwasakims merged PR #1094:
URL: https://github.com/apache/bigtop/pull/1094
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@bigtop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [bigtop] guluo2016 commented on a diff in pull request #1094: BIGTOP-3915:soft link error about jar of hadoop in {hbase_home}/lib
Posted by "guluo2016 (via GitHub)" <gi...@apache.org>.
guluo2016 commented on code in PR #1094:
URL: https://github.com/apache/bigtop/pull/1094#discussion_r1148560496
##########
bigtop-packages/src/deb/hbase/rules:
##########
@@ -50,15 +53,15 @@ override_dh_auto_install:
ln -f -s ${hadoop_home}/client/hadoop-annotations.jar debian/tmp/usr/lib/${hbase_pkg_name}/lib/
ln -f -s ${hadoop_home}/client/hadoop-auth.jar debian/tmp/usr/lib/${hbase_pkg_name}/lib/
ln -f -s ${hadoop_home}/client/hadoop-common.jar debian/tmp/usr/lib/${hbase_pkg_name}/lib/
- ln -f -s ${hadoop_home}/client/hadoop-hdfs.jar debian/tmp/usr/lib/${hbase_pkg_name}/lib/
- ln -f -s ${hadoop_home}/client/hadoop-mapreduce-client-app.jar debian/tmp/usr/lib/${hbase_pkg_name}/lib/
+ ln -f -s ${hadoop_hdfs_path}/hadoop-hdfs.jar debian/tmp/usr/lib/${hbase_pkg_name}/lib/
+ ln -f -s ${hadoop_mapreduce_path}/hadoop-mapreduce-client-app.jar debian/tmp/usr/lib/${hbase_pkg_name}/lib/
ln -f -s ${hadoop_home}/client/hadoop-mapreduce-client-common.jar debian/tmp/usr/lib/${hbase_pkg_name}/lib/
ln -f -s ${hadoop_home}/client/hadoop-mapreduce-client-core.jar debian/tmp/usr/lib/${hbase_pkg_name}/lib/
ln -f -s ${hadoop_home}/client/hadoop-mapreduce-client-jobclient.jar debian/tmp/usr/lib/${hbase_pkg_name}/lib/
- ln -f -s ${hadoop_home}/client/hadoop-mapreduce-client-shuffle.jar debian/tmp/usr/lib/${hbase_pkg_name}/lib/
+ ln -f -s ${hadoop_mapreduce_path}/hadoop-mapreduce-client-shuffle.jar debian/tmp/usr/lib/${hbase_pkg_name}/lib/
ln -f -s ${hadoop_home}/client/hadoop-yarn-api.jar debian/tmp/usr/lib/${hbase_pkg_name}/lib/
ln -f -s ${hadoop_home}/client/hadoop-yarn-client.jar debian/tmp/usr/lib/${hbase_pkg_name}/lib/
- ln -f -s ${hadoop_home}/client/hadoop-yarn-common.jar debian/tmp/usr/lib/${hbase_pkg_name}/lib/
+ ln -f -s ${hadoop_yarn_path}/hadoop-yarn-common.jar debian/tmp/usr/lib/${hbase_pkg_name}/lib/
Review Comment:
Thank you for your patient review, you are right,I will modify it again later
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@bigtop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [bigtop] iwasakims commented on pull request #1094: BIGTOP-3915:soft link error about jar of hadoop in {hbase_home}/lib
Posted by "iwasakims (via GitHub)" <gi...@apache.org>.
iwasakims commented on PR #1094:
URL: https://github.com/apache/bigtop/pull/1094#issuecomment-1483677920
> In addition, i found hbase is still normal even thoes soft link error, this is puzzling.
It works because [the classpath set by hbase CLI contains classpath of Hadoop](https://github.com/apache/hbase/blob/rel/2.4.13/bin/hbase#L337-L352), I guess.
```
[root@5593b54af887 hbase]# hbase classpath | sed -z 's/:/\n/g'
/etc/hbase/conf
/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.362.b09-2.el8_7.x86_64/lib/tools.jar
/usr/lib/hbase
/usr/lib/hbase/lib/shaded-clients/hbase-shaded-client-byo-hadoop-2.4.13.jar
/usr/lib/hbase/lib/client-facing-thirdparty/audience-annotations-0.5.0.jar
/usr/lib/hbase/lib/client-facing-thirdparty/commons-logging-1.2.jar
/usr/lib/hbase/lib/client-facing-thirdparty/htrace-core4-4.2.0-incubating.jar
/usr/lib/hbase/lib/client-facing-thirdparty/jcl-over-slf4j-1.7.33.jar
/usr/lib/hbase/lib/client-facing-thirdparty/jul-to-slf4j-1.7.33.jar
/usr/lib/hbase/lib/client-facing-thirdparty/reload4j-1.2.19.jar
/usr/lib/hbase/lib/client-facing-thirdparty/slf4j-api-1.7.33.jar
/usr/lib/hbase/lib/client-facing-thirdparty/slf4j-reload4j-1.7.33.jar
/etc/hadoop/conf
/usr/lib/hadoop/lib/*
/usr/lib/hadoop/.//*
/usr/lib/hadoop-hdfs/./
/usr/lib/hadoop-hdfs/lib/*
/usr/lib/hadoop-hdfs/.//*
/usr/lib/hadoop-mapreduce/.//*
/usr/lib/hadoop-yarn/./
/usr/lib/hadoop-yarn/lib/*
/usr/lib/hadoop-yarn/.//*
/etc/hadoop/conf
```
Let me investigate whether we still need those symlinks.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@bigtop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [bigtop] iwasakims commented on a diff in pull request #1094: BIGTOP-3915:soft link error about jar of hadoop in {hbase_home}/lib
Posted by "iwasakims (via GitHub)" <gi...@apache.org>.
iwasakims commented on code in PR #1094:
URL: https://github.com/apache/bigtop/pull/1094#discussion_r1148315448
##########
bigtop-packages/src/deb/hbase/rules:
##########
@@ -50,15 +53,15 @@ override_dh_auto_install:
ln -f -s ${hadoop_home}/client/hadoop-annotations.jar debian/tmp/usr/lib/${hbase_pkg_name}/lib/
ln -f -s ${hadoop_home}/client/hadoop-auth.jar debian/tmp/usr/lib/${hbase_pkg_name}/lib/
ln -f -s ${hadoop_home}/client/hadoop-common.jar debian/tmp/usr/lib/${hbase_pkg_name}/lib/
- ln -f -s ${hadoop_home}/client/hadoop-hdfs.jar debian/tmp/usr/lib/${hbase_pkg_name}/lib/
- ln -f -s ${hadoop_home}/client/hadoop-mapreduce-client-app.jar debian/tmp/usr/lib/${hbase_pkg_name}/lib/
+ ln -f -s ${hadoop_hdfs_path}/hadoop-hdfs.jar debian/tmp/usr/lib/${hbase_pkg_name}/lib/
+ ln -f -s ${hadoop_mapreduce_path}/hadoop-mapreduce-client-app.jar debian/tmp/usr/lib/${hbase_pkg_name}/lib/
ln -f -s ${hadoop_home}/client/hadoop-mapreduce-client-common.jar debian/tmp/usr/lib/${hbase_pkg_name}/lib/
ln -f -s ${hadoop_home}/client/hadoop-mapreduce-client-core.jar debian/tmp/usr/lib/${hbase_pkg_name}/lib/
ln -f -s ${hadoop_home}/client/hadoop-mapreduce-client-jobclient.jar debian/tmp/usr/lib/${hbase_pkg_name}/lib/
- ln -f -s ${hadoop_home}/client/hadoop-mapreduce-client-shuffle.jar debian/tmp/usr/lib/${hbase_pkg_name}/lib/
+ ln -f -s ${hadoop_mapreduce_path}/hadoop-mapreduce-client-shuffle.jar debian/tmp/usr/lib/${hbase_pkg_name}/lib/
ln -f -s ${hadoop_home}/client/hadoop-yarn-api.jar debian/tmp/usr/lib/${hbase_pkg_name}/lib/
ln -f -s ${hadoop_home}/client/hadoop-yarn-client.jar debian/tmp/usr/lib/${hbase_pkg_name}/lib/
- ln -f -s ${hadoop_home}/client/hadoop-yarn-common.jar debian/tmp/usr/lib/${hbase_pkg_name}/lib/
+ ln -f -s ${hadoop_yarn_path}/hadoop-yarn-common.jar debian/tmp/usr/lib/${hbase_pkg_name}/lib/
Review Comment:
hadoop-yarn-common.jar seems not to need the fix.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@bigtop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [bigtop] iwasakims commented on a diff in pull request #1094: BIGTOP-3915:soft link error about jar of hadoop in {hbase_home}/lib
Posted by "iwasakims (via GitHub)" <gi...@apache.org>.
iwasakims commented on code in PR #1094:
URL: https://github.com/apache/bigtop/pull/1094#discussion_r1148314525
##########
bigtop-packages/src/deb/hbase/rules:
##########
@@ -50,15 +53,15 @@ override_dh_auto_install:
ln -f -s ${hadoop_home}/client/hadoop-annotations.jar debian/tmp/usr/lib/${hbase_pkg_name}/lib/
ln -f -s ${hadoop_home}/client/hadoop-auth.jar debian/tmp/usr/lib/${hbase_pkg_name}/lib/
ln -f -s ${hadoop_home}/client/hadoop-common.jar debian/tmp/usr/lib/${hbase_pkg_name}/lib/
- ln -f -s ${hadoop_home}/client/hadoop-hdfs.jar debian/tmp/usr/lib/${hbase_pkg_name}/lib/
- ln -f -s ${hadoop_home}/client/hadoop-mapreduce-client-app.jar debian/tmp/usr/lib/${hbase_pkg_name}/lib/
+ ln -f -s ${hadoop_hdfs_path}/hadoop-hdfs.jar debian/tmp/usr/lib/${hbase_pkg_name}/lib/
+ ln -f -s ${hadoop_mapreduce_path}/hadoop-mapreduce-client-app.jar debian/tmp/usr/lib/${hbase_pkg_name}/lib/
ln -f -s ${hadoop_home}/client/hadoop-mapreduce-client-common.jar debian/tmp/usr/lib/${hbase_pkg_name}/lib/
ln -f -s ${hadoop_home}/client/hadoop-mapreduce-client-core.jar debian/tmp/usr/lib/${hbase_pkg_name}/lib/
ln -f -s ${hadoop_home}/client/hadoop-mapreduce-client-jobclient.jar debian/tmp/usr/lib/${hbase_pkg_name}/lib/
- ln -f -s ${hadoop_home}/client/hadoop-mapreduce-client-shuffle.jar debian/tmp/usr/lib/${hbase_pkg_name}/lib/
+ ln -f -s ${hadoop_mapreduce_path}/hadoop-mapreduce-client-shuffle.jar debian/tmp/usr/lib/${hbase_pkg_name}/lib/
ln -f -s ${hadoop_home}/client/hadoop-yarn-api.jar debian/tmp/usr/lib/${hbase_pkg_name}/lib/
ln -f -s ${hadoop_home}/client/hadoop-yarn-client.jar debian/tmp/usr/lib/${hbase_pkg_name}/lib/
- ln -f -s ${hadoop_home}/client/hadoop-yarn-common.jar debian/tmp/usr/lib/${hbase_pkg_name}/lib/
+ ln -f -s ${hadoop_yarn_path}/hadoop-yarn-common.jar debian/tmp/usr/lib/${hbase_pkg_name}/lib/
ln -f -s ${hadoop_home}/client/hadoop-yarn-server-common.jar debian/tmp/usr/lib/${hbase_pkg_name}/lib/
Review Comment:
hadoop-yarn-server-common.jar needs the same fix? I tested the patch like following. @guluo2016
```
[root@ec68ac35bd71 /]# for f in $(find /usr/lib/hbase/lib -type l) ; do test -e "$f" || echo "$f does not exist." ; done
/usr/lib/hbase/lib/hadoop-yarn-server-common.jar does not exist.
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@bigtop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [bigtop] iwasakims commented on pull request #1094: BIGTOP-3915:soft link error about jar of hadoop in {hbase_home}/lib
Posted by "iwasakims (via GitHub)" <gi...@apache.org>.
iwasakims commented on PR #1094:
URL: https://github.com/apache/bigtop/pull/1094#issuecomment-1483740274
> Let me investigate whether we still need those symlinks.
Anyway we can commit the patch here before further refactoring. The patch looks harmless.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@bigtop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org