You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@bigtop.apache.org by GitBox <gi...@apache.org> on 2022/09/19 11:17:40 UTC
[GitHub] [bigtop] vivostar opened a new pull request, #1017: BIGTOP-3818. Phoenix Don't Requre Hadoop and Zookeeper Installed
vivostar opened a new pull request, #1017:
URL: https://github.com/apache/bigtop/pull/1017
<!--
Thanks for sending a pull request!
1. If this is your first time, please read our contributor guidelines: https://cwiki.apache.org/confluence/display/BIGTOP/How+to+Contribute
2. Make sure your PR title starts with JIRA issue id, e.g., 'BIGTOP-3638: Your PR title ...'.
-->
### Description of PR
When we deploy Phoenix, we just need to move phoenix-server.jar to hbase lib dir.
So I think the package requires is redundant.
- [x] Does the title or this PR starts with the corresponding JIRA issue id (e.g. 'BIGTOP-3638. Your PR title ...')?
- [x] Make sure that newly added files do not have any licensing issues. When in doubt refer to https://www.apache.org/licenses/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@bigtop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [bigtop] iwasakims merged pull request #1017: BIGTOP-3818. Phoenix Don't Requre Hadoop and Zookeeper Installed
Posted by GitBox <gi...@apache.org>.
iwasakims merged PR #1017:
URL: https://github.com/apache/bigtop/pull/1017
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@bigtop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [bigtop] vivostar commented on a diff in pull request #1017: BIGTOP-3818. Phoenix Don't Requre Hadoop and Zookeeper Installed
Posted by GitBox <gi...@apache.org>.
vivostar commented on code in PR #1017:
URL: https://github.com/apache/bigtop/pull/1017#discussion_r975113541
##########
bigtop-packages/src/deb/phoenix/control:
##########
@@ -23,7 +23,7 @@ Homepage: http://phoenix.apache.org
Package: phoenix
Architecture: all
-Depends: zookeeper, hadoop, hadoop-mapreduce, hadoop-yarn, hbase, bigtop-utils (>= 0.7)
Review Comment:
thanks for your advice. the phoenix-client-hbase*.jar is a thick client for phoenix, It has contained all hadoop and hbase dependencies.
```shell
# jar tvf /usr/lib/phoenix/phoenix-client-hbase-2.4-5.1.2.jar | grep org.apache.hadoop.yarn | head -n 3
0 Fri May 06 23:35:42 UTC 2022 org/apache/hadoop/yarn/
0 Fri May 06 23:35:42 UTC 2022 org/apache/hadoop/yarn/api/
0 Fri May 06 23:35:42 UTC 2022 org/apache/hadoop/yarn/api/records/
# jar tvf /usr/lib/phoenix/phoenix-client-hbase-2.4-5.1.2.jar | grep org.apache.hadoop.yarn | wc -l
2714
# jar tvf /usr/lib/phoenix/phoenix-client-hbase-2.4-5.1.2.jar | grep org.apache.hadoop.hdfs | head -n 3
0 Fri May 06 23:29:32 UTC 2022 org/apache/hadoop/hdfs/
0 Fri May 06 23:29:32 UTC 2022 org/apache/hadoop/hdfs/client/
607 Fri May 06 23:29:32 UTC 2022 org/apache/hadoop/hdfs/client/package-info.class
# jar tvf /usr/lib/phoenix/phoenix-client-hbase-2.4-5.1.2.jar | grep org.apache.hadoop.hdfs | wc -l
4475
# jar tvf /usr/lib/phoenix/phoenix-client-hbase-2.4-5.1.2.jar | grep org.apache.hadoop.mapred | head -n 3
0 Fri May 06 23:41:52 UTC 2022 org/apache/hadoop/mapreduce/
0 Fri May 06 23:41:52 UTC 2022 org/apache/hadoop/mapreduce/lib/
0 Fri May 06 23:41:52 UTC 2022 org/apache/hadoop/mapreduce/lib/input/
# jar tvf /usr/lib/phoenix/phoenix-client-hbase-2.4-5.1.2.jar | grep org.apache.hadoop.mapred | wc -l
954
```
and the phoenix_util.py, I think it is for thin client like query server, however our current distros don't contain query server components. so I will remove the hbase dependency, too.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@bigtop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [bigtop] iwasakims commented on a diff in pull request #1017: BIGTOP-3818. Phoenix Don't Requre Hadoop and Zookeeper Installed
Posted by GitBox <gi...@apache.org>.
iwasakims commented on code in PR #1017:
URL: https://github.com/apache/bigtop/pull/1017#discussion_r974859823
##########
bigtop-packages/src/deb/phoenix/control:
##########
@@ -23,7 +23,7 @@ Homepage: http://phoenix.apache.org
Package: phoenix
Architecture: all
-Depends: zookeeper, hadoop, hadoop-mapreduce, hadoop-yarn, hbase, bigtop-utils (>= 0.7)
Review Comment:
If the fat jar contains Hadoop classes for running MR tools, we can remove this package dependencies. We do not need to add hbase dependency anyway.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@bigtop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [bigtop] vivostar commented on a diff in pull request #1017: BIGTOP-3818. Phoenix Don't Requre Hadoop and Zookeeper Installed
Posted by GitBox <gi...@apache.org>.
vivostar commented on code in PR #1017:
URL: https://github.com/apache/bigtop/pull/1017#discussion_r975113541
##########
bigtop-packages/src/deb/phoenix/control:
##########
@@ -23,7 +23,7 @@ Homepage: http://phoenix.apache.org
Package: phoenix
Architecture: all
-Depends: zookeeper, hadoop, hadoop-mapreduce, hadoop-yarn, hbase, bigtop-utils (>= 0.7)
Review Comment:
thanks for your advice. the phoenix-client-hbase*.jar is a thick client for phoenix, It has contained all hadoop and hbase dependencies.
```shell
# jar tvf /usr/lib/phoenix/phoenix-client-hbase-2.4-5.1.2.jar | grep org.apache.hadoop.yarn | head -n 3
0 Fri May 06 23:35:42 UTC 2022 org/apache/hadoop/yarn/
0 Fri May 06 23:35:42 UTC 2022 org/apache/hadoop/yarn/api/
0 Fri May 06 23:35:42 UTC 2022 org/apache/hadoop/yarn/api/records/
# jar tvf /usr/lib/phoenix/phoenix-client-hbase-2.4-5.1.2.jar | grep org.apache.hadoop.yarn | wc -l
2714
# jar tvf /usr/lib/phoenix/phoenix-client-hbase-2.4-5.1.2.jar | grep org.apache.hadoop.hdfs | head -n 3
0 Fri May 06 23:29:32 UTC 2022 org/apache/hadoop/hdfs/
0 Fri May 06 23:29:32 UTC 2022 org/apache/hadoop/hdfs/client/
607 Fri May 06 23:29:32 UTC 2022 org/apache/hadoop/hdfs/client/package-info.class
# jar tvf /usr/lib/phoenix/phoenix-client-hbase-2.4-5.1.2.jar | grep org.apache.hadoop.hdfs | wc -l
4475
# jar tvf /usr/lib/phoenix/phoenix-client-hbase-2.4-5.1.2.jar | grep org.apache.hadoop.mapred | head -n 3
0 Fri May 06 23:41:52 UTC 2022 org/apache/hadoop/mapreduce/
0 Fri May 06 23:41:52 UTC 2022 org/apache/hadoop/mapreduce/lib/
0 Fri May 06 23:41:52 UTC 2022 org/apache/hadoop/mapreduce/lib/input/
# jar tvf /usr/lib/phoenix/phoenix-client-hbase-2.4-5.1.2.jar | grep org.apache.hadoop.mapred | wc -l
954
```
and the phoenix_util.py, I think it is for thin client like query server, however our current distros don't contain query server components. so I will remote the hbase dependency, too.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@bigtop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [bigtop] iwasakims commented on a diff in pull request #1017: BIGTOP-3818. Phoenix Don't Requre Hadoop and Zookeeper Installed
Posted by GitBox <gi...@apache.org>.
iwasakims commented on code in PR #1017:
URL: https://github.com/apache/bigtop/pull/1017#discussion_r974859207
##########
bigtop-packages/src/deb/phoenix/control:
##########
@@ -23,7 +23,7 @@ Homepage: http://phoenix.apache.org
Package: phoenix
Architecture: all
-Depends: zookeeper, hadoop, hadoop-mapreduce, hadoop-yarn, hbase, bigtop-utils (>= 0.7)
Review Comment:
I guess the original author added the dependency for tools like [Phoenix Map Reduce](https://phoenix.apache.org/phoenix_mr.html) using libraries of Hadoop.
While tools installed under /usr/lib/phoenix/bin uses classpath assuming hadoop, it does not assume hbase classpath because fat cliant jar contains HBase classes.
```
-cp "/etc/hbase/conf:/usr/lib/phoenix/bin/../phoenix-client-hbase-2.4-5.1.2.jar:/etc/hadoop/conf:/etc/hadoop/conf:/usr/lib/hadoop/lib/*:/usr/lib/hadoop/.//*:/usr/lib/hadoop-hdfs/./:/usr/lib/hadoop-hdfs/lib/*:/usr/lib/hadoop-hdfs/.//*:/usr/lib/hadoop-mapreduce/.//*:/usr/lib/hadoop-yarn/./:/usr/lib/hadoop-yarn/lib/*:/usr/lib/hadoop-yarn/.//*"
```
```
# jar tvf /usr/lib/phoenix/phoenix-client-hbase-2.4-5.1.2.jar | grep org.apache.hadoop.hbase.client | head -n 3
0 Wed Jan 22 15:10:16 UTC 2020 org/apache/hadoop/hbase/client/
570 Wed Jan 22 15:10:16 UTC 2020 org/apache/hadoop/hbase/client/RegionInfoUtil.class
1312 Wed Jan 22 15:10:16 UTC 2020 org/apache/hadoop/hbase/client/AbstractClientScanner.class
# jar tvf /usr/lib/phoenix/phoenix-client-hbase-2.4-5.1.2.jar | grep org.apache.hadoop.hbase | wc -l
8380
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@bigtop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org