You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@bigtop.apache.org by "LiJie20190102 (via GitHub)" <gi...@apache.org> on 2023/10/20 07:22:52 UTC
[PR] BIGTOP-4028 : Excluding log dependencies corresponding to hbase in the hive service [bigtop]
LiJie20190102 opened a new pull request, #1200:
URL: https://github.com/apache/bigtop/pull/1200
…e hive service
<!--
Thanks for sending a pull request!
1. If this is your first time, please read our contributor guidelines: https://cwiki.apache.org/confluence/display/BIGTOP/How+to+Contribute
2. Make sure your PR title starts with JIRA issue id, e.g., 'BIGTOP-3638: Your PR title ...'.
-->
### Description of PR
Excluding log dependencies corresponding to hbase in the hive service
before:

after:

### How was this patch tested?
Launch hive CLI to view startup logs
### For code changes:
- [1 ] Does the title or this PR starts with the corresponding JIRA issue id (e.g. 'BIGTOP-3638. Your PR title ...')?
- [1] Make sure that newly added files do not have any licensing issues. When in doubt refer to https://www.apache.org/licenses/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@bigtop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] BIGTOP-4028 : Excluding log dependencies corresponding to hbase in the hive service [bigtop]
Posted by "iwasakims (via GitHub)" <gi...@apache.org>.
iwasakims commented on code in PR #1200:
URL: https://github.com/apache/bigtop/pull/1200#discussion_r1367914494
##########
bigtop-packages/src/common/hive/patch16-HIVE-SHELL.diff:
##########
@@ -0,0 +1,13 @@
+diff --git a/bin/hive b/bin/hive
+index 1ade51e..f645ef1 100755
+--- a/bin/hive
++++ b/bin/hive
+@@ -293,7 +293,7 @@ if [ "$SKIP_HBASECP" = false ]; then
+ # exclude ZK, PB, and Guava (See HIVE-2055)
+ # depends on HBASE-8438 (hbase-0.94.14+, hbase-0.96.1+) for `hbase mapredcp` command
+ for x in $($HBASE_BIN mapredcp 2>&2 | tr ':' '\n') ; do
+- if [[ $x == *zookeeper* || $x == *protobuf-java* || $x == *guava* ]] ; then
++ if [[ $x == *zookeeper* || $x == *protobuf-java* || $x == *guava* || $x == *log* || $x == *slf4j* ]] ; then
Review Comment:
`*log*` looks prone to future bug by matching unintentional files. What is the target jar here?
Same to `*slf4j*`. Are you really sure that all jars matching the pattern should be excluded?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@bigtop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] BIGTOP-4028 : Excluding log dependencies corresponding to hbase in the hive service [bigtop]
Posted by "LiJie20190102 (via GitHub)" <gi...@apache.org>.
LiJie20190102 commented on PR #1200:
URL: https://github.com/apache/bigtop/pull/1200#issuecomment-1772394593
> I meant the log messages related to HiveConf and FileUtils should be emitted even after fixing duplicate slf4j bindings.
I think fixing the duplicate SLf4j binding does not affect the previous log output. The following picture shows the scenario when I do not install Hbase:

--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@bigtop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] BIGTOP-4028 : Excluding log dependencies corresponding to hbase in the hive service [bigtop]
Posted by "iwasakims (via GitHub)" <gi...@apache.org>.
iwasakims commented on PR #1200:
URL: https://github.com/apache/bigtop/pull/1200#issuecomment-1772254274
Why ERROR and WARN messages are not shown at all after the patch is applied? Those messages seem not to be related to multiple slf4j bindings.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@bigtop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] BIGTOP-4028 : Excluding log dependencies corresponding to hbase in the hive service [bigtop]
Posted by "iwasakims (via GitHub)" <gi...@apache.org>.
iwasakims commented on PR #1200:
URL: https://github.com/apache/bigtop/pull/1200#issuecomment-1774107059
Hmm. I can not reproduce the issue by just installing both Hive and HBase. Is the issue already fixed in master (and maybe branch-3.2) by upgrading HBase? There is no slf4j-reload4j-1.7.33.jar in the output of `hbase mapredcp`.
```
root@0f9131e5418b:/# hbase mapredcp | sed -z 's/:/\n/g'
/usr/lib/hbase/lib/shaded-clients/hbase-shaded-mapreduce-2.4.17.jar
/usr/lib/hbase/lib/client-facing-thirdparty/audience-annotations-0.13.0.jar
/usr/lib/hbase/lib/client-facing-thirdparty/commons-logging-1.2.jar
/usr/lib/hbase/lib/client-facing-thirdparty/htrace-core4-4.2.0-incubating.jar
/usr/lib/hbase/lib/client-facing-thirdparty/jcl-over-slf4j-1.7.33.jar
/usr/lib/hbase/lib/client-facing-thirdparty/jul-to-slf4j-1.7.33.jar
/usr/lib/hbase/lib/client-facing-thirdparty/reload4j-1.2.22.jar
/usr/lib/hbase/lib/client-facing-thirdparty/slf4j-api-1.7.33.jar
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@bigtop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] BIGTOP-4028 : Excluding log dependencies corresponding to hbase in the hive service [bigtop]
Posted by "LiJie20190102 (via GitHub)" <gi...@apache.org>.
LiJie20190102 commented on PR #1200:
URL: https://github.com/apache/bigtop/pull/1200#issuecomment-1772306579
> Why ERROR and WARN messages are not shown at all after the patch is applied? Those messages seem not to be related to multiple slf4j bindings.
After installing hbase, there may be some log dependencies that conflict with hive's log dependencies:


--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@bigtop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] BIGTOP-4028 : Excluding log dependencies corresponding to hbase in the hive service [bigtop]
Posted by "iwasakims (via GitHub)" <gi...@apache.org>.
iwasakims commented on PR #1200:
URL: https://github.com/apache/bigtop/pull/1200#issuecomment-1772329516
I meant the log messages related to HiveConf and FileUtils should be emitted even after fixing duplicate slf4j bindings.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@bigtop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org