You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by GitBox <gi...@apache.org> on 2019/12/29 13:12:12 UTC
[GitHub] [spark] ajithme opened a new pull request #27042: [SPARK-30382]
start-thriftserver --help throws ClassNotFoundException
ajithme opened a new pull request #27042: [SPARK-30382] start-thriftserver --help throws ClassNotFoundException
URL: https://github.com/apache/spark/pull/27042
start-thriftserver.sh --help throws
```
.....
Thrift server options:
Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/logging/log4j/spi/LoggerContextFactory
at org.apache.hive.service.server.HiveServer2.main(HiveServer2.java:167)
at org.apache.spark.sql.hive.thriftserver.HiveThriftServer2$.main(HiveThriftServer2.scala:82)
at org.apache.spark.sql.hive.thriftserver.HiveThriftServer2.main(HiveThriftServer2.scala)
Caused by: java.lang.ClassNotFoundException: org.apache.logging.log4j.spi.LoggerContextFactory
at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
... 3 more
```
### What changes were proposed in this pull request?
Avoid hive log initialisation as https://github.com/apache/hive/blob/rel/release-2.3.5/common/src/java/org/apache/hadoop/hive/common/LogUtils.java introduces dependency over `org.apache.logging.log4j.core.impl.Log4jContextFactory` which is missing in our spark installer classpath directly. I believe the `LogUtils.initHiveLog4j()` code is here as the HiveServer2 class is copied from Hive.
### Why are the changes needed?
To make `start-thriftserver.sh --help` command success
### Does this PR introduce any user-facing change?
No
### How was this patch tested?
Checked Manually
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] srowen commented on issue #27042: [SPARK-30382]
start-thriftserver --help throws ClassNotFoundException
Posted by GitBox <gi...@apache.org>.
srowen commented on issue #27042: [SPARK-30382] start-thriftserver --help throws ClassNotFoundException
URL: https://github.com/apache/spark/pull/27042#issuecomment-569515852
So this only affects the 2.x code that is copied?
Yes, it's a copy, so we ideally don't want to modify it. I'm not sure what the implications are of not initializing the logging, right?
Is it not a matter of making sure the script or build includes log4j?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27042: [SPARK-30382]
start-thriftserver --help throws ClassNotFoundException
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27042: [SPARK-30382] start-thriftserver --help throws ClassNotFoundException
URL: https://github.com/apache/spark/pull/27042#issuecomment-569952852
Merged build finished. Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] ajithme commented on issue #27042: [SPARK-30382]
start-thriftserver --help throws ClassNotFoundException
Posted by GitBox <gi...@apache.org>.
ajithme commented on issue #27042: [SPARK-30382] start-thriftserver --help throws ClassNotFoundException
URL: https://github.com/apache/spark/pull/27042#issuecomment-569521497
> So this only affects the 2.x code that is copied?
> Yes, it's a copy, so we ideally don't want to modify it. I'm not sure what the implications are of not initializing the logging, right?
> Is it not a matter of making sure the script or build includes log4j?
Currently HiveServer2.main is called only to print help message, all other operations currently work without Hive log initialization. Please correct if wrong
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on issue #27042:
[SPARK-30382][SQL] Remove Hive LogUtils usage to prevent
ClassNotFoundException
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on issue #27042: [SPARK-30382][SQL] Remove Hive LogUtils usage to prevent ClassNotFoundException
URL: https://github.com/apache/spark/pull/27042#issuecomment-571629148
**[Test build #116244 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/116244/testReport)** for PR 27042 at commit [`d9935e6`](https://github.com/apache/spark/commit/d9935e6e34cd310d418b6cdac23df667cf08050a).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on issue #27042:
[SPARK-30382][SQL] Remove Hive LogUtils usage to prevent
ClassNotFoundException
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on issue #27042: [SPARK-30382][SQL] Remove Hive LogUtils usage to prevent ClassNotFoundException
URL: https://github.com/apache/spark/pull/27042#issuecomment-571610724
**[Test build #4988 has started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/4988/testReport)** for PR 27042 at commit [`d9935e6`](https://github.com/apache/spark/commit/d9935e6e34cd310d418b6cdac23df667cf08050a).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #27042: [SPARK-30382][SQL]
Remove Hive LogUtils usage to prevent ClassNotFoundException
Posted by GitBox <gi...@apache.org>.
SparkQA commented on issue #27042: [SPARK-30382][SQL] Remove Hive LogUtils usage to prevent ClassNotFoundException
URL: https://github.com/apache/spark/pull/27042#issuecomment-571629148
**[Test build #116244 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/116244/testReport)** for PR 27042 at commit [`d9935e6`](https://github.com/apache/spark/commit/d9935e6e34cd310d418b6cdac23df667cf08050a).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] srowen commented on issue #27042: [SPARK-30382]
start-thriftserver --help throws ClassNotFoundException
Posted by GitBox <gi...@apache.org>.
srowen commented on issue #27042: [SPARK-30382] start-thriftserver --help throws ClassNotFoundException
URL: https://github.com/apache/spark/pull/27042#issuecomment-569947647
Jenkins retest this please
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27042: [SPARK-30382][SQL]
Remove Hive LogUtils usage to prevent ClassNotFoundException
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27042: [SPARK-30382][SQL] Remove Hive LogUtils usage to prevent ClassNotFoundException
URL: https://github.com/apache/spark/pull/27042#issuecomment-571642623
Merged build finished. Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27042:
[SPARK-30382] start-thriftserver --help throws ClassNotFoundException
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27042: [SPARK-30382] start-thriftserver --help throws ClassNotFoundException
URL: https://github.com/apache/spark/pull/27042#issuecomment-569948244
Merged build finished. Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] srowen commented on a change in pull request #27042:
[SPARK-30382] start-thriftserver --help throws ClassNotFoundException
Posted by GitBox <gi...@apache.org>.
srowen commented on a change in pull request #27042: [SPARK-30382] start-thriftserver --help throws ClassNotFoundException
URL: https://github.com/apache/spark/pull/27042#discussion_r362008257
##########
File path: sql/hive-thriftserver/v2.3/src/main/java/org/apache/hive/service/server/HiveServer2.java
##########
@@ -158,25 +156,18 @@ private static void startHiveServer2() throws Throwable {
public static void main(String[] args) {
HiveConf.setLoadHiveServer2Config(true);
- try {
- ServerOptionsProcessor oproc = new ServerOptionsProcessor("hiveserver2");
- ServerOptionsProcessorResponse oprocResponse = oproc.parse(args);
+ ServerOptionsProcessor oproc = new ServerOptionsProcessor("hiveserver2");
+ ServerOptionsProcessorResponse oprocResponse = oproc.parse(args);
- // NOTE: It is critical to do this here so that log4j is reinitialized
- // before any of the other core hive classes are loaded
- String initLog4jMessage = LogUtils.initHiveLog4j();
- LOG.debug(initLog4jMessage);
- HiveStringUtils.startupShutdownMessage(HiveServer2.class, args, LOG);
+ // NOTE: It is critical to do this here so that log4j is reinitialized
Review comment:
Delete this comment?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] cloud-fan commented on a change in pull request #27042:
[SPARK-30382] start-thriftserver --help throws ClassNotFoundException
Posted by GitBox <gi...@apache.org>.
cloud-fan commented on a change in pull request #27042: [SPARK-30382] start-thriftserver --help throws ClassNotFoundException
URL: https://github.com/apache/spark/pull/27042#discussion_r362392825
##########
File path: sql/hive-thriftserver/v2.3/src/main/java/org/apache/hive/service/server/HiveServer2.java
##########
@@ -158,25 +156,16 @@ private static void startHiveServer2() throws Throwable {
public static void main(String[] args) {
HiveConf.setLoadHiveServer2Config(true);
- try {
- ServerOptionsProcessor oproc = new ServerOptionsProcessor("hiveserver2");
- ServerOptionsProcessorResponse oprocResponse = oproc.parse(args);
+ ServerOptionsProcessor oproc = new ServerOptionsProcessor("hiveserver2");
+ ServerOptionsProcessorResponse oprocResponse = oproc.parse(args);
- // NOTE: It is critical to do this here so that log4j is reinitialized
- // before any of the other core hive classes are loaded
- String initLog4jMessage = LogUtils.initHiveLog4j();
- LOG.debug(initLog4jMessage);
- HiveStringUtils.startupShutdownMessage(HiveServer2.class, args, LOG);
+ HiveStringUtils.startupShutdownMessage(HiveServer2.class, args, LOG);
- // Log debug message from "oproc" after log4j initialize properly
- LOG.debug(oproc.getDebugMessage().toString());
+ // Log debug message from "oproc" after log4j initialize properly
+ LOG.debug(oproc.getDebugMessage().toString());
Review comment:
is it OK to call `LOG.debug` without initialization?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] HeartSaVioR commented on issue #27042: [SPARK-30382]
start-thriftserver --help throws ClassNotFoundException
Posted by GitBox <gi...@apache.org>.
HeartSaVioR commented on issue #27042: [SPARK-30382] start-thriftserver --help throws ClassNotFoundException
URL: https://github.com/apache/spark/pull/27042#issuecomment-569568378
Looks like the problem is that Hive doesn't only leverage slf4j, but also leverages Log4j"2" API to programmatically configuring Log4j via configured in Hive conf. If I understand correctly, Spark uses Log4j 1, and that is also applied to hive-thriftserver module (since it should run with Spark dependencies).
We also seem to exclude everything for log4j2 as it conflicts with log4j 1:
```
<!-- org.apache.logging.log4j:* conflict with log4j 1.2.17 -->
<exclusion>
<groupId>org.apache.logging.log4j</groupId>
<artifactId>*</artifactId>
</exclusion>
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] ajithme removed a comment on issue #27042: [SPARK-30382]
start-thriftserver --help throws ClassNotFoundException
Posted by GitBox <gi...@apache.org>.
ajithme removed a comment on issue #27042: [SPARK-30382] start-thriftserver --help throws ClassNotFoundException
URL: https://github.com/apache/spark/pull/27042#issuecomment-569520422
> So this only affects the 2.x code that is copied?
> Yes, it's a copy, so we ideally don't want to modify it. I'm not sure what the implications are of not initializing the logging, right?
> Is it not a matter of making sure the script or build includes log4j?
I see HiveServer2.main being called only to print help message. Hive log initialisation is skipped for all other operations. pls correct me if wrong
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27042:
[SPARK-30382] start-thriftserver --help throws ClassNotFoundException
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27042: [SPARK-30382] start-thriftserver --help throws ClassNotFoundException
URL: https://github.com/apache/spark/pull/27042#issuecomment-569504830
Can one of the admins verify this patch?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] dongjoon-hyun closed pull request #27042:
[SPARK-30382][SQL] Remove Hive LogUtils usage to prevent
ClassNotFoundException
Posted by GitBox <gi...@apache.org>.
dongjoon-hyun closed pull request #27042: [SPARK-30382][SQL] Remove Hive LogUtils usage to prevent ClassNotFoundException
URL: https://github.com/apache/spark/pull/27042
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] ajithme edited a comment on issue #27042: [SPARK-30382]
start-thriftserver --help throws ClassNotFoundException
Posted by GitBox <gi...@apache.org>.
ajithme edited a comment on issue #27042: [SPARK-30382] start-thriftserver --help throws ClassNotFoundException
URL: https://github.com/apache/spark/pull/27042#issuecomment-569520422
> So this only affects the 2.x code that is copied?
> Yes, it's a copy, so we ideally don't want to modify it. I'm not sure what the implications are of not initializing the logging, right?
> Is it not a matter of making sure the script or build includes log4j?
I see HiveServer2.main being called only to print help message. Hive log initialisation is skipped for all other operations. pls correct me if wrong
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27042:
[SPARK-30382] start-thriftserver --help throws ClassNotFoundException
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27042: [SPARK-30382] start-thriftserver --help throws ClassNotFoundException
URL: https://github.com/apache/spark/pull/27042#issuecomment-569948248
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/20786/
Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #27042: [SPARK-30382][SQL]
Remove Hive LogUtils usage to prevent ClassNotFoundException
Posted by GitBox <gi...@apache.org>.
SparkQA commented on issue #27042: [SPARK-30382][SQL] Remove Hive LogUtils usage to prevent ClassNotFoundException
URL: https://github.com/apache/spark/pull/27042#issuecomment-571642455
**[Test build #116244 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/116244/testReport)** for PR 27042 at commit [`d9935e6`](https://github.com/apache/spark/commit/d9935e6e34cd310d418b6cdac23df667cf08050a).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27042: [SPARK-30382]
start-thriftserver --help throws ClassNotFoundException
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27042: [SPARK-30382] start-thriftserver --help throws ClassNotFoundException
URL: https://github.com/apache/spark/pull/27042#issuecomment-569948244
Merged build finished. Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #27042: [SPARK-30382][SQL]
Remove Hive LogUtils usage to prevent ClassNotFoundException
Posted by GitBox <gi...@apache.org>.
SparkQA commented on issue #27042: [SPARK-30382][SQL] Remove Hive LogUtils usage to prevent ClassNotFoundException
URL: https://github.com/apache/spark/pull/27042#issuecomment-571423601
**[Test build #116207 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/116207/testReport)** for PR 27042 at commit [`a30a678`](https://github.com/apache/spark/commit/a30a678f5d3fbe78b161aab4682b9e1c18fd0dd2).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27042:
[SPARK-30382][SQL] Remove Hive LogUtils usage to prevent
ClassNotFoundException
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27042: [SPARK-30382][SQL] Remove Hive LogUtils usage to prevent ClassNotFoundException
URL: https://github.com/apache/spark/pull/27042#issuecomment-571626324
Merged build finished. Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] HeartSaVioR edited a comment on issue #27042: [SPARK-30382]
start-thriftserver --help throws ClassNotFoundException
Posted by GitBox <gi...@apache.org>.
HeartSaVioR edited a comment on issue #27042: [SPARK-30382] start-thriftserver --help throws ClassNotFoundException
URL: https://github.com/apache/spark/pull/27042#issuecomment-569568378
Looks like the problem is that Hive doesn't only leverage slf4j, but also leverages Log4j"2" API to programmatically configuring Log4j via configured in Hive conf. If I understand correctly, Spark uses Log4j 1, and that is also applied to hive-thriftserver module (since it should run with Spark dependencies).
We also seem to exclude everything for log4j2 as it conflicts with log4j 1:
```
<!-- org.apache.logging.log4j:* conflict with log4j 1.2.17 -->
<exclusion>
<groupId>org.apache.logging.log4j</groupId>
<artifactId>*</artifactId>
</exclusion>
```
so the code may need to be removed if it requires log4j2 either directly or indirectly (transitive dependency).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27042: [SPARK-30382]
start-thriftserver --help throws ClassNotFoundException
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27042: [SPARK-30382] start-thriftserver --help throws ClassNotFoundException
URL: https://github.com/apache/spark/pull/27042#issuecomment-571415921
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/20997/
Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27042: [SPARK-30382]
start-thriftserver --help throws ClassNotFoundException
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27042: [SPARK-30382] start-thriftserver --help throws ClassNotFoundException
URL: https://github.com/apache/spark/pull/27042#issuecomment-569952858
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/115994/
Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #27042: [SPARK-30382]
start-thriftserver --help throws ClassNotFoundException
Posted by GitBox <gi...@apache.org>.
SparkQA commented on issue #27042: [SPARK-30382] start-thriftserver --help throws ClassNotFoundException
URL: https://github.com/apache/spark/pull/27042#issuecomment-569947589
**[Test build #4980 has finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/4980/testReport)** for PR 27042 at commit [`74a6b34`](https://github.com/apache/spark/commit/74a6b3495b8f11ae9dc3d9be942c307c4abeb37f).
* This patch **fails Spark unit tests**.
* This patch merges cleanly.
* This patch adds no public classes.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27042:
[SPARK-30382] start-thriftserver --help throws ClassNotFoundException
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27042: [SPARK-30382] start-thriftserver --help throws ClassNotFoundException
URL: https://github.com/apache/spark/pull/27042#issuecomment-569952858
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/115994/
Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on issue #27042: [SPARK-30382]
start-thriftserver --help throws ClassNotFoundException
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on issue #27042: [SPARK-30382] start-thriftserver --help throws ClassNotFoundException
URL: https://github.com/apache/spark/pull/27042#issuecomment-569943391
**[Test build #4980 has started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/4980/testReport)** for PR 27042 at commit [`74a6b34`](https://github.com/apache/spark/commit/74a6b3495b8f11ae9dc3d9be942c307c4abeb37f).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] ajithme commented on a change in pull request #27042:
[SPARK-30382] start-thriftserver --help throws ClassNotFoundException
Posted by GitBox <gi...@apache.org>.
ajithme commented on a change in pull request #27042: [SPARK-30382] start-thriftserver --help throws ClassNotFoundException
URL: https://github.com/apache/spark/pull/27042#discussion_r362021613
##########
File path: sql/hive-thriftserver/v2.3/src/main/java/org/apache/hive/service/server/HiveServer2.java
##########
@@ -158,25 +156,18 @@ private static void startHiveServer2() throws Throwable {
public static void main(String[] args) {
HiveConf.setLoadHiveServer2Config(true);
- try {
- ServerOptionsProcessor oproc = new ServerOptionsProcessor("hiveserver2");
- ServerOptionsProcessorResponse oprocResponse = oproc.parse(args);
+ ServerOptionsProcessor oproc = new ServerOptionsProcessor("hiveserver2");
+ ServerOptionsProcessorResponse oprocResponse = oproc.parse(args);
- // NOTE: It is critical to do this here so that log4j is reinitialized
- // before any of the other core hive classes are loaded
- String initLog4jMessage = LogUtils.initHiveLog4j();
- LOG.debug(initLog4jMessage);
- HiveStringUtils.startupShutdownMessage(HiveServer2.class, args, LOG);
+ // NOTE: It is critical to do this here so that log4j is reinitialized
Review comment:
Done
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27042: [SPARK-30382][SQL]
Remove Hive LogUtils usage to prevent ClassNotFoundException
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27042: [SPARK-30382][SQL] Remove Hive LogUtils usage to prevent ClassNotFoundException
URL: https://github.com/apache/spark/pull/27042#issuecomment-571423669
Merged build finished. Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27042: [SPARK-30382]
start-thriftserver --help throws ClassNotFoundException
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27042: [SPARK-30382] start-thriftserver --help throws ClassNotFoundException
URL: https://github.com/apache/spark/pull/27042#issuecomment-569504830
Can one of the admins verify this patch?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] ajithme commented on issue #27042: [SPARK-30382][SQL]
Remove Hive LogUtils usage to prevent ClassNotFoundException
Posted by GitBox <gi...@apache.org>.
ajithme commented on issue #27042: [SPARK-30382][SQL] Remove Hive LogUtils usage to prevent ClassNotFoundException
URL: https://github.com/apache/spark/pull/27042#issuecomment-571477332
> Hi, @ajithme . Thank you for your contribution. I have two comments.
>
> 1. This PR fixes only Hive 2.3 code path. We need to remove Hive 1.2 code path. We want to keep both source code path identical as much as possible.
>
> ```
> $ git grep 'org.apache.hadoop.hive.common.LogUtils'
> sql/hive-thriftserver/v1.2/src/main/java/org/apache/hive/service/server/HiveServer2.java:import org.apache.hadoop.hive.common.LogUtils;
> sql/hive-thriftserver/v1.2/src/main/java/org/apache/hive/service/server/HiveServer2.java:import org.apache.hadoop.hive.common.LogUtils.LogInitializationException;
> sql/hive-thriftserver/v2.3/src/main/java/org/apache/hive/service/server/HiveServer2.java:import org.apache.hadoop.hive.common.LogUtils;
> sql/hive-thriftserver/v2.3/src/main/java/org/apache/hive/service/server/HiveServer2.java:import org.apache.hadoop.hive.common.LogUtils.LogInitializationException;
> ```
>
> 1. It would be better describe your PR's content in the PR title. You don't need to copy the JIRA title. I updated a little for you at this time. Also, I updated the PR description, too.
@dongjoon-hyun Thanks for the inputs. I have updated the PR to Handle for Hive 1.2. Also thanks for updating PR details, will keep in mind next time.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27042:
[SPARK-30382][SQL] Remove Hive LogUtils usage to prevent
ClassNotFoundException
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27042: [SPARK-30382][SQL] Remove Hive LogUtils usage to prevent ClassNotFoundException
URL: https://github.com/apache/spark/pull/27042#issuecomment-571642629
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/116244/
Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27042:
[SPARK-30382] start-thriftserver --help throws ClassNotFoundException
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27042: [SPARK-30382] start-thriftserver --help throws ClassNotFoundException
URL: https://github.com/apache/spark/pull/27042#issuecomment-571415915
Merged build finished. Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27042: [SPARK-30382]
start-thriftserver --help throws ClassNotFoundException
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27042: [SPARK-30382] start-thriftserver --help throws ClassNotFoundException
URL: https://github.com/apache/spark/pull/27042#issuecomment-569504702
Can one of the admins verify this patch?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] ajithme commented on a change in pull request #27042:
[SPARK-30382] start-thriftserver --help throws ClassNotFoundException
Posted by GitBox <gi...@apache.org>.
ajithme commented on a change in pull request #27042: [SPARK-30382] start-thriftserver --help throws ClassNotFoundException
URL: https://github.com/apache/spark/pull/27042#discussion_r362437714
##########
File path: sql/hive-thriftserver/v2.3/src/main/java/org/apache/hive/service/server/HiveServer2.java
##########
@@ -158,25 +156,16 @@ private static void startHiveServer2() throws Throwable {
public static void main(String[] args) {
HiveConf.setLoadHiveServer2Config(true);
- try {
- ServerOptionsProcessor oproc = new ServerOptionsProcessor("hiveserver2");
- ServerOptionsProcessorResponse oprocResponse = oproc.parse(args);
+ ServerOptionsProcessor oproc = new ServerOptionsProcessor("hiveserver2");
+ ServerOptionsProcessorResponse oprocResponse = oproc.parse(args);
- // NOTE: It is critical to do this here so that log4j is reinitialized
- // before any of the other core hive classes are loaded
- String initLog4jMessage = LogUtils.initHiveLog4j();
- LOG.debug(initLog4jMessage);
- HiveStringUtils.startupShutdownMessage(HiveServer2.class, args, LOG);
+ HiveStringUtils.startupShutdownMessage(HiveServer2.class, args, LOG);
- // Log debug message from "oproc" after log4j initialize properly
- LOG.debug(oproc.getDebugMessage().toString());
+ // Log debug message from "oproc" after log4j initialize properly
+ LOG.debug(oproc.getDebugMessage().toString());
Review comment:
It will ignore the log message, but not throw any exception. As we see here, we initialise spark loggers after this call, https://github.com/apache/spark/blob/master/sql/hive-thriftserver/src/main/scala/org/apache/spark/sql/hive/thriftserver/HiveThriftServer2.scala#L87
So calling logger before this (even using commons.logging) will not print anything in logs as it doesn't have any appenders
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] ajithme commented on a change in pull request #27042:
[SPARK-30382] start-thriftserver --help throws ClassNotFoundException
Posted by GitBox <gi...@apache.org>.
ajithme commented on a change in pull request #27042: [SPARK-30382] start-thriftserver --help throws ClassNotFoundException
URL: https://github.com/apache/spark/pull/27042#discussion_r362448433
##########
File path: sql/hive-thriftserver/v2.3/src/main/java/org/apache/hive/service/server/HiveServer2.java
##########
@@ -158,25 +156,16 @@ private static void startHiveServer2() throws Throwable {
public static void main(String[] args) {
HiveConf.setLoadHiveServer2Config(true);
- try {
- ServerOptionsProcessor oproc = new ServerOptionsProcessor("hiveserver2");
- ServerOptionsProcessorResponse oprocResponse = oproc.parse(args);
+ ServerOptionsProcessor oproc = new ServerOptionsProcessor("hiveserver2");
+ ServerOptionsProcessorResponse oprocResponse = oproc.parse(args);
- // NOTE: It is critical to do this here so that log4j is reinitialized
- // before any of the other core hive classes are loaded
- String initLog4jMessage = LogUtils.initHiveLog4j();
- LOG.debug(initLog4jMessage);
- HiveStringUtils.startupShutdownMessage(HiveServer2.class, args, LOG);
+ HiveStringUtils.startupShutdownMessage(HiveServer2.class, args, LOG);
- // Log debug message from "oproc" after log4j initialize properly
- LOG.debug(oproc.getDebugMessage().toString());
+ // Log debug message from "oproc" after log4j initialize properly
+ LOG.debug(oproc.getDebugMessage().toString());
Review comment:
@steveloughran as i see the code `toString()` is over a `StringBuilder` which is always initialised in `org.apache.hive.service.server.HiveServer2.ServerOptionsProcessor#debugMessage` and as also this is java file dint use scala string expansion. Please correct me if wrong.
@cloud-fan removed as per suggestion
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27042:
[SPARK-30382] start-thriftserver --help throws ClassNotFoundException
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27042: [SPARK-30382] start-thriftserver --help throws ClassNotFoundException
URL: https://github.com/apache/spark/pull/27042#issuecomment-571415921
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/20997/
Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #27042: [SPARK-30382][SQL]
Remove Hive LogUtils usage to prevent ClassNotFoundException
Posted by GitBox <gi...@apache.org>.
SparkQA commented on issue #27042: [SPARK-30382][SQL] Remove Hive LogUtils usage to prevent ClassNotFoundException
URL: https://github.com/apache/spark/pull/27042#issuecomment-571417402
**[Test build #116207 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/116207/testReport)** for PR 27042 at commit [`a30a678`](https://github.com/apache/spark/commit/a30a678f5d3fbe78b161aab4682b9e1c18fd0dd2).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27042:
[SPARK-30382][SQL] Remove Hive LogUtils usage to prevent
ClassNotFoundException
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27042: [SPARK-30382][SQL] Remove Hive LogUtils usage to prevent ClassNotFoundException
URL: https://github.com/apache/spark/pull/27042#issuecomment-571626339
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/21037/
Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #27042: [SPARK-30382]
start-thriftserver --help throws ClassNotFoundException
Posted by GitBox <gi...@apache.org>.
SparkQA commented on issue #27042: [SPARK-30382] start-thriftserver --help throws ClassNotFoundException
URL: https://github.com/apache/spark/pull/27042#issuecomment-569947987
**[Test build #115994 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/115994/testReport)** for PR 27042 at commit [`74a6b34`](https://github.com/apache/spark/commit/74a6b3495b8f11ae9dc3d9be942c307c4abeb37f).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27042:
[SPARK-30382] start-thriftserver --help throws ClassNotFoundException
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27042: [SPARK-30382] start-thriftserver --help throws ClassNotFoundException
URL: https://github.com/apache/spark/pull/27042#issuecomment-569952852
Merged build finished. Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #27042: [SPARK-30382]
start-thriftserver --help throws ClassNotFoundException
Posted by GitBox <gi...@apache.org>.
SparkQA commented on issue #27042: [SPARK-30382] start-thriftserver --help throws ClassNotFoundException
URL: https://github.com/apache/spark/pull/27042#issuecomment-569943391
**[Test build #4980 has started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/4980/testReport)** for PR 27042 at commit [`74a6b34`](https://github.com/apache/spark/commit/74a6b3495b8f11ae9dc3d9be942c307c4abeb37f).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27042: [SPARK-30382]
start-thriftserver --help throws ClassNotFoundException
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27042: [SPARK-30382] start-thriftserver --help throws ClassNotFoundException
URL: https://github.com/apache/spark/pull/27042#issuecomment-571415915
Merged build finished. Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27042: [SPARK-30382]
start-thriftserver --help throws ClassNotFoundException
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27042: [SPARK-30382] start-thriftserver --help throws ClassNotFoundException
URL: https://github.com/apache/spark/pull/27042#issuecomment-569948248
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/20786/
Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] ajithme commented on issue #27042: [SPARK-30382]
start-thriftserver --help throws ClassNotFoundException
Posted by GitBox <gi...@apache.org>.
ajithme commented on issue #27042: [SPARK-30382] start-thriftserver --help throws ClassNotFoundException
URL: https://github.com/apache/spark/pull/27042#issuecomment-569504343
cc @srowen @wangyum @dongjoon-hyun @cloud-fan
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #27042: [SPARK-30382][SQL]
Remove Hive LogUtils usage to prevent ClassNotFoundException
Posted by GitBox <gi...@apache.org>.
SparkQA commented on issue #27042: [SPARK-30382][SQL] Remove Hive LogUtils usage to prevent ClassNotFoundException
URL: https://github.com/apache/spark/pull/27042#issuecomment-571610724
**[Test build #4988 has started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/4988/testReport)** for PR 27042 at commit [`d9935e6`](https://github.com/apache/spark/commit/d9935e6e34cd310d418b6cdac23df667cf08050a).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27042:
[SPARK-30382] start-thriftserver --help throws ClassNotFoundException
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27042: [SPARK-30382] start-thriftserver --help throws ClassNotFoundException
URL: https://github.com/apache/spark/pull/27042#issuecomment-569504702
Can one of the admins verify this patch?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27042: [SPARK-30382][SQL]
Remove Hive LogUtils usage to prevent ClassNotFoundException
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27042: [SPARK-30382][SQL] Remove Hive LogUtils usage to prevent ClassNotFoundException
URL: https://github.com/apache/spark/pull/27042#issuecomment-571626339
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/21037/
Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] ajithme commented on issue #27042: [SPARK-30382]
start-thriftserver --help throws ClassNotFoundException
Posted by GitBox <gi...@apache.org>.
ajithme commented on issue #27042: [SPARK-30382] start-thriftserver --help throws ClassNotFoundException
URL: https://github.com/apache/spark/pull/27042#issuecomment-569714382
> I doubt we can include log4j 2 successfully. Hm. So this code path doesn't get used unless --help is called? can you summarize why, because this looks like a main entry point. I don't know this code at all though. I'm worried it will break other logging.
Sure, currently for spark thrift server, the entrypoint class is `org.apache.spark.sql.hive.thriftserver.HiveThriftServer2` https://github.com/apache/spark/blob/master/sbin/start-thriftserver.sh#L32
In `org.apache.spark.sql.hive.thriftserver.HiveThriftServer2#main`, we can see, its calling hive's class `org.apache.hive.service.server.HiveServer2#main` only incase of argument passed to the HiveThriftServer2#main is `--help` or `-h`, referring to code at https://github.com/apache/spark/blob/master/sql/hive-thriftserver/src/main/scala/org/apache/spark/sql/hive/thriftserver/HiveThriftServer2.scala#L81 which inturn invokes our culprit `LogUtils.initHiveLog4j()`
So, Any other flow which is not passing help argument to `org.apache.spark.sql.hive.thriftserver.HiveThriftServer2`, it will not invoke `org.apache.hive.service.server.HiveServer2#main`
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27042:
[SPARK-30382][SQL] Remove Hive LogUtils usage to prevent
ClassNotFoundException
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27042: [SPARK-30382][SQL] Remove Hive LogUtils usage to prevent ClassNotFoundException
URL: https://github.com/apache/spark/pull/27042#issuecomment-571642623
Merged build finished. Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] ajithme commented on issue #27042: [SPARK-30382]
start-thriftserver --help throws ClassNotFoundException
Posted by GitBox <gi...@apache.org>.
ajithme commented on issue #27042: [SPARK-30382] start-thriftserver --help throws ClassNotFoundException
URL: https://github.com/apache/spark/pull/27042#issuecomment-569520422
> So this only affects the 2.x code that is copied?
> Yes, it's a copy, so we ideally don't want to modify it. I'm not sure what the implications are of not initializing the logging, right?
> Is it not a matter of making sure the script or build includes log4j?
I see HiveServer2.main being called only to print help message.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #27042: [SPARK-30382]
start-thriftserver --help throws ClassNotFoundException
Posted by GitBox <gi...@apache.org>.
SparkQA commented on issue #27042: [SPARK-30382] start-thriftserver --help throws ClassNotFoundException
URL: https://github.com/apache/spark/pull/27042#issuecomment-569952789
**[Test build #115994 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/115994/testReport)** for PR 27042 at commit [`74a6b34`](https://github.com/apache/spark/commit/74a6b3495b8f11ae9dc3d9be942c307c4abeb37f).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27042: [SPARK-30382][SQL]
Remove Hive LogUtils usage to prevent ClassNotFoundException
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27042: [SPARK-30382][SQL] Remove Hive LogUtils usage to prevent ClassNotFoundException
URL: https://github.com/apache/spark/pull/27042#issuecomment-571626324
Merged build finished. Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on issue #27042:
[SPARK-30382][SQL] Remove Hive LogUtils usage to prevent
ClassNotFoundException
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on issue #27042: [SPARK-30382][SQL] Remove Hive LogUtils usage to prevent ClassNotFoundException
URL: https://github.com/apache/spark/pull/27042#issuecomment-571417402
**[Test build #116207 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/116207/testReport)** for PR 27042 at commit [`a30a678`](https://github.com/apache/spark/commit/a30a678f5d3fbe78b161aab4682b9e1c18fd0dd2).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] steveloughran commented on a change in pull request #27042:
[SPARK-30382] start-thriftserver --help throws ClassNotFoundException
Posted by GitBox <gi...@apache.org>.
steveloughran commented on a change in pull request #27042: [SPARK-30382] start-thriftserver --help throws ClassNotFoundException
URL: https://github.com/apache/spark/pull/27042#discussion_r362443083
##########
File path: sql/hive-thriftserver/v2.3/src/main/java/org/apache/hive/service/server/HiveServer2.java
##########
@@ -158,25 +156,16 @@ private static void startHiveServer2() throws Throwable {
public static void main(String[] args) {
HiveConf.setLoadHiveServer2Config(true);
- try {
- ServerOptionsProcessor oproc = new ServerOptionsProcessor("hiveserver2");
- ServerOptionsProcessorResponse oprocResponse = oproc.parse(args);
+ ServerOptionsProcessor oproc = new ServerOptionsProcessor("hiveserver2");
+ ServerOptionsProcessorResponse oprocResponse = oproc.parse(args);
- // NOTE: It is critical to do this here so that log4j is reinitialized
- // before any of the other core hive classes are loaded
- String initLog4jMessage = LogUtils.initHiveLog4j();
- LOG.debug(initLog4jMessage);
- HiveStringUtils.startupShutdownMessage(HiveServer2.class, args, LOG);
+ HiveStringUtils.startupShutdownMessage(HiveServer2.class, args, LOG);
- // Log debug message from "oproc" after log4j initialize properly
- LOG.debug(oproc.getDebugMessage().toString());
+ // Log debug message from "oproc" after log4j initialize properly
+ LOG.debug(oproc.getDebugMessage().toString());
Review comment:
the toString() could trigger exceptions (NPE etc) If you aren't careful and there is the extra cost of that operation. Its why scala string expansion and SLF4J {} entries are better
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on issue #27042: [SPARK-30382]
start-thriftserver --help throws ClassNotFoundException
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on issue #27042: [SPARK-30382] start-thriftserver --help throws ClassNotFoundException
URL: https://github.com/apache/spark/pull/27042#issuecomment-569947987
**[Test build #115994 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/115994/testReport)** for PR 27042 at commit [`74a6b34`](https://github.com/apache/spark/commit/74a6b3495b8f11ae9dc3d9be942c307c4abeb37f).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] srowen commented on issue #27042: [SPARK-30382]
start-thriftserver --help throws ClassNotFoundException
Posted by GitBox <gi...@apache.org>.
srowen commented on issue #27042: [SPARK-30382] start-thriftserver --help throws ClassNotFoundException
URL: https://github.com/apache/spark/pull/27042#issuecomment-569530549
That I don't know. @wangyum @steveloughran maybe, do you have any insight into what this is used for?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] srowen commented on issue #27042: [SPARK-30382]
start-thriftserver --help throws ClassNotFoundException
Posted by GitBox <gi...@apache.org>.
srowen commented on issue #27042: [SPARK-30382] start-thriftserver --help throws ClassNotFoundException
URL: https://github.com/apache/spark/pull/27042#issuecomment-569943971
OK I think I buy it.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] cloud-fan commented on a change in pull request #27042:
[SPARK-30382] start-thriftserver --help throws ClassNotFoundException
Posted by GitBox <gi...@apache.org>.
cloud-fan commented on a change in pull request #27042: [SPARK-30382] start-thriftserver --help throws ClassNotFoundException
URL: https://github.com/apache/spark/pull/27042#discussion_r362444209
##########
File path: sql/hive-thriftserver/v2.3/src/main/java/org/apache/hive/service/server/HiveServer2.java
##########
@@ -158,25 +156,16 @@ private static void startHiveServer2() throws Throwable {
public static void main(String[] args) {
HiveConf.setLoadHiveServer2Config(true);
- try {
- ServerOptionsProcessor oproc = new ServerOptionsProcessor("hiveserver2");
- ServerOptionsProcessorResponse oprocResponse = oproc.parse(args);
+ ServerOptionsProcessor oproc = new ServerOptionsProcessor("hiveserver2");
+ ServerOptionsProcessorResponse oprocResponse = oproc.parse(args);
- // NOTE: It is critical to do this here so that log4j is reinitialized
- // before any of the other core hive classes are loaded
- String initLog4jMessage = LogUtils.initHiveLog4j();
- LOG.debug(initLog4jMessage);
- HiveStringUtils.startupShutdownMessage(HiveServer2.class, args, LOG);
+ HiveStringUtils.startupShutdownMessage(HiveServer2.class, args, LOG);
- // Log debug message from "oproc" after log4j initialize properly
- LOG.debug(oproc.getDebugMessage().toString());
+ // Log debug message from "oproc" after log4j initialize properly
+ LOG.debug(oproc.getDebugMessage().toString());
Review comment:
shall we just remove this line if it's noop?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] ajithme edited a comment on issue #27042: [SPARK-30382]
start-thriftserver --help throws ClassNotFoundException
Posted by GitBox <gi...@apache.org>.
ajithme edited a comment on issue #27042: [SPARK-30382] start-thriftserver --help throws ClassNotFoundException
URL: https://github.com/apache/spark/pull/27042#issuecomment-569520422
> So this only affects the 2.x code that is copied?
> Yes, it's a copy, so we ideally don't want to modify it. I'm not sure what the implications are of not initializing the logging, right?
> Is it not a matter of making sure the script or build includes log4j?
I see HiveServer2.main being called only to print help message. pls correct me if wrong
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27042: [SPARK-30382][SQL]
Remove Hive LogUtils usage to prevent ClassNotFoundException
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27042: [SPARK-30382][SQL] Remove Hive LogUtils usage to prevent ClassNotFoundException
URL: https://github.com/apache/spark/pull/27042#issuecomment-571423675
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/116207/
Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] dongjoon-hyun commented on a change in pull request #27042:
[SPARK-30382][SQL] Remove Hive LogUtils usage to prevent
ClassNotFoundException
Posted by GitBox <gi...@apache.org>.
dongjoon-hyun commented on a change in pull request #27042: [SPARK-30382][SQL] Remove Hive LogUtils usage to prevent ClassNotFoundException
URL: https://github.com/apache/spark/pull/27042#discussion_r363980733
##########
File path: sql/hive-thriftserver/v2.3/src/main/java/org/apache/hive/service/server/HiveServer2.java
##########
@@ -158,25 +156,13 @@ private static void startHiveServer2() throws Throwable {
public static void main(String[] args) {
HiveConf.setLoadHiveServer2Config(true);
- try {
- ServerOptionsProcessor oproc = new ServerOptionsProcessor("hiveserver2");
- ServerOptionsProcessorResponse oprocResponse = oproc.parse(args);
+ ServerOptionsProcessor oproc = new ServerOptionsProcessor("hiveserver2");
+ ServerOptionsProcessorResponse oprocResponse = oproc.parse(args);
- // NOTE: It is critical to do this here so that log4j is reinitialized
- // before any of the other core hive classes are loaded
- String initLog4jMessage = LogUtils.initHiveLog4j();
- LOG.debug(initLog4jMessage);
- HiveStringUtils.startupShutdownMessage(HiveServer2.class, args, LOG);
+ HiveStringUtils.startupShutdownMessage(HiveServer2.class, args, LOG);
- // Log debug message from "oproc" after log4j initialize properly
- LOG.debug(oproc.getDebugMessage().toString());
-
- // Call the executor which will execute the appropriate command based on the parsed options
- oprocResponse.getServerOptionsExecutor().execute();
- } catch (LogInitializationException e) {
- LOG.error("Error initializing log: " + e.getMessage(), e);
- System.exit(-1);
- }
+ // Call the executor which will execute s appropriate command based on the parsed options
Review comment:
I will fix this line during merging. This line seems to be changed accidentaly.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27042: [SPARK-30382][SQL]
Remove Hive LogUtils usage to prevent ClassNotFoundException
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27042: [SPARK-30382][SQL] Remove Hive LogUtils usage to prevent ClassNotFoundException
URL: https://github.com/apache/spark/pull/27042#issuecomment-571642629
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/116244/
Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27042:
[SPARK-30382][SQL] Remove Hive LogUtils usage to prevent
ClassNotFoundException
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27042: [SPARK-30382][SQL] Remove Hive LogUtils usage to prevent ClassNotFoundException
URL: https://github.com/apache/spark/pull/27042#issuecomment-571423675
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/116207/
Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] srowen commented on issue #27042: [SPARK-30382][SQL] Remove
Hive LogUtils usage to prevent ClassNotFoundException
Posted by GitBox <gi...@apache.org>.
srowen commented on issue #27042: [SPARK-30382][SQL] Remove Hive LogUtils usage to prevent ClassNotFoundException
URL: https://github.com/apache/spark/pull/27042#issuecomment-571625704
Jenkins retest this please
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27042:
[SPARK-30382][SQL] Remove Hive LogUtils usage to prevent
ClassNotFoundException
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27042: [SPARK-30382][SQL] Remove Hive LogUtils usage to prevent ClassNotFoundException
URL: https://github.com/apache/spark/pull/27042#issuecomment-571423669
Merged build finished. Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] dongjoon-hyun commented on issue #27042: [SPARK-30382]
start-thriftserver --help throws ClassNotFoundException
Posted by GitBox <gi...@apache.org>.
dongjoon-hyun commented on issue #27042: [SPARK-30382] start-thriftserver --help throws ClassNotFoundException
URL: https://github.com/apache/spark/pull/27042#issuecomment-571415663
Retest this please.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #27042: [SPARK-30382][SQL]
Remove Hive LogUtils usage to prevent ClassNotFoundException
Posted by GitBox <gi...@apache.org>.
SparkQA commented on issue #27042: [SPARK-30382][SQL] Remove Hive LogUtils usage to prevent ClassNotFoundException
URL: https://github.com/apache/spark/pull/27042#issuecomment-571623541
**[Test build #4988 has finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/4988/testReport)** for PR 27042 at commit [`d9935e6`](https://github.com/apache/spark/commit/d9935e6e34cd310d418b6cdac23df667cf08050a).
* This patch **fails Spark unit tests**.
* This patch merges cleanly.
* This patch adds no public classes.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] ajithme commented on issue #27042: [SPARK-30382]
start-thriftserver --help throws ClassNotFoundException
Posted by GitBox <gi...@apache.org>.
ajithme commented on issue #27042: [SPARK-30382] start-thriftserver --help throws ClassNotFoundException
URL: https://github.com/apache/spark/pull/27042#issuecomment-569594377
Thanks for the details @HeartSaVioR So what is the suggestion here, should we
1. include `log4j2` to complete the dependency OR
2. drop the code which leverages on log4j2,
personally after tracing the code i see log4j2 for hive is only initalized in help flow so it should be safe to delete the `LogUtils.initHiveLog4j()` , hence the PR follows approach2
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org