You are viewing a plain text version of this content. The canonical link for it is here.
Posted to gitbox@hive.apache.org by GitBox <gi...@apache.org> on 2020/08/04 16:09:00 UTC
[GitHub] [hive] belugabehr opened a new pull request #1241: HIVE-22415: Upgrade to Java 11
belugabehr opened a new pull request #1241:
URL: https://github.com/apache/hive/pull/1241
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] belugabehr commented on pull request #1241: HIVE-22415: Upgrade to Java 11
Posted by GitBox <gi...@apache.org>.
belugabehr commented on pull request #1241:
URL: https://github.com/apache/hive/pull/1241#issuecomment-665035885
@kgyrtkirk Let the build run and see what we get. Again, it's different to say that something compiles under JDK11 and something runs. Also, since Hive is a client application, the dependencies may not have experienced any changes.
I am not against an upgrade of Hadoop library. We just don't need to link these two efforts unless we absolutely have to.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] belugabehr commented on a change in pull request #1241: HIVE-22415: Upgrade to Java 11
Posted by GitBox <gi...@apache.org>.
belugabehr commented on a change in pull request #1241:
URL: https://github.com/apache/hive/pull/1241#discussion_r461565573
##########
File path: .travis.yml
##########
@@ -24,17 +23,14 @@ dist: bionic
language: java
jdk:
- - openjdk8
-
-# disabling cache for /home/travis/.m2/repository/org/apache/hive/hive-jdbc/3.0.0-SNAPSHOT/hive-jdbc-3.0.0-SNAPSHOT-standalone.jar (Permission denied)
-#cache:
-# directories:
-# - $HOME/.m2
+ - openjdk11
env:
MAVEN_SKIP_RC=true
MAVEN_OPTS="-Xmx2g"
install: skip
-script: mvn clean install -DskipTests -q -Pitests
+before_script: mvn --version
Review comment:
Ya, sorry, I mean from a feature standpoint : Travis v.s. Jenkins
##########
File path: .travis.yml
##########
@@ -24,17 +23,14 @@ dist: bionic
language: java
jdk:
- - openjdk8
-
-# disabling cache for /home/travis/.m2/repository/org/apache/hive/hive-jdbc/3.0.0-SNAPSHOT/hive-jdbc-3.0.0-SNAPSHOT-standalone.jar (Permission denied)
-#cache:
-# directories:
-# - $HOME/.m2
+ - openjdk11
env:
MAVEN_SKIP_RC=true
MAVEN_OPTS="-Xmx2g"
install: skip
-script: mvn clean install -DskipTests -q -Pitests
+before_script: mvn --version
Review comment:
Ya, sorry, I mean from a feature standpoint : Travis v.s. Jenkins
Can Travis run tests? If so, why setup this special purpose docker container when the same can be done more simply with Travis?
##########
File path: Jenkinsfile
##########
@@ -81,6 +81,7 @@ def buildHive(args) {
configFileProvider([configFile(fileId: 'artifactory', variable: 'SETTINGS')]) {
withEnv(["MULTIPLIER=$params.MULTIPLIER","M_OPTS=$params.OPTS"]) {
sh '''#!/bin/bash -e
+sw java 11
Review comment:
@kgyrtkirk
```
Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:3.8.1:compile (default-compile) on project hive-storage-api: Fatal error compiling: invalid target release: 11 -> [Help 1]
```
This didn't seem to work.
http://ci.hive.apache.org/blue/organizations/jenkins/hive-precommit/detail/PR-1241/5/pipeline
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] belugabehr commented on pull request #1241: HIVE-22415: Upgrade to Java 11
Posted by GitBox <gi...@apache.org>.
belugabehr commented on pull request #1241:
URL: https://github.com/apache/hive/pull/1241#issuecomment-669197244
@kgyrtkirk Cannot build `init-metastore` part of the pipeline. Does this somehow use a different docker image? How do I get it's `JAVA_HOME` to JDK 11?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] kgyrtkirk commented on a change in pull request #1241: HIVE-22415: Upgrade to Java 11
Posted by GitBox <gi...@apache.org>.
kgyrtkirk commented on a change in pull request #1241:
URL: https://github.com/apache/hive/pull/1241#discussion_r461355075
##########
File path: llap-server/src/java/org/apache/hadoop/hive/llap/cache/SimpleAllocator.java
##########
@@ -17,31 +17,17 @@
*/
package org.apache.hadoop.hive.llap.cache;
-import java.lang.reflect.Field;
import java.nio.ByteBuffer;
import org.apache.hadoop.conf.Configuration;
import org.apache.hadoop.hive.common.io.Allocator;
import org.apache.hadoop.hive.common.io.encoded.MemoryBuffer;
import org.apache.hadoop.hive.conf.HiveConf;
import org.apache.hadoop.hive.llap.io.api.impl.LlapIoImpl;
-
-import sun.misc.Cleaner;
+import org.apache.hive.common.util.CleanerUtil;
Review comment:
I think it's mandatory to upgrade to at least
https://issues.apache.org/jira/browse/HADOOP-15338 marks 3.3 for jdk11 support - I don't think we should try to run hive with hadoop 3.1 on jdk11...
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] belugabehr edited a comment on pull request #1241: HIVE-22415: Upgrade to Java 11
Posted by GitBox <gi...@apache.org>.
belugabehr edited a comment on pull request #1241:
URL: https://github.com/apache/hive/pull/1241#issuecomment-669197244
@kgyrtkirk Cannot build `init-metastore` part of the pipeline. Does this somehow use a different docker image? How do I get its `JAVA_HOME` to JDK 11?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] abstractdog commented on pull request #1241: HIVE-22415: Upgrade to Java 11
Posted by GitBox <gi...@apache.org>.
abstractdog commented on pull request #1241:
URL: https://github.com/apache/hive/pull/1241#issuecomment-718214905
FYI, HIVE-23930 is resolved, hive now depends on tez 0.10.0 which is JDK11 compliant
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] belugabehr commented on a change in pull request #1241: HIVE-22415: Upgrade to Java 11
Posted by GitBox <gi...@apache.org>.
belugabehr commented on a change in pull request #1241:
URL: https://github.com/apache/hive/pull/1241#discussion_r453030985
##########
File path: standalone-metastore/metastore-server/src/main/resources/datanucleus-log4j.properties
##########
@@ -15,3 +15,5 @@ log4j.category.DataNucleus.ValueGeneration=DEBUG, A1
log4j.category.DataNucleus.Enhancer=INFO, A1
log4j.category.DataNucleus.SchemaTool=INFO, A1
+
+log4j.category.DataNucleus.Persistence=INFO, A1
Review comment:
Remove this change.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] belugabehr commented on pull request #1241: HIVE-22415: Upgrade to Java 11
Posted by GitBox <gi...@apache.org>.
belugabehr commented on pull request #1241:
URL: https://github.com/apache/hive/pull/1241#issuecomment-665664081
@ashutoshc Happy to do that. Let me get it all working in this PR then I can start cutting into phases.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] github-actions[bot] commented on pull request #1241: HIVE-22415: Upgrade to Java 11
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on pull request #1241:
URL: https://github.com/apache/hive/pull/1241#issuecomment-703342991
This pull request has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs.
Feel free to reach out on the dev@hive.apache.org list if the patch is in need of reviews.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] github-actions[bot] closed pull request #1241: HIVE-22415: Upgrade to Java 11
Posted by GitBox <gi...@apache.org>.
github-actions[bot] closed pull request #1241:
URL: https://github.com/apache/hive/pull/1241
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] belugabehr commented on pull request #1241: HIVE-22415: Upgrade to Java 11
Posted by GitBox <gi...@apache.org>.
belugabehr commented on pull request #1241:
URL: https://github.com/apache/hive/pull/1241#issuecomment-657614154
@kgyrtkirk
Yup! OpenJDK and Oracle are the only ones Cloudera supports. For Java 11, it's OpenJDK 11 only.
https://docs.cloudera.com/cdp/latest/release-guide/topics/cdpdc-java-requirements.html
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] belugabehr commented on pull request #1241: HIVE-22415: Upgrade to Java 11
Posted by GitBox <gi...@apache.org>.
belugabehr commented on pull request #1241:
URL: https://github.com/apache/hive/pull/1241#issuecomment-657615065
@kgyrtkirk I'm not sure what you mean by `java 8 + java 11 testing`. If we go with JDK 11 for the build, it should be backwards compatible with Java 8 (which is defined in the Maven POM file for Hive 3.x and master currently).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] belugabehr commented on a change in pull request #1241: HIVE-22415: Upgrade to Java 11
Posted by GitBox <gi...@apache.org>.
belugabehr commented on a change in pull request #1241:
URL: https://github.com/apache/hive/pull/1241#discussion_r453722539
##########
File path: standalone-metastore/metastore-server/pom.xml
##########
@@ -726,8 +726,7 @@
<configuration>
<api>JDO</api>
<verbose>false</verbose>
- <log4jConfiguration>${basedir}/src/main/resources/datanucleus-log4j.properties
- </log4jConfiguration>
+ <log4jConfiguration>${basedir}/src/main/resources/datanucleus-log4j.properties</log4jConfiguration>
Review comment:
Remove this change.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] kgyrtkirk commented on pull request #1241: HIVE-22415: Upgrade to Java 11
Posted by GitBox <gi...@apache.org>.
kgyrtkirk commented on pull request #1241:
URL: https://github.com/apache/hive/pull/1241#issuecomment-657612371
how about we phasing in java 8 + java 11 testing at the same time?
@belugabehr would that make sense?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] belugabehr commented on pull request #1241: HIVE-22415: Upgrade to Java 11
Posted by GitBox <gi...@apache.org>.
belugabehr commented on pull request #1241:
URL: https://github.com/apache/hive/pull/1241#issuecomment-718748712
@abstractdog Re-opened PR at #1624
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] ashutoshc commented on pull request #1241: HIVE-22415: Upgrade to Java 11
Posted by GitBox <gi...@apache.org>.
ashutoshc commented on pull request #1241:
URL: https://github.com/apache/hive/pull/1241#issuecomment-665358916
My suggestion would be to split the changes in separate tickets. Ticket for code changes which are necessary for jdk11 but can be committed before the actual move to jdk11 like hadoop class/version, dependency exclusion, dependency upgrades etc. Commit those first and then have minimal changes after that just changes the compile and runtime version to 11. Advantage of doing it that way is it will make it easier downstream to cherry-pick up these changes if they are separate commits.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] belugabehr closed pull request #1241: HIVE-22415: Upgrade to Java 11
Posted by GitBox <gi...@apache.org>.
belugabehr closed pull request #1241:
URL: https://github.com/apache/hive/pull/1241
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] kgyrtkirk commented on pull request #1241: HIVE-22415: Upgrade to Java 11
Posted by GitBox <gi...@apache.org>.
kgyrtkirk commented on pull request #1241:
URL: https://github.com/apache/hive/pull/1241#issuecomment-657611666
oh...I see kgyrtkirk/hive-dev-box#5 is needed to test this
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org