You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@spark.apache.org by Michael Armbrust <mi...@databricks.com> on 2014/12/31 21:57:03 UTC
Re: Why the major.minor version of the new hive-exec is 51.0?
We actually do publish our own version of this jar, because the version
that the hive team publishes is an uber jar and this breaks all kinds of
things. As a result I'd file the JIRA against Spark.
On Wed, Dec 31, 2014 at 12:55 PM, Ted Yu <yu...@gmail.com> wrote:
> Michael:
> hive-exec-0.12.0-protobuf-2.5.jar is not generated from Spark source code,
> right ?
>
> What would be done after the JIRA is opened ?
>
> Cheers
>
> On Wed, Dec 31, 2014 at 12:16 PM, Michael Armbrust <michael@databricks.com
> > wrote:
>
>> This was not intended, can you open a JIRA?
>>
>> On Tue, Dec 30, 2014 at 8:40 PM, Ted Yu <yu...@gmail.com> wrote:
>>
>>> I extracted org/apache/hadoop/hive/common/CompressionUtils.class from the
>>> jar and used hexdump to view the class file.
>>> Bytes 6 and 7 are 00 and 33, respectively.
>>>
>>> According to http://en.wikipedia.org/wiki/Java_class_file, the jar was
>>> produced using Java 7.
>>>
>>> FYI
>>>
>>> On Tue, Dec 30, 2014 at 8:09 PM, Shixiong Zhu <zs...@gmail.com> wrote:
>>>
>>> > The major.minor version of the new org.spark-project.hive.hive-exec is
>>> > 51.0, so it will require people use JDK7. Is it intentional?
>>> >
>>> > <dependency>
>>> > <groupId>org.spark-project.hive</groupId>
>>> > <artifactId>hive-exec</artifactId>
>>> > <version>0.12.0-protobuf-2.5</version>
>>> > </dependency>
>>> >
>>> > You can use the following steps to reproduce it (Need to use JDK6):
>>> >
>>> > 1. Create a Test.java file with the following content:
>>> >
>>> > public class Test {
>>> >
>>> > public static void main(String[] args) throws Exception{
>>> > Class.forName("org.apache.hadoop.hive.conf.HiveConf");
>>> > }
>>> >
>>> > }
>>> >
>>> > 2. javac Test.java
>>> > 3. java -classpath
>>> >
>>> >
>>> ~/.m2/repository/org/spark-project/hive/hive-exec/0.12.0-protobuf-2.5/hive-exec-0.12.0-protobuf-2.5.jar:.
>>> > Test
>>> >
>>> > Exception in thread "main" java.lang.UnsupportedClassVersionError:
>>> > org/apache/hadoop/hive/conf/HiveConf : Unsupported major.minor version
>>> 51.0
>>> > at java.lang.ClassLoader.defineClass1(Native Method)
>>> > at java.lang.ClassLoader.defineClassCond(ClassLoader.java:631)
>>> > at java.lang.ClassLoader.defineClass(ClassLoader.java:615)
>>> > at
>>> java.security.SecureClassLoader.defineClass(SecureClassLoader.java:141)
>>> > at java.net.URLClassLoader.defineClass(URLClassLoader.java:283)
>>> > at java.net.URLClassLoader.access$000(URLClassLoader.java:58)
>>> > at java.net.URLClassLoader$1.run(URLClassLoader.java:197)
>>> > at java.security.AccessController.doPrivileged(Native Method)
>>> > at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
>>> > at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
>>> > at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
>>> > at java.lang.ClassLoader.loadClass(ClassLoader.java:247)
>>> > at java.lang.Class.forName0(Native Method)
>>> > at java.lang.Class.forName(Class.java:169)
>>> > at Test.main(Test.java:5)
>>> >
>>> >
>>> > Best Regards,
>>> > Shixiong Zhu
>>> >
>>>
>>
>>
>
Re: Why the major.minor version of the new hive-exec is 51.0?
Posted by Ted Yu <yu...@gmail.com>.
I see.
I logged SPARK-5041 which references this thread.
Thanks
On Wed, Dec 31, 2014 at 12:57 PM, Michael Armbrust <mi...@databricks.com>
wrote:
> We actually do publish our own version of this jar, because the version
> that the hive team publishes is an uber jar and this breaks all kinds of
> things. As a result I'd file the JIRA against Spark.
>
> On Wed, Dec 31, 2014 at 12:55 PM, Ted Yu <yu...@gmail.com> wrote:
>
>> Michael:
>> hive-exec-0.12.0-protobuf-2.5.jar is not generated from Spark source
>> code, right ?
>>
>> What would be done after the JIRA is opened ?
>>
>> Cheers
>>
>> On Wed, Dec 31, 2014 at 12:16 PM, Michael Armbrust <
>> michael@databricks.com> wrote:
>>
>>> This was not intended, can you open a JIRA?
>>>
>>> On Tue, Dec 30, 2014 at 8:40 PM, Ted Yu <yu...@gmail.com> wrote:
>>>
>>>> I extracted org/apache/hadoop/hive/common/CompressionUtils.class from
>>>> the
>>>> jar and used hexdump to view the class file.
>>>> Bytes 6 and 7 are 00 and 33, respectively.
>>>>
>>>> According to http://en.wikipedia.org/wiki/Java_class_file, the jar was
>>>> produced using Java 7.
>>>>
>>>> FYI
>>>>
>>>> On Tue, Dec 30, 2014 at 8:09 PM, Shixiong Zhu <zs...@gmail.com>
>>>> wrote:
>>>>
>>>> > The major.minor version of the new org.spark-project.hive.hive-exec is
>>>> > 51.0, so it will require people use JDK7. Is it intentional?
>>>> >
>>>> > <dependency>
>>>> > <groupId>org.spark-project.hive</groupId>
>>>> > <artifactId>hive-exec</artifactId>
>>>> > <version>0.12.0-protobuf-2.5</version>
>>>> > </dependency>
>>>> >
>>>> > You can use the following steps to reproduce it (Need to use JDK6):
>>>> >
>>>> > 1. Create a Test.java file with the following content:
>>>> >
>>>> > public class Test {
>>>> >
>>>> > public static void main(String[] args) throws Exception{
>>>> > Class.forName("org.apache.hadoop.hive.conf.HiveConf");
>>>> > }
>>>> >
>>>> > }
>>>> >
>>>> > 2. javac Test.java
>>>> > 3. java -classpath
>>>> >
>>>> >
>>>> ~/.m2/repository/org/spark-project/hive/hive-exec/0.12.0-protobuf-2.5/hive-exec-0.12.0-protobuf-2.5.jar:.
>>>> > Test
>>>> >
>>>> > Exception in thread "main" java.lang.UnsupportedClassVersionError:
>>>> > org/apache/hadoop/hive/conf/HiveConf : Unsupported major.minor
>>>> version 51.0
>>>> > at java.lang.ClassLoader.defineClass1(Native Method)
>>>> > at java.lang.ClassLoader.defineClassCond(ClassLoader.java:631)
>>>> > at java.lang.ClassLoader.defineClass(ClassLoader.java:615)
>>>> > at
>>>> java.security.SecureClassLoader.defineClass(SecureClassLoader.java:141)
>>>> > at java.net.URLClassLoader.defineClass(URLClassLoader.java:283)
>>>> > at java.net.URLClassLoader.access$000(URLClassLoader.java:58)
>>>> > at java.net.URLClassLoader$1.run(URLClassLoader.java:197)
>>>> > at java.security.AccessController.doPrivileged(Native Method)
>>>> > at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
>>>> > at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
>>>> > at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
>>>> > at java.lang.ClassLoader.loadClass(ClassLoader.java:247)
>>>> > at java.lang.Class.forName0(Native Method)
>>>> > at java.lang.Class.forName(Class.java:169)
>>>> > at Test.main(Test.java:5)
>>>> >
>>>> >
>>>> > Best Regards,
>>>> > Shixiong Zhu
>>>> >
>>>>
>>>
>>>
>>
>