You are viewing a plain text version of this content. The canonical link for it is here.

Posted to user@spark.apache.org by KhajaAsmath Mohammed <md...@gmail.com> on 2017/11/29 15:54:25 UTC

JDK1.8 for spark workers

Hi,

I am running cloudera version of spark2.1 and our cluster is on JDK1.7. For
some of the libraries, I need JDK1.8, is there a way to set to run Spark
worker in JDK1.8 without upgrading .

I was able run driver in JDK 1.8 by setting the path but not the workers.

17/11/28 20:22:27 WARN scheduler.TaskSetManager: Lost task 0.0 in stage 1.0
(TID 1, brksvl267.brk.navistar.com, executor 1):
java.lang.UnsupportedClassVersionError: org/wololo/geojson/GeoJSON :
Unsupported major.minor version 52.0

Thanks,
Asmath

Re: JDK1.8 for spark workers

Posted by KhajaAsmath Mohammed <md...@gmail.com>.

This didnt work. I tried it but no luck.

On Wed, Nov 29, 2017 at 7:49 PM, Vadim Semenov <va...@datadoghq.com>
wrote:

> You can pass `JAVA_HOME` environment variable
>
> `spark.executorEnv.JAVA_HOME=/usr/lib/jvm/java-1.8.0`
>
> On Wed, Nov 29, 2017 at 10:54 AM, KhajaAsmath Mohammed <
> mdkhajaasmath@gmail.com> wrote:
>
>> Hi,
>>
>> I am running cloudera version of spark2.1 and our cluster is on JDK1.7.
>> For some of the libraries, I need JDK1.8, is there a way to set to run
>> Spark worker in JDK1.8 without upgrading .
>>
>> I was able run driver in JDK 1.8 by setting the path but not the workers.
>>
>> 17/11/28 20:22:27 WARN scheduler.TaskSetManager: Lost task 0.0 in stage
>> 1.0 (TID 1, brksvl267.brk.navistar.com, executor 1):
>> java.lang.UnsupportedClassVersionError: org/wololo/geojson/GeoJSON :
>> Unsupported major.minor version 52.0
>>
>> Thanks,
>> Asmath
>>
>
>

Re: JDK1.8 for spark workers

Posted by Vadim Semenov <va...@datadoghq.com>.

You can pass `JAVA_HOME` environment variable

`spark.executorEnv.JAVA_HOME=/usr/lib/jvm/java-1.8.0`

On Wed, Nov 29, 2017 at 10:54 AM, KhajaAsmath Mohammed <
mdkhajaasmath@gmail.com> wrote:

> Hi,
>
> I am running cloudera version of spark2.1 and our cluster is on JDK1.7.
> For some of the libraries, I need JDK1.8, is there a way to set to run
> Spark worker in JDK1.8 without upgrading .
>
> I was able run driver in JDK 1.8 by setting the path but not the workers.
>
> 17/11/28 20:22:27 WARN scheduler.TaskSetManager: Lost task 0.0 in stage
> 1.0 (TID 1, brksvl267.brk.navistar.com, executor 1): java.lang.UnsupportedClassVersionError:
> org/wololo/geojson/GeoJSON : Unsupported major.minor version 52.0
>
> Thanks,
> Asmath
>