You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@zeppelin.apache.org by Joaquín Silva <jo...@gmail.com> on 2016/09/27 12:34:23 UTC
Spark with Kudu
Hello,
I'm running Spark with Kudu like this:
https://kudu.apache.org/docs/developing.html#_kudu_integration_with_spark
The spark-shell work fine by just adding the flag --jars. But in Zeppelin
I'm getting this error:
java.lang.NoSuchMethodError:
org.kududb.spark.kudu.package$KuduDataFrameReader.kudu()Lorg/apache/spark/sql/Dataset;
I added the jar file to the zeppelin.interpreter.localRepo
/root/zeppelin-0.6.1-bin-netinst/local-repo/2BZ9T2DCB and I can import the
libraries just fine. But I'm getting this error when I try to use Kudu's
classes.
What could it be?.
Regards,
Joaquín Silva
Re: Spark with Kudu
Posted by Joaquín Silva <jo...@gmail.com>.
Installed 0.6.0 and worked fine.
Thanks moon soo Lee.
2016-09-27 11:48 GMT-03:00 moon soo Lee <mo...@apache.org>:
> Yes, I think it's because of ZEPPELIN-1384.
> If you want to use zeppelin1.6.1 release, you will need to build from
> source with scala 2.10 profile.
>
> Spark interpreter in Current master branch and branch-0.6(which will be
> released as Zeppelin 0.6.2) has binary compatibility to scala 2.10 and 2.11.
>
> Thanks,
> moon
>
> On Tue, 27 Sep 2016 at 7:28 AM Joaquín Silva <
> joaquin.silva.vigenaux@gmail.com> wrote:
>
>> I just added SPARK_HOME to the zeppelin-env.sh and this error pops up:
>>
>> Exception in thread "pool-1-thread-6" java.lang.NoSuchMethodError:
>> scala.runtime.VolatileByteRef.create(B)Lscala/runtime/VolatileByteRef;
>>
>> could be this issue? https://issues.apache.org/jira/browse/ZEPPELIN-1384
>>
>> I have spark 1.6.0 with scala 2.10.5
>>
>> 2016-09-27 9:43 GMT-03:00 DuyHai Doan <do...@gmail.com>:
>>
>>> Did you add the jar dependency to the Spark interpreter too ?
>>>
>>> On Tue, Sep 27, 2016 at 2:34 PM, Joaquín Silva <
>>> joaquin.silva.vigenaux@gmail.com> wrote:
>>>
>>>> Hello,
>>>>
>>>> I'm running Spark with Kudu like this:
>>>> https://kudu.apache.org/docs/developing.html#_kudu_
>>>> integration_with_spark
>>>>
>>>> The spark-shell work fine by just adding the flag --jars. But in
>>>> Zeppelin I'm getting this error:
>>>>
>>>> java.lang.NoSuchMethodError: org.kududb.spark.kudu.package$
>>>> KuduDataFrameReader.kudu()Lorg/apache/spark/sql/Dataset;
>>>>
>>>> I added the jar file to the zeppelin.interpreter.localRepo
>>>> /root/zeppelin-0.6.1-bin-netinst/local-repo/2BZ9T2DCB and I can import
>>>> the libraries just fine. But I'm getting this error when I try to use
>>>> Kudu's classes.
>>>>
>>>> What could it be?.
>>>>
>>>> Regards,
>>>> Joaquín Silva
>>>>
>>>
>>>
>>
Re: Spark with Kudu
Posted by moon soo Lee <mo...@apache.org>.
Yes, I think it's because of ZEPPELIN-1384.
If you want to use zeppelin1.6.1 release, you will need to build from
source with scala 2.10 profile.
Spark interpreter in Current master branch and branch-0.6(which will be
released as Zeppelin 0.6.2) has binary compatibility to scala 2.10 and 2.11.
Thanks,
moon
On Tue, 27 Sep 2016 at 7:28 AM Joaquín Silva <
joaquin.silva.vigenaux@gmail.com> wrote:
> I just added SPARK_HOME to the zeppelin-env.sh and this error pops up:
>
> Exception in thread "pool-1-thread-6" java.lang.NoSuchMethodError:
> scala.runtime.VolatileByteRef.create(B)Lscala/runtime/VolatileByteRef;
>
> could be this issue? https://issues.apache.org/jira/browse/ZEPPELIN-1384
>
> I have spark 1.6.0 with scala 2.10.5
>
> 2016-09-27 9:43 GMT-03:00 DuyHai Doan <do...@gmail.com>:
>
>> Did you add the jar dependency to the Spark interpreter too ?
>>
>> On Tue, Sep 27, 2016 at 2:34 PM, Joaquín Silva <
>> joaquin.silva.vigenaux@gmail.com> wrote:
>>
>>> Hello,
>>>
>>> I'm running Spark with Kudu like this:
>>> https://kudu.apache.org/docs/developing.html#_kudu_integration_with_spark
>>>
>>> The spark-shell work fine by just adding the flag --jars. But in
>>> Zeppelin I'm getting this error:
>>>
>>> java.lang.NoSuchMethodError:
>>> org.kududb.spark.kudu.package$KuduDataFrameReader.kudu()Lorg/apache/spark/sql/Dataset;
>>>
>>> I added the jar file to the zeppelin.interpreter.localRepo
>>> /root/zeppelin-0.6.1-bin-netinst/local-repo/2BZ9T2DCB and I can import
>>> the libraries just fine. But I'm getting this error when I try to use
>>> Kudu's classes.
>>>
>>> What could it be?.
>>>
>>> Regards,
>>> Joaquín Silva
>>>
>>
>>
>
Re: Spark with Kudu
Posted by Joaquín Silva <jo...@gmail.com>.
I just added SPARK_HOME to the zeppelin-env.sh and this error pops up:
Exception in thread "pool-1-thread-6" java.lang.NoSuchMethodError:
scala.runtime.VolatileByteRef.create(B)Lscala/runtime/VolatileByteRef;
could be this issue? https://issues.apache.org/jira/browse/ZEPPELIN-1384
I have spark 1.6.0 with scala 2.10.5
2016-09-27 9:43 GMT-03:00 DuyHai Doan <do...@gmail.com>:
> Did you add the jar dependency to the Spark interpreter too ?
>
> On Tue, Sep 27, 2016 at 2:34 PM, Joaquín Silva <
> joaquin.silva.vigenaux@gmail.com> wrote:
>
>> Hello,
>>
>> I'm running Spark with Kudu like this:
>> https://kudu.apache.org/docs/developing.html#_kudu_integration_with_spark
>>
>> The spark-shell work fine by just adding the flag --jars. But in Zeppelin
>> I'm getting this error:
>>
>> java.lang.NoSuchMethodError: org.kududb.spark.kudu.package$
>> KuduDataFrameReader.kudu()Lorg/apache/spark/sql/Dataset;
>>
>> I added the jar file to the zeppelin.interpreter.localRepo
>> /root/zeppelin-0.6.1-bin-netinst/local-repo/2BZ9T2DCB and I can import
>> the libraries just fine. But I'm getting this error when I try to use
>> Kudu's classes.
>>
>> What could it be?.
>>
>> Regards,
>> Joaquín Silva
>>
>
>
Re: Spark with Kudu
Posted by DuyHai Doan <do...@gmail.com>.
Did you add the jar dependency to the Spark interpreter too ?
On Tue, Sep 27, 2016 at 2:34 PM, Joaquín Silva <
joaquin.silva.vigenaux@gmail.com> wrote:
> Hello,
>
> I'm running Spark with Kudu like this:
> https://kudu.apache.org/docs/developing.html#_kudu_integration_with_spark
>
> The spark-shell work fine by just adding the flag --jars. But in Zeppelin
> I'm getting this error:
>
> java.lang.NoSuchMethodError: org.kududb.spark.kudu.package$
> KuduDataFrameReader.kudu()Lorg/apache/spark/sql/Dataset;
>
> I added the jar file to the zeppelin.interpreter.localRepo
> /root/zeppelin-0.6.1-bin-netinst/local-repo/2BZ9T2DCB and I can import
> the libraries just fine. But I'm getting this error when I try to use
> Kudu's classes.
>
> What could it be?.
>
> Regards,
> Joaquín Silva
>