You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hbase.apache.org by Nkechi Achara <nk...@googlemail.com> on 2016/04/22 21:24:11 UTC
Good connector libraries for Spark connecting to Hbase for Spark 1.3
Hbase 1.0
Hi,
I am attempting to use both SpakOnHbase and Hbase-Spark, but I keep
receiving dependency isues, and I am not sure if any of these connectors
are available for Spark 1.3 / Hadoop 2.6 / Hbase 1.0.
Has anyone got a new library I can use, or a concrete example to use?
Thanks,
Keech
Re: Good connector libraries for Spark connecting to Hbase for Spark
1.3 Hbase 1.0
Posted by Sean Busbey <bu...@cloudera.com>.
The example in the HBase-Downstreamer project handles its own Kerberos
authentication for talking to HBase; it should not be impacted by
implementation issues in Spark (unless they are severely messing with
the Classloader, which I don't think they are).
That said, I have not tested it with Spark 1.3, so I'd be interested
in getting feedback.
On Fri, Apr 22, 2016 at 7:01 PM, sudhir patil <sp...@gmail.com> wrote:
> Connecting to kerborised hbase from spark is fixed in spark 1.4 , don't
> think 1.3 works because kerberos issues.
> https://issues.apache.org/jira/plugins/servlet/mobile#issue/SPARK-6918
> On Apr 23, 2016 5:35 AM, "Sean Busbey" <bu...@cloudera.com> wrote:
>
>> The HBase-Downstreamer project has an example that uses the Java API
>> for Spark Streaming on a secure cluster:
>>
>>
>> https://github.com/saintstack/hbase-downstreamer#spark-streaming-test-application
>>
>> https://s.apache.org/apvQ
>>
>> We'd greatly like a Scala version.
>>
>> On Fri, Apr 22, 2016 at 4:16 PM, Nkechi Achara <nk...@googlemail.com>
>> wrote:
>> > scala API, although Java should be easy to alter.... Also, the cluster is
>> > kerberized 😣
>> >
>> > On 22 April 2016 at 22:34, Sean Busbey <bu...@cloudera.com> wrote:
>> >
>> >> Are you attempting to use Spark's Java API or its Scala API? (or python,
>> >> etc?)
>> >>
>> >> On Fri, Apr 22, 2016 at 2:24 PM, Nkechi Achara <nkachara@googlemail.com
>> >
>> >> wrote:
>> >> > Hi,
>> >> >
>> >> > I am attempting to use both SpakOnHbase and Hbase-Spark, but I keep
>> >> > receiving dependency isues, and I am not sure if any of these
>> connectors
>> >> > are available for Spark 1.3 / Hadoop 2.6 / Hbase 1.0.
>> >> >
>> >> > Has anyone got a new library I can use, or a concrete example to use?
>> >> >
>> >> > Thanks,
>> >> >
>> >> > Keech
>> >>
>> >>
>> >>
>> >> --
>> >> busbey
>> >>
>>
>>
>>
>> --
>> busbey
>>
--
busbey
Re: Good connector libraries for Spark connecting to Hbase for Spark
1.3 Hbase 1.0
Posted by sudhir patil <sp...@gmail.com>.
Connecting to kerborised hbase from spark is fixed in spark 1.4 , don't
think 1.3 works because kerberos issues.
https://issues.apache.org/jira/plugins/servlet/mobile#issue/SPARK-6918
On Apr 23, 2016 5:35 AM, "Sean Busbey" <bu...@cloudera.com> wrote:
> The HBase-Downstreamer project has an example that uses the Java API
> for Spark Streaming on a secure cluster:
>
>
> https://github.com/saintstack/hbase-downstreamer#spark-streaming-test-application
>
> https://s.apache.org/apvQ
>
> We'd greatly like a Scala version.
>
> On Fri, Apr 22, 2016 at 4:16 PM, Nkechi Achara <nk...@googlemail.com>
> wrote:
> > scala API, although Java should be easy to alter.... Also, the cluster is
> > kerberized 😣
> >
> > On 22 April 2016 at 22:34, Sean Busbey <bu...@cloudera.com> wrote:
> >
> >> Are you attempting to use Spark's Java API or its Scala API? (or python,
> >> etc?)
> >>
> >> On Fri, Apr 22, 2016 at 2:24 PM, Nkechi Achara <nkachara@googlemail.com
> >
> >> wrote:
> >> > Hi,
> >> >
> >> > I am attempting to use both SpakOnHbase and Hbase-Spark, but I keep
> >> > receiving dependency isues, and I am not sure if any of these
> connectors
> >> > are available for Spark 1.3 / Hadoop 2.6 / Hbase 1.0.
> >> >
> >> > Has anyone got a new library I can use, or a concrete example to use?
> >> >
> >> > Thanks,
> >> >
> >> > Keech
> >>
> >>
> >>
> >> --
> >> busbey
> >>
>
>
>
> --
> busbey
>
Re: Good connector libraries for Spark connecting to Hbase for Spark
1.3 Hbase 1.0
Posted by Sean Busbey <bu...@cloudera.com>.
The HBase-Downstreamer project has an example that uses the Java API
for Spark Streaming on a secure cluster:
https://github.com/saintstack/hbase-downstreamer#spark-streaming-test-application
https://s.apache.org/apvQ
We'd greatly like a Scala version.
On Fri, Apr 22, 2016 at 4:16 PM, Nkechi Achara <nk...@googlemail.com> wrote:
> scala API, although Java should be easy to alter.... Also, the cluster is
> kerberized 😣
>
> On 22 April 2016 at 22:34, Sean Busbey <bu...@cloudera.com> wrote:
>
>> Are you attempting to use Spark's Java API or its Scala API? (or python,
>> etc?)
>>
>> On Fri, Apr 22, 2016 at 2:24 PM, Nkechi Achara <nk...@googlemail.com>
>> wrote:
>> > Hi,
>> >
>> > I am attempting to use both SpakOnHbase and Hbase-Spark, but I keep
>> > receiving dependency isues, and I am not sure if any of these connectors
>> > are available for Spark 1.3 / Hadoop 2.6 / Hbase 1.0.
>> >
>> > Has anyone got a new library I can use, or a concrete example to use?
>> >
>> > Thanks,
>> >
>> > Keech
>>
>>
>>
>> --
>> busbey
>>
--
busbey
Re: Good connector libraries for Spark connecting to Hbase for Spark
1.3 Hbase 1.0
Posted by Nkechi Achara <nk...@googlemail.com>.
scala API, although Java should be easy to alter.... Also, the cluster is
kerberized 😣
On 22 April 2016 at 22:34, Sean Busbey <bu...@cloudera.com> wrote:
> Are you attempting to use Spark's Java API or its Scala API? (or python,
> etc?)
>
> On Fri, Apr 22, 2016 at 2:24 PM, Nkechi Achara <nk...@googlemail.com>
> wrote:
> > Hi,
> >
> > I am attempting to use both SpakOnHbase and Hbase-Spark, but I keep
> > receiving dependency isues, and I am not sure if any of these connectors
> > are available for Spark 1.3 / Hadoop 2.6 / Hbase 1.0.
> >
> > Has anyone got a new library I can use, or a concrete example to use?
> >
> > Thanks,
> >
> > Keech
>
>
>
> --
> busbey
>
Re: Good connector libraries for Spark connecting to Hbase for Spark
1.3 Hbase 1.0
Posted by Sean Busbey <bu...@cloudera.com>.
Are you attempting to use Spark's Java API or its Scala API? (or python, etc?)
On Fri, Apr 22, 2016 at 2:24 PM, Nkechi Achara <nk...@googlemail.com> wrote:
> Hi,
>
> I am attempting to use both SpakOnHbase and Hbase-Spark, but I keep
> receiving dependency isues, and I am not sure if any of these connectors
> are available for Spark 1.3 / Hadoop 2.6 / Hbase 1.0.
>
> Has anyone got a new library I can use, or a concrete example to use?
>
> Thanks,
>
> Keech
--
busbey