You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hbase.apache.org by Rachana Srivastava <ra...@yahoo.com.INVALID> on 2016/03/10 15:36:18 UTC
How to Intgrate HBase With SparkStreaming
Hello all,
I am trying to integrate HBase with SparkStreaming new APIs mentioned here http://blog.cloudera.com/blog/2014/12/new-in-cloudera-labs-sparkonhbase/
I am using JavaHBaseContext hbaseContext = new JavaHBaseContext(jssc.sparkContext(), conf); Then called bulk Get API
hbaseContext.streamBulkGet(TableName.valueOf(tableName), 2, lines, new GetFunction2(), new ResultFunction());
But this bulk API is invoked during initialization not during each streaming message. Also used
hbaseContext.foreachPartition(jDStream,new VoidFunction<Tuple2<Iterator<String>, Connection>>() {
public void call(Tuple2<Iterator<String>, Connection> t)throws Exception { ...}
The API exists but somehow it is not working for streaming message.
Do we have example of how to get data using the spark streaming api.
Thanks,
Rachana
Re: How to Intgrate HBase With SparkStreaming
Posted by Jean-Marc Spaggiari <je...@spaggiari.org>.
Have you looked and tried this?
https://hbase.apache.org/book.html#_spark_streaming
It doesn't work for you?
JMS
2016-03-10 9:36 GMT-05:00 Rachana Srivastava <
rachanasrivastav@yahoo.com.invalid>:
> Hello all,
> I am trying to integrate HBase with SparkStreaming new APIs mentioned here
> http://blog.cloudera.com/blog/2014/12/new-in-cloudera-labs-sparkonhbase/
> I am using JavaHBaseContext hbaseContext = new
> JavaHBaseContext(jssc.sparkContext(), conf); Then called bulk Get API
> hbaseContext.streamBulkGet(TableName.valueOf(tableName), 2, lines, new
> GetFunction2(), new ResultFunction());
>
> But this bulk API is invoked during initialization not during each
> streaming message. Also used
>
> hbaseContext.foreachPartition(jDStream,new
> VoidFunction<Tuple2<Iterator<String>, Connection>>() {
> public void call(Tuple2<Iterator<String>, Connection> t)throws
> Exception { ...}
> The API exists but somehow it is not working for streaming message.
> Do we have example of how to get data using the spark streaming api.
> Thanks,
> Rachana
>
>
Re: How to Intgrate HBase With SparkStreaming
Posted by Sean Busbey <bu...@cloudera.com>.
Hi Rachana!
For help with vendor provided add-ons, please use the given vendor's
support mechanism.
For things in Cloudera Labs, your best best for a starting place is:
http://community.cloudera.com/t5/Cloudera-Labs/bd-p/ClouderaLabs
On Thu, Mar 10, 2016 at 6:36 AM, Rachana Srivastava <
rachanasrivastav@yahoo.com.invalid> wrote:
> Hello all,
> I am trying to integrate HBase with SparkStreaming new APIs mentioned here
> http://blog.cloudera.com/blog/2014/12/new-in-cloudera-labs-sparkonhbase/
> I am using JavaHBaseContext hbaseContext = new
> JavaHBaseContext(jssc.sparkContext(), conf); Then called bulk Get API
> hbaseContext.streamBulkGet(TableName.valueOf(tableName), 2, lines, new
> GetFunction2(), new ResultFunction());
>
> But this bulk API is invoked during initialization not during each
> streaming message. Also used
>
> hbaseContext.foreachPartition(jDStream,new
> VoidFunction<Tuple2<Iterator<String>, Connection>>() {
> public void call(Tuple2<Iterator<String>, Connection> t)throws
> Exception { ...}
> The API exists but somehow it is not working for streaming message.
> Do we have example of how to get data using the spark streaming api.
> Thanks,
> Rachana
>
>
--
busbey