You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Ted Malaska (JIRA)" <ji...@apache.org> on 2014/12/19 17:06:13 UTC
[jira] [Commented] (SPARK-2447) Add common solution for sending
upsert actions to HBase (put, deletes, and increment)
[ https://issues.apache.org/jira/browse/SPARK-2447?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14253586#comment-14253586 ]
Ted Malaska commented on SPARK-2447:
------------------------------------
Hey guy,
Just wanted to update this jira. In summery the Spark committers is still deciding how this will be or not be include in the external part of Spark.
For now because the demand is there and because the solution works I'm going to host the solution on Cloudera Labs.
Here is the blog post that walks through the solution.
http://blog.cloudera.com/blog/2014/12/new-in-cloudera-labs-sparkonhbase/
> Add common solution for sending upsert actions to HBase (put, deletes, and increment)
> -------------------------------------------------------------------------------------
>
> Key: SPARK-2447
> URL: https://issues.apache.org/jira/browse/SPARK-2447
> Project: Spark
> Issue Type: New Feature
> Components: Spark Core, Streaming
> Reporter: Ted Malaska
> Assignee: Ted Malaska
>
> Going to review the design with Tdas today.
> But first thoughts is to have an extension of VoidFunction that handles the connection to HBase and allows for options such as turning auto flush off for higher through put.
> Need to answer the following questions first.
> - Can it be written in Java or should it be written in Scala?
> - What is the best way to add the HBase dependency? (will review how Flume does this as the first option)
> - What is the best way to do testing? (will review how Flume does this as the first option)
> - How to support python? (python may be a different Jira it is unknown at this time)
> Goals:
> - Simple to use
> - Stable
> - Supports high load
> - Documented (May be in a separate Jira need to ask Tdas)
> - Supports Java, Scala, and hopefully Python
> - Supports Streaming and normal Spark
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org