You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@spark.apache.org by "darion yaphet (JIRA)" <ji...@apache.org> on 2017/06/28 05:47:00 UTC

[jira] [Created] (SPARK-21233) Support pluggable offset storage

darion yaphet created SPARK-21233:
-------------------------------------

             Summary: Support pluggable offset storage
                 Key: SPARK-21233
                 URL: https://issues.apache.org/jira/browse/SPARK-21233
             Project: Spark
          Issue Type: New Feature
          Components: DStreams
    Affects Versions: 2.1.1, 2.0.2
            Reporter: darion yaphet


Currently we using *ZooKeeper* to save the *Kafka Commit Offset* , when there are a lot of streaming program running in the cluster the ZooKeeper Cluster's loading is very high . Maybe Zookeeper is not very suitable to save offset periodicity.

This issue is wish to support a pluggable offset storage to avoid save it in the zookeeper 



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org