You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@spark.apache.org by "Reza Safi (JIRA)" <ji...@apache.org> on 2018/08/24 16:44:00 UTC

[jira] [Created] (SPARK-25233) Give the user the option of specifying a fixed minimum message per partition per batch when using kafka direct API with backpressure

Reza Safi created SPARK-25233:
---------------------------------

             Summary: Give the user the option of specifying a fixed minimum message per partition per batch when using kafka direct API with backpressure
                 Key: SPARK-25233
                 URL: https://issues.apache.org/jira/browse/SPARK-25233
             Project: Spark
          Issue Type: Improvement
          Components: DStreams
    Affects Versions: 2.4.0
            Reporter: Reza Safi


After SPARK-18371, it is guaranteed that there would be at least *one* message per partition per batch using direct kafka API with backpressure when new messages exist in the topics. It would be better if the user has the option of setting the minimum instead of just a hard coded 1 limit.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org