You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@spark.apache.org by "Apache Spark (JIRA)" <ji...@apache.org> on 2018/08/24 16:52:00 UTC

[jira] [Assigned] (SPARK-25233) Give the user the option of specifying a fixed minimum message per partition per batch when using kafka direct API with backpressure

     [ https://issues.apache.org/jira/browse/SPARK-25233?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Apache Spark reassigned SPARK-25233:
------------------------------------

    Assignee: Apache Spark

> Give the user the option of specifying a fixed minimum message per partition per batch when using kafka direct API with backpressure
> ------------------------------------------------------------------------------------------------------------------------------------
>
>                 Key: SPARK-25233
>                 URL: https://issues.apache.org/jira/browse/SPARK-25233
>             Project: Spark
>          Issue Type: Improvement
>          Components: DStreams
>    Affects Versions: 2.4.0
>            Reporter: Reza Safi
>            Assignee: Apache Spark
>            Priority: Major
>
> After SPARK-18371, it is guaranteed that there would be at least *one* message per partition per batch using direct kafka API with backpressure when new messages exist in the topics. It would be better if the user has the option of setting the minimum instead of just a hard coded 1 limit.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org