You are viewing a plain text version of this content. The canonical link for it is here.

Posted to reviews@spark.apache.org by GitBox <gi...@apache.org> on 2021/05/04 17:39:04 UTC

[GitHub] [spark] satishgopalani removed a comment on pull request #32434: [SPARK-35312] Introduce new Option in Kafka source to specify minimum number of records to read per trigger

satishgopalani removed a comment on pull request #32434:
URL: https://github.com/apache/spark/pull/32434#issuecomment-832065048


   Here are some more stats from the two run i.e. one normal run and other with minOffsetsperTrigger set:
   
   | Run | Data Size | Number of Batch Runs | Number of Files |
   | ------------- | ------------- |------------- |------------- |
   | Normal Run | 54.2 GB | 320 | 21968 |
   | Run with minOffsetsperTrigger | 54.2 GB | 120 | 12104 |


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org