You are viewing a plain text version of this content. The canonical link for it is here.

Posted to devnull@infra.apache.org by "kyle-winkelman (GitHub)" <gi...@apache.org> on 2018/08/27 15:17:29 UTC

[GitHub] [beam] kyle-winkelman commented on pull request #6181: [BEAM-4783] Add bundleSize for splitting BoundedSources.

This will cause the [Partitioner.defaultPartitioner](https://github.com/apache/spark/blob/master/core/src/main/scala/org/apache/spark/Partitioner.scala#L62) to be used. When called on a SourceRDD this should be a HashPartitioner with the number of partitions equal to the number of splits created by the bundleSize. When called on a SourceDStream this should be a HashPartitioner with the number of partitions equal to the defaultParallelism.

[ Full content available at: https://github.com/apache/beam/pull/6181 ]
This message was relayed via gitbox.apache.org for devnull@infra.apache.org