You are viewing a plain text version of this content. The canonical link for it is here.

Posted to commits@beam.apache.org by "Reuven Lax (JIRA)" <ji...@apache.org> on 2017/02/08 21:00:44 UTC

[jira] [Created] (BEAM-1438) The default behavior for the Write transform doesn't work well with the Dataflow streaming runner

Reuven Lax created BEAM-1438:
--------------------------------

             Summary: The default behavior for the Write transform doesn't work well with the Dataflow streaming runner
                 Key: BEAM-1438
                 URL: https://issues.apache.org/jira/browse/BEAM-1438
             Project: Beam
          Issue Type: Bug
          Components: runner-dataflow
            Reporter: Reuven Lax
            Assignee: Davor Bonaci


If a Write specifies 0 output shards, that implies the runner should pick an appropriate sharding. The default behavior is to write one shard per input bundle. This works well with the Dataflow batch runner, but not with the streaming runner which produces large numbers of small bundles.




--
This message was sent by Atlassian JIRA
(v6.3.15#6346)