You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@beam.apache.org by "Radosław Stankiewicz (Jira)" <ji...@apache.org> on 2020/01/24 14:14:00 UTC
[jira] [Created] (BEAM-9188) Improving speed of splitting for
Custom Sources
Radosław Stankiewicz created BEAM-9188:
------------------------------------------
Summary: Improving speed of splitting for Custom Sources
Key: BEAM-9188
URL: https://issues.apache.org/jira/browse/BEAM-9188
Project: Beam
Issue Type: Improvement
Components: runner-dataflow
Reporter: Radosław Stankiewicz
Assignee: Radosław Stankiewicz
At this moment Custom Source in being split and serialized in sequence. If there are many splits, it takes time to process all splits.
Example: it takes 2s to calculate size and serialize CassandraSource due to connection setup and teardown. With 100+ splits, it's a lot of time spent in 1 worker.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)