You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@beam.apache.org by "Ahmet Altay (Jira)" <ji...@apache.org> on 2019/10/26 05:07:00 UTC

[jira] [Created] (BEAM-8495) Change Dataflow byte size estimate for compressed files for FileBasedSources

Ahmet Altay created BEAM-8495:
---------------------------------

             Summary: Change Dataflow byte size estimate for compressed files for FileBasedSources
                 Key: BEAM-8495
                 URL: https://issues.apache.org/jira/browse/BEAM-8495
             Project: Beam
          Issue Type: Bug
          Components: io-java-files, io-py-files
            Reporter: Ahmet Altay
            Assignee: Chamikara Madhusanka Jayalath


Consider change the estimated byte size for file based sources for better initial autoscaling. Currently the estimate is based on total size, for compressed file we could add a factor multiplier (~5) for compressed files in a blob go produce a better estimate.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)