You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@beam.apache.org by "Kyle Winkelman (JIRA)" <ji...@apache.org> on 2019/04/04 15:11:00 UTC
[jira] [Closed] (BEAM-6713) FileIO and TextIO unable to alter
WriteFiles maxNumWritersPerBundle
[ https://issues.apache.org/jira/browse/BEAM-6713?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Kyle Winkelman closed BEAM-6713.
--------------------------------
Resolution: Won't Fix
Fix Version/s: Not applicable
> FileIO and TextIO unable to alter WriteFiles maxNumWritersPerBundle
> -------------------------------------------------------------------
>
> Key: BEAM-6713
> URL: https://issues.apache.org/jira/browse/BEAM-6713
> Project: Beam
> Issue Type: Improvement
> Components: sdk-java-core
> Reporter: Kyle Winkelman
> Priority: Major
> Fix For: Not applicable
>
> Time Spent: 1h 40m
> Remaining Estimate: 0h
>
> When attempting to run a batch workflow with a FileIO.write() I was getting job failures due to WriteFiles.DEFAULT_MAX_NUM_WRITERS_PER_BUNDLE causing a significant amount of data to be shuffled. My issues would be solved by increasing this and luckily WriteFiles already has withMaxNumWritersPerBundle but unfortunately FileIO and TextIO do not.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)