You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@beam.apache.org by "Kenneth Knowles (Jira)" <ji...@apache.org> on 2022/01/12 03:50:06 UTC
[jira] [Updated] (BEAM-6721) setting numshards dynamically for TextIO.write()
[ https://issues.apache.org/jira/browse/BEAM-6721?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Kenneth Knowles updated BEAM-6721:
----------------------------------
This Jira ticket has a pull request attached to it, but is still open. Did the pull request resolve the issue? If so, could you please mark it resolved? This will help the project have a clear view of its open issues.
> setting numshards dynamically for TextIO.write()
> -------------------------------------------------
>
> Key: BEAM-6721
> URL: https://issues.apache.org/jira/browse/BEAM-6721
> Project: Beam
> Issue Type: Improvement
> Components: io-java-text
> Reporter: Aditya Guru
> Priority: P3
> Time Spent: 2h 50m
> Remaining Estimate: 0h
>
> I go through code for TextIO. It uses WriteFiles in TypedWrite. Now, WriteFiles has the functionality withNumShards(ValueProvider<java.lang.Integer>) and [withSharding|https://beam.apache.org/releases/javadoc/2.9.0/org/apache/beam/sdk/io/WriteFiles.html#withSharding-org.apache.beam.sdk.transforms.PTransform-]([PTransform|https://beam.apache.org/releases/javadoc/2.9.0/org/apache/beam/sdk/transforms/PTransform.html]<[PCollection|https://beam.apache.org/releases/javadoc/2.9.0/org/apache/beam/sdk/values/PCollection.html]<[UserT|https://beam.apache.org/releases/javadoc/2.9.0/org/apache/beam/sdk/io/WriteFiles.html]>,[PCollectionView|https://beam.apache.org/releases/javadoc/2.9.0/org/apache/beam/sdk/values/PCollectionView.html]<java.lang.Integer>> sharding) but TextIO or its inner class TypedWrite doesn't have that. It would really be a great to expose that functionality to user.
> Are there some issues in implementation that this is not already available?
--
This message was sent by Atlassian Jira
(v8.20.1#820001)