You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@beam.apache.org by GitBox <gi...@apache.org> on 2021/02/02 05:06:48 UTC
[GitHub] [beam] nehsyc commented on a change in pull request #13805: [BEAM-10475] Add transform translator for GroupIntoBatches in Java.
nehsyc commented on a change in pull request #13805:
URL: https://github.com/apache/beam/pull/13805#discussion_r568325487
##########
File path: runners/google-cloud-dataflow-java/src/main/java/org/apache/beam/runners/dataflow/DataflowRunner.java
##########
@@ -1365,21 +1365,19 @@ void addPCollectionRequiringIndexedFormat(PCollection<?> pcol) {
}
void maybeRecordPCollectionWithAutoSharding(PCollection<?> pcol) {
- if (hasExperiment(options, "beam_fn_api")) {
+ if (!options.isEnableStreamingEngine()) {
LOG.warn(
- "Runner determined sharding not available in Dataflow for GroupIntoBatches for portable "
- + "jobs. Default sharding will be applied.");
+ "Runner determined sharding not available in Dataflow for GroupIntoBatches for "
+ + "non-Streaming Engine jobs. Default sharding will be applied.");
return;
}
- if (!options.isEnableStreamingEngine()) {
+ if (!hasExperiment(options, "enable_streaming_auto_sharding")) {
LOG.warn(
- "Runner determined sharding not available in Dataflow for GroupIntoBatches for Streaming "
- + "Appliance jobs. Default sharding will be applied.");
+ "Runner determined sharding not enabled in Dataflow for GroupIntoBatches for Streaming "
+ + "Engine jobs. Default sharding will be applied.");
return;
}
- if (hasExperiment(options, "enable_streaming_auto_sharding")) {
- pcollectionsRequiringAutoSharding.add(pcol);
- }
+ pcollectionsRequiringAutoSharding.add(pcol);
Review comment:
Good catch. Added the check.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org