[GitHub] [beam] kennknowles opened a new issue, #19311: AbstractGlobalCombineFn hierarchy is inconsistent

[GitBox <gi...@apache.org>]

kennknowles opened a new issue, #19311:
URL: https://github.com/apache/beam/issues/19311

   Subclasses of AbstractGlobalCombineFn seem to be arranged in a way that prevents them from being used with the DataflowRunner. 
   
   Subclasses of AbstractGlobalCombineFn are under either CombineFn or CombineFnWithContext, which seems to be in itself a CombineFn which has access to PipelineOptions and Side Inputs.
   
   However, the DataflowRunner casts all combiners passed from user code to CombineFn (see [1]) which prevents combiners that extend CombineFnWithContext from being used there. 
   
    
   
   For example: 
   public class CustomCombinerFn extends CombineWithContext.CombineFnWithContext<...\> \{...}
    
   final PCollectionView<SomeObject<String\>\> newCollection = oldCollection
     .apply("Custom Combiner", Combine.globally(new CustomCombinerFn(filter)) 
     .withSideInputs(filter)
     .withoutDefaults()
     .asSingletonView());
    
   IMHO either CombineFnWithContext should be a subclass of CombineFn or DataflowRunner should cast the combiner to AbstractGlobalCombineFn. 
    
   [1] https://github.com/apache/beam/blob/b83b302ef97767e4ca245ea24e8bd40a6692e72c/runners/google-cloud-dataflow-java/src/main/java/org/apache/beam/runners/dataflow/DataflowRunner.java#L514
   
   Imported from Jira [BEAM-6511](https://issues.apache.org/jira/browse/BEAM-6511). Original Jira may contain additional context.
   Reported by: nlofeudo.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org