[GitHub] [beam] BjornPrime commented on a diff in pull request #24435: slow draining fix

["BjornPrime (via GitHub)" <gi...@apache.org>]

BjornPrime commented on code in PR #24435:
URL: https://github.com/apache/beam/pull/24435#discussion_r1085832382


##########
sdks/python/apache_beam/transforms/core.py:
##########
@@ -2474,9 +2474,13 @@ def typed(transform):
       # Capture in closure (avoiding capturing self).
       args, kwargs = self.args, self.kwargs
 
+      #TODO: We need to add support for multi-window inputs to CombineGlobally()
       def inject_default(_, combined):
         if combined:
-          assert len(combined) == 1
+          if len(combined) > 1:
+            raise ValueError(
+                "Input from multiple simultaneous windows"
+                "isn't currently supported for CombineGlobally()")

Review Comment:
   My understanding is that, as defined, CombineGlobally is intended to return a single-element PCollection and I'm not sure how to interpret that in a multi-window context. My recollection is that attempting to implement an output with more than one element (e.g. "output = [1,2,3]"), as well as trying to nest that output in a list (e.g. "output = [[1,2,3]]"), both created broader issues for other parts of the execution.
   
   As it relates to the issue at hand, CombineGlobally was only seeing multiple windows due to an issue with closing windows not triggering the transform correctly (see https://github.com/apache/beam/issues/24682) that has now been resolved. I also created https://github.com/apache/beam/issues/24683 to address this gap in CombineGlobally's functionality, since I felt it stretched beyond the scope of this issue.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org