[GitHub] [beam] lukecwik commented on a change in pull request #12241: [BEAM-10420] Fix minor race condition related to splitting that will cause None has no method 'yyy'

[GitBox <gi...@apache.org>]

lukecwik commented on a change in pull request #12241:
URL: https://github.com/apache/beam/pull/12241#discussion_r454518553



##########
File path: sdks/python/apache_beam/runners/common.py
##########
@@ -842,29 +847,37 @@ def _invoke_process_per_window(self,
 
   def try_split(self, fraction):
     # type: (...) -> Optional[Tuple[SplitResultPrimary, SplitResultResidual]]
-    if self.threadsafe_restriction_tracker and self.current_windowed_value:
+    if not self.is_splittable:
+      return None
+
+    with self.splitting_lock:
+      # Make a local reference to member variables that change references during
+      # processing under lock before attempting to split so we have a consistent
+      # view of all the references.
+      current_windowed_value = self.current_windowed_value
+      threadsafe_restriction_tracker = self.threadsafe_restriction_tracker
+      threadsafe_watermark_estimator = self.threadsafe_watermark_estimator
+
+    if threadsafe_restriction_tracker:

Review comment:
       There is an assumption that if one is set then the other is set. Similarly, if one is unset then the other is unset.

##########
File path: sdks/python/apache_beam/runners/common.py
##########
@@ -842,29 +847,37 @@ def _invoke_process_per_window(self,
 
   def try_split(self, fraction):
     # type: (...) -> Optional[Tuple[SplitResultPrimary, SplitResultResidual]]
-    if self.threadsafe_restriction_tracker and self.current_windowed_value:
+    if not self.is_splittable:
+      return None
+
+    with self.splitting_lock:
+      # Make a local reference to member variables that change references during
+      # processing under lock before attempting to split so we have a consistent
+      # view of all the references.
+      current_windowed_value = self.current_windowed_value

Review comment:
       The purpose of the lock is to get a consistent point in time copy of the references to the current objects.
   
   We don't need a deep copy. Splitting will fail if the restriction becomes finished and the main processing loop moves onto the next element.
   




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org