You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@beam.apache.org by GitBox <gi...@apache.org> on 2020/10/28 23:35:04 UTC

[GitHub] [beam] tvalentyn commented on a change in pull request #13187: Updated shared.py comments

tvalentyn commented on a change in pull request #13187:
URL: https://github.com/apache/beam/pull/13187#discussion_r513823318



##########
File path: sdks/python/apache_beam/utils/shared.py
##########
@@ -26,22 +26,31 @@
 
 To share a very large list across all threads of each worker in a DoFn::
 
-  class GetNthStringFn(beam.DoFn):
-    def __init__(self, shared_handle):
-      self._shared_handle = shared_handle
-
-    def process(self, element):
-      def initialize_list():
-        # Build the giant initial list.
-        return [str(i) for i in range(1000000)]
-
-      giant_list = self._shared_handle.acquire(initialize_list)
-      yield giant_list[element]
-
-  p = beam.Pipeline()
-  shared_handle = shared.Shared()
-  (p | beam.Create([2, 4, 6, 8])
-     | beam.ParDo(GetNthStringFn(shared_handle)))
+# Several built-in types such as list and dict do not directly support weak

Review comment:
       Consider following wording for line 29 to prepare the reader for why weak references are discussed:
   
   Shared is a helper class for managing a single instance of an object shared by multiple threads within the same process. Instances of Shared are serializable objects that can be shared by all threads of each worker process. A Shared object encapsulates a weak reference to a singleton instance of the shared resource. The singleton is lazily initialized by calls to Shared.acquire().
   




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org