You are viewing a plain text version of this content. The canonical link for it is here.

Posted to commits@beam.apache.org by GitBox <gi...@apache.org> on 2020/03/05 00:40:56 UTC

[GitHub] [beam] KevinGG commented on issue #11050: [BEAM-8335] Implemented Capture Size limitation

KevinGG commented on issue #11050: [BEAM-8335] Implemented Capture Size limitation
URL: https://github.com/apache/beam/pull/11050#issuecomment-594967107
 
 
   R: @rohdesamuel 
   R: @aaltay 
   
   Data captured from sources are stored in cache just like intermediate PCollections that are assigned to variables.
   The capture_size limit is only applied to disk usage of data captured from sources.
   
   The implementation of getting a capture cache file's size
   `os.stat(self._path).st_size`
   
   The implementation of summing up all capture cache file's sizes
   `sum([sink.size_in_bytes for _, sink in self._capture_sinks.items()])`
   
   Everything else is wiring, logging and testing.
   
   
   

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services