[GitHub] [beam] KevinGG edited a comment on issue #11050: [BEAM-8335] Implemented Capture Size limitation

[GitBox <gi...@apache.org>]

KevinGG edited a comment on issue #11050: [BEAM-8335] Implemented Capture Size limitation
URL: https://github.com/apache/beam/pull/11050#issuecomment-594967107
 
 
   R: @rohdesamuel 
   R: @aaltay 
   
   Data captured from sources are stored in cache just like intermediate PCollections that are assigned to variables.
   The capture_size limit is only applied to disk usage of data captured from sources.
   
   The implementation of getting a capture cache file's size
   `os.stat(self._path).st_size`
   
   The implementation of summing up all capture cache file's sizes
   `sum([sink.size_in_bytes for _, sink in self._capture_sinks.items()])`
   
   They both locate in [this](https://github.com/apache/beam/pull/11050/commits/a6d9e2382eeea148b3f667726f8e8e8933a7196c#diff-e15d1558a3154511b759ef711deeaddb) change.
   
   Everything else is wiring, logging and testing.
   
   The first commit is a patch from Sam's ongoing PR, there is no need to review diff of it.
   
   

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services