You are viewing a plain text version of this content. The canonical link for it is here.

Posted to github@beam.apache.org by GitBox <gi...@apache.org> on 2021/08/11 13:38:03 UTC

[GitHub] [beam] scwhittle commented on pull request #15301: [BEAM-12740] [BEAM-8268] Improve error handling and retries for GCS rename used by FileBasedSink

scwhittle commented on pull request #15301:
URL: https://github.com/apache/beam/pull/15301#issuecomment-896834893


   PTAL, I removed the hacks for testing since the performance seemed greatly improved.
   Additionally I removed the unnecessary delete of renamed files in FileBasedSink by improving FileSystems.remove to always delete src files (even if the copy is skipped due to dest existing and that being filtered).
   I added a unit test for rename by adding support to inject a batch object. The apiary BatchRequest object is difficult to mock so I opted for mocking above that.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org