You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@beam.apache.org by GitBox <gi...@apache.org> on 2022/06/04 18:10:53 UTC
[GitHub] [beam] damccorm opened a new issue, #20557: fileIO issue on latest version of apache-beam to 2.23.0
damccorm opened a new issue, #20557:
URL: https://github.com/apache/beam/issues/20557
In Apache Beam , while using below P transformation -*fileio.MatchAll() and fileio.ReadMatches()* dataflow job on load performance .( processing 48 million resources)
For 2-3 minute Throughput(element/sec) goes to zero and no element was processed during that time , it comes back again back to average normal (0.25/sec) of my job
In between these three minutes. There is no log from apache beam , if some IO error came or any warning.
Please check for any fileIO known issue on latest version of *apache-beam to 2.23.0*
Imported from Jira [BEAM-11015](https://issues.apache.org/jira/browse/BEAM-11015). Original Jira may contain additional context.
Reported by: apoorva.e.sharma@accenture.com.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] Abacn commented on issue #20557: fileIO issue on latest version of apache-beam to 2.23.0
Posted by GitBox <gi...@apache.org>.
Abacn commented on issue #20557:
URL: https://github.com/apache/beam/issues/20557#issuecomment-1297291396
Intended behavior. Listing resources is not parallelized.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] Abacn commented on issue #20557: fileIO issue on latest version of apache-beam to 2.23.0
Posted by GitBox <gi...@apache.org>.
Abacn commented on issue #20557:
URL: https://github.com/apache/beam/issues/20557#issuecomment-1297291804
.close-issue not_planned
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] github-actions[bot] closed issue #20557: fileIO issue on latest version of apache-beam to 2.23.0
Posted by GitBox <gi...@apache.org>.
github-actions[bot] closed issue #20557: fileIO issue on latest version of apache-beam to 2.23.0
URL: https://github.com/apache/beam/issues/20557
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org