You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@beam.apache.org by GitBox <gi...@apache.org> on 2022/11/10 18:50:10 UTC
[GitHub] [beam] ahmedabu98 opened a new issue, #24096: [Bug]: BigQuery query reads are vulnerable to retries caused during temp table deletion.
ahmedabu98 opened a new issue, #24096:
URL: https://github.com/apache/beam/issues/24096
### What happened?
https://github.com/apache/beam/pull/4869 added a [random string](https://github.com/apache/beam/pull/4869/files#diff-b47580f7fcc36b9a014a20be2dd4dc3207ce1e17120358bbf32330d64097f32fR170) to the end of query job names to make this step resilient to retries. This query step creates a temp destination + table to output the query to. The next step would be to export data from this temp table to files, then delete the temp table+destination and read from those files. The problem was if a retry happened right after/during temp table+destination deletion, we would need to retry the query job. However, BQ would see the old successful query job name and return as so without actually querying and creating the temp table. The next step, export to files, would always fail because the table we are trying to export from does not exist.
The solution introduced by https://github.com/apache/beam/pull/4869 was written over in a code refactor in https://github.com/apache/beam/pull/12082 and the random string was lost [here](https://github.com/apache/beam/commit/9af54351ddc31c607d1634382704c8a52aaa78f3#diff-471ca9a2a4c3366613d75c1d98736b7d88c0873d2abc126e01577c937c3759dbL145). This re-introduced the problem of having a deterministic query job name that was vulnerable to retries.
### Issue Priority
Priority: 2
### Issue Component
Component: io-java-gcp
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] ahmedabu98 commented on issue #24096: [Bug]: BigQuery query reads are vulnerable to retries caused during temp table deletion.
Posted by GitBox <gi...@apache.org>.
ahmedabu98 commented on issue #24096:
URL: https://github.com/apache/beam/issues/24096#issuecomment-1310754612
Replaced with #24097
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] ahmedabu98 closed issue #24096: [Bug]: BigQuery query reads are vulnerable to retries caused during temp table deletion.
Posted by GitBox <gi...@apache.org>.
ahmedabu98 closed issue #24096: [Bug]: BigQuery query reads are vulnerable to retries caused during temp table deletion.
URL: https://github.com/apache/beam/issues/24096
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org