You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@beam.apache.org by GitBox <gi...@apache.org> on 2022/06/03 18:33:39 UTC
[GitHub] [beam] kennknowles opened a new issue, #18419: Log the reason for failure for BigQuery jobs
kennknowles opened a new issue, #18419:
URL: https://github.com/apache/beam/issues/18419
For BigQuery load jobs we currently retry few times and log the job ID.
It might be good to get the reason for the failure from BigQuery (prob. this is available through status poll) and log that as well. For example, failed load jobs are logged at following location.
https://github.com/apache/beam/blob/master/sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigquery/WriteTables.java#L202
Imported from Jira [BEAM-2843](https://issues.apache.org/jira/browse/BEAM-2843). Original Jira may contain additional context.
Reported by: chamikara.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Log the reason for failure for BigQuery jobs [beam]
Posted by "mburuian6 (via GitHub)" <gi...@apache.org>.
mburuian6 commented on issue #18419:
URL: https://github.com/apache/beam/issues/18419#issuecomment-3174932086
.take-issue
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Log the reason for failure for BigQuery jobs [beam]
Posted by "liferoad (via GitHub)" <gi...@apache.org>.
liferoad commented on issue #18419:
URL: https://github.com/apache/beam/issues/18419#issuecomment-3253396638
cc @ahmedabu98
I think we should always log errorResult. It is fine to log the last error more explicitly.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Log the reason for failure for BigQuery jobs [beam]
Posted by "mburuian6 (via GitHub)" <gi...@apache.org>.
mburuian6 commented on issue #18419:
URL: https://github.com/apache/beam/issues/18419#issuecomment-3252047411
- It seems BigQuery (BQ) promises to gives us the final error in [`errorResult`](https://cloud.google.com/bigquery/docs/reference/rest/v2/Job#jobstatus) field
- but this field actually contains the first error, not the last one from [`errors`](https://cloud.google.com/bigquery/docs/reference/rest/v2/Job#jobstatus) list
- looking at a [sample Dataflow Beam job](https://github.com/user-attachments/assets/8d65abad-7779-4e84-ba8f-e29af0b792c1) loading from one BQ table to another, the last error is the most meaningful and probably the one we should log @kennknowles , question is should we still log errorResult in case it's a BQ issue or just log the final error from `errors` list
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Log the reason for failure for BigQuery jobs [beam]
Posted by "damccorm (via GitHub)" <gi...@apache.org>.
damccorm closed issue #18419: Log the reason for failure for BigQuery jobs
URL: https://github.com/apache/beam/issues/18419
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org