You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@beam.apache.org by GitBox <gi...@apache.org> on 2022/07/21 19:56:03 UTC

[GitHub] [beam] bvolpato opened a new issue, #22401: [Bug]: BigQueryIO getFailedInserts fails when using Storage APIs

bvolpato opened a new issue, #22401:
URL: https://github.com/apache/beam/issues/22401

   ### What happened?
   
   Hello,
   
   I am having issues when using `getFailedInserts` / `getFailedInsertsWithErr` from the WriteResult of a **BigQueryIO** write using methods `STORAGE_WRITE_API` or `STORAGE_API_AT_LEAST_ONCE`.
   
   The message leads us to think there's something wrong with the usage of the API, but there's no combination that can make it work:
   
   
   .getFailedInserts():
   ```
   Exception in thread "main" java.lang.IllegalArgumentException: Cannot use getFailedInserts as this WriteResult uses extended errors information. Use getFailedInsertsWithErr instead
     at org.apache.beam.sdk.util.Preconditions.checkArgumentNotNull(Preconditions.java:68)
     at org.apache.beam.sdk.io.gcp.bigquery.WriteResult.getFailedInserts(WriteResult.java:167)
   ```
   
   .getFailedInsertsWithErr():
   ```
   Exception in thread "main" java.lang.IllegalArgumentException: Cannot use getFailedInsertsWithErr as this WriteResult does not use extended errors. Use getFailedInserts instead
     at org.apache.beam.sdk.util.Preconditions.checkArgumentNotNull(Preconditions.java:68)
     at org.apache.beam.sdk.io.gcp.bigquery.WriteResult.getFailedInsertsWithErr(WriteResult.java:185)
   ```
   
   .withExtendedErrorInfo().getFailedInsertsWithErr():
   ```
   Exception in thread "main" java.lang.IllegalArgumentException: Cannot use getFailedInsertsWithErr as this WriteResult does not use extended errors. Use getFailedInserts instead
     at org.apache.beam.sdk.util.Preconditions.checkArgumentNotNull(Preconditions.java:68)
     at org.apache.beam.sdk.io.gcp.bigquery.WriteResult.getFailedInsertsWithErr(WriteResult.java:185)
   ```
   
   
   .withExtendedErrorInfo().getFailedInserts():
   ```
   Exception in thread "main" java.lang.IllegalArgumentException: Cannot use getFailedInserts as this WriteResult uses extended errors information. Use getFailedInsertsWithErr instead
     at org.apache.beam.sdk.util.Preconditions.checkArgumentNotNull(Preconditions.java:68)
     at org.apache.beam.sdk.io.gcp.bigquery.WriteResult.getFailedInserts(WriteResult.java:167)
   ```
   
   
   
   It appears that they are mutually exclusive, but upon debugging, both `failedInserts` and `failedInsertsWithErr` are initialized as null.
   
   
   This error started after upgrading from 2.38.0 to 2.40.0, and I couldn't find a workaround.
   
   I'm trying to get more details on what's causing it and will update here if I see anything interesting.
   
   
   ### Issue Priority
   
   Priority: 1
   
   ### Issue Component
   
   Component: io-java-gcp


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [beam] johnjcasey closed issue #22401: [Bug]: BigQueryIO getFailedInserts fails when using Storage APIs

Posted by GitBox <gi...@apache.org>.
johnjcasey closed issue #22401: [Bug]: BigQueryIO getFailedInserts fails when using Storage APIs 
URL: https://github.com/apache/beam/issues/22401


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [beam] bvolpato commented on issue #22401: [Bug]: BigQueryIO getFailedInserts fails when using Storage APIs

Posted by GitBox <gi...@apache.org>.
bvolpato commented on issue #22401:
URL: https://github.com/apache/beam/issues/22401#issuecomment-1191886608

   Ok, I found out that there's a specific tag/PCollection for Storage API, which can be obtained through `getFailedStorageApiInserts`. 
   
   Not closing the issue because I feel there's room for improvement on the error message (as there are more exit points for errors now).
   Also, it would be good to have it listed as a breaking change -- I am not sure if it is still possible.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org