You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@beam.apache.org by "Beam JIRA Bot (Jira)" <ji...@apache.org> on 2022/03/20 16:59:00 UTC
[jira] [Updated] (BEAM-13962) Dataflow Pub/Sub to BigQuery NullPointerExceptionat
[ https://issues.apache.org/jira/browse/BEAM-13962?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Beam JIRA Bot updated BEAM-13962:
---------------------------------
Labels: google-cloud-dataflow pubsub stale-assigned (was: google-cloud-dataflow pubsub)
> Dataflow Pub/Sub to BigQuery NullPointerExceptionat
> ---------------------------------------------------
>
> Key: BEAM-13962
> URL: https://issues.apache.org/jira/browse/BEAM-13962
> Project: Beam
> Issue Type: Bug
> Components: io-java-gcp
> Affects Versions: 2.34.0
> Environment: GCP, Dataflow
> Reporter: Elon Chen
> Assignee: Kenneth Knowles
> Priority: P3
> Labels: google-cloud-dataflow, pubsub, stale-assigned
>
> Here is the error message when I run streaming Pub/Sub to BigQuery in Dataflow
>
> {code:java}
> java.lang.RuntimeException: java.lang.NullPointerExceptionat org.apache.beam.sdk.io.gcp.bigquery.BigQueryServicesImpl$DatasetServiceImpl.insertAll ( org/apache.beam.sdk.io.gcp.bigquery/BigQueryServicesImpl.java:1001 )at org.apache.beam.sdk.io.gcp.bigquery.BigQueryServicesImpl$DatasetServiceImpl.insertAll ( org/apache.beam.sdk.io.gcp.bigquery/BigQueryServicesImpl.java:1054 )at org.apache.beam.sdk.io.gcp.bigquery.BatchedStreamingWrite.flushRows ( org/apache.beam.sdk.io.gcp.bigquery/BatchedStreamingWrite.java:421 )at org.apache.beam.sdk.io.gcp.bigquery.BatchedStreamingWrite.access$900 ( org/apache.beam.sdk.io.gcp.bigquery/BatchedStreamingWrite.java:72 )at org.apache.beam.sdk.io.gcp.bigquery.BatchedStreamingWrite$BatchAndInsertElements.finishBundle ( org/apache.beam.sdk.io.gcp.bigquery/BatchedStreamingWrite.java:267 )Caused by: java.lang.NullPointerExceptionat org.apache.beam.sdk.io.gcp.bigquery.BigQueryServicesImpl$DatasetServiceImpl.getErrorInfo ( BigQueryServicesImpl.java:1075 )at org.apache.beam.sdk.io.gcp.bigquery.BigQueryServicesImpl$DatasetServiceImpl.lambda$insertAll$1 ( BigQueryServicesImpl.java:922 )at org.apache.beam.sdk.io.gcp.bigquery.BigQueryServicesImpl$BoundedExecutorService$SemaphoreCallable.call ( BigQueryServicesImpl.java:1594 )at java.util.concurrent.FutureTask.run ( FutureTask.java:264 )at java.util.concurrent.ThreadPoolExecutor.runWorker ( ThreadPoolExecutor.java:1128 )at java.util.concurrent.ThreadPoolExecutor$Worker.run ( ThreadPoolExecutor.java:628 )at java.lang.Thread.run ( Thread.java:834 ){code}
> And here is another one.
> {code:java}
> Error message from worker: java.lang.RuntimeException: com.google.api.client.googleapis.json.GoogleJsonResponseException: 400 Bad Request
> POST https://bigquery.googleapis.com/bigquery/v2/projects/{project-id}/datasets/{dataset-id}/tables/{table-id}/insertAll?prettyPrint=false
> {
> "code" : 400,
> "errors" : [ {
> "domain" : "global",
> "message" : "No rows present in the request.",
> "reason" : "invalid"
> } ],
> "message" : "No rows present in the request.",
> "status" : "INVALID_ARGUMENT"
> }
> org.apache.beam.sdk.io.gcp.bigquery.BigQueryServicesImpl$DatasetServiceImpl.insertAll(BigQueryServicesImpl.java:1001)
> org.apache.beam.sdk.io.gcp.bigquery.BigQueryServicesImpl$DatasetServiceImpl.insertAll(BigQueryServicesImpl.java:1054)
> org.apache.beam.sdk.io.gcp.bigquery.BatchedStreamingWrite.flushRows(BatchedStreamingWrite.java:421)
> org.apache.beam.sdk.io.gcp.bigquery.BatchedStreamingWrite.access$900(BatchedStreamingWrite.java:72)
> org.apache.beam.sdk.io.gcp.bigquery.BatchedStreamingWrite$BatchAndInsertElements.finishBundle(BatchedStreamingWrite.java:267)
> Caused by: com.google.api.client.googleapis.json.GoogleJsonResponseException: 400 Bad Request
> POST https://bigquery.googleapis.com/bigquery/v2/projects/{project-id}/datasets/{dataset-id}/tables/{table-id}/insertAll?prettyPrint=false
> {
> "code" : 400,
> "errors" : [ {
> "domain" : "global",
> "message" : "No rows present in the request.",
> "reason" : "invalid"
> } ],
> "message" : "No rows present in the request.",
> "status" : "INVALID_ARGUMENT"
> }
> com.google.api.client.googleapis.json.GoogleJsonResponseException.from(GoogleJsonResponseException.java:146)
> com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:118)
> com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:37)
> com.google.api.client.googleapis.services.AbstractGoogleClientRequest$1.interceptResponse(AbstractGoogleClientRequest.java:428)
> com.google.api.client.http.HttpRequest.execute(HttpRequest.java:1111)
> com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:514)
> com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:455)
> com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:565)
> org.apache.beam.sdk.io.gcp.bigquery.BigQueryServicesImpl$DatasetServiceImpl.lambda$insertAll$1(BigQueryServicesImpl.java:910)
> org.apache.beam.sdk.io.gcp.bigquery.BigQueryServicesImpl$BoundedExecutorService$SemaphoreCallable.call(BigQueryServicesImpl.java:1594)
> java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
> java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
> java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
> java.base/java.lang.Thread.run(Thread.java:834) {code}
> I'm wondering those errors will be affected my dataflow job? What kind of situation will happen? I just search my hold day, but I can't find out any similar problem. Thank you all.
>
--
This message was sent by Atlassian Jira
(v8.20.1#820001)