[jira] [Created] (BEAM-13612) When BigqueryIO's STORAGE_API_AT_LEAST_ONCE method is used and the table spec doesn't contain the project id the pipeline fails

["Sergei Lilichenko (Jira)" <ji...@apache.org>]

Sergei Lilichenko created BEAM-13612:
----------------------------------------

             Summary: When BigqueryIO's STORAGE_API_AT_LEAST_ONCE method is used and the table spec doesn't contain the project id the pipeline fails 
                 Key: BEAM-13612
                 URL: https://issues.apache.org/jira/browse/BEAM-13612
             Project: Beam
          Issue Type: Bug
          Components: io-java-gcp
    Affects Versions: 2.35.0
            Reporter: Sergei Lilichenko


The validation method correctly uses BigqueryIO's

getTableWithDefaultProject(BigQueryOptions bqOptions), but at the run time the table spec is not checked for absence of the project id, resulting in RuntimeException.

 

 

Stack trace:

Caused by: java.lang.NullPointerException: Required parameter projectId must be specified.
    at com.google.common.base.Preconditions.checkNotNull(Preconditions.java:897)
    at com.google.api.client.util.Preconditions.checkNotNull(Preconditions.java:138)
    at com.google.api.services.bigquery.Bigquery$Tables$Get.<init>(Bigquery.java:5325)
    at com.google.api.services.bigquery.Bigquery$Tables.get(Bigquery.java:5298)
    at org.apache.beam.sdk.io.gcp.bigquery.BigQueryServicesImpl$DatasetServiceImpl.getTable(BigQueryServicesImpl.java:553)
    at org.apache.beam.sdk.io.gcp.bigquery.BigQueryServicesImpl$DatasetServiceImpl.getTable(BigQueryServicesImpl.java:542)
    at org.apache.beam.sdk.io.gcp.bigquery.BigQueryServicesImpl$DatasetServiceImpl.getTable(BigQueryServicesImpl.java:536)
    at org.apache.beam.sdk.io.gcp.bigquery.StorageApiDynamicDestinationsTableRow$1.lambda$$0(StorageApiDynamicDestinationsTableRow.java:66)
    at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$LocalManualCache$1.load(LocalCache.java:4876)
    at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$LoadingValueReference.loadFuture(LocalCache.java:3528)
    at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$Segment.loadSync(LocalCache.java:2277)
    at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$Segment.lockedGetOrLoad(LocalCache.java:2154)
    at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2044)
    ... 35 more



--
This message was sent by Atlassian Jira
(v8.20.1#820001)