[GitHub] [beam] steveniemitz commented on pull request #17365: [BEAM-12482] Update Schema Destination during Bigquery load job when using temporary tables using zeroloadjob

[GitBox <gi...@apache.org>]

steveniemitz commented on PR #17365:
URL: https://github.com/apache/beam/pull/17365#issuecomment-1190690917

   This seems like it introduced a bug, this used to work in beam 2.39:
   ```
   BigQueryIO
      .write()
      .optimizedWrites()
      .to(OutputTable)
      .withAvroFormatFunction(abc)
      .withAvroSchemaFactory(xyz)
      .withCreateDisposition(CreateDisposition.CREATE_NEVER)
      .withWriteDisposition(WriteDisposition.WRITE_TRUNCATE))
   ```
   
   but now throws an exception if you end up in MultiPartitionsWriteTables:
   
   ```
   Caused by: java.lang.IllegalArgumentException: Unless create disposition is CREATE_NEVER, a schema must be specified, i.e. DynamicDestinations.getSchema() may not return null. However, create disposition is CREATE_IF_NEEDED, and org.apache.beam.sdk.io.gcp.bigquery.DynamicDestinationsHelpers$ConstantTableDestinations@52e95e1 returned null for destination tableSpec: <table>
   	at org.apache.beam.sdk.util.Preconditions.checkArgumentNotNull(Preconditions.java:436)
   	at org.apache.beam.sdk.io.gcp.bigquery.WriteTables$WriteTablesDoFn.processElement(WriteTables.java:207)
   ```
   
   The reason seems like now, the create disposition is set to `CreateDisposition.CREATE_IF_NEEDED` no matter what the user sets it to.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org