You are viewing a plain text version of this content. The canonical link for it is here.

Posted to github@beam.apache.org by "Abacn (via GitHub)" <gi...@apache.org> on 2023/02/09 22:41:19 UTC

[GitHub] [beam] Abacn commented on pull request #25410: Fix UpdateSchemaDestination breaking DynamicDestination in Bigquery BatchLoad

Abacn commented on PR #25410:
URL: https://github.com/apache/beam/pull/25410#issuecomment-1424934024

   The current UpdateSchemaDestination implementation in Java does not consider the dynamic destination. It simply takes the first seen destination. This causes problems in many ways. The integration test supposed to test this class did not work either. It should set `.withMaxFileSize()` instead of `.withMaxBytesPerPartition()` to get multiple files then multiple partitions. Otherwise there was a single file written and did not access to MultiPartitions branch in the pipeline.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org