[GitHub] [beam] RyuSA opened a new issue, #27298: [Task]: Update the document for BigQueryIO features

["RyuSA (via GitHub)" <gi...@apache.org>]

RyuSA opened a new issue, #27298:
URL: https://github.com/apache/beam/issues/27298

   ### What needs to happen?
   
   
   **Auto Sharding**
   
   Current document mentioned the autosharding with Storage Write API is not supported. 
   <img width="899" alt="image" src="https://github.com/apache/beam/assets/12961775/674652e6-05f6-469b-a264-3ac69bf235e6">
   https://beam.apache.org/documentation/io/built-in/google-bigquery/#exactly-once-semantics
   
   But this PR make AutoSharding available, and `Experimental` annotation was removed in Apache Beam 2.48.0.
   https://github.com/apache/beam/pull/16795
   
   I think it's time to announce that BigQueryIO can use Storage Write API with `withAutoSharding`!
   
   **ErrorHandling**
   
   Current documentation mentioned `WriteResult.getFailedInserts` does not work for Storage Write API. 
   
   <img width="876" alt="image" src="https://github.com/apache/beam/assets/12961775/0ce42ae3-419d-4ee8-ac53-767b8d0bb4a6">
   https://beam.apache.org/documentation/io/built-in/google-bigquery/#exactly-once-semantics
   
   But now WriteResult has another function `#getFailedStorageApiInserts`. I think it is better for users to mention this function to rescue failed rows.
   
   https://github.com/apache/beam/blob/v2.48.0/sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigquery/WriteResult.java#L214
   
   ---
   
   I'll create a PR if I can do that, please let me know. Thank you in advance.👋
   
   
   
   
   ### Issue Priority
   
   Priority: 3 (nice-to-have improvement)
   
   ### Issue Components
   
   - [ ] Component: Python SDK
   - [X] Component: Java SDK
   - [ ] Component: Go SDK
   - [ ] Component: Typescript SDK
   - [X] Component: IO connector
   - [ ] Component: Beam examples
   - [ ] Component: Beam playground
   - [ ] Component: Beam katas
   - [ ] Component: Website
   - [ ] Component: Spark Runner
   - [ ] Component: Flink Runner
   - [ ] Component: Samza Runner
   - [ ] Component: Twister2 Runner
   - [ ] Component: Hazelcast Jet Runner
   - [ ] Component: Google Cloud Dataflow Runner


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [I] [Task]: Update the document for BigQueryIO features [beam]

["ahmedabu98 (via GitHub)" <gi...@apache.org>]

ahmedabu98 closed issue #27298: [Task]: Update the document for BigQueryIO features
URL: https://github.com/apache/beam/issues/27298


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org