You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@beam.apache.org by "iht (via GitHub)" <gi...@apache.org> on 2023/11/02 18:14:24 UTC

[PR] [BigQueryIO] Upserts work with CREATE_IF_NEEDED -- doc only PR [beam]

iht opened a new pull request, #29276:
URL: https://github.com/apache/beam/pull/29276

   This is a doc only PR that updates `BigQueryIO` documentation.
   
   Upserts, in streaming, with STORAGE_AT_LEAST_ONCE work also using CREATE_IF_NEEDED if a primary key is specified. Actually, the code examples included in the documentation were already using CREATE_IF_NEEDED, I guess the bits of doc touched in this PR were not updated when the examples were written.
   
   GitHub Actions Tests Status (on master branch)
   ------------------------------------------------------------------------------------------------
   [![Build python source distribution and wheels](https://github.com/apache/beam/workflows/Build%20python%20source%20distribution%20and%20wheels/badge.svg?branch=master&event=schedule)](https://github.com/apache/beam/actions?query=workflow%3A%22Build+python+source+distribution+and+wheels%22+branch%3Amaster+event%3Aschedule)
   [![Python tests](https://github.com/apache/beam/workflows/Python%20tests/badge.svg?branch=master&event=schedule)](https://github.com/apache/beam/actions?query=workflow%3A%22Python+Tests%22+branch%3Amaster+event%3Aschedule)
   [![Java tests](https://github.com/apache/beam/workflows/Java%20Tests/badge.svg?branch=master&event=schedule)](https://github.com/apache/beam/actions?query=workflow%3A%22Java+Tests%22+branch%3Amaster+event%3Aschedule)
   [![Go tests](https://github.com/apache/beam/workflows/Go%20tests/badge.svg?branch=master&event=schedule)](https://github.com/apache/beam/actions?query=workflow%3A%22Go+tests%22+branch%3Amaster+event%3Aschedule)
   
   See [CI.md](https://github.com/apache/beam/blob/master/CI.md) for more information about GitHub Actions CI or the [workflows README](https://github.com/apache/beam/blob/master/.github/workflows/README.md) to see a list of phrases to trigger workflows.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] [BigQueryIO] Upserts work with CREATE_IF_NEEDED -- doc only PR [beam]

Posted by "iht (via GitHub)" <gi...@apache.org>.
iht commented on code in PR #29276:
URL: https://github.com/apache/beam/pull/29276#discussion_r1382438549


##########
sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigquery/BigQueryIO.java:
##########
@@ -486,8 +486,11 @@
  * <h3>Upserts and deletes</h3>
  *
  * The connector also supports streaming row updates to BigQuery, with the following qualifications:
- * - The CREATE_IF_NEEDED CreateDisposition is not supported. Tables must be precreated with primary
- * keys. - Only the STORAGE_WRITE_API_AT_LEAST_ONCE method is supported.
+ *
+ * <p>- If the table is not previously created and CREATE_IF_NEEDED is used, a primary key must be
+ * specified using {@link Write#withPrimaryKey}.
+ *
+ * <p>- Only the STORAGE_WRITE_API_AT_LEAST_ONCE method is supported.

Review Comment:
   Done



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] [BigQueryIO] Upserts work with CREATE_IF_NEEDED -- doc only PR [beam]

Posted by "Abacn (via GitHub)" <gi...@apache.org>.
Abacn commented on code in PR #29276:
URL: https://github.com/apache/beam/pull/29276#discussion_r1381803715


##########
sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigquery/BigQueryIO.java:
##########
@@ -486,8 +486,11 @@
  * <h3>Upserts and deletes</h3>
  *
  * The connector also supports streaming row updates to BigQuery, with the following qualifications:
- * - The CREATE_IF_NEEDED CreateDisposition is not supported. Tables must be precreated with primary
- * keys. - Only the STORAGE_WRITE_API_AT_LEAST_ONCE method is supported.
+ *
+ * <p>- If the table is not previously created and CREATE_IF_NEEDED is used, a primary key must be
+ * specified using {@link Write#withPrimaryKey}.
+ *
+ * <p>- Only the STORAGE_WRITE_API_AT_LEAST_ONCE method is supported.

Review Comment:
   It appears to me the second point is a more generic guideline, then consider switch the order of two points?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] [BigQueryIO] Upserts work with CREATE_IF_NEEDED -- doc only PR [beam]

Posted by "johnjcasey (via GitHub)" <gi...@apache.org>.
johnjcasey commented on PR #29276:
URL: https://github.com/apache/beam/pull/29276#issuecomment-1795078097

   LGTM


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] [BigQueryIO] Upserts work with CREATE_IF_NEEDED -- doc only PR [beam]

Posted by "iht (via GitHub)" <gi...@apache.org>.
iht commented on PR #29276:
URL: https://github.com/apache/beam/pull/29276#issuecomment-1792528774

   assign set of reviewers


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] [BigQueryIO] Upserts work with CREATE_IF_NEEDED -- doc only PR [beam]

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #29276:
URL: https://github.com/apache/beam/pull/29276#issuecomment-1792531040

   Assigning reviewers. If you would like to opt out of this review, comment `assign to next reviewer`:
   
   R: @Abacn for label java.
   R: @johnjcasey for label io.
   
   Available commands:
   - `stop reviewer notifications` - opt out of the automated review tooling
   - `remind me after tests pass` - tag the comment author after tests pass
   - `waiting on author` - shift the attention set back to the author (any comment or push by the author will return the attention set to the reviewers)
   
   The PR bot will only process comments in the main thread (not review comments).


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] [BigQueryIO] Upserts work with CREATE_IF_NEEDED -- doc only PR [beam]

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #29276:
URL: https://github.com/apache/beam/pull/29276#issuecomment-1791430990

   Checks are failing. Will not request review until checks are succeeding. If you'd like to override that behavior, comment `assign set of reviewers`


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] [BigQueryIO] Upserts work with CREATE_IF_NEEDED -- doc only PR [beam]

Posted by "johnjcasey (via GitHub)" <gi...@apache.org>.
johnjcasey merged PR #29276:
URL: https://github.com/apache/beam/pull/29276


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org