You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@beam.apache.org by GitBox <gi...@apache.org> on 2022/09/07 20:42:11 UTC

[GitHub] [beam] svetakvsundhar opened a new pull request, #23070: Support TIMESTAMP type in BigQueryIO with BEAM_ROW output type, and in read_gbq

svetakvsundhar opened a new pull request, #23070:
URL: https://github.com/apache/beam/pull/23070

   Issue: https://github.com/apache/beam/issues/22909
   
   This PR adds support for the `TIMESTAMP` time from BigQuery, and maps it to a `BEAM_ROW` type. This will be supported in Python's BigQuery IO implementation, as well as read_gbq.
   
   ------------------------
   
   Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily:
   
    - [ ] [**Choose reviewer(s)**](https://beam.apache.org/contribute/#make-your-change) and mention them in a comment (`R: @username`).
    - [ ] Mention the appropriate issue in your description (for example: `addresses #123`), if applicable. This will automatically add a link to the pull request in the issue. If you would like the issue to automatically close on merging the pull request, comment `fixes #<ISSUE NUMBER>` instead.
    - [ ] Update `CHANGES.md` with noteworthy changes.
    - [ ] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.pdf).
   
   See the [Contributor Guide](https://beam.apache.org/contribute) for more tips on [how to make review process smoother](https://beam.apache.org/contribute/get-started-contributing/#make-the-reviewers-job-easier).
   
   To check the build health, please visit [https://github.com/apache/beam/blob/master/.test-infra/BUILD_STATUS.md](https://github.com/apache/beam/blob/master/.test-infra/BUILD_STATUS.md)
   
   GitHub Actions Tests Status (on master branch)
   ------------------------------------------------------------------------------------------------
   [![Build python source distribution and wheels](https://github.com/apache/beam/workflows/Build%20python%20source%20distribution%20and%20wheels/badge.svg?branch=master&event=schedule)](https://github.com/apache/beam/actions?query=workflow%3A%22Build+python+source+distribution+and+wheels%22+branch%3Amaster+event%3Aschedule)
   [![Python tests](https://github.com/apache/beam/workflows/Python%20tests/badge.svg?branch=master&event=schedule)](https://github.com/apache/beam/actions?query=workflow%3A%22Python+Tests%22+branch%3Amaster+event%3Aschedule)
   [![Java tests](https://github.com/apache/beam/workflows/Java%20Tests/badge.svg?branch=master&event=schedule)](https://github.com/apache/beam/actions?query=workflow%3A%22Java+Tests%22+branch%3Amaster+event%3Aschedule)
   [![Go tests](https://github.com/apache/beam/workflows/Go%20tests/badge.svg?branch=master&event=schedule)](https://github.com/apache/beam/actions?query=workflow%3A%22Go+tests%22+branch%3Amaster+event%3Aschedule)
   
   See [CI.md](https://github.com/apache/beam/blob/master/CI.md) for more information about GitHub Actions CI.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [beam] codecov[bot] commented on pull request #23070: Support TIMESTAMP type in BigQueryIO with BEAM_ROW output type, and in read_gbq

Posted by GitBox <gi...@apache.org>.
codecov[bot] commented on PR #23070:
URL: https://github.com/apache/beam/pull/23070#issuecomment-1239886295

   # [Codecov](https://codecov.io/gh/apache/beam/pull/23070?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
   > Merging [#23070](https://codecov.io/gh/apache/beam/pull/23070?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (277cb40) into [master](https://codecov.io/gh/apache/beam/commit/25c6ed74c9846c89a92655c1e8d313ef87d6adb1?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (25c6ed7) will **decrease** coverage by `0.04%`.
   > The diff coverage is `100.00%`.
   
   ```diff
   @@            Coverage Diff             @@
   ##           master   #23070      +/-   ##
   ==========================================
   - Coverage   73.67%   73.63%   -0.05%     
   ==========================================
     Files         714      716       +2     
     Lines       95220    95292      +72     
   ==========================================
   + Hits        70154    70164      +10     
   - Misses      23769    23831      +62     
     Partials     1297     1297              
   ```
   
   | Flag | Coverage Δ | |
   |---|---|---|
   | python | `83.41% <100.00%> (-0.08%)` | :arrow_down: |
   
   Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#carryforward-flags-in-the-pull-request-comment) to find out more.
   
   | [Impacted Files](https://codecov.io/gh/apache/beam/pull/23070?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
   |---|---|---|
   | [...python/apache\_beam/io/gcp/bigquery\_schema\_tools.py](https://codecov.io/gh/apache/beam/pull/23070/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vaW8vZ2NwL2JpZ3F1ZXJ5X3NjaGVtYV90b29scy5weQ==) | `84.44% <100.00%> (+0.35%)` | :arrow_up: |
   | [sdks/python/apache\_beam/internal/pickler.py](https://codecov.io/gh/apache/beam/pull/23070/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vaW50ZXJuYWwvcGlja2xlci5weQ==) | `92.00% <0.00%> (-3.46%)` | :arrow_down: |
   | [.../apache\_beam/runners/interactive/dataproc/types.py](https://codecov.io/gh/apache/beam/pull/23070/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9pbnRlcmFjdGl2ZS9kYXRhcHJvYy90eXBlcy5weQ==) | `93.10% <0.00%> (-3.45%)` | :arrow_down: |
   | [...eam/runners/portability/fn\_api\_runner/execution.py](https://codecov.io/gh/apache/beam/pull/23070/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9wb3J0YWJpbGl0eS9mbl9hcGlfcnVubmVyL2V4ZWN1dGlvbi5weQ==) | `92.44% <0.00%> (-0.65%)` | :arrow_down: |
   | [...eam/runners/interactive/interactive\_environment.py](https://codecov.io/gh/apache/beam/pull/23070/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9pbnRlcmFjdGl2ZS9pbnRlcmFjdGl2ZV9lbnZpcm9ubWVudC5weQ==) | `91.71% <0.00%> (-0.31%)` | :arrow_down: |
   | [...hon/apache\_beam/runners/worker/bundle\_processor.py](https://codecov.io/gh/apache/beam/pull/23070/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy93b3JrZXIvYnVuZGxlX3Byb2Nlc3Nvci5weQ==) | `93.54% <0.00%> (-0.13%)` | :arrow_down: |
   | [...am/examples/inference/pytorch\_language\_modeling.py](https://codecov.io/gh/apache/beam/pull/23070/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vZXhhbXBsZXMvaW5mZXJlbmNlL3B5dG9yY2hfbGFuZ3VhZ2VfbW9kZWxpbmcucHk=) | `0.00% <0.00%> (ø)` | |
   | [...m/examples/inference/pytorch\_image\_segmentation.py](https://codecov.io/gh/apache/beam/pull/23070/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vZXhhbXBsZXMvaW5mZXJlbmNlL3B5dG9yY2hfaW1hZ2Vfc2VnbWVudGF0aW9uLnB5) | `0.00% <0.00%> (ø)` | |
   | [...examples/inference/pytorch\_image\_classification.py](https://codecov.io/gh/apache/beam/pull/23070/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vZXhhbXBsZXMvaW5mZXJlbmNlL3B5dG9yY2hfaW1hZ2VfY2xhc3NpZmljYXRpb24ucHk=) | `0.00% <0.00%> (ø)` | |
   | [...examples/inference/sklearn\_mnist\_classification.py](https://codecov.io/gh/apache/beam/pull/23070/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vZXhhbXBsZXMvaW5mZXJlbmNlL3NrbGVhcm5fbW5pc3RfY2xhc3NpZmljYXRpb24ucHk=) | `43.75% <0.00%> (ø)` | |
   | ... and [6 more](https://codecov.io/gh/apache/beam/pull/23070/diff?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | |
   
   :mega: We’re building smart automated test selection to slash your CI/CD build times. [Learn more](https://about.codecov.io/iterative-testing/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [beam] svetakvsundhar commented on pull request #23070: Support TIMESTAMP type in BigQueryIO with BEAM_ROW output type, and in read_gbq

Posted by GitBox <gi...@apache.org>.
svetakvsundhar commented on PR #23070:
URL: https://github.com/apache/beam/pull/23070#issuecomment-1284427849

   sg, looks like 3.7 ran the tests in question 
   <img width="1205" alt="Screen Shot 2022-10-19 at 1 13 11 PM" src="https://user-images.githubusercontent.com/26037657/196777091-6de9d8da-6c7c-4170-b09c-7a0027a9586c.png">
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [beam] ahmedabu98 commented on a diff in pull request #23070: Support TIMESTAMP type in BigQueryIO with BEAM_ROW output type, and in read_gbq

Posted by GitBox <gi...@apache.org>.
ahmedabu98 commented on code in PR #23070:
URL: https://github.com/apache/beam/pull/23070#discussion_r986013514


##########
sdks/python/apache_beam/io/gcp/bigquery_schema_tools_test.py:
##########
@@ -61,6 +61,25 @@ def test_check_conversion_with_empty_schema(self):
         the_table_schema=schema)
     self.assertEqual(usertype.__annotations__, {})
 
+  def test_check_schema_conversions_with_timestamp(self):
+    fields = [
+        bigquery.TableFieldSchema(name='stn', type='STRING', mode="NULLABLE"),
+        bigquery.TableFieldSchema(name='temp', type='FLOAT64', mode="REPEATED"),
+        bigquery.TableFieldSchema(
+            name='times', type='TIMESTAMP', mode="NULLABLE")
+    ]
+    schema = bigquery.TableSchema(fields=fields)
+
+    usertype = bigquery_schema_tools.generate_user_type_from_bq_schema(
+        the_table_schema=schema)
+    self.assertEqual(
+        usertype.__annotations__,
+        {
+            'stn': typing.Optional[str],
+            'temp': typing.Sequence[np.float64],
+            'times': typing.Any

Review Comment:
   ```
               'times': typing.Any
   ```
   Should this be testing for an Optional `datetime.datetime` type?



##########
sdks/python/apache_beam/io/gcp/bigquery_read_it_test.py:
##########
@@ -258,6 +258,123 @@ def test_table_schema_retrieve_with_direct_read(self):
               utype(id=4, name='customer2', type='test')
           ]))
 
+  @pytest.mark.it_postcommit
+  def test_table_schema_retrieve_with_timestamp(self):
+    the_table = bigquery_tools.BigQueryWrapper().get_table(
+        project_id="apache-beam-testing",
+        dataset_id="beam_bigquery_io_test",
+        table_id="taxi_small")
+    table = the_table.schema
+    utype = bigquery_schema_tools. \
+          generate_user_type_from_bq_schema(table)
+    with beam.Pipeline(argv=self.args) as p:
+      result = (
+          p | apache_beam.io.gcp.bigquery.ReadFromBigQuery(
+              gcs_location="gs://bqio_schema_test",
+              dataset="beam_bigquery_io_test",
+              table="taxi_small",
+              project="apache-beam-testing",
+              output_type='BEAM_ROW'))
+      assert_that(
+          result,
+          equal_to([
+              utype(
+                  event_timestamp=datetime.datetime(
+                      2021,
+                      11,
+                      5,
+                      20,
+                      57,
+                      43,
+                      612000,
+                      tzinfo=datetime.timezone.utc),
+                  ride_id='0004b5de-8db1-425b-8eec-7a25b04a1ee0',
+                  point_idx=9,
+                  latitude=40.757380000000005,
+                  timestamp='2021-11-05T16:57:43.60906-04:00',
+                  meter_reading=0.37846154,
+                  meter_increment=0.04205128,
+                  ride_status='enroute',
+                  passenger_count=1,
+                  longitude=-73.98969000000001),

Review Comment:
   Is it possible to only test for the timestamp field since that's what this test is focusing on? It would help make the test more concise and easy to read in the future. 
   
   One way is to change it so that ReadFromBigQuery queries for the timestamp field. If schema conversion isn't implemented yet for query reads, another way may be to add a step that filters `result` so you have a PCollection of timestamp elements.



##########
sdks/python/apache_beam/io/gcp/bigquery_read_it_test.py:
##########
@@ -258,6 +258,123 @@ def test_table_schema_retrieve_with_direct_read(self):
               utype(id=4, name='customer2', type='test')
           ]))
 
+  @pytest.mark.it_postcommit
+  def test_table_schema_retrieve_with_timestamp(self):
+    the_table = bigquery_tools.BigQueryWrapper().get_table(
+        project_id="apache-beam-testing",
+        dataset_id="beam_bigquery_io_test",
+        table_id="taxi_small")
+    table = the_table.schema
+    utype = bigquery_schema_tools. \
+          generate_user_type_from_bq_schema(table)

Review Comment:
   ```suggestion
       schema = the_table.schema
       utype = bigquery_schema_tools. \
             generate_user_type_from_bq_schema(schema)
   ```
   
   To avoid confusion



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [beam] svetakvsundhar commented on pull request #23070: Support TIMESTAMP type in BigQueryIO with BEAM_ROW output type, and in read_gbq

Posted by GitBox <gi...@apache.org>.
svetakvsundhar commented on PR #23070:
URL: https://github.com/apache/beam/pull/23070#issuecomment-1283944493

   Run Python PreCommit


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [beam] svetakvsundhar commented on pull request #23070: Support TIMESTAMP type in BigQueryIO with BEAM_ROW output type, and in read_gbq

Posted by GitBox <gi...@apache.org>.
svetakvsundhar commented on PR #23070:
URL: https://github.com/apache/beam/pull/23070#issuecomment-1284428817

   sg, looks like 3.7 ran the tests in question 
   <img width="1070" alt="Screen Shot 2022-10-19 at 2 41 37 PM" src="https://user-images.githubusercontent.com/26037657/196777252-d2f1c8a8-ab81-4b61-9fd3-a7a53dc0c187.png">
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [beam] github-actions[bot] commented on pull request #23070: Support TIMESTAMP type in BigQueryIO with BEAM_ROW output type, and in read_gbq

Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on PR #23070:
URL: https://github.com/apache/beam/pull/23070#issuecomment-1279733288

   Reminder, please take a look at this pr: @y1chi @johnjcasey 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [beam] tvalentyn commented on pull request #23070: Support TIMESTAMP type in BigQueryIO with BEAM_ROW output type, and in read_gbq

Posted by GitBox <gi...@apache.org>.
tvalentyn commented on PR #23070:
URL: https://github.com/apache/beam/pull/23070#issuecomment-1284482613

   > Py37 postcommits have been consistently timing out: Build timed out (after 240 minutes). Marking the build as aborted.
   
   This is strange, I have increased the timeout in: https://github.com/apache/beam/pull/23595


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [beam] svetakvsundhar commented on pull request #23070: Support TIMESTAMP type in BigQueryIO with BEAM_ROW output type, and in read_gbq

Posted by GitBox <gi...@apache.org>.
svetakvsundhar commented on PR #23070:
URL: https://github.com/apache/beam/pull/23070#issuecomment-1262469755

   Run Python 3.7 PostCommit


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [beam] svetakvsundhar commented on pull request #23070: Support TIMESTAMP type in BigQueryIO with BEAM_ROW output type, and in read_gbq

Posted by GitBox <gi...@apache.org>.
svetakvsundhar commented on PR #23070:
URL: https://github.com/apache/beam/pull/23070#issuecomment-1264535565

   Run Python 3.7 PostCommit


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [beam] github-actions[bot] commented on pull request #23070: Support TIMESTAMP type in BigQueryIO with BEAM_ROW output type, and in read_gbq

Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on PR #23070:
URL: https://github.com/apache/beam/pull/23070#issuecomment-1256159920

   Reminder, please take a look at this pr: @tvalentyn @ahmedabu98 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [beam] svetakvsundhar commented on pull request #23070: Support TIMESTAMP type in BigQueryIO with BEAM_ROW output type, and in read_gbq

Posted by GitBox <gi...@apache.org>.
svetakvsundhar commented on PR #23070:
URL: https://github.com/apache/beam/pull/23070#issuecomment-1263836996

   Run Python 3.7 PostCommit


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [beam] github-actions[bot] commented on pull request #23070: Support TIMESTAMP type in BigQueryIO with BEAM_ROW output type, and in read_gbq

Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on PR #23070:
URL: https://github.com/apache/beam/pull/23070#issuecomment-1240047319

   Assigning reviewers. If you would like to opt out of this review, comment `assign to next reviewer`:
   
   R: @tvalentyn for label python.
   R: @ahmedabu98 for label io.
   
   Available commands:
   - `stop reviewer notifications` - opt out of the automated review tooling
   - `remind me after tests pass` - tag the comment author after tests pass
   - `waiting on author` - shift the attention set back to the author (any comment or push by the author will return the attention set to the reviewers)
   
   The PR bot will only process comments in the main thread (not review comments).


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [beam] ahmedabu98 commented on a diff in pull request #23070: Support TIMESTAMP type in BigQueryIO with BEAM_ROW output type, and in read_gbq

Posted by GitBox <gi...@apache.org>.
ahmedabu98 commented on code in PR #23070:
URL: https://github.com/apache/beam/pull/23070#discussion_r966150744


##########
sdks/python/apache_beam/io/gcp/bigquery_read_it_test.py:
##########
@@ -258,6 +258,18 @@ def test_table_schema_retrieve_with_direct_read(self):
               utype(id=4, name='customer2', type='test')
           ]))
 
+  @pytest.mark.it_postcommit
+  def test_table_schema_retrieve_with_timestamp(self):
+    with beam.Pipeline(argv=self.args) as p:
+      result = (
+          p | apache_beam.io.gcp.bigquery.ReadFromBigQuery(
+              gcs_location="gs://bqio_schema_test",
+              dataset="beam_bigquery_io_test",
+              table="taxi",
+              project="apache-beam-testing",
+              output_type='BEAM_ROW'))
+      assert_that(result, equal_to([]))

Review Comment:
   What is this testing for?



##########
sdks/python/apache_beam/io/gcp/bigquery_schema_tools.py:
##########
@@ -80,7 +82,7 @@ def bq_field_to_type(field, mode):
     return Optional[BIG_QUERY_TO_PYTHON_TYPES[field]]
   elif mode == 'REPEATED':
     return Sequence[BIG_QUERY_TO_PYTHON_TYPES[field]]
-  elif mode is None or mode == '':
+  elif mode is None or mode == '' or mode == 'REQUIRED':

Review Comment:
   If mode isn't provided, we should [default to `NULLABLE` mode](https://developers.google.com/resources/api-libraries/documentation/bigquery/v2/java/latest/com/google/api/services/bigquery/model/TableFieldSchema.html#setMode-java.lang.String-) to stay consistent. WDYT?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [beam] svetakvsundhar commented on pull request #23070: Support TIMESTAMP type in BigQueryIO with BEAM_ROW output type, and in read_gbq

Posted by GitBox <gi...@apache.org>.
svetakvsundhar commented on PR #23070:
URL: https://github.com/apache/beam/pull/23070#issuecomment-1283945908

   Run Python 3.7 PostCommit


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [beam] ahmedabu98 commented on pull request #23070: Support TIMESTAMP type in BigQueryIO with BEAM_ROW output type, and in read_gbq

Posted by GitBox <gi...@apache.org>.
ahmedabu98 commented on PR #23070:
URL: https://github.com/apache/beam/pull/23070#issuecomment-1284411244

   Run Python 3.8 PostCommit


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [beam] svetakvsundhar commented on pull request #23070: Support TIMESTAMP type in BigQueryIO with BEAM_ROW output type, and in read_gbq

Posted by GitBox <gi...@apache.org>.
svetakvsundhar commented on PR #23070:
URL: https://github.com/apache/beam/pull/23070#issuecomment-1264535521

   Run Python 3.7 PreCommit


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [beam] TheNeuralBit commented on a diff in pull request #23070: Support TIMESTAMP type in BigQueryIO with BEAM_ROW output type, and in read_gbq

Posted by GitBox <gi...@apache.org>.
TheNeuralBit commented on code in PR #23070:
URL: https://github.com/apache/beam/pull/23070#discussion_r989326486


##########
sdks/python/apache_beam/io/gcp/bigquery_schema_tools.py:
##########
@@ -38,8 +39,10 @@
     "STRING": str,
     "INTEGER": np.int64,
     "FLOAT64": np.float64,
+    "FLOAT": np.float64,
     "BOOLEAN": bool,
     "BYTES": bytes,
+    "TIMESTAMP": apache_beam.utils.timestamp.Timestamp.from_utc_datetime

Review Comment:
   +1
   
   ```suggestion
       "TIMESTAMP": apache_beam.utils.timestamp.Timestamp
   ```
   
   should do it



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [beam] svetakvsundhar commented on pull request #23070: Support TIMESTAMP type in BigQueryIO with BEAM_ROW output type, and in read_gbq

Posted by GitBox <gi...@apache.org>.
svetakvsundhar commented on PR #23070:
URL: https://github.com/apache/beam/pull/23070#issuecomment-1240113518

   whoops, i didnt realize the reviewer would be auto-assigned.
   
   anyways, this integration test will fail with ` AttributeError: 'datetime.datetime' object has no attribute 'micros' [while running 'ReadFromBigQuery/ParDo(BeamSchemaConversionDoFn)']`
   
   I wanted to ask, should `apache_beam.utils.timestamp` be using `TimestampCoder`? Looks like it's using `RowCoder` now.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [beam] svetakvsundhar commented on pull request #23070: Support TIMESTAMP type in BigQueryIO with BEAM_ROW output type, and in read_gbq

Posted by GitBox <gi...@apache.org>.
svetakvsundhar commented on PR #23070:
URL: https://github.com/apache/beam/pull/23070#issuecomment-1283033264

   Thanks all for the suggestions. Because the ` BQ Timestamp` object was actually a `datetime.datetime`, we needed to modify the conversion ParDo logic to convert into an `apache_beam_utils.timestamp.Timestamp`


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [beam] svetakvsundhar commented on pull request #23070: Support TIMESTAMP type in BigQueryIO with BEAM_ROW output type, and in read_gbq

Posted by GitBox <gi...@apache.org>.
svetakvsundhar commented on PR #23070:
URL: https://github.com/apache/beam/pull/23070#issuecomment-1262909883

   Run Python 3.7 PostCommit


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [beam] TheNeuralBit commented on a diff in pull request #23070: Support TIMESTAMP type in BigQueryIO with BEAM_ROW output type, and in read_gbq

Posted by GitBox <gi...@apache.org>.
TheNeuralBit commented on code in PR #23070:
URL: https://github.com/apache/beam/pull/23070#discussion_r989325298


##########
sdks/python/apache_beam/io/gcp/bigquery_schema_tools_test.py:
##########
@@ -61,6 +61,25 @@ def test_check_conversion_with_empty_schema(self):
         the_table_schema=schema)
     self.assertEqual(usertype.__annotations__, {})
 
+  def test_check_schema_conversions_with_timestamp(self):
+    fields = [
+        bigquery.TableFieldSchema(name='stn', type='STRING', mode="NULLABLE"),
+        bigquery.TableFieldSchema(name='temp', type='FLOAT64', mode="REPEATED"),
+        bigquery.TableFieldSchema(
+            name='times', type='TIMESTAMP', mode="NULLABLE")
+    ]
+    schema = bigquery.TableSchema(fields=fields)
+
+    usertype = bigquery_schema_tools.generate_user_type_from_bq_schema(
+        the_table_schema=schema)
+    self.assertEqual(
+        usertype.__annotations__,
+        {
+            'stn': typing.Optional[str],
+            'temp': typing.Sequence[np.float64],
+            'times': typing.Any

Review Comment:
   It looks like somewhere we're creating a type `Optional[Timestamp.from_utc_datetime]`, which is not a real typehint, as Valentyn indicated it should be `Optional[Timestamp]`



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [beam] tvalentyn commented on pull request #23070: Support TIMESTAMP type in BigQueryIO with BEAM_ROW output type, and in read_gbq

Posted by GitBox <gi...@apache.org>.
tvalentyn commented on PR #23070:
URL: https://github.com/apache/beam/pull/23070#issuecomment-1261652980

   waiting on author


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [beam] svetakvsundhar commented on pull request #23070: Support TIMESTAMP type in BigQueryIO with BEAM_ROW output type, and in read_gbq

Posted by GitBox <gi...@apache.org>.
svetakvsundhar commented on PR #23070:
URL: https://github.com/apache/beam/pull/23070#issuecomment-1265798395

   > Thanks @svetakvsundhar, left a few comments
   > 
   > @tvalentyn does the use of `datetime.datetime` look good to you?
   
   Actually, I think we'd need to further convert this into a an `apache_beam.utils.timestamp.Timestamp` object. fix incoming...


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [beam] tvalentyn commented on a diff in pull request #23070: Support TIMESTAMP type in BigQueryIO with BEAM_ROW output type, and in read_gbq

Posted by GitBox <gi...@apache.org>.
tvalentyn commented on code in PR #23070:
URL: https://github.com/apache/beam/pull/23070#discussion_r965400966


##########
sdks/python/apache_beam/io/gcp/bigquery_read_it_test.py:
##########
@@ -258,6 +258,18 @@ def test_table_schema_retrieve_with_direct_read(self):
               utype(id=4, name='customer2', type='test')
           ]))
 
+  @pytest.mark.it_postcommit
+  def test_table_schema_retrieve_with_timestamp(self):
+    with beam.Pipeline(argv=self.args) as p:
+      result = (
+          p | apache_beam.io.gcp.bigquery.ReadFromBigQuery(
+              gcs_location="gs://bqio_schema_test",

Review Comment:
   nvm, looks like apache-beam-testing



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [beam] tvalentyn merged pull request #23070: Support TIMESTAMP type in BigQueryIO with BEAM_ROW output type, and in read_gbq

Posted by GitBox <gi...@apache.org>.
tvalentyn merged PR #23070:
URL: https://github.com/apache/beam/pull/23070


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [beam] tvalentyn commented on pull request #23070: Support TIMESTAMP type in BigQueryIO with BEAM_ROW output type, and in read_gbq

Posted by GitBox <gi...@apache.org>.
tvalentyn commented on PR #23070:
URL: https://github.com/apache/beam/pull/23070#issuecomment-1284419831

   other errors:
   apache_beam.runners.portability.fn_api_runner.fn_runner_test.FnApiRunnerTestWithGrpcAndMultiWorkers.test_batch_pardo_trigger_flush  --> https://github.com/apache/beam/issues/21104
   apache_beam.runners.portability.portable_runner_test.PortableRunnerTestWithExternalEnv.test_pardo_large_input -> https://github.com/apache/beam/issues/22321
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [beam] svetakvsundhar commented on pull request #23070: Support TIMESTAMP type in BigQueryIO with BEAM_ROW output type, and in read_gbq

Posted by GitBox <gi...@apache.org>.
svetakvsundhar commented on PR #23070:
URL: https://github.com/apache/beam/pull/23070#issuecomment-1283033805

   Run Python 3.7 PostCommit


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [beam] tvalentyn commented on a diff in pull request #23070: Support TIMESTAMP type in BigQueryIO with BEAM_ROW output type, and in read_gbq

Posted by GitBox <gi...@apache.org>.
tvalentyn commented on code in PR #23070:
URL: https://github.com/apache/beam/pull/23070#discussion_r965400670


##########
sdks/python/apache_beam/io/gcp/bigquery_read_it_test.py:
##########
@@ -258,6 +258,18 @@ def test_table_schema_retrieve_with_direct_read(self):
               utype(id=4, name='customer2', type='test')
           ]))
 
+  @pytest.mark.it_postcommit
+  def test_table_schema_retrieve_with_timestamp(self):
+    with beam.Pipeline(argv=self.args) as p:
+      result = (
+          p | apache_beam.io.gcp.bigquery.ReadFromBigQuery(
+              gcs_location="gs://bqio_schema_test",

Review Comment:
   which project does this bucket live in?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [beam] github-actions[bot] commented on pull request #23070: Support TIMESTAMP type in BigQueryIO with BEAM_ROW output type, and in read_gbq

Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on PR #23070:
URL: https://github.com/apache/beam/pull/23070#issuecomment-1259487581

   Assigning new set of reviewers because Pr has gone too long without review. If you would like to opt out of this review, comment `assign to next reviewer`:
   
   R: @y1chi for label python.
   R: @johnjcasey for label io.
   
   Available commands:
   - `stop reviewer notifications` - opt out of the automated review tooling
   - `remind me after tests pass` - tag the comment author after tests pass
   - `waiting on author` - shift the attention set back to the author (any comment or push by the author will return the attention set to the reviewers)


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [beam] tvalentyn commented on a diff in pull request #23070: Support TIMESTAMP type in BigQueryIO with BEAM_ROW output type, and in read_gbq

Posted by GitBox <gi...@apache.org>.
tvalentyn commented on code in PR #23070:
URL: https://github.com/apache/beam/pull/23070#discussion_r988341825


##########
sdks/python/apache_beam/io/gcp/bigquery_schema_tools.py:
##########
@@ -38,8 +39,10 @@
     "STRING": str,
     "INTEGER": np.int64,
     "FLOAT64": np.float64,
+    "FLOAT": np.float64,
     "BOOLEAN": bool,
     "BYTES": bytes,
+    "TIMESTAMP": apache_beam.utils.timestamp.Timestamp.from_utc_datetime

Review Comment:
   Thanks. It seems that we should drop `.from_utc_datetime` as that is a method, which returns the Timestamp.
   Any concerns with that @svetakvsundhar ?
   



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [beam] svetakvsundhar commented on pull request #23070: Support TIMESTAMP type in BigQueryIO with BEAM_ROW output type, and in read_gbq

Posted by GitBox <gi...@apache.org>.
svetakvsundhar commented on PR #23070:
URL: https://github.com/apache/beam/pull/23070#issuecomment-1263864275

   Run Python 3.7 PostCommit


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [beam] svetakvsundhar commented on pull request #23070: Support TIMESTAMP type in BigQueryIO with BEAM_ROW output type, and in read_gbq

Posted by GitBox <gi...@apache.org>.
svetakvsundhar commented on PR #23070:
URL: https://github.com/apache/beam/pull/23070#issuecomment-1263928480

   Run Python 3.7 PostCommit


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [beam] tvalentyn commented on a diff in pull request #23070: Support TIMESTAMP type in BigQueryIO with BEAM_ROW output type, and in read_gbq

Posted by GitBox <gi...@apache.org>.
tvalentyn commented on code in PR #23070:
URL: https://github.com/apache/beam/pull/23070#discussion_r988316445


##########
sdks/python/apache_beam/io/gcp/bigquery_schema_tools.py:
##########
@@ -38,8 +39,10 @@
     "STRING": str,
     "INTEGER": np.int64,
     "FLOAT64": np.float64,
+    "FLOAT": np.float64,
     "BOOLEAN": bool,
     "BYTES": bytes,
+    "TIMESTAMP": apache_beam.utils.timestamp.Timestamp.from_utc_datetime

Review Comment:
   cc: @Abacn @TheNeuralBit who worked on  https://github.com/apache/beam/blob/master/sdks/python/apache_beam/typehints/schemas.py#L36-L37 
   
   Folks, do you know what (if any) aspects we need to take care of for this change to correctly work with schemas & xlang?
   



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [beam] svetakvsundhar commented on pull request #23070: Support TIMESTAMP type in BigQueryIO with BEAM_ROW output type, and in read_gbq

Posted by GitBox <gi...@apache.org>.
svetakvsundhar commented on PR #23070:
URL: https://github.com/apache/beam/pull/23070#issuecomment-1264535142

   Run Python 3.7 PreCommit


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [beam] ahmedabu98 commented on pull request #23070: Support TIMESTAMP type in BigQueryIO with BEAM_ROW output type, and in read_gbq

Posted by GitBox <gi...@apache.org>.
ahmedabu98 commented on PR #23070:
URL: https://github.com/apache/beam/pull/23070#issuecomment-1284421172

   Py37 postcommits have been consistently timing out: `Build timed out (after 240 minutes). Marking the build as aborted.`


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [beam] ahmedabu98 commented on pull request #23070: Support TIMESTAMP type in BigQueryIO with BEAM_ROW output type, and in read_gbq

Posted by GitBox <gi...@apache.org>.
ahmedabu98 commented on PR #23070:
URL: https://github.com/apache/beam/pull/23070#issuecomment-1284421836

   Py38 more reliable as it isn't responsible for as many tests as Py37


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [beam] svetakvsundhar commented on pull request #23070: Support TIMESTAMP type in BigQueryIO with BEAM_ROW output type, and in read_gbq

Posted by GitBox <gi...@apache.org>.
svetakvsundhar commented on PR #23070:
URL: https://github.com/apache/beam/pull/23070#issuecomment-1283169869

   Run Python 3.7 PostCommit


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [beam] tvalentyn commented on pull request #23070: Support TIMESTAMP type in BigQueryIO with BEAM_ROW output type, and in read_gbq

Posted by GitBox <gi...@apache.org>.
tvalentyn commented on PR #23070:
URL: https://github.com/apache/beam/pull/23070#issuecomment-1284416772

   @AnandInguva is looking into py3.10 issue, looks unrelated, py3.8 has bq tests and should provide sufficient coverage, not sure why py3.7 failed, looks like infra error


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [beam] Abacn commented on a diff in pull request #23070: Support TIMESTAMP type in BigQueryIO with BEAM_ROW output type, and in read_gbq

Posted by GitBox <gi...@apache.org>.
Abacn commented on code in PR #23070:
URL: https://github.com/apache/beam/pull/23070#discussion_r988335821


##########
sdks/python/apache_beam/io/gcp/bigquery_schema_tools.py:
##########
@@ -38,8 +39,10 @@
     "STRING": str,
     "INTEGER": np.int64,
     "FLOAT64": np.float64,
+    "FLOAT": np.float64,
     "BOOLEAN": bool,
     "BYTES": bytes,
+    "TIMESTAMP": apache_beam.utils.timestamp.Timestamp.from_utc_datetime

Review Comment:
   I had some discussion with @svetakvsundhar before on the question whether implement TIMESTAMP as Python's native `datetime.datetime` or beam's `apache_beam.utils.timestamp.Timestamp`. Both should work within the Python SDK, but `datetime.datetime` is not portable. So I suggested the latter. In a xlang scenario it should just work.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [beam] tvalentyn commented on pull request #23070: Support TIMESTAMP type in BigQueryIO with BEAM_ROW output type, and in read_gbq

Posted by GitBox <gi...@apache.org>.
tvalentyn commented on PR #23070:
URL: https://github.com/apache/beam/pull/23070#issuecomment-1240050207

   Run Python 3.8 PostCommit


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [beam] tvalentyn commented on pull request #23070: Support TIMESTAMP type in BigQueryIO with BEAM_ROW output type, and in read_gbq

Posted by GitBox <gi...@apache.org>.
tvalentyn commented on PR #23070:
URL: https://github.com/apache/beam/pull/23070#issuecomment-1240050738

   will defer to @ahmedabu98 for the review assuming tests pass.
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [beam] svetakvsundhar commented on pull request #23070: Support TIMESTAMP type in BigQueryIO with BEAM_ROW output type, and in read_gbq

Posted by GitBox <gi...@apache.org>.
svetakvsundhar commented on PR #23070:
URL: https://github.com/apache/beam/pull/23070#issuecomment-1240892658

   cc: @TheNeuralBit 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org