You are viewing a plain text version of this content. The canonical link for it is here.

Posted to github@beam.apache.org by GitBox <gi...@apache.org> on 2022/06/03 22:57:41 UTC

[GitHub] [beam] kennknowles opened a new issue, #19205: Records including datetime to be saved as DATETIME or TIMESTAMP in BigQuery

kennknowles opened a new issue, #19205:
URL: https://github.com/apache/beam/issues/19205

When trying to write some row in BigQuery that include a python datetime object, the marshaling used to save a row in BigQuery is impossible.
```

File "/home/pascal/Wks/GitHub/PEAT-AI/Albatros/venv/local/lib/python2.7/site-packages/apache_beam/internal/gcp/json_value.py",
line 124, in to_json_value
raise TypeError('Cannot convert %s to a JSON value.' % repr(obj))
TypeError:
Cannot convert datetime.datetime(2018, 9, 25, 18, 57, 18, 108579) to a JSON value. [while running 'save/WriteToBigQuery']

```

However, this is something perfectly feasible, as `google-cloud-python` supports it since this issue has been solved: [https://github.com/GoogleCloudPlatform/google-cloud-python/issues/2957](https://github.com/GoogleCloudPlatform/google-cloud-python/issues/2957)

thanks to this pull request: [https://github.com/GoogleCloudPlatform/google-cloud-python/pull/3426/files](https://github.com/GoogleCloudPlatform/google-cloud-python/pull/3426/files)

As similar approach could be taken for the `json_value.py` helper.

Is there any workaround that can be applied to solve this issue?

Imported from Jira [BEAM-5510](https://issues.apache.org/jira/browse/BEAM-5510). Original Jira may contain additional context.
Reported by: GeoloeG_IsT.

--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [I] Records including datetime to be saved as DATETIME or TIMESTAMP in BigQuery [beam]

Posted by "lazarillo (via GitHub)" <gi...@apache.org>.

lazarillo commented on issue #19205:
URL: https://github.com/apache/beam/issues/19205#issuecomment-1876831689

   It seemed to me that this was more of a _Google_ issue than an Apache Beam issue.
   
   I created an issue here that summarizes the problem, and my humble opinion as to a means to resolve it that is hopefully flexible.
   
   https://issuetracker.google.com/issues/318478318
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [I] Records including datetime to be saved as DATETIME or TIMESTAMP in BigQuery [beam]

Posted by "kennknowles (via GitHub)" <gi...@apache.org>.

kennknowles commented on issue #19205:
URL: https://github.com/apache/beam/issues/19205#issuecomment-1877401194

   Sounds good to me! It is very old and a fresh report would be probably needed anyhow.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [I] Records including datetime to be saved as DATETIME or TIMESTAMP in BigQuery [beam]

Posted by "lazarillo (via GitHub)" <gi...@apache.org>.

lazarillo commented on issue #19205:
URL: https://github.com/apache/beam/issues/19205#issuecomment-1876832873

   If some core beam devs agree, then maybe we can close this issue. :smile: 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [I] Records including datetime to be saved as DATETIME or TIMESTAMP in BigQuery [beam]

Posted by "kennknowles (via GitHub)" <gi...@apache.org>.

kennknowles closed issue #19205: Records including datetime to be saved as DATETIME or TIMESTAMP in BigQuery
URL: https://github.com/apache/beam/issues/19205


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org