You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@arrow.apache.org by "alamb (via GitHub)" <gi...@apache.org> on 2023/02/02 17:29:10 UTC
[GitHub] [arrow-datafusion] alamb opened a new issue, #5164: Supporting casts to timestamptz: Eq Utf8 of binary physical should be same"
alamb opened a new issue, #5164:
URL: https://github.com/apache/arrow-datafusion/issues/5164
**Describe the bug**
After https://github.com/apache/arrow-datafusion/pull/5140 and https://github.com/apache/arrow-datafusion/pull/5117 we can cast strings to Timestamp without a timezone. However, casting with timezone is not working yet, as noticed by @waitingkuo
**To Reproduce**
In `datafusion-cli`:
```sql
❯ select '2000-01-01T00:00:00'::timestamp::timestamptz = '2000-01-01T00:00:00';
Internal("The type of Timestamp(Nanosecond, Some(\"+00:00\")) Eq Utf8 of binary physical should be same")
❯ select '2000-01-01T00:00:00'::timestamp = '2000-01-01T00:00:00';
+-----------------------------------------------------------+
| Utf8("2000-01-01T00:00:00") = Utf8("2000-01-01T00:00:00") |
+-----------------------------------------------------------+
| true |
+-----------------------------------------------------------+
1 row in set. Query took 0.004 seconds.
❯ select '2000-01-01T00:00:00'::timestamptz = '2000-01-01T00:00:00';
NotImplemented("Unsupported CAST from Utf8 to Timestamp(Nanosecond, Some(\"+00:00\"))")
````
**Expected behavior**
I expect all three queries to run and return `true`
**Additional context**
Add any other context about the problem here.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-datafusion] alamb closed issue #5164: Supporting casts to timestamptz: Eq Utf8 of binary physical should be same"
Posted by "alamb (via GitHub)" <gi...@apache.org>.
alamb closed issue #5164: Supporting casts to timestamptz: Eq Utf8 of binary physical should be same"
URL: https://github.com/apache/arrow-datafusion/issues/5164
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-datafusion] comphead commented on issue #5164: Supporting casts to timestamptz: Eq Utf8 of binary physical should be same"
Posted by "comphead (via GitHub)" <gi...@apache.org>.
comphead commented on issue #5164:
URL: https://github.com/apache/arrow-datafusion/issues/5164#issuecomment-1488815495
```
DataFusion CLI v21.0.0
❯ select '2000-01-01T00:00:00'::timestamp::timestamptz = '2000-01-01T00:00:00';
+-----------------------------------------------------------+
| Utf8("2000-01-01T00:00:00") = Utf8("2000-01-01T00:00:00") |
+-----------------------------------------------------------+
| true |
+-----------------------------------------------------------+
1 row in set. Query took 0.006 seconds.
❯
```
That finally works after Arrow 36.0.0, I'll create a PR with tests
@alamb
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-datafusion] alamb commented on issue #5164: Supporting casts to timestamptz: Eq Utf8 of binary physical should be same"
Posted by "alamb (via GitHub)" <gi...@apache.org>.
alamb commented on issue #5164:
URL: https://github.com/apache/arrow-datafusion/issues/5164#issuecomment-1419840322
In terms of `????????` I would expect that one could put any `None`, `Some("+07:00")` or `Some("+08:00")` and the resulting timestamps would be adjusted on input to reflect what timezone they appeared in
cc @tustvold in case he has additional thoughts
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-datafusion] comphead commented on issue #5164: Supporting casts to timestamptz: Eq Utf8 of binary physical should be same"
Posted by "comphead (via GitHub)" <gi...@apache.org>.
comphead commented on issue #5164:
URL: https://github.com/apache/arrow-datafusion/issues/5164#issuecomment-1414412353
@alamb I'll take it if noone else will. Need to finish this off
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-datafusion] tustvold commented on issue #5164: Supporting casts to timestamptz: Eq Utf8 of binary physical should be same"
Posted by "tustvold (via GitHub)" <gi...@apache.org>.
tustvold commented on issue #5164:
URL: https://github.com/apache/arrow-datafusion/issues/5164#issuecomment-1419856149
> cc @tustvold in case he has additional thoughts
I agree, we should adjust the timestamp from the timezone in the string, if any, to time since UTC epoch, and store this in the value. The timezone of the output can then be the DataFusion default, unless otherwise specified
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-datafusion] comphead commented on issue #5164: Supporting casts to timestamptz: Eq Utf8 of binary physical should be same"
Posted by "comphead (via GitHub)" <gi...@apache.org>.
comphead commented on issue #5164:
URL: https://github.com/apache/arrow-datafusion/issues/5164#issuecomment-1416887886
@alamb @waitingkuo likely we have some design issues on TimestampTz
please consider such simple case
```
let valid = StringArray::from(vec![
"2023-01-01 04:05:06.789-8",
"2023-01-01 04:05:06.789-7",
]);
let array = Arc::new(valid) as ArrayRef;
let b = cast(&array, &DataType::Timestamp(TimeUnit::Nanosecond,?????????)).unwrap();
```
Ive put **????????** as its not really clear what has to be there, `Some("+07:00")` or `Some("+08:00")`? cast works for array which can contain mixed timezones, however the output datatype is a single value with only 1 supported tz
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-datafusion] comphead commented on issue #5164: Supporting casts to timestamptz: Eq Utf8 of binary physical should be same"
Posted by "comphead (via GitHub)" <gi...@apache.org>.
comphead commented on issue #5164:
URL: https://github.com/apache/arrow-datafusion/issues/5164#issuecomment-1439147307
Waiting arrow 34.0.0
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org