You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@arrow.apache.org by "alamb (via GitHub)" <gi...@apache.org> on 2023/12/07 11:57:27 UTC
[I] Invalid trailing content after `Z` in timezone is ignored [arrow-rs]
alamb opened a new issue, #5182:
URL: https://github.com/apache/arrow-rs/issues/5182
**Describe the bug**
Timstamps like `2023-12-05T21:58:10.45ZZTOP` which are invalid (they have trailing content after the `Z`) should error when parsing (casting from String to Timestamp) but instead they ignore the trailing content
**To Reproduce**
This test case should pass in arrow-cast/src/cast.rs
```rust
#[test]
fn test_cast_string_to_timestamp_invalid_tz() {
// content after Z should be ignored
let bad_timestamp = "2023-12-05T21:58:10.45ZZTOP";
let array = StringArray::from(vec![Some(bad_timestamp)]);
let data_types = [
DataType::Timestamp(TimeUnit::Second, None),
DataType::Timestamp(TimeUnit::Millisecond, None),
DataType::Timestamp(TimeUnit::Microsecond, None),
DataType::Timestamp(TimeUnit::Nanosecond, None),
];
for dt in data_types {
assert_eq!(
cast(&array, &dt).unwrap_err().to_string(),
"Invalid timezone: ZZTOP"
);
}
}
```
**Expected behavior**
Test case should pass, instead it fails as the timestamp is correctly parsed
**Additional context**
Found upstream in https://github.com/apache/arrow-datafusion/issues/8452
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Invalid trailing content after `Z` in timezone is ignored [arrow-rs]
Posted by "tustvold (via GitHub)" <gi...@apache.org>.
tustvold closed issue #5182: Invalid trailing content after `Z` in timezone is ignored
URL: https://github.com/apache/arrow-rs/issues/5182
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Invalid trailing content after `Z` in timezone is ignored [arrow-rs]
Posted by "razeghi71 (via GitHub)" <gi...@apache.org>.
razeghi71 commented on issue #5182:
URL: https://github.com/apache/arrow-rs/issues/5182#issuecomment-1847217606
We need this to get released the fix the issue linked to it here: https://github.com/apache/arrow-datafusion/issues/8452. Is there a documentation on how the release process work in order to use it in `datafusion`?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Invalid trailing content after `Z` in timezone is ignored [arrow-rs]
Posted by "razeghi71 (via GitHub)" <gi...@apache.org>.
razeghi71 commented on issue #5182:
URL: https://github.com/apache/arrow-rs/issues/5182#issuecomment-1845618937
Can I give this a shot?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Invalid trailing content after `Z` in timezone is ignored [arrow-rs]
Posted by "alamb (via GitHub)" <gi...@apache.org>.
alamb commented on issue #5182:
URL: https://github.com/apache/arrow-rs/issues/5182#issuecomment-1846196659
> @alamb the previous behavior for `cast` was that when it returned `null` for the elements that it was unable to parse. this way if we pass it a good timestamp and a bad one, it will returned the parsed timestamp for good one and `null` for the bad one.
>
> In the test case you provided, you are expecting an error result. is that the behavior we want to change to? if yes what would happen to the case when we pass a good and a bad element to `cast`? again return error result?
Sorry, you are correct. I expect an error when [`CastOptions::safe`](https://docs.rs/arrow/latest/arrow/compute/struct.CastOptions.html#) is false. My test case should have called `cast_with_options`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Invalid trailing content after `Z` in timezone is ignored [arrow-rs]
Posted by "tustvold (via GitHub)" <gi...@apache.org>.
tustvold commented on issue #5182:
URL: https://github.com/apache/arrow-rs/issues/5182#issuecomment-1847241865
I anticipate cutting the next release in the coming weeks
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Invalid trailing content after `Z` in timezone is ignored [arrow-rs]
Posted by "razeghi71 (via GitHub)" <gi...@apache.org>.
razeghi71 commented on issue #5182:
URL: https://github.com/apache/arrow-rs/issues/5182#issuecomment-1846138689
@alamb the previous behavior for `cast` was that when it returned `null` for the elements that it was unable to parse. this way if we pass it a good timestamp and a bad one, it will returned the parsed timestamp for good one and `null` for the bad one.
In the test case you provided, you are expecting an error result. is that the behavior we want to change to? if yes what would happen to the case when we pass a good and a bad element to `cast`? again return error result?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Invalid trailing content after `Z` in timezone is ignored [arrow-rs]
Posted by "alamb (via GitHub)" <gi...@apache.org>.
alamb commented on issue #5182:
URL: https://github.com/apache/arrow-rs/issues/5182#issuecomment-1845215730
I think this will be a nice first issue as it has a test already written and should be a simple fix
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org