You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@arrow.apache.org by GitBox <gi...@apache.org> on 2021/07/21 17:04:43 UTC
[GitHub] [arrow-rs] alamb opened a new issue #587: Can't cast from string to timestamp:
alamb opened a new issue #587:
URL: https://github.com/apache/arrow-rs/issues/587
**Is your feature request related to a problem or challenge? Please describe what you are trying to do.**
I am trying to work with timestamps -- specifically converting data from csv, etc to timestamps
**Describe the solution you'd like**
I would like the `cast` kernel to support converting `StringArray` to `TimestampNanosecondArray` (and the other date / time / timestamp related types)
I would like the following Rust code to compile and produce a String array with value `"1970-01-01 00:00:00.000000100"`:
Reproducer:
```rust
let array = TimestampNanosecondArray::from(vec![100]);
let array: ArrayRef = Arc::new(array);
// array[0]: "1970-01-01 00:00:00.000000100"
println!("array[0]: {:?}", array_value_to_string(&array, 0).unwrap());
// cast to string array results in an error:
// thread 'main' panicked at 'called `Result::unwrap()` on an `Err` value: CastError("Casting from Timestamp(Nanosecond, None) to Utf8 not supported")', src/main.rs:26:54
let string_array = cast(&array, &DataType::Utf8).unwrap();
```
**Additional context**
The code exists in arrow to convert strings to timestamps (`string_to_timestamp_nanos`) and timestamps to strings, but the code is not connected to the cast kernel
Thus, it is not easy to convert an array of strings to an array of timestamps
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-rs] sum12 commented on issue #587: Can't cast from string to timestamp:
Posted by GitBox <gi...@apache.org>.
sum12 commented on issue #587:
URL: https://github.com/apache/arrow-rs/issues/587#issuecomment-892638678
Hello, Thanks for the awesome description.
I spent sometime reading through the code base to implement this cast.
- one way to would be have a helper `cast_timestamp_to_string` helper which iterates over each of elements and calls `array_value_to_string` and `collect` result and `into` it into `GenericStringArray`
- or one could replicate what `as_datetime` does in the cast.rs but limit it to `Timestamp(_,_)` type
- or one could just cast the array into a `PrimitiveArray` and call `value_as_time` and `push` into a `Vec<String>` and `into` a `GenericStringArray` (This seem to be the simplest approch but I am not sure how to bound it to `ArrowTimestampType` only since `value_as_datetime` expect to satisfy `ArrowNumericType` also)
I tried 2 and 3 but still wasnt sure.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-rs] alamb commented on issue #587: Can't cast from string to timestamp:
Posted by GitBox <gi...@apache.org>.
alamb commented on issue #587:
URL: https://github.com/apache/arrow-rs/issues/587#issuecomment-893079145
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-rs] alamb closed issue #587: Can't cast from timestamp array to string array
Posted by GitBox <gi...@apache.org>.
alamb closed issue #587:
URL: https://github.com/apache/arrow-rs/issues/587
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-rs] sum12 commented on issue #587: Can't cast from string to timestamp:
Posted by GitBox <gi...@apache.org>.
sum12 commented on issue #587:
URL: https://github.com/apache/arrow-rs/issues/587#issuecomment-892638678
Hello, Thanks for the awesome description.
I spent sometime reading through the code base to implement this cast.
- one way to would be have a helper `cast_timestamp_to_string` helper which iterates over each of elements and calls `array_value_to_string` and `collect` result and `into` it into `GenericStringArray`
- or one could replicate what `as_datetime` does in the cast.rs but limit it to `Timestamp(_,_)` type
- or one could just cast the array into a `PrimitiveArray` and call `value_as_time` and `push` into a `Vec<String>` and `into` a `GenericStringArray` (This seem to be the simplest approch but I am not sure how to bound it to `ArrowTimestampType` only since `value_as_datetime` expect to satisfy `ArrowNumericType` also)
I tried 2 and 3 but still wasnt sure.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-rs] alamb commented on issue #587: Can't cast from string to timestamp:
Posted by GitBox <gi...@apache.org>.
alamb commented on issue #587:
URL: https://github.com/apache/arrow-rs/issues/587#issuecomment-893079145
Thanks @sum12 ! think the trait you want is `ArrowTemporalType` https://docs.rs/arrow/5.1.0/arrow/datatypes/trait.ArrowTemporalType.html
So then you could write a function like:
```rust
fn temporal_array_to_string<T>(array: &PrimitiveArray<T>) -> Result<ArrayRef>
where
T: ArrowTemporalType,
{
// ... call array.value_as_datetime here as appropriate
}
```
And then to call it you might do something like:
```rust
let timestamp_array = array.as_any().downcast_ref::<TimestampNanosecondArray>().unwrap();
temporal_array_to_string(timestamp_array)
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-rs] alamb commented on issue #587: Can't cast from timestamp array to string array
Posted by GitBox <gi...@apache.org>.
alamb commented on issue #587:
URL: https://github.com/apache/arrow-rs/issues/587#issuecomment-893079793
I also updated the description to be consistent (the example code and description said the opposite)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org