You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@arrow.apache.org by "gruuya (via GitHub)" <gi...@apache.org> on 2023/02/23 11:43:23 UTC
[GitHub] [arrow-rs] gruuya opened a new pull request, #3752: Enable casting of string to timestamp with microsecond resolution
gruuya opened a new pull request, #3752:
URL: https://github.com/apache/arrow-rs/pull/3752
# Which issue does this PR close?
Closes #3751.
# Rationale for this change
Enable casting strings to timestamps with microsecond resolution.
# What changes are included in this PR?
Implement conversion of `Utf8`/`LargeUtf8` timestamp strings to `Timestamp(TimeUnit::Microsecond, _)` by making `cast_string_to_timestamp` generic.
# Are there any user-facing changes?
No more casting errors for the said type pairs.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-rs] tustvold commented on pull request #3752: Enable casting of string to timestamp with microsecond resolution
Posted by "tustvold (via GitHub)" <gi...@apache.org>.
tustvold commented on PR #3752:
URL: https://github.com/apache/arrow-rs/pull/3752#issuecomment-1441862221
I have confirmed that the integration test in array_cast already contains examples of other timestamp array types
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-rs] tustvold commented on pull request #3752: Enable casting of string to timestamp with microsecond resolution
Posted by "tustvold (via GitHub)" <gi...@apache.org>.
tustvold commented on PR #3752:
URL: https://github.com/apache/arrow-rs/pull/3752#issuecomment-1442034331
Thank you
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-rs] tustvold commented on a diff in pull request #3752: Enable casting of string to timestamp with microsecond resolution
Posted by "tustvold (via GitHub)" <gi...@apache.org>.
tustvold commented on code in PR #3752:
URL: https://github.com/apache/arrow-rs/pull/3752#discussion_r1115745193
##########
arrow-cast/src/cast.rs:
##########
@@ -2562,26 +2573,35 @@ fn cast_string_to_timestamp_ns<Offset: OffsetSizeTrait>(
.downcast_ref::<GenericStringArray<Offset>>()
.unwrap();
+ let scale_factor = match TimestampType::get_time_unit() {
+ TimeUnit::Microsecond => 1_000,
+ TimeUnit::Nanosecond => 1,
+ _ => return Err(ArrowError::CastError("String to timestamp casting supports only microsecond or nanosecond resolution".to_string()))
Review Comment:
Is there a particular reason not to just support the full set whilst here?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-rs] tustvold merged pull request #3752: Enable casting of string to timestamp with microsecond resolution
Posted by "tustvold (via GitHub)" <gi...@apache.org>.
tustvold merged PR #3752:
URL: https://github.com/apache/arrow-rs/pull/3752
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-rs] gruuya commented on a diff in pull request #3752: Enable casting of string to timestamp with microsecond resolution
Posted by "gruuya (via GitHub)" <gi...@apache.org>.
gruuya commented on code in PR #3752:
URL: https://github.com/apache/arrow-rs/pull/3752#discussion_r1115765616
##########
arrow-cast/src/cast.rs:
##########
@@ -2562,26 +2573,35 @@ fn cast_string_to_timestamp_ns<Offset: OffsetSizeTrait>(
.downcast_ref::<GenericStringArray<Offset>>()
.unwrap();
+ let scale_factor = match TimestampType::get_time_unit() {
+ TimeUnit::Microsecond => 1_000,
+ TimeUnit::Nanosecond => 1,
+ _ => return Err(ArrowError::CastError("String to timestamp casting supports only microsecond or nanosecond resolution".to_string()))
Review Comment:
Not to my knowledge; I can add them too if no one objects.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-rs] ursabot commented on pull request #3752: Enable casting of string to timestamp with microsecond resolution
Posted by "ursabot (via GitHub)" <gi...@apache.org>.
ursabot commented on PR #3752:
URL: https://github.com/apache/arrow-rs/pull/3752#issuecomment-1442050612
Benchmark runs are scheduled for baseline = 9699e1df7c7e0b83c8ec8be6678ee17d77a17f47 and contender = 57f79c03a8dee9d8bf8601bf555aa271746913fe. 57f79c03a8dee9d8bf8601bf555aa271746913fe is a master commit associated with this PR. Results will be available as each benchmark for each run completes.
Conbench compare runs links:
[Skipped :warning: Benchmarking of arrow-rs-commits is not supported on ec2-t3-xlarge-us-east-2] [ec2-t3-xlarge-us-east-2](https://conbench.ursa.dev/compare/runs/8af6ee307db84287a065f768918521a5...70b7842350394d53b5cb33533ea9ef2b/)
[Skipped :warning: Benchmarking of arrow-rs-commits is not supported on test-mac-arm] [test-mac-arm](https://conbench.ursa.dev/compare/runs/cd87a4834cb8418b829ff5cf42ad59e6...53fb0801d46e4e8d8a6b91788fee736f/)
[Skipped :warning: Benchmarking of arrow-rs-commits is not supported on ursa-i9-9960x] [ursa-i9-9960x](https://conbench.ursa.dev/compare/runs/59cdbed71ae54eb4925afb92f529035c...977ca52a509d459b929470c2a427b76f/)
[Skipped :warning: Benchmarking of arrow-rs-commits is not supported on ursa-thinkcentre-m75q] [ursa-thinkcentre-m75q](https://conbench.ursa.dev/compare/runs/4a14eea932af4eee99b94eaebb5a6d34...96dcbcdccc7f4995a565fa3c921f6576/)
Buildkite builds:
Supported benchmarks:
ec2-t3-xlarge-us-east-2: Supported benchmark langs: Python, R. Runs only benchmarks with cloud = True
test-mac-arm: Supported benchmark langs: C++, Python, R
ursa-i9-9960x: Supported benchmark langs: Python, R, JavaScript
ursa-thinkcentre-m75q: Supported benchmark langs: C++, Java
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org