You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@arrow.apache.org by "izveigor (via GitHub)" <gi...@apache.org> on 2023/05/08 19:33:19 UTC
[GitHub] [arrow-rs] izveigor opened a new pull request, #4180: feat: Float16 cast
izveigor opened a new pull request, #4180:
URL: https://github.com/apache/arrow-rs/pull/4180
# Which issue does this PR close?
# Rationale for this change
Support Float16Type in `cast`.
# What changes are included in this PR?
# Are there any user-facing changes?
Yes
<!--
If there are user-facing changes then we may require documentation to be updated before approving the PR.
-->
<!---
If there are any breaking changes to public APIs, please add the `breaking change` label.
-->
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-rs] tustvold commented on a diff in pull request #4180: feat: Float16 cast
Posted by "tustvold (via GitHub)" <gi...@apache.org>.
tustvold commented on code in PR #4180:
URL: https://github.com/apache/arrow-rs/pull/4180#discussion_r1188781408
##########
arrow-cast/src/cast.rs:
##########
@@ -6399,6 +6644,110 @@ mod tests {
);
}
+ #[test]
+ fn test_cast_from_f16() {
+ let f16_values: Vec<f16> = vec![
+ f16::from_f32(i32::MIN as f32),
+ f16::from_f32(i32::MIN as f32),
+ f16::from_f32(i16::MIN as f32),
+ f16::from_f32(i8::MIN as f32),
+ f16::from_f32(0_f32),
+ f16::from_f32(u8::MAX as f32),
+ f16::from_f32(u16::MAX as f32),
+ f16::from_f32(u32::MAX as f32),
+ f16::from_f32(u32::MAX as f32),
+ ];
+ let f16_array: ArrayRef = Arc::new(Float16Array::from(f16_values));
+
+ let f64_expected = vec![
+ "-inf", "-inf", "-32768.0", "-128.0", "0.0", "255.0", "inf", "inf", "inf",
Review Comment:
-inf seems off here, unless the original value was f16::NEG_INFINITY ?
##########
arrow-cast/src/cast.rs:
##########
@@ -6399,6 +6644,110 @@ mod tests {
);
}
+ #[test]
+ fn test_cast_from_f16() {
Review Comment:
I think this test would be clearer if it used f16::Inf, instead of relying on i32::MIN overflowing (which is what I think is going on)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-rs] izveigor commented on a diff in pull request #4180: feat: Float16 cast
Posted by "izveigor (via GitHub)" <gi...@apache.org>.
izveigor commented on code in PR #4180:
URL: https://github.com/apache/arrow-rs/pull/4180#discussion_r1210783826
##########
arrow-cast/src/cast.rs:
##########
@@ -4821,6 +5011,29 @@ mod tests {
let array: &Decimal256Array = array.as_primitive();
assert!(array.is_null(4));
+ // test f16 to decimal type
+ let array = Float16Array::from(vec![
+ Some(f16::from_f32(1.1)),
+ Some(f16::from_f32(2.2)),
+ Some(f16::from_f32(4.4)),
+ None,
+ Some(f16::from_f32(1.125_4)), // round down
+ Some(f16::from_f32(1.165_4)), // round up
+ ]);
+ generate_cast_test_case!(
+ &array,
+ Decimal256Array,
+ &decimal_type,
+ vec![
+ Some(i256::from_i128(1099609_i128)), // round down
Review Comment:
I can't explain why it is not a round number.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-rs] izveigor commented on a diff in pull request #4180: feat: Float16 cast
Posted by "izveigor (via GitHub)" <gi...@apache.org>.
izveigor commented on code in PR #4180:
URL: https://github.com/apache/arrow-rs/pull/4180#discussion_r1210783826
##########
arrow-cast/src/cast.rs:
##########
@@ -4821,6 +5011,29 @@ mod tests {
let array: &Decimal256Array = array.as_primitive();
assert!(array.is_null(4));
+ // test f16 to decimal type
+ let array = Float16Array::from(vec![
+ Some(f16::from_f32(1.1)),
+ Some(f16::from_f32(2.2)),
+ Some(f16::from_f32(4.4)),
+ None,
+ Some(f16::from_f32(1.125_4)), // round down
+ Some(f16::from_f32(1.165_4)), // round up
+ ]);
+ generate_cast_test_case!(
+ &array,
+ Decimal256Array,
+ &decimal_type,
+ vec![
+ Some(i256::from_i128(1099609_i128)), // round down
Review Comment:
I can't explain why it is not a round number.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-rs] izveigor commented on pull request #4180: feat: Float16 cast
Posted by "izveigor (via GitHub)" <gi...@apache.org>.
izveigor commented on PR #4180:
URL: https://github.com/apache/arrow-rs/pull/4180#issuecomment-1593479372
So, what about this PR @tustvold.
If you're worried about the "infinity" and "decimal" problem, I can tell that they are part of the algorithm and are considered correct.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-rs] izveigor commented on a diff in pull request #4180: feat: Float16 cast
Posted by "izveigor (via GitHub)" <gi...@apache.org>.
izveigor commented on code in PR #4180:
URL: https://github.com/apache/arrow-rs/pull/4180#discussion_r1210783118
##########
arrow-cast/src/cast.rs:
##########
@@ -6596,6 +6836,85 @@ mod tests {
);
}
+ #[test]
+ fn test_cast_from_f16() {
+ let f16_values: Vec<f16> = vec![
+ f16::NEG_INFINITY,
+ f16::from_f32(i16::MIN as f32),
+ f16::from_f32(i8::MIN as f32),
+ f16::from_f32(0_f32),
+ f16::from_f32(u8::MAX as f32),
+ f16::INFINITY,
+ ];
+ let f16_array: ArrayRef = Arc::new(Float16Array::from(f16_values));
+
+ let f64_expected = vec!["-inf", "-32768.0", "-128.0", "0.0", "255.0", "inf"];
Review Comment:
I didn't find the way in the package `half` how to replace `inf` with `null. Is this a serious problem?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-rs] izveigor commented on pull request #4180: feat: Float16 cast
Posted by "izveigor (via GitHub)" <gi...@apache.org>.
izveigor commented on PR #4180:
URL: https://github.com/apache/arrow-rs/pull/4180#issuecomment-1569041053
@tustvold Sorry for the long delay. I left a few questions on this PR. Can you pay attention to them if you have time.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-rs] izveigor closed pull request #4180: feat: Float16 cast
Posted by "izveigor (via GitHub)" <gi...@apache.org>.
izveigor closed pull request #4180: feat: Float16 cast
URL: https://github.com/apache/arrow-rs/pull/4180
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org