You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@arrow.apache.org by GitBox <gi...@apache.org> on 2020/12/09 18:38:49 UTC
[GitHub] [arrow] Dandandan opened a new pull request #8882: ARROW-10864: [Rust] Use standard ordering for floats [WIP]
Dandandan opened a new pull request #8882:
URL: https://github.com/apache/arrow/pull/8882
This implements ordering using IEEE 754 ordering as mentioned in this discussion (only for sort now)
https://github.com/apache/arrow/pull/8685
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] codecov-io edited a comment on pull request #8882: ARROW-10864: [Rust] Use standard ordering for floats
Posted by GitBox <gi...@apache.org>.
codecov-io edited a comment on pull request #8882:
URL: https://github.com/apache/arrow/pull/8882#issuecomment-741980279
# [Codecov](https://codecov.io/gh/apache/arrow/pull/8882?src=pr&el=h1) Report
> Merging [#8882](https://codecov.io/gh/apache/arrow/pull/8882?src=pr&el=desc) (40c9754) into [master](https://codecov.io/gh/apache/arrow/commit/399245678c1be5d7f45fb350cb09baa36a35797b?el=desc) (3992456) will **increase** coverage by `0.00%`.
> The diff coverage is `68.62%`.
[](https://codecov.io/gh/apache/arrow/pull/8882?src=pr&el=tree)
```diff
@@ Coverage Diff @@
## master #8882 +/- ##
=======================================
Coverage 82.56% 82.56%
=======================================
Files 203 203
Lines 50042 50032 -10
=======================================
- Hits 41315 41308 -7
+ Misses 8727 8724 -3
```
| [Impacted Files](https://codecov.io/gh/apache/arrow/pull/8882?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [rust/arrow/src/compute/kernels/sort.rs](https://codecov.io/gh/apache/arrow/pull/8882/diff?src=pr&el=tree#diff-cnVzdC9hcnJvdy9zcmMvY29tcHV0ZS9rZXJuZWxzL3NvcnQucnM=) | `93.58% <68.62%> (+0.15%)` | :arrow_up: |
| [rust/parquet/src/encodings/encoding.rs](https://codecov.io/gh/apache/arrow/pull/8882/diff?src=pr&el=tree#diff-cnVzdC9wYXJxdWV0L3NyYy9lbmNvZGluZ3MvZW5jb2RpbmcucnM=) | `95.43% <0.00%> (+0.19%)` | :arrow_up: |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/arrow/pull/8882?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/arrow/pull/8882?src=pr&el=footer). Last update [3992456...c01932b](https://codecov.io/gh/apache/arrow/pull/8882?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] codecov-io edited a comment on pull request #8882: ARROW-10864: [Rust] Use standard ordering for floats
Posted by GitBox <gi...@apache.org>.
codecov-io edited a comment on pull request #8882:
URL: https://github.com/apache/arrow/pull/8882#issuecomment-741980279
# [Codecov](https://codecov.io/gh/apache/arrow/pull/8882?src=pr&el=h1) Report
> Merging [#8882](https://codecov.io/gh/apache/arrow/pull/8882?src=pr&el=desc) (4d9c35c) into [master](https://codecov.io/gh/apache/arrow/commit/399245678c1be5d7f45fb350cb09baa36a35797b?el=desc) (3992456) will **increase** coverage by `0.00%`.
> The diff coverage is `68.62%`.
[](https://codecov.io/gh/apache/arrow/pull/8882?src=pr&el=tree)
```diff
@@ Coverage Diff @@
## master #8882 +/- ##
=======================================
Coverage 82.56% 82.56%
=======================================
Files 203 203
Lines 50042 50032 -10
=======================================
- Hits 41315 41308 -7
+ Misses 8727 8724 -3
```
| [Impacted Files](https://codecov.io/gh/apache/arrow/pull/8882?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [rust/arrow/src/compute/kernels/sort.rs](https://codecov.io/gh/apache/arrow/pull/8882/diff?src=pr&el=tree#diff-cnVzdC9hcnJvdy9zcmMvY29tcHV0ZS9rZXJuZWxzL3NvcnQucnM=) | `93.58% <68.62%> (+0.15%)` | :arrow_up: |
| [rust/parquet/src/encodings/encoding.rs](https://codecov.io/gh/apache/arrow/pull/8882/diff?src=pr&el=tree#diff-cnVzdC9wYXJxdWV0L3NyYy9lbmNvZGluZ3MvZW5jb2RpbmcucnM=) | `95.43% <0.00%> (+0.19%)` | :arrow_up: |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/arrow/pull/8882?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/arrow/pull/8882?src=pr&el=footer). Last update [3992456...d04f893](https://codecov.io/gh/apache/arrow/pull/8882?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] alamb commented on a change in pull request #8882: ARROW-10864: [Rust] Use standard ordering for floats
Posted by GitBox <gi...@apache.org>.
alamb commented on a change in pull request #8882:
URL: https://github.com/apache/arrow/pull/8882#discussion_r552202489
##########
File path: rust/arrow/src/compute/kernels/sort.rs
##########
@@ -41,40 +41,33 @@ pub fn sort(values: &ArrayRef, options: Option<SortOptions>) -> Result<ArrayRef>
take(values.as_ref(), &indices, None)
}
-// partition indices into non-NaN and NaN
-fn partition_nan<T: ArrowPrimitiveType>(
- array: &ArrayRef,
- v: Vec<u32>,
-) -> (Vec<u32>, Vec<u32>) {
- // partition by nan for float types
- if matches!(T::DATA_TYPE, DataType::Float32) {
- // T::Native has no `is_nan` and thus we need to downcast
- let array = array
- .as_any()
- .downcast_ref::<Float32Array>()
- .expect("Unable to downcast array");
- let has_nan = v.iter().any(|index| array.value(*index as usize).is_nan());
- if has_nan {
- v.into_iter()
- .partition(|index| !array.value(*index as usize).is_nan())
- } else {
- (v, vec![])
- }
- } else if matches!(T::DATA_TYPE, DataType::Float64) {
- let array = array
- .as_any()
- .downcast_ref::<Float64Array>()
- .expect("Unable to downcast array");
- let has_nan = v.iter().any(|index| array.value(*index as usize).is_nan());
- if has_nan {
- v.into_iter()
- .partition(|index| !array.value(*index as usize).is_nan())
- } else {
- (v, vec![])
- }
- } else {
- unreachable!("Partition by nan is only applicable to float types")
- }
+// sorts f32 in IEEE 754 total ordering
Review comment:
It might be good here to provide a pointer to the original source implementation (e.g. https://doc.rust-lang.org/std/primitive.f64.html#method.total_cmp) and a TODO to change to use that API when is stabilized
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] Dandandan commented on pull request #8882: ARROW-10864: [Rust] Use standard ordering for floats
Posted by GitBox <gi...@apache.org>.
Dandandan commented on pull request #8882:
URL: https://github.com/apache/arrow/pull/8882#issuecomment-753613551
@nevi-me rebased, but looks like needs-rebase label isn't removed
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] nevi-me commented on pull request #8882: ARROW-10864: [Rust] Use standard ordering for floats [WIP]
Posted by GitBox <gi...@apache.org>.
nevi-me commented on pull request #8882:
URL: https://github.com/apache/arrow/pull/8882#issuecomment-753443369
@Dandandan what more work do you still need to do on this PR?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] alamb closed pull request #8882: ARROW-10864: [Rust] Use standard ordering for floats
Posted by GitBox <gi...@apache.org>.
alamb closed pull request #8882:
URL: https://github.com/apache/arrow/pull/8882
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] codecov-io edited a comment on pull request #8882: ARROW-10864: [Rust] Use standard ordering for floats
Posted by GitBox <gi...@apache.org>.
codecov-io edited a comment on pull request #8882:
URL: https://github.com/apache/arrow/pull/8882#issuecomment-741980279
# [Codecov](https://codecov.io/gh/apache/arrow/pull/8882?src=pr&el=h1) Report
> Merging [#8882](https://codecov.io/gh/apache/arrow/pull/8882?src=pr&el=desc) (eac99bd) into [master](https://codecov.io/gh/apache/arrow/commit/dfef236f7587e4168ac1e07bd09e42d9373beb70?el=desc) (dfef236) will **decrease** coverage by `0.00%`.
> The diff coverage is `68.62%`.
[](https://codecov.io/gh/apache/arrow/pull/8882?src=pr&el=tree)
```diff
@@ Coverage Diff @@
## master #8882 +/- ##
==========================================
- Coverage 82.60% 82.60% -0.01%
==========================================
Files 204 204
Lines 50189 50177 -12
==========================================
- Hits 41459 41448 -11
+ Misses 8730 8729 -1
```
| [Impacted Files](https://codecov.io/gh/apache/arrow/pull/8882?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [rust/arrow/src/compute/kernels/sort.rs](https://codecov.io/gh/apache/arrow/pull/8882/diff?src=pr&el=tree#diff-cnVzdC9hcnJvdy9zcmMvY29tcHV0ZS9rZXJuZWxzL3NvcnQucnM=) | `93.56% <68.62%> (+0.14%)` | :arrow_up: |
| [rust/parquet/src/encodings/encoding.rs](https://codecov.io/gh/apache/arrow/pull/8882/diff?src=pr&el=tree#diff-cnVzdC9wYXJxdWV0L3NyYy9lbmNvZGluZ3MvZW5jb2RpbmcucnM=) | `95.24% <0.00%> (-0.20%)` | :arrow_down: |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/arrow/pull/8882?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/arrow/pull/8882?src=pr&el=footer). Last update [dfef236...eac99bd](https://codecov.io/gh/apache/arrow/pull/8882?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] codecov-io edited a comment on pull request #8882: ARROW-10864: [Rust] Use standard ordering for floats
Posted by GitBox <gi...@apache.org>.
codecov-io edited a comment on pull request #8882:
URL: https://github.com/apache/arrow/pull/8882#issuecomment-741980279
# [Codecov](https://codecov.io/gh/apache/arrow/pull/8882?src=pr&el=h1) Report
> Merging [#8882](https://codecov.io/gh/apache/arrow/pull/8882?src=pr&el=desc) (d04f893) into [master](https://codecov.io/gh/apache/arrow/commit/399245678c1be5d7f45fb350cb09baa36a35797b?el=desc) (3992456) will **decrease** coverage by `0.00%`.
> The diff coverage is `68.62%`.
[](https://codecov.io/gh/apache/arrow/pull/8882?src=pr&el=tree)
```diff
@@ Coverage Diff @@
## master #8882 +/- ##
==========================================
- Coverage 82.56% 82.56% -0.01%
==========================================
Files 203 203
Lines 50042 50030 -12
==========================================
- Hits 41315 41305 -10
+ Misses 8727 8725 -2
```
| [Impacted Files](https://codecov.io/gh/apache/arrow/pull/8882?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [rust/arrow/src/compute/kernels/sort.rs](https://codecov.io/gh/apache/arrow/pull/8882/diff?src=pr&el=tree#diff-cnVzdC9hcnJvdy9zcmMvY29tcHV0ZS9rZXJuZWxzL3NvcnQucnM=) | `93.56% <68.62%> (+0.14%)` | :arrow_up: |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/arrow/pull/8882?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/arrow/pull/8882?src=pr&el=footer). Last update [3992456...d04f893](https://codecov.io/gh/apache/arrow/pull/8882?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] github-actions[bot] commented on pull request #8882: ARROW-10864: [Rust] Use standard ordering for floats [WIP]
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on pull request #8882:
URL: https://github.com/apache/arrow/pull/8882#issuecomment-741979653
https://issues.apache.org/jira/browse/ARROW-10864
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] codecov-io commented on pull request #8882: ARROW-10864: [Rust] Use standard ordering for floats [WIP]
Posted by GitBox <gi...@apache.org>.
codecov-io commented on pull request #8882:
URL: https://github.com/apache/arrow/pull/8882#issuecomment-741980279
# [Codecov](https://codecov.io/gh/apache/arrow/pull/8882?src=pr&el=h1) Report
> Merging [#8882](https://codecov.io/gh/apache/arrow/pull/8882?src=pr&el=desc) (4ebd8c3) into [master](https://codecov.io/gh/apache/arrow/commit/db94f24f15ad783efe676fef8883b7cab9a82f88?el=desc) (db94f24) will **decrease** coverage by `22.37%`.
> The diff coverage is `0.00%`.
[](https://codecov.io/gh/apache/arrow/pull/8882?src=pr&el=tree)
```diff
@@ Coverage Diff @@
## master #8882 +/- ##
===========================================
- Coverage 77.03% 54.66% -22.38%
===========================================
Files 173 177 +4
Lines 40090 31254 -8836
===========================================
- Hits 30885 17084 -13801
- Misses 9205 14170 +4965
```
| [Impacted Files](https://codecov.io/gh/apache/arrow/pull/8882?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [rust/arrow/src/compute/kernels/sort.rs](https://codecov.io/gh/apache/arrow/pull/8882/diff?src=pr&el=tree#diff-cnVzdC9hcnJvdy9zcmMvY29tcHV0ZS9rZXJuZWxzL3NvcnQucnM=) | `0.00% <0.00%> (-95.43%)` | :arrow_down: |
| [rust/arrow/src/array/cast.rs](https://codecov.io/gh/apache/arrow/pull/8882/diff?src=pr&el=tree#diff-cnVzdC9hcnJvdy9zcmMvYXJyYXkvY2FzdC5ycw==) | `0.00% <0.00%> (-100.00%)` | :arrow_down: |
| [rust/arrow/src/array/equal/decimal.rs](https://codecov.io/gh/apache/arrow/pull/8882/diff?src=pr&el=tree#diff-cnVzdC9hcnJvdy9zcmMvYXJyYXkvZXF1YWwvZGVjaW1hbC5ycw==) | `0.00% <0.00%> (-100.00%)` | :arrow_down: |
| [rust/arrow/src/array/transform/utils.rs](https://codecov.io/gh/apache/arrow/pull/8882/diff?src=pr&el=tree#diff-cnVzdC9hcnJvdy9zcmMvYXJyYXkvdHJhbnNmb3JtL3V0aWxzLnJz) | `0.00% <0.00%> (-100.00%)` | :arrow_down: |
| [rust/arrow/src/compute/kernels/limit.rs](https://codecov.io/gh/apache/arrow/pull/8882/diff?src=pr&el=tree#diff-cnVzdC9hcnJvdy9zcmMvY29tcHV0ZS9rZXJuZWxzL2xpbWl0LnJz) | `0.00% <0.00%> (-100.00%)` | :arrow_down: |
| [rust/arrow/src/array/equal/fixed\_list.rs](https://codecov.io/gh/apache/arrow/pull/8882/diff?src=pr&el=tree#diff-cnVzdC9hcnJvdy9zcmMvYXJyYXkvZXF1YWwvZml4ZWRfbGlzdC5ycw==) | `0.00% <0.00%> (-100.00%)` | :arrow_down: |
| [rust/arrow/src/compute/kernels/length.rs](https://codecov.io/gh/apache/arrow/pull/8882/diff?src=pr&el=tree#diff-cnVzdC9hcnJvdy9zcmMvY29tcHV0ZS9rZXJuZWxzL2xlbmd0aC5ycw==) | `0.00% <0.00%> (-100.00%)` | :arrow_down: |
| [rust/arrow/src/array/transform/primitive.rs](https://codecov.io/gh/apache/arrow/pull/8882/diff?src=pr&el=tree#diff-cnVzdC9hcnJvdy9zcmMvYXJyYXkvdHJhbnNmb3JtL3ByaW1pdGl2ZS5ycw==) | `0.00% <0.00%> (-100.00%)` | :arrow_down: |
| [rust/arrow/src/array/transform/variable\_size.rs](https://codecov.io/gh/apache/arrow/pull/8882/diff?src=pr&el=tree#diff-cnVzdC9hcnJvdy9zcmMvYXJyYXkvdHJhbnNmb3JtL3ZhcmlhYmxlX3NpemUucnM=) | `0.00% <0.00%> (-100.00%)` | :arrow_down: |
| [rust/arrow/src/compute/util.rs](https://codecov.io/gh/apache/arrow/pull/8882/diff?src=pr&el=tree#diff-cnVzdC9hcnJvdy9zcmMvY29tcHV0ZS91dGlsLnJz) | `0.00% <0.00%> (-99.14%)` | :arrow_down: |
| ... and [74 more](https://codecov.io/gh/apache/arrow/pull/8882/diff?src=pr&el=tree-more) | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/arrow/pull/8882?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/arrow/pull/8882?src=pr&el=footer). Last update [db94f24...4ebd8c3](https://codecov.io/gh/apache/arrow/pull/8882?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] alamb commented on pull request #8882: ARROW-10864: [Rust] Use standard ordering for floats
Posted by GitBox <gi...@apache.org>.
alamb commented on pull request #8882:
URL: https://github.com/apache/arrow/pull/8882#issuecomment-754640846
I will try and look at this PR later today
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] Dandandan commented on pull request #8882: ARROW-10864: [Rust] Use standard ordering for floats
Posted by GitBox <gi...@apache.org>.
Dandandan commented on pull request #8882:
URL: https://github.com/apache/arrow/pull/8882#issuecomment-753454663
Thanks @nevi-me for remembering me. I updated and finalized the PR.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] codecov-io edited a comment on pull request #8882: ARROW-10864: [Rust] Use standard ordering for floats
Posted by GitBox <gi...@apache.org>.
codecov-io edited a comment on pull request #8882:
URL: https://github.com/apache/arrow/pull/8882#issuecomment-741980279
# [Codecov](https://codecov.io/gh/apache/arrow/pull/8882?src=pr&el=h1) Report
> Merging [#8882](https://codecov.io/gh/apache/arrow/pull/8882?src=pr&el=desc) (c01932b) into [master](https://codecov.io/gh/apache/arrow/commit/399245678c1be5d7f45fb350cb09baa36a35797b?el=desc) (3992456) will **increase** coverage by `0.00%`.
> The diff coverage is `68.62%`.
[](https://codecov.io/gh/apache/arrow/pull/8882?src=pr&el=tree)
```diff
@@ Coverage Diff @@
## master #8882 +/- ##
=======================================
Coverage 82.56% 82.56%
=======================================
Files 203 203
Lines 50042 50032 -10
=======================================
- Hits 41315 41307 -8
+ Misses 8727 8725 -2
```
| [Impacted Files](https://codecov.io/gh/apache/arrow/pull/8882?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [rust/arrow/src/compute/kernels/sort.rs](https://codecov.io/gh/apache/arrow/pull/8882/diff?src=pr&el=tree#diff-cnVzdC9hcnJvdy9zcmMvY29tcHV0ZS9rZXJuZWxzL3NvcnQucnM=) | `93.58% <68.62%> (+0.15%)` | :arrow_up: |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/arrow/pull/8882?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/arrow/pull/8882?src=pr&el=footer). Last update [3992456...d04f893](https://codecov.io/gh/apache/arrow/pull/8882?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] codecov-io edited a comment on pull request #8882: ARROW-10864: [Rust] Use standard ordering for floats
Posted by GitBox <gi...@apache.org>.
codecov-io edited a comment on pull request #8882:
URL: https://github.com/apache/arrow/pull/8882#issuecomment-741980279
# [Codecov](https://codecov.io/gh/apache/arrow/pull/8882?src=pr&el=h1) Report
> Merging [#8882](https://codecov.io/gh/apache/arrow/pull/8882?src=pr&el=desc) (bfb14d7) into [master](https://codecov.io/gh/apache/arrow/commit/5228ede9abf8ecf9b4bb68a06075cd16af3523e9?el=desc) (5228ede) will **increase** coverage by `0.01%`.
> The diff coverage is `68.62%`.
[](https://codecov.io/gh/apache/arrow/pull/8882?src=pr&el=tree)
```diff
@@ Coverage Diff @@
## master #8882 +/- ##
==========================================
+ Coverage 82.59% 82.60% +0.01%
==========================================
Files 204 204
Lines 50169 50177 +8
==========================================
+ Hits 41436 41449 +13
+ Misses 8733 8728 -5
```
| [Impacted Files](https://codecov.io/gh/apache/arrow/pull/8882?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [rust/arrow/src/compute/kernels/sort.rs](https://codecov.io/gh/apache/arrow/pull/8882/diff?src=pr&el=tree#diff-cnVzdC9hcnJvdy9zcmMvY29tcHV0ZS9rZXJuZWxzL3NvcnQucnM=) | `93.56% <68.62%> (+0.14%)` | :arrow_up: |
| [rust/arrow/src/array/array\_string.rs](https://codecov.io/gh/apache/arrow/pull/8882/diff?src=pr&el=tree#diff-cnVzdC9hcnJvdy9zcmMvYXJyYXkvYXJyYXlfc3RyaW5nLnJz) | `89.94% <0.00%> (-0.22%)` | :arrow_down: |
| [rust/arrow/src/array/transform/mod.rs](https://codecov.io/gh/apache/arrow/pull/8882/diff?src=pr&el=tree#diff-cnVzdC9hcnJvdy9zcmMvYXJyYXkvdHJhbnNmb3JtL21vZC5ycw==) | `89.01% <0.00%> (+0.72%)` | :arrow_up: |
| [rust/arrow/src/ffi.rs](https://codecov.io/gh/apache/arrow/pull/8882/diff?src=pr&el=tree#diff-cnVzdC9hcnJvdy9zcmMvZmZpLnJz) | `72.22% <0.00%> (+2.22%)` | :arrow_up: |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/arrow/pull/8882?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/arrow/pull/8882?src=pr&el=footer). Last update [5228ede...bfb14d7](https://codecov.io/gh/apache/arrow/pull/8882?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] jhorstmann commented on pull request #8882: ARROW-10864: [Rust] Use standard ordering for floats
Posted by GitBox <gi...@apache.org>.
jhorstmann commented on pull request #8882:
URL: https://github.com/apache/arrow/pull/8882#issuecomment-753647897
@Dandandan there is another implementation of float comparison in use for lexicographical ordering (sorting by multiple columns) in `ord.rs`. That would also need to be updated to be consistent.
Ideally the `min`/`max` kernels would also follow the same ordering. For the simd implementations that could be tricky because they currently delegate all types to the packed_simd `lt` implementation. I didn't have time to look into it in more detail yet.
In my understanding, the only difference in behaviour should be around negative NaN and that small difference shouldn't block this PR.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] alamb commented on a change in pull request #8882: ARROW-10864: [Rust] Use standard ordering for floats
Posted by GitBox <gi...@apache.org>.
alamb commented on a change in pull request #8882:
URL: https://github.com/apache/arrow/pull/8882#discussion_r553458570
##########
File path: rust/arrow/src/compute/kernels/sort.rs
##########
@@ -41,40 +41,33 @@ pub fn sort(values: &ArrayRef, options: Option<SortOptions>) -> Result<ArrayRef>
take(values.as_ref(), &indices, None)
}
-// partition indices into non-NaN and NaN
-fn partition_nan<T: ArrowPrimitiveType>(
- array: &ArrayRef,
- v: Vec<u32>,
-) -> (Vec<u32>, Vec<u32>) {
- // partition by nan for float types
- if matches!(T::DATA_TYPE, DataType::Float32) {
- // T::Native has no `is_nan` and thus we need to downcast
- let array = array
- .as_any()
- .downcast_ref::<Float32Array>()
- .expect("Unable to downcast array");
- let has_nan = v.iter().any(|index| array.value(*index as usize).is_nan());
- if has_nan {
- v.into_iter()
- .partition(|index| !array.value(*index as usize).is_nan())
- } else {
- (v, vec![])
- }
- } else if matches!(T::DATA_TYPE, DataType::Float64) {
- let array = array
- .as_any()
- .downcast_ref::<Float64Array>()
- .expect("Unable to downcast array");
- let has_nan = v.iter().any(|index| array.value(*index as usize).is_nan());
- if has_nan {
- v.into_iter()
- .partition(|index| !array.value(*index as usize).is_nan())
- } else {
- (v, vec![])
- }
- } else {
- unreachable!("Partition by nan is only applicable to float types")
- }
+// sorts f32 in IEEE 754 total ordering
Review comment:
I added comments here: https://github.com/apache/arrow/pull/9129
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] Dandandan commented on pull request #8882: ARROW-10864: [Rust] Use standard ordering for floats
Posted by GitBox <gi...@apache.org>.
Dandandan commented on pull request #8882:
URL: https://github.com/apache/arrow/pull/8882#issuecomment-753602953
@nevi-me yes, that is the info I also found.
There are also nightly implementations of `total_cmp`, we could switch whenever it is stabilized.
https://doc.rust-lang.org/std/primitive.f64.html#method.total_cmp
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org