You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@arrow.apache.org by GitBox <gi...@apache.org> on 2020/12/09 18:38:49 UTC

[GitHub] [arrow] Dandandan opened a new pull request #8882: ARROW-10864: [Rust] Use standard ordering for floats [WIP]

Dandandan opened a new pull request #8882:
URL: https://github.com/apache/arrow/pull/8882


   This implements ordering using IEEE 754 ordering as mentioned in this discussion (only for sort now)
   
   https://github.com/apache/arrow/pull/8685


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [arrow] codecov-io edited a comment on pull request #8882: ARROW-10864: [Rust] Use standard ordering for floats

Posted by GitBox <gi...@apache.org>.
codecov-io edited a comment on pull request #8882:
URL: https://github.com/apache/arrow/pull/8882#issuecomment-741980279


   # [Codecov](https://codecov.io/gh/apache/arrow/pull/8882?src=pr&el=h1) Report
   > Merging [#8882](https://codecov.io/gh/apache/arrow/pull/8882?src=pr&el=desc) (40c9754) into [master](https://codecov.io/gh/apache/arrow/commit/399245678c1be5d7f45fb350cb09baa36a35797b?el=desc) (3992456) will **increase** coverage by `0.00%`.
   > The diff coverage is `68.62%`.
   
   [![Impacted file tree graph](https://codecov.io/gh/apache/arrow/pull/8882/graphs/tree.svg?width=650&height=150&src=pr&token=LpTCFbqVT1)](https://codecov.io/gh/apache/arrow/pull/8882?src=pr&el=tree)
   
   ```diff
   @@           Coverage Diff           @@
   ##           master    #8882   +/-   ##
   =======================================
     Coverage   82.56%   82.56%           
   =======================================
     Files         203      203           
     Lines       50042    50032   -10     
   =======================================
   - Hits        41315    41308    -7     
   + Misses       8727     8724    -3     
   ```
   
   
   | [Impacted Files](https://codecov.io/gh/apache/arrow/pull/8882?src=pr&el=tree) | Coverage Δ | |
   |---|---|---|
   | [rust/arrow/src/compute/kernels/sort.rs](https://codecov.io/gh/apache/arrow/pull/8882/diff?src=pr&el=tree#diff-cnVzdC9hcnJvdy9zcmMvY29tcHV0ZS9rZXJuZWxzL3NvcnQucnM=) | `93.58% <68.62%> (+0.15%)` | :arrow_up: |
   | [rust/parquet/src/encodings/encoding.rs](https://codecov.io/gh/apache/arrow/pull/8882/diff?src=pr&el=tree#diff-cnVzdC9wYXJxdWV0L3NyYy9lbmNvZGluZ3MvZW5jb2RpbmcucnM=) | `95.43% <0.00%> (+0.19%)` | :arrow_up: |
   
   ------
   
   [Continue to review full report at Codecov](https://codecov.io/gh/apache/arrow/pull/8882?src=pr&el=continue).
   > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
   > `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
   > Powered by [Codecov](https://codecov.io/gh/apache/arrow/pull/8882?src=pr&el=footer). Last update [3992456...c01932b](https://codecov.io/gh/apache/arrow/pull/8882?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [arrow] codecov-io edited a comment on pull request #8882: ARROW-10864: [Rust] Use standard ordering for floats

Posted by GitBox <gi...@apache.org>.
codecov-io edited a comment on pull request #8882:
URL: https://github.com/apache/arrow/pull/8882#issuecomment-741980279


   # [Codecov](https://codecov.io/gh/apache/arrow/pull/8882?src=pr&el=h1) Report
   > Merging [#8882](https://codecov.io/gh/apache/arrow/pull/8882?src=pr&el=desc) (4d9c35c) into [master](https://codecov.io/gh/apache/arrow/commit/399245678c1be5d7f45fb350cb09baa36a35797b?el=desc) (3992456) will **increase** coverage by `0.00%`.
   > The diff coverage is `68.62%`.
   
   [![Impacted file tree graph](https://codecov.io/gh/apache/arrow/pull/8882/graphs/tree.svg?width=650&height=150&src=pr&token=LpTCFbqVT1)](https://codecov.io/gh/apache/arrow/pull/8882?src=pr&el=tree)
   
   ```diff
   @@           Coverage Diff           @@
   ##           master    #8882   +/-   ##
   =======================================
     Coverage   82.56%   82.56%           
   =======================================
     Files         203      203           
     Lines       50042    50032   -10     
   =======================================
   - Hits        41315    41308    -7     
   + Misses       8727     8724    -3     
   ```
   
   
   | [Impacted Files](https://codecov.io/gh/apache/arrow/pull/8882?src=pr&el=tree) | Coverage Δ | |
   |---|---|---|
   | [rust/arrow/src/compute/kernels/sort.rs](https://codecov.io/gh/apache/arrow/pull/8882/diff?src=pr&el=tree#diff-cnVzdC9hcnJvdy9zcmMvY29tcHV0ZS9rZXJuZWxzL3NvcnQucnM=) | `93.58% <68.62%> (+0.15%)` | :arrow_up: |
   | [rust/parquet/src/encodings/encoding.rs](https://codecov.io/gh/apache/arrow/pull/8882/diff?src=pr&el=tree#diff-cnVzdC9wYXJxdWV0L3NyYy9lbmNvZGluZ3MvZW5jb2RpbmcucnM=) | `95.43% <0.00%> (+0.19%)` | :arrow_up: |
   
   ------
   
   [Continue to review full report at Codecov](https://codecov.io/gh/apache/arrow/pull/8882?src=pr&el=continue).
   > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
   > `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
   > Powered by [Codecov](https://codecov.io/gh/apache/arrow/pull/8882?src=pr&el=footer). Last update [3992456...d04f893](https://codecov.io/gh/apache/arrow/pull/8882?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [arrow] alamb commented on a change in pull request #8882: ARROW-10864: [Rust] Use standard ordering for floats

Posted by GitBox <gi...@apache.org>.
alamb commented on a change in pull request #8882:
URL: https://github.com/apache/arrow/pull/8882#discussion_r552202489



##########
File path: rust/arrow/src/compute/kernels/sort.rs
##########
@@ -41,40 +41,33 @@ pub fn sort(values: &ArrayRef, options: Option<SortOptions>) -> Result<ArrayRef>
     take(values.as_ref(), &indices, None)
 }
 
-// partition indices into non-NaN and NaN
-fn partition_nan<T: ArrowPrimitiveType>(
-    array: &ArrayRef,
-    v: Vec<u32>,
-) -> (Vec<u32>, Vec<u32>) {
-    // partition by nan for float types
-    if matches!(T::DATA_TYPE, DataType::Float32) {
-        // T::Native has no `is_nan` and thus we need to downcast
-        let array = array
-            .as_any()
-            .downcast_ref::<Float32Array>()
-            .expect("Unable to downcast array");
-        let has_nan = v.iter().any(|index| array.value(*index as usize).is_nan());
-        if has_nan {
-            v.into_iter()
-                .partition(|index| !array.value(*index as usize).is_nan())
-        } else {
-            (v, vec![])
-        }
-    } else if matches!(T::DATA_TYPE, DataType::Float64) {
-        let array = array
-            .as_any()
-            .downcast_ref::<Float64Array>()
-            .expect("Unable to downcast array");
-        let has_nan = v.iter().any(|index| array.value(*index as usize).is_nan());
-        if has_nan {
-            v.into_iter()
-                .partition(|index| !array.value(*index as usize).is_nan())
-        } else {
-            (v, vec![])
-        }
-    } else {
-        unreachable!("Partition by nan is only applicable to float types")
-    }
+// sorts f32 in IEEE 754 total ordering

Review comment:
       It might be good here to provide a pointer to the original source implementation (e.g. https://doc.rust-lang.org/std/primitive.f64.html#method.total_cmp) and a TODO to change to use that API when is stabilized




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [arrow] Dandandan commented on pull request #8882: ARROW-10864: [Rust] Use standard ordering for floats

Posted by GitBox <gi...@apache.org>.
Dandandan commented on pull request #8882:
URL: https://github.com/apache/arrow/pull/8882#issuecomment-753613551


   @nevi-me rebased, but looks like needs-rebase label isn't removed


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [arrow] nevi-me commented on pull request #8882: ARROW-10864: [Rust] Use standard ordering for floats [WIP]

Posted by GitBox <gi...@apache.org>.
nevi-me commented on pull request #8882:
URL: https://github.com/apache/arrow/pull/8882#issuecomment-753443369


   @Dandandan what more work do you still need to do on this PR?


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [arrow] alamb closed pull request #8882: ARROW-10864: [Rust] Use standard ordering for floats

Posted by GitBox <gi...@apache.org>.
alamb closed pull request #8882:
URL: https://github.com/apache/arrow/pull/8882


   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [arrow] codecov-io edited a comment on pull request #8882: ARROW-10864: [Rust] Use standard ordering for floats

Posted by GitBox <gi...@apache.org>.
codecov-io edited a comment on pull request #8882:
URL: https://github.com/apache/arrow/pull/8882#issuecomment-741980279


   # [Codecov](https://codecov.io/gh/apache/arrow/pull/8882?src=pr&el=h1) Report
   > Merging [#8882](https://codecov.io/gh/apache/arrow/pull/8882?src=pr&el=desc) (eac99bd) into [master](https://codecov.io/gh/apache/arrow/commit/dfef236f7587e4168ac1e07bd09e42d9373beb70?el=desc) (dfef236) will **decrease** coverage by `0.00%`.
   > The diff coverage is `68.62%`.
   
   [![Impacted file tree graph](https://codecov.io/gh/apache/arrow/pull/8882/graphs/tree.svg?width=650&height=150&src=pr&token=LpTCFbqVT1)](https://codecov.io/gh/apache/arrow/pull/8882?src=pr&el=tree)
   
   ```diff
   @@            Coverage Diff             @@
   ##           master    #8882      +/-   ##
   ==========================================
   - Coverage   82.60%   82.60%   -0.01%     
   ==========================================
     Files         204      204              
     Lines       50189    50177      -12     
   ==========================================
   - Hits        41459    41448      -11     
   + Misses       8730     8729       -1     
   ```
   
   
   | [Impacted Files](https://codecov.io/gh/apache/arrow/pull/8882?src=pr&el=tree) | Coverage Δ | |
   |---|---|---|
   | [rust/arrow/src/compute/kernels/sort.rs](https://codecov.io/gh/apache/arrow/pull/8882/diff?src=pr&el=tree#diff-cnVzdC9hcnJvdy9zcmMvY29tcHV0ZS9rZXJuZWxzL3NvcnQucnM=) | `93.56% <68.62%> (+0.14%)` | :arrow_up: |
   | [rust/parquet/src/encodings/encoding.rs](https://codecov.io/gh/apache/arrow/pull/8882/diff?src=pr&el=tree#diff-cnVzdC9wYXJxdWV0L3NyYy9lbmNvZGluZ3MvZW5jb2RpbmcucnM=) | `95.24% <0.00%> (-0.20%)` | :arrow_down: |
   
   ------
   
   [Continue to review full report at Codecov](https://codecov.io/gh/apache/arrow/pull/8882?src=pr&el=continue).
   > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
   > `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
   > Powered by [Codecov](https://codecov.io/gh/apache/arrow/pull/8882?src=pr&el=footer). Last update [dfef236...eac99bd](https://codecov.io/gh/apache/arrow/pull/8882?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [arrow] codecov-io edited a comment on pull request #8882: ARROW-10864: [Rust] Use standard ordering for floats

Posted by GitBox <gi...@apache.org>.
codecov-io edited a comment on pull request #8882:
URL: https://github.com/apache/arrow/pull/8882#issuecomment-741980279


   # [Codecov](https://codecov.io/gh/apache/arrow/pull/8882?src=pr&el=h1) Report
   > Merging [#8882](https://codecov.io/gh/apache/arrow/pull/8882?src=pr&el=desc) (d04f893) into [master](https://codecov.io/gh/apache/arrow/commit/399245678c1be5d7f45fb350cb09baa36a35797b?el=desc) (3992456) will **decrease** coverage by `0.00%`.
   > The diff coverage is `68.62%`.
   
   [![Impacted file tree graph](https://codecov.io/gh/apache/arrow/pull/8882/graphs/tree.svg?width=650&height=150&src=pr&token=LpTCFbqVT1)](https://codecov.io/gh/apache/arrow/pull/8882?src=pr&el=tree)
   
   ```diff
   @@            Coverage Diff             @@
   ##           master    #8882      +/-   ##
   ==========================================
   - Coverage   82.56%   82.56%   -0.01%     
   ==========================================
     Files         203      203              
     Lines       50042    50030      -12     
   ==========================================
   - Hits        41315    41305      -10     
   + Misses       8727     8725       -2     
   ```
   
   
   | [Impacted Files](https://codecov.io/gh/apache/arrow/pull/8882?src=pr&el=tree) | Coverage Δ | |
   |---|---|---|
   | [rust/arrow/src/compute/kernels/sort.rs](https://codecov.io/gh/apache/arrow/pull/8882/diff?src=pr&el=tree#diff-cnVzdC9hcnJvdy9zcmMvY29tcHV0ZS9rZXJuZWxzL3NvcnQucnM=) | `93.56% <68.62%> (+0.14%)` | :arrow_up: |
   
   ------
   
   [Continue to review full report at Codecov](https://codecov.io/gh/apache/arrow/pull/8882?src=pr&el=continue).
   > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
   > `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
   > Powered by [Codecov](https://codecov.io/gh/apache/arrow/pull/8882?src=pr&el=footer). Last update [3992456...d04f893](https://codecov.io/gh/apache/arrow/pull/8882?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [arrow] github-actions[bot] commented on pull request #8882: ARROW-10864: [Rust] Use standard ordering for floats [WIP]

Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on pull request #8882:
URL: https://github.com/apache/arrow/pull/8882#issuecomment-741979653


   https://issues.apache.org/jira/browse/ARROW-10864


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [arrow] codecov-io commented on pull request #8882: ARROW-10864: [Rust] Use standard ordering for floats [WIP]

Posted by GitBox <gi...@apache.org>.
codecov-io commented on pull request #8882:
URL: https://github.com/apache/arrow/pull/8882#issuecomment-741980279


   # [Codecov](https://codecov.io/gh/apache/arrow/pull/8882?src=pr&el=h1) Report
   > Merging [#8882](https://codecov.io/gh/apache/arrow/pull/8882?src=pr&el=desc) (4ebd8c3) into [master](https://codecov.io/gh/apache/arrow/commit/db94f24f15ad783efe676fef8883b7cab9a82f88?el=desc) (db94f24) will **decrease** coverage by `22.37%`.
   > The diff coverage is `0.00%`.
   
   [![Impacted file tree graph](https://codecov.io/gh/apache/arrow/pull/8882/graphs/tree.svg?width=650&height=150&src=pr&token=LpTCFbqVT1)](https://codecov.io/gh/apache/arrow/pull/8882?src=pr&el=tree)
   
   ```diff
   @@             Coverage Diff             @@
   ##           master    #8882       +/-   ##
   ===========================================
   - Coverage   77.03%   54.66%   -22.38%     
   ===========================================
     Files         173      177        +4     
     Lines       40090    31254     -8836     
   ===========================================
   - Hits        30885    17084    -13801     
   - Misses       9205    14170     +4965     
   ```
   
   
   | [Impacted Files](https://codecov.io/gh/apache/arrow/pull/8882?src=pr&el=tree) | Coverage Δ | |
   |---|---|---|
   | [rust/arrow/src/compute/kernels/sort.rs](https://codecov.io/gh/apache/arrow/pull/8882/diff?src=pr&el=tree#diff-cnVzdC9hcnJvdy9zcmMvY29tcHV0ZS9rZXJuZWxzL3NvcnQucnM=) | `0.00% <0.00%> (-95.43%)` | :arrow_down: |
   | [rust/arrow/src/array/cast.rs](https://codecov.io/gh/apache/arrow/pull/8882/diff?src=pr&el=tree#diff-cnVzdC9hcnJvdy9zcmMvYXJyYXkvY2FzdC5ycw==) | `0.00% <0.00%> (-100.00%)` | :arrow_down: |
   | [rust/arrow/src/array/equal/decimal.rs](https://codecov.io/gh/apache/arrow/pull/8882/diff?src=pr&el=tree#diff-cnVzdC9hcnJvdy9zcmMvYXJyYXkvZXF1YWwvZGVjaW1hbC5ycw==) | `0.00% <0.00%> (-100.00%)` | :arrow_down: |
   | [rust/arrow/src/array/transform/utils.rs](https://codecov.io/gh/apache/arrow/pull/8882/diff?src=pr&el=tree#diff-cnVzdC9hcnJvdy9zcmMvYXJyYXkvdHJhbnNmb3JtL3V0aWxzLnJz) | `0.00% <0.00%> (-100.00%)` | :arrow_down: |
   | [rust/arrow/src/compute/kernels/limit.rs](https://codecov.io/gh/apache/arrow/pull/8882/diff?src=pr&el=tree#diff-cnVzdC9hcnJvdy9zcmMvY29tcHV0ZS9rZXJuZWxzL2xpbWl0LnJz) | `0.00% <0.00%> (-100.00%)` | :arrow_down: |
   | [rust/arrow/src/array/equal/fixed\_list.rs](https://codecov.io/gh/apache/arrow/pull/8882/diff?src=pr&el=tree#diff-cnVzdC9hcnJvdy9zcmMvYXJyYXkvZXF1YWwvZml4ZWRfbGlzdC5ycw==) | `0.00% <0.00%> (-100.00%)` | :arrow_down: |
   | [rust/arrow/src/compute/kernels/length.rs](https://codecov.io/gh/apache/arrow/pull/8882/diff?src=pr&el=tree#diff-cnVzdC9hcnJvdy9zcmMvY29tcHV0ZS9rZXJuZWxzL2xlbmd0aC5ycw==) | `0.00% <0.00%> (-100.00%)` | :arrow_down: |
   | [rust/arrow/src/array/transform/primitive.rs](https://codecov.io/gh/apache/arrow/pull/8882/diff?src=pr&el=tree#diff-cnVzdC9hcnJvdy9zcmMvYXJyYXkvdHJhbnNmb3JtL3ByaW1pdGl2ZS5ycw==) | `0.00% <0.00%> (-100.00%)` | :arrow_down: |
   | [rust/arrow/src/array/transform/variable\_size.rs](https://codecov.io/gh/apache/arrow/pull/8882/diff?src=pr&el=tree#diff-cnVzdC9hcnJvdy9zcmMvYXJyYXkvdHJhbnNmb3JtL3ZhcmlhYmxlX3NpemUucnM=) | `0.00% <0.00%> (-100.00%)` | :arrow_down: |
   | [rust/arrow/src/compute/util.rs](https://codecov.io/gh/apache/arrow/pull/8882/diff?src=pr&el=tree#diff-cnVzdC9hcnJvdy9zcmMvY29tcHV0ZS91dGlsLnJz) | `0.00% <0.00%> (-99.14%)` | :arrow_down: |
   | ... and [74 more](https://codecov.io/gh/apache/arrow/pull/8882/diff?src=pr&el=tree-more) | |
   
   ------
   
   [Continue to review full report at Codecov](https://codecov.io/gh/apache/arrow/pull/8882?src=pr&el=continue).
   > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
   > `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
   > Powered by [Codecov](https://codecov.io/gh/apache/arrow/pull/8882?src=pr&el=footer). Last update [db94f24...4ebd8c3](https://codecov.io/gh/apache/arrow/pull/8882?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [arrow] alamb commented on pull request #8882: ARROW-10864: [Rust] Use standard ordering for floats

Posted by GitBox <gi...@apache.org>.
alamb commented on pull request #8882:
URL: https://github.com/apache/arrow/pull/8882#issuecomment-754640846


   I will try and look at this PR later today


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [arrow] Dandandan commented on pull request #8882: ARROW-10864: [Rust] Use standard ordering for floats

Posted by GitBox <gi...@apache.org>.
Dandandan commented on pull request #8882:
URL: https://github.com/apache/arrow/pull/8882#issuecomment-753454663


   Thanks @nevi-me for remembering me. I updated and finalized the PR.


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [arrow] codecov-io edited a comment on pull request #8882: ARROW-10864: [Rust] Use standard ordering for floats

Posted by GitBox <gi...@apache.org>.
codecov-io edited a comment on pull request #8882:
URL: https://github.com/apache/arrow/pull/8882#issuecomment-741980279


   # [Codecov](https://codecov.io/gh/apache/arrow/pull/8882?src=pr&el=h1) Report
   > Merging [#8882](https://codecov.io/gh/apache/arrow/pull/8882?src=pr&el=desc) (c01932b) into [master](https://codecov.io/gh/apache/arrow/commit/399245678c1be5d7f45fb350cb09baa36a35797b?el=desc) (3992456) will **increase** coverage by `0.00%`.
   > The diff coverage is `68.62%`.
   
   [![Impacted file tree graph](https://codecov.io/gh/apache/arrow/pull/8882/graphs/tree.svg?width=650&height=150&src=pr&token=LpTCFbqVT1)](https://codecov.io/gh/apache/arrow/pull/8882?src=pr&el=tree)
   
   ```diff
   @@           Coverage Diff           @@
   ##           master    #8882   +/-   ##
   =======================================
     Coverage   82.56%   82.56%           
   =======================================
     Files         203      203           
     Lines       50042    50032   -10     
   =======================================
   - Hits        41315    41307    -8     
   + Misses       8727     8725    -2     
   ```
   
   
   | [Impacted Files](https://codecov.io/gh/apache/arrow/pull/8882?src=pr&el=tree) | Coverage Δ | |
   |---|---|---|
   | [rust/arrow/src/compute/kernels/sort.rs](https://codecov.io/gh/apache/arrow/pull/8882/diff?src=pr&el=tree#diff-cnVzdC9hcnJvdy9zcmMvY29tcHV0ZS9rZXJuZWxzL3NvcnQucnM=) | `93.58% <68.62%> (+0.15%)` | :arrow_up: |
   
   ------
   
   [Continue to review full report at Codecov](https://codecov.io/gh/apache/arrow/pull/8882?src=pr&el=continue).
   > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
   > `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
   > Powered by [Codecov](https://codecov.io/gh/apache/arrow/pull/8882?src=pr&el=footer). Last update [3992456...d04f893](https://codecov.io/gh/apache/arrow/pull/8882?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [arrow] codecov-io edited a comment on pull request #8882: ARROW-10864: [Rust] Use standard ordering for floats

Posted by GitBox <gi...@apache.org>.
codecov-io edited a comment on pull request #8882:
URL: https://github.com/apache/arrow/pull/8882#issuecomment-741980279


   # [Codecov](https://codecov.io/gh/apache/arrow/pull/8882?src=pr&el=h1) Report
   > Merging [#8882](https://codecov.io/gh/apache/arrow/pull/8882?src=pr&el=desc) (bfb14d7) into [master](https://codecov.io/gh/apache/arrow/commit/5228ede9abf8ecf9b4bb68a06075cd16af3523e9?el=desc) (5228ede) will **increase** coverage by `0.01%`.
   > The diff coverage is `68.62%`.
   
   [![Impacted file tree graph](https://codecov.io/gh/apache/arrow/pull/8882/graphs/tree.svg?width=650&height=150&src=pr&token=LpTCFbqVT1)](https://codecov.io/gh/apache/arrow/pull/8882?src=pr&el=tree)
   
   ```diff
   @@            Coverage Diff             @@
   ##           master    #8882      +/-   ##
   ==========================================
   + Coverage   82.59%   82.60%   +0.01%     
   ==========================================
     Files         204      204              
     Lines       50169    50177       +8     
   ==========================================
   + Hits        41436    41449      +13     
   + Misses       8733     8728       -5     
   ```
   
   
   | [Impacted Files](https://codecov.io/gh/apache/arrow/pull/8882?src=pr&el=tree) | Coverage Δ | |
   |---|---|---|
   | [rust/arrow/src/compute/kernels/sort.rs](https://codecov.io/gh/apache/arrow/pull/8882/diff?src=pr&el=tree#diff-cnVzdC9hcnJvdy9zcmMvY29tcHV0ZS9rZXJuZWxzL3NvcnQucnM=) | `93.56% <68.62%> (+0.14%)` | :arrow_up: |
   | [rust/arrow/src/array/array\_string.rs](https://codecov.io/gh/apache/arrow/pull/8882/diff?src=pr&el=tree#diff-cnVzdC9hcnJvdy9zcmMvYXJyYXkvYXJyYXlfc3RyaW5nLnJz) | `89.94% <0.00%> (-0.22%)` | :arrow_down: |
   | [rust/arrow/src/array/transform/mod.rs](https://codecov.io/gh/apache/arrow/pull/8882/diff?src=pr&el=tree#diff-cnVzdC9hcnJvdy9zcmMvYXJyYXkvdHJhbnNmb3JtL21vZC5ycw==) | `89.01% <0.00%> (+0.72%)` | :arrow_up: |
   | [rust/arrow/src/ffi.rs](https://codecov.io/gh/apache/arrow/pull/8882/diff?src=pr&el=tree#diff-cnVzdC9hcnJvdy9zcmMvZmZpLnJz) | `72.22% <0.00%> (+2.22%)` | :arrow_up: |
   
   ------
   
   [Continue to review full report at Codecov](https://codecov.io/gh/apache/arrow/pull/8882?src=pr&el=continue).
   > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
   > `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
   > Powered by [Codecov](https://codecov.io/gh/apache/arrow/pull/8882?src=pr&el=footer). Last update [5228ede...bfb14d7](https://codecov.io/gh/apache/arrow/pull/8882?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [arrow] jhorstmann commented on pull request #8882: ARROW-10864: [Rust] Use standard ordering for floats

Posted by GitBox <gi...@apache.org>.
jhorstmann commented on pull request #8882:
URL: https://github.com/apache/arrow/pull/8882#issuecomment-753647897


   @Dandandan there is another implementation of float comparison in use for lexicographical ordering (sorting by multiple columns) in `ord.rs`. That would also need to be updated to be consistent.
   
   Ideally the `min`/`max` kernels would also follow the same ordering. For the simd implementations that could be tricky because they currently delegate all types to the packed_simd `lt` implementation. I didn't have time to look into it in more detail yet.
   
   In my understanding, the only difference in behaviour should be around negative NaN and that small difference shouldn't block this PR.


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [arrow] alamb commented on a change in pull request #8882: ARROW-10864: [Rust] Use standard ordering for floats

Posted by GitBox <gi...@apache.org>.
alamb commented on a change in pull request #8882:
URL: https://github.com/apache/arrow/pull/8882#discussion_r553458570



##########
File path: rust/arrow/src/compute/kernels/sort.rs
##########
@@ -41,40 +41,33 @@ pub fn sort(values: &ArrayRef, options: Option<SortOptions>) -> Result<ArrayRef>
     take(values.as_ref(), &indices, None)
 }
 
-// partition indices into non-NaN and NaN
-fn partition_nan<T: ArrowPrimitiveType>(
-    array: &ArrayRef,
-    v: Vec<u32>,
-) -> (Vec<u32>, Vec<u32>) {
-    // partition by nan for float types
-    if matches!(T::DATA_TYPE, DataType::Float32) {
-        // T::Native has no `is_nan` and thus we need to downcast
-        let array = array
-            .as_any()
-            .downcast_ref::<Float32Array>()
-            .expect("Unable to downcast array");
-        let has_nan = v.iter().any(|index| array.value(*index as usize).is_nan());
-        if has_nan {
-            v.into_iter()
-                .partition(|index| !array.value(*index as usize).is_nan())
-        } else {
-            (v, vec![])
-        }
-    } else if matches!(T::DATA_TYPE, DataType::Float64) {
-        let array = array
-            .as_any()
-            .downcast_ref::<Float64Array>()
-            .expect("Unable to downcast array");
-        let has_nan = v.iter().any(|index| array.value(*index as usize).is_nan());
-        if has_nan {
-            v.into_iter()
-                .partition(|index| !array.value(*index as usize).is_nan())
-        } else {
-            (v, vec![])
-        }
-    } else {
-        unreachable!("Partition by nan is only applicable to float types")
-    }
+// sorts f32 in IEEE 754 total ordering

Review comment:
       I added comments here: https://github.com/apache/arrow/pull/9129 




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [arrow] Dandandan commented on pull request #8882: ARROW-10864: [Rust] Use standard ordering for floats

Posted by GitBox <gi...@apache.org>.
Dandandan commented on pull request #8882:
URL: https://github.com/apache/arrow/pull/8882#issuecomment-753602953


   @nevi-me yes, that is the info I also found.
   
   There are also nightly implementations of `total_cmp`, we could switch whenever it is stabilized.
    https://doc.rust-lang.org/std/primitive.f64.html#method.total_cmp


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org