You are viewing a plain text version of this content. The canonical link for it is here.
Posted to jira@arrow.apache.org by "Ritchie (Jira)" <ji...@apache.org> on 2020/09/11 12:24:00 UTC
[jira] [Updated] (ARROW-9975) min and max aggregate kernels are
inconsistent for floating point types.
[ https://issues.apache.org/jira/browse/ARROW-9975?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ritchie updated ARROW-9975:
---------------------------
Description:
The aggregation kernels min and max yield inconsistent results if there are NaN values in the arrays.
Related issue: [https://github.com/ritchie46/polars/issues/71]
{code:c++}
use arrow::{
datatypes::Float64Type,
compute::kernels::aggregate::min,
array::{
PrimitiveBuilder
}
};
let mut builder_1 = PrimitiveBuilder::<Float64Type>::new(2);
builder_1.append_value(1.0).unwrap();
builder_1.append_value(f64::NAN).unwrap();
let array_1 = builder_1.finish();
let mut builder_2 = PrimitiveBuilder::<Float64Type>::new(2);
builder_2.append_value(f64::NAN).unwrap();
builder_2.append_value(1.0).unwrap();
let array_2 = builder_2.finish();
// expect both nan, or both 1.0
assert_eq!(min(&array_1), min(&array_2));
{code}
Outputs:
{code:bash}
Left: Some(1.0)
Right: Some(NaN)
<Click to see difference>
thread 'frame::ser::csv::test::test_' panicked at 'assertion failed: `(left == right)`
left: `Some(1.0)`,
right: `Some(NaN)`', polars/src/frame/ser/csv.rs:328:9
stack backtrace:
0: rust_begin_unwind
at /rustc/73dc675b9437c2a51a975a9f58cc66f05463c351/library/std/src/panicking.rs:483
1: std::panicking::begin_panic_fmt
at /rustc/73dc675b9437c2a51a975a9f58cc66f05463c351/library/std/src/panicking.rs:437
2: polars::frame::ser::csv::test::test_
at ./src/frame/ser/csv.rs:328
3: polars::frame::ser::csv::test::test_::{{closure}}
at ./src/frame/ser/csv.rs:310
4: core::ops::function::FnOnce::call_once
at /home/ritchie46/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/core/src/ops/function.rs:227
5: core::ops::function::FnOnce::call_once
at /rustc/73dc675b9437c2a51a975a9f58cc66f05463c351/library/core/src/ops/function.rs:227
note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.
error: test failed, to rerun pass '-p polars --lib'
{code}
was:
The aggregation kernels min and max yield inconsistent results if there are NaN values in the arrays.
Related issue: https://github.com/ritchie46/polars/issues/71
{code:c++}
use arrow::{
datatypes::Float32Type,
compute::kernels::aggregate::min,
array::{
PrimitiveBuilder
}
};
let mut builder_1 = PrimitiveBuilder::<Float64Type>::new(2);
builder_1.append_value(1.0).unwrap();
builder_1.append_value(f64::NAN).unwrap();
let array_1 = builder_1.finish();
let mut builder_2 = PrimitiveBuilder::<Float64Type>::new(2);
builder_2.append_value(f64::NAN).unwrap();
builder_2.append_value(1.0).unwrap();
let array_2 = builder_2.finish();
// expect both nan, or both 1.0
assert_eq!(min(&array_1), min(&array_2));
{code}
Outputs:
{code:bash}
Left: Some(1.0)
Right: Some(NaN)
<Click to see difference>
thread 'frame::ser::csv::test::test_' panicked at 'assertion failed: `(left == right)`
left: `Some(1.0)`,
right: `Some(NaN)`', polars/src/frame/ser/csv.rs:328:9
stack backtrace:
0: rust_begin_unwind
at /rustc/73dc675b9437c2a51a975a9f58cc66f05463c351/library/std/src/panicking.rs:483
1: std::panicking::begin_panic_fmt
at /rustc/73dc675b9437c2a51a975a9f58cc66f05463c351/library/std/src/panicking.rs:437
2: polars::frame::ser::csv::test::test_
at ./src/frame/ser/csv.rs:328
3: polars::frame::ser::csv::test::test_::{{closure}}
at ./src/frame/ser/csv.rs:310
4: core::ops::function::FnOnce::call_once
at /home/ritchie46/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/core/src/ops/function.rs:227
5: core::ops::function::FnOnce::call_once
at /rustc/73dc675b9437c2a51a975a9f58cc66f05463c351/library/core/src/ops/function.rs:227
note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.
error: test failed, to rerun pass '-p polars --lib'
{code}
> min and max aggregate kernels are inconsistent for floating point types.
> -------------------------------------------------------------------------
>
> Key: ARROW-9975
> URL: https://issues.apache.org/jira/browse/ARROW-9975
> Project: Apache Arrow
> Issue Type: Bug
> Components: Rust
> Affects Versions: 1.0.1
> Reporter: Ritchie
> Priority: Major
>
> The aggregation kernels min and max yield inconsistent results if there are NaN values in the arrays.
> Related issue: [https://github.com/ritchie46/polars/issues/71]
> {code:c++}
> use arrow::{
> datatypes::Float64Type,
> compute::kernels::aggregate::min,
> array::{
> PrimitiveBuilder
> }
> };
> let mut builder_1 = PrimitiveBuilder::<Float64Type>::new(2);
> builder_1.append_value(1.0).unwrap();
> builder_1.append_value(f64::NAN).unwrap();
> let array_1 = builder_1.finish();
> let mut builder_2 = PrimitiveBuilder::<Float64Type>::new(2);
> builder_2.append_value(f64::NAN).unwrap();
> builder_2.append_value(1.0).unwrap();
> let array_2 = builder_2.finish();
>
> // expect both nan, or both 1.0
> assert_eq!(min(&array_1), min(&array_2));
> {code}
> Outputs:
> {code:bash}
> Left: Some(1.0)
> Right: Some(NaN)
> <Click to see difference>
> thread 'frame::ser::csv::test::test_' panicked at 'assertion failed: `(left == right)`
> left: `Some(1.0)`,
> right: `Some(NaN)`', polars/src/frame/ser/csv.rs:328:9
> stack backtrace:
> 0: rust_begin_unwind
> at /rustc/73dc675b9437c2a51a975a9f58cc66f05463c351/library/std/src/panicking.rs:483
> 1: std::panicking::begin_panic_fmt
> at /rustc/73dc675b9437c2a51a975a9f58cc66f05463c351/library/std/src/panicking.rs:437
> 2: polars::frame::ser::csv::test::test_
> at ./src/frame/ser/csv.rs:328
> 3: polars::frame::ser::csv::test::test_::{{closure}}
> at ./src/frame/ser/csv.rs:310
> 4: core::ops::function::FnOnce::call_once
> at /home/ritchie46/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/core/src/ops/function.rs:227
> 5: core::ops::function::FnOnce::call_once
> at /rustc/73dc675b9437c2a51a975a9f58cc66f05463c351/library/core/src/ops/function.rs:227
> note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.
> error: test failed, to rerun pass '-p polars --lib'
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)