You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@arrow.apache.org by "Jefffrey (via GitHub)" <gi...@apache.org> on 2026/08/27 10:33:51 UTC
[PR] chore: ensure miri ignores are properly commented, add guidelines [arrow-rs]
Jefffrey opened a new pull request, #10884:
URL: https://github.com/apache/arrow-rs/pull/10884
Make sure we always have a clear reason why certain tests are ignored; also try enabling some, and add guidelines for these annotations so we have a reference
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] chore: ensure miri ignores are properly commented, add guidelines [arrow-rs]
Posted by "Jefffrey (via GitHub)" <gi...@apache.org>.
Jefffrey commented on PR #10884:
URL: https://github.com/apache/arrow-rs/pull/10884#issuecomment-5461330136
thanks all
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] chore: ensure miri ignores are properly commented, add guidelines [arrow-rs]
Posted by "ethantang93 (via GitHub)" <gi...@apache.org>.
ethantang93 commented on code in PR #10884:
URL: https://github.com/apache/arrow-rs/pull/10884#discussion_r3883409279
##########
CONTRIBUTING.md:
##########
@@ -166,6 +166,38 @@ parquet files run the following from the top-level `arrow-rs` directory:
cargo fmt -p parquet -- --check --config skip_children=true `find ./parquet -name "*.rs" \! -name format.rs`
```
+## Miri
+
+Run tests under [`Miri`](https://github.com/rust-lang/miri) like so, assuming
+[`cargo-nextest`](https://nexte.st/) is available:
+
+```sh
+# Run all tests
+MIRIFLAGS="-Zmiri-disable-isolation -Zmiri-no-extra-rounding-error" cargo +nightly miri nextest run
+# Run specific tests
+MIRIFLAGS="-Zmiri-disable-isolation -Zmiri-no-extra-rounding-error" cargo +nightly miri nextest run -p arrow-buffer --lib bigint
Review Comment:
```suggestion
# Run specific tests
MIRIFLAGS="-Zmiri-disable-isolation -Zmiri-no-extra-rounding-error" cargo +nightly miri nextest run -p arrow-buffer --lib bigint
# Run specific crate
MIRIFLAGS="-Zmiri-disable-isolation -Zmiri-no-extra-rounding-error" cargo +nightly miri nextest run -p arrow-buffer
```
maybe add this too for convenience
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] chore: ensure miri ignores are properly commented, add guidelines [arrow-rs]
Posted by "alamb (via GitHub)" <gi...@apache.org>.
alamb commented on code in PR #10884:
URL: https://github.com/apache/arrow-rs/pull/10884#discussion_r3874383940
##########
CONTRIBUTING.md:
##########
@@ -166,6 +166,38 @@ parquet files run the following from the top-level `arrow-rs` directory:
cargo fmt -p parquet -- --check --config skip_children=true `find ./parquet -name "*.rs" \! -name format.rs`
```
+## Miri
+
+Run tests under [`Miri`](https://github.com/rust-lang/miri) like so, assuming
+[`cargo-nextest`](https://nexte.st/) is available:
+
+```sh
+# Run all tests
+MIRIFLAGS="-Zmiri-disable-isolation" cargo +nightly miri nextest run
+# Run specific tests
+MIRIFLAGS="-Zmiri-disable-isolation" cargo +nightly miri nextest run -p arrow-buffer --lib bigint
+```
+
+The whole suite will take a long time to run so it's suggested to run individual
+suites/tests when required.
+
+Add a `cfg_attr` to tests in cases where the test should be ignored by Miri:
+
+```rust
+#[test]
+#[cfg_attr(miri, ignore)] // Takes too long
+fn test123() {}
+```
+
+Ensuring to annotate with a comment why the test is being ignored. Common cases
Review Comment:
```suggestion
Please ensure you include a comment why the test is being ignored. Common cases
```
##########
arrow-array/src/array/union_array.rs:
##########
@@ -1128,7 +1128,6 @@ mod tests {
}
#[test]
- #[cfg_attr(miri, ignore)]
Review Comment:
took about 10 for me (m3 max)
```shell
andrewlamb@Andrews-MacBook-Pro-3:~/Software/arrow-rs$ MIRIFLAGS="-Zmiri-disable-isolation" cargo +nightly miri nextest run -p arrow-array --lib test_dense_i32_large
Finished `test` profile [unoptimized + debuginfo] target(s) in 0.06s
info: for the target platform, using target runner `/Users/andrewlamb/.rustup/toolchains/nightly-aarch64-apple-darwin/bin/cargo-miri runner` defined by `target.cfg(all()).runner` specified by `--config`
info: for the host platform, using target runner `/Users/andrewlamb/.rustup/toolchains/nightly-aarch64-apple-darwin/bin/cargo-miri runner` defined by `target.cfg(all()).runner` specified by `--config`
────────────
Nextest run ID ea4df5d8-defd-43a9-9620-15356db3bfcc with nextest profile: default-miri
Starting 1 test across 1 binary (718 tests skipped)
PASS [ 9.937s] arrow-array array::union_array::tests::test_dense_i32_large
────────────
Summary [ 9.938s] 1 test run: 1 passed, 718 skipped
andrewlamb@Andrews-MacBook-Pro-3:~/Software/arrow-rs$
```
##########
arrow-cast/src/parse.rs:
##########
@@ -1611,7 +1611,6 @@ mod tests {
}
#[test]
- #[cfg_attr(miri, ignore)] // unsupported operation: can't call foreign function: mktime
Review Comment:
Worked good for me too
```shell
Nextest run ID 8245ef2e-512b-4969-8b20-2043c2b5768b with nextest profile: default-miri
Starting 35 tests across 1 binary (340 tests skipped)
PASS [ 3.694s] arrow-cast cast::decimal::tests::test_parse_string_to_decimal_native_long_fraction
PASS [ 3.698s] arrow-cast cast::decimal::tests::test_parse_string_to_decimal_native_zero_with_large_scale
PASS [ 3.732s] arrow-cast cast::decimal::tests::test_parse_string_to_decimal_native_rounding_and_padding
PASS [ 3.771s] arrow-cast parse::tests::parse_time64_nanos
PASS [ 3.796s] arrow-cast parse::tests::parse_time32_millis
PASS [ 3.822s] arrow-cast parse::tests::parse_time32_secs
PASS [ 3.836s] arrow-cast parse::tests::parse_date32_extended_year
PASS [ 3.864s] arrow-cast parse::tests::parse_time64_micros
LEAK [ 3.901s] arrow-cast cast::decimal::tests::test_parse_string_to_decimal_native
PASS [ 3.912s] arrow-cast cast::decimal::tests::test_parse_string_to_decimal_native_integer_widths
PASS [ 3.913s] arrow-cast cast::decimal::tests::test_parse_string_to_decimal_native_invalid_syntax
PASS [ 3.959s] arrow-cast cast::tests::test_parse_string_to_decimal
PASS [ 3.983s] arrow-cast cast::decimal::tests::test_parse_string_to_decimal_native_overflow_not_wrapped
PASS [ 4.567s] arrow-cast parse::tests::string_to_timestamp_chrono
PASS [ 4.576s] arrow-cast parse::tests::parse_date32
PASS [ 4.604s] arrow-cast cast::decimal::tests::test_parse_string_to_decimal_native_rounding_overflow
PASS [ 3.397s] arrow-cast parse::tests::string_to_timestamp_no_timezone
PASS [ 3.419s] arrow-cast parse::tests::string_to_timestamp_timezone
PASS [ 3.437s] arrow-cast parse::tests::string_to_timestamp_timezone_space
PASS [ 3.373s] arrow-cast parse::tests::test_interval_precision
PASS [ 3.406s] arrow-cast parse::tests::test_interval_addition
PASS [ 3.543s] arrow-cast parse::tests::string_to_timestamp_old
PASS [ 3.622s] arrow-cast parse::tests::string_to_timestamp_naive
PASS [ 3.510s] arrow-cast parse::tests::string_without_timezone_to_timestamp
PASS [ 3.486s] arrow-cast parse::tests::test_duplicate_interval_type
PASS [ 3.462s] arrow-cast parse::tests::test_interval_amount_parsing
PASS [ 3.397s] arrow-cast parse::tests::test_parse_empty
PASS [ 3.733s] arrow-cast parse::tests::string_to_timestamp_invalid
PASS [ 3.015s] arrow-cast parse::tests::test_parse_nanos
PASS [ 3.058s] arrow-cast parse::tests::test_parse_interval_month_day_nano_config
PASS [ 4.582s] arrow-cast parse::tests::test_parse_decimal_with_parameter
PASS [ 4.130s] arrow-cast parse::tests::test_parse_interval
PASS [ 2.469s] arrow-cast parse::tests::test_parse_prefix_white_space
PASS [ 2.364s] arrow-cast parse::tests::test_string_to_time_invalid
PASS [ 2.671s] arrow-cast parse::tests::test_string_to_time_chrono
────────────
Summary [ 9.888s] 35 tests run: 35 passed (1 leaky), 340 skipped
``
##########
CONTRIBUTING.md:
##########
@@ -166,6 +166,38 @@ parquet files run the following from the top-level `arrow-rs` directory:
cargo fmt -p parquet -- --check --config skip_children=true `find ./parquet -name "*.rs" \! -name format.rs`
```
+## Miri
+
+Run tests under [`Miri`](https://github.com/rust-lang/miri) like so, assuming
+[`cargo-nextest`](https://nexte.st/) is available:
+
+```sh
+# Run all tests
+MIRIFLAGS="-Zmiri-disable-isolation" cargo +nightly miri nextest run
Review Comment:
TIL -- this is a much faster way to use miri
##########
arrow-buffer/src/util/bit_chunk_iterator.rs:
##########
@@ -730,7 +730,6 @@ mod tests {
}
#[test]
- #[cfg_attr(miri, ignore)]
Review Comment:
This one took 55s on my machine
```shell
andrewlamb@Andrews-MacBook-Pro-3:~/Software/arrow-rs$ MIRIFLAGS="-Zmiri-disable-isolation" cargo +nightly miri nextest run -p arrow-buffer --lib fuzz_unaligned_bit_chunk_iterator
Finished `test` profile [unoptimized + debuginfo] target(s) in 0.05s
info: for the target platform, using target runner `/Users/andrewlamb/.rustup/toolchains/nightly-aarch64-apple-darwin/bin/cargo-miri runner` defined by `target.cfg(all()).runner` specified by `--config`
info: for the host platform, using target runner `/Users/andrewlamb/.rustup/toolchains/nightly-aarch64-apple-darwin/bin/cargo-miri runner` defined by `target.cfg(all()).runner` specified by `--config`
────────────
Nextest run ID 2e28df64-bd94-4d87-95de-bccba15bda86 with nextest profile: default-miri
Starting 1 test across 1 binary (343 tests skipped)
PASS [ 54.627s] arrow-buffer util::bit_chunk_iterator::tests::fuzz_unaligned_bit_chunk_iterator
────────────
Summary [ 54.630s] 1 test run: 1 passed, 343 skipped
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] chore: ensure miri ignores are properly commented, add guidelines [arrow-rs]
Posted by "Jefffrey (via GitHub)" <gi...@apache.org>.
Jefffrey merged PR #10884:
URL: https://github.com/apache/arrow-rs/pull/10884
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] chore: ensure miri ignores are properly commented, add guidelines [arrow-rs]
Posted by "Jefffrey (via GitHub)" <gi...@apache.org>.
Jefffrey commented on code in PR #10884:
URL: https://github.com/apache/arrow-rs/pull/10884#discussion_r3870915248
##########
arrow-array/src/array/union_array.rs:
##########
@@ -1128,7 +1128,6 @@ mod tests {
}
#[test]
- #[cfg_attr(miri, ignore)]
Review Comment:
i think it took 30 seconds on my M4 mac machine, so I think its fine to enable
##########
arrow-buffer/src/bigint/mod.rs:
##########
@@ -1819,7 +1819,7 @@ mod tests {
}
#[test]
- #[cfg_attr(miri, ignore)]
+ #[cfg_attr(miri, ignore)] // conversion to float isn't accurate under miri
Review Comment:
```
thread 'bigint::tests::test_decimal256_to_f64_typical_values' (82672) panicked at arrow-buffer/src/bigint/mod.rs:1825:9:
assertion `left == right` failed
left: 41.99999999999997
right: 42.0
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
note: in Miri, you may have to set `MIRIFLAGS=-Zmiri-env-forward=RUST_BACKTRACE` for the environment variable to have an effect
```
didnt spend any time looking into this, im sure theres a deeper reason here
##########
arrow-cast/src/parse.rs:
##########
@@ -1611,7 +1611,6 @@ mod tests {
}
#[test]
- #[cfg_attr(miri, ignore)] // unsupported operation: can't call foreign function: mktime
Review Comment:
seems to run fine, at least on macos
##########
arrow-buffer/src/util/bit_chunk_iterator.rs:
##########
@@ -730,7 +730,6 @@ mod tests {
}
#[test]
- #[cfg_attr(miri, ignore)]
Review Comment:
similarly here it took about 30 seconds or so
##########
arrow-schema/src/datatype.rs:
##########
@@ -1271,7 +1271,7 @@ mod tests {
}
#[test]
- #[cfg_attr(miri, ignore)] // Can't handle the inlined strings of the assert_debug_snapshot macro
+ #[cfg_attr(miri, ignore)] // fork is not supported
Review Comment:
more clear what is not supported here
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] chore: ensure miri ignores are properly commented, add guidelines [arrow-rs]
Posted by "Jefffrey (via GitHub)" <gi...@apache.org>.
Jefffrey commented on code in PR #10884:
URL: https://github.com/apache/arrow-rs/pull/10884#discussion_r3877400780
##########
arrow-buffer/src/util/bit_chunk_iterator.rs:
##########
@@ -730,7 +730,6 @@ mod tests {
}
#[test]
- #[cfg_attr(miri, ignore)]
Review Comment:
that sounds correct, i think i just forgot which test took how long
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] chore: ensure miri ignores are properly commented, add guidelines [arrow-rs]
Posted by "Jefffrey (via GitHub)" <gi...@apache.org>.
Jefffrey commented on code in PR #10884:
URL: https://github.com/apache/arrow-rs/pull/10884#discussion_r3877404564
##########
arrow-buffer/src/bigint/mod.rs:
##########
@@ -1819,7 +1819,7 @@ mod tests {
}
#[test]
- #[cfg_attr(miri, ignore)]
+ #[cfg_attr(miri, ignore)] // conversion to float isn't accurate under miri
Review Comment:
oh this is quite nice
looking at the miri flags i decided to go with `-Zmiri-no-extra-rounding-error`, seems to pass locally now 👍
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] chore: ensure miri ignores are properly commented, add guidelines [arrow-rs]
Posted by "jhorstmann (via GitHub)" <gi...@apache.org>.
jhorstmann commented on code in PR #10884:
URL: https://github.com/apache/arrow-rs/pull/10884#discussion_r3873627645
##########
arrow-buffer/src/bigint/mod.rs:
##########
@@ -1819,7 +1819,7 @@ mod tests {
}
#[test]
- #[cfg_attr(miri, ignore)]
+ #[cfg_attr(miri, ignore)] // conversion to float isn't accurate under miri
Review Comment:
There is a flag `-Zmiri-deterministic-floats` to disable this artificial precision loss.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org