You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@arrow.apache.org by GitBox <gi...@apache.org> on 2021/05/24 21:29:03 UTC
[GitHub] [arrow-rs] alamb opened a new issue #345: MIRI CI check fails intermittently with `thread 'main' panicked at 'invalid time'`
alamb opened a new issue #345:
URL: https://github.com/apache/arrow-rs/issues/345
**Describe the bug**
Ever since we have enabled MIRI checks on CI in https://github.com/apache/arrow-rs/pull/323 (epic thanks to @roee88 ) we have started to see intermittent failures of the check
**To Reproduce**
Run CI checks on a PR
Here are some example failures
https://github.com/apache/arrow-rs/runs/2656075800
https://github.com/apache/arrow-rs/runs/2658362992
An excerpt from
https://github.com/alamb/arrow-rs/runs/2659329294?check_suite_focus=true
```
test array::array_primitive::tests::test_time32_millisecond_array_from_vec ... ok
test array::array_primitive::tests::test_time32second_fmt_debug ... ok
thread 'main' panicked at 'invalid time', /home/runner/.cargo/registry/src/github.com-1ecc6299db9ec823/chrono-0.4.19/src/naive/time.rs:412:67
stack backtrace:
0: 0xa9fa3ef - std::backtrace_rs::backtrace::miri::trace_unsynchronized::<&mut [closure@std::sys_common::backtrace::_print_fmt::{closure#1}]>
at /usr/share/rust/.rustup/toolchains/nightly-2021-03-24-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/std/src/../../backtrace/src/backtrace/miri.rs:65:18
1: 0xaa266e9 - std::backtrace_rs::backtrace::miri::trace::<&mut [closure@std::sys_common::backtrace::_print_fmt::{closure#1}]>
at /usr/share/rust/.rustup/toolchains/nightly-2021-03-24-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/std/src/../../backtrace/src/backtrace/miri.rs:51:14
2: 0xa9dc49b - std::backtrace_rs::backtrace::trace_unsynchronized::<[closure@std::sys_common::backtrace::_print_fmt::{closure#1}]>
at /usr/share/rust/.rustup/toolchains/nightly-2021-03-24-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/std/src/../../backtrace/src/backtrace/mod.rs:66:5
3: 0x298f21f - std::sys_common::backtrace::_print_fmt
at /usr/share/rust/.rustup/toolchains/nightly-2021-03-24-x86_64-unknown-linux-g
```
**Expected behavior**
A clear and concise description of what you expected to happen.
**Additional context**
Add any other context about the problem here.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-rs] alamb commented on issue #345: MIRI CI check fails intermittently with `thread 'main' panicked at 'invalid time'`
Posted by GitBox <gi...@apache.org>.
alamb commented on issue #345:
URL: https://github.com/apache/arrow-rs/issues/345#issuecomment-847352360
```
Caused by:
process didn't exit successfully: `/usr/share/rust/.rustup/toolchains/nightly-2021-03-24-x86_64-unknown-linux-gnu/bin/cargo-miri /home/runner/work/arrow-rs/arrow-rs/target/x86_64-unknown-linux-gnu/debug/deps/arrow-d73b9ed71d5fffd0 --skip csv --skip ipc --skip json` (exit code: 255)
test compute::kernels::length::tests::bit_length_test_string ...
Error: Process completed with exit code 255.
```
🤔
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-rs] alamb closed issue #345: MIRI CI check fails intermittently with `thread 'main' panicked at 'invalid time'`
Posted by GitBox <gi...@apache.org>.
alamb closed issue #345:
URL: https://github.com/apache/arrow-rs/issues/345
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-rs] alamb commented on issue #345: MIRI CI check fails intermittently with `thread 'main' panicked at 'invalid time'`
Posted by GitBox <gi...@apache.org>.
alamb commented on issue #345:
URL: https://github.com/apache/arrow-rs/issues/345#issuecomment-848666988
> I see that it's (almost?) always in bit_length_test_string. Did you consider skipping this test? Should we open an issue in miri to help us debug this? It might reveal something important about this test.
@roee88 I did not -- I tried to skip `array::array_primitive::tests::test_time32second_invalid_neg` in #346 but that did not seem to help. I don't have much time to devote to this issue at the moment (most of my arrow bandwidth is going towards getting the releases in shape) so any help here would be most appreciated
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-rs] roee88 commented on issue #345: MIRI CI check fails intermittently with `thread 'main' panicked at 'invalid time'`
Posted by GitBox <gi...@apache.org>.
roee88 commented on issue #345:
URL: https://github.com/apache/arrow-rs/issues/345#issuecomment-849055081
I think that the issue description here is wrong and this is expected for test_time32second_invalid_neg.
I did a quick memory usage check and identifier the following:
1. bit_length_test_string consumes on its own ~8g accumulated to a peak of ~10g that are not released after the test is done
2. bit_length_test_large_string has similar memory consumption but the memory is freed after the test is done
3. match_single_group consumes 2.7g accumulated to a peak of ~4.5G that are not released after the test is done
I tested ignoring bit_length_test_string and github actions checks pass (5 re-runs). I think it might be worth to ignore all three though. Some questions:
1. Is it possible that there is a memory leak in bit_length_test_string and match_single_group?
2. Should we disable bit_length_test_large_string just because it consumes a lot of RAM while running?
3. Should a support ticket be opened in Miri?
I will submit a PR based on the answers.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-rs] jorgecarleitao commented on issue #345: MIRI CI check fails intermittently with `thread 'main' panicked at 'invalid time'`
Posted by GitBox <gi...@apache.org>.
jorgecarleitao commented on issue #345:
URL: https://github.com/apache/arrow-rs/issues/345#issuecomment-849082680
Just to understand, the memory consumption is only when running the test via MIRI, right?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-rs] alamb commented on issue #345: MIRI CI check fails intermittently with `thread 'main' panicked at 'invalid time'`
Posted by GitBox <gi...@apache.org>.
alamb commented on issue #345:
URL: https://github.com/apache/arrow-rs/issues/345#issuecomment-849080451
Thank you @roee88
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-rs] alamb commented on issue #345: MIRI CI check fails intermittently with `thread 'main' panicked at 'invalid time'`
Posted by GitBox <gi...@apache.org>.
alamb commented on issue #345:
URL: https://github.com/apache/arrow-rs/issues/345#issuecomment-849086836
> Is it possible that there is a memory leak in bit_length_test_string and match_single_group?
I don't know but it sounds like it is worth more investigation
> Should we disable bit_length_test_large_string just because it consumes a lot of RAM while running?
I think we should disable the test under MIRI
> Should a support ticket be opened in MIRI?
If there is some bug or improvement you can articulate, I ams ure the MIRI developers would appreciate a ticket
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-rs] roee88 commented on issue #345: MIRI CI check fails intermittently with `thread 'main' panicked at 'invalid time'`
Posted by GitBox <gi...@apache.org>.
roee88 commented on issue #345:
URL: https://github.com/apache/arrow-rs/issues/345#issuecomment-848485806
I see that it's (almost?) always in bit_length_test_string. Did you consider skipping this test? Should we open an issue in miri to help us debug this? It might reveal something important about this test.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org