You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@arrow.apache.org by "bmwiedemann (via GitHub)" <gi...@apache.org> on 2023/08/21 13:44:41 UTC
[GitHub] [arrow] bmwiedemann opened a new issue, #37276: Tests fail on 1-core VM
bmwiedemann opened a new issue, #37276:
URL: https://github.com/apache/arrow/issues/37276
### Describe the bug, including details regarding any error messages, version, and platform.
While working on [reproducible builds](https://reproducible-builds.org/) for [openSUSE](https://en.opensuse.org/openSUSE:Reproducible_Builds), I found that
our `apache-arrow-12.0.1` package failed tests when run on a 1-core VM:
Here is an extract from the log:
```
/home/abuild/rpmbuild/BUILD/arrow-apache-arrow-12.0.1/cpp/src/arrow/csv/reader_test.cc:190: Failure
Expected equality of these values:
-1
row.number
Which is: 961
/home/abuild/rpmbuild/BUILD/arrow-apache-arrow-12.0.1/cpp/src/arrow/csv/reader_test.cc:190: Failure
Expected equality of these values:
-1
row.number
Which is: 981
/home/abuild/rpmbuild/BUILD/arrow-apache-arrow-12.0.1/cpp/src/arrow/csv/reader_test.cc:190: Failure
Expected equality of these values:
-1
row.number
Which is: 1001
[ FAILED ] StreamingReaderTests.InvalidRowsSkippedAsync (2 ms)
[ RUN ] StreamingReaderTests.BytesRead
[ OK ] StreamingReaderTests.BytesRead (0 ms)
[==========] 253 tests from 44 test suites ran. (978 ms total)
[ PASSED ] 252 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] StreamingReaderTests.InvalidRowsSkippedAsync
1 FAILED TEST
~/rpmbuild/BUILD/arrow-apache-arrow-12.0.1/cpp/build/src/arrow/csv
Start 39: arrow-acero-plan-test
39/78 Test #39: arrow-acero-plan-test ....................***Failed 1.78 sec
Running arrow-acero-plan-test, redirecting output into /home/abuild/rpmbuild/BUILD/arrow-apache-arrow-12.0.1/cpp/build/build/test-logs/arrow-acero-plan-test.txt (attempt 1/1)
Running main() from /home/abuild/rpmbuild/BUILD/googletest-release-1.12.1/googletest/src/gtest_main.cc
[ RUN ] ExecPlanExecution.SegmentedAggregationWithMultiThreading
/home/abuild/rpmbuild/BUILD/arrow-apache-arrow-12.0.1/cpp/src/arrow/acero/plan_test.cc:1615: Failure
Value of: _st.IsNotImplemented()
Actual: false
Expected: true
Expected 'DeclarationToExecBatches(std::move(plan))' to fail with NotImplemented, but got Invalid: Group-by aggregation with field 'i32' as both key and segment key
/home/abuild/rpmbuild/BUILD/arrow-apache-arrow-12.0.1/cpp/src/arrow/acero/plan_test.cc:1615: Failure
Value of: _st.ToString()
Expected: has substring "multi-threaded"
Actual: "Invalid: Group-by aggregation with field 'i32' as both key and segment key"
[ FAILED ] ExecPlanExecution.SegmentedAggregationWithMultiThreading (0 ms)
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20390831 20:38:32.254076 25012 source_node.cc:76] An input buffer was poorly aligned. This could lead to crashes or poor performance on some hardware. Please ensure that all Acero sources generate aligned buffers, or change the unaligned buffer handling configuration to silence this warning.
[ OK ] ExecPlanExecution.UnalignedInput (0 ms)
[----------] 50 tests from ExecPlanExecution (1584 ms total)
[----------] Global test environment tear-down
[==========] 60 tests from 4 test suites ran. (1716 ms total)
[ PASSED ] 59 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] ExecPlanExecution.SegmentedAggregationWithMultiThreading
1 FAILED TEST
~/rpmbuild/BUILD/arrow-apache-arrow-12.0.1/cpp/build/src/arrow/acero
```
Steps to reproduce on Debian or openSUSE:
```bash
osc co openSUSE:Factory/apache-arrow && cd $_
osc build --vm-type=kvm --noservice -j1 standard
```
### Component(s)
C++, Other
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@arrow.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] pitrou commented on issue #37276: Tests fail on 1-core VM
Posted by "pitrou (via GitHub)" <gi...@apache.org>.
pitrou commented on issue #37276:
URL: https://github.com/apache/arrow/issues/37276#issuecomment-1689527619
Ok, so these are the same errors that can be reproduced locally as described [above](https://github.com/apache/arrow/issues/37276#issuecomment-1688443943).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] pitrou closed issue #37276: Tests fail on 1-core VM
Posted by "pitrou (via GitHub)" <gi...@apache.org>.
pitrou closed issue #37276: Tests fail on 1-core VM
URL: https://github.com/apache/arrow/issues/37276
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] js8544 commented on issue #37276: Tests fail on 1-core VM
Posted by "js8544 (via GitHub)" <gi...@apache.org>.
js8544 commented on issue #37276:
URL: https://github.com/apache/arrow/issues/37276#issuecomment-1689579326
Both tests assume a multi-threaded environment without checking the actual threading capacity. Given that they intend to check the behavior of multi-threaded execution, I think they should be skipped if there is only one thread available:
```cpp
if (internal::GetCpuThreadPool()->GetCapacity() < 2) {
GTEST_SKIP() << "Test requires at least 2 threads";
}
```
The above patch works on my machine. Should I submit a PR for this? @pitrou
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] js8544 commented on issue #37276: Tests fail on 1-core VM
Posted by "js8544 (via GitHub)" <gi...@apache.org>.
js8544 commented on issue #37276:
URL: https://github.com/apache/arrow/issues/37276#issuecomment-1689614885
> In principle you can run multiple threads on a single CPU through the kernel scheduling one part after another. But that might already be covered in a different test, then skipping should be fine.
True. These two tests use the default capacity which is infered with [this logic](https://github.com/apache/arrow/blob/main/cpp/src/arrow/util/thread_pool.cc#L705). We can also force them to use a custom threadpool with multiple threads, which will need some refactoring. I'm not sure if it's worth it though.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] bmwiedemann commented on issue #37276: Tests fail on 1-core VM
Posted by "bmwiedemann (via GitHub)" <gi...@apache.org>.
bmwiedemann commented on issue #37276:
URL: https://github.com/apache/arrow/issues/37276#issuecomment-1689594904
In principle you can run multiple threads on a single CPU through the kernel scheduling one part after another. But that might already be covered in a different test, then skipping should be fine.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] pitrou commented on issue #37276: Tests fail on 1-core VM
Posted by "pitrou (via GitHub)" <gi...@apache.org>.
pitrou commented on issue #37276:
URL: https://github.com/apache/arrow/issues/37276#issuecomment-1688444492
Also @bmwiedemann could you give a link to the full test logs?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] pitrou commented on issue #37276: Tests fail on 1-core VM
Posted by "pitrou (via GitHub)" <gi...@apache.org>.
pitrou commented on issue #37276:
URL: https://github.com/apache/arrow/issues/37276#issuecomment-1688431940
cc @westonpace
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] pitrou commented on issue #37276: Tests fail on 1-core VM
Posted by "pitrou (via GitHub)" <gi...@apache.org>.
pitrou commented on issue #37276:
URL: https://github.com/apache/arrow/issues/37276#issuecomment-1689618732
> We can also force them to use a custom threadpool with multiple threads, which will need some refactoring. I'm not sure if it's worth it though.
Probably not. It's interesting that no public CI platform seems to provide single-CPU VMs then.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] pitrou commented on issue #37276: Tests fail on 1-core VM
Posted by "pitrou (via GitHub)" <gi...@apache.org>.
pitrou commented on issue #37276:
URL: https://github.com/apache/arrow/issues/37276#issuecomment-1688443943
It seems `OMP_THREAD_LIMIT=1 taskset -c 0 <test command>` should be enough to reproduce locally without relying on the OpenSUSE build infrastructure.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] bmwiedemann commented on issue #37276: Tests fail on 1-core VM
Posted by "bmwiedemann (via GitHub)" <gi...@apache.org>.
bmwiedemann commented on issue #37276:
URL: https://github.com/apache/arrow/issues/37276#issuecomment-1689520514
[full build+test log](https://rb.zq1.de/temp/apache-arrow-log.txt)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org