You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@arrow.apache.org by "pitrou (via GitHub)" <gi...@apache.org> on 2023/05/22 16:47:38 UTC
[GitHub] [arrow] pitrou opened a new issue, #35712: [CI][C++] arrow-array-test slow on macOS
pitrou opened a new issue, #35712:
URL: https://github.com/apache/arrow/issues/35712
### Describe the bug, including details regarding any error messages, version, and platform.
This is rather minor, but `arrow-array-test` takes more than one minute on our C++ macOS CI. I even saw it sporadically time out.
It would be nice to find out the cause, as it may be an underlying performance problem in Arrow C++.
```
25/101 Test #1: arrow-array-test .......................... Passed 103.65 sec
```
### Component(s)
C++
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@arrow.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] mapleFU commented on issue #35712: [CI][C++] arrow-array-test slow on macOS
Posted by "mapleFU (via GitHub)" <gi...@apache.org>.
mapleFU commented on issue #35712:
URL: https://github.com/apache/arrow/issues/35712#issuecomment-1559383325
take
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] pitrou commented on issue #35712: [CI][C++] arrow-array-test slow on macOS
Posted by "pitrou (via GitHub)" <gi...@apache.org>.
pitrou commented on issue #35712:
URL: https://github.com/apache/arrow/issues/35712#issuecomment-1559363242
Thanks for the investigation @mapleFU ! I think we can just disable test on macOS, since other platforms don't seem to have that problem.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] mapleFU commented on issue #35712: [CI][C++] arrow-array-test slow on macOS
Posted by "mapleFU (via GitHub)" <gi...@apache.org>.
mapleFU commented on issue #35712:
URL: https://github.com/apache/arrow/issues/35712#issuecomment-1557586565
Hi, pitrou, on My MacOS with M1 Pro, test runs much more faster, here are some slow ones:
```
1: [ RUN ] TestDictionary.Validate
1: [ OK ] TestDictionary.Validate (8382 ms)
1: [ RUN ] TestArray.BuildLargeInMemoryArray
1: [ OK ] TestArray.BuildLargeInMemoryArray (2636 ms)
```
The whole tests are finished within 15secs... So maybe we can print the result in CI?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] pitrou closed issue #35712: [CI][C++] arrow-array-test slow on macOS
Posted by "pitrou (via GitHub)" <gi...@apache.org>.
pitrou closed issue #35712: [CI][C++] arrow-array-test slow on macOS
URL: https://github.com/apache/arrow/issues/35712
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] pitrou commented on issue #35712: [CI][C++] arrow-array-test slow on macOS
Posted by "pitrou (via GitHub)" <gi...@apache.org>.
pitrou commented on issue #35712:
URL: https://github.com/apache/arrow/issues/35712#issuecomment-1559251011
Can you validate on the macOS CI?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] mapleFU commented on issue #35712: [CI][C++] arrow-array-test slow on macOS
Posted by "mapleFU (via GitHub)" <gi...@apache.org>.
mapleFU commented on issue #35712:
URL: https://github.com/apache/arrow/issues/35712#issuecomment-1558539713
Hi pitrou, I've tried it here: https://github.com/apache/arrow/actions/runs/5053338938/jobs/9067090574?pr=35719
```
Top 20 slowest tests:
Test: TestDictionary 47785(ms)
Test: TestArray 764(ms)
Test: TestPrimitiveBuilder/12 63(ms)
Test: TestPrimitiveBuilder/3 54(ms)
Test: EncodedArrayTests/TestRunEndEncodedArray 52(ms)
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] mapleFU commented on issue #35712: [CI][C++] arrow-array-test slow on macOS
Posted by "mapleFU (via GitHub)" <gi...@apache.org>.
mapleFU commented on issue #35712:
URL: https://github.com/apache/arrow/issues/35712#issuecomment-1559249180
After remove ASSERT_DEATH:
```
Top 20 slowest tests:
Test: TestArray 756(ms)
Test: EncodedArrayTests/TestRunEndEncodedArray 78(ms)
Test: Decimal128Test/Decimal128Test 64(ms)
Test: TestPrimitiveBuilder/12 58(ms)
Test: TestPrimitiveBuilder/0 52(ms)
Test: TestDictionary 47(ms)
Test: TestPrimitiveBuilder/8 42(ms)
Test: TestPrimitiveBuilder/11 41(ms)
Test: TestStringDictionaryBuilder 39(ms)
Test: TestPrimitiveBuilder/4 39(ms)
Test: TestPrimitiveBuilder/6 38(ms)
Test: TestPrimitiveBuilder/10 35(ms)
Test: TestPrimitiveBuilder/7 34(ms)
Test: TestPrimitiveBuilder/3 34(ms)
Test: TestPrimitiveBuilder/2 32(ms)
Test: TestDecimal256DictionaryBuilder 31(ms)
Test: TestPrimitiveBuilder/1 31(ms)
Test: TestPrimitiveBuilder/9 30(ms)
Test: TestAdaptiveIntBuilder 29(ms)
Test: TestPrimitiveBuilder/5 29(ms)
~/work/arrow/arrow/build/cpp/src/arrow
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] mapleFU commented on issue #35712: [CI][C++] arrow-array-test slow on macOS
Posted by "mapleFU (via GitHub)" <gi...@apache.org>.
mapleFU commented on issue #35712:
URL: https://github.com/apache/arrow/issues/35712#issuecomment-1559383175
Okay, let me submit a quick fixing reuse that patch
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] mapleFU commented on issue #35712: [CI][C++] arrow-array-test slow on macOS
Posted by "mapleFU (via GitHub)" <gi...@apache.org>.
mapleFU commented on issue #35712:
URL: https://github.com/apache/arrow/issues/35712#issuecomment-1557604615
I'm glad to but it's a bit late in my timezone, if nobody tries it tonight, I'll have a try in the morning :-)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] mapleFU commented on issue #35712: [CI][C++] arrow-array-test slow on macOS
Posted by "mapleFU (via GitHub)" <gi...@apache.org>.
mapleFU commented on issue #35712:
URL: https://github.com/apache/arrow/issues/35712#issuecomment-1558712746
The test is here: https://github.com/apache/arrow/pull/35719
And I force the test failed, sort and output the testing time. I've no idea why `TestDictionary` slow, maybe I can take a look tonight
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] pitrou commented on issue #35712: [CI][C++] arrow-array-test slow on macOS
Posted by "pitrou (via GitHub)" <gi...@apache.org>.
pitrou commented on issue #35712:
URL: https://github.com/apache/arrow/issues/35712#issuecomment-1559156824
Also cc @felipecrv
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] pitrou commented on issue #35712: [CI][C++] arrow-array-test slow on macOS
Posted by "pitrou (via GitHub)" <gi...@apache.org>.
pitrou commented on issue #35712:
URL: https://github.com/apache/arrow/issues/35712#issuecomment-1557591658
Perhaps you can submit a draft PR to run the tests in verbose mode indeed.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] mapleFU commented on issue #35712: [CI][C++] arrow-array-test slow on macOS
Posted by "mapleFU (via GitHub)" <gi...@apache.org>.
mapleFU commented on issue #35712:
URL: https://github.com/apache/arrow/issues/35712#issuecomment-1559193457
I guess I've found out the reason. Maybe it's `ASSERT_DEATH` making testing slowly:
```c++
ASSERT_DEATH(
{
std::shared_ptr<Array> null_dict_arr =
std::make_shared<DictionaryArray>(dict_type, indices, nullptr);
},
"");
```
In `TestDictionary.Validate` @pitrou . Can I remove this if I can confirm it?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] mapleFU commented on issue #35712: [CI][C++] arrow-array-test slow on macOS
Posted by "mapleFU (via GitHub)" <gi...@apache.org>.
mapleFU commented on issue #35712:
URL: https://github.com/apache/arrow/issues/35712#issuecomment-1559254017
Already done, see https://github.com/apache/arrow/actions/runs/5057218042/jobs/9075698907?pr=35719 and https://github.com/apache/arrow/issues/35712#issuecomment-1559249180 @pitrou
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] pitrou commented on issue #35712: [CI][C++] arrow-array-test slow on macOS
Posted by "pitrou (via GitHub)" <gi...@apache.org>.
pitrou commented on issue #35712:
URL: https://github.com/apache/arrow/issues/35712#issuecomment-1558688552
> Seems that we should optimize `TestDictionary`?
Definitely!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org