You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@lucene.apache.org by GitBox <gi...@apache.org> on 2022/11/18 17:19:37 UTC
[GitHub] [lucene] rmuir opened a new issue, #11951: TestStressIndexing can sometime take minutes
rmuir opened a new issue, #11951:
URL: https://github.com/apache/lucene/issues/11951
### Description
I've seen this happen several times, so i think it may not be hard to reproduce, have not tried to use the seed yet:
```
> Task :randomizationInfo
Running tests with randomization seed: tests.seed=6D4E6284011DCBC9
...
The slowest tests (exceeding 500 ms) during this run:
212.58s TestStressIndexing.testStressIndexAndSearching (:lucene:core)
```
### Gradle command to reproduce
_No response_
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org
[GitHub] [lucene] rmuir commented on issue #11951: TestStressIndexing can sometime take minutes
Posted by GitBox <gi...@apache.org>.
rmuir commented on issue #11951:
URL: https://github.com/apache/lucene/issues/11951#issuecomment-1320945227
I can conditionalize the crazy virus-checker+fsdir stuff to `TEST_NIGHTLY` and just use ByteBuffersDirectory otherwise, but even after that fix i still find sluggish seeds (e.g. 15s) easily. so I'm digging into this one a bit more.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org
[GitHub] [lucene] rmuir commented on issue #11951: TestStressIndexing can sometime take minutes
Posted by GitBox <gi...@apache.org>.
rmuir commented on issue #11951:
URL: https://github.com/apache/lucene/issues/11951#issuecomment-1320942537
Looks to me like it may be doing way too many actual filesystem operations? Or the profiler is just lying to us :)
```
./gradlew -p lucene/core test --tests TestStressIndexing -Dtests.seed=6D4E6284011DCBC9 -Ptests.profile=true -Ptests.profile.stacksize=4
> Task :lucene:core:test
WARNING: A command line option has enabled the Security Manager
WARNING: The Security Manager is deprecated and will be removed in a future release
[0.394s][info][jfr,startup] Started recording 1.
[0.394s][info][jfr,startup]
[0.394s][info][jfr,startup] Use jcmd 18175 JFR.dump name=1 filename=FILEPATH to copy recording data to file.
:lucene:core:test (SUCCESS): 1 test(s)
The slowest tests (exceeding 500 ms) during this run:
127.62s TestStressIndexing.testStressIndexAndSearching (:lucene:core)
The slowest suites (exceeding 1s) during this run:
127.97s TestStressIndexing (:lucene:core)
PROFILE SUMMARY from 70032 events (total: 70032)
tests.profile.mode=cpu
tests.profile.count=10
tests.profile.stacksize=4
tests.profile.linenumbers=false
PERCENT CPU SAMPLES STACK
15.55% 10892 sun.nio.fs.UnixPath#initOffsets()
at sun.nio.fs.UnixPath#getFileName()
at sun.nio.fs.UnixPath#getFileName()
at org.apache.lucene.tests.mockfile.FilterPath#getFileName()
10.56% 7395 sun.nio.fs.UnixNativeDispatcher#readdir()
at sun.nio.fs.UnixDirectoryStream$UnixDirectoryIterator#readNextEntry()
at sun.nio.fs.UnixDirectoryStream$UnixDirectoryIterator#hasNext()
at org.apache.lucene.tests.mockfile.FilterDirectoryStream$1#hasNext()
10.33% 7236 sun.nio.fs.UnixPath#getFileName()
at sun.nio.fs.UnixPath#getFileName()
at org.apache.lucene.tests.mockfile.FilterPath#getFileName()
at org.apache.lucene.tests.mockfile.FilterPath#getFileName()
7.61% 5326 java.lang.StringCoding#hasNegatives()
at java.lang.String#<init>()
at java.lang.String#<init>()
at sun.nio.fs.Util#toString()
5.55% 3890 java.lang.StringLatin1#compareTo()
at java.lang.StringLatin1#compareTo()
at java.lang.String#compareTo()
at java.lang.String#compareTo()
4.05% 2837 sun.nio.fs.UnixPath#toString()
at org.apache.lucene.tests.mockfile.FilterPath#toString()
at org.apache.lucene.tests.mockfile.FilterPath#toString()
at org.apache.lucene.tests.mockfile.ShuffleFS#lambda$newDirectoryStream$0()
3.10% 2170 java.util.Arrays#copyOfRange()
at java.lang.String#<init>()
at java.lang.String#<init>()
at sun.nio.fs.Util#toString()
3.05% 2135 java.lang.StringLatin1#compareTo()
at java.lang.StringLatin1#compareTo()
at java.lang.String#compareTo()
at org.apache.lucene.tests.mockfile.ShuffleFS#lambda$newDirectoryStream$0()
3.00% 2103 java.lang.String#<init>()
at java.lang.String#<init>()
at sun.nio.fs.Util#toString()
at sun.nio.fs.UnixPath#toString()
1.86% 1300 java.util.ComparableTimSort#binarySort()
at java.util.ComparableTimSort#sort()
at java.util.Arrays#sort()
at org.apache.lucene.store.FSDirectory#listAll()
BUILD SUCCESSFUL in 2m 25s
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org
[GitHub] [lucene] rmuir closed issue #11951: TestStressIndexing can sometime take minutes
Posted by GitBox <gi...@apache.org>.
rmuir closed issue #11951: TestStressIndexing can sometime take minutes
URL: https://github.com/apache/lucene/issues/11951
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org