You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by Uwe Schindler <uw...@thetaphi.de> on 2023/11/08 18:48:51 UTC
Re: [JENKINS] Lucene-9.x-Linux (64bit/hotspot/jdk-20) - Build # 13968 - Unstable!
Hi,
this is caused by the change to better rethrow exception:
https://github.com/apache/lucene/pull/12707
The internals of MemorySegment's ScopedMemoryAccess throw
IllegalStateException("Already closed") when the memory segment was
closed by another thread.
Since the above PR, we check the exception type and rethrow it if the
extra check does not confirm that the Arena/MemorySession's scope.
Unfortunately it looks like isAlive() does sometimes still returns true,
when another thread has closes the Arena. The issue is that the check is
not volatile, so isAlive is just a informational method to check if an
Arena/MemorySession are alive. It may still return true.
We should maybe add another check to the alreadyClosed method to also
rethrow as AlreadyClosedException when the exception message equals
"Already closed". This is also not 100% safe, but works.
I will open a PR for Lucene 9.9 to improve the detection in
multithreaded code.
Uwe
Am 08.11.2023 um 17:43 schrieb Policeman Jenkins Server:
> Build: https://jenkins.thetaphi.de/job/Lucene-9.x-Linux/13968/
> Java: 64bit/hotspot/jdk-20 -XX:-UseCompressedOops -XX:+UseG1GC
>
> 1 tests failed.
> FAILED: org.apache.lucene.store.TestMmapDirectory.testAceWithThreads
>
> Error Message:
> com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=3961, name=Thread-3701, state=RUNNABLE, group=TGRP-TestMmapDirectory]
>
> Stack Trace:
> com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=3961, name=Thread-3701, state=RUNNABLE, group=TGRP-TestMmapDirectory]
> at __randomizedtesting.SeedInfo.seed([58DC34EC2F093F06:866507D737434A23]:0)
> Caused by: java.lang.IllegalStateException: Already closed
> at __randomizedtesting.SeedInfo.seed([58DC34EC2F093F06]:0)
> at java.base/jdk.internal.foreign.MemorySessionImpl.alreadyClosed(MemorySessionImpl.java:312)
> at java.base/jdk.internal.misc.ScopedMemoryAccess$ScopedAccessError.newRuntimeException(ScopedMemoryAccess.java:113)
> at java.base/jdk.internal.misc.ScopedMemoryAccess.copyMemory(ScopedMemoryAccess.java:131)
> at java.base/jdk.internal.foreign.AbstractMemorySegmentImpl.copy(AbstractMemorySegmentImpl.java:589)
> at java.base/java.lang.foreign.MemorySegment.copy(MemorySegment.java:2152)
> at org.apache.lucene.store.MemorySegmentIndexInput.readBytes(MemorySegmentIndexInput.java:146)
> at org.apache.lucene.store.TestMmapDirectory.lambda$testAceWithThreads$1(TestMmapDirectory.java:83)
> at java.base/java.lang.Thread.run(Thread.java:1623)
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: builds-unsubscribe@lucene.apache.org
> For additional commands, e-mail: builds-help@lucene.apache.org
--
Uwe Schindler
Achterdiek 19, D-28357 Bremen
https://www.thetaphi.de
eMail: uwe@thetaphi.de
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
Re: [JENKINS] Lucene-9.x-Linux (64bit/hotspot/jdk-20) - Build # 13968 - Unstable!
Posted by Uwe Schindler <uw...@thetaphi.de>.
See this PR: https://github.com/apache/lucene/pull/12785
The OpenJDK issue is: https://bugs.openjdk.org/browse/JDK-8319756
Uwe
Am 08.11.2023 um 20:31 schrieb Uwe Schindler:
> I have seen this error multiple times, so we should fix it before 9.9.
>
> Uwe
>
> Am 08.11.2023 um 19:48 schrieb Uwe Schindler:
>> Hi,
>>
>> this is caused by the change to better rethrow exception:
>> https://github.com/apache/lucene/pull/12707
>>
>> The internals of MemorySegment's ScopedMemoryAccess throw
>> IllegalStateException("Already closed") when the memory segment was
>> closed by another thread.
>>
>> Since the above PR, we check the exception type and rethrow it if the
>> extra check does not confirm that the Arena/MemorySession's scope.
>> Unfortunately it looks like isAlive() does sometimes still returns
>> true, when another thread has closes the Arena. The issue is that the
>> check is not volatile, so isAlive is just a informational method to
>> check if an Arena/MemorySession are alive. It may still return true.
>>
>> We should maybe add another check to the alreadyClosed method to also
>> rethrow as AlreadyClosedException when the exception message equals
>> "Already closed". This is also not 100% safe, but works.
>>
>> I will open a PR for Lucene 9.9 to improve the detection in
>> multithreaded code.
>>
>> Uwe
>>
>> Am 08.11.2023 um 17:43 schrieb Policeman Jenkins Server:
>>> Build: https://jenkins.thetaphi.de/job/Lucene-9.x-Linux/13968/
>>> Java: 64bit/hotspot/jdk-20 -XX:-UseCompressedOops -XX:+UseG1GC
>>>
>>> 1 tests failed.
>>> FAILED: org.apache.lucene.store.TestMmapDirectory.testAceWithThreads
>>>
>>> Error Message:
>>> com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured
>>> an uncaught exception in thread: Thread[id=3961, name=Thread-3701,
>>> state=RUNNABLE, group=TGRP-TestMmapDirectory]
>>>
>>> Stack Trace:
>>> com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured
>>> an uncaught exception in thread: Thread[id=3961, name=Thread-3701,
>>> state=RUNNABLE, group=TGRP-TestMmapDirectory]
>>> at
>>> __randomizedtesting.SeedInfo.seed([58DC34EC2F093F06:866507D737434A23]:0)
>>>
>>> Caused by: java.lang.IllegalStateException: Already closed
>>> at __randomizedtesting.SeedInfo.seed([58DC34EC2F093F06]:0)
>>> at
>>> java.base/jdk.internal.foreign.MemorySessionImpl.alreadyClosed(MemorySessionImpl.java:312)
>>> at
>>> java.base/jdk.internal.misc.ScopedMemoryAccess$ScopedAccessError.newRuntimeException(ScopedMemoryAccess.java:113)
>>> at
>>> java.base/jdk.internal.misc.ScopedMemoryAccess.copyMemory(ScopedMemoryAccess.java:131)
>>> at
>>> java.base/jdk.internal.foreign.AbstractMemorySegmentImpl.copy(AbstractMemorySegmentImpl.java:589)
>>> at
>>> java.base/java.lang.foreign.MemorySegment.copy(MemorySegment.java:2152)
>>> at
>>> org.apache.lucene.store.MemorySegmentIndexInput.readBytes(MemorySegmentIndexInput.java:146)
>>> at
>>> org.apache.lucene.store.TestMmapDirectory.lambda$testAceWithThreads$1(TestMmapDirectory.java:83)
>>> at java.base/java.lang.Thread.run(Thread.java:1623)
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: builds-unsubscribe@lucene.apache.org
>>> For additional commands, e-mail: builds-help@lucene.apache.org
>>
--
Uwe Schindler
Achterdiek 19, D-28357 Bremen
https://www.thetaphi.de
eMail: uwe@thetaphi.de
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
Re: [JENKINS] Lucene-9.x-Linux (64bit/hotspot/jdk-20) - Build # 13968 - Unstable!
Posted by Uwe Schindler <uw...@thetaphi.de>.
I have seen this error multiple times, so we should fix it before 9.9.
Uwe
Am 08.11.2023 um 19:48 schrieb Uwe Schindler:
> Hi,
>
> this is caused by the change to better rethrow exception:
> https://github.com/apache/lucene/pull/12707
>
> The internals of MemorySegment's ScopedMemoryAccess throw
> IllegalStateException("Already closed") when the memory segment was
> closed by another thread.
>
> Since the above PR, we check the exception type and rethrow it if the
> extra check does not confirm that the Arena/MemorySession's scope.
> Unfortunately it looks like isAlive() does sometimes still returns
> true, when another thread has closes the Arena. The issue is that the
> check is not volatile, so isAlive is just a informational method to
> check if an Arena/MemorySession are alive. It may still return true.
>
> We should maybe add another check to the alreadyClosed method to also
> rethrow as AlreadyClosedException when the exception message equals
> "Already closed". This is also not 100% safe, but works.
>
> I will open a PR for Lucene 9.9 to improve the detection in
> multithreaded code.
>
> Uwe
>
> Am 08.11.2023 um 17:43 schrieb Policeman Jenkins Server:
>> Build: https://jenkins.thetaphi.de/job/Lucene-9.x-Linux/13968/
>> Java: 64bit/hotspot/jdk-20 -XX:-UseCompressedOops -XX:+UseG1GC
>>
>> 1 tests failed.
>> FAILED: org.apache.lucene.store.TestMmapDirectory.testAceWithThreads
>>
>> Error Message:
>> com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured
>> an uncaught exception in thread: Thread[id=3961, name=Thread-3701,
>> state=RUNNABLE, group=TGRP-TestMmapDirectory]
>>
>> Stack Trace:
>> com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured
>> an uncaught exception in thread: Thread[id=3961, name=Thread-3701,
>> state=RUNNABLE, group=TGRP-TestMmapDirectory]
>> at
>> __randomizedtesting.SeedInfo.seed([58DC34EC2F093F06:866507D737434A23]:0)
>> Caused by: java.lang.IllegalStateException: Already closed
>> at __randomizedtesting.SeedInfo.seed([58DC34EC2F093F06]:0)
>> at
>> java.base/jdk.internal.foreign.MemorySessionImpl.alreadyClosed(MemorySessionImpl.java:312)
>> at
>> java.base/jdk.internal.misc.ScopedMemoryAccess$ScopedAccessError.newRuntimeException(ScopedMemoryAccess.java:113)
>> at
>> java.base/jdk.internal.misc.ScopedMemoryAccess.copyMemory(ScopedMemoryAccess.java:131)
>> at
>> java.base/jdk.internal.foreign.AbstractMemorySegmentImpl.copy(AbstractMemorySegmentImpl.java:589)
>> at
>> java.base/java.lang.foreign.MemorySegment.copy(MemorySegment.java:2152)
>> at
>> org.apache.lucene.store.MemorySegmentIndexInput.readBytes(MemorySegmentIndexInput.java:146)
>> at
>> org.apache.lucene.store.TestMmapDirectory.lambda$testAceWithThreads$1(TestMmapDirectory.java:83)
>> at java.base/java.lang.Thread.run(Thread.java:1623)
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: builds-unsubscribe@lucene.apache.org
>> For additional commands, e-mail: builds-help@lucene.apache.org
>
--
Uwe Schindler
Achterdiek 19, D-28357 Bremen
https://www.thetaphi.de
eMail: uwe@thetaphi.de
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org