You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by Erick Erickson <er...@gmail.com> on 2018/09/04 15:02:51 UTC
SolrMetricReporterTest failures
Locally, this test fails 100% of the time but not on Jenkins and I'm
clueless. This is on a current pull of master. I think it's related to
SOLR-12055: make loggers async by default.
The _test_ succeeds, but the _run_ fails with a leaked thread. The
stack trace mentions "disruptor" which is definitely the async change.
Plus, if I revert log4j2.xml in test-files to synch logging,
everything's fine.
Anyone have a clue where to even start looking? Or why this isn't
failing on Jenkins? I've cleaned my ivy cache, re-cloned master and it
still fails.
Partial stack trace:
SEVERE: 1 thread leaked from SUITE scope at
org.apache.solr.metrics.SolrMetricReporterTest:
1) Thread[id=14, name=Log4j2-TF-1-AsyncLoggerConfig--1,
state=TIMED_WAITING, group=TGRP-SolrMetricReporterTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
at com.lmax.disruptor.TimeoutBlockingWaitStrategy.waitFor(TimeoutBlockingWaitStrategy.java:38)
at com.lmax.disruptor.ProcessingSequenceBarrier.waitFor(ProcessingSequenceBarrier.java:56)
at com.lmax.disruptor.BatchEventProcessor.run(BatchEventProcessor.java:128)
at java.lang.Thread.run(Thread.java:748)
Meanwhile I'm digging....
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
Re: SolrMetricReporterTest failures
Posted by Erick Erickson <er...@gmail.com>.
Steve:
Thanks at least I'm reassured that this isn't something weird in my
local environment. Still clueless why it doesn't seem to be happening
on Jenkins, and Hoss' rollups don't show it failing either. Digging...
On Tue, Sep 4, 2018 at 8:34 AM Steve Rowe <sa...@gmail.com> wrote:
>
> 20/20 beasting iterations failed for me.
>
> --
> Steve
> www.lucidworks.com
>
> > On Sep 4, 2018, at 11:22 AM, Steve Rowe <sa...@gmail.com> wrote:
> >
> > I’ll run the test locally to see if I can reproduce the failures.
> >
> > --
> > Steve
> > www.lucidworks.com
> >
> >> On Sep 4, 2018, at 11:02 AM, Erick Erickson <er...@gmail.com> wrote:
> >>
> >> Locally, this test fails 100% of the time but not on Jenkins and I'm
> >> clueless. This is on a current pull of master. I think it's related to
> >> SOLR-12055: make loggers async by default.
> >>
> >> The _test_ succeeds, but the _run_ fails with a leaked thread. The
> >> stack trace mentions "disruptor" which is definitely the async change.
> >> Plus, if I revert log4j2.xml in test-files to synch logging,
> >> everything's fine.
> >>
> >> Anyone have a clue where to even start looking? Or why this isn't
> >> failing on Jenkins? I've cleaned my ivy cache, re-cloned master and it
> >> still fails.
> >>
> >> Partial stack trace:
> >> SEVERE: 1 thread leaked from SUITE scope at
> >> org.apache.solr.metrics.SolrMetricReporterTest:
> >> 1) Thread[id=14, name=Log4j2-TF-1-AsyncLoggerConfig--1,
> >> state=TIMED_WAITING, group=TGRP-SolrMetricReporterTest]
> >> at sun.misc.Unsafe.park(Native Method)
> >> at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
> >> at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
> >> at com.lmax.disruptor.TimeoutBlockingWaitStrategy.waitFor(TimeoutBlockingWaitStrategy.java:38)
> >> at com.lmax.disruptor.ProcessingSequenceBarrier.waitFor(ProcessingSequenceBarrier.java:56)
> >> at com.lmax.disruptor.BatchEventProcessor.run(BatchEventProcessor.java:128)
> >> at java.lang.Thread.run(Thread.java:748)
> >>
> >> Meanwhile I'm digging....
> >>
> >> ---------------------------------------------------------------------
> >> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> >> For additional commands, e-mail: dev-help@lucene.apache.org
> >>
> >
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: dev-help@lucene.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
Re: SolrMetricReporterTest failures
Posted by Steve Rowe <sa...@gmail.com>.
20/20 beasting iterations failed for me.
--
Steve
www.lucidworks.com
> On Sep 4, 2018, at 11:22 AM, Steve Rowe <sa...@gmail.com> wrote:
>
> I’ll run the test locally to see if I can reproduce the failures.
>
> --
> Steve
> www.lucidworks.com
>
>> On Sep 4, 2018, at 11:02 AM, Erick Erickson <er...@gmail.com> wrote:
>>
>> Locally, this test fails 100% of the time but not on Jenkins and I'm
>> clueless. This is on a current pull of master. I think it's related to
>> SOLR-12055: make loggers async by default.
>>
>> The _test_ succeeds, but the _run_ fails with a leaked thread. The
>> stack trace mentions "disruptor" which is definitely the async change.
>> Plus, if I revert log4j2.xml in test-files to synch logging,
>> everything's fine.
>>
>> Anyone have a clue where to even start looking? Or why this isn't
>> failing on Jenkins? I've cleaned my ivy cache, re-cloned master and it
>> still fails.
>>
>> Partial stack trace:
>> SEVERE: 1 thread leaked from SUITE scope at
>> org.apache.solr.metrics.SolrMetricReporterTest:
>> 1) Thread[id=14, name=Log4j2-TF-1-AsyncLoggerConfig--1,
>> state=TIMED_WAITING, group=TGRP-SolrMetricReporterTest]
>> at sun.misc.Unsafe.park(Native Method)
>> at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
>> at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
>> at com.lmax.disruptor.TimeoutBlockingWaitStrategy.waitFor(TimeoutBlockingWaitStrategy.java:38)
>> at com.lmax.disruptor.ProcessingSequenceBarrier.waitFor(ProcessingSequenceBarrier.java:56)
>> at com.lmax.disruptor.BatchEventProcessor.run(BatchEventProcessor.java:128)
>> at java.lang.Thread.run(Thread.java:748)
>>
>> Meanwhile I'm digging....
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
>> For additional commands, e-mail: dev-help@lucene.apache.org
>>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
Re: SolrMetricReporterTest failures
Posted by Steve Rowe <sa...@gmail.com>.
I’ll run the test locally to see if I can reproduce the failures.
--
Steve
www.lucidworks.com
> On Sep 4, 2018, at 11:02 AM, Erick Erickson <er...@gmail.com> wrote:
>
> Locally, this test fails 100% of the time but not on Jenkins and I'm
> clueless. This is on a current pull of master. I think it's related to
> SOLR-12055: make loggers async by default.
>
> The _test_ succeeds, but the _run_ fails with a leaked thread. The
> stack trace mentions "disruptor" which is definitely the async change.
> Plus, if I revert log4j2.xml in test-files to synch logging,
> everything's fine.
>
> Anyone have a clue where to even start looking? Or why this isn't
> failing on Jenkins? I've cleaned my ivy cache, re-cloned master and it
> still fails.
>
> Partial stack trace:
> SEVERE: 1 thread leaked from SUITE scope at
> org.apache.solr.metrics.SolrMetricReporterTest:
> 1) Thread[id=14, name=Log4j2-TF-1-AsyncLoggerConfig--1,
> state=TIMED_WAITING, group=TGRP-SolrMetricReporterTest]
> at sun.misc.Unsafe.park(Native Method)
> at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
> at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
> at com.lmax.disruptor.TimeoutBlockingWaitStrategy.waitFor(TimeoutBlockingWaitStrategy.java:38)
> at com.lmax.disruptor.ProcessingSequenceBarrier.waitFor(ProcessingSequenceBarrier.java:56)
> at com.lmax.disruptor.BatchEventProcessor.run(BatchEventProcessor.java:128)
> at java.lang.Thread.run(Thread.java:748)
>
> Meanwhile I'm digging....
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: dev-help@lucene.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org