You are viewing a plain text version of this content. The canonical link for it is here.
Posted to builds@lucene.apache.org by Apache Jenkins Server <je...@builds.apache.org> on 2020/02/19 06:33:45 UTC
[JENKINS] Lucene-Solr-NightlyTests-master - Build # 2108 - Failure
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-master/2108/
All tests passed
Build Log:
[...truncated 67936 lines...]
BUILD FAILED
/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/build.xml:653: The following error occurred while executing this line:
/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/build.xml:101: The following error occurred while executing this line:
/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build.xml:625: The following error occurred while executing this line:
/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build.xml:157: The following error occurred while executing this line:
/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/solr-ref-guide/build.xml:289: Java returned: 255
Total time: 286 minutes 40 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any
[JENKINS] Lucene-Solr-NightlyTests-master - Build # 2111 - Still
Unstable
Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-master/2111/
1 tests failed.
FAILED: org.apache.solr.cloud.hdfs.HdfsSyncSliceTest.test
Error Message:
expected:<5> but was:<4>
Stack Trace:
java.lang.AssertionError: expected:<5> but was:<4>
at __randomizedtesting.SeedInfo.seed([B17B537CC2F9B000:392F6CA66C05DDF8]:0)
at org.junit.Assert.fail(Assert.java:88)
at org.junit.Assert.failNotEquals(Assert.java:834)
at org.junit.Assert.assertEquals(Assert.java:645)
at org.junit.Assert.assertEquals(Assert.java:631)
at org.apache.solr.cloud.SyncSliceTest.test(SyncSliceTest.java:150)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:566)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1754)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:942)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:978)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:992)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:1082)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:1054)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:370)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:819)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:470)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:951)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:836)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:887)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:898)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:370)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.lambda$forkTimeoutingTask$0(ThreadLeakControl.java:826)
at java.base/java.lang.Thread.run(Thread.java:834)
Build Log:
[...truncated 14919 lines...]
[junit4] Suite: org.apache.solr.cloud.hdfs.HdfsSyncSliceTest
[junit4] 2> 4487230 INFO (SUITE-HdfsSyncSliceTest-seed#[B17B537CC2F9B000]-worker) [ ] o.a.s.SolrTestCase Setting 'solr.default.confdir' system property to test-framework derived value of '/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/server/solr/configsets/_default/conf'
[junit4] 2> 4487230 INFO (SUITE-HdfsSyncSliceTest-seed#[B17B537CC2F9B000]-worker) [ ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
[junit4] 2> 4487230 INFO (SUITE-HdfsSyncSliceTest-seed#[B17B537CC2F9B000]-worker) [ ] o.a.s.SolrTestCaseJ4 Created dataDir: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsSyncSliceTest_B17B537CC2F9B000-001/data-dir-221-001
[junit4] 2> 4487230 WARN (SUITE-HdfsSyncSliceTest-seed#[B17B537CC2F9B000]-worker) [ ] o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=20 numCloses=20
[junit4] 2> 4487230 INFO (SUITE-HdfsSyncSliceTest-seed#[B17B537CC2F9B000]-worker) [ ] o.a.s.SolrTestCaseJ4 Using TrieFields (NUMERIC_POINTS_SYSPROP=false) w/NUMERIC_DOCVALUES_SYSPROP=true
[junit4] 2> 4487231 INFO (SUITE-HdfsSyncSliceTest-seed#[B17B537CC2F9B000]-worker) [ ] o.a.s.SolrTestCaseJ4 Randomized ssl (true) and clientAuth (true) via: @org.apache.solr.util.RandomizeSSL(reason="", value=0.0/0.0, ssl=0.0/0.0, clientAuth=0.0/0.0)
[junit4] 2> 4487232 INFO (SUITE-HdfsSyncSliceTest-seed#[B17B537CC2F9B000]-worker) [ ] o.a.s.BaseDistributedSearchTestCase Setting hostContext system property: /
[junit4] 1> Formatting using clusterid: testClusterID
[junit4] 2> 4487292 WARN (SUITE-HdfsSyncSliceTest-seed#[B17B537CC2F9B000]-worker) [ ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
[junit4] 2> 4487294 INFO (SUITE-HdfsSyncSliceTest-seed#[B17B537CC2F9B000]-worker) [ ] o.e.j.s.Server jetty-9.4.24.v20191120; built: 2019-11-20T21:37:49.771Z; git: 363d5f2df3a8a28de40604320230664b9c793c16; jvm 11.0.4+10-LTS
[junit4] 2> 4487295 INFO (SUITE-HdfsSyncSliceTest-seed#[B17B537CC2F9B000]-worker) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 4487295 INFO (SUITE-HdfsSyncSliceTest-seed#[B17B537CC2F9B000]-worker) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 4487295 INFO (SUITE-HdfsSyncSliceTest-seed#[B17B537CC2F9B000]-worker) [ ] o.e.j.s.session node0 Scavenging every 660000ms
[junit4] 2> 4487295 INFO (SUITE-HdfsSyncSliceTest-seed#[B17B537CC2F9B000]-worker) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@a0d58a2{static,/static,jar:file:/home/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-3.2.0-tests.jar!/webapps/static,AVAILABLE}
[junit4] 2> 4487449 INFO (SUITE-HdfsSyncSliceTest-seed#[B17B537CC2F9B000]-worker) [ ] o.e.j.s.h.ContextHandler Started o.e.j.w.WebAppContext@5d0297f3{hdfs,/,file:///home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J2/temp/jetty-localhost_localdomain-35887-hadoop-hdfs-3_2_0-tests_jar-_-any-5368395901943143748.dir/webapp/,AVAILABLE}{jar:file:/home/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-3.2.0-tests.jar!/webapps/hdfs}
[junit4] 2> 4487449 INFO (SUITE-HdfsSyncSliceTest-seed#[B17B537CC2F9B000]-worker) [ ] o.e.j.s.AbstractConnector Started ServerConnector@578493af{HTTP/1.1,[http/1.1]}{localhost.localdomain:35887}
[junit4] 2> 4487449 INFO (SUITE-HdfsSyncSliceTest-seed#[B17B537CC2F9B000]-worker) [ ] o.e.j.s.Server Started @4487486ms
[junit4] 2> 4487507 WARN (SUITE-HdfsSyncSliceTest-seed#[B17B537CC2F9B000]-worker) [ ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
[junit4] 2> 4487508 INFO (SUITE-HdfsSyncSliceTest-seed#[B17B537CC2F9B000]-worker) [ ] o.e.j.s.Server jetty-9.4.24.v20191120; built: 2019-11-20T21:37:49.771Z; git: 363d5f2df3a8a28de40604320230664b9c793c16; jvm 11.0.4+10-LTS
[junit4] 2> 4487510 INFO (SUITE-HdfsSyncSliceTest-seed#[B17B537CC2F9B000]-worker) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 4487510 INFO (SUITE-HdfsSyncSliceTest-seed#[B17B537CC2F9B000]-worker) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 4487510 INFO (SUITE-HdfsSyncSliceTest-seed#[B17B537CC2F9B000]-worker) [ ] o.e.j.s.session node0 Scavenging every 600000ms
[junit4] 2> 4487510 INFO (SUITE-HdfsSyncSliceTest-seed#[B17B537CC2F9B000]-worker) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@4fff8a02{static,/static,jar:file:/home/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-3.2.0-tests.jar!/webapps/static,AVAILABLE}
[junit4] 2> 4487629 INFO (SUITE-HdfsSyncSliceTest-seed#[B17B537CC2F9B000]-worker) [ ] o.e.j.s.h.ContextHandler Started o.e.j.w.WebAppContext@3e6b8a27{datanode,/,file:///home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J2/temp/jetty-localhost-36295-hadoop-hdfs-3_2_0-tests_jar-_-any-12697710436294043592.dir/webapp/,AVAILABLE}{jar:file:/home/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-3.2.0-tests.jar!/webapps/datanode}
[junit4] 2> 4487629 INFO (SUITE-HdfsSyncSliceTest-seed#[B17B537CC2F9B000]-worker) [ ] o.e.j.s.AbstractConnector Started ServerConnector@61c41385{HTTP/1.1,[http/1.1]}{localhost:36295}
[junit4] 2> 4487629 INFO (SUITE-HdfsSyncSliceTest-seed#[B17B537CC2F9B000]-worker) [ ] o.e.j.s.Server Started @4487666ms
[junit4] 2> 4487766 INFO (Block report processor) [ ] BlockStateChange BLOCK* processReport 0x5ae9a75e2d6a2bc5: Processing first storage report for DS-37604381-eafc-42ea-9bc8-b8670b8ab45a from datanode 83f6d46f-5e34-4418-9dbd-b9676bf8fdec
[junit4] 2> 4487766 INFO (Block report processor) [ ] BlockStateChange BLOCK* processReport 0x5ae9a75e2d6a2bc5: from storage DS-37604381-eafc-42ea-9bc8-b8670b8ab45a node DatanodeRegistration(127.0.0.1:45461, datanodeUuid=83f6d46f-5e34-4418-9dbd-b9676bf8fdec, infoPort=38735, infoSecurePort=0, ipcPort=35043, storageInfo=lv=-57;cid=testClusterID;nsid=1426251164;c=1582354540363), blocks: 0, hasStaleStorage: true, processing time: 0 msecs, invalidatedBlocks: 0
[junit4] 2> 4487766 INFO (Block report processor) [ ] BlockStateChange BLOCK* processReport 0x5ae9a75e2d6a2bc5: Processing first storage report for DS-db09cbef-8e30-4914-89f5-8ec452d37f87 from datanode 83f6d46f-5e34-4418-9dbd-b9676bf8fdec
[junit4] 2> 4487766 INFO (Block report processor) [ ] BlockStateChange BLOCK* processReport 0x5ae9a75e2d6a2bc5: from storage DS-db09cbef-8e30-4914-89f5-8ec452d37f87 node DatanodeRegistration(127.0.0.1:45461, datanodeUuid=83f6d46f-5e34-4418-9dbd-b9676bf8fdec, infoPort=38735, infoSecurePort=0, ipcPort=35043, storageInfo=lv=-57;cid=testClusterID;nsid=1426251164;c=1582354540363), blocks: 0, hasStaleStorage: false, processing time: 0 msecs, invalidatedBlocks: 0
[junit4] 2> 4487856 INFO (TEST-HdfsSyncSliceTest.test-seed#[B17B537CC2F9B000]) [ ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
[junit4] 2> 4487856 INFO (ZkTestServer Run Thread) [ ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 4487856 INFO (ZkTestServer Run Thread) [ ] o.a.s.c.ZkTestServer Starting server
[junit4] 2> 4487956 INFO (TEST-HdfsSyncSliceTest.test-seed#[B17B537CC2F9B000]) [ ] o.a.s.c.ZkTestServer start zk server on port:43667
[junit4] 2> 4487956 INFO (TEST-HdfsSyncSliceTest.test-seed#[B17B537CC2F9B000]) [ ] o.a.s.c.ZkTestServer waitForServerUp: 127.0.0.1:43667
[junit4] 2> 4487956 INFO (TEST-HdfsSyncSliceTest.test-seed#[B17B537CC2F9B000]) [ ] o.a.s.c.ZkTestServer parse host and port list: 127.0.0.1:43667
[junit4] 2> 4487956 INFO (TEST-HdfsSyncSliceTest.test-seed#[B17B537CC2F9B000]) [ ] o.a.s.c.ZkTestServer connecting to 127.0.0.1 43667
[junit4] 2> 4487958 INFO (TEST-HdfsSyncSliceTest.test-seed#[B17B537CC2F9B000]) [ ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
[junit4] 2> 4487960 INFO (zkConnectionManagerCallback-20817-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 4487960 INFO (TEST-HdfsSyncSliceTest.test-seed#[B17B537CC2F9B000]) [ ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
[junit4] 2> 4487962 INFO (TEST-HdfsSyncSliceTest.test-seed#[B17B537CC2F9B000]) [ ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
[junit4] 2> 4487963 INFO (zkConnectionManagerCallback-20819-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 4487963 INFO (TEST-HdfsSyncSliceTest.test-seed#[B17B537CC2F9B000]) [ ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
[junit4] 2> 4487964 INFO (TEST-HdfsSyncSliceTest.test-seed#[B17B537CC2F9B000]) [ ] o.a.s.c.ZkTestServer put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
[junit4] 2> 4487965 INFO (TEST-HdfsSyncSliceTest.test-seed#[B17B537CC2F9B000]) [ ] o.a.s.c.ZkTestServer put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/core/src/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
[junit4] 2> 4487966 INFO (TEST-HdfsSyncSliceTest.test-seed#[B17B537CC2F9B000]) [ ] o.a.s.c.ZkTestServer put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
[junit4] 2> 4487967 INFO (TEST-HdfsSyncSliceTest.test-seed#[B17B537CC2F9B000]) [ ] o.a.s.c.ZkTestServer put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
[junit4] 2> 4487968 INFO (TEST-HdfsSyncSliceTest.test-seed#[B17B537CC2F9B000]) [ ] o.a.s.c.ZkTestServer put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
[junit4] 2> 4487969 INFO (TEST-HdfsSyncSliceTest.test-seed#[B17B537CC2F9B000]) [ ] o.a.s.c.ZkTestServer put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
[junit4] 2> 4487970 INFO (TEST-HdfsSyncSliceTest.test-seed#[B17B537CC2F9B000]) [ ] o.a.s.c.ZkTestServer put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
[junit4] 2> 4487971 INFO (TEST-HdfsSyncSliceTest.test-seed#[B17B537CC2F9B000]) [ ] o.a.s.c.ZkTestServer put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
[junit4] 2> 4487971 INFO (TEST-HdfsSyncSliceTest.test-seed#[B17B537CC2F9B000]) [ ] o.a.s.c.ZkTestServer put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4] 2> 4487972 INFO (TEST-HdfsSyncSliceTest.test-seed#[B17B537CC2F9B000]) [ ] o.a.s.c.ZkTestServer put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
[junit4] 2> 4487973 INFO (TEST-HdfsSyncSliceTest.test-seed#[B17B537CC2F9B000]) [ ] o.a.s.c.ZkTestServer put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
[junit4] 2> 4487975 INFO (TEST-HdfsSyncSliceTest.test-seed#[B17B537CC2F9B000]) [ ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
[junit4] 2> 4487975 INFO (zkConnectionManagerCallback-20823-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 4487975 INFO (TEST-HdfsSyncSliceTest.test-seed#[B17B537CC2F9B000]) [ ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
[junit4] 2> 4488077 INFO (TEST-HdfsSyncSliceTest.test-seed#[B17B537CC2F9B000]) [ ] o.a.s.c.AbstractFullDistribZkTestBase Will use NRT replicas unless explicitly asked otherwise
[junit4] 2> 4488229 INFO (TEST-HdfsSyncSliceTest.test-seed#[B17B537CC2F9B000]) [ ] o.a.s.c.s.e.JettySolrRunner Start Jetty (configured port=0, binding port=0)
[junit4] 2> 4488229 INFO (TEST-HdfsSyncSliceTest.test-seed#[B17B537CC2F9B000]) [ ] o.a.s.c.s.e.JettySolrRunner Trying to start Jetty on port 0 try number 1 ...
[junit4] 2> 4488229 INFO (TEST-HdfsSyncSliceTest.test-seed#[B17B537CC2F9B000]) [ ] o.e.j.s.Server jetty-9.4.24.v20191120; built: 2019-11-20T21:37:49.771Z; git: 363d5f2df3a8a28de40604320230664b9c793c16; jvm 11.0.4+10-LTS
[junit4] 2> 4488235 INFO (TEST-HdfsSyncSliceTest.test-seed#[B17B537CC2F9B000]) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 4488235 INFO (TEST-HdfsSyncSliceTest.test-seed#[B17B537CC2F9B000]) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 4488235 INFO (TEST-HdfsSyncSliceTest.test-seed#[B17B537CC2F9B000]) [ ] o.e.j.s.session node0 Scavenging every 660000ms
[junit4] 2> 4488236 INFO (TEST-HdfsSyncSliceTest.test-seed#[B17B537CC2F9B000]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@839cc1d{/,null,AVAILABLE}
[junit4] 2> 4488237 INFO (TEST-HdfsSyncSliceTest.test-seed#[B17B537CC2F9B000]) [ ] o.e.j.s.AbstractConnector Started ServerConnector@3d4aed52{ssl,[ssl, alpn, http/1.1, h2]}{127.0.0.1:46199}
[junit4] 2> 4488237 INFO (TEST-HdfsSyncSliceTest.test-seed#[B17B537CC2F9B000]) [ ] o.e.j.s.Server Started @4488274ms
[junit4] 2> 4488237 INFO (TEST-HdfsSyncSliceTest.test-seed#[B17B537CC2F9B000]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/, solr.data.dir=hdfs://localhost.localdomain:35847/hdfs__localhost.localdomain_35847__home_jenkins_jenkins-slave_workspace_Lucene-Solr-NightlyTests-master_checkout_solr_build_solr-core_test_J2_temp_solr.cloud.hdfs.HdfsSyncSliceTest_B17B537CC2F9B000-001_tempDir-002_control_data, hostPort=46199, coreRootDirectory=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J2/../../../../../../../../../../../home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsSyncSliceTest_B17B537CC2F9B000-001/control-001/cores}
[junit4] 2> 4488237 ERROR (TEST-HdfsSyncSliceTest.test-seed#[B17B537CC2F9B000]) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 4488237 INFO (TEST-HdfsSyncSliceTest.test-seed#[B17B537CC2F9B000]) [ ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
[junit4] 2> 4488237 INFO (TEST-HdfsSyncSliceTest.test-seed#[B17B537CC2F9B000]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version 9.0.0
[junit4] 2> 4488237 INFO (TEST-HdfsSyncSliceTest.test-seed#[B17B537CC2F9B000]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 4488237 INFO (TEST-HdfsSyncSliceTest.test-seed#[B17B537CC2F9B000]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr
[junit4] 2> 4488237 INFO (TEST-HdfsSyncSliceTest.test-seed#[B17B537CC2F9B000]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2020-02-22T06:55:41.348846Z
[junit4] 2> 4488240 INFO (TEST-HdfsSyncSliceTest.test-seed#[B17B537CC2F9B000]) [ ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
[junit4] 2> 4488245 INFO (zkConnectionManagerCallback-20825-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 4488245 INFO (TEST-HdfsSyncSliceTest.test-seed#[B17B537CC2F9B000]) [ ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
[junit4] 2> 4488347 INFO (TEST-HdfsSyncSliceTest.test-seed#[B17B537CC2F9B000]) [ ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
[junit4] 2> 4488347 INFO (TEST-HdfsSyncSliceTest.test-seed#[B17B537CC2F9B000]) [ ] o.a.s.c.SolrXmlConfig Loading container configuration from /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsSyncSliceTest_B17B537CC2F9B000-001/control-001/solr.xml
[junit4] 2> 4488350 INFO (TEST-HdfsSyncSliceTest.test-seed#[B17B537CC2F9B000]) [ ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverWorkLoopDelay is ignored
[junit4] 2> 4488350 INFO (TEST-HdfsSyncSliceTest.test-seed#[B17B537CC2F9B000]) [ ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverBadNodeExpiration is ignored
[junit4] 2> 4488351 INFO (TEST-HdfsSyncSliceTest.test-seed#[B17B537CC2F9B000]) [ ] o.a.s.c.SolrXmlConfig MBean server found: com.sun.jmx.mbeanserver.JmxMBeanServer@7425d76c, but no JMX reporters were configured - adding default JMX reporter.
[junit4] 2> 4489146 INFO (TEST-HdfsSyncSliceTest.test-seed#[B17B537CC2F9B000]) [ ] o.a.s.h.c.HttpShardHandlerFactory Host whitelist initialized: WhitelistHostChecker [whitelistHosts=null, whitelistHostCheckingEnabled=false]
[junit4] 2> 4489148 WARN (TEST-HdfsSyncSliceTest.test-seed#[B17B537CC2F9B000]) [ ] o.e.j.u.s.S.config Trusting all certificates configured for Client@3890c06d[provider=null,keyStore=null,trustStore=null]
[junit4] 2> 4489148 WARN (TEST-HdfsSyncSliceTest.test-seed#[B17B537CC2F9B000]) [ ] o.e.j.u.s.S.config No Client EndPointIdentificationAlgorithm configured for Client@3890c06d[provider=null,keyStore=null,trustStore=null]
[junit4] 2> 4489151 WARN (TEST-HdfsSyncSliceTest.test-seed#[B17B537CC2F9B000]) [ ] o.e.j.u.s.S.config Trusting all certificates configured for Client@54bfa283[provider=null,keyStore=null,trustStore=null]
[junit4] 2> 4489151 WARN (TEST-HdfsSyncSliceTest.test-seed#[B17B537CC2F9B000]) [ ] o.e.j.u.s.S.config No Client EndPointIdentificationAlgorithm configured for Client@54bfa283[provider=null,keyStore=null,trustStore=null]
[junit4] 2> 4489152 INFO (TEST-HdfsSyncSliceTest.test-seed#[B17B537CC2F9B000]) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:43667/solr
[junit4] 2> 4489153 INFO (TEST-HdfsSyncSliceTest.test-seed#[B17B537CC2F9B000]) [ ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
[junit4] 2> 4489156 INFO (zkConnectionManagerCallback-20832-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 4489156 INFO (TEST-HdfsSyncSliceTest.test-seed#[B17B537CC2F9B000]) [ ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
[junit4] 2> 4489258 INFO (TEST-HdfsSyncSliceTest.test-seed#[B17B537CC2F9B000]) [n:127.0.0.1:46199_ ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
[junit4] 2> 4489259 INFO (zkConnectionManagerCallback-20834-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 4489259 INFO (TEST-HdfsSyncSliceTest.test-seed#[B17B537CC2F9B000]) [n:127.0.0.1:46199_ ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
[junit4] 2> 4489329 INFO (TEST-HdfsSyncSliceTest.test-seed#[B17B537CC2F9B000]) [n:127.0.0.1:46199_ ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:46199_
[junit4] 2> 4489330 INFO (TEST-HdfsSyncSliceTest.test-seed#[B17B537CC2F9B000]) [n:127.0.0.1:46199_ ] o.a.s.c.Overseer Overseer (id=73288106769711109-127.0.0.1:46199_-n_0000000000) starting
[junit4] 2> 4489334 INFO (OverseerStateUpdate-73288106769711109-127.0.0.1:46199_-n_0000000000) [n:127.0.0.1:46199_ ] o.a.s.c.Overseer Starting to work on the main queue : 127.0.0.1:46199_
[junit4] 2> 4489335 INFO (TEST-HdfsSyncSliceTest.test-seed#[B17B537CC2F9B000]) [n:127.0.0.1:46199_ ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:46199_
[junit4] 2> 4489336 INFO (zkCallback-20833-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 4489337 INFO (TEST-HdfsSyncSliceTest.test-seed#[B17B537CC2F9B000]) [n:127.0.0.1:46199_ ] o.a.s.p.PackageLoader /packages.json updated to version -1
[junit4] 2> 4489338 WARN (TEST-HdfsSyncSliceTest.test-seed#[B17B537CC2F9B000]) [n:127.0.0.1:46199_ ] o.a.s.c.CoreContainer Not all security plugins configured! authentication=disabled authorization=disabled. Solr is only as secure as you make it. Consider configuring authentication/authorization before exposing Solr to users internal or external. See https://s.apache.org/solrsecurity for more info
[junit4] 2> 4489356 INFO (TEST-HdfsSyncSliceTest.test-seed#[B17B537CC2F9B000]) [n:127.0.0.1:46199_ ] o.a.s.h.a.MetricsHistoryHandler No .system collection, keeping metrics history in memory.
[junit4] 2> 4489378 INFO (TEST-HdfsSyncSliceTest.test-seed#[B17B537CC2F9B000]) [n:127.0.0.1:46199_ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@7425d76c
[junit4] 2> 4489386 INFO (TEST-HdfsSyncSliceTest.test-seed#[B17B537CC2F9B000]) [n:127.0.0.1:46199_ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@7425d76c
[junit4] 2> 4489386 INFO (TEST-HdfsSyncSliceTest.test-seed#[B17B537CC2F9B000]) [n:127.0.0.1:46199_ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@7425d76c
[junit4] 2> 4489387 INFO (TEST-HdfsSyncSliceTest.test-seed#[B17B537CC2F9B000]) [n:127.0.0.1:46199_ ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J2/../../../../../../../../../../../home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsSyncSliceTest_B17B537CC2F9B000-001/control-001/cores
[junit4] 2> 4489403 INFO (TEST-HdfsSyncSliceTest.test-seed#[B17B537CC2F9B000]) [ ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
[junit4] 2> 4489405 INFO (zkConnectionManagerCallback-20843-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 4489405 INFO (TEST-HdfsSyncSliceTest.test-seed#[B17B537CC2F9B000]) [ ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
[junit4] 2> 4489412 INFO (TEST-HdfsSyncSliceTest.test-seed#[B17B537CC2F9B000]) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 4489412 INFO (TEST-HdfsSyncSliceTest.test-seed#[B17B537CC2F9B000]) [ ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:43667/solr ready
[junit4] 2> 4489448 INFO (qtp1032895805-151220) [n:127.0.0.1:46199_ ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params collection.configName=conf1&name=control_collection&nrtReplicas=1&action=CREATE&numShards=1&createNodeSet=127.0.0.1:46199_&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 4489451 INFO (OverseerThreadFactory-15818-thread-1-processing-n:127.0.0.1:46199_) [n:127.0.0.1:46199_ ] o.a.s.c.a.c.CreateCollectionCmd Create collection control_collection
[junit4] 2> 4489564 INFO (qtp1032895805-151219) [n:127.0.0.1:46199_ x:control_collection_shard1_replica_n1 ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf1&newCollection=true&name=control_collection_shard1_replica_n1&action=CREATE&numShards=1&collection=control_collection&shard=shard1&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 4489564 INFO (qtp1032895805-151219) [n:127.0.0.1:46199_ x:control_collection_shard1_replica_n1 ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 4 transient cores
[junit4] 2> 4490574 INFO (qtp1032895805-151219) [n:127.0.0.1:46199_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1 ] o.a.s.c.SolrConfig Using Lucene MatchVersion: 9.0.0
[junit4] 2> 4490620 INFO (qtp1032895805-151219) [n:127.0.0.1:46199_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1 ] o.a.s.s.IndexSchema [control_collection_shard1_replica_n1] Schema name=test
[junit4] 2> 4490718 INFO (qtp1032895805-151219) [n:127.0.0.1:46199_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1 ] o.a.s.s.IndexSchema Loaded schema test/1.0 with uniqueid field id
[junit4] 2> 4490736 INFO (qtp1032895805-151219) [n:127.0.0.1:46199_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1 ] o.a.s.c.CoreContainer Creating SolrCore 'control_collection_shard1_replica_n1' using configuration from configset conf1, trusted=true
[junit4] 2> 4490736 INFO (qtp1032895805-151219) [n:127.0.0.1:46199_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1 ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.control_collection.shard1.replica_n1' (registry 'solr.core.control_collection.shard1.replica_n1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@7425d76c
[junit4] 2> 4490736 INFO (qtp1032895805-151219) [n:127.0.0.1:46199_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1 ] o.a.s.c.HdfsDirectoryFactory solr.hdfs.home=hdfs://localhost.localdomain:35847/solr_hdfs_home
[junit4] 2> 4490736 INFO (qtp1032895805-151219) [n:127.0.0.1:46199_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1 ] o.a.s.c.HdfsDirectoryFactory Solr Kerberos Authentication disabled
[junit4] 2> 4490736 INFO (qtp1032895805-151219) [n:127.0.0.1:46199_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1 ] o.a.s.c.SolrCore [[control_collection_shard1_replica_n1] ] Opening new SolrCore at [/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsSyncSliceTest_B17B537CC2F9B000-001/control-001/cores/control_collection_shard1_replica_n1], dataDir=[hdfs://localhost.localdomain:35847/solr_hdfs_home/control_collection/core_node2/data/]
[junit4] 2> 4490737 INFO (qtp1032895805-151219) [n:127.0.0.1:46199_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1 ] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost.localdomain:35847/solr_hdfs_home/control_collection/core_node2/data/snapshot_metadata
[junit4] 2> 4490745 INFO (qtp1032895805-151219) [n:127.0.0.1:46199_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1 ] o.a.s.c.HdfsDirectoryFactory Number of slabs of block cache [1] with direct memory allocation set to [true]
[junit4] 2> 4490745 INFO (qtp1032895805-151219) [n:127.0.0.1:46199_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1 ] o.a.s.c.HdfsDirectoryFactory Block cache target memory usage, slab size of [4194304] will allocate [1] slabs and use ~[4194304] bytes
[junit4] 2> 4490745 INFO (qtp1032895805-151219) [n:127.0.0.1:46199_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1 ] o.a.s.c.HdfsDirectoryFactory Creating new single instance HDFS BlockCache
[junit4] 2> 4490748 WARN (qtp1032895805-151219) [n:127.0.0.1:46199_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1 ] o.a.s.s.h.HdfsDirectory The NameNode is in SafeMode - Solr will wait 5 seconds and try again.
[junit4] 2> 4495750 INFO (qtp1032895805-151219) [n:127.0.0.1:46199_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1 ] o.a.s.s.b.BlockDirectory Block cache on write is disabled
[junit4] 2> 4495751 INFO (qtp1032895805-151219) [n:127.0.0.1:46199_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1 ] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost.localdomain:35847/solr_hdfs_home/control_collection/core_node2/data
[junit4] 2> 4495766 INFO (qtp1032895805-151219) [n:127.0.0.1:46199_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1 ] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost.localdomain:35847/solr_hdfs_home/control_collection/core_node2/data/index
[junit4] 2> 4495772 INFO (qtp1032895805-151219) [n:127.0.0.1:46199_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1 ] o.a.s.c.HdfsDirectoryFactory Number of slabs of block cache [1] with direct memory allocation set to [true]
[junit4] 2> 4495772 INFO (qtp1032895805-151219) [n:127.0.0.1:46199_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1 ] o.a.s.c.HdfsDirectoryFactory Block cache target memory usage, slab size of [4194304] will allocate [1] slabs and use ~[4194304] bytes
[junit4] 2> 4495772 INFO (qtp1032895805-151219) [n:127.0.0.1:46199_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1 ] o.a.s.c.HdfsDirectoryFactory Creating new single instance HDFS BlockCache
[junit4] 2> 4495775 INFO (qtp1032895805-151219) [n:127.0.0.1:46199_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1 ] o.a.s.s.b.BlockDirectory Block cache on write is disabled
[junit4] 2> 4495775 INFO (qtp1032895805-151219) [n:127.0.0.1:46199_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1 ] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=23, maxMergeAtOnceExplicit=21, maxMergedSegmentMB=77.716796875, floorSegmentMB=1.43359375, forceMergeDeletesPctAllowed=3.3339192782120284, segmentsPerTier=44.0, maxCFSSegmentSizeMB=0.9423828125, noCFSRatio=1.0, deletesPctAllowed=38.58366471507388
[junit4] 2> 4495784 WARN (qtp1032895805-151219) [n:127.0.0.1:46199_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1 ] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,attributes = {initParams=a, name=/dump, class=DumpRequestHandler},args = {defaults={a=A, b=B}}}
[junit4] 2> 4495843 INFO (qtp1032895805-151219) [n:127.0.0.1:46199_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1 ] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.HdfsUpdateLog
[junit4] 2> 4495843 INFO (qtp1032895805-151219) [n:127.0.0.1:46199_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1 ] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 4495843 INFO (qtp1032895805-151219) [n:127.0.0.1:46199_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1 ] o.a.s.u.HdfsUpdateLog Initializing HdfsUpdateLog: tlogDfsReplication=2
[junit4] 2> 4495852 INFO (qtp1032895805-151219) [n:127.0.0.1:46199_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1 ] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 4495852 INFO (qtp1032895805-151219) [n:127.0.0.1:46199_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1 ] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 4495853 INFO (qtp1032895805-151219) [n:127.0.0.1:46199_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1 ] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=8, maxMergeAtOnceExplicit=6, maxMergedSegmentMB=106.478515625, floorSegmentMB=0.568359375, forceMergeDeletesPctAllowed=17.92212604825535, segmentsPerTier=32.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0, deletesPctAllowed=22.927839180350503
[junit4] 2> 4495856 INFO (qtp1032895805-151219) [n:127.0.0.1:46199_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1 ] o.a.s.s.SolrIndexSearcher Opening [Searcher@5085aa3f[control_collection_shard1_replica_n1] main]
[junit4] 2> 4495857 INFO (qtp1032895805-151219) [n:127.0.0.1:46199_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1 ] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 4495858 INFO (qtp1032895805-151219) [n:127.0.0.1:46199_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1 ] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 4495858 INFO (qtp1032895805-151219) [n:127.0.0.1:46199_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1 ] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
[junit4] 2> 4495858 INFO (qtp1032895805-151219) [n:127.0.0.1:46199_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1 ] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1659219003539718144
[junit4] 2> 4495861 INFO (searcherExecutor-15823-thread-1-processing-n:127.0.0.1:46199_ x:control_collection_shard1_replica_n1 c:control_collection s:shard1) [n:127.0.0.1:46199_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1 ] o.a.s.c.SolrCore [control_collection_shard1_replica_n1] Registered new searcher Searcher@5085aa3f[control_collection_shard1_replica_n1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 4495863 INFO (qtp1032895805-151219) [n:127.0.0.1:46199_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1 ] o.a.s.c.ZkShardTerms Successful update of terms at /collections/control_collection/terms/shard1 to Terms{values={core_node2=0}, version=0}
[junit4] 2> 4495863 INFO (qtp1032895805-151219) [n:127.0.0.1:46199_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1 ] o.a.s.c.ShardLeaderElectionContextBase make sure parent is created /collections/control_collection/leaders/shard1
[junit4] 2> 4495865 INFO (qtp1032895805-151219) [n:127.0.0.1:46199_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1 ] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
[junit4] 2> 4495865 INFO (qtp1032895805-151219) [n:127.0.0.1:46199_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1 ] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
[junit4] 2> 4495865 INFO (qtp1032895805-151219) [n:127.0.0.1:46199_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1 ] o.a.s.c.SyncStrategy Sync replicas to https://127.0.0.1:46199/control_collection_shard1_replica_n1/
[junit4] 2> 4495865 INFO (qtp1032895805-151219) [n:127.0.0.1:46199_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1 ] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
[junit4] 2> 4495866 INFO (qtp1032895805-151219) [n:127.0.0.1:46199_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1 ] o.a.s.c.SyncStrategy https://127.0.0.1:46199/control_collection_shard1_replica_n1/ has no replicas
[junit4] 2> 4495866 INFO (qtp1032895805-151219) [n:127.0.0.1:46199_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1 ] o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node /collections/control_collection/leaders/shard1/leader after winning as /collections/control_collection/leader_elect/shard1/election/73288106769711109-core_node2-n_0000000000
[junit4] 2> 4495867 INFO (qtp1032895805-151219) [n:127.0.0.1:46199_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1 ] o.a.s.c.ShardLeaderElectionContext I am the new leader: https://127.0.0.1:46199/control_collection_shard1_replica_n1/ shard1
[junit4] 2> 4495868 INFO (zkCallback-20833-thread-1) [ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/control_collection/state.json] for collection [control_collection] has occurred - updating... (live nodes size: [1])
[junit4] 2> 4495868 INFO (zkCallback-20833-thread-2) [ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/control_collection/state.json] for collection [control_collection] has occurred - updating... (live nodes size: [1])
[junit4] 2> 4495868 INFO (qtp1032895805-151219) [n:127.0.0.1:46199_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1 ] o.a.s.c.ZkController I am the leader, no recovery necessary
[junit4] 2> 4495870 INFO (qtp1032895805-151219) [n:127.0.0.1:46199_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1 ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=conf1&newCollection=true&name=control_collection_shard1_replica_n1&action=CREATE&numShards=1&collection=control_collection&shard=shard1&wt=javabin&version=2&replicaType=NRT} status=0 QTime=6306
[junit4] 2> 4495871 INFO (qtp1032895805-151220) [n:127.0.0.1:46199_ ] o.a.s.h.a.CollectionsHandler Wait for new collection to be active for at most 45 seconds. Check all shard replicas
[junit4] 2> 4495970 INFO (zkCallback-20833-thread-1) [ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/control_collection/state.json] for collection [control_collection] has occurred - updating... (live nodes size: [1])
[junit4] 2> 4495970 INFO (zkCallback-20833-thread-2) [ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/control_collection/state.json] for collection [control_collection] has occurred - updating... (live nodes size: [1])
[junit4] 2> 4495970 INFO (zkCallback-20833-thread-3) [ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/control_collection/state.json] for collection [control_collection] has occurred - updating... (live nodes size: [1])
[junit4] 2> 4495971 INFO (qtp1032895805-151220) [n:127.0.0.1:46199_ ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={collection.configName=conf1&name=control_collection&nrtReplicas=1&action=CREATE&numShards=1&createNodeSet=127.0.0.1:46199_&wt=javabin&version=2} status=0 QTime=6523
[junit4] 2> 4495971 INFO (TEST-HdfsSyncSliceTest.test-seed#[B17B537CC2F9B000]) [ ] o.a.s.c.AbstractFullDistribZkTestBase Waiting to see 1 active replicas in collection: control_collection
[junit4] 2> 4496076 INFO (TEST-HdfsSyncSliceTest.test-seed#[B17B537CC2F9B000]) [ ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
[junit4] 2> 4496076 INFO (zkConnectionManagerCallback-20849-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 4496076 INFO (TEST-HdfsSyncSliceTest.test-seed#[B17B537CC2F9B000]) [ ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
[junit4] 2> 4496078 INFO (TEST-HdfsSyncSliceTest.test-seed#[B17B537CC2F9B000]) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 4496079 INFO (TEST-HdfsSyncSliceTest.test-seed#[B17B537CC2F9B000]) [ ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:43667/solr ready
[junit4] 2> 4496079 INFO (TEST-HdfsSyncSliceTest.test-seed#[B17B537CC2F9B000]) [ ] o.a.s.c.ChaosMonkey monkey: init - expire sessions:false cause connection loss:false
[junit4] 2> 4496114 INFO (qtp1032895805-151218) [n:127.0.0.1:46199_ ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params collection.configName=conf1&name=collection1&nrtReplicas=1&action=CREATE&numShards=1&createNodeSet=&stateFormat=2&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 4496116 INFO (OverseerThreadFactory-15818-thread-2-processing-n:127.0.0.1:46199_) [n:127.0.0.1:46199_ ] o.a.s.c.a.c.CreateCollectionCmd Create collection collection1
[junit4] 2> 4496116 INFO (OverseerCollectionConfigSetProcessor-73288106769711109-127.0.0.1:46199_-n_0000000000) [n:127.0.0.1:46199_ ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000000 doesn't exist. Requestor may have disconnected from ZooKeeper
[junit4] 2> 4496319 WARN (OverseerThreadFactory-15818-thread-2-processing-n:127.0.0.1:46199_) [n:127.0.0.1:46199_ ] o.a.s.c.a.c.CreateCollectionCmd It is unusual to create a collection (collection1) without cores.
[junit4] 2> 4496320 INFO (qtp1032895805-151218) [n:127.0.0.1:46199_ ] o.a.s.h.a.CollectionsHandler Wait for new collection to be active for at most 45 seconds. Check all shard replicas
[junit4] 2> 4496320 INFO (qtp1032895805-151218) [n:127.0.0.1:46199_ ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={collection.configName=conf1&name=collection1&nrtReplicas=1&action=CREATE&numShards=1&createNodeSet=&stateFormat=2&wt=javabin&version=2} status=0 QTime=206
[junit4] 2> 4496332 INFO (TEST-HdfsSyncSliceTest.test-seed#[B17B537CC2F9B000]) [ ] o.a.s.c.SolrCloudTestCase active slice count: 1 expected:1
[junit4] 2> 4496332 INFO (TEST-HdfsSyncSliceTest.test-seed#[B17B537CC2F9B000]) [ ] o.a.s.c.SolrCloudTestCase active replica count: 0 expected replica count: 0
[junit4] 2> 4496332 INFO (TEST-HdfsSyncSliceTest.test-seed#[B17B537CC2F9B000]) [ ] o.a.s.c.SolrCloudTestCase active slice count: 1 expected:1
[junit4] 2> 4496332 INFO (TEST-HdfsSyncSliceTest.test-seed#[B17B537CC2F9B000]) [ ] o.a.s.c.SolrCloudTestCase active replica count: 0 expected replica count: 0
[junit4] 2> 4496332 INFO (TEST-HdfsSyncSliceTest.test-seed#[B17B537CC2F9B000]) [ ] o.a.s.c.SolrCloudTestCase active slice count: 1 expected:1
[junit4] 2> 4496332 INFO (TEST-HdfsSyncSliceTest.test-seed#[B17B537CC2F9B000]) [ ] o.a.s.c.SolrCloudTestCase active replica count: 0 expected replica count: 0
[junit4] 2> 4496333 INFO (TEST-HdfsSyncSliceTest.test-seed#[B17B537CC2F9B000]) [ ] o.a.s.c.AbstractFullDistribZkTestBase Creating jetty instances pullReplicaCount=0 numOtherReplicas=7
[junit4] 2> 4496490 INFO (TEST-HdfsSyncSliceTest.test-seed#[B17B537CC2F9B000]) [ ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 1 in directory /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsSyncSliceTest_B17B537CC2F9B000-001/shard-1-001 of type NRT for shard1
[junit4] 2> 4496492 INFO (closeThreadPool-20850-thread-1) [ ] o.a.s.c.s.e.JettySolrRunner Start Jetty (configured port=0, binding port=0)
[junit4] 2> 4496492 INFO (closeThreadPool-20850-thread-1) [ ] o.a.s.c.s.e.JettySolrRunner Trying to start Jetty on port 0 try number 1 ...
[junit4] 2> 4496492 INFO (closeThreadPool-20850-thread-1) [ ] o.e.j.s.Server jetty-9.4.24.v20191120; built: 2019-11-20T21:37:49.771Z; git: 363d5f2df3a8a28de40604320230664b9c793c16; jvm 11.0.4+10-LTS
[junit4] 2> 4496495 INFO (closeThreadPool-20850-thread-1) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 4496495 INFO (closeThreadPool-20850-thread-1) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 4496495 INFO (closeThreadPool-20850-thread-1) [ ] o.e.j.s.session node0 Scavenging every 660000ms
[junit4] 2> 4496495 INFO (closeThreadPool-20850-thread-1) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@3310d5f4{/,null,AVAILABLE}
[junit4] 2> 4496496 INFO (closeThreadPool-20850-thread-1) [ ] o.e.j.s.AbstractConnector Started ServerConnector@4c63d985{ssl,[ssl, alpn, http/1.1, h2]}{127.0.0.1:38143}
[junit4] 2> 4496496 INFO (closeThreadPool-20850-thread-1) [ ] o.e.j.s.Server Started @4496533ms
[junit4] 2> 4496496 INFO (closeThreadPool-20850-thread-1) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/, solrconfig=solrconfig.xml, solr.data.dir=hdfs://localhost.localdomain:35847/hdfs__localhost.localdomain_35847__home_jenkins_jenkins-slave_workspace_Lucene-Solr-NightlyTests-master_checkout_solr_build_solr-core_test_J2_temp_solr.cloud.hdfs.HdfsSyncSliceTest_B17B537CC2F9B000-001_tempDir-002_jetty1, hostPort=38143, coreRootDirectory=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J2/../../../../../../../../../../../home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsSyncSliceTest_B17B537CC2F9B000-001/shard-1-001/cores, replicaType=NRT}
[junit4] 2> 4496496 ERROR (closeThreadPool-20850-thread-1) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 4496496 INFO (closeThreadPool-20850-thread-1) [ ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
[junit4] 2> 4496496 INFO (closeThreadPool-20850-thread-1) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version 9.0.0
[junit4] 2> 4496496 INFO (closeThreadPool-20850-thread-1) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 4496496 INFO (closeThreadPool-20850-thread-1) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr
[junit4] 2> 4496496 INFO (closeThreadPool-20850-thread-1) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2020-02-22T06:55:49.607936Z
[junit4] 2> 4496498 INFO (closeThreadPool-20850-thread-1) [ ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
[junit4] 2> 4496498 INFO (zkConnectionManagerCallback-20852-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 4496498 INFO (closeThreadPool-20850-thread-1) [ ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
[junit4] 2> 4496599 INFO (closeThreadPool-20850-thread-1) [ ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
[junit4] 2> 4496599 INFO (closeThreadPool-20850-thread-1) [ ] o.a.s.c.SolrXmlConfig Loading container configuration from /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsSyncSliceTest_B17B537CC2F9B000-001/shard-1-001/solr.xml
[junit4] 2> 4496602 INFO (closeThreadPool-20850-thread-1) [ ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverWorkLoopDelay is ignored
[junit4] 2> 4496602 INFO (closeThreadPool-20850-thread-1) [ ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverBadNodeExpiration is ignored
[junit4] 2> 4496603 INFO (closeThreadPool-20850-thread-1) [ ] o.a.s.c.SolrXmlConfig MBean server found: com.sun.jmx.mbeanserver.JmxMBeanServer@7425d76c, but no JMX reporters were configured - adding default JMX reporter.
[junit4] 2> 4496649 INFO (TEST-HdfsSyncSliceTest.test-seed#[B17B537CC2F9B000]) [ ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 2 in directory /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsSyncSliceTest_B17B537CC2F9B000-001/shard-2-001 of type NRT for shard1
[junit4] 2> 4496654 INFO (closeThreadPool-20850-thread-2) [ ] o.a.s.c.s.e.JettySolrRunner Start Jetty (configured port=0, binding port=0)
[junit4] 2> 4496654 INFO (closeThreadPool-20850-thread-2) [ ] o.a.s.c.s.e.JettySolrRunner Trying to start Jetty on port 0 try number 1 ...
[junit4] 2> 4496654 INFO (closeThreadPool-20850-thread-2) [ ] o.e.j.s.Server jetty-9.4.24.v20191120; built: 2019-11-20T21:37:49.771Z; git: 363d5f2df3a8a28de40604320230664b9c793c16; jvm 11.0.4+10-LTS
[junit4] 2> 4496679 INFO (closeThreadPool-20850-thread-2) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 4496679 INFO (closeThreadPool-20850-thread-2) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 4496679 INFO (closeThreadPool-20850-thread-2) [ ] o.e.j.s.session node0 Scavenging every 660000ms
[junit4] 2> 4496680 INFO (closeThreadPool-20850-thread-2) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@42237d0a{/,null,AVAILABLE}
[junit4] 2> 4496681 INFO (closeThreadPool-20850-thread-2) [ ] o.e.j.s.AbstractConnector Started ServerConnector@65b7f20a{ssl,[ssl, alpn, http/1.1, h2]}{127.0.0.1:46737}
[junit4] 2> 4496681 INFO (closeThreadPool-20850-thread-2) [ ] o.e.j.s.Server Started @4496718ms
[junit4] 2> 4496681 INFO (closeThreadPool-20850-thread-2) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/, solrconfig=solrconfig.xml, solr.data.dir=hdfs://localhost.localdomain:35847/hdfs__localhost.localdomain_35847__home_jenkins_jenkins-slave_workspace_Lucene-Solr-NightlyTests-master_checkout_solr_build_solr-core_test_J2_temp_solr.cloud.hdfs.HdfsSyncSliceTest_B17B537CC2F9B000-001_tempDir-002_jetty2, hostPort=46737, coreRootDirectory=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J2/../../../../../../../../../../../home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsSyncSliceTest_B17B537CC2F9B000-001/shard-2-001/cores, replicaType=NRT}
[junit4] 2> 4496681 ERROR (closeThreadPool-20850-thread-2) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 4496681 INFO (closeThreadPool-20850-thread-2) [ ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
[junit4] 2> 4496681 INFO (closeThreadPool-20850-thread-2) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version 9.0.0
[junit4] 2> 4496681 INFO (closeThreadPool-20850-thread-2) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 4496681 INFO (closeThreadPool-20850-thread-2) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr
[junit4] 2> 4496681 INFO (closeThreadPool-20850-thread-2) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2020-02-22T06:55:49.792698Z
[junit4] 2> 4496685 INFO (closeThreadPool-20850-thread-2) [ ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
[junit4] 2> 4496687 INFO (zkConnectionManagerCallback-20855-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 4496687 INFO (closeThreadPool-20850-thread-2) [ ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
[junit4] 2> 4496789 INFO (closeThreadPool-20850-thread-2) [ ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
[junit4] 2> 4496789 INFO (closeThreadPool-20850-thread-2) [ ] o.a.s.c.SolrXmlConfig Loading container configuration from /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsSyncSliceTest_B17B537CC2F9B000-001/shard-2-001/solr.xml
[junit4] 2> 4496792 INFO (closeThreadPool-20850-thread-2) [ ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverWorkLoopDelay is ignored
[junit4] 2> 4496792 INFO (closeThreadPool-20850-thread-2) [ ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverBadNodeExpiration is ignored
[junit4] 2> 4496800 INFO (closeThreadPool-20850-thread-2) [ ] o.a.s.c.SolrXmlConfig MBean server found: com.sun.jmx.mbeanserver.JmxMBeanServer@7425d76c, but no JMX reporters were configured - adding default JMX reporter.
[junit4] 2> 4496838 INFO (TEST-HdfsSyncSliceTest.test-seed#[B17B537CC2F9B000]) [ ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 3 in directory /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsSyncSliceTest_B17B537CC2F9B000-001/shard-3-001 of type NRT for shard1
[junit4] 2> 4496846 INFO (closeThreadPool-20850-thread-3) [ ] o.a.s.c.s.e.JettySolrRunner Start Jetty (configured port=0, binding port=0)
[junit4] 2> 4496846 INFO (closeThreadPool-20850-thread-3) [ ] o.a.s.c.s.e.JettySolrRunner Trying to start Jetty on port 0 try number 1 ...
[junit4] 2> 4496847 INFO (closeThreadPool-20850-thread-3) [ ] o.e.j.s.Server jetty-9.4.24.v20191120; built: 2019-11-20T21:37:49.771Z; git: 363d5f2df3a8a28de40604320230664b9c793c16; jvm 11.0.4+10-LTS
[junit4] 2> 4496847 INFO (closeThreadPool-20850-thread-3) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 4496847 INFO (closeThreadPool-20850-thread-3) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 4496847 INFO (closeThreadPool-20850-thread-3) [ ] o.e.j.s.session node0 Scavenging every 660000ms
[junit4] 2> 4496847 INFO (closeThreadPool-20850-thread-3) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@724b3f8d{/,null,AVAILABLE}
[junit4] 2> 4496848 INFO (closeThreadPool-20850-thread-3) [ ] o.e.j.s.AbstractConnector Started ServerConnector@de7f2cc{ssl,[ssl, alpn, http/1.1, h2]}{127.0.0.1:41591}
[junit4] 2> 4496848 INFO (closeThreadPool-20850-thread-3) [ ] o.e.j.s.Server Started @4496885ms
[junit4] 2> 4496848 INFO (closeThreadPool-20850-thread-3) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/, solrconfig=solrconfig.xml, solr.data.dir=hdfs://localhost.localdomain:35847/hdfs__localhost.localdomain_35847__home_jenkins_jenkins-slave_workspace_Lucene-Solr-NightlyTests-master_checkout_solr_build_solr-core_test_J2_temp_solr.cloud.hdfs.HdfsSyncSliceTest_B17B537CC2F9B000-001_tempDir-002_jetty3, hostPort=41591, coreRootDirectory=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J2/../../../../../../../../../../../home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsSyncSliceTest_B17B537CC2F9B000-001/shard-3-001/cores, replicaType=NRT}
[junit4] 2> 4496848 ERROR (closeThreadPool-20850-thread-3) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 4496848 INFO (closeThreadPool-20850-thread-3) [ ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
[junit4] 2> 4496848 INFO (closeThreadPool-20850-thread-3) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version 9.0.0
[junit4] 2> 4496848 INFO (closeThreadPool-20850-thread-3) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 4496848 INFO (closeThreadPool-20850-thread-3) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr
[junit4] 2> 4496848 INFO (closeThreadPool-20850-thread-3) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2020-02-22T06:55:49.959885Z
[junit4] 2> 4496849 INFO (closeThreadPool-20850-thread-3) [ ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
[junit4] 2> 4496850 INFO (zkConnectionManagerCallback-20858-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 4496850 INFO (closeThreadPool-20850-thread-3) [ ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
[junit4] 2> 4496956 INFO (closeThreadPool-20850-thread-3) [ ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
[junit4] 2> 4496956 INFO (closeThreadPool-20850-thread-3) [ ] o.a.s.c.SolrXmlConfig Loading container configuration from /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsSyncSliceTest_B17B537CC2F9B000-001/shard-3-001/solr.xml
[junit4] 2> 4496958 INFO (closeThreadPool-20850-thread-3) [ ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverWorkLoopDelay is ignored
[junit4] 2> 4496958 INFO (closeThreadPool-20850-thread-3) [ ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverBadNodeExpiration is ignored
[junit4] 2> 4496960 INFO (closeThreadPool-20850-thread-3) [ ] o.a.s.c.SolrXmlConfig MBean server found: com.sun.jmx.mbeanserver.JmxMBeanServer@7425d76c, but no JMX reporters were configured - adding default JMX reporter.
[junit4] 2> 4496962 INFO (closeThreadPool-20850-thread-1) [ ] o.a.s.h.c.HttpShardHandlerFactory Host whitelist initialized: WhitelistHostChecker [whitelistHosts=null, whitelistHostCheckingEnabled=false]
[junit4] 2> 4496963 WARN (closeThreadPool-20850-thread-1) [ ] o.e.j.u.s.S.config Trusting all certificates configured for Client@2bc6b41[provider=null,keyStore=null,trustStore=null]
[junit4] 2> 4496963 WARN (closeThreadPool-20850-thread-1) [ ] o.e.j.u.s.S.config No Client EndPointIdentificationAlgorithm configured for Client@2bc6b41[provider=null,keyStore=null,trustStore=null]
[junit4] 2> 4496974 WARN (closeThreadPool-20850-thread-1) [ ] o.e.j.u.s.S.config Trusting all certificates configured for Client@2e50f11d[provider=null,keyStore=null,trustStore=null]
[junit4] 2> 4496974 WARN (closeThreadPool-20850-thread-1) [ ] o.e.j.u.s.S.config No Client EndPointIdentificationAlgorithm configured for Client@2e50f11d[provider=null,keyStore=null,trustStore=null]
[junit4] 2> 4496975 INFO (closeThreadPool-20850-thread-1) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:43667/solr
[junit4] 2> 4496977 INFO (closeThreadPool-20850-thread-1) [ ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
[junit4] 2> 4496978 INFO (zkConnectionManagerCallback-20865-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 4496978 INFO (closeThreadPool-20850-thread-1) [ ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
[junit4] 2> 4496990 INFO (TEST-HdfsSyncSliceTest.test-seed#[B17B537CC2F9B000]) [ ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 4 in directory /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsSyncSliceTest_B17B537CC2F9B000-001/shard-4-001 of type NRT for shard1
[junit4] 2> 4496994 INFO (closeThreadPool-20850-thread-4) [ ] o.a.s.c.s.e.JettySolrRunner Start Jetty (configured port=0, binding port=0)
[junit4] 2> 4496994 INFO (closeThreadPool-20850-thread-4) [ ] o.a.s.c.s.e.JettySolrRunner Trying to start Jetty on port 0 try number 1 ...
[junit4] 2> 4496994 INFO (closeThreadPool-20850-thread-4) [ ] o.e.j.s.Server jetty-9.4.24.v20191120; built: 2019-11-20T21:37:49.771Z; git: 363d5f2df3a8a28de40604320230664b9c793c16; jvm 11.0.4+10-LTS
[junit4] 2> 4497006 INFO (closeThreadPool-20850-thread-4) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 4497006 INFO (closeThreadPool-20850-thread-4) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 4497006 INFO (closeThreadPool-20850-thread-4) [ ] o.e.j.s.session node0 Scavenging every 600000ms
[junit4] 2> 4497007 INFO (closeThreadPool-20850-thread-4) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@6fa2348{/,null,AVAILABLE}
[junit4] 2> 4497008 INFO (closeThreadPool-20850-thread-4) [ ] o.e.j.s.AbstractConnector Started ServerConnector@637c1954{ssl,[ssl, alpn, http/1.1, h2]}{127.0.0.1:34811}
[junit4] 2> 4497008 INFO (closeThreadPool-20850-thread-4) [ ] o.e.j.s.Server Started @4497045ms
[junit4] 2> 4497008 INFO (closeThreadPool-20850-thread-4) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/, solrconfig=solrconfig.xml, solr.data.dir=hdfs://localhost.localdomain:35847/hdfs__localhost.localdomain_35847__home_jenkins_jenkins-slave_workspace_Lucene-Solr-NightlyTests-master_checkout_solr_build_solr-core_test_J2_temp_solr.cloud.hdfs.HdfsSyncSliceTest_B17B537CC2F9B000-001_tempDir-002_jetty4, hostPort=34811, coreRootDirectory=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J2/../../../../../../../../../../../home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsSyncSliceTest_B17B537CC2F9B000-001/shard-4-001/cores, replicaType=NRT}
[junit4] 2> 4497008 ERROR (closeThreadPool-20850-thread-4) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 4497008 INFO (closeThreadPool-20850-thread-4) [ ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
[junit4] 2> 4497008 INFO (closeThreadPool-20850-thread-4) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version 9.0.0
[junit4] 2> 4497008 INFO (closeThreadPool-20850-thread-4) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 4497008 INFO (closeThreadPool-20850-thread-4) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr
[junit4] 2> 4497008 INFO (closeThreadPool-20850-thread-4) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2020-02-22T06:55:50.119590Z
[junit4] 2> 4497010 INFO (closeThreadPool-20850-thread-4) [ ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
[junit4] 2> 4497010 INFO (zkConnectionManagerCallback-20867-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 4497010 INFO (closeThreadPool-20850-thread-4) [ ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
[junit4] 2> 4497080 INFO (closeThreadPool-20850-thread-1) [n:127.0.0.1:38143_ ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
[junit4] 2> 4497080 INFO (zkConnectionManagerCallback-20869-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 4497080 INFO (closeThreadPool-20850-thread-1) [n:127.0.0.1:38143_ ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
[junit4] 2> 4497085 INFO (closeThreadPool-20850-thread-1) [n:127.0.0.1:38143_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 4497087 INFO (closeThreadPool-20850-thread-1) [n:127.0.0.1:38143_ ] o.a.s.c.ZkController Publish node=127.0.0.1:38143_ as DOWN
[junit4] 2> 4497088 INFO (closeThreadPool-20850-thread-1) [n:127.0.0.1:38143_ ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 4 transient cores
[junit4] 2> 4497088 INFO (closeThreadPool-20850-thread-1) [n:127.0.0.1:38143_ ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:38143_
[junit4] 2> 4497089 INFO (zkCallback-20833-thread-2) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 4497090 INFO (closeThreadPool-20850-thread-1) [n:127.0.0.1:38143_ ] o.a.s.p.PackageLoader /packages.json updated to version -1
[junit4] 2> 4497090 WARN (closeThreadPool-20850-thread-1) [n:127.0.0.1:38143_ ] o.a.s.c.CoreContainer Not all security plugins configured! authentication=disabled authorization=disabled. Solr is only as secure as you make it. Consider configuring authentication/authorization before exposing Solr to users internal or external. See https://s.apache.org/solrsecurity for more info
[junit4] 2> 4497101 INFO (zkCallback-20848-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 4497111 INFO (zkCallback-20868-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 4497120 INFO (closeThreadPool-20850-thread-1) [n:127.0.0.1:38143_ ] o.a.s.h.a.MetricsHistoryHandler No .system collection, keeping metrics history in memory.
[junit4] 2> 4497153 INFO (closeThreadPool-20850-thread-1) [n:127.0.0.1:38143_ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@7425d76c
[junit4] 2> 4497164 INFO (closeThreadPool-20850-thread-1) [n:127.0.0.1:38143_ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@7425d76c
[junit4] 2> 4497164 INFO (closeThreadPool-20850-thread-1) [n:127.0.0.1:38143_ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@7425d76c
[junit4] 2> 4497165 INFO (closeThreadPool-20850-thread-1) [n:127.0.0.1:38143_ ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J2/../../../../../../../../../../../home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsSyncSliceTest_B17B537CC2F9B000-001/shard-1-001/cores
[junit4] 2> 4497179 INFO (closeThreadPool-20850-thread-4) [ ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
[junit4] 2> 4497179 INFO (closeThreadPool-20850-thread-4) [ ] o.a.s.c.SolrXmlConfig Loading container configuration from /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsSyncSliceTest_B17B537CC2F9B000-001/shard-4-001/solr.xml
[junit4] 2> 4497182 INFO (closeThreadPool-20850-thread-4) [ ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverWorkLoopDelay is ignored
[junit4] 2> 4497182 INFO (closeThreadPool-20850-thread-4) [ ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverBadNodeExpiration is ignored
[junit4] 2> 4497183 INFO (closeThreadPool-20850-thread-4) [ ] o.a.s.c.SolrXmlConfig MBean server found: com.sun.jmx.mbeanserver.JmxMBeanServer@7425d76c, but no JMX reporters were configured - adding default JMX reporter.
[junit4] 2> 4497210 INFO (TEST-HdfsSyncSliceTest.test-seed#[B17B537CC2F9B000]) [ ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 5 in directory /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsSyncSliceTest_B17B537CC2F9B000-001/shard-5-001 of type NRT for shard1
[junit4] 2> 4497222 INFO (closeThreadPool-20850-thread-5) [ ] o.a.s.c.s.e.JettySolrRunner Start Jetty (configured port=0, binding port=0)
[junit4] 2> 4497222 INFO (closeThreadPool-20850-thread-5) [ ] o.a.s.c.s.e.JettySolrRunner Trying to start Jetty on port 0 try number 1 ...
[junit4] 2> 4497222 INFO (closeThreadPool-20850-thread-5) [ ] o.e.j.s.Server jetty-9.4.24.v20191120; built: 2019-11-20T21:37:49.771Z; git: 363d5f2df3a8a28de40604320230664b9c793c16; jvm 11.0.4+10-LTS
[junit4] 2> 4497225 INFO (closeThreadPool-20850-thread-1) [ ] o.a.s.c.AbstractFullDistribZkTestBase waitForLiveNode: 127.0.0.1:38143_
[junit4] 2> 4497273 INFO (closeThreadPool-20850-thread-2) [ ] o.a.s.h.c.HttpShardHandlerFactory Host whitelist initialized: WhitelistHostChecker [whitelistHosts=null, whitelistHostCheckingEnabled=false]
[junit4] 2> 4497344 WARN (closeThreadPool-20850-thread-2) [ ] o.e.j.u.s.S.config Trusting all certificates configured for Client@51c3e91c[provider=null,keyStore=null,trustStore=null]
[junit4] 2> 4497344 WARN (closeThreadPool-20850-thread-2) [ ] o.e.j.u.s.S.config No Client EndPointIdentificationAlgorithm configured for Client@51c3e91c[provider=null,keyStore=null,trustStore=null]
[junit4] 2> 4497346 INFO (closeThreadPool-20850-thread-5) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 4497346 INFO (closeThreadPool-20850-thread-5) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 4497346 INFO (closeThreadPool-20850-thread-5) [ ] o.e.j.s.session node0 Scavenging every 600000ms
[junit4] 2> 4497346 INFO (closeThreadPool-20850-thread-5) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@44fdcaff{/,null,AVAILABLE}
[junit4] 2> 4497349 INFO (closeThreadPool-20850-thread-5) [ ] o.e.j.s.AbstractConnector Started ServerConnector@668d984e{ssl,[ssl, alpn, http/1.1, h2]}{127.0.0.1:37639}
[junit4] 2> 4497349 INFO (closeThreadPool-20850-thread-5) [ ] o.e.j.s.Server Started @4497386ms
[junit4] 2> 4497349 INFO (closeThreadPool-20850-thread-5) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/, solrconfig=solrconfig.xml, solr.data.dir=hdfs://localhost.localdomain:35847/hdfs__localhost.localdomain_35847__home_jenkins_jenkins-slave_workspace_Lucene-Solr-NightlyTests-master_checkout_solr_build_solr-core_test_J2_temp_solr.cloud.hdfs.HdfsSyncSliceTest_B17B537CC2F9B000-001_tempDir-002_jetty5, hostPort=37639, coreRootDirectory=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J2/../../../../../../../../../../../home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsSyncSliceTest_B17B537CC2F9B000-001/shard-5-001/cores, replicaType=NRT}
[junit4] 2> 4497349 ERROR (closeThreadPool-20850-thread-5) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 4497349 INFO (closeThreadPool-20850-thread-5) [ ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
[junit4] 2> 4497349 INFO (closeThreadPool-20850-thread-5) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version 9.0.0
[junit4] 2> 4497349 INFO (closeThreadPool-20850-thread-5) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 4497349 INFO (closeThreadPool-20850-thread-5) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr
[junit4] 2> 4497349 INFO (closeThreadPool-20850-thread-5) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2020-02-22T06:55:50.460350Z
[junit4] 2> 4497384 WARN (closeThreadPool-20850-thread-2) [ ] o.e.j.u.s.S.config Trusting all certificates configured for Client@700417a3[provider=null,keyStore=null,trustStore=null]
[junit4] 2> 4497384 WARN (closeThreadPool-20850-thread-2) [ ] o.e.j.u.s.S.config No Client EndPointIdentificationAlgorithm configured for Client@700417a3[provider=null,keyStore=null,trustStore=null]
[junit4] 2> 4497386 INFO (closeThreadPool-20850-thread-2) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:43667/solr
[junit4] 2> 4497404 INFO (closeThreadPool-20850-thread-5) [ ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
[junit4] 2> 4497418 INFO (closeThreadPool-20850-thread-2) [ ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
[junit4] 2> 4497424 INFO (zkConnectionManagerCallback-20878-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 4497424 INFO (closeThreadPool-20850-thread-5) [ ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
[junit4] 2> 4497425 INFO (zkConnectionManagerCallback-20882-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 4497425 INFO (closeThreadPool-20850-thread-2) [ ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
[junit4] 2> 4497461 INFO (TEST-HdfsSyncSliceTest.test-seed#[B17B537CC2F9B000]) [ ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 6 in directory /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsSyncSliceTest_B17B537CC2F9B000-001/shard-6-001 of type NRT for shard1
[junit4] 2> 4497463 INFO (closeThreadPool-20850-thread-1) [ ] o.a.s.c.s.e.JettySolrRunner Start Jetty (configured port=0, binding port=0)
[junit4] 2> 4497463 INFO (closeThreadPool-20850-thread-1) [ ] o.a.s.c.s.e.JettySolrRunner Trying to start Jetty on port 0 try number 1 ...
[junit4] 2> 4497463 INFO (closeThreadPool-20850-thread-1) [ ] o.e.j.s.Server jetty-9.4.24.v20191120; built: 2019-11-20T21:37:49.771Z; git: 363d5f2df3a8a28de40604320230664b9c793c16; jvm 11.0.4+10-LTS
[junit4] 2> 4497492 INFO (closeThreadPool-20850-thread-1) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 4497492 INFO (closeThreadPool-20850-thread-1) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 4497492 INFO (closeThreadPool-20850-thread-1) [ ] o.e.j.s.session node0 Scavenging every 600000ms
[junit4] 2> 4497493 INFO (closeThreadPool-20850-thread-1) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@41791929{/,null,AVAILABLE}
[junit4] 2> 4497494 INFO (closeThreadPool-20850-thread-1) [ ] o.e.j.s.AbstractConnector Started ServerConnector@1eff724c{ssl,[ssl, alpn, http/1.1, h2]}{127.0.0.1:41001}
[junit4] 2> 4497494 INFO (closeThreadPool-20850-thread-1) [ ] o.e.j.s.Server Started @4497531ms
[junit4] 2> 4497494 INFO (closeThreadPool-20850-thread-1) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/, solrconfig=solrconfig.xml, solr.data.dir=hdfs://localhost.localdomain:35847/hdfs__localhost.localdomain_35847__home_jenkins_jenkins-slave_workspace_Lucene-Solr-NightlyTests-master_checkout_solr_build_solr-core_test_J2_temp_solr.cloud.hdfs.HdfsSyncSliceTest_B17B537CC2F9B000-001_tempDir-002_jetty6, hostPort=41001, coreRootDirectory=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsSyncSliceTest_B17B537CC2F9B000-001/shard-6-001/cores}
[junit4] 2> 4497494 ERROR (closeThreadPool-20850-thread-1) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 4497494 INFO (closeThreadPool-20850-thread-1) [ ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
[junit4] 2> 4497494 INFO (closeThreadPool-20850-thread-1) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version 9.0.0
[junit4] 2> 4497494 INFO (closeThreadPool-20850-thread-1) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 4497494 INFO (closeThreadPool-20850-thread-1) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr
[junit4] 2> 4497494 INFO (closeThreadPool-20850-thread-1) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2020-02-22T06:55:50.605452Z
[junit4] 2> 4497496 INFO (closeThreadPool-20850-thread-1) [ ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
[junit4] 2> 4497498 INFO (zkConnectionManagerCallback-20884-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 4497498 INFO (closeThreadPool-20850-thread-1) [ ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
[junit4] 2> 4497525 INFO (closeThreadPool-20850-thread-5) [ ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
[junit4] 2> 4497525 INFO (closeThreadPool-20850-thread-5) [ ] o.a.s.c.SolrXmlConfig Loading container configuration from /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsSyncSliceTest_B17B537CC2F9B000-001/shard-5-001/solr.xml
[junit4] 2> 4497528 INFO (closeThreadPool-20850-thread-5) [ ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverWorkLoopDelay is ignored
[junit4] 2> 4497528 INFO (closeThreadPool-20850-thread-5) [ ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverBadNodeExpiration is ignored
[junit4] 2> 4497530 INFO (closeThreadPool-20850-thread-5) [ ] o.a.s.c.SolrXmlConfig MBean server found: com.sun.jmx.mbeanserver.JmxMBeanServer@7425d76c, but no JMX reporters were configured - adding default JMX reporter.
[junit4] 2> 4497626 INFO (closeThreadPool-20850-thread-1) [ ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
[junit4] 2> 4497626 INFO (closeThreadPool-20850-thread-1) [ ] o.a.s.c.SolrXmlConfig Loading container configuration from /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsSyncSliceTest_B17B537CC2F9B000-001/shard-6-001/solr.xml
[junit4] 2> 4497641 INFO (closeThreadPool-20850-thread-1) [ ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverWorkLoopDelay is ignored
[junit4] 2> 4497641 INFO (closeThreadPool-20850-thread-1) [ ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverBadNodeExpiration is ignored
[junit4] 2> 4497642 INFO (closeThreadPool-20850-thread-1) [ ] o.a.s.c.SolrXmlConfig MBean server found: com.sun.jmx.mbeanserver.JmxMBeanServer@7425d76c, but no JMX reporters were configured - adding default JMX reporter.
[junit4] 2> 4497661 INFO (TEST-HdfsSyncSliceTest.test-seed#[B17B537CC2F9B000]) [ ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 7 in directory /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsSyncSliceTest_B17B537CC2F9B000-001/shard-7-001 of type NRT for shard1
[junit4] 2> 4497663 INFO (closeThreadPool-20850-thread-6) [ ] o.a.s.c.s.e.JettySolrRunner Start Jetty (configured port=0, binding port=0)
[junit4] 2> 4497663 INFO (closeThreadPool-20850-thread-6) [ ] o.a.s.c.s.e.JettySolrRunner Trying to start Jetty on port 0 try number 1 ...
[junit4] 2> 4497663 INFO (closeThreadPool-20850-thread-6) [ ] o.e.j.s.Server jetty-9.4.24.v20191120; built: 2019-11-20T21:37:49.771Z; git: 363d5f2df3a8a28de40604320230664b9c793c16; jvm 11.0.4+10-LTS
[junit4] 2> 4497704 INFO (closeThreadPool-20850-thread-6) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 4497704 INFO (closeThreadPool-20850-thread-6) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 4497704 INFO (closeThreadPool-20850-thread-6) [ ] o.e.j.s.session node0 Scavenging every 600000ms
[junit4] 2> 4497709 INFO (closeThreadPool-20850-thread-6) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@78dc3173{/,null,AVAILABLE}
[junit4] 2> 4497710 INFO (closeThreadPool-20850-thread-6) [ ] o.e.j.s.AbstractConnector Started ServerConnector@39ace372{ssl,[ssl, alpn, http/1.1, h2]}{127.0.0.1:39491}
[junit4] 2> 4497710 INFO (closeThreadPool-20850-thread-6) [ ] o.e.j.s.Server Started @4497747ms
[junit4] 2> 4497710 INFO (closeThreadPool-20850-thread-6) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/, solrconfig=solrconfig.xml, solr.data.dir=hdfs://localhost.localdomain:35847/hdfs__localhost.localdomain_35847__home_jenkins_jenkins-slave_workspace_Lucene-Solr-NightlyTests-master_checkout_solr_build_solr-core_test_J2_temp_solr.cloud.hdfs.HdfsSyncSliceTest_B17B537CC2F9B000-001_tempDir-002_jetty7, hostPort=39491, coreRootDirectory=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J2/../../../../../../../../../../../home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsSyncSliceTest_B17B537CC2F9B000-001/shard-7-001/cores, replicaType=NRT}
[junit4] 2> 4497710 ERROR (closeThreadPool-20850-thread-6) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 4497710 INFO (closeThreadPool-20850-thread-6) [ ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
[junit4] 2> 4497710 INFO (closeThreadPool-20850-thread-6) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version 9.0.0
[junit4] 2> 4497710 INFO (closeThreadPool-20850-thread-6) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 4497710 INFO (closeThreadPool-20850-thread-6) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr
[junit4] 2> 4497711 INFO (closeThreadPool-20850-thread-6) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2020-02-22T06:55:50.821992Z
[junit4] 2> 4497731 INFO (closeThreadPool-20850-thread-6) [ ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
[junit4] 2> 4497743 INFO (closeThreadPool-20850-thread-2) [n:127.0.0.1:46737_ ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
[junit4] 2> 4497747 INFO (zkConnectionManagerCallback-20888-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 4497747 INFO (closeThreadPool-20850-thread-6) [ ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
[junit4] 2> 4497754 INFO (zkConnectionManagerCallback-20890-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 4497754 INFO (closeThreadPool-20850-thread-2) [n:127.0.0.1:46737_ ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
[junit4] 2> 4497767 INFO (closeThreadPool-20850-thread-4) [ ] o.a.s.h.c.HttpShardHandlerFactory Host whitelist initialized: WhitelistHostChecker [whitelistHosts=null, whitelistHostCheckingEnabled=false]
[junit4] 2> 4497774 WARN (closeThreadPool-20850-thread-4) [ ] o.e.j.u.s.S.config Trusting all certificates configured for Client@5b43fa34[provider=null,keyStore=null,trustStore=null]
[junit4] 2> 4497774 WARN (closeThreadPool-20850-thread-4) [ ] o.e.j.u.s.S.config No Client EndPointIdentificationAlgorithm configured for Client@5b43fa34[provider=null,keyStore=null,trustStore=null]
[junit4] 2> 4497780 WARN (closeThreadPool-20850-thread-4) [ ] o.e.j.u.s.S.config Trusting all certificates configured for Client@5017c928[provider=null,keyStore=null,trustStore=null]
[junit4] 2> 4497780 WARN (closeThreadPool-20850-thread-4) [ ] o.e.j.u.s.S.config No Client EndPointIdentificationAlgorith
[...truncated too long message...]
iveMethodAccessorImpl.invoke0(Native Method)
[junit4] > at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
[junit4] > at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
[junit4] > at java.base/java.lang.reflect.Method.invoke(Method.java:566)
[junit4] > at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:1082)
[junit4] > at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:1054)
[junit4] > at java.base/java.lang.Thread.run(Thread.java:834)
[junit4] 2> 4538110 WARN (SUITE-HdfsSyncSliceTest-seed#[B17B537CC2F9B000]-worker) [ ] o.a.h.h.s.d.DirectoryScanner DirectoryScanner: shutdown has been called
[junit4] 2> 4538114 INFO (SUITE-HdfsSyncSliceTest-seed#[B17B537CC2F9B000]-worker) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.w.WebAppContext@3e6b8a27{datanode,/,null,UNAVAILABLE}{jar:file:/home/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-3.2.0-tests.jar!/webapps/datanode}
[junit4] 2> 4538114 INFO (SUITE-HdfsSyncSliceTest-seed#[B17B537CC2F9B000]-worker) [ ] o.e.j.s.AbstractConnector Stopped ServerConnector@61c41385{HTTP/1.1,[http/1.1]}{localhost:0}
[junit4] 2> 4538114 INFO (SUITE-HdfsSyncSliceTest-seed#[B17B537CC2F9B000]-worker) [ ] o.e.j.s.session node0 Stopped scavenging
[junit4] 2> 4538114 INFO (SUITE-HdfsSyncSliceTest-seed#[B17B537CC2F9B000]-worker) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@4fff8a02{static,/static,jar:file:/home/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-3.2.0-tests.jar!/webapps/static,UNAVAILABLE}
[junit4] 2> 4538115 WARN (BP-1373479189-127.0.0.1-1582354540363 heartbeating to localhost.localdomain/127.0.0.1:35847) [ ] o.a.h.h.s.d.IncrementalBlockReportManager IncrementalBlockReportManager interrupted
[junit4] 2> 4538115 WARN (BP-1373479189-127.0.0.1-1582354540363 heartbeating to localhost.localdomain/127.0.0.1:35847) [ ] o.a.h.h.s.d.DataNode Ending block pool service for: Block pool BP-1373479189-127.0.0.1-1582354540363 (Datanode Uuid 83f6d46f-5e34-4418-9dbd-b9676bf8fdec) service to localhost.localdomain/127.0.0.1:35847
[junit4] 2> 4538135 INFO (SUITE-HdfsSyncSliceTest-seed#[B17B537CC2F9B000]-worker) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.w.WebAppContext@5d0297f3{hdfs,/,null,UNAVAILABLE}{jar:file:/home/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-3.2.0-tests.jar!/webapps/hdfs}
[junit4] 2> 4538135 INFO (SUITE-HdfsSyncSliceTest-seed#[B17B537CC2F9B000]-worker) [ ] o.e.j.s.AbstractConnector Stopped ServerConnector@578493af{HTTP/1.1,[http/1.1]}{localhost.localdomain:0}
[junit4] 2> 4538135 INFO (SUITE-HdfsSyncSliceTest-seed#[B17B537CC2F9B000]-worker) [ ] o.e.j.s.session node0 Stopped scavenging
[junit4] 2> 4538135 INFO (SUITE-HdfsSyncSliceTest-seed#[B17B537CC2F9B000]-worker) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@a0d58a2{static,/static,jar:file:/home/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-3.2.0-tests.jar!/webapps/static,UNAVAILABLE}
[junit4] 2> NOTE: leaving temporary files on disk at: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsSyncSliceTest_B17B537CC2F9B000-001
[junit4] 2> Feb 22, 2020 6:56:31 AM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
[junit4] 2> WARNING: Will linger awaiting termination of 17 leaked thread(s).
[junit4] 2> NOTE: test params are: codec=Asserting(Lucene84): {other_tl1=PostingsFormat(name=Direct), range_facet_l_dv=PostingsFormat(name=LuceneVarGapFixedInterval), multiDefault=PostingsFormat(name=MockRandom), _root_=PostingsFormat(name=Direct), intDefault=PostingsFormat(name=Direct), a_i1=PostingsFormat(name=Direct), range_facet_l=PostingsFormat(name=Direct), rnd_b=PostingsFormat(name=LuceneVarGapFixedInterval), a_t=PostingsFormat(name=Direct), id_i1=PostingsFormat(name=MockRandom), id=PostingsFormat(name=LuceneVarGapFixedInterval), range_facet_i_dv=PostingsFormat(name=Direct), text=PostingsFormat(name=Asserting), timestamp=PostingsFormat(name=Direct)}, docValues:{other_tl1=DocValuesFormat(name=Asserting), range_facet_l_dv=DocValuesFormat(name=Lucene80), _version_=DocValuesFormat(name=Asserting), intDefault=DocValuesFormat(name=Asserting), id_i1=DocValuesFormat(name=Asserting), a_i1=DocValuesFormat(name=Asserting), range_facet_i_dv=DocValuesFormat(name=Asserting), intDvoDefault=DocValuesFormat(name=Lucene80), timestamp=DocValuesFormat(name=Asserting), range_facet_l=DocValuesFormat(name=Asserting)}, maxPointsInLeafNode=1656, maxMBSortInHeap=6.453153857082324, sim=Asserting(org.apache.lucene.search.similarities.AssertingSimilarity@2a89604a), locale=ug, timezone=Africa/Mogadishu
[junit4] 2> NOTE: Linux 4.15.0-54-generic amd64/Oracle Corporation 11.0.4 (64-bit)/cpus=4,threads=4,free=175436000,total=536870912
[junit4] 2> NOTE: All tests run in this JVM: [TestZkAclsWithHadoopAuth, TestManagedSynonymFilterFactory, TestClassicSimilarityFactory, SaslZkACLProviderTest, TestPHPSerializedResponseWriter, TestSolrJ, TestBinaryResponseWriter, ParsingFieldUpdateProcessorsTest, TestMissingGroups, TestJmxIntegration, BadIndexSchemaTest, TestFastOutputStream, HighlighterConfigTest, TestUnifiedSolrHighlighterWithoutStoredId, HttpPartitionOnCommitTest, TestDocumentBuilder, IndexSchemaRuntimeFieldTest, TestRawTransformer, PolyFieldTest, TestDynamicLoading, RequestHandlersTest, TestPerFieldSimilarityWithDefaultOverride, TestCorePropertiesReload, BlockDirectoryTest, TestSimUtils, LeaderFailoverAfterPartitionTest, MoreLikeThisHandlerTest, TestBulkSchemaAPI, StatsReloadRaceTest, InfoHandlerTest, TimeRoutedAliasUpdateProcessorTest, CopyFieldTest, SearchRateTriggerIntegrationTest, AuditLoggerPluginTest, TestCollectionAPI, TestPolicyCloud, TestPseudoReturnFields, OverseerTaskQueueTest, TestSSLRandomization, ComputePlanActionTest, TestChildDocTransformer, NodeAddedTriggerIntegrationTest, FacetPivotSmallTest, MetricsHistoryWithAuthIntegrationTest, DocumentAnalysisRequestHandlerTest, TestConfigSetsAPIExclusivity, WrapperMergePolicyFactoryTest, TestReplicationHandler, ReplaceNodeTest, TestLocalFSCloudBackupRestore, TestPKIAuthenticationPlugin, ClassificationUpdateProcessorIntegrationTest, DistributedDebugComponentTest, HttpTriggerListenerTest, TestDynamicFieldResource, SortByFunctionTest, ConcurrentDeleteAndCreateCollectionTest, TestSearchPerf, TestCollectionAPIs, HdfsTlogReplayBufferedWhileIndexingTest, TestRTGBase, ClusterStateMockUtilTest, SharedFSAutoReplicaFailoverTest, TestExactStatsCache, UnloadDistributedZkTest, OverseerCollectionConfigSetProcessorTest, TestDistributedTracing, HttpSolrCallGetCoreTest, ConvertedLegacyTest, SuggestComponentTest, TestSolrJacksonAnnotation, AdminHandlersProxyTest, SolrJmxReporterCloudTest, SolrTestCaseJ4Test, SolrCloudExampleTest, TestCloudJSONFacetJoinDomain, TestDefaultStatsCache, AsyncCallRequestStatusResponseTest, TestValueSourceCache, SolrCloudReportersTest, TestConfigOverlay, DistributedFacetPivotSmallTest, BigEndianAscendingWordSerializerTest, DeleteLastCustomShardedReplicaTest, TestCollationField, LeaderElectionContextKeyTest, PKIAuthenticationIntegrationTest, TestIBSimilarityFactory, CreateCollectionCleanupTest, CoreMergeIndexesAdminHandlerTest, IgnoreLargeDocumentProcessorFactoryTest, TestHighFrequencyDictionaryFactory, SpatialRPTFieldTypeTest, TestNumericTokenStream, TestFieldCacheSortRandom, ConcurrentCreateRoutedAliasTest, TestPackages, TestStandardQParsers, TestSuggestSpellingConverter, TestDistributedGrouping, TestTextField, SubstringBytesRefFilterTest, ZookeeperStatusHandlerTest, TestOverriddenPrefixQueryForCustomFieldType, TestDeleteCollectionOnDownNodes, TestSubQueryTransformer, UpdateLogTest, TestLegacyFieldCache, PluginInfoTest, DistributedVersionInfoTest, DistributedIntervalFacetingTest, CollectionPropsTest, SignificantTermsQParserPluginTest, DateRangeFieldTest, DOMUtilTest, CdcrBidirectionalTest, TestSolrConfigHandler, ShardSplitTest, ChaosMonkeyNothingIsSafeWithPullReplicasTest, AtomicUpdatesTest, StressHdfsTest, TestMacros, TestMultiValuedNumericRangeQuery, TestSimComputePlanAction, SchemaApiFailureTest, MBeansHandlerTest, SolrRequestParserTest, CdcrWithNodesRestartsTest, DistributedFacetSimpleRefinementLongTailTest, TestCloudSearcherWarming, DocValuesTest, CustomHighlightComponentTest, V2StandaloneTest, DistribDocExpirationUpdateProcessorTest, TestUtils, TestOmitPositions, TermsComponentTest, LeaderVoteWaitTimeoutTest, TestAddFieldRealTimeGet, TestAnalyzeInfixSuggestions, SolrLogPostToolTest, TestExactStatsCacheCloud, PropertiesRequestHandlerTest, NodeMutatorTest, CdcrReplicationHandlerTest, TestTlogReplayVsRecovery, TestTlogReplica, TestTolerantUpdateProcessorCloud, TestTolerantUpdateProcessorRandomCloud, TestUtilizeNode, TestWaitForStateWithJettyShutdowns, TestWithCollection, TestZkChroot, TlogReplayBufferedWhileIndexingTest, ZkControllerTest, CollectionTooManyReplicasTest, CollectionsAPIDistributedZkTest, TestRequestStatusCollectionAPI, AutoAddReplicasPlanActionTest, IndexSizeTriggerSizeEstimationTest, IndexSizeTriggerTest, MetricTriggerIntegrationTest, NodeMarkersRegistrationTest, RestoreTriggerStateTest, SearchRateTriggerTest, TriggerSetPropertiesIntegrationTest, TestSimDistributedQueue, TestSimExecutePlanAction, TestSimExtremeIndexing, TestSimGenericDistributedQueue, TestSimLargeCluster, HdfsBasicDistributedZk2Test, HdfsChaosMonkeySafeLeaderTest, HdfsNNFailoverTest, HdfsRecoveryZkTest, HdfsRestartWhileUpdatingTest, HdfsSyncSliceTest]
[junit4] Completed [602/904 (1!)] on J2 in 54.12s, 1 test, 1 failure <<< FAILURES!
[...truncated 53516 lines...]
[JENKINS] Lucene-Solr-NightlyTests-master - Build # 2110 - Still
unstable
Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-master/2110/
1 tests failed.
FAILED: org.apache.solr.cloud.api.collections.CollectionsAPIAsyncDistributedZkTest.testAsyncIdBackCompat
Error Message:
Expecting exception
Stack Trace:
java.lang.AssertionError: Expecting exception
at __randomizedtesting.SeedInfo.seed([EC7974BA469EE123:3249E716EA720573]:0)
at org.junit.Assert.fail(Assert.java:88)
at org.apache.solr.cloud.api.collections.CollectionsAPIAsyncDistributedZkTest.testAsyncIdBackCompat(CollectionsAPIAsyncDistributedZkTest.java:288)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:566)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1754)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:942)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:978)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:992)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:370)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:819)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:470)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:951)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:836)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:887)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:898)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:370)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.lambda$forkTimeoutingTask$0(ThreadLeakControl.java:826)
at java.base/java.lang.Thread.run(Thread.java:834)
Build Log:
[...truncated 14962 lines...]
[junit4] Suite: org.apache.solr.cloud.api.collections.CollectionsAPIAsyncDistributedZkTest
[junit4] 2> 5893310 INFO (SUITE-CollectionsAPIAsyncDistributedZkTest-seed#[EC7974BA469EE123]-worker) [ ] o.a.s.SolrTestCase Setting 'solr.default.confdir' system property to test-framework derived value of '/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/server/solr/configsets/_default/conf'
[junit4] 2> 5893310 INFO (SUITE-CollectionsAPIAsyncDistributedZkTest-seed#[EC7974BA469EE123]-worker) [ ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
[junit4] 2> 5893311 INFO (SUITE-CollectionsAPIAsyncDistributedZkTest-seed#[EC7974BA469EE123]-worker) [ ] o.a.s.SolrTestCaseJ4 Created dataDir: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.api.collections.CollectionsAPIAsyncDistributedZkTest_EC7974BA469EE123-001/data-dir-198-001
[junit4] 2> 5893311 WARN (SUITE-CollectionsAPIAsyncDistributedZkTest-seed#[EC7974BA469EE123]-worker) [ ] o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=24 numCloses=24
[junit4] 2> 5893311 INFO (SUITE-CollectionsAPIAsyncDistributedZkTest-seed#[EC7974BA469EE123]-worker) [ ] o.a.s.SolrTestCaseJ4 Using PointFields (NUMERIC_POINTS_SYSPROP=true) w/NUMERIC_DOCVALUES_SYSPROP=false
[junit4] 2> 5893312 INFO (SUITE-CollectionsAPIAsyncDistributedZkTest-seed#[EC7974BA469EE123]-worker) [ ] o.a.s.SolrTestCaseJ4 Randomized ssl (true) and clientAuth (false) via: @org.apache.solr.util.RandomizeSSL(reason="", value=0.0/0.0, ssl=0.0/0.0, clientAuth=0.0/0.0)
[junit4] 2> 5893324 INFO (TEST-CollectionsAPIAsyncDistributedZkTest.testSolrJAPICalls-seed#[EC7974BA469EE123]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testSolrJAPICalls
[junit4] 2> 5893325 INFO (TEST-CollectionsAPIAsyncDistributedZkTest.testSolrJAPICalls-seed#[EC7974BA469EE123]) [ ] o.a.s.c.MiniSolrCloudCluster Starting cluster of 2 servers in /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.api.collections.CollectionsAPIAsyncDistributedZkTest_EC7974BA469EE123-001/tempDir-001
[junit4] 2> 5893325 INFO (TEST-CollectionsAPIAsyncDistributedZkTest.testSolrJAPICalls-seed#[EC7974BA469EE123]) [ ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
[junit4] 2> 5893326 INFO (ZkTestServer Run Thread) [ ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 5893326 INFO (ZkTestServer Run Thread) [ ] o.a.s.c.ZkTestServer Starting server
[junit4] 2> 5893426 INFO (TEST-CollectionsAPIAsyncDistributedZkTest.testSolrJAPICalls-seed#[EC7974BA469EE123]) [ ] o.a.s.c.ZkTestServer start zk server on port:46687
[junit4] 2> 5893426 INFO (TEST-CollectionsAPIAsyncDistributedZkTest.testSolrJAPICalls-seed#[EC7974BA469EE123]) [ ] o.a.s.c.ZkTestServer waitForServerUp: 127.0.0.1:46687
[junit4] 2> 5893426 INFO (TEST-CollectionsAPIAsyncDistributedZkTest.testSolrJAPICalls-seed#[EC7974BA469EE123]) [ ] o.a.s.c.ZkTestServer parse host and port list: 127.0.0.1:46687
[junit4] 2> 5893426 INFO (TEST-CollectionsAPIAsyncDistributedZkTest.testSolrJAPICalls-seed#[EC7974BA469EE123]) [ ] o.a.s.c.ZkTestServer connecting to 127.0.0.1 46687
[junit4] 2> 5893428 INFO (TEST-CollectionsAPIAsyncDistributedZkTest.testSolrJAPICalls-seed#[EC7974BA469EE123]) [ ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
[junit4] 2> 5893430 INFO (zkConnectionManagerCallback-14510-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 5893430 INFO (TEST-CollectionsAPIAsyncDistributedZkTest.testSolrJAPICalls-seed#[EC7974BA469EE123]) [ ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
[junit4] 2> 5893433 INFO (TEST-CollectionsAPIAsyncDistributedZkTest.testSolrJAPICalls-seed#[EC7974BA469EE123]) [ ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
[junit4] 2> 5893434 INFO (zkConnectionManagerCallback-14512-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 5893434 INFO (TEST-CollectionsAPIAsyncDistributedZkTest.testSolrJAPICalls-seed#[EC7974BA469EE123]) [ ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
[junit4] 2> 5893435 INFO (TEST-CollectionsAPIAsyncDistributedZkTest.testSolrJAPICalls-seed#[EC7974BA469EE123]) [ ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
[junit4] 2> 5893436 INFO (zkConnectionManagerCallback-14514-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 5893436 INFO (TEST-CollectionsAPIAsyncDistributedZkTest.testSolrJAPICalls-seed#[EC7974BA469EE123]) [ ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
[junit4] 2> 5893548 INFO (jetty-launcher-14515-thread-1) [ ] o.a.s.c.s.e.JettySolrRunner Start Jetty (configured port=0, binding port=0)
[junit4] 2> 5893548 INFO (jetty-launcher-14515-thread-2) [ ] o.a.s.c.s.e.JettySolrRunner Start Jetty (configured port=0, binding port=0)
[junit4] 2> 5893548 INFO (jetty-launcher-14515-thread-1) [ ] o.a.s.c.s.e.JettySolrRunner Trying to start Jetty on port 0 try number 1 ...
[junit4] 2> 5893548 INFO (jetty-launcher-14515-thread-2) [ ] o.a.s.c.s.e.JettySolrRunner Trying to start Jetty on port 0 try number 1 ...
[junit4] 2> 5893548 INFO (jetty-launcher-14515-thread-1) [ ] o.e.j.s.Server jetty-9.4.24.v20191120; built: 2019-11-20T21:37:49.771Z; git: 363d5f2df3a8a28de40604320230664b9c793c16; jvm 11.0.4+10-LTS
[junit4] 2> 5893548 INFO (jetty-launcher-14515-thread-2) [ ] o.e.j.s.Server jetty-9.4.24.v20191120; built: 2019-11-20T21:37:49.771Z; git: 363d5f2df3a8a28de40604320230664b9c793c16; jvm 11.0.4+10-LTS
[junit4] 2> 5893549 INFO (jetty-launcher-14515-thread-1) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 5893549 INFO (jetty-launcher-14515-thread-1) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 5893549 INFO (jetty-launcher-14515-thread-1) [ ] o.e.j.s.session node0 Scavenging every 600000ms
[junit4] 2> 5893549 INFO (jetty-launcher-14515-thread-2) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 5893549 INFO (jetty-launcher-14515-thread-2) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 5893549 INFO (jetty-launcher-14515-thread-2) [ ] o.e.j.s.session node0 Scavenging every 600000ms
[junit4] 2> 5893550 INFO (jetty-launcher-14515-thread-1) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@31e5a38c{/solr,null,AVAILABLE}
[junit4] 2> 5893550 INFO (jetty-launcher-14515-thread-2) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@2a27e895{/solr,null,AVAILABLE}
[junit4] 2> 5893563 INFO (jetty-launcher-14515-thread-2) [ ] o.e.j.s.AbstractConnector Started ServerConnector@6f9901db{ssl,[ssl, alpn, http/1.1, h2]}{127.0.0.1:34409}
[junit4] 2> 5893563 INFO (jetty-launcher-14515-thread-2) [ ] o.e.j.s.Server Started @5893601ms
[junit4] 2> 5893563 INFO (jetty-launcher-14515-thread-2) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=34409}
[junit4] 2> 5893563 ERROR (jetty-launcher-14515-thread-2) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 5893563 INFO (jetty-launcher-14515-thread-2) [ ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
[junit4] 2> 5893563 INFO (jetty-launcher-14515-thread-2) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 9.0.0
[junit4] 2> 5893564 INFO (jetty-launcher-14515-thread-2) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 5893564 INFO (jetty-launcher-14515-thread-2) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr
[junit4] 2> 5893564 INFO (jetty-launcher-14515-thread-2) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2020-02-21T07:18:25.914057Z
[junit4] 2> 5893565 INFO (jetty-launcher-14515-thread-2) [ ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
[junit4] 2> 5893566 INFO (zkConnectionManagerCallback-14517-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 5893566 INFO (jetty-launcher-14515-thread-2) [ ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
[junit4] 2> 5893566 INFO (jetty-launcher-14515-thread-2) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 5893571 INFO (jetty-launcher-14515-thread-1) [ ] o.e.j.s.AbstractConnector Started ServerConnector@556182c5{ssl,[ssl, alpn, http/1.1, h2]}{127.0.0.1:40724}
[junit4] 2> 5893571 INFO (jetty-launcher-14515-thread-1) [ ] o.e.j.s.Server Started @5893609ms
[junit4] 2> 5893571 INFO (jetty-launcher-14515-thread-1) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=40724}
[junit4] 2> 5893571 ERROR (jetty-launcher-14515-thread-1) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 5893571 INFO (jetty-launcher-14515-thread-1) [ ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
[junit4] 2> 5893571 INFO (jetty-launcher-14515-thread-1) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 9.0.0
[junit4] 2> 5893572 INFO (jetty-launcher-14515-thread-1) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 5893572 INFO (jetty-launcher-14515-thread-1) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr
[junit4] 2> 5893572 INFO (jetty-launcher-14515-thread-1) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2020-02-21T07:18:25.922054Z
[junit4] 2> 5893572 INFO (jetty-launcher-14515-thread-1) [ ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
[junit4] 2> 5893574 INFO (zkConnectionManagerCallback-14519-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 5893574 INFO (jetty-launcher-14515-thread-1) [ ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
[junit4] 2> 5893574 INFO (jetty-launcher-14515-thread-1) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 5895738 INFO (jetty-launcher-14515-thread-2) [ ] o.a.s.h.c.HttpShardHandlerFactory Host whitelist initialized: WhitelistHostChecker [whitelistHosts=null, whitelistHostCheckingEnabled=true]
[junit4] 2> 5895740 WARN (jetty-launcher-14515-thread-2) [ ] o.e.j.u.s.S.config Trusting all certificates configured for Client@e3ade7[provider=null,keyStore=null,trustStore=null]
[junit4] 2> 5895740 WARN (jetty-launcher-14515-thread-2) [ ] o.e.j.u.s.S.config No Client EndPointIdentificationAlgorithm configured for Client@e3ade7[provider=null,keyStore=null,trustStore=null]
[junit4] 2> 5895743 WARN (jetty-launcher-14515-thread-2) [ ] o.e.j.u.s.S.config Trusting all certificates configured for Client@1ad070b8[provider=null,keyStore=null,trustStore=null]
[junit4] 2> 5895743 WARN (jetty-launcher-14515-thread-2) [ ] o.e.j.u.s.S.config No Client EndPointIdentificationAlgorithm configured for Client@1ad070b8[provider=null,keyStore=null,trustStore=null]
[junit4] 2> 5895744 INFO (jetty-launcher-14515-thread-2) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:46687/solr
[junit4] 2> 5895745 INFO (jetty-launcher-14515-thread-2) [ ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
[junit4] 2> 5895746 INFO (zkConnectionManagerCallback-14527-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 5895746 INFO (jetty-launcher-14515-thread-2) [ ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
[junit4] 2> 5895849 INFO (jetty-launcher-14515-thread-2) [n:127.0.0.1:34409_solr ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
[junit4] 2> 5895849 INFO (zkConnectionManagerCallback-14529-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 5895850 INFO (jetty-launcher-14515-thread-2) [n:127.0.0.1:34409_solr ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
[junit4] 2> 5896013 INFO (jetty-launcher-14515-thread-2) [n:127.0.0.1:34409_solr ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:34409_solr
[junit4] 2> 5896014 INFO (jetty-launcher-14515-thread-2) [n:127.0.0.1:34409_solr ] o.a.s.c.Overseer Overseer (id=72322689862139910-127.0.0.1:34409_solr-n_0000000000) starting
[junit4] 2> 5896020 INFO (OverseerStateUpdate-72322689862139910-127.0.0.1:34409_solr-n_0000000000) [n:127.0.0.1:34409_solr ] o.a.s.c.Overseer Starting to work on the main queue : 127.0.0.1:34409_solr
[junit4] 2> 5896021 INFO (jetty-launcher-14515-thread-2) [n:127.0.0.1:34409_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:34409_solr
[junit4] 2> 5896030 INFO (zkCallback-14528-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 5896036 INFO (jetty-launcher-14515-thread-2) [n:127.0.0.1:34409_solr ] o.a.s.p.PackageLoader /packages.json updated to version -1
[junit4] 2> 5896036 WARN (jetty-launcher-14515-thread-2) [n:127.0.0.1:34409_solr ] o.a.s.c.CoreContainer Not all security plugins configured! authentication=disabled authorization=disabled. Solr is only as secure as you make it. Consider configuring authentication/authorization before exposing Solr to users internal or external. See https://s.apache.org/solrsecurity for more info
[junit4] 2> 5896089 INFO (jetty-launcher-14515-thread-2) [n:127.0.0.1:34409_solr ] o.a.s.h.a.MetricsHistoryHandler No .system collection, keeping metrics history in memory.
[junit4] 2> 5896163 INFO (jetty-launcher-14515-thread-2) [n:127.0.0.1:34409_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_34409.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@5ef91bdc
[junit4] 2> 5896218 INFO (jetty-launcher-14515-thread-2) [n:127.0.0.1:34409_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_34409.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@5ef91bdc
[junit4] 2> 5896218 INFO (jetty-launcher-14515-thread-2) [n:127.0.0.1:34409_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_34409.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@5ef91bdc
[junit4] 2> 5896220 INFO (jetty-launcher-14515-thread-2) [n:127.0.0.1:34409_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.api.collections.CollectionsAPIAsyncDistributedZkTest_EC7974BA469EE123-001/tempDir-001/node2/.
[junit4] 2> 5896391 INFO (jetty-launcher-14515-thread-1) [ ] o.a.s.h.c.HttpShardHandlerFactory Host whitelist initialized: WhitelistHostChecker [whitelistHosts=null, whitelistHostCheckingEnabled=true]
[junit4] 2> 5896392 WARN (jetty-launcher-14515-thread-1) [ ] o.e.j.u.s.S.config Trusting all certificates configured for Client@6fded1dc[provider=null,keyStore=null,trustStore=null]
[junit4] 2> 5896392 WARN (jetty-launcher-14515-thread-1) [ ] o.e.j.u.s.S.config No Client EndPointIdentificationAlgorithm configured for Client@6fded1dc[provider=null,keyStore=null,trustStore=null]
[junit4] 2> 5896395 WARN (jetty-launcher-14515-thread-1) [ ] o.e.j.u.s.S.config Trusting all certificates configured for Client@4fc0d905[provider=null,keyStore=null,trustStore=null]
[junit4] 2> 5896396 WARN (jetty-launcher-14515-thread-1) [ ] o.e.j.u.s.S.config No Client EndPointIdentificationAlgorithm configured for Client@4fc0d905[provider=null,keyStore=null,trustStore=null]
[junit4] 2> 5896397 INFO (jetty-launcher-14515-thread-1) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:46687/solr
[junit4] 2> 5896398 INFO (jetty-launcher-14515-thread-1) [ ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
[junit4] 2> 5896399 INFO (zkConnectionManagerCallback-14539-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 5896399 INFO (jetty-launcher-14515-thread-1) [ ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
[junit4] 2> 5896502 INFO (jetty-launcher-14515-thread-1) [n:127.0.0.1:40724_solr ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
[junit4] 2> 5896503 INFO (zkConnectionManagerCallback-14541-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 5896503 INFO (jetty-launcher-14515-thread-1) [n:127.0.0.1:40724_solr ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
[junit4] 2> 5896508 INFO (jetty-launcher-14515-thread-1) [n:127.0.0.1:40724_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 5896513 INFO (jetty-launcher-14515-thread-1) [n:127.0.0.1:40724_solr ] o.a.s.c.ZkController Publish node=127.0.0.1:40724_solr as DOWN
[junit4] 2> 5896514 INFO (jetty-launcher-14515-thread-1) [n:127.0.0.1:40724_solr ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 5896514 INFO (jetty-launcher-14515-thread-1) [n:127.0.0.1:40724_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:40724_solr
[junit4] 2> 5896515 INFO (zkCallback-14528-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 5896515 INFO (zkCallback-14540-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 5896517 INFO (jetty-launcher-14515-thread-1) [n:127.0.0.1:40724_solr ] o.a.s.p.PackageLoader /packages.json updated to version -1
[junit4] 2> 5896517 WARN (jetty-launcher-14515-thread-1) [n:127.0.0.1:40724_solr ] o.a.s.c.CoreContainer Not all security plugins configured! authentication=disabled authorization=disabled. Solr is only as secure as you make it. Consider configuring authentication/authorization before exposing Solr to users internal or external. See https://s.apache.org/solrsecurity for more info
[junit4] 2> 5896684 INFO (jetty-launcher-14515-thread-1) [n:127.0.0.1:40724_solr ] o.a.s.h.a.MetricsHistoryHandler No .system collection, keeping metrics history in memory.
[junit4] 2> 5896736 INFO (jetty-launcher-14515-thread-1) [n:127.0.0.1:40724_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_40724.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@5ef91bdc
[junit4] 2> 5896751 INFO (jetty-launcher-14515-thread-1) [n:127.0.0.1:40724_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_40724.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@5ef91bdc
[junit4] 2> 5896751 INFO (jetty-launcher-14515-thread-1) [n:127.0.0.1:40724_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_40724.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@5ef91bdc
[junit4] 2> 5896753 INFO (jetty-launcher-14515-thread-1) [n:127.0.0.1:40724_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.api.collections.CollectionsAPIAsyncDistributedZkTest_EC7974BA469EE123-001/tempDir-001/node1/.
[junit4] 2> 5896789 INFO (TEST-CollectionsAPIAsyncDistributedZkTest.testSolrJAPICalls-seed#[EC7974BA469EE123]) [ ] o.a.s.c.MiniSolrCloudCluster waitForAllNodes: numServers=2
[junit4] 2> 5896790 INFO (TEST-CollectionsAPIAsyncDistributedZkTest.testSolrJAPICalls-seed#[EC7974BA469EE123]) [ ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
[junit4] 2> 5896792 INFO (zkConnectionManagerCallback-14550-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 5896792 INFO (TEST-CollectionsAPIAsyncDistributedZkTest.testSolrJAPICalls-seed#[EC7974BA469EE123]) [ ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
[junit4] 2> 5896793 INFO (TEST-CollectionsAPIAsyncDistributedZkTest.testSolrJAPICalls-seed#[EC7974BA469EE123]) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
[junit4] 2> 5896795 INFO (TEST-CollectionsAPIAsyncDistributedZkTest.testSolrJAPICalls-seed#[EC7974BA469EE123]) [ ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:46687/solr ready
[junit4] 2> 5896851 INFO (qtp1603922357-51731) [n:127.0.0.1:40724_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params async=631967a1-fbbc-4419-93ff-adf14a59e025&collection.configName=conf1&name=testasynccollectioncreation&nrtReplicas=1&action=CREATE&numShards=1&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 5896853 INFO (qtp1603922357-51731) [n:127.0.0.1:40724_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={async=631967a1-fbbc-4419-93ff-adf14a59e025&collection.configName=conf1&name=testasynccollectioncreation&nrtReplicas=1&action=CREATE&numShards=1&wt=javabin&version=2} status=0 QTime=2
[junit4] 2> 5896854 INFO (qtp1603922357-51735) [n:127.0.0.1:40724_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :requeststatus with params requestid=631967a1-fbbc-4419-93ff-adf14a59e025&action=REQUESTSTATUS&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 5896856 INFO (qtp1603922357-51735) [n:127.0.0.1:40724_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={requestid=631967a1-fbbc-4419-93ff-adf14a59e025&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=1
[junit4] 2> 5896856 INFO (OverseerThreadFactory-11366-thread-1-processing-n:127.0.0.1:34409_solr) [n:127.0.0.1:34409_solr ] o.a.s.c.a.c.CreateCollectionCmd Create collection testasynccollectioncreation
[junit4] 2> 5896965 INFO (OverseerStateUpdate-72322689862139910-127.0.0.1:34409_solr-n_0000000000) [n:127.0.0.1:34409_solr ] o.a.s.c.o.SliceMutator createReplica() {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"testasynccollectioncreation",
[junit4] 2> "shard":"shard1",
[junit4] 2> "core":"testasynccollectioncreation_shard1_replica_n1",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"https://127.0.0.1:40724/solr",
[junit4] 2> "node_name":"127.0.0.1:40724_solr",
[junit4] 2> "type":"NRT",
[junit4] 2> "waitForFinalState":"false"}
[junit4] 2> 5897179 INFO (qtp1603922357-51735) [n:127.0.0.1:40724_solr x:testasynccollectioncreation_shard1_replica_n1 ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=conf1&newCollection=true&collection=testasynccollectioncreation&version=2&replicaType=NRT&async=631967a1-fbbc-4419-93ff-adf14a59e0254045044898596921&coreNodeName=core_node2&name=testasynccollectioncreation_shard1_replica_n1&action=CREATE&numShards=1&shard=shard1&wt=javabin} status=0 QTime=0
[junit4] 2> 5897179 INFO (parallelCoreAdminExecutor-11374-thread-1-processing-n:127.0.0.1:40724_solr x:testasynccollectioncreation_shard1_replica_n1 631967a1-fbbc-4419-93ff-adf14a59e0254045044898596921 CREATE) [n:127.0.0.1:40724_solr x:testasynccollectioncreation_shard1_replica_n1 ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf1&newCollection=true&collection=testasynccollectioncreation&version=2&replicaType=NRT&async=631967a1-fbbc-4419-93ff-adf14a59e0254045044898596921&coreNodeName=core_node2&name=testasynccollectioncreation_shard1_replica_n1&action=CREATE&numShards=1&shard=shard1&wt=javabin
[junit4] 2> 5897238 INFO (parallelCoreAdminExecutor-11374-thread-1-processing-n:127.0.0.1:40724_solr x:testasynccollectioncreation_shard1_replica_n1 631967a1-fbbc-4419-93ff-adf14a59e0254045044898596921 CREATE) [n:127.0.0.1:40724_solr c:testasynccollectioncreation s:shard1 r:core_node2 x:testasynccollectioncreation_shard1_replica_n1 ] o.a.s.c.SolrConfig Using Lucene MatchVersion: 9.0.0
[junit4] 2> 5897277 INFO (parallelCoreAdminExecutor-11374-thread-1-processing-n:127.0.0.1:40724_solr x:testasynccollectioncreation_shard1_replica_n1 631967a1-fbbc-4419-93ff-adf14a59e0254045044898596921 CREATE) [n:127.0.0.1:40724_solr c:testasynccollectioncreation s:shard1 r:core_node2 x:testasynccollectioncreation_shard1_replica_n1 ] o.a.s.s.IndexSchema [testasynccollectioncreation_shard1_replica_n1] Schema name=minimal
[junit4] 2> 5897280 INFO (parallelCoreAdminExecutor-11374-thread-1-processing-n:127.0.0.1:40724_solr x:testasynccollectioncreation_shard1_replica_n1 631967a1-fbbc-4419-93ff-adf14a59e0254045044898596921 CREATE) [n:127.0.0.1:40724_solr c:testasynccollectioncreation s:shard1 r:core_node2 x:testasynccollectioncreation_shard1_replica_n1 ] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
[junit4] 2> 5897280 INFO (parallelCoreAdminExecutor-11374-thread-1-processing-n:127.0.0.1:40724_solr x:testasynccollectioncreation_shard1_replica_n1 631967a1-fbbc-4419-93ff-adf14a59e0254045044898596921 CREATE) [n:127.0.0.1:40724_solr c:testasynccollectioncreation s:shard1 r:core_node2 x:testasynccollectioncreation_shard1_replica_n1 ] o.a.s.c.CoreContainer Creating SolrCore 'testasynccollectioncreation_shard1_replica_n1' using configuration from configset conf1, trusted=true
[junit4] 2> 5897280 INFO (parallelCoreAdminExecutor-11374-thread-1-processing-n:127.0.0.1:40724_solr x:testasynccollectioncreation_shard1_replica_n1 631967a1-fbbc-4419-93ff-adf14a59e0254045044898596921 CREATE) [n:127.0.0.1:40724_solr c:testasynccollectioncreation s:shard1 r:core_node2 x:testasynccollectioncreation_shard1_replica_n1 ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_40724.solr.core.testasynccollectioncreation.shard1.replica_n1' (registry 'solr.core.testasynccollectioncreation.shard1.replica_n1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@5ef91bdc
[junit4] 2> 5897280 INFO (parallelCoreAdminExecutor-11374-thread-1-processing-n:127.0.0.1:40724_solr x:testasynccollectioncreation_shard1_replica_n1 631967a1-fbbc-4419-93ff-adf14a59e0254045044898596921 CREATE) [n:127.0.0.1:40724_solr c:testasynccollectioncreation s:shard1 r:core_node2 x:testasynccollectioncreation_shard1_replica_n1 ] o.a.s.c.SolrCore [[testasynccollectioncreation_shard1_replica_n1] ] Opening new SolrCore at [/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.api.collections.CollectionsAPIAsyncDistributedZkTest_EC7974BA469EE123-001/tempDir-001/node1/testasynccollectioncreation_shard1_replica_n1], dataDir=[/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.api.collections.CollectionsAPIAsyncDistributedZkTest_EC7974BA469EE123-001/tempDir-001/node1/./testasynccollectioncreation_shard1_replica_n1/data/]
[junit4] 2> 5897329 INFO (qtp1603922357-51735) [n:127.0.0.1:40724_solr ] o.a.s.h.a.CoreAdminOperation Checking request status for : 631967a1-fbbc-4419-93ff-adf14a59e0254045044898596921
[junit4] 2> 5897329 INFO (qtp1603922357-51735) [n:127.0.0.1:40724_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&requestid=631967a1-fbbc-4419-93ff-adf14a59e0254045044898596921&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=0
[junit4] 2> 5897535 INFO (parallelCoreAdminExecutor-11374-thread-1-processing-n:127.0.0.1:40724_solr x:testasynccollectioncreation_shard1_replica_n1 631967a1-fbbc-4419-93ff-adf14a59e0254045044898596921 CREATE) [n:127.0.0.1:40724_solr c:testasynccollectioncreation s:shard1 r:core_node2 x:testasynccollectioncreation_shard1_replica_n1 ] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 5897535 INFO (parallelCoreAdminExecutor-11374-thread-1-processing-n:127.0.0.1:40724_solr x:testasynccollectioncreation_shard1_replica_n1 631967a1-fbbc-4419-93ff-adf14a59e0254045044898596921 CREATE) [n:127.0.0.1:40724_solr c:testasynccollectioncreation s:shard1 r:core_node2 x:testasynccollectioncreation_shard1_replica_n1 ] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 5897538 INFO (parallelCoreAdminExecutor-11374-thread-1-processing-n:127.0.0.1:40724_solr x:testasynccollectioncreation_shard1_replica_n1 631967a1-fbbc-4419-93ff-adf14a59e0254045044898596921 CREATE) [n:127.0.0.1:40724_solr c:testasynccollectioncreation s:shard1 r:core_node2 x:testasynccollectioncreation_shard1_replica_n1 ] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 5897538 INFO (parallelCoreAdminExecutor-11374-thread-1-processing-n:127.0.0.1:40724_solr x:testasynccollectioncreation_shard1_replica_n1 631967a1-fbbc-4419-93ff-adf14a59e0254045044898596921 CREATE) [n:127.0.0.1:40724_solr c:testasynccollectioncreation s:shard1 r:core_node2 x:testasynccollectioncreation_shard1_replica_n1 ] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 5897592 INFO (parallelCoreAdminExecutor-11374-thread-1-processing-n:127.0.0.1:40724_solr x:testasynccollectioncreation_shard1_replica_n1 631967a1-fbbc-4419-93ff-adf14a59e0254045044898596921 CREATE) [n:127.0.0.1:40724_solr c:testasynccollectioncreation s:shard1 r:core_node2 x:testasynccollectioncreation_shard1_replica_n1 ] o.a.s.s.SolrIndexSearcher Opening [Searcher@49db9c2c[testasynccollectioncreation_shard1_replica_n1] main]
[junit4] 2> 5897604 INFO (parallelCoreAdminExecutor-11374-thread-1-processing-n:127.0.0.1:40724_solr x:testasynccollectioncreation_shard1_replica_n1 631967a1-fbbc-4419-93ff-adf14a59e0254045044898596921 CREATE) [n:127.0.0.1:40724_solr c:testasynccollectioncreation s:shard1 r:core_node2 x:testasynccollectioncreation_shard1_replica_n1 ] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 5897604 INFO (parallelCoreAdminExecutor-11374-thread-1-processing-n:127.0.0.1:40724_solr x:testasynccollectioncreation_shard1_replica_n1 631967a1-fbbc-4419-93ff-adf14a59e0254045044898596921 CREATE) [n:127.0.0.1:40724_solr c:testasynccollectioncreation s:shard1 r:core_node2 x:testasynccollectioncreation_shard1_replica_n1 ] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 5897605 INFO (parallelCoreAdminExecutor-11374-thread-1-processing-n:127.0.0.1:40724_solr x:testasynccollectioncreation_shard1_replica_n1 631967a1-fbbc-4419-93ff-adf14a59e0254045044898596921 CREATE) [n:127.0.0.1:40724_solr c:testasynccollectioncreation s:shard1 r:core_node2 x:testasynccollectioncreation_shard1_replica_n1 ] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
[junit4] 2> 5897605 INFO (parallelCoreAdminExecutor-11374-thread-1-processing-n:127.0.0.1:40724_solr x:testasynccollectioncreation_shard1_replica_n1 631967a1-fbbc-4419-93ff-adf14a59e0254045044898596921 CREATE) [n:127.0.0.1:40724_solr c:testasynccollectioncreation s:shard1 r:core_node2 x:testasynccollectioncreation_shard1_replica_n1 ] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1659129833670574080
[junit4] 2> 5897609 INFO (searcherExecutor-11378-thread-1-processing-x:testasynccollectioncreation_shard1_replica_n1 631967a1-fbbc-4419-93ff-adf14a59e0254045044898596921 r:core_node2 CREATE n:127.0.0.1:40724_solr c:testasynccollectioncreation s:shard1) [n:127.0.0.1:40724_solr c:testasynccollectioncreation s:shard1 r:core_node2 x:testasynccollectioncreation_shard1_replica_n1 ] o.a.s.c.SolrCore [testasynccollectioncreation_shard1_replica_n1] Registered new searcher Searcher@49db9c2c[testasynccollectioncreation_shard1_replica_n1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 5897612 INFO (parallelCoreAdminExecutor-11374-thread-1-processing-n:127.0.0.1:40724_solr x:testasynccollectioncreation_shard1_replica_n1 631967a1-fbbc-4419-93ff-adf14a59e0254045044898596921 CREATE) [n:127.0.0.1:40724_solr c:testasynccollectioncreation s:shard1 r:core_node2 x:testasynccollectioncreation_shard1_replica_n1 ] o.a.s.c.ZkShardTerms Successful update of terms at /collections/testasynccollectioncreation/terms/shard1 to Terms{values={core_node2=0}, version=0}
[junit4] 2> 5897612 INFO (parallelCoreAdminExecutor-11374-thread-1-processing-n:127.0.0.1:40724_solr x:testasynccollectioncreation_shard1_replica_n1 631967a1-fbbc-4419-93ff-adf14a59e0254045044898596921 CREATE) [n:127.0.0.1:40724_solr c:testasynccollectioncreation s:shard1 r:core_node2 x:testasynccollectioncreation_shard1_replica_n1 ] o.a.s.c.ShardLeaderElectionContextBase make sure parent is created /collections/testasynccollectioncreation/leaders/shard1
[junit4] 2> 5897624 INFO (parallelCoreAdminExecutor-11374-thread-1-processing-n:127.0.0.1:40724_solr x:testasynccollectioncreation_shard1_replica_n1 631967a1-fbbc-4419-93ff-adf14a59e0254045044898596921 CREATE) [n:127.0.0.1:40724_solr c:testasynccollectioncreation s:shard1 r:core_node2 x:testasynccollectioncreation_shard1_replica_n1 ] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
[junit4] 2> 5897624 INFO (parallelCoreAdminExecutor-11374-thread-1-processing-n:127.0.0.1:40724_solr x:testasynccollectioncreation_shard1_replica_n1 631967a1-fbbc-4419-93ff-adf14a59e0254045044898596921 CREATE) [n:127.0.0.1:40724_solr c:testasynccollectioncreation s:shard1 r:core_node2 x:testasynccollectioncreation_shard1_replica_n1 ] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
[junit4] 2> 5897624 INFO (parallelCoreAdminExecutor-11374-thread-1-processing-n:127.0.0.1:40724_solr x:testasynccollectioncreation_shard1_replica_n1 631967a1-fbbc-4419-93ff-adf14a59e0254045044898596921 CREATE) [n:127.0.0.1:40724_solr c:testasynccollectioncreation s:shard1 r:core_node2 x:testasynccollectioncreation_shard1_replica_n1 ] o.a.s.c.SyncStrategy Sync replicas to https://127.0.0.1:40724/solr/testasynccollectioncreation_shard1_replica_n1/
[junit4] 2> 5897624 INFO (parallelCoreAdminExecutor-11374-thread-1-processing-n:127.0.0.1:40724_solr x:testasynccollectioncreation_shard1_replica_n1 631967a1-fbbc-4419-93ff-adf14a59e0254045044898596921 CREATE) [n:127.0.0.1:40724_solr c:testasynccollectioncreation s:shard1 r:core_node2 x:testasynccollectioncreation_shard1_replica_n1 ] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
[junit4] 2> 5897625 INFO (parallelCoreAdminExecutor-11374-thread-1-processing-n:127.0.0.1:40724_solr x:testasynccollectioncreation_shard1_replica_n1 631967a1-fbbc-4419-93ff-adf14a59e0254045044898596921 CREATE) [n:127.0.0.1:40724_solr c:testasynccollectioncreation s:shard1 r:core_node2 x:testasynccollectioncreation_shard1_replica_n1 ] o.a.s.c.SyncStrategy https://127.0.0.1:40724/solr/testasynccollectioncreation_shard1_replica_n1/ has no replicas
[junit4] 2> 5897625 INFO (parallelCoreAdminExecutor-11374-thread-1-processing-n:127.0.0.1:40724_solr x:testasynccollectioncreation_shard1_replica_n1 631967a1-fbbc-4419-93ff-adf14a59e0254045044898596921 CREATE) [n:127.0.0.1:40724_solr c:testasynccollectioncreation s:shard1 r:core_node2 x:testasynccollectioncreation_shard1_replica_n1 ] o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node /collections/testasynccollectioncreation/leaders/shard1/leader after winning as /collections/testasynccollectioncreation/leader_elect/shard1/election/72322689862139912-core_node2-n_0000000000
[junit4] 2> 5897636 INFO (parallelCoreAdminExecutor-11374-thread-1-processing-n:127.0.0.1:40724_solr x:testasynccollectioncreation_shard1_replica_n1 631967a1-fbbc-4419-93ff-adf14a59e0254045044898596921 CREATE) [n:127.0.0.1:40724_solr c:testasynccollectioncreation s:shard1 r:core_node2 x:testasynccollectioncreation_shard1_replica_n1 ] o.a.s.c.ShardLeaderElectionContext I am the new leader: https://127.0.0.1:40724/solr/testasynccollectioncreation_shard1_replica_n1/ shard1
[junit4] 2> 5897739 INFO (zkCallback-14540-thread-1) [ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testasynccollectioncreation/state.json] for collection [testasynccollectioncreation] has occurred - updating... (live nodes size: [2])
[junit4] 2> 5897739 INFO (zkCallback-14540-thread-2) [ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testasynccollectioncreation/state.json] for collection [testasynccollectioncreation] has occurred - updating... (live nodes size: [2])
[junit4] 2> 5897740 INFO (parallelCoreAdminExecutor-11374-thread-1-processing-n:127.0.0.1:40724_solr x:testasynccollectioncreation_shard1_replica_n1 631967a1-fbbc-4419-93ff-adf14a59e0254045044898596921 CREATE) [n:127.0.0.1:40724_solr c:testasynccollectioncreation s:shard1 r:core_node2 x:testasynccollectioncreation_shard1_replica_n1 ] o.a.s.c.ZkController I am the leader, no recovery necessary
[junit4] 2> 5897843 INFO (zkCallback-14540-thread-2) [ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testasynccollectioncreation/state.json] for collection [testasynccollectioncreation] has occurred - updating... (live nodes size: [2])
[junit4] 2> 5897843 INFO (zkCallback-14540-thread-1) [ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testasynccollectioncreation/state.json] for collection [testasynccollectioncreation] has occurred - updating... (live nodes size: [2])
[junit4] 2> 5897843 INFO (zkCallback-14540-thread-3) [ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testasynccollectioncreation/state.json] for collection [testasynccollectioncreation] has occurred - updating... (live nodes size: [2])
[junit4] 2> 5897858 INFO (qtp1603922357-51731) [n:127.0.0.1:40724_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :requeststatus with params requestid=631967a1-fbbc-4419-93ff-adf14a59e025&action=REQUESTSTATUS&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 5897859 INFO (qtp1603922357-51731) [n:127.0.0.1:40724_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={requestid=631967a1-fbbc-4419-93ff-adf14a59e025&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=1
[junit4] 2> 5898331 INFO (qtp1603922357-51733) [n:127.0.0.1:40724_solr ] o.a.s.h.a.CoreAdminOperation Checking request status for : 631967a1-fbbc-4419-93ff-adf14a59e0254045044898596921
[junit4] 2> 5898331 INFO (qtp1603922357-51733) [n:127.0.0.1:40724_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&requestid=631967a1-fbbc-4419-93ff-adf14a59e0254045044898596921&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=0
[junit4] 2> 5898347 INFO (OverseerThreadFactory-11366-thread-1-processing-n:127.0.0.1:34409_solr) [n:127.0.0.1:34409_solr ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000000 doesn't exist. Requestor may have disconnected from ZooKeeper
[junit4] 2> 5898869 INFO (qtp1603922357-51735) [n:127.0.0.1:40724_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :requeststatus with params requestid=631967a1-fbbc-4419-93ff-adf14a59e025&action=REQUESTSTATUS&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 5898869 INFO (OverseerCollectionConfigSetProcessor-72322689862139910-127.0.0.1:34409_solr-n_0000000000) [n:127.0.0.1:34409_solr ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000000 doesn't exist. Requestor may have disconnected from ZooKeeper
[junit4] 2> 5898870 INFO (qtp1603922357-51735) [n:127.0.0.1:40724_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={requestid=631967a1-fbbc-4419-93ff-adf14a59e025&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=1
[junit4] 2> 5898871 INFO (qtp1603922357-51731) [n:127.0.0.1:40724_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :deletestatus with params requestid=631967a1-fbbc-4419-93ff-adf14a59e025&action=DELETESTATUS&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 5898872 INFO (qtp1603922357-51731) [n:127.0.0.1:40724_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={requestid=631967a1-fbbc-4419-93ff-adf14a59e025&action=DELETESTATUS&wt=javabin&version=2} status=0 QTime=1
[junit4] 2> 5898892 INFO (qtp1603922357-51733) [n:127.0.0.1:40724_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params async=a53b48f9-49ff-49a3-b5bb-f28cbe58d783&collection.configName=conf1&name=testasynccollectioncreation&nrtReplicas=1&action=CREATE&numShards=1&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 5898895 INFO (qtp1603922357-51733) [n:127.0.0.1:40724_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={async=a53b48f9-49ff-49a3-b5bb-f28cbe58d783&collection.configName=conf1&name=testasynccollectioncreation&nrtReplicas=1&action=CREATE&numShards=1&wt=javabin&version=2} status=0 QTime=3
[junit4] 2> 5898913 INFO (qtp1603922357-51735) [n:127.0.0.1:40724_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :requeststatus with params requestid=a53b48f9-49ff-49a3-b5bb-f28cbe58d783&action=REQUESTSTATUS&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 5898962 INFO (qtp1603922357-51735) [n:127.0.0.1:40724_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={requestid=a53b48f9-49ff-49a3-b5bb-f28cbe58d783&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=49
[junit4] 2> 5898963 INFO (OverseerThreadFactory-11366-thread-2-processing-n:127.0.0.1:34409_solr) [n:127.0.0.1:34409_solr ] o.a.s.c.a.c.CreateCollectionCmd Create collection testasynccollectioncreation
[junit4] 2> 5898965 ERROR (OverseerThreadFactory-11366-thread-2-processing-n:127.0.0.1:34409_solr) [n:127.0.0.1:34409_solr ] o.a.s.c.a.c.OverseerCollectionMessageHandler Collection: testasynccollectioncreation operation: create failed:org.apache.solr.common.SolrException: collection already exists: testasynccollectioncreation
[junit4] 2> at org.apache.solr.cloud.api.collections.CreateCollectionCmd.call(CreateCollectionCmd.java:116)
[junit4] 2> at org.apache.solr.cloud.api.collections.OverseerCollectionMessageHandler.processMessage(OverseerCollectionMessageHandler.java:263)
[junit4] 2> at org.apache.solr.cloud.OverseerTaskProcessor$Runner.run(OverseerTaskProcessor.java:504)
[junit4] 2> at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:210)
[junit4] 2> at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
[junit4] 2> at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
[junit4] 2> at java.base/java.lang.Thread.run(Thread.java:834)
[junit4] 2>
[junit4] 2> 5899080 INFO (OverseerThreadFactory-11366-thread-2-processing-n:127.0.0.1:34409_solr) [n:127.0.0.1:34409_solr ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000001 doesn't exist. Requestor may have disconnected from ZooKeeper
[junit4] 2> 5899964 INFO (qtp1603922357-51731) [n:127.0.0.1:40724_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :requeststatus with params requestid=a53b48f9-49ff-49a3-b5bb-f28cbe58d783&action=REQUESTSTATUS&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 5899965 INFO (qtp1603922357-51731) [n:127.0.0.1:40724_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={requestid=a53b48f9-49ff-49a3-b5bb-f28cbe58d783&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=1
[junit4] 2> 5899966 INFO (qtp1603922357-51733) [n:127.0.0.1:40724_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :deletestatus with params requestid=a53b48f9-49ff-49a3-b5bb-f28cbe58d783&action=DELETESTATUS&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 5899968 INFO (qtp1603922357-51733) [n:127.0.0.1:40724_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={requestid=a53b48f9-49ff-49a3-b5bb-f28cbe58d783&action=DELETESTATUS&wt=javabin&version=2} status=0 QTime=1
[junit4] 2> 5899969 INFO (qtp1603922357-51735) [n:127.0.0.1:40724_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :addreplica with params async=970c73ed-e9af-47a3-81e7-29fbaab20b2f&action=ADDREPLICA&collection=testasynccollectioncreation&shard=shard1&type=NRT&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 5899971 INFO (qtp1603922357-51735) [n:127.0.0.1:40724_solr c:testasynccollectioncreation ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={async=970c73ed-e9af-47a3-81e7-29fbaab20b2f&action=ADDREPLICA&collection=testasynccollectioncreation&shard=shard1&type=NRT&wt=javabin&version=2} status=0 QTime=2
[junit4] 2> 5899971 INFO (qtp1603922357-51731) [n:127.0.0.1:40724_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :requeststatus with params requestid=970c73ed-e9af-47a3-81e7-29fbaab20b2f&action=REQUESTSTATUS&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 5899973 INFO (qtp1603922357-51731) [n:127.0.0.1:40724_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={requestid=970c73ed-e9af-47a3-81e7-29fbaab20b2f&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=1
[junit4] 2> 5899974 INFO (OverseerCollectionConfigSetProcessor-72322689862139910-127.0.0.1:34409_solr-n_0000000000) [n:127.0.0.1:34409_solr ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000001 doesn't exist. Requestor may have disconnected from ZooKeeper
[junit4] 2> 5899976 INFO (OverseerThreadFactory-11366-thread-3-processing-n:127.0.0.1:34409_solr) [n:127.0.0.1:34409_solr c:testasynccollectioncreation s:shard1 ] o.a.s.c.a.c.AddReplicaCmd Node Identified 127.0.0.1:34409_solr for creating new replica of shard shard1 for collection testasynccollectioncreation
[junit4] 2> 5899977 INFO (OverseerThreadFactory-11366-thread-3-processing-n:127.0.0.1:34409_solr) [n:127.0.0.1:34409_solr c:testasynccollectioncreation s:shard1 ] o.a.s.c.a.c.AddReplicaCmd Returning CreateReplica command.
[junit4] 2> 5899979 INFO (OverseerStateUpdate-72322689862139910-127.0.0.1:34409_solr-n_0000000000) [n:127.0.0.1:34409_solr ] o.a.s.c.o.SliceMutator createReplica() {
[junit4] 2> "operation":"addreplica",
[junit4] 2> "collection":"testasynccollectioncreation",
[junit4] 2> "shard":"shard1",
[junit4] 2> "core":"testasynccollectioncreation_shard1_replica_n3",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"https://127.0.0.1:34409/solr",
[junit4] 2> "node_name":"127.0.0.1:34409_solr",
[junit4] 2> "type":"NRT"}
[junit4] 2> 5899982 INFO (zkCallback-14540-thread-3) [ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testasynccollectioncreation/state.json] for collection [testasynccollectioncreation] has occurred - updating... (live nodes size: [2])
[junit4] 2> 5899982 INFO (zkCallback-14540-thread-2) [ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testasynccollectioncreation/state.json] for collection [testasynccollectioncreation] has occurred - updating... (live nodes size: [2])
[junit4] 2> 5899982 INFO (zkCallback-14540-thread-1) [ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testasynccollectioncreation/state.json] for collection [testasynccollectioncreation] has occurred - updating... (live nodes size: [2])
[junit4] 2> 5900132 INFO (qtp1700883115-51736) [n:127.0.0.1:34409_solr x:testasynccollectioncreation_shard1_replica_n3 ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={async=970c73ed-e9af-47a3-81e7-29fbaab20b2f4045048015462210&qt=/admin/cores&coreNodeName=core_node4&collection.configName=conf1&name=testasynccollectioncreation_shard1_replica_n3&action=CREATE&collection=testasynccollectioncreation&shard=shard1&wt=javabin&version=2&replicaType=NRT} status=0 QTime=0
[junit4] 2> 5900132 INFO (parallelCoreAdminExecutor-11367-thread-1-processing-n:127.0.0.1:34409_solr x:testasynccollectioncreation_shard1_replica_n3 970c73ed-e9af-47a3-81e7-29fbaab20b2f4045048015462210 CREATE) [n:127.0.0.1:34409_solr x:testasynccollectioncreation_shard1_replica_n3 ] o.a.s.h.a.CoreAdminOperation core create command async=970c73ed-e9af-47a3-81e7-29fbaab20b2f4045048015462210&qt=/admin/cores&coreNodeName=core_node4&collection.configName=conf1&name=testasynccollectioncreation_shard1_replica_n3&action=CREATE&collection=testasynccollectioncreation&shard=shard1&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 5900132 INFO (parallelCoreAdminExecutor-11367-thread-1-processing-n:127.0.0.1:34409_solr x:testasynccollectioncreation_shard1_replica_n3 970c73ed-e9af-47a3-81e7-29fbaab20b2f4045048015462210 CREATE) [n:127.0.0.1:34409_solr x:testasynccollectioncreation_shard1_replica_n3 ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 5900133 INFO (qtp1700883115-51732) [n:127.0.0.1:34409_solr ] o.a.s.h.a.CoreAdminOperation Checking request status for : 970c73ed-e9af-47a3-81e7-29fbaab20b2f4045048015462210
[junit4] 2> 5900133 INFO (qtp1700883115-51732) [n:127.0.0.1:34409_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&requestid=970c73ed-e9af-47a3-81e7-29fbaab20b2f4045048015462210&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=0
[junit4] 2> 5900155 INFO (parallelCoreAdminExecutor-11367-thread-1-processing-n:127.0.0.1:34409_solr x:testasynccollectioncreation_shard1_replica_n3 970c73ed-e9af-47a3-81e7-29fbaab20b2f4045048015462210 CREATE) [n:127.0.0.1:34409_solr c:testasynccollectioncreation s:shard1 r:core_node4 x:testasynccollectioncreation_shard1_replica_n3 ] o.a.s.c.SolrConfig Using Lucene MatchVersion: 9.0.0
[junit4] 2> 5900175 INFO (parallelCoreAdminExecutor-11367-thread-1-processing-n:127.0.0.1:34409_solr x:testasynccollectioncreation_shard1_replica_n3 970c73ed-e9af-47a3-81e7-29fbaab20b2f4045048015462210 CREATE) [n:127.0.0.1:34409_solr c:testasynccollectioncreation s:shard1 r:core_node4 x:testasynccollectioncreation_shard1_replica_n3 ] o.a.s.s.IndexSchema [testasynccollectioncreation_shard1_replica_n3] Schema name=minimal
[junit4] 2> 5900177 INFO (parallelCoreAdminExecutor-11367-thread-1-processing-n:127.0.0.1:34409_solr x:testasynccollectioncreation_shard1_replica_n3 970c73ed-e9af-47a3-81e7-29fbaab20b2f4045048015462210 CREATE) [n:127.0.0.1:34409_solr c:testasynccollectioncreation s:shard1 r:core_node4 x:testasynccollectioncreation_shard1_replica_n3 ] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
[junit4] 2> 5900177 INFO (parallelCoreAdminExecutor-11367-thread-1-processing-n:127.0.0.1:34409_solr x:testasynccollectioncreation_shard1_replica_n3 970c73ed-e9af-47a3-81e7-29fbaab20b2f4045048015462210 CREATE) [n:127.0.0.1:34409_solr c:testasynccollectioncreation s:shard1 r:core_node4 x:testasynccollectioncreation_shard1_replica_n3 ] o.a.s.c.CoreContainer Creating SolrCore 'testasynccollectioncreation_shard1_replica_n3' using configuration from configset conf1, trusted=true
[junit4] 2> 5900178 INFO (parallelCoreAdminExecutor-11367-thread-1-processing-n:127.0.0.1:34409_solr x:testasynccollectioncreation_shard1_replica_n3 970c73ed-e9af-47a3-81e7-29fbaab20b2f4045048015462210 CREATE) [n:127.0.0.1:34409_solr c:testasynccollectioncreation s:shard1 r:core_node4 x:testasynccollectioncreation_shard1_replica_n3 ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_34409.solr.core.testasynccollectioncreation.shard1.replica_n3' (registry 'solr.core.testasynccollectioncreation.shard1.replica_n3') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@5ef91bdc
[junit4] 2> 5900178 INFO (parallelCoreAdminExecutor-11367-thread-1-processing-n:127.0.0.1:34409_solr x:testasynccollectioncreation_shard1_replica_n3 970c73ed-e9af-47a3-81e7-29fbaab20b2f4045048015462210 CREATE) [n:127.0.0.1:34409_solr c:testasynccollectioncreation s:shard1 r:core_node4 x:testasynccollectioncreation_shard1_replica_n3 ] o.a.s.c.SolrCore [[testasynccollectioncreation_shard1_replica_n3] ] Opening new SolrCore at [/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.api.collections.CollectionsAPIAsyncDistributedZkTest_EC7974BA469EE123-001/tempDir-001/node2/testasynccollectioncreation_shard1_replica_n3], dataDir=[/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.api.collections.CollectionsAPIAsyncDistributedZkTest_EC7974BA469EE123-001/tempDir-001/node2/./testasynccollectioncreation_shard1_replica_n3/data/]
[junit4] 2> 5900248 INFO (zkCallback-14540-thread-2) [ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testasynccollectioncreation/state.json] for collection [testasynccollectioncreation] has occurred - updating... (live nodes size: [2])
[junit4] 2> 5900248 INFO (zkCallback-14540-thread-1) [ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testasynccollectioncreation/state.json] for collection [testasynccollectioncreation] has occurred - updating... (live nodes size: [2])
[junit4] 2> 5900248 INFO (zkCallback-14540-thread-3) [ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testasynccollectioncreation/state.json] for collection [testasynccollectioncreation] has occurred - updating... (live nodes size: [2])
[junit4] 2> 5900334 INFO (parallelCoreAdminExecutor-11367-thread-1-processing-n:127.0.0.1:34409_solr x:testasynccollectioncreation_shard1_replica_n3 970c73ed-e9af-47a3-81e7-29fbaab20b2f4045048015462210 CREATE) [n:127.0.0.1:34409_solr c:testasynccollectioncreation s:shard1 r:core_node4 x:testasynccollectioncreation_shard1_replica_n3 ] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 5900334 INFO (parallelCoreAdminExecutor-11367-thread-1-processing-n:127.0.0.1:34409_solr x:testasynccollectioncreation_shard1_replica_n3 970c73ed-e9af-47a3-81e7-29fbaab20b2f4045048015462210 CREATE) [n:127.0.0.1:34409_solr c:testasynccollectioncreation s:shard1 r:core_node4 x:testasynccollectioncreation_shard1_replica_n3 ] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 5900338 INFO (parallelCoreAdminExecutor-11367-thread-1-processing-n:127.0.0.1:34409_solr x:testasynccollectioncreation_shard1_replica_n3 970c73ed-e9af-47a3-81e7-29fbaab20b2f4045048015462210 CREATE) [n:127.0.0.1:34409_solr c:testasynccollectioncreation s:shard1 r:core_node4 x:testasynccollectioncreation_shard1_replica_n3 ] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 5900338 INFO (parallelCoreAdminExecutor-11367-thread-1-processing-n:127.0.0.1:34409_solr x:testasynccollectioncreation_shard1_replica_n3 970c73ed-e9af-47a3-81e7-29fbaab20b2f4045048015462210 CREATE) [n:127.0.0.1:34409_solr c:testasynccollectioncreation s:shard1 r:core_node4 x:testasynccollectioncreation_shard1_replica_n3 ] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 5900392 INFO (parallelCoreAdminExecutor-11367-thread-1-processing-n:127.0.0.1:34409_solr x:testasynccollectioncreation_shard1_replica_n3 970c73ed-e9af-47a3-81e7-29fbaab20b2f4045048015462210 CREATE) [n:127.0.0.1:34409_solr c:testasynccollectioncreation s:shard1 r:core_node4 x:testasynccollectioncreation_shard1_replica_n3 ] o.a.s.s.SolrIndexSearcher Opening [Searcher@3b94bda5[testasynccollectioncreation_shard1_replica_n3] main]
[junit4] 2> 5900402 INFO (parallelCoreAdminExecutor-11367-thread-1-processing-n:127.0.0.1:34409_solr x:testasynccollectioncreation_shard1_replica_n3 970c73ed-e9af-47a3-81e7-29fbaab20b2f4045048015462210 CREATE) [n:127.0.0.1:34409_solr c:testasynccollectioncreation s:shard1 r:core_node4 x:testasynccollectioncreation_shard1_replica_n3 ] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 5900402 INFO (parallelCoreAdminExecutor-11367-thread-1-processing-n:127.0.0.1:34409_solr x:testasynccollectioncreation_shard1_replica_n3 970c73ed-e9af-47a3-81e7-29fbaab20b2f4045048015462210 CREATE) [n:127.0.0.1:34409_solr c:testasynccollectioncreation s:shard1 r:core_node4 x:testasynccollectioncreation_shard1_replica_n3 ] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 5900403 INFO (parallelCoreAdminExecutor-11367-thread-1-processing-n:127.0.0.1:34409_solr x:testasynccollectioncreation_shard1_replica_n3 970c73ed-e9af-47a3-81e7-29fbaab20b2f4045048015462210 CREATE) [n:127.0.0.1:34409_solr c:testasynccollectioncreation s:shard1 r:core_node4 x:testasynccollectioncreation_shard1_replica_n3 ] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
[junit4] 2> 5900403 INFO (parallelCoreAdminExecutor-11367-thread-1-processing-n:127.0.0.1:34409_solr x:testasynccollectioncreation_shard1_replica_n3 970c73ed-e9af-47a3-81e7-29fbaab20b2f4045048015462210 CREATE) [n:127.0.0.1:34409_solr c:testasynccollectioncreation s:shard1 r:core_node4 x:testasynccollectioncreation_shard1_replica_n3 ] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1659129836604489728
[junit4] 2> 5900407 INFO (searcherExecutor-11383-thread-1-processing-x:testasynccollectioncreation_shard1_replica_n3 970c73ed-e9af-47a3-81e7-29fbaab20b2f4045048015462210 r:core_node4 CREATE n:127.0.0.1:34409_solr c:testasynccollectioncreation s:shard1) [n:127.0.0.1:34409_solr c:testasynccollectioncreation s:shard1 r:core_node4 x:testasynccollectioncreation_shard1_replica_n3 ] o.a.s.c.SolrCore [testasynccollectioncreation_shard1_replica_n3] Registered new searcher Searcher@3b94bda5[testasynccollectioncreation_shard1_replica_n3] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 5900422 INFO (parallelCoreAdminExecutor-11367-thread-1-processing-n:127.0.0.1:34409_solr x:testasynccollectioncreation_shard1_replica_n3 970c73ed-e9af-47a3-81e7-29fbaab20b2f4045048015462210 CREATE) [n:127.0.0.1:34409_solr c:testasynccollectioncreation s:shard1 r:core_node4 x:testasynccollectioncreation_shard1_replica_n3 ] o.a.s.c.ZkShardTerms Successful update of terms at /collections/testasynccollectioncreation/terms/shard1 to Terms{values={core_node2=0, core_node4=0}, version=1}
[junit4] 2> 5900422 INFO (parallelCoreAdminExecutor-11367-thread-1-processing-n:127.0.0.1:34409_solr x:testasynccollectioncreation_shard1_replica_n3 970c73ed-e9af-47a3-81e7-29fbaab20b2f4045048015462210 CREATE) [n:127.0.0.1:34409_solr c:testasynccollectioncreation s:shard1 r:core_node4 x:testasynccollectioncreation_shard1_replica_n3 ] o.a.s.c.ShardLeaderElectionContextBase make sure parent is created /collections/testasynccollectioncreation/leaders/shard1
[junit4] 2> 5900425 INFO (parallelCoreAdminExecutor-11367-thread-1-processing-n:127.0.0.1:34409_solr x:testasynccollectioncreation_shard1_replica_n3 970c73ed-e9af-47a3-81e7-29fbaab20b2f4045048015462210 CREATE) [n:127.0.0.1:34409_solr c:testasynccollectioncreation s:shard1 r:core_node4 x:testasynccollectioncreation_shard1_replica_n3 ] o.a.s.c.ZkController Core needs to recover:testasynccollectioncreation_shard1_replica_n3
[junit4] 2> 5900426 INFO (updateExecutor-14523-thread-1-processing-x:testasynccollectioncreation_shard1_replica_n3 970c73ed-e9af-47a3-81e7-29fbaab20b2f4045048015462210 r:core_node4 CREATE n:127.0.0.1:34409_solr c:testasynccollectioncreation s:shard1) [n:127.0.0.1:34409_solr c:testasynccollectioncreation s:shard1 r:core_node4 x:testasynccollectioncreation_shard1_replica_n3 ] o.a.s.u.DefaultSolrCoreState Running recovery
[junit4] 2> 5900426 INFO (recoveryExecutor-14525-thread-1-processing-x:testasynccollectioncreation_shard1_replica_n3 970c73ed-e9af-47a3-81e7-29fbaab20b2f4045048015462210 r:core_node4 CREATE n:127.0.0.1:34409_solr c:testasynccollectioncreation s:shard1) [n:127.0.0.1:34409_solr c:testasynccollectioncreation s:shard1 r:core_node4 x:testasynccollectioncreation_shard1_replica_n3 ] o.a.s.c.RecoveryStrategy Starting recovery process. recoveringAfterStartup=true
[junit4] 2> 5900426 INFO (recoveryExecutor-14525-thread-1-processing-x:testasynccollectioncreation_shard1_replica_n3 970c73ed-e9af-47a3-81e7-29fbaab20b2f4045048015462210 r:core_node4 CREATE n:127.0.0.1:34409_solr c:testasynccollectioncreation s:shard1) [n:127.0.0.1:34409_solr c:testasynccollectioncreation s:shard1 r:core_node4 x:testasynccollectioncreation_shard1_replica_n3 ] o.a.s.c.RecoveryStrategy startupVersions is empty
[junit4] 2> 5900463 INFO (qtp1603922357-51731) [n:127.0.0.1:40724_solr c:testasynccollectioncreation s:shard1 r:core_node2 x:testasynccollectioncreation_shard1_replica_n1 ] o.a.s.c.S.Request [testasynccollectioncreation_shard1_replica_n1] webapp=/solr path=/admin/ping params={wt=javabin&version=2} hits=0 status=0 QTime=0
[junit4] 2> 5900463 INFO (qtp1603922357-51731) [n:127.0.0.1:40724_solr c:testasynccollectioncreation s:shard1 r:core_node2 x:testasynccollectioncreation_shard1_replica_n1 ] o.a.s.c.S.Request [testasynccollectioncreation_shard1_replica_n1] webapp=/solr path=/admin/ping params={wt=javabin&version=2} status=0 QTime=0
[junit4] 2> 5900478 INFO (recoveryExecutor-14525-thread-1-processing-x:testasynccollectioncreation_shard1_replica_n3 970c73ed-e9af-47a3-81e7-29fbaab20b2f4045048015462210 r:core_node4 CREATE n:127.0.0.1:34409_solr c:testasynccollectioncreation s:shard1) [n:127.0.0.1:34409_solr c:testasynccollectioncreation s:shard1 r:core_node4 x:testasynccollectioncreation_shard1_replica_n3 ] o.a.s.c.RecoveryStrategy Begin buffering updates. core=[testasynccollectioncreation_shard1_replica_n3]
[junit4] 2> 5900479 INFO (recoveryExecutor-14525-thread-1-processing-x:testasynccollectioncreation_shard1_replica_n3 970c73ed-e9af-47a3-81e7-29fbaab20b2f4045048015462210 r:core_node4 CREATE n:127.0.0.1:34409_solr c:testasynccollectioncreation s:shard1) [n:127.0.0.1:34409_solr c:testasynccollectioncreation s:shard1 r:core_node4 x:testasynccollectioncreation_shard1_replica_n3 ] o.a.s.u.UpdateLog Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null}
[junit4] 2> 5900479 INFO (recoveryExecutor-14525-thread-1-processing-x:testasynccollectioncreation_shard1_replica_n3 970c73ed-e9af-47a3-81e7-29fbaab20b2f4045048015462210 r:core_node4 CREATE n:127.0.0.1:34409_solr c:testasynccollectioncreation s:shard1) [n:127.0.0.1:34409_solr c:testasynccollectioncreation s:shard1 r:core_node4 x:testasynccollectioncreation_shard1_replica_n3 ] o.a.s.c.RecoveryStrategy Publishing state of core [testasynccollectioncreation_shard1_replica_n3] as recovering, leader is [https://127.0.0.1:40724/solr/testasynccollectioncreation_shard1_replica_n1/] and I am [https://127.0.0.1:34409/solr/testasynccollectioncreation_shard1_replica_n3/]
[junit4] 2> 5900480 INFO (recoveryExecutor-14525-thread-1-processing-x:testasynccollectioncreation_shard1_replica_n3 970c73ed-e9af-47a3-81e7-29fbaab20b2f4045048015462210 r:core_node4 CREATE n:127.0.0.1:34409_solr c:testasynccollectioncreation s:shard1) [n:127.0.0.1:34409_solr c:testasynccollectioncreation s:shard1 r:core_node4 x:testasynccollectioncreation_shard1_replica_n3 ] o.a.s.c.RecoveryStrategy Sending prep recovery command to [https://127.0.0.1:40724/solr]; [WaitForState: action=PREPRECOVERY&core=testasynccollectioncreation_shard1_replica_n1&nodeName=127.0.0.1:34409_solr&coreNodeName=core_node4&state=recovering&checkLive=true&onlyIfLeader=true&onlyIfLeaderActive=true]
[junit4] 2> 5900481 INFO (qtp1603922357-51735) [n:127.0.0.1:40724_solr x:testasynccollectioncreation_shard1_replica_n1 ] o.a.s.h.a.PrepRecoveryOp Going to wait for coreNodeName: core_node4, state: recovering, checkLive: true, onlyIfLeader: true, onlyIfLeaderActive: true
[junit4] 2> 5900481 INFO (qtp1603922357-51735) [n:127.0.0.1:40724_solr x:testasynccollectioncreation_shard1_replica_n1 ] o.a.s.h.a.PrepRecoveryOp In WaitForState(recovering): collection=testasynccollectioncreation, shard=shard1, thisCore=testasynccollectioncreation_shard1_replica_n1, leaderDoesNotNeedRecovery=false, isLeader? true, live=true, checkLive=true, currentState=down, localState=active, nodeName=127.0.0.1:34409_solr, coreNodeName=core_node4, onlyIfActiveCheckResult=false, nodeProps: core_node4:{
[junit4] 2> "core":"testasynccollectioncreation_shard1_replica_n3",
[junit4] 2> "base_url":"https://127.0.0.1:34409/solr",
[junit4] 2> "node_name":"127.0.0.1:34409_solr",
[junit4] 2> "state":"down",
[junit4] 2> "type":"NRT",
[junit4] 2> "force_set_state":"false"}
[junit4] 2> 5900481 INFO (qtp1603922357-51735) [n:127.0.0.1:40724_solr x:testasynccollectioncreation_shard1_replica_n1 ] o.a.s.h.a.PrepRecoveryOp In WaitForState(recovering): collection=testasynccollectioncreation, shard=shard1, thisCore=testasynccollectioncreation_shard1_replica_n1, leaderDoesNotNeedRecovery=false, isLeader? true, live=true, checkLive=true, currentState=down, localState=active, nodeName=127.0.0.1:34409_solr, coreNodeName=core_node4, onlyIfActiveCheckResult=false, nodeProps: core_node4:{
[junit4] 2> "core":"testasynccollectioncreation_shard1_replica_n3",
[junit4] 2> "base_url":"https://127.0.0.1:34409/solr",
[junit4] 2> "node_name":"127.0.0.1:34409_solr",
[junit4] 2> "state":"down",
[junit4] 2> "type":"NRT",
[junit4] 2> "force_set_state":"false"}
[junit4] 2> 5900481 INFO (qtp1603922357-51735) [n:127.0.0.1:40724_solr x:testasynccollectioncreation_shard1_replica_n1 ] o.a.s.h.a.PrepRecoveryOp In WaitForState(recovering): collection=testasynccollectioncreation, shard=shard1, thisCore=testasynccollectioncreation_shard1_replica_n1, leaderDoesNotNeedRecovery=false, isLeader? true, live=true, checkLive=true, currentState=down, localState=active, nodeName=127.0.0.1:34409_solr, coreNodeName=core_node4, onlyIfActiveCheckResult=false, nodeProps: core_node4:{
[junit4] 2> "core":"testasynccollectioncreation_shard1_replica_n3",
[junit4] 2> "base_url":"https://127.0.0.1:34409/solr",
[junit4] 2> "node_name":"127.0.0.1:34409_solr",
[junit4] 2> "state":"down",
[junit4] 2> "type":"NRT",
[junit4] 2> "force_set_state":"false"}
[junit4] 2> 5900582 INFO (zkCallback-14528-thread-1) [ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testasynccollectioncreation/state.json] for collection [testasynccollectioncreation] has occurred - updating... (live nodes size: [2])
[junit4] 2> 5900582 INFO (zkCallback-14540-thread-1) [ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testasynccollectioncreation/state.json] for collection [testasynccollectioncreation] has occurred - updating... (live nodes size: [2])
[junit4] 2> 5900582 INFO (zkCallback-14540-thread-2) [ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testasynccollectioncreation/state.json] for collection [testasynccollectioncreation] has occurred - updating... (live nodes size: [2])
[junit4] 2> 5900582 INFO (zkCallback-14540-thread-3) [ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testasynccollectioncreation/state.json] for collection [testasynccollectioncreation] has occurred - updating... (live nodes size: [2])
[junit4] 2> 5900582 INFO (zkCallback-14528-thread-2) [ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testasynccollectioncreation/state.json] for collection [testasynccollectioncreation] has occurred - updating... (live nodes size: [2])
[junit4] 2> 5900582 INFO (zkCallback-14528-thread-3) [ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testasynccollectioncreation/state.json] for collection [testasynccollectioncreation] has occurred - updating... (live nodes size: [2])
[junit4] 2> 5900583 INFO (watches-14542-thread-1) [ ] o.a.s.h.a.PrepRecoveryOp In WaitForState(recovering): collection=testasynccollectioncreation, shard=shard1, thisCore=testasynccollectioncreation_shard1_replica_n1, leaderDoesNotNeedRecovery=false, isLeader? true, live=true, checkLive=true, currentState=recovering, localState=active, nodeName=127.0.0.1:34409_solr, coreNodeName=core_node4, onlyIfActiveCheckResult=false, nodeProps: core_node4:{
[junit4] 2> "core":"testasynccollectioncreation_shard1_replica_n3",
[junit4] 2> "base_url":"https://127.0.0.1:34409/solr",
[junit4] 2> "node_name":"127.0.0.1:34409_solr",
[junit4] 2> "state":"recovering",
[junit4] 2> "type":"NRT",
[junit4] 2> "force_set_state":"false"}
[junit4] 2> 5900583 INFO (qtp1603922357-51735) [n:127.0.0.1:40724_solr x:testasynccollectioncreation_shard1_replica_n1 ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={nodeName=127.0.0.1:34409_solr&onlyIfLeaderActive=true&core=testasynccollectioncreation_shard1_replica_n1&coreNodeName=core_node4&action=PREPRECOVERY&checkLive=true&state=recovering&onlyIfLeader=true&wt=javabin&version=2} status=0 QTime=102
[junit4] 2> 5900975 INFO (qtp1603922357-51733) [n:127.0.0.1:40724_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :requeststatus with params requestid=970c73ed-e9af-47a3-81e7-29fbaab20b2f&action=REQUESTSTATUS&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 5900976 INFO (qtp1603922357-51733) [n:127.0.0.1:40724_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={requestid=970c73ed-e9af-47a3-81e7-29fbaab20b2f&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=1
[junit4] 2> 5901084 INFO (recoveryExecutor-14525-thread-1-processing-x:testasynccollectioncreation_shard1_replica_n3 970c73ed-e9af-47a3-81e7-29fbaab20b2f4045048015462210 r:core_node4 CREATE n:127.0.0.1:34409_solr c:testasynccollectioncreation s:shard1) [n:127.0.0.1:34409_solr c:testasynccollectioncreation s:shard1 r:core_node4 x:testasynccollectioncreation_shard1_replica_n3 ] o.a.s.c.RecoveryStrategy Attempting to PeerSync from [https://127.0.0.1:40724/solr/testasynccollectioncreation_shard1_replica_n1/] - recoveringAfterStartup=[true]
[junit4] 2> 5901084 WARN (recoveryExecutor-14525-thread-1-processing-x:testasynccollectioncreation_shard1_replica_n3 970c73ed-e9af-47a3-81e7-29fbaab20b2f4045048015462210 r:core_node4 CREATE n:127.0.0.1:34409_solr c:testasynccollectioncreation s:shard1) [n:127.0.0.1:34409_solr c:testasynccollectioncreation s:shard1 r:core_node4 x:testasynccollectioncreation_shard1_replica_n3 ] o.a.s.u.PeerSyncWithLeader no frame of reference to tell if we've missed updates
[junit4] 2> 5901084 INFO (recoveryExecutor-14525-thread-1-processing-x:testasynccollectioncreation_shard1_replica_n3 970c73ed-e9af-47a3-81e7-29fbaab20b2f4045048015462210 r:core_node4 CREATE n:127.0.0.1:34409_solr c:testasynccollectioncreation s:shard1) [n:127.0.0.1:34409_solr c:testasynccollectioncreation s:shard1 r:core_node4 x:testasynccollectioncreation_shard1_replica_n3 ] o.a.s.c.RecoveryStrategy PeerSync Recovery was not successful - trying replication.
[junit4] 2> 5901084 INFO (recoveryExecutor-14525-thread-1-processing-x:testasynccollectioncreation_shard1_replica_n3 970c73ed-e9af-47a3-81e7-29fbaab20b2f4045048015462210 r:core_node4 CREATE n:127.0.0.1:34409_solr c:testasynccollectioncreation s:shard1) [n:127.0.0.1:34409_solr c:testasynccollectioncreation s:shard1 r:core_node4 x:testasynccollectioncreation_shard1_replica_n3 ] o.a.s.c.RecoveryStrategy Starting Replication Recovery.
[junit4] 2> 5901084 INFO (recoveryExecutor-14525-thread-1-processing-x:testasynccollectioncreation_shard1_replica_n3 970c73ed-e9af-47a3-81e7-29fbaab20b2f4045048015462210 r:core_node4 CREATE n:127.0.0.1:34409_solr c:testasynccollectioncreation s:shard1) [n:127.0.0.1:34409_solr c:testasynccollectioncreation s:shard1 r:core_node4 x:testasynccollectioncreation_shard1_replica_n3 ] o.a.s.c.RecoveryStrategy Attempting to replicate from [https://127.0.0.1:40724/solr/testasynccollectioncreation_shard1_replica_n1/].
[junit4] 2> 5901087 INFO (qtp1603922357-51731) [n:127.0.0.1:40724_solr c:testasynccollectioncreation s:shard1 r:core_node2 x:testasynccollectioncreation_shard1_replica_n1 ] o.a.s.u.DirectUpdateHandler2 start commit{_version_=1659129837321715712,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4] 2> 5901089 INFO (qtp1603922357-51731) [n:127.0.0.1:40724_solr c:testasynccollectioncreation s:shard1 r:core_node2 x:testasynccollectioncreation_shard1_replica_n1 ] o.a.s.u.DirectUpdateHandler2 No uncommitted changes. Skipping IW.commit.
[junit4] 2> 5901090 INFO (qtp1603922357-51731) [n:127.0.0.1:40724_solr c:testasynccollectioncreation s:shard1 r:core_node2 x:testasynccollectioncreation_shard1_replica_n1 ] o.a.s.u.DirectUpdateHandler2 end_commit_flush
[junit4] 2> 5901135 INFO (qtp1700883115-51732) [n:127.0.0.1:34409_solr ] o.a.s.h.a.CoreAdminOperation Checking request status for : 970c73ed-e9af-47a3-81e7-29fbaab20b2f4045048015462210
[junit4] 2> 5901135 INFO (qtp1700883115-51732) [n:127.0.0.1:34409_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&requestid=970c73ed-e9af-47a3-81e7-29fbaab20b2f4045048015462210&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=0
[junit4] 2> 5901137 INFO (qtp1700883115-51734) [n:127.0.0.1:34409_solr c:testasynccollectioncreation s:shard1 r:core_node4 x:testasynccollectioncreation_shard1_replica_n3 ] o.a.s.u.p.DistributedUpdateProcessor Ignoring commit while not ACTIVE - state: BUFFERING replay: false
[junit4] 2> 5901137 INFO (qtp1700883115-51734) [n:127.0.0.1:34409_solr c:testasynccollectioncreation s:shard1 r:core_node4 x:testasynccollectioncreation_shard1_replica_n3 ] o.a.s.u.p.LogUpdateProcessorFactory [testasynccollectioncreation_shard1_replica_n3] webapp=/solr path=/update params={update.distrib=FROMLEADER&waitSearcher=true&openSearcher=false&commit=true&softCommit=false&distrib.from=https://127.0.0.1:40724/solr/testasynccollectioncreation_shard1_replica_n1/&commit_end_point=replicas&wt=javabin&version=2&expungeDeletes=false}{commit=} 0 1
[junit4] 2> 5901137 INFO (qtp1603922357-51731) [n:127.0.0.1:40724_solr c:testasynccollectioncreation s:shard1 r:core_node2 x:testasynccollectioncreation_shard1_replica_n1 ] o.a.s.u.p.LogUpdateProcessorFactory [testasynccollectioncreation_shard1_replica_n1] webapp=/solr path=/update params={waitSearcher=true&openSearcher=false&commit=true&softCommit=false&wt=javabin&version=2}{commit=} 0 52
[junit4] 2> 5901139 INFO (OverseerThreadFactory-11366-thread-3-processing-n:127.0.0.1:34409_solr) [n:127.0.0.1:34409_solr c:testasynccollectioncreation s:shard1 ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000002 doesn't exist. Requestor may have disconnected from ZooKeeper
[junit4] 2> 5901139 INFO (qtp1603922357-51735) [n:127.0.0.1:40724_solr c:testasynccollectioncreation s:shard1 r:core_node2 x:testasynccollectioncreation_shard1_replica_n1 ] o.a.s.c.S.Request [testasynccollectioncreation_shard1_replica_n1] webapp=/solr path=/replication params={qt=/replication&wt=javabin&version=2&command=indexversion} status=0 QTime=0
[junit4] 2> 5901139 INFO (recoveryExecutor-14525-thread-1-processing-x:testasynccollectioncreation_shard1_replica_n3 970c73ed-e9af-47a3-81e7-29fbaab20b2f4045048015462210 r:core_node4 CREATE n:127.0.0.1:34409_solr c:testasynccollectioncreation s:shard1) [n:127.0.0.1:34409_solr c:testasynccollectioncreation s:shard1 r:core_node4 x:testasynccollectioncreation_shard1_replica_n3 ] o.a.s.h.IndexFetcher Master's generation: 1
[junit4] 2> 5901139 INFO (recoveryExecutor-14525-thread-1-processing-x:testasynccollectioncreation_shard1_replica_n3 970c73ed-e9af-47a3-81e7-29fbaab20b2f4045048015462210 r:core_node4 CREATE n:127.0.0.1:34409_solr c:testasynccollectioncreation s:shard1) [n:127.0.0.1:34409_solr c:testasynccollectioncreation s:shard1 r:core_node4 x:testasynccollectioncreation_shard1_replica_n3 ] o.a.s.h.IndexFetcher Master's version: 0
[junit4] 2> 5901139 INFO (recoveryExecutor-14525-thread-1-processing-x:testasynccollectioncreation_shard1_replica_n3 970c73ed-e9af-47a3-81e7-29fbaab20b2f4045048015462210 r:core_node4 CREATE n:127.0.0.1:34409_solr c:testasynccollectioncreation s:shard1) [n:127.0.0.1:34409_solr c:testasynccollectioncreation s:shard1 r:core_node4 x:testasynccollectioncreation_shard1_replica_n3 ] o.a.s.h.IndexFetcher Slave's generation: 1
[junit4] 2> 5901139 INFO (recoveryExecutor-14525-thread-1-processing-x:testasynccollectioncreation_shard1_replica_n3 970c73ed-e9af-47a3-81e7-29fbaab20b2f4045048015462210 r:core_node4 CREATE n:127.0.0.1:34409_solr c:testasynccollectioncreation s:shard1) [n:127.0.0.1:34409_solr c:testasynccollectioncreation s:shard1 r:core_node4 x:testasynccollectioncreation_shard1_replica_n3 ] o.a.s.h.IndexFetcher Slave's version: 0
[junit4] 2> 5901139 INFO (recoveryExecutor-14525-thread-1-processing-x:testasynccollectioncreation_shard1_replica_n3 970c73ed-e9af-47a3-81e7-29fbaab20b2f4045048015462210 r:core_node4 CREATE n:127.0.0.1:34409_solr c:testasynccollectioncreation s:shard1) [n:127.0.0.1:34409_solr c:testasynccollectioncreation s:shard1 r:core_node4 x:testasynccollectioncreation_shard1_replica_n3 ] o.a.s.h.IndexFetcher New index in Master. Deleting mine...
[junit4] 2> 5901140 INFO (recoveryExecutor-14525-thread-1-processing-x:testasynccollectioncreation_shard1_replica_n3 970c73ed-e9af-47a3-81e7-29fbaab20b2f4045048015462210 r:core_node4 CREATE n:127.0.0.1:34409_solr c:testasynccollectioncreation s:shard1) [n:127.0.0.1:34409_solr c:testasynccollectioncreation s:shard1 r:core_node4 x:testasynccollectioncreation_shard1_replica_n3 ] o.a.s.s.SolrIndexSearcher Opening [Searcher@529b6faf[testasynccollectioncreation_shard1_replica_n3] main]
[junit4] 2> 5901143 INFO (searcherExecutor-11383-thread-1-processing-x:testasynccollectioncreation_shard1_replica_n3 970c73ed-e9af-47a3-81e7-29fbaab20b2f4045048015462210 r:core_node4 CREATE n:127.0.0.1:34409_solr c:testasynccollectioncreation s:shard1) [n:127.0.0.1:34409_solr c:testasynccollectioncreation s:shard1 r:core_node4 x:testasynccollectioncreation_shard1_replica_n3 ] o.a.s.c.SolrCore [testasynccollectioncreation_shard1_replica_n3] Registered new searcher Searcher@529b6faf[testasynccollectioncreation_shard1_replica_n3] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 5901143 INFO (recoveryExecutor-14525-thread-1-processing-x:testasynccollectioncreation_shard1_replica_n3 970c73ed-e9af-47a3-81e7-29fbaab20b2f4045048015462210 r:core_node4 CREATE n:127.0.0.1:34409_solr c:testasynccollectioncreation s:shard1) [n:127.0.0.1:34409_solr c:testasynccollectioncreation s:shard1 r:core_node4 x:testasynccollectioncreation_shard1_replica_n3 ] o.a.s.c.RecoveryStrategy No replay needed.
[junit4] 2> 5901143 INFO (recoveryExecutor-14525-thread-1-processing-x:testasynccollectioncreation_shard1_replica_n3 970c73ed-e9af-47a3-81e7-29fbaab20b2f4045048015462210 r:core_node4 CREATE n:127.0.0.1:34409_solr c:testasynccollectioncreation s:shard1) [n:127.0.0.1:34409_solr c:testasynccollectioncreation s:shard1 r:core_node4 x:testasynccollectioncreation_shard1_replica_n3 ] o.a.s.c.RecoveryStrategy Replication Recovery was successful.
[junit4] 2> 5901143 INFO (recoveryExecutor-14525-thread-1-processing-x:testasynccollectioncreation_shard1_replica_n3 970c73ed-e9af-47a3-81e7-29fbaab20b2f4045048015462210 r:core_node4 CREATE n:127.0.0.1:34409_solr c:testasynccollectioncreation s:shard1) [n:127.0.0.1:34409_solr c:testasynccollectioncreation s:shard1 r:core_node4 x:testasynccollectioncreation_shard1_replica_n3 ] o.a.s.c.RecoveryStrategy Registering as Active after recovery.
[junit4] 2> 5901144 INFO (recoveryExecutor-14525-thread-1-processing-x:testasynccollectioncreation_shard1_replica_n3 970c73ed-e9af-47a3-81e7-29fbaab20b2f4045048015462210 r:core_node4 CREATE n:127.0.0.1:34409_solr c:testasynccollectioncreation s:shard1) [n:127.0.0.1:34409_solr c:testasynccollectioncreation s:shard1 r:core_node4 x:testasynccollectioncreation_shard1_replica_n3 ] o.a.s.c.RecoveryStrategy Updating version bucket highest from index after successful recovery.
[junit4] 2> 5901144 INFO (recoveryExecutor-14525-thread-1-processing-x:testasynccollectioncreation_shard1_replica_n3 970c73ed-e9af-47a3-81e7-29fbaab20b2f4045048015462210 r:core_node4 CREATE n:127.0.0.1:34409_solr c:testasynccollectioncreation s:shard1) [n:127.0.0.1:34409_solr c:testasynccollectioncreation s:shard1 r:core_node4 x:testasynccollectioncreation_shard1_replica_n3 ] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1659129837381484544
[junit4] 2> 5901145 INFO (recoveryExecutor-14525-thread-1-processing-x:testasynccollectioncreation_shard1_replica_n3 970c73ed-e9af-47a3-81e7-29fbaab20b2f4045048015462210 r:core_node4 CREATE n:127.0.0.1:34409_solr c:testasynccollectioncreation s:shard1) [n:127.0.0.1:34409_solr c:testasynccollectioncreation s:shard1 r:core_node4 x:testasynccollectioncreation_shard1_replica_n3 ] o.a.s.c.RecoveryStrategy Finished recovery process, successful=[true]
[junit4] 2> 5901246 INFO (zkCallback-14540-thread-3) [ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testasynccollectioncreation/state.json] for collection [testasynccollectioncreation] has occurred - updating... (live nodes size: [2])
[junit4] 2> 5901246 INFO (zkCallback-14540-thread-1) [ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testasynccollectioncreation/state.json] for collection [testasynccollectioncreation] has occurred - updating... (live nodes size: [2])
[junit4] 2> 5901246 INFO (zkCallback-14528-thread-3) [ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testasynccollectioncreation/state.json] for collection [testasynccollectioncreation] has occurred - updating... (live nodes size: [2])
[junit4] 2> 5901246 INFO (zkCallback-14528-thread-1) [ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testasynccollectioncreation/state.json] for collection [testasynccollectioncreation] has occurred - updating.
[...truncated too long message...]
43013_solr ] o.a.s.c.OverseerTaskProcessor :org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /overseer/collection-map-running/mn-repeatedId
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:134)
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:54)
[junit4] 2> at org.apache.zookeeper.ZooKeeper.delete(ZooKeeper.java:1793)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.lambda$delete$1(SolrZkClient.java:263)
[junit4] 2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:71)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.delete(SolrZkClient.java:262)
[junit4] 2> at org.apache.solr.cloud.DistributedMap.remove(DistributedMap.java:98)
[junit4] 2> at org.apache.solr.cloud.OverseerTaskProcessor$Runner.resetTaskWithException(OverseerTaskProcessor.java:579)
[junit4] 2> at org.apache.solr.cloud.OverseerTaskProcessor$Runner.run(OverseerTaskProcessor.java:545)
[junit4] 2> at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:210)
[junit4] 2> at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
[junit4] 2> at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
[junit4] 2> at java.base/java.lang.Thread.run(Thread.java:834)
[junit4] 2>
[junit4] 2> 5947660 INFO (OverseerAutoScalingTriggerThread-72322693050073096-127.0.0.1:43013_solr-n_0000000000) [ ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread woken up but we are closed, exiting.
[junit4] 2> 5947661 INFO (jetty-closer-14722-thread-2) [ ] o.a.s.c.Overseer Overseer (id=72322693050073096-127.0.0.1:43013_solr-n_0000000000) closing
[junit4] 2> 5947661 INFO (closeThreadPool-14729-thread-1) [ ] o.a.s.c.Overseer Overseer (id=72322693050073096-127.0.0.1:43013_solr-n_0000000000) closing
[junit4] 2> 5947662 INFO (jetty-closer-14722-thread-2) [ ] o.e.j.s.AbstractConnector Stopped ServerConnector@72f16fe1{ssl,[ssl, alpn, http/1.1, h2]}{127.0.0.1:0}
[junit4] 2> 5947662 INFO (jetty-closer-14722-thread-2) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@4f600052{/solr,null,UNAVAILABLE}
[junit4] 2> 5947663 INFO (jetty-closer-14722-thread-2) [ ] o.e.j.s.session node0 Stopped scavenging
[junit4] 2> 5947663 INFO (TEST-CollectionsAPIAsyncDistributedZkTest.testAsyncIdRaceCondition-seed#[EC7974BA469EE123]) [ ] o.a.s.c.ZkTestServer Shutting down ZkTestServer.
[junit4] 2> 5947867 WARN (ZkTestServer Run Thread) [ ] o.a.s.c.ZkTestServer Watch limit violations:
[junit4] 2> Maximum concurrent create/delete watches above limit:
[junit4] 2>
[junit4] 2> 3 /solr/aliases.json
[junit4] 2> 2 /solr/packages.json
[junit4] 2> 2 /solr/security.json
[junit4] 2> 2 /solr/collections/testAsyncIdRaceCondition/terms/shard1
[junit4] 2>
[junit4] 2> Maximum concurrent data watches above limit:
[junit4] 2>
[junit4] 2> 9 /solr/collections/testAsyncIdRaceCondition/state.json
[junit4] 2> 3 /solr/clusterstate.json
[junit4] 2> 3 /solr/clusterprops.json
[junit4] 2>
[junit4] 2> Maximum concurrent children watches above limit:
[junit4] 2>
[junit4] 2> 15 /solr/overseer/queue
[junit4] 2> 7 /solr/live_nodes
[junit4] 2> 6 /solr/collections
[junit4] 2> 4 /solr/overseer/collection-queue-work
[junit4] 2>
[junit4] 2> 5947869 INFO (TEST-CollectionsAPIAsyncDistributedZkTest.testAsyncIdRaceCondition-seed#[EC7974BA469EE123]) [ ] o.a.s.c.ZkTestServer waitForServerDown: 127.0.0.1:38666
[junit4] 2> 5947869 INFO (TEST-CollectionsAPIAsyncDistributedZkTest.testAsyncIdRaceCondition-seed#[EC7974BA469EE123]) [ ] o.a.s.c.ZkTestServer parse host and port list: 127.0.0.1:38666
[junit4] 2> 5947869 INFO (TEST-CollectionsAPIAsyncDistributedZkTest.testAsyncIdRaceCondition-seed#[EC7974BA469EE123]) [ ] o.a.s.c.ZkTestServer connecting to 127.0.0.1 38666
[junit4] 2> NOTE: leaving temporary files on disk at: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.api.collections.CollectionsAPIAsyncDistributedZkTest_EC7974BA469EE123-001
[junit4] 2> Feb 21, 2020 7:19:20 AM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
[junit4] 2> WARNING: Will linger awaiting termination of 1 leaked thread(s).
[junit4] 2> NOTE: test params are: codec=CheapBastard, sim=Asserting(org.apache.lucene.search.similarities.AssertingSimilarity@37f522be), locale=es-BR, timezone=America/Manaus
[junit4] 2> NOTE: Linux 4.4.0-170-generic amd64/Oracle Corporation 11.0.4 (64-bit)/cpus=8,threads=1,free=361678384,total=536870912
[junit4] 2> NOTE: All tests run in this JVM: [DirectUpdateHandlerTest, TestFuzzyAnalyzedSuggestions, TestCloudDeleteByQuery, JvmMetricsTest, HighlighterTest, ExplicitHLLTest, TriLevelCompositeIdRoutingTest, DistanceFunctionTest, ReplaceNodeNoTargetTest, TestDFRSimilarityFactory, AnalyticsQueryTest, TestInitParams, TestRawResponseWriter, ClusterStateUpdateTest, DistanceUnitsTest, SimpleFacetsTest, SOLR749Test, OverriddenZkACLAndCredentialsProvidersTest, BasicAuthOnSingleNodeTest, TestFilteredDocIdSet, TestHighlightDedupGrouping, TestManagedSchemaThreadSafety, TestSortableTextField, TestConfig, TestElisionMultitermQuery, PeerSyncTest, XsltUpdateRequestHandlerTest, V2ApiIntegrationTest, TestConfigSetsAPI, SolrRrdBackendFactoryTest, BadCopyFieldTest, TestExactSharedStatsCache, TestSolrCoreSnapshots, VersionInfoTest, UninvertDocValuesMergePolicyTest, PeerSyncWithLeaderAndIndexFingerprintCachingTest, HLLUtilTest, CoreSorterTest, NestedShardedAtomicUpdateTest, TestFieldCacheSort, SpatialFilterTest, TestWithCollection, ZkCollectionPropsCachingTest, BlockCacheTest, TemplateUpdateProcessorTest, TestDynamicURP, HdfsWriteToMultipleCollectionsTest, ResponseHeaderTest, OpenExchangeRatesOrgProviderTest, ForceLeaderTest, TestSchemaField, SplitShardTest, TaggerTest, FuzzySearchTest, AnalysisErrorHandlingTest, DirectSolrSpellCheckerTest, TestReloadDeadlock, TestSimTriggerIntegration, AssignTest, RestoreTriggerStateTest, TestRestManager, SolrIndexMetricsTest, TestMinMaxOnMultiValuedField, TestUseDocValuesAsStored2, IgnoreCommitOptimizeUpdateProcessorFactoryTest, CloudExitableDirectoryReaderTest, BigEndianAscendingWordDeserializerTest, OverseerStatusTest, TestSolrCloudWithHadoopAuthPlugin, PathHierarchyTokenizerFactoryTest, TestEmbeddedSolrServerAdminHandler, TestPartialUpdateDeduplication, TestInPlaceUpdatesDistrib, TestFieldResource, TestStressVersions, TestSubQueryTransformerDistrib, SmileWriterTest, HdfsCollectionsAPIDistributedZkTest, TestRequestForwarding, TestTestInjection, SegmentsInfoRequestHandlerTest, HdfsAutoAddReplicasIntegrationTest, HdfsDirectoryTest, OrderedExecutorTest, TestQueryUtils, ShardsWhitelistTest, SignatureUpdateProcessorFactoryTest, SplitByPrefixTest, TestCloudPivotFacet, CdcrVersionReplicationTest, CdcrOpsAndBoundariesTest, TestTlogReplayVsRecovery, TestCloudNestedDocsSort, UpdateRequestProcessorFactoryTest, TestBM25SimilarityFactory, HttpPartitionWithTlogReplicasTest, TestHalfAndHalfDocValues, IndexSizeTriggerSizeEstimationTest, HdfsThreadLeakTest, MissingSegmentRecoveryTest, TestSolr4Spatial2, AlternateDirectoryTest, UtilsToolTest, BasicDistributedZk2Test, TestCloudSchemaless, SuggesterFSTTest, TestNumericTerms64, TestUnifiedSolrHighlighter, DistributedQueueTest, ScheduledTriggerTest, TestSolrQueryResponse, LeaderElectionTest, TestCursorMarkWithoutUniqueKey, TestMiniSolrCloudClusterSSL, ChangedSchemaMergeTest, TestSimClusterStateProvider, HealthCheckHandlerTest, ReturnFieldsTest, MetricTriggerTest, NestedAtomicUpdateTest, TestTrieFacet, ClassificationUpdateProcessorFactoryTest, TestMergePolicyConfig, TestDocTermOrdsUninvertLimit, TestCollapseQParserPlugin, TestRandomFlRTGCloud, MetricUtilsTest, OverseerRolesTest, DistributedUpdateProcessorTest, URLClassifyProcessorTest, TestStressCloudBlindAtomicUpdates, TriggerEventQueueTest, TestGraphTermsQParserPlugin, PackageManagerCLITest, TestExceedMaxTermLength, TestBlendedInfixSuggestions, TestSchemaNameResource, TestPerFieldSimilarity, ByteBuffersDirectoryFactoryTest, TestMinHashQParser, ShardRoutingTest, TestXIncludeConfig, AutoScalingHandlerTest, FastVectorHighlighterTest, TestSolrTestCaseJ4, RequestLoggingTest, TestApiFramework, CSVRequestHandlerTest, TestCryptoKeys, BasicFunctionalityTest, ConvertedLegacyTest, CursorPagingTest, DisMaxRequestHandlerTest, DistributedIntervalFacetingTest, MinimalSchemaTest, OutputWriterTest, SampleTest, SolrInfoBeanTest, SolrTestCaseJ4Test, TestDistributedMissingSort, TestDocumentBuilder, TestGroupingSearch, TestRandomDVFaceting, TestRandomFaceting, ConcurrentCreateRoutedAliasTest, ConnectionManagerTest, CreateCollectionCleanupTest, DeleteInactiveReplicaTest, DeleteLastCustomShardedReplicaTest, DeleteNodeTest, DeleteReplicaTest, LeaderElectionIntegrationTest, LegacyCloudClusterPropTest, MetricsHistoryIntegrationTest, ReindexCollectionTest, SolrCLIZkUtilsTest, SystemCollectionCompatTest, TestAuthenticationFramework, TestCloudRecovery2, TestCloudSearcherWarming, TestConfigSetsAPIZkFailure, TestDistribDocBasedVersion, TestHashPartitioner, TestLocalStatsCacheCloud, TestPullReplica, TestPullReplicaErrorHandling, TestShortCircuitedRequests, TestSkipOverseerOperations, TestTolerantUpdateProcessorCloud, TestTolerantUpdateProcessorRandomCloud, TestUtilizeNode, TestWaitForStateWithJettyShutdowns, ZkCLITest, ZkControllerTest, ZkNodePropsTest, ZkShardTermsTest, ZkSolrClientTest, CollectionTooManyReplicasTest, CollectionsAPIAsyncDistributedZkTest]
[junit4] Completed [561/904 (1!)] on J1 in 54.89s, 4 tests, 1 failure <<< FAILURES!
[...truncated 53733 lines...]
[JENKINS] Lucene-Solr-NightlyTests-master - Build # 2109 - Still
Failing
Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-master/2109/
2 tests failed.
FAILED: org.apache.lucene.index.TestIndexingSequenceNumbers.testStressConcurrentCommit
Error Message:
this IndexWriter is closed
Stack Trace:
org.apache.lucene.store.AlreadyClosedException: this IndexWriter is closed
at org.apache.lucene.index.IndexWriter.ensureOpen(IndexWriter.java:681)
at org.apache.lucene.index.IndexWriter.ensureOpen(IndexWriter.java:695)
at org.apache.lucene.index.IndexWriter.commit(IndexWriter.java:3392)
at org.apache.lucene.index.TestIndexingSequenceNumbers.testStressConcurrentCommit(TestIndexingSequenceNumbers.java:228)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:566)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1754)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:942)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:978)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:992)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:370)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:819)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:470)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:951)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:836)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:887)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:898)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:370)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.lambda$forkTimeoutingTask$0(ThreadLeakControl.java:826)
at java.base/java.lang.Thread.run(Thread.java:834)
Caused by: java.lang.OutOfMemoryError: Java heap space
at org.apache.lucene.index.DocumentsWriterPerThread$IntBlockAllocator.getIntBlock(DocumentsWriterPerThread.java:660)
at org.apache.lucene.util.IntBlockPool.nextBuffer(IntBlockPool.java:155)
at org.apache.lucene.index.TermsHashPerField.add(TermsHashPerField.java:159)
at org.apache.lucene.index.DefaultIndexingChain$PerField.invert(DefaultIndexingChain.java:862)
at org.apache.lucene.index.DefaultIndexingChain.processField(DefaultIndexingChain.java:442)
at org.apache.lucene.index.DefaultIndexingChain.processDocument(DefaultIndexingChain.java:406)
at org.apache.lucene.index.DocumentsWriterPerThread.updateDocument(DocumentsWriterPerThread.java:250)
at org.apache.lucene.index.DocumentsWriter.updateDocument(DocumentsWriter.java:495)
at org.apache.lucene.index.IndexWriter.updateDocument(IndexWriter.java:1594)
at org.apache.lucene.index.IndexWriter.updateDocument(IndexWriter.java:1586)
at org.apache.lucene.index.TestIndexingSequenceNumbers$2.run(TestIndexingSequenceNumbers.java:208)
FAILED: org.apache.solr.cloud.RemoteQueryErrorTest.test
Error Message:
Expected: a string containing "Document is missing mandatory uniqueKey field: id" but: was "Error from server at http://127.0.0.1:34518/solr: No active replicas found for collection: collection"
Stack Trace:
java.lang.AssertionError:
Expected: a string containing "Document is missing mandatory uniqueKey field: id"
but: was "Error from server at http://127.0.0.1:34518/solr: No active replicas found for collection: collection"
at __randomizedtesting.SeedInfo.seed([4E094E65EF364989:C65D71BF41CA2471]:0)
at org.hamcrest.MatcherAssert.assertThat(MatcherAssert.java:20)
at org.junit.Assert.assertThat(Assert.java:956)
at org.junit.Assert.assertThat(Assert.java:923)
at org.apache.solr.cloud.RemoteQueryErrorTest.test(RemoteQueryErrorTest.java:53)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:566)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1754)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:942)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:978)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:992)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:370)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:819)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:470)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:951)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:836)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:887)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:898)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:370)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.lambda$forkTimeoutingTask$0(ThreadLeakControl.java:826)
at java.base/java.lang.Thread.run(Thread.java:834)
Build Log:
[...truncated 842 lines...]
[junit4] Suite: org.apache.lucene.index.TestIndexingSequenceNumbers
[junit4] 2> Feb 19, 2020 9:19:00 PM com.carrotsearch.randomizedtesting.RandomizedRunner$QueueUncaughtExceptionsHandler uncaughtException
[junit4] 2> WARNING: Uncaught exception in thread: Thread[Thread-3343,5,TGRP-TestIndexingSequenceNumbers]
[junit4] 2> java.lang.OutOfMemoryError: Java heap space
[junit4] 2>
[junit4] 2> Feb 19, 2020 9:19:00 PM com.carrotsearch.randomizedtesting.RandomizedRunner$QueueUncaughtExceptionsHandler uncaughtException
[junit4] 2> WARNING: Uncaught exception in thread: Thread[Thread-3345,5,TGRP-TestIndexingSequenceNumbers]
[junit4] 2> java.lang.OutOfMemoryError: Java heap space
[junit4] 2>
[junit4] 2> Feb 19, 2020 9:19:00 PM com.carrotsearch.randomizedtesting.RandomizedRunner$QueueUncaughtExceptionsHandler uncaughtException
[junit4] 2> WARNING: Uncaught exception in thread: Thread[Thread-3344,5,TGRP-TestIndexingSequenceNumbers]
[junit4] 2> java.lang.OutOfMemoryError: Java heap space
[junit4] 2>
[junit4] 2> Feb 19, 2020 9:19:00 PM com.carrotsearch.randomizedtesting.RandomizedRunner$QueueUncaughtExceptionsHandler uncaughtException
[junit4] 2> WARNING: Uncaught exception in thread: Thread[Thread-3338,5,TGRP-TestIndexingSequenceNumbers]
[junit4] 2> java.lang.AssertionError: pendingNumDocs 1837 != 1094 totalMaxDoc
[junit4] 2> at __randomizedtesting.SeedInfo.seed([542FC45EC20059D4]:0)
[junit4] 2> at org.apache.lucene.index.IndexWriter.rollbackInternal(IndexWriter.java:2255)
[junit4] 2> at org.apache.lucene.index.IndexWriter.maybeCloseOnTragicEvent(IndexWriter.java:4894)
[junit4] 2> at org.apache.lucene.index.IndexWriter.tragicEvent(IndexWriter.java:4884)
[junit4] 2> at org.apache.lucene.index.IndexWriter.updateDocuments(IndexWriter.java:1292)
[junit4] 2> at org.apache.lucene.index.IndexWriter.updateDocuments(IndexWriter.java:1277)
[junit4] 2> at org.apache.lucene.index.TestIndexingSequenceNumbers$2.run(TestIndexingSequenceNumbers.java:206)
[junit4] 2>
[junit4] 2> NOTE: download the large Jenkins line-docs file by running 'ant get-jenkins-line-docs' in the lucene directory.
[junit4] 2> NOTE: reproduce with: ant test -Dtestcase=TestIndexingSequenceNumbers -Dtests.method=testStressConcurrentCommit -Dtests.seed=542FC45EC20059D4 -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/test-data/enwiki.random.lines.txt -Dtests.locale=en-FJ -Dtests.timezone=America/Coral_Harbour -Dtests.asserts=true -Dtests.file.encoding=UTF-8
[junit4] ERROR 668s J2 | TestIndexingSequenceNumbers.testStressConcurrentCommit <<<
[junit4] > Throwable #1: org.apache.lucene.store.AlreadyClosedException: this IndexWriter is closed
[junit4] > at org.apache.lucene.index.IndexWriter.ensureOpen(IndexWriter.java:681)
[junit4] > at org.apache.lucene.index.IndexWriter.ensureOpen(IndexWriter.java:695)
[junit4] > at org.apache.lucene.index.IndexWriter.commit(IndexWriter.java:3392)
[junit4] > at org.apache.lucene.index.TestIndexingSequenceNumbers.testStressConcurrentCommit(TestIndexingSequenceNumbers.java:228)
[junit4] > at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
[junit4] > at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
[junit4] > at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
[junit4] > at java.base/java.lang.reflect.Method.invoke(Method.java:566)
[junit4] > at java.base/java.lang.Thread.run(Thread.java:834)
[junit4] > Caused by: java.lang.OutOfMemoryError: Java heap space
[junit4] > at org.apache.lucene.index.DocumentsWriterPerThread$IntBlockAllocator.getIntBlock(DocumentsWriterPerThread.java:660)
[junit4] > at org.apache.lucene.util.IntBlockPool.nextBuffer(IntBlockPool.java:155)
[junit4] > at org.apache.lucene.index.TermsHashPerField.add(TermsHashPerField.java:159)
[junit4] > at org.apache.lucene.index.DefaultIndexingChain$PerField.invert(DefaultIndexingChain.java:862)
[junit4] > at org.apache.lucene.index.DefaultIndexingChain.processField(DefaultIndexingChain.java:442)
[junit4] > at org.apache.lucene.index.DefaultIndexingChain.processDocument(DefaultIndexingChain.java:406)
[junit4] > at org.apache.lucene.index.DocumentsWriterPerThread.updateDocument(DocumentsWriterPerThread.java:250)
[junit4] > at org.apache.lucene.index.DocumentsWriter.updateDocument(DocumentsWriter.java:495)
[junit4] > at org.apache.lucene.index.IndexWriter.updateDocument(IndexWriter.java:1594)
[junit4] > at org.apache.lucene.index.IndexWriter.updateDocument(IndexWriter.java:1586)
[junit4] > at org.apache.lucene.index.TestIndexingSequenceNumbers$2.run(TestIndexingSequenceNumbers.java:208)Throwable #2: com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=3430, name=Thread-3343, state=RUNNABLE, group=TGRP-TestIndexingSequenceNumbers]
[junit4] > Caused by: java.lang.OutOfMemoryError: Java heap spaceThrowable #3: com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=3431, name=Thread-3344, state=RUNNABLE, group=TGRP-TestIndexingSequenceNumbers]
[junit4] > Caused by: java.lang.OutOfMemoryError: Java heap spaceThrowable #4: com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=3432, name=Thread-3345, state=RUNNABLE, group=TGRP-TestIndexingSequenceNumbers]
[junit4] > Caused by: java.lang.OutOfMemoryError: Java heap spaceThrowable #5: com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=3425, name=Thread-3338, state=RUNNABLE, group=TGRP-TestIndexingSequenceNumbers]
[junit4] > Caused by: java.lang.AssertionError: pendingNumDocs 1837 != 1094 totalMaxDoc
[junit4] > at __randomizedtesting.SeedInfo.seed([542FC45EC20059D4]:0)
[junit4] > at org.apache.lucene.index.IndexWriter.rollbackInternal(IndexWriter.java:2255)
[junit4] > at org.apache.lucene.index.IndexWriter.maybeCloseOnTragicEvent(IndexWriter.java:4894)
[junit4] > at org.apache.lucene.index.IndexWriter.tragicEvent(IndexWriter.java:4884)
[junit4] > at org.apache.lucene.index.IndexWriter.updateDocuments(IndexWriter.java:1292)
[junit4] > at org.apache.lucene.index.IndexWriter.updateDocuments(IndexWriter.java:1277)
[junit4] > at org.apache.lucene.index.TestIndexingSequenceNumbers$2.run(TestIndexingSequenceNumbers.java:206)
[junit4] 2> NOTE: leaving temporary files on disk at: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/lucene/build/core/test/J2/temp/lucene.index.TestIndexingSequenceNumbers_542FC45EC20059D4-001
[junit4] 2> NOTE: test params are: codec=Asserting(Lucene84): {id=PostingsFormat(name=LuceneVarGapDocFreqInterval)}, docValues:{thread=DocValuesFormat(name=Lucene80), id=DocValuesFormat(name=Asserting), ___soft_deletes=DocValuesFormat(name=Lucene80)}, maxPointsInLeafNode=1417, maxMBSortInHeap=5.778054691683031, sim=Asserting(org.apache.lucene.search.similarities.AssertingSimilarity@5b2e96e7), locale=en-FJ, timezone=America/Coral_Harbour
[junit4] 2> NOTE: Linux 4.4.0-170-generic amd64/Oracle Corporation 11.0.4 (64-bit)/cpus=8,threads=1,free=40220120,total=536870912
[junit4] 2> NOTE: All tests run in this JVM: [TestDistributionLL, TestLatLonLineShapeQueries, TestFeatureSort, TestRateLimiter, TestAllFilesCheckIndexHeader, TestIntroSelector, TestDocIdSetIterator, TestCharArrayMap, TestAutomatonQueryUnicode, TestIndexWriterExceptions, TestMultiLevelSkipList, TestNot, TestStressIndexing2, TestDocIdSetBuilder, TestXYMultiPointShapeQueries, TestCompetitiveFreqNormAccumulator, TestMatchAllDocsQuery, TestBooleanOr, TestPoint2D, TestStringMSBRadixSorter, TestNewestSegment, TestElevationComparator, TestSpanTermQuery, TestIndexWriterOutOfFileDescriptors, TestDocsAndPositions, TestTryDelete, TestTermdocPerf, TestLongPostings, TestQueryRescorer, TestMaxTermFrequency, Test2BTerms, TestBytesRefArray, TestBoolean2, Test2BBinaryDocValues, TestFixedBitSet, TestTermsEnum2, TestIndexingSequenceNumbers]
[junit4] Completed [172/560 (1!)] on J2 in 892.84s, 8 tests, 1 error <<< FAILURES!
[...truncated 1495 lines...]
[junit4] JVM J2: stdout was not empty, see: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/lucene/build/core/test/temp/junit4-J2-20200220_014607_80815275572708860027501.sysout
[junit4] >>> JVM J2 emitted unexpected output (verbatim) ----
[junit4] java.lang.OutOfMemoryError: Java heap space
[junit4] Dumping heap to /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/heapdumps/java_pid1913.hprof ...
[junit4] Heap dump file created [629315008 bytes in 33.845 secs]
[junit4] <<< JVM J2: EOF ----
[junit4] JVM J2: stderr was not empty, see: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/lucene/build/core/test/temp/junit4-J2-20200220_014607_80813675417066397877257.syserr
[junit4] >>> JVM J2 emitted unexpected output (verbatim) ----
[junit4]
[junit4] Exception: java.lang.IllegalArgumentException thrown from the UncaughtExceptionHandler in thread "Thread-3339"
[junit4] <<< JVM J2: EOF ----
[...truncated 12413 lines...]
[junit4] Suite: org.apache.solr.cloud.RemoteQueryErrorTest
[junit4] 2> 5896951 INFO (SUITE-RemoteQueryErrorTest-seed#[4E094E65EF364989]-worker) [ ] o.a.s.SolrTestCase Setting 'solr.default.confdir' system property to test-framework derived value of '/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/server/solr/configsets/_default/conf'
[junit4] 2> 5896952 INFO (SUITE-RemoteQueryErrorTest-seed#[4E094E65EF364989]-worker) [ ] o.a.s.SolrTestCaseJ4 Created dataDir: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.RemoteQueryErrorTest_4E094E65EF364989-001/data-dir-210-001
[junit4] 2> 5896952 INFO (SUITE-RemoteQueryErrorTest-seed#[4E094E65EF364989]-worker) [ ] o.a.s.SolrTestCaseJ4 Using PointFields (NUMERIC_POINTS_SYSPROP=true) w/NUMERIC_DOCVALUES_SYSPROP=true
[junit4] 2> 5896969 INFO (SUITE-RemoteQueryErrorTest-seed#[4E094E65EF364989]-worker) [ ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (true) via: @org.apache.solr.util.RandomizeSSL(reason="", value=0.0/0.0, ssl=0.0/0.0, clientAuth=0.0/0.0)
[junit4] 2> 5896969 INFO (SUITE-RemoteQueryErrorTest-seed#[4E094E65EF364989]-worker) [ ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
[junit4] 2> 5896970 INFO (SUITE-RemoteQueryErrorTest-seed#[4E094E65EF364989]-worker) [ ] o.a.s.c.MiniSolrCloudCluster Starting cluster of 3 servers in /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.RemoteQueryErrorTest_4E094E65EF364989-001/tempDir-001
[junit4] 2> 5896970 INFO (SUITE-RemoteQueryErrorTest-seed#[4E094E65EF364989]-worker) [ ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
[junit4] 2> 5896971 INFO (ZkTestServer Run Thread) [ ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 5896971 INFO (ZkTestServer Run Thread) [ ] o.a.s.c.ZkTestServer Starting server
[junit4] 2> 5897078 INFO (SUITE-RemoteQueryErrorTest-seed#[4E094E65EF364989]-worker) [ ] o.a.s.c.ZkTestServer start zk server on port:43070
[junit4] 2> 5897078 INFO (SUITE-RemoteQueryErrorTest-seed#[4E094E65EF364989]-worker) [ ] o.a.s.c.ZkTestServer waitForServerUp: 127.0.0.1:43070
[junit4] 2> 5897078 INFO (SUITE-RemoteQueryErrorTest-seed#[4E094E65EF364989]-worker) [ ] o.a.s.c.ZkTestServer parse host and port list: 127.0.0.1:43070
[junit4] 2> 5897078 INFO (SUITE-RemoteQueryErrorTest-seed#[4E094E65EF364989]-worker) [ ] o.a.s.c.ZkTestServer connecting to 127.0.0.1 43070
[junit4] 2> 5897080 INFO (SUITE-RemoteQueryErrorTest-seed#[4E094E65EF364989]-worker) [ ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
[junit4] 2> 5897081 INFO (zkConnectionManagerCallback-20719-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 5897082 INFO (SUITE-RemoteQueryErrorTest-seed#[4E094E65EF364989]-worker) [ ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
[junit4] 2> 5897084 INFO (SUITE-RemoteQueryErrorTest-seed#[4E094E65EF364989]-worker) [ ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
[junit4] 2> 5897085 INFO (zkConnectionManagerCallback-20721-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 5897086 INFO (SUITE-RemoteQueryErrorTest-seed#[4E094E65EF364989]-worker) [ ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
[junit4] 2> 5897086 INFO (SUITE-RemoteQueryErrorTest-seed#[4E094E65EF364989]-worker) [ ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
[junit4] 2> 5897087 INFO (zkConnectionManagerCallback-20723-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 5897087 INFO (SUITE-RemoteQueryErrorTest-seed#[4E094E65EF364989]-worker) [ ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
[junit4] 2> 5897191 WARN (jetty-launcher-20724-thread-2) [ ] o.e.j.s.h.g.GzipHandler minGzipSize of 0 is inefficient for short content, break even is size 23
[junit4] 2> 5897191 WARN (jetty-launcher-20724-thread-3) [ ] o.e.j.s.h.g.GzipHandler minGzipSize of 0 is inefficient for short content, break even is size 23
[junit4] 2> 5897191 WARN (jetty-launcher-20724-thread-1) [ ] o.e.j.s.h.g.GzipHandler minGzipSize of 0 is inefficient for short content, break even is size 23
[junit4] 2> 5897191 INFO (jetty-launcher-20724-thread-3) [ ] o.a.s.c.s.e.JettySolrRunner Start Jetty (configured port=0, binding port=0)
[junit4] 2> 5897191 INFO (jetty-launcher-20724-thread-3) [ ] o.a.s.c.s.e.JettySolrRunner Trying to start Jetty on port 0 try number 1 ...
[junit4] 2> 5897191 INFO (jetty-launcher-20724-thread-1) [ ] o.a.s.c.s.e.JettySolrRunner Start Jetty (configured port=0, binding port=0)
[junit4] 2> 5897191 INFO (jetty-launcher-20724-thread-1) [ ] o.a.s.c.s.e.JettySolrRunner Trying to start Jetty on port 0 try number 1 ...
[junit4] 2> 5897191 INFO (jetty-launcher-20724-thread-2) [ ] o.a.s.c.s.e.JettySolrRunner Start Jetty (configured port=0, binding port=0)
[junit4] 2> 5897191 INFO (jetty-launcher-20724-thread-2) [ ] o.a.s.c.s.e.JettySolrRunner Trying to start Jetty on port 0 try number 1 ...
[junit4] 2> 5897191 INFO (jetty-launcher-20724-thread-3) [ ] o.e.j.s.Server jetty-9.4.24.v20191120; built: 2019-11-20T21:37:49.771Z; git: 363d5f2df3a8a28de40604320230664b9c793c16; jvm 11.0.4+10-LTS
[junit4] 2> 5897191 INFO (jetty-launcher-20724-thread-1) [ ] o.e.j.s.Server jetty-9.4.24.v20191120; built: 2019-11-20T21:37:49.771Z; git: 363d5f2df3a8a28de40604320230664b9c793c16; jvm 11.0.4+10-LTS
[junit4] 2> 5897191 INFO (jetty-launcher-20724-thread-2) [ ] o.e.j.s.Server jetty-9.4.24.v20191120; built: 2019-11-20T21:37:49.771Z; git: 363d5f2df3a8a28de40604320230664b9c793c16; jvm 11.0.4+10-LTS
[junit4] 2> 5897193 INFO (jetty-launcher-20724-thread-2) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 5897193 INFO (jetty-launcher-20724-thread-2) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 5897193 INFO (jetty-launcher-20724-thread-2) [ ] o.e.j.s.session node0 Scavenging every 600000ms
[junit4] 2> 5897194 INFO (jetty-launcher-20724-thread-2) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@c49d1de{/solr,null,AVAILABLE}
[junit4] 2> 5897194 INFO (jetty-launcher-20724-thread-3) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 5897194 INFO (jetty-launcher-20724-thread-3) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 5897194 INFO (jetty-launcher-20724-thread-3) [ ] o.e.j.s.session node0 Scavenging every 660000ms
[junit4] 2> 5897195 INFO (jetty-launcher-20724-thread-3) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@7681c073{/solr,null,AVAILABLE}
[junit4] 2> 5897195 INFO (jetty-launcher-20724-thread-1) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 5897195 INFO (jetty-launcher-20724-thread-1) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 5897195 INFO (jetty-launcher-20724-thread-1) [ ] o.e.j.s.session node0 Scavenging every 600000ms
[junit4] 2> 5897195 INFO (jetty-launcher-20724-thread-3) [ ] o.e.j.s.AbstractConnector Started ServerConnector@132aa614{HTTP/1.1,[http/1.1, h2c]}{127.0.0.1:44398}
[junit4] 2> 5897195 INFO (jetty-launcher-20724-thread-3) [ ] o.e.j.s.Server Started @5897234ms
[junit4] 2> 5897195 INFO (jetty-launcher-20724-thread-3) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=44398}
[junit4] 2> 5897195 INFO (jetty-launcher-20724-thread-1) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@754185a1{/solr,null,AVAILABLE}
[junit4] 2> 5897196 ERROR (jetty-launcher-20724-thread-3) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 5897196 INFO (jetty-launcher-20724-thread-3) [ ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
[junit4] 2> 5897196 INFO (jetty-launcher-20724-thread-2) [ ] o.e.j.s.AbstractConnector Started ServerConnector@8e71959{HTTP/1.1,[http/1.1, h2c]}{127.0.0.1:34518}
[junit4] 2> 5897196 INFO (jetty-launcher-20724-thread-2) [ ] o.e.j.s.Server Started @5897234ms
[junit4] 2> 5897196 INFO (jetty-launcher-20724-thread-2) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=34518}
[junit4] 2> 5897196 INFO (jetty-launcher-20724-thread-3) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version 9.0.0
[junit4] 2> 5897196 INFO (jetty-launcher-20724-thread-3) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 5897196 INFO (jetty-launcher-20724-thread-3) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr
[junit4] 2> 5897196 INFO (jetty-launcher-20724-thread-3) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2020-02-20T07:44:05.198185Z
[junit4] 2> 5897196 ERROR (jetty-launcher-20724-thread-2) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 5897196 INFO (jetty-launcher-20724-thread-2) [ ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
[junit4] 2> 5897196 INFO (jetty-launcher-20724-thread-1) [ ] o.e.j.s.AbstractConnector Started ServerConnector@558413a6{HTTP/1.1,[http/1.1, h2c]}{127.0.0.1:33252}
[junit4] 2> 5897196 INFO (jetty-launcher-20724-thread-1) [ ] o.e.j.s.Server Started @5897234ms
[junit4] 2> 5897196 INFO (jetty-launcher-20724-thread-2) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version 9.0.0
[junit4] 2> 5897196 INFO (jetty-launcher-20724-thread-1) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=33252}
[junit4] 2> 5897196 ERROR (jetty-launcher-20724-thread-1) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 5897196 INFO (jetty-launcher-20724-thread-1) [ ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
[junit4] 2> 5897196 INFO (jetty-launcher-20724-thread-1) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version 9.0.0
[junit4] 2> 5897196 INFO (jetty-launcher-20724-thread-1) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 5897196 INFO (jetty-launcher-20724-thread-1) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr
[junit4] 2> 5897196 INFO (jetty-launcher-20724-thread-1) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2020-02-20T07:44:05.198584Z
[junit4] 2> 5897212 INFO (jetty-launcher-20724-thread-2) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 5897212 INFO (jetty-launcher-20724-thread-2) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr
[junit4] 2> 5897212 INFO (jetty-launcher-20724-thread-2) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2020-02-20T07:44:05.214737Z
[junit4] 2> 5897230 INFO (jetty-launcher-20724-thread-1) [ ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
[junit4] 2> 5897230 INFO (jetty-launcher-20724-thread-2) [ ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
[junit4] 2> 5897231 INFO (zkConnectionManagerCallback-20726-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 5897231 INFO (jetty-launcher-20724-thread-1) [ ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
[junit4] 2> 5897232 INFO (jetty-launcher-20724-thread-1) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 5897238 INFO (jetty-launcher-20724-thread-3) [ ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
[junit4] 2> 5897250 INFO (zkConnectionManagerCallback-20730-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 5897250 INFO (jetty-launcher-20724-thread-3) [ ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
[junit4] 2> 5897250 INFO (jetty-launcher-20724-thread-3) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 5897310 INFO (zkConnectionManagerCallback-20728-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 5897310 INFO (jetty-launcher-20724-thread-2) [ ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
[junit4] 2> 5897322 INFO (jetty-launcher-20724-thread-2) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 5899962 INFO (jetty-launcher-20724-thread-2) [ ] o.a.s.h.c.HttpShardHandlerFactory Host whitelist initialized: WhitelistHostChecker [whitelistHosts=null, whitelistHostCheckingEnabled=true]
[junit4] 2> 5899963 WARN (jetty-launcher-20724-thread-2) [ ] o.e.j.u.s.S.config Trusting all certificates configured for Client@18ee0a8f[provider=null,keyStore=null,trustStore=null]
[junit4] 2> 5899963 WARN (jetty-launcher-20724-thread-2) [ ] o.e.j.u.s.S.config No Client EndPointIdentificationAlgorithm configured for Client@18ee0a8f[provider=null,keyStore=null,trustStore=null]
[junit4] 2> 5900027 WARN (jetty-launcher-20724-thread-2) [ ] o.e.j.u.s.S.config Trusting all certificates configured for Client@5bf12d70[provider=null,keyStore=null,trustStore=null]
[junit4] 2> 5900027 WARN (jetty-launcher-20724-thread-2) [ ] o.e.j.u.s.S.config No Client EndPointIdentificationAlgorithm configured for Client@5bf12d70[provider=null,keyStore=null,trustStore=null]
[junit4] 2> 5900029 INFO (jetty-launcher-20724-thread-2) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:43070/solr
[junit4] 2> 5900071 INFO (jetty-launcher-20724-thread-2) [ ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
[junit4] 2> 5900128 INFO (zkConnectionManagerCallback-20739-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 5900144 INFO (jetty-launcher-20724-thread-2) [ ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
[junit4] 2> 5900424 INFO (jetty-launcher-20724-thread-2) [n:127.0.0.1:34518_solr ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
[junit4] 2> 5900501 INFO (zkConnectionManagerCallback-20741-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 5900510 INFO (jetty-launcher-20724-thread-2) [n:127.0.0.1:34518_solr ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
[junit4] 2> 5901158 INFO (jetty-launcher-20724-thread-2) [n:127.0.0.1:34518_solr ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:34518_solr
[junit4] 2> 5901175 INFO (jetty-launcher-20724-thread-2) [n:127.0.0.1:34518_solr ] o.a.s.c.Overseer Overseer (id=72317128431894535-127.0.0.1:34518_solr-n_0000000000) starting
[junit4] 2> 5901204 INFO (OverseerStateUpdate-72317128431894535-127.0.0.1:34518_solr-n_0000000000) [n:127.0.0.1:34518_solr ] o.a.s.c.Overseer Starting to work on the main queue : 127.0.0.1:34518_solr
[junit4] 2> 5901253 INFO (jetty-launcher-20724-thread-2) [n:127.0.0.1:34518_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:34518_solr
[junit4] 2> 5901273 INFO (jetty-launcher-20724-thread-2) [n:127.0.0.1:34518_solr ] o.a.s.p.PackageLoader /packages.json updated to version -1
[junit4] 2> 5901273 WARN (jetty-launcher-20724-thread-2) [n:127.0.0.1:34518_solr ] o.a.s.c.CoreContainer Not all security plugins configured! authentication=disabled authorization=disabled. Solr is only as secure as you make it. Consider configuring authentication/authorization before exposing Solr to users internal or external. See https://s.apache.org/solrsecurity for more info
[junit4] 2> 5901399 INFO (zkCallback-20740-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 5901533 INFO (jetty-launcher-20724-thread-2) [n:127.0.0.1:34518_solr ] o.a.s.h.a.MetricsHistoryHandler No .system collection, keeping metrics history in memory.
[junit4] 2> 5901786 INFO (jetty-launcher-20724-thread-2) [n:127.0.0.1:34518_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_34518.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@74c51b81
[junit4] 2> 5902164 INFO (jetty-launcher-20724-thread-2) [n:127.0.0.1:34518_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_34518.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@74c51b81
[junit4] 2> 5902165 INFO (jetty-launcher-20724-thread-2) [n:127.0.0.1:34518_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_34518.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@74c51b81
[junit4] 2> 5902183 INFO (jetty-launcher-20724-thread-2) [n:127.0.0.1:34518_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.RemoteQueryErrorTest_4E094E65EF364989-001/tempDir-001/node2/.
[junit4] 2> 5902709 INFO (jetty-launcher-20724-thread-3) [ ] o.a.s.h.c.HttpShardHandlerFactory Host whitelist initialized: WhitelistHostChecker [whitelistHosts=null, whitelistHostCheckingEnabled=true]
[junit4] 2> 5902731 WARN (jetty-launcher-20724-thread-3) [ ] o.e.j.u.s.S.config Trusting all certificates configured for Client@2f4a3a70[provider=null,keyStore=null,trustStore=null]
[junit4] 2> 5902731 WARN (jetty-launcher-20724-thread-3) [ ] o.e.j.u.s.S.config No Client EndPointIdentificationAlgorithm configured for Client@2f4a3a70[provider=null,keyStore=null,trustStore=null]
[junit4] 2> 5902738 WARN (jetty-launcher-20724-thread-3) [ ] o.e.j.u.s.S.config Trusting all certificates configured for Client@385a3118[provider=null,keyStore=null,trustStore=null]
[junit4] 2> 5902738 WARN (jetty-launcher-20724-thread-3) [ ] o.e.j.u.s.S.config No Client EndPointIdentificationAlgorithm configured for Client@385a3118[provider=null,keyStore=null,trustStore=null]
[junit4] 2> 5902760 INFO (jetty-launcher-20724-thread-3) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:43070/solr
[junit4] 2> 5902761 INFO (jetty-launcher-20724-thread-3) [ ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
[junit4] 2> 5902762 INFO (zkConnectionManagerCallback-20751-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 5902762 INFO (jetty-launcher-20724-thread-3) [ ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
[junit4] 2> 5902986 INFO (jetty-launcher-20724-thread-3) [n:127.0.0.1:44398_solr ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
[junit4] 2> 5902998 INFO (zkConnectionManagerCallback-20753-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 5903010 INFO (jetty-launcher-20724-thread-3) [n:127.0.0.1:44398_solr ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
[junit4] 2> 5903034 INFO (jetty-launcher-20724-thread-3) [n:127.0.0.1:44398_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 5903205 INFO (jetty-launcher-20724-thread-3) [n:127.0.0.1:44398_solr ] o.a.s.c.ZkController Publish node=127.0.0.1:44398_solr as DOWN
[junit4] 2> 5903257 INFO (jetty-launcher-20724-thread-3) [n:127.0.0.1:44398_solr ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 5903257 INFO (jetty-launcher-20724-thread-3) [n:127.0.0.1:44398_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:44398_solr
[junit4] 2> 5903259 INFO (zkCallback-20740-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 5903260 INFO (jetty-launcher-20724-thread-3) [n:127.0.0.1:44398_solr ] o.a.s.p.PackageLoader /packages.json updated to version -1
[junit4] 2> 5903261 WARN (jetty-launcher-20724-thread-3) [n:127.0.0.1:44398_solr ] o.a.s.c.CoreContainer Not all security plugins configured! authentication=disabled authorization=disabled. Solr is only as secure as you make it. Consider configuring authentication/authorization before exposing Solr to users internal or external. See https://s.apache.org/solrsecurity for more info
[junit4] 2> 5903347 INFO (zkCallback-20752-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 5903444 INFO (jetty-launcher-20724-thread-3) [n:127.0.0.1:44398_solr ] o.a.s.h.a.MetricsHistoryHandler No .system collection, keeping metrics history in memory.
[junit4] 2> 5903576 INFO (jetty-launcher-20724-thread-3) [n:127.0.0.1:44398_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_44398.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@74c51b81
[junit4] 2> 5903618 INFO (jetty-launcher-20724-thread-3) [n:127.0.0.1:44398_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_44398.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@74c51b81
[junit4] 2> 5903618 INFO (jetty-launcher-20724-thread-3) [n:127.0.0.1:44398_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_44398.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@74c51b81
[junit4] 2> 5903632 INFO (jetty-launcher-20724-thread-3) [n:127.0.0.1:44398_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.RemoteQueryErrorTest_4E094E65EF364989-001/tempDir-001/node3/.
[junit4] 2> 5904719 INFO (jetty-launcher-20724-thread-1) [ ] o.a.s.h.c.HttpShardHandlerFactory Host whitelist initialized: WhitelistHostChecker [whitelistHosts=null, whitelistHostCheckingEnabled=true]
[junit4] 2> 5904720 WARN (jetty-launcher-20724-thread-1) [ ] o.e.j.u.s.S.config Trusting all certificates configured for Client@32e7ce77[provider=null,keyStore=null,trustStore=null]
[junit4] 2> 5904720 WARN (jetty-launcher-20724-thread-1) [ ] o.e.j.u.s.S.config No Client EndPointIdentificationAlgorithm configured for Client@32e7ce77[provider=null,keyStore=null,trustStore=null]
[junit4] 2> 5904723 WARN (jetty-launcher-20724-thread-1) [ ] o.e.j.u.s.S.config Trusting all certificates configured for Client@5f5b5fb[provider=null,keyStore=null,trustStore=null]
[junit4] 2> 5904724 WARN (jetty-launcher-20724-thread-1) [ ] o.e.j.u.s.S.config No Client EndPointIdentificationAlgorithm configured for Client@5f5b5fb[provider=null,keyStore=null,trustStore=null]
[junit4] 2> 5904725 INFO (jetty-launcher-20724-thread-1) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:43070/solr
[junit4] 2> 5904726 INFO (jetty-launcher-20724-thread-1) [ ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
[junit4] 2> 5904727 INFO (zkConnectionManagerCallback-20763-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 5904727 INFO (jetty-launcher-20724-thread-1) [ ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
[junit4] 2> 5904830 INFO (jetty-launcher-20724-thread-1) [n:127.0.0.1:33252_solr ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
[junit4] 2> 5904831 INFO (zkConnectionManagerCallback-20765-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 5904831 INFO (jetty-launcher-20724-thread-1) [n:127.0.0.1:33252_solr ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
[junit4] 2> 5904846 INFO (jetty-launcher-20724-thread-1) [n:127.0.0.1:33252_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
[junit4] 2> 5904851 INFO (jetty-launcher-20724-thread-1) [n:127.0.0.1:33252_solr ] o.a.s.c.ZkController Publish node=127.0.0.1:33252_solr as DOWN
[junit4] 2> 5904852 INFO (jetty-launcher-20724-thread-1) [n:127.0.0.1:33252_solr ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 5904852 INFO (jetty-launcher-20724-thread-1) [n:127.0.0.1:33252_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:33252_solr
[junit4] 2> 5904853 INFO (zkCallback-20752-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 5904853 INFO (zkCallback-20740-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 5904853 INFO (zkCallback-20764-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 5904854 INFO (jetty-launcher-20724-thread-1) [n:127.0.0.1:33252_solr ] o.a.s.p.PackageLoader /packages.json updated to version -1
[junit4] 2> 5904855 WARN (jetty-launcher-20724-thread-1) [n:127.0.0.1:33252_solr ] o.a.s.c.CoreContainer Not all security plugins configured! authentication=disabled authorization=disabled. Solr is only as secure as you make it. Consider configuring authentication/authorization before exposing Solr to users internal or external. See https://s.apache.org/solrsecurity for more info
[junit4] 2> 5904989 INFO (jetty-launcher-20724-thread-1) [n:127.0.0.1:33252_solr ] o.a.s.h.a.MetricsHistoryHandler No .system collection, keeping metrics history in memory.
[junit4] 2> 5905058 INFO (jetty-launcher-20724-thread-1) [n:127.0.0.1:33252_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_33252.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@74c51b81
[junit4] 2> 5905072 INFO (jetty-launcher-20724-thread-1) [n:127.0.0.1:33252_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_33252.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@74c51b81
[junit4] 2> 5905072 INFO (jetty-launcher-20724-thread-1) [n:127.0.0.1:33252_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_33252.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@74c51b81
[junit4] 2> 5905074 INFO (jetty-launcher-20724-thread-1) [n:127.0.0.1:33252_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.RemoteQueryErrorTest_4E094E65EF364989-001/tempDir-001/node1/.
[junit4] 2> 5905123 INFO (SUITE-RemoteQueryErrorTest-seed#[4E094E65EF364989]-worker) [ ] o.a.s.c.MiniSolrCloudCluster waitForAllNodes: numServers=3
[junit4] 2> 5905124 INFO (SUITE-RemoteQueryErrorTest-seed#[4E094E65EF364989]-worker) [ ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
[junit4] 2> 5905125 INFO (zkConnectionManagerCallback-20774-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 5905126 INFO (SUITE-RemoteQueryErrorTest-seed#[4E094E65EF364989]-worker) [ ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
[junit4] 2> 5905127 INFO (SUITE-RemoteQueryErrorTest-seed#[4E094E65EF364989]-worker) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (3)
[junit4] 2> 5905129 INFO (SUITE-RemoteQueryErrorTest-seed#[4E094E65EF364989]-worker) [ ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:43070/solr ready
[junit4] 2> 5905176 INFO (TEST-RemoteQueryErrorTest.test-seed#[4E094E65EF364989]) [ ] o.a.s.SolrTestCaseJ4 ###Starting test
[junit4] 2> 5905203 INFO (qtp1531180734-111674) [n:127.0.0.1:33252_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params collection.configName=conf&name=collection&nrtReplicas=1&action=CREATE&numShards=2&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 5905207 INFO (OverseerThreadFactory-14112-thread-1-processing-n:127.0.0.1:34518_solr) [n:127.0.0.1:34518_solr ] o.a.s.c.a.c.CreateCollectionCmd Create collection collection
[junit4] 2> 5905317 INFO (OverseerStateUpdate-72317128431894535-127.0.0.1:34518_solr-n_0000000000) [n:127.0.0.1:34518_solr ] o.a.s.c.o.SliceMutator createReplica() {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"collection",
[junit4] 2> "shard":"shard1",
[junit4] 2> "core":"collection_shard1_replica_n1",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"http://127.0.0.1:34518/solr",
[junit4] 2> "node_name":"127.0.0.1:34518_solr",
[junit4] 2> "type":"NRT",
[junit4] 2> "waitForFinalState":"false"}
[junit4] 2> 5905358 INFO (OverseerStateUpdate-72317128431894535-127.0.0.1:34518_solr-n_0000000000) [n:127.0.0.1:34518_solr ] o.a.s.c.o.SliceMutator createReplica() {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"collection",
[junit4] 2> "shard":"shard2",
[junit4] 2> "core":"collection_shard2_replica_n2",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"http://127.0.0.1:44398/solr",
[junit4] 2> "node_name":"127.0.0.1:44398_solr",
[junit4] 2> "type":"NRT",
[junit4] 2> "waitForFinalState":"false"}
[junit4] 2> 5905562 INFO (qtp102823917-111667) [n:127.0.0.1:34518_solr x:collection_shard1_replica_n1 ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node3&collection.configName=conf&newCollection=true&name=collection_shard1_replica_n1&action=CREATE&numShards=2&collection=collection&shard=shard1&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 5905562 INFO (qtp502090713-111675) [n:127.0.0.1:44398_solr x:collection_shard2_replica_n2 ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node4&collection.configName=conf&newCollection=true&name=collection_shard2_replica_n2&action=CREATE&numShards=2&collection=collection&shard=shard2&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 5905563 INFO (qtp102823917-111667) [n:127.0.0.1:34518_solr x:collection_shard1_replica_n1 ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 5905590 INFO (qtp502090713-111675) [n:127.0.0.1:44398_solr c:collection s:shard2 r:core_node4 x:collection_shard2_replica_n2 ] o.a.s.c.SolrConfig Using Lucene MatchVersion: 9.0.0
[junit4] 2> 5905629 INFO (qtp102823917-111667) [n:127.0.0.1:34518_solr c:collection s:shard1 r:core_node3 x:collection_shard1_replica_n1 ] o.a.s.c.SolrConfig Using Lucene MatchVersion: 9.0.0
[junit4] 2> 5905642 INFO (qtp502090713-111675) [n:127.0.0.1:44398_solr c:collection s:shard2 r:core_node4 x:collection_shard2_replica_n2 ] o.a.s.s.IndexSchema [collection_shard2_replica_n2] Schema name=minimal
[junit4] 2> 5905653 INFO (qtp502090713-111675) [n:127.0.0.1:44398_solr c:collection s:shard2 r:core_node4 x:collection_shard2_replica_n2 ] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
[junit4] 2> 5905653 INFO (qtp502090713-111675) [n:127.0.0.1:44398_solr c:collection s:shard2 r:core_node4 x:collection_shard2_replica_n2 ] o.a.s.c.CoreContainer Creating SolrCore 'collection_shard2_replica_n2' using configuration from configset conf, trusted=true
[junit4] 2> 5905653 INFO (qtp502090713-111675) [n:127.0.0.1:44398_solr c:collection s:shard2 r:core_node4 x:collection_shard2_replica_n2 ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_44398.solr.core.collection.shard2.replica_n2' (registry 'solr.core.collection.shard2.replica_n2') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@74c51b81
[junit4] 2> 5905654 INFO (qtp502090713-111675) [n:127.0.0.1:44398_solr c:collection s:shard2 r:core_node4 x:collection_shard2_replica_n2 ] o.a.s.c.SolrCore [[collection_shard2_replica_n2] ] Opening new SolrCore at [/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.RemoteQueryErrorTest_4E094E65EF364989-001/tempDir-001/node3/collection_shard2_replica_n2], dataDir=[/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.RemoteQueryErrorTest_4E094E65EF364989-001/tempDir-001/node3/./collection_shard2_replica_n2/data/]
[junit4] 2> 5905656 INFO (qtp102823917-111667) [n:127.0.0.1:34518_solr c:collection s:shard1 r:core_node3 x:collection_shard1_replica_n1 ] o.a.s.s.IndexSchema [collection_shard1_replica_n1] Schema name=minimal
[junit4] 2> 5905659 INFO (qtp102823917-111667) [n:127.0.0.1:34518_solr c:collection s:shard1 r:core_node3 x:collection_shard1_replica_n1 ] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
[junit4] 2> 5905659 INFO (qtp102823917-111667) [n:127.0.0.1:34518_solr c:collection s:shard1 r:core_node3 x:collection_shard1_replica_n1 ] o.a.s.c.CoreContainer Creating SolrCore 'collection_shard1_replica_n1' using configuration from configset conf, trusted=true
[junit4] 2> 5905659 INFO (qtp102823917-111667) [n:127.0.0.1:34518_solr c:collection s:shard1 r:core_node3 x:collection_shard1_replica_n1 ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_34518.solr.core.collection.shard1.replica_n1' (registry 'solr.core.collection.shard1.replica_n1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@74c51b81
[junit4] 2> 5905660 INFO (qtp102823917-111667) [n:127.0.0.1:34518_solr c:collection s:shard1 r:core_node3 x:collection_shard1_replica_n1 ] o.a.s.c.SolrCore [[collection_shard1_replica_n1] ] Opening new SolrCore at [/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.RemoteQueryErrorTest_4E094E65EF364989-001/tempDir-001/node2/collection_shard1_replica_n1], dataDir=[/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.RemoteQueryErrorTest_4E094E65EF364989-001/tempDir-001/node2/./collection_shard1_replica_n1/data/]
[junit4] 2> 5906009 INFO (qtp102823917-111667) [n:127.0.0.1:34518_solr c:collection s:shard1 r:core_node3 x:collection_shard1_replica_n1 ] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 5906009 INFO (qtp102823917-111667) [n:127.0.0.1:34518_solr c:collection s:shard1 r:core_node3 x:collection_shard1_replica_n1 ] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 5906010 INFO (qtp502090713-111675) [n:127.0.0.1:44398_solr c:collection s:shard2 r:core_node4 x:collection_shard2_replica_n2 ] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 5906010 INFO (qtp502090713-111675) [n:127.0.0.1:44398_solr c:collection s:shard2 r:core_node4 x:collection_shard2_replica_n2 ] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 5906013 INFO (qtp502090713-111675) [n:127.0.0.1:44398_solr c:collection s:shard2 r:core_node4 x:collection_shard2_replica_n2 ] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 5906013 INFO (qtp502090713-111675) [n:127.0.0.1:44398_solr c:collection s:shard2 r:core_node4 x:collection_shard2_replica_n2 ] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 5906028 INFO (qtp102823917-111667) [n:127.0.0.1:34518_solr c:collection s:shard1 r:core_node3 x:collection_shard1_replica_n1 ] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 5906028 INFO (qtp102823917-111667) [n:127.0.0.1:34518_solr c:collection s:shard1 r:core_node3 x:collection_shard1_replica_n1 ] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 5906029 INFO (qtp102823917-111667) [n:127.0.0.1:34518_solr c:collection s:shard1 r:core_node3 x:collection_shard1_replica_n1 ] o.a.s.s.SolrIndexSearcher Opening [Searcher@58978704[collection_shard1_replica_n1] main]
[junit4] 2> 5906039 INFO (qtp502090713-111675) [n:127.0.0.1:44398_solr c:collection s:shard2 r:core_node4 x:collection_shard2_replica_n2 ] o.a.s.s.SolrIndexSearcher Opening [Searcher@3d3e8efe[collection_shard2_replica_n2] main]
[junit4] 2> 5906041 INFO (qtp502090713-111675) [n:127.0.0.1:44398_solr c:collection s:shard2 r:core_node4 x:collection_shard2_replica_n2 ] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf
[junit4] 2> 5906041 INFO (qtp502090713-111675) [n:127.0.0.1:44398_solr c:collection s:shard2 r:core_node4 x:collection_shard2_replica_n2 ] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf
[junit4] 2> 5906042 INFO (qtp502090713-111675) [n:127.0.0.1:44398_solr c:collection s:shard2 r:core_node4 x:collection_shard2_replica_n2 ] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
[junit4] 2> 5906042 INFO (qtp502090713-111675) [n:127.0.0.1:44398_solr c:collection s:shard2 r:core_node4 x:collection_shard2_replica_n2 ] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1659040855798841344
[junit4] 2> 5906046 INFO (searcherExecutor-14131-thread-1-processing-n:127.0.0.1:44398_solr x:collection_shard2_replica_n2 c:collection s:shard2 r:core_node4) [n:127.0.0.1:44398_solr c:collection s:shard2 r:core_node4 x:collection_shard2_replica_n2 ] o.a.s.c.SolrCore [collection_shard2_replica_n2] Registered new searcher Searcher@3d3e8efe[collection_shard2_replica_n2] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 5906068 INFO (qtp102823917-111667) [n:127.0.0.1:34518_solr c:collection s:shard1 r:core_node3 x:collection_shard1_replica_n1 ] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf
[junit4] 2> 5906068 INFO (qtp102823917-111667) [n:127.0.0.1:34518_solr c:collection s:shard1 r:core_node3 x:collection_shard1_replica_n1 ] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf
[junit4] 2> 5906069 INFO (qtp102823917-111667) [n:127.0.0.1:34518_solr c:collection s:shard1 r:core_node3 x:collection_shard1_replica_n1 ] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
[junit4] 2> 5906069 INFO (qtp102823917-111667) [n:127.0.0.1:34518_solr c:collection s:shard1 r:core_node3 x:collection_shard1_replica_n1 ] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1659040855827152896
[junit4] 2> 5906069 INFO (qtp502090713-111675) [n:127.0.0.1:44398_solr c:collection s:shard2 r:core_node4 x:collection_shard2_replica_n2 ] o.a.s.c.ZkShardTerms Successful update of terms at /collections/collection/terms/shard2 to Terms{values={core_node4=0}, version=0}
[junit4] 2> 5906069 INFO (qtp502090713-111675) [n:127.0.0.1:44398_solr c:collection s:shard2 r:core_node4 x:collection_shard2_replica_n2 ] o.a.s.c.ShardLeaderElectionContextBase make sure parent is created /collections/collection/leaders/shard2
[junit4] 2> 5906072 INFO (searcherExecutor-14132-thread-1-processing-n:127.0.0.1:34518_solr x:collection_shard1_replica_n1 c:collection s:shard1 r:core_node3) [n:127.0.0.1:34518_solr c:collection s:shard1 r:core_node3 x:collection_shard1_replica_n1 ] o.a.s.c.SolrCore [collection_shard1_replica_n1] Registered new searcher Searcher@58978704[collection_shard1_replica_n1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 5906073 INFO (qtp502090713-111675) [n:127.0.0.1:44398_solr c:collection s:shard2 r:core_node4 x:collection_shard2_replica_n2 ] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
[junit4] 2> 5906073 INFO (qtp502090713-111675) [n:127.0.0.1:44398_solr c:collection s:shard2 r:core_node4 x:collection_shard2_replica_n2 ] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
[junit4] 2> 5906073 INFO (qtp502090713-111675) [n:127.0.0.1:44398_solr c:collection s:shard2 r:core_node4 x:collection_shard2_replica_n2 ] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:44398/solr/collection_shard2_replica_n2/
[junit4] 2> 5906074 INFO (qtp502090713-111675) [n:127.0.0.1:44398_solr c:collection s:shard2 r:core_node4 x:collection_shard2_replica_n2 ] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
[junit4] 2> 5906074 INFO (qtp502090713-111675) [n:127.0.0.1:44398_solr c:collection s:shard2 r:core_node4 x:collection_shard2_replica_n2 ] o.a.s.c.SyncStrategy http://127.0.0.1:44398/solr/collection_shard2_replica_n2/ has no replicas
[junit4] 2> 5906074 INFO (qtp502090713-111675) [n:127.0.0.1:44398_solr c:collection s:shard2 r:core_node4 x:collection_shard2_replica_n2 ] o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node /collections/collection/leaders/shard2/leader after winning as /collections/collection/leader_elect/shard2/election/72317128431894537-core_node4-n_0000000000
[junit4] 2> 5906091 INFO (qtp102823917-111667) [n:127.0.0.1:34518_solr c:collection s:shard1 r:core_node3 x:collection_shard1_replica_n1 ] o.a.s.c.ZkShardTerms Successful update of terms at /collections/collection/terms/shard1 to Terms{values={core_node3=0}, version=0}
[junit4] 2> 5906091 INFO (qtp102823917-111667) [n:127.0.0.1:34518_solr c:collection s:shard1 r:core_node3 x:collection_shard1_replica_n1 ] o.a.s.c.ShardLeaderElectionContextBase make sure parent is created /collections/collection/leaders/shard1
[junit4] 2> 5906092 INFO (qtp502090713-111675) [n:127.0.0.1:44398_solr c:collection s:shard2 r:core_node4 x:collection_shard2_replica_n2 ] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:44398/solr/collection_shard2_replica_n2/ shard2
[junit4] 2> 5906103 INFO (qtp102823917-111667) [n:127.0.0.1:34518_solr c:collection s:shard1 r:core_node3 x:collection_shard1_replica_n1 ] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
[junit4] 2> 5906103 INFO (qtp102823917-111667) [n:127.0.0.1:34518_solr c:collection s:shard1 r:core_node3 x:collection_shard1_replica_n1 ] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
[junit4] 2> 5906103 INFO (qtp102823917-111667) [n:127.0.0.1:34518_solr c:collection s:shard1 r:core_node3 x:collection_shard1_replica_n1 ] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:34518/solr/collection_shard1_replica_n1/
[junit4] 2> 5906104 INFO (qtp102823917-111667) [n:127.0.0.1:34518_solr c:collection s:shard1 r:core_node3 x:collection_shard1_replica_n1 ] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
[junit4] 2> 5906104 INFO (qtp102823917-111667) [n:127.0.0.1:34518_solr c:collection s:shard1 r:core_node3 x:collection_shard1_replica_n1 ] o.a.s.c.SyncStrategy http://127.0.0.1:34518/solr/collection_shard1_replica_n1/ has no replicas
[junit4] 2> 5906104 INFO (qtp102823917-111667) [n:127.0.0.1:34518_solr c:collection s:shard1 r:core_node3 x:collection_shard1_replica_n1 ] o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node /collections/collection/leaders/shard1/leader after winning as /collections/collection/leader_elect/shard1/election/72317128431894535-core_node3-n_0000000000
[junit4] 2> 5906106 INFO (qtp102823917-111667) [n:127.0.0.1:34518_solr c:collection s:shard1 r:core_node3 x:collection_shard1_replica_n1 ] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:34518/solr/collection_shard1_replica_n1/ shard1
[junit4] 2> 5906208 INFO (zkCallback-20740-thread-1) [ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection/state.json] for collection [collection] has occurred - updating... (live nodes size: [3])
[junit4] 2> 5906208 INFO (zkCallback-20752-thread-1) [ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection/state.json] for collection [collection] has occurred - updating... (live nodes size: [3])
[junit4] 2> 5906208 INFO (zkCallback-20740-thread-2) [ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection/state.json] for collection [collection] has occurred - updating... (live nodes size: [3])
[junit4] 2> 5906208 INFO (zkCallback-20752-thread-2) [ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection/state.json] for collection [collection] has occurred - updating... (live nodes size: [3])
[junit4] 2> 5906210 INFO (qtp102823917-111667) [n:127.0.0.1:34518_solr c:collection s:shard1 r:core_node3 x:collection_shard1_replica_n1 ] o.a.s.c.ZkController I am the leader, no recovery necessary
[junit4] 2> 5906211 INFO (qtp502090713-111675) [n:127.0.0.1:44398_solr c:collection s:shard2 r:core_node4 x:collection_shard2_replica_n2 ] o.a.s.c.ZkController I am the leader, no recovery necessary
[junit4] 2> 5906213 INFO (qtp102823917-111667) [n:127.0.0.1:34518_solr c:collection s:shard1 r:core_node3 x:collection_shard1_replica_n1 ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&coreNodeName=core_node3&collection.configName=conf&newCollection=true&name=collection_shard1_replica_n1&action=CREATE&numShards=2&collection=collection&shard=shard1&wt=javabin&version=2&replicaType=NRT} status=0 QTime=651
[junit4] 2> 5906215 INFO (qtp502090713-111675) [n:127.0.0.1:44398_solr c:collection s:shard2 r:core_node4 x:collection_shard2_replica_n2 ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&coreNodeName=core_node4&collection.configName=conf&newCollection=true&name=collection_shard2_replica_n2&action=CREATE&numShards=2&collection=collection&shard=shard2&wt=javabin&version=2&replicaType=NRT} status=0 QTime=653
[junit4] 2> 5906233 INFO (qtp1531180734-111674) [n:127.0.0.1:33252_solr ] o.a.s.h.a.CollectionsHandler Wait for new collection to be active for at most 45 seconds. Check all shard replicas
[junit4] 2> 5906315 INFO (zkCallback-20752-thread-2) [ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection/state.json] for collection [collection] has occurred - updating... (live nodes size: [3])
[junit4] 2> 5906315 INFO (zkCallback-20752-thread-1) [ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection/state.json] for collection [collection] has occurred - updating... (live nodes size: [3])
[junit4] 2> 5906315 INFO (zkCallback-20764-thread-1) [ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection/state.json] for collection [collection] has occurred - updating... (live nodes size: [3])
[junit4] 2> 5906315 INFO (zkCallback-20740-thread-1) [ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection/state.json] for collection [collection] has occurred - updating... (live nodes size: [3])
[junit4] 2> 5906315 INFO (zkCallback-20752-thread-3) [ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection/state.json] for collection [collection] has occurred - updating... (live nodes size: [3])
[junit4] 2> 5906315 INFO (zkCallback-20740-thread-2) [ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection/state.json] for collection [collection] has occurred - updating... (live nodes size: [3])
[junit4] 2> 5906315 INFO (zkCallback-20740-thread-3) [ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection/state.json] for collection [collection] has occurred - updating... (live nodes size: [3])
[junit4] 2> 5906316 INFO (qtp1531180734-111674) [n:127.0.0.1:33252_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={collection.configName=conf&name=collection&nrtReplicas=1&action=CREATE&numShards=2&wt=javabin&version=2} status=0 QTime=1113
[junit4] 2> 5906333 INFO (TEST-RemoteQueryErrorTest.test-seed#[4E094E65EF364989]) [ ] o.a.s.SolrTestCaseJ4 ###Ending test
[junit4] 2> NOTE: download the large Jenkins line-docs file by running 'ant get-jenkins-line-docs' in the lucene directory.
[junit4] 2> NOTE: reproduce with: ant test -Dtestcase=RemoteQueryErrorTest -Dtests.method=test -Dtests.seed=4E094E65EF364989 -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/test-data/enwiki.random.lines.txt -Dtests.locale=en-PK -Dtests.timezone=Pacific/Bougainville -Dtests.asserts=true -Dtests.file.encoding=UTF-8
[junit4] FAILURE 1.22s J1 | RemoteQueryErrorTest.test <<<
[junit4] > Throwable #1: java.lang.AssertionError:
[junit4] > Expected: a string containing "Document is missing mandatory uniqueKey field: id"
[junit4] > but: was "Error from server at http://127.0.0.1:34518/solr: No active replicas found for collection: collection"
[junit4] > at __randomizedtesting.SeedInfo.seed([4E094E65EF364989:C65D71BF41CA2471]:0)
[junit4] > at org.hamcrest.MatcherAssert.assertThat(MatcherAssert.java:20)
[junit4] > at org.apache.solr.cloud.RemoteQueryErrorTest.test(RemoteQueryErrorTest.java:53)
[junit4] > at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
[junit4] > at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
[junit4] > at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
[junit4] > at java.base/java.lang.reflect.Method.invoke(Method.java:566)
[junit4] > at java.base/java.lang.Thread.run(Thread.java:834)
[junit4] 2> 5906453 INFO (jetty-closer-20777-thread-1) [ ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=568958718
[junit4] 2> 5906453 INFO (jetty-closer-20777-thread-1) [ ] o.a.s.c.ZkController Remove node as live in ZooKeeper:/live_nodes/127.0.0.1:34518_solr
[junit4] 2> 5906454 INFO (jetty-closer-20777-thread-2) [ ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=336465441
[junit4] 2> 5906454 INFO (jetty-closer-20777-thread-2) [ ] o.a.s.c.ZkController Remove node as live in ZooKeeper:/live_nodes/127.0.0.1:44398_solr
[junit4] 2> 5906454 INFO (jetty-closer-20777-thread-1) [ ] o.a.s.c.ZkController Publish this node as DOWN...
[junit4] 2> 5906454 INFO (jetty-closer-20777-thread-1) [ ] o.a.s.c.ZkController Publish node=127.0.0.1:34518_solr as DOWN
[junit4] 2> 5906454 INFO (jetty-closer-20777-thread-2) [ ] o.a.s.c.ZkController Publish this node as DOWN...
[junit4] 2> 5906454 INFO (jetty-closer-20777-thread-2) [ ] o.a.s.c.ZkController Publish node=127.0.0.1:44398_solr as DOWN
[junit4] 2> 5906456 INFO (coreCloseExecutor-14142-thread-1) [n:127.0.0.1:44398_solr c:collection s:shard2 r:core_node4 x:collection_shard2_replica_n2 ] o.a.s.c.SolrCore [collection_shard2_replica_n2] CLOSING SolrCore org.apache.solr.core.SolrCore@a52692c
[junit4] 2> 5906456 INFO (coreCloseExecutor-14142-thread-1) [n:127.0.0.1:44398_solr c:collection s:shard2 r:core_node4 x:collection_shard2_replica_n2 ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.collection.shard2.replica_n2, tag=SolrCore@a52692c
[junit4] 2> 5906456 INFO (coreCloseExecutor-14142-thread-1) [n:127.0.0.1:44398_solr c:collection s:shard2 r:core_node4 x:collection_shard2_replica_n2 ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@466da6b4: rootName = solr_44398, domain = solr.core.collection.shard2.replica_n2, service url = null, agent id = null] for registry solr.core.collection.shard2.replica_n2 / com.codahale.metrics.MetricRegistry@4159c306
[junit4] 2> 5906457 INFO (coreCloseExecutor-14141-thread-1) [n:127.0.0.1:34518_solr c:collection s:shard1 r:core_node3 x:collection_shard1_replica_n1 ] o.a.s.c.SolrCore [collection_shard1_replica_n1] CLOSING SolrCore org.apache.solr.core.SolrCore@63aa86cf
[junit4] 2> 5906457 INFO (coreCloseExecutor-14141-thread-1) [n:127.0.0.1:34518_solr c:collection s:shard1 r:core_node3 x:collection_shard1_replica_n1 ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.collection.shard1.replica_n1, tag=SolrCore@63aa86cf
[junit4] 2> 5906457 INFO (coreCloseExecutor-14141-thread-1) [n:127.0.0.1:34518_solr c:collection s:shard1 r:core_node3 x:collection_shard1_replica_n1 ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@d809492: rootName = solr_34518, domain = solr.core.collection.shard1.replica_n1, service url = null, agent id = null] for registry solr.core.collection.shard1.replica_n1 / com.codahale.metrics.MetricRegistry@3e671159
[junit4] 2> 5906459 INFO (jetty-closer-20777-thread-3) [ ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=1017680840
[junit4] 2> 5906459 INFO (jetty-closer-20777-thread-3) [ ] o.a.s.c.ZkController Remove node as live in ZooKeeper:/live_nodes/127.0.0.1:33252_solr
[junit4] 2> 5906459 INFO (jetty-closer-20777-thread-3) [ ] o.a.s.c.ZkController Publish this node as DOWN...
[junit4] 2> 5906459 INFO (jetty-closer-20777-thread-3) [ ] o.a.s.c.ZkController Publish node=127.0.0.1:33252_solr as DOWN
[junit4] 2> 5906651 INFO (zkCallback-20752-thread-3) [ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection/state.json] for collection [collection] has occurred - updating... (live nodes size: [3])
[junit4] 2> 5906651 INFO (zkCallback-20752-thread-1) [ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection/state.json] for collection [collection] has occurred - updating... (live nodes size: [3])
[junit4] 2> 5906651 INFO (zkCallback-20752-thread-2) [ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection/state.json] for collection [collection] has occurred - updating... (live nodes size: [3])
[junit4] 2> 5906668 INFO (zkCallback-20740-thread-3) [ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection/state.json] for collection [collection] has occurred - updating... (live nodes size: [3])
[junit4] 2> 5906668 INFO (zkCallback-20740-thread-1) [ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection/state.json] for collection [collection] has occurred - updating... (live nodes size: [3])
[junit4] 2> 5906670 INFO (zkCallback-20740-thread-4) [ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection/state.json] for collection [collection] has occurred - updating... (live nodes size: [3])
[junit4] 2> 5906744 INFO (jetty-closer-20777-thread-3) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.node, tag=null
[junit4] 2> 5906744 INFO (jetty-closer-20777-thread-3) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@483df448: rootName = solr_33252, domain = solr.node, service url = null, agent id = null] for registry solr.node / com.codahale.metrics.MetricRegistry@46ce6ea
[junit4] 2> 5906745 INFO (coreCloseExecutor-14141-thread-1) [n:127.0.0.1:34518_solr c:collection s:shard1 r:core_node3 x:collection_shard1_replica_n1 ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.collection.shard1.leader, tag=SolrCore@63aa86cf
[junit4] 2> 5906777 INFO (coreCloseExecutor-14141-thread-1) [n:127.0.0.1:34518_solr c:collection s:shard1 r:core_node3 x:collection_shard1_replica_n1 ] o.a.s.u.DirectUpdateHandler2 Committing on IndexWriter.close() ... SKIPPED (unnecessary).
[junit4] 2> 5906784 INFO (coreCloseExecutor-14142-thread-1) [n:127.0.0.1:44398_solr c:collection s:shard2 r:core_node4 x:collection_shard2_replica_n2 ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.collection.shard2.leader, tag=SolrCore@a52692c
[junit4] 2> 5906806 INFO (jetty-closer-20777-thread-1) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.node, tag=null
[junit4] 2> 5906806 INFO (jetty-closer-20777-thread-1) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@3ed4c4e1: rootName = solr_34518, domain = solr.node, service url = null, agent id = null] for registry solr.node / com.codahale.metrics.MetricRegistry@15352b3d
[junit4] 2> 5906824 INFO (jetty-closer-20777-thread-3) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jvm, tag=null
[junit4] 2> 5906825 INFO (jetty-closer-20777-thread-3) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@f6537c2: rootName = solr_33252, domain = solr.jvm, service url = null, agent id = null] for registry solr.jvm / com.codahale.metrics.MetricRegistry@5111173e
[junit4] 2> 5906825 INFO (coreCloseExecutor-14142-thread-1) [n:127.0.0.1:44398_solr c:collection s:shard2 r:core_node4 x:collection_shard2_replica_n2 ] o.a.s.u.DirectUpdateHandler2 Committing on IndexWriter.close() ... SKIPPED (unnecessary).
[junit4] 2> 5906832 INFO (jetty-closer-20777-thread-1) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jvm, tag=null
[junit4] 2> 5906833 INFO (jetty-closer-20777-thread-1) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@1adc00e1: rootName = solr_34518, domain = solr.jvm, service url = null, agent id = null] for registry solr.jvm / com.codahale.metrics.MetricRegistry@5111173e
[junit4] 2> 5906845 INFO (jetty-closer-20777-thread-3) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jetty, tag=null
[junit4] 2> 5906845 INFO (jetty-closer-20777-thread-3) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@11fb85a9: rootName = solr_33252, domain = solr.jetty, service url = null, agent id = null] for registry solr.jetty / com.codahale.metrics.MetricRegistry@1bd00ff6
[junit4] 2> 5906845 INFO (jetty-closer-20777-thread-3) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.cluster, tag=null
[junit4] 2> 5906853 INFO (jetty-closer-20777-thread-1) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jetty, tag=null
[junit4] 2> 5906853 INFO (jetty-closer-20777-thread-1) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@4f022e8c: rootName = solr_34518, domain = solr.jetty, service url = null, agent id = null] for registry solr.jetty / com.codahale.metrics.MetricRegistry@1bd00ff6
[junit4] 2> 5906853 INFO (jetty-closer-20777-thread-1) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.cluster, tag=null
[junit4] 2> 5906888 INFO (closeThreadPool-20787-thread-1) [ ] o.a.s.c.Overseer Overseer (id=72317128431894535-127.0.0.1:34518_solr-n_0000000000) closing
[junit4] 2> 5906899 INFO (OverseerAutoScalingTriggerThread-72317128431894535-127.0.0.1:34518_solr-n_0000000000) [ ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread woken up but we are closed, exiting.
[junit4] 2> 5906911 INFO (OverseerStateUpdate-72317128431894535-127.0.0.1:34518_solr-n_0000000000) [n:127.0.0.1:34518_solr ] o.a.s.c.Overseer Overseer Loop exiting : 127.0.0.1:34518_solr
[junit4] 2> 5906942 INFO (zkCallback-20764-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (2)
[junit4] 2> 5906942 INFO (zkCallback-20752-thread-2) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (2)
[junit4] 2> 5906943 INFO (jetty-closer-20777-thread-2) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.node, tag=null
[junit4] 2> 5906943 INFO (jetty-closer-20777-thread-2) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@6818dd7f: rootName = solr_44398, domain = solr.node, service url = null, agent id = null] for registry solr.node / com.codahale.metrics.MetricRegistry@2d403c71
[junit4] 2> 5906973 INFO (closeThreadPool-20787-thread-2) [ ] o.a.s.c.Overseer Overseer (id=72317128431894535-127.0.0.1:34518_solr-n_0000000000) closing
[junit4] 2> 5906995 INFO (jetty-closer-20777-thread-2) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jvm, tag=null
[junit4] 2> 5906995 INFO (jetty-closer-20777-thread-2) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@519bc49d: rootName = solr_44398, domain = solr.jvm, service url = null, agent id = null] for registry solr.jvm / com.codahale.metrics.MetricRegistry@5111173e
[junit4] 2> 5907005 INFO (zkCallback-20752-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (1)
[junit4] 2> 5907015 INFO (jetty-closer-20777-thread-2) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jetty, tag=null
[junit4] 2> 5907015 INFO (jetty-closer-20777-thread-2) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@3e2b6976: rootName = solr_44398, domain = solr.jetty, service url = null, agent id = null] for registry solr.jetty / com.codahale.metrics.MetricRegistry@1bd00ff6
[junit4] 2> 5907015 INFO (jetty-closer-20777-thread-2) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.cluster, tag=null
[junit4] 2> 5907049 INFO (jetty-closer-20777-thread-1) [ ] o.a.s.c.Overseer Overseer (id=72317128431894535-127.0.0.1:34518_solr-n_0000000000) closing
[junit4] 2> 5907139 INFO (jetty-closer-20777-thread-3) [ ] o.e.j.s.AbstractConnector Stopped ServerConnector@558413a6{HTTP/1.1,[http/1.1, h2c]}{127.0.0.1:0}
[junit4] 2> 5907139 INFO (jetty-closer-20777-thread-3) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@754185a1{/solr,null,UNAVAILABLE}
[junit4] 2> 5907139 INFO (jetty-closer-20777-thread-3) [ ] o.e.j.s.session node0 Stopped scavenging
[junit4] 2> 5907140 INFO (jetty-closer-20777-thread-1) [ ] o.e.j.s.AbstractConnector Stopped ServerConnector@8e71959{HTTP/1.1,[http/1.1, h2c]}{127.0.0.1:0}
[junit4] 2> 5907140 INFO (jetty-closer-20777-thread-1) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@c49d1de{/solr,null,UNAVAILABLE}
[junit4] 2> 5907141 INFO (jetty-closer-20777-thread-1) [ ] o.e.j.s.session node0 Stopped scavenging
[junit4] 2> 5907707 INFO (jetty-closer-20777-thread-2) [ ] o.e.j.s.AbstractConnector Stopped ServerConnector@132aa614{HTTP/1.1,[http/1.1, h2c]}{127.0.0.1:0}
[junit4] 2> 5907707 INFO (jetty-closer-20777-thread-2) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@7681c073{/solr,null,UNAVAILABLE}
[junit4] 2> 5907752 INFO (jetty-closer-20777-thread-2) [ ] o.e.j.s.session node0 Stopped scavenging
[junit4] 2> 5907788 INFO (SUITE-RemoteQueryErrorTest-seed#[4E094E65EF364989]-worker) [ ] o.a.s.c.ZkTestServer Shutting down ZkTestServer.
[junit4] 2> 5908094 WARN (ZkTestServer Run Thread) [ ] o.a.s.c.ZkTestServer Watch limit violations:
[junit4] 2> Maximum concurrent create/delete watches above limit:
[junit4] 2>
[junit4] 2> 4 /solr/aliases.json
[junit4] 2> 4 /solr/clusterprops.json
[junit4] 2> 3 /solr/packages.json
[junit4] 2> 3 /solr/security.json
[junit4] 2> 2 /solr/collections/collection/terms/shard1
[junit4] 2> 2 /solr/collections/collection/terms/shard2
[junit4] 2> 2 /solr/configs/conf
[junit4] 2>
[junit4] 2> Maximum concurrent data watches above limit:
[junit4] 2>
[junit4] 2> 26 /solr/collections/collection/state.json
[junit4] 2> 4 /solr/clusterstate.json
[junit4] 2>
[junit4] 2> Maximum concurrent children watches above limit:
[junit4] 2>
[junit4] 2> 21 /solr/overseer/queue
[junit4] 2> 13 /solr/live_nodes
[junit4] 2> 8 /solr/collections
[junit4] 2> 4 /solr/overseer/collection-queue-work
[junit4] 2>
[junit4] 2> 5908096 INFO (SUITE-RemoteQueryErrorTest-seed#[4E094E65EF364989]-worker) [ ] o.a.s.c.ZkTestServer waitForServerDown: 127.0.0.1:43070
[junit4] 2> 5908096 INFO (SUITE-RemoteQueryErrorTest-seed#[4E094E65EF364989]-worker) [ ] o.a.s.c.ZkTestServer parse host and port list: 127.0.0.1:43070
[junit4] 2> 5908096 INFO (SUITE-RemoteQueryErrorTest-seed#[4E094E65EF364989]-worker) [ ] o.a.s.c.ZkTestServer connecting to 127.0.0.1 43070
[junit4] 2> NOTE: leaving temporary files on disk at: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.RemoteQueryErrorTest_4E094E65EF364989-001
[junit4] 2> Feb 20, 2020 7:44:16 AM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
[junit4] 2> WARNING: Will linger awaiting termination of 1 leaked thread(s).
[junit4] 2> NOTE: test params are: codec=FastCompressingStoredFieldsData(storedFieldsFormat=CompressingStoredFieldsFormat(compressionMode=FAST, chunkSize=7, maxDocsPerChunk=577, blockShift=8), termVectorsFormat=CompressingTermVectorsFormat(compressionMode=FAST, chunkSize=7, blockSize=8)), sim=Asserting(org.apache.lucene.search.similarities.AssertingSimilarity@76b43195), locale=en-PK, timezone=Pacific/Bougainville
[junit4] 2> NOTE: Linux 4.4.0-170-generic amd64/Oracle Corporation 11.0.4 (64-bit)/cpus=8,threads=1,free=365625032,total=536870912
[junit4] 2> NOTE: All tests run in this JVM: [AnalyticsQueryTest, TestHighlightDedupGrouping, PeerSyncWithLeaderAndIndexFingerprintCachingTest, TestSchemaField, JsonLoaderTest, DeleteReplicaTest, TestFieldCacheWithThreads, TestManagedStopFilterFactory, HdfsThreadLeakTest, LukeRequestHandlerTest, BigEndianAscendingWordDeserializerTest, LeaderTragicEventTest, CloudExitableDirectoryReaderTest, CollectionsAPISolrJTest, TestBlendedInfixSuggestions, IndexSizeTriggerSizeEstimationTest, TestPullReplica, HdfsAutoAddReplicasIntegrationTest, TestExactSharedStatsCache, TestSchemaVersionResource, TemplateUpdateProcessorTest, AddReplicaTest, TestFaceting, TestPayloadScoreQParserPlugin, TestShortCircuitedRequests, AlternateDirectoryTest, TestAuthenticationFramework, TestJettySolrRunner, TestCoreDiscovery, TestSQLHandler, CdcrOpsAndBoundariesTest, MetricTriggerTest, SuggesterWFSTTest, TestWithCollection, UninvertDocValuesMergePolicyTest, RAMDirectoryFactoryTest, TestMacroExpander, TestSimClusterStateProvider, TestQuerySenderNoQuery, SpatialFilterTest, DocValuesMultiTest, TestPerFieldSimilarity, TriggerEventQueueTest, HdfsDirectoryTest, TestSolrQueryResponse, TestDFRSimilarityFactory, SolrCLIZkUtilsTest, ByteBuffersDirectoryFactoryTest, SystemLogListenerTest, TestCloudNestedDocsSort, SignatureUpdateProcessorFactoryTest, AuditLoggerIntegrationTest, TestUseDocValuesAsStored2, TestLegacyField, TestDistributedSearch, TestHdfsBackupRestoreCore, TestNumericRangeQuery64, XsltUpdateRequestHandlerTest, JWTAuthPluginTest, TestSolrTestCaseJ4, TestJsonFacetsWithNestedObjects, TestQueryUtils, ConnectionManagerTest, HdfsCollectionsAPIDistributedZkTest, TestFuzzyAnalyzedSuggestions, TestTrieFacet, TestConfigSetsAPI, TestRequestStatusCollectionAPI, TestCollapseQParserPlugin, UtilsToolTest, MetricsHistoryIntegrationTest, ShardsWhitelistTest, SpatialHeatmapFacetsTest, TestNumericTerms64, CloneFieldUpdateProcessorFactoryTest, OverriddenZkACLAndCredentialsProvidersTest, TestCloudSchemaless, DebugComponentTest, TestScoreJoinQPScore, ReturnFieldsTest, DistributedFacetPivotSmallAdvancedTest, NestedShardedAtomicUpdateTest, TestElisionMultitermQuery, TestCSVLoader, TestReloadDeadlock, TestSimTriggerIntegration, ZkCollectionPropsCachingTest, TestSolrDeletionPolicy2, TestRandomFlRTGCloud, TestExceedMaxTermLength, ChaosMonkeySafeLeaderWithPullReplicasTest, TestNoOpRegenerator, TestDirectoryFactory, FuzzySearchTest, CoreAdminCreateDiscoverTest, BasicFunctionalityTest, ResponseBuilderTest, SuggesterFSTTest, ResolveAnalyzerByNameTest, TestTlogReplayVsRecovery, BasicDistributedZk2Test, TestRangeQuery, TestSolr4Spatial2, VersionInfoTest, TestTolerantUpdateProcessorRandomCloud, SpellCheckCollatorTest, DeleteInactiveReplicaTest, CloudReplicaSourceTest, TestSubQueryTransformerDistrib, ScheduledTriggerTest, TestXmlQParserPlugin, V2ApiIntegrationTest, DistributedQueryElevationComponentTest, HLLUtilTest, AnalysisErrorHandlingTest, TestJsonFacetRefinement, TestGraphTermsQParserPlugin, TestSimpleQParserPlugin, DirectSolrConnectionTest, CoreSorterTest, TaggerTest, TestDeprecatedFilters, MinimalSchemaTest, TestSimpleTextCodec, SimpleFacetsTest, ClusterStateUpdateTest, CdcrVersionReplicationTest, LeaderElectionTest, TestHalfAndHalfDocValues, TestSolrCloudWithHadoopAuthPlugin, RegexBoostProcessorTest, TestCoreAdminApis, BadCopyFieldTest, RequestLoggingTest, TestCryptoKeys, FastVectorHighlighterTest, TestStressRecovery, CollectionTooManyReplicasTest, RecoveryZkTest, TestStressVersions, TestSolrCoreSnapshots, OpenExchangeRatesOrgProviderTest, TestRecovery, TestSolrConfigHandlerConcurrent, TestLeaderElectionZkExpiry, CursorPagingTest, DisMaxRequestHandlerTest, DistributedIntervalFacetingTest, EchoParamsTest, SolrInfoBeanTest, SolrTestCaseJ4Test, TestCursorMarkWithoutUniqueKey, TestDistributedGrouping, TestRandomDVFaceting, TestRandomFaceting, TestSimpleTrackingShardHandler, PathHierarchyTokenizerFactoryTest, ProtectedTermFilterFactoryTest, TestCharFilters, TestReversedWildcardFilterFactory, TestWordDelimiterFilterFactory, TokenizerChainTest, TestLuceneIndexBackCompat, TestEmbeddedSolrServerAdminHandler, TestEmbeddedSolrServerConstructors, TestEmbeddedSolrServerSchemaAPI, ActionThrottleTest, AssignBackwardCompatibilityTest, BasicZkTest, ChaosMonkeyNothingIsSafeTest, ChaosMonkeyNothingIsSafeWithPullReplicasTest, ChaosMonkeySafeLeaderTest, ClusterStateTest, CollectionStateFormat2Test, ConfigSetsAPITest, DeleteStatusTest, DistribCursorPagingTest, DistribDocExpirationUpdateProcessorTest, DistribJoinFromCollectionTest, DistributedQueueTest, DistributedVersionInfoTest, ForceLeaderTest, ForceLeaderWithTlogReplicasTest, HealthCheckHandlerTest, HttpPartitionOnCommitTest, HttpPartitionTest, HttpPartitionWithTlogReplicasTest, MoveReplicaTest, MultiSolrCloudTestCaseTest, OutOfBoxZkACLAndCredentialsProvidersTest, OverseerRolesTest, OverseerTest, RemoteQueryErrorTest]
[junit4] Completed [505/904 (1!)] on J1 in 11.69s, 1 test, 1 failure <<< FAILURES!
[...truncated 10937 lines...]
BUILD FAILED
/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/build.xml:653: The following error occurred while executing this line:
/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/build.xml:587: Some of the tests produced a heap dump, but did not fail. Maybe a suppressed OutOfMemoryError? Dumps created:
* java_pid1913.hprof
Total time: 454 minutes 9 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any