You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by Policeman Jenkins Server <je...@thetaphi.de> on 2017/12/31 10:23:46 UTC
[JENKINS] Lucene-Solr-7.x-Linux (64bit/jdk-9.0.1) - Build # 1087 -
Failure!
Build: https://jenkins.thetaphi.de/job/Lucene-Solr-7.x-Linux/1087/
Java: 64bit/jdk-9.0.1 -XX:-UseCompressedOops -XX:+UseParallelGC
2 tests failed.
FAILED: org.apache.solr.cloud.AssignBackwardCompatibilityTest.test
Error Message:
Expected 4 active replicas null Live Nodes: [127.0.0.1:33887_solr, 127.0.0.1:36055_solr, 127.0.0.1:42197_solr, 127.0.0.1:44111_solr] Last available state: DocCollection(collection1//collections/collection1/state.json/7)={ "pullReplicas":"0", "replicationFactor":"4", "shards":{"shard1":{ "range":"80000000-7fffffff", "state":"active", "replicas":{ "core_node2":{ "core":"collection1_shard1_replica_n1", "base_url":"http://127.0.0.1:33887/solr", "node_name":"127.0.0.1:33887_solr", "state":"active", "type":"NRT", "leader":"true"}, "core_node4":{ "core":"collection1_shard1_replica_n3", "base_url":"http://127.0.0.1:44111/solr", "node_name":"127.0.0.1:44111_solr", "state":"active", "type":"NRT"}, "core_node8":{ "core":"collection1_shard1_replica_n7", "base_url":"http://127.0.0.1:42197/solr", "node_name":"127.0.0.1:42197_solr", "state":"active", "type":"NRT"}, "core_node10":{ "core":"collection1_shard1_replica_n9", "base_url":"http://127.0.0.1:36055/solr", "state":"down", "node_name":"127.0.0.1:36055_solr", "type":"NRT"}}}}, "router":{"name":"compositeId"}, "maxShardsPerNode":"1000", "autoAddReplicas":"false", "nrtReplicas":"4", "tlogReplicas":"0"}
Stack Trace:
java.lang.AssertionError: Expected 4 active replicas
null
Live Nodes: [127.0.0.1:33887_solr, 127.0.0.1:36055_solr, 127.0.0.1:42197_solr, 127.0.0.1:44111_solr]
Last available state: DocCollection(collection1//collections/collection1/state.json/7)={
"pullReplicas":"0",
"replicationFactor":"4",
"shards":{"shard1":{
"range":"80000000-7fffffff",
"state":"active",
"replicas":{
"core_node2":{
"core":"collection1_shard1_replica_n1",
"base_url":"http://127.0.0.1:33887/solr",
"node_name":"127.0.0.1:33887_solr",
"state":"active",
"type":"NRT",
"leader":"true"},
"core_node4":{
"core":"collection1_shard1_replica_n3",
"base_url":"http://127.0.0.1:44111/solr",
"node_name":"127.0.0.1:44111_solr",
"state":"active",
"type":"NRT"},
"core_node8":{
"core":"collection1_shard1_replica_n7",
"base_url":"http://127.0.0.1:42197/solr",
"node_name":"127.0.0.1:42197_solr",
"state":"active",
"type":"NRT"},
"core_node10":{
"core":"collection1_shard1_replica_n9",
"base_url":"http://127.0.0.1:36055/solr",
"state":"down",
"node_name":"127.0.0.1:36055_solr",
"type":"NRT"}}}},
"router":{"name":"compositeId"},
"maxShardsPerNode":"1000",
"autoAddReplicas":"false",
"nrtReplicas":"4",
"tlogReplicas":"0"}
at __randomizedtesting.SeedInfo.seed([23F1118A31631A7F:ABA52E509F9F7787]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.apache.solr.cloud.SolrCloudTestCase.waitForState(SolrCloudTestCase.java:269)
at org.apache.solr.cloud.AssignBackwardCompatibilityTest.test(AssignBackwardCompatibilityTest.java:92)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:564)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.base/java.lang.Thread.run(Thread.java:844)
FAILED: org.apache.solr.cloud.TestHdfsCloudBackupRestore.test
Error Message:
Stack Trace:
java.lang.AssertionError
at __randomizedtesting.SeedInfo.seed([23F1118A31631A7F:ABA52E509F9F7787]:0)
at org.junit.Assert.fail(Assert.java:92)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.junit.Assert.assertTrue(Assert.java:54)
at org.apache.solr.cloud.AbstractCloudBackupRestoreTestCase.test(AbstractCloudBackupRestoreTestCase.java:133)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:564)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.base/java.lang.Thread.run(Thread.java:844)
Build Log:
[...truncated 11883 lines...]
[junit4] Suite: org.apache.solr.cloud.TestHdfsCloudBackupRestore
[junit4] 2> 202989 INFO (SUITE-TestHdfsCloudBackupRestore-seed#[23F1118A31631A7F]-worker) [ ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
[junit4] 2> Creating dataDir: /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J2/temp/solr.cloud.TestHdfsCloudBackupRestore_23F1118A31631A7F-001/init-core-data-001
[junit4] 2> 202989 WARN (SUITE-TestHdfsCloudBackupRestore-seed#[23F1118A31631A7F]-worker) [ ] o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=32 numCloses=32
[junit4] 2> 202989 INFO (SUITE-TestHdfsCloudBackupRestore-seed#[23F1118A31631A7F]-worker) [ ] o.a.s.SolrTestCaseJ4 Using PointFields (NUMERIC_POINTS_SYSPROP=true) w/NUMERIC_DOCVALUES_SYSPROP=true
[junit4] 2> 202990 INFO (SUITE-TestHdfsCloudBackupRestore-seed#[23F1118A31631A7F]-worker) [ ] o.a.s.SolrTestCaseJ4 Randomized ssl (true) and clientAuth (false) via: @org.apache.solr.util.RandomizeSSL(reason="", value=0.0/0.0, ssl=0.0/0.0, clientAuth=0.0/0.0)
[junit4] 2> 203309 WARN (SUITE-TestHdfsCloudBackupRestore-seed#[23F1118A31631A7F]-worker) [ ] o.a.h.u.NativeCodeLoader Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
[junit4] 1> Formatting using clusterid: testClusterID
[junit4] 2> 203625 WARN (SUITE-TestHdfsCloudBackupRestore-seed#[23F1118A31631A7F]-worker) [ ] o.a.h.m.i.MetricsConfig Cannot locate configuration: tried hadoop-metrics2-namenode.properties,hadoop-metrics2.properties
[junit4] 2> 203722 INFO (SUITE-TestHdfsCloudBackupRestore-seed#[23F1118A31631A7F]-worker) [ ] o.m.log Logging to org.slf4j.impl.Log4jLoggerAdapter(org.mortbay.log) via org.mortbay.log.Slf4jLog
[junit4] 2> 203734 WARN (SUITE-TestHdfsCloudBackupRestore-seed#[23F1118A31631A7F]-worker) [ ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
[junit4] 2> 203858 INFO (SUITE-TestHdfsCloudBackupRestore-seed#[23F1118A31631A7F]-worker) [ ] o.m.log jetty-6.1.x
[junit4] 2> 203882 INFO (SUITE-TestHdfsCloudBackupRestore-seed#[23F1118A31631A7F]-worker) [ ] o.m.log Extract jar:file:/home/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.7.4-tests.jar!/webapps/hdfs to ./temp/Jetty_localhost_localdomain_40517_hdfs____.ut03yf/webapp
[junit4] 2> 203996 INFO (SUITE-TestHdfsCloudBackupRestore-seed#[23F1118A31631A7F]-worker) [ ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost.localdomain:40517
[junit4] 2> 204334 WARN (SUITE-TestHdfsCloudBackupRestore-seed#[23F1118A31631A7F]-worker) [ ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
[junit4] 2> 204336 INFO (SUITE-TestHdfsCloudBackupRestore-seed#[23F1118A31631A7F]-worker) [ ] o.m.log jetty-6.1.x
[junit4] 2> 204340 INFO (SUITE-TestHdfsCloudBackupRestore-seed#[23F1118A31631A7F]-worker) [ ] o.m.log Extract jar:file:/home/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.7.4-tests.jar!/webapps/datanode to ./temp/Jetty_localhost_34583_datanode____785ee/webapp
[junit4] 2> 204405 INFO (SUITE-TestHdfsCloudBackupRestore-seed#[23F1118A31631A7F]-worker) [ ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:34583
[junit4] 2> 204416 INFO (SUITE-TestHdfsCloudBackupRestore-seed#[23F1118A31631A7F]-worker) [ ] i.n.u.i.PlatformDependent Your platform does not provide complete low-level API for accessing direct buffers reliably. Unless explicitly requested, heap buffer will always be preferred to avoid potential system unstability.
[junit4] 2> 204540 WARN (SUITE-TestHdfsCloudBackupRestore-seed#[23F1118A31631A7F]-worker) [ ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
[junit4] 2> 204541 INFO (SUITE-TestHdfsCloudBackupRestore-seed#[23F1118A31631A7F]-worker) [ ] o.m.log jetty-6.1.x
[junit4] 2> 204546 INFO (SUITE-TestHdfsCloudBackupRestore-seed#[23F1118A31631A7F]-worker) [ ] o.m.log Extract jar:file:/home/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.7.4-tests.jar!/webapps/datanode to ./temp/Jetty_localhost_35709_datanode____8d4ukl/webapp
[junit4] 2> 204622 INFO (SUITE-TestHdfsCloudBackupRestore-seed#[23F1118A31631A7F]-worker) [ ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:35709
[junit4] 2> 204870 ERROR (DataNode: [[[DISK]file:/home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J2/temp/solr.cloud.TestHdfsCloudBackupRestore_23F1118A31631A7F-001/tempDir-001/hdfsBaseDir/data/data3/, [DISK]file:/home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J2/temp/solr.cloud.TestHdfsCloudBackupRestore_23F1118A31631A7F-001/tempDir-001/hdfsBaseDir/data/data4/]] heartbeating to localhost.localdomain/127.0.0.1:39171) [ ] o.a.h.h.s.d.DirectoryScanner dfs.datanode.directoryscan.throttle.limit.ms.per.sec set to value below 1 ms/sec. Assuming default value of 1000
[junit4] 2> 204871 ERROR (DataNode: [[[DISK]file:/home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J2/temp/solr.cloud.TestHdfsCloudBackupRestore_23F1118A31631A7F-001/tempDir-001/hdfsBaseDir/data/data1/, [DISK]file:/home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J2/temp/solr.cloud.TestHdfsCloudBackupRestore_23F1118A31631A7F-001/tempDir-001/hdfsBaseDir/data/data2/]] heartbeating to localhost.localdomain/127.0.0.1:39171) [ ] o.a.h.h.s.d.DirectoryScanner dfs.datanode.directoryscan.throttle.limit.ms.per.sec set to value below 1 ms/sec. Assuming default value of 1000
[junit4] 2> 204933 INFO (Block report processor) [ ] BlockStateChange BLOCK* processReport 0x8932542f522db: from storage DS-ed93d7f5-e2db-4867-974a-57eb987109aa node DatanodeRegistration(127.0.0.1:33859, datanodeUuid=f4d784a5-ff8d-49ed-85ea-fc2889b5682a, infoPort=35893, infoSecurePort=0, ipcPort=32773, storageInfo=lv=-56;cid=testClusterID;nsid=1233212690;c=0), blocks: 0, hasStaleStorage: true, processing time: 2 msecs
[junit4] 2> 204933 INFO (Block report processor) [ ] BlockStateChange BLOCK* processReport 0x8932542e6b017: from storage DS-5bc826a6-1bec-480b-8d03-4dddb2658e7f node DatanodeRegistration(127.0.0.1:40755, datanodeUuid=b31d512d-ac59-4eb2-8d5b-2ab546f9d50a, infoPort=37089, infoSecurePort=0, ipcPort=36015, storageInfo=lv=-56;cid=testClusterID;nsid=1233212690;c=0), blocks: 0, hasStaleStorage: true, processing time: 0 msecs
[junit4] 2> 204933 INFO (Block report processor) [ ] BlockStateChange BLOCK* processReport 0x8932542f522db: from storage DS-c40fafb0-07e8-4d44-9b80-5e8d04143908 node DatanodeRegistration(127.0.0.1:33859, datanodeUuid=f4d784a5-ff8d-49ed-85ea-fc2889b5682a, infoPort=35893, infoSecurePort=0, ipcPort=32773, storageInfo=lv=-56;cid=testClusterID;nsid=1233212690;c=0), blocks: 0, hasStaleStorage: false, processing time: 0 msecs
[junit4] 2> 204934 INFO (Block report processor) [ ] BlockStateChange BLOCK* processReport 0x8932542e6b017: from storage DS-2ef24a75-fc6e-4cc7-85c2-5c8a6dadefad node DatanodeRegistration(127.0.0.1:40755, datanodeUuid=b31d512d-ac59-4eb2-8d5b-2ab546f9d50a, infoPort=37089, infoSecurePort=0, ipcPort=36015, storageInfo=lv=-56;cid=testClusterID;nsid=1233212690;c=0), blocks: 0, hasStaleStorage: false, processing time: 0 msecs
[junit4] 2> 205089 INFO (SUITE-TestHdfsCloudBackupRestore-seed#[23F1118A31631A7F]-worker) [ ] o.a.s.c.MiniSolrCloudCluster Starting cluster of 2 servers in /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J2/temp/solr.cloud.TestHdfsCloudBackupRestore_23F1118A31631A7F-001/tempDir-002
[junit4] 2> 205090 INFO (SUITE-TestHdfsCloudBackupRestore-seed#[23F1118A31631A7F]-worker) [ ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
[junit4] 2> 205090 INFO (Thread-369) [ ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 205090 INFO (Thread-369) [ ] o.a.s.c.ZkTestServer Starting server
[junit4] 2> 205091 ERROR (Thread-369) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 205190 INFO (SUITE-TestHdfsCloudBackupRestore-seed#[23F1118A31631A7F]-worker) [ ] o.a.s.c.ZkTestServer start zk server on port:43707
[junit4] 2> 205192 INFO (zkConnectionManagerCallback-295-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 205217 INFO (jetty-launcher-292-thread-2) [ ] o.e.j.s.Server jetty-9.3.20.v20170531
[junit4] 2> 205217 INFO (jetty-launcher-292-thread-1) [ ] o.e.j.s.Server jetty-9.3.20.v20170531
[junit4] 2> 205218 INFO (jetty-launcher-292-thread-2) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@121ecf11{/solr,null,AVAILABLE}
[junit4] 2> 205218 INFO (jetty-launcher-292-thread-1) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@7211c42{/solr,null,AVAILABLE}
[junit4] 2> 205219 INFO (jetty-launcher-292-thread-2) [ ] o.e.j.s.AbstractConnector Started ServerConnector@246af3ca{SSL,[ssl, http/1.1]}{127.0.0.1:45539}
[junit4] 2> 205219 INFO (jetty-launcher-292-thread-1) [ ] o.e.j.s.AbstractConnector Started ServerConnector@86b5cae{SSL,[ssl, http/1.1]}{127.0.0.1:40065}
[junit4] 2> 205219 INFO (jetty-launcher-292-thread-2) [ ] o.e.j.s.Server Started @207479ms
[junit4] 2> 205219 INFO (jetty-launcher-292-thread-1) [ ] o.e.j.s.Server Started @207479ms
[junit4] 2> 205219 INFO (jetty-launcher-292-thread-2) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=45539}
[junit4] 2> 205219 INFO (jetty-launcher-292-thread-1) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=40065}
[junit4] 2> 205219 ERROR (jetty-launcher-292-thread-1) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 205219 ERROR (jetty-launcher-292-thread-2) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 205219 INFO (jetty-launcher-292-thread-2) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 7.3.0
[junit4] 2> 205219 INFO (jetty-launcher-292-thread-1) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 7.3.0
[junit4] 2> 205219 INFO (jetty-launcher-292-thread-2) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 205219 INFO (jetty-launcher-292-thread-1) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 205219 INFO (jetty-launcher-292-thread-2) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 205219 INFO (jetty-launcher-292-thread-1) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 205219 INFO (jetty-launcher-292-thread-1) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2017-12-31T09:27:26.381842Z
[junit4] 2> 205219 INFO (jetty-launcher-292-thread-2) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2017-12-31T09:27:26.381833Z
[junit4] 2> 205222 INFO (zkConnectionManagerCallback-298-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 205222 INFO (zkConnectionManagerCallback-299-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 205223 INFO (jetty-launcher-292-thread-1) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 205223 INFO (jetty-launcher-292-thread-2) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 205226 INFO (jetty-launcher-292-thread-1) [ ] o.a.s.c.SolrXmlConfig MBean server found: com.sun.jmx.mbeanserver.JmxMBeanServer@50aae5fe, but no JMX reporters were configured - adding default JMX reporter.
[junit4] 2> 205226 INFO (jetty-launcher-292-thread-2) [ ] o.a.s.c.SolrXmlConfig MBean server found: com.sun.jmx.mbeanserver.JmxMBeanServer@50aae5fe, but no JMX reporters were configured - adding default JMX reporter.
[junit4] 2> 205227 WARN (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [ ] o.a.z.s.NIOServerCnxn caught end of stream exception
[junit4] 2> EndOfStreamException: Unable to read additional data from client sessionid 0x160abe5b52c0001, likely client has closed socket
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:239)
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:203)
[junit4] 2> at java.base/java.lang.Thread.run(Thread.java:844)
[junit4] 2> 205228 INFO (jetty-launcher-292-thread-1) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:43707/solr
[junit4] 2> 205229 INFO (jetty-launcher-292-thread-2) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:43707/solr
[junit4] 2> 205230 INFO (zkConnectionManagerCallback-304-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 205230 INFO (zkConnectionManagerCallback-307-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 205232 INFO (zkConnectionManagerCallback-309-thread-1-processing-n:127.0.0.1:45539_solr) [n:127.0.0.1:45539_solr ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 205233 INFO (zkConnectionManagerCallback-311-thread-1-processing-n:127.0.0.1:40065_solr) [n:127.0.0.1:40065_solr ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 205276 INFO (jetty-launcher-292-thread-2) [n:127.0.0.1:45539_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 205276 INFO (jetty-launcher-292-thread-1) [n:127.0.0.1:40065_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 205277 INFO (jetty-launcher-292-thread-2) [n:127.0.0.1:45539_solr ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:45539_solr
[junit4] 2> 205277 INFO (jetty-launcher-292-thread-1) [n:127.0.0.1:40065_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:40065_solr
[junit4] 2> 205277 INFO (jetty-launcher-292-thread-2) [n:127.0.0.1:45539_solr ] o.a.s.c.Overseer Overseer (id=99268194877571077-127.0.0.1:45539_solr-n_0000000000) starting
[junit4] 2> 205278 INFO (zkCallback-310-thread-1-processing-n:127.0.0.1:40065_solr) [n:127.0.0.1:40065_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 205278 INFO (zkCallback-308-thread-1-processing-n:127.0.0.1:45539_solr) [n:127.0.0.1:45539_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 205282 INFO (jetty-launcher-292-thread-2) [n:127.0.0.1:45539_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:45539_solr
[junit4] 2> 205282 INFO (zkCallback-308-thread-1-processing-n:127.0.0.1:45539_solr) [n:127.0.0.1:45539_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 205282 INFO (zkCallback-310-thread-1-processing-n:127.0.0.1:40065_solr) [n:127.0.0.1:40065_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 205299 INFO (jetty-launcher-292-thread-1) [n:127.0.0.1:40065_solr ] o.a.s.c.b.r.BackupRepositoryFactory Added backup repository with configuration params {type = repository,name = hdfs,class = org.apache.solr.core.backup.repository.HdfsBackupRepository,attributes = {name=hdfs, class=org.apache.solr.core.backup.repository.HdfsBackupRepository},args = {location=/backup,solr.hdfs.home=hdfs://localhost.localdomain:39171/solr,solr.hdfs.confdir=}}
[junit4] 2> 205299 INFO (jetty-launcher-292-thread-1) [n:127.0.0.1:40065_solr ] o.a.s.c.b.r.BackupRepositoryFactory Default configuration for backup repository is with configuration params {type = repository,name = hdfs,class = org.apache.solr.core.backup.repository.HdfsBackupRepository,attributes = {name=hdfs, class=org.apache.solr.core.backup.repository.HdfsBackupRepository},args = {location=/backup,solr.hdfs.home=hdfs://localhost.localdomain:39171/solr,solr.hdfs.confdir=}}
[junit4] 2> 205301 INFO (jetty-launcher-292-thread-2) [n:127.0.0.1:45539_solr ] o.a.s.c.b.r.BackupRepositoryFactory Added backup repository with configuration params {type = repository,name = hdfs,class = org.apache.solr.core.backup.repository.HdfsBackupRepository,attributes = {name=hdfs, class=org.apache.solr.core.backup.repository.HdfsBackupRepository},args = {location=/backup,solr.hdfs.home=hdfs://localhost.localdomain:39171/solr,solr.hdfs.confdir=}}
[junit4] 2> 205301 INFO (jetty-launcher-292-thread-2) [n:127.0.0.1:45539_solr ] o.a.s.c.b.r.BackupRepositoryFactory Default configuration for backup repository is with configuration params {type = repository,name = hdfs,class = org.apache.solr.core.backup.repository.HdfsBackupRepository,attributes = {name=hdfs, class=org.apache.solr.core.backup.repository.HdfsBackupRepository},args = {location=/backup,solr.hdfs.home=hdfs://localhost.localdomain:39171/solr,solr.hdfs.confdir=}}
[junit4] 2> 205317 INFO (jetty-launcher-292-thread-1) [n:127.0.0.1:40065_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@50aae5fe
[junit4] 2> 205320 INFO (jetty-launcher-292-thread-2) [n:127.0.0.1:45539_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@50aae5fe
[junit4] 2> 205326 INFO (jetty-launcher-292-thread-1) [n:127.0.0.1:40065_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@50aae5fe
[junit4] 2> 205326 INFO (jetty-launcher-292-thread-1) [n:127.0.0.1:40065_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@50aae5fe
[junit4] 2> 205327 INFO (jetty-launcher-292-thread-1) [n:127.0.0.1:40065_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J2/temp/solr.cloud.TestHdfsCloudBackupRestore_23F1118A31631A7F-001/tempDir-002/node1/.
[junit4] 2> 205330 INFO (jetty-launcher-292-thread-2) [n:127.0.0.1:45539_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@50aae5fe
[junit4] 2> 205330 INFO (jetty-launcher-292-thread-2) [n:127.0.0.1:45539_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@50aae5fe
[junit4] 2> 205331 INFO (jetty-launcher-292-thread-2) [n:127.0.0.1:45539_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J2/temp/solr.cloud.TestHdfsCloudBackupRestore_23F1118A31631A7F-001/tempDir-002/node2/.
[junit4] 2> 205388 INFO (zkConnectionManagerCallback-317-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 205395 INFO (zkConnectionManagerCallback-321-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 205395 INFO (SUITE-TestHdfsCloudBackupRestore-seed#[23F1118A31631A7F]-worker) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
[junit4] 2> 205396 INFO (SUITE-TestHdfsCloudBackupRestore-seed#[23F1118A31631A7F]-worker) [ ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:43707/solr ready
[junit4] 2> 205415 INFO (TEST-TestHdfsCloudBackupRestore.test-seed#[23F1118A31631A7F]) [ ] o.a.s.SolrTestCaseJ4 ###Starting test
[junit4] 2> 205425 INFO (qtp1171428704-1686) [n:127.0.0.1:40065_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params pullReplicas=0&replicationFactor=2&property.customKey=customValue&collection.configName=conf1&maxShardsPerNode=6&name=hdfsbackuprestore&nrtReplicas=2&action=CREATE&numShards=2&tlogReplicas=1&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 205430 INFO (OverseerThreadFactory-605-thread-1-processing-n:127.0.0.1:45539_solr) [n:127.0.0.1:45539_solr ] o.a.s.c.CreateCollectionCmd Create collection hdfsbackuprestore
[junit4] 2> 205430 WARN (OverseerThreadFactory-605-thread-1-processing-n:127.0.0.1:45539_solr) [n:127.0.0.1:45539_solr ] o.a.s.c.CreateCollectionCmd Specified number of replicas of 3 on collection hdfsbackuprestore is higher than the number of Solr instances currently live or live and part of your createNodeSet(2). It's unusual to run two replica of the same slice on the same Solr-instance.
[junit4] 2> 205533 INFO (OverseerStateUpdate-99268194877571077-127.0.0.1:45539_solr-n_0000000000) [n:127.0.0.1:45539_solr ] o.a.s.c.o.SliceMutator createReplica() {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"hdfsbackuprestore",
[junit4] 2> "shard":"shard1",
[junit4] 2> "core":"hdfsbackuprestore_shard1_replica_n1",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"https://127.0.0.1:40065/solr",
[junit4] 2> "type":"NRT",
[junit4] 2> "waitForFinalState":"false"}
[junit4] 2> 205534 INFO (OverseerStateUpdate-99268194877571077-127.0.0.1:45539_solr-n_0000000000) [n:127.0.0.1:45539_solr ] o.a.s.c.o.SliceMutator createReplica() {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"hdfsbackuprestore",
[junit4] 2> "shard":"shard1",
[junit4] 2> "core":"hdfsbackuprestore_shard1_replica_n2",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"https://127.0.0.1:45539/solr",
[junit4] 2> "type":"NRT",
[junit4] 2> "waitForFinalState":"false"}
[junit4] 2> 205535 INFO (OverseerStateUpdate-99268194877571077-127.0.0.1:45539_solr-n_0000000000) [n:127.0.0.1:45539_solr ] o.a.s.c.o.SliceMutator createReplica() {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"hdfsbackuprestore",
[junit4] 2> "shard":"shard1",
[junit4] 2> "core":"hdfsbackuprestore_shard1_replica_t4",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"https://127.0.0.1:40065/solr",
[junit4] 2> "type":"TLOG",
[junit4] 2> "waitForFinalState":"false"}
[junit4] 2> 205536 INFO (OverseerStateUpdate-99268194877571077-127.0.0.1:45539_solr-n_0000000000) [n:127.0.0.1:45539_solr ] o.a.s.c.o.SliceMutator createReplica() {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"hdfsbackuprestore",
[junit4] 2> "shard":"shard2",
[junit4] 2> "core":"hdfsbackuprestore_shard2_replica_n6",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"https://127.0.0.1:45539/solr",
[junit4] 2> "type":"NRT",
[junit4] 2> "waitForFinalState":"false"}
[junit4] 2> 205537 INFO (OverseerStateUpdate-99268194877571077-127.0.0.1:45539_solr-n_0000000000) [n:127.0.0.1:45539_solr ] o.a.s.c.o.SliceMutator createReplica() {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"hdfsbackuprestore",
[junit4] 2> "shard":"shard2",
[junit4] 2> "core":"hdfsbackuprestore_shard2_replica_n8",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"https://127.0.0.1:40065/solr",
[junit4] 2> "type":"NRT",
[junit4] 2> "waitForFinalState":"false"}
[junit4] 2> 205537 INFO (OverseerStateUpdate-99268194877571077-127.0.0.1:45539_solr-n_0000000000) [n:127.0.0.1:45539_solr ] o.a.s.c.o.SliceMutator createReplica() {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"hdfsbackuprestore",
[junit4] 2> "shard":"shard2",
[junit4] 2> "core":"hdfsbackuprestore_shard2_replica_t10",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"https://127.0.0.1:45539/solr",
[junit4] 2> "type":"TLOG",
[junit4] 2> "waitForFinalState":"false"}
[junit4] 2> 205756 INFO (qtp1171428704-1684) [n:127.0.0.1:40065_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf1&newCollection=true&collection=hdfsbackuprestore&version=2&replicaType=NRT&property.customKey=customValue&coreNodeName=core_node3&name=hdfsbackuprestore_shard1_replica_n1&action=CREATE&numShards=2&shard=shard1&wt=javabin
[junit4] 2> 205756 INFO (qtp1171428704-1684) [n:127.0.0.1:40065_solr ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 205758 INFO (qtp1171428704-1682) [n:127.0.0.1:40065_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf1&newCollection=true&collection=hdfsbackuprestore&version=2&replicaType=TLOG&property.customKey=customValue&coreNodeName=core_node7&name=hdfsbackuprestore_shard1_replica_t4&action=CREATE&numShards=2&shard=shard1&wt=javabin
[junit4] 2> 205779 INFO (qtp1171428704-1743) [n:127.0.0.1:40065_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf1&newCollection=true&collection=hdfsbackuprestore&version=2&replicaType=NRT&property.customKey=customValue&coreNodeName=core_node11&name=hdfsbackuprestore_shard2_replica_n8&action=CREATE&numShards=2&shard=shard2&wt=javabin
[junit4] 2> 205790 INFO (qtp1887539743-1675) [n:127.0.0.1:45539_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf1&newCollection=true&collection=hdfsbackuprestore&version=2&replicaType=NRT&property.customKey=customValue&coreNodeName=core_node9&name=hdfsbackuprestore_shard2_replica_n6&action=CREATE&numShards=2&shard=shard2&wt=javabin
[junit4] 2> 205790 INFO (qtp1887539743-1675) [n:127.0.0.1:45539_solr ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 205807 INFO (qtp1887539743-1671) [n:127.0.0.1:45539_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf1&newCollection=true&collection=hdfsbackuprestore&version=2&replicaType=TLOG&property.customKey=customValue&coreNodeName=core_node12&name=hdfsbackuprestore_shard2_replica_t10&action=CREATE&numShards=2&shard=shard2&wt=javabin
[junit4] 2> 205807 INFO (qtp1887539743-1747) [n:127.0.0.1:45539_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf1&newCollection=true&collection=hdfsbackuprestore&version=2&replicaType=NRT&property.customKey=customValue&coreNodeName=core_node5&name=hdfsbackuprestore_shard1_replica_n2&action=CREATE&numShards=2&shard=shard1&wt=javabin
[junit4] 2> 205909 INFO (zkCallback-308-thread-1-processing-n:127.0.0.1:45539_solr) [n:127.0.0.1:45539_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/hdfsbackuprestore/state.json] for collection [hdfsbackuprestore] has occurred - updating... (live nodes size: [2])
[junit4] 2> 205909 INFO (zkCallback-310-thread-1-processing-n:127.0.0.1:40065_solr) [n:127.0.0.1:40065_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/hdfsbackuprestore/state.json] for collection [hdfsbackuprestore] has occurred - updating... (live nodes size: [2])
[junit4] 2> 205910 INFO (zkCallback-308-thread-2-processing-n:127.0.0.1:45539_solr) [n:127.0.0.1:45539_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/hdfsbackuprestore/state.json] for collection [hdfsbackuprestore] has occurred - updating... (live nodes size: [2])
[junit4] 2> 206769 INFO (qtp1171428704-1682) [n:127.0.0.1:40065_solr c:hdfsbackuprestore s:shard1 r:core_node7 x:hdfsbackuprestore_shard1_replica_t4] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.3.0
[junit4] 2> 206776 INFO (qtp1171428704-1682) [n:127.0.0.1:40065_solr c:hdfsbackuprestore s:shard1 r:core_node7 x:hdfsbackuprestore_shard1_replica_t4] o.a.s.s.IndexSchema [hdfsbackuprestore_shard1_replica_t4] Schema name=minimal
[junit4] 2> 206778 INFO (qtp1171428704-1682) [n:127.0.0.1:40065_solr c:hdfsbackuprestore s:shard1 r:core_node7 x:hdfsbackuprestore_shard1_replica_t4] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
[junit4] 2> 206778 INFO (qtp1171428704-1682) [n:127.0.0.1:40065_solr c:hdfsbackuprestore s:shard1 r:core_node7 x:hdfsbackuprestore_shard1_replica_t4] o.a.s.c.CoreContainer Creating SolrCore 'hdfsbackuprestore_shard1_replica_t4' using configuration from collection hdfsbackuprestore, trusted=true
[junit4] 2> 206778 INFO (qtp1171428704-1682) [n:127.0.0.1:40065_solr c:hdfsbackuprestore s:shard1 r:core_node7 x:hdfsbackuprestore_shard1_replica_t4] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.hdfsbackuprestore.shard1.replica_t4' (registry 'solr.core.hdfsbackuprestore.shard1.replica_t4') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@50aae5fe
[junit4] 2> 206783 INFO (qtp1171428704-1682) [n:127.0.0.1:40065_solr c:hdfsbackuprestore s:shard1 r:core_node7 x:hdfsbackuprestore_shard1_replica_t4] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 206784 INFO (qtp1171428704-1682) [n:127.0.0.1:40065_solr c:hdfsbackuprestore s:shard1 r:core_node7 x:hdfsbackuprestore_shard1_replica_t4] o.a.s.c.SolrCore [[hdfsbackuprestore_shard1_replica_t4] ] Opening new SolrCore at [/home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J2/temp/solr.cloud.TestHdfsCloudBackupRestore_23F1118A31631A7F-001/tempDir-002/node1/hdfsbackuprestore_shard1_replica_t4], dataDir=[/home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J2/temp/solr.cloud.TestHdfsCloudBackupRestore_23F1118A31631A7F-001/tempDir-002/node1/./hdfsbackuprestore_shard1_replica_t4/data/]
[junit4] 2> 206787 INFO (qtp1171428704-1684) [n:127.0.0.1:40065_solr c:hdfsbackuprestore s:shard1 r:core_node3 x:hdfsbackuprestore_shard1_replica_n1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.3.0
[junit4] 2> 206788 INFO (qtp1171428704-1743) [n:127.0.0.1:40065_solr c:hdfsbackuprestore s:shard2 r:core_node11 x:hdfsbackuprestore_shard2_replica_n8] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.3.0
[junit4] 2> 206801 INFO (qtp1171428704-1684) [n:127.0.0.1:40065_solr c:hdfsbackuprestore s:shard1 r:core_node3 x:hdfsbackuprestore_shard1_replica_n1] o.a.s.s.IndexSchema [hdfsbackuprestore_shard1_replica_n1] Schema name=minimal
[junit4] 2> 206801 INFO (qtp1171428704-1743) [n:127.0.0.1:40065_solr c:hdfsbackuprestore s:shard2 r:core_node11 x:hdfsbackuprestore_shard2_replica_n8] o.a.s.s.IndexSchema [hdfsbackuprestore_shard2_replica_n8] Schema name=minimal
[junit4] 2> 206803 INFO (qtp1171428704-1684) [n:127.0.0.1:40065_solr c:hdfsbackuprestore s:shard1 r:core_node3 x:hdfsbackuprestore_shard1_replica_n1] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
[junit4] 2> 206803 INFO (qtp1171428704-1684) [n:127.0.0.1:40065_solr c:hdfsbackuprestore s:shard1 r:core_node3 x:hdfsbackuprestore_shard1_replica_n1] o.a.s.c.CoreContainer Creating SolrCore 'hdfsbackuprestore_shard1_replica_n1' using configuration from collection hdfsbackuprestore, trusted=true
[junit4] 2> 206803 INFO (qtp1171428704-1743) [n:127.0.0.1:40065_solr c:hdfsbackuprestore s:shard2 r:core_node11 x:hdfsbackuprestore_shard2_replica_n8] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
[junit4] 2> 206803 INFO (qtp1171428704-1743) [n:127.0.0.1:40065_solr c:hdfsbackuprestore s:shard2 r:core_node11 x:hdfsbackuprestore_shard2_replica_n8] o.a.s.c.CoreContainer Creating SolrCore 'hdfsbackuprestore_shard2_replica_n8' using configuration from collection hdfsbackuprestore, trusted=true
[junit4] 2> 206804 INFO (qtp1171428704-1684) [n:127.0.0.1:40065_solr c:hdfsbackuprestore s:shard1 r:core_node3 x:hdfsbackuprestore_shard1_replica_n1] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.hdfsbackuprestore.shard1.replica_n1' (registry 'solr.core.hdfsbackuprestore.shard1.replica_n1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@50aae5fe
[junit4] 2> 206804 INFO (qtp1171428704-1684) [n:127.0.0.1:40065_solr c:hdfsbackuprestore s:shard1 r:core_node3 x:hdfsbackuprestore_shard1_replica_n1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 206804 INFO (qtp1171428704-1684) [n:127.0.0.1:40065_solr c:hdfsbackuprestore s:shard1 r:core_node3 x:hdfsbackuprestore_shard1_replica_n1] o.a.s.c.SolrCore [[hdfsbackuprestore_shard1_replica_n1] ] Opening new SolrCore at [/home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J2/temp/solr.cloud.TestHdfsCloudBackupRestore_23F1118A31631A7F-001/tempDir-002/node1/hdfsbackuprestore_shard1_replica_n1], dataDir=[/home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J2/temp/solr.cloud.TestHdfsCloudBackupRestore_23F1118A31631A7F-001/tempDir-002/node1/./hdfsbackuprestore_shard1_replica_n1/data/]
[junit4] 2> 206804 INFO (qtp1171428704-1743) [n:127.0.0.1:40065_solr c:hdfsbackuprestore s:shard2 r:core_node11 x:hdfsbackuprestore_shard2_replica_n8] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.hdfsbackuprestore.shard2.replica_n8' (registry 'solr.core.hdfsbackuprestore.shard2.replica_n8') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@50aae5fe
[junit4] 2> 206804 INFO (qtp1171428704-1743) [n:127.0.0.1:40065_solr c:hdfsbackuprestore s:shard2 r:core_node11 x:hdfsbackuprestore_shard2_replica_n8] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 206804 INFO (qtp1171428704-1743) [n:127.0.0.1:40065_solr c:hdfsbackuprestore s:shard2 r:core_node11 x:hdfsbackuprestore_shard2_replica_n8] o.a.s.c.SolrCore [[hdfsbackuprestore_shard2_replica_n8] ] Opening new SolrCore at [/home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J2/temp/solr.cloud.TestHdfsCloudBackupRestore_23F1118A31631A7F-001/tempDir-002/node1/hdfsbackuprestore_shard2_replica_n8], dataDir=[/home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J2/temp/solr.cloud.TestHdfsCloudBackupRestore_23F1118A31631A7F-001/tempDir-002/node1/./hdfsbackuprestore_shard2_replica_n8/data/]
[junit4] 2> 206808 INFO (qtp1887539743-1675) [n:127.0.0.1:45539_solr c:hdfsbackuprestore s:shard2 r:core_node9 x:hdfsbackuprestore_shard2_replica_n6] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.3.0
[junit4] 2> 206812 INFO (qtp1887539743-1675) [n:127.0.0.1:45539_solr c:hdfsbackuprestore s:shard2 r:core_node9 x:hdfsbackuprestore_shard2_replica_n6] o.a.s.s.IndexSchema [hdfsbackuprestore_shard2_replica_n6] Schema name=minimal
[junit4] 2> 206814 INFO (qtp1887539743-1671) [n:127.0.0.1:45539_solr c:hdfsbackuprestore s:shard2 r:core_node12 x:hdfsbackuprestore_shard2_replica_t10] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.3.0
[junit4] 2> 206819 INFO (qtp1887539743-1747) [n:127.0.0.1:45539_solr c:hdfsbackuprestore s:shard1 r:core_node5 x:hdfsbackuprestore_shard1_replica_n2] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.3.0
[junit4] 2> 206819 INFO (qtp1887539743-1675) [n:127.0.0.1:45539_solr c:hdfsbackuprestore s:shard2 r:core_node9 x:hdfsbackuprestore_shard2_replica_n6] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
[junit4] 2> 206819 INFO (qtp1887539743-1675) [n:127.0.0.1:45539_solr c:hdfsbackuprestore s:shard2 r:core_node9 x:hdfsbackuprestore_shard2_replica_n6] o.a.s.c.CoreContainer Creating SolrCore 'hdfsbackuprestore_shard2_replica_n6' using configuration from collection hdfsbackuprestore, trusted=true
[junit4] 2> 206820 INFO (qtp1887539743-1675) [n:127.0.0.1:45539_solr c:hdfsbackuprestore s:shard2 r:core_node9 x:hdfsbackuprestore_shard2_replica_n6] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.hdfsbackuprestore.shard2.replica_n6' (registry 'solr.core.hdfsbackuprestore.shard2.replica_n6') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@50aae5fe
[junit4] 2> 206820 INFO (qtp1887539743-1675) [n:127.0.0.1:45539_solr c:hdfsbackuprestore s:shard2 r:core_node9 x:hdfsbackuprestore_shard2_replica_n6] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 206820 INFO (qtp1887539743-1675) [n:127.0.0.1:45539_solr c:hdfsbackuprestore s:shard2 r:core_node9 x:hdfsbackuprestore_shard2_replica_n6] o.a.s.c.SolrCore [[hdfsbackuprestore_shard2_replica_n6] ] Opening new SolrCore at [/home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J2/temp/solr.cloud.TestHdfsCloudBackupRestore_23F1118A31631A7F-001/tempDir-002/node2/hdfsbackuprestore_shard2_replica_n6], dataDir=[/home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J2/temp/solr.cloud.TestHdfsCloudBackupRestore_23F1118A31631A7F-001/tempDir-002/node2/./hdfsbackuprestore_shard2_replica_n6/data/]
[junit4] 2> 206823 INFO (qtp1887539743-1671) [n:127.0.0.1:45539_solr c:hdfsbackuprestore s:shard2 r:core_node12 x:hdfsbackuprestore_shard2_replica_t10] o.a.s.s.IndexSchema [hdfsbackuprestore_shard2_replica_t10] Schema name=minimal
[junit4] 2> 206824 INFO (qtp1887539743-1747) [n:127.0.0.1:45539_solr c:hdfsbackuprestore s:shard1 r:core_node5 x:hdfsbackuprestore_shard1_replica_n2] o.a.s.s.IndexSchema [hdfsbackuprestore_shard1_replica_n2] Schema name=minimal
[junit4] 2> 206825 INFO (qtp1887539743-1671) [n:127.0.0.1:45539_solr c:hdfsbackuprestore s:shard2 r:core_node12 x:hdfsbackuprestore_shard2_replica_t10] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
[junit4] 2> 206825 INFO (qtp1887539743-1671) [n:127.0.0.1:45539_solr c:hdfsbackuprestore s:shard2 r:core_node12 x:hdfsbackuprestore_shard2_replica_t10] o.a.s.c.CoreContainer Creating SolrCore 'hdfsbackuprestore_shard2_replica_t10' using configuration from collection hdfsbackuprestore, trusted=true
[junit4] 2> 206825 INFO (qtp1887539743-1671) [n:127.0.0.1:45539_solr c:hdfsbackuprestore s:shard2 r:core_node12 x:hdfsbackuprestore_shard2_replica_t10] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.hdfsbackuprestore.shard2.replica_t10' (registry 'solr.core.hdfsbackuprestore.shard2.replica_t10') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@50aae5fe
[junit4] 2> 206825 INFO (qtp1887539743-1671) [n:127.0.0.1:45539_solr c:hdfsbackuprestore s:shard2 r:core_node12 x:hdfsbackuprestore_shard2_replica_t10] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 206825 INFO (qtp1887539743-1671) [n:127.0.0.1:45539_solr c:hdfsbackuprestore s:shard2 r:core_node12 x:hdfsbackuprestore_shard2_replica_t10] o.a.s.c.SolrCore [[hdfsbackuprestore_shard2_replica_t10] ] Opening new SolrCore at [/home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J2/temp/solr.cloud.TestHdfsCloudBackupRestore_23F1118A31631A7F-001/tempDir-002/node2/hdfsbackuprestore_shard2_replica_t10], dataDir=[/home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J2/temp/solr.cloud.TestHdfsCloudBackupRestore_23F1118A31631A7F-001/tempDir-002/node2/./hdfsbackuprestore_shard2_replica_t10/data/]
[junit4] 2> 206826 INFO (qtp1887539743-1747) [n:127.0.0.1:45539_solr c:hdfsbackuprestore s:shard1 r:core_node5 x:hdfsbackuprestore_shard1_replica_n2] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
[junit4] 2> 206826 INFO (qtp1887539743-1747) [n:127.0.0.1:45539_solr c:hdfsbackuprestore s:shard1 r:core_node5 x:hdfsbackuprestore_shard1_replica_n2] o.a.s.c.CoreContainer Creating SolrCore 'hdfsbackuprestore_shard1_replica_n2' using configuration from collection hdfsbackuprestore, trusted=true
[junit4] 2> 206827 INFO (qtp1887539743-1747) [n:127.0.0.1:45539_solr c:hdfsbackuprestore s:shard1 r:core_node5 x:hdfsbackuprestore_shard1_replica_n2] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.hdfsbackuprestore.shard1.replica_n2' (registry 'solr.core.hdfsbackuprestore.shard1.replica_n2') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@50aae5fe
[junit4] 2> 206827 INFO (qtp1887539743-1747) [n:127.0.0.1:45539_solr c:hdfsbackuprestore s:shard1 r:core_node5 x:hdfsbackuprestore_shard1_replica_n2] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 206827 INFO (qtp1887539743-1747) [n:127.0.0.1:45539_solr c:hdfsbackuprestore s:shard1 r:core_node5 x:hdfsbackuprestore_shard1_replica_n2] o.a.s.c.SolrCore [[hdfsbackuprestore_shard1_replica_n2] ] Opening new SolrCore at [/home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J2/temp/solr.cloud.TestHdfsCloudBackupRestore_23F1118A31631A7F-001/tempDir-002/node2/hdfsbackuprestore_shard1_replica_n2], dataDir=[/home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J2/temp/solr.cloud.TestHdfsCloudBackupRestore_23F1118A31631A7F-001/tempDir-002/node2/./hdfsbackuprestore_shard1_replica_n2/data/]
[junit4] 2> 206894 INFO (qtp1171428704-1682) [n:127.0.0.1:40065_solr c:hdfsbackuprestore s:shard1 r:core_node7 x:hdfsbackuprestore_shard1_replica_t4] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 206894 INFO (qtp1171428704-1682) [n:127.0.0.1:40065_solr c:hdfsbackuprestore s:shard1 r:core_node7 x:hdfsbackuprestore_shard1_replica_t4] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 206895 INFO (qtp1171428704-1682) [n:127.0.0.1:40065_solr c:hdfsbackuprestore s:shard1 r:core_node7 x:hdfsbackuprestore_shard1_replica_t4] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 206895 INFO (qtp1171428704-1682) [n:127.0.0.1:40065_solr c:hdfsbackuprestore s:shard1 r:core_node7 x:hdfsbackuprestore_shard1_replica_t4] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 206896 INFO (qtp1171428704-1743) [n:127.0.0.1:40065_solr c:hdfsbackuprestore s:shard2 r:core_node11 x:hdfsbackuprestore_shard2_replica_n8] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 206896 INFO (qtp1171428704-1743) [n:127.0.0.1:40065_solr c:hdfsbackuprestore s:shard2 r:core_node11 x:hdfsbackuprestore_shard2_replica_n8] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 206897 INFO (qtp1171428704-1743) [n:127.0.0.1:40065_solr c:hdfsbackuprestore s:shard2 r:core_node11 x:hdfsbackuprestore_shard2_replica_n8] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 206897 INFO (qtp1171428704-1743) [n:127.0.0.1:40065_solr c:hdfsbackuprestore s:shard2 r:core_node11 x:hdfsbackuprestore_shard2_replica_n8] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 206898 INFO (qtp1171428704-1682) [n:127.0.0.1:40065_solr c:hdfsbackuprestore s:shard1 r:core_node7 x:hdfsbackuprestore_shard1_replica_t4] o.a.s.s.SolrIndexSearcher Opening [Searcher@7a217b3b[hdfsbackuprestore_shard1_replica_t4] main]
[junit4] 2> 206898 INFO (qtp1171428704-1682) [n:127.0.0.1:40065_solr c:hdfsbackuprestore s:shard1 r:core_node7 x:hdfsbackuprestore_shard1_replica_t4] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 206899 INFO (qtp1171428704-1682) [n:127.0.0.1:40065_solr c:hdfsbackuprestore s:shard1 r:core_node7 x:hdfsbackuprestore_shard1_replica_t4] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 206899 INFO (qtp1171428704-1743) [n:127.0.0.1:40065_solr c:hdfsbackuprestore s:shard2 r:core_node11 x:hdfsbackuprestore_shard2_replica_n8] o.a.s.s.SolrIndexSearcher Opening [Searcher@1becd60e[hdfsbackuprestore_shard2_replica_n8] main]
[junit4] 2> 206899 INFO (qtp1171428704-1682) [n:127.0.0.1:40065_solr c:hdfsbackuprestore s:shard1 r:core_node7 x:hdfsbackuprestore_shard1_replica_t4] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
[junit4] 2> 206900 INFO (qtp1171428704-1743) [n:127.0.0.1:40065_solr c:hdfsbackuprestore s:shard2 r:core_node11 x:hdfsbackuprestore_shard2_replica_n8] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 206900 INFO (qtp1171428704-1743) [n:127.0.0.1:40065_solr c:hdfsbackuprestore s:shard2 r:core_node11 x:hdfsbackuprestore_shard2_replica_n8] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 206900 INFO (qtp1171428704-1682) [n:127.0.0.1:40065_solr c:hdfsbackuprestore s:shard1 r:core_node7 x:hdfsbackuprestore_shard1_replica_t4] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1588291119939059712
[junit4] 2> 206900 INFO (qtp1171428704-1743) [n:127.0.0.1:40065_solr c:hdfsbackuprestore s:shard2 r:core_node11 x:hdfsbackuprestore_shard2_replica_n8] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
[junit4] 2> 206901 INFO (qtp1171428704-1743) [n:127.0.0.1:40065_solr c:hdfsbackuprestore s:shard2 r:core_node11 x:hdfsbackuprestore_shard2_replica_n8] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1588291119940108288
[junit4] 2> 206902 INFO (searcherExecutor-610-thread-1-processing-n:127.0.0.1:40065_solr x:hdfsbackuprestore_shard1_replica_t4 s:shard1 c:hdfsbackuprestore r:core_node7) [n:127.0.0.1:40065_solr c:hdfsbackuprestore s:shard1 r:core_node7 x:hdfsbackuprestore_shard1_replica_t4] o.a.s.c.SolrCore [hdfsbackuprestore_shard1_replica_t4] Registered new searcher Searcher@7a217b3b[hdfsbackuprestore_shard1_replica_t4] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 206905 INFO (qtp1171428704-1682) [n:127.0.0.1:40065_solr c:hdfsbackuprestore s:shard1 r:core_node7 x:hdfsbackuprestore_shard1_replica_t4] o.a.s.c.ShardLeaderElectionContext Waiting until we see more replicas up for shard shard1: total=3 found=1 timeoutin=9999ms
[junit4] 2> 206905 INFO (qtp1171428704-1743) [n:127.0.0.1:40065_solr c:hdfsbackuprestore s:shard2 r:core_node11 x:hdfsbackuprestore_shard2_replica_n8] o.a.s.c.ShardLeaderElectionContext Waiting until we see more replicas up for shard shard2: total=3 found=1 timeoutin=9999ms
[junit4] 2> 206906 INFO (searcherExecutor-612-thread-1-processing-n:127.0.0.1:40065_solr x:hdfsbackuprestore_shard2_replica_n8 s:shard2 c:hdfsbackuprestore r:core_node11) [n:127.0.0.1:40065_solr c:hdfsbackuprestore s:shard2 r:core_node11 x:hdfsbackuprestore_shard2_replica_n8] o.a.s.c.SolrCore [hdfsbackuprestore_shard2_replica_n8] Registered new searcher Searcher@1becd60e[hdfsbackuprestore_shard2_replica_n8] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 206914 INFO (qtp1887539743-1671) [n:127.0.0.1:45539_solr c:hdfsbackuprestore s:shard2 r:core_node12 x:hdfsbackuprestore_shard2_replica_t10] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 206914 INFO (qtp1887539743-1671) [n:127.0.0.1:45539_solr c:hdfsbackuprestore s:shard2 r:core_node12 x:hdfsbackuprestore_shard2_replica_t10] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 206914 INFO (qtp1887539743-1671) [n:127.0.0.1:45539_solr c:hdfsbackuprestore s:shard2 r:core_node12 x:hdfsbackuprestore_shard2_replica_t10] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 206914 INFO (qtp1887539743-1671) [n:127.0.0.1:45539_solr c:hdfsbackuprestore s:shard2 r:core_node12 x:hdfsbackuprestore_shard2_replica_t10] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 206915 INFO (qtp1887539743-1675) [n:127.0.0.1:45539_solr c:hdfsbackuprestore s:shard2 r:core_node9 x:hdfsbackuprestore_shard2_replica_n6] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 206915 INFO (qtp1887539743-1675) [n:127.0.0.1:45539_solr c:hdfsbackuprestore s:shard2 r:core_node9 x:hdfsbackuprestore_shard2_replica_n6] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 206915 INFO (qtp1171428704-1684) [n:127.0.0.1:40065_solr c:hdfsbackuprestore s:shard1 r:core_node3 x:hdfsbackuprestore_shard1_replica_n1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 206915 INFO (qtp1171428704-1684) [n:127.0.0.1:40065_solr c:hdfsbackuprestore s:shard1 r:core_node3 x:hdfsbackuprestore_shard1_replica_n1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 206916 INFO (qtp1887539743-1671) [n:127.0.0.1:45539_solr c:hdfsbackuprestore s:shard2 r:core_node12 x:hdfsbackuprestore_shard2_replica_t10] o.a.s.s.SolrIndexSearcher Opening [Searcher@25422840[hdfsbackuprestore_shard2_replica_t10] main]
[junit4] 2> 206916 INFO (qtp1887539743-1675) [n:127.0.0.1:45539_solr c:hdfsbackuprestore s:shard2 r:core_node9 x:hdfsbackuprestore_shard2_replica_n6] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 206916 INFO (qtp1171428704-1684) [n:127.0.0.1:40065_solr c:hdfsbackuprestore s:shard1 r:core_node3 x:hdfsbackuprestore_shard1_replica_n1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 206916 INFO (qtp1887539743-1675) [n:127.0.0.1:45539_solr c:hdfsbackuprestore s:shard2 r:core_node9 x:hdfsbackuprestore_shard2_replica_n6] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 206916 INFO (qtp1171428704-1684) [n:127.0.0.1:40065_solr c:hdfsbackuprestore s:shard1 r:core_node3 x:hdfsbackuprestore_shard1_replica_n1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 206917 INFO (qtp1887539743-1671) [n:127.0.0.1:45539_solr c:hdfsbackuprestore s:shard2 r:core_node12 x:hdfsbackuprestore_shard2_replica_t10] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 206917 INFO (qtp1887539743-1671) [n:127.0.0.1:45539_solr c:hdfsbackuprestore s:shard2 r:core_node12 x:hdfsbackuprestore_shard2_replica_t10] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 206917 INFO (qtp1887539743-1671) [n:127.0.0.1:45539_solr c:hdfsbackuprestore s:shard2 r:core_node12 x:hdfsbackuprestore_shard2_replica_t10] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
[junit4] 2> 206917 INFO (qtp1887539743-1671) [n:127.0.0.1:45539_solr c:hdfsbackuprestore s:shard2 r:core_node12 x:hdfsbackuprestore_shard2_replica_t10] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1588291119956885504
[junit4] 2> 206918 INFO (searcherExecutor-614-thread-1-processing-n:127.0.0.1:45539_solr x:hdfsbackuprestore_shard2_replica_t10 s:shard2 c:hdfsbackuprestore r:core_node12) [n:127.0.0.1:45539_solr c:hdfsbackuprestore s:shard2 r:core_node12 x:hdfsbackuprestore_shard2_replica_t10] o.a.s.c.SolrCore [hdfsbackuprestore_shard2_replica_t10] Registered new searcher Searcher@25422840[hdfsbackuprestore_shard2_replica_t10] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 206919 INFO (qtp1887539743-1675) [n:127.0.0.1:45539_solr c:hdfsbackuprestore s:shard2 r:core_node9 x:hdfsbackuprestore_shard2_replica_n6] o.a.s.s.SolrIndexSearcher Opening [Searcher@10c55a98[hdfsbackuprestore_shard2_replica_n6] main]
[junit4] 2> 206919 INFO (qtp1171428704-1684) [n:127.0.0.1:40065_solr c:hdfsbackuprestore s:shard1 r:core_node3 x:hdfsbackuprestore_shard1_replica_n1] o.a.s.s.SolrIndexSearcher Opening [Searcher@70c33a4[hdfsbackuprestore_shard1_replica_n1] main]
[junit4] 2> 206919 INFO (qtp1887539743-1675) [n:127.0.0.1:45539_solr c:hdfsbackuprestore s:shard2 r:core_node9 x:hdfsbackuprestore_shard2_replica_n6] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 206920 INFO (qtp1887539743-1675) [n:127.0.0.1:45539_solr c:hdfsbackuprestore s:shard2 r:core_node9 x:hdfsbackuprestore_shard2_replica_n6] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 206920 INFO (qtp1171428704-1684) [n:127.0.0.1:40065_solr c:hdfsbackuprestore s:shard1 r:core_node3 x:hdfsbackuprestore_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 206920 INFO (qtp1887539743-1675) [n:127.0.0.1:45539_solr c:hdfsbackuprestore s:shard2 r:core_node9 x:hdfsbackuprestore_shard2_replica_n6] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
[junit4] 2> 206920 INFO (qtp1171428704-1684) [n:127.0.0.1:40065_solr c:hdfsbackuprestore s:shard1 r:core_node3 x:hdfsbackuprestore_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 206921 INFO (searcherExecutor-613-thread-1-processing-n:127.0.0.1:45539_solr x:hdfsbackuprestore_shard2_replica_n6 s:shard2 c:hdfsbackuprestore r:core_node9) [n:127.0.0.1:45539_solr c:hdfsbackuprestore s:shard2 r:core_node9 x:hdfsbackuprestore_shard2_replica_n6] o.a.s.c.SolrCore [hdfsbackuprestore_shard2_replica_n6] Registered new searcher Searcher@10c55a98[hdfsbackuprestore_shard2_replica_n6] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 206921 INFO (qtp1887539743-1675) [n:127.0.0.1:45539_solr c:hdfsbackuprestore s:shard2 r:core_node9 x:hdfsbackuprestore_shard2_replica_n6] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1588291119961079808
[junit4] 2> 206921 INFO (qtp1171428704-1684) [n:127.0.0.1:40065_solr c:hdfsbackuprestore s:shard1 r:core_node3 x:hdfsbackuprestore_shard1_replica_n1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
[junit4] 2> 206921 INFO (searcherExecutor-611-thread-1-processing-n:127.0.0.1:40065_solr x:hdfsbackuprestore_shard1_replica_n1 s:shard1 c:hdfsbackuprestore r:core_node3) [n:127.0.0.1:40065_solr c:hdfsbackuprestore s:shard1 r:core_node3 x:hdfsbackuprestore_shard1_replica_n1] o.a.s.c.SolrCore [hdfsbackuprestore_shard1_replica_n1] Registered new searcher Searcher@70c33a4[hdfsbackuprestore_shard1_replica_n1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 206921 INFO (qtp1171428704-1684) [n:127.0.0.1:40065_solr c:hdfsbackuprestore s:shard1 r:core_node3 x:hdfsbackuprestore_shard1_replica_n1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1588291119961079808
[junit4] 2> 206926 INFO (qtp1887539743-1747) [n:127.0.0.1:45539_solr c:hdfsbackuprestore s:shard1 r:core_node5 x:hdfsbackuprestore_shard1_replica_n2] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 206926 INFO (qtp1887539743-1747) [n:127.0.0.1:45539_solr c:hdfsbackuprestore s:shard1 r:core_node5 x:hdfsbackuprestore_shard1_replica_n2] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 206927 INFO (qtp1887539743-1747) [n:127.0.0.1:45539_solr c:hdfsbackuprestore s:shard1 r:core_node5 x:hdfsbackuprestore_shard1_replica_n2] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 206927 INFO (qtp1887539743-1747) [n:127.0.0.1:45539_solr c:hdfsbackuprestore s:shard1 r:core_node5 x:hdfsbackuprestore_shard1_replica_n2] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 206929 INFO (qtp1887539743-1747) [n:127.0.0.1:45539_solr c:hdfsbackuprestore s:shard1 r:core_node5 x:hdfsbackuprestore_shard1_replica_n2] o.a.s.s.SolrIndexSearcher Opening [Searcher@378e1cda[hdfsbackuprestore_shard1_replica_n2] main]
[junit4] 2> 206930 INFO (qtp1887539743-1747) [n:127.0.0.1:45539_solr c:hdfsbackuprestore s:shard1 r:core_node5 x:hdfsbackuprestore_shard1_replica_n2] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 206930 INFO (qtp1887539743-1747) [n:127.0.0.1:45539_solr c:hdfsbackuprestore s:shard1 r:core_node5 x:hdfsbackuprestore_shard1_replica_n2] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 206931 INFO (qtp1887539743-1747) [n:127.0.0.1:45539_solr c:hdfsbackuprestore s:shard1 r:core_node5 x:hdfsbackuprestore_shard1_replica_n2] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
[junit4] 2> 206931 INFO (searcherExecutor-615-thread-1-processing-n:127.0.0.1:45539_solr x:hdfsbackuprestore_shard1_replica_n2 s:shard1 c:hdfsbackuprestore r:core_node5) [n:127.0.0.1:45539_solr c:hdfsbackuprestore s:shard1 r:core_node5 x:hdfsbackuprestore_shard1_replica_n2] o.a.s.c.SolrCore [hdfsbackuprestore_shard1_replica_n2] Registered new searcher Searcher@378e1cda[hdfsbackuprestore_shard1_replica_n2] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 206931 INFO (qtp1887539743-1747) [n:127.0.0.1:45539_solr c:hdfsbackuprestore s:shard1 r:core_node5 x:hdfsbackuprestore_shard1_replica_n2] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1588291119971565568
[junit4] 2> 207006 INFO (zkCallback-308-thread-2-processing-n:127.0.0.1:45539_solr) [n:127.0.0.1:45539_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/hdfsbackuprestore/state.json] for collection [hdfsbackuprestore] has occurred - updating... (live nodes size: [2])
[junit4] 2> 207006 INFO (zkCallback-308-thread-3-processing-n:127.0.0.1:45539_solr) [n:127.0.0.1:45539_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/hdfsbackuprestore/state.json] for collection [hdfsbackuprestore] has occurred - updating... (live nodes size: [2])
[junit4] 2> 207006 INFO (zkCallback-310-thread-1-processing-n:127.0.0.1:40065_solr) [n:127.0.0.1:40065_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/hdfsbackuprestore/state.json] for collection [hdfsbackuprestore] has occurred - updating... (live nodes size: [2])
[junit4] 2> 207405 INFO (qtp1171428704-1743) [n:127.0.0.1:40065_solr c:hdfsbackuprestore s:shard2 r:core_node11 x:hdfsbackuprestore_shard2_replica_n8] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
[junit4] 2> 207405 INFO (qtp1171428704-1682) [n:127.0.0.1:40065_solr c:hdfsbackuprestore s:shard1 r:core_node7 x:hdfsbackuprestore_shard1_replica_t4] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
[junit4] 2> 207406 INFO (qtp1171428704-1743) [n:127.0.0.1:40065_solr c:hdfsbackuprestore s:shard2 r:core_node11 x:hdfsbackuprestore_shard2_replica_n8] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
[junit4] 2> 207406 INFO (qtp1171428704-1682) [n:127.0.0.1:40065_solr c:hdfsbackuprestore s:shard1 r:core_node7 x:hdfsbackuprestore_shard1_replica_t4] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
[junit4] 2> 207406 INFO (qtp1171428704-1743) [n:127.0.0.1:40065_solr c:hdfsbackuprestore s:shard2 r:core_node11 x:hdfsbackuprestore_shard2_replica_n8] o.a.s.c.SyncStrategy Sync replicas to https://127.0.0.1:40065/solr/hdfsbackuprestore_shard2_replica_n8/
[junit4] 2> 207406 INFO (qtp1171428704-1682) [n:127.0.0.1:40065_solr c:hdfsbackuprestore s:shard1 r:core_node7 x:hdfsbackuprestore_shard1_replica_t4] o.a.s.c.SyncStrategy Sync replicas to https://127.0.0.1:40065/solr/hdfsbackuprestore_shard1_replica_t4/
[junit4] 2> 207406 INFO (qtp1171428704-1682) [n:127.0.0.1:40065_solr c:hdfsbackuprestore s:shard1 r:core_node7 x:hdfsbackuprestore_shard1_replica_t4] o.a.s.u.PeerSync PeerSync: core=hdfsbackuprestore_shard1_replica_t4 url=https://127.0.0.1:40065/solr START replicas=[https://127.0.0.1:40065/solr/hdfsbackuprestore_shard1_replica_n1/, https://127.0.0.1:45539/solr/hdfsbackuprestore_shard1_replica_n2/] nUpdates=100
[junit4] 2> 207406 INFO (qtp1171428704-1743) [n:127.0.0.1:40065_solr c:hdfsbackuprestore s:shard2 r:core_node11 x:hdfsbackuprestore_shard2_replica_n8] o.a.s.u.PeerSync PeerSync: core=hdfsbackuprestore_shard2_replica_n8 url=https://127.0.0.1:40065/solr START replicas=[https://127.0.0.1:45539/solr/hdfsbackuprestore_shard2_replica_n6/, https://127.0.0.1:45539/solr/hdfsbackuprestore_shard2_replica_t10/] nUpdates=100
[junit4] 2> 207419 INFO (qtp1887539743-1683) [n:127.0.0.1:45539_solr c:hdfsbackuprestore s:shard2 r:core_node12 x:hdfsbackuprestore_shard2_replica_t10] o.a.s.c.S.Request [hdfsbackuprestore_shard2_replica_t10] webapp=/solr path=/get params={distrib=false&qt=/get&fingerprint=false&getVersions=100&wt=javabin&version=2} status=0 QTime=1
[junit4] 2> 207423 INFO (qtp1887539743-1677) [n:127.0.0.1:45539_solr c:hdfsbackuprestore s:shard1 r:core_node5 x:hdfsbackuprestore_shard1_replica_n2] o.a.s.c.S.Request [hdfsbackuprestore_shard1_replica_n2] webapp=/solr path=/get params={distrib=false&qt=/get&fingerprint=false&getVersions=100&wt=javabin&version=2} status=0 QTime=2
[junit4] 2> 207425 INFO (qtp1171428704-1742) [n:127.0.0.1:40065_solr c:hdfsbackuprestore s:shard1 r:core_node3 x:hdfsbackuprestore_shard1_replica_n1] o.a.s.c.S.Request [hdfsbackuprestore_shard1_replica_n1] webapp=/solr path=/get params={distrib=false&qt=/get&fingerprint=false&getVersions=100&wt=javabin&version=2} status=0 QTime=1
[junit4] 2> 207427 INFO (qtp1887539743-1685) [n:127.0.0.1:45539_solr c:hdfsbackuprestore s:shard2 r:core_node9 x:hdfsbackuprestore_shard2_replica_n6] o.a.s.c.S.Request [hdfsbackuprestore_shard2_replica_n6] webapp=/solr path=/get params={distrib=false&qt=/get&fingerprint=false&getVersions=100&wt=javabin&version=2} status=0 QTime=0
[junit4] 2> 207707 INFO (qtp1171428704-1682) [n:127.0.0.1:40065_solr c:hdfsbackuprestore s:shard1 r:core_node7 x:hdfsbackuprestore_shard1_replica_t4] o.a.s.u.PeerSync PeerSync: core=hdfsbackuprestore_shard1_replica_t4 url=https://127.0.0.1:40065/solr DONE. We have no versions. sync failed.
[junit4] 2> 207708 INFO (qtp1171428704-1682) [n:127.0.0.1:40065_solr c:hdfsbackuprestore s:shard1 r:core_node7 x:hdfsbackuprestore_shard1_replica_t4] o.a.s.c.SyncStrategy Leader's attempt to sync with shard failed, moving to the next candidate
[junit4] 2> 207708 INFO (qtp1171428704-1682) [n:127.0.0.1:40065_solr c:hdfsbackuprestore s:shard1 r:core_node7 x:hdfsbackuprestore_shard1_replica_t4] o.a.s.c.ShardLeaderElectionContext We failed sync, but we have no versions - we can't sync in that case - we were active before, so become leader anyway
[junit4] 2> 207708 INFO (qtp1171428704-1682) [n:127.0.0.1:40065_solr c:hdfsbackuprestore s:shard1 r:core_node7 x:hdfsbackuprestore_shard1_replica_t4] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
[junit4] 2> 207708 INFO (qtp1171428704-1743) [n:127.0.0.1:40065_solr c:hdfsbackuprestore s:shard2 r:core_node11 x:hdfsbackuprestore_shard2_replica_n8] o.a.s.u.PeerSync PeerSync: core=hdfsbackuprestore_shard2_replica_n8 url=https://127.0.0.1:40065/solr DONE. We have no versions. sync failed.
[junit4] 2> 207708 INFO (qtp1171428704-1743) [n:127.0.0.1:40065_solr c:hdfsbackuprestore s:shard2 r:core_node11 x:hdfsbackuprestore_shard2_replica_n8] o.a.s.c.SyncStrategy Leader's attempt to sync with shard failed, moving to the next candidate
[junit4] 2> 207708 INFO (qtp1171428704-1743) [n:127.0.0.1:40065_solr c:hdfsbackuprestore s:shard2 r:core_node11 x:hdfsbackuprestore_shard2_replica_n8] o.a.s.c.ShardLeaderElectionContext We failed sync, but we have no versions - we can't sync in that case - we were active before, so become leader anyway
[junit4] 2> 207708 INFO (qtp1171428704-1743) [n:127.0.0.1:40065_solr c:hdfsbackuprestore s:shard2 r:core_node11 x:hdfsbackuprestore_shard2_replica_n8] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
[junit4] 2> 207709 INFO (qtp1171428704-1682) [n:127.0.0.1:40065_solr c:hdfsbackuprestore s:shard1 r:core_node7 x:hdfsbackuprestore_shard1_replica_t4] o.a.s.c.ZkController hdfsbackuprestore_shard1_replica_t4 stopping background replication from leader
[junit4] 2> 207712 INFO (qtp1171428704-1682) [n:127.0.0.1:40065_solr c:hdfsbackuprestore s:shard1 r:core_node7 x:hdfsbackuprestore_shard1_replica_t4] o.a.s.c.ShardLeaderElectionContext I am the new leader: https://127.0.0.1:40065/solr/hdfsbackuprestore_shard1_replica_t4/ shard1
[junit4] 2> 207712 INFO (qtp1171428704-1743) [n:127.0.0.1:40065_solr c:hdfsbackuprestore s:shard2 r:core_node11 x:hdfsbackuprestore_shard2_replica_n8] o.a.s.c.ShardLeaderElectionContext I am the new leader: https://127.0.0.1:40065/solr/hdfsbackuprestore_shard2_replica_n8/ shard2
[junit4] 2> 207813 INFO (zkCallback-310-thread-1-processing-n:127.0.0.1:40065_solr) [n:127.0.0.1:40065_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/hdfsbackuprestore/state.json] for collection [hdfsbackuprestore] has occurred - updating... (live nodes size: [2])
[junit4] 2> 207813 INFO (zkCallback-308-thread-2-processing-n:127.0.0.1:45539_solr) [n:127.0.0.1:45539_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/hdfsbackuprestore/state.json] for collection [hdfsbackuprestore] has occurred - updating... (live nodes size: [2])
[junit4] 2> 207813 INFO (zkCallback-308-thread-3-processing-n:127.0.0.1:45539_solr) [n:127.0.0.1:45539_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/hdfsbackuprestore/state.json] for collection [hdfsbackuprestore] has occurred - updating... (live nodes size: [2])
[junit4] 2> 207863 INFO (qtp1171428704-1682) [n:127.0.0.1:40065_solr c:hdfsbackuprestore s:shard1 r:core_node7 x:hdfsbackuprestore_shard1_replica_t4] o.a.s.c.ZkController I am the leader, no recovery necessary
[junit4] 2> 207863 INFO (qtp1171428704-1743) [n:127.0.0.1:40065_solr c:hdfsbackuprestore s:shard2 r:core_node11 x:hdfsbackuprestore_shard2_replica_n8] o.a.s.c.ZkController I am the leader, no recovery necessary
[junit4] 2> 207871 INFO (qtp1171428704-1682) [n:127.0.0.1:40065_solr c:hdfsbackuprestore s:shard1 r:core_node7 x:hdfsbackuprestore_shard1_replica_t4] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=conf1&newCollection=true&collection=hdfsbackuprestore&version=2&replicaType=TLOG&property.customKey=customValue&coreNodeName=core_node7&name=hdfsbackuprestore_shard1_replica_t4&action=CREATE&numShards=2&shard=shard1&wt=javabin} status=0 QTime=2113
[junit4] 2> 207871 INFO (qtp1171428704-1743) [n:127.0.0.1:40065_solr c:hdfsbackuprestore s:shard2 r:core_node11 x:hdfsbackuprestore_shard2_replica_n8] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=conf1&newCollection=true&collection=hdfsbackuprestore&version=2&replicaType=NRT&property.customKey=customValue&coreNodeName=core_node11&name=hdfsbackuprestore_shard2_replica_n8&action=CREATE&numShards=2&shard=shard2&wt=javabin} status=0 QTime=2092
[junit4] 2> 207920 INFO (qtp1887539743-1671) [n:127.0.0.1:45539_solr c:hdfsbackuprestore s:shard2 r:core_node12 x:hdfsbackuprestore_shard2_replica_t10] o.a.s.c.ZkController hdfsbackuprestore_shard2_replica_t10 starting background replication from leader
[junit4] 2> 207920 INFO (qtp1887539743-1671) [n:127.0.0.1:45539_solr c:hdfsbackuprestore s:shard2 r:core_node12 x:hdfsbackuprestore_shard2_replica_t10] o.a.s.c.ReplicateFromLeader Will start replication from leader with poll interval: 00:00:03
[junit4] 2> 207921 INFO (qtp1887539743-1671) [n:127.0.0.1:45539_solr c:hdfsbackuprestore s:shard2 r:core_node12 x:hdfsbackuprestore_shard2_replica_t10] o.a.s.h.ReplicationHandler Poll scheduled at an interval of 3000ms
[junit4] 2> 207921 INFO (qtp1887539743-1671) [n:127.0.0.1:45539_solr c:hdfsbackuprestore s:shard2 r:core_node12 x:hdfsbackuprestore_shard2_replica_t10] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
[junit4] 2> 207922 INFO (qtp1887539743-1671) [n:127.0.0.1:45539_solr c:hdfsbackuprestore s:shard2 r:core_node12 x:hdfsbackuprestore_shard2_replica_t10] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=conf1&newCollection=true&collection=hdfsbackuprestore&version=2&replicaType=TLOG&property.customKey=customValue&coreNodeName=core_node12&name=hdfsbackuprestore_shard2_replica_t10&action=CREATE&numShards=2&shard=shard2&wt=javabin} status=0 QTime=2115
[junit4] 2> 207924 INFO (qtp1887539743-1675) [n:127.0.0.1:45539_solr c:hdfsbackuprestore s:shard2 r:core_node9 x:hdfsbackuprestore_shard2_replica_n6] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=conf1&newCollection=true&collection=hdfsbackuprestore&version=2&replicaType=NRT&property.customKey=customValue&coreNodeName=core_node9&name=hdfsbackuprestore_shard2_replica_n6&action=CREATE&numShards=2&shard=shard2&wt=javabin} status=0 QTime=2134
[junit4] 2> 207925 INFO (qtp1171428704-1684) [n:127.0.0.1:40065_solr c:hdfsbackuprestore s:shard1 r:core_node3 x:hdfsbackuprestore_shard1_replica_n1] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=conf1&newCollection=true&collection=hdfsbackuprestore&version=2&replicaType=NRT&property.customKey=customValue&coreNodeName=core_node3&name=hdfsbackuprestore_shard1_replica_n1&action=CREATE&numShards=2&shard=shard1&wt=javabin} status=0 QTime=2168
[junit4] 2> 207935 INFO (qtp1887539743-1747) [n:127.0.0.1:45539_solr c:hdfsbackuprestore s:shard1 r:core_node5 x:hdfsbackuprestore_shard1_replica_n2] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=conf1&newCollection=true&collection=hdfsbackuprestore&version=2&replicaType=NRT&property.customKey=customValue&coreNodeName=core_node5&name=hdfsbackuprestore_shard1_replica_n2&action=CREATE&numShards=2&shard=shard1&wt=javabin} status=0 QTime=2128
[junit4] 2> 207937 INFO (qtp1171428704-1686) [n:127.0.0.1:40065_solr ] o.a.s.h.a.CollectionsHandler Wait for new collection to be active for at most 30 seconds. Check all shard replicas
[junit4] 2> 208036 INFO (zkCallback-308-thread-3-processing-n:127.0.0.1:45539_solr) [n:127.0.0.1:45539_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/hdfsbackuprestore/state.json] for collection [hdfsbackuprestore] has occurred - updating... (live nodes size: [2])
[junit4] 2> 208036 INFO (zkCallback-308-thread-2-processing-n:127.0.0.1:45539_solr) [n:127.0.0.1:45539_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/hdfsbackuprestore/state.json] for collection [hdfsbackuprestore] has occurred - updating... (live nodes size: [2])
[junit4] 2> 208036 INFO (zkCallback-310-thread-1-processing-n:127.0.0.1:40065_solr) [n:127.0.0.1:40065_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/hdfsbackuprestore/state.json] for collection [hdfsbackuprestore] has occurred - updating... (live nodes size: [2])
[junit4] 2> 208937 INFO (qtp1171428704-1686) [n:127.0.0.1:40065_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={pullReplicas=0&replicationFactor=2&property.customKey=customValue&collection.configName=conf1&maxShardsPerNode=6&name=hdfsbackuprestore&nrtReplicas=2&action=CREATE&numShards=2&tlogReplicas=1&wt=javabin&version=2} status=0 QTime=3512
[junit4] 2> 208963 INFO (qtp1171428704-1743) [n:127.0.0.1:40065_solr c:hdfsbackuprestore s:shard1 r:core_node3 x:hdfsbackuprestore_shard1_replica_n1] o.a.s.u.p.LogUpdateProcessorFactory [hdfsbackuprestore_shard1_replica_n1] webapp=/solr path=/update params={update.distrib=FROMLEADER&distrib.from=https://127.0.0.1:40065/solr/hdfsbackuprestore_shard1_replica_t4/&wt=javabin&version=2}{add=[0 (1588291122080251904), 1 (1588291122089689088), 4 (1588291122089689089), 8 (1588291122090737664), 10 (1588291122090737665), 11 (1588291122091786240), 12 (1588291122091786241), 13 (1588291122092834816), 14 (1588291122094931968), 15 (1588291122094931969), ... (12 adds)]} 0 10
[junit4] 2> 208964 INFO (qtp1887539743-1746) [n:127.0.0.1:45539_solr c:hdfsbackuprestore s:shard2 r:core_node12 x:hdfsbackuprestore_shard2_replica_t10] o.a.s.u.p.LogUpdateProcessorFactory [hdfsbackuprestore_shard2_replica_t10] webapp=/solr path=/update params={update.distrib=FROMLEADER&distrib.from=https://127.0.0.1:40065/solr/hdfsbackuprestore_shard2_replica_n8/&wt=javabin&version=2}{add=[2 (1588291122086543360), 3 (1588291122097029120), 5 (1588291122097029121), 6 (1588291122097029122), 7 (1588291122100174848), 9 (1588291122101223424), 17 (1588291122101223425), 18 (1588291122101223426), 19 (1588291122101223427)]} 0 2
[junit4] 2> 208969 INFO (qtp1887539743-1677) [n:127.0.0.1:45539_solr c:hdfsbackuprestore s:shard1 r:core_node5 x:hdfsbackuprestore_shard1_replica_n2] o.a.s.u.p.LogUpdateProcessorFactory [hdfsbackuprestore_shard1_replica_n2] webapp=/solr path=/update params={update.distrib=FROMLEADER&distrib.from=https://127.0.0.1:40065/solr/hdfsbackuprestore_shard1_replica_t4/&wt=javabin&version=2}{add=[0 (1588291122080251904), 1 (1588291122089689088), 4 (1588291122089689089), 8 (1588291122090737664), 10 (1588291122090737665), 11 (1588291122091786240), 12 (1588291122091786241), 13 (1588291122092834816), 14 (1588291122094931968), 15 (1588291122094931969), ... (12 adds)]} 0 16
[junit4] 2> 208969 INFO (qtp1887539743-1683) [n:127.0.0.1:45539_solr c:hdfsbackuprestore s:shard2 r:core_node9 x:hdfsbackuprestore_shard2_replica_n6] o.a.s.u.p.LogUpdateProcessorFactory [hdfsbackuprestore_shard2_replica_n6] webapp=/solr path=/update params={update.distrib=FROMLEADER&distrib.from=https://127.0.0.1:40065/solr/hdfsbackuprestore_shard2_replica_n8/&wt=javabin&version=2}{add=[2 (1588291122086543360), 3 (1588291122097029120), 5 (1588291122097029121), 6 (1588291122097029122), 7 (1588291122100174848), 9 (1588291122101223424), 17 (1588291122101223425), 18 (1588291122101223426), 19 (1588291122101223427)]} 0 9
[junit4] 2> 208970 INFO (qtp1171428704-1672) [n:127.0.0.1:40065_solr c:hdfsbackuprestore s:shard1 r:core_node7 x:hdfsbackuprestore_shard1_replica_t4] o.a.s.u.p.LogUpdateProcessorFactory [hdfsbackuprestore_shard1_replica_t4] webapp=/solr path=/update params={wt=javabin&version=2}{add=[0 (1588291122080251904), 1 (1588291122089689088), 4 (1588291122089689089), 8 (1588291122090737664), 10 (1588291122090737665), 11 (1588291122091786240), 12 (1588291122091786241), 13 (1588291122092834816), 14 (1588291122094931968), 15 (1588291122094931969), ... (12 adds)]} 0 28
[junit4] 2> 208970 INFO (qtp1171428704-1682) [n:127.0.0.1:40065_solr c:hdfsbackuprestore s:shard2 r:core_node11 x:hdfsbackuprestore_shard2_replica_n8] o.a.s.u.p.LogUpdateProcessorFactory [hdfsbackuprestore_shard2_replica_n8] webapp=/solr path=/update params={wt=javabin&version=2}{add=[2 (1588291122086543360), 3 (1588291122097029120), 5 (1588291122097029121), 6 (1588291122097029122), 7 (1588291122100174848), 9 (1588291122101223424), 17 (1588291122101223425), 18 (1588291122101223426), 19 (1588291122101223427)]} 0 22
[junit4] 2> 208974 INFO (qtp1887539743-1771) [n:127.0.0.1:45539_solr c:hdfsbackuprestore s:shard2 r:core_node9 x:hdfsbackuprestore_shard2_replica_n6] o.a.s.u.DirectUpdateHandler2 start commit{_version_=1588291122113806336,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4] 2> 208974 INFO (qtp1171428704-1684) [n:127.0.0.1:40065_solr c:hdfsbackuprestore s:shard1 r:core_node7 x:hdfsbackuprestore_shard1_replica_t4] o.a.s.u.DirectUpdateHandler2 start commit{_version_=1588291122113806336,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4] 2> 208974 INFO (qtp1887539743-1771) [n:127.0.0.1:45539_solr c:hdfsbackuprestore s:shard2 r:core_node9 x:hdfsbackuprestore_shard2_replica_n6] o.a.s.u.SolrIndexWriter Calling setCommitData with IW:org.apache.solr.update.SolrIndexWriter@656e2c78 commitCommandVersion:1588291122113806336
[junit4] 2> 208975 INFO (qtp1171428704-1684) [n:127.0.0.1:40065_solr c:hdfsbackuprestore s:shard1 r:core_node7 x:hdfsbackuprestore_shard1_replica_t4] o.a.s.u.SolrIndexWriter Calling setCommitData with IW:org.apache.solr.update.SolrIndexWriter@33053554 commitCommandVersion:1588291122113806336
[junit4] 2> 208979 INFO (qtp1887539743-1685) [n:127.0.0.1:45539_solr c:hdfsbackuprestore s:shard1 r:core_node5 x:hdfsbackuprestore_shard1_replica_n2] o.a.s.u.DirectUpdateHandler2 start commit{_version_=1588291122119049216,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4] 2> 208979 INFO (qtp1887539743-1
[...truncated too long message...]
it4] 2> 257347 INFO (coreCloseExecutor-458-thread-1) [n:127.0.0.1:33887_solr c:collection1 s:shard1 r:core_node2 x:collection1_shard1_replica_n1] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@3413e33f: rootName = solr_33887, domain = solr.core.collection1.shard1.replica_n1, service url = null, agent id = null] for registry solr.core.collection1.shard1.replica_n1 / com.codahale.metrics.MetricRegistry@6eccc50b
[junit4] 2> 257347 INFO (zkCallback-213-thread-3-processing-n:127.0.0.1:33887_solr) [n:127.0.0.1:33887_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (0)
[junit4] 2> 257348 INFO (zkCallback-215-thread-4-processing-n:127.0.0.1:42197_solr) [n:127.0.0.1:42197_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (0)
[junit4] 2> 257355 INFO (coreCloseExecutor-455-thread-1) [n:127.0.0.1:44111_solr c:collection1 s:shard1 r:core_node4 x:collection1_shard1_replica_n3] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.collection1.shard1.leader, tag=1581781992
[junit4] 2> 257357 INFO (coreCloseExecutor-456-thread-1) [n:127.0.0.1:42197_solr c:collection1 s:shard1 r:core_node8 x:collection1_shard1_replica_n7] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.collection1.shard1.leader, tag=2126708523
[junit4] 2> 257358 INFO (jetty-closer-184-thread-4) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@7690dbe{/solr,null,UNAVAILABLE}
[junit4] 2> 257359 INFO (jetty-closer-184-thread-2) [ ] o.a.s.c.Overseer Overseer (id=99268192019742730-127.0.0.1:42197_solr-n_0000000000) closing
[junit4] 2> 257359 INFO (OverseerStateUpdate-99268192019742730-127.0.0.1:42197_solr-n_0000000000) [n:127.0.0.1:42197_solr ] o.a.s.c.Overseer Overseer Loop exiting : 127.0.0.1:42197_solr
[junit4] 2> 257359 INFO (coreCloseExecutor-458-thread-1) [n:127.0.0.1:33887_solr c:collection1 s:shard1 r:core_node2 x:collection1_shard1_replica_n1] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.collection1.shard1.leader, tag=1658691855
[junit4] 2> 257359 WARN (OverseerAutoScalingTriggerThread-99268192019742730-127.0.0.1:42197_solr-n_0000000000) [n:127.0.0.1:42197_solr ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread woken up but we are closed, exiting.
[junit4] 2> 257361 WARN (zkCallback-219-thread-3-processing-n:127.0.0.1:36055_solr) [n:127.0.0.1:36055_solr ] o.a.s.c.LeaderElector Our node is no longer in line to be leader
[junit4] 2> 257361 INFO (jetty-closer-184-thread-2) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@216c6533{/solr,null,UNAVAILABLE}
[junit4] 2> 257362 INFO (zkCallback-219-thread-5-processing-n:127.0.0.1:36055_solr) [n:127.0.0.1:36055_solr ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:36055_solr
[junit4] 2> 257362 INFO (jetty-closer-184-thread-3) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@6aab6fb8{/solr,null,UNAVAILABLE}
[junit4] 2> 262317 INFO (recoveryExecutor-205-thread-1-processing-n:127.0.0.1:36055_solr x:collection1_shard1_replica_n9 s:shard1 c:collection1 r:core_node10) [n:127.0.0.1:36055_solr c:collection1 s:shard1 r:core_node10 x:collection1_shard1_replica_n9] o.a.s.c.RecoveryStrategy RecoveryStrategy has been closed
[junit4] 2> 262317 INFO (recoveryExecutor-205-thread-1-processing-n:127.0.0.1:36055_solr x:collection1_shard1_replica_n9 s:shard1 c:collection1 r:core_node10) [n:127.0.0.1:36055_solr c:collection1 s:shard1 r:core_node10 x:collection1_shard1_replica_n9] o.a.s.c.RecoveryStrategy Finished recovery process, successful=[false]
[junit4] 2> 262317 INFO (recoveryExecutor-205-thread-1-processing-n:127.0.0.1:36055_solr x:collection1_shard1_replica_n9 s:shard1 c:collection1 r:core_node10) [n:127.0.0.1:36055_solr c:collection1 s:shard1 r:core_node10 x:collection1_shard1_replica_n9] o.a.s.c.SolrCore [collection1_shard1_replica_n9] CLOSING SolrCore org.apache.solr.core.SolrCore@31cee06d
[junit4] 2> 262317 INFO (recoveryExecutor-205-thread-1-processing-n:127.0.0.1:36055_solr x:collection1_shard1_replica_n9 s:shard1 c:collection1 r:core_node10) [n:127.0.0.1:36055_solr c:collection1 s:shard1 r:core_node10 x:collection1_shard1_replica_n9] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.collection1.shard1.replica_n9, tag=835641453
[junit4] 2> 262318 INFO (recoveryExecutor-205-thread-1-processing-n:127.0.0.1:36055_solr x:collection1_shard1_replica_n9 s:shard1 c:collection1 r:core_node10) [n:127.0.0.1:36055_solr c:collection1 s:shard1 r:core_node10 x:collection1_shard1_replica_n9] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@524b13cb: rootName = solr_36055, domain = solr.core.collection1.shard1.replica_n9, service url = null, agent id = null] for registry solr.core.collection1.shard1.replica_n9 / com.codahale.metrics.MetricRegistry@7e35bde2
[junit4] 2> 262324 INFO (recoveryExecutor-205-thread-1-processing-n:127.0.0.1:36055_solr x:collection1_shard1_replica_n9 s:shard1 c:collection1 r:core_node10) [n:127.0.0.1:36055_solr c:collection1 s:shard1 r:core_node10 x:collection1_shard1_replica_n9] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.collection1.shard1.leader, tag=835641453
[junit4] 2> 262324 WARN (recoveryExecutor-205-thread-1-processing-n:127.0.0.1:36055_solr x:collection1_shard1_replica_n9 s:shard1 c:collection1 r:core_node10) [n:127.0.0.1:36055_solr c:collection1 s:shard1 r:core_node10 x:collection1_shard1_replica_n9] o.a.s.c.RecoveryStrategy Stopping recovery for core=[collection1_shard1_replica_n9] coreNodeName=[core_node10]
[junit4] 2> 262326 INFO (jetty-closer-184-thread-1) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@57949819{/solr,null,UNAVAILABLE}
[junit4] 2> 262328 ERROR (SUITE-AssignBackwardCompatibilityTest-seed#[23F1118A31631A7F]-worker) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 262328 INFO (SUITE-AssignBackwardCompatibilityTest-seed#[23F1118A31631A7F]-worker) [ ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:39543 39543
[junit4] 2> 272374 INFO (Thread-162) [ ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:39543 39543
[junit4] 2> 272374 WARN (Thread-162) [ ] o.a.s.c.ZkTestServer Watch limit violations:
[junit4] 2> Maximum concurrent create/delete watches above limit:
[junit4] 2>
[junit4] 2> 5 /solr/configs/collection1
[junit4] 2> 5 /solr/aliases.json
[junit4] 2> 5 /solr/configs/collection1/managed-schema
[junit4] 2> 5 /solr/clusterprops.json
[junit4] 2> 4 /solr/security.json
[junit4] 2>
[junit4] 2> Maximum concurrent data watches above limit:
[junit4] 2>
[junit4] 2> 32 /solr/collections/collection1/state.json
[junit4] 2> 5 /solr/clusterstate.json
[junit4] 2> 3 /solr/collections/collection1/leader_elect/shard1/election/99268192019742730-core_node8-n_0000000001
[junit4] 2> 2 /solr/overseer_elect/election/99268192019742732-127.0.0.1:36055_solr-n_0000000001
[junit4] 2>
[junit4] 2> Maximum concurrent children watches above limit:
[junit4] 2>
[junit4] 2> 5 /solr/live_nodes
[junit4] 2> 5 /solr/collections
[junit4] 2>
[junit4] 2> NOTE: leaving temporary files on disk at: /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.AssignBackwardCompatibilityTest_23F1118A31631A7F-001
[junit4] 2> NOTE: test params are: codec=Asserting(Lucene70): {}, docValues:{}, maxPointsInLeafNode=1543, maxMBSortInHeap=6.583153993051027, sim=RandomSimilarity(queryNorm=true): {}, locale=ms-BN, timezone=America/Juneau
[junit4] 2> NOTE: Linux 4.10.0-40-generic amd64/Oracle Corporation 9.0.1 (64-bit)/cpus=8,threads=1,free=262072616,total=525336576
[junit4] 2> NOTE: All tests run in this JVM: [DocValuesTest, DirectoryFactoryTest, TermsComponentTest, CoreMergeIndexesAdminHandlerTest, IgnoreCommitOptimizeUpdateProcessorFactoryTest, ManagedSchemaRoundRobinCloudTest, BaseCdcrDistributedZkTest, TestOverriddenPrefixQueryForCustomFieldType, LeaderInitiatedRecoveryOnShardRestartTest, CleanupOldIndexTest, TestPostingsSolrHighlighter, TestSolrDeletionPolicy1, DateRangeFieldTest, URLClassifyProcessorTest, TestNumericTerms32, ResourceLoaderTest, RAMDirectoryFactoryTest, TestEmbeddedSolrServerSchemaAPI, TestLargeCluster, SyncSliceTest, AssignBackwardCompatibilityTest]
[junit4] Completed [67/764 (2!)] on J0 in 110.39s, 1 test, 1 failure <<< FAILURES!
[...truncated 2350 lines...]
[junit4] JVM J1: stdout was not empty, see: /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/temp/junit4-J1-20171231_092358_8437864630989356919796.sysout
[junit4] >>> JVM J1 emitted unexpected output (verbatim) ----
[junit4] java.lang.OutOfMemoryError: Java heap space
[junit4] Dumping heap to /home/jenkins/workspace/Lucene-Solr-7.x-Linux/heapdumps/java_pid32029.hprof ...
[junit4] Heap dump file created [142179397 bytes in 0.453 secs]
[junit4] <<< JVM J1: EOF ----
[...truncated 8571 lines...]
BUILD FAILED
/home/jenkins/workspace/Lucene-Solr-7.x-Linux/build.xml:835: The following error occurred while executing this line:
/home/jenkins/workspace/Lucene-Solr-7.x-Linux/build.xml:787: Some of the tests produced a heap dump, but did not fail. Maybe a suppressed OutOfMemoryError? Dumps created:
* java_pid32029.hprof
Total time: 80 minutes 33 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
[WARNINGS] Skipping publisher since build result is FAILURE
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any
[JENKINS-EA] Lucene-Solr-7.x-Linux (64bit/jdk-10-ea+37) - Build #
1091 - Still unstable!
Posted by Policeman Jenkins Server <je...@thetaphi.de>.
Build: https://jenkins.thetaphi.de/job/Lucene-Solr-7.x-Linux/1091/
Java: 64bit/jdk-10-ea+37 -XX:+UseCompressedOops -XX:+UseParallelGC
2 tests failed.
FAILED: org.apache.solr.cloud.MoveReplicaHDFSTest.testFailedMove
Error Message:
No live SolrServers available to handle this request:[http://127.0.0.1:44913/solr/MoveReplicaHDFSTest_failed_coll_true, http://127.0.0.1:40241/solr/MoveReplicaHDFSTest_failed_coll_true]
Stack Trace:
org.apache.solr.client.solrj.SolrServerException: No live SolrServers available to handle this request:[http://127.0.0.1:44913/solr/MoveReplicaHDFSTest_failed_coll_true, http://127.0.0.1:40241/solr/MoveReplicaHDFSTest_failed_coll_true]
at __randomizedtesting.SeedInfo.seed([C3CBB6E52F2E74B5:6906651798FDA165]:0)
at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:462)
at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1104)
at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:884)
at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:991)
at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:817)
at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:194)
at org.apache.solr.client.solrj.SolrClient.query(SolrClient.java:942)
at org.apache.solr.cloud.MoveReplicaTest.testFailedMove(MoveReplicaTest.java:306)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:564)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.base/java.lang.Thread.run(Thread.java:844)
Caused by: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:40241/solr/MoveReplicaHDFSTest_failed_coll_true: no servers hosting shard: shard1
at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:643)
at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:255)
at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:244)
at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:483)
at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:413)
... 45 more
FAILED: org.apache.solr.cloud.autoscaling.sim.TestExecutePlanAction.testIntegration
Error Message:
Stack Trace:
java.util.ConcurrentModificationException
at __randomizedtesting.SeedInfo.seed([C3CBB6E52F2E74B5:73AAB8C90A11D590]:0)
at java.base/java.util.ArrayList$Itr.checkForComodification(ArrayList.java:937)
at java.base/java.util.ArrayList$Itr.next(ArrayList.java:891)
at org.apache.solr.cloud.autoscaling.sim.SimSolrCloudTestCase.tearDown(SimSolrCloudTestCase.java:141)
at jdk.internal.reflect.GeneratedMethodAccessor11.invoke(Unknown Source)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:564)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:992)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.base/java.lang.Thread.run(Thread.java:844)
Build Log:
[...truncated 12134 lines...]
[junit4] Suite: org.apache.solr.cloud.MoveReplicaHDFSTest
[junit4] 2> 405430 INFO (SUITE-MoveReplicaHDFSTest-seed#[C3CBB6E52F2E74B5]-worker) [ ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
[junit4] 2> Creating dataDir: /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J2/temp/solr.cloud.MoveReplicaHDFSTest_C3CBB6E52F2E74B5-001/init-core-data-001
[junit4] 2> 405431 WARN (SUITE-MoveReplicaHDFSTest-seed#[C3CBB6E52F2E74B5]-worker) [ ] o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=4 numCloses=4
[junit4] 2> 405431 INFO (SUITE-MoveReplicaHDFSTest-seed#[C3CBB6E52F2E74B5]-worker) [ ] o.a.s.SolrTestCaseJ4 Using TrieFields (NUMERIC_POINTS_SYSPROP=false) w/NUMERIC_DOCVALUES_SYSPROP=false
[junit4] 2> 405431 INFO (SUITE-MoveReplicaHDFSTest-seed#[C3CBB6E52F2E74B5]-worker) [ ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (true) via: @org.apache.solr.util.RandomizeSSL(reason="", ssl=0.0/0.0, value=0.0/0.0, clientAuth=0.0/0.0)
[junit4] 2> 405431 INFO (SUITE-MoveReplicaHDFSTest-seed#[C3CBB6E52F2E74B5]-worker) [ ] o.a.s.c.MiniSolrCloudCluster Starting cluster of 4 servers in /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J2/temp/solr.cloud.MoveReplicaHDFSTest_C3CBB6E52F2E74B5-001/tempDir-001
[junit4] 2> 405431 INFO (SUITE-MoveReplicaHDFSTest-seed#[C3CBB6E52F2E74B5]-worker) [ ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
[junit4] 2> 405432 INFO (Thread-588) [ ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 405432 INFO (Thread-588) [ ] o.a.s.c.ZkTestServer Starting server
[junit4] 2> 405434 ERROR (Thread-588) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 405532 INFO (SUITE-MoveReplicaHDFSTest-seed#[C3CBB6E52F2E74B5]-worker) [ ] o.a.s.c.ZkTestServer start zk server on port:44389
[junit4] 2> 405533 INFO (zkConnectionManagerCallback-563-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 405534 INFO (jetty-launcher-560-thread-2) [ ] o.e.j.s.Server jetty-9.3.20.v20170531
[junit4] 2> 405534 INFO (jetty-launcher-560-thread-1) [ ] o.e.j.s.Server jetty-9.3.20.v20170531
[junit4] 2> 405535 INFO (jetty-launcher-560-thread-4) [ ] o.e.j.s.Server jetty-9.3.20.v20170531
[junit4] 2> 405536 INFO (jetty-launcher-560-thread-3) [ ] o.e.j.s.Server jetty-9.3.20.v20170531
[junit4] 2> 405536 INFO (jetty-launcher-560-thread-3) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@531e40af{/solr,null,AVAILABLE}
[junit4] 2> 405537 INFO (jetty-launcher-560-thread-3) [ ] o.e.j.s.AbstractConnector Started ServerConnector@46cf1b91{HTTP/1.1,[http/1.1]}{127.0.0.1:40241}
[junit4] 2> 405537 INFO (jetty-launcher-560-thread-3) [ ] o.e.j.s.Server Started @407286ms
[junit4] 2> 405537 INFO (jetty-launcher-560-thread-3) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=40241}
[junit4] 2> 405537 ERROR (jetty-launcher-560-thread-3) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 405537 INFO (jetty-launcher-560-thread-3) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version 7.3.0
[junit4] 2> 405537 INFO (jetty-launcher-560-thread-3) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 405537 INFO (jetty-launcher-560-thread-3) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 405537 INFO (jetty-launcher-560-thread-3) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2017-12-31T22:20:07.151527Z
[junit4] 2> 405538 INFO (jetty-launcher-560-thread-1) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@175d05d1{/solr,null,AVAILABLE}
[junit4] 2> 405538 INFO (jetty-launcher-560-thread-1) [ ] o.e.j.s.AbstractConnector Started ServerConnector@5fc693dd{HTTP/1.1,[http/1.1]}{127.0.0.1:38399}
[junit4] 2> 405538 INFO (jetty-launcher-560-thread-1) [ ] o.e.j.s.Server Started @407287ms
[junit4] 2> 405538 INFO (jetty-launcher-560-thread-1) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=38399}
[junit4] 2> 405538 ERROR (jetty-launcher-560-thread-1) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 405538 INFO (jetty-launcher-560-thread-1) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version 7.3.0
[junit4] 2> 405538 INFO (jetty-launcher-560-thread-1) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 405538 INFO (jetty-launcher-560-thread-1) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 405538 INFO (jetty-launcher-560-thread-1) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2017-12-31T22:20:07.152693Z
[junit4] 2> 405541 INFO (jetty-launcher-560-thread-2) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@2bac944e{/solr,null,AVAILABLE}
[junit4] 2> 405541 INFO (jetty-launcher-560-thread-2) [ ] o.e.j.s.AbstractConnector Started ServerConnector@2a527d20{HTTP/1.1,[http/1.1]}{127.0.0.1:43723}
[junit4] 2> 405541 INFO (jetty-launcher-560-thread-2) [ ] o.e.j.s.Server Started @407291ms
[junit4] 2> 405541 INFO (jetty-launcher-560-thread-2) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=43723}
[junit4] 2> 405542 ERROR (jetty-launcher-560-thread-2) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 405542 INFO (jetty-launcher-560-thread-2) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version 7.3.0
[junit4] 2> 405542 INFO (jetty-launcher-560-thread-2) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 405542 INFO (jetty-launcher-560-thread-2) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 405542 INFO (jetty-launcher-560-thread-2) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2017-12-31T22:20:07.156139Z
[junit4] 2> 405542 INFO (zkConnectionManagerCallback-565-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 405543 INFO (zkConnectionManagerCallback-569-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 405543 INFO (jetty-launcher-560-thread-3) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 405543 INFO (jetty-launcher-560-thread-2) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 405543 INFO (jetty-launcher-560-thread-4) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@4950924b{/solr,null,AVAILABLE}
[junit4] 2> 405543 INFO (zkConnectionManagerCallback-567-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 405543 INFO (jetty-launcher-560-thread-4) [ ] o.e.j.s.AbstractConnector Started ServerConnector@5af60239{HTTP/1.1,[http/1.1]}{127.0.0.1:45429}
[junit4] 2> 405544 INFO (jetty-launcher-560-thread-4) [ ] o.e.j.s.Server Started @407293ms
[junit4] 2> 405544 INFO (jetty-launcher-560-thread-4) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=45429}
[junit4] 2> 405544 ERROR (jetty-launcher-560-thread-4) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 405544 INFO (jetty-launcher-560-thread-4) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version 7.3.0
[junit4] 2> 405544 INFO (jetty-launcher-560-thread-4) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 405544 INFO (jetty-launcher-560-thread-4) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 405544 INFO (jetty-launcher-560-thread-4) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2017-12-31T22:20:07.158259Z
[junit4] 2> 405545 INFO (jetty-launcher-560-thread-1) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 405546 INFO (zkConnectionManagerCallback-571-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 405547 INFO (jetty-launcher-560-thread-4) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 405547 INFO (jetty-launcher-560-thread-3) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:44389/solr
[junit4] 2> 405547 INFO (jetty-launcher-560-thread-2) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:44389/solr
[junit4] 2> 405548 INFO (zkConnectionManagerCallback-577-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 405558 INFO (zkConnectionManagerCallback-579-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 405558 WARN (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [ ] o.a.z.s.NIOServerCnxn caught end of stream exception
[junit4] 2> EndOfStreamException: Unable to read additional data from client sessionid 0x160aea91e070006, likely client has closed socket
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:239)
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:203)
[junit4] 2> at java.base/java.lang.Thread.run(Thread.java:844)
[junit4] 2> 405559 INFO (jetty-launcher-560-thread-1) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:44389/solr
[junit4] 2> 405559 INFO (zkConnectionManagerCallback-582-thread-1-processing-n:127.0.0.1:43723_solr) [n:127.0.0.1:43723_solr ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 405560 INFO (jetty-launcher-560-thread-4) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:44389/solr
[junit4] 2> 405561 INFO (zkConnectionManagerCallback-588-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 405561 INFO (zkConnectionManagerCallback-585-thread-1-processing-n:127.0.0.1:40241_solr) [n:127.0.0.1:40241_solr ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 405561 INFO (zkConnectionManagerCallback-591-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 405563 INFO (zkConnectionManagerCallback-595-thread-1-processing-n:127.0.0.1:45429_solr) [n:127.0.0.1:45429_solr ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 405563 INFO (zkConnectionManagerCallback-593-thread-1-processing-n:127.0.0.1:38399_solr) [n:127.0.0.1:38399_solr ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 405610 INFO (jetty-launcher-560-thread-2) [n:127.0.0.1:43723_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 405610 INFO (jetty-launcher-560-thread-3) [n:127.0.0.1:40241_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 405610 INFO (jetty-launcher-560-thread-1) [n:127.0.0.1:38399_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 405610 INFO (jetty-launcher-560-thread-4) [n:127.0.0.1:45429_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 405610 INFO (jetty-launcher-560-thread-2) [n:127.0.0.1:43723_solr ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:43723_solr
[junit4] 2> 405610 INFO (jetty-launcher-560-thread-3) [n:127.0.0.1:40241_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:40241_solr
[junit4] 2> 405611 INFO (jetty-launcher-560-thread-4) [n:127.0.0.1:45429_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:45429_solr
[junit4] 2> 405611 INFO (jetty-launcher-560-thread-1) [n:127.0.0.1:38399_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:38399_solr
[junit4] 2> 405611 INFO (jetty-launcher-560-thread-2) [n:127.0.0.1:43723_solr ] o.a.s.c.Overseer Overseer (id=99271233178632199-127.0.0.1:43723_solr-n_0000000000) starting
[junit4] 2> 405626 INFO (zkCallback-584-thread-1-processing-n:127.0.0.1:40241_solr) [n:127.0.0.1:40241_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (3)
[junit4] 2> 405632 INFO (zkCallback-592-thread-1-processing-n:127.0.0.1:38399_solr) [n:127.0.0.1:38399_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (3)
[junit4] 2> 405636 INFO (zkCallback-594-thread-1-processing-n:127.0.0.1:45429_solr) [n:127.0.0.1:45429_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (3)
[junit4] 2> 405636 INFO (zkCallback-581-thread-1-processing-n:127.0.0.1:43723_solr) [n:127.0.0.1:43723_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (3)
[junit4] 2> 405645 INFO (jetty-launcher-560-thread-3) [n:127.0.0.1:40241_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_40241.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@48ad14a0
[junit4] 2> 405658 INFO (jetty-launcher-560-thread-2) [n:127.0.0.1:43723_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:43723_solr
[junit4] 2> 405662 INFO (zkCallback-581-thread-1-processing-n:127.0.0.1:43723_solr) [n:127.0.0.1:43723_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 405662 INFO (zkCallback-592-thread-1-processing-n:127.0.0.1:38399_solr) [n:127.0.0.1:38399_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 405662 INFO (zkCallback-584-thread-1-processing-n:127.0.0.1:40241_solr) [n:127.0.0.1:40241_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 405662 INFO (zkCallback-594-thread-1-processing-n:127.0.0.1:45429_solr) [n:127.0.0.1:45429_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 405663 INFO (jetty-launcher-560-thread-3) [n:127.0.0.1:40241_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_40241.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@48ad14a0
[junit4] 2> 405663 INFO (jetty-launcher-560-thread-3) [n:127.0.0.1:40241_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_40241.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@48ad14a0
[junit4] 2> 405664 INFO (jetty-launcher-560-thread-3) [n:127.0.0.1:40241_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J2/temp/solr.cloud.MoveReplicaHDFSTest_C3CBB6E52F2E74B5-001/tempDir-001/node3/.
[junit4] 2> 405672 INFO (jetty-launcher-560-thread-4) [n:127.0.0.1:45429_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_45429.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@48ad14a0
[junit4] 2> 405676 INFO (jetty-launcher-560-thread-1) [n:127.0.0.1:38399_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_38399.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@48ad14a0
[junit4] 2> 405678 INFO (jetty-launcher-560-thread-4) [n:127.0.0.1:45429_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_45429.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@48ad14a0
[junit4] 2> 405678 INFO (jetty-launcher-560-thread-4) [n:127.0.0.1:45429_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_45429.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@48ad14a0
[junit4] 2> 405679 INFO (jetty-launcher-560-thread-4) [n:127.0.0.1:45429_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J2/temp/solr.cloud.MoveReplicaHDFSTest_C3CBB6E52F2E74B5-001/tempDir-001/node4/.
[junit4] 2> 405682 INFO (jetty-launcher-560-thread-1) [n:127.0.0.1:38399_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_38399.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@48ad14a0
[junit4] 2> 405682 INFO (jetty-launcher-560-thread-1) [n:127.0.0.1:38399_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_38399.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@48ad14a0
[junit4] 2> 405683 INFO (jetty-launcher-560-thread-1) [n:127.0.0.1:38399_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J2/temp/solr.cloud.MoveReplicaHDFSTest_C3CBB6E52F2E74B5-001/tempDir-001/node1/.
[junit4] 2> 405695 INFO (jetty-launcher-560-thread-2) [n:127.0.0.1:43723_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_43723.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@48ad14a0
[junit4] 2> 405699 WARN (jetty-launcher-560-thread-2) [n:127.0.0.1:43723_solr ] o.a.s.m.r.j.JmxMetricsReporter Unable to register gauge
[junit4] 2> javax.management.InstanceNotFoundException: solr_43723:dom1=solr,dom2=jvm,name0=threads,name=deadlocks
[junit4] 2> at java.management/com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.getMBean(DefaultMBeanServerInterceptor.java:1083)
[junit4] 2> at java.management/com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.exclusiveUnregisterMBean(DefaultMBeanServerInterceptor.java:423)
[junit4] 2> at java.management/com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.unregisterMBean(DefaultMBeanServerInterceptor.java:411)
[junit4] 2> at java.management/com.sun.jmx.mbeanserver.JmxMBeanServer.unregisterMBean(JmxMBeanServer.java:546)
[junit4] 2> at org.apache.solr.metrics.reporters.jmx.JmxMetricsReporter$JmxListener.registerMBean(JmxMetricsReporter.java:530)
[junit4] 2> at org.apache.solr.metrics.reporters.jmx.JmxMetricsReporter$JmxListener.onGaugeAdded(JmxMetricsReporter.java:565)
[junit4] 2> at org.apache.solr.metrics.reporters.jmx.JmxMetricsReporter.lambda$start$0(JmxMetricsReporter.java:738)
[junit4] 2> at java.base/java.util.HashMap.forEach(HashMap.java:1341)
[junit4] 2> at org.apache.solr.metrics.reporters.jmx.JmxMetricsReporter.start(JmxMetricsReporter.java:728)
[junit4] 2> at org.apache.solr.metrics.reporters.SolrJmxReporter.doInit(SolrJmxReporter.java:109)
[junit4] 2> at org.apache.solr.metrics.SolrMetricReporter.init(SolrMetricReporter.java:70)
[junit4] 2> at org.apache.solr.metrics.SolrMetricManager.loadReporter(SolrMetricManager.java:881)
[junit4] 2> at org.apache.solr.metrics.SolrMetricManager.loadReporters(SolrMetricManager.java:817)
[junit4] 2> at org.apache.solr.core.CoreContainer.load(CoreContainer.java:547)
[junit4] 2> at org.apache.solr.servlet.SolrDispatchFilter.createCoreContainer(SolrDispatchFilter.java:263)
[junit4] 2> at org.apache.solr.servlet.SolrDispatchFilter.init(SolrDispatchFilter.java:183)
[junit4] 2> at org.eclipse.jetty.servlet.FilterHolder.initialize(FilterHolder.java:139)
[junit4] 2> at org.eclipse.jetty.servlet.ServletHandler.initialize(ServletHandler.java:873)
[junit4] 2> at org.eclipse.jetty.servlet.ServletHandler.updateMappings(ServletHandler.java:1596)
[junit4] 2> at org.eclipse.jetty.servlet.ServletHandler.setFilterMappings(ServletHandler.java:1659)
[junit4] 2> at org.eclipse.jetty.servlet.ServletHandler.addFilterMapping(ServletHandler.java:1316)
[junit4] 2> at org.eclipse.jetty.servlet.ServletHandler.addFilterWithMapping(ServletHandler.java:1145)
[junit4] 2> at org.eclipse.jetty.servlet.ServletContextHandler.addFilter(ServletContextHandler.java:448)
[junit4] 2> at org.apache.solr.client.solrj.embedded.JettySolrRunner$1.lifeCycleStarted(JettySolrRunner.java:306)
[junit4] 2> at org.eclipse.jetty.util.component.AbstractLifeCycle.setStarted(AbstractLifeCycle.java:179)
[junit4] 2> at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:69)
[junit4] 2> at org.apache.solr.client.solrj.embedded.JettySolrRunner.start(JettySolrRunner.java:394)
[junit4] 2> at org.apache.solr.client.solrj.embedded.JettySolrRunner.start(JettySolrRunner.java:367)
[junit4] 2> at org.apache.solr.cloud.MiniSolrCloudCluster.startJettySolrRunner(MiniSolrCloudCluster.java:384)
[junit4] 2> at org.apache.solr.cloud.MiniSolrCloudCluster.lambda$new$0(MiniSolrCloudCluster.java:247)
[junit4] 2> at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
[junit4] 2> at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:188)
[junit4] 2> at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1135)
[junit4] 2> at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
[junit4] 2> at java.base/java.lang.Thread.run(Thread.java:844)
[junit4] 2> 405701 INFO (jetty-launcher-560-thread-2) [n:127.0.0.1:43723_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_43723.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@48ad14a0
[junit4] 2> 405701 INFO (jetty-launcher-560-thread-2) [n:127.0.0.1:43723_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_43723.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@48ad14a0
[junit4] 2> 405702 INFO (jetty-launcher-560-thread-2) [n:127.0.0.1:43723_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J2/temp/solr.cloud.MoveReplicaHDFSTest_C3CBB6E52F2E74B5-001/tempDir-001/node2/.
[junit4] 2> 405787 INFO (zkConnectionManagerCallback-605-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 405790 INFO (zkConnectionManagerCallback-609-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 405790 INFO (SUITE-MoveReplicaHDFSTest-seed#[C3CBB6E52F2E74B5]-worker) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (4)
[junit4] 2> 405791 INFO (SUITE-MoveReplicaHDFSTest-seed#[C3CBB6E52F2E74B5]-worker) [ ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:44389/solr ready
[junit4] 2> 405793 INFO (qtp1590780120-2750) [n:127.0.0.1:43723_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :overseerstatus with params action=OVERSEERSTATUS&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 405797 INFO (qtp1590780120-2750) [n:127.0.0.1:43723_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={action=OVERSEERSTATUS&wt=javabin&version=2} status=0 QTime=3
[junit4] 1> Formatting using clusterid: testClusterID
[junit4] 2> 405840 WARN (SUITE-MoveReplicaHDFSTest-seed#[C3CBB6E52F2E74B5]-worker) [ ] o.a.h.m.i.MetricsConfig Cannot locate configuration: tried hadoop-metrics2-namenode.properties,hadoop-metrics2.properties
[junit4] 2> 405847 WARN (SUITE-MoveReplicaHDFSTest-seed#[C3CBB6E52F2E74B5]-worker) [ ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
[junit4] 2> 405848 INFO (SUITE-MoveReplicaHDFSTest-seed#[C3CBB6E52F2E74B5]-worker) [ ] o.m.log jetty-6.1.26
[junit4] 2> 405851 INFO (SUITE-MoveReplicaHDFSTest-seed#[C3CBB6E52F2E74B5]-worker) [ ] o.m.log Extract jar:file:/home/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.7.4-tests.jar!/webapps/hdfs to ./temp/Jetty_localhost_localdomain_42859_hdfs____om8tbs/webapp
[junit4] 2> 405923 INFO (SUITE-MoveReplicaHDFSTest-seed#[C3CBB6E52F2E74B5]-worker) [ ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost.localdomain:42859
[junit4] 2> 406163 WARN (SUITE-MoveReplicaHDFSTest-seed#[C3CBB6E52F2E74B5]-worker) [ ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
[junit4] 2> 406164 INFO (SUITE-MoveReplicaHDFSTest-seed#[C3CBB6E52F2E74B5]-worker) [ ] o.m.log jetty-6.1.26
[junit4] 2> 406167 INFO (SUITE-MoveReplicaHDFSTest-seed#[C3CBB6E52F2E74B5]-worker) [ ] o.m.log Extract jar:file:/home/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.7.4-tests.jar!/webapps/datanode to ./temp/Jetty_localhost_41915_datanode____.gmo80p/webapp
[junit4] 2> 406234 INFO (SUITE-MoveReplicaHDFSTest-seed#[C3CBB6E52F2E74B5]-worker) [ ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:41915
[junit4] 2> 406402 WARN (SUITE-MoveReplicaHDFSTest-seed#[C3CBB6E52F2E74B5]-worker) [ ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
[junit4] 2> 406402 INFO (SUITE-MoveReplicaHDFSTest-seed#[C3CBB6E52F2E74B5]-worker) [ ] o.m.log jetty-6.1.26
[junit4] 2> 406406 INFO (SUITE-MoveReplicaHDFSTest-seed#[C3CBB6E52F2E74B5]-worker) [ ] o.m.log Extract jar:file:/home/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.7.4-tests.jar!/webapps/datanode to ./temp/Jetty_localhost_40037_datanode____p4cvb3/webapp
[junit4] 2> 406424 ERROR (DataNode: [[[DISK]file:/home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J2/temp/solr.cloud.MoveReplicaHDFSTest_C3CBB6E52F2E74B5-001/tempDir-002/hdfsBaseDir/data/data1/, [DISK]file:/home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J2/temp/solr.cloud.MoveReplicaHDFSTest_C3CBB6E52F2E74B5-001/tempDir-002/hdfsBaseDir/data/data2/]] heartbeating to localhost.localdomain/127.0.0.1:33659) [ ] o.a.h.h.s.d.DirectoryScanner dfs.datanode.directoryscan.throttle.limit.ms.per.sec set to value below 1 ms/sec. Assuming default value of 1000
[junit4] 2> 406433 INFO (Block report processor) [ ] BlockStateChange BLOCK* processReport 0x8bd4fbf038a98: from storage DS-2280b15a-61fd-4edb-99b6-2454a46cb439 node DatanodeRegistration(127.0.0.1:39789, datanodeUuid=e7360f28-848f-4231-951e-289243c4d1ae, infoPort=45937, infoSecurePort=0, ipcPort=37031, storageInfo=lv=-56;cid=testClusterID;nsid=659095415;c=0), blocks: 0, hasStaleStorage: true, processing time: 0 msecs
[junit4] 2> 406433 INFO (Block report processor) [ ] BlockStateChange BLOCK* processReport 0x8bd4fbf038a98: from storage DS-5b37ce92-bdcd-4db5-aabe-7d161445570a node DatanodeRegistration(127.0.0.1:39789, datanodeUuid=e7360f28-848f-4231-951e-289243c4d1ae, infoPort=45937, infoSecurePort=0, ipcPort=37031, storageInfo=lv=-56;cid=testClusterID;nsid=659095415;c=0), blocks: 0, hasStaleStorage: false, processing time: 0 msecs
[junit4] 2> 406477 INFO (SUITE-MoveReplicaHDFSTest-seed#[C3CBB6E52F2E74B5]-worker) [ ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:40037
[junit4] 2> 406692 ERROR (DataNode: [[[DISK]file:/home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J2/temp/solr.cloud.MoveReplicaHDFSTest_C3CBB6E52F2E74B5-001/tempDir-002/hdfsBaseDir/data/data3/, [DISK]file:/home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J2/temp/solr.cloud.MoveReplicaHDFSTest_C3CBB6E52F2E74B5-001/tempDir-002/hdfsBaseDir/data/data4/]] heartbeating to localhost.localdomain/127.0.0.1:33659) [ ] o.a.h.h.s.d.DirectoryScanner dfs.datanode.directoryscan.throttle.limit.ms.per.sec set to value below 1 ms/sec. Assuming default value of 1000
[junit4] 2> 406699 INFO (Block report processor) [ ] BlockStateChange BLOCK* processReport 0x8bd4fcee51f86: from storage DS-c93db8db-8a2f-495a-9c56-b493e77e9f1b node DatanodeRegistration(127.0.0.1:46631, datanodeUuid=66c0cec4-0bbf-4b27-a649-dda3125db45e, infoPort=40797, infoSecurePort=0, ipcPort=33469, storageInfo=lv=-56;cid=testClusterID;nsid=659095415;c=0), blocks: 0, hasStaleStorage: true, processing time: 0 msecs
[junit4] 2> 406699 INFO (Block report processor) [ ] BlockStateChange BLOCK* processReport 0x8bd4fcee51f86: from storage DS-3b7d1f1e-14d5-4aff-bae3-57c758864fdb node DatanodeRegistration(127.0.0.1:46631, datanodeUuid=66c0cec4-0bbf-4b27-a649-dda3125db45e, infoPort=40797, infoSecurePort=0, ipcPort=33469, storageInfo=lv=-56;cid=testClusterID;nsid=659095415;c=0), blocks: 0, hasStaleStorage: false, processing time: 0 msecs
[junit4] 2> 406739 INFO (TEST-MoveReplicaHDFSTest.test-seed#[C3CBB6E52F2E74B5]) [ ] o.a.s.SolrTestCaseJ4 ###Starting test
[junit4] 2> 406740 INFO (TEST-MoveReplicaHDFSTest.test-seed#[C3CBB6E52F2E74B5]) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (4)
[junit4] 2> 406740 INFO (TEST-MoveReplicaHDFSTest.test-seed#[C3CBB6E52F2E74B5]) [ ] o.e.j.s.Server jetty-9.3.20.v20170531
[junit4] 2> 406741 INFO (TEST-MoveReplicaHDFSTest.test-seed#[C3CBB6E52F2E74B5]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@3b9a4858{/solr,null,AVAILABLE}
[junit4] 2> 406742 INFO (TEST-MoveReplicaHDFSTest.test-seed#[C3CBB6E52F2E74B5]) [ ] o.e.j.s.AbstractConnector Started ServerConnector@38b1dfe8{HTTP/1.1,[http/1.1]}{127.0.0.1:44913}
[junit4] 2> 406742 INFO (TEST-MoveReplicaHDFSTest.test-seed#[C3CBB6E52F2E74B5]) [ ] o.e.j.s.Server Started @408491ms
[junit4] 2> 406742 INFO (TEST-MoveReplicaHDFSTest.test-seed#[C3CBB6E52F2E74B5]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=44913}
[junit4] 2> 406742 ERROR (TEST-MoveReplicaHDFSTest.test-seed#[C3CBB6E52F2E74B5]) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 406742 INFO (TEST-MoveReplicaHDFSTest.test-seed#[C3CBB6E52F2E74B5]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version 7.3.0
[junit4] 2> 406742 INFO (TEST-MoveReplicaHDFSTest.test-seed#[C3CBB6E52F2E74B5]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 406742 INFO (TEST-MoveReplicaHDFSTest.test-seed#[C3CBB6E52F2E74B5]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 406742 INFO (TEST-MoveReplicaHDFSTest.test-seed#[C3CBB6E52F2E74B5]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2017-12-31T22:20:08.356656Z
[junit4] 2> 406744 INFO (zkConnectionManagerCallback-612-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 406745 INFO (TEST-MoveReplicaHDFSTest.test-seed#[C3CBB6E52F2E74B5]) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 406751 INFO (TEST-MoveReplicaHDFSTest.test-seed#[C3CBB6E52F2E74B5]) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:44389/solr
[junit4] 2> 406754 INFO (zkConnectionManagerCallback-616-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 406755 INFO (zkConnectionManagerCallback-618-thread-1-processing-n:127.0.0.1:44913_solr) [n:127.0.0.1:44913_solr ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 406757 INFO (TEST-MoveReplicaHDFSTest.test-seed#[C3CBB6E52F2E74B5]) [n:127.0.0.1:44913_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (4)
[junit4] 2> 406758 INFO (TEST-MoveReplicaHDFSTest.test-seed#[C3CBB6E52F2E74B5]) [n:127.0.0.1:44913_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 406759 INFO (TEST-MoveReplicaHDFSTest.test-seed#[C3CBB6E52F2E74B5]) [n:127.0.0.1:44913_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:44913_solr
[junit4] 2> 406759 INFO (zkCallback-584-thread-1-processing-n:127.0.0.1:40241_solr) [n:127.0.0.1:40241_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5)
[junit4] 2> 406759 INFO (zkCallback-594-thread-1-processing-n:127.0.0.1:45429_solr) [n:127.0.0.1:45429_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5)
[junit4] 2> 406760 INFO (zkCallback-592-thread-1-processing-n:127.0.0.1:38399_solr) [n:127.0.0.1:38399_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5)
[junit4] 2> 406761 INFO (zkCallback-608-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5)
[junit4] 2> 406761 INFO (zkCallback-608-thread-2) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5)
[junit4] 2> 406761 INFO (zkCallback-581-thread-2-processing-n:127.0.0.1:43723_solr) [n:127.0.0.1:43723_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5)
[junit4] 2> 406762 INFO (zkCallback-617-thread-1-processing-n:127.0.0.1:44913_solr) [n:127.0.0.1:44913_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5)
[junit4] 2> 406797 INFO (TEST-MoveReplicaHDFSTest.test-seed#[C3CBB6E52F2E74B5]) [n:127.0.0.1:44913_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_44913.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@48ad14a0
[junit4] 2> 406804 INFO (TEST-MoveReplicaHDFSTest.test-seed#[C3CBB6E52F2E74B5]) [n:127.0.0.1:44913_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_44913.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@48ad14a0
[junit4] 2> 406804 INFO (TEST-MoveReplicaHDFSTest.test-seed#[C3CBB6E52F2E74B5]) [n:127.0.0.1:44913_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_44913.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@48ad14a0
[junit4] 2> 406806 INFO (TEST-MoveReplicaHDFSTest.test-seed#[C3CBB6E52F2E74B5]) [n:127.0.0.1:44913_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J2/temp/solr.cloud.MoveReplicaHDFSTest_C3CBB6E52F2E74B5-001/tempDir-001/node5/.
[junit4] 2> 406848 INFO (zkConnectionManagerCallback-622-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 406849 INFO (TEST-MoveReplicaHDFSTest.test-seed#[C3CBB6E52F2E74B5]) [ ] o.a.s.c.MoveReplicaTest total_jettys: 5
[junit4] 2> 406849 INFO (qtp1425932665-2729) [n:127.0.0.1:38399_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params replicationFactor=2&collection.configName=conf1&maxShardsPerNode=2&autoAddReplicas=false&name=MoveReplicaHDFSTest_coll_true&nrtReplicas=2&action=CREATE&numShards=2&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 406850 INFO (OverseerThreadFactory-1167-thread-2-processing-n:127.0.0.1:43723_solr) [n:127.0.0.1:43723_solr ] o.a.s.c.CreateCollectionCmd Create collection MoveReplicaHDFSTest_coll_true
[junit4] 2> 406850 INFO (OverseerCollectionConfigSetProcessor-99271233178632199-127.0.0.1:43723_solr-n_0000000000) [n:127.0.0.1:43723_solr ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000000 doesn't exist. Requestor may have disconnected from ZooKeeper
[junit4] 2> 406953 INFO (OverseerStateUpdate-99271233178632199-127.0.0.1:43723_solr-n_0000000000) [n:127.0.0.1:43723_solr ] o.a.s.c.o.SliceMutator createReplica() {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"MoveReplicaHDFSTest_coll_true",
[junit4] 2> "shard":"shard1",
[junit4] 2> "core":"MoveReplicaHDFSTest_coll_true_shard1_replica_n1",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"http://127.0.0.1:40241/solr",
[junit4] 2> "type":"NRT",
[junit4] 2> "waitForFinalState":"false"}
[junit4] 2> 406954 INFO (OverseerStateUpdate-99271233178632199-127.0.0.1:43723_solr-n_0000000000) [n:127.0.0.1:43723_solr ] o.a.s.c.o.SliceMutator createReplica() {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"MoveReplicaHDFSTest_coll_true",
[junit4] 2> "shard":"shard1",
[junit4] 2> "core":"MoveReplicaHDFSTest_coll_true_shard1_replica_n2",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"http://127.0.0.1:43723/solr",
[junit4] 2> "type":"NRT",
[junit4] 2> "waitForFinalState":"false"}
[junit4] 2> 406955 INFO (OverseerStateUpdate-99271233178632199-127.0.0.1:43723_solr-n_0000000000) [n:127.0.0.1:43723_solr ] o.a.s.c.o.SliceMutator createReplica() {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"MoveReplicaHDFSTest_coll_true",
[junit4] 2> "shard":"shard2",
[junit4] 2> "core":"MoveReplicaHDFSTest_coll_true_shard2_replica_n4",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"http://127.0.0.1:45429/solr",
[junit4] 2> "type":"NRT",
[junit4] 2> "waitForFinalState":"false"}
[junit4] 2> 406955 INFO (OverseerStateUpdate-99271233178632199-127.0.0.1:43723_solr-n_0000000000) [n:127.0.0.1:43723_solr ] o.a.s.c.o.SliceMutator createReplica() {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"MoveReplicaHDFSTest_coll_true",
[junit4] 2> "shard":"shard2",
[junit4] 2> "core":"MoveReplicaHDFSTest_coll_true_shard2_replica_n6",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"http://127.0.0.1:38399/solr",
[junit4] 2> "type":"NRT",
[junit4] 2> "waitForFinalState":"false"}
[junit4] 2> 407158 INFO (qtp1828228189-2756) [n:127.0.0.1:45429_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node7&collection.configName=conf1&newCollection=true&name=MoveReplicaHDFSTest_coll_true_shard2_replica_n4&action=CREATE&numShards=2&collection=MoveReplicaHDFSTest_coll_true&shard=shard2&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 407159 INFO (qtp1828228189-2756) [n:127.0.0.1:45429_solr ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 407159 INFO (qtp1590780120-2757) [n:127.0.0.1:43723_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node5&collection.configName=conf1&newCollection=true&name=MoveReplicaHDFSTest_coll_true_shard1_replica_n2&action=CREATE&numShards=2&collection=MoveReplicaHDFSTest_coll_true&shard=shard1&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 407158 INFO (qtp706789920-2737) [n:127.0.0.1:40241_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node3&collection.configName=conf1&newCollection=true&name=MoveReplicaHDFSTest_coll_true_shard1_replica_n1&action=CREATE&numShards=2&collection=MoveReplicaHDFSTest_coll_true&shard=shard1&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 407158 INFO (qtp1425932665-2732) [n:127.0.0.1:38399_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node8&collection.configName=conf1&newCollection=true&name=MoveReplicaHDFSTest_coll_true_shard2_replica_n6&action=CREATE&numShards=2&collection=MoveReplicaHDFSTest_coll_true&shard=shard2&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 407159 INFO (qtp706789920-2737) [n:127.0.0.1:40241_solr ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 407159 INFO (qtp1590780120-2757) [n:127.0.0.1:43723_solr ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 407159 INFO (qtp1425932665-2732) [n:127.0.0.1:38399_solr ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 407261 INFO (zkCallback-581-thread-1-processing-n:127.0.0.1:43723_solr) [n:127.0.0.1:43723_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/MoveReplicaHDFSTest_coll_true/state.json] for collection [MoveReplicaHDFSTest_coll_true] has occurred - updating... (live nodes size: [5])
[junit4] 2> 407261 INFO (zkCallback-592-thread-1-processing-n:127.0.0.1:38399_solr) [n:127.0.0.1:38399_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/MoveReplicaHDFSTest_coll_true/state.json] for collection [MoveReplicaHDFSTest_coll_true] has occurred - updating... (live nodes size: [5])
[junit4] 2> 407261 INFO (zkCallback-594-thread-1-processing-n:127.0.0.1:45429_solr) [n:127.0.0.1:45429_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/MoveReplicaHDFSTest_coll_true/state.json] for collection [MoveReplicaHDFSTest_coll_true] has occurred - updating... (live nodes size: [5])
[junit4] 2> 407261 INFO (zkCallback-584-thread-2-processing-n:127.0.0.1:40241_solr) [n:127.0.0.1:40241_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/MoveReplicaHDFSTest_coll_true/state.json] for collection [MoveReplicaHDFSTest_coll_true] has occurred - updating... (live nodes size: [5])
[junit4] 2> 407261 INFO (zkCallback-581-thread-2-processing-n:127.0.0.1:43723_solr) [n:127.0.0.1:43723_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/MoveReplicaHDFSTest_coll_true/state.json] for collection [MoveReplicaHDFSTest_coll_true] has occurred - updating... (live nodes size: [5])
[junit4] 2> 407261 INFO (zkCallback-594-thread-2-processing-n:127.0.0.1:45429_solr) [n:127.0.0.1:45429_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/MoveReplicaHDFSTest_coll_true/state.json] for collection [MoveReplicaHDFSTest_coll_true] has occurred - updating... (live nodes size: [5])
[junit4] 2> 407261 INFO (zkCallback-584-thread-1-processing-n:127.0.0.1:40241_solr) [n:127.0.0.1:40241_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/MoveReplicaHDFSTest_coll_true/state.json] for collection [MoveReplicaHDFSTest_coll_true] has occurred - updating... (live nodes size: [5])
[junit4] 2> 407262 INFO (zkCallback-592-thread-2-processing-n:127.0.0.1:38399_solr) [n:127.0.0.1:38399_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/MoveReplicaHDFSTest_coll_true/state.json] for collection [MoveReplicaHDFSTest_coll_true] has occurred - updating... (live nodes size: [5])
[junit4] 2> 408166 INFO (qtp706789920-2737) [n:127.0.0.1:40241_solr c:MoveReplicaHDFSTest_coll_true s:shard1 r:core_node3 x:MoveReplicaHDFSTest_coll_true_shard1_replica_n1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.3.0
[junit4] 2> 408166 INFO (qtp1828228189-2756) [n:127.0.0.1:45429_solr c:MoveReplicaHDFSTest_coll_true s:shard2 r:core_node7 x:MoveReplicaHDFSTest_coll_true_shard2_replica_n4] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.3.0
[junit4] 2> 408166 INFO (qtp1590780120-2757) [n:127.0.0.1:43723_solr c:MoveReplicaHDFSTest_coll_true s:shard1 r:core_node5 x:MoveReplicaHDFSTest_coll_true_shard1_replica_n2] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.3.0
[junit4] 2> 408170 INFO (qtp706789920-2737) [n:127.0.0.1:40241_solr c:MoveReplicaHDFSTest_coll_true s:shard1 r:core_node3 x:MoveReplicaHDFSTest_coll_true_shard1_replica_n1] o.a.s.s.IndexSchema [MoveReplicaHDFSTest_coll_true_shard1_replica_n1] Schema name=minimal
[junit4] 2> 408393 INFO (qtp1828228189-2756) [n:127.0.0.1:45429_solr c:MoveReplicaHDFSTest_coll_true s:shard2 r:core_node7 x:MoveReplicaHDFSTest_coll_true_shard2_replica_n4] o.a.s.s.IndexSchema [MoveReplicaHDFSTest_coll_true_shard2_replica_n4] Schema name=minimal
[junit4] 2> 408394 INFO (qtp1425932665-2732) [n:127.0.0.1:38399_solr c:MoveReplicaHDFSTest_coll_true s:shard2 r:core_node8 x:MoveReplicaHDFSTest_coll_true_shard2_replica_n6] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.3.0
[junit4] 2> 408395 INFO (qtp706789920-2737) [n:127.0.0.1:40241_solr c:MoveReplicaHDFSTest_coll_true s:shard1 r:core_node3 x:MoveReplicaHDFSTest_coll_true_shard1_replica_n1] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
[junit4] 2> 408395 INFO (qtp706789920-2737) [n:127.0.0.1:40241_solr c:MoveReplicaHDFSTest_coll_true s:shard1 r:core_node3 x:MoveReplicaHDFSTest_coll_true_shard1_replica_n1] o.a.s.c.CoreContainer Creating SolrCore 'MoveReplicaHDFSTest_coll_true_shard1_replica_n1' using configuration from collection MoveReplicaHDFSTest_coll_true, trusted=true
[junit4] 2> 408396 INFO (qtp706789920-2737) [n:127.0.0.1:40241_solr c:MoveReplicaHDFSTest_coll_true s:shard1 r:core_node3 x:MoveReplicaHDFSTest_coll_true_shard1_replica_n1] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_40241.solr.core.MoveReplicaHDFSTest_coll_true.shard1.replica_n1' (registry 'solr.core.MoveReplicaHDFSTest_coll_true.shard1.replica_n1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@48ad14a0
[junit4] 2> 408396 INFO (qtp1590780120-2757) [n:127.0.0.1:43723_solr c:MoveReplicaHDFSTest_coll_true s:shard1 r:core_node5 x:MoveReplicaHDFSTest_coll_true_shard1_replica_n2] o.a.s.s.IndexSchema [MoveReplicaHDFSTest_coll_true_shard1_replica_n2] Schema name=minimal
[junit4] 2> 408399 INFO (qtp1828228189-2756) [n:127.0.0.1:45429_solr c:MoveReplicaHDFSTest_coll_true s:shard2 r:core_node7 x:MoveReplicaHDFSTest_coll_true_shard2_replica_n4] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
[junit4] 2> 408399 INFO (qtp1828228189-2756) [n:127.0.0.1:45429_solr c:MoveReplicaHDFSTest_coll_true s:shard2 r:core_node7 x:MoveReplicaHDFSTest_coll_true_shard2_replica_n4] o.a.s.c.CoreContainer Creating SolrCore 'MoveReplicaHDFSTest_coll_true_shard2_replica_n4' using configuration from collection MoveReplicaHDFSTest_coll_true, trusted=true
[junit4] 2> 408399 INFO (qtp1828228189-2756) [n:127.0.0.1:45429_solr c:MoveReplicaHDFSTest_coll_true s:shard2 r:core_node7 x:MoveReplicaHDFSTest_coll_true_shard2_replica_n4] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_45429.solr.core.MoveReplicaHDFSTest_coll_true.shard2.replica_n4' (registry 'solr.core.MoveReplicaHDFSTest_coll_true.shard2.replica_n4') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@48ad14a0
[junit4] 2> 408400 INFO (qtp1590780120-2757) [n:127.0.0.1:43723_solr c:MoveReplicaHDFSTest_coll_true s:shard1 r:core_node5 x:MoveReplicaHDFSTest_coll_true_shard1_replica_n2] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
[junit4] 2> 408400 INFO (qtp1590780120-2757) [n:127.0.0.1:43723_solr c:MoveReplicaHDFSTest_coll_true s:shard1 r:core_node5 x:MoveReplicaHDFSTest_coll_true_shard1_replica_n2] o.a.s.c.CoreContainer Creating SolrCore 'MoveReplicaHDFSTest_coll_true_shard1_replica_n2' using configuration from collection MoveReplicaHDFSTest_coll_true, trusted=true
[junit4] 2> 408400 INFO (qtp1590780120-2757) [n:127.0.0.1:43723_solr c:MoveReplicaHDFSTest_coll_true s:shard1 r:core_node5 x:MoveReplicaHDFSTest_coll_true_shard1_replica_n2] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_43723.solr.core.MoveReplicaHDFSTest_coll_true.shard1.replica_n2' (registry 'solr.core.MoveReplicaHDFSTest_coll_true.shard1.replica_n2') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@48ad14a0
[junit4] 2> 408405 INFO (qtp1425932665-2732) [n:127.0.0.1:38399_solr c:MoveReplicaHDFSTest_coll_true s:shard2 r:core_node8 x:MoveReplicaHDFSTest_coll_true_shard2_replica_n6] o.a.s.s.IndexSchema [MoveReplicaHDFSTest_coll_true_shard2_replica_n6] Schema name=minimal
[junit4] 2> 408408 INFO (qtp1425932665-2732) [n:127.0.0.1:38399_solr c:MoveReplicaHDFSTest_coll_true s:shard2 r:core_node8 x:MoveReplicaHDFSTest_coll_true_shard2_replica_n6] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
[junit4] 2> 408408 INFO (qtp1425932665-2732) [n:127.0.0.1:38399_solr c:MoveReplicaHDFSTest_coll_true s:shard2 r:core_node8 x:MoveReplicaHDFSTest_coll_true_shard2_replica_n6] o.a.s.c.CoreContainer Creating SolrCore 'MoveReplicaHDFSTest_coll_true_shard2_replica_n6' using configuration from collection MoveReplicaHDFSTest_coll_true, trusted=true
[junit4] 2> 408408 INFO (qtp706789920-2737) [n:127.0.0.1:40241_solr c:MoveReplicaHDFSTest_coll_true s:shard1 r:core_node3 x:MoveReplicaHDFSTest_coll_true_shard1_replica_n1] o.a.s.c.HdfsDirectoryFactory solr.hdfs.home=hdfs://localhost.localdomain:33659/data
[junit4] 2> 408408 INFO (qtp706789920-2737) [n:127.0.0.1:40241_solr c:MoveReplicaHDFSTest_coll_true s:shard1 r:core_node3 x:MoveReplicaHDFSTest_coll_true_shard1_replica_n1] o.a.s.c.HdfsDirectoryFactory Solr Kerberos Authentication disabled
[junit4] 2> 408408 INFO (qtp706789920-2737) [n:127.0.0.1:40241_solr c:MoveReplicaHDFSTest_coll_true s:shard1 r:core_node3 x:MoveReplicaHDFSTest_coll_true_shard1_replica_n1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 408408 INFO (qtp1425932665-2732) [n:127.0.0.1:38399_solr c:MoveReplicaHDFSTest_coll_true s:shard2 r:core_node8 x:MoveReplicaHDFSTest_coll_true_shard2_replica_n6] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_38399.solr.core.MoveReplicaHDFSTest_coll_true.shard2.replica_n6' (registry 'solr.core.MoveReplicaHDFSTest_coll_true.shard2.replica_n6') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@48ad14a0
[junit4] 2> 408408 INFO (qtp706789920-2737) [n:127.0.0.1:40241_solr c:MoveReplicaHDFSTest_coll_true s:shard1 r:core_node3 x:MoveReplicaHDFSTest_coll_true_shard1_replica_n1] o.a.s.c.SolrCore [[MoveReplicaHDFSTest_coll_true_shard1_replica_n1] ] Opening new SolrCore at [/home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J2/temp/solr.cloud.MoveReplicaHDFSTest_C3CBB6E52F2E74B5-001/tempDir-001/node3/MoveReplicaHDFSTest_coll_true_shard1_replica_n1], dataDir=[hdfs://localhost.localdomain:33659/data/MoveReplicaHDFSTest_coll_true/core_node3/data/]
[junit4] 2> 408408 INFO (qtp1425932665-2732) [n:127.0.0.1:38399_solr c:MoveReplicaHDFSTest_coll_true s:shard2 r:core_node8 x:MoveReplicaHDFSTest_coll_true_shard2_replica_n6] o.a.s.c.HdfsDirectoryFactory solr.hdfs.home=hdfs://localhost.localdomain:33659/data
[junit4] 2> 408408 INFO (qtp1425932665-2732) [n:127.0.0.1:38399_solr c:MoveReplicaHDFSTest_coll_true s:shard2 r:core_node8 x:MoveReplicaHDFSTest_coll_true_shard2_replica_n6] o.a.s.c.HdfsDirectoryFactory Solr Kerberos Authentication disabled
[junit4] 2> 408409 INFO (qtp1425932665-2732) [n:127.0.0.1:38399_solr c:MoveReplicaHDFSTest_coll_true s:shard2 r:core_node8 x:MoveReplicaHDFSTest_coll_true_shard2_replica_n6] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 408409 INFO (qtp1828228189-2756) [n:127.0.0.1:45429_solr c:MoveReplicaHDFSTest_coll_true s:shard2 r:core_node7 x:MoveReplicaHDFSTest_coll_true_shard2_replica_n4] o.a.s.c.HdfsDirectoryFactory solr.hdfs.home=hdfs://localhost.localdomain:33659/data
[junit4] 2> 408409 INFO (qtp1590780120-2757) [n:127.0.0.1:43723_solr c:MoveReplicaHDFSTest_coll_true s:shard1 r:core_node5 x:MoveReplicaHDFSTest_coll_true_shard1_replica_n2] o.a.s.c.HdfsDirectoryFactory solr.hdfs.home=hdfs://localhost.localdomain:33659/data
[junit4] 2> 408409 INFO (qtp1590780120-2757) [n:127.0.0.1:43723_solr c:MoveReplicaHDFSTest_coll_true s:shard1 r:core_node5 x:MoveReplicaHDFSTest_coll_true_shard1_replica_n2] o.a.s.c.HdfsDirectoryFactory Solr Kerberos Authentication disabled
[junit4] 2> 408409 INFO (qtp1590780120-2757) [n:127.0.0.1:43723_solr c:MoveReplicaHDFSTest_coll_true s:shard1 r:core_node5 x:MoveReplicaHDFSTest_coll_true_shard1_replica_n2] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 408409 INFO (qtp1425932665-2732) [n:127.0.0.1:38399_solr c:MoveReplicaHDFSTest_coll_true s:shard2 r:core_node8 x:MoveReplicaHDFSTest_coll_true_shard2_replica_n6] o.a.s.c.SolrCore [[MoveReplicaHDFSTest_coll_true_shard2_replica_n6] ] Opening new SolrCore at [/home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J2/temp/solr.cloud.MoveReplicaHDFSTest_C3CBB6E52F2E74B5-001/tempDir-001/node1/MoveReplicaHDFSTest_coll_true_shard2_replica_n6], dataDir=[hdfs://localhost.localdomain:33659/data/MoveReplicaHDFSTest_coll_true/core_node8/data/]
[junit4] 2> 408409 INFO (qtp1590780120-2757) [n:127.0.0.1:43723_solr c:MoveReplicaHDFSTest_coll_true s:shard1 r:core_node5 x:MoveReplicaHDFSTest_coll_true_shard1_replica_n2] o.a.s.c.SolrCore [[MoveReplicaHDFSTest_coll_true_shard1_replica_n2] ] Opening new SolrCore at [/home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J2/temp/solr.cloud.MoveReplicaHDFSTest_C3CBB6E52F2E74B5-001/tempDir-001/node2/MoveReplicaHDFSTest_coll_true_shard1_replica_n2], dataDir=[hdfs://localhost.localdomain:33659/data/MoveReplicaHDFSTest_coll_true/core_node5/data/]
[junit4] 2> 408409 INFO (qtp1828228189-2756) [n:127.0.0.1:45429_solr c:MoveReplicaHDFSTest_coll_true s:shard2 r:core_node7 x:MoveReplicaHDFSTest_coll_true_shard2_replica_n4] o.a.s.c.HdfsDirectoryFactory Solr Kerberos Authentication disabled
[junit4] 2> 408409 INFO (qtp1828228189-2756) [n:127.0.0.1:45429_solr c:MoveReplicaHDFSTest_coll_true s:shard2 r:core_node7 x:MoveReplicaHDFSTest_coll_true_shard2_replica_n4] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 408410 INFO (qtp1828228189-2756) [n:127.0.0.1:45429_solr c:MoveReplicaHDFSTest_coll_true s:shard2 r:core_node7 x:MoveReplicaHDFSTest_coll_true_shard2_replica_n4] o.a.s.c.SolrCore [[MoveReplicaHDFSTest_coll_true_shard2_replica_n4] ] Opening new SolrCore at [/home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J2/temp/solr.cloud.MoveReplicaHDFSTest_C3CBB6E52F2E74B5-001/tempDir-001/node4/MoveReplicaHDFSTest_coll_true_shard2_replica_n4], dataDir=[hdfs://localhost.localdomain:33659/data/MoveReplicaHDFSTest_coll_true/core_node7/data/]
[junit4] 2> 408410 INFO (qtp1590780120-2757) [n:127.0.0.1:43723_solr c:MoveReplicaHDFSTest_coll_true s:shard1 r:core_node5 x:MoveReplicaHDFSTest_coll_true_shard1_replica_n2] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost.localdomain:33659/data/MoveReplicaHDFSTest_coll_true/core_node5/data/snapshot_metadata
[junit4] 2> 408410 INFO (qtp1425932665-2732) [n:127.0.0.1:38399_solr c:MoveReplicaHDFSTest_coll_true s:shard2 r:core_node8 x:MoveReplicaHDFSTest_coll_true_shard2_replica_n6] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost.localdomain:33659/data/MoveReplicaHDFSTest_coll_true/core_node8/data/snapshot_metadata
[junit4] 2> 408410 INFO (qtp706789920-2737) [n:127.0.0.1:40241_solr c:MoveReplicaHDFSTest_coll_true s:shard1 r:core_node3 x:MoveReplicaHDFSTest_coll_true_shard1_replica_n1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost.localdomain:33659/data/MoveReplicaHDFSTest_coll_true/core_node3/data/snapshot_metadata
[junit4] 2> 408411 INFO (qtp1828228189-2756) [n:127.0.0.1:45429_solr c:MoveReplicaHDFSTest_coll_true s:shard2 r:core_node7 x:MoveReplicaHDFSTest_coll_true_shard2_replica_n4] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost.localdomain:33659/data/MoveReplicaHDFSTest_coll_true/core_node7/data/snapshot_metadata
[junit4] 2> 408432 WARN (qtp1425932665-2732) [n:127.0.0.1:38399_solr c:MoveReplicaHDFSTest_coll_true s:shard2 r:core_node8 x:MoveReplicaHDFSTest_coll_true_shard2_replica_n6] o.a.s.s.h.HdfsDirectory The NameNode is in SafeMode - Solr will wait 5 seconds and try again.
[junit4] 2> 408432 WARN (qtp706789920-2737) [n:127.0.0.1:40241_solr c:MoveReplicaHDFSTest_coll_true s:shard1 r:core_node3 x:MoveReplicaHDFSTest_coll_true_shard1_replica_n1] o.a.s.s.h.HdfsDirectory The NameNode is in SafeMode - Solr will wait 5 seconds and try again.
[junit4] 2> 408432 WARN (qtp1590780120-2757) [n:127.0.0.1:43723_solr c:MoveReplicaHDFSTest_coll_true s:shard1 r:core_node5 x:MoveReplicaHDFSTest_coll_true_shard1_replica_n2] o.a.s.s.h.HdfsDirectory The NameNode is in SafeMode - Solr will wait 5 seconds and try again.
[junit4] 2> 408433 WARN (qtp1828228189-2756) [n:127.0.0.1:45429_solr c:MoveReplicaHDFSTest_coll_true s:shard2 r:core_node7 x:MoveReplicaHDFSTest_coll_true_shard2_replica_n4] o.a.s.s.h.HdfsDirectory The NameNode is in SafeMode - Solr will wait 5 seconds and try again.
[junit4] 2> 413433 WARN (qtp1425932665-2732) [n:127.0.0.1:38399_solr c:MoveReplicaHDFSTest_coll_true s:shard2 r:core_node8 x:MoveReplicaHDFSTest_coll_true_shard2_replica_n6] o.a.s.s.h.HdfsDirectory The NameNode is in SafeMode - Solr will wait 5 seconds and try again.
[junit4] 2> 413434 WARN (qtp706789920-2737) [n:127.0.0.1:40241_solr c:MoveReplicaHDFSTest_coll_true s:shard1 r:core_node3 x:MoveReplicaHDFSTest_coll_true_shard1_replica_n1] o.a.s.s.h.HdfsDirectory The NameNode is in SafeMode - Solr will wait 5 seconds and try again.
[junit4] 2> 413434 WARN (qtp1590780120-2757) [n:127.0.0.1:43723_solr c:MoveReplicaHDFSTest_coll_true s:shard1 r:core_node5 x:MoveReplicaHDFSTest_coll_true_shard1_replica_n2] o.a.s.s.h.HdfsDirectory The NameNode is in SafeMode - Solr will wait 5 seconds and try again.
[junit4] 2> 413434 WARN (qtp1828228189-2756) [n:127.0.0.1:45429_solr c:MoveReplicaHDFSTest_coll_true s:shard2 r:core_node7 x:MoveReplicaHDFSTest_coll_true_shard2_replica_n4] o.a.s.s.h.HdfsDirectory The NameNode is in SafeMode - Solr will wait 5 seconds and try again.
[junit4] 2> 418466 INFO (qtp706789920-2737) [n:127.0.0.1:40241_solr c:MoveReplicaHDFSTest_coll_true s:shard1 r:core_node3 x:MoveReplicaHDFSTest_coll_true_shard1_replica_n1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost.localdomain:33659/data/MoveReplicaHDFSTest_coll_true/core_node3/data
[junit4] 2> 418466 INFO (qtp1828228189-2756) [n:127.0.0.1:45429_solr c:MoveReplicaHDFSTest_coll_true s:shard2 r:core_node7 x:MoveReplicaHDFSTest_coll_true_shard2_replica_n4] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost.localdomain:33659/data/MoveReplicaHDFSTest_coll_true/core_node7/data
[junit4] 2> 418473 INFO (qtp1425932665-2732) [n:127.0.0.1:38399_solr c:MoveReplicaHDFSTest_coll_true s:shard2 r:core_node8 x:MoveReplicaHDFSTest_coll_true_shard2_replica_n6] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost.localdomain:33659/data/MoveReplicaHDFSTest_coll_true/core_node8/data
[junit4] 2> 418473 INFO (qtp1590780120-2757) [n:127.0.0.1:43723_solr c:MoveReplicaHDFSTest_coll_true s:shard1 r:core_node5 x:MoveReplicaHDFSTest_coll_true_shard1_replica_n2] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost.localdomain:33659/data/MoveReplicaHDFSTest_coll_true/core_node5/data
[junit4] 2> 418512 INFO (qtp1828228189-2756) [n:127.0.0.1:45429_solr c:MoveReplicaHDFSTest_coll_true s:shard2 r:core_node7 x:MoveReplicaHDFSTest_coll_true_shard2_replica_n4] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost.localdomain:33659/data/MoveReplicaHDFSTest_coll_true/core_node7/data/index
[junit4] 2> 418516 INFO (qtp706789920-2737) [n:127.0.0.1:40241_solr c:MoveReplicaHDFSTest_coll_true s:shard1 r:core_node3 x:MoveReplicaHDFSTest_coll_true_shard1_replica_n1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost.localdomain:33659/data/MoveReplicaHDFSTest_coll_true/core_node3/data/index
[junit4] 2> 418530 INFO (qtp1425932665-2732) [n:127.0.0.1:38399_solr c:MoveReplicaHDFSTest_coll_true s:shard2 r:core_node8 x:MoveReplicaHDFSTest_coll_true_shard2_replica_n6] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost.localdomain:33659/data/MoveReplicaHDFSTest_coll_true/core_node8/data/index
[junit4] 2> 418531 INFO (qtp1590780120-2757) [n:127.0.0.1:43723_solr c:MoveReplicaHDFSTest_coll_true s:shard1 r:core_node5 x:MoveReplicaHDFSTest_coll_true_shard1_replica_n2] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost.localdomain:33659/data/MoveReplicaHDFSTest_coll_true/core_node5/data/index
[junit4] 2> 418726 INFO (Block report processor) [ ] BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:39789 is added to blk_1073741825_1001{UCState=UNDER_CONSTRUCTION, truncateBlock=null, primaryNodeIndex=-1, replicas=[ReplicaUC[[DISK]DS-c93db8db-8a2f-495a-9c56-b493e77e9f1b:NORMAL:127.0.0.1:46631|RBW], ReplicaUC[[DISK]DS-2280b15a-61fd-4edb-99b6-2454a46cb439:NORMAL:127.0.0.1:39789|FINALIZED]]} size 0
[junit4] 2> 418728 INFO (Block report processor) [ ] BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:46631 is added to blk_1073741825_1001 size 69
[junit4] 2> 418735 INFO (Block report processor) [ ] BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:46631 is added to blk_1073741827_1003{UCState=UNDER_CONSTRUCTION, truncateBlock=null, primaryNodeIndex=-1, replicas=[ReplicaUC[[DISK]DS-c93db8db-8a2f-495a-9c56-b493e77e9f1b:NORMAL:127.0.0.1:46631|RBW], ReplicaUC[[DISK]DS-2280b15a-61fd-4edb-99b6-2454a46cb439:NORMAL:127.0.0.1:39789|RBW]]} size 0
[junit4] 2> 418746 WARN (DataStreamer for file /data/MoveReplicaHDFSTest_coll_true/core_node3/data/index/pending_segments_1) [n:127.0.0.1:40241_solr c:MoveReplicaHDFSTest_coll_true s:shard1 r:core_node3 x:MoveReplicaHDFSTest_coll_true_shard1_replica_n1] o.a.h.h.DFSClient Caught exception
[junit4] 2> java.lang.InterruptedException
[junit4] 2> at java.base/java.lang.Object.wait(Native Method)
[junit4] 2> at java.base/java.lang.Thread.join(Thread.java:1353)
[junit4] 2> at java.base/java.lang.Thread.join(Thread.java:1427)
[junit4] 2> at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.closeResponder(DFSOutputStream.java:716)
[junit4] 2> at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.endBlock(DFSOutputStream.java:476)
[junit4] 2> at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.run(DFSOutputStream.java:652)
[junit4] 2> 418755 INFO (Block report processor) [ ] BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:39789 is added to blk_1073741827_1003 size 69
[junit4] 2> 418755 INFO (Block report processor) [ ] BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:39789 is added to blk_1073741826_1002{UCState=UNDER_CONSTRUCTION, truncateBlock=null, primaryNodeIndex=-1, replicas=[ReplicaUC[[DISK]DS-5b37ce92-bdcd-4db5-aabe-7d161445570a:NORMAL:127.0.0.1:39789|RBW], ReplicaUC[[DISK]DS-3b7d1f1e-14d5-4aff-bae3-57c758864fdb:NORMAL:127.0.0.1:46631|RBW]]} size 0
[junit4] 2> 418756 INFO (Block report processor) [ ] BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:46631 is added to blk_1073741826_1002{UCState=UNDER_CONSTRUCTION, truncateBlock=null, primaryNodeIndex=-1, replicas=[ReplicaUC[[DISK]DS-5b37ce92-bdcd-4db5-aabe-7d161445570a:NORMAL:127.0.0.1:39789|RBW], ReplicaUC[[DISK]DS-3b7d1f1e-14d5-4aff-bae3-57c758864fdb:NORMAL:127.0.0.1:46631|RBW]]} size 0
[junit4] 2> 418770 INFO (Block report processor) [ ] BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:46631 is added to blk_1073741828_1004{UCState=UNDER_CONSTRUCTION, truncateBlock=null, primaryNodeIndex=-1, replicas=[ReplicaUC[[DISK]DS-3b7d1f1e-14d5-4aff-bae3-57c758864fdb:NORMAL:127.0.0.1:46631|RBW], ReplicaUC[[DISK]DS-5b37ce92-bdcd-4db5-aabe-7d161445570a:NORMAL:127.0.0.1:39789|RBW]]} size 0
[junit4] 2> 418799 INFO (Block report processor) [ ] BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:39789 is added to blk_1073741828_1004{UCState=UNDER_CONSTRUCTION, truncateBlock=null, primaryNodeIndex=-1, replicas=[ReplicaUC[[DISK]DS-3b7d1f1e-14d5-4aff-bae3-57c758864fdb:NORMAL:127.0.0.1:46631|RBW], ReplicaUC[[DISK]DS-5b37ce92-bdcd-4db5-aabe-7d161445570a:NORMAL:127.0.0.1:39789|RBW]]} size 0
[junit4] 2> 418924 INFO (qtp1425932665-2732) [n:127.0.0.1:38399_solr c:MoveReplicaHDFSTest_coll_true s:shard2 r:core_node8 x:MoveReplicaHDFSTest_coll_true_shard2_replica_n6] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.HdfsUpdateLog
[junit4] 2> 418924 INFO (qtp1425932665-2732) [n:127.0.0.1:38399_solr c:MoveReplicaHDFSTest_coll_true s:shard2 r:core_node8 x:MoveReplicaHDFSTest_coll_true_shard2_replica_n6] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 418924 INFO (qtp1425932665-2732) [n:127.0.0.1:38399_solr c:MoveReplicaHDFSTest_coll_true s:shard2 r:core_node8 x:MoveReplicaHDFSTest_coll_true_shard2_replica_n6] o.a.s.u.HdfsUpdateLog Initializing HdfsUpdateLog: tlogDfsReplication=3
[junit4] 2> 418940 INFO (qtp1425932665-2732) [n:127.0.0.1:38399_solr c:MoveReplicaHDFSTest_coll_true s:shard2 r:core_node8 x:MoveReplicaHDFSTest_coll_true_shard2_replica_n6] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 418940 INFO (qtp1425932665-2732) [n:127.0.0.1:38399_solr c:MoveReplicaHDFSTest_coll_true s:shard2 r:core_node8 x:MoveReplicaHDFSTest_coll_true_shard2_replica_n6] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 418950 INFO (qtp1828228189-2756) [n:127.0.0.1:45429_solr c:MoveReplicaHDFSTest_coll_true s:shard2 r:core_node7 x:MoveReplicaHDFSTest_coll_true_shard2_replica_n4] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.HdfsUpdateLog
[junit4] 2> 418950 INFO (qtp1828228189-2756) [n:127.0.0.1:45429_solr c:MoveReplicaHDFSTest_coll_true s:shard2 r:core_node7 x:MoveReplicaHDFSTest_coll_true_shard2_replica_n4] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 418950 INFO (qtp1828228189-2756) [n:127.0.0.1:45429_solr c:MoveReplicaHDFSTest_coll_true s:shard2 r:core_node7 x:MoveReplicaHDFSTest_coll_true_shard2_replica_n4] o.a.s.u.HdfsUpdateLog Initializing HdfsUpdateLog: tlogDfsReplication=3
[junit4] 2> 418954 INFO (qtp706789920-2737) [n:127.0.0.1:40241_solr c:MoveReplicaHDFSTest_coll_true s:shard1 r:core_node3 x:MoveReplicaHDFSTest_coll_true_shard1_replica_n1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.HdfsUpdateLog
[junit4] 2> 418954 INFO (qtp706789920-2737) [n:127.0.0.1:40241_solr c:MoveReplicaHDFSTest_coll_true s:shard1 r:core_node3 x:MoveReplicaHDFSTest_coll_true_shard1_replica_n1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 418954 INFO (qtp706789920-2737) [n:127.0.0.1:40241_solr c:MoveReplicaHDFSTest_coll_true s:shard1 r:core_node3 x:MoveReplicaHDFSTest_coll_true_shard1_replica_n1] o.a.s.u.HdfsUpdateLog Initializing HdfsUpdateLog: tlogDfsReplication=3
[junit4] 2> 418962 INFO (qtp1590780120-2757) [n:127.0.0.1:43723_solr c:MoveReplicaHDFSTest_coll_true s:shard1 r:core_node5 x:MoveReplicaHDFSTest_coll_true_shard1_replica_n2] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.HdfsUpdateLog
[junit4] 2> 418962 INFO (qtp1590780120-2757) [n:127.0.0.1:43723_solr c:MoveReplicaHDFSTest_coll_true s:shard1 r:core_node5 x:MoveReplicaHDFSTest_coll_true_shard1_replica_n2] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 418962 INFO (qtp1590780120-2757) [n:127.0.0.1:43723_solr c:MoveReplicaHDFSTest_coll_true s:shard1 r:core_node5 x:MoveReplicaHDFSTest_coll_true_shard1_replica_n2] o.a.s.u.HdfsUpdateLog Initializing HdfsUpdateLog: tlogDfsReplication=3
[junit4] 2> 418968 INFO (qtp1828228189-2756) [n:127.0.0.1:45429_solr c:MoveReplicaHDFSTest_coll_true s:shard2 r:core_node7 x:MoveReplicaHDFSTest_coll_true_shard2_replica_n4] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 418968 INFO (qtp1828228189-2756) [n:127.0.0.1:45429_solr c:MoveReplicaHDFSTest_coll_true s:shard2 r:core_node7 x:MoveReplicaHDFSTest_coll_true_shard2_replica_n4] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 418975 INFO (qtp706789920-2737) [n:127.0.0.1:40241_solr c:MoveReplicaHDFSTest_coll_true s:shard1 r:core_node3 x:MoveReplicaHDFSTest_coll_true_shard1_replica_n1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 418975 INFO (qtp706789920-2737) [n:127.0.0.1:40241_solr c:MoveReplicaHDFSTest_coll_true s:shard1 r:core_node3 x:MoveReplicaHDFSTest_coll_true_shard1_replica_n1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 419031 INFO (qtp1590780120-2757) [n:127.0.0.1:43723_solr c:MoveReplicaHDFSTest_coll_true s:shard1 r:core_node5 x:MoveReplicaHDFSTest_coll_true_shard1_replica_n2] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 419031 INFO (qtp1590780120-2757) [n:127.0.0.1:43723_solr c:MoveReplicaHDFSTest_coll_true s:shard1 r:core_node5 x:MoveReplicaHDFSTest_coll_true_shard1_replica_n2] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 419077 INFO (qtp1425932665-2732) [n:127.0.0.1:38399_solr c:MoveReplicaHDFSTest_coll_true s:shard2 r:core_node8 x:MoveReplicaHDFSTest_coll_true_shard2_replica_n6] o.a.s.s.SolrIndexSearcher Opening [Searcher@57047965[MoveReplicaHDFSTest_coll_true_shard2_replica_n6] main]
[junit4] 2> 419078 INFO (qtp1425932665-2732) [n:127.0.0.1:38399_solr c:MoveReplicaHDFSTest_coll_true s:shard2 r:core_node8 x:MoveReplicaHDFSTest_coll_true_shard2_replica_n6] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 419079 INFO (qtp1425932665-2732) [n:127.0.0.1:38399_solr c:MoveReplicaHDFSTest_coll_true s:shard2 r:core_node8 x:MoveReplicaHDFSTest_coll_true_shard2_replica_n6] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 419081 INFO (qtp1425932665-2732) [n:127.0.0.1:38399_solr c:MoveReplicaHDFSTest_coll_true s:shard2 r:core_node8 x:MoveReplicaHDFSTest_coll_true_shard2_replica_n6] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
[junit4] 2> 419104 INFO (searcherExecutor-1182-thread-1-processing-n:127.0.0.1:38399_solr x:MoveReplicaHDFSTest_coll_true_shard2_replica_n6 s:shard2 c:MoveReplicaHDFSTest_coll_true r:core_node8) [n:127.0.0.1:38399_solr c:MoveReplicaHDFSTest_coll_true s:shard2 r:core_node8 x:MoveReplicaHDFSTest_coll_true_shard2_replica_n6] o.a.s.c.SolrCore [MoveReplicaHDFSTest_coll_true_shard2_replica_n6] Registered new searcher Searcher@57047965[MoveReplicaHDFSTest_coll_true_shard2_replica_n6] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 419104 INFO (qtp1425932665-2732) [n:127.0.0.1:38399_solr c:MoveReplicaHDFSTest_coll_true s:shard2 r:core_node8 x:MoveReplicaHDFSTest_coll_true_shard2_replica_n6] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1588339745193197568
[junit4] 2> 419114 INFO (qtp706789920-2737) [n:127.0.0.1:40241_solr c:MoveReplicaHDFSTest_coll_true s:shard1 r:core_node3 x:MoveReplicaHDFSTest_coll_true_shard1_replica_n1] o.a.s.s.SolrIndexSearcher Opening [Searcher@2d1203ba[MoveReplicaHDFSTest_coll_true_shard1_replica_n1] main]
[junit4] 2> 419115 INFO (qtp706789920-2737) [n:127.0.0.1:40241_solr c:MoveReplicaHDFSTest_coll_true s:shard1 r:core_node3 x:MoveReplicaHDFSTest_coll_true_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 419115 INFO (qtp706789920-2737) [n:127.0.0.1:40241_solr c:MoveReplicaHDFSTest_coll_true s:shard1 r:core_node3 x:MoveReplicaHDFSTest_coll_true_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 419116 INFO (qtp706789920-2737) [n:127.0.0.1:40241_solr c:MoveReplicaHDFSTest_coll_true s:shard1 r:core_node3 x:MoveReplicaHDFSTest_coll_true_shard1_replica_n1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
[junit4] 2> 419123 INFO (qtp1828228189-2756) [n:127.0.0.1:45429_solr c:MoveReplicaHDFSTest_coll_true s:shard2 r:core_node7 x:MoveReplicaHDFSTest_coll_true_shard2_replica_n4] o.a.s.s.SolrIndexSearcher Opening [Searcher@2011cff3[MoveReplicaHDFSTest_coll_true_shard2_replica_n4] main]
[junit4] 2> 419124 INFO (qtp1828228189-2756) [n:127.0.0.1:45429_solr c:MoveReplicaHDFSTest_coll_true s:shard2 r:core_node7 x:MoveReplicaHDFSTest_coll_true_shard2_replica_n4] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 419124 INFO (qtp1828228189-2756) [n:127.0.0.1:45429_solr c:MoveReplicaHDFSTest_coll_true s:shard2 r:core_node7 x:MoveReplicaHDFSTest_coll_true_shard2_replica_n4] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 419125 INFO (qtp1828228189-2756) [n:127.0.0.1:45429_solr c:MoveReplicaHDFSTest_coll_true s:shard2 r:core_node7 x:MoveReplicaHDFSTest_coll_true_shard2_replica_n4] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
[junit4] 2> 419125 INFO (qtp1590780120-2757) [n:127.0.0.1:43723_solr c:MoveReplicaHDFSTest_coll_true s:shard1 r:core_node5 x:MoveReplicaHDFSTest_coll_true_shard1_replica_n2] o.a.s.s.SolrIndexSearcher Opening [Searcher@13861d5f[MoveReplicaHDFSTest_coll_true_shard1_replica_n2] main]
[junit4] 2> 419126 INFO (qtp1828228189-2756) [n:127.0.0.1:45429_solr c:MoveReplicaHDFSTest_coll_true s:shard2 r:core_node7 x:MoveReplicaHDFSTest_coll_true_shard2_replica_n4] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1588339745216266240
[junit4] 2> 419128 INFO (qtp1590780120-2757) [n:127.0.0.1:43723_solr c:MoveReplicaHDFSTest_coll_true s:shard1 r:core_node5 x:MoveReplicaHDFSTest_coll_true_shard1_replica_n2] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 419128 INFO (qtp1590780120-2757) [n:127.0.0.1:43723_solr c:MoveReplicaHDFSTest_coll_true s:shard1 r:core_node5 x:MoveReplicaHDFSTest_coll_true_shard1_replica_n2] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 419128 INFO (qtp1425932665-2732) [n:127.0.0.1:38399_solr c:MoveReplicaHDFSTest_coll_true s:shard2 r:core_node8 x:MoveReplicaHDFSTest_coll_true_shard2_replica_n6] o.a.s.c.ShardLeaderElectionContext Waiting until we see more replicas up for shard shard2: total=2 found=1 timeoutin=9999ms
[junit4] 2> 419129 INFO (qtp1590780120-2757) [n:127.0.0.1:43723_solr c:MoveReplicaHDFSTest_coll_true s:shard1 r:core_node5 x:MoveReplicaHDFSTest_coll_true_shard1_replica_n2] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
[junit4] 2> 419129 INFO (zkCallback-594-thread-2-processing-n:127.0.0.1:45429_solr) [n:127.0.0.1:45429_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/MoveReplicaHDFSTest_coll_true/state.json] for collection [MoveReplicaHDFSTest_coll_true] has occurred - updating... (live nodes size: [5])
[junit4] 2> 419130 INFO (zkCallback-581-thread-2-processing-n:127.0.0.1:43723_solr) [n:127.0.0.1:43723_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/MoveReplicaHDFSTest_coll_true/state.json] for collection [MoveReplicaHDFSTest_coll_true] has occurred - updating... (live nodes size: [5])
[junit4] 2> 419130 INFO (zkCallback-581-thread-3-processing-n:127.0.0.1:43723_solr) [n:127.0.0.1:43723_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/MoveReplicaHDFSTest_coll_true/state.json] for collection [MoveReplicaHDFSTest_coll_true] has occurred - updating... (live nodes size: [5])
[junit4] 2> 419129 INFO (zkCallback-584-thread-1-processing-n:127.0.0.1:40241_solr) [n:127.0.0.1:40241_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/MoveReplicaHDFSTest_coll_true/state.json] for collection [MoveReplicaHDFSTest_coll_true] has occurred - updating... (live nodes size: [5])
[junit4] 2> 419132 INFO (zkCallback-594-thread-1-processing-n:127.0.0.1:45429_solr) [n:127.0.0.1:45429_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/MoveReplicaHDFSTest_coll_true/state.json] for collection [MoveReplicaHDFSTest_coll_true] has occurred - updating... (live nodes size: [5])
[junit4] 2> 419132 INFO (zkCallback-592-thread-2-processing-n:127.0.0.1:38399_solr) [n:127.0.0.1:38399_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/MoveReplicaHDFSTest_coll_true/state.json] for collection [MoveReplicaHDFSTest_coll_true] has occurred - updating... (live nodes size: [5])
[junit4] 2> 419133 INFO (searcherExecutor-1179-thread-1-processing-n:127.0.0.1:40241_solr x:MoveReplicaHDFSTest_coll_true_shard1_replica_n1 s:shard1 c:MoveReplicaHDFSTest_coll_true r:core_node3) [n:127.0.0.1:40241_solr c:MoveReplicaHDFSTest_coll_true s:shard1 r:core_node3 x:MoveReplicaHDFSTest_coll_true_shard1_replica_n1] o.a.s.c.SolrCore [MoveReplicaHDFSTest_coll_true_shard1_replica_n1] Registered new searcher Searcher@2d1203ba[MoveReplicaHDFSTest_coll_true_shard1_replica_n1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 419130 INFO (zkCallback-584-thread-2-processing-n:127.0.0.1:40241_solr) [n:127.0.0.1:40241_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/MoveReplicaHDFSTest_coll_true/state.json] for collection [MoveReplicaHDFSTest_coll_true] has occurred - updating... (live nodes size: [5])
[junit4] 2> 419132 INFO (zkCallback-592-thread-1-processing-n:127.0.0.1:38399_solr) [n:127.0.0.1:38399_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/MoveReplicaHDFSTest_coll_true/state.json] for collection [MoveReplicaHDFSTest_coll_true] has occurred - updating... (live nodes size: [5])
[junit4] 2> 419134 INFO (qtp706789920-2737) [n:127.0.0.1:40241_solr c:MoveReplicaHDFSTest_coll_true s:shard1 r:core_node3 x:MoveReplicaHDFSTest_coll_true_shard1_replica_n1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1588339745224654848
[junit4] 2> 419134 INFO (searcherExecutor-1180-thread-1-processing-n:127.0.0.1:45429_solr x:MoveReplicaHDFSTest_coll_true_shard2_replica_n4 s:shard2 c:MoveReplicaHDFSTest_coll_true r:core_node7) [n:127.0.0.
[...truncated too long message...]
2> 751835 INFO (TEST-TestExecutePlanAction.testExecute-seed#[C3CBB6E52F2E74B5]) [ ] o.a.s.c.a.s.TestExecutePlanAction -------------_ FINAL STATE --------------
[junit4] 2> 751835 INFO (TEST-TestExecutePlanAction.testExecute-seed#[C3CBB6E52F2E74B5]) [ ] o.a.s.c.a.s.TestExecutePlanAction * Node values: {
[junit4] 2> "127.0.0.1:10006_solr":{
[junit4] 2> "sysprop.java.vendor":"Oracle Corporation",
[junit4] 2> "cores":2,
[junit4] 2> "host":"127.0.0.1",
[junit4] 2> "freedisk":960,
[junit4] 2> "sysprop.java.version":"10-ea",
[junit4] 2> "ip_1":"127",
[junit4] 2> "ip_3":"0",
[junit4] 2> "ip_2":"0",
[junit4] 2> "ip_4":"1",
[junit4] 2> "node":"127.0.0.1:10006_solr",
[junit4] 2> "metrics:solr.jvm:buffers.direct.Count":0,
[junit4] 2> "port":10006,
[junit4] 2> "heapUsage":123450000,
[junit4] 2> "sysLoadAvg":1.0,
[junit4] 2> "metrics:solr.node:ADMIN./admin/authorization.clientErrors:count":0},
[junit4] 2> "127.0.0.1:10008_solr":{
[junit4] 2> "sysprop.java.vendor":"Oracle Corporation",
[junit4] 2> "cores":0,
[junit4] 2> "host":"127.0.0.1",
[junit4] 2> "freedisk":1000,
[junit4] 2> "sysprop.java.version":"10-ea",
[junit4] 2> "ip_1":"127",
[junit4] 2> "ip_3":"0",
[junit4] 2> "ip_2":"0",
[junit4] 2> "ip_4":"1",
[junit4] 2> "node":"127.0.0.1:10008_solr",
[junit4] 2> "metrics:solr.jvm:buffers.direct.Count":0,
[junit4] 2> "port":10008,
[junit4] 2> "heapUsage":123450000,
[junit4] 2> "sysLoadAvg":1.0,
[junit4] 2> "metrics:solr.node:ADMIN./admin/authorization.clientErrors:count":0}}
[junit4] 2> 751835 INFO (TEST-TestExecutePlanAction.testExecute-seed#[C3CBB6E52F2E74B5]) [ ] o.a.s.c.a.s.TestExecutePlanAction * Live nodes: [127.0.0.1:10006_solr, 127.0.0.1:10008_solr]
[junit4] 2> 751835 INFO (TEST-TestExecutePlanAction.testExecute-seed#[C3CBB6E52F2E74B5]) [ ] o.a.s.c.a.s.TestExecutePlanAction * Collection testExecute state: DocCollection(testExecute//clusterstate.json/10)={
[junit4] 2> "replicationFactor":"2",
[junit4] 2> "pullReplicas":"0",
[junit4] 2> "router":{"name":"compositeId"},
[junit4] 2> "maxShardsPerNode":"1",
[junit4] 2> "autoAddReplicas":"false",
[junit4] 2> "nrtReplicas":"2",
[junit4] 2> "tlogReplicas":"0",
[junit4] 2> "autoCreated":"true",
[junit4] 2> "shards":{"shard1":{
[junit4] 2> "replicas":{
[junit4] 2> "core_node2":{
[junit4] 2> "core":"testExecute_shard1_replica_n2",
[junit4] 2> "leader":"true",
[junit4] 2> "INDEX.sizeInBytes":123450000,
[junit4] 2> "node_name":"127.0.0.1:10006_solr",
[junit4] 2> "state":"active",
[junit4] 2> "type":"NRT"},
[junit4] 2> "core_node4":{
[junit4] 2> "node_name":"127.0.0.1:10006_solr",
[junit4] 2> "core":"testExecute_shard1_replica_n3",
[junit4] 2> "state":"active",
[junit4] 2> "INDEX.sizeInBytes":123450000,
[junit4] 2> "type":"NRT"}},
[junit4] 2> "range":"80000000-7fffffff",
[junit4] 2> "state":"active"}}}
[junit4] 2> 751835 INFO (TEST-TestExecutePlanAction.testExecute-seed#[C3CBB6E52F2E74B5]) [ ] o.a.s.SolrTestCaseJ4 ###Ending testExecute
[junit4] 2> 751835 INFO (TEST-TestExecutePlanAction.testExecute-seed#[C3CBB6E52F2E74B5]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase
[junit4] 2>
[junit4] 2> 751835 INFO (TEST-TestExecutePlanAction.testExecute-seed#[C3CBB6E52F2E74B5]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase #############################################
[junit4] 2> 751835 INFO (TEST-TestExecutePlanAction.testExecute-seed#[C3CBB6E52F2E74B5]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ############ FINAL CLUSTER STATS ############
[junit4] 2> 751835 INFO (TEST-TestExecutePlanAction.testExecute-seed#[C3CBB6E52F2E74B5]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase #############################################
[junit4] 2>
[junit4] 2> 751835 INFO (TEST-TestExecutePlanAction.testExecute-seed#[C3CBB6E52F2E74B5]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## Live nodes: 2
[junit4] 2> 751835 INFO (TEST-TestExecutePlanAction.testExecute-seed#[C3CBB6E52F2E74B5]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## Empty nodes: 1
[junit4] 2> 751835 INFO (TEST-TestExecutePlanAction.testExecute-seed#[C3CBB6E52F2E74B5]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## Dead nodes: 0
[junit4] 2> 751835 INFO (TEST-TestExecutePlanAction.testExecute-seed#[C3CBB6E52F2E74B5]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## Collections: [testExecute]
[junit4] 2> 751835 INFO (TEST-TestExecutePlanAction.testExecute-seed#[C3CBB6E52F2E74B5]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## Max replicas per node: 2
[junit4] 2> 751835 INFO (TEST-TestExecutePlanAction.testExecute-seed#[C3CBB6E52F2E74B5]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## Min replicas per node: 0
[junit4] 2> 751835 INFO (TEST-TestExecutePlanAction.testExecute-seed#[C3CBB6E52F2E74B5]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## Total replicas: 2
[junit4] 2> 751835 INFO (TEST-TestExecutePlanAction.testExecute-seed#[C3CBB6E52F2E74B5]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## * testExecute 2
[junit4] 2> 751835 INFO (TEST-TestExecutePlanAction.testExecute-seed#[C3CBB6E52F2E74B5]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## - active 2
[junit4] 2> 751835 INFO (TEST-TestExecutePlanAction.testExecute-seed#[C3CBB6E52F2E74B5]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ######### Final Solr op counts ##########
[junit4] 2> 751835 INFO (TEST-TestExecutePlanAction.testExecute-seed#[C3CBB6E52F2E74B5]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## - CREATE 1
[junit4] 2> 751835 INFO (TEST-TestExecutePlanAction.testExecute-seed#[C3CBB6E52F2E74B5]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## - DELETESTATUS 2
[junit4] 2> 751835 INFO (TEST-TestExecutePlanAction.testExecute-seed#[C3CBB6E52F2E74B5]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## - MOVEREPLICA 1
[junit4] 2> 751835 INFO (TEST-TestExecutePlanAction.testExecute-seed#[C3CBB6E52F2E74B5]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## - OVERSEERSTATUS 1
[junit4] 2> 751835 INFO (TEST-TestExecutePlanAction.testExecute-seed#[C3CBB6E52F2E74B5]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## - REQUESTSTATUS 2
[junit4] 2> 751835 INFO (TEST-TestExecutePlanAction.testExecute-seed#[C3CBB6E52F2E74B5]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ######### Autoscaling event counts ###########
[junit4] 2> 751836 WARN (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread woken up but we are closed, exiting.
[junit4] 2> 751836 DEBUG (SUITE-TestExecutePlanAction-seed#[C3CBB6E52F2E74B5]-worker) [ ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread has been closed explicitly
[junit4] 2> NOTE: leaving temporary files on disk at: /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.autoscaling.sim.TestExecutePlanAction_C3CBB6E52F2E74B5-001
[junit4] 2> NOTE: test params are: codec=Asserting(Lucene70): {}, docValues:{}, maxPointsInLeafNode=1186, maxMBSortInHeap=7.876462070873278, sim=RandomSimilarity(queryNorm=true): {}, locale=fa, timezone=US/Samoa
[junit4] 2> NOTE: Linux 4.10.0-40-generic amd64/Oracle Corporation 10-ea (64-bit)/cpus=8,threads=1,free=162454880,total=508035072
[junit4] 2> NOTE: All tests run in this JVM: [SuggesterFSTTest, PathHierarchyTokenizerFactoryTest, ReplaceNodeTest, TestSolrCoreSnapshots, TestZkAclsWithHadoopAuth, TolerantUpdateProcessorTest, ConfigureRecoveryStrategyTest, DeleteShardTest, TestNumericTerms64, ShardRoutingTest, HighlighterConfigTest, TestConfigSetProperties, PreAnalyzedFieldManagedSchemaCloudTest, DocValuesMissingTest, DistributedIntervalFacetingTest, NodeAddedTriggerTest, TestClusterStateProvider, TestLeaderElectionZkExpiry, TestConfigSetsAPIExclusivity, TestFastWriter, TestCollectionAPI, TestStreamBody, TestSolrQueryParser, HdfsAutoAddReplicasIntegrationTest, BinaryUpdateRequestHandlerTest, CloudExitableDirectoryReaderTest, TestRestoreCore, FullHLLTest, TestCollationField, TestNonDefinedSimilarityFactory, TestDeleteCollectionOnDownNodes, TestCustomDocTransformer, InfoHandlerTest, TestHdfsBackupRestoreCore, BasicAuthStandaloneTest, TestAnalyzedSuggestions, PolyFieldTest, TestJsonFacets, TestOmitPositions, TestSolrConfigHandler, SignatureUpdateProcessorFactoryTest, AutoScalingHandlerTest, RequestHandlersTest, TestDelegationWithHadoopAuth, PrimUtilsTest, TestSolrCloudWithKerberosAlt, UtilsToolTest, TestBackupRepositoryFactory, TestCustomSort, TestInPlaceUpdatesStandalone, TestSortByMinMaxFunction, DistributedVersionInfoTest, TestStressRecovery, NumericFieldsTest, TestLuceneMatchVersion, HdfsChaosMonkeySafeLeaderTest, DeleteLastCustomShardedReplicaTest, AddSchemaFieldsUpdateProcessorFactoryTest, TestNodeAddedTrigger, TestMultiWordSynonyms, HdfsSyncSliceTest, OpenExchangeRatesOrgProviderTest, TestJmxIntegration, TestExclusionRuleCollectionAccess, TestManagedResourceStorage, CollectionsAPIDistributedZkTest, TestBlobHandler, TestCustomStream, TestFieldCacheSort, SolrIndexConfigTest, SOLR749Test, TestReplicaProperties, OpenCloseCoreStressTest, CdcrReplicationHandlerTest, SimpleMLTQParserTest, FacetPivotSmallTest, BigEndianAscendingWordSerializerTest, LukeRequestHandlerTest, CoreAdminCreateDiscoverTest, TestDocSet, TestLegacyNumericUtils, TestSimpleQParserPlugin, HttpPartitionTest, CollectionTooManyReplicasTest, TestExecutePlanAction]
[junit4] Completed [229/764 (2!)] on J1 in 0.74s, 2 tests, 1 error <<< FAILURES!
[...truncated 42734 lines...]
[JENKINS-EA] Lucene-Solr-7.x-Linux (64bit/jdk-10-ea+37) - Build #
1090 - Failure!
Posted by Policeman Jenkins Server <je...@thetaphi.de>.
Build: https://jenkins.thetaphi.de/job/Lucene-Solr-7.x-Linux/1090/
Java: 64bit/jdk-10-ea+37 -XX:-UseCompressedOops -XX:+UseSerialGC
1 tests failed.
FAILED: org.apache.solr.cloud.LegacyCloudClusterPropTest.testCreateCollectionSwitchLegacyCloud
Error Message:
Error from server at https://127.0.0.1:35301/solr: create the collection time out:180s
Stack Trace:
org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at https://127.0.0.1:35301/solr: create the collection time out:180s
at __randomizedtesting.SeedInfo.seed([E08BDD6E79B89D5E:318C2FEBDDB7166C]:0)
at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:643)
at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:255)
at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:244)
at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:483)
at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:413)
at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1104)
at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:884)
at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:817)
at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:194)
at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:211)
at org.apache.solr.cloud.LegacyCloudClusterPropTest.createAndTest(LegacyCloudClusterPropTest.java:86)
at org.apache.solr.cloud.LegacyCloudClusterPropTest.testCreateCollectionSwitchLegacyCloud(LegacyCloudClusterPropTest.java:69)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:564)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.base/java.lang.Thread.run(Thread.java:844)
Build Log:
[...truncated 11921 lines...]
[junit4] Suite: org.apache.solr.cloud.LegacyCloudClusterPropTest
[junit4] 2> Creating dataDir: /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.LegacyCloudClusterPropTest_E08BDD6E79B89D5E-001/init-core-data-001
[junit4] 2> 151553 WARN (SUITE-LegacyCloudClusterPropTest-seed#[E08BDD6E79B89D5E]-worker) [ ] o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=135 numCloses=135
[junit4] 2> 151553 INFO (SUITE-LegacyCloudClusterPropTest-seed#[E08BDD6E79B89D5E]-worker) [ ] o.a.s.SolrTestCaseJ4 Using PointFields (NUMERIC_POINTS_SYSPROP=true) w/NUMERIC_DOCVALUES_SYSPROP=false
[junit4] 2> 151554 INFO (SUITE-LegacyCloudClusterPropTest-seed#[E08BDD6E79B89D5E]-worker) [ ] o.a.s.SolrTestCaseJ4 Randomized ssl (true) and clientAuth (false) via: @org.apache.solr.util.RandomizeSSL(reason="", value=0.0/0.0, ssl=0.0/0.0, clientAuth=0.0/0.0)
[junit4] 2> 151555 INFO (SUITE-LegacyCloudClusterPropTest-seed#[E08BDD6E79B89D5E]-worker) [ ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
[junit4] 2> 151555 INFO (SUITE-LegacyCloudClusterPropTest-seed#[E08BDD6E79B89D5E]-worker) [ ] o.a.s.c.MiniSolrCloudCluster Starting cluster of 1 servers in /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.LegacyCloudClusterPropTest_E08BDD6E79B89D5E-001/tempDir-001
[junit4] 2> 151555 INFO (SUITE-LegacyCloudClusterPropTest-seed#[E08BDD6E79B89D5E]-worker) [ ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
[junit4] 2> 151555 INFO (Thread-159) [ ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 151555 INFO (Thread-159) [ ] o.a.s.c.ZkTestServer Starting server
[junit4] 2> 151562 ERROR (Thread-159) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 151655 INFO (SUITE-LegacyCloudClusterPropTest-seed#[E08BDD6E79B89D5E]-worker) [ ] o.a.s.c.ZkTestServer start zk server on port:40221
[junit4] 2> 151658 INFO (zkConnectionManagerCallback-199-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 151682 INFO (jetty-launcher-196-thread-1) [ ] o.e.j.s.Server jetty-9.3.20.v20170531
[junit4] 2> 151683 INFO (jetty-launcher-196-thread-1) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@33748fd6{/solr,null,AVAILABLE}
[junit4] 2> 151685 INFO (jetty-launcher-196-thread-1) [ ] o.e.j.s.AbstractConnector Started ServerConnector@608989f1{SSL,[ssl, http/1.1]}{127.0.0.1:35301}
[junit4] 2> 151685 INFO (jetty-launcher-196-thread-1) [ ] o.e.j.s.Server Started @153130ms
[junit4] 2> 151685 INFO (jetty-launcher-196-thread-1) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=35301}
[junit4] 2> 151685 ERROR (jetty-launcher-196-thread-1) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 151685 INFO (jetty-launcher-196-thread-1) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version 7.3.0
[junit4] 2> 151685 INFO (jetty-launcher-196-thread-1) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 151685 INFO (jetty-launcher-196-thread-1) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 151685 INFO (jetty-launcher-196-thread-1) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2017-12-31T19:17:50.373880Z
[junit4] 2> 151687 INFO (zkConnectionManagerCallback-201-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 151688 INFO (jetty-launcher-196-thread-1) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 151693 INFO (jetty-launcher-196-thread-1) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:40221/solr
[junit4] 2> 151695 INFO (zkConnectionManagerCallback-205-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 151696 INFO (zkConnectionManagerCallback-207-thread-1-processing-n:127.0.0.1:35301_solr) [n:127.0.0.1:35301_solr ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 151727 INFO (jetty-launcher-196-thread-1) [n:127.0.0.1:35301_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 151727 INFO (jetty-launcher-196-thread-1) [n:127.0.0.1:35301_solr ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:35301_solr
[junit4] 2> 151727 INFO (jetty-launcher-196-thread-1) [n:127.0.0.1:35301_solr ] o.a.s.c.Overseer Overseer (id=99270516424507395-127.0.0.1:35301_solr-n_0000000000) starting
[junit4] 2> 151732 INFO (jetty-launcher-196-thread-1) [n:127.0.0.1:35301_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:35301_solr
[junit4] 2> 151733 INFO (zkCallback-206-thread-1-processing-n:127.0.0.1:35301_solr) [n:127.0.0.1:35301_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 151761 INFO (jetty-launcher-196-thread-1) [n:127.0.0.1:35301_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_35301.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@3c1fbe5c
[junit4] 2> 151765 INFO (jetty-launcher-196-thread-1) [n:127.0.0.1:35301_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_35301.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@3c1fbe5c
[junit4] 2> 151765 INFO (jetty-launcher-196-thread-1) [n:127.0.0.1:35301_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_35301.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@3c1fbe5c
[junit4] 2> 151766 INFO (jetty-launcher-196-thread-1) [n:127.0.0.1:35301_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.LegacyCloudClusterPropTest_E08BDD6E79B89D5E-001/tempDir-001/node1/.
[junit4] 2> 151777 INFO (zkConnectionManagerCallback-211-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 151779 INFO (zkConnectionManagerCallback-215-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 151780 INFO (SUITE-LegacyCloudClusterPropTest-seed#[E08BDD6E79B89D5E]-worker) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 151780 INFO (SUITE-LegacyCloudClusterPropTest-seed#[E08BDD6E79B89D5E]-worker) [ ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:40221/solr ready
[junit4] 2> 151783 INFO (TEST-LegacyCloudClusterPropTest.testCreateCollectionSwitchLegacyCloud-seed#[E08BDD6E79B89D5E]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testCreateCollectionSwitchLegacyCloud
[junit4] 2> 151794 INFO (qtp536175940-971) [n:127.0.0.1:35301_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :clusterprop with params val=true&name=legacyCloud&action=CLUSTERPROP&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 151795 INFO (qtp536175940-971) [n:127.0.0.1:35301_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={val=true&name=legacyCloud&action=CLUSTERPROP&wt=javabin&version=2} status=0 QTime=1
[junit4] 2> 151797 INFO (qtp536175940-972) [n:127.0.0.1:35301_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params replicationFactor=1&collection.configName=conf&maxShardsPerNode=1&name=legacyTrue&nrtReplicas=1&action=CREATE&numShards=1&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 151798 INFO (OverseerThreadFactory-347-thread-1-processing-n:127.0.0.1:35301_solr) [n:127.0.0.1:35301_solr ] o.a.s.c.CreateCollectionCmd Create collection legacyTrue
[junit4] 2> 151906 INFO (qtp536175940-967) [n:127.0.0.1:35301_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf&newCollection=true&name=legacyTrue_shard1_replica_n1&action=CREATE&numShards=1&collection=legacyTrue&shard=shard1&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 151906 INFO (qtp536175940-967) [n:127.0.0.1:35301_solr ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 152009 INFO (zkCallback-206-thread-1-processing-n:127.0.0.1:35301_solr) [n:127.0.0.1:35301_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/legacyTrue/state.json] for collection [legacyTrue] has occurred - updating... (live nodes size: [1])
[junit4] 2> 152918 INFO (qtp536175940-967) [n:127.0.0.1:35301_solr c:legacyTrue s:shard1 x:legacyTrue_shard1_replica_n1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.3.0
[junit4] 2> 152928 INFO (qtp536175940-967) [n:127.0.0.1:35301_solr c:legacyTrue s:shard1 x:legacyTrue_shard1_replica_n1] o.a.s.s.IndexSchema [legacyTrue_shard1_replica_n1] Schema name=minimal
[junit4] 2> 152931 INFO (qtp536175940-967) [n:127.0.0.1:35301_solr c:legacyTrue s:shard1 x:legacyTrue_shard1_replica_n1] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
[junit4] 2> 152931 INFO (qtp536175940-967) [n:127.0.0.1:35301_solr c:legacyTrue s:shard1 x:legacyTrue_shard1_replica_n1] o.a.s.c.CoreContainer Creating SolrCore 'legacyTrue_shard1_replica_n1' using configuration from collection legacyTrue, trusted=true
[junit4] 2> 152932 INFO (qtp536175940-967) [n:127.0.0.1:35301_solr c:legacyTrue s:shard1 x:legacyTrue_shard1_replica_n1] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_35301.solr.core.legacyTrue.shard1.replica_n1' (registry 'solr.core.legacyTrue.shard1.replica_n1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@3c1fbe5c
[junit4] 2> 152932 INFO (qtp536175940-967) [n:127.0.0.1:35301_solr c:legacyTrue s:shard1 x:legacyTrue_shard1_replica_n1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 152932 INFO (qtp536175940-967) [n:127.0.0.1:35301_solr c:legacyTrue s:shard1 x:legacyTrue_shard1_replica_n1] o.a.s.c.SolrCore [[legacyTrue_shard1_replica_n1] ] Opening new SolrCore at [/home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.LegacyCloudClusterPropTest_E08BDD6E79B89D5E-001/tempDir-001/node1/legacyTrue_shard1_replica_n1], dataDir=[/home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.LegacyCloudClusterPropTest_E08BDD6E79B89D5E-001/tempDir-001/node1/./legacyTrue_shard1_replica_n1/data/]
[junit4] 2> 152979 INFO (qtp536175940-967) [n:127.0.0.1:35301_solr c:legacyTrue s:shard1 x:legacyTrue_shard1_replica_n1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 152979 INFO (qtp536175940-967) [n:127.0.0.1:35301_solr c:legacyTrue s:shard1 x:legacyTrue_shard1_replica_n1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 152980 INFO (qtp536175940-967) [n:127.0.0.1:35301_solr c:legacyTrue s:shard1 x:legacyTrue_shard1_replica_n1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 152980 INFO (qtp536175940-967) [n:127.0.0.1:35301_solr c:legacyTrue s:shard1 x:legacyTrue_shard1_replica_n1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 152981 INFO (qtp536175940-967) [n:127.0.0.1:35301_solr c:legacyTrue s:shard1 x:legacyTrue_shard1_replica_n1] o.a.s.s.SolrIndexSearcher Opening [Searcher@463405fa[legacyTrue_shard1_replica_n1] main]
[junit4] 2> 152981 INFO (qtp536175940-967) [n:127.0.0.1:35301_solr c:legacyTrue s:shard1 x:legacyTrue_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf
[junit4] 2> 152982 INFO (qtp536175940-967) [n:127.0.0.1:35301_solr c:legacyTrue s:shard1 x:legacyTrue_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf
[junit4] 2> 152982 INFO (qtp536175940-967) [n:127.0.0.1:35301_solr c:legacyTrue s:shard1 x:legacyTrue_shard1_replica_n1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
[junit4] 2> 152983 INFO (searcherExecutor-350-thread-1-processing-n:127.0.0.1:35301_solr x:legacyTrue_shard1_replica_n1 s:shard1 c:legacyTrue) [n:127.0.0.1:35301_solr c:legacyTrue s:shard1 x:legacyTrue_shard1_replica_n1] o.a.s.c.SolrCore [legacyTrue_shard1_replica_n1] Registered new searcher Searcher@463405fa[legacyTrue_shard1_replica_n1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 152983 INFO (qtp536175940-967) [n:127.0.0.1:35301_solr c:legacyTrue s:shard1 x:legacyTrue_shard1_replica_n1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1588328264285290496
[junit4] 2> 152985 INFO (qtp536175940-967) [n:127.0.0.1:35301_solr c:legacyTrue s:shard1 x:legacyTrue_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
[junit4] 2> 152985 INFO (qtp536175940-967) [n:127.0.0.1:35301_solr c:legacyTrue s:shard1 x:legacyTrue_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
[junit4] 2> 152985 INFO (qtp536175940-967) [n:127.0.0.1:35301_solr c:legacyTrue s:shard1 x:legacyTrue_shard1_replica_n1] o.a.s.c.SyncStrategy Sync replicas to https://127.0.0.1:35301/solr/legacyTrue_shard1_replica_n1/
[junit4] 2> 152985 INFO (qtp536175940-967) [n:127.0.0.1:35301_solr c:legacyTrue s:shard1 x:legacyTrue_shard1_replica_n1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
[junit4] 2> 152986 INFO (qtp536175940-967) [n:127.0.0.1:35301_solr c:legacyTrue s:shard1 x:legacyTrue_shard1_replica_n1] o.a.s.c.SyncStrategy https://127.0.0.1:35301/solr/legacyTrue_shard1_replica_n1/ has no replicas
[junit4] 2> 152986 INFO (qtp536175940-967) [n:127.0.0.1:35301_solr c:legacyTrue s:shard1 x:legacyTrue_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
[junit4] 2> 152987 INFO (qtp536175940-967) [n:127.0.0.1:35301_solr c:legacyTrue s:shard1 x:legacyTrue_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext I am the new leader: https://127.0.0.1:35301/solr/legacyTrue_shard1_replica_n1/ shard1
[junit4] 2> 331799 ERROR (qtp536175940-972) [n:127.0.0.1:35301_solr ] o.a.s.h.RequestHandlerBase org.apache.solr.common.SolrException: create the collection time out:180s
[junit4] 2> at org.apache.solr.handler.admin.CollectionsHandler.handleResponse(CollectionsHandler.java:314)
[junit4] 2> at org.apache.solr.handler.admin.CollectionsHandler.invokeAction(CollectionsHandler.java:246)
[junit4] 2> at org.apache.solr.handler.admin.CollectionsHandler.handleRequestBody(CollectionsHandler.java:224)
[junit4] 2> at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:177)
[junit4] 2> at org.apache.solr.servlet.HttpSolrCall.handleAdmin(HttpSolrCall.java:736)
[junit4] 2> at org.apache.solr.servlet.HttpSolrCall.handleAdminRequest(HttpSolrCall.java:717)
[junit4] 2> at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:498)
[junit4] 2> at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:382)
[junit4] 2> at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:326)
[junit4] 2> at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1759)
[junit4] 2> at org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:139)
[junit4] 2> at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1759)
[junit4] 2> at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:582)
[junit4] 2> at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:224)
[junit4] 2> at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1180)
[junit4] 2> at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:512)
[junit4] 2> at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
[junit4] 2> at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1112)
[junit4] 2> at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
[junit4] 2> at org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:493)
[junit4] 2> at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134)
[junit4] 2> at org.eclipse.jetty.server.Server.handle(Server.java:534)
[junit4] 2> at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:320)
[junit4] 2> at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:251)
[junit4] 2> at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:283)
[junit4] 2> at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:108)
[junit4] 2> at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:251)
[junit4] 2> at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:283)
[junit4] 2> at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:108)
[junit4] 2> at org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:93)
[junit4] 2> at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:303)
[junit4] 2> at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148)
[junit4] 2> at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:136)
[junit4] 2> at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671)
[junit4] 2> at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589)
[junit4] 2> at java.base/java.lang.Thread.run(Thread.java:844)
[junit4] 2>
[junit4] 2> 331800 INFO (qtp536175940-972) [n:127.0.0.1:35301_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={replicationFactor=1&collection.configName=conf&maxShardsPerNode=1&name=legacyTrue&nrtReplicas=1&action=CREATE&numShards=1&wt=javabin&version=2} status=500 QTime=180002
[junit4] 2> 331800 ERROR (qtp536175940-972) [n:127.0.0.1:35301_solr ] o.a.s.s.HttpSolrCall null:org.apache.solr.common.SolrException: create the collection time out:180s
[junit4] 2> at org.apache.solr.handler.admin.CollectionsHandler.handleResponse(CollectionsHandler.java:314)
[junit4] 2> at org.apache.solr.handler.admin.CollectionsHandler.invokeAction(CollectionsHandler.java:246)
[junit4] 2> at org.apache.solr.handler.admin.CollectionsHandler.handleRequestBody(CollectionsHandler.java:224)
[junit4] 2> at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:177)
[junit4] 2> at org.apache.solr.servlet.HttpSolrCall.handleAdmin(HttpSolrCall.java:736)
[junit4] 2> at org.apache.solr.servlet.HttpSolrCall.handleAdminRequest(HttpSolrCall.java:717)
[junit4] 2> at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:498)
[junit4] 2> at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:382)
[junit4] 2> at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:326)
[junit4] 2> at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1759)
[junit4] 2> at org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:139)
[junit4] 2> at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1759)
[junit4] 2> at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:582)
[junit4] 2> at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:224)
[junit4] 2> at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1180)
[junit4] 2> at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:512)
[junit4] 2> at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
[junit4] 2> at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1112)
[junit4] 2> at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
[junit4] 2> at org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:493)
[junit4] 2> at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134)
[junit4] 2> at org.eclipse.jetty.server.Server.handle(Server.java:534)
[junit4] 2> at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:320)
[junit4] 2> at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:251)
[junit4] 2> at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:283)
[junit4] 2> at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:108)
[junit4] 2> at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:251)
[junit4] 2> at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:283)
[junit4] 2> at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:108)
[junit4] 2> at org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:93)
[junit4] 2> at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:303)
[junit4] 2> at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148)
[junit4] 2> at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:136)
[junit4] 2> at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671)
[junit4] 2> at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589)
[junit4] 2> at java.base/java.lang.Thread.run(Thread.java:844)
[junit4] 2>
[junit4] 2> 331801 INFO (TEST-LegacyCloudClusterPropTest.testCreateCollectionSwitchLegacyCloud-seed#[E08BDD6E79B89D5E]) [ ] o.a.s.SolrTestCaseJ4 ###Ending testCreateCollectionSwitchLegacyCloud
[junit4] 2> NOTE: reproduce with: ant test -Dtestcase=LegacyCloudClusterPropTest -Dtests.method=testCreateCollectionSwitchLegacyCloud -Dtests.seed=E08BDD6E79B89D5E -Dtests.multiplier=3 -Dtests.slow=true -Dtests.locale=ja-JP -Dtests.timezone=America/Argentina/ComodRivadavia -Dtests.asserts=true -Dtests.file.encoding=UTF-8
[junit4] ERROR 180s J1 | LegacyCloudClusterPropTest.testCreateCollectionSwitchLegacyCloud <<<
[junit4] > Throwable #1: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at https://127.0.0.1:35301/solr: create the collection time out:180s
[junit4] > at __randomizedtesting.SeedInfo.seed([E08BDD6E79B89D5E:318C2FEBDDB7166C]:0)
[junit4] > at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:643)
[junit4] > at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:255)
[junit4] > at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:244)
[junit4] > at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:483)
[junit4] > at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:413)
[junit4] > at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1104)
[junit4] > at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:884)
[junit4] > at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:817)
[junit4] > at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:194)
[junit4] > at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:211)
[junit4] > at org.apache.solr.cloud.LegacyCloudClusterPropTest.createAndTest(LegacyCloudClusterPropTest.java:86)
[junit4] > at org.apache.solr.cloud.LegacyCloudClusterPropTest.testCreateCollectionSwitchLegacyCloud(LegacyCloudClusterPropTest.java:69)
[junit4] > at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
[junit4] > at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
[junit4] > at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
[junit4] > at java.base/java.lang.reflect.Method.invoke(Method.java:564)
[junit4] > at java.base/java.lang.Thread.run(Thread.java:844)
[junit4] 2> 331805 INFO (jetty-closer-197-thread-1) [ ] o.e.j.s.AbstractConnector Stopped ServerConnector@608989f1{SSL,[ssl, http/1.1]}{127.0.0.1:0}
[junit4] 2> 331805 INFO (jetty-closer-197-thread-1) [ ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=971232790
[junit4] 2> 331805 INFO (jetty-closer-197-thread-1) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.node, tag=null
[junit4] 2> 331805 INFO (jetty-closer-197-thread-1) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@492726d9: rootName = solr_35301, domain = solr.node, service url = null, agent id = null] for registry solr.node / com.codahale.metrics.MetricRegistry@27e4e851
[junit4] 2> 331806 ERROR (OverseerThreadFactory-347-thread-1-processing-n:127.0.0.1:35301_solr) [n:127.0.0.1:35301_solr ] o.a.s.c.OverseerCollectionMessageHandler Error from shard: https://127.0.0.1:35301/solr
[junit4] 2> org.apache.solr.client.solrj.SolrServerException: IOException occured when talking to server at: https://127.0.0.1:35301/solr
[junit4] 2> at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:657)
[junit4] 2> at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:255)
[junit4] 2> at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:244)
[junit4] 2> at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1219)
[junit4] 2> at org.apache.solr.handler.component.HttpShardHandler.lambda$submit$0(HttpShardHandler.java:172)
[junit4] 2> at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
[junit4] 2> at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:514)
[junit4] 2> at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
[junit4] 2> at com.codahale.metrics.InstrumentedExecutorService$InstrumentedRunnable.run(InstrumentedExecutorService.java:176)
[junit4] 2> at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:188)
[junit4] 2> at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1135)
[junit4] 2> at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
[junit4] 2> at java.base/java.lang.Thread.run(Thread.java:844)
[junit4] 2> Caused by: org.apache.http.NoHttpResponseException: 127.0.0.1:35301 failed to respond
[junit4] 2> at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:141)
[junit4] 2> at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:56)
[junit4] 2> at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:259)
[junit4] 2> at org.apache.http.impl.DefaultBHttpClientConnection.receiveResponseHeader(DefaultBHttpClientConnection.java:163)
[junit4] 2> at org.apache.http.impl.conn.CPoolProxy.receiveResponseHeader(CPoolProxy.java:165)
[junit4] 2> at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:273)
[junit4] 2> at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:125)
[junit4] 2> at org.apache.solr.util.stats.InstrumentedHttpRequestExecutor.execute(InstrumentedHttpRequestExecutor.java:118)
[junit4] 2> at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:272)
[junit4] 2> at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:185)
[junit4] 2> at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89)
[junit4] 2> at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:111)
[junit4] 2> at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185)
[junit4] 2> at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83)
[junit4] 2> at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:56)
[junit4] 2> at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:542)
[junit4] 2> ... 12 more
[junit4] 2> 331808 ERROR (OverseerThreadFactory-347-thread-1-processing-n:127.0.0.1:35301_solr) [n:127.0.0.1:35301_solr ] o.a.s.c.OverseerCollectionMessageHandler Cleaning up collection [legacyTrue].
[junit4] 2> 331810 INFO (OverseerThreadFactory-347-thread-1-processing-n:127.0.0.1:35301_solr) [n:127.0.0.1:35301_solr ] o.a.s.c.OverseerCollectionMessageHandler Executing Collection Cmd : action=UNLOAD&deleteInstanceDir=true&deleteDataDir=true
[junit4] 2> 331812 ERROR (OverseerThreadFactory-347-thread-1-processing-n:127.0.0.1:35301_solr) [n:127.0.0.1:35301_solr ] o.a.s.c.OverseerCollectionMessageHandler Error from shard: https://127.0.0.1:35301/solr
[junit4] 2> org.apache.solr.client.solrj.SolrServerException: Server refused connection at: https://127.0.0.1:35301/solr
[junit4] 2> at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:650)
[junit4] 2> at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:255)
[junit4] 2> at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:244)
[junit4] 2> at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1219)
[junit4] 2> at org.apache.solr.handler.component.HttpShardHandler.lambda$submit$0(HttpShardHandler.java:172)
[junit4] 2> at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
[junit4] 2> at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:514)
[junit4] 2> at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
[junit4] 2> at com.codahale.metrics.InstrumentedExecutorService$InstrumentedRunnable.run(InstrumentedExecutorService.java:176)
[junit4] 2> at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:188)
[junit4] 2> at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1135)
[junit4] 2> at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
[junit4] 2> at java.base/java.lang.Thread.run(Thread.java:844)
[junit4] 2> Caused by: org.apache.http.conn.HttpHostConnectException: Connect to 127.0.0.1:35301 [/127.0.0.1] failed: Connection refused (Connection refused)
[junit4] 2> at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:159)
[junit4] 2> at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:359)
[junit4] 2> at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:381)
[junit4] 2> at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:237)
[junit4] 2> at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:185)
[junit4] 2> at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89)
[junit4] 2> at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:111)
[junit4] 2> at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185)
[junit4] 2> at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83)
[junit4] 2> at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:56)
[junit4] 2> at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:542)
[junit4] 2> ... 12 more
[junit4] 2> Caused by: java.net.ConnectException: Connection refused (Connection refused)
[junit4] 2> at java.base/java.net.PlainSocketImpl.socketConnect(Native Method)
[junit4] 2> at java.base/java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:400)
[junit4] 2> at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:243)
[junit4] 2> at java.base/java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:225)
[junit4] 2> at java.base/java.net.SocksSocketImpl.connect(SocksSocketImpl.java:402)
[junit4] 2> at java.base/java.net.Socket.connect(Socket.java:591)
[junit4] 2> at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:339)
[junit4] 2> at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142)
[junit4] 2> ... 22 more
[junit4] 2> 331814 INFO (jetty-closer-197-thread-1) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jvm, tag=null
[junit4] 2> 331814 INFO (jetty-closer-197-thread-1) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@58a58e79: rootName = solr_35301, domain = solr.jvm, service url = null, agent id = null] for registry solr.jvm / com.codahale.metrics.MetricRegistry@67c53bea
[junit4] 2> 331818 INFO (jetty-closer-197-thread-1) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jetty, tag=null
[junit4] 2> 331818 INFO (jetty-closer-197-thread-1) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@3ac2c458: rootName = solr_35301, domain = solr.jetty, service url = null, agent id = null] for registry solr.jetty / com.codahale.metrics.MetricRegistry@20e5ad86
[junit4] 2> 331819 INFO (jetty-closer-197-thread-1) [ ] o.a.s.c.ZkController Remove node as live in ZooKeeper:/live_nodes/127.0.0.1:35301_solr
[junit4] 2> 331820 INFO (jetty-closer-197-thread-1) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.cluster, tag=null
[junit4] 2> 331820 INFO (zkCallback-206-thread-4-processing-n:127.0.0.1:35301_solr) [n:127.0.0.1:35301_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (0)
[junit4] 2> 331821 INFO (coreCloseExecutor-355-thread-1) [n:127.0.0.1:35301_solr c:legacyTrue s:shard1 r:core_node2 x:legacyTrue_shard1_replica_n1] o.a.s.c.SolrCore [legacyTrue_shard1_replica_n1] CLOSING SolrCore org.apache.solr.core.SolrCore@342f1208
[junit4] 2> 331821 INFO (coreCloseExecutor-355-thread-1) [n:127.0.0.1:35301_solr c:legacyTrue s:shard1 r:core_node2 x:legacyTrue_shard1_replica_n1] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.legacyTrue.shard1.replica_n1, tag=875500040
[junit4] 2> 331821 INFO (coreCloseExecutor-355-thread-1) [n:127.0.0.1:35301_solr c:legacyTrue s:shard1 r:core_node2 x:legacyTrue_shard1_replica_n1] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@602dcc12: rootName = solr_35301, domain = solr.core.legacyTrue.shard1.replica_n1, service url = null, agent id = null] for registry solr.core.legacyTrue.shard1.replica_n1 / com.codahale.metrics.MetricRegistry@4eb2ceea
[junit4] 2> 331830 INFO (coreCloseExecutor-355-thread-1) [n:127.0.0.1:35301_solr c:legacyTrue s:shard1 r:core_node2 x:legacyTrue_shard1_replica_n1] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.legacyTrue.shard1.leader, tag=875500040
[junit4] 2> 331833 INFO (jetty-closer-197-thread-1) [ ] o.a.s.c.Overseer Overseer (id=99270516424507395-127.0.0.1:35301_solr-n_0000000000) closing
[junit4] 2> 331833 INFO (OverseerStateUpdate-99270516424507395-127.0.0.1:35301_solr-n_0000000000) [n:127.0.0.1:35301_solr ] o.a.s.c.Overseer Overseer Loop exiting : 127.0.0.1:35301_solr
[junit4] 2> 361848 INFO (zkCallback-206-thread-4-processing-n:127.0.0.1:35301_solr) [n:127.0.0.1:35301_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDeleted path:/collections/legacyTrue/state.json] for collection [legacyTrue] has occurred - updating... (live nodes size: [0])
[junit4] 2> 361849 ERROR (OverseerThreadFactory-347-thread-1-processing-n:127.0.0.1:35301_solr) [n:127.0.0.1:35301_solr ] o.a.s.c.OverseerCollectionMessageHandler Collection: legacyTrue operation: create failed:org.apache.solr.common.SolrException: Could not fully remove collection: legacyTrue
[junit4] 2> at org.apache.solr.cloud.DeleteCollectionCmd.call(DeleteCollectionCmd.java:112)
[junit4] 2> at org.apache.solr.cloud.OverseerCollectionMessageHandler.cleanupCollection(OverseerCollectionMessageHandler.java:688)
[junit4] 2> at org.apache.solr.cloud.CreateCollectionCmd.call(CreateCollectionCmd.java:240)
[junit4] 2> at org.apache.solr.cloud.OverseerCollectionMessageHandler.processMessage(OverseerCollectionMessageHandler.java:242)
[junit4] 2> at org.apache.solr.cloud.OverseerTaskProcessor$Runner.run(OverseerTaskProcessor.java:464)
[junit4] 2> at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:188)
[junit4] 2> at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1135)
[junit4] 2> at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
[junit4] 2> at java.base/java.lang.Thread.run(Thread.java:844)
[junit4] 2>
[junit4] 2> 361858 INFO (OverseerThreadFactory-347-thread-1-processing-n:127.0.0.1:35301_solr) [n:127.0.0.1:35301_solr ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000000 doesn't exist. Requestor may have disconnected from ZooKeeper
[junit4] 2> 361859 WARN (OverseerAutoScalingTriggerThread-99270516424507395-127.0.0.1:35301_solr-n_0000000000) [n:127.0.0.1:35301_solr ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread woken up but we are closed, exiting.
[junit4] 2> 361860 INFO (jetty-closer-197-thread-1) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@33748fd6{/solr,null,UNAVAILABLE}
[junit4] 2> 361890 ERROR (qtp536175940-967) [n:127.0.0.1:35301_solr c:legacyTrue s:shard1 x:legacyTrue_shard1_replica_n1] o.a.s.c.ZkController Error getting leader from zk
[junit4] 2> org.apache.solr.common.SolrException: No registered leader was found after waiting for 1220000ms , collection: legacyTrue slice: shard1 saw state=null with live_nodes=[]
[junit4] 2> at org.apache.solr.common.cloud.ZkStateReader.getLeaderRetry(ZkStateReader.java:771)
[junit4] 2> at org.apache.solr.common.cloud.ZkStateReader.getLeaderUrl(ZkStateReader.java:736)
[junit4] 2> at org.apache.solr.cloud.ZkController.getLeader(ZkController.java:1178)
[junit4] 2> at org.apache.solr.cloud.ZkController.register(ZkController.java:1079)
[junit4] 2> at org.apache.solr.cloud.ZkController.register(ZkController.java:1014)
[junit4] 2> at org.apache.solr.core.ZkContainer.lambda$registerInZk$0(ZkContainer.java:181)
[junit4] 2> at org.apache.solr.core.ZkContainer.registerInZk(ZkContainer.java:208)
[junit4] 2> at org.apache.solr.core.CoreContainer.registerCore(CoreContainer.java:888)
[junit4] 2> at org.apache.solr.core.CoreContainer.createFromDescriptor(CoreContainer.java:1049)
[junit4] 2> at org.apache.solr.core.CoreContainer.create(CoreContainer.java:950)
[junit4] 2> at org.apache.solr.handler.admin.CoreAdminOperation.lambda$static$0(CoreAdminOperation.java:91)
[junit4] 2> at org.apache.solr.handler.admin.CoreAdminOperation.execute(CoreAdminOperation.java:384)
[junit4] 2> at org.apache.solr.handler.admin.CoreAdminHandler$CallInfo.call(CoreAdminHandler.java:389)
[junit4] 2> at org.apache.solr.handler.admin.CoreAdminHandler.handleRequestBody(CoreAdminHandler.java:174)
[junit4] 2> at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:177)
[junit4] 2> at org.apache.solr.servlet.HttpSolrCall.handleAdmin(HttpSolrCall.java:736)
[junit4] 2> at org.apache.solr.servlet.HttpSolrCall.handleAdminRequest(HttpSolrCall.java:717)
[junit4] 2> at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:498)
[junit4] 2> at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:382)
[junit4] 2> at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:326)
[junit4] 2> at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1759)
[junit4] 2> at org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:139)
[junit4] 2> at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1759)
[junit4] 2> at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:582)
[junit4] 2> at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:224)
[junit4] 2> at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1180)
[junit4] 2> at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:512)
[junit4] 2> at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
[junit4] 2> at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1112)
[junit4] 2> at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
[junit4] 2> at org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:426)
[junit4] 2> at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134)
[junit4] 2> at org.eclipse.jetty.server.Server.handle(Server.java:534)
[junit4] 2> at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:320)
[junit4] 2> at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:251)
[junit4] 2> at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:283)
[junit4] 2> at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:108)
[junit4] 2> at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:251)
[junit4] 2> at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:283)
[junit4] 2> at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:108)
[junit4] 2> at org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:93)
[junit4] 2> at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:303)
[junit4] 2> at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148)
[junit4] 2> at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:136)
[junit4] 2> at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671)
[junit4] 2> at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589)
[junit4] 2> at java.base/java.lang.Thread.run(Thread.java:844)
[junit4] 2> 361891 ERROR (qtp536175940-967) [n:127.0.0.1:35301_solr c:legacyTrue s:shard1 x:legacyTrue_shard1_replica_n1] o.a.s.c.ZkContainer :org.apache.solr.common.SolrException: Error getting leader from zk for shard shard1
[junit4] 2> at org.apache.solr.cloud.ZkController.getLeader(ZkController.java:1207)
[junit4] 2> at org.apache.solr.cloud.ZkController.register(ZkController.java:1079)
[junit4] 2> at org.apache.solr.cloud.ZkController.register(ZkController.java:1014)
[junit4] 2> at org.apache.solr.core.ZkContainer.lambda$registerInZk$0(ZkContainer.java:181)
[junit4] 2> at org.apache.solr.core.ZkContainer.registerInZk(ZkContainer.java:208)
[junit4] 2> at org.apache.solr.core.CoreContainer.registerCore(CoreContainer.java:888)
[junit4] 2> at org.apache.solr.core.CoreContainer.createFromDescriptor(CoreContainer.java:1049)
[junit4] 2> at org.apache.solr.core.CoreContainer.create(CoreContainer.java:950)
[junit4] 2> at org.apache.solr.handler.admin.CoreAdminOperation.lambda$static$0(CoreAdminOperation.java:91)
[junit4] 2> at org.apache.solr.handler.admin.CoreAdminOperation.execute(CoreAdminOperation.java:384)
[junit4] 2> at org.apache.solr.handler.admin.CoreAdminHandler$CallInfo.call(CoreAdminHandler.java:389)
[junit4] 2> at org.apache.solr.handler.admin.CoreAdminHandler.handleRequestBody(CoreAdminHandler.java:174)
[junit4] 2> at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:177)
[junit4] 2> at org.apache.solr.servlet.HttpSolrCall.handleAdmin(HttpSolrCall.java:736)
[junit4] 2> at org.apache.solr.servlet.HttpSolrCall.handleAdminRequest(HttpSolrCall.java:717)
[junit4] 2> at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:498)
[junit4] 2> at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:382)
[junit4] 2> at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:326)
[junit4] 2> at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1759)
[junit4] 2> at org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:139)
[junit4] 2> at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1759)
[junit4] 2> at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:582)
[junit4] 2> at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:224)
[junit4] 2> at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1180)
[junit4] 2> at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:512)
[junit4] 2> at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
[junit4] 2> at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1112)
[junit4] 2> at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
[junit4] 2> at org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:426)
[junit4] 2> at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134)
[junit4] 2> at org.eclipse.jetty.server.Server.handle(Server.java:534)
[junit4] 2> at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:320)
[junit4] 2> at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:251)
[junit4] 2> at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:283)
[junit4] 2> at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:108)
[junit4] 2> at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:251)
[junit4] 2> at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:283)
[junit4] 2> at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:108)
[junit4] 2> at org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:93)
[junit4] 2> at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:303)
[junit4] 2> at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148)
[junit4] 2> at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:136)
[junit4] 2> at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671)
[junit4] 2> at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589)
[junit4] 2> at java.base/java.lang.Thread.run(Thread.java:844)
[junit4] 2> Caused by: org.apache.solr.common.SolrException: No registered leader was found after waiting for 1220000ms , collection: legacyTrue slice: shard1 saw state=null with live_nodes=[]
[junit4] 2> at org.apache.solr.common.cloud.ZkStateReader.getLeaderRetry(ZkStateReader.java:771)
[junit4] 2> at org.apache.solr.common.cloud.ZkStateReader.getLeaderUrl(ZkStateReader.java:736)
[junit4] 2> at org.apache.solr.cloud.ZkController.getLeader(ZkController.java:1178)
[junit4] 2> ... 44 more
[junit4] 2>
[junit4] 2> 361897 INFO (qtp536175940-967) [n:127.0.0.1:35301_solr c:legacyTrue s:shard1 x:legacyTrue_shard1_replica_n1] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=conf&newCollection=true&name=legacyTrue_shard1_replica_n1&action=CREATE&numShards=1&collection=legacyTrue&shard=shard1&wt=javabin&version=2&replicaType=NRT} status=0 QTime=209991
[junit4] 2> 361898 ERROR (SUITE-LegacyCloudClusterPropTest-seed#[E08BDD6E79B89D5E]-worker) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 361899 INFO (SUITE-LegacyCloudClusterPropTest-seed#[E08BDD6E79B89D5E]-worker) [ ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:40221 40221
[junit4] 2> 362371 INFO (Thread-159) [ ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:40221 40221
[junit4] 2> 362372 WARN (Thread-159) [ ] o.a.s.c.ZkTestServer Watch limit violations:
[junit4] 2> Maximum concurrent create/delete watches above limit:
[junit4] 2>
[junit4] 2> 2 /solr/aliases.json
[junit4] 2>
[junit4] 2> Maximum concurrent data watches above limit:
[junit4] 2>
[junit4] 2> 2 /solr/clusterstate.json
[junit4] 2> 2 /solr/clusterprops.json
[junit4] 2>
[junit4] 2> Maximum concurrent children watches above limit:
[junit4] 2>
[junit4] 2> 2 /solr/live_nodes
[junit4] 2> 2 /solr/collections
[junit4] 2>
[junit4] 2> NOTE: leaving temporary files on disk at: /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.LegacyCloudClusterPropTest_E08BDD6E79B89D5E-001
[junit4] 2> NOTE: test params are: codec=CheapBastard, sim=RandomSimilarity(queryNorm=false): {}, locale=ja-JP, timezone=America/Argentina/ComodRivadavia
[junit4] 2> NOTE: Linux 4.10.0-40-generic amd64/Oracle Corporation 10-ea (64-bit)/cpus=8,threads=1,free=330630248,total=518979584
[junit4] 2> NOTE: All tests run in this JVM: [ShardRoutingCustomTest, TestFieldTypeResource, TestSurroundQueryParser, TestCloudRecovery, SolrCloudReportersTest, TestInPlaceUpdatesStandalone, TestSubQueryTransformerCrossCore, TestSolr4Spatial, ConfigureRecoveryStrategyTest, DocValuesMissingTest, LegacyCloudClusterPropTest]
[junit4] Completed [61/764 (1!)] on J1 in 210.83s, 1 test, 1 error <<< FAILURES!
[...truncated 2354 lines...]
[junit4] JVM J0: stdout was not empty, see: /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/temp/junit4-J0-20171231_191517_20915380220158523334868.sysout
[junit4] >>> JVM J0 emitted unexpected output (verbatim) ----
[junit4] java.lang.OutOfMemoryError: Java heap space
[junit4] Dumping heap to /home/jenkins/workspace/Lucene-Solr-7.x-Linux/heapdumps/java_pid26426.hprof ...
[junit4] Heap dump file created [212727682 bytes in 0.698 secs]
[junit4] <<< JVM J0: EOF ----
[...truncated 8595 lines...]
BUILD FAILED
/home/jenkins/workspace/Lucene-Solr-7.x-Linux/build.xml:835: The following error occurred while executing this line:
/home/jenkins/workspace/Lucene-Solr-7.x-Linux/build.xml:787: Some of the tests produced a heap dump, but did not fail. Maybe a suppressed OutOfMemoryError? Dumps created:
* java_pid26426.hprof
Total time: 69 minutes 44 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
[WARNINGS] Skipping publisher since build result is FAILURE
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any
[JENKINS] Lucene-Solr-7.x-Linux (32bit/jdk1.8.0_144) - Build # 1089
- Still unstable!
Posted by Policeman Jenkins Server <je...@thetaphi.de>.
Build: https://jenkins.thetaphi.de/job/Lucene-Solr-7.x-Linux/1089/
Java: 32bit/jdk1.8.0_144 -server -XX:+UseG1GC
2 tests failed.
FAILED: junit.framework.TestSuite.org.apache.solr.schema.PreAnalyzedFieldManagedSchemaCloudTest
Error Message:
Stack Trace:
java.util.concurrent.TimeoutException
at __randomizedtesting.SeedInfo.seed([10F0E6577DBF52E]:0)
at org.apache.solr.common.cloud.ZkStateReader.waitForState(ZkStateReader.java:1261)
at org.apache.solr.client.solrj.impl.CloudSolrClient.waitForState(CloudSolrClient.java:449)
at org.apache.solr.schema.PreAnalyzedFieldManagedSchemaCloudTest.setupCluster(PreAnalyzedFieldManagedSchemaCloudTest.java:45)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:874)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: org.apache.solr.cloud.ShardSplitTest.test
Error Message:
Timeout occured while waiting response from server at: http://127.0.0.1:45083/i/r
Stack Trace:
org.apache.solr.client.solrj.SolrServerException: Timeout occured while waiting response from server at: http://127.0.0.1:45083/i/r
at __randomizedtesting.SeedInfo.seed([10F0E6577DBF52E:895B31BFD92798D6]:0)
at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:654)
at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:255)
at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:244)
at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:483)
at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:413)
at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1104)
at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:884)
at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:817)
at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1219)
at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createCollection(AbstractFullDistribZkTestBase.java:1675)
at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createCollection(AbstractFullDistribZkTestBase.java:1614)
at org.apache.solr.cloud.ShardSplitTest.splitByRouteKeyTest(ShardSplitTest.java:751)
at org.apache.solr.cloud.ShardSplitTest.test(ShardSplitTest.java:102)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:993)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:968)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.net.SocketTimeoutException: Read timed out
at java.net.SocketInputStream.socketRead0(Native Method)
at java.net.SocketInputStream.socketRead(SocketInputStream.java:116)
at java.net.SocketInputStream.read(SocketInputStream.java:171)
at java.net.SocketInputStream.read(SocketInputStream.java:141)
at org.apache.http.impl.io.SessionInputBufferImpl.streamRead(SessionInputBufferImpl.java:137)
at org.apache.http.impl.io.SessionInputBufferImpl.fillBuffer(SessionInputBufferImpl.java:153)
at org.apache.http.impl.io.SessionInputBufferImpl.readLine(SessionInputBufferImpl.java:282)
at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:138)
at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:56)
at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:259)
at org.apache.http.impl.DefaultBHttpClientConnection.receiveResponseHeader(DefaultBHttpClientConnection.java:163)
at org.apache.http.impl.conn.CPoolProxy.receiveResponseHeader(CPoolProxy.java:165)
at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:273)
at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:125)
at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:272)
at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:185)
at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89)
at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:111)
at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185)
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83)
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:56)
at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:542)
... 53 more
Build Log:
[...truncated 12528 lines...]
[junit4] Suite: org.apache.solr.schema.PreAnalyzedFieldManagedSchemaCloudTest
[junit4] 2> 583081 INFO (SUITE-PreAnalyzedFieldManagedSchemaCloudTest-seed#[10F0E6577DBF52E]-worker) [ ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
[junit4] 2> Creating dataDir: /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J1/temp/solr.schema.PreAnalyzedFieldManagedSchemaCloudTest_10F0E6577DBF52E-001/init-core-data-001
[junit4] 2> 583081 WARN (SUITE-PreAnalyzedFieldManagedSchemaCloudTest-seed#[10F0E6577DBF52E]-worker) [ ] o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=2 numCloses=2
[junit4] 2> 583081 INFO (SUITE-PreAnalyzedFieldManagedSchemaCloudTest-seed#[10F0E6577DBF52E]-worker) [ ] o.a.s.SolrTestCaseJ4 Using PointFields (NUMERIC_POINTS_SYSPROP=true) w/NUMERIC_DOCVALUES_SYSPROP=true
[junit4] 2> 583082 INFO (SUITE-PreAnalyzedFieldManagedSchemaCloudTest-seed#[10F0E6577DBF52E]-worker) [ ] o.a.s.SolrTestCaseJ4 Randomized ssl (true) and clientAuth (true) via: @org.apache.solr.util.RandomizeSSL(reason=, ssl=NaN, value=NaN, clientAuth=NaN)
[junit4] 2> 583083 INFO (SUITE-PreAnalyzedFieldManagedSchemaCloudTest-seed#[10F0E6577DBF52E]-worker) [ ] o.a.s.c.MiniSolrCloudCluster Starting cluster of 2 servers in /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J1/temp/solr.schema.PreAnalyzedFieldManagedSchemaCloudTest_10F0E6577DBF52E-001/tempDir-001
[junit4] 2> 583083 INFO (SUITE-PreAnalyzedFieldManagedSchemaCloudTest-seed#[10F0E6577DBF52E]-worker) [ ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
[junit4] 2> 583083 INFO (Thread-3151) [ ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 583083 INFO (Thread-3151) [ ] o.a.s.c.ZkTestServer Starting server
[junit4] 2> 583085 ERROR (Thread-3151) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 583183 INFO (SUITE-PreAnalyzedFieldManagedSchemaCloudTest-seed#[10F0E6577DBF52E]-worker) [ ] o.a.s.c.ZkTestServer start zk server on port:37327
[junit4] 2> 583201 INFO (zkConnectionManagerCallback-1195-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 583206 WARN (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [ ] o.a.z.s.NIOServerCnxn caught end of stream exception
[junit4] 2> EndOfStreamException: Unable to read additional data from client sessionid 0x160ad62466e0000, likely client has closed socket
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:239)
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:203)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2> 583218 INFO (jetty-launcher-1192-thread-1) [ ] o.e.j.s.Server jetty-9.3.20.v20170531
[junit4] 2> 583219 INFO (jetty-launcher-1192-thread-2) [ ] o.e.j.s.Server jetty-9.3.20.v20170531
[junit4] 2> 583220 INFO (jetty-launcher-1192-thread-1) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@d3b46f{/solr,null,AVAILABLE}
[junit4] 2> 583220 INFO (jetty-launcher-1192-thread-2) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@195ca9c{/solr,null,AVAILABLE}
[junit4] 2> 583228 INFO (jetty-launcher-1192-thread-1) [ ] o.e.j.s.AbstractConnector Started ServerConnector@777320{SSL,[ssl, http/1.1]}{127.0.0.1:40043}
[junit4] 2> 583228 INFO (jetty-launcher-1192-thread-2) [ ] o.e.j.s.AbstractConnector Started ServerConnector@2f4946{SSL,[ssl, http/1.1]}{127.0.0.1:33731}
[junit4] 2> 583228 INFO (jetty-launcher-1192-thread-1) [ ] o.e.j.s.Server Started @585045ms
[junit4] 2> 583228 INFO (jetty-launcher-1192-thread-2) [ ] o.e.j.s.Server Started @585046ms
[junit4] 2> 583228 INFO (jetty-launcher-1192-thread-1) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=40043}
[junit4] 2> 583229 INFO (jetty-launcher-1192-thread-2) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=33731}
[junit4] 2> 583229 ERROR (jetty-launcher-1192-thread-1) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 583229 ERROR (jetty-launcher-1192-thread-2) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 583229 INFO (jetty-launcher-1192-thread-1) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 7.3.0
[junit4] 2> 583229 INFO (jetty-launcher-1192-thread-2) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 7.3.0
[junit4] 2> 583229 INFO (jetty-launcher-1192-thread-1) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 583229 INFO (jetty-launcher-1192-thread-2) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 583229 INFO (jetty-launcher-1192-thread-2) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 583229 INFO (jetty-launcher-1192-thread-1) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 583229 INFO (jetty-launcher-1192-thread-2) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2017-12-31T16:23:07.263Z
[junit4] 2> 583229 INFO (jetty-launcher-1192-thread-1) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2017-12-31T16:23:07.263Z
[junit4] 2> 583234 INFO (zkConnectionManagerCallback-1199-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 583234 INFO (zkConnectionManagerCallback-1198-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 583238 INFO (jetty-launcher-1192-thread-2) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 583238 INFO (jetty-launcher-1192-thread-1) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 583247 INFO (jetty-launcher-1192-thread-2) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:37327/solr
[junit4] 2> 583247 INFO (jetty-launcher-1192-thread-1) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:37327/solr
[junit4] 2> 583251 INFO (zkConnectionManagerCallback-1205-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 583251 INFO (zkConnectionManagerCallback-1207-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 583253 INFO (zkConnectionManagerCallback-1209-thread-1-processing-n:127.0.0.1:40043_solr) [n:127.0.0.1:40043_solr ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 583264 INFO (zkConnectionManagerCallback-1211-thread-1-processing-n:127.0.0.1:33731_solr) [n:127.0.0.1:33731_solr ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 583270 INFO (jetty-launcher-1192-thread-2) [n:127.0.0.1:33731_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 583271 INFO (jetty-launcher-1192-thread-2) [n:127.0.0.1:33731_solr ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:33731_solr
[junit4] 2> 583271 INFO (jetty-launcher-1192-thread-2) [n:127.0.0.1:33731_solr ] o.a.s.c.Overseer Overseer (id=99269829402165254-127.0.0.1:33731_solr-n_0000000000) starting
[junit4] 2> 583282 INFO (jetty-launcher-1192-thread-2) [n:127.0.0.1:33731_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:33731_solr
[junit4] 2> 583291 INFO (zkCallback-1210-thread-1-processing-n:127.0.0.1:33731_solr) [n:127.0.0.1:33731_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 583306 INFO (jetty-launcher-1192-thread-1) [n:127.0.0.1:40043_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 583307 INFO (jetty-launcher-1192-thread-1) [n:127.0.0.1:40043_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 583308 INFO (jetty-launcher-1192-thread-1) [n:127.0.0.1:40043_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:40043_solr
[junit4] 2> 583308 INFO (zkCallback-1210-thread-1-processing-n:127.0.0.1:33731_solr) [n:127.0.0.1:33731_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 583309 INFO (zkCallback-1208-thread-1-processing-n:127.0.0.1:40043_solr) [n:127.0.0.1:40043_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 583393 INFO (jetty-launcher-1192-thread-1) [n:127.0.0.1:40043_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_40043.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@178480c
[junit4] 2> 583401 INFO (jetty-launcher-1192-thread-1) [n:127.0.0.1:40043_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_40043.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@178480c
[junit4] 2> 583401 INFO (jetty-launcher-1192-thread-1) [n:127.0.0.1:40043_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_40043.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@178480c
[junit4] 2> 583403 INFO (jetty-launcher-1192-thread-1) [n:127.0.0.1:40043_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J1/temp/solr.schema.PreAnalyzedFieldManagedSchemaCloudTest_10F0E6577DBF52E-001/tempDir-001/node1/.
[junit4] 2> 583410 INFO (jetty-launcher-1192-thread-2) [n:127.0.0.1:33731_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_33731.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@178480c
[junit4] 2> 583417 INFO (jetty-launcher-1192-thread-2) [n:127.0.0.1:33731_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_33731.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@178480c
[junit4] 2> 583417 INFO (jetty-launcher-1192-thread-2) [n:127.0.0.1:33731_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_33731.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@178480c
[junit4] 2> 583418 INFO (jetty-launcher-1192-thread-2) [n:127.0.0.1:33731_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J1/temp/solr.schema.PreAnalyzedFieldManagedSchemaCloudTest_10F0E6577DBF52E-001/tempDir-001/node2/.
[junit4] 2> 583444 INFO (zkConnectionManagerCallback-1217-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 583453 INFO (zkConnectionManagerCallback-1221-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 583453 INFO (SUITE-PreAnalyzedFieldManagedSchemaCloudTest-seed#[10F0E6577DBF52E]-worker) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
[junit4] 2> 583454 INFO (SUITE-PreAnalyzedFieldManagedSchemaCloudTest-seed#[10F0E6577DBF52E]-worker) [ ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:37327/solr ready
[junit4] 2> 583497 INFO (qtp13209148-7511) [n:127.0.0.1:40043_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params replicationFactor=1&collection.configName=cloud-managed-preanalyzed&maxShardsPerNode=1&name=managed-preanalyzed&nrtReplicas=1&action=CREATE&numShards=2&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 583498 INFO (OverseerThreadFactory-2525-thread-1-processing-n:127.0.0.1:33731_solr) [n:127.0.0.1:33731_solr ] o.a.s.c.CreateCollectionCmd Create collection managed-preanalyzed
[junit4] 2> 583608 INFO (OverseerStateUpdate-99269829402165254-127.0.0.1:33731_solr-n_0000000000) [n:127.0.0.1:33731_solr ] o.a.s.c.o.SliceMutator createReplica() {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"managed-preanalyzed",
[junit4] 2> "shard":"shard1",
[junit4] 2> "core":"managed-preanalyzed_shard1_replica_n1",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"https://127.0.0.1:40043/solr",
[junit4] 2> "type":"NRT",
[junit4] 2> "waitForFinalState":"false"}
[junit4] 2> 583609 INFO (OverseerStateUpdate-99269829402165254-127.0.0.1:33731_solr-n_0000000000) [n:127.0.0.1:33731_solr ] o.a.s.c.o.SliceMutator createReplica() {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"managed-preanalyzed",
[junit4] 2> "shard":"shard2",
[junit4] 2> "core":"managed-preanalyzed_shard2_replica_n2",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"https://127.0.0.1:33731/solr",
[junit4] 2> "type":"NRT",
[junit4] 2> "waitForFinalState":"false"}
[junit4] 2> 583827 INFO (qtp13209148-7517) [n:127.0.0.1:40043_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node3&collection.configName=cloud-managed-preanalyzed&newCollection=true&name=managed-preanalyzed_shard1_replica_n1&action=CREATE&numShards=2&collection=managed-preanalyzed&shard=shard1&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 583828 INFO (qtp13209148-7517) [n:127.0.0.1:40043_solr ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 583874 INFO (qtp8779174-7518) [n:127.0.0.1:33731_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node4&collection.configName=cloud-managed-preanalyzed&newCollection=true&name=managed-preanalyzed_shard2_replica_n2&action=CREATE&numShards=2&collection=managed-preanalyzed&shard=shard2&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 583874 INFO (qtp8779174-7518) [n:127.0.0.1:33731_solr ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 583976 INFO (zkCallback-1208-thread-1-processing-n:127.0.0.1:40043_solr) [n:127.0.0.1:40043_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/managed-preanalyzed/state.json] for collection [managed-preanalyzed] has occurred - updating... (live nodes size: [2])
[junit4] 2> 583977 INFO (zkCallback-1208-thread-2-processing-n:127.0.0.1:40043_solr) [n:127.0.0.1:40043_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/managed-preanalyzed/state.json] for collection [managed-preanalyzed] has occurred - updating... (live nodes size: [2])
[junit4] 2> 583977 INFO (zkCallback-1210-thread-1-processing-n:127.0.0.1:33731_solr) [n:127.0.0.1:33731_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/managed-preanalyzed/state.json] for collection [managed-preanalyzed] has occurred - updating... (live nodes size: [2])
[junit4] 2> 583977 INFO (zkCallback-1210-thread-2-processing-n:127.0.0.1:33731_solr) [n:127.0.0.1:33731_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/managed-preanalyzed/state.json] for collection [managed-preanalyzed] has occurred - updating... (live nodes size: [2])
[junit4] 2> 584841 INFO (qtp13209148-7517) [n:127.0.0.1:40043_solr c:managed-preanalyzed s:shard1 r:core_node3 x:managed-preanalyzed_shard1_replica_n1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.3.0
[junit4] 2> 584847 INFO (qtp13209148-7517) [n:127.0.0.1:40043_solr c:managed-preanalyzed s:shard1 r:core_node3 x:managed-preanalyzed_shard1_replica_n1] o.a.s.s.IndexSchema [managed-preanalyzed_shard1_replica_n1] Schema name=managed-preanalyzed
[junit4] 2> 584850 INFO (qtp13209148-7517) [n:127.0.0.1:40043_solr c:managed-preanalyzed s:shard1 r:core_node3 x:managed-preanalyzed_shard1_replica_n1] o.a.s.s.IndexSchema Loaded schema managed-preanalyzed/1.6 with uniqueid field id
[junit4] 2> 584850 INFO (qtp13209148-7517) [n:127.0.0.1:40043_solr c:managed-preanalyzed s:shard1 r:core_node3 x:managed-preanalyzed_shard1_replica_n1] o.a.s.c.CoreContainer Creating SolrCore 'managed-preanalyzed_shard1_replica_n1' using configuration from collection managed-preanalyzed, trusted=true
[junit4] 2> 584850 INFO (qtp13209148-7517) [n:127.0.0.1:40043_solr c:managed-preanalyzed s:shard1 r:core_node3 x:managed-preanalyzed_shard1_replica_n1] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_40043.solr.core.managed-preanalyzed.shard1.replica_n1' (registry 'solr.core.managed-preanalyzed.shard1.replica_n1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@178480c
[junit4] 2> 584850 INFO (qtp13209148-7517) [n:127.0.0.1:40043_solr c:managed-preanalyzed s:shard1 r:core_node3 x:managed-preanalyzed_shard1_replica_n1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 584850 INFO (qtp13209148-7517) [n:127.0.0.1:40043_solr c:managed-preanalyzed s:shard1 r:core_node3 x:managed-preanalyzed_shard1_replica_n1] o.a.s.c.SolrCore [[managed-preanalyzed_shard1_replica_n1] ] Opening new SolrCore at [/home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J1/temp/solr.schema.PreAnalyzedFieldManagedSchemaCloudTest_10F0E6577DBF52E-001/tempDir-001/node1/managed-preanalyzed_shard1_replica_n1], dataDir=[/home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J1/temp/solr.schema.PreAnalyzedFieldManagedSchemaCloudTest_10F0E6577DBF52E-001/tempDir-001/node1/./managed-preanalyzed_shard1_replica_n1/data/]
[junit4] 2> 584883 INFO (qtp8779174-7518) [n:127.0.0.1:33731_solr c:managed-preanalyzed s:shard2 r:core_node4 x:managed-preanalyzed_shard2_replica_n2] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.3.0
[junit4] 2> 584890 INFO (qtp8779174-7518) [n:127.0.0.1:33731_solr c:managed-preanalyzed s:shard2 r:core_node4 x:managed-preanalyzed_shard2_replica_n2] o.a.s.s.IndexSchema [managed-preanalyzed_shard2_replica_n2] Schema name=managed-preanalyzed
[junit4] 2> 584893 INFO (qtp8779174-7518) [n:127.0.0.1:33731_solr c:managed-preanalyzed s:shard2 r:core_node4 x:managed-preanalyzed_shard2_replica_n2] o.a.s.s.IndexSchema Loaded schema managed-preanalyzed/1.6 with uniqueid field id
[junit4] 2> 584893 INFO (qtp8779174-7518) [n:127.0.0.1:33731_solr c:managed-preanalyzed s:shard2 r:core_node4 x:managed-preanalyzed_shard2_replica_n2] o.a.s.c.CoreContainer Creating SolrCore 'managed-preanalyzed_shard2_replica_n2' using configuration from collection managed-preanalyzed, trusted=true
[junit4] 2> 584893 INFO (qtp13209148-7517) [n:127.0.0.1:40043_solr c:managed-preanalyzed s:shard1 r:core_node3 x:managed-preanalyzed_shard1_replica_n1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 584893 INFO (qtp13209148-7517) [n:127.0.0.1:40043_solr c:managed-preanalyzed s:shard1 r:core_node3 x:managed-preanalyzed_shard1_replica_n1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 584893 INFO (qtp8779174-7518) [n:127.0.0.1:33731_solr c:managed-preanalyzed s:shard2 r:core_node4 x:managed-preanalyzed_shard2_replica_n2] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_33731.solr.core.managed-preanalyzed.shard2.replica_n2' (registry 'solr.core.managed-preanalyzed.shard2.replica_n2') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@178480c
[junit4] 2> 584893 INFO (qtp8779174-7518) [n:127.0.0.1:33731_solr c:managed-preanalyzed s:shard2 r:core_node4 x:managed-preanalyzed_shard2_replica_n2] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 584894 INFO (qtp8779174-7518) [n:127.0.0.1:33731_solr c:managed-preanalyzed s:shard2 r:core_node4 x:managed-preanalyzed_shard2_replica_n2] o.a.s.c.SolrCore [[managed-preanalyzed_shard2_replica_n2] ] Opening new SolrCore at [/home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J1/temp/solr.schema.PreAnalyzedFieldManagedSchemaCloudTest_10F0E6577DBF52E-001/tempDir-001/node2/managed-preanalyzed_shard2_replica_n2], dataDir=[/home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J1/temp/solr.schema.PreAnalyzedFieldManagedSchemaCloudTest_10F0E6577DBF52E-001/tempDir-001/node2/./managed-preanalyzed_shard2_replica_n2/data/]
[junit4] 2> 584894 INFO (qtp13209148-7517) [n:127.0.0.1:40043_solr c:managed-preanalyzed s:shard1 r:core_node3 x:managed-preanalyzed_shard1_replica_n1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 584894 INFO (qtp13209148-7517) [n:127.0.0.1:40043_solr c:managed-preanalyzed s:shard1 r:core_node3 x:managed-preanalyzed_shard1_replica_n1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 584895 INFO (qtp13209148-7517) [n:127.0.0.1:40043_solr c:managed-preanalyzed s:shard1 r:core_node3 x:managed-preanalyzed_shard1_replica_n1] o.a.s.s.SolrIndexSearcher Opening [Searcher@185c4c5[managed-preanalyzed_shard1_replica_n1] main]
[junit4] 2> 584896 INFO (qtp13209148-7517) [n:127.0.0.1:40043_solr c:managed-preanalyzed s:shard1 r:core_node3 x:managed-preanalyzed_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/cloud-managed-preanalyzed
[junit4] 2> 584896 INFO (qtp13209148-7517) [n:127.0.0.1:40043_solr c:managed-preanalyzed s:shard1 r:core_node3 x:managed-preanalyzed_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/cloud-managed-preanalyzed
[junit4] 2> 584896 INFO (qtp13209148-7517) [n:127.0.0.1:40043_solr c:managed-preanalyzed s:shard1 r:core_node3 x:managed-preanalyzed_shard1_replica_n1] o.a.s.s.ZkIndexSchemaReader Creating ZooKeeper watch for the managed schema at /configs/cloud-managed-preanalyzed/managed-schema
[junit4] 2> 584897 INFO (qtp13209148-7517) [n:127.0.0.1:40043_solr c:managed-preanalyzed s:shard1 r:core_node3 x:managed-preanalyzed_shard1_replica_n1] o.a.s.s.ZkIndexSchemaReader Current schema version 0 is already the latest
[junit4] 2> 584897 INFO (qtp13209148-7517) [n:127.0.0.1:40043_solr c:managed-preanalyzed s:shard1 r:core_node3 x:managed-preanalyzed_shard1_replica_n1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
[junit4] 2> 584897 INFO (qtp13209148-7517) [n:127.0.0.1:40043_solr c:managed-preanalyzed s:shard1 r:core_node3 x:managed-preanalyzed_shard1_replica_n1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1588317272335712256
[junit4] 2> 584898 INFO (searcherExecutor-2530-thread-1-processing-n:127.0.0.1:40043_solr x:managed-preanalyzed_shard1_replica_n1 s:shard1 c:managed-preanalyzed r:core_node3) [n:127.0.0.1:40043_solr c:managed-preanalyzed s:shard1 r:core_node3 x:managed-preanalyzed_shard1_replica_n1] o.a.s.c.SolrCore [managed-preanalyzed_shard1_replica_n1] Registered new searcher Searcher@185c4c5[managed-preanalyzed_shard1_replica_n1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 584902 INFO (qtp13209148-7517) [n:127.0.0.1:40043_solr c:managed-preanalyzed s:shard1 r:core_node3 x:managed-preanalyzed_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
[junit4] 2> 584902 INFO (qtp13209148-7517) [n:127.0.0.1:40043_solr c:managed-preanalyzed s:shard1 r:core_node3 x:managed-preanalyzed_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
[junit4] 2> 584902 INFO (qtp13209148-7517) [n:127.0.0.1:40043_solr c:managed-preanalyzed s:shard1 r:core_node3 x:managed-preanalyzed_shard1_replica_n1] o.a.s.c.SyncStrategy Sync replicas to https://127.0.0.1:40043/solr/managed-preanalyzed_shard1_replica_n1/
[junit4] 2> 584902 INFO (qtp13209148-7517) [n:127.0.0.1:40043_solr c:managed-preanalyzed s:shard1 r:core_node3 x:managed-preanalyzed_shard1_replica_n1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
[junit4] 2> 584902 INFO (qtp13209148-7517) [n:127.0.0.1:40043_solr c:managed-preanalyzed s:shard1 r:core_node3 x:managed-preanalyzed_shard1_replica_n1] o.a.s.c.SyncStrategy https://127.0.0.1:40043/solr/managed-preanalyzed_shard1_replica_n1/ has no replicas
[junit4] 2> 584902 INFO (qtp13209148-7517) [n:127.0.0.1:40043_solr c:managed-preanalyzed s:shard1 r:core_node3 x:managed-preanalyzed_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
[junit4] 2> 584904 INFO (qtp13209148-7517) [n:127.0.0.1:40043_solr c:managed-preanalyzed s:shard1 r:core_node3 x:managed-preanalyzed_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext I am the new leader: https://127.0.0.1:40043/solr/managed-preanalyzed_shard1_replica_n1/ shard1
[junit4] 2> 584998 INFO (qtp8779174-7518) [n:127.0.0.1:33731_solr c:managed-preanalyzed s:shard2 r:core_node4 x:managed-preanalyzed_shard2_replica_n2] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 584998 INFO (qtp8779174-7518) [n:127.0.0.1:33731_solr c:managed-preanalyzed s:shard2 r:core_node4 x:managed-preanalyzed_shard2_replica_n2] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 584999 INFO (qtp8779174-7518) [n:127.0.0.1:33731_solr c:managed-preanalyzed s:shard2 r:core_node4 x:managed-preanalyzed_shard2_replica_n2] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 584999 INFO (qtp8779174-7518) [n:127.0.0.1:33731_solr c:managed-preanalyzed s:shard2 r:core_node4 x:managed-preanalyzed_shard2_replica_n2] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 585000 INFO (qtp8779174-7518) [n:127.0.0.1:33731_solr c:managed-preanalyzed s:shard2 r:core_node4 x:managed-preanalyzed_shard2_replica_n2] o.a.s.s.SolrIndexSearcher Opening [Searcher@126ebae[managed-preanalyzed_shard2_replica_n2] main]
[junit4] 2> 585001 INFO (qtp8779174-7518) [n:127.0.0.1:33731_solr c:managed-preanalyzed s:shard2 r:core_node4 x:managed-preanalyzed_shard2_replica_n2] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/cloud-managed-preanalyzed
[junit4] 2> 585001 INFO (qtp8779174-7518) [n:127.0.0.1:33731_solr c:managed-preanalyzed s:shard2 r:core_node4 x:managed-preanalyzed_shard2_replica_n2] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/cloud-managed-preanalyzed
[junit4] 2> 585001 INFO (qtp8779174-7518) [n:127.0.0.1:33731_solr c:managed-preanalyzed s:shard2 r:core_node4 x:managed-preanalyzed_shard2_replica_n2] o.a.s.s.ZkIndexSchemaReader Creating ZooKeeper watch for the managed schema at /configs/cloud-managed-preanalyzed/managed-schema
[junit4] 2> 585002 INFO (qtp8779174-7518) [n:127.0.0.1:33731_solr c:managed-preanalyzed s:shard2 r:core_node4 x:managed-preanalyzed_shard2_replica_n2] o.a.s.s.ZkIndexSchemaReader Current schema version 0 is already the latest
[junit4] 2> 585002 INFO (qtp8779174-7518) [n:127.0.0.1:33731_solr c:managed-preanalyzed s:shard2 r:core_node4 x:managed-preanalyzed_shard2_replica_n2] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
[junit4] 2> 585003 INFO (qtp8779174-7518) [n:127.0.0.1:33731_solr c:managed-preanalyzed s:shard2 r:core_node4 x:managed-preanalyzed_shard2_replica_n2] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1588317272446861312
[junit4] 2> 585005 INFO (searcherExecutor-2532-thread-1-processing-n:127.0.0.1:33731_solr x:managed-preanalyzed_shard2_replica_n2 s:shard2 c:managed-preanalyzed r:core_node4) [n:127.0.0.1:33731_solr c:managed-preanalyzed s:shard2 r:core_node4 x:managed-preanalyzed_shard2_replica_n2] o.a.s.c.SolrCore [managed-preanalyzed_shard2_replica_n2] Registered new searcher Searcher@126ebae[managed-preanalyzed_shard2_replica_n2] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 585005 INFO (zkCallback-1210-thread-2-processing-n:127.0.0.1:33731_solr) [n:127.0.0.1:33731_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/managed-preanalyzed/state.json] for collection [managed-preanalyzed] has occurred - updating... (live nodes size: [2])
[junit4] 2> 585005 INFO (zkCallback-1208-thread-2-processing-n:127.0.0.1:40043_solr) [n:127.0.0.1:40043_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/managed-preanalyzed/state.json] for collection [managed-preanalyzed] has occurred - updating... (live nodes size: [2])
[junit4] 2> 585006 INFO (zkCallback-1210-thread-1-processing-n:127.0.0.1:33731_solr) [n:127.0.0.1:33731_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/managed-preanalyzed/state.json] for collection [managed-preanalyzed] has occurred - updating... (live nodes size: [2])
[junit4] 2> 585006 INFO (zkCallback-1208-thread-1-processing-n:127.0.0.1:40043_solr) [n:127.0.0.1:40043_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/managed-preanalyzed/state.json] for collection [managed-preanalyzed] has occurred - updating... (live nodes size: [2])
[junit4] 2> 585007 INFO (qtp8779174-7518) [n:127.0.0.1:33731_solr c:managed-preanalyzed s:shard2 r:core_node4 x:managed-preanalyzed_shard2_replica_n2] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
[junit4] 2> 585007 INFO (qtp8779174-7518) [n:127.0.0.1:33731_solr c:managed-preanalyzed s:shard2 r:core_node4 x:managed-preanalyzed_shard2_replica_n2] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
[junit4] 2> 585007 INFO (qtp8779174-7518) [n:127.0.0.1:33731_solr c:managed-preanalyzed s:shard2 r:core_node4 x:managed-preanalyzed_shard2_replica_n2] o.a.s.c.SyncStrategy Sync replicas to https://127.0.0.1:33731/solr/managed-preanalyzed_shard2_replica_n2/
[junit4] 2> 585007 INFO (qtp8779174-7518) [n:127.0.0.1:33731_solr c:managed-preanalyzed s:shard2 r:core_node4 x:managed-preanalyzed_shard2_replica_n2] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
[junit4] 2> 585007 INFO (qtp8779174-7518) [n:127.0.0.1:33731_solr c:managed-preanalyzed s:shard2 r:core_node4 x:managed-preanalyzed_shard2_replica_n2] o.a.s.c.SyncStrategy https://127.0.0.1:33731/solr/managed-preanalyzed_shard2_replica_n2/ has no replicas
[junit4] 2> 585007 INFO (qtp8779174-7518) [n:127.0.0.1:33731_solr c:managed-preanalyzed s:shard2 r:core_node4 x:managed-preanalyzed_shard2_replica_n2] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
[junit4] 2> 585011 INFO (qtp8779174-7518) [n:127.0.0.1:33731_solr c:managed-preanalyzed s:shard2 r:core_node4 x:managed-preanalyzed_shard2_replica_n2] o.a.s.c.ShardLeaderElectionContext I am the new leader: https://127.0.0.1:33731/solr/managed-preanalyzed_shard2_replica_n2/ shard2
[junit4] 2> 585046 INFO (qtp13209148-7517) [n:127.0.0.1:40043_solr c:managed-preanalyzed s:shard1 r:core_node3 x:managed-preanalyzed_shard1_replica_n1] o.a.s.c.ZkController I am the leader, no recovery necessary
[junit4] 2> 585054 INFO (qtp13209148-7517) [n:127.0.0.1:40043_solr c:managed-preanalyzed s:shard1 r:core_node3 x:managed-preanalyzed_shard1_replica_n1] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&coreNodeName=core_node3&collection.configName=cloud-managed-preanalyzed&newCollection=true&name=managed-preanalyzed_shard1_replica_n1&action=CREATE&numShards=2&collection=managed-preanalyzed&shard=shard1&wt=javabin&version=2&replicaType=NRT} status=0 QTime=1226
[junit4] 2> 585148 INFO (zkCallback-1210-thread-1-processing-n:127.0.0.1:33731_solr) [n:127.0.0.1:33731_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/managed-preanalyzed/state.json] for collection [managed-preanalyzed] has occurred - updating... (live nodes size: [2])
[junit4] 2> 585148 INFO (zkCallback-1210-thread-2-processing-n:127.0.0.1:33731_solr) [n:127.0.0.1:33731_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/managed-preanalyzed/state.json] for collection [managed-preanalyzed] has occurred - updating... (live nodes size: [2])
[junit4] 2> 585148 INFO (zkCallback-1208-thread-1-processing-n:127.0.0.1:40043_solr) [n:127.0.0.1:40043_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/managed-preanalyzed/state.json] for collection [managed-preanalyzed] has occurred - updating... (live nodes size: [2])
[junit4] 2> 585149 INFO (zkCallback-1208-thread-2-processing-n:127.0.0.1:40043_solr) [n:127.0.0.1:40043_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/managed-preanalyzed/state.json] for collection [managed-preanalyzed] has occurred - updating... (live nodes size: [2])
[junit4] 2> 585161 INFO (qtp8779174-7518) [n:127.0.0.1:33731_solr c:managed-preanalyzed s:shard2 r:core_node4 x:managed-preanalyzed_shard2_replica_n2] o.a.s.c.ZkController I am the leader, no recovery necessary
[junit4] 2> 585163 INFO (qtp8779174-7518) [n:127.0.0.1:33731_solr c:managed-preanalyzed s:shard2 r:core_node4 x:managed-preanalyzed_shard2_replica_n2] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&coreNodeName=core_node4&collection.configName=cloud-managed-preanalyzed&newCollection=true&name=managed-preanalyzed_shard2_replica_n2&action=CREATE&numShards=2&collection=managed-preanalyzed&shard=shard2&wt=javabin&version=2&replicaType=NRT} status=0 QTime=1289
[junit4] 2> 585164 INFO (qtp13209148-7511) [n:127.0.0.1:40043_solr ] o.a.s.h.a.CollectionsHandler Wait for new collection to be active for at most 30 seconds. Check all shard replicas
[junit4] 2> 585499 INFO (OverseerCollectionConfigSetProcessor-99269829402165254-127.0.0.1:33731_solr-n_0000000000) [n:127.0.0.1:33731_solr ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000000 doesn't exist. Requestor may have disconnected from ZooKeeper
[junit4] 2> 615170 ERROR (qtp13209148-7511) [n:127.0.0.1:40043_solr ] o.a.s.h.a.CollectionsHandler Timed out waiting for new collection's replicas to become ACTIVE replica https://127.0.0.1:33731/solr/managed-preanalyzed_shard2_replica_n2/ is in state of down with timeout=30
[junit4] 2> 615170 INFO (qtp13209148-7511) [n:127.0.0.1:40043_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={replicationFactor=1&collection.configName=cloud-managed-preanalyzed&maxShardsPerNode=1&name=managed-preanalyzed&nrtReplicas=1&action=CREATE&numShards=2&wt=javabin&version=2} status=0 QTime=31672
[junit4] 2> 705173 INFO (jetty-closer-1193-thread-2) [ ] o.e.j.s.AbstractConnector Stopped ServerConnector@2f4946{SSL,[ssl, http/1.1]}{127.0.0.1:0}
[junit4] 2> 705173 INFO (jetty-closer-1193-thread-2) [ ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=14209301
[junit4] 2> 705173 INFO (jetty-closer-1193-thread-2) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.node, tag=null
[junit4] 2> 705173 INFO (jetty-closer-1193-thread-2) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@1ec9b17: rootName = solr_33731, domain = solr.node, service url = null, agent id = null] for registry solr.node / com.codahale.metrics.MetricRegistry@9907db
[junit4] 2> 705173 INFO (jetty-closer-1193-thread-1) [ ] o.e.j.s.AbstractConnector Stopped ServerConnector@777320{SSL,[ssl, http/1.1]}{127.0.0.1:0}
[junit4] 2> 705174 INFO (jetty-closer-1193-thread-1) [ ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=16651721
[junit4] 2> 705174 INFO (jetty-closer-1193-thread-1) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.node, tag=null
[junit4] 2> 705174 INFO (jetty-closer-1193-thread-1) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@11cdb6c: rootName = solr_40043, domain = solr.node, service url = null, agent id = null] for registry solr.node / com.codahale.metrics.MetricRegistry@9a2d5f
[junit4] 2> 705177 INFO (jetty-closer-1193-thread-2) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jvm, tag=null
[junit4] 2> 705177 INFO (jetty-closer-1193-thread-2) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@1b4ea1a: rootName = solr_33731, domain = solr.jvm, service url = null, agent id = null] for registry solr.jvm / com.codahale.metrics.MetricRegistry@1e23c2d
[junit4] 2> 705179 INFO (jetty-closer-1193-thread-1) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jvm, tag=null
[junit4] 2> 705180 INFO (jetty-closer-1193-thread-1) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@fb3eb2: rootName = solr_40043, domain = solr.jvm, service url = null, agent id = null] for registry solr.jvm / com.codahale.metrics.MetricRegistry@1e23c2d
[junit4] 2> 705180 INFO (jetty-closer-1193-thread-2) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jetty, tag=null
[junit4] 2> 705180 INFO (jetty-closer-1193-thread-2) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@c19bb4: rootName = solr_33731, domain = solr.jetty, service url = null, agent id = null] for registry solr.jetty / com.codahale.metrics.MetricRegistry@1672313
[junit4] 2> 705181 INFO (jetty-closer-1193-thread-2) [ ] o.a.s.c.ZkController Remove node as live in ZooKeeper:/live_nodes/127.0.0.1:33731_solr
[junit4] 2> 705181 INFO (jetty-closer-1193-thread-2) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.cluster, tag=null
[junit4] 2> 705185 INFO (jetty-closer-1193-thread-1) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jetty, tag=null
[junit4] 2> 705185 INFO (jetty-closer-1193-thread-1) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@bf4919: rootName = solr_40043, domain = solr.jetty, service url = null, agent id = null] for registry solr.jetty / com.codahale.metrics.MetricRegistry@1672313
[junit4] 2> 705185 INFO (coreCloseExecutor-2540-thread-1) [n:127.0.0.1:33731_solr c:managed-preanalyzed s:shard2 r:core_node4 x:managed-preanalyzed_shard2_replica_n2] o.a.s.c.SolrCore [managed-preanalyzed_shard2_replica_n2] CLOSING SolrCore org.apache.solr.core.SolrCore@1654d72
[junit4] 2> 705185 INFO (coreCloseExecutor-2540-thread-1) [n:127.0.0.1:33731_solr c:managed-preanalyzed s:shard2 r:core_node4 x:managed-preanalyzed_shard2_replica_n2] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.managed-preanalyzed.shard2.replica_n2, tag=23416178
[junit4] 2> 705185 INFO (zkCallback-1208-thread-3-processing-n:127.0.0.1:40043_solr) [n:127.0.0.1:40043_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (1)
[junit4] 2> 705185 INFO (coreCloseExecutor-2540-thread-1) [n:127.0.0.1:33731_solr c:managed-preanalyzed s:shard2 r:core_node4 x:managed-preanalyzed_shard2_replica_n2] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@1dbe9a: rootName = solr_33731, domain = solr.core.managed-preanalyzed.shard2.replica_n2, service url = null, agent id = null] for registry solr.core.managed-preanalyzed.shard2.replica_n2 / com.codahale.metrics.MetricRegistry@1b0ebfc
[junit4] 2> 705185 INFO (zkCallback-1210-thread-4-processing-n:127.0.0.1:33731_solr) [n:127.0.0.1:33731_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (1)
[junit4] 2> 705190 INFO (jetty-closer-1193-thread-1) [ ] o.a.s.c.ZkController Remove node as live in ZooKeeper:/live_nodes/127.0.0.1:40043_solr
[junit4] 2> 705190 INFO (jetty-closer-1193-thread-1) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.cluster, tag=null
[junit4] 2> 705194 INFO (coreCloseExecutor-2541-thread-1) [n:127.0.0.1:40043_solr c:managed-preanalyzed s:shard1 r:core_node3 x:managed-preanalyzed_shard1_replica_n1] o.a.s.c.SolrCore [managed-preanalyzed_shard1_replica_n1] CLOSING SolrCore org.apache.solr.core.SolrCore@f4ff57
[junit4] 2> 705194 INFO (coreCloseExecutor-2541-thread-1) [n:127.0.0.1:40043_solr c:managed-preanalyzed s:shard1 r:core_node3 x:managed-preanalyzed_shard1_replica_n1] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.managed-preanalyzed.shard1.replica_n1, tag=16056151
[junit4] 2> 705194 INFO (coreCloseExecutor-2541-thread-1) [n:127.0.0.1:40043_solr c:managed-preanalyzed s:shard1 r:core_node3 x:managed-preanalyzed_shard1_replica_n1] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@15131ce: rootName = solr_40043, domain = solr.core.managed-preanalyzed.shard1.replica_n1, service url = null, agent id = null] for registry solr.core.managed-preanalyzed.shard1.replica_n1 / com.codahale.metrics.MetricRegistry@665ae
[junit4] 2> 705198 INFO (coreCloseExecutor-2540-thread-1) [n:127.0.0.1:33731_solr c:managed-preanalyzed s:shard2 r:core_node4 x:managed-preanalyzed_shard2_replica_n2] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.managed-preanalyzed.shard2.leader, tag=23416178
[junit4] 2> 705203 INFO (zkCallback-1208-thread-3-processing-n:127.0.0.1:40043_solr) [n:127.0.0.1:40043_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (0)
[junit4] 2> 705204 INFO (zkCallback-1210-thread-4-processing-n:127.0.0.1:33731_solr) [n:127.0.0.1:33731_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (0)
[junit4] 2> 705212 INFO (jetty-closer-1193-thread-2) [ ] o.a.s.c.Overseer Overseer (id=99269829402165254-127.0.0.1:33731_solr-n_0000000000) closing
[junit4] 2> 705212 INFO (OverseerStateUpdate-99269829402165254-127.0.0.1:33731_solr-n_0000000000) [n:127.0.0.1:33731_solr ] o.a.s.c.Overseer Overseer Loop exiting : 127.0.0.1:33731_solr
[junit4] 2> 705215 INFO (coreCloseExecutor-2541-thread-1) [n:127.0.0.1:40043_solr c:managed-preanalyzed s:shard1 r:core_node3 x:managed-preanalyzed_shard1_replica_n1] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.managed-preanalyzed.shard1.leader, tag=16056151
[junit4] 2> 705216 WARN (OverseerAutoScalingTriggerThread-99269829402165254-127.0.0.1:33731_solr-n_0000000000) [n:127.0.0.1:33731_solr ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread woken up but we are closed, exiting.
[junit4] 2> 705218 INFO (zkCallback-1208-thread-3-processing-n:127.0.0.1:40043_solr) [n:127.0.0.1:40043_solr ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:40043_solr
[junit4] 2> 705219 INFO (jetty-closer-1193-thread-2) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@195ca9c{/solr,null,UNAVAILABLE}
[junit4] 2> 705220 INFO (jetty-closer-1193-thread-1) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@d3b46f{/solr,null,UNAVAILABLE}
[junit4] 2> 705220 ERROR (SUITE-PreAnalyzedFieldManagedSchemaCloudTest-seed#[10F0E6577DBF52E]-worker) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 705221 INFO (SUITE-PreAnalyzedFieldManagedSchemaCloudTest-seed#[10F0E6577DBF52E]-worker) [ ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:37327 37327
[junit4] 2> 707137 INFO (Thread-3151) [ ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:37327 37327
[junit4] 2> 707141 WARN (Thread-3151) [ ] o.a.s.c.ZkTestServer Watch limit violations:
[junit4] 2> Maximum concurrent create/delete watches above limit:
[junit4] 2>
[junit4] 2> 3 /solr/aliases.json
[junit4] 2> 2 /solr/configs/cloud-managed-preanalyzed/managed-schema
[junit4] 2> 2 /solr/security.json
[junit4] 2> 2 /solr/configs/cloud-managed-preanalyzed
[junit4] 2>
[junit4] 2> Maximum concurrent data watches above limit:
[junit4] 2>
[junit4] 2> 11 /solr/collections/managed-preanalyzed/state.json
[junit4] 2> 3 /solr/clusterstate.json
[junit4] 2> 3 /solr/clusterprops.json
[junit4] 2>
[junit4] 2> Maximum concurrent children watches above limit:
[junit4] 2>
[junit4] 2> 3 /solr/live_nodes
[junit4] 2> 3 /solr/collections
[junit4] 2>
[junit4] 2> 707142 INFO (SUITE-PreAnalyzedFieldManagedSchemaCloudTest-seed#[10F0E6577DBF52E]-worker) [ ] o.a.s.SolrTestCaseJ4 ------------------------------------------------------- Done waiting for tracked resources to be released
[junit4] 2> NOTE: test params are: codec=Asserting(Lucene70): {}, docValues:{}, maxPointsInLeafNode=106, maxMBSortInHeap=6.284114899467414, sim=RandomSimilarity(queryNorm=false): {}, locale=zh-CN, timezone=Africa/Bujumbura
[junit4] 2> NOTE: Linux 4.10.0-40-generic i386/Oracle Corporation 1.8.0_144 (32-bit)/cpus=8,threads=1,free=305950768,total=536870912
[junit4] 2> NOTE: All tests run in this JVM: [TestLegacyNumericRangeQueryBuilder, TestUniqueKeyFieldResource, DistributedFacetExistsSmallTest, TestUseDocValuesAsStored2, TestExecutePlanAction, PolyFieldTest, HdfsRecoverLeaseTest, TestManagedSynonymGraphFilterFactory, TestExactStatsCache, TestNumericRangeQuery32, CreateCollectionCleanupTest, ClusterStateUpdateTest, OpenExchangeRatesOrgProviderTest, TestLockTree, ParsingFieldUpdateProcessorsTest, TestPseudoReturnFields, ShardRoutingTest, ChaosMonkeySafeLeaderTest, TestSolrCoreParser, DocValuesMultiTest, TestCursorMarkWithoutUniqueKey, TestPrepRecovery, TestSolrQueryParser, CollectionsAPIDistributedZkTest, TestFastLRUCache, TestQuerySenderNoQuery, TestPerFieldSimilarity, TestLeaderElectionZkExpiry, MBeansHandlerTest, TestStreamBody, TestDelegationWithHadoopAuth, TestClusterStateProvider, AutoScalingHandlerTest, AssignTest, TestFaceting, TestPayloadCheckQParserPlugin, TlogReplayBufferedWhileIndexingTest, TestDefaultStatsCache, MoreLikeThisHandlerTest, DocValuesMissingTest, SolrMetricsIntegrationTest, TestSortByMinMaxFunction, TestRTGBase, TestCloudJSONFacetJoinDomain, BigEndianAscendingWordSerializerTest, HdfsUnloadDistributedZkTest, TestSchemaSimilarityResource, HighlighterConfigTest, CdcrReplicationHandlerTest, TestFieldTypeResource, TestBM25SimilarityFactory, TestExclusionRuleCollectionAccess, SuggesterFSTTest, BasicAuthStandaloneTest, AddSchemaFieldsUpdateProcessorFactoryTest, RegexBoostProcessorTest, TestGraphTermsQParserPlugin, TestCollationField, TestStressRecovery, TestConfigSetsAPIExclusivity, SecurityConfHandlerTest, TestSchemaManager, RollingRestartTest, TestSurroundQueryParser, TestCustomSort, TestSlowCompositeReaderWrapper, LegacyCloudClusterPropTest, RankQueryTest, FacetPivotSmallTest, TestApiFramework, ConvertedLegacyTest, TestZkAclsWithHadoopAuth, TestAuthenticationFramework, TestCoreAdminApis, CloudExitableDirectoryReaderTest, TestSolr4Spatial, TestAtomicUpdateErrorCases, TestFastWriter, SolrCLIZkUtilsTest, TestHighFrequencyDictionaryFactory, SolrIndexConfigTest, SolrRequestParserTest, TestDocumentBuilder, TestLuceneMatchVersion, PathHierarchyTokenizerFactoryTest, TestStressLucene, TolerantUpdateProcessorTest, TestRecoveryHdfs, SolrIndexSplitterTest, LukeRequestHandlerTest, TestSubQueryTransformerCrossCore, TestMultiWordSynonyms, TestDeleteCollectionOnDownNodes, ShowFileRequestHandlerTest, PreAnalyzedFieldManagedSchemaCloudTest]
[junit4] 2> NOTE: reproduce with: ant test -Dtestcase=PreAnalyzedFieldManagedSchemaCloudTest -Dtests.seed=10F0E6577DBF52E -Dtests.multiplier=3 -Dtests.slow=true -Dtests.locale=zh-CN -Dtests.timezone=Africa/Bujumbura -Dtests.asserts=true -Dtests.file.encoding=US-ASCII
[junit4] ERROR 0.00s J1 | PreAnalyzedFieldManagedSchemaCloudTest (suite) <<<
[junit4] > Throwable #1: java.util.concurrent.TimeoutException
[junit4] > at __randomizedtesting.SeedInfo.seed([10F0E6577DBF52E]:0)
[junit4] > at org.apache.solr.common.cloud.ZkStateReader.waitForState(ZkStateReader.java:1261)
[junit4] > at org.apache.solr.client.solrj.impl.CloudSolrClient.waitForState(CloudSolrClient.java:449)
[junit4] > at org.apache.solr.schema.PreAnalyzedFieldManagedSchemaCloudTest.setupCluster(PreAnalyzedFieldManagedSchemaCloudTest.java:45)
[junit4] > at java.lang.Thread.run(Thread.java:748)
[junit4] Completed [259/764 (1!)] on J1 in 124.07s, 0 tests, 1 error <<< FAILURES!
[...truncated 1279 lines...]
[junit4] Suite: org.apache.solr.cloud.ShardSplitTest
[junit4] 2> 2065110 INFO (SUITE-ShardSplitTest-seed#[10F0E6577DBF52E]-worker) [ ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
[junit4] 2> Creating dataDir: /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J2/temp/solr.cloud.ShardSplitTest_10F0E6577DBF52E-001/init-core-data-001
[junit4] 2> 2065111 WARN (SUITE-ShardSplitTest-seed#[10F0E6577DBF52E]-worker) [ ] o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=30 numCloses=30
[junit4] 2> 2065111 INFO (SUITE-ShardSplitTest-seed#[10F0E6577DBF52E]-worker) [ ] o.a.s.SolrTestCaseJ4 Using PointFields (NUMERIC_POINTS_SYSPROP=true) w/NUMERIC_DOCVALUES_SYSPROP=true
[junit4] 2> 2065112 INFO (SUITE-ShardSplitTest-seed#[10F0E6577DBF52E]-worker) [ ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false) via: @org.apache.solr.SolrTestCaseJ4$SuppressSSL(bugUrl=https://issues.apache.org/jira/browse/SOLR-5776)
[junit4] 2> 2065112 INFO (SUITE-ShardSplitTest-seed#[10F0E6577DBF52E]-worker) [ ] o.a.s.BaseDistributedSearchTestCase Setting hostContext system property: /i/r
[junit4] 2> 2065115 INFO (TEST-ShardSplitTest.testSplitShardWithRule-seed#[10F0E6577DBF52E]) [ ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
[junit4] 2> 2065115 INFO (Thread-6107) [ ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 2065115 INFO (Thread-6107) [ ] o.a.s.c.ZkTestServer Starting server
[junit4] 2> 2065116 ERROR (Thread-6107) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 2065215 INFO (TEST-ShardSplitTest.testSplitShardWithRule-seed#[10F0E6577DBF52E]) [ ] o.a.s.c.ZkTestServer start zk server on port:33965
[junit4] 2> 2065217 INFO (zkConnectionManagerCallback-10030-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2065219 INFO (zkConnectionManagerCallback-10032-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2065220 INFO (TEST-ShardSplitTest.testSplitShardWithRule-seed#[10F0E6577DBF52E]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
[junit4] 2> 2065221 INFO (TEST-ShardSplitTest.testSplitShardWithRule-seed#[10F0E6577DBF52E]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/core/src/test-files/solr/collection1/conf/schema15.xml to /configs/conf1/schema.xml
[junit4] 2> 2065222 INFO (TEST-ShardSplitTest.testSplitShardWithRule-seed#[10F0E6577DBF52E]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
[junit4] 2> 2065222 INFO (TEST-ShardSplitTest.testSplitShardWithRule-seed#[10F0E6577DBF52E]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
[junit4] 2> 2065222 INFO (TEST-ShardSplitTest.testSplitShardWithRule-seed#[10F0E6577DBF52E]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
[junit4] 2> 2065223 INFO (TEST-ShardSplitTest.testSplitShardWithRule-seed#[10F0E6577DBF52E]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
[junit4] 2> 2065223 INFO (TEST-ShardSplitTest.testSplitShardWithRule-seed#[10F0E6577DBF52E]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
[junit4] 2> 2065223 INFO (TEST-ShardSplitTest.testSplitShardWithRule-seed#[10F0E6577DBF52E]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
[junit4] 2> 2065224 INFO (TEST-ShardSplitTest.testSplitShardWithRule-seed#[10F0E6577DBF52E]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4] 2> 2065224 INFO (TEST-ShardSplitTest.testSplitShardWithRule-seed#[10F0E6577DBF52E]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
[junit4] 2> 2065225 INFO (TEST-ShardSplitTest.testSplitShardWithRule-seed#[10F0E6577DBF52E]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
[junit4] 2> 2065225 INFO (TEST-ShardSplitTest.testSplitShardWithRule-seed#[10F0E6577DBF52E]) [ ] o.a.s.c.AbstractFullDistribZkTestBase Will use NRT replicas unless explicitly asked otherwise
[junit4] 2> 2065291 INFO (TEST-ShardSplitTest.testSplitShardWithRule-seed#[10F0E6577DBF52E]) [ ] o.e.j.s.Server jetty-9.3.20.v20170531
[junit4] 2> 2065292 INFO (TEST-ShardSplitTest.testSplitShardWithRule-seed#[10F0E6577DBF52E]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@dc36c3{/i/r,null,AVAILABLE}
[junit4] 2> 2065293 INFO (TEST-ShardSplitTest.testSplitShardWithRule-seed#[10F0E6577DBF52E]) [ ] o.e.j.s.AbstractConnector Started ServerConnector@c634d{HTTP/1.1,[http/1.1]}{127.0.0.1:46811}
[junit4] 2> 2065293 INFO (TEST-ShardSplitTest.testSplitShardWithRule-seed#[10F0E6577DBF52E]) [ ] o.e.j.s.Server Started @2067019ms
[junit4] 2> 2065293 INFO (TEST-ShardSplitTest.testSplitShardWithRule-seed#[10F0E6577DBF52E]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J2/temp/solr.cloud.ShardSplitTest_10F0E6577DBF52E-001/tempDir-001/control/data, hostContext=/i/r, hostPort=46811, coreRootDirectory=/home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J2/../../../../../../../../../home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J2/temp/solr.cloud.ShardSplitTest_10F0E6577DBF52E-001/control-001/cores}
[junit4] 2> 2065293 ERROR (TEST-ShardSplitTest.testSplitShardWithRule-seed#[10F0E6577DBF52E]) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 2065293 INFO (TEST-ShardSplitTest.testSplitShardWithRule-seed#[10F0E6577DBF52E]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 7.3.0
[junit4] 2> 2065293 INFO (TEST-ShardSplitTest.testSplitShardWithRule-seed#[10F0E6577DBF52E]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 2065293 INFO (TEST-ShardSplitTest.testSplitShardWithRule-seed#[10F0E6577DBF52E]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 2065293 INFO (TEST-ShardSplitTest.testSplitShardWithRule-seed#[10F0E6577DBF52E]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2017-12-31T16:47:49.235Z
[junit4] 2> 2065295 INFO (zkConnectionManagerCallback-10034-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2065295 INFO (TEST-ShardSplitTest.testSplitShardWithRule-seed#[10F0E6577DBF52E]) [ ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
[junit4] 2> 2065295 INFO (TEST-ShardSplitTest.testSplitShardWithRule-seed#[10F0E6577DBF52E]) [ ] o.a.s.c.SolrXmlConfig Loading container configuration from /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J2/temp/solr.cloud.ShardSplitTest_10F0E6577DBF52E-001/control-001/solr.xml
[junit4] 2> 2065298 INFO (TEST-ShardSplitTest.testSplitShardWithRule-seed#[10F0E6577DBF52E]) [ ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverWorkLoopDelay is ignored
[junit4] 2> 2065298 INFO (TEST-ShardSplitTest.testSplitShardWithRule-seed#[10F0E6577DBF52E]) [ ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverBadNodeExpiration is ignored
[junit4] 2> 2065299 INFO (TEST-ShardSplitTest.testSplitShardWithRule-seed#[10F0E6577DBF52E]) [ ] o.a.s.c.SolrXmlConfig MBean server found: com.sun.jmx.mbeanserver.JmxMBeanServer@17403fc, but no JMX reporters were configured - adding default JMX reporter.
[junit4] 2> 2065301 INFO (TEST-ShardSplitTest.testSplitShardWithRule-seed#[10F0E6577DBF52E]) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:33965/solr
[junit4] 2> 2065301 INFO (zkConnectionManagerCallback-10038-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2065302 WARN (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [ ] o.a.z.s.NIOServerCnxn caught end of stream exception
[junit4] 2> EndOfStreamException: Unable to read additional data from client sessionid 0x160ad78e3420003, likely client has closed socket
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:239)
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:203)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2> 2065302 INFO (zkConnectionManagerCallback-10040-thread-1-processing-n:127.0.0.1:46811_i%2Fr) [n:127.0.0.1:46811_i%2Fr ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2065328 INFO (TEST-ShardSplitTest.testSplitShardWithRule-seed#[10F0E6577DBF52E]) [n:127.0.0.1:46811_i%2Fr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 2065328 INFO (TEST-ShardSplitTest.testSplitShardWithRule-seed#[10F0E6577DBF52E]) [n:127.0.0.1:46811_i%2Fr ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:46811_i%2Fr
[junit4] 2> 2065329 INFO (TEST-ShardSplitTest.testSplitShardWithRule-seed#[10F0E6577DBF52E]) [n:127.0.0.1:46811_i%2Fr ] o.a.s.c.Overseer Overseer (id=99269926522585092-127.0.0.1:46811_i%2Fr-n_0000000000) starting
[junit4] 2> 2065332 INFO (TEST-ShardSplitTest.testSplitShardWithRule-seed#[10F0E6577DBF52E]) [n:127.0.0.1:46811_i%2Fr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:46811_i%2Fr
[junit4] 2> 2065332 INFO (zkCallback-10039-thread-1-processing-n:127.0.0.1:46811_i%2Fr) [n:127.0.0.1:46811_i%2Fr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 2065454 INFO (TEST-ShardSplitTest.testSplitShardWithRule-seed#[10F0E6577DBF52E]) [n:127.0.0.1:46811_i%2Fr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@17403fc
[junit4] 2> 2065462 INFO (TEST-ShardSplitTest.testSplitShardWithRule-seed#[10F0E6577DBF52E]) [n:127.0.0.1:46811_i%2Fr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@17403fc
[junit4] 2> 2065462 INFO (TEST-ShardSplitTest.testSplitShardWithRule-seed#[10F0E6577DBF52E]) [n:127.0.0.1:46811_i%2Fr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@17403fc
[junit4] 2> 2065463 INFO (TEST-ShardSplitTest.testSplitShardWithRule-seed#[10F0E6577DBF52E]) [n:127.0.0.1:46811_i%2Fr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J2/../../../../../../../../../home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J2/temp/solr.cloud.ShardSplitTest_10F0E6577DBF52E-001/control-001/cores
[junit4] 2> 2065479 INFO (zkConnectionManagerCallback-10046-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2065480 INFO (TEST-ShardSplitTest.testSplitShardWithRule-seed#[10F0E6577DBF52E]) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 2065480 INFO (TEST-ShardSplitTest.testSplitShardWithRule-seed#[10F0E6577DBF52E]) [ ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:33965/solr ready
[junit4] 2> 2065481 INFO (qtp1242750-31135) [n:127.0.0.1:46811_i%2Fr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params replicationFactor=1&collection.configName=conf1&name=control_collection&nrtReplicas=1&action=CREATE&numShards=1&createNodeSet=127.0.0.1:46811_i%252Fr&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 2065482 INFO (OverseerThreadFactory-9832-thread-1-processing-n:127.0.0.1:46811_i%2Fr) [n:127.0.0.1:46811_i%2Fr ] o.a.s.c.CreateCollectionCmd Create collection control_collection
[junit4] 2> 2065585 INFO (qtp1242750-31137) [n:127.0.0.1:46811_i%2Fr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf1&newCollection=true&name=control_collection_shard1_replica_n1&action=CREATE&numShards=1&collection=control_collection&shard=shard1&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 2065585 INFO (qtp1242750-31137) [n:127.0.0.1:46811_i%2Fr ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 4 transient cores
[junit4] 2> 2065687 INFO (zkCallback-10039-thread-1-processing-n:127.0.0.1:46811_i%2Fr) [n:127.0.0.1:46811_i%2Fr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/control_collection/state.json] for collection [control_collection] has occurred - updating... (live nodes size: [1])
[junit4] 2> 2066602 INFO (qtp1242750-31137) [n:127.0.0.1:46811_i%2Fr c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.3.0
[junit4] 2> 2066627 INFO (qtp1242750-31137) [n:127.0.0.1:46811_i%2Fr c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.s.IndexSchema [control_collection_shard1_replica_n1] Schema name=test
[junit4] 2> 2066674 INFO (qtp1242750-31137) [n:127.0.0.1:46811_i%2Fr c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.s.IndexSchema Loaded schema test/1.6 with uniqueid field id
[junit4] 2> 2066681 INFO (qtp1242750-31137) [n:127.0.0.1:46811_i%2Fr c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.CoreContainer Creating SolrCore 'control_collection_shard1_replica_n1' using configuration from collection control_collection, trusted=true
[junit4] 2> 2066681 INFO (qtp1242750-31137) [n:127.0.0.1:46811_i%2Fr c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.control_collection.shard1.replica_n1' (registry 'solr.core.control_collection.shard1.replica_n1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@17403fc
[junit4] 2> 2066681 INFO (qtp1242750-31137) [n:127.0.0.1:46811_i%2Fr c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 2066681 INFO (qtp1242750-31137) [n:127.0.0.1:46811_i%2Fr c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.SolrCore [[control_collection_shard1_replica_n1] ] Opening new SolrCore at [/home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J2/temp/solr.cloud.ShardSplitTest_10F0E6577DBF52E-001/control-001/cores/control_collection_shard1_replica_n1], dataDir=[/home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J2/../../../../../../../../../home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J2/temp/solr.cloud.ShardSplitTest_10F0E6577DBF52E-001/control-001/cores/control_collection_shard1_replica_n1/data/]
[junit4] 2> 2066684 INFO (qtp1242750-31137) [n:127.0.0.1:46811_i%2Fr c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=40, maxMergeAtOnceExplicit=40, maxMergedSegmentMB=97.796875, floorSegmentMB=2.0595703125, forceMergeDeletesPctAllowed=19.957366809104393, segmentsPerTier=41.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0
[junit4] 2> 2066705 WARN (qtp1242750-31137) [n:127.0.0.1:46811_i%2Fr c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,attributes = {initParams=a, name=/dump, class=DumpRequestHandler},args = {defaults={a=A,b=B}}}
[junit4] 2> 2066731 INFO (qtp1242750-31137) [n:127.0.0.1:46811_i%2Fr c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 2066732 INFO (qtp1242750-31137) [n:127.0.0.1:46811_i%2Fr c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 2066732 INFO (qtp1242750-31137) [n:127.0.0.1:46811_i%2Fr c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 2066732 INFO (qtp1242750-31137) [n:127.0.0.1:46811_i%2Fr c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 2066733 INFO (qtp1242750-31137) [n:127.0.0.1:46811_i%2Fr c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=43, maxMergeAtOnceExplicit=41, maxMergedSegmentMB=53.7001953125, floorSegmentMB=1.2431640625, forceMergeDeletesPctAllowed=28.59210968916453, segmentsPerTier=31.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.4757805355459702
[junit4] 2> 2066734 INFO (qtp1242750-31137) [n:127.0.0.1:46811_i%2Fr c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.s.SolrIndexSearcher Opening [Searcher@88f9b4[control_collection_shard1_replica_n1] main]
[junit4] 2> 2066734 INFO (qtp1242750-31137) [n:127.0.0.1:46811_i%2Fr c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 2066735 INFO (qtp1242750-31137) [n:127.0.0.1:46811_i%2Fr c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 2066735 INFO (qtp1242750-31137) [n:127.0.0.1:46811_i%2Fr c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
[junit4] 2> 2066736 INFO (searcherExecutor-9835-thread-1-processing-n:127.0.0.1:46811_i%2Fr x:control_collection_shard1_replica_n1 s:shard1 c:control_collection) [n:127.0.0.1:46811_i%2Fr c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.SolrCore [control_collection_shard1_replica_n1] Registered new searcher Searcher@88f9b4[control_collection_shard1_replica_n1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 2066736 INFO (qtp1242750-31137) [n:127.0.0.1:46811_i%2Fr c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1588318826060054528
[junit4] 2> 2066739 INFO (qtp1242750-31137) [n:127.0.0.1:46811_i%2Fr c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
[junit4] 2> 2066739 INFO (qtp1242750-31137) [n:127.0.0.1:46811_i%2Fr c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
[junit4] 2> 2066739 INFO (qtp1242750-31137) [n:127.0.0.1:46811_i%2Fr c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:46811/i/r/control_collection_shard1_replica_n1/
[junit4] 2> 2066739 INFO (qtp1242750-31137) [n:127.0.0.1:46811_i%2Fr c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
[junit4] 2> 2066739 INFO (qtp1242750-31137) [n:127.0.0.1:46811_i%2Fr c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.SyncStrategy http://127.0.0.1:46811/i/r/control_collection_shard1_replica_n1/ has no replicas
[junit4] 2> 2066739 INFO (qtp1242750-31137) [n:127.0.0.1:46811_i%2Fr c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
[junit4] 2> 2066740 INFO (qtp1242750-31137) [n:127.0.0.1:46811_i%2Fr c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:46811/i/r/control_collection_shard1_replica_n1/ shard1
[junit4] 2> 2066841 INFO (zkCallback-10039-thread-1-processing-n:127.0.0.1:46811_i%2Fr) [n:127.0.0.1:46811_i%2Fr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/control_collection/state.json] for collection [control_collection] has occurred - updating... (live nodes size: [1])
[junit4] 2> 2066891 INFO (qtp1242750-31137) [n:127.0.0.1:46811_i%2Fr c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.ZkController I am the leader, no recovery necessary
[junit4] 2> 2066895 INFO (qtp1242750-31137) [n:127.0.0.1:46811_i%2Fr c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=conf1&newCollection=true&name=control_collection_shard1_replica_n1&action=CREATE&numShards=1&collection=control_collection&shard=shard1&wt=javabin&version=2&replicaType=NRT} status=0 QTime=1310
[junit4] 2> 2066897 INFO (qtp1242750-31135) [n:127.0.0.1:46811_i%2Fr ] o.a.s.h.a.CollectionsHandler Wait for new collection to be active for at most 30 seconds. Check all shard replicas
[junit4] 2> 2066992 INFO (zkCallback-10039-thread-1-processing-n:127.0.0.1:46811_i%2Fr) [n:127.0.0.1:46811_i%2Fr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/control_collection/state.json] for collection [control_collection] has occurred - updating... (live nodes size: [1])
[junit4] 2> 2067483 INFO (OverseerCollectionConfigSetProcessor-99269926522585092-127.0.0.1:46811_i%2Fr-n_0000000000) [n:127.0.0.1:46811_i%2Fr ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000000 doesn't exist. Requestor may have disconnected from ZooKeeper
[junit4] 2> 2067897 INFO (qtp1242750-31135) [n:127.0.0.1:46811_i%2Fr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={replicationFactor=1&collection.configName=conf1&name=control_collection&nrtReplicas=1&action=CREATE&numShards=1&createNodeSet=127.0.0.1:46811_i%252Fr&wt=javabin&version=2} status=0 QTime=2416
[junit4] 2> 2067901 INFO (zkConnectionManagerCallback-10050-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2067901 INFO (TEST-ShardSplitTest.testSplitShardWithRule-seed#[10F0E6577DBF52E]) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 2067901 INFO (TEST-ShardSplitTest.testSplitShardWithRule-seed#[10F0E6577DBF52E]) [ ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:33965/solr ready
[junit4] 2> 2067901 INFO (TEST-ShardSplitTest.testSplitShardWithRule-seed#[10F0E6577DBF52E]) [ ] o.a.s.c.ChaosMonkey monkey: init - expire sessions:false cause connection loss:false
[junit4] 2> 2067902 INFO (qtp1242750-31135) [n:127.0.0.1:46811_i%2Fr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params replicationFactor=1&collection.configName=conf1&name=collection1&nrtReplicas=1&action=CREATE&numShards=2&createNodeSet=&stateFormat=1&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 2067903 INFO (OverseerThreadFactory-9832-thread-2-processing-n:127.0.0.1:46811_i%2Fr) [n:127.0.0.1:46811_i%2Fr ] o.a.s.c.CreateCollectionCmd Create collection collection1
[junit4] 2> 2067903 WARN (OverseerThreadFactory-9832-thread-2-processing-n:127.0.0.1:46811_i%2Fr) [n:127.0.0.1:46811_i%2Fr ] o.a.s.c.CreateCollectionCmd It is unusual to create a collection (collection1) without cores.
[junit4] 2> 2068105 INFO (qtp1242750-31135) [n:127.0.0.1:46811_i%2Fr ] o.a.s.h.a.CollectionsHandler Wait for new collection to be active for at most 30 seconds. Check all shard replicas
[junit4] 2> 2068105 INFO (qtp1242750-31135) [n:127.0.0.1:46811_i%2Fr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={replicationFactor=1&collection.configName=conf1&name=collection1&nrtReplicas=1&action=CREATE&numShards=2&createNodeSet=&stateFormat=1&wt=javabin&version=2} status=0 QTime=203
[junit4] 2> 2068168 INFO (TEST-ShardSplitTest.testSplitShardWithRule-seed#[10F0E6577DBF52E]) [ ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 1 in directory /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J2/temp/solr.cloud.ShardSplitTest_10F0E6577DBF52E-001/shard-1-001 of type NRT
[junit4] 2> 2068168 INFO (TEST-ShardSplitTest.testSplitShardWithRule-seed#[10F0E6577DBF52E]) [ ] o.e.j.s.Server jetty-9.3.20.v20170531
[junit4] 2> 2068169 INFO (TEST-ShardSplitTest.testSplitShardWithRule-seed#[10F0E6577DBF52E]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@e20f89{/i/r,null,AVAILABLE}
[junit4] 2> 2068169 INFO (TEST-ShardSplitTest.testSplitShardWithRule-seed#[10F0E6577DBF52E]) [ ] o.e.j.s.AbstractConnector Started ServerConnector@34c715{HTTP/1.1,[http/1.1]}{127.0.0.1:43799}
[junit4] 2> 2068169 INFO (TEST-ShardSplitTest.testSplitShardWithRule-seed#[10F0E6577DBF52E]) [ ] o.e.j.s.Server Started @2069895ms
[junit4] 2> 2068170 INFO (TEST-ShardSplitTest.testSplitShardWithRule-seed#[10F0E6577DBF52E]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J2/temp/solr.cloud.ShardSplitTest_10F0E6577DBF52E-001/tempDir-001/jetty1, solrconfig=solrconfig.xml, hostContext=/i/r, hostPort=43799, coreRootDirectory=/home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J2/temp/solr.cloud.ShardSplitTest_10F0E6577DBF52E-001/shard-1-001/cores}
[junit4] 2> 2068170 ERROR (TEST-ShardSplitTest.testSplitShardWithRule-seed#[10F0E6577DBF52E]) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 2068170 INFO (TEST-ShardSplitTest.testSplitShardWithRule-seed#[10F0E6577DBF52E]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 7.3.0
[junit4] 2> 2068170 INFO (TEST-ShardSplitTest.testSplitShardWithRule-seed#[10F0E6577DBF52E]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 2068170 INFO (TEST-ShardSplitTest.testSplitShardWithRule-seed#[10F0E6577DBF52E]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 2068170 INFO (TEST-ShardSplitTest.testSplitShardWithRule-seed#[10F0E6577DBF52E]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2017-12-31T16:47:52.112Z
[junit4] 2> 2068171 INFO (zkConnectionManagerCallback-10052-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2068171 INFO (TEST-ShardSplitTest.testSplitShardWithRule-seed#[10F0E6577DBF52E]) [ ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
[junit4] 2> 2068171 INFO (TEST-ShardSplitTest.testSplitShardWithRule-seed#[10F0E6577DBF52E]) [ ] o.a.s.c.SolrXmlConfig Loading container configuration from /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J2/temp/solr.cloud.ShardSplitTest_10F0E6577DBF52E-001/shard-1-001/solr.xml
[junit4] 2> 206
[...truncated too long message...]
sing metric reporters for registry=solr.collection.collection1.shard1.leader, tag=30265812
[junit4] 2> 3010434 INFO (TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[10F0E6577DBF52E]) [ ] o.a.s.c.Overseer Overseer (id=99269985967276050-127.0.0.1:35723_i%2Fr-n_0000000004) closing
[junit4] 2> 3010434 WARN (OverseerAutoScalingTriggerThread-99269985967276050-127.0.0.1:35723_i%2Fr-n_0000000004) [n:127.0.0.1:35723_i%2Fr ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread woken up but we are closed, exiting.
[junit4] 2> 3010442 INFO (OverseerStateUpdate-99269985967276050-127.0.0.1:35723_i%2Fr-n_0000000004) [n:127.0.0.1:35723_i%2Fr ] o.a.s.c.Overseer Overseer Loop exiting : 127.0.0.1:35723_i%2Fr
[junit4] 2> 3010482 INFO (TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[10F0E6577DBF52E]) [ ] o.e.j.s.AbstractConnector Stopped ServerConnector@a73076{HTTP/1.1,[http/1.1]}{127.0.0.1:0}
[junit4] 2> 3010482 INFO (TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[10F0E6577DBF52E]) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@1763a23{/i/r,null,UNAVAILABLE}
[junit4] 2> 3010483 INFO (TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[10F0E6577DBF52E]) [ ] o.a.s.c.ChaosMonkey monkey: stop jetty! 42167
[junit4] 2> 3010483 ERROR (TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[10F0E6577DBF52E]) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 3010484 INFO (TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[10F0E6577DBF52E]) [ ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:42595 42595
[junit4] 2> 3011935 INFO (Thread-6359) [ ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:42595 42595
[junit4] 2> 3011935 WARN (Thread-6359) [ ] o.a.s.c.ZkTestServer Watch limit violations:
[junit4] 2> Maximum concurrent create/delete watches above limit:
[junit4] 2>
[junit4] 2> 8 /solr/aliases.json
[junit4] 2> 7 /solr/collections/collection1/state.json
[junit4] 2> 6 /solr/security.json
[junit4] 2> 6 /solr/configs/conf1
[junit4] 2>
[junit4] 2> Maximum concurrent data watches above limit:
[junit4] 2>
[junit4] 2> 8 /solr/clusterprops.json
[junit4] 2> 8 /solr/clusterstate.json
[junit4] 2> 4 /solr/autoscaling.json
[junit4] 2> 2 /solr/overseer_elect/election/99269985967276041-127.0.0.1:46297_i%2Fr-n_0000000001
[junit4] 2>
[junit4] 2> Maximum concurrent children watches above limit:
[junit4] 2>
[junit4] 2> 8 /solr/live_nodes
[junit4] 2> 8 /solr/collections
[junit4] 2> 4 /solr/overseer/queue
[junit4] 2> 4 /solr/autoscaling/events/.auto_add_replicas
[junit4] 2> 4 /solr/overseer/queue-work
[junit4] 2> 4 /solr/overseer/collection-queue-work
[junit4] 2>
[junit4] 2> NOTE: leaving temporary files on disk at: /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J2/temp/solr.cloud.ShardSplitTest_10F0E6577DBF52E-001
[junit4] 2> NOTE: test params are: codec=Asserting(Lucene70): {rnd_b=PostingsFormat(name=LuceneFixedGap), shard_s=PostingsFormat(name=LuceneVarGapDocFreqInterval), a_t=PostingsFormat(name=LuceneVarGapDocFreqInterval), id=PostingsFormat(name=LuceneFixedGap)}, docValues:{rnd_b=DocValuesFormat(name=Lucene70), shard_s=DocValuesFormat(name=Direct), _version_=DocValuesFormat(name=Direct), n_ti=DocValuesFormat(name=Direct), a_t=DocValuesFormat(name=Direct), a_i=DocValuesFormat(name=Lucene70), id=DocValuesFormat(name=Lucene70)}, maxPointsInLeafNode=1804, maxMBSortInHeap=6.719549478275708, sim=RandomSimilarity(queryNorm=false): {}, locale=sr-RS, timezone=Europe/Samara
[junit4] 2> NOTE: Linux 4.10.0-40-generic i386/Oracle Corporation 1.8.0_144 (32-bit)/cpus=8,threads=1,free=145029792,total=536870912
[junit4] 2> NOTE: All tests run in this JVM: [UpdateLogTest, SpatialHeatmapFacetsTest, BadCopyFieldTest, UniqFieldsUpdateProcessorFactoryTest, TestSolrCLIRunExample, ZkStateReaderTest, TestLegacyFieldCache, DistributedSpellCheckComponentTest, TestRequestForwarding, BJQParserTest, TestClassicSimilarityFactory, TestSearchPerf, HLLSerializationTest, TestConfigSets, HdfsDirectoryTest, XmlUpdateRequestHandlerTest, MoveReplicaHDFSTest, TestSmileRequest, TestJavabinTupleStreamParser, TestPerFieldSimilarityWithDefaultOverride, ClusterStateTest, TestUninvertingReader, TestFiltering, JvmMetricsTest, TestHalfAndHalfDocValues, TestCloudDeleteByQuery, CdcrBootstrapTest, DistributedDebugComponentTest, TestLFUCache, FieldAnalysisRequestHandlerTest, TestTrie, TestComputePlanAction, TestFunctionQuery, TestGroupingSearch, TestZkChroot, RequiredFieldsTest, OverseerTest, TestMiniSolrCloudClusterSSL, TestQueryTypes, SolrIndexMetricsTest, SolrShardReporterTest, BitVectorTest, TestJsonRequest, DefaultValueUpdateProcessorTest, SolrCoreCheckLockOnStartupTest, TestEmbeddedSolrServerConstructors, BadComponentTest, BlockCacheTest, SortByFunctionTest, TestFuzzyAnalyzedSuggestions, TestPayloadScoreQParserPlugin, TestShardHandlerFactory, SpellPossibilityIteratorTest, TestMaxScoreQueryParser, TestNestedDocsSort, CdcrRequestHandlerTest, TestSolrCloudWithSecureImpersonation, TestNamedUpdateProcessors, TestReplicationHandlerBackup, CoreAdminHandlerTest, TestStressLiveNodes, SharedFSAutoReplicaFailoverTest, MetricsHandlerTest, TestRestManager, TestExportWriter, OverseerCollectionConfigSetProcessorTest, DistributedFacetPivotSmallTest, WordBreakSolrSpellCheckerTest, SpellCheckCollatorWithCollapseTest, TestSQLHandler, TestReloadDeadlock, TestConfigOverlay, TestCollectionsAPIViaSolrCloudCluster, PreAnalyzedUpdateProcessorTest, FieldMutatingUpdateProcessorTest, UnloadDistributedZkTest, TestReload, TestTriggerIntegration, ConnectionReuseTest, SolrGangliaReporterTest, TestQuerySenderListener, TestMaxTokenLenTokenizer, TestLMDirichletSimilarityFactory, SmileWriterTest, ChaosMonkeyNothingIsSafeTest, TestFieldCacheSortRandom, RecoveryZkTest, RuleEngineTest, TestDistribDocBasedVersion, InfixSuggestersTest, FileBasedSpellCheckerTest, SegmentsInfoRequestHandlerTest, TestManagedSchemaThreadSafety, TestTrieFacet, CopyFieldTest, TestSolrDeletionPolicy2, ReturnFieldsTest, TestComplexPhraseLeadingWildcard, TestConfigReload, DistributedFacetSimpleRefinementLongTailTest, ComputePlanActionTest, TestMacroExpander, MigrateRouteKeyTest, NodeLostTriggerTest, CoreSorterTest, TestPullReplica, TestDistributedStatsComponentCardinality, TestGenericDistributedQueue, TestSchemaVersionResource, TestSegmentSorting, TriLevelCompositeIdRoutingTest, TestTolerantUpdateProcessorRandomCloud, UUIDUpdateProcessorFallbackTest, TestCoreDiscovery, TestRealTimeGet, TestImpersonationWithHadoopAuth, CoreAdminRequestStatusTest, CurrencyFieldTypeTest, CollectionsAPIAsyncDistributedZkTest, HdfsCollectionsAPIDistributedZkTest, LeaderElectionContextKeyTest, TestChildDocTransformer, TestRandomFaceting, LeaderElectionTest, TestRandomDVFaceting, TestDistributedGrouping, TestRecovery, TestHashPartitioner, TestStressReorder, TestCoreContainer, SimpleFacetsTest, StatsComponentTest, BadIndexSchemaTest, BasicFunctionalityTest, TestLazyCores, DirectUpdateHandlerTest, TestBadConfig, HighlighterTest, SimplePostToolTest, SignatureUpdateProcessorFactoryTest, TestExtendedDismaxParser, SuggesterTest, SpatialFilterTest, SuggesterWFSTTest, TestCSVLoader, NoCacheHeaderTest, SchemaVersionSpecificBehaviorTest, QueryEqualityTest, DirectUpdateHandlerOptimizeTest, TestRemoteStreaming, DistanceFunctionTest, XsltUpdateRequestHandlerTest, DebugComponentTest, IndexBasedSpellCheckerTest, DisMaxRequestHandlerTest, TestWriterPerf, DirectSolrSpellCheckerTest, DocumentAnalysisRequestHandlerTest, PrimitiveFieldTypeTest, TestOmitPositions, TermsComponentTest, RequestHandlersTest, DocumentBuilderTest, TestValueSourceCache, LoggingHandlerTest, SolrPluginUtilsTest, IndexSchemaTest, TestJmxIntegration, UpdateRequestProcessorFactoryTest, TestCSVResponseWriter, QueryParsingTest, TestAnalyzedSuggestions, JsonLoaderTest, BinaryUpdateRequestHandlerTest, TestPartialUpdateDeduplication, PingRequestHandlerTest, CSVRequestHandlerTest, SearchHandlerTest, SOLR749Test, AlternateDirectoryTest, TestSolrIndexConfig, ResponseLogComponentTest, TestMergePolicyConfig, MultiTermTest, SampleTest, TestBinaryField, MinimalSchemaTest, TestElisionMultitermQuery, OutputWriterTest, TestSolrCoreProperties, TestPostingsSolrHighlighter, DirectSolrConnectionTest, TestCharFilters, TestCodecSupport, SynonymTokenizerTest, TestXIncludeConfig, EchoParamsTest, TestSweetSpotSimilarityFactory, TestDFRSimilarityFactory, TestLMJelinekMercerSimilarityFactory, TestIBSimilarityFactory, TimeZoneUtilsTest, ResourceLoaderTest, TestFastOutputStream, ScriptEngineTest, URLClassifyProcessorTest, PluginInfoTest, PreAnalyzedFieldTest, TestSystemIdResolver, PrimUtilsTest, SpellingQueryConverterTest, DOMUtilTest, RAMDirectoryFactoryTest, TestSolrJ, TestLRUCache, ZkNodePropsTest, SliceStateTest, SystemInfoHandlerTest, UUIDFieldTest, FileUtilsTest, CircularListTest, DistributedMLTComponentTest, CursorPagingTest, DistributedIntervalFacetingTest, SolrTestCaseJ4Test, TestDistributedMissingSort, TestTolerantSearch, TestEmbeddedSolrServerAdminHandler, TestEmbeddedSolrServerSchemaAPI, TestJettySolrRunner, AliasIntegrationTest, ChaosMonkeyNothingIsSafeWithPullReplicasTest, CollectionsAPISolrJTest, DeleteLastCustomShardedReplicaTest, DeleteNodeTest, DeleteShardTest, DeleteStatusTest, DistribCursorPagingTest, DistribJoinFromCollectionTest, DocValuesNotIndexedTest, ForceLeaderTest, LeaderFailoverAfterPartitionTest, LeaderInitiatedRecoveryOnShardRestartTest, MissingSegmentRecoveryTest, MultiThreadedOCPTest, NodeMutatorTest, OverriddenZkACLAndCredentialsProvidersTest, PeerSyncReplicationTest, ReplicationFactorTest, SSLMigrationTest, ShardSplitTest]
[junit4] Completed [622/764 (2!)] on J2 in 946.85s, 5 tests, 1 error <<< FAILURES!
[...truncated 44374 lines...]
[JENKINS] Lucene-Solr-7.x-Linux (32bit/jdk1.8.0_144) - Build # 1088
- Still Failing!
Posted by Policeman Jenkins Server <je...@thetaphi.de>.
Build: https://jenkins.thetaphi.de/job/Lucene-Solr-7.x-Linux/1088/
Java: 32bit/jdk1.8.0_144 -client -XX:+UseConcMarkSweepGC
2 tests failed.
FAILED: org.apache.solr.cloud.autoscaling.AutoAddReplicasPlanActionTest.testSimple
Error Message:
No live SolrServers available to handle this request:[http://127.0.0.1:42023/solr]
Stack Trace:
org.apache.solr.client.solrj.SolrServerException: No live SolrServers available to handle this request:[http://127.0.0.1:42023/solr]
at __randomizedtesting.SeedInfo.seed([D85FA2E647359F41:E0EC861860C64B90]:0)
at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:462)
at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1104)
at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:884)
at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:817)
at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1219)
at org.apache.solr.cloud.autoscaling.AutoAddReplicasPlanActionTest.testSimple(AutoAddReplicasPlanActionTest.java:110)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
Caused by: org.apache.solr.client.solrj.SolrServerException: IOException occured when talking to server at: http://127.0.0.1:42023/solr
at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:657)
at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:255)
at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:244)
at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:483)
at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:413)
... 44 more
Caused by: org.apache.http.NoHttpResponseException: 127.0.0.1:42023 failed to respond
at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:141)
at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:56)
at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:259)
at org.apache.http.impl.DefaultBHttpClientConnection.receiveResponseHeader(DefaultBHttpClientConnection.java:163)
at org.apache.http.impl.conn.CPoolProxy.receiveResponseHeader(CPoolProxy.java:165)
at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:273)
at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:125)
at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:272)
at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:185)
at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89)
at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:111)
at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185)
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83)
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:56)
at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:542)
... 48 more
FAILED: org.apache.solr.schema.TestCloudSchemaless.test
Error Message:
Timeout occured while waiting response from server at: http://127.0.0.1:38307
Stack Trace:
org.apache.solr.client.solrj.SolrServerException: Timeout occured while waiting response from server at: http://127.0.0.1:38307
at __randomizedtesting.SeedInfo.seed([D85FA2E647359F41:500B9D3CE9C9F2B9]:0)
at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:654)
at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:255)
at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:244)
at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:483)
at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:413)
at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1104)
at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:884)
at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:817)
at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:194)
at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:211)
at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createServers(AbstractFullDistribZkTestBase.java:320)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:991)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:968)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.net.SocketTimeoutException: Read timed out
at java.net.SocketInputStream.socketRead0(Native Method)
at java.net.SocketInputStream.socketRead(SocketInputStream.java:116)
at java.net.SocketInputStream.read(SocketInputStream.java:171)
at java.net.SocketInputStream.read(SocketInputStream.java:141)
at org.apache.http.impl.io.SessionInputBufferImpl.streamRead(SessionInputBufferImpl.java:137)
at org.apache.http.impl.io.SessionInputBufferImpl.fillBuffer(SessionInputBufferImpl.java:153)
at org.apache.http.impl.io.SessionInputBufferImpl.readLine(SessionInputBufferImpl.java:282)
at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:138)
at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:56)
at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:259)
at org.apache.http.impl.DefaultBHttpClientConnection.receiveResponseHeader(DefaultBHttpClientConnection.java:163)
at org.apache.http.impl.conn.CPoolProxy.receiveResponseHeader(CPoolProxy.java:165)
at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:273)
at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:125)
at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:272)
at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:185)
at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89)
at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:111)
at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185)
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83)
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:56)
at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:542)
... 43 more
Build Log:
[...truncated 11988 lines...]
[junit4] Suite: org.apache.solr.schema.TestCloudSchemaless
[junit4] 2> 143973 INFO (SUITE-TestCloudSchemaless-seed#[D85FA2E647359F41]-worker) [ ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
[junit4] 2> Creating dataDir: /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J0/temp/solr.schema.TestCloudSchemaless_D85FA2E647359F41-001/init-core-data-001
[junit4] 2> 143974 WARN (SUITE-TestCloudSchemaless-seed#[D85FA2E647359F41]-worker) [ ] o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=21 numCloses=21
[junit4] 2> 143974 INFO (SUITE-TestCloudSchemaless-seed#[D85FA2E647359F41]-worker) [ ] o.a.s.SolrTestCaseJ4 Using PointFields (NUMERIC_POINTS_SYSPROP=true) w/NUMERIC_DOCVALUES_SYSPROP=true
[junit4] 2> 143975 INFO (SUITE-TestCloudSchemaless-seed#[D85FA2E647359F41]-worker) [ ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false) via: @org.apache.solr.SolrTestCaseJ4$SuppressSSL(bugUrl=https://issues.apache.org/jira/browse/SOLR-5776)
[junit4] 2> 143975 INFO (SUITE-TestCloudSchemaless-seed#[D85FA2E647359F41]-worker) [ ] o.a.s.BaseDistributedSearchTestCase Setting hostContext system property: /
[junit4] 2> 143977 INFO (TEST-TestCloudSchemaless.test-seed#[D85FA2E647359F41]) [ ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
[junit4] 2> 143977 INFO (Thread-221) [ ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 143978 INFO (Thread-221) [ ] o.a.s.c.ZkTestServer Starting server
[junit4] 2> 143980 ERROR (Thread-221) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 144078 INFO (TEST-TestCloudSchemaless.test-seed#[D85FA2E647359F41]) [ ] o.a.s.c.ZkTestServer start zk server on port:34683
[junit4] 2> 144091 INFO (zkConnectionManagerCallback-198-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 144101 INFO (zkConnectionManagerCallback-200-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 144120 INFO (TEST-TestCloudSchemaless.test-seed#[D85FA2E647359F41]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/core/src/test-files/solr/collection1/conf/solrconfig-schemaless.xml to /configs/conf1/solrconfig.xml
[junit4] 2> 144125 INFO (TEST-TestCloudSchemaless.test-seed#[D85FA2E647359F41]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/core/src/test-files/solr/collection1/conf/schema-add-schema-fields-update-processor.xml to /configs/conf1/schema.xml
[junit4] 2> 144129 INFO (TEST-TestCloudSchemaless.test-seed#[D85FA2E647359F41]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
[junit4] 2> 144133 INFO (TEST-TestCloudSchemaless.test-seed#[D85FA2E647359F41]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
[junit4] 2> 144137 INFO (TEST-TestCloudSchemaless.test-seed#[D85FA2E647359F41]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
[junit4] 2> 144141 INFO (TEST-TestCloudSchemaless.test-seed#[D85FA2E647359F41]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
[junit4] 2> 144146 INFO (TEST-TestCloudSchemaless.test-seed#[D85FA2E647359F41]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
[junit4] 2> 144151 INFO (TEST-TestCloudSchemaless.test-seed#[D85FA2E647359F41]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
[junit4] 2> 144154 INFO (TEST-TestCloudSchemaless.test-seed#[D85FA2E647359F41]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4] 2> 144157 INFO (TEST-TestCloudSchemaless.test-seed#[D85FA2E647359F41]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
[junit4] 2> 144160 INFO (TEST-TestCloudSchemaless.test-seed#[D85FA2E647359F41]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
[junit4] 2> 144167 INFO (TEST-TestCloudSchemaless.test-seed#[D85FA2E647359F41]) [ ] o.a.s.c.AbstractFullDistribZkTestBase Will use NRT replicas unless explicitly asked otherwise
[junit4] 2> 144371 INFO (TEST-TestCloudSchemaless.test-seed#[D85FA2E647359F41]) [ ] o.e.j.s.Server jetty-9.3.20.v20170531
[junit4] 2> 144383 INFO (TEST-TestCloudSchemaless.test-seed#[D85FA2E647359F41]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@13a8395{/,null,AVAILABLE}
[junit4] 2> 144386 INFO (TEST-TestCloudSchemaless.test-seed#[D85FA2E647359F41]) [ ] o.e.j.s.AbstractConnector Started ServerConnector@1646950{HTTP/1.1,[http/1.1]}{127.0.0.1:38307}
[junit4] 2> 144386 INFO (TEST-TestCloudSchemaless.test-seed#[D85FA2E647359F41]) [ ] o.e.j.s.Server Started @145501ms
[junit4] 2> 144386 INFO (TEST-TestCloudSchemaless.test-seed#[D85FA2E647359F41]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J0/temp/solr.schema.TestCloudSchemaless_D85FA2E647359F41-001/tempDir-001/control/data, hostContext=/, hostPort=38307, coreRootDirectory=/home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J0/../../../../../../../../../home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J0/temp/solr.schema.TestCloudSchemaless_D85FA2E647359F41-001/control-001/cores}
[junit4] 2> 144387 ERROR (TEST-TestCloudSchemaless.test-seed#[D85FA2E647359F41]) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 144387 INFO (TEST-TestCloudSchemaless.test-seed#[D85FA2E647359F41]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 7.3.0
[junit4] 2> 144387 INFO (TEST-TestCloudSchemaless.test-seed#[D85FA2E647359F41]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 144387 INFO (TEST-TestCloudSchemaless.test-seed#[D85FA2E647359F41]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 144387 INFO (TEST-TestCloudSchemaless.test-seed#[D85FA2E647359F41]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2017-12-31T12:32:44.820Z
[junit4] 2> 144396 INFO (zkConnectionManagerCallback-202-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 144398 INFO (TEST-TestCloudSchemaless.test-seed#[D85FA2E647359F41]) [ ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
[junit4] 2> 144398 INFO (TEST-TestCloudSchemaless.test-seed#[D85FA2E647359F41]) [ ] o.a.s.c.SolrXmlConfig Loading container configuration from /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J0/temp/solr.schema.TestCloudSchemaless_D85FA2E647359F41-001/control-001/solr.xml
[junit4] 2> 144429 INFO (TEST-TestCloudSchemaless.test-seed#[D85FA2E647359F41]) [ ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverWorkLoopDelay is ignored
[junit4] 2> 144429 INFO (TEST-TestCloudSchemaless.test-seed#[D85FA2E647359F41]) [ ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverBadNodeExpiration is ignored
[junit4] 2> 144441 INFO (TEST-TestCloudSchemaless.test-seed#[D85FA2E647359F41]) [ ] o.a.s.c.SolrXmlConfig MBean server found: com.sun.jmx.mbeanserver.JmxMBeanServer@1a67344, but no JMX reporters were configured - adding default JMX reporter.
[junit4] 2> 144448 INFO (TEST-TestCloudSchemaless.test-seed#[D85FA2E647359F41]) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:34683/solr
[junit4] 2> 144456 INFO (zkConnectionManagerCallback-206-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 144470 INFO (zkConnectionManagerCallback-208-thread-1-processing-n:127.0.0.1:38307_) [n:127.0.0.1:38307_ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 144611 INFO (TEST-TestCloudSchemaless.test-seed#[D85FA2E647359F41]) [n:127.0.0.1:38307_ ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 144612 INFO (TEST-TestCloudSchemaless.test-seed#[D85FA2E647359F41]) [n:127.0.0.1:38307_ ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:38307_
[junit4] 2> 144613 INFO (TEST-TestCloudSchemaless.test-seed#[D85FA2E647359F41]) [n:127.0.0.1:38307_ ] o.a.s.c.Overseer Overseer (id=99268923517304836-127.0.0.1:38307_-n_0000000000) starting
[junit4] 2> 144627 INFO (TEST-TestCloudSchemaless.test-seed#[D85FA2E647359F41]) [n:127.0.0.1:38307_ ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:38307_
[junit4] 2> 144633 INFO (OverseerStateUpdate-99268923517304836-127.0.0.1:38307_-n_0000000000) [n:127.0.0.1:38307_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 144955 INFO (TEST-TestCloudSchemaless.test-seed#[D85FA2E647359F41]) [n:127.0.0.1:38307_ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@1a67344
[junit4] 2> 145010 INFO (TEST-TestCloudSchemaless.test-seed#[D85FA2E647359F41]) [n:127.0.0.1:38307_ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@1a67344
[junit4] 2> 145010 INFO (TEST-TestCloudSchemaless.test-seed#[D85FA2E647359F41]) [n:127.0.0.1:38307_ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@1a67344
[junit4] 2> 145013 INFO (TEST-TestCloudSchemaless.test-seed#[D85FA2E647359F41]) [n:127.0.0.1:38307_ ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J0/../../../../../../../../../home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J0/temp/solr.schema.TestCloudSchemaless_D85FA2E647359F41-001/control-001/cores
[junit4] 2> 145079 INFO (zkConnectionManagerCallback-214-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 145091 INFO (TEST-TestCloudSchemaless.test-seed#[D85FA2E647359F41]) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 145092 INFO (TEST-TestCloudSchemaless.test-seed#[D85FA2E647359F41]) [ ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:34683/solr ready
[junit4] 2> 145098 INFO (qtp5701713-883) [n:127.0.0.1:38307_ ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params replicationFactor=1&collection.configName=conf1&name=control_collection&nrtReplicas=1&action=CREATE&numShards=1&createNodeSet=127.0.0.1:38307_&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 145100 INFO (OverseerThreadFactory-365-thread-1-processing-n:127.0.0.1:38307_) [n:127.0.0.1:38307_ ] o.a.s.c.CreateCollectionCmd Create collection control_collection
[junit4] 2> 145220 INFO (qtp5701713-879) [n:127.0.0.1:38307_ ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf1&newCollection=true&name=control_collection_shard1_replica_n1&action=CREATE&numShards=1&collection=control_collection&shard=shard1&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 145220 INFO (qtp5701713-879) [n:127.0.0.1:38307_ ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 4 transient cores
[junit4] 2> 145331 INFO (zkCallback-207-thread-1-processing-n:127.0.0.1:38307_) [n:127.0.0.1:38307_ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/control_collection/state.json] for collection [control_collection] has occurred - updating... (live nodes size: [1])
[junit4] 2> 146287 INFO (qtp5701713-879) [n:127.0.0.1:38307_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.3.0
[junit4] 2> 146376 INFO (qtp5701713-879) [n:127.0.0.1:38307_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.s.ManagedIndexSchemaFactory The schema is configured as managed, but managed schema resource managed-schema not found - loading non-managed schema schema.xml instead
[junit4] 2> 146389 INFO (qtp5701713-879) [n:127.0.0.1:38307_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.s.IndexSchema [control_collection_shard1_replica_n1] Schema name=add-schema-fields-update-processor
[junit4] 2> 146502 INFO (qtp5701713-879) [n:127.0.0.1:38307_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.s.IndexSchema Loaded schema add-schema-fields-update-processor/1.6 with uniqueid field id
[junit4] 2> 146527 INFO (qtp5701713-879) [n:127.0.0.1:38307_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.s.ManagedIndexSchema Created and persisted managed schema znode at /configs/conf1/managed-schema
[junit4] 2> 146532 INFO (qtp5701713-879) [n:127.0.0.1:38307_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.s.ManagedIndexSchemaFactory After upgrading to managed schema in ZooKeeper, renamed the non-managed schema /configs/conf1/schema.xml to /configs/conf1/schema.xml.bak
[junit4] 2> 146532 INFO (qtp5701713-879) [n:127.0.0.1:38307_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.CoreContainer Creating SolrCore 'control_collection_shard1_replica_n1' using configuration from collection control_collection, trusted=true
[junit4] 2> 146532 INFO (qtp5701713-879) [n:127.0.0.1:38307_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.control_collection.shard1.replica_n1' (registry 'solr.core.control_collection.shard1.replica_n1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@1a67344
[junit4] 2> 146534 INFO (qtp5701713-879) [n:127.0.0.1:38307_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 146534 INFO (qtp5701713-879) [n:127.0.0.1:38307_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.SolrCore [[control_collection_shard1_replica_n1] ] Opening new SolrCore at [/home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J0/temp/solr.schema.TestCloudSchemaless_D85FA2E647359F41-001/control-001/cores/control_collection_shard1_replica_n1], dataDir=[/home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J0/../../../../../../../../../home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J0/temp/solr.schema.TestCloudSchemaless_D85FA2E647359F41-001/control-001/cores/control_collection_shard1_replica_n1/data/]
[junit4] 2> 146554 INFO (qtp5701713-879) [n:127.0.0.1:38307_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=50, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0]
[junit4] 2> 146902 INFO (qtp5701713-879) [n:127.0.0.1:38307_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 146902 INFO (qtp5701713-879) [n:127.0.0.1:38307_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 146941 INFO (qtp5701713-879) [n:127.0.0.1:38307_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 146941 INFO (qtp5701713-879) [n:127.0.0.1:38307_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 146943 INFO (qtp5701713-879) [n:127.0.0.1:38307_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=46, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0]
[junit4] 2> 146944 INFO (qtp5701713-879) [n:127.0.0.1:38307_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.s.SolrIndexSearcher Opening [Searcher@1009e14[control_collection_shard1_replica_n1] main]
[junit4] 2> 146957 INFO (qtp5701713-879) [n:127.0.0.1:38307_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 146967 INFO (qtp5701713-879) [n:127.0.0.1:38307_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 146969 INFO (qtp5701713-879) [n:127.0.0.1:38307_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.s.ZkIndexSchemaReader Creating ZooKeeper watch for the managed schema at /configs/conf1/managed-schema
[junit4] 2> 146971 INFO (qtp5701713-879) [n:127.0.0.1:38307_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.s.ZkIndexSchemaReader Current schema version 0 is already the latest
[junit4] 2> 146973 INFO (qtp5701713-879) [n:127.0.0.1:38307_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
[junit4] 2> 146975 INFO (searcherExecutor-368-thread-1-processing-n:127.0.0.1:38307_ x:control_collection_shard1_replica_n1 s:shard1 c:control_collection) [n:127.0.0.1:38307_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.SolrCore [control_collection_shard1_replica_n1] Registered new searcher Searcher@1009e14[control_collection_shard1_replica_n1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 146975 INFO (qtp5701713-879) [n:127.0.0.1:38307_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1588302779418411008
[junit4] 2> 146981 INFO (qtp5701713-879) [n:127.0.0.1:38307_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
[junit4] 2> 146981 INFO (qtp5701713-879) [n:127.0.0.1:38307_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
[junit4] 2> 146981 INFO (qtp5701713-879) [n:127.0.0.1:38307_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:38307/control_collection_shard1_replica_n1/
[junit4] 2> 146981 INFO (qtp5701713-879) [n:127.0.0.1:38307_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
[junit4] 2> 146981 INFO (qtp5701713-879) [n:127.0.0.1:38307_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.SyncStrategy http://127.0.0.1:38307/control_collection_shard1_replica_n1/ has no replicas
[junit4] 2> 146981 INFO (qtp5701713-879) [n:127.0.0.1:38307_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
[junit4] 2> 146987 INFO (qtp5701713-879) [n:127.0.0.1:38307_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:38307/control_collection_shard1_replica_n1/ shard1
[junit4] 2> 205122 INFO (TEST-TestCloudSchemaless.test-seed#[D85FA2E647359F41]) [ ] o.a.s.c.ChaosMonkey monkey: stop jetty! 38307
[junit4] 2> 205122 INFO (TEST-TestCloudSchemaless.test-seed#[D85FA2E647359F41]) [ ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=21805513
[junit4] 2> 205127 INFO (TEST-TestCloudSchemaless.test-seed#[D85FA2E647359F41]) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.node, tag=null
[junit4] 2> 205127 INFO (TEST-TestCloudSchemaless.test-seed#[D85FA2E647359F41]) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@a23877: rootName = null, domain = solr.node, service url = null, agent id = null] for registry solr.node / com.codahale.metrics.MetricRegistry@18542d5
[junit4] 2> 205133 INFO (TEST-TestCloudSchemaless.test-seed#[D85FA2E647359F41]) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jvm, tag=null
[junit4] 2> 205134 INFO (TEST-TestCloudSchemaless.test-seed#[D85FA2E647359F41]) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@9b7f11: rootName = null, domain = solr.jvm, service url = null, agent id = null] for registry solr.jvm / com.codahale.metrics.MetricRegistry@11bedd5
[junit4] 2> 205155 INFO (TEST-TestCloudSchemaless.test-seed#[D85FA2E647359F41]) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jetty, tag=null
[junit4] 2> 205156 INFO (TEST-TestCloudSchemaless.test-seed#[D85FA2E647359F41]) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@1fca93d: rootName = null, domain = solr.jetty, service url = null, agent id = null] for registry solr.jetty / com.codahale.metrics.MetricRegistry@1246a6e
[junit4] 2> 205159 INFO (TEST-TestCloudSchemaless.test-seed#[D85FA2E647359F41]) [ ] o.a.s.c.ZkController Remove node as live in ZooKeeper:/live_nodes/127.0.0.1:38307_
[junit4] 2> 205160 INFO (zkCallback-207-thread-2-processing-n:127.0.0.1:38307_) [n:127.0.0.1:38307_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (0)
[junit4] 2> 205161 INFO (TEST-TestCloudSchemaless.test-seed#[D85FA2E647359F41]) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.cluster, tag=null
[junit4] 2> 205163 INFO (coreCloseExecutor-373-thread-1) [n:127.0.0.1:38307_ c:control_collection s:shard1 r:core_node2 x:control_collection_shard1_replica_n1] o.a.s.c.SolrCore [control_collection_shard1_replica_n1] CLOSING SolrCore org.apache.solr.core.SolrCore@f8d325
[junit4] 2> 205163 INFO (coreCloseExecutor-373-thread-1) [n:127.0.0.1:38307_ c:control_collection s:shard1 r:core_node2 x:control_collection_shard1_replica_n1] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.control_collection.shard1.replica_n1, tag=16306981
[junit4] 2> 205163 INFO (coreCloseExecutor-373-thread-1) [n:127.0.0.1:38307_ c:control_collection s:shard1 r:core_node2 x:control_collection_shard1_replica_n1] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@1fbb8c5: rootName = null, domain = solr.core.control_collection.shard1.replica_n1, service url = null, agent id = null] for registry solr.core.control_collection.shard1.replica_n1 / com.codahale.metrics.MetricRegistry@1b86d23
[junit4] 2> 205182 INFO (coreCloseExecutor-373-thread-1) [n:127.0.0.1:38307_ c:control_collection s:shard1 r:core_node2 x:control_collection_shard1_replica_n1] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.control_collection.shard1.leader, tag=16306981
[junit4] 2> 215659 INFO (ScheduledTrigger-363-thread-1) [n:127.0.0.1:38307_ ] o.a.s.c.a.SystemLogListener Collection .system does not exist, disabling logging.
[junit4] 2> 215675 INFO (AutoscalingActionExecutor-364-thread-1-processing-n:127.0.0.1:38307_) [n:127.0.0.1:38307_ ] o.a.s.c.s.c.a.PolicyHelper returnSession, curr-time 2424778080 sessionWrapper.createTime 2424778075801170, this.sessionWrapper.createTime 2424778075801170
[junit4] 2> 215675 INFO (AutoscalingActionExecutor-364-thread-1-processing-n:127.0.0.1:38307_) [n:127.0.0.1:38307_ ] o.a.s.c.a.ExecutePlanAction No operations to execute for event: {
[junit4] 2> "id":"89d504ee42a06T31dfcn7cf49c5jglg7m94gmxp",
[junit4] 2> "source":".auto_add_replicas",
[junit4] 2> "eventTime":2424768060205574,
[junit4] 2> "eventType":"NODELOST",
[junit4] 2> "properties":{
[junit4] 2> "eventTimes":[2424768060205574],
[junit4] 2> "_enqueue_time_":2424778062807864,
[junit4] 2> "nodeNames":["127.0.0.1:38307_"]}}
[junit4] 2> 325101 ERROR (qtp5701713-883) [n:127.0.0.1:38307_ ] o.a.s.h.RequestHandlerBase org.apache.solr.common.SolrException: create the collection time out:180s
[junit4] 2> at org.apache.solr.handler.admin.CollectionsHandler.handleResponse(CollectionsHandler.java:314)
[junit4] 2> at org.apache.solr.handler.admin.CollectionsHandler.invokeAction(CollectionsHandler.java:246)
[junit4] 2> at org.apache.solr.handler.admin.CollectionsHandler.handleRequestBody(CollectionsHandler.java:224)
[junit4] 2> at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:177)
[junit4] 2> at org.apache.solr.servlet.HttpSolrCall.handleAdmin(HttpSolrCall.java:736)
[junit4] 2> at org.apache.solr.servlet.HttpSolrCall.handleAdminRequest(HttpSolrCall.java:717)
[junit4] 2> at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:498)
[junit4] 2> at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:382)
[junit4] 2> at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:326)
[junit4] 2> at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1759)
[junit4] 2> at org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:139)
[junit4] 2> at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1759)
[junit4] 2> at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:582)
[junit4] 2> at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:224)
[junit4] 2> at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1180)
[junit4] 2> at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:512)
[junit4] 2> at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
[junit4] 2> at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1112)
[junit4] 2> at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
[junit4] 2> at org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:493)
[junit4] 2> at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134)
[junit4] 2> at org.eclipse.jetty.server.Server.handle(Server.java:534)
[junit4] 2> at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:320)
[junit4] 2> at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:251)
[junit4] 2> at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:283)
[junit4] 2> at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:108)
[junit4] 2> at org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:93)
[junit4] 2> at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:303)
[junit4] 2> at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148)
[junit4] 2> at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:136)
[junit4] 2> at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671)
[junit4] 2> at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2>
[junit4] 2> 325101 INFO (qtp5701713-883) [n:127.0.0.1:38307_ ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={replicationFactor=1&collection.configName=conf1&name=control_collection&nrtReplicas=1&action=CREATE&numShards=1&createNodeSet=127.0.0.1:38307_&wt=javabin&version=2} status=500 QTime=180003
[junit4] 2> 325103 ERROR (qtp5701713-883) [n:127.0.0.1:38307_ ] o.a.s.s.HttpSolrCall null:org.apache.solr.common.SolrException: create the collection time out:180s
[junit4] 2> at org.apache.solr.handler.admin.CollectionsHandler.handleResponse(CollectionsHandler.java:314)
[junit4] 2> at org.apache.solr.handler.admin.CollectionsHandler.invokeAction(CollectionsHandler.java:246)
[junit4] 2> at org.apache.solr.handler.admin.CollectionsHandler.handleRequestBody(CollectionsHandler.java:224)
[junit4] 2> at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:177)
[junit4] 2> at org.apache.solr.servlet.HttpSolrCall.handleAdmin(HttpSolrCall.java:736)
[junit4] 2> at org.apache.solr.servlet.HttpSolrCall.handleAdminRequest(HttpSolrCall.java:717)
[junit4] 2> at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:498)
[junit4] 2> at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:382)
[junit4] 2> at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:326)
[junit4] 2> at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1759)
[junit4] 2> at org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:139)
[junit4] 2> at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1759)
[junit4] 2> at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:582)
[junit4] 2> at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:224)
[junit4] 2> at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1180)
[junit4] 2> at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:512)
[junit4] 2> at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
[junit4] 2> at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1112)
[junit4] 2> at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
[junit4] 2> at org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:493)
[junit4] 2> at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134)
[junit4] 2> at org.eclipse.jetty.server.Server.handle(Server.java:534)
[junit4] 2> at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:320)
[junit4] 2> at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:251)
[junit4] 2> at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:283)
[junit4] 2> at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:108)
[junit4] 2> at org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:93)
[junit4] 2> at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:303)
[junit4] 2> at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148)
[junit4] 2> at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:136)
[junit4] 2> at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671)
[junit4] 2> at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2>
[junit4] 2> 745265 ERROR (OverseerThreadFactory-365-thread-1-processing-n:127.0.0.1:38307_) [n:127.0.0.1:38307_ ] o.a.s.c.OverseerCollectionMessageHandler Error from shard: http://127.0.0.1:38307
[junit4] 2> org.apache.solr.client.solrj.SolrServerException: Timeout occured while waiting response from server at: http://127.0.0.1:38307
[junit4] 2> at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:654)
[junit4] 2> at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:255)
[junit4] 2> at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:244)
[junit4] 2> at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1219)
[junit4] 2> at org.apache.solr.handler.component.HttpShardHandler.lambda$submit$0(HttpShardHandler.java:172)
[junit4] 2> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
[junit4] 2> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
[junit4] 2> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
[junit4] 2> at com.codahale.metrics.InstrumentedExecutorService$InstrumentedRunnable.run(InstrumentedExecutorService.java:176)
[junit4] 2> at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:188)
[junit4] 2> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
[junit4] 2> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2> Caused by: java.net.SocketTimeoutException: Read timed out
[junit4] 2> at java.net.SocketInputStream.socketRead0(Native Method)
[junit4] 2> at java.net.SocketInputStream.socketRead(SocketInputStream.java:116)
[junit4] 2> at java.net.SocketInputStream.read(SocketInputStream.java:171)
[junit4] 2> at java.net.SocketInputStream.read(SocketInputStream.java:141)
[junit4] 2> at org.apache.http.impl.io.SessionInputBufferImpl.streamRead(SessionInputBufferImpl.java:137)
[junit4] 2> at org.apache.http.impl.io.SessionInputBufferImpl.fillBuffer(SessionInputBufferImpl.java:153)
[junit4] 2> at org.apache.http.impl.io.SessionInputBufferImpl.readLine(SessionInputBufferImpl.java:282)
[junit4] 2> at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:138)
[junit4] 2> at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:56)
[junit4] 2> at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:259)
[junit4] 2> at org.apache.http.impl.DefaultBHttpClientConnection.receiveResponseHeader(DefaultBHttpClientConnection.java:163)
[junit4] 2> at org.apache.http.impl.conn.CPoolProxy.receiveResponseHeader(CPoolProxy.java:165)
[junit4] 2> at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:273)
[junit4] 2> at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:125)
[junit4] 2> at org.apache.solr.util.stats.InstrumentedHttpRequestExecutor.execute(InstrumentedHttpRequestExecutor.java:118)
[junit4] 2> at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:272)
[junit4] 2> at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:185)
[junit4] 2> at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89)
[junit4] 2> at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:111)
[junit4] 2> at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185)
[junit4] 2> at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83)
[junit4] 2> at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:56)
[junit4] 2> at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:542)
[junit4] 2> ... 12 more
[junit4] 2> 745266 INFO (TEST-TestCloudSchemaless.test-seed#[D85FA2E647359F41]) [ ] o.a.s.c.Overseer Overseer (id=99268923517304836-127.0.0.1:38307_-n_0000000000) closing
[junit4] 2> 745266 ERROR (OverseerThreadFactory-365-thread-1-processing-n:127.0.0.1:38307_) [n:127.0.0.1:38307_ ] o.a.s.c.OverseerCollectionMessageHandler Cleaning up collection [control_collection].
[junit4] 2> 745266 INFO (OverseerStateUpdate-99268923517304836-127.0.0.1:38307_-n_0000000000) [n:127.0.0.1:38307_ ] o.a.s.c.Overseer Overseer Loop exiting : 127.0.0.1:38307_
[junit4] 2> 745269 INFO (OverseerThreadFactory-365-thread-1-processing-n:127.0.0.1:38307_) [n:127.0.0.1:38307_ ] o.a.s.c.OverseerCollectionMessageHandler Executing Collection Cmd : action=UNLOAD&deleteInstanceDir=true&deleteDataDir=true
[junit4] 2> 775303 INFO (zkCallback-207-thread-5-processing-n:127.0.0.1:38307_) [n:127.0.0.1:38307_ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDeleted path:/collections/control_collection/state.json] for collection [control_collection] has occurred - updating... (live nodes size: [0])
[junit4] 2> 775304 ERROR (OverseerThreadFactory-365-thread-1-processing-n:127.0.0.1:38307_) [n:127.0.0.1:38307_ ] o.a.s.c.OverseerCollectionMessageHandler Collection: control_collection operation: create failed:org.apache.solr.common.SolrException: Could not fully remove collection: control_collection
[junit4] 2> at org.apache.solr.cloud.DeleteCollectionCmd.call(DeleteCollectionCmd.java:112)
[junit4] 2> at org.apache.solr.cloud.OverseerCollectionMessageHandler.cleanupCollection(OverseerCollectionMessageHandler.java:688)
[junit4] 2> at org.apache.solr.cloud.CreateCollectionCmd.call(CreateCollectionCmd.java:240)
[junit4] 2> at org.apache.solr.cloud.OverseerCollectionMessageHandler.processMessage(OverseerCollectionMessageHandler.java:242)
[junit4] 2> at org.apache.solr.cloud.OverseerTaskProcessor$Runner.run(OverseerTaskProcessor.java:464)
[junit4] 2> at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:188)
[junit4] 2> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
[junit4] 2> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2>
[junit4] 2> 775310 INFO (OverseerThreadFactory-365-thread-1-processing-n:127.0.0.1:38307_) [n:127.0.0.1:38307_ ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000000 doesn't exist. Requestor may have disconnected from ZooKeeper
[junit4] 2> 775310 WARN (OverseerAutoScalingTriggerThread-99268923517304836-127.0.0.1:38307_-n_0000000000) [n:127.0.0.1:38307_ ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread woken up but we are closed, exiting.
[junit4] 2> 775315 INFO (TEST-TestCloudSchemaless.test-seed#[D85FA2E647359F41]) [ ] o.e.j.s.AbstractConnector Stopped ServerConnector@1646950{HTTP/1.1,[http/1.1]}{127.0.0.1:0}
[junit4] 2> 775315 INFO (TEST-TestCloudSchemaless.test-seed#[D85FA2E647359F41]) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@13a8395{/,null,UNAVAILABLE}
[junit4] 2> 775337 ERROR (qtp5701713-879) [n:127.0.0.1:38307_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.ZkController Error getting leader from zk
[junit4] 2> org.apache.solr.common.SolrException: No registered leader was found after waiting for 1220000ms , collection: control_collection slice: shard1 saw state=null with live_nodes=[]
[junit4] 2> at org.apache.solr.common.cloud.ZkStateReader.getLeaderRetry(ZkStateReader.java:771)
[junit4] 2> at org.apache.solr.common.cloud.ZkStateReader.getLeaderUrl(ZkStateReader.java:736)
[junit4] 2> at org.apache.solr.cloud.ZkController.getLeader(ZkController.java:1178)
[junit4] 2> at org.apache.solr.cloud.ZkController.register(ZkController.java:1079)
[junit4] 2> at org.apache.solr.cloud.ZkController.register(ZkController.java:1014)
[junit4] 2> at org.apache.solr.core.ZkContainer.lambda$registerInZk$0(ZkContainer.java:181)
[junit4] 2> at org.apache.solr.core.ZkContainer.registerInZk(ZkContainer.java:208)
[junit4] 2> at org.apache.solr.core.CoreContainer.registerCore(CoreContainer.java:888)
[junit4] 2> at org.apache.solr.core.CoreContainer.createFromDescriptor(CoreContainer.java:1049)
[junit4] 2> at org.apache.solr.core.CoreContainer.create(CoreContainer.java:950)
[junit4] 2> at org.apache.solr.handler.admin.CoreAdminOperation.lambda$static$0(CoreAdminOperation.java:91)
[junit4] 2> at org.apache.solr.handler.admin.CoreAdminOperation.execute(CoreAdminOperation.java:384)
[junit4] 2> at org.apache.solr.handler.admin.CoreAdminHandler$CallInfo.call(CoreAdminHandler.java:389)
[junit4] 2> at org.apache.solr.handler.admin.CoreAdminHandler.handleRequestBody(CoreAdminHandler.java:174)
[junit4] 2> at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:177)
[junit4] 2> at org.apache.solr.servlet.HttpSolrCall.handleAdmin(HttpSolrCall.java:736)
[junit4] 2> at org.apache.solr.servlet.HttpSolrCall.handleAdminRequest(HttpSolrCall.java:717)
[junit4] 2> at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:498)
[junit4] 2> at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:382)
[junit4] 2> at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:326)
[junit4] 2> at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1759)
[junit4] 2> at org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:139)
[junit4] 2> at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1759)
[junit4] 2> at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:582)
[junit4] 2> at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:224)
[junit4] 2> at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1180)
[junit4] 2> at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:512)
[junit4] 2> at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
[junit4] 2> at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1112)
[junit4] 2> at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
[junit4] 2> at org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:426)
[junit4] 2> at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134)
[junit4] 2> at org.eclipse.jetty.server.Server.handle(Server.java:534)
[junit4] 2> at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:320)
[junit4] 2> at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:251)
[junit4] 2> at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:283)
[junit4] 2> at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:108)
[junit4] 2> at org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:93)
[junit4] 2> at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:303)
[junit4] 2> at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148)
[junit4] 2> at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.execute(ExecuteProduceConsume.java:100)
[junit4] 2> at org.eclipse.jetty.io.ManagedSelector.run(ManagedSelector.java:147)
[junit4] 2> at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671)
[junit4] 2> at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2> 775338 ERROR (qtp5701713-879) [n:127.0.0.1:38307_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.ZkContainer :org.apache.solr.common.SolrException: Error getting leader from zk for shard shard1
[junit4] 2> at org.apache.solr.cloud.ZkController.getLeader(ZkController.java:1207)
[junit4] 2> at org.apache.solr.cloud.ZkController.register(ZkController.java:1079)
[junit4] 2> at org.apache.solr.cloud.ZkController.register(ZkController.java:1014)
[junit4] 2> at org.apache.solr.core.ZkContainer.lambda$registerInZk$0(ZkContainer.java:181)
[junit4] 2> at org.apache.solr.core.ZkContainer.registerInZk(ZkContainer.java:208)
[junit4] 2> at org.apache.solr.core.CoreContainer.registerCore(CoreContainer.java:888)
[junit4] 2> at org.apache.solr.core.CoreContainer.createFromDescriptor(CoreContainer.java:1049)
[junit4] 2> at org.apache.solr.core.CoreContainer.create(CoreContainer.java:950)
[junit4] 2> at org.apache.solr.handler.admin.CoreAdminOperation.lambda$static$0(CoreAdminOperation.java:91)
[junit4] 2> at org.apache.solr.handler.admin.CoreAdminOperation.execute(CoreAdminOperation.java:384)
[junit4] 2> at org.apache.solr.handler.admin.CoreAdminHandler$CallInfo.call(CoreAdminHandler.java:389)
[junit4] 2> at org.apache.solr.handler.admin.CoreAdminHandler.handleRequestBody(CoreAdminHandler.java:174)
[junit4] 2> at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:177)
[junit4] 2> at org.apache.solr.servlet.HttpSolrCall.handleAdmin(HttpSolrCall.java:736)
[junit4] 2> at org.apache.solr.servlet.HttpSolrCall.handleAdminRequest(HttpSolrCall.java:717)
[junit4] 2> at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:498)
[junit4] 2> at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:382)
[junit4] 2> at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:326)
[junit4] 2> at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1759)
[junit4] 2> at org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:139)
[junit4] 2> at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1759)
[junit4] 2> at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:582)
[junit4] 2> at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:224)
[junit4] 2> at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1180)
[junit4] 2> at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:512)
[junit4] 2> at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
[junit4] 2> at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1112)
[junit4] 2> at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
[junit4] 2> at org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:426)
[junit4] 2> at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134)
[junit4] 2> at org.eclipse.jetty.server.Server.handle(Server.java:534)
[junit4] 2> at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:320)
[junit4] 2> at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:251)
[junit4] 2> at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:283)
[junit4] 2> at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:108)
[junit4] 2> at org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:93)
[junit4] 2> at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:303)
[junit4] 2> at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148)
[junit4] 2> at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.execute(ExecuteProduceConsume.java:100)
[junit4] 2> at org.eclipse.jetty.io.ManagedSelector.run(ManagedSelector.java:147)
[junit4] 2> at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671)
[junit4] 2> at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2> Caused by: org.apache.solr.common.SolrException: No registered leader was found after waiting for 1220000ms , collection: control_collection slice: shard1 saw state=null with live_nodes=[]
[junit4] 2> at org.apache.solr.common.cloud.ZkStateReader.getLeaderRetry(ZkStateReader.java:771)
[junit4] 2> at org.apache.solr.common.cloud.ZkStateReader.getLeaderUrl(ZkStateReader.java:736)
[junit4] 2> at org.apache.solr.cloud.ZkController.getLeader(ZkController.java:1178)
[junit4] 2> ... 42 more
[junit4] 2>
[junit4] 2> 775368 INFO (qtp5701713-879) [n:127.0.0.1:38307_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=conf1&newCollection=true&name=control_collection_shard1_replica_n1&action=CREATE&numShards=1&collection=control_collection&shard=shard1&wt=javabin&version=2&replicaType=NRT} status=0 QTime=630148
[junit4] 2> 775368 ERROR (TEST-TestCloudSchemaless.test-seed#[D85FA2E647359F41]) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 775369 INFO (TEST-TestCloudSchemaless.test-seed#[D85FA2E647359F41]) [ ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:34683 34683
[junit4] 2> 779940 INFO (Thread-221) [ ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:34683 34683
[junit4] 2> 779941 WARN (Thread-221) [ ] o.a.s.c.ZkTestServer Watch limit violations:
[junit4] 2> Maximum concurrent create/delete watches above limit:
[junit4] 2>
[junit4] 2> 2 /solr/aliases.json
[junit4] 2>
[junit4] 2> Maximum concurrent data watches above limit:
[junit4] 2>
[junit4] 2> 2 /solr/clusterstate.json
[junit4] 2> 2 /solr/clusterprops.json
[junit4] 2>
[junit4] 2> Maximum concurrent children watches above limit:
[junit4] 2>
[junit4] 2> 2 /solr/live_nodes
[junit4] 2> 2 /solr/collections
[junit4] 2>
[junit4] 2> NOTE: reproduce with: ant test -Dtestcase=TestCloudSchemaless -Dtests.method=test -Dtests.seed=D85FA2E647359F41 -Dtests.multiplier=3 -Dtests.slow=true -Dtests.locale=cs-CZ -Dtests.timezone=America/Buenos_Aires -Dtests.asserts=true -Dtests.file.encoding=ISO-8859-1
[junit4] ERROR 636s J0 | TestCloudSchemaless.test <<<
[junit4] > Throwable #1: org.apache.solr.client.solrj.SolrServerException: Timeout occured while waiting response from server at: http://127.0.0.1:38307
[junit4] > at __randomizedtesting.SeedInfo.seed([D85FA2E647359F41:500B9D3CE9C9F2B9]:0)
[junit4] > at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:654)
[junit4] > at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:255)
[junit4] > at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:244)
[junit4] > at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:483)
[junit4] > at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:413)
[junit4] > at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1104)
[junit4] > at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:884)
[junit4] > at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:817)
[junit4] > at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:194)
[junit4] > at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:211)
[junit4] > at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createServers(AbstractFullDistribZkTestBase.java:320)
[junit4] > at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:991)
[junit4] > at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:968)
[junit4] > at java.lang.Thread.run(Thread.java:748)
[junit4] > Caused by: java.net.SocketTimeoutException: Read timed out
[junit4] > at java.net.SocketInputStream.socketRead0(Native Method)
[junit4] > at java.net.SocketInputStream.socketRead(SocketInputStream.java:116)
[junit4] > at java.net.SocketInputStream.read(SocketInputStream.java:171)
[junit4] > at java.net.SocketInputStream.read(SocketInputStream.java:141)
[junit4] > at org.apache.http.impl.io.SessionInputBufferImpl.streamRead(SessionInputBufferImpl.java:137)
[junit4] > at org.apache.http.impl.io.SessionInputBufferImpl.fillBuffer(SessionInputBufferImpl.java:153)
[junit4] > at org.apache.http.impl.io.SessionInputBufferImpl.readLine(SessionInputBufferImpl.java:282)
[junit4] > at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:138)
[junit4] > at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:56)
[junit4] > at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:259)
[junit4] > at org.apache.http.impl.DefaultBHttpClientConnection.receiveResponseHeader(DefaultBHttpClientConnection.java:163)
[junit4] > at org.apache.http.impl.conn.CPoolProxy.receiveResponseHeader(CPoolProxy.java:165)
[junit4] > at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:273)
[junit4] > at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:125)
[junit4] > at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:272)
[junit4] > at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:185)
[junit4] > at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89)
[junit4] > at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:111)
[junit4] > at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185)
[junit4] > at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83)
[junit4] > at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:56)
[junit4] > at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:542)
[junit4] > ... 43 more
[junit4] 2> NOTE: leaving temporary files on disk at: /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J0/temp/solr.schema.TestCloudSchemaless_D85FA2E647359F41-001
[junit4] 2> NOTE: test params are: codec=CheapBastard, sim=RandomSimilarity(queryNorm=true): {}, locale=cs-CZ, timezone=America/Buenos_Aires
[junit4] 2> NOTE: Linux 4.10.0-40-generic i386/Oracle Corporation 1.8.0_144 (32-bit)/cpus=8,threads=1,free=20278936,total=85323776
[junit4] 2> NOTE: All tests run in this JVM: [IgnoreCommitOptimizeUpdateProcessorFactoryTest, TestExactSharedStatsCache, SpellCheckComponentTest, CloudMLTQParserTest, DateMathParserTest, FastVectorHighlighterTest, ResourceLoaderTest, TestSolr4Spatial2, PropertiesRequestHandlerTest, MetricTriggerTest, DeleteInactiveReplicaTest, ReplicaListTransformerTest, TestScoreJoinQPNoScore, SaslZkACLProviderTest, RemoteQueryErrorTest, CursorPagingTest, SolrMetricReporterTest, TestCloudSchemaless]
[junit4] Completed [95/764 (1!)] on J0 in 636.00s, 1 test, 1 error <<< FAILURES!
[...truncated 1383 lines...]
[junit4] Suite: org.apache.solr.cloud.autoscaling.AutoAddReplicasPlanActionTest
[junit4] 2> Creating dataDir: /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.autoscaling.AutoAddReplicasPlanActionTest_D85FA2E647359F41-001/init-core-data-001
[junit4] 2> 2515973 INFO (SUITE-AutoAddReplicasPlanActionTest-seed#[D85FA2E647359F41]-worker) [ ] o.a.s.SolrTestCaseJ4 Using TrieFields (NUMERIC_POINTS_SYSPROP=false) w/NUMERIC_DOCVALUES_SYSPROP=false
[junit4] 2> 2515974 INFO (SUITE-AutoAddReplicasPlanActionTest-seed#[D85FA2E647359F41]-worker) [ ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false) via: @org.apache.solr.util.RandomizeSSL(reason=, ssl=NaN, value=NaN, clientAuth=NaN)
[junit4] 2> 2515974 INFO (SUITE-AutoAddReplicasPlanActionTest-seed#[D85FA2E647359F41]-worker) [ ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
[junit4] 2> 2515975 INFO (SUITE-AutoAddReplicasPlanActionTest-seed#[D85FA2E647359F41]-worker) [ ] o.a.s.c.MiniSolrCloudCluster Starting cluster of 3 servers in /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.autoscaling.AutoAddReplicasPlanActionTest_D85FA2E647359F41-001/tempDir-001
[junit4] 2> 2515975 INFO (SUITE-AutoAddReplicasPlanActionTest-seed#[D85FA2E647359F41]-worker) [ ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
[junit4] 2> 2515975 INFO (Thread-2983) [ ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 2515975 INFO (Thread-2983) [ ] o.a.s.c.ZkTestServer Starting server
[junit4] 2> 2515991 ERROR (Thread-2983) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 2516075 INFO (SUITE-AutoAddReplicasPlanActionTest-seed#[D85FA2E647359F41]-worker) [ ] o.a.s.c.ZkTestServer start zk server on port:44833
[junit4] 2> 2516080 INFO (zkConnectionManagerCallback-2084-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2516088 INFO (jetty-launcher-2081-thread-1) [ ] o.e.j.s.Server jetty-9.3.20.v20170531
[junit4] 2> 2516088 INFO (jetty-launcher-2081-thread-2) [ ] o.e.j.s.Server jetty-9.3.20.v20170531
[junit4] 2> 2516088 INFO (jetty-launcher-2081-thread-3) [ ] o.e.j.s.Server jetty-9.3.20.v20170531
[junit4] 2> 2516089 INFO (jetty-launcher-2081-thread-1) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@f37ca9{/solr,null,AVAILABLE}
[junit4] 2> 2516089 INFO (jetty-launcher-2081-thread-2) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@ec4fd9{/solr,null,AVAILABLE}
[junit4] 2> 2516089 INFO (jetty-launcher-2081-thread-3) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@b73377{/solr,null,AVAILABLE}
[junit4] 2> 2516092 INFO (jetty-launcher-2081-thread-2) [ ] o.e.j.s.AbstractConnector Started ServerConnector@92972a{HTTP/1.1,[http/1.1]}{127.0.0.1:42023}
[junit4] 2> 2516092 INFO (jetty-launcher-2081-thread-3) [ ] o.e.j.s.AbstractConnector Started ServerConnector@1084f17{HTTP/1.1,[http/1.1]}{127.0.0.1:43225}
[junit4] 2> 2516092 INFO (jetty-launcher-2081-thread-2) [ ] o.e.j.s.Server Started @2517206ms
[junit4] 2> 2516092 INFO (jetty-launcher-2081-thread-3) [ ] o.e.j.s.Server Started @2517206ms
[junit4] 2> 2516092 INFO (jetty-launcher-2081-thread-2) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=42023}
[junit4] 2> 2516092 INFO (jetty-launcher-2081-thread-3) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=43225}
[junit4] 2> 2516092 ERROR (jetty-launcher-2081-thread-3) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 2516092 ERROR (jetty-launcher-2081-thread-2) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 2516092 INFO (jetty-launcher-2081-thread-3) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 7.3.0
[junit4] 2> 2516092 INFO (jetty-launcher-2081-thread-2) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 7.3.0
[junit4] 2> 2516092 INFO (jetty-launcher-2081-thread-3) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 2516092 INFO (jetty-launcher-2081-thread-2) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 2516092 INFO (jetty-launcher-2081-thread-3) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 2516092 INFO (jetty-launcher-2081-thread-2) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 2516093 INFO (jetty-launcher-2081-thread-3) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2017-12-31T13:12:16.526Z
[junit4] 2> 2516093 INFO (jetty-launcher-2081-thread-2) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2017-12-31T13:12:16.526Z
[junit4] 2> 2516093 INFO (jetty-launcher-2081-thread-1) [ ] o.e.j.s.AbstractConnector Started ServerConnector@133162d{HTTP/1.1,[http/1.1]}{127.0.0.1:33917}
[junit4] 2> 2516093 INFO (jetty-launcher-2081-thread-1) [ ] o.e.j.s.Server Started @2517208ms
[junit4] 2> 2516093 INFO (jetty-launcher-2081-thread-1) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=33917}
[junit4] 2> 2516093 ERROR (jetty-launcher-2081-thread-1) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 2516093 INFO (jetty-launcher-2081-thread-1) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 7.3.0
[junit4] 2> 2516093 INFO (jetty-launcher-2081-thread-1) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 2516093 INFO (jetty-launcher-2081-thread-1) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 2516094 INFO (jetty-launcher-2081-thread-1) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2017-12-31T13:12:16.526Z
[junit4] 2> 2516101 INFO (zkConnectionManagerCallback-2090-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2516102 INFO (zkConnectionManagerCallback-2086-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2516102 INFO (jetty-launcher-2081-thread-1) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 2516102 INFO (zkConnectionManagerCallback-2088-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2516105 INFO (jetty-launcher-2081-thread-3) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 2516105 INFO (jetty-launcher-2081-thread-2) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 2516114 INFO (jetty-launcher-2081-thread-1) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:44833/solr
[junit4] 2> 2516116 INFO (zkConnectionManagerCallback-2094-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2516118 INFO (jetty-launcher-2081-thread-3) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:44833/solr
[junit4] 2> 2516118 INFO (jetty-launcher-2081-thread-2) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:44833/solr
[junit4] 2> 2516122 INFO (zkConnectionManagerCallback-2100-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2516127 INFO (zkConnectionManagerCallback-2102-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2516129 WARN (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [ ] o.a.z.s.NIOServerCnxn caught end of stream exception
[junit4] 2> EndOfStreamException: Unable to read additional data from client sessionid 0x160acb38d200006, likely client has closed socket
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:239)
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:203)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2> 2516129 WARN (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [ ] o.a.z.s.NIOServerCnxn caught end of stream exception
[junit4] 2> EndOfStreamException: Unable to read additional data from client sessionid 0x160acb38d200005, likely client has closed socket
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:239)
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:203)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2> 2516129 INFO (zkConnectionManagerCallback-2104-thread-1-processing-n:127.0.0.1:33917_solr) [n:127.0.0.1:33917_solr ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2516132 INFO (zkConnectionManagerCallback-2106-thread-1-processing-n:127.0.0.1:42023_solr) [n:127.0.0.1:42023_solr ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2516132 INFO (zkConnectionManagerCallback-2108-thread-1-processing-n:127.0.0.1:43225_solr) [n:127.0.0.1:43225_solr ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2516220 INFO (jetty-launcher-2081-thread-1) [n:127.0.0.1:33917_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 2516220 INFO (jetty-launcher-2081-thread-1) [n:127.0.0.1:33917_solr ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:33917_solr
[junit4] 2> 2516221 INFO (jetty-launcher-2081-thread-1) [n:127.0.0.1:33917_solr ] o.a.s.c.Overseer Overseer (id=99269078968958983-127.0.0.1:33917_solr-n_0000000000) starting
[junit4] 2> 2516222 INFO (jetty-launcher-2081-thread-2) [n:127.0.0.1:42023_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 2516223 INFO (jetty-launcher-2081-thread-2) [n:127.0.0.1:42023_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:42023_solr
[junit4] 2> 2516230 INFO (zkCallback-2103-thread-1-processing-n:127.0.0.1:33917_solr) [n:127.0.0.1:33917_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 2516230 INFO (zkCallback-2105-thread-1-processing-n:127.0.0.1:42023_solr) [n:127.0.0.1:42023_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 2516238 INFO (jetty-launcher-2081-thread-3) [n:127.0.0.1:43225_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 2516240 INFO (jetty-launcher-2081-thread-3) [n:127.0.0.1:43225_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 2516240 INFO (jetty-launcher-2081-thread-1) [n:127.0.0.1:33917_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:33917_solr
[junit4] 2> 2516242 INFO (jetty-launcher-2081-thread-3) [n:127.0.0.1:43225_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:43225_solr
[junit4] 2> 2516249 INFO (zkCallback-2103-thread-1-processing-n:127.0.0.1:33917_solr) [n:127.0.0.1:33917_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (3)
[junit4] 2> 2516255 INFO (zkCallback-2105-thread-1-processing-n:127.0.0.1:42023_solr) [n:127.0.0.1:42023_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (3)
[junit4] 2> 2516261 INFO (zkCallback-2107-thread-1-processing-n:127.0.0.1:43225_solr) [n:127.0.0.1:43225_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (3)
[junit4] 2> 2516372 INFO (jetty-launcher-2081-thread-3) [n:127.0.0.1:43225_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_43225.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@1a67344
[junit4] 2> 2516381 INFO (jetty-launcher-2081-thread-3) [n:127.0.0.1:43225_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_43225.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@1a67344
[junit4] 2> 2516382 INFO (jetty-launcher-2081-thread-3) [n:127.0.0.1:43225_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_43225.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@1a67344
[junit4] 2> 2516383 INFO (jetty-launcher-2081-thread-3) [n:127.0.0.1:43225_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.autoscaling.AutoAddReplicasPlanActionTest_D85FA2E647359F41-001/tempDir-001/node2/.
[junit4] 2> 2516534 INFO (jetty-launcher-2081-thread-2) [n:127.0.0.1:42023_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_42023.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@1a67344
[junit4] 2> 2516550 INFO (jetty-launcher-2081-thread-2) [n:127.0.0.1:42023_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_42023.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@1a67344
[junit4] 2> 2516550 INFO (jetty-launcher-2081-thread-2) [n:127.0.0.1:42023_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_42023.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@1a67344
[junit4] 2> 2516552 INFO (jetty-launcher-2081-thread-2) [n:127.0.0.1:42023_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.autoscaling.AutoAddReplicasPlanActionTest_D85FA2E647359F41-001/tempDir-001/node3/.
[junit4] 2> 2516559 INFO (jetty-launcher-2081-thread-1) [n:127.0.0.1:33917_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_33917.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@1a67344
[junit4] 2> 2516568 INFO (jetty-launcher-2081-thread-1) [n:127.0.0.1:33917_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_33917.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@1a67344
[junit4] 2> 2516568 INFO (jetty-launcher-2081-thread-1) [n:127.0.0.1:33917_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_33917.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@1a67344
[junit4] 2> 2516569 INFO (jetty-launcher-2081-thread-1) [n:127.0.0.1:33917_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.autoscaling.AutoAddReplicasPlanActionTest_D85FA2E647359F41-001/tempDir-001/node1/.
[junit4] 2> 2516643 INFO (zkConnectionManagerCallback-2116-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2516645 WARN (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [ ] o.a.z.s.NIOServerCnxn caught end of stream exception
[junit4] 2> EndOfStreamException: Unable to read additional data from client sessionid 0x160acb38d20000a, likely client has closed socket
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:239)
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:203)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2> 2516648 INFO (zkConnectionManagerCallback-2120-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2516649 INFO (SUITE-AutoAddReplicasPlanActionTest-seed#[D85FA2E647359F41]-worker) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (3)
[junit4] 2> 2516650 INFO (SUITE-AutoAddReplicasPlanActionTest-seed#[D85FA2E647359F41]-worker) [ ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:44833/solr ready
[junit4] 2> 2516680 INFO (TEST-AutoAddReplicasPlanActionTest.testSimple-seed#[D85FA2E647359F41]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testSimple
[junit4] 2> 2516684 INFO (qtp10477421-11588) [n:127.0.0.1:42023_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params replicationFactor=2&collection.configName=conf&maxShardsPerNode=2&autoAddReplicas=true&name=testSimple1&nrtReplicas=2&action=CREATE&numShards=2&createNodeSet=127.0.0.1:43225_solr,127.0.0.1:42023_solr&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 2516686 INFO (OverseerThreadFactory-5374-thread-1-processing-n:127.0.0.1:33917_solr) [n:127.0.0.1:33917_solr ] o.a.s.c.CreateCollectionCmd Create collection testSimple1
[junit4] 2> 2516792 INFO (OverseerStateUpdate-99269078968958983-127.0.0.1:33917_solr-n_0000000000) [n:127.0.0.1:33917_solr ] o.a.s.c.o.SliceMutator createReplica() {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"testSimple1",
[junit4] 2> "shard":"shard1",
[junit4] 2> "core":"testSimple1_shard1_replica_n1",
[junit4] 2> "state":"down
[...truncated too long message...]
metric reporters for registry=solr.collection.testSimple1.shard2.leader, tag=28129834
[junit4] 2> 2529077 INFO (jetty-closer-2082-thread-2) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@286c05{/solr,null,UNAVAILABLE}
[junit4] 2> 2529086 INFO (coreCloseExecutor-5449-thread-4) [n:127.0.0.1:43225_solr c:testSimple3 s:shard3 r:core_node6 x:testSimple3_shard3_replica_n4] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.testSimple3.shard3.replica_n4, tag=3838554
[junit4] 2> 2529087 INFO (coreCloseExecutor-5449-thread-4) [n:127.0.0.1:43225_solr c:testSimple3 s:shard3 r:core_node6 x:testSimple3_shard3_replica_n4] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@13549cf: rootName = solr_43225, domain = solr.core.testSimple3.shard3.replica_n4, service url = null, agent id = null] for registry solr.core.testSimple3.shard3.replica_n4 / com.codahale.metrics.MetricRegistry@e43671
[junit4] 2> 2529098 INFO (coreCloseExecutor-5449-thread-4) [n:127.0.0.1:43225_solr c:testSimple3 s:shard3 r:core_node6 x:testSimple3_shard3_replica_n4] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.testSimple3.shard3.leader, tag=3838554
[junit4] 2> 2529099 INFO (coreCloseExecutor-5449-thread-5) [n:127.0.0.1:43225_solr c:testSimple3 s:shard2 r:core_node5 x:testSimple3_shard2_replica_n2] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.testSimple3.shard2.replica_n2, tag=19001138
[junit4] 2> 2529099 INFO (coreCloseExecutor-5449-thread-5) [n:127.0.0.1:43225_solr c:testSimple3 s:shard2 r:core_node5 x:testSimple3_shard2_replica_n2] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@144d843: rootName = solr_43225, domain = solr.core.testSimple3.shard2.replica_n2, service url = null, agent id = null] for registry solr.core.testSimple3.shard2.replica_n2 / com.codahale.metrics.MetricRegistry@114e25f
[junit4] 2> 2529111 INFO (coreCloseExecutor-5449-thread-2) [n:127.0.0.1:43225_solr c:testSimple1 s:shard2 r:core_node8 x:testSimple1_shard2_replica_n6] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.testSimple1.shard2.leader, tag=20539413
[junit4] 2> 2529112 INFO (coreCloseExecutor-5449-thread-1) [n:127.0.0.1:43225_solr c:testSimple1 s:shard1 r:core_node5 x:testSimple1_shard1_replica_n2] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.testSimple1.shard1.leader, tag=29453985
[junit4] 2> 2529112 INFO (coreCloseExecutor-5449-thread-3) [n:127.0.0.1:43225_solr c:testSimple3 s:shard1 r:core_node3 x:testSimple3_shard1_replica_n1] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.testSimple3.shard1.leader, tag=31763734
[junit4] 2> 2529112 INFO (coreCloseExecutor-5449-thread-5) [n:127.0.0.1:43225_solr c:testSimple3 s:shard2 r:core_node5 x:testSimple3_shard2_replica_n2] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.testSimple3.shard2.leader, tag=19001138
[junit4] 2> 2529114 INFO (jetty-closer-2082-thread-1) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@b73377{/solr,null,UNAVAILABLE}
[junit4] 2> 2529115 ERROR (SUITE-AutoAddReplicasPlanActionTest-seed#[D85FA2E647359F41]-worker) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 2529115 INFO (SUITE-AutoAddReplicasPlanActionTest-seed#[D85FA2E647359F41]-worker) [ ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:44833 44833
[junit4] 2> 2535519 INFO (Thread-2983) [ ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:44833 44833
[junit4] 2> 2535520 WARN (Thread-2983) [ ] o.a.s.c.ZkTestServer Watch limit violations:
[junit4] 2> Maximum concurrent create/delete watches above limit:
[junit4] 2>
[junit4] 2> 5 /solr/aliases.json
[junit4] 2> 5 /solr/clusterprops.json
[junit4] 2> 4 /solr/security.json
[junit4] 2> 4 /solr/configs/conf
[junit4] 2>
[junit4] 2> Maximum concurrent data watches above limit:
[junit4] 2>
[junit4] 2> 30 /solr/collections/testSimple1/state.json
[junit4] 2> 19 /solr/collections/testSimple2/state.json
[junit4] 2> 9 /solr/collections/testSimple3/state.json
[junit4] 2> 5 /solr/clusterstate.json
[junit4] 2> 2 /solr/collections/testSimple1/leader_elect/shard2/election/99269078968958985-core_node8-n_0000000000
[junit4] 2> 2 /solr/overseer_elect/election/99269078968958983-127.0.0.1:33917_solr-n_0000000000
[junit4] 2> 2 /solr/collections/testSimple2/leader_elect/shard1/election/99269078968958983-core_node3-n_0000000000
[junit4] 2> 2 /solr/collections/testSimple1/leader_elect/shard1/election/99269078968958985-core_node5-n_0000000000
[junit4] 2>
[junit4] 2> Maximum concurrent children watches above limit:
[junit4] 2>
[junit4] 2> 5 /solr/live_nodes
[junit4] 2> 5 /solr/collections
[junit4] 2>
[junit4] 2> NOTE: leaving temporary files on disk at: /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.autoscaling.AutoAddReplicasPlanActionTest_D85FA2E647359F41-001
[junit4] 2> NOTE: test params are: codec=Asserting(Lucene70): {}, docValues:{}, maxPointsInLeafNode=75, maxMBSortInHeap=7.867552646375433, sim=RandomSimilarity(queryNorm=true): {}, locale=da-DK, timezone=America/Detroit
[junit4] 2> NOTE: Linux 4.10.0-40-generic i386/Oracle Corporation 1.8.0_144 (32-bit)/cpus=8,threads=1,free=229087384,total=518979584
[junit4] 2> NOTE: All tests run in this JVM: [IgnoreCommitOptimizeUpdateProcessorFactoryTest, TestExactSharedStatsCache, SpellCheckComponentTest, CloudMLTQParserTest, DateMathParserTest, FastVectorHighlighterTest, ResourceLoaderTest, TestSolr4Spatial2, PropertiesRequestHandlerTest, MetricTriggerTest, DeleteInactiveReplicaTest, ReplicaListTransformerTest, TestScoreJoinQPNoScore, SaslZkACLProviderTest, RemoteQueryErrorTest, CursorPagingTest, SolrMetricReporterTest, TestCloudSchemaless, SpatialRPTFieldTypeTest, BasicFunctionalityTest, OverseerModifyCollectionTest, SolrGraphiteReporterTest, MinimalSchemaTest, TestDFISimilarityFactory, RulesTest, TestCloudInspectUtil, TestShortCircuitedRequests, TestConfig, TestManagedResource, RAMDirectoryFactoryTest, TestUpdate, TestUtils, TestMultiValuedNumericRangeQuery, TestDFRSimilarityFactory, TestHighlightDedupGrouping, ChaosMonkeyNothingIsSafeWithPullReplicasTest, AutoscalingHistoryHandlerTest, TestFreeTextSuggestions, AddBlockUpdateTest, TestOrdValues, TestWordDelimiterFilterFactory, VersionInfoTest, ZkControllerTest, TestTlogReplica, TestIndexingPerformance, AtomicUpdatesTest, TestSolrIndexConfig, SearchHandlerTest, TestDynamicFieldResource, AssignBackwardCompatibilityTest, IndexSchemaRuntimeFieldTest, SimpleFacetsTest, DataDrivenBlockJoinTest, TestSolrCloudWithDelegationTokens, HdfsNNFailoverTest, AnalyticsQueryTest, DistanceUnitsTest, TestXmlQParser, TestNumericTerms32, TestBinaryResponseWriter, TestRandomFaceting, TestIBSimilarityFactory, DistributedQueueTest, TestFastOutputStream, ExplicitHLLTest, TestTrackingShardHandlerFactory, TestStressCloudBlindAtomicUpdates, TestObjectReleaseTracker, TestSolrCloudSnapshots, LeaderFailoverAfterPartitionTest, TestLargeCluster, TestAddFieldRealTimeGet, TestCrossCoreJoin, TriggerIntegrationTest, TestSystemCollAutoCreate, TestUnifiedSolrHighlighter, SolrCoreTest, TestLegacyFieldReuse, SolrXmlInZkTest, HdfsChaosMonkeyNothingIsSafeTest, SoftAutoCommitTest, HdfsTlogReplayBufferedWhileIndexingTest, TestLuceneIndexBackCompat, DateFieldTest, TestSearcherReuse, SearchHandlerTest, DistributedFacetPivotWhiteBoxTest, TestFilteredDocIdSet, TestPointFields, TestSerializedLuceneMatchVersion, CollectionsAPISolrJTest, TestRandomDVFaceting, TestSimpleTextCodec, CacheHeaderTest, TestManagedSchema, BaseCdcrDistributedZkTest, TestUseDocValuesAsStored, StressHdfsTest, ExternalFileFieldSortTest, TestPhraseSuggestions, TestPostingsSolrHighlighter, NoCacheHeaderTest, DocumentBuilderTest, TestQueryUtils, TestOnReconnectListenerSupport, DistributedSuggestComponentTest, BasicAuthIntegrationTest, CdcrReplicationDistributedZkTest, TestRetrieveFieldsOptimizer, QueryResultKeyTest, StatsReloadRaceTest, LeaderInitiatedRecoveryOnShardRestartTest, TestReqParamsAPI, WrapperMergePolicyFactoryTest, QueryParsingTest, TestStandardQParsers, TestJsonFacetRefinement, TestSQLHandlerNonCloud, TestSchemaResource, TestEmbeddedSolrServerSchemaAPI, TestReplicationHandler, SliceStateTest, TestSolrJ, FileUtilsTest, AutoAddReplicasPlanActionTest]
[junit4] Completed [504/764 (2!)] on J0 in 19.57s, 1 test, 1 error <<< FAILURES!
[...truncated 921 lines...]
[junit4] JVM J0: stdout was not empty, see: /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/temp/junit4-J0-20171231_123019_2923084408049109781140.sysout
[junit4] >>> JVM J0 emitted unexpected output (verbatim) ----
[junit4] java.lang.OutOfMemoryError: Java heap space
[junit4] Dumping heap to /home/jenkins/workspace/Lucene-Solr-7.x-Linux/heapdumps/java_pid11282.hprof ...
[junit4] Heap dump file created [189919040 bytes in 1.080 secs]
[junit4] <<< JVM J0: EOF ----
[...truncated 8573 lines...]
BUILD FAILED
/home/jenkins/workspace/Lucene-Solr-7.x-Linux/build.xml:835: The following error occurred while executing this line:
/home/jenkins/workspace/Lucene-Solr-7.x-Linux/build.xml:787: Some of the tests produced a heap dump, but did not fail. Maybe a suppressed OutOfMemoryError? Dumps created:
* java_pid11282.hprof
Total time: 102 minutes 37 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
[WARNINGS] Skipping publisher since build result is FAILURE
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any