You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by Apache Jenkins Server <je...@builds.apache.org> on 2018/10/06 12:33:37 UTC
[JENKINS] Lucene-Solr-NightlyTests-master - Build # 1659 - Failure
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-master/1659/
3 tests failed.
FAILED: org.apache.solr.cloud.FullSolrCloudDistribCmdsTest.test
Error Message:
Java heap space
Stack Trace:
java.lang.OutOfMemoryError: Java heap space
FAILED: org.apache.solr.cloud.autoscaling.sim.TestSimGenericDistributedQueue.testDistributedQueue {#2}
Error Message:
Test abandoned because suite timeout was reached.
Stack Trace:
java.lang.Exception: Test abandoned because suite timeout was reached.
at __randomizedtesting.SeedInfo.seed([690FCFDA6B8134D2]:0)
FAILED: junit.framework.TestSuite.org.apache.solr.cloud.autoscaling.sim.TestSimGenericDistributedQueue
Error Message:
Suite timeout exceeded (>= 7200000 msec).
Stack Trace:
java.lang.Exception: Suite timeout exceeded (>= 7200000 msec).
at __randomizedtesting.SeedInfo.seed([690FCFDA6B8134D2]:0)
Build Log:
[...truncated 14339 lines...]
[junit4] Suite: org.apache.solr.cloud.FullSolrCloudDistribCmdsTest
[junit4] 2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.FullSolrCloudDistribCmdsTest_690FCFDA6B8134D2-001/init-core-data-001
[junit4] 2> 1911582 WARN (SUITE-FullSolrCloudDistribCmdsTest-seed#[690FCFDA6B8134D2]-worker) [ ] o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=42 numCloses=42
[junit4] 2> 1911582 INFO (SUITE-FullSolrCloudDistribCmdsTest-seed#[690FCFDA6B8134D2]-worker) [ ] o.a.s.SolrTestCaseJ4 Using TrieFields (NUMERIC_POINTS_SYSPROP=false) w/NUMERIC_DOCVALUES_SYSPROP=true
[junit4] 2> 1911584 INFO (SUITE-FullSolrCloudDistribCmdsTest-seed#[690FCFDA6B8134D2]-worker) [ ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false) via: @org.apache.solr.SolrTestCaseJ4$SuppressSSL(bugUrl=https://issues.apache.org/jira/browse/SOLR-5776)
[junit4] 2> 1911584 INFO (SUITE-FullSolrCloudDistribCmdsTest-seed#[690FCFDA6B8134D2]-worker) [ ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
[junit4] 2> 1911584 INFO (SUITE-FullSolrCloudDistribCmdsTest-seed#[690FCFDA6B8134D2]-worker) [ ] o.a.s.BaseDistributedSearchTestCase Setting hostContext system property: /
[junit4] 2> 1911589 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[690FCFDA6B8134D2]) [ ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
[junit4] 2> 1911589 INFO (Thread-55391) [ ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 1911589 INFO (Thread-55391) [ ] o.a.s.c.ZkTestServer Starting server
[junit4] 2> 1911591 ERROR (Thread-55391) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 1911689 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[690FCFDA6B8134D2]) [ ] o.a.s.c.ZkTestServer start zk server on port:42889
[junit4] 2> 1911697 INFO (zkConnectionManagerCallback-4819-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1911701 INFO (zkConnectionManagerCallback-4821-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1911705 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[690FCFDA6B8134D2]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
[junit4] 2> 1911707 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[690FCFDA6B8134D2]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/core/src/test-files/solr/collection1/conf/schema15.xml to /configs/conf1/schema.xml
[junit4] 2> 1911708 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[690FCFDA6B8134D2]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
[junit4] 2> 1911709 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[690FCFDA6B8134D2]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
[junit4] 2> 1911710 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[690FCFDA6B8134D2]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
[junit4] 2> 1911711 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[690FCFDA6B8134D2]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
[junit4] 2> 1911712 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[690FCFDA6B8134D2]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
[junit4] 2> 1911713 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[690FCFDA6B8134D2]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
[junit4] 2> 1911714 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[690FCFDA6B8134D2]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4] 2> 1911715 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[690FCFDA6B8134D2]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
[junit4] 2> 1911716 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[690FCFDA6B8134D2]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
[junit4] 2> 1911718 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[690FCFDA6B8134D2]) [ ] o.a.s.c.AbstractFullDistribZkTestBase Will use NRT replicas unless explicitly asked otherwise
[junit4] 2> 1911831 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[690FCFDA6B8134D2]) [ ] o.e.j.s.Server jetty-9.4.11.v20180605; built: 2018-06-05T18:24:03.829Z; git: d5fc0523cfa96bfebfbda19606cad384d772f04c; jvm 1.8.0_172-b11
[junit4] 2> 1911832 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[690FCFDA6B8134D2]) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 1911832 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[690FCFDA6B8134D2]) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 1911832 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[690FCFDA6B8134D2]) [ ] o.e.j.s.session node0 Scavenging every 660000ms
[junit4] 2> 1911832 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[690FCFDA6B8134D2]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@7858dc0d{/,null,AVAILABLE}
[junit4] 2> 1911833 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[690FCFDA6B8134D2]) [ ] o.e.j.s.AbstractConnector Started ServerConnector@10e2e8ff{HTTP/1.1,[http/1.1]}{127.0.0.1:37722}
[junit4] 2> 1911833 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[690FCFDA6B8134D2]) [ ] o.e.j.s.Server Started @1911872ms
[junit4] 2> 1911833 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[690FCFDA6B8134D2]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.FullSolrCloudDistribCmdsTest_690FCFDA6B8134D2-001/tempDir-001/control/data, replicaType=NRT, hostContext=/, hostPort=37722, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.FullSolrCloudDistribCmdsTest_690FCFDA6B8134D2-001/control-001/cores}
[junit4] 2> 1911833 ERROR (TEST-FullSolrCloudDistribCmdsTest.test-seed#[690FCFDA6B8134D2]) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 1911833 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[690FCFDA6B8134D2]) [ ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
[junit4] 2> 1911833 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[690FCFDA6B8134D2]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 8.0.0
[junit4] 2> 1911833 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[690FCFDA6B8134D2]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 1911833 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[690FCFDA6B8134D2]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 1911834 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[690FCFDA6B8134D2]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-10-06T09:56:18.739Z
[junit4] 2> 1911835 INFO (zkConnectionManagerCallback-4823-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1911836 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[690FCFDA6B8134D2]) [ ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
[junit4] 2> 1911836 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[690FCFDA6B8134D2]) [ ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.FullSolrCloudDistribCmdsTest_690FCFDA6B8134D2-001/control-001/solr.xml
[junit4] 2> 1911839 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[690FCFDA6B8134D2]) [ ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverWorkLoopDelay is ignored
[junit4] 2> 1911839 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[690FCFDA6B8134D2]) [ ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverBadNodeExpiration is ignored
[junit4] 2> 1911841 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[690FCFDA6B8134D2]) [ ] o.a.s.c.SolrXmlConfig MBean server found: com.sun.jmx.mbeanserver.JmxMBeanServer@67453e75, but no JMX reporters were configured - adding default JMX reporter.
[junit4] 2> 1911973 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[690FCFDA6B8134D2]) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:42889/solr
[junit4] 2> 1911974 INFO (zkConnectionManagerCallback-4827-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1911976 INFO (zkConnectionManagerCallback-4829-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1912061 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[690FCFDA6B8134D2]) [n:127.0.0.1:37722_ ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:37722_
[junit4] 2> 1912062 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[690FCFDA6B8134D2]) [n:127.0.0.1:37722_ ] o.a.s.c.Overseer Overseer (id=72457920339378180-127.0.0.1:37722_-n_0000000000) starting
[junit4] 2> 1912069 INFO (zkConnectionManagerCallback-4836-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1912070 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[690FCFDA6B8134D2]) [n:127.0.0.1:37722_ ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:42889/solr ready
[junit4] 2> 1912071 INFO (OverseerStateUpdate-72457920339378180-127.0.0.1:37722_-n_0000000000) [n:127.0.0.1:37722_ ] o.a.s.c.Overseer Starting to work on the main queue : 127.0.0.1:37722_
[junit4] 2> 1912072 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[690FCFDA6B8134D2]) [n:127.0.0.1:37722_ ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:37722_
[junit4] 2> 1912076 INFO (OverseerStateUpdate-72457920339378180-127.0.0.1:37722_-n_0000000000) [n:127.0.0.1:37722_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 1912085 INFO (zkCallback-4835-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 1912093 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[690FCFDA6B8134D2]) [n:127.0.0.1:37722_ ] o.a.s.h.a.MetricsHistoryHandler No .system collection, keeping metrics history in memory.
[junit4] 2> 1912108 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[690FCFDA6B8134D2]) [n:127.0.0.1:37722_ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@67453e75
[junit4] 2> 1912117 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[690FCFDA6B8134D2]) [n:127.0.0.1:37722_ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@67453e75
[junit4] 2> 1912117 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[690FCFDA6B8134D2]) [n:127.0.0.1:37722_ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@67453e75
[junit4] 2> 1912118 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[690FCFDA6B8134D2]) [n:127.0.0.1:37722_ ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.FullSolrCloudDistribCmdsTest_690FCFDA6B8134D2-001/control-001/cores
[junit4] 2> 1912138 INFO (zkConnectionManagerCallback-4842-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1912138 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[690FCFDA6B8134D2]) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 1912139 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[690FCFDA6B8134D2]) [ ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:42889/solr ready
[junit4] 2> 1912140 INFO (qtp1443383752-70377) [n:127.0.0.1:37722_ ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params collection.configName=conf1&name=control_collection&nrtReplicas=1&action=CREATE&numShards=1&createNodeSet=127.0.0.1:37722_&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 1912142 INFO (OverseerThreadFactory-7558-thread-1-processing-n:127.0.0.1:37722_) [n:127.0.0.1:37722_ ] o.a.s.c.a.c.CreateCollectionCmd Create collection control_collection
[junit4] 2> 1912145 DEBUG (OverseerStateUpdate-72457920339378180-127.0.0.1:37722_-n_0000000000) [n:127.0.0.1:37722_ ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
[junit4] 2> "name":"control_collection",
[junit4] 2> "fromApi":"true",
[junit4] 2> "collection.configName":"conf1",
[junit4] 2> "numShards":"1",
[junit4] 2> "createNodeSet":"127.0.0.1:37722_",
[junit4] 2> "nrtReplicas":"1",
[junit4] 2> "stateFormat":"2",
[junit4] 2> "replicationFactor":"1",
[junit4] 2> "operation":"create"} current state version: 0
[junit4] 2> 1912249 INFO (qtp1443383752-70372) [n:127.0.0.1:37722_ x:control_collection_shard1_replica_n1] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf1&newCollection=true&name=control_collection_shard1_replica_n1&action=CREATE&numShards=1&collection=control_collection&shard=shard1&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 1912250 INFO (qtp1443383752-70372) [n:127.0.0.1:37722_ x:control_collection_shard1_replica_n1] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 4 transient cores
[junit4] 2> 1912253 DEBUG (OverseerStateUpdate-72457920339378180-127.0.0.1:37722_-n_0000000000) [n:127.0.0.1:37722_ ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
[junit4] 2> "core":"control_collection_shard1_replica_n1",
[junit4] 2> "roles":null,
[junit4] 2> "base_url":"http://127.0.0.1:37722",
[junit4] 2> "node_name":"127.0.0.1:37722_",
[junit4] 2> "numShards":"1",
[junit4] 2> "state":"down",
[junit4] 2> "shard":"shard1",
[junit4] 2> "collection":"control_collection",
[junit4] 2> "type":"NRT",
[junit4] 2> "operation":"state"} current state version: 0
[junit4] 2> 1913274 INFO (qtp1443383752-70372) [n:127.0.0.1:37722_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
[junit4] 2> 1913286 INFO (qtp1443383752-70372) [n:127.0.0.1:37722_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.s.IndexSchema [control_collection_shard1_replica_n1] Schema name=test
[junit4] 2> 1913355 INFO (qtp1443383752-70372) [n:127.0.0.1:37722_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.s.IndexSchema Loaded schema test/1.6 with uniqueid field id
[junit4] 2> 1913371 INFO (qtp1443383752-70372) [n:127.0.0.1:37722_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.CoreContainer Creating SolrCore 'control_collection_shard1_replica_n1' using configuration from collection control_collection, trusted=true
[junit4] 2> 1913371 INFO (qtp1443383752-70372) [n:127.0.0.1:37722_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.control_collection.shard1.replica_n1' (registry 'solr.core.control_collection.shard1.replica_n1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@67453e75
[junit4] 2> 1913372 INFO (qtp1443383752-70372) [n:127.0.0.1:37722_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.SolrCore [[control_collection_shard1_replica_n1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.FullSolrCloudDistribCmdsTest_690FCFDA6B8134D2-001/control-001/cores/control_collection_shard1_replica_n1], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.FullSolrCloudDistribCmdsTest_690FCFDA6B8134D2-001/control-001/cores/control_collection_shard1_replica_n1/data/]
[junit4] 2> 1913374 INFO (qtp1443383752-70372) [n:127.0.0.1:37722_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=27, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=0.541015625, noCFSRatio=0.0]
[junit4] 2> 1913375 WARN (qtp1443383752-70372) [n:127.0.0.1:37722_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,attributes = {initParams=a, name=/dump, class=DumpRequestHandler},args = {defaults={a=A,b=B}}}
[junit4] 2> 1913413 INFO (qtp1443383752-70372) [n:127.0.0.1:37722_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 1913413 INFO (qtp1443383752-70372) [n:127.0.0.1:37722_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 1913414 INFO (qtp1443383752-70372) [n:127.0.0.1:37722_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 1913414 INFO (qtp1443383752-70372) [n:127.0.0.1:37722_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 1913414 INFO (qtp1443383752-70372) [n:127.0.0.1:37722_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=31, maxMergeAtOnceExplicit=47, maxMergedSegmentMB=0.7861328125, floorSegmentMB=1.216796875, forceMergeDeletesPctAllowed=0.6729811829427002, segmentsPerTier=30.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.47373358407785326, deletesPctAllowed=45.33519292309924
[junit4] 2> 1913415 INFO (qtp1443383752-70372) [n:127.0.0.1:37722_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.s.SolrIndexSearcher Opening [Searcher@6d8f0158[control_collection_shard1_replica_n1] main]
[junit4] 2> 1913416 INFO (qtp1443383752-70372) [n:127.0.0.1:37722_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 1913416 INFO (qtp1443383752-70372) [n:127.0.0.1:37722_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 1913416 INFO (qtp1443383752-70372) [n:127.0.0.1:37722_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
[junit4] 2> 1913417 INFO (searcherExecutor-7563-thread-1-processing-n:127.0.0.1:37722_ x:control_collection_shard1_replica_n1 c:control_collection s:shard1) [n:127.0.0.1:37722_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.SolrCore [control_collection_shard1_replica_n1] Registered new searcher Searcher@6d8f0158[control_collection_shard1_replica_n1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 1913418 INFO (qtp1443383752-70372) [n:127.0.0.1:37722_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1613569489973018624
[junit4] 2> 1913423 INFO (qtp1443383752-70372) [n:127.0.0.1:37722_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.ZkShardTerms Successful update of terms at /collections/control_collection/terms/shard1 to Terms{values={core_node2=0}, version=0}
[junit4] 2> 1913426 INFO (qtp1443383752-70372) [n:127.0.0.1:37722_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
[junit4] 2> 1913426 INFO (qtp1443383752-70372) [n:127.0.0.1:37722_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
[junit4] 2> 1913426 INFO (qtp1443383752-70372) [n:127.0.0.1:37722_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:37722/control_collection_shard1_replica_n1/
[junit4] 2> 1913426 INFO (qtp1443383752-70372) [n:127.0.0.1:37722_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
[junit4] 2> 1913426 INFO (qtp1443383752-70372) [n:127.0.0.1:37722_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.SyncStrategy http://127.0.0.1:37722/control_collection_shard1_replica_n1/ has no replicas
[junit4] 2> 1913427 INFO (qtp1443383752-70372) [n:127.0.0.1:37722_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
[junit4] 2> 1913430 INFO (qtp1443383752-70372) [n:127.0.0.1:37722_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:37722/control_collection_shard1_replica_n1/ shard1
[junit4] 2> 1913430 DEBUG (OverseerStateUpdate-72457920339378180-127.0.0.1:37722_-n_0000000000) [n:127.0.0.1:37722_ ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
[junit4] 2> "operation":"leader",
[junit4] 2> "shard":"shard1",
[junit4] 2> "collection":"control_collection",
[junit4] 2> "base_url":"http://127.0.0.1:37722",
[junit4] 2> "core":"control_collection_shard1_replica_n1",
[junit4] 2> "state":"active"} current state version: 0
[junit4] 2> 1913532 INFO (qtp1443383752-70372) [n:127.0.0.1:37722_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.ZkController I am the leader, no recovery necessary
[junit4] 2> 1913534 DEBUG (OverseerStateUpdate-72457920339378180-127.0.0.1:37722_-n_0000000000) [n:127.0.0.1:37722_ ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
[junit4] 2> "core":"control_collection_shard1_replica_n1",
[junit4] 2> "core_node_name":"core_node2",
[junit4] 2> "roles":null,
[junit4] 2> "base_url":"http://127.0.0.1:37722",
[junit4] 2> "node_name":"127.0.0.1:37722_",
[junit4] 2> "numShards":"1",
[junit4] 2> "state":"active",
[junit4] 2> "shard":"shard1",
[junit4] 2> "collection":"control_collection",
[junit4] 2> "type":"NRT",
[junit4] 2> "operation":"state"} current state version: 0
[junit4] 2> 1913543 INFO (qtp1443383752-70372) [n:127.0.0.1:37722_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=conf1&newCollection=true&name=control_collection_shard1_replica_n1&action=CREATE&numShards=1&collection=control_collection&shard=shard1&wt=javabin&version=2&replicaType=NRT} status=0 QTime=1294
[junit4] 2> 1913547 INFO (qtp1443383752-70377) [n:127.0.0.1:37722_ ] o.a.s.h.a.CollectionsHandler Wait for new collection to be active for at most 30 seconds. Check all shard replicas
[junit4] 2> 1913635 INFO (zkCallback-4828-thread-1) [ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/control_collection/state.json] for collection [control_collection] has occurred - updating... (live nodes size: [1])
[junit4] 2> 1914147 INFO (OverseerCollectionConfigSetProcessor-72457920339378180-127.0.0.1:37722_-n_0000000000) [n:127.0.0.1:37722_ ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000000 doesn't exist. Requestor may have disconnected from ZooKeeper
[junit4] 2> 1914547 INFO (qtp1443383752-70377) [n:127.0.0.1:37722_ ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={collection.configName=conf1&name=control_collection&nrtReplicas=1&action=CREATE&numShards=1&createNodeSet=127.0.0.1:37722_&wt=javabin&version=2} status=0 QTime=2407
[junit4] 2> 1914552 INFO (zkConnectionManagerCallback-4847-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1914556 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[690FCFDA6B8134D2]) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 1914557 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[690FCFDA6B8134D2]) [ ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:42889/solr ready
[junit4] 2> 1914557 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[690FCFDA6B8134D2]) [ ] o.a.s.c.ChaosMonkey monkey: init - expire sessions:false cause connection loss:false
[junit4] 2> 1914558 INFO (qtp1443383752-70379) [n:127.0.0.1:37722_ ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params collection.configName=conf1&name=collection1&nrtReplicas=1&action=CREATE&numShards=3&createNodeSet=&stateFormat=1&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 1914561 INFO (OverseerThreadFactory-7558-thread-2-processing-n:127.0.0.1:37722_) [n:127.0.0.1:37722_ ] o.a.s.c.a.c.CreateCollectionCmd Create collection collection1
[junit4] 2> 1914565 DEBUG (OverseerStateUpdate-72457920339378180-127.0.0.1:37722_-n_0000000000) [n:127.0.0.1:37722_ ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
[junit4] 2> "name":"collection1",
[junit4] 2> "fromApi":"true",
[junit4] 2> "collection.configName":"conf1",
[junit4] 2> "numShards":"3",
[junit4] 2> "createNodeSet":"",
[junit4] 2> "stateFormat":"1",
[junit4] 2> "nrtReplicas":"1",
[junit4] 2> "replicationFactor":"1",
[junit4] 2> "operation":"create"} current state version: 0
[junit4] 2> 1914771 WARN (OverseerThreadFactory-7558-thread-2-processing-n:127.0.0.1:37722_) [n:127.0.0.1:37722_ ] o.a.s.c.a.c.CreateCollectionCmd It is unusual to create a collection (collection1) without cores.
[junit4] 2> 1914773 INFO (qtp1443383752-70379) [n:127.0.0.1:37722_ ] o.a.s.h.a.CollectionsHandler Wait for new collection to be active for at most 30 seconds. Check all shard replicas
[junit4] 2> 1914774 INFO (qtp1443383752-70379) [n:127.0.0.1:37722_ ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={collection.configName=conf1&name=collection1&nrtReplicas=1&action=CREATE&numShards=3&createNodeSet=&stateFormat=1&wt=javabin&version=2} status=0 QTime=215
[junit4] 2> 1914884 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[690FCFDA6B8134D2]) [ ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 1 in directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.FullSolrCloudDistribCmdsTest_690FCFDA6B8134D2-001/shard-1-001 of type NRT
[junit4] 2> 1914885 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[690FCFDA6B8134D2]) [ ] o.e.j.s.Server jetty-9.4.11.v20180605; built: 2018-06-05T18:24:03.829Z; git: d5fc0523cfa96bfebfbda19606cad384d772f04c; jvm 1.8.0_172-b11
[junit4] 2> 1914885 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[690FCFDA6B8134D2]) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 1914886 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[690FCFDA6B8134D2]) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 1914886 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[690FCFDA6B8134D2]) [ ] o.e.j.s.session node0 Scavenging every 660000ms
[junit4] 2> 1914886 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[690FCFDA6B8134D2]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@22b7039a{/,null,AVAILABLE}
[junit4] 2> 1914892 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[690FCFDA6B8134D2]) [ ] o.e.j.s.AbstractConnector Started ServerConnector@18b2b7b7{HTTP/1.1,[http/1.1]}{127.0.0.1:41583}
[junit4] 2> 1914892 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[690FCFDA6B8134D2]) [ ] o.e.j.s.Server Started @1914931ms
[junit4] 2> 1914892 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[690FCFDA6B8134D2]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.FullSolrCloudDistribCmdsTest_690FCFDA6B8134D2-001/tempDir-001/jetty1, replicaType=NRT, solrconfig=solrconfig.xml, hostContext=/, hostPort=41583, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.FullSolrCloudDistribCmdsTest_690FCFDA6B8134D2-001/shard-1-001/cores}
[junit4] 2> 1914892 ERROR (TEST-FullSolrCloudDistribCmdsTest.test-seed#[690FCFDA6B8134D2]) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 1914892 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[690FCFDA6B8134D2]) [ ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
[junit4] 2> 1914892 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[690FCFDA6B8134D2]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 8.0.0
[junit4] 2> 1914892 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[690FCFDA6B8134D2]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 1914892 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[690FCFDA6B8134D2]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 1914892 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[690FCFDA6B8134D2]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-10-06T09:56:21.798Z
[junit4] 2> 1914904 INFO (zkConnectionManagerCallback-4849-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1914905 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[690FCFDA6B8134D2]) [ ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
[junit4] 2> 1914905 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[690FCFDA6B8134D2]) [ ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.FullSolrCloudDistribCmdsTest_690FCFDA6B8134D2-001/shard-1-001/solr.xml
[junit4] 2> 1914908 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[690FCFDA6B8134D2]) [ ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverWorkLoopDelay is ignored
[junit4] 2> 1914908 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[690FCFDA6B8134D2]) [ ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverBadNodeExpiration is ignored
[junit4] 2> 1914910 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[690FCFDA6B8134D2]) [ ] o.a.s.c.SolrXmlConfig MBean server found: com.sun.jmx.mbeanserver.JmxMBeanServer@67453e75, but no JMX reporters were configured - adding default JMX reporter.
[junit4] 2> 1915151 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[690FCFDA6B8134D2]) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:42889/solr
[junit4] 2> 1915152 INFO (zkConnectionManagerCallback-4853-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1915155 INFO (zkConnectionManagerCallback-4855-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1915162 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[690FCFDA6B8134D2]) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 1915165 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[690FCFDA6B8134D2]) [ ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 4 transient cores
[junit4] 2> 1915165 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[690FCFDA6B8134D2]) [ ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:41583_
[junit4] 2> 1915166 DEBUG (OverseerStateUpdate-72457920339378180-127.0.0.1:37722_-n_0000000000) [n:127.0.0.1:37722_ ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
[junit4] 2> "operation":"downnode",
[junit4] 2> "node_name":"127.0.0.1:41583_"} current state version: 1
[junit4] 2> 1915166 INFO (zkCallback-4835-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 1915166 INFO (zkCallback-4846-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 1915166 INFO (zkCallback-4828-thread-2) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 1915168 INFO (zkCallback-4854-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 1915186 INFO (zkConnectionManagerCallback-4862-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1915187 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[690FCFDA6B8134D2]) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
[junit4] 2> 1915188 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[690FCFDA6B8134D2]) [ ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:42889/solr ready
[junit4] 2> 1915188 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[690FCFDA6B8134D2]) [ ] o.a.s.h.a.MetricsHistoryHandler No .system collection, keeping metrics history in memory.
[junit4] 2> 1915206 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[690FCFDA6B8134D2]) [ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@67453e75
[junit4] 2> 1915216 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[690FCFDA6B8134D2]) [ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@67453e75
[junit4] 2> 1915216 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[690FCFDA6B8134D2]) [ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@67453e75
[junit4] 2> 1915217 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[690FCFDA6B8134D2]) [ ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.FullSolrCloudDistribCmdsTest_690FCFDA6B8134D2-001/shard-1-001/cores
[junit4] 2> 1915245 INFO (qtp1443383752-70373) [n:127.0.0.1:37722_ ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :addreplica with params node=127.0.0.1:41583_&action=ADDREPLICA&collection=collection1&shard=shard2&type=NRT&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 1915248 INFO (OverseerCollectionConfigSetProcessor-72457920339378180-127.0.0.1:37722_-n_0000000000) [n:127.0.0.1:37722_ ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000002 doesn't exist. Requestor may have disconnected from ZooKeeper
[junit4] 2> 1915249 INFO (OverseerThreadFactory-7558-thread-3-processing-n:127.0.0.1:37722_) [n:127.0.0.1:37722_ c:collection1 s:shard2 ] o.a.s.c.a.c.AddReplicaCmd Node Identified 127.0.0.1:41583_ for creating new replica of shard shard2 for collection collection1
[junit4] 2> 1915251 INFO (qtp2023914897-70432) [n:127.0.0.1:41583_ x:collection1_shard2_replica_n61] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf1&name=collection1_shard2_replica_n61&action=CREATE&collection=collection1&shard=shard2&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 1915253 DEBUG (OverseerStateUpdate-72457920339378180-127.0.0.1:37722_-n_0000000000) [n:127.0.0.1:37722_ ] o.a.s.c.Overseer processMessage: queueSize: 2, message = {
[junit4] 2> "core":"collection1_shard2_replica_n61",
[junit4] 2> "roles":null,
[junit4] 2> "base_url":"http://127.0.0.1:41583",
[junit4] 2> "node_name":"127.0.0.1:41583_",
[junit4] 2> "numShards":"3",
[junit4] 2> "state":"down",
[junit4] 2> "shard":"shard2",
[junit4] 2> "collection":"collection1",
[junit4] 2> "type":"NRT",
[junit4] 2> "operation":"state"} current state version: 1
[junit4] 2> 1916266 INFO (qtp2023914897-70432) [n:127.0.0.1:41583_ c:collection1 s:shard2 x:collection1_shard2_replica_n61] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
[junit4] 2> 1916314 INFO (qtp2023914897-70432) [n:127.0.0.1:41583_ c:collection1 s:shard2 x:collection1_shard2_replica_n61] o.a.s.s.IndexSchema [collection1_shard2_replica_n61] Schema name=test
[junit4] 2> 1916419 INFO (qtp2023914897-70432) [n:127.0.0.1:41583_ c:collection1 s:shard2 x:collection1_shard2_replica_n61] o.a.s.s.IndexSchema Loaded schema test/1.6 with uniqueid field id
[junit4] 2> 1916480 INFO (qtp2023914897-70432) [n:127.0.0.1:41583_ c:collection1 s:shard2 x:collection1_shard2_replica_n61] o.a.s.c.CoreContainer Creating SolrCore 'collection1_shard2_replica_n61' using configuration from collection collection1, trusted=true
[junit4] 2> 1916481 INFO (qtp2023914897-70432) [n:127.0.0.1:41583_ c:collection1 s:shard2 x:collection1_shard2_replica_n61] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.collection1.shard2.replica_n61' (registry 'solr.core.collection1.shard2.replica_n61') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@67453e75
[junit4] 2> 1916481 INFO (qtp2023914897-70432) [n:127.0.0.1:41583_ c:collection1 s:shard2 x:collection1_shard2_replica_n61] o.a.s.c.SolrCore [[collection1_shard2_replica_n61] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.FullSolrCloudDistribCmdsTest_690FCFDA6B8134D2-001/shard-1-001/cores/collection1_shard2_replica_n61], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.FullSolrCloudDistribCmdsTest_690FCFDA6B8134D2-001/shard-1-001/cores/collection1_shard2_replica_n61/data/]
[junit4] 2> 1916483 INFO (qtp2023914897-70432) [n:127.0.0.1:41583_ c:collection1 s:shard2 x:collection1_shard2_replica_n61] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=27, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=0.541015625, noCFSRatio=0.0]
[junit4] 2> 1916497 WARN (qtp2023914897-70432) [n:127.0.0.1:41583_ c:collection1 s:shard2 x:collection1_shard2_replica_n61] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,attributes = {initParams=a, name=/dump, class=DumpRequestHandler},args = {defaults={a=A,b=B}}}
[junit4] 2> 1916553 INFO (qtp2023914897-70432) [n:127.0.0.1:41583_ c:collection1 s:shard2 x:collection1_shard2_replica_n61] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 1916554 INFO (qtp2023914897-70432) [n:127.0.0.1:41583_ c:collection1 s:shard2 x:collection1_shard2_replica_n61] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 1916555 INFO (qtp2023914897-70432) [n:127.0.0.1:41583_ c:collection1 s:shard2 x:collection1_shard2_replica_n61] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 1916555 INFO (qtp2023914897-70432) [n:127.0.0.1:41583_ c:collection1 s:shard2 x:collection1_shard2_replica_n61] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 1916555 INFO (qtp2023914897-70432) [n:127.0.0.1:41583_ c:collection1 s:shard2 x:collection1_shard2_replica_n61] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=31, maxMergeAtOnceExplicit=47, maxMergedSegmentMB=0.7861328125, floorSegmentMB=1.216796875, forceMergeDeletesPctAllowed=0.6729811829427002, segmentsPerTier=30.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.47373358407785326, deletesPctAllowed=45.33519292309924
[junit4] 2> 1916556 INFO (qtp2023914897-70432) [n:127.0.0.1:41583_ c:collection1 s:shard2 x:collection1_shard2_replica_n61] o.a.s.s.SolrIndexSearcher Opening [Searcher@74dde355[collection1_shard2_replica_n61] main]
[junit4] 2> 1916557 INFO (qtp2023914897-70432) [n:127.0.0.1:41583_ c:collection1 s:shard2 x:collection1_shard2_replica_n61] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 1916557 INFO (qtp2023914897-70432) [n:127.0.0.1:41583_ c:collection1 s:shard2 x:collection1_shard2_replica_n61] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 1916558 INFO (qtp2023914897-70432) [n:127.0.0.1:41583_ c:collection1 s:shard2 x:collection1_shard2_replica_n61] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
[junit4] 2> 1916558 INFO (qtp2023914897-70432) [n:127.0.0.1:41583_ c:collection1 s:shard2 x:collection1_shard2_replica_n61] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1613569493265547264
[junit4] 2> 1916560 INFO (searcherExecutor-7577-thread-1-processing-n:127.0.0.1:41583_ x:collection1_shard2_replica_n61 c:collection1 s:shard2) [n:127.0.0.1:41583_ c:collection1 s:shard2 x:collection1_shard2_replica_n61] o.a.s.c.SolrCore [collection1_shard2_replica_n61] Registered new searcher Searcher@74dde355[collection1_shard2_replica_n61] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 1916575 INFO (qtp2023914897-70432) [n:127.0.0.1:41583_ c:collection1 s:shard2 x:collection1_shard2_replica_n61] o.a.s.c.ZkShardTerms Successful update of terms at /collections/collection1/terms/shard2 to Terms{values={core_node62=0}, version=0}
[junit4] 2> 1916578 INFO (qtp2023914897-70432) [n:127.0.0.1:41583_ c:collection1 s:shard2 x:collection1_shard2_replica_n61] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
[junit4] 2> 1916578 INFO (qtp2023914897-70432) [n:127.0.0.1:41583_ c:collection1 s:shard2 x:collection1_shard2_replica_n61] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
[junit4] 2> 1916578 INFO (qtp2023914897-70432) [n:127.0.0.1:41583_ c:collection1 s:shard2 x:collection1_shard2_replica_n61] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:41583/collection1_shard2_replica_n61/
[junit4] 2> 1916578 INFO (qtp2023914897-70432) [n:127.0.0.1:41583_ c:collection1 s:shard2 x:collection1_shard2_replica_n61] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
[junit4] 2> 1916578 INFO (qtp2023914897-70432) [n:127.0.0.1:41583_ c:collection1 s:shard2 x:collection1_shard2_replica_n61] o.a.s.c.SyncStrategy http://127.0.0.1:41583/collection1_shard2_replica_n61/ has no replicas
[junit4] 2> 1916578 INFO (qtp2023914897-70432) [n:127.0.0.1:41583_ c:collection1 s:shard2 x:collection1_shard2_replica_n61] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
[junit4] 2> 1916582 INFO (qtp2023914897-70432) [n:127.0.0.1:41583_ c:collection1 s:shard2 x:collection1_shard2_replica_n61] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:41583/collection1_shard2_replica_n61/ shard2
[junit4] 2> 1916584 DEBUG (OverseerStateUpdate-72457920339378180-127.0.0.1:37722_-n_0000000000) [n:127.0.0.1:37722_ ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
[junit4] 2> "operation":"leader",
[junit4] 2> "shard":"shard2",
[junit4] 2> "collection":"collection1",
[junit4] 2> "base_url":"http://127.0.0.1:41583",
[junit4] 2> "core":"collection1_shard2_replica_n61",
[junit4] 2> "state":"active"} current state version: 2
[junit4] 2> 1916733 INFO (qtp2023914897-70432) [n:127.0.0.1:41583_ c:collection1 s:shard2 x:collection1_shard2_replica_n61] o.a.s.c.ZkController I am the leader, no recovery necessary
[junit4] 2> 1916734 DEBUG (OverseerStateUpdate-72457920339378180-127.0.0.1:37722_-n_0000000000) [n:127.0.0.1:37722_ ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
[junit4] 2> "core":"collection1_shard2_replica_n61",
[junit4] 2> "core_node_name":"core_node62",
[junit4] 2> "roles":null,
[junit4] 2> "base_url":"http://127.0.0.1:41583",
[junit4] 2> "node_name":"127.0.0.1:41583_",
[junit4] 2> "numShards":"3",
[junit4] 2> "state":"active",
[junit4] 2> "shard":"shard2",
[junit4] 2> "collection":"collection1",
[junit4] 2> "type":"NRT",
[junit4] 2> "operation":"state"} current state version: 3
[junit4] 2> 1916748 INFO (qtp2023914897-70432) [n:127.0.0.1:41583_ c:collection1 s:shard2 x:collection1_shard2_replica_n61] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=conf1&name=collection1_shard2_replica_n61&action=CREATE&collection=collection1&shard=shard2&wt=javabin&version=2&replicaType=NRT} status=0 QTime=1497
[junit4] 2> 1916750 INFO (qtp1443383752-70373) [n:127.0.0.1:37722_ c:collection1 ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={node=127.0.0.1:41583_&action=ADDREPLICA&collection=collection1&shard=shard2&type=NRT&wt=javabin&version=2} status=0 QTime=1504
[junit4] 2> 1916886 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[690FCFDA6B8134D2]) [ ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 2 in directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.FullSolrCloudDistribCmdsTest_690FCFDA6B8134D2-001/shard-2-001 of type NRT
[junit4] 2> 1916887 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[690FCFDA6B8134D2]) [ ] o.e.j.s.Server jetty-9.4.11.v20180605; built: 2018-06-05T18:24:03.829Z; git: d5fc0523cfa96bfebfbda19606cad384d772f04c; jvm 1.8.0_172-b11
[junit4] 2> 1916888 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[690FCFDA6B8134D2]) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 1916888 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[690FCFDA6B8134D2]) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 1916888 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[690FCFDA6B8134D2]) [ ] o.e.j.s.session node0 Scavenging every 660000ms
[junit4] 2> 1916888 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[690FCFDA6B8134D2]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@7c456d2e{/,null,AVAILABLE}
[junit4] 2> 1916889 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[690FCFDA6B8134D2]) [ ] o.e.j.s.AbstractConnector Started ServerConnector@3e21c36{HTTP/1.1,[http/1.1]}{127.0.0.1:46212}
[junit4] 2> 1916889 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[690FCFDA6B8134D2]) [ ] o.e.j.s.Server Started @1916928ms
[junit4] 2> 1916889 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[690FCFDA6B8134D2]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.FullSolrCloudDistribCmdsTest_690FCFDA6B8134D2-001/tempDir-001/jetty2, replicaType=NRT, solrconfig=solrconfig.xml, hostContext=/, hostPort=46212, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J1/../../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.FullSolrCloudDistribCmdsTest_690FCFDA6B8134D2-001/shard-2-001/cores}
[junit4] 2> 1916889 ERROR (TEST-FullSolrCloudDistribCmdsTest.test-seed#[690FCFDA6B8134D2]) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 1916889 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[690FCFDA6B8134D2]) [ ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
[junit4] 2> 1916889 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[690FCFDA6B8134D2]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 8.0.0
[junit4] 2> 1916889 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[690FCFDA6B8134D2]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 1916889 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[690FCFDA6B8134D2]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 1916889 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[690FCFDA6B8134D2]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-10-06T09:56:23.795Z
[junit4] 2> 1916896 INFO (zkConnectionManagerCallback-4865-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1916897 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[690FCFDA6B8134D2]) [ ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
[junit4] 2> 1916897 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[690FCFDA6B8134D2]) [ ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.FullSolrCloudDistribCmdsTest_690FCFDA6B8134D2-001/shard-2-001/solr.xml
[junit4] 2> 1916901 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[690FCFDA6B8134D2]) [ ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverWorkLoopDelay is ignored
[junit4] 2> 1916901 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[690FCFDA6B8134D2]) [ ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverBadNodeExpiration is ignored
[junit4] 2> 1916902 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[690FCFDA6B8134D2]) [ ] o.a.s.c.SolrXmlConfig MBean server found: com.sun.jmx.mbeanserver.JmxMBeanServer@67453e75, but no JMX reporters were configured - adding default JMX reporter.
[junit4] 2> 1917065 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[690FCFDA6B8134D2]) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:42889/solr
[junit4] 2> 1917067 INFO (zkConnectionManagerCallback-4869-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1917069 INFO (zkConnectionManagerCallback-4871-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1917074 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[690FCFDA6B8134D2]) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
[junit4] 2> 1917078 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[690FCFDA6B8134D2]) [ ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 4 transient cores
[junit4] 2> 1917078 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[690FCFDA6B8134D2]) [ ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:46212_
[junit4] 2> 1917079 DEBUG (OverseerStateUpdate-72457920339378180-127.0.0.1:37722_-n_0000000000) [n:127.0.0.1:37722_ ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
[junit4] 2> "operation":"downnode",
[junit4] 2> "node_name":"127.0.0.1:46212_"} current state version: 4
[junit4] 2> 1917080 INFO (zkCallback-4828-thread-2) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 1917080 INFO (zkCallback-4846-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 1917080 INFO (zkCallback-4870-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 1917080 INFO (zkCallback-4854-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 1917080 INFO (zkCallback-4861-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 1917080 INFO (zkCallback-4835-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 1917098 INFO (zkConnectionManagerCallback-4878-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1917099 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[690FCFDA6B8134D2]) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (3)
[junit4] 2> 1917100 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[690FCFDA6B8134D2]) [ ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:42889/solr ready
[junit4] 2> 1917101 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[690FCFDA6B8134D2]) [ ] o.a.s.h.a.MetricsHistoryHandler No .system collection, keeping metrics history in memory.
[junit4] 2> 1917121 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[690FCFDA6B8134D2]) [ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@67453e75
[junit4] 2> 1917131 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[690FCFDA6B8134D2]) [ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@67453e75
[junit4] 2> 1917131 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[690FCFDA6B8134D2]) [ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@67453e75
[junit4] 2> 1917133 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[690FCFDA6B8134D2]) [ ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J1/../../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.FullSolrCloudDistribCmdsTest_690FCFDA6B8134D2-001/shard-2-001/cores
[junit4] 2> 1917168 INFO (qtp1443383752-70379) [n:127.0.0.1:37722_ ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :addreplica with params node=127.0.0.1:46212_&action=ADDREPLICA&collection=collection1&shard=shard3&type=NRT&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 1917170 INFO (OverseerCollectionConfigSetProcessor-72457920339378180-127.0.0.1:37722_-n_0000000000) [n:127.0.0.1:37722_ ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000004 doesn't exist. Requestor may have disconnected from ZooKeeper
[junit4] 2> 1917170 INFO (OverseerThreadFactory-7558-thread-4-processing-n:127.0.0.1:37722_) [n:127.0.0.1:37722_ c:collection1 s:shard3 ] o.a.s.c.a.c.AddReplicaCmd Node Identified 127.0.0.1:46212_ for creating new replica of shard shard3 for collection collection1
[junit4] 2> 1917173 INFO (qtp321590564-70469) [n:127.0.0.1:46212_ x:collection1_shard3_replica_n63] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf1&name=collection1_shard3_replica_n63&action=CREATE&collection=collection1&shard=shard3&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 1917175 DEBUG (OverseerStateUpdate-72457920339378180-127.0.0.1:37722_-n_0000000000) [n:127.0.0.1:37722_ ] o.a.s.c.Overseer processMessage: queueSize: 2, message = {
[junit4] 2> "core":"collection1_shard3_replica_n63",
[junit4] 2> "roles":null,
[junit4] 2> "base_url":"http://127.0.0.1:46212",
[junit4] 2> "node_name":"127.0.0.1:46212_",
[junit4] 2> "numShards":"3",
[junit4] 2> "state":"down",
[junit4] 2> "shard":"shard3",
[junit4] 2> "collection":"collection1",
[junit4] 2> "type":"NRT",
[junit4] 2> "operation":"state"} current state version: 4
[junit4] 2> 1918190 INFO (qtp321590564-70469) [n:127.0.0.1:46212_ c:collection1 s:shard3 x:collection1_shard3_replica_n63] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
[junit4] 2> 1918203 INFO (qtp321590564-70469) [n:127.0.0.1:46212_ c:collection1 s:shard3 x:collection1_shard3_replica_n63] o.a.s.s.IndexSchema [collection1_shard3_replica_n63] Schema name=test
[junit4] 2> 1918282 INFO (qtp321590564-70469) [n:127.0.0.1:46212_ c:collection1 s:shard3 x:collection1_shard3_replica_n63] o.a.s.s.IndexSchema Loaded schema test/1.6 with uniqueid field id
[junit4] 2> 1918294 INFO (qtp321590564-70469) [n:127.0.0.1:46212_ c:collection1 s:shard3 x:collection1_shard3_replica_n63] o.a.s.c.CoreContainer Creating SolrCore 'collection1_shard3_replica_n63' using configuration from collection collection1, trusted=true
[junit4] 2> 1918295 INFO (qtp321590564-70469) [n:127.0.0.1:46212_ c:collection1 s:shard3 x:collection1_shard3_replica_n63] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.collection1.shard3.replica_n63' (registry 'solr.core.collection1.shard3.replica_n63') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@67453e75
[junit4] 2> 1918295 INFO (qtp321590564-70469) [n:127.0.0.1:46212_ c:collection1 s:shard3 x:collection1_shard3_replica_n63] o.a.s.c.SolrCore [[collection1_shard3_replica_n63] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.FullSolrCloudDistribCmdsTest_690FCFDA6B8134D2-001/shard-2-001/cores/collection1_shard3_replica_n63], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J1/../../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.FullSolrCloudDistribCmdsTest_690FCFDA6B8134D2-001/shard-2-001/cores/collection1_shard3_replica_n63/data/]
[junit4] 2> 1918297 INFO (qtp321590564-70469) [n:127.0.0.1:46212_ c:collection1 s:shard3 x:collection1_shard3_replica_n63] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=27, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=0.541015625, noCFSRatio=0.0]
[junit4] 2> 1918298 WARN (qtp321590564-70469) [n:127.0.0.1:46212_ c:collection1 s:shard3 x:collection1_shard3_replica_n63] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,attributes = {initParams=a, name=/dump, class=DumpRequestHandler},args = {defaults={a=A,b=B}}}
[junit4] 2> 1918335 INFO (qtp321590564-70469) [n:127.0.0.1:46212_ c:collection1 s:shard3 x:collection1_shard3_replica_n63] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 1918335 INFO (qtp321590564-70469) [n:127.0.0.1:46212_ c:collection1 s:shard3 x:collection1_shard3_replica_n63] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 1918337 INFO (qtp321590564-70469) [n:127.0.0.1:46212_ c:collection1 s:shard3 x:collection1_shard3_replica_n63] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 1918337 INFO (qtp321590564-70469) [n:127.0.0.1:46212_ c:collection1 s:shard3 x:collection1_shard3_replica_n63] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 1918337 INFO (qtp321590564-70469) [n:127.0.0.1:46212_ c:collection1 s:shard3 x:collection1_shard3_replica_n63] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=31, maxMergeAtOnceExplicit=47, maxMergedSegmentMB=0.7861328125, floorSegmentMB=1.216796875, forceMergeDeletesPctAllowed=0.6729811829427002, segmentsPerTier=30.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.47373358407785326, deletesPctAllowed=45.33519292309924
[junit4] 2> 1918338 INFO (qtp321590564-70469) [n:127.0.0.1:46212_ c:collection1 s:shard3 x:collection1_shard3_replica_n63] o.a.s.s.SolrIndexSearcher Opening [Searcher@379b9080[collection1_shard3_replica_n63] main]
[junit4] 2> 1918339 INFO (qtp321590564-70469) [n:127.0.0.1:46212_ c:collection1 s:shard3 x:collection1_shard3_replica_n63] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 1918339 INFO (qtp321590564-70469) [n:127.0.0.1:46212_ c:collection1 s:shard3 x:collection1_shard3_replica_n63] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 1918340 INFO (qtp321590564-70469) [n:127.0.0.1:46212_ c:collection1 s:shard3 x:collection1_shard3_replica_n63] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
[junit4] 2> 1918340 INFO (qtp321590564-70469) [n:127.0.0.1:46212_ c:collection1 s:shard3 x:collection1_shard3_replica_n63] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1613569495134109696
[junit4] 2> 1918340 INFO (searcherExecutor-7591-thread-1-processing-n:127.0.0.1:46212_ x:collection1_shard3_replica_n63 c:collection1 s:shard3) [n:127.0.0.1:46212_ c:collection1 s:shard3 x:collection1_shard3_replica_n63] o.a.s.c.SolrCore [collection1_shard3_replica_n63] Registered new searcher Searcher@379b9080[collection1_shard3_replica_n63] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 1918345 INFO (qtp321590564-70469) [n:127.0.0.1:46212_ c:collection1 s:shard3 x:collection1_shard3_replica_n63] o.a.s.c.ZkShardTerms Successful update of terms at /collections/collection1/terms/shard3 to Terms{values={core_node64=0}, version=0}
[junit4] 2> 1918347 INFO (qtp321590564-70469) [n:127.0.0.1:46212_ c:collection1 s:shard3 x:collection1_shard3_replica_n63] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
[junit4] 2> 1918347 INFO (qtp321590564-70469) [n:127.0.0.1:46212_ c:collection1 s:shard3 x:collection1_shard3_replica_n63] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
[junit4] 2> 1918347 INFO (qtp321590564-70469) [n:127.0.0.1:46212_ c:collection1 s:shard3 x:collection1_shard3_replica_n63] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:46212/collection1_shard3_replica_n63/
[junit4] 2> 1918347 INFO (qtp321590564-70469) [n:127.0.0.1:46212_ c:collection1 s:shard3 x:collection1_shard3_replica_n63] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
[junit4] 2> 1918347 INFO (qtp321590564-70469) [n:127.0.0.1:46212_ c:collection1 s:shard3 x:collection1_shard3_replica_n63] o.a.s.c.SyncStrategy http://127.0.0.1:46212/collection1_shard3_replica_n63/ has no replicas
[junit4] 2> 1918347 INFO (qtp321590564-70469) [n:127.0.0.1:46212_ c:collection1 s:shard3 x:collection1_shard3_replica_n63] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
[junit4] 2> 1918350 DEBUG (OverseerStateUpdate-72457920339378180-127.0.0.1:37722_-n_0000000000) [n:127.0.0.1:37722_ ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
[junit4] 2> "operation":"leader",
[junit4] 2> "shard":"shard3",
[junit4] 2> "collection":"collection1",
[junit4] 2> "base_url":"http://127.0.0.1:46212",
[junit4] 2> "core":"collection1_shard3_replica_n63",
[junit4] 2> "state":"active"} current state version: 5
[junit4] 2> 1918350 INFO (qtp321590564-70469) [n:127.0.0.1:46212_ c:collection1 s:shard3 x:collection1_shard3_replica_n63] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:46212/collection1_shard3_replica_n63/ shard3
[junit4] 2> 1918501 INFO (qtp321590564-70469) [n:127.0.0.1:46212_ c:collection1 s:shard3 x:collection1_shard3_replica_n63] o.a.s.c.ZkController I am the leader, no recovery necessary
[junit4] 2> 1918503 DEBUG (OverseerStateUpdate-72457920339378180-127.0.0.1:37722_-n_0000000000) [n:127.0.0.1:37722_ ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
[junit4] 2> "core":"collection1_shard3_replica_n63",
[junit4] 2> "core_node_name":"core_node64",
[junit4] 2> "roles":null,
[junit4] 2> "base_url":"http://127.0.0.1:46212",
[junit4] 2> "node_name":"127.0.0.1:46212_",
[junit4] 2> "numShards":"3",
[junit4] 2> "state":"active",
[junit4] 2> "shard":"shard3",
[junit4] 2> "collection":"collection1",
[junit4] 2> "type":"NRT",
[junit4] 2> "operation":"state"} current state version: 6
[junit4] 2> 1918506 INFO (qtp321590564-70469) [n:127.0.0.1:46212_ c:collection1 s:shard3 x:collection1_shard3_replica_n63] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=conf1&name=collection1_shard3_replica_n63&action=CREATE&collection=collection1&shard=shard3&wt=javabin&version=2&replicaType=NRT} status=0 QTime=1333
[junit4] 2> 1918508 INFO (qtp1443383752-70379) [n:127.0.0.1:37722_ c:collection1 ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={node=127.0.0.1:46212_&action=ADDREPLICA&collection=collection1&shard=shard3&type=NRT&wt=javabin&version=2} status=0 QTime=1340
[junit4] 2> 1918620 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[690FCFDA6B8134D2]) [ ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 3 in directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.FullSolrCloudDistribCmdsTest_690FCFDA6B8134D2-001/shard-3-001 of type NRT
[junit4] 2> 1918621 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[690FCFDA6B8134D2]) [ ] o.e.j.s.Server jetty-9.4.11.v20180605; built: 2018-06-05T18:24:03.829Z; git: d5fc0523cfa96bfebfbda19606cad384d772f04c; jvm 1.8.0_172-b11
[junit4] 2> 1918623 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[690FCFDA6B8134D2]) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 1918623 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[690FCFDA6B8134D2]) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 1918623 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[690FCFDA6B8134D2]) [ ] o.e.j.s.session node0 Scavenging every 600000ms
[junit4] 2> 1918624 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[690FCFDA6B8134D2]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@6a0e8493{/,null,AVAILABLE}
[junit4] 2> 1918624 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[690FCFDA6B8134D2]) [ ] o.e.j.s.AbstractConnector Started ServerConnector@4f3b8cbc{HTTP/1.1,[http/1.1]}{127.0.0.1:40466}
[junit4] 2> 1918624 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[690FCFDA6B8134D2]) [ ] o.e.j.s.Server Started @1918663ms
[junit4] 2> 1918624 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[690FCFDA6B8134D2]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.FullSolrCloudDistribCmdsTest_690FCFDA6B8134D2-001/tempDir-001/jetty3, replicaType=NRT, solrconfig=solrconfig.xml, hostContext=/, hostPort=40466, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J1/../../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.FullSolrCloudDistribCmdsTest_690FCFDA6B8134D2-001/shard-3-001/cores}
[junit4] 2> 1918624 ERROR (TEST-FullSolrCloudDistribCmdsTest.test-seed#[690FCFDA6B8134D2]) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 1918624 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[690FCFDA6B8134D2]) [ ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
[junit4] 2> 1918624 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[690FCFDA6B8134D2]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 8.0.0
[junit4] 2> 1918624 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[690FCFDA6B8134D2]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 1918625 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[690FCFDA6B8134D2]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 1918625 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[690FCFDA6B8134D2]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-10-06T09:56:25.531Z
[junit4] 2> 1918626 INFO (zkConnectionManagerCallback-4881-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1918627 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[690FCFDA6B8134D2]) [ ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
[junit4] 2> 1918627 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[690FCFDA6B8134D2]) [ ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.FullSolrCloudDistribCmdsTest_690FCFDA6B8134D2-001/shard-3-001/solr.xml
[junit4] 2> 1918631 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[690FCFDA6B8134D2]) [ ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverWorkLoopDelay is ignored
[junit4] 2> 1918631 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[690FCFDA6B8134D2]) [ ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverBadNodeExpiration is ignored
[junit4] 2> 1918632 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[690FCFDA6B8134D2]) [ ] o.a.s.c.SolrXmlConfig MBean server found: com.sun.jmx.mbeanserver.JmxMBeanServer@67453e75, but no JMX reporters were configured - adding default JMX reporter.
[junit4] 2> 1918875 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[690FCFDA6B8134D2]) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:42889/solr
[junit4] 2> 1918876 INFO (zkConnectionManagerCallback-4885-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1918878 INFO (zkConnectionManagerCallback-4887-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1918884 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[690FCFDA6B8134D2]) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (3)
[junit4] 2> 1918888 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[690FCFDA6B8134D2]) [ ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 4 transient cores
[junit4] 2> 1918888 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[690FCFDA6B8134D2]) [ ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:40466_
[junit4] 2> 1918888 DEBUG (OverseerStateUpdate-72457920339378180-127.0.0.1:37722_-n_0000000000) [n:127.0.0.1:37722_ ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
[junit4] 2> "operation":"downnode",
[junit4] 2> "node_name":"127.0.0.1:40466_"} current state version: 7
[junit4] 2> 1918889 INFO (zkCallback-4835-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 1918889 INFO (zkCallback-4854-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 1918889 INFO (zkCallback-4846-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 1918889 INFO (zkCallback-4870-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 1918892 INFO (zkCallback-4861-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 1918893 INFO (zkCallback-4886-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 1918893 INFO (zkCallback-4877-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 1918893 INFO (zkCallback-4828-thread-2) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 1918907 INFO (zkConnectionManagerCallback-4894-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1918909 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[690FCFDA6B8134D2]) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (4)
[junit4] 2> 1918910 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[690FCFDA6B8134D2]) [ ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:42889/solr ready
[junit4] 2> 1918910 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[690FCFDA6B8134D2]) [ ] o.a.s.h.a.MetricsHistoryHandler No .system collection, keeping metrics history in memory.
[junit4] 2> 1918929 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[690FCFDA6B8134D2]) [ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@67453e75
[junit4] 2> 1918940 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[690FCFDA6B8134D2]) [ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@67453e75
[junit4] 2> 1918940 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[690FCFDA6B8134D2]) [ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@67453e75
[junit4] 2> 1918941 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[690FCFDA6B8134D2]) [ ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J1/../../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.FullSolrCloudDistribCmdsTest_690FCFDA6B8134D2-001/shard-3-001/cores
[junit4] 2> 1919000 INFO (qtp1443383752-70373) [n:127.0.0.1:37722_ ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :addreplica with params node=127.0.0.1:40466_&action=ADDREPLICA&collection=collection1&shard=shard1&type=NRT&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 1919002 INFO (OverseerCollectionConfigSetProcessor-72457920339378180-127.0.0.1:37722_-n_0000000000) [n:127.0.0.1:37722_ ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000006 doesn't exist. Requestor may have disconnected from ZooKeeper
[junit4] 2> 1919005 INFO (OverseerThreadFactory-7558-thread-5-processing-n:127.0.0.1:37722_) [n:127.0.0.1:37722_ c:collection1 s:shard1 ] o.a.s.c.a.c.AddReplicaCmd Node Identified 127.0.0.1:40466_ for creating new replica of shard shard1 for collection collection1
[junit4] 2> 1919009 INFO (qtp333792483-70506) [n:127.0.0.1:40466_ x:collection1_shard1_replica_n65] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf1&name=collection1_shard1_replica_n65&action=CREATE&collection=collection1&shard=shard1&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 1919013 DEBUG (OverseerStateUpdate-72457920339378180-127.0.0.1:37722_-n_0000000000) [n:127.0.0.1:37722_ ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
[junit4] 2> "core":"collection1_shard1_replica_n65",
[junit4] 2> "roles":null,
[junit4] 2> "base_url":"http://127.0.0.1:40466",
[junit4] 2> "node_name":"127.0.0.1:40466_",
[junit4] 2> "numShards":"3",
[junit4] 2> "state":"down",
[junit4] 2> "shard":"shard1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "type":"NRT",
[junit4] 2> "operation":"state"} current state version: 7
[junit4] 2> 1920026 INFO (qtp333792483-70506) [n:127.0.0.1:40466_ c:collection1 s:shard1 x:collection1_shard1_replica_n65] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
[junit4] 2> 1920037 INFO (qtp333792483-70506) [n:127.0.0.1:40466_ c:collection1 s:shard1 x:collection1_shard1_replica_n65] o.a.s.s.IndexSchema [collection1_shard1_replica_n65] Schema name=test
[junit4] 2> 1920120 INFO (qtp333792483-70506) [n:127.0.0.1:40466_ c:collection1 s:shard1 x:collection1_shard1_replica_n65] o.a.s.s.IndexSchema Loaded schema test/1.6 with uniqueid field id
[junit4] 2> 1920132 INFO (qtp333792483-70506) [n:127.0.0.1:40466_ c:collection1 s:shard1 x:collection1_shard1_replica_n65] o.a.s.c.CoreContainer Creating SolrCore 'collection1_shard1_replica_n65' using configuration from collection collection1, trusted=true
[junit4] 2> 1920133 INFO (qtp333792483-70506) [n:127.0.0.1:40466_ c:collection1 s:shard1 x:collection1_shard1_replica_n65] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.collection1.shard1.replica_n65' (registry 'solr.core.collection1.shard1.replica_n65') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@67453e75
[junit4] 2> 1920133 INFO (qtp333792483-70506) [n:127.0.0.1:40466_ c:collection1 s:shard1 x:collection1_shard1_replica_n65] o.a.s.c.SolrCore [[collection1_shard1_replica_n65] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.FullSolrCloudDistribCmdsTest_690FCFDA6B8134D2-001/shard-3-001/cores/collection1_shard1_replica_n65], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J1/../../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.FullSolrCloudDistribCmdsTest_690FCFDA6B8134D2-001/shard-3-001/cores/collection1_shard1_replica_n65/data/]
[junit4] 2> 1920135 INFO (qtp333792483-70506) [n:127.0.0.1:40466_ c:collection1 s:shard1 x:collection1_shard1_replica_n65] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=27, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=0.541015625, noCFSRatio=0.0]
[junit4] 2> 1920141 WARN (qtp333792483-70506) [n:127.0.0.1:40466_ c:collection1 s:shard1 x:collection1_shard1_replica_n65] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,attributes = {initParams=a, name=/dump, class=DumpRequestHandler},args = {defaults={a=A,b=B}}}
[junit4] 2> 1920178 INFO (qtp333792483-70506) [n:127.0.0.1:40466_ c:collection1 s:shard1 x:collection1_shard1_replica_n65] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 1920178 INFO (qtp333792483-70506) [n:127.0.0.1:40466_ c:collection1 s:shard1 x:collection1_shard1_replica_n65] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 1920179 INFO (qtp333792483-70506) [n:127.0.0.1:40466_ c:collection1 s:shard1 x:collection1_shard1_replica_n65] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 1920179 INFO (qtp333792483-70506) [n:127.0.0.1:40466_ c:collection1 s:shard1 x:collection1_shard1_replica_n65] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 1920180 INFO (qtp333792483-70506) [n:127.0.0.1:40466_ c:collection1 s:shard1 x:collection1_shard1_replica_n65] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=31, maxMergeAtOnceExplicit=47, maxMergedSegmentMB=0.7861328125, floorSegmentMB=1.216796875, forceMergeDeletesPctAllowed=0.6729811829427002, segmentsPerTier=30.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.47373358407785326, deletesPctAllowed=45.33519292309924
[junit4] 2> 1920180 INFO (qtp333792483-70506) [n:127.0.0.1:40466_ c:collection1 s:shard1 x:collection1_shard1_replica_n65] o.a.s.s.SolrIndexSearcher Opening [Searcher@b6f9712[collection1_shard1_replica_n65] main]
[junit4] 2> 1920181 INFO (qtp333792483-70506) [n:127.0.0.1:40466_ c:collection1 s:shard1 x:collection1_shard1_replica_n65] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 1920181 INFO (qtp333792483-70506) [n:127.0.0.1:40466_ c:collection1 s:shard1 x:collection1_shard1_replica_n65] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 1920181 INFO (qtp333792483-70506) [n:127.0.0.1:40466_ c:collection1 s:shard1 x:collection1_shard1_replica_n65] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
[junit4] 2> 1920182 INFO (searcherExecutor-7605-thread-1-processing-n:127.0.0.1:40466_ x:collection1_shard1_replica_n65 c:collection1 s:shard1) [n:127.0.0.1:40466_ c:collection1 s:shard1 x:collection1_shard1_replica_n65] o.a.s.c.SolrCore [collection1_shard1_replica_n65] Registered new searcher Searcher@b6f9712[collection1_shard1_replica_n65] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 1920182 INFO (qtp333792483-70506) [n:127.0.0.1:40466_ c:collection1 s:shard1 x:collection1_shard1_replica_n65] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1613569497065586688
[junit4] 2> 1920190 INFO (qtp333792483-70506) [n:127.0.0.1:40466_ c:collection1 s:shard1 x:collection1_shard1_replica_n65] o.a.s.c.ZkShardTerms Successful update of terms at /collections/collection1/terms/shard1 to Terms{values={core_node66=0}, version=0}
[junit4] 2> 1920193 INFO (qtp333792483-70506) [n:127.0.0.1:40466_ c:collection1 s:shard1 x:collection1_shard1_replica_n65] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
[junit4] 2> 1920193 INFO (qtp333792483-70506) [n:127.0.0.1:40466_ c:collection1 s:shard1 x:collection1_shard1_replica_n65] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
[junit4] 2> 1920193 INFO (qtp333792483-70506) [n:127.0.0.1:40466_ c:collection1 s:shard1 x:collection1_shard1_replica_n65] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:40466/collection1_shard1_replica_n65/
[junit4] 2> 1920194 INFO (qtp333792483-70506) [n:127.0.0.1:40466_ c:collection1 s:shard1 x:collection1_shard1_replica_n65] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
[junit4] 2> 1920194 INFO (qtp333792483-70506) [n:127.0.0.1:40466_ c:collection1 s:shard1 x:collection1_shard1_replica_n65] o.a.s.c.SyncStrategy http://127.0.0.1:40466/collection1_shard1_replica_n65/ has no replicas
[junit4] 2> 1920194 INFO (qtp333792483-70506) [n:127.0.0.1:40466_ c:collection1 s:shard1 x:collection1_shard1_replica_n65] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
[junit4] 2> 1920198 INFO (qtp333792483-70506) [n:127.0.0.1:40466_ c:collection1 s:shard1 x:collection1_shard1_replica_n65] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:40466/collection1_shard1_replica_n65/ shard1
[junit4] 2> 1920198 DEBUG (OverseerStateUpdate-72457920339378180-127.0.0.1:37722_-n_0000000000) [n:127.0.0.1:37722_ ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
[junit4] 2> "operation":"leader",
[junit4] 2> "shard":"shard1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "base_url":"http://127.0.0.1:40466",
[junit4] 2> "core":"collection1_shard1_replica_n65",
[junit4] 2> "state":"active"} current state version: 8
[junit4] 2> 1920353 INFO (qtp333792483-70506) [n:127.0.0.1:40466_ c:collection1 s:shard1 x:collection1_shard1_replica_n65] o.a.s.c.ZkController I am the leader, no recovery necessary
[junit4] 2> 1920354 DEBUG (OverseerStateUpdate-72457920339378180-127.0.0.1:37722_-n_0000000000) [n:127.0.0.1:37722_ ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
[junit4] 2> "core":"collection1_shard1_replica_n65",
[junit4] 2> "core_node_name":"core_node66",
[junit4] 2> "roles":null,
[junit4] 2> "base_url":"http://127.0.0.1:40466",
[junit4] 2> "node_name":"127.0.0.1:40466_",
[junit4] 2> "numShards":"3",
[junit4]
[...truncated too long message...]
(NoShadowingOrOverridesOnMethodsRule.java:40)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
[junit4] 2> at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
[junit4] 2> at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
[junit4] 2> at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2> oct. 06, 2018 8:24:26 PM com.carrotsearch.randomizedtesting.ThreadLeakControl tryToInterruptAll
[junit4] 2> INFOS: Starting to interrupt leaked threads:
[junit4] 2> 1) Thread[id=82102, name=TEST-TestSimGenericDistributedQueue.testDistributedQueue-seed#[690FCFDA6B8134D2], state=TIMED_WAITING, group=TGRP-TestSimGenericDistributedQueue]
[junit4] 2> 10799173 INFO (TEST-TestSimGenericDistributedQueue.testDistributedQueue-seed#[690FCFDA6B8134D2]) [ ] o.a.s.SolrTestCaseJ4 ###Ending testDistributedQueue {#2}
[junit4] 2> NOTE: leaving temporary files on disk at: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.autoscaling.sim.TestSimGenericDistributedQueue_690FCFDA6B8134D2-001
[junit4] 2> Oct 06, 2018 12:24:26 PM com.carrotsearch.randomizedtesting.ThreadLeakControl tryToInterruptAll
[junit4] 2> INFO: All leaked threads terminated.
[junit4] 2> NOTE: test params are: codec=Asserting(Lucene80), sim=Asserting(org.apache.lucene.search.similarities.AssertingSimilarity@1daa8152), locale=fr-CA, timezone=Asia/Hong_Kong
[junit4] 2> NOTE: Linux 4.4.0-130-generic amd64/Oracle Corporation 1.8.0_172 (64-bit)/cpus=4,threads=1,free=188333088,total=497025024
[junit4] 2> NOTE: All tests run in this JVM: [TestCloudDeleteByQuery, BadIndexSchemaTest, TestDistributedStatsComponentCardinality, TestExclusionRuleCollectionAccess, DistributedFacetSimpleRefinementLongTailTest, DefaultValueUpdateProcessorTest, TestStressInPlaceUpdates, ShardRoutingTest, ImplicitSnitchTest, TestAuthorizationFramework, TestClusterProperties, DisMaxRequestHandlerTest, TestLuceneIndexBackCompat, TestCloudJSONFacetSKG, TriLevelCompositeIdRoutingTest, UpdateRequestProcessorFactoryTest, CopyFieldTest, TestConfig, AdminHandlersProxyTest, SuggesterWFSTTest, SimplePostToolTest, DeleteNodeTest, WrapperMergePolicyFactoryTest, SpellCheckCollatorTest, TestConfigSetProperties, CoreMergeIndexesAdminHandlerTest, MultiTermTest, TestSimNodeAddedTrigger, TestRealTimeGet, ZkCLITest, ReplicationFactorTest, OpenExchangeRatesOrgProviderTest, SchemaApiFailureTest, SpellingQueryConverterTest, TestFaceting, TestCloudNestedDocsSort, TestSolr4Spatial, TestBulkSchemaAPI, TestClusterStateMutator, TestHdfsCloudBackupRestore, TestUnifiedSolrHighlighter, URLClassifyProcessorTest, TestOmitPositions, V2StandaloneTest, SOLR749Test, TestComplexPhraseQParserPlugin, ResponseHeaderTest, ScriptEngineTest, UtilsToolTest, IndexSchemaRuntimeFieldTest, TestHashQParserPlugin, BlockJoinFacetRandomTest, TestQueryUtils, BadComponentTest, TestFieldCacheSort, TriggerSetPropertiesIntegrationTest, TestConfigSetsAPIZkFailure, HealthCheckHandlerTest, TestNRTOpen, JsonLoaderTest, DateMathParserTest, TestPayloadScoreQParserPlugin, SolrGangliaReporterTest, NodeMutatorTest, ScheduledTriggerIntegrationTest, TestSolrQueryResponse, TestMiniSolrCloudClusterSSL, TestMaxTokenLenTokenizer, DateRangeFieldTest, MinimalSchemaTest, QueryParsingTest, TestSolrFieldCacheBean, TestSolrCloudSnapshots, SimpleFacetsTest, TestSchemalessBufferedUpdates, TestSimDistribStateManager, TestValueSourceCache, Tagger2Test, BlockCacheTest, TestBulkSchemaConcurrent, TestStressVersions, TestSolrIndexConfig, TestExceedMaxTermLength, SearchHandlerTest, DistributedQueryComponentCustomSortTest, TestEmbeddedSolrServerSchemaAPI, TestSSLRandomization, TestMinMaxOnMultiValuedField, TestDFISimilarityFactory, TestJsonFacetRefinement, SimpleCollectionCreateDeleteTest, TestReversedWildcardFilterFactory, SolrShardReporterTest, XmlInterpolationTest, OverseerCollectionConfigSetProcessorTest, ClassificationUpdateProcessorIntegrationTest, TestCursorMarkWithoutUniqueKey, MetricsHistoryHandlerTest, AutoScalingHandlerTest, SyncSliceTest, TestJsonFacetsWithNestedObjects, TestPivotHelperCode, CreateRoutedAliasTest, TestImplicitCoreProperties, TestSizeLimitedDistributedMap, StatsComponentTest, TestJavabinTupleStreamParser, HdfsDirectoryTest, TestExactSharedStatsCache, NodeAddedTriggerTest, ProtectedTermFilterFactoryTest, MissingSegmentRecoveryTest, AlternateDirectoryTest, AssignTest, TestTolerantUpdateProcessorCloud, TestAddFieldRealTimeGet, SimpleMLTQParserTest, CollectionStateFormat2Test, TestFiltering, MoveReplicaTest, TestSubQueryTransformer, TestFieldCacheWithThreads, TestNumericTerms64, SuggesterFSTTest, RequestLoggingTest, TestSolrConfigHandler, TestCoreDiscovery, OverseerStatusTest, TestRandomRequestDistribution, DistributedSpellCheckComponentTest, HdfsChaosMonkeyNothingIsSafeTest, SmileWriterTest, HdfsUnloadDistributedZkTest, TestDocSet, TestCollationField, NumberUtilsTest, TestSimTriggerIntegration, TestBackupRepositoryFactory, DirectSolrSpellCheckerTest, BasicFunctionalityTest, ConvertedLegacyTest, CursorPagingTest, SolrInfoBeanTest, SolrTestCaseJ4Test, TestDistributedSearch, TestDocumentBuilder, TestGroupingSearch, TestHighlightDedupGrouping, TestJoin, TestRandomFaceting, TestSimpleTrackingShardHandler, TestSolrCoreProperties, TestTolerantSearch, TestTrie, PathHierarchyTokenizerFactoryTest, TestLuceneMatchVersion, TokenizerChainTest, ActionThrottleTest, AddReplicaTest, AliasIntegrationTest, AssignBackwardCompatibilityTest, AsyncCallRequestStatusResponseTest, BasicDistributedZkTest, ChaosMonkeyNothingIsSafeWithPullReplicasTest, ClusterStateTest, ClusterStateUpdateTest, ConcurrentCreateRoutedAliasTest, ConfigSetsAPITest, ConnectionManagerTest, CreateCollectionCleanupTest, DistribCursorPagingTest, DistribJoinFromCollectionTest, DocValuesNotIndexedTest, ForceLeaderTest, FullSolrCloudDistribCmdsTest, OverseerTest, RecoveryAfterSoftCommitTest, RecoveryZkTest, RemoteQueryErrorTest, ReplaceNodeNoTargetTest, ReplaceNodeTest, SolrCloudExampleTest, TestCloudRecovery, TestConfigSetsAPIExclusivity, TestDeleteCollectionOnDownNodes, TestDistributedMap, TestLeaderElectionWithEmptyReplica, TestPullReplica, TestRequestForwarding, TestShortCircuitedRequests, TestSolrCloudWithDelegationTokens, TestSolrCloudWithKerberosAlt, TestSolrCloudWithSecureImpersonation, TestStressCloudBlindAtomicUpdates, TestZkChroot, TlogReplayBufferedWhileIndexingTest, CollectionsAPIAsyncDistributedZkTest, ConcurrentDeleteAndCreateCollectionTest, CustomCollectionTest, TestReplicaProperties, TestRequestStatusCollectionAPI, AutoAddReplicasIntegrationTest, ExecutePlanActionTest, HdfsAutoAddReplicasIntegrationTest, NodeLostTriggerTest, RestoreTriggerStateTest, SearchRateTriggerTest, TriggerCooldownIntegrationTest, TestSimExtremeIndexing, TestSimGenericDistributedQueue]
[junit4] 2> NOTE: download the large Jenkins line-docs file by running 'ant get-jenkins-line-docs' in the lucene directory.
[junit4] 2> NOTE: reproduce with: ant test -Dtestcase=TestSimGenericDistributedQueue -Dtests.seed=690FCFDA6B8134D2 -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/test-data/enwiki.random.lines.txt -Dtests.locale=fr-CA -Dtests.timezone=Asia/Hong_Kong -Dtests.asserts=true -Dtests.file.encoding=ISO-8859-1
[junit4] ERROR 0.00s J1 | TestSimGenericDistributedQueue (suite) <<<
[junit4] > Throwable #1: java.lang.Exception: Suite timeout exceeded (>= 7200000 msec).
[junit4] > at __randomizedtesting.SeedInfo.seed([690FCFDA6B8134D2]:0)
[junit4] Completed [835/835 (2!)] on J1 in 7280.38s, 4 tests, 2 errors <<< FAILURES!
[...truncated 1 lines...]
[junit4] JVM J1: stdout was not empty, see: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/temp/junit4-J1-20181006_092426_814545575240458603412.sysout
[junit4] >>> JVM J1 emitted unexpected output (verbatim) ----
[junit4] java.lang.OutOfMemoryError: Java heap space
[junit4] Dumping heap to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/heapdumps/java_pid24953.hprof ...
[junit4] Heap dump file created [594292755 bytes in 8.552 secs]
[junit4] <<< JVM J1: EOF ----
[...truncated 8961 lines...]
BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/build.xml:651: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/build.xml:585: Some of the tests produced a heap dump, but did not fail. Maybe a suppressed OutOfMemoryError? Dumps created:
* java_pid24953.hprof
Total time: 369 minutes 25 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any
[JENKINS] Lucene-Solr-NightlyTests-master - Build # 1660 - Still
Failing
Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-master/1660/
1 tests failed.
FAILED: org.apache.solr.uninverting.TestDocTermOrdsUninvertLimit.testTriggerUnInvertLimit
Error Message:
Java heap space
Stack Trace:
java.lang.OutOfMemoryError: Java heap space
at __randomizedtesting.SeedInfo.seed([3F0C366A88442659:CBE1EAE85F3FCEE]:0)
at org.apache.lucene.store.RAMFile.newBuffer(RAMFile.java:84)
at org.apache.lucene.store.RAMFile.addBuffer(RAMFile.java:57)
at org.apache.lucene.store.RAMOutputStream.switchCurrentBuffer(RAMOutputStream.java:168)
at org.apache.lucene.store.RAMOutputStream.writeBytes(RAMOutputStream.java:154)
at org.apache.lucene.store.DataOutput.copyBytes(DataOutput.java:278)
at org.apache.lucene.store.MockIndexOutputWrapper.copyBytes(MockIndexOutputWrapper.java:165)
at org.apache.lucene.codecs.lucene50.Lucene50CompoundFormat.write(Lucene50CompoundFormat.java:96)
at org.apache.lucene.index.IndexWriter.createCompoundFile(IndexWriter.java:5004)
at org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:4517)
at org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:4075)
at org.apache.lucene.index.SerialMergeScheduler.merge(SerialMergeScheduler.java:40)
at org.apache.lucene.index.IndexWriter.maybeMerge(IndexWriter.java:2178)
at org.apache.lucene.index.IndexWriter.processEvents(IndexWriter.java:5110)
at org.apache.lucene.index.IndexWriter.updateDocument(IndexWriter.java:1620)
at org.apache.lucene.index.IndexWriter.addDocument(IndexWriter.java:1236)
at org.apache.lucene.index.RandomIndexWriter.addDocument(RandomIndexWriter.java:189)
at org.apache.solr.uninverting.TestDocTermOrdsUninvertLimit.testTriggerUnInvertLimit(TestDocTermOrdsUninvertLimit.java:64)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1742)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:935)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:971)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:985)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
Build Log:
[...truncated 15675 lines...]
[junit4] Suite: org.apache.solr.uninverting.TestDocTermOrdsUninvertLimit
[junit4] 2> NOTE: download the large Jenkins line-docs file by running 'ant get-jenkins-line-docs' in the lucene directory.
[junit4] 2> NOTE: reproduce with: ant test -Dtestcase=TestDocTermOrdsUninvertLimit -Dtests.method=testTriggerUnInvertLimit -Dtests.seed=3F0C366A88442659 -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/test-data/enwiki.random.lines.txt -Dtests.locale=sv -Dtests.timezone=Iran -Dtests.asserts=true -Dtests.file.encoding=US-ASCII
[junit4] ERROR 29.1s J1 | TestDocTermOrdsUninvertLimit.testTriggerUnInvertLimit <<<
[junit4] > Throwable #1: java.lang.OutOfMemoryError: Java heap space
[junit4] > at __randomizedtesting.SeedInfo.seed([3F0C366A88442659:CBE1EAE85F3FCEE]:0)
[junit4] > at org.apache.lucene.store.RAMFile.newBuffer(RAMFile.java:84)
[junit4] > at org.apache.lucene.store.RAMFile.addBuffer(RAMFile.java:57)
[junit4] > at org.apache.lucene.store.RAMOutputStream.switchCurrentBuffer(RAMOutputStream.java:168)
[junit4] > at org.apache.lucene.store.RAMOutputStream.writeBytes(RAMOutputStream.java:154)
[junit4] > at org.apache.lucene.store.DataOutput.copyBytes(DataOutput.java:278)
[junit4] > at org.apache.lucene.store.MockIndexOutputWrapper.copyBytes(MockIndexOutputWrapper.java:165)
[junit4] > at org.apache.lucene.codecs.lucene50.Lucene50CompoundFormat.write(Lucene50CompoundFormat.java:96)
[junit4] > at org.apache.lucene.index.IndexWriter.createCompoundFile(IndexWriter.java:5004)
[junit4] > at org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:4517)
[junit4] > at org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:4075)
[junit4] > at org.apache.lucene.index.SerialMergeScheduler.merge(SerialMergeScheduler.java:40)
[junit4] > at org.apache.lucene.index.IndexWriter.maybeMerge(IndexWriter.java:2178)
[junit4] > at org.apache.lucene.index.IndexWriter.processEvents(IndexWriter.java:5110)
[junit4] > at org.apache.lucene.index.IndexWriter.updateDocument(IndexWriter.java:1620)
[junit4] > at org.apache.lucene.index.IndexWriter.addDocument(IndexWriter.java:1236)
[junit4] > at org.apache.lucene.index.RandomIndexWriter.addDocument(RandomIndexWriter.java:189)
[junit4] > at org.apache.solr.uninverting.TestDocTermOrdsUninvertLimit.testTriggerUnInvertLimit(TestDocTermOrdsUninvertLimit.java:64)
[junit4] 2> NOTE: test params are: codec=Asserting(Lucene80): {field=PostingsFormat(name=MockRandom)}, docValues:{}, maxPointsInLeafNode=1845, maxMBSortInHeap=5.330281823018872, sim=Asserting(org.apache.lucene.search.similarities.AssertingSimilarity@3e761570), locale=sv, timezone=Iran
[junit4] 2> NOTE: Linux 4.4.0-137-generic amd64/Oracle Corporation 1.8.0_172 (64-bit)/cpus=4,threads=1,free=119282080,total=477626368
[junit4] 2> NOTE: All tests run in this JVM: [RequestLoggingTest, SolrInfoBeanTest, DefaultValueUpdateProcessorTest, TestSafeXMLParsing, OpenExchangeRatesOrgProviderTest, TestSubQueryTransformer, NodeMutatorTest, CopyFieldTest, TestHighlightDedupGrouping, TestCloudJSONFacetSKG, TestFuzzyAnalyzedSuggestions, SchemaApiFailureTest, ReplicationFactorTest, TestMiniSolrCloudClusterSSL, BlockJoinFacetRandomTest, SmileWriterTest, SolrCLIZkUtilsTest, JsonLoaderTest, TestDistributedStatsComponentCardinality, TestSchemalessBufferedUpdates, LegacyCloudClusterPropTest, TestSolrConfigHandlerConcurrent, TestUpdate, FacetPivotSmallTest, TestLegacyField, ProtectedTermFilterFactoryTest, TestWithCollection, TestPullReplica, StatelessScriptUpdateProcessorFactoryTest, HdfsChaosMonkeyNothingIsSafeTest, MetricsHistoryHandlerTest, FieldMutatingUpdateProcessorTest, SimpleFacetsTest, TestCursorMarkWithoutUniqueKey, StatsComponentTest, TestFastLRUCache, TestSolrFieldCacheBean, TestLMDirichletSimilarityFactory, AnalysisErrorHandlingTest, DistribJoinFromCollectionTest, DistributedFacetSimpleRefinementLongTailTest, MetricTriggerIntegrationTest, TestStreamBody, OverseerCollectionConfigSetProcessorTest, TestSearcherReuse, UtilsToolTest, ChaosMonkeyNothingIsSafeWithPullReplicasTest, TestCloudPseudoReturnFields, PeerSyncReplicationTest, TestBlobHandler, TestExclusionRuleCollectionAccess, BasicDistributedZkTest, TestMacroExpander, DocValuesNotIndexedTest, TestRandomRequestDistribution, TestSolrIndexConfig, DateMathParserTest, ScriptEngineTest, PeerSyncWithLeaderAndIndexFingerprintCachingTest, RemoteQueryErrorTest, TestTestInjection, SystemInfoHandlerTest, TestSolrConfigHandler, TestDocSet, TestFieldCacheWithThreads, TestExactSharedStatsCache, MultiTermTest, TestSort, SimplePostToolTest, CSVRequestHandlerTest, ScheduledTriggerIntegrationTest, TestFieldCacheSort, TestManagedResource, NodeMarkersRegistrationTest, TestNumericTerms64, TestFaceting, CdcrBootstrapTest, TestLeaderElectionZkExpiry, TestCollationField, SuggesterWFSTTest, DateRangeFieldTest, OverseerStatusTest, CreateRoutedAliasTest, TestValueSourceCache, SyncSliceTest, TestStressRecovery, TestReversedWildcardFilterFactory, HDFSCollectionsAPITest, SearchHandlerTest, SuggesterFSTTest, DistributedQueryComponentCustomSortTest, TriLevelCompositeIdRoutingTest, NodeAddedTriggerTest, TestManagedSynonymFilterFactory, TestTolerantUpdateProcessorCloud, TestExceedMaxTermLength, ConfigSetsAPITest, LeaderFailureAfterFreshStartTest, TestDirectoryFactory, TestJsonFacetRefinement, DirectoryFactoryTest, SOLR749Test, SSLMigrationTest, TestCoreBackup, TestSizeLimitedDistributedMap, TestFiltering, TestSSLRandomization, ResponseHeaderTest, TestHdfsCloudBackupRestore, TestRequestStatusCollectionAPI, AnalyticsQueryTest, JvmMetricsTest, TestAuthorizationFramework, BadIndexSchemaTest, ZkNodePropsTest, DisMaxRequestHandlerTest, TestNoOpRegenerator, DirectSolrSpellCheckerTest, DistributedFacetPivotLargeTest, SolrCmdDistributorTest, TestReplicationHandlerBackup, LIROnShardRestartTest, HLLUtilTest, TermVectorComponentDistributedTest, TestHashQParserPlugin, ExplicitHLLTest, TestSolrCloudWithHadoopAuthPlugin, HighlighterMaxOffsetTest, TestSimTriggerIntegration, TestConfigsApi, TestRangeQuery, ImplicitSnitchTest, RAMDirectoryFactoryTest, TestLFUCache, TestSolrCloudWithDelegationTokens, TestSolrCloudWithKerberosAlt, TestSolrCloudWithSecureImpersonation, ZkCLITest, ZkControllerTest, ZkFailoverTest, AssignTest, ConcurrentDeleteAndCreateCollectionTest, CustomCollectionTest, ShardSplitTest, TriggerCooldownIntegrationTest, TestSimClusterStateProvider, TestSimComputePlanAction, TestSimDistribStateManager, TestSimDistributedQueue, TestSimExecutePlanAction, TestSimExtremeIndexing, TestSimNodeAddedTrigger, TestSimNodeLostTrigger, CdcrVersionReplicationTest, HdfsRecoverLeaseTest, HdfsRecoveryZkTest, HdfsSyncSliceTest, HdfsThreadLeakTest, HdfsUnloadDistributedZkTest, HdfsWriteToMultipleCollectionsTest, TestClusterStateMutator, ZkStateReaderTest, AlternateDirectoryTest, BlobRepositoryCloudTest, ConfigureRecoveryStrategyTest, CoreSorterTest, HdfsDirectoryFactoryTest, OpenCloseCoreStressTest, CoreAdminCreateDiscoverTest, CoreAdminHandlerTest, CoreAdminOperationTest, CoreMergeIndexesAdminHandlerTest, InfoHandlerTest, LoggingHandlerTest, LukeRequestHandlerTest, SecurityConfHandlerTest, SegmentsInfoRequestHandlerTest, ShowFileRequestHandlerTest, TestCollectionAPIs, ZookeeperStatusHandlerTest, BadComponentTest, CustomHighlightComponentTest, DebugComponentTest, DistributedDebugComponentTest, DistributedExpandComponentTest, DistributedFacetExistsSmallTest, DistributedFacetPivotWhiteBoxTest, DistributedSpellCheckComponentTest, QueryElevationComponentTest, TestTrackingShardHandlerFactory, WrapperMergePolicyFactoryTest, CheckHdfsIndexTest, TestMultiValuedNumericRangeQuery, TestNumericRangeQuery64, SolrGraphiteReporterTest, SolrSlf4jReporterTest, SolrShardReporterTest, TestCustomDocTransformer, TestJavabinTupleStreamParser, TestPHPSerializedResponseWriter, TestRawResponseWriter, TestSolrQueryResponse, TestChildDocTransformerHierarchy, TestBulkSchemaAPI, TestUniqueKeyFieldResource, BadCopyFieldTest, DocValuesMultiTest, DocValuesTest, TestCloudManagedSchema, TestManagedSchema, TestPointFields, TestPseudoReturnFields, TestRecovery, TestSolr4Spatial2, TestSolrCoreParser, TestStressLucene, TestSurroundQueryParser, TestTrieFacet, TestXmlQParserPlugin, TestJsonFacetsWithNestedObjects, TestFunctionQuery, BJQParserTest, TestCloudNestedDocsSort, TestBM25SimilarityFactory, TestIBSimilarityFactory, BasicAuthIntegrationTest, SpellCheckCollatorTest, SpellingQueryConverterTest, TestSuggestSpellingConverter, WordBreakSolrSpellCheckerTest, TestBlendedInfixSuggestions, BlockCacheTest, BlockDirectoryTest, TestDocTermOrdsUninvertLimit]
[junit4] Completed [784/835 (1!)] on J1 in 29.13s, 1 test, 1 error <<< FAILURES!
[...truncated 162 lines...]
[junit4] JVM J1: stdout was not empty, see: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/temp/junit4-J1-20181007_100446_2613512726185708172650.sysout
[junit4] >>> JVM J1 emitted unexpected output (verbatim) ----
[junit4] java.lang.OutOfMemoryError: Java heap space
[junit4] Dumping heap to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/heapdumps/java_pid22089.hprof ...
[junit4] Heap dump file created [517335321 bytes in 1.654 secs]
[junit4] <<< JVM J1: EOF ----
[...truncated 8951 lines...]
BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/build.xml:651: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/build.xml:585: Some of the tests produced a heap dump, but did not fail. Maybe a suppressed OutOfMemoryError? Dumps created:
* java_pid22089.hprof
Total time: 294 minutes 31 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any