You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by Apache Jenkins Server <je...@builds.apache.org> on 2015/05/16 18:38:02 UTC
[JENKINS] Lucene-Solr-NightlyTests-5.x - Build # 849 - Still
Failing
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-5.x/849/
1 tests failed.
REGRESSION: org.apache.solr.cloud.CollectionsAPIDistributedZkTest.test
Error Message:
Captured an uncaught exception in thread: Thread[id=1555, name=collection1, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]
Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=1555, name=collection1, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]
Caused by: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at https://127.0.0.1:27649: Could not find collection : awholynewstresscollection_collection1_4
at __randomizedtesting.SeedInfo.seed([25585D626C46C0BA]:0)
at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:560)
at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:235)
at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:227)
at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:376)
at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:328)
at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1086)
at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:856)
at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:799)
at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1220)
at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:884)
Build Log:
[...truncated 10442 lines...]
[junit4] Suite: org.apache.solr.cloud.CollectionsAPIDistributedZkTest
[junit4] 2> Creating dataDir: /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J3/temp/solr.cloud.CollectionsAPIDistributedZkTest 25585D626C46C0BA-001/init-core-data-001
[junit4] 2> 275460 T991 oas.SolrTestCaseJ4.buildSSLConfig Randomized ssl (true) and clientAuth (false)
[junit4] 2> 275460 T991 oas.BaseDistributedSearchTestCase.initHostContext Setting hostContext system property: /
[junit4] 2> 275467 T991 oasc.ZkTestServer.run STARTING ZK TEST SERVER
[junit4] 2> 275468 T992 oasc.ZkTestServer$2$1.setClientPort client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 275468 T992 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
[junit4] 2> 275568 T991 oasc.ZkTestServer.run start zk server on port:27618
[junit4] 2> 275569 T991 oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default ZkCredentialsProvider
[junit4] 2> 275570 T991 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 275574 T999 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@13bf5121 name:ZooKeeperConnection Watcher:127.0.0.1:27618 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 275575 T991 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 275575 T991 oascc.SolrZkClient.createZkACLProvider Using default ZkACLProvider
[junit4] 2> 275575 T991 oascc.SolrZkClient.makePath makePath: /solr
[junit4] 2> 275578 T991 oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default ZkCredentialsProvider
[junit4] 2> 275579 T991 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 275580 T1002 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1288f074 name:ZooKeeperConnection Watcher:127.0.0.1:27618/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 275581 T991 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 275581 T991 oascc.SolrZkClient.createZkACLProvider Using default ZkACLProvider
[junit4] 2> 275581 T991 oascc.SolrZkClient.makePath makePath: /collections/collection1
[junit4] 2> 275583 T991 oascc.SolrZkClient.makePath makePath: /collections/collection1/shards
[junit4] 2> 275585 T991 oascc.SolrZkClient.makePath makePath: /collections/control_collection
[junit4] 2> 275586 T991 oascc.SolrZkClient.makePath makePath: /collections/control_collection/shards
[junit4] 2> 275588 T991 oasc.AbstractZkTestCase.putConfig put /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
[junit4] 2> 275589 T991 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.xml
[junit4] 2> 275592 T991 oasc.AbstractZkTestCase.putConfig put /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
[junit4] 2> 275593 T991 oascc.SolrZkClient.makePath makePath: /configs/conf1/schema.xml
[junit4] 2> 275596 T991 oasc.AbstractZkTestCase.putConfig put /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
[junit4] 2> 275596 T991 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.snippet.randomindexconfig.xml
[junit4] 2> 275599 T991 oasc.AbstractZkTestCase.putConfig put /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
[junit4] 2> 275599 T991 oascc.SolrZkClient.makePath makePath: /configs/conf1/stopwords.txt
[junit4] 2> 275602 T991 oasc.AbstractZkTestCase.putConfig put /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
[junit4] 2> 275602 T991 oascc.SolrZkClient.makePath makePath: /configs/conf1/protwords.txt
[junit4] 2> 275605 T991 oasc.AbstractZkTestCase.putConfig put /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
[junit4] 2> 275605 T991 oascc.SolrZkClient.makePath makePath: /configs/conf1/currency.xml
[junit4] 2> 275608 T991 oasc.AbstractZkTestCase.putConfig put /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
[junit4] 2> 275608 T991 oascc.SolrZkClient.makePath makePath: /configs/conf1/enumsConfig.xml
[junit4] 2> 275610 T991 oasc.AbstractZkTestCase.putConfig put /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
[junit4] 2> 275611 T991 oascc.SolrZkClient.makePath makePath: /configs/conf1/open-exchange-rates.json
[junit4] 2> 275613 T991 oasc.AbstractZkTestCase.putConfig put /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4] 2> 275614 T991 oascc.SolrZkClient.makePath makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4] 2> 275616 T991 oasc.AbstractZkTestCase.putConfig put /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
[junit4] 2> 275617 T991 oascc.SolrZkClient.makePath makePath: /configs/conf1/old_synonyms.txt
[junit4] 2> 275620 T991 oasc.AbstractZkTestCase.putConfig put /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
[junit4] 2> 275620 T991 oascc.SolrZkClient.makePath makePath: /configs/conf1/synonyms.txt
[junit4] 2> 275623 T991 oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default ZkCredentialsProvider
[junit4] 2> 275624 T991 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 275625 T1005 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@355688be name:ZooKeeperConnection Watcher:127.0.0.1:27618/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 275625 T991 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 275626 T991 oascc.SolrZkClient.createZkACLProvider Using default ZkACLProvider
[junit4] 2> 277132 T991 oas.SolrTestCaseJ4.writeCoreProperties Writing core.properties file to /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J3/temp/solr.cloud.CollectionsAPIDistributedZkTest 25585D626C46C0BA-001/control-001/cores/collection1
[junit4] 2> 277220 T991 oejs.Server.doStart jetty-9.2.10.v20150310
[junit4] 2> 277222 T991 oejsh.ContextHandler.doStart Started o.e.j.s.ServletContextHandler@32fbc784{/,null,AVAILABLE}
[junit4] 2> 277374 T991 oejs.AbstractConnector.doStart Started ServerConnector@6116dd53{SSL-http/1.1}{127.0.0.1:27629}
[junit4] 2> 277374 T991 oejs.Server.doStart Started @282461ms
[junit4] 2> 277375 T991 oascse.JettySolrRunner$1.lifeCycleStarted Jetty properties: {coreRootDirectory=/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J3/temp/solr.cloud.CollectionsAPIDistributedZkTest 25585D626C46C0BA-001/control-001/cores, hostPort=27629, hostContext=/}
[junit4] 2> 277420 T991 oass.SolrDispatchFilter.init SolrDispatchFilter.init()sun.misc.Launcher$AppClassLoader@17072b90
[junit4] 2> 277421 T991 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J3/temp/solr.cloud.CollectionsAPIDistributedZkTest 25585D626C46C0BA-001/control-001/'
[junit4] 2> 277469 T991 oasc.SolrXmlConfig.fromFile Loading container configuration from /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J3/temp/solr.cloud.CollectionsAPIDistributedZkTest 25585D626C46C0BA-001/control-001/solr.xml
[junit4] 2> 277534 T991 oasc.CorePropertiesLocator.<init> Config-defined core root directory: /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J3/temp/solr.cloud.CollectionsAPIDistributedZkTest 25585D626C46C0BA-001/control-001/cores
[junit4] 2> 277534 T991 oasc.CoreContainer.<init> New CoreContainer 882917919
[junit4] 2> 277534 T991 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J3/temp/solr.cloud.CollectionsAPIDistributedZkTest 25585D626C46C0BA-001/control-001/]
[junit4] 2> 277535 T991 oasc.CoreContainer.load loading shared library: /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J3/temp/solr.cloud.CollectionsAPIDistributedZkTest 25585D626C46C0BA-001/control-001/lib
[junit4] 2> 277535 T991 oasc.SolrResourceLoader.addToClassLoader WARN Can't find (or read) directory to add to classloader: lib (resolved as: /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J3/temp/solr.cloud.CollectionsAPIDistributedZkTest 25585D626C46C0BA-001/control-001/lib).
[junit4] 2> 277547 T991 oashc.HttpShardHandlerFactory.init created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
[junit4] 2> 277558 T991 oasu.UpdateShardHandler.<init> Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
[junit4] 2> 277562 T991 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
[junit4] 2> 277562 T991 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
[junit4] 2> 277562 T991 oasc.CoreContainer.load Node Name: 127.0.0.1
[junit4] 2> 277562 T991 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:27618/solr
[junit4] 2> 277562 T991 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4] 2> 277563 T991 oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default ZkCredentialsProvider
[junit4] 2> 277563 T991 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 277565 T1019 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@401e4ce0 name:ZooKeeperConnection Watcher:127.0.0.1:27618 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 277565 T991 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 277566 T991 oascc.SolrZkClient.createZkACLProvider Using default ZkACLProvider
[junit4] 2> 277567 T991 n:127.0.0.1:27629_ oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 277568 T1022 n:127.0.0.1:27629_ oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@773af7dc name:ZooKeeperConnection Watcher:127.0.0.1:27618/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 277569 T991 n:127.0.0.1:27629_ oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 277570 T991 n:127.0.0.1:27629_ oascc.SolrZkClient.makePath makePath: /overseer/queue
[junit4] 2> 277571 T991 n:127.0.0.1:27629_ oascc.SolrZkClient.makePath makePath: /overseer/collection-queue-work
[junit4] 2> 277573 T991 n:127.0.0.1:27629_ oascc.SolrZkClient.makePath makePath: /overseer/collection-map-running
[junit4] 2> 277575 T991 n:127.0.0.1:27629_ oascc.SolrZkClient.makePath makePath: /overseer/collection-map-completed
[junit4] 2> 277576 T991 n:127.0.0.1:27629_ oascc.SolrZkClient.makePath makePath: /overseer/collection-map-failure
[junit4] 2> 277579 T991 n:127.0.0.1:27629_ oascc.SolrZkClient.makePath makePath: /live_nodes
[junit4] 2> 277580 T991 n:127.0.0.1:27629_ oascc.SolrZkClient.makePath makePath: /aliases.json
[junit4] 2> 277581 T991 n:127.0.0.1:27629_ oascc.SolrZkClient.makePath makePath: /clusterstate.json
[junit4] 2> 277582 T991 n:127.0.0.1:27629_ oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:27629_
[junit4] 2> 277582 T991 n:127.0.0.1:27629_ oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:27629_
[junit4] 2> 277584 T991 n:127.0.0.1:27629_ oascc.SolrZkClient.makePath makePath: /overseer_elect
[junit4] 2> 277585 T991 n:127.0.0.1:27629_ oascc.SolrZkClient.makePath makePath: /overseer_elect/election
[junit4] 2> 277586 T991 n:127.0.0.1:27629_ oasc.Overseer.close Overseer (id=null) closing
[junit4] 2> 277587 T991 n:127.0.0.1:27629_ oasc.OverseerElectionContext.runLeaderProcess I am going to be the leader 127.0.0.1:27629_
[junit4] 2> 277587 T991 n:127.0.0.1:27629_ oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
[junit4] 2> 277588 T991 n:127.0.0.1:27629_ oasc.Overseer.start Overseer (id=93833746392416260-127.0.0.1:27629_-n_0000000000) starting
[junit4] 2> 277590 T991 n:127.0.0.1:27629_ oascc.SolrZkClient.makePath makePath: /overseer/queue-work
[junit4] 2> 277595 T991 n:127.0.0.1:27629_ oasc.OverseerAutoReplicaFailoverThread.<init> Starting OverseerAutoReplicaFailoverThread autoReplicaFailoverWorkLoopDelay=10000 autoReplicaFailoverWaitAfterExpiration=10000 autoReplicaFailoverBadNodeExpiration=60000
[junit4] 2> 277595 T1024 n:127.0.0.1:27629_ oasc.OverseerCollectionProcessor.run Process current queue of collection creations
[junit4] 2> 277595 T991 n:127.0.0.1:27629_ oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 277595 T1023 n:127.0.0.1:27629_ oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
[junit4] 2> 277598 T991 n:127.0.0.1:27629_ oasc.CoreContainer.intializeAuthorizationPlugin Security conf doesn't exist. Skipping setup for authorization module.
[junit4] 2> 277599 T991 n:127.0.0.1:27629_ oasc.CorePropertiesLocator.discover Looking for core definitions underneath /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J3/temp/solr.cloud.CollectionsAPIDistributedZkTest 25585D626C46C0BA-001/control-001/cores
[junit4] 2> 277602 T991 n:127.0.0.1:27629_ oasc.CoreDescriptor.<init> CORE DESCRIPTOR: {dataDir=data/, transient=false, coreNodeName=, collection=control_collection, config=solrconfig.xml, loadOnStartup=true, shard=, schema=schema.xml, absoluteInstDir=/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J3/temp/solr.cloud.CollectionsAPIDistributedZkTest 25585D626C46C0BA-001/control-001/cores/collection1/, name=collection1, instanceDir=/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J3/temp/solr.cloud.CollectionsAPIDistributedZkTest 25585D626C46C0BA-001/control-001/cores/collection1}
[junit4] 2> 277603 T991 n:127.0.0.1:27629_ oasc.CorePropertiesLocator.discoverUnder Found core collection1 in /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J3/temp/solr.cloud.CollectionsAPIDistributedZkTest 25585D626C46C0BA-001/control-001/cores/collection1/
[junit4] 2> 277603 T991 n:127.0.0.1:27629_ oasc.CorePropertiesLocator.discover Found 1 core definitions
[junit4] 2> 277604 T1026 n:127.0.0.1:27629_ c:control_collection x:collection1 oasc.ZkController.publish publishing core=collection1 state=down collection=control_collection
[junit4] 2> 277604 T1026 n:127.0.0.1:27629_ c:control_collection x:collection1 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 277605 T1026 n:127.0.0.1:27629_ oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4] 2> 277605 T1022 n:127.0.0.1:27629_ oasc.DistributedQueue$LatchWatcher.process NodeChildrenChanged fired on path /overseer/queue state SyncConnected
[junit4] 2> 277606 T1023 n:127.0.0.1:27629_ oasc.Overseer$ClusterStateUpdater.run processMessage: queueSize: 1, message = {
[junit4] 2> "roles":null,
[junit4] 2> "state":"down",
[junit4] 2> "numShards":"1",
[junit4] 2> "collection":"control_collection",
[junit4] 2> "operation":"state",
[junit4] 2> "node_name":"127.0.0.1:27629_",
[junit4] 2> "shard":null,
[junit4] 2> "core":"collection1",
[junit4] 2> "base_url":"https://127.0.0.1:27629"} current state version: 0
[junit4] 2> 277606 T1023 n:127.0.0.1:27629_ oasco.ReplicaMutator.updateState Update state numShards=1 message={
[junit4] 2> "roles":null,
[junit4] 2> "state":"down",
[junit4] 2> "numShards":"1",
[junit4] 2> "collection":"control_collection",
[junit4] 2> "operation":"state",
[junit4] 2> "node_name":"127.0.0.1:27629_",
[junit4] 2> "shard":null,
[junit4] 2> "core":"collection1",
[junit4] 2> "base_url":"https://127.0.0.1:27629"}
[junit4] 2> 277607 T1023 n:127.0.0.1:27629_ oasco.ClusterStateMutator.createCollection building a new cName: control_collection
[junit4] 2> 277607 T1023 n:127.0.0.1:27629_ oasco.ReplicaMutator.updateState Assigning new node to shard shard=shard1
[junit4] 2> 277608 T1022 n:127.0.0.1:27629_ oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 278605 T1026 n:127.0.0.1:27629_ oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
[junit4] 2> 278606 T1026 n:127.0.0.1:27629_ oasc.ZkController.createCollectionZkNode Check for collection zkNode:control_collection
[junit4] 2> 278606 T1026 n:127.0.0.1:27629_ oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4] 2> 278607 T1026 n:127.0.0.1:27629_ oascc.ZkStateReader.readConfigName Load collection config from:/collections/control_collection
[junit4] 2> 278607 T1026 n:127.0.0.1:27629_ oascc.ZkStateReader.readConfigName path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
[junit4] 2> 278607 T1026 n:127.0.0.1:27629_ oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J3/temp/solr.cloud.CollectionsAPIDistributedZkTest 25585D626C46C0BA-001/control-001/cores/collection1/'
[junit4] 2> 278623 T1026 n:127.0.0.1:27629_ oasc.Config.<init> loaded config solrconfig.xml with version 0
[junit4] 2> 278633 T1026 n:127.0.0.1:27629_ oasc.SolrConfig.refreshRequestParams current version of requestparams : -1
[junit4] 2> 278653 T1026 n:127.0.0.1:27629_ oasc.SolrConfig.<init> Using Lucene MatchVersion: 5.2.0
[junit4] 2> 278693 T1026 n:127.0.0.1:27629_ oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4] 2> 278795 T1026 n:127.0.0.1:27629_ oass.IndexSchema.readSchema Reading Solr Schema from /configs/conf1/schema.xml
[junit4] 2> 278807 T1026 n:127.0.0.1:27629_ oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4] 2> 279213 T1026 n:127.0.0.1:27629_ oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4] 2> 279231 T1026 n:127.0.0.1:27629_ oass.IndexSchema.readSchema default search field in schema is text
[junit4] 2> 279234 T1026 n:127.0.0.1:27629_ oass.IndexSchema.readSchema unique key field: id
[junit4] 2> 279245 T1026 n:127.0.0.1:27629_ oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 279250 T1026 n:127.0.0.1:27629_ oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 279254 T1026 n:127.0.0.1:27629_ oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4] 2> 279255 T1026 n:127.0.0.1:27629_ oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4] 2> 279255 T1026 n:127.0.0.1:27629_ oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4] 2> 279256 T1026 n:127.0.0.1:27629_ oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4] 2> 279256 T1026 n:127.0.0.1:27629_ oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4] 2> 279257 T1026 n:127.0.0.1:27629_ oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4] 2> 279257 T1026 n:127.0.0.1:27629_ oasc.CoreContainer.create Creating SolrCore 'collection1' using configuration from collection control_collection
[junit4] 2> 279257 T1026 n:127.0.0.1:27629_ x:collection1 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4] 2> 279258 T1026 n:127.0.0.1:27629_ x:collection1 oasc.SolrCore.<init> [[collection1] ] Opening new SolrCore at [/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J3/temp/solr.cloud.CollectionsAPIDistributedZkTest 25585D626C46C0BA-001/control-001/cores/collection1/], dataDir=[null]
[junit4] 2> 279258 T1026 n:127.0.0.1:27629_ x:collection1 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@580a298b
[junit4] 2> 279259 T1026 n:127.0.0.1:27629_ x:collection1 oasc.CachingDirectoryFactory.get return new directory for /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J3/temp/solr.cloud.CollectionsAPIDistributedZkTest 25585D626C46C0BA-001/control-001/cores/collection1/data
[junit4] 2> 279259 T1026 n:127.0.0.1:27629_ x:collection1 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J3/temp/solr.cloud.CollectionsAPIDistributedZkTest 25585D626C46C0BA-001/control-001/cores/collection1/data/index/
[junit4] 2> 279259 T1026 n:127.0.0.1:27629_ x:collection1 oasc.SolrCore.initIndex WARN [collection1] Solr index directory '/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J3/temp/solr.cloud.CollectionsAPIDistributedZkTest 25585D626C46C0BA-001/control-001/cores/collection1/data/index' doesn't exist. Creating new index...
[junit4] 2> 279260 T1026 n:127.0.0.1:27629_ x:collection1 oasc.CachingDirectoryFactory.get return new directory for /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J3/temp/solr.cloud.CollectionsAPIDistributedZkTest 25585D626C46C0BA-001/control-001/cores/collection1/data/index
[junit4] 2> 279260 T1026 n:127.0.0.1:27629_ x:collection1 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.MockRandomMergePolicy: org.apache.lucene.index.MockRandomMergePolicy@1c194e76
[junit4] 2> 279261 T1026 n:127.0.0.1:27629_ x:collection1 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4] 2> commit{dir=MockDirectoryWrapper(RAMDirectory@416e3363 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@29efca24),segFN=segments_1,generation=1}
[junit4] 2> 279261 T1026 n:127.0.0.1:27629_ x:collection1 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 279268 T1026 n:127.0.0.1:27629_ x:collection1 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "nodistrib"
[junit4] 2> 279268 T1026 n:127.0.0.1:27629_ x:collection1 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe"
[junit4] 2> 279269 T1026 n:127.0.0.1:27629_ x:collection1 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4] 2> 279269 T1026 n:127.0.0.1:27629_ x:collection1 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "stored_sig"
[junit4] 2> 279269 T1026 n:127.0.0.1:27629_ x:collection1 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4] 2> 279270 T1026 n:127.0.0.1:27629_ x:collection1 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
[junit4] 2> 279270 T1026 n:127.0.0.1:27629_ x:collection1 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 279270 T1026 n:127.0.0.1:27629_ x:collection1 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 279271 T1026 n:127.0.0.1:27629_ x:collection1 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
[junit4] 2> 279294 T1026 n:127.0.0.1:27629_ x:collection1 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 279297 T1026 n:127.0.0.1:27629_ x:collection1 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 279299 T1026 n:127.0.0.1:27629_ x:collection1 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 279302 T1026 n:127.0.0.1:27629_ x:collection1 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 279308 T1026 n:127.0.0.1:27629_ x:collection1 oasc.RequestHandlers.initHandlersFromConfig Registered paths: /admin/threads,/replication,/config,/admin/segments,/admin/plugins,/admin/mbeans,/admin/luke,/admin/properties,/admin/system,/update/json,/admin/logging,/update/csv,/admin/file,/update,standard,/update/json/docs,/admin/ping,/schema,/get
[junit4] 2> 279310 T1026 n:127.0.0.1:27629_ x:collection1 oasc.SolrCore.initStatsCache Using default statsCache cache: org.apache.solr.search.stats.LocalStatsCache
[junit4] 2> 279312 T1026 n:127.0.0.1:27629_ x:collection1 oasu.UpdateHandler.<init> Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 279312 T1026 n:127.0.0.1:27629_ x:collection1 oasu.UpdateLog.init Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10
[junit4] 2> 279314 T1026 n:127.0.0.1:27629_ x:collection1 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4] 2> 279314 T1026 n:127.0.0.1:27629_ x:collection1 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4] 2> 279315 T1026 n:127.0.0.1:27629_ x:collection1 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=9, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.2682451494377761]
[junit4] 2> 279316 T1026 n:127.0.0.1:27629_ x:collection1 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4] 2> commit{dir=MockDirectoryWrapper(RAMDirectory@416e3363 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@29efca24),segFN=segments_1,generation=1}
[junit4] 2> 279316 T1026 n:127.0.0.1:27629_ x:collection1 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 279316 T1026 n:127.0.0.1:27629_ x:collection1 oass.SolrIndexSearcher.<init> Opening Searcher@219b66f[collection1] main
[junit4] 2> 279317 T1026 n:127.0.0.1:27629_ x:collection1 oascc.ZkStateReader.readConfigName Load collection config from:/collections/control_collection
[junit4] 2> 279318 T1026 n:127.0.0.1:27629_ x:collection1 oascc.ZkStateReader.readConfigName path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
[junit4] 2> 279318 T1026 n:127.0.0.1:27629_ x:collection1 oasr.ManagedResourceStorage.newStorageIO Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
[junit4] 2> 279318 T1026 n:127.0.0.1:27629_ x:collection1 oasr.ManagedResourceStorage$ZooKeeperStorageIO.configure Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 279319 T1026 n:127.0.0.1:27629_ x:collection1 oasr.RestManager.init Initializing RestManager with initArgs: {}
[junit4] 2> 279319 T1026 n:127.0.0.1:27629_ x:collection1 oasr.ManagedResourceStorage.load Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 279319 T1026 n:127.0.0.1:27629_ x:collection1 oasr.ManagedResourceStorage$ZooKeeperStorageIO.openInputStream No data found for znode /configs/conf1/_rest_managed.json
[junit4] 2> 279320 T1026 n:127.0.0.1:27629_ x:collection1 oasr.ManagedResourceStorage.load Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 279320 T1026 n:127.0.0.1:27629_ x:collection1 oasr.RestManager.init Initializing 0 registered ManagedResources
[junit4] 2> 279320 T1026 n:127.0.0.1:27629_ x:collection1 oash.ReplicationHandler.inform Commits will be reserved for 10000
[junit4] 2> 279321 T1027 n:127.0.0.1:27629_ x:collection1 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@219b66f[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 279322 T1026 n:127.0.0.1:27629_ x:collection1 oasc.ZkController.getConfDirListeners watch zkdir /configs/conf1
[junit4] 2> 279322 T1026 n:127.0.0.1:27629_ x:collection1 oasc.CoreContainer.registerCore registering core: collection1
[junit4] 2> 279323 T1030 n:127.0.0.1:27629_ c:control_collection s:shard1 x:collection1 oasc.ZkController.register Register replica - core:collection1 address:https://127.0.0.1:27629 collection:control_collection shard:shard1
[junit4] 2> 279324 T991 n:127.0.0.1:27629_ oass.SolrDispatchFilter.init user.dir=/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J3
[junit4] 2> 279324 T991 n:127.0.0.1:27629_ oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4] 2> 279324 T1030 n:127.0.0.1:27629_ c:control_collection s:shard1 x:collection1 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leader_elect/shard1/election
[junit4] 2> 279328 T1030 n:127.0.0.1:27629_ c:control_collection s:shard1 x:collection1 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard1
[junit4] 2> 279329 T1022 n:127.0.0.1:27629_ oasc.DistributedQueue$LatchWatcher.process NodeChildrenChanged fired on path /overseer/queue state SyncConnected
[junit4] 2> 279329 T1030 n:127.0.0.1:27629_ c:control_collection s:shard1 x:collection1 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4] 2> 279330 T1030 n:127.0.0.1:27629_ c:control_collection s:shard1 x:collection1 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4] 2> ASYNC NEW_CORE C159 name=collection1 org.apache.solr.core.SolrCore@5b72de50 url=https://127.0.0.1:27629/collection1 node=127.0.0.1:27629_ C159_STATE=coll:control_collection core:collection1 props:{state=down, node_name=127.0.0.1:27629_, core=collection1, base_url=https://127.0.0.1:27629}
[junit4] 2> 279330 T1030 n:127.0.0.1:27629_ c:control_collection s:shard1 x:collection1 C159 oasc.SyncStrategy.sync Sync replicas to https://127.0.0.1:27629/collection1/
[junit4] 2> 279330 T1030 n:127.0.0.1:27629_ c:control_collection s:shard1 x:collection1 C159 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4] 2> 279330 T1023 n:127.0.0.1:27629_ oasc.Overseer$ClusterStateUpdater.run processMessage: queueSize: 1, message = {
[junit4] 2> "operation":"leader",
[junit4] 2> "shard":"shard1",
[junit4] 2> "collection":"control_collection"} current state version: 1
[junit4] 2> 279330 T1030 n:127.0.0.1:27629_ c:control_collection s:shard1 x:collection1 C159 oasc.SyncStrategy.syncToMe https://127.0.0.1:27629/collection1/ has no replicas
[junit4] 2> 279331 T1030 n:127.0.0.1:27629_ c:control_collection s:shard1 x:collection1 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: https://127.0.0.1:27629/collection1/ shard1
[junit4] 2> 279331 T1030 n:127.0.0.1:27629_ c:control_collection s:shard1 x:collection1 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leaders/shard1
[junit4] 2> 279331 T991 oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default ZkCredentialsProvider
[junit4] 2> 279332 T1022 n:127.0.0.1:27629_ oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 279332 T991 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 279333 T1033 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@cd52850 name:ZooKeeperConnection Watcher:127.0.0.1:27618/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 279334 T991 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 279334 T991 oascc.SolrZkClient.createZkACLProvider Using default ZkACLProvider
[junit4] 2> 279335 T991 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 279335 T1023 n:127.0.0.1:27629_ oasc.Overseer$ClusterStateUpdater.run processMessage: queueSize: 1, message = {
[junit4] 2> "operation":"leader",
[junit4] 2> "shard":"shard1",
[junit4] 2> "collection":"control_collection",
[junit4] 2> "base_url":"https://127.0.0.1:27629",
[junit4] 2> "core":"collection1",
[junit4] 2> "state":"active"} current state version: 2
[junit4] 2> 279337 T1022 n:127.0.0.1:27629_ oasc.DistributedQueue$LatchWatcher.process NodeChildrenChanged fired on path /overseer/queue state SyncConnected
[junit4] 2> 279337 T991 oasc.ChaosMonkey.monkeyLog monkey: init - expire sessions:false cause connection loss:false
[junit4] 2> 279338 T1033 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 279338 T1022 n:127.0.0.1:27629_ oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 279386 T1030 n:127.0.0.1:27629_ c:control_collection s:shard1 x:collection1 oasc.ZkController.register We are https://127.0.0.1:27629/collection1/ and leader is https://127.0.0.1:27629/collection1/
[junit4] 2> 279387 T1030 n:127.0.0.1:27629_ c:control_collection s:shard1 x:collection1 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=https://127.0.0.1:27629
[junit4] 2> 279387 T1030 n:127.0.0.1:27629_ c:control_collection s:shard1 x:collection1 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4] 2> 279387 T1030 n:127.0.0.1:27629_ c:control_collection s:shard1 x:collection1 oasc.ZkController.publish publishing core=collection1 state=active collection=control_collection
[junit4] 2> 279388 T1030 n:127.0.0.1:27629_ c:control_collection s:shard1 x:collection1 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 279389 T1022 n:127.0.0.1:27629_ oasc.DistributedQueue$LatchWatcher.process NodeChildrenChanged fired on path /overseer/queue state SyncConnected
[junit4] 2> 279390 T1023 n:127.0.0.1:27629_ oasc.Overseer$ClusterStateUpdater.run processMessage: queueSize: 1, message = {
[junit4] 2> "roles":null,
[junit4] 2> "state":"active",
[junit4] 2> "core_node_name":"core_node1",
[junit4] 2> "numShards":"2",
[junit4] 2> "collection":"control_collection",
[junit4] 2> "operation":"state",
[junit4] 2> "node_name":"127.0.0.1:27629_",
[junit4] 2> "shard":"shard1",
[junit4] 2> "core":"collection1",
[junit4] 2> "base_url":"https://127.0.0.1:27629"} current state version: 3
[junit4] 2> 279391 T1023 n:127.0.0.1:27629_ oasco.ReplicaMutator.updateState Update state numShards=2 message={
[junit4] 2> "roles":null,
[junit4] 2> "state":"active",
[junit4] 2> "core_node_name":"core_node1",
[junit4] 2> "numShards":"2",
[junit4] 2> "collection":"control_collection",
[junit4] 2> "operation":"state",
[junit4] 2> "node_name":"127.0.0.1:27629_",
[junit4] 2> "shard":"shard1",
[junit4] 2> "core":"collection1",
[junit4] 2> "base_url":"https://127.0.0.1:27629"}
[junit4] 2> 279494 T1022 n:127.0.0.1:27629_ oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 279494 T1033 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 281410 T991 oas.SolrTestCaseJ4.writeCoreProperties Writing core.properties file to /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J3/temp/solr.cloud.CollectionsAPIDistributedZkTest 25585D626C46C0BA-001/shard-1-001/cores/collection1
[junit4] 2> 281415 T991 oasc.AbstractFullDistribZkTestBase.createJettys create jetty 1 in directory /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J3/temp/solr.cloud.CollectionsAPIDistributedZkTest 25585D626C46C0BA-001/shard-1-001
[junit4] 2> 281416 T991 oejs.Server.doStart jetty-9.2.10.v20150310
[junit4] 2> 281419 T991 oejsh.ContextHandler.doStart Started o.e.j.s.ServletContextHandler@56b30577{/,null,AVAILABLE}
[junit4] 2> 281430 T991 oejs.AbstractConnector.doStart Started ServerConnector@664968be{SSL-http/1.1}{127.0.0.1:27649}
[junit4] 2> 281430 T991 oejs.Server.doStart Started @286517ms
[junit4] 2> 281431 T991 oascse.JettySolrRunner$1.lifeCycleStarted Jetty properties: {hostContext=/, hostPort=27649, solrconfig=solrconfig.xml, coreRootDirectory=/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J3/../../../../../../../../../../../usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J3/temp/solr.cloud.CollectionsAPIDistributedZkTest 25585D626C46C0BA-001/shard-1-001/cores}
[junit4] 2> 281436 T991 oass.SolrDispatchFilter.init SolrDispatchFilter.init()sun.misc.Launcher$AppClassLoader@17072b90
[junit4] 2> 281436 T991 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J3/../../../../../../../../../../../usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J3/temp/solr.cloud.CollectionsAPIDistributedZkTest 25585D626C46C0BA-001/shard-1-001/'
[junit4] 2> 281505 T991 oasc.SolrXmlConfig.fromFile Loading container configuration from /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J3/../../../../../../../../../../../usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J3/temp/solr.cloud.CollectionsAPIDistributedZkTest 25585D626C46C0BA-001/shard-1-001/solr.xml
[junit4] 2> 281559 T991 oasc.CorePropertiesLocator.<init> Config-defined core root directory: /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J3/../../../../../../../../../../../usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J3/temp/solr.cloud.CollectionsAPIDistributedZkTest 25585D626C46C0BA-001/shard-1-001/cores
[junit4] 2> 281560 T991 oasc.CoreContainer.<init> New CoreContainer 1903129407
[junit4] 2> 281560 T991 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J3/../../../../../../../../../../../usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J3/temp/solr.cloud.CollectionsAPIDistributedZkTest 25585D626C46C0BA-001/shard-1-001/]
[junit4] 2> 281561 T991 oasc.CoreContainer.load loading shared library: /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J3/../../../../../../../../../../../usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J3/temp/solr.cloud.CollectionsAPIDistributedZkTest 25585D626C46C0BA-001/shard-1-001/lib
[junit4] 2> 281562 T991 oasc.SolrResourceLoader.addToClassLoader WARN Can't find (or read) directory to add to classloader: lib (resolved as: /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J3/../../../../../../../../../../../usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J3/temp/solr.cloud.CollectionsAPIDistributedZkTest 25585D626C46C0BA-001/shard-1-001/lib).
[junit4] 2> 281575 T991 oashc.HttpShardHandlerFactory.init created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
[junit4] 2> 281586 T991 oasu.UpdateShardHandler.<init> Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
[junit4] 2> 281590 T991 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
[junit4] 2> 281591 T991 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
[junit4] 2> 281591 T991 oasc.CoreContainer.load Node Name: 127.0.0.1
[junit4] 2> 281591 T991 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:27618/solr
[junit4] 2> 281592 T991 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4] 2> 281592 T991 oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default ZkCredentialsProvider
[junit4] 2> 281593 T991 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 281595 T1047 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@4b41d9eb name:ZooKeeperConnection Watcher:127.0.0.1:27618 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 281596 T991 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 281597 T991 oascc.SolrZkClient.createZkACLProvider Using default ZkACLProvider
[junit4] 2> 281598 T991 n:127.0.0.1:27649_ oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 281600 T1050 n:127.0.0.1:27649_ oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@31e4f732 name:ZooKeeperConnection Watcher:127.0.0.1:27618/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 281600 T991 n:127.0.0.1:27649_ oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 281605 T991 n:127.0.0.1:27649_ oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 282610 T991 n:127.0.0.1:27649_ oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:27649_
[junit4] 2> 282611 T991 n:127.0.0.1:27649_ oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:27649_
[junit4] 2> 282613 T991 n:127.0.0.1:27649_ oasc.Overseer.close Overseer (id=null) closing
[junit4] 2> 282615 T991 n:127.0.0.1:27649_ oasc.CoreContainer.intializeAuthorizationPlugin Security conf doesn't exist. Skipping setup for authorization module.
[junit4] 2> 282616 T991 n:127.0.0.1:27649_ oasc.CorePropertiesLocator.discover Looking for core definitions underneath /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J3/../../../../../../../../../../../usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J3/temp/solr.cloud.CollectionsAPIDistributedZkTest 25585D626C46C0BA-001/shard-1-001/cores
[junit4] 2> 282620 T991 n:127.0.0.1:27649_ oasc.CoreDescriptor.<init> CORE DESCRIPTOR: {name=collection1, absoluteInstDir=/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J3/../../../../../../../../../../../usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J3/temp/solr.cloud.CollectionsAPIDistributedZkTest 25585D626C46C0BA-001/shard-1-001/cores/collection1/, shard=, collection=collection1, config=solrconfig.xml, schema=schema.xml, dataDir=data/, loadOnStartup=true, transient=false, coreNodeName=, instanceDir=/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J3/../../../../../../../../../../../usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J3/temp/solr.cloud.CollectionsAPIDistributedZkTest 25585D626C46C0BA-001/shard-1-001/cores/collection1}
[junit4] 2> 282620 T991 n:127.0.0.1:27649_ oasc.CorePropertiesLocator.discoverUnder Found core collection1 in /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J3/../../../../../../../../../../../usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J3/temp/solr.cloud.CollectionsAPIDistributedZkTest 25585D626C46C0BA-001/shard-1-001/cores/collection1/
[junit4] 2> 282621 T991 n:127.0.0.1:27649_ oasc.CorePropertiesLocator.discover Found 1 core definitions
[junit4] 2> 282622 T1051 n:127.0.0.1:27649_ c:collection1 x:collection1 oasc.ZkController.publish publishing core=collection1 state=down collection=collection1
[junit4] 2> 282622 T1051 n:127.0.0.1:27649_ c:collection1 x:collection1 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 282623 T1051 n:127.0.0.1:27649_ oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4] 2> 282623 T1022 n:127.0.0.1:27629_ oasc.DistributedQueue$LatchWatcher.process NodeChildrenChanged fired on path /overseer/queue state SyncConnected
[junit4] 2> 282624 T1023 n:127.0.0.1:27629_ oasc.Overseer$ClusterStateUpdater.run processMessage: queueSize: 1, message = {
[junit4] 2> "roles":null,
[junit4] 2> "state":"down",
[junit4] 2> "numShards":"2",
[junit4] 2> "collection":"collection1",
[junit4] 2> "operation":"state",
[junit4] 2> "node_name":"127.0.0.1:27649_",
[junit4] 2> "shard":null,
[junit4] 2> "core":"collection1",
[junit4] 2> "base_url":"https://127.0.0.1:27649"} current state version: 4
[junit4] 2> 282625 T1023 n:127.0.0.1:27629_ oasco.ReplicaMutator.updateState Update state numShards=2 message={
[junit4] 2> "roles":null,
[junit4] 2> "state":"down",
[junit4] 2> "numShards":"2",
[junit4] 2> "collection":"collection1",
[junit4] 2> "operation":"state",
[junit4] 2> "node_name":"127.0.0.1:27649_",
[junit4] 2> "shard":null,
[junit4] 2> "core":"collection1",
[junit4] 2> "base_url":"https://127.0.0.1:27649"}
[junit4] 2> 282625 T1023 n:127.0.0.1:27629_ oasco.ClusterStateMutator.createCollection building a new cName: collection1
[junit4] 2> 282625 T1023 n:127.0.0.1:27629_ oasco.ReplicaMutator.updateState Assigning new node to shard shard=shard2
[junit4] 2> 282626 T1050 n:127.0.0.1:27649_ oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 282626 T1022 n:127.0.0.1:27629_ oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 282626 T1033 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 283623 T1051 n:127.0.0.1:27649_ oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
[junit4] 2> 283624 T1051 n:127.0.0.1:27649_ oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4] 2> 283625 T1051 n:127.0.0.1:27649_ oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4] 2> 283625 T1051 n:127.0.0.1:27649_ oascc.ZkStateReader.readConfigName Load collection config from:/collections/collection1
[junit4] 2> 283626 T1051 n:127.0.0.1:27649_ oascc.ZkStateReader.readConfigName path=/collections/collection1 configName=conf1 specified config exists in ZooKeeper
[junit4] 2> 283626 T1051 n:127.0.0.1:27649_ oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J3/../../../../../../../../../../../usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J3/temp/solr.cloud.CollectionsAPIDistributedZkTest 25585D626C46C0BA-001/shard-1-001/cores/collection1/'
[junit4] 2> 283646 T1051 n:127.0.0.1:27649_ oasc.Config.<init> loaded config solrconfig.xml with version 0
[junit4] 2> 283658 T1051 n:127.0.0.1:27649_ oasc.SolrConfig.refreshRequestParams current version of requestparams : -1
[junit4] 2> 283694 T1051 n:127.0.0.1:27649_ oasc.SolrConfig.<init> Using Lucene MatchVersion: 5.2.0
[junit4] 2> 283741 T1051 n:127.0.0.1:27649_ oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4] 2> 283843 T1051 n:127.0.0.1:27649_ oass.IndexSchema.readSchema Reading Solr Schema from /configs/conf1/schema.xml
[junit4] 2> 283856 T1051 n:127.0.0.1:27649_ oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4] 2> 284269 T1051 n:127.0.0.1:27649_ oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4] 2> 284287 T1051 n:127.0.0.1:27649_ oass.IndexSchema.readSchema default search field in schema is text
[junit4] 2> 284291 T1051 n:127.0.0.1:27649_ oass.IndexSchema.readSchema unique key field: id
[junit4] 2> 284303 T1051 n:127.0.0.1:27649_ oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 284308 T1051 n:127.0.0.1:27649_ oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 284312 T1051 n:127.0.0.1:27649_ oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4] 2> 284313 T1051 n:127.0.0.1:27649_ oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4] 2> 284314 T1051 n:127.0.0.1:27649_ oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4] 2> 284314 T1051 n:127.0.0.1:27649_ oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4] 2> 284315 T1051 n:127.0.0.1:27649_ oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4] 2> 284315 T1051 n:127.0.0.1:27649_ oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4] 2> 284315 T1051 n:127.0.0.1:27649_ oasc.CoreContainer.create Creating SolrCore 'collection1' using configuration from collection collection1
[junit4] 2> 284316 T1051 n:127.0.0.1:27649_ x:collection1 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4] 2> 284316 T1051 n:127.0.0.1:27649_ x:collection1 oasc.SolrCore.<init> [[collection1] ] Opening new SolrCore at [/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J3/../../../../../../../../../../../usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J3/temp/solr.cloud.CollectionsAPIDistributedZkTest 25585D626C46C0BA-001/shard-1-001/cores/collection1/], dataDir=[null]
[junit4] 2> 284316 T1051 n:127.0.0.1:27649_ x:collection1 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@580a298b
[junit4] 2> 284317 T1051 n:127.0.0.1:27649_ x:collection1 oasc.CachingDirectoryFactory.get return new directory for /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J3/../../../../../../../../../../../usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J3/temp/solr.cloud.CollectionsAPIDistributedZkTest 25585D626C46C0BA-001/shard-1-001/cores/collection1/data
[junit4] 2> 284318 T1051 n:127.0.0.1:27649_ x:collection1 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J3/../../../../../../../../../../../usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J3/temp/solr.cloud.CollectionsAPIDistributedZkTest 25585D626C46C0BA-001/shard-1-001/cores/collection1/data/index/
[junit4] 2> 284318 T1051 n:127.0.0.1:27649_ x:collection1 oasc.SolrCore.initIndex WARN [collection1] Solr index directory '/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J3/../../../../../../../../../../../usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J3/temp/solr.cloud.CollectionsAPIDistributedZkTest 25585D626C46C0BA-001/shard-1-001/cores/collection1/data/index' doesn't exist. Creating new index...
[junit4] 2> 284318 T1051 n:127.0.0.1:27649_ x:collection1 oasc.CachingDirectoryFactory.get return new directory for /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J3/../../../../../../../../../../../usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J3/temp/solr.cloud.CollectionsAPIDistributedZkTest 25585D626C46C0BA-001/shard-1-001/cores/collection1/data/index
[junit4] 2> 284319 T1051 n:127.0.0.1:27649_ x:collection1 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.MockRandomMergePolicy: org.apache.lucene.index.MockRandomMergePolicy@60206c1
[junit4] 2> 284320 T1051 n:127.0.0.1:27649_ x:collection1 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4] 2> commit{dir=MockDirectoryWrapper(RAMDirectory@6f096bfb lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@539064b0),segFN=segments_1,generation=1}
[junit4] 2> 284320 T1051 n:127.0.0.1:27649_ x:collection1 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 284328 T1051 n:127.0.0.1:27649_ x:collection1 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "nodistrib"
[junit4] 2> 284329 T1051 n:127.0.0.1:27649_ x:collection1 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe"
[junit4] 2> 284329 T1051 n:127.0.0.1:27649_ x:collection1 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4] 2> 284329 T1051 n:127.0.0.1:27649_ x:collection1 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "stored_sig"
[junit4] 2> 284330 T1051 n:127.0.0.1:27649_ x:collection1 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4] 2> 284330 T1051 n:127.0.0.1:27649_ x:collection1 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
[junit4] 2> 284330 T1051 n:127.0.0.1:27649_ x:collection1 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 284331 T1051 n:127.0.0.1:27649_ x:collection1 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 284331 T1051 n:127.0.0.1:27649_ x:collection1 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
[junit4] 2> 284350 T1051 n:127.0.0.1:27649_ x:collection1 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 284353 T1051 n:127.0.0.1:27649_ x:collection1 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 284356 T1051 n:127.0.0.1:27649_ x:collection1 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 284358 T1051 n:127.0.0.1:27649_ x:collection1 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 284366 T1051 n:127.0.0.1:27649_ x:collection1 oasc.RequestHandlers.initHandlersFromConfig Registered paths: /admin/threads,/replication,/config,/admin/segments,/admin/plugins,/admin/mbeans,/admin/luke,/admin/properties,/admin/system,/update/json,/admin/logging,/update/csv,/admin/file,/update,standard,/update/json/docs,/admin/ping,/schema,/get
[junit4] 2> 284369 T1051 n:127.0.0.1:27649_ x:collection1 oasc.SolrCore.initStatsCache Using default statsCache cache: org.apache.solr.search.stats.LocalStatsCache
[junit4] 2> 284371 T1051 n:127.0.0.1:27649_ x:collection1 oasu.UpdateHandler.<init> Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 284371 T1051 n:127.0.0.1:27649_ x:collection1 oasu.UpdateLog.init Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10
[junit4] 2> 284374 T1051 n:127.0.0.1:27649_ x:collection1 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4] 2> 284374 T1051 n:127.0.0.1:27649_ x:collection1 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4] 2> 284375 T1051 n:127.0.0.1:27649_ x:collection1 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=9, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.2682451494377761]
[junit4] 2> 284376 T1051 n:127.0.0.1:27649_ x:collection1 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4] 2> commit{dir=MockDirectoryWrapper(RAMDirectory@6f096bfb lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@539064b0),segFN=segments_1,generation=1}
[junit4] 2> 284376 T1051 n:127.0.0.1:27649_ x:collection1 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 284376 T1051 n:127.0.0.1:27649_ x:collection1 oass.SolrIndexSearcher.<init> Opening Searcher@697d227f[collection1] main
[junit4] 2> 284377 T1051 n:127.0.0.1:27649_ x:collection1 oascc.ZkStateReader.readConfigName Load collection config from:/collections/collection1
[junit4] 2> 284378 T1051 n:127.0.0.1:27649_ x:collection1 oascc.ZkStateReader.readConfigName path=/collections/collection1 configName=conf1 specified config exists in ZooKeeper
[junit4] 2> 284378 T1051 n:127.0.0.1:27649_ x:collection1 oasr.ManagedResourceStorage.newStorageIO Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
[junit4] 2> 284378 T1051 n:127.0.0.1:27649_ x:collection1 oasr.ManagedResourceStorage$ZooKeeperStorageIO.configure Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 284379 T1051 n:127.0.0.1:27649_ x:collection1 oasr.RestManager.init Initializing RestManager with initArgs: {}
[junit4] 2> 284379 T1051 n:127.0.0.1:27649_ x:collection1 oasr.ManagedResourceStorage.load Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 284379 T1051 n:127.0.0.1:27649_ x:collection1 oasr.ManagedResourceStorage$ZooKeeperStorageIO.openInputStream No data found for znode /configs/conf1/_rest_managed.json
[junit4] 2> 284380 T1051 n:127.0.0.1:27649_ x:collection1 oasr.ManagedResourceStorage.load Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 284380 T1051 n:127.0.0.1:27649_ x:collection1 oasr.RestManager.init Initializing 0 registered ManagedResources
[junit4] 2> 284380 T1051 n:127.0.0.1:27649_ x:collection1 oash.ReplicationHandler.inform Commits will be reserved for 10000
[junit4] 2> 284381 T1052 n:127.0.0.1:27649_ x:collection1 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@697d227f[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 284382 T1051 n:127.0.0.1:27649_ x:collection1 oasc.ZkController.getConfDirListeners watch zkdir /configs/conf1
[junit4] 2> 284382 T1051 n:127.0.0.1:27649_ x:collection1 oasc.CoreContainer.registerCore registering core: collection1
[junit4] 2> 284383 T1055 n:127.0.0.1:27649_ c:collection1 s:shard2 x:collection1 oasc.ZkController.register Register replica - core:collection1 address:https://127.0.0.1:27649 collection:collection1 shard:shard2
[junit4] 2> 284383 T991 n:127.0.0.1:27649_ oass.SolrDispatchFilter.init user.dir=/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J3
[junit4] 2> 284384 T991 n:127.0.0.1:27649_ oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4] 2> 284384 T1055 n:127.0.0.1:27649_ c:collection1 s:shard2 x:collection1 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard2/election
[junit4] 2> 284388 T1055 n:127.0.0.1:27649_ c:collection1 s:shard2 x:collection1 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard2
[junit4] 2> 284389 T1022 n:127.0.0.1:27629_ oasc.DistributedQueue$LatchWatcher.process NodeChildrenChanged fired on path /overseer/queue state SyncConnected
[junit4] 2> 284389 T1055 n:127.0.0.1:27649_ c:collection1 s:shard2 x:collection1 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4] 2> 284390 T1055 n:127.0.0.1:27649_ c:collection1 s:shard2 x:collection1 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4] 2> ASYNC NEW_CORE C160 name=collection1 org.apache.solr.core.SolrCore@1da02950 url=https://127.0.0.1:27649/collection1 node=127.0.0.1:27649_ C160_STATE=coll:collection1 core:collection1 props:{state=down, node_name=127.0.0.1:27649_, core=collection1, base_url=https://127.0.0.1:27649}
[junit4] 2> 284390 T1055 n:127.0.0.1:27649_ c:collection1 s:shard2 x:collection1 C160 oasc.SyncStrategy.sync Sync replicas to https://127.0.0.1:27649/collection1/
[junit4] 2> 284390 T1055 n:127.0.0.1:27649_ c:collection1 s:shard2 x:collection1 C160 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4] 2> 284390 T1023 n:127.0.0.1:27629_ oasc.Overseer$ClusterStateUpdater.run processMessage: queueSize: 1, message = {
[junit4] 2> "operation":"leader",
[junit4] 2> "shard":"shard2",
[junit4] 2> "collection":"collection1"} current state version: 5
[junit4] 2> 284390 T1055 n:127.0.0.1:27649_ c:collection1 s:shard2 x:collection1 C160 oasc.SyncStrategy.syncToMe https://127.0.0.1:27649/collection1/ has no replicas
[junit4] 2> 284391 T1055 n:127.0.0.1:27649_ c:collection1 s:shard2 x:collection1 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: https://127.0.0.1:27649/collection1/ shard2
[junit4] 2> 284391 T1055 n:127.0.0.1:27649_ c:collection1 s:shard2 x:collection1 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard2
[junit4] 2> 284392 T1022 n:127.0.0.1:27629_ oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 284392 T1033 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 284392 T1050 n:127.0.0.1:27649_ oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 284395 T1023 n:127.0.0.1:27629_ oasc.Overseer$ClusterStateUpdater.run processMessage: queueSize: 1, message = {
[junit4] 2> "operation":"leader",
[junit4] 2> "shard":"shard2",
[junit4] 2> "collection":"collection1",
[junit4] 2> "base_url":"https://127.0.0.1:27649",
[junit4] 2> "core":"collection1",
[junit4] 2> "state":"active"} current state version: 6
[junit4] 2> 284397 T1022 n:127.0.0.1:27629_ oasc.DistributedQueue$LatchWatcher.process NodeChildrenChanged fired on path /overseer/queue state SyncConnected
[junit4] 2> 284398 T1033 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 284398 T1050 n:127.0.0.1:27649_ oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 284398 T1022 n:127.0.0.1:27629_ oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 284446 T1055 n:127.0.0.1:27649_ c:collection1 s:shard2 x:collection1 oasc.ZkController.register We are https://127.0.0.1:27649/collection1/ and leader is https://127.0.0.1:27649/collection1/
[junit4] 2> 284447 T1055 n:127.0.0.1:27649_ c:collection1 s:shard2 x:collection1 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=https://127.0.0.1:27649
[junit4] 2> 284447 T1055 n:127.0.0.1:27649_ c:collection1 s:shard2 x:collection1 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4] 2> 284447 T1055 n:127.0.0.1:27649_ c:collection1 s:shard2 x:collection1 oasc.ZkController.publish publishing core=collection1 state=active collection=collection1
[junit4] 2> 284447 T1055 n:127.0.0.1:27649_ c:collection1 s:shard2 x:collection1 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 284449 T1022 n:127.0.0.1:27629_ oasc.DistributedQueue$LatchWatcher.process NodeChildrenChanged fired on path /overseer/queue state SyncConnected
[junit4] 2> 284450 T1023 n:127.0.0.1:27629_ oasc.Overseer$ClusterStateUpdater.run processMessage: queueSize: 1, message = {
[junit4] 2> "roles":null,
[junit4] 2> "state":"active",
[junit4] 2> "core_node_name":"core_node1",
[junit4] 2> "numShards":"2",
[junit4] 2> "collection":"collection1",
[junit4] 2> "operation":"state",
[junit4] 2> "node_name":"127.0.0.1:27649_",
[junit4] 2> "shard":"shard2",
[junit4] 2> "core":"collection1",
[junit4] 2> "base_url":"https://127.0.0.1:27649"} current state version: 7
[junit4] 2> 284451 T1023 n:127.0.0.1:27629_ oasco.ReplicaMutator.updateState Update state numShards=2 message={
[junit4] 2> "roles":null,
[junit4] 2> "state":"active",
[junit4] 2> "core_node_name":"core_node1",
[junit4] 2> "numShards":"2",
[junit4] 2> "collection":"collection1",
[junit4] 2> "operation":"state",
[junit4] 2> "node_name":"127.0.0.1:27649_",
[junit4] 2> "shard":"shard2",
[junit4] 2> "core":"collection1",
[junit4] 2> "base_url":"https://127.0.0.1:27649"}
[junit4] 2> 284554 T1033 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 284554 T1050 n:127.0.0.1:27649_ oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 284554 T1022 n:127.0.0.1:27629_ oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 286006 T991 oas.SolrTestCaseJ4.writeCoreProperties Writing core.properties file to /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J3/temp/solr.cloud.CollectionsAPIDistributedZkTest 25585D626C46C0BA-001/shard-2-001/cores/collection1
[junit4] 2> 286009 T991 oasc.AbstractFullDistribZkTestBase.createJettys create jetty 2 in directory /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J3/temp/solr.cloud.CollectionsAPIDistributedZkTest 25585D626C46C0BA-001/shard-2-001
[junit4] 2> 286011 T991 oejs.Server.doStart jetty-9.2.10.v20150310
[junit4] 2> 286012 T991 oejsh.ContextHandler.doStart Started o.e.j.s.ServletContextHandler@78299767{/,null,AVAILABLE}
[junit4] 2> 286020 T991 oejs.AbstractConnector.doStart Started ServerConnector@42db79bc{SSL-http/1.1}{127.0.0.1:27681}
[junit4] 2> 286020 T991 oejs.Server.doStart Started @291107ms
[junit4] 2> 286020 T991 oascse.JettySolrRunner$1.lifeCycleStarted Jetty properties: {coreRootDirectory=/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J3/../../../../../../../../../../../usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J3/temp/solr.cloud.CollectionsAPIDistributedZkTest 25585D626C46C0BA-001/shard-2-001/cores, hostPort=27681, solrconfig=solrconfig.xml, hostContext=/}
[junit4] 2> 286023 T991 oass.SolrDispatchFilter.init SolrDispatchFilter.init()sun.misc.Launcher$AppClassLoader@17072b90
[junit4] 2> 286023 T991 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J3/../../../../../../../../../../../usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J3/temp/solr.cloud.CollectionsAPIDistributedZkTest 25585D626C46C0BA-001/shard-2-001/'
[junit4] 2> 286065 T991 oasc.SolrXmlConfig.fromFile Loading container configuration from /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J3/../../../../../../../../../../../usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J3/temp/solr.cloud.CollectionsAPIDistributedZkTest 25585D626C46C0BA-001/shard-2-001/solr.xml
[junit4] 2> 286118 T991 oasc.CorePropertiesLocator.<init> Config-defined core root directory: /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J3/../../../../../../../../../../../usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J3/temp/solr.cloud.CollectionsAPIDistributedZkTest 25585D626C46C0BA-001/shard-2-001/cores
[junit4] 2> 286119 T991 oasc.CoreContainer.<init> New CoreContainer 354538572
[junit4] 2> 286119 T991 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J3/../../../../../../../../../../../usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J3/temp/solr.cloud.CollectionsAPIDistributedZkTest 25585D626C46C0BA-001/shard-2-001/]
[junit4] 2> 286119 T991 oasc.CoreContainer.load loading shared library: /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J3/../../../../../../../../../../../usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J3/temp/solr.cloud.CollectionsAPIDistributedZkTest 25585D626C46C0BA-001/shard-2-001/lib
[junit4] 2> 286120 T991 oasc.SolrResourceLoader.addToClassLoader WARN Can't find (or read) directory to add to classloader: lib (resolved as: /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J3/../../../../../../../../../../../usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J3/temp/solr.cloud.CollectionsAPIDistributedZkTest 25585D626C46C0BA-001/shard-2-001/lib).
[junit4] 2> 286131 T991 oashc.HttpShardHandlerFactory.init created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
[junit4] 2> 286140 T991 oasu.UpdateShardHandler.<init> Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
[junit4] 2> 286144 T991 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
[junit4] 2> 286144 T991 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
[junit4] 2> 286144 T991 oasc.CoreContainer.load Node Name: 127.0.0.1
[junit4] 2> 286144 T991 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:27618/solr
[junit4] 2> 286144 T991 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4] 2> 286145 T991 oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default ZkCredentialsProvider
[junit4] 2> 286145 T991 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 286147 T1069 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@7d9fb841 name:ZooKeeperConnection Watcher:127.0.0.1:27618 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 286148 T991 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 286148 T991 oascc.SolrZkClient.createZkACLProvider Using default ZkACLProvider
[junit4] 2> 286150 T991 n:127.0.0.1:27681_ oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 286151 T1072 n:127.0.0.1:27681_ oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@6691f91d name:ZooKeeperConnection Watcher:127.0.0.1:27618/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 286151 T991 n:127.0.0.1:27681_ oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 286156 T991 n:127.0.0.1:27681_ oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 287160 T991 n:127.0.0.1:27681_ oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:27681_
[junit4] 2> 287161 T991 n:127.0.0.1:27681_ oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:27681_
[junit4] 2> 287163 T991 n:127.0.0.1:27681_ oasc.Overseer.close Overseer (id=null) closing
[junit4] 2> 287165 T991 n:127.0.0.1:27681_ oasc.CoreContainer.intializeAuthorizationPlugin Security conf doesn't exist. Skipping setup for authorization module.
[junit4] 2> 287166 T991 n:127.0.0.1:27681_ oasc.CorePropertiesLocator.discover Looking for core definitions underneath /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J3/../../../../../../../../../../../usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J3/temp/solr.cloud.CollectionsAPIDistributedZkTest 25585D626C46C0BA-001/shard-2-001/cores
[junit4] 2> 287171 T991 n:127.0.0.1:27681_ oasc.CoreDescriptor.<init> CORE DESCRIPTOR: {dataDir=data/, collection=collection1, coreNodeName=, loadOnStartup=true, shard=, schema=schema.xml, config=solrconfig.xml, absoluteInstDir=/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J3/../../../../../../../../../../../usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J3/temp/solr.cloud.CollectionsAPIDistributedZkTest 25585D626C46C0BA-001/shard-2-001/cores/collection1/, instanceDir=/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J3/../../../../../../../../../../../usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J3/temp/solr.cloud.CollectionsAPIDistributedZkTest 25585D626C46C0BA-001/shard-2-001/cores/collection1, name=collection1, transient=false}
[junit4] 2> 287171 T991 n:127.0.0.1:27681_ oasc.CorePropertiesLocator.discoverUnder Found core collection1 in /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J3/../../../../../../../../../../../usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J3/temp/solr.cloud.CollectionsAPIDistributedZkTest 25585D626C46C0BA-001/shard-2-001/cores/collection1/
[junit4] 2> 287171 T991 n:127.0.0.1:27681_ oasc.CorePropertiesLocator.discover Found 1 core definitions
[junit4] 2> 287173 T1073 n:127.0.0.1:27681_ c:collection1 x:collection1 oasc.ZkController.publish publishing core=collection1 state=down collection=collection1
[junit4] 2> 287173 T1073 n:127.0.0.1:27681_ c:collection1 x:collection1 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 287174 T1073 n:127.0.0.1:27681_ oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4] 2> 287174 T1022 n:127.0.0.1:27629_ oasc.DistributedQueue$LatchWatcher.process NodeChildrenChanged fired on path /overseer/queue state SyncConnected
[junit4] 2> 287175 T1023 n:127.0.0.1:27629_ oasc.Overseer$ClusterStateUpdater.run processMessage: queueSize: 1, message = {
[junit4] 2> "roles":null,
[junit4] 2> "state":"down",
[junit4] 2> "numShards":"2",
[junit4] 2> "collection":"collection1",
[junit4] 2> "operation":"state",
[junit4] 2> "node_name":"127.0.0.1:27681_",
[junit4] 2> "shard":null,
[junit4] 2> "core":"collection1",
[junit4] 2> "base_url":"https://127.0.0.1:27681"} current state version: 8
[junit4] 2> 287176 T1023 n:127.0.0.1:27629_ oasco.ReplicaMutator.updateState Update state numShards=2 message={
[junit4] 2> "roles":null,
[junit4] 2> "state":"down",
[junit4] 2> "numShards":"2",
[junit4] 2> "collection":"collection1",
[junit4] 2> "operation":"state",
[junit4] 2> "node_name":"127.0.0.1:27681_",
[junit4] 2> "shard":null,
[junit4] 2> "core":"collection1",
[junit4] 2> "base_url":"https://127.0.0.1:27681"}
[junit4] 2> 287176 T1023 n:127.0.0.1:27629_ oasco.ReplicaMutator.updateState Collection already exists with numShards=2
[junit4] 2> 287177 T1023 n:127.0.0.1:27629_ oasco.ReplicaMutator.updateState Assigning new node to shard shard=shard1
[junit4] 2> 287178 T1072 n:127.0.0.1:27681_ oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 287178 T1022 n:127.0.0.1:27629_ oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 287178 T1050 n:127.0.0.1:27649_ oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 287178 T1033 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 288174 T1073 n:127.0.0.1:27681_ oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
[junit4] 2> 288175 T1073 n:127.0.0.1:27681_ oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4] 2> 288175 T1073 n:127.0.0.1:27681_ oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4] 2> 288176 T1073 n:127.0.0.1:27681_ oascc.ZkStateReader.readConfigName Load collection config from:/collections/collection1
[junit4] 2> 288176 T1073 n:127.0.0.1:27681_ oascc.ZkStateReader.readConfigName path=/collections/collection1 configName=conf1 specified config exists in ZooKeeper
[junit4] 2> 288177 T1073 n:127.0.0.1:27681_ oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J3/../../../../../../../../../../../usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J3/temp/solr.cloud.CollectionsAPIDistributedZkTest 25585D626C46C0BA-001/shard-2-001/cores/collection1/'
[junit4] 2> 288196 T1073 n:127.0.0.1:27681_ oasc.Config.<init> loaded config solrconfig.xml with version 0
[junit4] 2> 288208 T1073 n:127.0.0.1:27681_ oasc.SolrConfig.refreshRequestParams current version of requestparams : -1
[junit4] 2> 288229 T1073 n:127.0.0.1:27681_ oasc.SolrConfig.<init> Using Lucene MatchVersion: 5.2.0
[junit4] 2> 288276 T1073 n:127.0.0.1:27681_ oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4] 2> 288378 T1073 n:127.0.0.1:27681_ oass.IndexSchema.readSchema Reading Solr Schema from /configs/conf1/schema.xml
[junit4] 2> 288392 T1073 n:127.0.0.1:27681_ oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4] 2> 288786 T1073 n:127.0.0.1:27681_ oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4] 2> 288804 T1073 n:127.0.0.1:27681_ oass.IndexSchema.readSchema default search field in schema is text
[junit4] 2> 288807 T1073 n:127.0.0.1:27681_ oass.IndexSchema.readSchema unique key field: id
[junit4] 2> 288818 T1073 n:127.0.0.1:27681_ oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 288823 T1073 n:127.0.0.1:27681_ oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 288827 T1073 n:127.0.0.1:27681_ oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4] 2> 288828 T1073 n:127.0.0.1:27681_ oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4] 2> 288829 T1073 n:127.0.0.1:27681_ oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4] 2> 288829 T1073 n:127.0.0.1:27681_ oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4] 2> 288830 T1073 n:127.0.0.1:27681_ oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4] 2> 288830 T1073 n:127.0.0.1:27681_ oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4] 2> 288830 T1073 n:127.0.0.1:27681_ oasc.CoreContainer.create Creating SolrCore 'collection1' using configuration from collection collection1
[junit4] 2> 288830 T1073 n:127.0.0.1:27681_ x:collection1 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4] 2> 288831 T1073 n:127.0.0.1:27681_ x:collection1 oasc.SolrCore.<init> [[collection1] ] Opening new SolrCore at [/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J3/../../../../../../../../../../../usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J3/temp/solr.cloud.CollectionsAPIDistributedZkTest 25585D626C46C0BA-001/shard-2-001/cores/collection1/], dataDir=[null]
[junit4] 2> 288831 T1073 n:127.0.0.1:27681_ x:collection1 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@580a298b
[junit4] 2> 288832 T1073 n:127.0.0.1:27681_ x:collection1 oasc.CachingDirectoryFactory.get return new directory for /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J3/../../../../../../../../../../../usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J3/temp/solr.cloud.CollectionsAPIDistributedZkTest 25585D626C46C0BA-001/shard-2-001/cores/collection1/data
[junit4] 2> 288832 T1073 n:127.0.0.1:27681_ x:collection1 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J3/../../../../../../../../../../../usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J3/temp/solr.cloud.CollectionsAPIDistributedZkTest 25585D626C46C0BA-001/shard-2-001/cores/collection1/data/index/
[junit4] 2> 2888
[...truncated too long message...]
elect/shard2/election/93833746392416269-core_node8-n_0000000000
[junit4] 2> 2 /solr/collections/awholynewstresscollection_collection0_5/leader_elect/shard1/election/93833746392416269-core_node20-n_0000000001
[junit4] 2> 2 /solr/collections/awholynewstresscollection_collection1_4/leader_elect/shard5/election/93833746392416260-core_node4-n_0000000000
[junit4] 2> 2 /solr/collections/awholynewcollection_1/leader_elect/shard2/election/93833746392416260-core_node11-n_0000000000
[junit4] 2> 2 /solr/collections/halfdeletedcollection2/leader_elect/shard3/election/93833746392416269-core_node5-n_0000000000
[junit4] 2> 2 /solr/collections/awholynewcollection_1/leader_elect/shard1/election/93833746392416269-core_node2-n_0000000000
[junit4] 2> 2 /solr/collections/awholynewstresscollection_collection4_5/leader_elect/shard8/election/93833746392416263-core_node12-n_0000000001
[junit4] 2> 2 /solr/collections/awholynewstresscollection_collection2_4/leader_elect/shard9/election/93833746392416265-core_node17-n_0000000000
[junit4] 2> 2 /solr/collections/awholynewcollection_0/leader_elect/shard4/election/93833746392416265-core_node8-n_0000000001
[junit4] 2> 2 /solr/collections/awholynewstresscollection_collection5_9/leader_elect/shard3/election/93833746392416269-core_node11-n_0000000000
[junit4] 2> 2 /solr/collections/awholynewstresscollection_collection3_4/leader_elect/shard3/election/93833746392416265-core_node7-n_0000000000
[junit4] 2> 2 /solr/collections/awholynewstresscollection_collection3_5/leader_elect/shard5/election/93833746392416263-core_node11-n_0000000001
[junit4] 2> 2 /solr/collections/awholynewstresscollection_collection5_5/leader_elect/shard1/election/93833746392416263-core_node6-n_0000000001
[junit4] 2> 2 /solr/collections/awholynewstresscollection_collection3_5/leader_elect/shard8/election/93833746392416267-core_node2-n_0000000000
[junit4] 2>
[junit4] 2> Maximum concurrent children watches above limit:
[junit4] 2>
[junit4] 2> 69 /solr/live_nodes
[junit4] 2> 3 /solr/overseer/collection-queue-work
[junit4] 2> 3 /solr/overseer/queue
[junit4] 2>
[junit4] 2> NOTE: download the large Jenkins line-docs file by running 'ant get-jenkins-line-docs' in the lucene directory.
[junit4] 2> NOTE: reproduce with: ant test -Dtestcase=CollectionsAPIDistributedZkTest -Dtests.method=test -Dtests.seed=25585D626C46C0BA -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/home/jenkins/lucene-data/enwiki.random.lines.txt -Dtests.locale=en_MT -Dtests.timezone=Europe/Vatican -Dtests.asserts=true -Dtests.file.encoding=US-ASCII
[junit4] ERROR 542s J3 | CollectionsAPIDistributedZkTest.test <<<
[junit4] > Throwable #1: com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=1555, name=collection1, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]
[junit4] > Caused by: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at https://127.0.0.1:27649: Could not find collection : awholynewstresscollection_collection1_4
[junit4] > at __randomizedtesting.SeedInfo.seed([25585D626C46C0BA]:0)
[junit4] > at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:560)
[junit4] > at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:235)
[junit4] > at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:227)
[junit4] > at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:376)
[junit4] > at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:328)
[junit4] > at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1086)
[junit4] > at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:856)
[junit4] > at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:799)
[junit4] > at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1220)
[junit4] > at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:884)Throwable #2: com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=1554, name=collection0, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]
[junit4] > Caused by: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at https://127.0.0.1:27649: Could not find collection : awholynewstresscollection_collection0_5
[junit4] > at __randomizedtesting.SeedInfo.seed([25585D626C46C0BA]:0)
[junit4] > at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:560)
[junit4] > at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:235)
[junit4] > at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:227)
[junit4] > at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:376)
[junit4] > at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:328)
[junit4] > at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1086)
[junit4] > at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:856)
[junit4] > at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:799)
[junit4] > at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1220)
[junit4] > at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:884)Throwable #3: com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=1558, name=collection4, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]
[junit4] > Caused by: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at https://127.0.0.1:27649: Could not find collection : awholynewstresscollection_collection4_5
[junit4] > at __randomizedtesting.SeedInfo.seed([25585D626C46C0BA]:0)
[junit4] > at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:560)
[junit4] > at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:235)
[junit4] > at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:227)
[junit4] > at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:376)
[junit4] > at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:328)
[junit4] > at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1086)
[junit4] > at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:856)
[junit4] > at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:799)
[junit4] > at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1220)
[junit4] > at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:884)
[junit4] 2> 817955 T991 c:collection1 s:shard2 x:collection1 oas.SolrTestCaseJ4.deleteCore ###deleteCore
[junit4] 2> NOTE: test params are: codec=Asserting(Lucene50), sim=DefaultSimilarity, locale=en_MT, timezone=Europe/Vatican
[junit4] 2> NOTE: FreeBSD 9.1-RELEASE-p3 amd64/Oracle Corporation 1.7.0_65 (64-bit)/cpus=16,threads=1,free=117416096,total=532676608
[junit4] 2> NOTE: All tests run in this JVM: [TestSolrConfigHandler, CollectionReloadTest, TestGroupingSearch, TermsComponentTest, TestUtils, TestIndexingPerformance, ActionThrottleTest, UnloadDistributedZkTest, LeaderElectionTest, TestWriterPerf, CollectionsAPIDistributedZkTest]
[junit4] Completed [122/490] on J3 in 546.34s, 1 test, 1 error <<< FAILURES!
[...truncated 1199 lines...]
BUILD FAILED
/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:543: The following error occurred while executing this line:
/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:484: The following error occurred while executing this line:
/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:61: The following error occurred while executing this line:
/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/extra-targets.xml:39: The following error occurred while executing this line:
/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build.xml:229: The following error occurred while executing this line:
/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/common-build.xml:511: The following error occurred while executing this line:
/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1433: The following error occurred while executing this line:
/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:991: There were test failures: 490 suites, 1960 tests, 1 error, 65 ignored (11 assumptions)
Total time: 151 minutes 19 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Sending artifact delta relative to Lucene-Solr-NightlyTests-5.x #846
Archived 5 artifacts
Archive block size is 32768
Received 0 blocks and 988886321 bytes
Compression is 0.0%
Took 2 min 51 sec
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure
[JENKINS] Lucene-Solr-NightlyTests-5.x - Build # 851 - Still
Failing
Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-5.x/851/
1 tests failed.
FAILED: org.apache.solr.cloud.RecoveryZkTest.test
Error Message:
shard1 is not consistent. Got 878 from http://127.0.0.1:26089/collection1lastClient and got 271 from http://127.0.0.1:26098/collection1
Stack Trace:
java.lang.AssertionError: shard1 is not consistent. Got 878 from http://127.0.0.1:26089/collection1lastClient and got 271 from http://127.0.0.1:26098/collection1
at __randomizedtesting.SeedInfo.seed([EEE64A9CBACB3BAD:66B2754614375655]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.apache.solr.cloud.RecoveryZkTest.test(RecoveryZkTest.java:123)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1627)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:836)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:872)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:886)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:960)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:935)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:845)
at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:747)
at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:781)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:792)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
at java.lang.Thread.run(Thread.java:745)
Build Log:
[...truncated 10395 lines...]
[junit4] Suite: org.apache.solr.cloud.RecoveryZkTest
[junit4] 2> Creating dataDir: /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.RecoveryZkTest EEE64A9CBACB3BAD-001/init-core-data-001
[junit4] 2> 673906 T5561 oas.SolrTestCaseJ4.buildSSLConfig Randomized ssl (false) and clientAuth (false)
[junit4] 2> 673907 T5561 oas.BaseDistributedSearchTestCase.initHostContext Setting hostContext system property: /
[junit4] 2> 673914 T5561 oasc.ZkTestServer.run STARTING ZK TEST SERVER
[junit4] 2> 673915 T5562 oasc.ZkTestServer$2$1.setClientPort client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 673915 T5562 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
[junit4] 2> 674015 T5561 oasc.ZkTestServer.run start zk server on port:26081
[junit4] 2> 674030 T5561 oasc.AbstractZkTestCase.putConfig put /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
[junit4] 2> 674033 T5561 oasc.AbstractZkTestCase.putConfig put /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/schema15.xml to /configs/conf1/schema.xml
[junit4] 2> 674137 T5561 oasc.AbstractZkTestCase.putConfig put /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
[junit4] 2> 674141 T5561 oasc.AbstractZkTestCase.putConfig put /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
[junit4] 2> 674144 T5561 oasc.AbstractZkTestCase.putConfig put /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
[junit4] 2> 674148 T5561 oasc.AbstractZkTestCase.putConfig put /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
[junit4] 2> 674151 T5561 oasc.AbstractZkTestCase.putConfig put /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
[junit4] 2> 674153 T5561 oasc.AbstractZkTestCase.putConfig put /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
[junit4] 2> 674155 T5561 oasc.AbstractZkTestCase.putConfig put /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4] 2> 674157 T5561 oasc.AbstractZkTestCase.putConfig put /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
[junit4] 2> 674160 T5561 oasc.AbstractZkTestCase.putConfig put /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
[junit4] 2> 675411 T5561 oas.SolrTestCaseJ4.writeCoreProperties Writing core.properties file to /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.RecoveryZkTest EEE64A9CBACB3BAD-001/control-001/cores/collection1
[junit4] 2> 675414 T5561 oejs.Server.doStart jetty-9.2.10.v20150310
[junit4] 2> 675416 T5561 oejsh.ContextHandler.doStart Started o.e.j.s.ServletContextHandler@5b5602d1{/,null,AVAILABLE}
[junit4] 2> 675416 T5561 oejs.AbstractConnector.doStart Started ServerConnector@6f559ef6{HTTP/1.1}{127.0.0.1:26084}
[junit4] 2> 675416 T5561 oejs.Server.doStart Started @681099ms
[junit4] 2> 675417 T5561 oascse.JettySolrRunner$1.lifeCycleStarted Jetty properties: {solr.data.dir=/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.RecoveryZkTest EEE64A9CBACB3BAD-001/tempDir-001/control/data, coreRootDirectory=/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../../usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.RecoveryZkTest EEE64A9CBACB3BAD-001/control-001/cores, hostPort=26084, hostContext=/}
[junit4] 2> 675417 T5561 oass.SolrDispatchFilter.init SolrDispatchFilter.init()sun.misc.Launcher$AppClassLoader@17072b90
[junit4] 2> 675417 T5561 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../../usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.RecoveryZkTest EEE64A9CBACB3BAD-001/control-001/'
[junit4] 2> 675449 T5561 oasc.SolrXmlConfig.fromFile Loading container configuration from /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../../usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.RecoveryZkTest EEE64A9CBACB3BAD-001/control-001/solr.xml
[junit4] 2> 675485 T5561 oasc.CorePropertiesLocator.<init> Config-defined core root directory: /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../../usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.RecoveryZkTest EEE64A9CBACB3BAD-001/control-001/cores
[junit4] 2> 675485 T5561 oasc.CoreContainer.<init> New CoreContainer 2110373052
[junit4] 2> 675486 T5561 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../../usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.RecoveryZkTest EEE64A9CBACB3BAD-001/control-001/]
[junit4] 2> 675486 T5561 oasc.CoreContainer.load loading shared library: /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../../usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.RecoveryZkTest EEE64A9CBACB3BAD-001/control-001/lib
[junit4] 2> 675487 T5561 oasc.SolrResourceLoader.addToClassLoader WARN Can't find (or read) directory to add to classloader: lib (resolved as: /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../../usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.RecoveryZkTest EEE64A9CBACB3BAD-001/control-001/lib).
[junit4] 2> 675496 T5561 oashc.HttpShardHandlerFactory.init created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
[junit4] 2> 675501 T5561 oasu.UpdateShardHandler.<init> Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
[junit4] 2> 675502 T5561 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
[junit4] 2> 675502 T5561 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
[junit4] 2> 675502 T5561 oasc.CoreContainer.load Node Name: 127.0.0.1
[junit4] 2> 675502 T5561 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:26081/solr
[junit4] 2> 675503 T5561 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4] 2> 675517 T5561 n:127.0.0.1:26084_ oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:26084_
[junit4] 2> 675521 T5561 n:127.0.0.1:26084_ oasc.Overseer.close Overseer (id=null) closing
[junit4] 2> 675522 T5561 n:127.0.0.1:26084_ oasc.OverseerElectionContext.runLeaderProcess I am going to be the leader 127.0.0.1:26084_
[junit4] 2> 675523 T5561 n:127.0.0.1:26084_ oasc.Overseer.start Overseer (id=93845081397067779-127.0.0.1:26084_-n_0000000000) starting
[junit4] 2> 675529 T5561 n:127.0.0.1:26084_ oasc.OverseerAutoReplicaFailoverThread.<init> Starting OverseerAutoReplicaFailoverThread autoReplicaFailoverWorkLoopDelay=10000 autoReplicaFailoverWaitAfterExpiration=30000 autoReplicaFailoverBadNodeExpiration=60000
[junit4] 2> 675530 T5591 n:127.0.0.1:26084_ oasc.OverseerCollectionProcessor.run Process current queue of collection creations
[junit4] 2> 675531 T5590 n:127.0.0.1:26084_ oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
[junit4] 2> 675533 T5561 n:127.0.0.1:26084_ oasc.CoreContainer.intializeAuthorizationPlugin Security conf doesn't exist. Skipping setup for authorization module.
[junit4] 2> 675534 T5561 n:127.0.0.1:26084_ oasc.CorePropertiesLocator.discover Looking for core definitions underneath /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../../usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.RecoveryZkTest EEE64A9CBACB3BAD-001/control-001/cores
[junit4] 2> 675536 T5561 n:127.0.0.1:26084_ oasc.CoreDescriptor.<init> CORE DESCRIPTOR: {name=collection1, config=solrconfig.xml, absoluteInstDir=/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../../usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.RecoveryZkTest EEE64A9CBACB3BAD-001/control-001/cores/collection1/, loadOnStartup=true, coreNodeName=, collection=control_collection, dataDir=data/, schema=schema.xml, shard=, transient=false, instanceDir=/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../../usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.RecoveryZkTest EEE64A9CBACB3BAD-001/control-001/cores/collection1}
[junit4] 2> 675537 T5561 n:127.0.0.1:26084_ oasc.CorePropertiesLocator.discoverUnder Found core collection1 in /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../../usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.RecoveryZkTest EEE64A9CBACB3BAD-001/control-001/cores/collection1/
[junit4] 2> 675537 T5561 n:127.0.0.1:26084_ oasc.CorePropertiesLocator.discover Found 1 core definitions
[junit4] 2> 675538 T5593 n:127.0.0.1:26084_ c:control_collection x:collection1 oasc.ZkController.publish publishing core=collection1 state=down collection=control_collection
[junit4] 2> 675538 T5593 n:127.0.0.1:26084_ c:control_collection x:collection1 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 675539 T5593 n:127.0.0.1:26084_ oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4] 2> 675539 T5589 n:127.0.0.1:26084_ oasc.DistributedQueue$LatchWatcher.process NodeChildrenChanged fired on path /overseer/queue state SyncConnected
[junit4] 2> 675539 T5590 n:127.0.0.1:26084_ oasc.Overseer$ClusterStateUpdater.run processMessage: queueSize: 1, message = {
[junit4] 2> "core":"collection1",
[junit4] 2> "state":"down",
[junit4] 2> "node_name":"127.0.0.1:26084_",
[junit4] 2> "collection":"control_collection",
[junit4] 2> "operation":"state",
[junit4] 2> "numShards":"1",
[junit4] 2> "roles":null,
[junit4] 2> "base_url":"http://127.0.0.1:26084",
[junit4] 2> "shard":null} current state version: 0
[junit4] 2> 675540 T5590 n:127.0.0.1:26084_ oasco.ReplicaMutator.updateState Update state numShards=1 message={
[junit4] 2> "core":"collection1",
[junit4] 2> "state":"down",
[junit4] 2> "node_name":"127.0.0.1:26084_",
[junit4] 2> "collection":"control_collection",
[junit4] 2> "operation":"state",
[junit4] 2> "numShards":"1",
[junit4] 2> "roles":null,
[junit4] 2> "base_url":"http://127.0.0.1:26084",
[junit4] 2> "shard":null}
[junit4] 2> 675540 T5590 n:127.0.0.1:26084_ oasco.ClusterStateMutator.createCollection building a new cName: control_collection
[junit4] 2> 675540 T5590 n:127.0.0.1:26084_ oasco.ReplicaMutator.updateState Assigning new node to shard shard=shard1
[junit4] 2> 676539 T5593 n:127.0.0.1:26084_ oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
[junit4] 2> 676540 T5593 n:127.0.0.1:26084_ oasc.ZkController.createCollectionZkNode Check for collection zkNode:control_collection
[junit4] 2> 676540 T5593 n:127.0.0.1:26084_ oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4] 2> 676541 T5593 n:127.0.0.1:26084_ oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../../usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.RecoveryZkTest EEE64A9CBACB3BAD-001/control-001/cores/collection1/'
[junit4] 2> 676558 T5593 n:127.0.0.1:26084_ oasc.Config.<init> loaded config solrconfig.xml with version 0
[junit4] 2> 676568 T5593 n:127.0.0.1:26084_ oasc.SolrConfig.refreshRequestParams current version of requestparams : -1
[junit4] 2> 676586 T5593 n:127.0.0.1:26084_ oasc.SolrConfig.<init> Using Lucene MatchVersion: 5.2.0
[junit4] 2> 676634 T5593 n:127.0.0.1:26084_ oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4] 2> 676735 T5593 n:127.0.0.1:26084_ oass.IndexSchema.readSchema Reading Solr Schema from /configs/conf1/schema.xml
[junit4] 2> 676745 T5593 n:127.0.0.1:26084_ oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4] 2> 677112 T5593 n:127.0.0.1:26084_ oass.IndexSchema.readSchema default search field in schema is text
[junit4] 2> 677115 T5593 n:127.0.0.1:26084_ oass.IndexSchema.readSchema unique key field: id
[junit4] 2> 677117 T5593 n:127.0.0.1:26084_ oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 677122 T5593 n:127.0.0.1:26084_ oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 677141 T5593 n:127.0.0.1:26084_ oasc.CoreContainer.create Creating SolrCore 'collection1' using configuration from collection control_collection
[junit4] 2> 677141 T5593 n:127.0.0.1:26084_ x:collection1 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4] 2> 677142 T5593 n:127.0.0.1:26084_ x:collection1 oasc.SolrCore.<init> [[collection1] ] Opening new SolrCore at [/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../../usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.RecoveryZkTest EEE64A9CBACB3BAD-001/control-001/cores/collection1/], dataDir=[null]
[junit4] 2> 677142 T5593 n:127.0.0.1:26084_ x:collection1 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@6eafcd05
[junit4] 2> 677143 T5593 n:127.0.0.1:26084_ x:collection1 oasc.CachingDirectoryFactory.get return new directory for /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../../usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.RecoveryZkTest EEE64A9CBACB3BAD-001/control-001/cores/collection1/data
[junit4] 2> 677143 T5593 n:127.0.0.1:26084_ x:collection1 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../../usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.RecoveryZkTest EEE64A9CBACB3BAD-001/control-001/cores/collection1/data/index/
[junit4] 2> 677143 T5593 n:127.0.0.1:26084_ x:collection1 oasc.SolrCore.initIndex WARN [collection1] Solr index directory '/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../../usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.RecoveryZkTest EEE64A9CBACB3BAD-001/control-001/cores/collection1/data/index' doesn't exist. Creating new index...
[junit4] 2> 677144 T5593 n:127.0.0.1:26084_ x:collection1 oasc.CachingDirectoryFactory.get return new directory for /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../../usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.RecoveryZkTest EEE64A9CBACB3BAD-001/control-001/cores/collection1/data/index
[junit4] 2> 677144 T5593 n:127.0.0.1:26084_ x:collection1 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=26, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.6440467782656305]
[junit4] 2> 677145 T5593 n:127.0.0.1:26084_ x:collection1 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4] 2> commit{dir=MockDirectoryWrapper(RAMDirectory@2efb7104 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@7db8d5b4),segFN=segments_1,generation=1}
[junit4] 2> 677145 T5593 n:127.0.0.1:26084_ x:collection1 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 677151 T5593 n:127.0.0.1:26084_ x:collection1 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "nodistrib"
[junit4] 2> 677152 T5593 n:127.0.0.1:26084_ x:collection1 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe"
[junit4] 2> 677152 T5593 n:127.0.0.1:26084_ x:collection1 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4] 2> 677153 T5593 n:127.0.0.1:26084_ x:collection1 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "stored_sig"
[junit4] 2> 677153 T5593 n:127.0.0.1:26084_ x:collection1 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4] 2> 677153 T5593 n:127.0.0.1:26084_ x:collection1 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
[junit4] 2> 677154 T5593 n:127.0.0.1:26084_ x:collection1 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 677154 T5593 n:127.0.0.1:26084_ x:collection1 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 677154 T5593 n:127.0.0.1:26084_ x:collection1 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
[junit4] 2> 677174 T5593 n:127.0.0.1:26084_ x:collection1 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 677177 T5593 n:127.0.0.1:26084_ x:collection1 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 677179 T5593 n:127.0.0.1:26084_ x:collection1 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 677182 T5593 n:127.0.0.1:26084_ x:collection1 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 677187 T5593 n:127.0.0.1:26084_ x:collection1 oasc.RequestHandlers.initHandlersFromConfig Registered paths: /update,standard,/admin/plugins,/admin/mbeans,/schema,/admin/system,/get,/admin/ping,/admin/threads,/update/json/docs,/admin/luke,/replication,/admin/logging,/admin/segments,/config,/admin/properties,/update/json,/admin/file,/update/csv
[junit4] 2> 677602 T5593 n:127.0.0.1:26084_ x:collection1 oasc.SolrCore.initStatsCache Using default statsCache cache: org.apache.solr.search.stats.LocalStatsCache
[junit4] 2> 677605 T5593 n:127.0.0.1:26084_ x:collection1 oasu.UpdateHandler.<init> Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 677606 T5593 n:127.0.0.1:26084_ x:collection1 oasu.UpdateLog.init Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10
[junit4] 2> 677609 T5593 n:127.0.0.1:26084_ x:collection1 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4] 2> 677610 T5593 n:127.0.0.1:26084_ x:collection1 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4] 2> 677611 T5593 n:127.0.0.1:26084_ x:collection1 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=38, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.8592471579064755]
[junit4] 2> 677612 T5593 n:127.0.0.1:26084_ x:collection1 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4] 2> commit{dir=MockDirectoryWrapper(RAMDirectory@2efb7104 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@7db8d5b4),segFN=segments_1,generation=1}
[junit4] 2> 677613 T5593 n:127.0.0.1:26084_ x:collection1 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 677613 T5593 n:127.0.0.1:26084_ x:collection1 oass.SolrIndexSearcher.<init> Opening Searcher@4b149f76[collection1] main
[junit4] 2> 677614 T5593 n:127.0.0.1:26084_ x:collection1 oasr.ManagedResourceStorage.newStorageIO Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
[junit4] 2> 677615 T5593 n:127.0.0.1:26084_ x:collection1 oasr.ManagedResourceStorage$ZooKeeperStorageIO.configure Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 677615 T5593 n:127.0.0.1:26084_ x:collection1 oasr.RestManager.init Initializing RestManager with initArgs: {}
[junit4] 2> 677615 T5593 n:127.0.0.1:26084_ x:collection1 oasr.ManagedResourceStorage.load Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 677616 T5593 n:127.0.0.1:26084_ x:collection1 oasr.ManagedResourceStorage$ZooKeeperStorageIO.openInputStream No data found for znode /configs/conf1/_rest_managed.json
[junit4] 2> 677616 T5593 n:127.0.0.1:26084_ x:collection1 oasr.ManagedResourceStorage.load Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 677616 T5593 n:127.0.0.1:26084_ x:collection1 oasr.RestManager.init Initializing 0 registered ManagedResources
[junit4] 2> 677617 T5593 n:127.0.0.1:26084_ x:collection1 oash.ReplicationHandler.inform Commits will be reserved for 10000
[junit4] 2> 677617 T5593 n:127.0.0.1:26084_ x:collection1 oasc.ZkController.getConfDirListeners watch zkdir /configs/conf1
[junit4] 2> 677618 T5594 n:127.0.0.1:26084_ x:collection1 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@4b149f76[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 677618 T5593 n:127.0.0.1:26084_ x:collection1 oasc.CoreContainer.registerCore registering core: collection1
[junit4] 2> 677619 T5597 n:127.0.0.1:26084_ c:control_collection s:shard1 x:collection1 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:26084 collection:control_collection shard:shard1
[junit4] 2> 677619 T5561 n:127.0.0.1:26084_ oass.SolrDispatchFilter.init user.dir=/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0
[junit4] 2> 677620 T5561 n:127.0.0.1:26084_ oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4] 2> 677626 T5597 n:127.0.0.1:26084_ c:control_collection s:shard1 x:collection1 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard1
[junit4] 2> 677628 T5561 oasc.ChaosMonkey.monkeyLog monkey: init - expire sessions:false cause connection loss:false
[junit4] 2> 677628 T5589 n:127.0.0.1:26084_ oasc.DistributedQueue$LatchWatcher.process NodeChildrenChanged fired on path /overseer/queue state SyncConnected
[junit4] 2> 677628 T5561 oasc.AbstractFullDistribZkTestBase.createJettys Creating collection1 with stateFormat=2
[junit4] 2> 677628 T5597 n:127.0.0.1:26084_ c:control_collection s:shard1 x:collection1 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4] 2> 677629 T5597 n:127.0.0.1:26084_ c:control_collection s:shard1 x:collection1 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4] 2> 677629 T5590 n:127.0.0.1:26084_ oasc.Overseer$ClusterStateUpdater.run processMessage: queueSize: 1, message = {
[junit4] 2> "operation":"leader",
[junit4] 2> "shard":"shard1",
[junit4] 2> "collection":"control_collection"} current state version: 1
[junit4] 2> ASYNC NEW_CORE C2117 name=collection1 org.apache.solr.core.SolrCore@4d9aa7f0 url=http://127.0.0.1:26084/collection1 node=127.0.0.1:26084_ C2117_STATE=coll:control_collection core:collection1 props:{core=collection1, state=down, node_name=127.0.0.1:26084_, base_url=http://127.0.0.1:26084}
[junit4] 2> 677629 T5597 n:127.0.0.1:26084_ c:control_collection s:shard1 x:collection1 C2117 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:26084/collection1/
[junit4] 2> 677630 T5597 n:127.0.0.1:26084_ c:control_collection s:shard1 x:collection1 C2117 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4] 2> 677630 T5597 n:127.0.0.1:26084_ c:control_collection s:shard1 x:collection1 C2117 oasc.SyncStrategy.syncToMe http://127.0.0.1:26084/collection1/ has no replicas
[junit4] 2> 677631 T5597 n:127.0.0.1:26084_ c:control_collection s:shard1 x:collection1 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:26084/collection1/ shard1
[junit4] 2> 677636 T5589 n:127.0.0.1:26084_ oasc.DistributedQueue$LatchWatcher.process NodeChildrenChanged fired on path /overseer/queue state SyncConnected
[junit4] 2> 677636 T5590 n:127.0.0.1:26084_ oasc.Overseer$ClusterStateUpdater.run processMessage: queueSize: 1, message = {
[junit4] 2> "operation":"create",
[junit4] 2> "name":"collection1",
[junit4] 2> "numShards":"1",
[junit4] 2> "stateFormat":"2"} current state version: 2
[junit4] 2> 677637 T5590 n:127.0.0.1:26084_ oasco.ClusterStateMutator.createCollection building a new cName: collection1
[junit4] 2> 677639 T5590 n:127.0.0.1:26084_ oasc.Overseer$ClusterStateUpdater.run processMessage: queueSize: 1, message = {
[junit4] 2> "operation":"leader",
[junit4] 2> "shard":"shard1",
[junit4] 2> "collection":"control_collection",
[junit4] 2> "base_url":"http://127.0.0.1:26084",
[junit4] 2> "core":"collection1",
[junit4] 2> "state":"active"} current state version: 2
[junit4] 2> 677640 T5590 n:127.0.0.1:26084_ oasco.ZkStateWriter.writePendingUpdates going to create_collection /collections/collection1/state.json
[junit4] 2> 677645 T5589 n:127.0.0.1:26084_ oasc.DistributedQueue$LatchWatcher.process NodeChildrenChanged fired on path /overseer/queue state SyncConnected
[junit4] 2> 677687 T5597 n:127.0.0.1:26084_ c:control_collection s:shard1 x:collection1 oasc.ZkController.register We are http://127.0.0.1:26084/collection1/ and leader is http://127.0.0.1:26084/collection1/
[junit4] 2> 677688 T5597 n:127.0.0.1:26084_ c:control_collection s:shard1 x:collection1 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:26084
[junit4] 2> 677688 T5597 n:127.0.0.1:26084_ c:control_collection s:shard1 x:collection1 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4] 2> 677688 T5597 n:127.0.0.1:26084_ c:control_collection s:shard1 x:collection1 oasc.ZkController.publish publishing core=collection1 state=active collection=control_collection
[junit4] 2> 677688 T5597 n:127.0.0.1:26084_ c:control_collection s:shard1 x:collection1 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 677690 T5589 n:127.0.0.1:26084_ oasc.DistributedQueue$LatchWatcher.process NodeChildrenChanged fired on path /overseer/queue state SyncConnected
[junit4] 2> 677691 T5590 n:127.0.0.1:26084_ oasc.Overseer$ClusterStateUpdater.run processMessage: queueSize: 1, message = {
[junit4] 2> "core":"collection1",
[junit4] 2> "state":"active",
[junit4] 2> "node_name":"127.0.0.1:26084_",
[junit4] 2> "collection":"control_collection",
[junit4] 2> "operation":"state",
[junit4] 2> "numShards":"1",
[junit4] 2> "roles":null,
[junit4] 2> "base_url":"http://127.0.0.1:26084",
[junit4] 2> "core_node_name":"core_node1",
[junit4] 2> "shard":"shard1"} current state version: 4
[junit4] 2> 677692 T5590 n:127.0.0.1:26084_ oasco.ReplicaMutator.updateState Update state numShards=1 message={
[junit4] 2> "core":"collection1",
[junit4] 2> "state":"active",
[junit4] 2> "node_name":"127.0.0.1:26084_",
[junit4] 2> "collection":"control_collection",
[junit4] 2> "operation":"state",
[junit4] 2> "numShards":"1",
[junit4] 2> "roles":null,
[junit4] 2> "base_url":"http://127.0.0.1:26084",
[junit4] 2> "core_node_name":"core_node1",
[junit4] 2> "shard":"shard1"}
[junit4] 2> 678900 T5561 oas.SolrTestCaseJ4.writeCoreProperties Writing core.properties file to /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.RecoveryZkTest EEE64A9CBACB3BAD-001/shard-1-001/cores/collection1
[junit4] 2> 678902 T5561 oasc.AbstractFullDistribZkTestBase.createJettys create jetty 1 in directory /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.RecoveryZkTest EEE64A9CBACB3BAD-001/shard-1-001
[junit4] 2> 678903 T5561 oejs.Server.doStart jetty-9.2.10.v20150310
[junit4] 2> 678905 T5561 oejsh.ContextHandler.doStart Started o.e.j.s.ServletContextHandler@536dee1a{/,null,AVAILABLE}
[junit4] 2> 678906 T5561 oejs.AbstractConnector.doStart Started ServerConnector@27246a72{HTTP/1.1}{127.0.0.1:26089}
[junit4] 2> 678906 T5561 oejs.Server.doStart Started @684589ms
[junit4] 2> 678906 T5561 oascse.JettySolrRunner$1.lifeCycleStarted Jetty properties: {coreRootDirectory=/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../../usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.RecoveryZkTest EEE64A9CBACB3BAD-001/shard-1-001/cores, solrconfig=solrconfig.xml, hostPort=26089, hostContext=/, solr.data.dir=/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.RecoveryZkTest EEE64A9CBACB3BAD-001/tempDir-001/jetty1}
[junit4] 2> 678907 T5561 oass.SolrDispatchFilter.init SolrDispatchFilter.init()sun.misc.Launcher$AppClassLoader@17072b90
[junit4] 2> 678907 T5561 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../../usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.RecoveryZkTest EEE64A9CBACB3BAD-001/shard-1-001/'
[junit4] 2> 678937 T5561 oasc.SolrXmlConfig.fromFile Loading container configuration from /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../../usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.RecoveryZkTest EEE64A9CBACB3BAD-001/shard-1-001/solr.xml
[junit4] 2> 678973 T5561 oasc.CorePropertiesLocator.<init> Config-defined core root directory: /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../../usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.RecoveryZkTest EEE64A9CBACB3BAD-001/shard-1-001/cores
[junit4] 2> 678973 T5561 oasc.CoreContainer.<init> New CoreContainer 1389186062
[junit4] 2> 678973 T5561 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../../usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.RecoveryZkTest EEE64A9CBACB3BAD-001/shard-1-001/]
[junit4] 2> 678974 T5561 oasc.CoreContainer.load loading shared library: /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../../usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.RecoveryZkTest EEE64A9CBACB3BAD-001/shard-1-001/lib
[junit4] 2> 678975 T5561 oasc.SolrResourceLoader.addToClassLoader WARN Can't find (or read) directory to add to classloader: lib (resolved as: /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../../usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.RecoveryZkTest EEE64A9CBACB3BAD-001/shard-1-001/lib).
[junit4] 2> 678984 T5561 oashc.HttpShardHandlerFactory.init created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
[junit4] 2> 678989 T5561 oasu.UpdateShardHandler.<init> Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
[junit4] 2> 678990 T5561 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
[junit4] 2> 678990 T5561 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
[junit4] 2> 678990 T5561 oasc.CoreContainer.load Node Name: 127.0.0.1
[junit4] 2> 678990 T5561 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:26081/solr
[junit4] 2> 678991 T5561 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4] 2> 680004 T5561 n:127.0.0.1:26089_ oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:26089_
[junit4] 2> 680006 T5561 n:127.0.0.1:26089_ oasc.Overseer.close Overseer (id=null) closing
[junit4] 2> 680008 T5561 n:127.0.0.1:26089_ oasc.CoreContainer.intializeAuthorizationPlugin Security conf doesn't exist. Skipping setup for authorization module.
[junit4] 2> 680009 T5561 n:127.0.0.1:26089_ oasc.CorePropertiesLocator.discover Looking for core definitions underneath /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../../usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.RecoveryZkTest EEE64A9CBACB3BAD-001/shard-1-001/cores
[junit4] 2> 680012 T5561 n:127.0.0.1:26089_ oasc.CoreDescriptor.<init> CORE DESCRIPTOR: {loadOnStartup=true, transient=false, shard=, schema=schema.xml, instanceDir=/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../../usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.RecoveryZkTest EEE64A9CBACB3BAD-001/shard-1-001/cores/collection1, config=solrconfig.xml, dataDir=data/, collection=collection1, name=collection1, coreNodeName=, absoluteInstDir=/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../../usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.RecoveryZkTest EEE64A9CBACB3BAD-001/shard-1-001/cores/collection1/}
[junit4] 2> 680012 T5561 n:127.0.0.1:26089_ oasc.CorePropertiesLocator.discoverUnder Found core collection1 in /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../../usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.RecoveryZkTest EEE64A9CBACB3BAD-001/shard-1-001/cores/collection1/
[junit4] 2> 680013 T5561 n:127.0.0.1:26089_ oasc.CorePropertiesLocator.discover Found 1 core definitions
[junit4] 2> 680015 T5621 n:127.0.0.1:26089_ c:collection1 x:collection1 oasc.ZkController.publish publishing core=collection1 state=down collection=collection1
[junit4] 2> 680016 T5621 n:127.0.0.1:26089_ c:collection1 x:collection1 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 680018 T5589 n:127.0.0.1:26084_ oasc.DistributedQueue$LatchWatcher.process NodeChildrenChanged fired on path /overseer/queue state SyncConnected
[junit4] 2> 680018 T5621 n:127.0.0.1:26089_ oasc.ZkController.preRegister Registering watch for external collection collection1
[junit4] 2> 680019 T5590 n:127.0.0.1:26084_ oasc.Overseer$ClusterStateUpdater.run processMessage: queueSize: 1, message = {
[junit4] 2> "core":"collection1",
[junit4] 2> "state":"down",
[junit4] 2> "node_name":"127.0.0.1:26089_",
[junit4] 2> "collection":"collection1",
[junit4] 2> "operation":"state",
[junit4] 2> "numShards":"1",
[junit4] 2> "roles":null,
[junit4] 2> "base_url":"http://127.0.0.1:26089",
[junit4] 2> "shard":null} current state version: 5
[junit4] 2> 680019 T5590 n:127.0.0.1:26084_ oasco.ReplicaMutator.updateState Update state numShards=1 message={
[junit4] 2> "core":"collection1",
[junit4] 2> "state":"down",
[junit4] 2> "node_name":"127.0.0.1:26089_",
[junit4] 2> "collection":"collection1",
[junit4] 2> "operation":"state",
[junit4] 2> "numShards":"1",
[junit4] 2> "roles":null,
[junit4] 2> "base_url":"http://127.0.0.1:26089",
[junit4] 2> "shard":null}
[junit4] 2> 680019 T5590 n:127.0.0.1:26084_ oasco.ReplicaMutator.updateState Collection already exists with numShards=1
[junit4] 2> 680019 T5590 n:127.0.0.1:26084_ oasco.ReplicaMutator.updateState Assigning new node to shard shard=shard1
[junit4] 2> 680020 T5621 n:127.0.0.1:26089_ oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4] 2> 680021 T5590 n:127.0.0.1:26084_ oasco.ZkStateWriter.writePendingUpdates going to update_collection /collections/collection1/state.json version: 0
[junit4] 2> 681020 T5621 n:127.0.0.1:26089_ oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
[junit4] 2> 681021 T5621 n:127.0.0.1:26089_ oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4] 2> 681021 T5621 n:127.0.0.1:26089_ oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4] 2> 681022 T5621 n:127.0.0.1:26089_ oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../../usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.RecoveryZkTest EEE64A9CBACB3BAD-001/shard-1-001/cores/collection1/'
[junit4] 2> 681037 T5621 n:127.0.0.1:26089_ oasc.Config.<init> loaded config solrconfig.xml with version 0
[junit4] 2> 681045 T5621 n:127.0.0.1:26089_ oasc.SolrConfig.refreshRequestParams current version of requestparams : -1
[junit4] 2> 681061 T5621 n:127.0.0.1:26089_ oasc.SolrConfig.<init> Using Lucene MatchVersion: 5.2.0
[junit4] 2> 681095 T5621 n:127.0.0.1:26089_ oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4] 2> 681196 T5621 n:127.0.0.1:26089_ oass.IndexSchema.readSchema Reading Solr Schema from /configs/conf1/schema.xml
[junit4] 2> 681203 T5621 n:127.0.0.1:26089_ oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4] 2> 681479 T5621 n:127.0.0.1:26089_ oass.IndexSchema.readSchema default search field in schema is text
[junit4] 2> 681482 T5621 n:127.0.0.1:26089_ oass.IndexSchema.readSchema unique key field: id
[junit4] 2> 681483 T5621 n:127.0.0.1:26089_ oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 681487 T5621 n:127.0.0.1:26089_ oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 681501 T5621 n:127.0.0.1:26089_ oasc.CoreContainer.create Creating SolrCore 'collection1' using configuration from collection collection1
[junit4] 2> 681501 T5621 n:127.0.0.1:26089_ x:collection1 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4] 2> 681501 T5621 n:127.0.0.1:26089_ x:collection1 oasc.SolrCore.<init> [[collection1] ] Opening new SolrCore at [/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../../usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.RecoveryZkTest EEE64A9CBACB3BAD-001/shard-1-001/cores/collection1/], dataDir=[null]
[junit4] 2> 681502 T5621 n:127.0.0.1:26089_ x:collection1 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@6eafcd05
[junit4] 2> 681502 T5621 n:127.0.0.1:26089_ x:collection1 oasc.CachingDirectoryFactory.get return new directory for /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../../usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.RecoveryZkTest EEE64A9CBACB3BAD-001/shard-1-001/cores/collection1/data
[junit4] 2> 681502 T5621 n:127.0.0.1:26089_ x:collection1 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../../usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.RecoveryZkTest EEE64A9CBACB3BAD-001/shard-1-001/cores/collection1/data/index/
[junit4] 2> 681503 T5621 n:127.0.0.1:26089_ x:collection1 oasc.SolrCore.initIndex WARN [collection1] Solr index directory '/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../../usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.RecoveryZkTest EEE64A9CBACB3BAD-001/shard-1-001/cores/collection1/data/index' doesn't exist. Creating new index...
[junit4] 2> 681503 T5621 n:127.0.0.1:26089_ x:collection1 oasc.CachingDirectoryFactory.get return new directory for /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../../usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.RecoveryZkTest EEE64A9CBACB3BAD-001/shard-1-001/cores/collection1/data/index
[junit4] 2> 681503 T5621 n:127.0.0.1:26089_ x:collection1 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=26, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.6440467782656305]
[junit4] 2> 681504 T5621 n:127.0.0.1:26089_ x:collection1 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4] 2> commit{dir=MockDirectoryWrapper(RAMDirectory@6062903e lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@66ec07e1),segFN=segments_1,generation=1}
[junit4] 2> 681504 T5621 n:127.0.0.1:26089_ x:collection1 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 681509 T5621 n:127.0.0.1:26089_ x:collection1 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "nodistrib"
[junit4] 2> 681509 T5621 n:127.0.0.1:26089_ x:collection1 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe"
[junit4] 2> 681509 T5621 n:127.0.0.1:26089_ x:collection1 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4] 2> 681509 T5621 n:127.0.0.1:26089_ x:collection1 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "stored_sig"
[junit4] 2> 681509 T5621 n:127.0.0.1:26089_ x:collection1 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4] 2> 681510 T5621 n:127.0.0.1:26089_ x:collection1 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
[junit4] 2> 681510 T5621 n:127.0.0.1:26089_ x:collection1 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 681510 T5621 n:127.0.0.1:26089_ x:collection1 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 681510 T5621 n:127.0.0.1:26089_ x:collection1 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
[junit4] 2> 681523 T5621 n:127.0.0.1:26089_ x:collection1 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 681525 T5621 n:127.0.0.1:26089_ x:collection1 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 681528 T5621 n:127.0.0.1:26089_ x:collection1 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 681531 T5621 n:127.0.0.1:26089_ x:collection1 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 681537 T5621 n:127.0.0.1:26089_ x:collection1 oasc.RequestHandlers.initHandlersFromConfig Registered paths: /update,standard,/admin/plugins,/admin/mbeans,/schema,/admin/system,/get,/admin/ping,/admin/threads,/update/json/docs,/admin/luke,/replication,/admin/logging,/admin/segments,/config,/admin/properties,/update/json,/admin/file,/update/csv
[junit4] 2> 681539 T5621 n:127.0.0.1:26089_ x:collection1 oasc.SolrCore.initStatsCache Using default statsCache cache: org.apache.solr.search.stats.LocalStatsCache
[junit4] 2> 681540 T5621 n:127.0.0.1:26089_ x:collection1 oasu.UpdateHandler.<init> Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 681541 T5621 n:127.0.0.1:26089_ x:collection1 oasu.UpdateLog.init Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10
[junit4] 2> 681542 T5621 n:127.0.0.1:26089_ x:collection1 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4] 2> 681542 T5621 n:127.0.0.1:26089_ x:collection1 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4] 2> 681543 T5621 n:127.0.0.1:26089_ x:collection1 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=38, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.8592471579064755]
[junit4] 2> 681544 T5621 n:127.0.0.1:26089_ x:collection1 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4] 2> commit{dir=MockDirectoryWrapper(RAMDirectory@6062903e lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@66ec07e1),segFN=segments_1,generation=1}
[junit4] 2> 681544 T5621 n:127.0.0.1:26089_ x:collection1 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 681545 T5621 n:127.0.0.1:26089_ x:collection1 oass.SolrIndexSearcher.<init> Opening Searcher@127cf3c[collection1] main
[junit4] 2> 681545 T5621 n:127.0.0.1:26089_ x:collection1 oasr.ManagedResourceStorage.newStorageIO Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
[junit4] 2> 681546 T5621 n:127.0.0.1:26089_ x:collection1 oasr.ManagedResourceStorage$ZooKeeperStorageIO.configure Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 681546 T5621 n:127.0.0.1:26089_ x:collection1 oasr.RestManager.init Initializing RestManager with initArgs: {}
[junit4] 2> 681546 T5621 n:127.0.0.1:26089_ x:collection1 oasr.ManagedResourceStorage.load Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 681547 T5621 n:127.0.0.1:26089_ x:collection1 oasr.ManagedResourceStorage$ZooKeeperStorageIO.openInputStream No data found for znode /configs/conf1/_rest_managed.json
[junit4] 2> 681547 T5621 n:127.0.0.1:26089_ x:collection1 oasr.ManagedResourceStorage.load Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 681547 T5621 n:127.0.0.1:26089_ x:collection1 oasr.RestManager.init Initializing 0 registered ManagedResources
[junit4] 2> 681547 T5621 n:127.0.0.1:26089_ x:collection1 oash.ReplicationHandler.inform Commits will be reserved for 10000
[junit4] 2> 681548 T5622 n:127.0.0.1:26089_ x:collection1 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@127cf3c[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 681550 T5621 n:127.0.0.1:26089_ x:collection1 oasc.ZkController.getConfDirListeners watch zkdir /configs/conf1
[junit4] 2> 681551 T5621 n:127.0.0.1:26089_ x:collection1 oasc.CoreContainer.registerCore registering core: collection1
[junit4] 2> 681552 T5625 n:127.0.0.1:26089_ c:collection1 s:shard1 x:collection1 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:26089 collection:collection1 shard:shard1
[junit4] 2> 681552 T5561 n:127.0.0.1:26089_ oass.SolrDispatchFilter.init user.dir=/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0
[junit4] 2> 681552 T5561 n:127.0.0.1:26089_ oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4] 2> 681557 T5625 n:127.0.0.1:26089_ c:collection1 s:shard1 x:collection1 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard1
[junit4] 2> 681558 T5589 n:127.0.0.1:26084_ oasc.DistributedQueue$LatchWatcher.process NodeChildrenChanged fired on path /overseer/queue state SyncConnected
[junit4] 2> 681558 T5625 n:127.0.0.1:26089_ c:collection1 s:shard1 x:collection1 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4] 2> 681558 T5625 n:127.0.0.1:26089_ c:collection1 s:shard1 x:collection1 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4] 2> ASYNC NEW_CORE C2118 name=collection1 org.apache.solr.core.SolrCore@1a5708f3 url=http://127.0.0.1:26089/collection1 node=127.0.0.1:26089_ C2118_STATE=coll:collection1 core:collection1 props:{core=collection1, state=down, node_name=127.0.0.1:26089_, base_url=http://127.0.0.1:26089}
[junit4] 2> 681559 T5625 n:127.0.0.1:26089_ c:collection1 s:shard1 x:collection1 C2118 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:26089/collection1/
[junit4] 2> 681559 T5625 n:127.0.0.1:26089_ c:collection1 s:shard1 x:collection1 C2118 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4] 2> 681559 T5590 n:127.0.0.1:26084_ oasc.Overseer$ClusterStateUpdater.run processMessage: queueSize: 1, message = {
[junit4] 2> "operation":"leader",
[junit4] 2> "shard":"shard1",
[junit4] 2> "collection":"collection1"} current state version: 5
[junit4] 2> 681559 T5625 n:127.0.0.1:26089_ c:collection1 s:shard1 x:collection1 C2118 oasc.SyncStrategy.syncToMe http://127.0.0.1:26089/collection1/ has no replicas
[junit4] 2> 681560 T5625 n:127.0.0.1:26089_ c:collection1 s:shard1 x:collection1 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:26089/collection1/ shard1
[junit4] 2> 681560 T5590 n:127.0.0.1:26084_ oasco.ZkStateWriter.writePendingUpdates going to update_collection /collections/collection1/state.json version: 1
[junit4] 2> 681564 T5590 n:127.0.0.1:26084_ oasc.Overseer$ClusterStateUpdater.run processMessage: queueSize: 1, message = {
[junit4] 2> "operation":"leader",
[junit4] 2> "shard":"shard1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "base_url":"http://127.0.0.1:26089",
[junit4] 2> "core":"collection1",
[junit4] 2> "state":"active"} current state version: 5
[junit4] 2> 681565 T5590 n:127.0.0.1:26084_ oasco.ZkStateWriter.writePendingUpdates going to update_collection /collections/collection1/state.json version: 2
[junit4] 2> 681569 T5589 n:127.0.0.1:26084_ oasc.DistributedQueue$LatchWatcher.process NodeChildrenChanged fired on path /overseer/queue state SyncConnected
[junit4] 2> 681614 T5625 n:127.0.0.1:26089_ c:collection1 s:shard1 x:collection1 oasc.ZkController.register We are http://127.0.0.1:26089/collection1/ and leader is http://127.0.0.1:26089/collection1/
[junit4] 2> 681615 T5625 n:127.0.0.1:26089_ c:collection1 s:shard1 x:collection1 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:26089
[junit4] 2> 681615 T5625 n:127.0.0.1:26089_ c:collection1 s:shard1 x:collection1 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4] 2> 681615 T5625 n:127.0.0.1:26089_ c:collection1 s:shard1 x:collection1 oasc.ZkController.publish publishing core=collection1 state=active collection=collection1
[junit4] 2> 681615 T5625 n:127.0.0.1:26089_ c:collection1 s:shard1 x:collection1 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 681616 T5589 n:127.0.0.1:26084_ oasc.DistributedQueue$LatchWatcher.process NodeChildrenChanged fired on path /overseer/queue state SyncConnected
[junit4] 2> 681617 T5590 n:127.0.0.1:26084_ oasc.Overseer$ClusterStateUpdater.run processMessage: queueSize: 1, message = {
[junit4] 2> "core":"collection1",
[junit4] 2> "state":"active",
[junit4] 2> "node_name":"127.0.0.1:26089_",
[junit4] 2> "collection":"collection1",
[junit4] 2> "operation":"state",
[junit4] 2> "numShards":"1",
[junit4] 2> "roles":null,
[junit4] 2> "base_url":"http://127.0.0.1:26089",
[junit4] 2> "core_node_name":"core_node1",
[junit4] 2> "shard":"shard1"} current state version: 5
[junit4] 2> 681618 T5590 n:127.0.0.1:26084_ oasco.ReplicaMutator.updateState Update state numShards=1 message={
[junit4] 2> "core":"collection1",
[junit4] 2> "state":"active",
[junit4] 2> "node_name":"127.0.0.1:26089_",
[junit4] 2> "collection":"collection1",
[junit4] 2> "operation":"state",
[junit4] 2> "numShards":"1",
[junit4] 2> "roles":null,
[junit4] 2> "base_url":"http://127.0.0.1:26089",
[junit4] 2> "core_node_name":"core_node1",
[junit4] 2> "shard":"shard1"}
[junit4] 2> 681619 T5590 n:127.0.0.1:26084_ oasco.ZkStateWriter.writePendingUpdates going to update_collection /collections/collection1/state.json version: 3
[junit4] 2> 682840 T5561 oas.SolrTestCaseJ4.writeCoreProperties Writing core.properties file to /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.RecoveryZkTest EEE64A9CBACB3BAD-001/shard-2-001/cores/collection1
[junit4] 2> 682843 T5561 oasc.AbstractFullDistribZkTestBase.createJettys create jetty 2 in directory /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.RecoveryZkTest EEE64A9CBACB3BAD-001/shard-2-001
[junit4] 2> 682844 T5561 oejs.Server.doStart jetty-9.2.10.v20150310
[junit4] 2> 682845 T5561 oejsh.ContextHandler.doStart Started o.e.j.s.ServletContextHandler@6441caeb{/,null,AVAILABLE}
[junit4] 2> 682846 T5561 oejs.AbstractConnector.doStart Started ServerConnector@278b4ac2{HTTP/1.1}{127.0.0.1:26098}
[junit4] 2> 682846 T5561 oejs.Server.doStart Started @688529ms
[junit4] 2> 682846 T5561 oascse.JettySolrRunner$1.lifeCycleStarted Jetty properties: {coreRootDirectory=/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../../usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.RecoveryZkTest EEE64A9CBACB3BAD-001/shard-2-001/cores, solrconfig=solrconfig.xml, hostContext=/, hostPort=26098, solr.data.dir=/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.RecoveryZkTest EEE64A9CBACB3BAD-001/tempDir-001/jetty2}
[junit4] 2> 682847 T5561 oass.SolrDispatchFilter.init SolrDispatchFilter.init()sun.misc.Launcher$AppClassLoader@17072b90
[junit4] 2> 682847 T5561 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../../usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.RecoveryZkTest EEE64A9CBACB3BAD-001/shard-2-001/'
[junit4] 2> 682887 T5561 oasc.SolrXmlConfig.fromFile Loading container configuration from /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../../usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.RecoveryZkTest EEE64A9CBACB3BAD-001/shard-2-001/solr.xml
[junit4] 2> 682929 T5561 oasc.CorePropertiesLocator.<init> Config-defined core root directory: /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../../usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.RecoveryZkTest EEE64A9CBACB3BAD-001/shard-2-001/cores
[junit4] 2> 682929 T5561 oasc.CoreContainer.<init> New CoreContainer 49027204
[junit4] 2> 682930 T5561 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../../usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.RecoveryZkTest EEE64A9CBACB3BAD-001/shard-2-001/]
[junit4] 2> 682930 T5561 oasc.CoreContainer.load loading shared library: /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../../usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.RecoveryZkTest EEE64A9CBACB3BAD-001/shard-2-001/lib
[junit4] 2> 682931 T5561 oasc.SolrResourceLoader.addToClassLoader WARN Can't find (or read) directory to add to classloader: lib (resolved as: /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../../usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.RecoveryZkTest EEE64A9CBACB3BAD-001/shard-2-001/lib).
[junit4] 2> 682946 T5561 oashc.HttpShardHandlerFactory.init created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
[junit4] 2> 682954 T5561 oasu.UpdateShardHandler.<init> Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
[junit4] 2> 682955 T5561 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
[junit4] 2> 682955 T5561 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
[junit4] 2> 682955 T5561 oasc.CoreContainer.load Node Name: 127.0.0.1
[junit4] 2> 682956 T5561 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:26081/solr
[junit4] 2> 682956 T5561 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4] 2> 683967 T5561 n:127.0.0.1:26098_ oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:26098_
[junit4] 2> 683970 T5561 n:127.0.0.1:26098_ oasc.Overseer.close Overseer (id=null) closing
[junit4] 2> 683971 T5561 n:127.0.0.1:26098_ oasc.CoreContainer.intializeAuthorizationPlugin Security conf doesn't exist. Skipping setup for authorization module.
[junit4] 2> 683972 T5561 n:127.0.0.1:26098_ oasc.CorePropertiesLocator.discover Looking for core definitions underneath /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../../usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.RecoveryZkTest EEE64A9CBACB3BAD-001/shard-2-001/cores
[junit4] 2> 683975 T5561 n:127.0.0.1:26098_ oasc.CoreDescriptor.<init> CORE DESCRIPTOR: {name=collection1, absoluteInstDir=/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../../usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.RecoveryZkTest EEE64A9CBACB3BAD-001/shard-2-001/cores/collection1/, config=solrconfig.xml, loadOnStartup=true, instanceDir=/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../../usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.RecoveryZkTest EEE64A9CBACB3BAD-001/shard-2-001/cores/collection1, schema=schema.xml, shard=, dataDir=data/, transient=false, coreNodeName=, collection=collection1}
[junit4] 2> 683975 T5561 n:127.0.0.1:26098_ oasc.CorePropertiesLocator.discoverUnder Found core collection1 in /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../../usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.RecoveryZkTest EEE64A9CBACB3BAD-001/shard-2-001/cores/collection1/
[junit4] 2> 683975 T5561 n:127.0.0.1:26098_ oasc.CorePropertiesLocator.discover Found 1 core definitions
[junit4] 2> 683977 T5643 n:127.0.0.1:26098_ c:collection1 x:collection1 oasc.ZkController.publish publishing core=collection1 state=down collection=collection1
[junit4] 2> 683978 T5643 n:127.0.0.1:26098_ c:collection1 x:collection1 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 683980 T5589 n:127.0.0.1:26084_ oasc.DistributedQueue$LatchWatcher.process NodeChildrenChanged fired on path /overseer/queue state SyncConnected
[junit4] 2> 683980 T5643 n:127.0.0.1:26098_ oasc.ZkController.preRegister Registering watch for external collection collection1
[junit4] 2> 683980 T5590 n:127.0.0.1:26084_ oasc.Overseer$ClusterStateUpdater.run processMessage: queueSize: 1, message = {
[junit4] 2> "core":"collection1",
[junit4] 2> "state":"down",
[junit4] 2> "node_name":"127.0.0.1:26098_",
[junit4] 2> "collection":"collection1",
[junit4] 2> "operation":"state",
[junit4] 2> "numShards":"1",
[junit4] 2> "roles":null,
[junit4] 2> "base_url":"http://127.0.0.1:26098",
[junit4] 2> "shard":null} current state version: 5
[junit4] 2> 683981 T5590 n:127.0.0.1:26084_ oasco.ReplicaMutator.updateState Update state numShards=1 message={
[junit4] 2> "core":"collection1",
[junit4] 2> "state":"down",
[junit4] 2> "node_name":"127.0.0.1:26098_",
[junit4] 2> "collection":"collection1",
[junit4] 2> "operation":"state",
[junit4] 2> "numShards":"1",
[junit4] 2> "roles":null,
[junit4] 2> "base_url":"http://127.0.0.1:26098",
[junit4] 2> "shard":null}
[junit4] 2> 683981 T5643 n:127.0.0.1:26098_ oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4] 2> 683981 T5590 n:127.0.0.1:26084_ oasco.ReplicaMutator.updateState Collection already exists with numShards=1
[junit4] 2> 683981 T5590 n:127.0.0.1:26084_ oasco.ReplicaMutator.updateState Assigning new node to shard shard=shard1
[junit4] 2> 683982 T5590 n:127.0.0.1:26084_ oasco.ZkStateWriter.writePendingUpdates going to update_collection /collections/collection1/state.json version: 4
[junit4] 2> 684981 T5643 n:127.0.0.1:26098_ oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
[junit4] 2> 684982 T5643 n:127.0.0.1:26098_ oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4] 2> 684982 T5643 n:127.0.0.1:26098_ oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4] 2> 684983 T5643 n:127.0.0.1:26098_ oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../../usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.RecoveryZkTest EEE64A9CBACB3BAD-001/shard-2-001/cores/collection1/'
[junit4] 2> 685004 T5643 n:127.0.0.1:26098_ oasc.Config.<init> loaded config solrconfig.xml with version 0
[junit4] 2> 685014 T5643 n:127.0.0.1:26098_ oasc.SolrConfig.refreshRequestParams current version of requestparams : -1
[junit4] 2> 685037 T5643 n:127.0.0.1:26098_ oasc.SolrConfig.<init> Using Lucene MatchVersion: 5.2.0
[junit4] 2> 685087 T5643 n:127.0.0.1:26098_ oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4] 2> 685188 T5643 n:127.0.0.1:26098_ oass.IndexSchema.readSchema Reading Solr Schema from /configs/conf1/schema.xml
[junit4] 2> 685195 T5643 n:127.0.0.1:26098_ oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4] 2> 685440 T5643 n:127.0.0.1:26098_ oass.IndexSchema.readSchema default search field in schema is text
[junit4] 2> 685442 T5643 n:127.0.0.1:26098_ oass.IndexSchema.readSchema unique key field: id
[junit4] 2> 685443 T5643 n:127.0.0.1:26098_ oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 685447 T5643 n:127.0.0.1:26098_ oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 685463 T5643 n:127.0.0.1:26098_ oasc.CoreContainer.create Creating SolrCore 'collection1' using configuration from collection collection1
[junit4] 2> 685463 T5643 n:127.0.0.1:26098_ x:collection1 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4] 2> 685463 T5643 n:127.0.0.1:26098_ x:collection1 oasc.SolrCore.<init> [[collection1] ] Opening new SolrCore at [/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../../usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.RecoveryZkTest EEE64A9CBACB3BAD-001/shard-2-001/cores/collection1/], dataDir=[null]
[junit4] 2> 685464 T5643 n:127.0.0.1:26098_ x:collection1 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@6eafcd05
[junit4] 2> 685464 T5643 n:127.0.0.1:26098_ x:collection1 oasc.CachingDirectoryFactory.get return new directory for /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../../usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.RecoveryZkTest EEE64A9CBACB3BAD-001/shard-2-001/cores/collection1/data
[junit4] 2> 685465 T5643 n:127.0.0.1:26098_ x:collection1 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../../usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.RecoveryZkTest EEE64A9CBACB3BAD-001/shard-2-001/cores/collection1/data/index/
[junit4] 2> 685465 T5643 n:127.0.0.1:26098_ x:collection1 oasc.SolrCore.initIndex WARN [collection1] Solr index directory '/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../../usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.RecoveryZkTest EEE64A9CBACB3BAD-001/shard-2-001/cores/collection1/data/index' doesn't exist. Creating new index...
[junit4] 2> 685465 T5643 n:127.0.0.1:26098_ x:collection1 oasc.CachingDirectoryFactory.get return new directory for /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../../usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.RecoveryZkTest EEE64A9CBACB3BAD-001/shard-2-001/cores/collection1/data/index
[junit4] 2> 685465 T5643 n:127.0.0.1:26098_ x:collection1 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=26, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.6440467782656305]
[junit4] 2> 685466 T5643 n:127.0.0.1:26098_ x:collection1 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4] 2> commit{dir=MockDirectoryWrapper(RAMDirectory@32028677 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@43ca0847),segFN=segments_1,generation=1}
[junit4] 2> 685466 T5643 n:127.0.0.1:26098_ x:collection1 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 685471 T5643 n:127.0.0.1:26098_ x:collection1 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "nodistrib"
[junit4] 2> 685471 T5643 n:127.0.0.1:26098_ x:collection1 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe"
[junit4] 2> 685471 T5643 n:127.0.0.1:26098_ x:collection1 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4] 2> 685471 T5643 n:127.0.0.1:26098_ x:collection1 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "stored_sig"
[junit4] 2> 685472 T5643 n:127.0.0.1:26098_ x:collection1 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4] 2> 685472 T5643 n:127.0.0.1:26098_ x:collection1 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
[junit4] 2> 685472 T5643 n:127.0.0.1:26098_ x:collection1 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 685472 T5643 n:127.0.0.1:26098_ x:collection1 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 685473 T5643 n:127.0.0.1:26098_ x:collection1 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
[junit4] 2> 685485 T5643 n:127.0.0.1:26098_ x:collection1 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 685487 T5643 n:127.0.0.1:26098_ x:collection1 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 685488 T5643 n:127.0.0.1:26098_ x:collection1 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 685490 T5643 n:127.0.0.1:26098_ x:collection1 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 685496 T5643 n:127.0.0.1:26098_ x:collection1 oasc.RequestHandlers.initHandlersFromConfig Registered paths: /update,standard,/admin/plugins,/admin/mbeans,/schema,/admin/system,/get,/admin/ping,/admin/threads,/update/json/docs,/admin/luke,/replication,/admin/logging,/admin/segments,/config,/admin/properties,/update/json,/admin/file,/update/csv
[junit4] 2> 685498 T5643 n:127.0.0.1:26098_ x:collection1 oasc.SolrCore.initStatsCache Using default statsCache cache: org.apache.solr.search.stats.LocalStatsCache
[junit4] 2> 685499 T5643 n:127.0.0.1:26098_ x:collection1 oasu.UpdateHandler.<init> Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 685499 T5643 n:127.0.0.1:26098_ x:collection1 oasu.UpdateLog.init Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10
[junit4] 2> 685501 T5643 n:127.0.0.1:26098_ x:collection1 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4] 2> 685501 T5643 n:127.0.0.1:26098_ x:collection1 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4] 2> 685502 T5643 n:127.0.0.1:26098_ x:collection1 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=38, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.8592471579064755]
[junit4] 2> 685503 T5643 n:127.0.0.1:26098_ x:collection1 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4] 2> commit{dir=MockDirectoryWrapper(RAMDirectory@32028677 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@43ca0847),segFN=segments_1,generation=1}
[junit4] 2> 685503 T5643 n:127.0.0.1:26098_ x:collection1 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 685503 T5643 n:127.0.0.1:26098_ x:collection1 oass.SolrIndexSearcher.<init> Opening Searcher@2df157b3[collection1] main
[junit4] 2> 685504 T5643 n:127.0.0.1:26098_ x:collection1 oasr.ManagedResourceStorage.newStorageIO Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
[junit4] 2> 685504 T5643 n:127.0.0.1:26098_ x:collection1 oasr.ManagedResourceStorage$ZooKeeperStorageIO.configure Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 685504 T5643 n:127.0.0.1:26098_ x:collection1 oasr.RestManager.init Initializing RestManager with initArgs: {}
[junit4] 2> 685505 T5643 n:127.0.0.1:26098_ x:collection1 oasr.ManagedResourceStorage.load Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 685505 T5643 n:127.0.0.1:26098_ x:collection1 oasr.ManagedResourceStorage$ZooKeeperStorageIO.openInputStream No data found for znode /configs/conf1/_rest_managed.json
[junit4] 2> 685505 T5643 n:127.0.0.1:26098_ x:collection1 oasr.ManagedResourceStorage.load Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 685505 T5643 n:127.0.0.1:26098_ x:collection1 oasr.RestManager.init Initializing 0 registered ManagedResources
[junit4] 2> 685506 T5643 n:127.0.0.1:26098_ x:collection1 oash.ReplicationHandler.inform Commits will be reserved for 10000
[junit4] 2> 685507 T5644 n:127.0.0.1:26098_ x:collection1 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@2df157b3[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 685507 T5643 n:127.0.0.1:26098_ x:collection1 oasc.ZkController.getConfDirListeners watch zkdir /configs/conf1
[junit4] 2> 685508 T5643 n:127.0.0.1:26098_ x:collection1 oasc.CoreContainer.registerCore registering core: collection1
[junit4] 2> 685508 T5647 n:127.0.0.1:26098_ c:collection1 s:shard1 x:collection1 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:26098 collection:collection1 shard:shard1
[junit4] 2> 685509 T5561 n:127.0.0.1:26098_ oass.SolrDispatchFilter.init user.dir=/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0
[junit4] 2> 685509 T5561 n:127.0.0.1:26098_ oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4] 2> 685511 T5647 n:127.0.0.1:26098_ c:collection1 s:shard1 x:collection1 oasc.ZkController.register We are http://127.0.0.1:26098/collection1/ and leader is http://127.0.0.1:26089/collection1/
[junit4] 2> 685511 T5647 n:127.0.0.1:26098_ c:collection1 s:shard1 x:collection1 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:26098
[junit4] 2> 685511 T5647 n:127.0.0.1:26098_ c:collection1 s:shard1 x:collection1 oasc.ZkController.checkRecovery Core needs to recover:collection1
[junit4] 2> 685511 T5647 n:127.0.0.1:26098_ c:collection1 s:shard1 x:collection1 oasu.DefaultSolrCoreState.doRecovery Running recovery - first canceling any ongoing recovery
[junit4] 2> ASYNC NEW_CORE C2119 name=collection1 org.apache.solr.core.SolrCore@6e2722e4 url=http://127.0.0.1:26098/collection1 node=127.0.0.1:26098_ C2119_STATE=coll:collection1 core:collection1 props:{core=collection1, state=down, node_name=127.0.0.1:26098_, base_url=http://127.0.0.1:26098}
[junit4] 2> 685511 T5648 n:127.0.0.1:26098_ c:collection1 s:shard1 x:collection1 C2119 oasc.RecoveryStrategy.run Starting recovery process. core=collection1 recoveringAfterStartup=true
[junit4] 2> 685512 T5648 n:127.0.0.1:26098_ c:collection1 s:shard1 x:collection1 C2119 oasc.RecoveryStrategy.doRecovery ###### startupVersions=[]
[junit4] 2> 685513 T5648 n:127.0.0.1:26098_ c:collection1 s:shard1 x:collection1 C2119 oasc.RecoveryStrategy.doRecovery Publishing state of core collection1 as recovering, leader is http://127.0.0.1:26089/collection1/ and I am http://127.0.0.1:26098/collection1/
[junit4] 2> 685513 T5648 n:127.0.0.1:26098_ c:collection1 s:shard1 x:collection1 C2119 oasc.ZkController.publish publishing core=collection1 state=recovering collection=collection1
[junit4] 2> 685513 T5648 n:127.0.0.1:26098_ c:collection1 s:shard1 x:collection1 C2119 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 685514 T5589 n:127.0.0.1:26084_ oasc.DistributedQueue$LatchWatcher.process NodeChildrenChanged fired on path /overseer/queue state SyncConnected
[junit4] 2> 685515 T5561 oas.SolrTestCaseJ4.setUp ###Starting test
[junit4] 2> 685515 T5590 n:127.0.0.1:26084_ oasc.Overseer$ClusterStateUpdater.run processMessage: queueSize: 1, message = {
[junit4] 2> "core":"collection1",
[junit4] 2> "state":"recovering",
[junit4] 2> "node_name":"127.0.0.1:26098_",
[junit4] 2> "collection":"collection1",
[junit4] 2> "operation":"state",
[junit4] 2> "numShards":"1",
[junit4] 2> "roles":null,
[junit4] 2> "base_url":"http://127.0.0.1:26098",
[junit4] 2> "core_node_name":"core_node2",
[junit4] 2> "shard":"shard1"} current state version: 5
[junit4] 2> 685516 T5648 n:127.0.0.1:26098_ c:collection1 s:shard1 x:collection1 C2119 oasc.RecoveryStrategy.sendPrepRecoveryCmd Sending prep recovery command to http://127.0.0.1:26089; WaitForState: action=PREPRECOVERY&core=collection1&nodeName=127.0.0.1%3A26098_&coreNodeName=core_node2&state=recovering&checkLive=true&onlyIfLeader=true&onlyIfLeaderActive=true
[junit4] 2> 685516 T5590 n:127.0.0.1:26084_ oasco.ReplicaMutator.updateState Update state numShards=1 message={
[junit4] 2> "core":"collection1",
[junit4] 2> "state":"recovering",
[junit4] 2> "node_name":"127.0.0.1:26098_",
[junit4] 2> "collection":"collection1",
[junit4] 2> "operation":"state",
[junit4] 2> "numShards":"1",
[junit4] 2> "roles":null,
[junit4] 2> "base_url":"http://127.0.0.1:26098",
[junit4] 2> "core_node_name":"core_node2",
[junit4] 2> "shard":"shard1"}
[junit4] 2> 685518 T5590 n:127.0.0.1:26084_ oasco.ZkStateWriter.writePendingUpdates going to update_collection /collections/collection1/state.json version: 5
[junit4] 2> 685520 T5611 n:127.0.0.1:26089_ oasha.CoreAdminHandler.handleWaitForStateAction Going to wait for coreNodeName: core_node2, state: recovering, checkLive: true, onlyIfLeader: true, onlyIfLeaderActive: true
[junit4] 2> ASYNC NEW_CORE C2120 name=collection1 org.apache.solr.core.SolrCore@4d9aa7f0 url=http://127.0.0.1:26084/collection1 node=127.0.0.1:26084_ C2120_STATE=coll:control_collection core:collection1 props:{core=collection1, state=active, node_name=127.0.0.1:26084_, base_url=http://127.0.0.1:26084, leader=true}
[junit4] 2> 685527 T5580 n:127.0.0.1:26084_ c:control_collection s:shard1 r:core_node1 x:collection1 C2120 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={version=2&wt=javabin&CONTROL=TRUE} {add=[1-0 (1501521314517614592)]} 0 6
[junit4] 2> 685527 T5611 n:127.0.0.1:26089_ oasha.CoreAdminHandler.handleWaitForStateAction Will wait a max of 183 seconds to see collection1 (shard1 of collection1) have state: recovering
[junit4] 2> 685527 T5581 n:127.0.0.1:26084_ c:control_collection s:shard1 r:core_node1 x
[...truncated too long message...]
41 T5734 n:127.0.0.1:26098_ oasc.ShardLeaderElectionContext.rejoinLeaderElection Not rejoining election because CoreContainer is closed
[junit4] 2> 847042 T5734 n:127.0.0.1:26098_ oasc.SolrCore.close [collection1] CLOSING SolrCore org.apache.solr.core.SolrCore@40ab1044
[junit4] 2> 847042 T5734 n:127.0.0.1:26098_ oasc.ZkController.unregisterConfListener removed listener for config directory [/configs/conf1]
[junit4] 2> 847042 T5734 n:127.0.0.1:26098_ oasc.ZkController.unregisterConfListener No more listeners for config directory [/configs/conf1]
[junit4] 2> 847043 T5734 n:127.0.0.1:26098_ oasu.DirectUpdateHandler2.close closing DirectUpdateHandler2{commits=63,autocommits=0,soft autocommits=0,optimizes=0,rollbacks=0,expungeDeletes=0,docsPending=0,adds=0,deletesById=0,deletesByQuery=0,errors=0,cumulative_adds=412,cumulative_deletesById=180,cumulative_deletesByQuery=0,cumulative_errors=0,transaction_logs_total_size=38231,transaction_logs_total_number=1}
[junit4] 2> 847044 T5734 n:127.0.0.1:26098_ oasu.SolrCoreState.decrefSolrCoreState Closing SolrCoreState
[junit4] 2> 847044 T5734 n:127.0.0.1:26098_ oasu.DefaultSolrCoreState.closeIndexWriter SolrCoreState ref count has reached 0 - closing IndexWriter
[junit4] 2> 847044 T5734 n:127.0.0.1:26098_ oasu.DefaultSolrCoreState.closeIndexWriter closing IndexWriter with IndexWriterCloser
[junit4] 2> 847048 T5734 n:127.0.0.1:26098_ oasc.SolrCore.closeSearcher [collection1] Closing main searcher on request.
[junit4] 2> 847064 T5734 n:127.0.0.1:26098_ oasc.CachingDirectoryFactory.close Closing MockDirectoryFactory - 2 directories currently being tracked
[junit4] 2> 847065 T5734 n:127.0.0.1:26098_ oasc.CachingDirectoryFactory.closeCacheValue looking to close /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../../usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.RecoveryZkTest EEE64A9CBACB3BAD-001/shard-2-001/cores/collection1/data [CachedDir<<refCount=0;path=/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../../usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.RecoveryZkTest EEE64A9CBACB3BAD-001/shard-2-001/cores/collection1/data;done=false>>]
[junit4] 2> 847065 T5734 n:127.0.0.1:26098_ oasc.CachingDirectoryFactory.close Closing directory: /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../../usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.RecoveryZkTest EEE64A9CBACB3BAD-001/shard-2-001/cores/collection1/data
[junit4] 2> 847066 T5734 n:127.0.0.1:26098_ oasc.CachingDirectoryFactory.closeCacheValue looking to close /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../../usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.RecoveryZkTest EEE64A9CBACB3BAD-001/shard-2-001/cores/collection1/data/index [CachedDir<<refCount=0;path=/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../../usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.RecoveryZkTest EEE64A9CBACB3BAD-001/shard-2-001/cores/collection1/data/index;done=false>>]
[junit4] 2> 847066 T5734 n:127.0.0.1:26098_ oasc.CachingDirectoryFactory.close Closing directory: /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/../../../../../../../../../../../usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.RecoveryZkTest EEE64A9CBACB3BAD-001/shard-2-001/cores/collection1/data/index
[junit4] 2> 847522 T5730 n:127.0.0.1:26098_ oascc.ZkStateReader$3.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4] 2> 847547 T5561 oejs.AbstractConnector.doStop Stopped ServerConnector@567ad500{HTTP/1.1}{127.0.0.1:26098}
[junit4] 2> 847548 T5561 oejsh.ContextHandler.doStop Stopped o.e.j.s.ServletContextHandler@4ec05bf5{/,null,UNAVAILABLE}
[junit4] 2> 847551 T5561 c:control_collection s:shard1 x:collection1 oasc.ZkTestServer.send4LetterWord connecting to 127.0.0.1:26081 26081
[junit4] 2> 848594 T5562 oasc.ZkTestServer.send4LetterWord connecting to 127.0.0.1:26081 26081
[junit4] 2> 848596 T5562 oasc.ZkTestServer$ZKServerMain.runFromConfig WARN Watch limit violations:
[junit4] 2> Maximum concurrent create/delete watches above limit:
[junit4] 2>
[junit4] 2> 5 /solr/aliases.json
[junit4] 2> 5 /solr/clusterstate.json
[junit4] 2> 4 /solr/configs/conf1
[junit4] 2> 3 /solr/collections/collection1/state.json
[junit4] 2>
[junit4] 2> Maximum concurrent data watches above limit:
[junit4] 2>
[junit4] 2> 2 /solr/overseer_elect/election/93845081397067783-127.0.0.1:26089_-n_0000000001
[junit4] 2> 2 /solr/collections/collection1/leader_elect/shard1/election/93845081397067783-core_node1-n_0000000000
[junit4] 2>
[junit4] 2> Maximum concurrent children watches above limit:
[junit4] 2>
[junit4] 2> 4 /solr/live_nodes
[junit4] 2> 3 /solr/overseer/collection-queue-work
[junit4] 2> 2 /solr/overseer/queue
[junit4] 2>
[junit4] 2> NOTE: download the large Jenkins line-docs file by running 'ant get-jenkins-line-docs' in the lucene directory.
[junit4] 2> NOTE: reproduce with: ant test -Dtestcase=RecoveryZkTest -Dtests.method=test -Dtests.seed=EEE64A9CBACB3BAD -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/home/jenkins/lucene-data/enwiki.random.lines.txt -Dtests.locale=zh -Dtests.timezone=America/Antigua -Dtests.asserts=true -Dtests.file.encoding=US-ASCII
[junit4] FAILURE 175s J0 | RecoveryZkTest.test <<<
[junit4] > Throwable #1: java.lang.AssertionError: shard1 is not consistent. Got 878 from http://127.0.0.1:26089/collection1lastClient and got 271 from http://127.0.0.1:26098/collection1
[junit4] > at __randomizedtesting.SeedInfo.seed([EEE64A9CBACB3BAD:66B2754614375655]:0)
[junit4] > at org.apache.solr.cloud.RecoveryZkTest.test(RecoveryZkTest.java:123)
[junit4] > at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:960)
[junit4] > at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:935)
[junit4] > at java.lang.Thread.run(Thread.java:745)
[junit4] 2> 848628 T5561 c:control_collection s:shard1 x:collection1 oas.SolrTestCaseJ4.deleteCore ###deleteCore
[junit4] 2> NOTE: leaving temporary files on disk at: /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.RecoveryZkTest EEE64A9CBACB3BAD-001
[junit4] 2> NOTE: test params are: codec=Asserting(Lucene50), sim=DefaultSimilarity, locale=zh, timezone=America/Antigua
[junit4] 2> NOTE: FreeBSD 9.1-RELEASE-p3 amd64/Oracle Corporation 1.7.0_65 (64-bit)/cpus=16,threads=1,free=205995568,total=379060224
[junit4] 2> NOTE: All tests run in this JVM: [TestJsonFacets, OverseerTest, TestInfoStreamLogging, TestExactSharedStatsCache, StressHdfsTest, TestCloudSchemaless, SolrCloudExampleTest, EnumFieldTest, ParsingFieldUpdateProcessorsTest, CollectionsAPIAsyncDistributedZkTest, AliasIntegrationTest, TestObjectReleaseTracker, PrimitiveFieldTypeTest, TestSolrConfigHandlerCloud, TestFaceting, TestAtomicUpdateErrorCases, TestXIncludeConfig, TestUpdate, DOMUtilTest, TestRequestStatusCollectionAPI, TestSolrJ, TestManagedStopFilterFactory, TestSortingResponseWriter, TestCloudInspectUtil, QueryParsingTest, TestTrackingShardHandlerFactory, TestDocSet, TestSolrIndexConfig, RecoveryZkTest]
[junit4] Completed [114/490] on J0 in 174.77s, 1 test, 1 failure <<< FAILURES!
[...truncated 1218 lines...]
BUILD FAILED
/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:543: The following error occurred while executing this line:
/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:484: The following error occurred while executing this line:
/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:61: The following error occurred while executing this line:
/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/extra-targets.xml:39: The following error occurred while executing this line:
/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build.xml:229: The following error occurred while executing this line:
/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/common-build.xml:511: The following error occurred while executing this line:
/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1433: The following error occurred while executing this line:
/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:991: There were test failures: 490 suites, 1960 tests, 1 failure, 65 ignored (11 assumptions)
Total time: 147 minutes 47 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Sending artifact delta relative to Lucene-Solr-NightlyTests-5.x #846
Archived 5 artifacts
Archive block size is 32768
Received 0 blocks and 813062149 bytes
Compression is 0.0%
Took 2 min 24 sec
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure
[JENKINS] Lucene-Solr-NightlyTests-5.x - Build # 850 - Still
Failing
Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-5.x/850/
3 tests failed.
REGRESSION: org.apache.solr.cloud.ChaosMonkeySafeLeaderTest.test
Error Message:
shard1 is not consistent. Got 1026 from https://127.0.0.1:54805/collection1lastClient and got 522 from https://127.0.0.1:54817/collection1
Stack Trace:
java.lang.AssertionError: shard1 is not consistent. Got 1026 from https://127.0.0.1:54805/collection1lastClient and got 522 from https://127.0.0.1:54817/collection1
at __randomizedtesting.SeedInfo.seed([8689C038CB394CF:803CA3D9224FF937]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.apache.solr.cloud.AbstractFullDistribZkTestBase.checkShardConsistency(AbstractFullDistribZkTestBase.java:1293)
at org.apache.solr.cloud.AbstractFullDistribZkTestBase.checkShardConsistency(AbstractFullDistribZkTestBase.java:1272)
at org.apache.solr.cloud.ChaosMonkeySafeLeaderTest.test(ChaosMonkeySafeLeaderTest.java:174)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1627)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:836)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:872)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:886)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:960)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:935)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:845)
at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:747)
at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:781)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:792)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
at java.lang.Thread.run(Thread.java:745)
REGRESSION: org.apache.solr.cloud.RecoveryZkTest.test
Error Message:
shard1 is not consistent. Got 835 from http://127.0.0.1:46708/collection1lastClient and got 215 from http://127.0.0.1:46723/collection1
Stack Trace:
java.lang.AssertionError: shard1 is not consistent. Got 835 from http://127.0.0.1:46708/collection1lastClient and got 215 from http://127.0.0.1:46723/collection1
at __randomizedtesting.SeedInfo.seed([8689C038CB394CF:803CA3D9224FF937]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.apache.solr.cloud.RecoveryZkTest.test(RecoveryZkTest.java:123)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1627)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:836)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:872)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:886)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:960)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:935)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:845)
at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:747)
at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:781)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:792)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
at java.lang.Thread.run(Thread.java:745)
FAILED: junit.framework.TestSuite.org.apache.solr.search.QueryEqualityTest
Error Message:
testParserCoverage was run w/o any other method explicitly testing val parser: agg_hll
Stack Trace:
java.lang.AssertionError: testParserCoverage was run w/o any other method explicitly testing val parser: agg_hll
at __randomizedtesting.SeedInfo.seed([8689C038CB394CF]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.apache.solr.search.QueryEqualityTest.afterClassParserCoverageTest(QueryEqualityTest.java:63)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1627)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:799)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
at java.lang.Thread.run(Thread.java:745)
Build Log:
[...truncated 10164 lines...]
[junit4] Suite: org.apache.solr.search.QueryEqualityTest
[junit4] 2> Creating dataDir: /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.search.QueryEqualityTest 8689C038CB394CF-001/init-core-data-001
[junit4] 2> 212245 T573 oas.SolrTestCaseJ4.buildSSLConfig Randomized ssl (true) and clientAuth (true)
[junit4] 2> 212248 T573 oas.SolrTestCaseJ4.initCore ####initCore
[junit4] 2> 212248 T573 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/'
[junit4] 2> 212250 T573 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/lib/classes/' to classloader
[junit4] 2> 212251 T573 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/lib/.svn/' to classloader
[junit4] 2> 212251 T573 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/lib/README' to classloader
[junit4] 2> 212434 T573 oasc.SolrConfig.refreshRequestParams current version of requestparams : -1
[junit4] 2> 212479 T573 oasc.SolrConfig.<init> Using Lucene MatchVersion: 5.2.0
[junit4] 2> 212559 T573 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4] 2> 212560 T573 oass.IndexSchema.readSchema Reading Solr Schema from /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/schema15.xml
[junit4] 2> 212589 T573 oass.IndexSchema.readSchema [null] Schema name=test
[junit4] 2> 213003 T573 oass.IndexSchema.readSchema default search field in schema is text
[junit4] 2> 213007 T573 oass.IndexSchema.readSchema unique key field: id
[junit4] 2> 213009 T573 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 213015 T573 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 213040 T573 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4] 2> 213040 T573 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr
[junit4] 2> 213040 T573 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/'
[junit4] 2> 213060 T573 oasc.CoreContainer.<init> New CoreContainer 2008849676
[junit4] 2> 213060 T573 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/]
[junit4] 2> 213061 T573 oasc.CoreContainer.load loading shared library: /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/lib
[junit4] 2> 213062 T573 oasc.SolrResourceLoader.addToClassLoader WARN Can't find (or read) directory to add to classloader: lib (resolved as: /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/lib).
[junit4] 2> 213076 T573 oashc.HttpShardHandlerFactory.init created with socketTimeout : 600000,connTimeout : 60000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
[junit4] 2> 213095 T573 oasu.UpdateShardHandler.<init> Creating UpdateShardHandler HTTP client with params: socketTimeout=30000&connTimeout=30000&retry=true
[junit4] 2> 213100 T573 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
[junit4] 2> 213100 T573 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
[junit4] 2> 213101 T573 oasc.CoreContainer.load Node Name: testNode
[junit4] 2> 213103 T573 oasc.CoreDescriptor.<init> CORE DESCRIPTOR: {collection=collection1, shard=shard1, name=collection1, config=solrconfig.xml, schema=schema15.xml, absoluteInstDir=/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/, instanceDir=collection1, transient=false, loadOnStartup=true, dataDir=/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.search.QueryEqualityTest 8689C038CB394CF-001/init-core-data-001}
[junit4] 2> 213104 T574 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/'
[junit4] 2> 213106 T574 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/lib/classes/' to classloader
[junit4] 2> 213106 T574 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/lib/.svn/' to classloader
[junit4] 2> 213106 T574 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/lib/README' to classloader
[junit4] 2> 213150 T574 oasc.SolrConfig.refreshRequestParams current version of requestparams : -1
[junit4] 2> 213186 T574 oasc.SolrConfig.<init> Using Lucene MatchVersion: 5.2.0
[junit4] 2> 213273 T574 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4] 2> 213274 T574 oass.IndexSchema.readSchema Reading Solr Schema from /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/schema15.xml
[junit4] 2> 213299 T574 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4] 2> 213674 T574 oass.IndexSchema.readSchema default search field in schema is text
[junit4] 2> 213676 T574 oass.IndexSchema.readSchema unique key field: id
[junit4] 2> 213677 T574 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 213681 T574 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 213700 T574 oasc.CoreContainer.create Creating SolrCore 'collection1' using configuration from instancedir /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/
[junit4] 2> 213700 T574 x:collection1 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4] 2> 213701 T574 x:collection1 oasc.SolrCore.<init> [[collection1] ] Opening new SolrCore at [/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/], dataDir=[null]
[junit4] 2> 213701 T574 x:collection1 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@2defdf33
[junit4] 2> 213701 T574 x:collection1 oasc.CachingDirectoryFactory.get return new directory for /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.search.QueryEqualityTest 8689C038CB394CF-001/init-core-data-001
[junit4] 2> 213702 T574 x:collection1 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.search.QueryEqualityTest 8689C038CB394CF-001/init-core-data-001/index/
[junit4] 2> 213702 T574 x:collection1 oasc.SolrCore.initIndex WARN [collection1] Solr index directory '/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.search.QueryEqualityTest 8689C038CB394CF-001/init-core-data-001/index' doesn't exist. Creating new index...
[junit4] 2> 213702 T574 x:collection1 oasc.CachingDirectoryFactory.get return new directory for /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.search.QueryEqualityTest 8689C038CB394CF-001/init-core-data-001/index
[junit4] 2> 213703 T574 x:collection1 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.MockRandomMergePolicy: org.apache.lucene.index.MockRandomMergePolicy@528dea14
[junit4] 2> 213703 T574 x:collection1 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4] 2> commit{dir=MockDirectoryWrapper(NRTCachingDirectory(RAMDirectory@6302f163 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@5635cd5d; maxCacheMB=0.029296875 maxMergeSizeMB=0.53125)),segFN=segments_1,generation=1}
[junit4] 2> 213704 T574 x:collection1 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 213710 T574 x:collection1 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe"
[junit4] 2> 213710 T574 x:collection1 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4] 2> 213711 T574 x:collection1 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe-allfields"
[junit4] 2> 213711 T574 x:collection1 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe-allfields"
[junit4] 2> 213711 T574 x:collection1 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "stored_sig"
[junit4] 2> 213711 T574 x:collection1 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4] 2> 213711 T574 x:collection1 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "uniq-fields"
[junit4] 2> 213712 T574 x:collection1 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "uniq-fields"
[junit4] 2> 213712 T574 x:collection1 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
[junit4] 2> 213712 T574 x:collection1 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 213713 T574 x:collection1 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 213713 T574 x:collection1 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
[junit4] 2> 213726 T574 x:collection1 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 213728 T574 x:collection1 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 213730 T574 x:collection1 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 213732 T574 x:collection1 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 213738 T574 x:collection1 oasc.PluginBag.createPlugin requestHandler : 'lazy' created with startup=lazy
[junit4] 2> 213741 T574 x:collection1 oasc.RequestHandlers.initHandlersFromConfig Registered paths: /debug/dump,dismax,/admin/threads,/mlt,/update/json/docs,/config,/admin/system,standard,/get,/replication,defaults,/admin/luke,/search-facet-def,/admin/logging,spellCheckCompRH_Direct,spellCheckWithWordbreak,spellCheckCompRH,lazy,/search-facet-invariants,/admin/file,/admin/plugins,tvrh,/terms,/admin/segments,spellCheckCompRH1,/update,/admin/ping,/update/csv,/admin/properties,mock,spellCheckWithWordbreak_Direct,/update/json,/schema,mltrh,/admin/mbeans
[junit4] 2> 213742 T574 x:collection1 oasc.SolrCore.initDeprecatedSupport WARN solrconfig.xml uses deprecated <admin/gettableFiles>, Please update your config to use the ShowFileRequestHandler.
[junit4] 2> 213743 T574 x:collection1 oasc.SolrCore.initStatsCache Using default statsCache cache: org.apache.solr.search.stats.LocalStatsCache
[junit4] 2> 213744 T574 x:collection1 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4] 2> 213744 T574 x:collection1 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4] 2> 213745 T574 x:collection1 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=47, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.8306424123035618]
[junit4] 2> 213745 T574 x:collection1 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4] 2> commit{dir=MockDirectoryWrapper(NRTCachingDirectory(RAMDirectory@6302f163 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@5635cd5d; maxCacheMB=0.029296875 maxMergeSizeMB=0.53125)),segFN=segments_1,generation=1}
[junit4] 2> 213745 T574 x:collection1 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 213746 T574 x:collection1 oass.SolrIndexSearcher.<init> Opening Searcher@5754ae8e[collection1] main
[junit4] 2> 213746 T574 x:collection1 oasr.RestManager.init Initializing RestManager with initArgs: {storageIO=org.apache.solr.rest.ManagedResourceStorage$InMemoryStorageIO}
[junit4] 2> 213747 T574 x:collection1 oasr.ManagedResourceStorage.load Reading _rest_managed.json using InMemoryStorage
[junit4] 2> 213747 T574 x:collection1 oasr.RestManager.init Initializing 0 registered ManagedResources
[junit4] 2> 213747 T574 x:collection1 oashc.SpellCheckComponent.inform Initializing spell checkers
[junit4] 2> 213776 T574 x:collection1 oass.DirectSolrSpellChecker.init init: {name=direct,classname=DirectSolrSpellChecker,field=lowerfilt,minQueryLength=3}
[junit4] 2> 213827 T574 x:collection1 oash.ReplicationHandler.inform Commits will be reserved for 10000
[junit4] 2> 213827 T575 x:collection1 oashc.SpellCheckComponent$SpellCheckerListener.newSearcher Loading spell index for spellchecker: default
[junit4] 2> 213828 T575 x:collection1 oashc.SpellCheckComponent$SpellCheckerListener.newSearcher Loading spell index for spellchecker: direct
[junit4] 2> 213828 T575 x:collection1 oashc.SpellCheckComponent$SpellCheckerListener.newSearcher Loading spell index for spellchecker: wordbreak
[junit4] 2> 213828 T575 x:collection1 oashc.SpellCheckComponent$SpellCheckerListener.newSearcher Loading spell index for spellchecker: multipleFields
[junit4] 2> 213828 T574 x:collection1 oasc.CoreContainer.registerCore registering core: collection1
[junit4] 2> 213829 T575 x:collection1 oashc.SpellCheckComponent$SpellCheckerListener.newSearcher Loading spell index for spellchecker: jarowinkler
[junit4] 2> 213830 T575 x:collection1 oashc.SpellCheckComponent$SpellCheckerListener.newSearcher Loading spell index for spellchecker: external
[junit4] 2> 213830 T575 x:collection1 oashc.SpellCheckComponent$SpellCheckerListener.newSearcher Loading spell index for spellchecker: freq
[junit4] 2> 213831 T573 oas.SolrTestCaseJ4.initCore ####initCore end
[junit4] 2> 213831 T575 x:collection1 oashc.SpellCheckComponent$SpellCheckerListener.newSearcher Loading spell index for spellchecker: fqcn
[junit4] 2> 213832 T575 x:collection1 oashc.SpellCheckComponent$SpellCheckerListener.newSearcher Loading spell index for spellchecker: perDict
[junit4] 2> 213833 T575 x:collection1 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@5754ae8e[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 213835 T573 oas.SolrTestCaseJ4.setUp ###Starting testQueryLucenePlusSort
[junit4] 2> 213854 T573 oas.SolrTestCaseJ4.tearDown ###Ending testQueryLucenePlusSort
[junit4] 2> 213858 T573 oas.SolrTestCaseJ4.setUp ###Starting testFuncSingleValueMathFuncs
[junit4] 2> 213876 T573 oas.SolrTestCaseJ4.tearDown ###Ending testFuncSingleValueMathFuncs
[junit4] 2> 213880 T573 oas.SolrTestCaseJ4.setUp ###Starting testFuncField
[junit4] 2> 213881 T573 oas.SolrTestCaseJ4.tearDown ###Ending testFuncField
[junit4] 2> 213884 T573 oas.SolrTestCaseJ4.setUp ###Starting testQueryComplexPhrase
[junit4] 2> 213987 T573 oas.SolrTestCaseJ4.tearDown ###Ending testQueryComplexPhrase
[junit4] 2> 213991 T573 oas.SolrTestCaseJ4.setUp ###Starting testBlockJoin
[junit4] 2> 214018 T573 oas.SolrTestCaseJ4.tearDown ###Ending testBlockJoin
[junit4] 2> 214022 T573 oas.SolrTestCaseJ4.setUp ###Starting testFuncTerms
[junit4] 2> 214072 T573 oas.SolrTestCaseJ4.tearDown ###Ending testFuncTerms
[junit4] 2> 214076 T573 oas.SolrTestCaseJ4.setUp ###Starting testFuncSqedist
[junit4] 2> 214110 T573 oas.SolrTestCaseJ4.tearDown ###Ending testFuncSqedist
[junit4] 2> 214114 T573 oas.SolrTestCaseJ4.setUp ###Starting testFuncMin
[junit4] 2> 214115 T573 oas.SolrTestCaseJ4.tearDown ###Ending testFuncMin
[junit4] 2> 214118 T573 oas.SolrTestCaseJ4.setUp ###Starting testFuncNumdocs
[junit4] 2> 214118 T573 oas.SolrTestCaseJ4.tearDown ###Ending testFuncNumdocs
[junit4] 2> 214122 T573 oas.SolrTestCaseJ4.setUp ###Starting testFuncOrd
[junit4] 2> 214130 T573 oas.SolrTestCaseJ4.tearDown ###Ending testFuncOrd
[junit4] 2> 214134 T573 oas.SolrTestCaseJ4.setUp ###Starting testFuncTestfunc
[junit4] 2> 214135 T573 oas.SolrTestCaseJ4.tearDown ###Ending testFuncTestfunc
[junit4] 2> 214138 T573 oas.SolrTestCaseJ4.setUp ###Starting testAggs
[junit4] 2> 214328 T573 oas.SolrTestCaseJ4.tearDown ###Ending testAggs
[junit4] 2> 214333 T573 oas.SolrTestCaseJ4.setUp ###Starting testFuncGeohash
[junit4] 2> 214343 T573 oas.SolrTestCaseJ4.tearDown ###Ending testFuncGeohash
[junit4] 2> 214346 T573 oas.SolrTestCaseJ4.setUp ###Starting testQuerySimple
[junit4] 2> 214376 T573 oas.SolrTestCaseJ4.tearDown ###Ending testQuerySimple
[junit4] 2> 214379 T573 oas.SolrTestCaseJ4.setUp ###Starting testFuncLinear
[junit4] 2> 214390 T573 oas.SolrTestCaseJ4.tearDown ###Ending testFuncLinear
[junit4] 2> 214394 T573 oas.SolrTestCaseJ4.setUp ###Starting testQueryRaw
[junit4] 2> 214394 T573 oas.SolrTestCaseJ4.tearDown ###Ending testQueryRaw
[junit4] 2> 214398 T573 oas.SolrTestCaseJ4.setUp ###Starting testQueryLucene
[junit4] 2> 214399 T573 oas.SolrTestCaseJ4.tearDown ###Ending testQueryLucene
[junit4] 2> 214403 T573 oas.SolrTestCaseJ4.setUp ###Starting testFuncGeodist
[junit4] 2> 214423 T573 oas.SolrTestCaseJ4.tearDown ###Ending testFuncGeodist
[junit4] 2> 214427 T573 oas.SolrTestCaseJ4.setUp ###Starting testTerms
[junit4] 2> 214473 T573 oas.SolrTestCaseJ4.tearDown ###Ending testTerms
[junit4] 2> 214476 T573 oas.SolrTestCaseJ4.setUp ###Starting testFuncDiv
[junit4] 2> 214477 T573 oas.SolrTestCaseJ4.tearDown ###Ending testFuncDiv
[junit4] 2> 214481 T573 oas.SolrTestCaseJ4.setUp ###Starting testFuncVector
[junit4] 2> 214482 T573 oas.SolrTestCaseJ4.tearDown ###Ending testFuncVector
[junit4] 2> 214485 T573 oas.SolrTestCaseJ4.setUp ###Starting testFuncDist
[junit4] 2> 214486 T573 oas.SolrTestCaseJ4.tearDown ###Ending testFuncDist
[junit4] 2> 214489 T573 oas.SolrTestCaseJ4.setUp ###Starting testFuncHsin
[junit4] 2> 214498 T573 oas.SolrTestCaseJ4.tearDown ###Ending testFuncHsin
[junit4] 2> 214502 T573 oas.SolrTestCaseJ4.setUp ###Starting testFuncIf
[junit4] 2> 214525 T573 oas.SolrTestCaseJ4.tearDown ###Ending testFuncIf
[junit4] 2> 214529 T573 oas.SolrTestCaseJ4.setUp ###Starting testQueryGeofilt
[junit4] 2> 214562 T573 oas.SolrTestCaseJ4.tearDown ###Ending testQueryGeofilt
[junit4] 2> 214566 T573 oas.SolrTestCaseJ4.setUp ###Starting testFuncNorm
[junit4] 2> 214577 T573 oas.SolrTestCaseJ4.tearDown ###Ending testFuncNorm
[junit4] 2> 214580 T573 oas.SolrTestCaseJ4.setUp ###Starting testFuncBools
[junit4] 2> 214581 T573 oas.SolrTestCaseJ4.tearDown ###Ending testFuncBools
[junit4] 2> 214584 T573 oas.SolrTestCaseJ4.setUp ###Starting testQueryPrefix
[junit4] 2> 214585 T573 oas.SolrTestCaseJ4.tearDown ###Ending testQueryPrefix
[junit4] 2> 214588 T573 oas.SolrTestCaseJ4.setUp ###Starting testParserCoverage
[junit4] 2> 214588 T573 oas.SolrTestCaseJ4.tearDown ###Ending testParserCoverage
[junit4] 2> 214591 T573 oas.SolrTestCaseJ4.setUp ###Starting testQueryBbox
[junit4] 2> 214596 T573 oas.SolrTestCaseJ4.tearDown ###Ending testQueryBbox
[junit4] 2> 214600 T573 oas.SolrTestCaseJ4.setUp ###Starting testReRankQuery
[junit4] 2> 214619 T573 oas.SolrTestCaseJ4.tearDown ###Ending testReRankQuery
[junit4] 2> 214623 T573 oas.SolrTestCaseJ4.setUp ###Starting testQuerySurround
[junit4] 2> 214929 T573 oas.SolrTestCaseJ4.tearDown ###Ending testQuerySurround
[junit4] 2> 214933 T573 oas.SolrTestCaseJ4.setUp ###Starting testQueryCollapse
[junit4] 2> 214940 T573 oas.SolrTestCaseJ4.tearDown ###Ending testQueryCollapse
[junit4] 2> 214943 T573 oas.SolrTestCaseJ4.setUp ###Starting testQueryNested
[junit4] 2> 214944 T573 oas.SolrTestCaseJ4.tearDown ###Ending testQueryNested
[junit4] 2> 214948 T573 oas.SolrTestCaseJ4.setUp ###Starting testFuncRord
[junit4] 2> 214956 T573 oas.SolrTestCaseJ4.tearDown ###Ending testFuncRord
[junit4] 2> 214960 T573 oas.SolrTestCaseJ4.setUp ###Starting testFuncScale
[junit4] 2> 214971 T573 oas.SolrTestCaseJ4.tearDown ###Ending testFuncScale
[junit4] 2> 214975 T573 oas.SolrTestCaseJ4.setUp ###Starting testField
[junit4] 2> 214975 T573 oas.SolrTestCaseJ4.tearDown ###Ending testField
[junit4] 2> 214978 T573 oas.SolrTestCaseJ4.setUp ###Starting testFuncGhhsin
[junit4] 2> 214979 T573 oas.SolrTestCaseJ4.tearDown ###Ending testFuncGhhsin
[junit4] 2> 214982 T573 oas.SolrTestCaseJ4.setUp ###Starting testFuncMax
[junit4] 2> 214983 T573 oas.SolrTestCaseJ4.tearDown ###Ending testFuncMax
[junit4] 2> 214986 T573 oas.SolrTestCaseJ4.setUp ###Starting testQueryJoin
[junit4] 2> 215002 T573 oas.SolrTestCaseJ4.tearDown ###Ending testQueryJoin
[junit4] 2> 215006 T573 oas.SolrTestCaseJ4.setUp ###Starting testFuncRecip
[junit4] 2> 215018 T573 oas.SolrTestCaseJ4.tearDown ###Ending testFuncRecip
[junit4] 2> 215021 T573 oas.SolrTestCaseJ4.setUp ###Starting testFuncMod
[junit4] 2> 215022 T573 oas.SolrTestCaseJ4.tearDown ###Ending testFuncMod
[junit4] 2> 215026 T573 oas.SolrTestCaseJ4.setUp ###Starting testFuncMap
[junit4] 2> 215036 T573 oas.SolrTestCaseJ4.tearDown ###Ending testFuncMap
[junit4] 2> 215040 T573 oas.SolrTestCaseJ4.setUp ###Starting testFuncNot
[junit4] 2> 215041 T573 oas.SolrTestCaseJ4.tearDown ###Ending testFuncNot
[junit4] 2> 215045 T573 oas.SolrTestCaseJ4.setUp ###Starting testQueryMaxScore
[junit4] 2> 215048 T573 oas.SolrTestCaseJ4.tearDown ###Ending testQueryMaxScore
[junit4] 2> 215051 T573 oas.SolrTestCaseJ4.setUp ###Starting testQueryMLT
[junit4] 2> ASYNC NEW_CORE C65 name=collection1 org.apache.solr.core.SolrCore@11589f6
[junit4] 2> 215053 T573 C65 oasup.LogUpdateProcessor.finish [collection1] webapp=null path=null params={} {add=[1]} 0 1
[junit4] 2> 215083 T573 C65 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4] 2> 215086 T573 C65 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4] 2> commit{dir=MockDirectoryWrapper(NRTCachingDirectory(RAMDirectory@6302f163 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@5635cd5d; maxCacheMB=0.029296875 maxMergeSizeMB=0.53125)),segFN=segments_1,generation=1}
[junit4] 2> commit{dir=MockDirectoryWrapper(NRTCachingDirectory(RAMDirectory@6302f163 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@5635cd5d; maxCacheMB=0.029296875 maxMergeSizeMB=0.53125)),segFN=segments_2,generation=2}
[junit4] 2> 215087 T573 C65 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 2
[junit4] 2> 215088 T573 C65 oass.SolrIndexSearcher.<init> Opening Searcher@78e763da[collection1] main
[junit4] 2> 215088 T573 C65 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4] 2> 215090 T575 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@78e763da[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_0(5.2.0):C1)))}
[junit4] 2> 215091 T573 C65 oasup.LogUpdateProcessor.finish [collection1] webapp=null path=null params={} {commit=} 0 8
[junit4] 2> 215102 T573 C65 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4] 2> 215103 T573 C65 oasu.DirectUpdateHandler2.commit No uncommitted changes. Skipping IW.commit.
[junit4] 2> 215103 T573 C65 oasc.SolrCore.openNewSearcher SolrIndexSearcher has not changed - not re-opening: org.apache.solr.search.SolrIndexSearcher
[junit4] 2> 215103 T573 C65 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4] 2> 215104 T573 C65 oasup.LogUpdateProcessor.finish [collection1] webapp=null path=null params={} {commit=} 0 2
[junit4] 2> 215105 T573 oas.SolrTestCaseJ4.tearDown ###Ending testQueryMLT
[junit4] 2> 215109 T573 oas.SolrTestCaseJ4.setUp ###Starting testFuncStrdist
[junit4] 2> 215128 T573 oas.SolrTestCaseJ4.tearDown ###Ending testFuncStrdist
[junit4] 2> 215132 T573 oas.SolrTestCaseJ4.setUp ###Starting testFuncLiteral
[junit4] 2> 215133 T573 oas.SolrTestCaseJ4.tearDown ###Ending testFuncLiteral
[junit4] 2> 215136 T573 oas.SolrTestCaseJ4.setUp ###Starting testQueryFunc
[junit4] 2> 215138 T573 oas.SolrTestCaseJ4.tearDown ###Ending testQueryFunc
[junit4] 2> 215141 T573 oas.SolrTestCaseJ4.setUp ###Starting testFuncCurrency
[junit4] 2> 215157 T573 oas.SolrTestCaseJ4.tearDown ###Ending testFuncCurrency
[junit4] 2> 215161 T573 oas.SolrTestCaseJ4.setUp ###Starting testFuncBoost
[junit4] 2> 215181 T573 oas.SolrTestCaseJ4.tearDown ###Ending testFuncBoost
[junit4] 2> 215185 T573 oas.SolrTestCaseJ4.setUp ###Starting testFuncCscore
[junit4] 2> 215193 T573 oas.SolrTestCaseJ4.tearDown ###Ending testFuncCscore
[junit4] 2> 215197 T573 oas.SolrTestCaseJ4.setUp ###Starting testQueryBoost
[junit4] 2> 215198 T573 oas.SolrTestCaseJ4.tearDown ###Ending testQueryBoost
[junit4] 2> 215201 T573 oas.SolrTestCaseJ4.setUp ###Starting testDateMathParsingEquality
[junit4] 2> 215204 T573 oas.SolrTestCaseJ4.tearDown ###Ending testDateMathParsingEquality
[junit4] 2> 215207 T573 oas.SolrTestCaseJ4.setUp ###Starting testFuncDef
[junit4] 2> 215229 T573 oas.SolrTestCaseJ4.tearDown ###Ending testFuncDef
[junit4] 2> 215232 T573 oas.SolrTestCaseJ4.setUp ###Starting testFuncQuery
[junit4] 2> 215233 T573 oas.SolrTestCaseJ4.tearDown ###Ending testFuncQuery
[junit4] 2> 215237 T573 oas.SolrTestCaseJ4.setUp ###Starting testFuncDoubleValueBools
[junit4] 2> 215250 T573 oas.SolrTestCaseJ4.tearDown ###Ending testFuncDoubleValueBools
[junit4] 2> 215254 T573 oas.SolrTestCaseJ4.setUp ###Starting testFuncTop
[junit4] 2> 215254 T573 oas.SolrTestCaseJ4.tearDown ###Ending testFuncTop
[junit4] 2> 215257 T573 oas.SolrTestCaseJ4.setUp ###Starting testExportQuery
[junit4] 2> 215270 T573 oas.SolrTestCaseJ4.tearDown ###Ending testExportQuery
[junit4] 2> 215274 T573 oas.SolrTestCaseJ4.setUp ###Starting testTestFuncs
[junit4] 2> 215278 T573 oas.SolrTestCaseJ4.tearDown ###Ending testTestFuncs
[junit4] 2> 215281 T573 oas.SolrTestCaseJ4.setUp ###Starting testFuncMs
[junit4] 2> 215282 T573 oas.SolrTestCaseJ4.tearDown ###Ending testFuncMs
[junit4] 2> 215286 T573 oas.SolrTestCaseJ4.setUp ###Starting testFuncSub
[junit4] 2> 215286 T573 oas.SolrTestCaseJ4.tearDown ###Ending testFuncSub
[junit4] 2> 215290 T573 oas.SolrTestCaseJ4.setUp ###Starting testQueryFrange
[junit4] 2> 215290 T573 oas.SolrTestCaseJ4.tearDown ###Ending testQueryFrange
[junit4] 2> 215293 T573 oas.SolrTestCaseJ4.setUp ###Starting testFuncMathConsts
[junit4] 2> 215294 T573 oas.SolrTestCaseJ4.tearDown ###Ending testFuncMathConsts
[junit4] 2> 215297 T573 oas.SolrTestCaseJ4.setUp ###Starting testFuncMaxdoc
[junit4] 2> 215297 T573 oas.SolrTestCaseJ4.tearDown ###Ending testFuncMaxdoc
[junit4] 2> 215301 T573 oas.SolrTestCaseJ4.setUp ###Starting testFuncExists
[junit4] 2> 215319 T573 oas.SolrTestCaseJ4.tearDown ###Ending testFuncExists
[junit4] 2> 215322 T573 oas.SolrTestCaseJ4.setUp ###Starting testFuncSum
[junit4] 2> 215323 T573 oas.SolrTestCaseJ4.tearDown ###Ending testFuncSum
[junit4] 2> 215327 T573 oas.SolrTestCaseJ4.setUp ###Starting testHash
[junit4] 2> 215350 T573 oas.SolrTestCaseJ4.tearDown ###Ending testHash
[junit4] 2> 215354 T573 oas.SolrTestCaseJ4.setUp ###Starting testQueryTerm
[junit4] 2> 215355 T573 oas.SolrTestCaseJ4.tearDown ###Ending testQueryTerm
[junit4] 2> 215358 T573 oas.SolrTestCaseJ4.setUp ###Starting testQueryDismax
[junit4] 2> 215364 T573 oas.SolrTestCaseJ4.tearDown ###Ending testQueryDismax
[junit4] 2> 215368 T573 oas.SolrTestCaseJ4.setUp ###Starting testFuncJoindf
[junit4] 2> 215380 T573 oas.SolrTestCaseJ4.tearDown ###Ending testFuncJoindf
[junit4] 2> 215383 T573 oas.SolrTestCaseJ4.setUp ###Starting testQuerySwitch
[junit4] 2> 215386 T573 oas.SolrTestCaseJ4.tearDown ###Ending testQuerySwitch
[junit4] 2> 215389 T573 oas.SolrTestCaseJ4.setUp ###Starting testFuncSttf
[junit4] 2> 215390 T573 oas.SolrTestCaseJ4.tearDown ###Ending testFuncSttf
[junit4] 2> 215393 T573 oas.SolrTestCaseJ4.setUp ###Starting testFuncProduct
[junit4] 2> 215394 T573 oas.SolrTestCaseJ4.tearDown ###Ending testFuncProduct
[junit4] 2> 215398 T573 oas.SolrTestCaseJ4.setUp ###Starting testFuncDoubleValueMathFuncs
[junit4] 2> 215405 T573 oas.SolrTestCaseJ4.tearDown ###Ending testFuncDoubleValueMathFuncs
[junit4] 2> 215406 T573 oas.SolrTestCaseJ4.deleteCore ###deleteCore
[junit4] 2> 215407 T573 oasc.CoreContainer.shutdown Shutting down CoreContainer instance=2008849676
[junit4] 2> 215407 T573 oasc.SolrCore.close [collection1] CLOSING SolrCore org.apache.solr.core.SolrCore@11589f6
[junit4] 2> 215407 T573 oasu.DirectUpdateHandler2.close closing DirectUpdateHandler2{commits=2,autocommits=0,soft autocommits=0,optimizes=0,rollbacks=0,expungeDeletes=0,docsPending=0,adds=0,deletesById=0,deletesByQuery=0,errors=0,cumulative_adds=1,cumulative_deletesById=0,cumulative_deletesByQuery=0,cumulative_errors=0}
[junit4] 2> 215407 T573 oasu.SolrCoreState.decrefSolrCoreState Closing SolrCoreState
[junit4] 2> 215408 T573 oasu.DefaultSolrCoreState.closeIndexWriter SolrCoreState ref count has reached 0 - closing IndexWriter
[junit4] 2> 215408 T573 oasu.DefaultSolrCoreState.closeIndexWriter closing IndexWriter with IndexWriterCloser
[junit4] 2> 215409 T573 oasc.SolrCore.closeSearcher [collection1] Closing main searcher on request.
[junit4] 2> 215425 T573 oasc.CachingDirectoryFactory.close Closing MockDirectoryFactory - 2 directories currently being tracked
[junit4] 2> 215426 T573 oasc.CachingDirectoryFactory.closeCacheValue looking to close /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.search.QueryEqualityTest 8689C038CB394CF-001/init-core-data-001 [CachedDir<<refCount=0;path=/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.search.QueryEqualityTest 8689C038CB394CF-001/init-core-data-001;done=false>>]
[junit4] 2> 215426 T573 oasc.CachingDirectoryFactory.close Closing directory: /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.search.QueryEqualityTest 8689C038CB394CF-001/init-core-data-001
[junit4] 2> 215426 T573 oasc.CachingDirectoryFactory.closeCacheValue looking to close /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.search.QueryEqualityTest 8689C038CB394CF-001/init-core-data-001/index [CachedDir<<refCount=0;path=/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.search.QueryEqualityTest 8689C038CB394CF-001/init-core-data-001/index;done=false>>]
[junit4] 2> 215426 T573 oasc.CachingDirectoryFactory.close Closing directory: /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.search.QueryEqualityTest 8689C038CB394CF-001/init-core-data-001/index
[junit4] 2> NOTE: test params are: codec=FastCompressingStoredFields(storedFieldsFormat=CompressingStoredFieldsFormat(compressionMode=FAST, chunkSize=988, maxDocsPerChunk=5, blockSize=10), termVectorsFormat=CompressingTermVectorsFormat(compressionMode=FAST, chunkSize=988, blockSize=10)), sim=DefaultSimilarity, locale=hu_HU, timezone=Africa/Timbuktu
[junit4] 2> NOTE: FreeBSD 9.1-RELEASE-p3 amd64/Oracle Corporation 1.7.0_65 (64-bit)/cpus=16,threads=1,free=110259680,total=332922880
[junit4] 2> NOTE: All tests run in this JVM: [SolrPluginUtilsTest, MoreLikeThisHandlerTest, TestExtendedDismaxParser, TestFoldingMultitermQuery, DocValuesMissingTest, TestRandomMergePolicy, TestHighlightDedupGrouping, TestLuceneMatchVersion, RAMDirectoryFactoryTest, CloudMLTQParserTest, LukeRequestHandlerTest, TestRemoveLastDynamicCopyField, TestElisionMultitermQuery, TestPivotHelperCode, DirectUpdateHandlerOptimizeTest, TestCSVResponseWriter, DistributedFacetPivotLargeTest, SolrRequestParserTest, QueryEqualityTest]
[junit4] 2> NOTE: download the large Jenkins line-docs file by running 'ant get-jenkins-line-docs' in the lucene directory.
[junit4] 2> NOTE: reproduce with: ant test -Dtestcase=QueryEqualityTest -Dtests.seed=8689C038CB394CF -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/home/jenkins/lucene-data/enwiki.random.lines.txt -Dtests.locale=hu_HU -Dtests.timezone=Africa/Timbuktu -Dtests.asserts=true -Dtests.file.encoding=ISO-8859-1
[junit4] ERROR 0.00s J2 | QueryEqualityTest (suite) <<<
[junit4] > Throwable #1: java.lang.AssertionError: testParserCoverage was run w/o any other method explicitly testing val parser: agg_hll
[junit4] > at __randomizedtesting.SeedInfo.seed([8689C038CB394CF]:0)
[junit4] > at org.apache.solr.search.QueryEqualityTest.afterClassParserCoverageTest(QueryEqualityTest.java:63)
[junit4] > at java.lang.Thread.run(Thread.java:745)
[junit4] Completed [49/490] on J2 in 3.32s, 75 tests, 1 failure <<< FAILURES!
[...truncated 485 lines...]
[junit4] Suite: org.apache.solr.cloud.ChaosMonkeySafeLeaderTest
[junit4] 2> Creating dataDir: /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeySafeLeaderTest 8689C038CB394CF-001/init-core-data-001
[junit4] 2> 776554 T9705 oas.SolrTestCaseJ4.buildSSLConfig Randomized ssl (true) and clientAuth (false)
[junit4] 2> 776555 T9705 oas.BaseDistributedSearchTestCase.initHostContext Setting hostContext system property: /
[junit4] 2> 776566 T9705 oasc.ZkTestServer.run STARTING ZK TEST SERVER
[junit4] 2> 776566 T9706 oasc.ZkTestServer$2$1.setClientPort client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 776567 T9706 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
[junit4] 2> 776666 T9705 oasc.ZkTestServer.run start zk server on port:54778
[junit4] 2> 776667 T9705 oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default ZkCredentialsProvider
[junit4] 2> 776668 T9705 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 776674 T9713 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@6a4799c8 name:ZooKeeperConnection Watcher:127.0.0.1:54778 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 776675 T9705 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 776676 T9705 oascc.SolrZkClient.createZkACLProvider Using default ZkACLProvider
[junit4] 2> 776676 T9705 oascc.SolrZkClient.makePath makePath: /solr
[junit4] 2> 776680 T9705 oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default ZkCredentialsProvider
[junit4] 2> 776681 T9705 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 776683 T9716 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@7f3e78a6 name:ZooKeeperConnection Watcher:127.0.0.1:54778/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 776684 T9705 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 776684 T9705 oascc.SolrZkClient.createZkACLProvider Using default ZkACLProvider
[junit4] 2> 776685 T9705 oascc.SolrZkClient.makePath makePath: /collections/collection1
[junit4] 2> 776688 T9705 oascc.SolrZkClient.makePath makePath: /collections/collection1/shards
[junit4] 2> 776690 T9705 oascc.SolrZkClient.makePath makePath: /collections/control_collection
[junit4] 2> 776692 T9705 oascc.SolrZkClient.makePath makePath: /collections/control_collection/shards
[junit4] 2> 776694 T9705 oasc.AbstractZkTestCase.putConfig put /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
[junit4] 2> 776695 T9705 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.xml
[junit4] 2> 776699 T9705 oasc.AbstractZkTestCase.putConfig put /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/schema15.xml to /configs/conf1/schema.xml
[junit4] 2> 776700 T9705 oascc.SolrZkClient.makePath makePath: /configs/conf1/schema.xml
[junit4] 2> 776802 T9705 oasc.AbstractZkTestCase.putConfig put /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
[junit4] 2> 776803 T9705 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.snippet.randomindexconfig.xml
[junit4] 2> 776806 T9705 oasc.AbstractZkTestCase.putConfig put /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
[junit4] 2> 776807 T9705 oascc.SolrZkClient.makePath makePath: /configs/conf1/stopwords.txt
[junit4] 2> 776809 T9705 oasc.AbstractZkTestCase.putConfig put /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
[junit4] 2> 776810 T9705 oascc.SolrZkClient.makePath makePath: /configs/conf1/protwords.txt
[junit4] 2> 776812 T9705 oasc.AbstractZkTestCase.putConfig put /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
[junit4] 2> 776813 T9705 oascc.SolrZkClient.makePath makePath: /configs/conf1/currency.xml
[junit4] 2> 776815 T9705 oasc.AbstractZkTestCase.putConfig put /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
[junit4] 2> 776816 T9705 oascc.SolrZkClient.makePath makePath: /configs/conf1/enumsConfig.xml
[junit4] 2> 776822 T9705 oasc.AbstractZkTestCase.putConfig put /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
[junit4] 2> 776822 T9705 oascc.SolrZkClient.makePath makePath: /configs/conf1/open-exchange-rates.json
[junit4] 2> 776824 T9705 oasc.AbstractZkTestCase.putConfig put /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4] 2> 776825 T9705 oascc.SolrZkClient.makePath makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4] 2> 776827 T9705 oasc.AbstractZkTestCase.putConfig put /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
[junit4] 2> 776828 T9705 oascc.SolrZkClient.makePath makePath: /configs/conf1/old_synonyms.txt
[junit4] 2> 776831 T9705 oasc.AbstractZkTestCase.putConfig put /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
[junit4] 2> 776832 T9705 oascc.SolrZkClient.makePath makePath: /configs/conf1/synonyms.txt
[junit4] 2> 776835 T9705 oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default ZkCredentialsProvider
[junit4] 2> 776836 T9705 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 776838 T9719 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@e19a476 name:ZooKeeperConnection Watcher:127.0.0.1:54778/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 776838 T9705 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 776839 T9705 oascc.SolrZkClient.createZkACLProvider Using default ZkACLProvider
[junit4] 2> 778787 T9705 oas.SolrTestCaseJ4.writeCoreProperties Writing core.properties file to /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeySafeLeaderTest 8689C038CB394CF-001/control-001/cores/collection1
[junit4] 2> 778794 T9705 oejs.Server.doStart jetty-9.2.10.v20150310
[junit4] 2> 778797 T9705 oejsh.ContextHandler.doStart Started o.e.j.s.ServletContextHandler@30ec0677{/,null,AVAILABLE}
[junit4] 2> 778813 T9705 oejs.AbstractConnector.doStart Started ServerConnector@65926df7{SSL-http/1.1}{127.0.0.1:54788}
[junit4] 2> 778814 T9705 oejs.Server.doStart Started @786803ms
[junit4] 2> 778814 T9705 oascse.JettySolrRunner$1.lifeCycleStarted Jetty properties: {hostPort=54788, coreRootDirectory=/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeySafeLeaderTest 8689C038CB394CF-001/control-001/cores, solr.data.dir=/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeySafeLeaderTest 8689C038CB394CF-001/tempDir-001/control/data, hostContext=/}
[junit4] 2> 778820 T9705 oass.SolrDispatchFilter.init SolrDispatchFilter.init()sun.misc.Launcher$AppClassLoader@749cd006
[junit4] 2> 778820 T9705 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeySafeLeaderTest 8689C038CB394CF-001/control-001/'
[junit4] 2> 778892 T9705 oasc.SolrXmlConfig.fromFile Loading container configuration from /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeySafeLeaderTest 8689C038CB394CF-001/control-001/solr.xml
[junit4] 2> 778993 T9705 oasc.CorePropertiesLocator.<init> Config-defined core root directory: /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeySafeLeaderTest 8689C038CB394CF-001/control-001/cores
[junit4] 2> 778994 T9705 oasc.CoreContainer.<init> New CoreContainer 1181255293
[junit4] 2> 778994 T9705 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeySafeLeaderTest 8689C038CB394CF-001/control-001/]
[junit4] 2> 778994 T9705 oasc.CoreContainer.load loading shared library: /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeySafeLeaderTest 8689C038CB394CF-001/control-001/lib
[junit4] 2> 778995 T9705 oasc.SolrResourceLoader.addToClassLoader WARN Can't find (or read) directory to add to classloader: lib (resolved as: /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeySafeLeaderTest 8689C038CB394CF-001/control-001/lib).
[junit4] 2> 779011 T9705 oashc.HttpShardHandlerFactory.init created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
[junit4] 2> 779031 T9705 oasu.UpdateShardHandler.<init> Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
[junit4] 2> 779035 T9705 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
[junit4] 2> 779035 T9705 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
[junit4] 2> 779036 T9705 oasc.CoreContainer.load Node Name: 127.0.0.1
[junit4] 2> 779036 T9705 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:54778/solr
[junit4] 2> 779036 T9705 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4] 2> 779036 T9705 oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default ZkCredentialsProvider
[junit4] 2> 779037 T9705 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 779039 T9733 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@3291676e name:ZooKeeperConnection Watcher:127.0.0.1:54778 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 779040 T9705 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 779040 T9705 oascc.SolrZkClient.createZkACLProvider Using default ZkACLProvider
[junit4] 2> 779043 T9705 n:127.0.0.1:54788_ oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 779045 T9736 n:127.0.0.1:54788_ oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@4eb815aa name:ZooKeeperConnection Watcher:127.0.0.1:54778/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 779045 T9705 n:127.0.0.1:54788_ oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 779047 T9705 n:127.0.0.1:54788_ oascc.SolrZkClient.makePath makePath: /overseer/queue
[junit4] 2> 779050 T9705 n:127.0.0.1:54788_ oascc.SolrZkClient.makePath makePath: /overseer/collection-queue-work
[junit4] 2> 779052 T9705 n:127.0.0.1:54788_ oascc.SolrZkClient.makePath makePath: /overseer/collection-map-running
[junit4] 2> 779055 T9705 n:127.0.0.1:54788_ oascc.SolrZkClient.makePath makePath: /overseer/collection-map-completed
[junit4] 2> 779057 T9705 n:127.0.0.1:54788_ oascc.SolrZkClient.makePath makePath: /overseer/collection-map-failure
[junit4] 2> 779060 T9705 n:127.0.0.1:54788_ oascc.SolrZkClient.makePath makePath: /live_nodes
[junit4] 2> 779062 T9705 n:127.0.0.1:54788_ oascc.SolrZkClient.makePath makePath: /aliases.json
[junit4] 2> 779064 T9705 n:127.0.0.1:54788_ oascc.SolrZkClient.makePath makePath: /clusterstate.json
[junit4] 2> 779065 T9705 n:127.0.0.1:54788_ oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:54788_
[junit4] 2> 779066 T9705 n:127.0.0.1:54788_ oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:54788_
[junit4] 2> 779068 T9705 n:127.0.0.1:54788_ oascc.SolrZkClient.makePath makePath: /overseer_elect
[junit4] 2> 779070 T9705 n:127.0.0.1:54788_ oascc.SolrZkClient.makePath makePath: /overseer_elect/election
[junit4] 2> 779072 T9705 n:127.0.0.1:54788_ oasc.Overseer.close Overseer (id=null) closing
[junit4] 2> 779074 T9705 n:127.0.0.1:54788_ oasc.OverseerElectionContext.runLeaderProcess I am going to be the leader 127.0.0.1:54788_
[junit4] 2> 779074 T9705 n:127.0.0.1:54788_ oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
[junit4] 2> 779076 T9705 n:127.0.0.1:54788_ oasc.Overseer.start Overseer (id=93840094544396292-127.0.0.1:54788_-n_0000000000) starting
[junit4] 2> 779079 T9705 n:127.0.0.1:54788_ oascc.SolrZkClient.makePath makePath: /overseer/queue-work
[junit4] 2> 779085 T9705 n:127.0.0.1:54788_ oasc.OverseerAutoReplicaFailoverThread.<init> Starting OverseerAutoReplicaFailoverThread autoReplicaFailoverWorkLoopDelay=10000 autoReplicaFailoverWaitAfterExpiration=30000 autoReplicaFailoverBadNodeExpiration=60000
[junit4] 2> 779087 T9738 n:127.0.0.1:54788_ oasc.OverseerCollectionProcessor.run Process current queue of collection creations
[junit4] 2> 779087 T9705 n:127.0.0.1:54788_ oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 779087 T9737 n:127.0.0.1:54788_ oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
[junit4] 2> 779091 T9705 n:127.0.0.1:54788_ oasc.CoreContainer.intializeAuthorizationPlugin Security conf doesn't exist. Skipping setup for authorization module.
[junit4] 2> 779094 T9705 n:127.0.0.1:54788_ oasc.CorePropertiesLocator.discover Looking for core definitions underneath /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeySafeLeaderTest 8689C038CB394CF-001/control-001/cores
[junit4] 2> 779098 T9705 n:127.0.0.1:54788_ oasc.CoreDescriptor.<init> CORE DESCRIPTOR: {transient=false, coreNodeName=, collection=control_collection, loadOnStartup=true, shard=, dataDir=data/, instanceDir=/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeySafeLeaderTest 8689C038CB394CF-001/control-001/cores/collection1, name=collection1, schema=schema.xml, absoluteInstDir=/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeySafeLeaderTest 8689C038CB394CF-001/control-001/cores/collection1/, config=solrconfig.xml}
[junit4] 2> 779098 T9705 n:127.0.0.1:54788_ oasc.CorePropertiesLocator.discoverUnder Found core collection1 in /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeySafeLeaderTest 8689C038CB394CF-001/control-001/cores/collection1/
[junit4] 2> 779099 T9705 n:127.0.0.1:54788_ oasc.CorePropertiesLocator.discover Found 1 core definitions
[junit4] 2> 779101 T9740 n:127.0.0.1:54788_ c:control_collection x:collection1 oasc.ZkController.publish publishing core=collection1 state=down collection=control_collection
[junit4] 2> 779101 T9740 n:127.0.0.1:54788_ c:control_collection x:collection1 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 779102 T9740 n:127.0.0.1:54788_ oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4] 2> 779102 T9736 n:127.0.0.1:54788_ oasc.DistributedQueue$LatchWatcher.process NodeChildrenChanged fired on path /overseer/queue state SyncConnected
[junit4] 2> 779104 T9737 n:127.0.0.1:54788_ oasc.Overseer$ClusterStateUpdater.run processMessage: queueSize: 1, message = {
[junit4] 2> "operation":"state",
[junit4] 2> "collection":"control_collection",
[junit4] 2> "numShards":"1",
[junit4] 2> "roles":null,
[junit4] 2> "base_url":"https://127.0.0.1:54788",
[junit4] 2> "state":"down",
[junit4] 2> "shard":null,
[junit4] 2> "node_name":"127.0.0.1:54788_",
[junit4] 2> "core":"collection1"} current state version: 0
[junit4] 2> 779105 T9737 n:127.0.0.1:54788_ oasco.ReplicaMutator.updateState Update state numShards=1 message={
[junit4] 2> "operation":"state",
[junit4] 2> "collection":"control_collection",
[junit4] 2> "numShards":"1",
[junit4] 2> "roles":null,
[junit4] 2> "base_url":"https://127.0.0.1:54788",
[junit4] 2> "state":"down",
[junit4] 2> "shard":null,
[junit4] 2> "node_name":"127.0.0.1:54788_",
[junit4] 2> "core":"collection1"}
[junit4] 2> 779105 T9737 n:127.0.0.1:54788_ oasco.ClusterStateMutator.createCollection building a new cName: control_collection
[junit4] 2> 779106 T9737 n:127.0.0.1:54788_ oasco.ReplicaMutator.updateState Assigning new node to shard shard=shard1
[junit4] 2> 779109 T9736 n:127.0.0.1:54788_ oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 780106 T9740 n:127.0.0.1:54788_ oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
[junit4] 2> 780106 T9740 n:127.0.0.1:54788_ oasc.ZkController.createCollectionZkNode Check for collection zkNode:control_collection
[junit4] 2> 780107 T9740 n:127.0.0.1:54788_ oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4] 2> 780108 T9740 n:127.0.0.1:54788_ oascc.ZkStateReader.readConfigName Load collection config from:/collections/control_collection
[junit4] 2> 780108 T9740 n:127.0.0.1:54788_ oascc.ZkStateReader.readConfigName path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
[junit4] 2> 780108 T9740 n:127.0.0.1:54788_ oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeySafeLeaderTest 8689C038CB394CF-001/control-001/cores/collection1/'
[junit4] 2> 780141 T9740 n:127.0.0.1:54788_ oasc.Config.<init> loaded config solrconfig.xml with version 0
[junit4] 2> 780163 T9740 n:127.0.0.1:54788_ oasc.SolrConfig.refreshRequestParams current version of requestparams : -1
[junit4] 2> 780205 T9740 n:127.0.0.1:54788_ oasc.SolrConfig.<init> Using Lucene MatchVersion: 5.2.0
[junit4] 2> 780290 T9740 n:127.0.0.1:54788_ oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4] 2> 780392 T9740 n:127.0.0.1:54788_ oass.IndexSchema.readSchema Reading Solr Schema from /configs/conf1/schema.xml
[junit4] 2> 780403 T9740 n:127.0.0.1:54788_ oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4] 2> 780930 T9740 n:127.0.0.1:54788_ oass.IndexSchema.readSchema default search field in schema is text
[junit4] 2> 780934 T9740 n:127.0.0.1:54788_ oass.IndexSchema.readSchema unique key field: id
[junit4] 2> 780936 T9740 n:127.0.0.1:54788_ oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 780945 T9740 n:127.0.0.1:54788_ oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 780976 T9740 n:127.0.0.1:54788_ oasc.CoreContainer.create Creating SolrCore 'collection1' using configuration from collection control_collection
[junit4] 2> 780976 T9740 n:127.0.0.1:54788_ x:collection1 oasc.SolrCore.initDirectoryFactory solr.StandardDirectoryFactory
[junit4] 2> 781126 T9740 n:127.0.0.1:54788_ x:collection1 oasc.SolrCore.<init> [[collection1] ] Opening new SolrCore at [/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeySafeLeaderTest 8689C038CB394CF-001/control-001/cores/collection1/], dataDir=[null]
[junit4] 2> 781127 T9740 n:127.0.0.1:54788_ x:collection1 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@49d456d0
[junit4] 2> 781128 T9740 n:127.0.0.1:54788_ x:collection1 oasc.CachingDirectoryFactory.get return new directory for /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeySafeLeaderTest 8689C038CB394CF-001/control-001/cores/collection1/data
[junit4] 2> 781129 T9740 n:127.0.0.1:54788_ x:collection1 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeySafeLeaderTest 8689C038CB394CF-001/control-001/cores/collection1/data/index/
[junit4] 2> 781130 T9740 n:127.0.0.1:54788_ x:collection1 oasc.SolrCore.initIndex WARN [collection1] Solr index directory '/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeySafeLeaderTest 8689C038CB394CF-001/control-001/cores/collection1/data/index' doesn't exist. Creating new index...
[junit4] 2> 781131 T9740 n:127.0.0.1:54788_ x:collection1 oasc.CachingDirectoryFactory.get return new directory for /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeySafeLeaderTest 8689C038CB394CF-001/control-001/cores/collection1/data/index
[junit4] 2> 781132 T9740 n:127.0.0.1:54788_ x:collection1 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=24, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.5732994702890314]
[junit4] 2> 781140 T9740 n:127.0.0.1:54788_ x:collection1 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4] 2> commit{dir=/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeySafeLeaderTest 8689C038CB394CF-001/control-001/cores/collection1/data/index,segFN=segments_1,generation=1}
[junit4] 2> 781141 T9740 n:127.0.0.1:54788_ x:collection1 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 781151 T9740 n:127.0.0.1:54788_ x:collection1 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "nodistrib"
[junit4] 2> 781151 T9740 n:127.0.0.1:54788_ x:collection1 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe"
[junit4] 2> 781152 T9740 n:127.0.0.1:54788_ x:collection1 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4] 2> 781152 T9740 n:127.0.0.1:54788_ x:collection1 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "stored_sig"
[junit4] 2> 781152 T9740 n:127.0.0.1:54788_ x:collection1 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4] 2> 781153 T9740 n:127.0.0.1:54788_ x:collection1 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
[junit4] 2> 781153 T9740 n:127.0.0.1:54788_ x:collection1 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 781154 T9740 n:127.0.0.1:54788_ x:collection1 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 781154 T9740 n:127.0.0.1:54788_ x:collection1 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
[junit4] 2> 781194 T9740 n:127.0.0.1:54788_ x:collection1 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 781199 T9740 n:127.0.0.1:54788_ x:collection1 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 781203 T9740 n:127.0.0.1:54788_ x:collection1 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 781207 T9740 n:127.0.0.1:54788_ x:collection1 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 781216 T9740 n:127.0.0.1:54788_ x:collection1 oasc.RequestHandlers.initHandlersFromConfig Registered paths: /admin/plugins,/admin/system,/config,/admin/properties,/admin/luke,/admin/logging,/admin/mbeans,/update/csv,/schema,/update/json,/get,/admin/file,/update,/admin/threads,standard,/update/json/docs,/replication,/admin/ping,/admin/segments
[junit4] 2> 781219 T9740 n:127.0.0.1:54788_ x:collection1 oasc.SolrCore.initStatsCache Using default statsCache cache: org.apache.solr.search.stats.LocalStatsCache
[junit4] 2> 781219 T9740 n:127.0.0.1:54788_ x:collection1 oasu.UpdateHandler.<init> Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 781220 T9740 n:127.0.0.1:54788_ x:collection1 oasu.UpdateLog.init Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10
[junit4] 2> 781221 T9740 n:127.0.0.1:54788_ x:collection1 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4] 2> 781222 T9740 n:127.0.0.1:54788_ x:collection1 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4] 2> 781222 T9740 n:127.0.0.1:54788_ x:collection1 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.MockRandomMergePolicy: org.apache.lucene.index.MockRandomMergePolicy@195cd52b
[junit4] 2> 781224 T9740 n:127.0.0.1:54788_ x:collection1 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4] 2> commit{dir=/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeySafeLeaderTest 8689C038CB394CF-001/control-001/cores/collection1/data/index,segFN=segments_1,generation=1}
[junit4] 2> 781224 T9740 n:127.0.0.1:54788_ x:collection1 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 781225 T9740 n:127.0.0.1:54788_ x:collection1 oass.SolrIndexSearcher.<init> Opening Searcher@441ed54[collection1] main
[junit4] 2> 781225 T9740 n:127.0.0.1:54788_ x:collection1 oascc.ZkStateReader.readConfigName Load collection config from:/collections/control_collection
[junit4] 2> 781226 T9740 n:127.0.0.1:54788_ x:collection1 oascc.ZkStateReader.readConfigName path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
[junit4] 2> 781227 T9740 n:127.0.0.1:54788_ x:collection1 oasr.ManagedResourceStorage.newStorageIO Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
[junit4] 2> 781227 T9740 n:127.0.0.1:54788_ x:collection1 oasr.ManagedResourceStorage$ZooKeeperStorageIO.configure Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 781228 T9740 n:127.0.0.1:54788_ x:collection1 oasr.RestManager.init Initializing RestManager with initArgs: {}
[junit4] 2> 781228 T9740 n:127.0.0.1:54788_ x:collection1 oasr.ManagedResourceStorage.load Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 781229 T9740 n:127.0.0.1:54788_ x:collection1 oasr.ManagedResourceStorage$ZooKeeperStorageIO.openInputStream No data found for znode /configs/conf1/_rest_managed.json
[junit4] 2> 781229 T9740 n:127.0.0.1:54788_ x:collection1 oasr.ManagedResourceStorage.load Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 781230 T9740 n:127.0.0.1:54788_ x:collection1 oasr.RestManager.init Initializing 0 registered ManagedResources
[junit4] 2> 781230 T9740 n:127.0.0.1:54788_ x:collection1 oash.ReplicationHandler.inform Commits will be reserved for 10000
[junit4] 2> 781231 T9740 n:127.0.0.1:54788_ x:collection1 oasc.ZkController.getConfDirListeners watch zkdir /configs/conf1
[junit4] 2> 781232 T9740 n:127.0.0.1:54788_ x:collection1 oasc.CoreContainer.registerCore registering core: collection1
[junit4] 2> 781233 T9741 n:127.0.0.1:54788_ x:collection1 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@441ed54[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 781233 T9744 n:127.0.0.1:54788_ c:control_collection s:shard1 x:collection1 oasc.ZkController.register Register replica - core:collection1 address:https://127.0.0.1:54788 collection:control_collection shard:shard1
[junit4] 2> 781234 T9705 n:127.0.0.1:54788_ oass.SolrDispatchFilter.init user.dir=/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1
[junit4] 2> 781234 T9705 n:127.0.0.1:54788_ oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4] 2> 781235 T9744 n:127.0.0.1:54788_ c:control_collection s:shard1 x:collection1 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leader_elect/shard1/election
[junit4] 2> 781241 T9744 n:127.0.0.1:54788_ c:control_collection s:shard1 x:collection1 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard1
[junit4] 2> 781244 T9736 n:127.0.0.1:54788_ oasc.DistributedQueue$LatchWatcher.process NodeChildrenChanged fired on path /overseer/queue state SyncConnected
[junit4] 2> 781244 T9744 n:127.0.0.1:54788_ c:control_collection s:shard1 x:collection1 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4] 2> 781245 T9744 n:127.0.0.1:54788_ c:control_collection s:shard1 x:collection1 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4] 2> 781245 T9737 n:127.0.0.1:54788_ oasc.Overseer$ClusterStateUpdater.run processMessage: queueSize: 1, message = {
[junit4] 2> "operation":"leader",
[junit4] 2> "shard":"shard1",
[junit4] 2> "collection":"control_collection"} current state version: 1
[junit4] 2> ASYNC NEW_CORE C4450 name=collection1 org.apache.solr.core.SolrCore@4843488e url=https://127.0.0.1:54788/collection1 node=127.0.0.1:54788_ C4450_STATE=coll:control_collection core:collection1 props:{base_url=https://127.0.0.1:54788, state=down, node_name=127.0.0.1:54788_, core=collection1}
[junit4] 2> 781245 T9744 n:127.0.0.1:54788_ c:control_collection s:shard1 x:collection1 C4450 oasc.SyncStrategy.sync Sync replicas to https://127.0.0.1:54788/collection1/
[junit4] 2> 781246 T9744 n:127.0.0.1:54788_ c:control_collection s:shard1 x:collection1 C4450 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4] 2> 781246 T9744 n:127.0.0.1:54788_ c:control_collection s:shard1 x:collection1 C4450 oasc.SyncStrategy.syncToMe https://127.0.0.1:54788/collection1/ has no replicas
[junit4] 2> 781247 T9744 n:127.0.0.1:54788_ c:control_collection s:shard1 x:collection1 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: https://127.0.0.1:54788/collection1/ shard1
[junit4] 2> 781247 T9744 n:127.0.0.1:54788_ c:control_collection s:shard1 x:collection1 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leaders/shard1
[junit4] 2> 781247 T9736 n:127.0.0.1:54788_ oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 781248 T9705 oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default ZkCredentialsProvider
[junit4] 2> 781249 T9705 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 781252 T9747 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@78a3f7c name:ZooKeeperConnection Watcher:127.0.0.1:54778/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 781252 T9705 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 781253 T9705 oascc.SolrZkClient.createZkACLProvider Using default ZkACLProvider
[junit4] 2> 781253 T9705 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 781256 T9736 n:127.0.0.1:54788_ oasc.DistributedQueue$LatchWatcher.process NodeChildrenChanged fired on path /overseer/queue state SyncConnected
[junit4] 2> 781261 T9737 n:127.0.0.1:54788_ oasc.Overseer$ClusterStateUpdater.run processMessage: queueSize: 1, message = {
[junit4] 2> "operation":"leader",
[junit4] 2> "shard":"shard1",
[junit4] 2> "collection":"control_collection",
[junit4] 2> "base_url":"https://127.0.0.1:54788",
[junit4] 2> "core":"collection1",
[junit4] 2> "state":"active"} current state version: 2
[junit4] 2> 781262 T9705 oasc.ChaosMonkey.monkeyLog monkey: init - expire sessions:false cause connection loss:false
[junit4] 2> 781367 T9747 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 781367 T9736 n:127.0.0.1:54788_ oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 781413 T9744 n:127.0.0.1:54788_ c:control_collection s:shard1 x:collection1 oasc.ZkController.register We are https://127.0.0.1:54788/collection1/ and leader is https://127.0.0.1:54788/collection1/
[junit4] 2> 781414 T9744 n:127.0.0.1:54788_ c:control_collection s:shard1 x:collection1 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=https://127.0.0.1:54788
[junit4] 2> 781414 T9744 n:127.0.0.1:54788_ c:control_collection s:shard1 x:collection1 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4] 2> 781414 T9744 n:127.0.0.1:54788_ c:control_collection s:shard1 x:collection1 oasc.ZkController.publish publishing core=collection1 state=active collection=control_collection
[junit4] 2> 781415 T9744 n:127.0.0.1:54788_ c:control_collection s:shard1 x:collection1 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 781416 T9736 n:127.0.0.1:54788_ oasc.DistributedQueue$LatchWatcher.process NodeChildrenChanged fired on path /overseer/queue state SyncConnected
[junit4] 2> 781417 T9737 n:127.0.0.1:54788_ oasc.Overseer$ClusterStateUpdater.run processMessage: queueSize: 1, message = {
[junit4] 2> "operation":"state",
[junit4] 2> "collection":"control_collection",
[junit4] 2> "numShards":"1",
[junit4] 2> "roles":null,
[junit4] 2> "base_url":"https://127.0.0.1:54788",
[junit4] 2> "state":"active",
[junit4] 2> "core_node_name":"core_node1",
[junit4] 2> "shard":"shard1",
[junit4] 2> "node_name":"127.0.0.1:54788_",
[junit4] 2> "core":"collection1"} current state version: 3
[junit4] 2> 781418 T9737 n:127.0.0.1:54788_ oasco.ReplicaMutator.updateState Update state numShards=1 message={
[junit4] 2> "operation":"state",
[junit4] 2> "collection":"control_collection",
[junit4] 2> "numShards":"1",
[junit4] 2> "roles":null,
[junit4] 2> "base_url":"https://127.0.0.1:54788",
[junit4] 2> "state":"active",
[junit4] 2> "core_node_name":"core_node1",
[junit4] 2> "shard":"shard1",
[junit4] 2> "node_name":"127.0.0.1:54788_",
[junit4] 2> "core":"collection1"}
[junit4] 2> 781523 T9747 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 781523 T9736 n:127.0.0.1:54788_ oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 783953 T9705 oas.SolrTestCaseJ4.writeCoreProperties Writing core.properties file to /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeySafeLeaderTest 8689C038CB394CF-001/shard-1-001/cores/collection1
[junit4] 2> 783960 T9705 oasc.AbstractFullDistribZkTestBase.createJettys create jetty 1 in directory /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeySafeLeaderTest 8689C038CB394CF-001/shard-1-001
[junit4] 2> 783965 T9705 oejs.Server.doStart jetty-9.2.10.v20150310
[junit4] 2> 783968 T9705 oejsh.ContextHandler.doStart Started o.e.j.s.ServletContextHandler@3314c73d{/,null,AVAILABLE}
[junit4] 2> 783980 T9705 oejs.AbstractConnector.doStart Started ServerConnector@2f697997{SSL-http/1.1}{127.0.0.1:54805}
[junit4] 2> 783981 T9705 oejs.Server.doStart Started @791970ms
[junit4] 2> 783981 T9705 oascse.JettySolrRunner$1.lifeCycleStarted Jetty properties: {coreRootDirectory=/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../../usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeySafeLeaderTest 8689C038CB394CF-001/shard-1-001/cores, solrconfig=solrconfig.xml, hostPort=54805, hostContext=/, solr.data.dir=/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeySafeLeaderTest 8689C038CB394CF-001/tempDir-001/jetty1}
[junit4] 2> 783988 T9705 oass.SolrDispatchFilter.init SolrDispatchFilter.init()sun.misc.Launcher$AppClassLoader@749cd006
[junit4] 2> 783988 T9705 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../../usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeySafeLeaderTest 8689C038CB394CF-001/shard-1-001/'
[junit4] 2> 784091 T9705 oasc.SolrXmlConfig.fromFile Loading container configuration from /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../../usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeySafeLeaderTest 8689C038CB394CF-001/shard-1-001/solr.xml
[junit4] 2> 784144 T9705 oasc.CorePropertiesLocator.<init> Config-defined core root directory: /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../../usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeySafeLeaderTest 8689C038CB394CF-001/shard-1-001/cores
[junit4] 2> 784144 T9705 oasc.CoreContainer.<init> New CoreContainer 463513870
[junit4] 2> 784144 T9705 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../../usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeySafeLeaderTest 8689C038CB394CF-001/shard-1-001/]
[junit4] 2> 784145 T9705 oasc.CoreContainer.load loading shared library: /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J1/../../../../../../../../../../../usr/home/jenkins/j
[...truncated too long message...]
7753 T7847 n:127.0.0.1:46723_ oasc.ZkController.unregisterConfListener removed listener for config directory [/configs/conf1]
[junit4] 2> 1327753 T7847 n:127.0.0.1:46723_ oasc.ZkController.unregisterConfListener No more listeners for config directory [/configs/conf1]
[junit4] 2> 1327754 T7847 n:127.0.0.1:46723_ oasu.DirectUpdateHandler2.close closing DirectUpdateHandler2{commits=63,autocommits=0,soft autocommits=0,optimizes=0,rollbacks=0,expungeDeletes=0,docsPending=0,adds=0,deletesById=0,deletesByQuery=0,errors=0,cumulative_adds=329,cumulative_deletesById=173,cumulative_deletesByQuery=0,cumulative_errors=0,transaction_logs_total_size=38399,transaction_logs_total_number=1}
[junit4] 2> 1327754 T7847 n:127.0.0.1:46723_ oasu.SolrCoreState.decrefSolrCoreState Closing SolrCoreState
[junit4] 2> 1327754 T7847 n:127.0.0.1:46723_ oasu.DefaultSolrCoreState.closeIndexWriter SolrCoreState ref count has reached 0 - closing IndexWriter
[junit4] 2> 1327754 T7847 n:127.0.0.1:46723_ oasu.DefaultSolrCoreState.closeIndexWriter closing IndexWriter with IndexWriterCloser
[junit4] 2> 1327757 T7847 n:127.0.0.1:46723_ oasc.SolrCore.closeSearcher [collection1] Closing main searcher on request.
[junit4] 2> 1327767 T7847 n:127.0.0.1:46723_ oasc.CachingDirectoryFactory.close Closing MockDirectoryFactory - 2 directories currently being tracked
[junit4] 2> 1327767 T7847 n:127.0.0.1:46723_ oasc.CachingDirectoryFactory.closeCacheValue looking to close /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.RecoveryZkTest 8689C038CB394CF-001/shard-2-001/cores/collection1/data [CachedDir<<refCount=0;path=/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.RecoveryZkTest 8689C038CB394CF-001/shard-2-001/cores/collection1/data;done=false>>]
[junit4] 2> 1327767 T7847 n:127.0.0.1:46723_ oasc.CachingDirectoryFactory.close Closing directory: /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.RecoveryZkTest 8689C038CB394CF-001/shard-2-001/cores/collection1/data
[junit4] 2> 1327767 T7847 n:127.0.0.1:46723_ oasc.CachingDirectoryFactory.closeCacheValue looking to close /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.RecoveryZkTest 8689C038CB394CF-001/shard-2-001/cores/collection1/data/index [CachedDir<<refCount=0;path=/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.RecoveryZkTest 8689C038CB394CF-001/shard-2-001/cores/collection1/data/index;done=false>>]
[junit4] 2> 1327768 T7847 n:127.0.0.1:46723_ oasc.CachingDirectoryFactory.close Closing directory: /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.RecoveryZkTest 8689C038CB394CF-001/shard-2-001/cores/collection1/data/index
[junit4] 2> 1327769 T7682 oejs.AbstractConnector.doStop Stopped ServerConnector@136735ed{HTTP/1.1}{127.0.0.1:46723}
[junit4] 2> 1327770 T7682 oejsh.ContextHandler.doStop Stopped o.e.j.s.ServletContextHandler@3a996ab5{/,null,UNAVAILABLE}
[junit4] 2> 1327772 T7682 c:control_collection s:shard1 x:collection1 oasc.ZkTestServer.send4LetterWord connecting to 127.0.0.1:46697 46697
[junit4] 2> 1327831 T7683 oasc.ZkTestServer.send4LetterWord connecting to 127.0.0.1:46697 46697
[junit4] 2> 1327832 T7683 oasc.ZkTestServer$ZKServerMain.runFromConfig WARN Watch limit violations:
[junit4] 2> Maximum concurrent create/delete watches above limit:
[junit4] 2>
[junit4] 2> 5 /solr/clusterstate.json
[junit4] 2> 5 /solr/aliases.json
[junit4] 2> 4 /solr/configs/conf1
[junit4] 2>
[junit4] 2> Maximum concurrent data watches above limit:
[junit4] 2>
[junit4] 2> 2 /solr/collections/collection1/leader_elect/shard1/election/93840119164370950-core_node1-n_0000000000
[junit4] 2> 2 /solr/overseer_elect/election/93840119164370950-127.0.0.1:46708_-n_0000000001
[junit4] 2>
[junit4] 2> Maximum concurrent children watches above limit:
[junit4] 2>
[junit4] 2> 4 /solr/live_nodes
[junit4] 2> 2 /solr/overseer/queue
[junit4] 2> 2 /solr/overseer/collection-queue-work
[junit4] 2>
[junit4] 2> NOTE: download the large Jenkins line-docs file by running 'ant get-jenkins-line-docs' in the lucene directory.
[junit4] 2> NOTE: reproduce with: ant test -Dtestcase=RecoveryZkTest -Dtests.method=test -Dtests.seed=8689C038CB394CF -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/home/jenkins/lucene-data/enwiki.random.lines.txt -Dtests.locale=no_NO_NY -Dtests.timezone=Europe/Lisbon -Dtests.asserts=true -Dtests.file.encoding=ISO-8859-1
[junit4] FAILURE 172s J0 | RecoveryZkTest.test <<<
[junit4] > Throwable #1: java.lang.AssertionError: shard1 is not consistent. Got 835 from http://127.0.0.1:46708/collection1lastClient and got 215 from http://127.0.0.1:46723/collection1
[junit4] > at __randomizedtesting.SeedInfo.seed([8689C038CB394CF:803CA3D9224FF937]:0)
[junit4] > at org.apache.solr.cloud.RecoveryZkTest.test(RecoveryZkTest.java:123)
[junit4] > at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:960)
[junit4] > at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:935)
[junit4] > at java.lang.Thread.run(Thread.java:745)
[junit4] 2> 1327841 T7682 c:control_collection s:shard1 x:collection1 oas.SolrTestCaseJ4.deleteCore ###deleteCore
[junit4] 2> NOTE: leaving temporary files on disk at: /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J0/temp/solr.cloud.RecoveryZkTest 8689C038CB394CF-001
[junit4] 2> 172228 T7681 ccr.ThreadLeakControl.checkThreadLeaks WARNING Will linger awaiting termination of 1 leaked thread(s).
[junit4] 2> NOTE: test params are: codec=Asserting(Lucene50): {_version_=FSTOrd50, a_i=PostingsFormat(name=LuceneVarGapDocFreqInterval), id=PostingsFormat(name=LuceneVarGapDocFreqInterval), rnd_b=PostingsFormat(name=LuceneVarGapDocFreqInterval), a_t=PostingsFormat(name=LuceneFixedGap)}, docValues:{}, sim=DefaultSimilarity, locale=no_NO_NY, timezone=Europe/Lisbon
[junit4] 2> NOTE: FreeBSD 9.1-RELEASE-p3 amd64/Oracle Corporation 1.7.0_65 (64-bit)/cpus=16,threads=1,free=100379240,total=421003264
[junit4] 2> NOTE: All tests run in this JVM: [QueryParsingTest, TestFileDictionaryLookup, TestCloudPivotFacet, RequestHandlersTest, ZkNodePropsTest, HdfsDirectoryFactoryTest, StatelessScriptUpdateProcessorFactoryTest, TestSchemaManager, TestSolrCoreProperties, BadCopyFieldTest, TestRandomFaceting, SpellCheckCollatorTest, ParsingFieldUpdateProcessorsTest, RegexBoostProcessorTest, BasicFunctionalityTest, TestCloudInspectUtil, TestStressUserVersions, SolrIndexConfigTest, OpenExchangeRatesOrgProviderTest, TestJettySolrRunner, SolrCmdDistributorTest, TestSchemaNameResource, TestAnalyzeInfixSuggestions, SignatureUpdateProcessorFactoryTest, AnalyticsMergeStrategyTest, TestSolrDeletionPolicy2, MinimalSchemaTest, TestImplicitCoreProperties, BlockDirectoryTest, IndexBasedSpellCheckerTest, ShardRoutingCustomTest, TestIBSimilarityFactory, BasicDistributedZkTest, UUIDUpdateProcessorFallbackTest, TestAtomicUpdateErrorCases, ReplicationFactorTest, TestSolrConfigHandlerCloud, TestMacros, TestSchemaResource, RequiredFieldsTest, StressHdfsTest, TestDocSet, TestCollapseQParserPlugin, CollectionsAPIAsyncDistributedZkTest, TestDynamicFieldCollectionResource, TestSolrJ, TestBulkSchemaAPI, TestRequestStatusCollectionAPI, MBeansHandlerTest, SuggesterTest, TestReRankQParserPlugin, AliasIntegrationTest, SegmentsInfoRequestHandlerTest, TestManagedStopFilterFactory, TestSolrXml, OverseerTest, FastVectorHighlighterTest, TestCloudSchemaless, SimpleFacetsTest, UUIDFieldTest, HdfsNNFailoverTest, CollectionsAPIDistributedZkTest, ShardRoutingTest, RecoveryZkTest]
[junit4] Completed [239/490] on J0 in 173.29s, 1 test, 1 failure <<< FAILURES!
[...truncated 839 lines...]
BUILD FAILED
/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:543: The following error occurred while executing this line:
/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:484: The following error occurred while executing this line:
/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/build.xml:61: The following error occurred while executing this line:
/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/extra-targets.xml:39: The following error occurred while executing this line:
/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build.xml:229: The following error occurred while executing this line:
/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/common-build.xml:511: The following error occurred while executing this line:
/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:1433: The following error occurred while executing this line:
/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/lucene/common-build.xml:991: There were test failures: 490 suites, 1960 tests, 1 suite-level error, 2 failures, 65 ignored (11 assumptions)
Total time: 156 minutes 44 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Sending artifact delta relative to Lucene-Solr-NightlyTests-5.x #846
Archived 5 artifacts
Archive block size is 32768
Received 0 blocks and 760369533 bytes
Compression is 0.0%
Took 2 min 17 sec
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure