You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by Policeman Jenkins Server <je...@thetaphi.de> on 2014/01/21 23:20:23 UTC

[JENKINS] Lucene-Solr-trunk-Linux (64bit/jdk1.8.0-ea-b123) - Build # 9185 - Failure!

Build: http://jenkins.thetaphi.de/job/Lucene-Solr-trunk-Linux/9185/
Java: 64bit/jdk1.8.0-ea-b123 -XX:-UseCompressedOops -XX:+UseConcMarkSweepGC

1 tests failed.
REGRESSION:  org.apache.solr.cloud.OverseerRolesTest.testDistribSearch

Error Message:
Captured an uncaught exception in thread: Thread[id=4307, name=Overseer-91117502065934346-127.0.0.1:36018__-n_0000000018, state=RUNNABLE, group=Overseer collection creation process.]

Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=4307, name=Overseer-91117502065934346-127.0.0.1:36018__-n_0000000018, state=RUNNABLE, group=Overseer collection creation process.]
Caused by: org.apache.solr.common.cloud.ZooKeeperException: 
	at __randomizedtesting.SeedInfo.seed([2BF5834E30E33A80]:0)
	at org.apache.solr.cloud.OverseerCollectionProcessor.run(OverseerCollectionProcessor.java:203)
	at java.lang.Thread.run(Thread.java:744)
Caused by: org.apache.zookeeper.KeeperException$NoNodeException: KeeperErrorCode = NoNode for /overseer/collection-queue-work/qn-0000000004
	at org.apache.zookeeper.KeeperException.create(KeeperException.java:111)
	at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
	at org.apache.zookeeper.ZooKeeper.delete(ZooKeeper.java:873)
	at org.apache.solr.common.cloud.SolrZkClient$2.execute(SolrZkClient.java:173)
	at org.apache.solr.common.cloud.SolrZkClient$2.execute(SolrZkClient.java:170)
	at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:73)
	at org.apache.solr.common.cloud.SolrZkClient.delete(SolrZkClient.java:170)
	at org.apache.solr.cloud.DistributedQueue.remove(DistributedQueue.java:193)
	at org.apache.solr.cloud.OverseerCollectionProcessor.run(OverseerCollectionProcessor.java:194)
	... 1 more




Build Log:
[...truncated 10935 lines...]
   [junit4] Suite: org.apache.solr.cloud.OverseerRolesTest
   [junit4]   2> 1154362 T4129 oas.BaseDistributedSearchTestCase.initHostContext Setting hostContext system property: /_/
   [junit4]   2> 1154364 T4129 oas.SolrTestCaseJ4.setUp ###Starting testDistribSearch
   [junit4]   2> Creating dataDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./solrtest-OverseerRolesTest-1390342743924
   [junit4]   2> 1154365 T4129 oasc.ZkTestServer.run STARTING ZK TEST SERVER
   [junit4]   2> 1154365 T4130 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
   [junit4]   2> 1154466 T4129 oasc.ZkTestServer.run start zk server on port:54548
   [junit4]   2> 1154466 T4129 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 1154478 T4136 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@104d28a2 name:ZooKeeperConnection Watcher:127.0.0.1:54548 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1154478 T4129 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 1154479 T4129 oascc.SolrZkClient.makePath makePath: /solr
   [junit4]   2> 1154483 T4129 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 1154486 T4138 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@2f6eee38 name:ZooKeeperConnection Watcher:127.0.0.1:54548/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1154486 T4129 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 1154486 T4129 oascc.SolrZkClient.makePath makePath: /collections/collection1
   [junit4]   2> 1154489 T4129 oascc.SolrZkClient.makePath makePath: /collections/collection1/shards
   [junit4]   2> 1154491 T4129 oascc.SolrZkClient.makePath makePath: /collections/control_collection
   [junit4]   2> 1154492 T4129 oascc.SolrZkClient.makePath makePath: /collections/control_collection/shards
   [junit4]   2> 1154494 T4129 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
   [junit4]   2> 1154495 T4129 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.xml
   [junit4]   2> 1154499 T4129 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
   [junit4]   2> 1154499 T4129 oascc.SolrZkClient.makePath makePath: /configs/conf1/schema.xml
   [junit4]   2> 1154503 T4129 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 1154503 T4129 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 1154505 T4129 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
   [junit4]   2> 1154506 T4129 oascc.SolrZkClient.makePath makePath: /configs/conf1/stopwords.txt
   [junit4]   2> 1154513 T4129 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
   [junit4]   2> 1154513 T4129 oascc.SolrZkClient.makePath makePath: /configs/conf1/protwords.txt
   [junit4]   2> 1154518 T4129 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
   [junit4]   2> 1154519 T4129 oascc.SolrZkClient.makePath makePath: /configs/conf1/currency.xml
   [junit4]   2> 1154521 T4129 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
   [junit4]   2> 1154522 T4129 oascc.SolrZkClient.makePath makePath: /configs/conf1/enumsConfig.xml
   [junit4]   2> 1154525 T4129 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
   [junit4]   2> 1154525 T4129 oascc.SolrZkClient.makePath makePath: /configs/conf1/open-exchange-rates.json
   [junit4]   2> 1154534 T4129 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 1154534 T4129 oascc.SolrZkClient.makePath makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 1154537 T4129 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
   [junit4]   2> 1154537 T4129 oascc.SolrZkClient.makePath makePath: /configs/conf1/old_synonyms.txt
   [junit4]   2> 1154539 T4129 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
   [junit4]   2> 1154540 T4129 oascc.SolrZkClient.makePath makePath: /configs/conf1/synonyms.txt
   [junit4]   2> 1154604 T4129 oejs.Server.doStart jetty-8.1.10.v20130312
   [junit4]   2> 1154607 T4129 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:35412
   [junit4]   2> 1154607 T4129 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
   [junit4]   2> 1154608 T4129 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 1154608 T4129 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ./org.apache.solr.cloud.OverseerRolesTest-controljetty-1390342744102
   [junit4]   2> 1154609 T4129 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: './org.apache.solr.cloud.OverseerRolesTest-controljetty-1390342744102/'
   [junit4]   2> 1154620 T4129 oasc.ConfigSolr.fromFile Loading container configuration from /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.OverseerRolesTest-controljetty-1390342744102/solr.xml
   [junit4]   2> 1154627 T4129 oasc.ConfigSolrXml.<init> Config-defined core root directory: 
   [junit4]   2> 1154628 T4129 oasc.CoreContainer.<init> New CoreContainer 1846258941
   [junit4]   2> 1154628 T4129 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=./org.apache.solr.cloud.OverseerRolesTest-controljetty-1390342744102/]
   [junit4]   2> 1154629 T4129 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
   [junit4]   2> 1154629 T4129 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
   [junit4]   2> 1154629 T4129 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
   [junit4]   2> 1154629 T4129 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
   [junit4]   2> 1154630 T4129 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
   [junit4]   2> 1154630 T4129 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
   [junit4]   2> 1154630 T4129 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
   [junit4]   2> 1154630 T4129 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
   [junit4]   2> 1154630 T4129 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
   [junit4]   2> 1154632 T4129 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 1154632 T4129 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 1154633 T4129 oasc.CoreContainer.load Host Name: 127.0.0.1
   [junit4]   2> 1154633 T4129 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:54548/solr
   [junit4]   2> 1154633 T4129 oasc.ZkController.checkChrootPath zkHost includes chroot
   [junit4]   2> 1154634 T4129 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 1154635 T4149 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@25e4a60c name:ZooKeeperConnection Watcher:127.0.0.1:54548 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1154635 T4129 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 1154637 T4129 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 1154638 T4151 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@3a1ad827 name:ZooKeeperConnection Watcher:127.0.0.1:54548/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1154639 T4129 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 1154640 T4129 oascc.SolrZkClient.makePath makePath: /overseer/queue
   [junit4]   2> 1154643 T4129 oascc.SolrZkClient.makePath makePath: /overseer/collection-queue-work
   [junit4]   2> 1154645 T4129 oascc.SolrZkClient.makePath makePath: /live_nodes
   [junit4]   2> 1154646 T4129 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:35412__
   [junit4]   2> 1154647 T4129 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:35412__
   [junit4]   2> 1154649 T4129 oascc.SolrZkClient.makePath makePath: /overseer_elect
   [junit4]   2> 1154650 T4129 oascc.SolrZkClient.makePath makePath: /overseer_elect/election
   [junit4]   2> 1154653 T4129 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
   [junit4]   2> 1154654 T4129 oasc.Overseer.start Overseer (id=91117502065934339-127.0.0.1:35412__-n_0000000000) starting
   [junit4]   2> 1154657 T4129 oascc.SolrZkClient.makePath makePath: /overseer/queue-work
   [junit4]   2> 1154659 T4153 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
   [junit4]   2> 1154659 T4129 oascc.SolrZkClient.makePath makePath: /clusterstate.json
   [junit4]   2> 1154660 T4153 oasc.OverseerCollectionProcessor.prioritizeOverseerNodes prioritizing overseer nodes
   [junit4]   2> 1154661 T4129 oascc.SolrZkClient.makePath makePath: /aliases.json
   [junit4]   2> 1154662 T4129 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 1154664 T4152 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
   [junit4]   2> 1154665 T4129 oasc.CorePropertiesLocator.discover Looking for core definitions underneath /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.OverseerRolesTest-controljetty-1390342744102
   [junit4]   2> 1154666 T4129 oasc.CorePropertiesLocator.discoverUnder Found core conf in /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.OverseerRolesTest-controljetty-1390342744102/conf/
   [junit4]   2> 1154690 T4129 oasc.CorePropertiesLocator.discover Found 1 core definitions
   [junit4]   2> 1154691 T4129 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0
   [junit4]   2> 1154691 T4129 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
   [junit4]   2> 1154692 T4129 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 1154713 T4156 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@11b763db name:ZooKeeperConnection Watcher:127.0.0.1:54548/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1154713 T4129 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 1154714 T4129 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 1154715 T4129 oasc.ChaosMonkey.monkeyLog monkey: init - expire sessions:false cause connection loss:false
   [junit4]   2> 1154779 T4129 oejs.Server.doStart jetty-8.1.10.v20130312
   [junit4]   2> 1154780 T4129 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:34661
   [junit4]   2> 1154780 T4129 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
   [junit4]   2> 1154781 T4129 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 1154781 T4129 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.OverseerRolesTest-jetty1-1390342744274
   [junit4]   2> 1154781 T4129 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.OverseerRolesTest-jetty1-1390342744274/'
   [junit4]   2> 1154790 T4129 oasc.ConfigSolr.fromFile Loading container configuration from /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.OverseerRolesTest-jetty1-1390342744274/solr.xml
   [junit4]   2> 1154801 T4129 oasc.ConfigSolrXml.<init> Config-defined core root directory: 
   [junit4]   2> 1154802 T4129 oasc.CoreContainer.<init> New CoreContainer 1426538808
   [junit4]   2> 1154802 T4129 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.OverseerRolesTest-jetty1-1390342744274/]
   [junit4]   2> 1154802 T4129 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
   [junit4]   2> 1154802 T4129 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
   [junit4]   2> 1154803 T4129 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
   [junit4]   2> 1154803 T4129 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
   [junit4]   2> 1154803 T4129 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
   [junit4]   2> 1154803 T4129 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
   [junit4]   2> 1154804 T4129 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
   [junit4]   2> 1154804 T4129 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
   [junit4]   2> 1154804 T4129 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
   [junit4]   2> 1154806 T4129 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 1154806 T4129 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 1154806 T4129 oasc.CoreContainer.load Host Name: 127.0.0.1
   [junit4]   2> 1154806 T4129 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:54548/solr
   [junit4]   2> 1154807 T4129 oasc.ZkController.checkChrootPath zkHost includes chroot
   [junit4]   2> 1154807 T4129 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 1154809 T4167 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@c629a65 name:ZooKeeperConnection Watcher:127.0.0.1:54548 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1154809 T4129 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 1154810 T4129 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 1154811 T4169 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@685915b4 name:ZooKeeperConnection Watcher:127.0.0.1:54548/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1154811 T4129 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 1154814 T4129 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 1155816 T4129 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:34661__
   [junit4]   2> 1155817 T4129 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:34661__
   [junit4]   2> 1155820 T4156 oascc.ZkStateReader$3.process Updating live nodes... (2)
   [junit4]   2> 1155820 T4151 oascc.ZkStateReader$3.process Updating live nodes... (2)
   [junit4]   2> 1155820 T4169 oascc.ZkStateReader$3.process Updating live nodes... (2)
   [junit4]   2> 1155823 T4129 oasc.CorePropertiesLocator.discover Looking for core definitions underneath /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.OverseerRolesTest-jetty1-1390342744274
   [junit4]   2> 1155824 T4129 oasc.CorePropertiesLocator.discoverUnder Found core conf in /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.OverseerRolesTest-jetty1-1390342744274/conf/
   [junit4]   2> 1155853 T4129 oasc.CorePropertiesLocator.discover Found 1 core definitions
   [junit4]   2> 1155854 T4129 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0
   [junit4]   2> 1155854 T4129 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
   [junit4]   2> 1155917 T4129 oejs.Server.doStart jetty-8.1.10.v20130312
   [junit4]   2> 1155918 T4129 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:34311
   [junit4]   2> 1155919 T4129 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
   [junit4]   2> 1155919 T4129 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 1155919 T4129 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ./org.apache.solr.cloud.OverseerRolesTest-jetty2-1390342745414
   [junit4]   2> 1155920 T4129 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: './org.apache.solr.cloud.OverseerRolesTest-jetty2-1390342745414/'
   [junit4]   2> 1155928 T4129 oasc.ConfigSolr.fromFile Loading container configuration from /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.OverseerRolesTest-jetty2-1390342745414/solr.xml
   [junit4]   2> 1155935 T4129 oasc.ConfigSolrXml.<init> Config-defined core root directory: 
   [junit4]   2> 1155936 T4129 oasc.CoreContainer.<init> New CoreContainer 1440000271
   [junit4]   2> 1155936 T4129 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=./org.apache.solr.cloud.OverseerRolesTest-jetty2-1390342745414/]
   [junit4]   2> 1155936 T4129 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
   [junit4]   2> 1155937 T4129 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
   [junit4]   2> 1155937 T4129 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
   [junit4]   2> 1155937 T4129 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
   [junit4]   2> 1155937 T4129 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
   [junit4]   2> 1155938 T4129 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
   [junit4]   2> 1155938 T4129 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
   [junit4]   2> 1155938 T4129 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
   [junit4]   2> 1155938 T4129 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
   [junit4]   2> 1155940 T4129 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 1155940 T4129 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 1155940 T4129 oasc.CoreContainer.load Host Name: 127.0.0.1
   [junit4]   2> 1155941 T4129 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:54548/solr
   [junit4]   2> 1155941 T4129 oasc.ZkController.checkChrootPath zkHost includes chroot
   [junit4]   2> 1155942 T4129 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 1155943 T4181 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@37b3a50d name:ZooKeeperConnection Watcher:127.0.0.1:54548 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1155943 T4129 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 1155945 T4129 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 1155946 T4183 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@11dd9b1e name:ZooKeeperConnection Watcher:127.0.0.1:54548/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1155946 T4129 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 1155952 T4129 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 1156955 T4129 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:34311__
   [junit4]   2> 1156957 T4129 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:34311__
   [junit4]   2> 1156959 T4151 oascc.ZkStateReader$3.process Updating live nodes... (3)
   [junit4]   2> 1156959 T4156 oascc.ZkStateReader$3.process Updating live nodes... (3)
   [junit4]   2> 1156959 T4183 oascc.ZkStateReader$3.process Updating live nodes... (3)
   [junit4]   2> 1156959 T4169 oascc.ZkStateReader$3.process Updating live nodes... (3)
   [junit4]   2> 1156962 T4129 oasc.CorePropertiesLocator.discover Looking for core definitions underneath /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.OverseerRolesTest-jetty2-1390342745414
   [junit4]   2> 1156963 T4129 oasc.CorePropertiesLocator.discoverUnder Found core conf in /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.OverseerRolesTest-jetty2-1390342745414/conf/
   [junit4]   2> 1156993 T4129 oasc.CorePropertiesLocator.discover Found 1 core definitions
   [junit4]   2> 1156994 T4129 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0
   [junit4]   2> 1156994 T4129 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
   [junit4]   2> 1157056 T4129 oejs.Server.doStart jetty-8.1.10.v20130312
   [junit4]   2> 1157057 T4129 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:36018
   [junit4]   2> 1157058 T4129 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
   [junit4]   2> 1157058 T4129 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 1157058 T4129 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ./org.apache.solr.cloud.OverseerRolesTest-jetty3-1390342746553
   [junit4]   2> 1157059 T4129 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: './org.apache.solr.cloud.OverseerRolesTest-jetty3-1390342746553/'
   [junit4]   2> 1157068 T4129 oasc.ConfigSolr.fromFile Loading container configuration from /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.OverseerRolesTest-jetty3-1390342746553/solr.xml
   [junit4]   2> 1157079 T4129 oasc.ConfigSolrXml.<init> Config-defined core root directory: 
   [junit4]   2> 1157079 T4129 oasc.CoreContainer.<init> New CoreContainer 1005667471
   [junit4]   2> 1157079 T4129 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=./org.apache.solr.cloud.OverseerRolesTest-jetty3-1390342746553/]
   [junit4]   2> 1157080 T4129 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
   [junit4]   2> 1157080 T4129 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
   [junit4]   2> 1157080 T4129 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
   [junit4]   2> 1157081 T4129 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
   [junit4]   2> 1157081 T4129 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
   [junit4]   2> 1157081 T4129 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
   [junit4]   2> 1157081 T4129 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
   [junit4]   2> 1157081 T4129 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
   [junit4]   2> 1157082 T4129 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
   [junit4]   2> 1157083 T4129 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 1157084 T4129 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 1157084 T4129 oasc.CoreContainer.load Host Name: 127.0.0.1
   [junit4]   2> 1157084 T4129 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:54548/solr
   [junit4]   2> 1157084 T4129 oasc.ZkController.checkChrootPath zkHost includes chroot
   [junit4]   2> 1157085 T4129 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 1157086 T4195 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@27cfd9dc name:ZooKeeperConnection Watcher:127.0.0.1:54548 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1157087 T4129 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 1157090 T4129 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 1157092 T4197 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@2675ad3d name:ZooKeeperConnection Watcher:127.0.0.1:54548/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1157093 T4129 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 1157095 T4129 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 1158097 T4129 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:36018__
   [junit4]   2> 1158098 T4129 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:36018__
   [junit4]   2> 1158101 T4183 oascc.ZkStateReader$3.process Updating live nodes... (4)
   [junit4]   2> 1158101 T4151 oascc.ZkStateReader$3.process Updating live nodes... (4)
   [junit4]   2> 1158101 T4156 oascc.ZkStateReader$3.process Updating live nodes... (4)
   [junit4]   2> 1158101 T4197 oascc.ZkStateReader$3.process Updating live nodes... (4)
   [junit4]   2> 1158101 T4169 oascc.ZkStateReader$3.process Updating live nodes... (4)
   [junit4]   2> 1158104 T4129 oasc.CorePropertiesLocator.discover Looking for core definitions underneath /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.OverseerRolesTest-jetty3-1390342746553
   [junit4]   2> 1158105 T4129 oasc.CorePropertiesLocator.discoverUnder Found core conf in /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.OverseerRolesTest-jetty3-1390342746553/conf/
   [junit4]   2> 1158127 T4129 oasc.CorePropertiesLocator.discover Found 1 core definitions
   [junit4]   2> 1158128 T4129 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0
   [junit4]   2> 1158128 T4129 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
   [junit4]   2> 1158190 T4129 oejs.Server.doStart jetty-8.1.10.v20130312
   [junit4]   2> 1158192 T4129 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:43499
   [junit4]   2> 1158192 T4129 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
   [junit4]   2> 1158192 T4129 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 1158193 T4129 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.OverseerRolesTest-jetty4-1390342747687
   [junit4]   2> 1158193 T4129 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.OverseerRolesTest-jetty4-1390342747687/'
   [junit4]   2> 1158202 T4129 oasc.ConfigSolr.fromFile Loading container configuration from /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.OverseerRolesTest-jetty4-1390342747687/solr.xml
   [junit4]   2> 1158209 T4129 oasc.ConfigSolrXml.<init> Config-defined core root directory: 
   [junit4]   2> 1158209 T4129 oasc.CoreContainer.<init> New CoreContainer 1863191596
   [junit4]   2> 1158209 T4129 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.OverseerRolesTest-jetty4-1390342747687/]
   [junit4]   2> 1158210 T4129 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
   [junit4]   2> 1158210 T4129 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
   [junit4]   2> 1158210 T4129 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
   [junit4]   2> 1158211 T4129 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
   [junit4]   2> 1158211 T4129 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
   [junit4]   2> 1158211 T4129 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
   [junit4]   2> 1158211 T4129 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
   [junit4]   2> 1158211 T4129 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
   [junit4]   2> 1158212 T4129 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
   [junit4]   2> 1158213 T4129 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 1158213 T4129 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 1158214 T4129 oasc.CoreContainer.load Host Name: 127.0.0.1
   [junit4]   2> 1158214 T4129 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:54548/solr
   [junit4]   2> 1158214 T4129 oasc.ZkController.checkChrootPath zkHost includes chroot
   [junit4]   2> 1158215 T4129 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 1158216 T4209 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@503ad833 name:ZooKeeperConnection Watcher:127.0.0.1:54548 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1158216 T4129 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 1158218 T4129 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 1158219 T4211 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@3f2434b0 name:ZooKeeperConnection Watcher:127.0.0.1:54548/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1158219 T4129 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 1158224 T4129 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 1159226 T4129 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:43499__
   [junit4]   2> 1159227 T4129 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:43499__
   [junit4]   2> 1159230 T4151 oascc.ZkStateReader$3.process Updating live nodes... (5)
   [junit4]   2> 1159230 T4156 oascc.ZkStateReader$3.process Updating live nodes... (5)
   [junit4]   2> 1159230 T4197 oascc.ZkStateReader$3.process Updating live nodes... (5)
   [junit4]   2> 1159231 T4169 oascc.ZkStateReader$3.process Updating live nodes... (5)
   [junit4]   2> 1159231 T4183 oascc.ZkStateReader$3.process Updating live nodes... (5)
   [junit4]   2> 1159235 T4211 oascc.ZkStateReader$3.process Updating live nodes... (5)
   [junit4]   2> 1159238 T4129 oasc.CorePropertiesLocator.discover Looking for core definitions underneath /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.OverseerRolesTest-jetty4-1390342747687
   [junit4]   2> 1159239 T4129 oasc.CorePropertiesLocator.discoverUnder Found core conf in /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.OverseerRolesTest-jetty4-1390342747687/conf/
   [junit4]   2> 1159268 T4129 oasc.CorePropertiesLocator.discover Found 1 core definitions
   [junit4]   2> 1159269 T4129 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0
   [junit4]   2> 1159269 T4129 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
   [junit4]   2> 1159331 T4129 oejs.Server.doStart jetty-8.1.10.v20130312
   [junit4]   2> 1159333 T4129 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:54183
   [junit4]   2> 1159333 T4129 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
   [junit4]   2> 1159334 T4129 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 1159334 T4129 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.OverseerRolesTest-jetty5-1390342748828
   [junit4]   2> 1159334 T4129 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.OverseerRolesTest-jetty5-1390342748828/'
   [junit4]   2> 1159343 T4129 oasc.ConfigSolr.fromFile Loading container configuration from /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.OverseerRolesTest-jetty5-1390342748828/solr.xml
   [junit4]   2> 1159350 T4129 oasc.ConfigSolrXml.<init> Config-defined core root directory: 
   [junit4]   2> 1159351 T4129 oasc.CoreContainer.<init> New CoreContainer 948644837
   [junit4]   2> 1159351 T4129 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.OverseerRolesTest-jetty5-1390342748828/]
   [junit4]   2> 1159351 T4129 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
   [junit4]   2> 1159351 T4129 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
   [junit4]   2> 1159352 T4129 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
   [junit4]   2> 1159352 T4129 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
   [junit4]   2> 1159352 T4129 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
   [junit4]   2> 1159352 T4129 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
   [junit4]   2> 1159353 T4129 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
   [junit4]   2> 1159353 T4129 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
   [junit4]   2> 1159353 T4129 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
   [junit4]   2> 1159354 T4129 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 1159355 T4129 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 1159355 T4129 oasc.CoreContainer.load Host Name: 127.0.0.1
   [junit4]   2> 1159355 T4129 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:54548/solr
   [junit4]   2> 1159355 T4129 oasc.ZkController.checkChrootPath zkHost includes chroot
   [junit4]   2> 1159356 T4129 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 1159357 T4223 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@653a65e2 name:ZooKeeperConnection Watcher:127.0.0.1:54548 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1159358 T4129 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 1159359 T4129 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 1159361 T4225 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@23e8d002 name:ZooKeeperConnection Watcher:127.0.0.1:54548/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1159362 T4129 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 1159365 T4129 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 1160367 T4129 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:54183__
   [junit4]   2> 1160368 T4129 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:54183__
   [junit4]   2> 1160371 T4197 oascc.ZkStateReader$3.process Updating live nodes... (6)
   [junit4]   2> 1160371 T4211 oascc.ZkStateReader$3.process Updating live nodes... (6)
   [junit4]   2> 1160371 T4183 oascc.ZkStateReader$3.process Updating live nodes... (6)
   [junit4]   2> 1160371 T4156 oascc.ZkStateReader$3.process Updating live nodes... (6)
   [junit4]   2> 1160371 T4225 oascc.ZkStateReader$3.process Updating live nodes... (6)
   [junit4]   2> 1160371 T4169 oascc.ZkStateReader$3.process Updating live nodes... (6)
   [junit4]   2> 1160371 T4151 oascc.ZkStateReader$3.process Updating live nodes... (6)
   [junit4]   2> 1160374 T4129 oasc.CorePropertiesLocator.discover Looking for core definitions underneath /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.OverseerRolesTest-jetty5-1390342748828
   [junit4]   2> 1160375 T4129 oasc.CorePropertiesLocator.discoverUnder Found core conf in /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.OverseerRolesTest-jetty5-1390342748828/conf/
   [junit4]   2> 1160403 T4129 oasc.CorePropertiesLocator.discover Found 1 core definitions
   [junit4]   2> 1160404 T4129 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0
   [junit4]   2> 1160404 T4129 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
   [junit4]   2> 1160466 T4129 oejs.Server.doStart jetty-8.1.10.v20130312
   [junit4]   2> 1160468 T4129 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:37223
   [junit4]   2> 1160468 T4129 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
   [junit4]   2> 1160469 T4129 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 1160469 T4129 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.OverseerRolesTest-jetty6-1390342749964
   [junit4]   2> 1160469 T4129 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.OverseerRolesTest-jetty6-1390342749964/'
   [junit4]   2> 1160478 T4129 oasc.ConfigSolr.fromFile Loading container configuration from /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.OverseerRolesTest-jetty6-1390342749964/solr.xml
   [junit4]   2> 1160492 T4129 oasc.ConfigSolrXml.<init> Config-defined core root directory: 
   [junit4]   2> 1160492 T4129 oasc.CoreContainer.<init> New CoreContainer 2104530343
   [junit4]   2> 1160492 T4129 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.OverseerRolesTest-jetty6-1390342749964/]
   [junit4]   2> 1160493 T4129 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
   [junit4]   2> 1160493 T4129 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
   [junit4]   2> 1160493 T4129 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
   [junit4]   2> 1160493 T4129 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
   [junit4]   2> 1160494 T4129 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
   [junit4]   2> 1160494 T4129 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
   [junit4]   2> 1160494 T4129 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
   [junit4]   2> 1160494 T4129 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
   [junit4]   2> 1160494 T4129 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
   [junit4]   2> 1160496 T4129 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 1160496 T4129 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 1160496 T4129 oasc.CoreContainer.load Host Name: 127.0.0.1
   [junit4]   2> 1160497 T4129 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:54548/solr
   [junit4]   2> 1160497 T4129 oasc.ZkController.checkChrootPath zkHost includes chroot
   [junit4]   2> 1160498 T4129 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 1160499 T4237 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@207ee712 name:ZooKeeperConnection Watcher:127.0.0.1:54548 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1160499 T4129 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 1160501 T4129 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 1160502 T4239 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@57c9eb59 name:ZooKeeperConnection Watcher:127.0.0.1:54548/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1160502 T4129 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 1160505 T4129 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 1161507 T4129 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:37223__
   [junit4]   2> 1161508 T4129 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:37223__
   [junit4]   2> 1161511 T4156 oascc.ZkStateReader$3.process Updating live nodes... (7)
   [junit4]   2> 1161511 T4151 oascc.ZkStateReader$3.process Updating live nodes... (7)
   [junit4]   2> 1161511 T4239 oascc.ZkStateReader$3.process Updating live nodes... (7)
   [junit4]   2> 1161511 T4183 oascc.ZkStateReader$3.process Updating live nodes... (7)
   [junit4]   2> 1161511 T4169 oascc.ZkStateReader$3.process Updating live nodes... (7)
   [junit4]   2> 1161511 T4211 oascc.ZkStateReader$3.process Updating live nodes... (7)
   [junit4]   2> 1161511 T4197 oascc.ZkStateReader$3.process Updating live nodes... (7)
   [junit4]   2> 1161511 T4225 oascc.ZkStateReader$3.process Updating live nodes... (7)
   [junit4]   2> 1161514 T4129 oasc.CorePropertiesLocator.discover Looking for core definitions underneath /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.OverseerRolesTest-jetty6-1390342749964
   [junit4]   2> 1161515 T4129 oasc.CorePropertiesLocator.discoverUnder Found core conf in /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.OverseerRolesTest-jetty6-1390342749964/conf/
   [junit4]   2> 1161547 T4129 oasc.CorePropertiesLocator.discover Found 1 core definitions
   [junit4]   2> 1161547 T4129 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0
   [junit4]   2> 1161547 T4129 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
   [junit4]   2> 1161548 T4129 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 1161549 T4242 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@61b8dceb name:ZooKeeperConnection Watcher:127.0.0.1:54548/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1161550 T4129 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 1161551 T4129 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 1161552 T4129 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 1161554 T4244 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@181ca0d6 name:ZooKeeperConnection Watcher:127.0.0.1:54548/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1161554 T4129 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 1161555 T4129 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 1161557 T4219 oasha.CollectionsHandler.handleCreateAction Creating Collection : replicationFactor=2&maxShardsPerNode=2&name=testOverseerCol&action=CREATE&numShards=4&wt=javabin&version=2
   [junit4]   2> 1161559 T4151 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/collection-queue-work state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 1161563 T4151 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/collection-queue-work state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 1161563 T4153 oasc.OverseerCollectionProcessor.run Overseer Collection Processor: Get the message id:/overseer/collection-queue-work/qn-0000000000 message:{
   [junit4]   2> 	  "operation":"createcollection",
   [junit4]   2> 	  "fromApi":"true",
   [junit4]   2> 	  "name":"testOverseerCol",
   [junit4]   2> 	  "replicationFactor":"2",
   [junit4]   2> 	  "numShards":"4",
   [junit4]   2> 	  "maxShardsPerNode":"2"}
   [junit4]   2> 1161563 T4153 oasc.OverseerCollectionProcessor.processMessage WARN OverseerCollectionProcessor.processMessage : createcollection , {
   [junit4]   2> 	  "operation":"createcollection",
   [junit4]   2> 	  "fromApi":"true",
   [junit4]   2> 	  "name":"testOverseerCol",
   [junit4]   2> 	  "replicationFactor":"2",
   [junit4]   2> 	  "numShards":"4",
   [junit4]   2> 	  "maxShardsPerNode":"2"}
   [junit4]   2> 1161564 T4153 oasc.OverseerCollectionProcessor.createConfNode Only one config set found in zk - using it:conf1
   [junit4]   2> 1161564 T4153 oasc.OverseerCollectionProcessor.createConfNode creating collections conf node /collections/testOverseerCol 
   [junit4]   2> 1161564 T4153 oascc.SolrZkClient.makePath makePath: /collections/testOverseerCol
   [junit4]   2> 1161567 T4151 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 1161567 T4152 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 1161568 T4152 oasc.Overseer$ClusterStateUpdater.buildCollection building a new collection: testOverseerCol
   [junit4]   2> 1161568 T4152 oasc.Overseer$ClusterStateUpdater.createCollection Create collection testOverseerCol with shards [shard1, shard2, shard3, shard4]
   [junit4]   2> 1161569 T4151 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 1161570 T4225 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 7)
   [junit4]   2> 1161570 T4169 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 7)
   [junit4]   2> 1161570 T4244 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 7)
   [junit4]   2> 1161570 T4151 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 7)
   [junit4]   2> 1161570 T4156 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 7)
   [junit4]   2> 1161570 T4197 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 7)
   [junit4]   2> 1161571 T4242 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 7)
   [junit4]   2> 1161570 T4239 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 7)
   [junit4]   2> 1161570 T4183 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 7)
   [junit4]   2> 1161574 T4211 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 7)
   [junit4]   2> 1161667 T4153 oasc.OverseerCollectionProcessor.createCollection going to create cores replicas shardNames [shard1, shard2, shard3, shard4] , repFactor : 2
   [junit4]   2> 1161667 T4153 oasc.OverseerCollectionProcessor.createCollection Creating shard testOverseerCol_shard1_replica1 as part of slice shard1 of collection testOverseerCol on 127.0.0.1:54183__
   [junit4]   2> 1161668 T4153 oasc.OverseerCollectionProcessor.createCollection Creating shard testOverseerCol_shard1_replica2 as part of slice shard1 of collection testOverseerCol on 127.0.0.1:37223__
   [junit4]   2> 1161668 T4153 oasc.OverseerCollectionProcessor.createCollection Creating shard testOverseerCol_shard2_replica1 as part of slice shard2 of collection testOverseerCol on 127.0.0.1:35412__
   [junit4]   2> 1161669 T4153 oasc.OverseerCollectionProcessor.createCollection Creating shard testOverseerCol_shard2_replica2 as part of slice shard2 of collection testOverseerCol on 127.0.0.1:36018__
   [junit4]   2> 1161669 T4153 oasc.OverseerCollectionProcessor.createCollection Creating shard testOverseerCol_shard3_replica1 as part of slice shard3 of collection testOverseerCol on 127.0.0.1:34311__
   [junit4]   2> 1161669 T4153 oasc.OverseerCollectionProcessor.createCollection Creating shard testOverseerCol_shard3_replica2 as part of slice shard3 of collection testOverseerCol on 127.0.0.1:34661__
   [junit4]   2> 1161670 T4153 oasc.OverseerCollectionProcessor.createCollection Creating shard testOverseerCol_shard4_replica1 as part of slice shard4 of collection testOverseerCol on 127.0.0.1:43499__
   [junit4]   2> 1161670 T4147 oasc.ZkController.publish publishing core=testOverseerCol_shard2_replica1 state=down
   [junit4]   2> 1161671 T4178 oasc.ZkController.publish publishing core=testOverseerCol_shard3_replica1 state=down
   [junit4]   2> 1161670 T4153 oasc.OverseerCollectionProcessor.createCollection Creating shard testOverseerCol_shard4_replica2 as part of slice shard4 of collection testOverseerCol on 127.0.0.1:54183__
   [junit4]   2> 1161671 T4234 oasc.ZkController.publish publishing core=testOverseerCol_shard1_replica2 state=down
   [junit4]   2> 1161672 T4205 oasc.ZkController.publish publishing core=testOverseerCol_shard4_replica1 state=down
   [junit4]   2> 1161671 T4221 oasc.ZkController.publish publishing core=testOverseerCol_shard1_replica1 state=down
   [junit4]   2> 1161673 T4220 oasc.ZkController.publish publishing core=testOverseerCol_shard4_replica2 state=down
   [junit4]   2> 1161673 T4164 oasc.ZkController.publish publishing core=testOverseerCol_shard3_replica2 state=down
   [junit4]   2> 1161674 T4151 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 1161674 T4205 oasc.ZkController.waitForCoreNodeName look for our core node name
   [junit4]   2> 1161674 T4190 oasc.ZkController.publish publishing core=testOverseerCol_shard2_replica2 state=down
   [junit4]   2> 1161674 T4147 oasc.ZkController.waitForCoreNodeName look for our core node name
   [junit4]   2> 1161675 T4221 oasc.ZkController.waitForCoreNodeName look for our core node name
   [junit4]   2> 1161676 T4164 oasc.ZkController.waitForCoreNodeName look for our core node name
   [junit4]   2> 1161676 T4151 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 1161675 T4234 oasc.ZkController.waitForCoreNodeName look for our core node name
   [junit4]   2> 1161675 T4220 oasc.ZkController.waitForCoreNodeName look for our core node name
   [junit4]   2> 1161677 T4178 oasc.ZkController.waitForCoreNodeName look for our core node name
   [junit4]   2> 1161677 T4152 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 1161676 T4190 oasc.ZkController.waitForCoreNodeName look for our core node name
   [junit4]   2> 1161678 T4152 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=4 message={
   [junit4]   2> 	  "operation":"state",
   [junit4]   2> 	  "state":"down",
   [junit4]   2> 	  "base_url":"http://127.0.0.1:35412/_",
   [junit4]   2> 	  "core":"testOverseerCol_shard2_replica1",
   [junit4]   2> 	  "roles":null,
   [junit4]   2> 	  "node_name":"127.0.0.1:35412__",
   [junit4]   2> 	  "shard":"shard2",
   [junit4]   2> 	  "shard_range":null,
   [junit4]   2> 	  "shard_state":"active",
   [junit4]   2> 	  "shard_parent":null,
   [junit4]   2> 	  "collection":"testOverseerCol",
   [junit4]   2> 	  "numShards":"4",
   [junit4]   2> 	  "core_node_name":null}
   [junit4]   2> 1161680 T4152 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=4 message={
   [junit4]   2> 	  "operation":"state",
   [junit4]   2> 	  "state":"down",
   [junit4]   2> 	  "base_url":"http://127.0.0.1:34311/_",
   [junit4]   2> 	  "core":"testOverseerCol_shard3_replica1",
   [junit4]   2> 	  "roles":null,
   [junit4]   2> 	  "node_name":"127.0.0.1:34311__",
   [junit4]   2> 	  "shard":"shard3",
   [junit4]   2> 	  "shard_range":null,
   [junit4]   2> 	  "shard_state":"active",
   [junit4]   2> 	  "shard_parent":null,
   [junit4]   2> 	  "collection":"testOverseerCol",
   [junit4]   2> 	  "numShards":"4",
   [junit4]   2> 	  "core_node_name":null}
   [junit4]   2> 1161682 T4151 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 1161683 T4152 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=4 message={
   [junit4]   2> 	  "operation":"state",
   [junit4]   2> 	  "state":"down",
   [junit4]   2> 	  "base_url":"http://127.0.0.1:37223/_",
   [junit4]   2> 	  "core":"testOverseerCol_shard1_replica2",
   [junit4]   2> 	  "roles":null,
   [junit4]   2> 	  "node_name":"127.0.0.1:37223__",
   [junit4]   2> 	  "shard":"shard1",
   [junit4]   2> 	  "shard_range":null,
   [junit4]   2> 	  "shard_state":"active",
   [junit4]   2> 	  "shard_parent":null,
   [junit4]   2> 	  "collection":"testOverseerCol",
   [junit4]   2> 	  "numShards":"4",
   [junit4]   2> 	  "core_node_name":null}
   [junit4]   2> 1161685 T4151 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 1161686 T4152 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=4 message={
   [junit4]   2> 	  "operation":"state",
   [junit4]   2> 	  "state":"down",
   [junit4]   2> 	  "base_url":"http://127.0.0.1:43499/_",
   [junit4]   2> 	  "core":"testOverseerCol_shard4_replica1",
   [junit4]   2> 	  "roles":null,
   [junit4]   2> 	  "node_name":"127.0.0.1:43499__",
   [junit4]   2> 	  "shard":"shard4",
   [junit4]   2> 	  "shard_range":null,
   [junit4]   2> 	  "shard_state":"active",
   [junit4]   2> 	  "shard_parent":null,
   [junit4]   2> 	  "collection":"testOverseerCol",
   [junit4]   2> 	  "numShards":"4",
   [junit4]   2> 	  "core_node_name":null}
   [junit4]   2> 1161688 T4151 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 1161689 T4152 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=4 message={
   [junit4]   2> 	  "operation":"state",
   [junit4]   2> 	  "state":"down",
   [junit4]   2> 	  "base_url":"http://127.0.0.1:54183/_",
   [junit4]   2> 	  "core":"testOverseerCol_shard1_replica1",
   [junit4]   2> 	  "roles":null,
   [junit4]   2> 	  "node_name":"127.0.0.1:54183__",
   [junit4]   2> 	  "shard":"shard1",
   [junit4]   2> 	  "shard_range":null,
   [junit4]   2> 	  "shard_state":"active",
   [junit4]   2> 	  "shard_parent":null,
   [junit4]   2> 	  "collection":"testOverseerCol",
   [junit4]   2> 	  "numShards":"4",
   [junit4]   2> 	  "core_node_name":null}
   [junit4]   2> 1161691 T4151 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 1161692 T4152 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=4 message={
   [junit4]   2> 	  "operation":"state",
   [junit4]   2> 	  "state":"down",
   [junit4]   2> 	  "base_url":"http://127.0.0.1:54183/_",
   [junit4]   2> 	  "core":"testOverseerCol_shard4_replica2",
   [junit4]   2> 	  "roles":null,
   [junit4]   2> 	  "node_name":"127.0.0.1:54183__",
   [junit4]   2> 	  "shard":"shard4",
   [junit4]   2> 	  "shard_range":null,
   [junit4]   2> 	  "shard_state":"active",
   [junit4]   2> 	  "shard_parent":null,
   [junit4]   2> 	  "collection":"testOverseerCol",
   [junit4]   2> 	  "numShards":"4",
   [junit4]   2> 	  "core_node_name":null}
   [junit4]   2> 1161694 T4151 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 1161694 T4152 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=4 message={
   [junit4]   2> 	  "operation":"state",
   [junit4]   2> 	  "state":"down",
   [junit4]   2> 	  "base_url":"http://127.0.0.1:34661/_",
   [junit4]   2> 	  "core":"testOverseerCol_shard3_replica2",
   [junit4]   2> 	  "roles":null,
   [junit4]   2> 	  "node_name":"127.0.0.1:34661__",
   [junit4]   2> 	  "shard":"shard3",
   [junit4]   2> 	  "shard_range":null,
   [junit4]   2> 	  "shard_state":"active",
   [junit4]   2> 	  "shard_parent":null,
   [junit4]   2> 	  "collection":"testOverseerCol",
   [junit4]   2> 	  "numShards":"4",
   [junit4]   2> 	  "core_node_name":null}
   [junit4]   2> 1161696 T4151 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 1161697 T4152 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=4 message={
   [junit4]   2> 	  "operation":"state",
   [junit4]   2> 	  "state":"down",
   [junit4]   2> 	  "base_url":"http://127.0.0.1:36018/_",
   [junit4]   2> 	  "core":"testOverseerCol_shard2_replica2",
   [junit4]   2> 	  "roles":null,
   [junit4]   2> 	  "node_name":"127.0.0.1:36018__",
   [junit4]   2> 	  "shard":"shard2",
   [junit4]   2> 	  "shard_range":null,
   [junit4]   2> 	  "shard_state":"active",
   [junit4]   2> 	  "shard_parent":null,
   [junit4]   2> 	  "collection":"testOverseerCol",
   [junit4]   2> 	  "numShards":"4",
   [junit4]   2> 	  "core_node_name":null}
   [junit4]   2> 1161698 T4151 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 1161801 T4169 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 7)
   [junit4]   2> 1161802 T4211 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 7)
   [junit4]   2> 1161802 T4183 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 7)
   [junit4]   2> 1161802 T4151 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 7)
   [junit4]   2> 1161802 T4239 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 7)
   [junit4]   2> 1161802 T4244 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 7)
   [junit4]   2> 1161802 T4242 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 7)
   [junit4]   2> 1161802 T4156 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 7)
   [junit4]   2> 1161801 T4225 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 7)
   [junit4]   2> 1161801 T4197 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 7)
   [junit4]   2> 1162674 T4205 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for testOverseerCol_shard4_replica1
   [junit4]   2> 1162675 T4147 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for testOverseerCol_shard2_replica1
   [junit4]   2> 1162676 T4221 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for testOverseerCol_shard1_replica1
   [junit4]   2> 1162676 T4205 oasc.CoreContainer.create Creating SolrCore 'testOverseerCol_shard4_replica1' using instanceDir: ../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.OverseerRolesTest-jetty4-1390342747687/testOverseerCol_shard4_replica1
   [junit4]   2> 1162676 T4147 oasc.CoreContainer.create Creating SolrCore 'testOverseerCol_shard2_replica1' using instanceDir: ./org.apache.solr.cloud.OverseerRolesTest-controljetty-1390342744102/testOverseerCol_shard2_replica1
   [junit4]   2> 1162676 T4147 oasc.ZkController.createCollectionZkNode Check for collection zkNode:testOverseerCol
   [junit4]   2> 1162677 T4234 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for testOverseerCol_shard1_replica2
   [junit4]   2> 1162677 T4220 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for testOverseerCol_shard4_replica2
   [junit4]   2> 1162678 T4234 oasc.CoreContainer.create Creating SolrCore 'testOverseerCol_shard1_replica2' using instanceDir: ../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.OverseerRolesTest-jetty6-1390342749964/testOverseerCol_shard1_replica2
   [junit4]   2> 1162678 T4234 oasc.ZkController.createCollectionZkNode Check for collection zkNode:testOverseerCol
   [junit4]   2> 1162676 T4205 oasc.ZkController.createCollectionZkNode Check for collection zkNode:testOverseerCol
   [junit4]   2> 1162678 T4220 oasc.CoreContainer.create Creating SolrCore 'testOverseerCol_shard4_replica2' using instanceDir: ../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.OverseerRolesTest-jetty5-1390342748828/testOverseerCol_shard4_replica2
   [junit4]   2> 1162678 T4190 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for testOverseerCol_shard2_replica2
   [junit4]   2> 1162677 T4178 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for testOverseerCol_shard3_replica1
   [junit4]   2> 1162677 T4147 oasc.ZkController.createCollectionZkNode Collection zkNode exists
   [junit4]   2> 1162679 T4190 oasc.CoreContainer.create Creating SolrCore 'testOverseerCol_shard2_replica2' using instanceDir: ./org.apache.solr.cloud.OverseerRolesTest-jetty3-1390342746553/testOverseerCol_shard2_replica2
   [junit4]   2> 1162680 T4190 oasc.ZkController.createCollectionZkNode Check for collection zkNode:testOverseerCol
   [junit4]   2> 1162677 T4164 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for testOverseerCol_shard3_replica2
   [junit4]   2> 1162680 T4190 oasc.ZkController.createCollectionZkNode Collection zkNode exists
   [junit4]   2> 1162681 T4190 oascc.ZkStateReader.readConfigName Load collection config from:/collections/testOverseerCol
   [junit4]   2> 1162677 T4221 oasc.CoreContainer.create Creating SolrCore 'testOverseerCol_shard1_replica1' using instanceDir: ../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.OverseerRolesTest-jetty5-1390342748828/testOverseerCol_shard1_replica1
   [junit4]   2> 1162681 T4164 oasc.CoreContainer.create Creating SolrCore 'testOverseerCol_shard3_replica2' using instanceDir: ../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.OverseerRolesTest-jetty1-1390342744274/testOverseerCol_shard3_replica2
   [junit4]   2> 1162680 T4178 oasc.CoreContainer.create Creating SolrCore 'testOverseerCol_shard3_replica1' using instanceDir: ./org.apache.solr.cloud.OverseerRolesTest-jetty2-1390342745414/testOverseerCol_shard3_replica1
   [junit4]   2> 1162679 T4147 oascc.ZkStateReader.readConfigName Load collection config from:/collections/testOverseerCol
   [junit4]   2> 1162679 T4205 oasc.ZkController.createCollectionZkNode Collection zkNode exists
   [junit4]   2> 1162679 T4220 oasc.ZkController.createCollectionZkNode Check for collection zkNode:testOverseerCol
   [junit4]   2> 1162678 T4234 oasc.ZkController.createCollectionZkNode Collection zkNode exists
   [junit4]   2> 1162683 T4147 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: './org.apache.solr.cloud.OverseerRolesTest-controljetty-1390342744102/testOverseerCol_shard2_replica1/'
   [junit4]   2> 1162683 T4220 oasc.ZkController.createCollectionZkNode Collection zkNode exists
   [junit4]   2> 1162683 T4234 oascc.ZkStateReader.readConfigName Load collection config from:/collections/testOverseerCol
   [junit4]   2> 1162682 T4205 oascc.ZkStateReader.readConfigName Load collection config from:/collections/testOverseerCol
   [junit4]   2> 1162682 T4190 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: './org.apache.solr.cloud.OverseerRolesTest-jetty3-1390342746553/testOverseerCol_shard2_replica2/'
   [junit4]   2> 1162682 T4178 oasc.ZkController.createCollectionZkNode Check for collection zkNode:testOverseerCol
   [junit4]   2> 1162681 T4164 oasc.ZkController.createCollectionZkNode Check for collection zkNode:testOverseerCol
   [junit4]   2> 1162681 T4221 oasc.ZkController.createCollectionZkNode Check for collection zkNode:testOverseerCol
   [junit4]   2> 1162685 T4178 oasc.ZkController.createCollectionZkNode Collection zkNode exists
   [junit4]   2> 1162685 T4234 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.OverseerRolesTest-jetty6-1390342749964/testOverseerCol_shard1_replica2/'
   [junit4]   2> 1162686 T4221 oasc.ZkController.createCollectionZkNode Collection zkNode exists
   [junit4]   2> 1162686 T4221 oascc.ZkStateReader.readConfigName Load collection config from:/collections/testOverseerCol
   [junit4]   2> 1162685 T4205 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.OverseerRolesTest-jetty4-1390342747687/testOverseerCol_shard4_replica1/'
   [junit4]   2> 1162683 T4220 oascc.ZkStateReader.readConfigName Load collection config from:/collections/testOverseerCol
   [junit4]   2> 1162691 T4164 oasc.ZkController.createCollectionZkNode Collection zkNode exists
   [junit4]   2> 1162691 T4164 oascc.ZkStateReader.readConfigName Load collection config from:/collections/testOverseerCol
   [junit4]   2> 1162691 T4221 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.OverseerRolesTest-jetty5-1390342748828/testOverseerCol_shard1_replica1/'
   [junit4]   2> 1162685 T4178 oascc.ZkStateReader.readConfigName Load collection config from:/collections/testOverseerCol
   [junit4]   2> 1162693 T4220 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.OverseerRolesTest-jetty5-1390342748828/testOverseerCol_shard4_replica2/'
   [junit4]   2> 1162693 T4164 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.OverseerRolesTest-jetty1-1390342744274/testOverseerCol_shard3_replica2/'
   [junit4]   2> 1162694 T4178 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: './org.apache.solr.cloud.OverseerRolesTest-jetty2-1390342745414/testOverseerCol_shard3_replica1/'
   [junit4]   2> 1162708 T4147 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
   [junit4]   2> 1162733 T4147 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 1162735 T4147 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
   [junit4]   2> 1162736 T4164 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
   [junit4]   2> 1162742 T4147 oass.IndexSchema.readSchema [testOverseerCol_shard2_replica1] Schema name=test
   [junit4]   2> 1162747 T4190 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
   [junit4]   2> 1162769 T4205 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
   [junit4]   2> 1162771 T4164 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 1162771 T4234 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
   [junit4]   2> 1162784 T4205 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 1162787 T4164 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
   [junit4]   2> 1162799 T4164 oass.IndexSchema.readSchema [testOverseerCol_shard3_replica2] Schema name=test
   [junit4]   2> 1162804 T4234 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 1162804 T4205 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
   [junit4]   2> 1162806 T4190 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 1162809 T4190 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
   [junit4]   2> 1162822 T4234 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
   [junit4]   2> 1162823 T4205 oass.IndexSchema.readSchema [testOverseerCol_shard4_replica1] Schema name=test
   [junit4]   2> 1162848 T4190 oass.IndexSchema.readSchema [testOverseerCol_shard2_replica2] Schema name=test
   [junit4]   2> 1162850 T4220 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
   [junit4]   2> 1162851 T4234 oass.IndexSchema.readSchema [testOverseerCol_shard1_replica2] Schema name=test
   [junit4]   2> 1162862 T4178 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
   [junit4]   2> 1162900 T4220 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 1162902 T4220 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
   [junit4]   2> 1162904 T4221 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
   [junit4]   2> 1162910 T4220 oass.IndexSchema.readSchema [testOverseerCol_shard4_replica2] Schema name=test
   [junit4]   2> 1162942 T4178 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 1162947 T4178 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
   [junit4]   2> 1162965 T4221 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 1162967 T4221 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
   [junit4]   2> 1162973 T4221 oass.IndexSchema.readSchema [testOverseerCol_shard1_replica1] Schema name=test
   [junit4]   2> 1163000 T4178 oass.IndexSchema.readSchema [testOverseerCol_shard3_replica1] Schema name=test
   [junit4]   2> 1163289 T4164 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 1163310 T4147 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 1163314 T4164 oass.IndexSchema.readSchema default search field in schema is text
   [junit4]   2> 1163317 T4147 oass.IndexSchema.readSchema default search field in schema is text
   [junit4]   2> 1163320 T4147 oass.IndexSchema.readSchema unique key field: id
   [junit4]   2> 1163332 T4164 oass.IndexSchema.readSchema unique key field: id
   [junit4]   2> 1163341 T4220 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 1163347 T4220 oass.IndexSchema.readSchema default search field in schema is text
   [junit4]   2> 1163350 T4220 oass.IndexSchema.readSchema unique key field: id
   [junit4]   2> 1163350 T4164 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 1163355 T4164 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 1163350 T4147 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 1163359 T4164 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 1163366 T4147 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 1163373 T4147 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 1163375 T4164 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
   [junit4]   2> 1163375 T4147 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
   [junit4]   2> 1163379 T4147 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
   [junit4]   2> 1163378 T4164 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
   [junit4]   2> 1163379 T4220 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 1163379 T4147 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 1163379 T4164 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 1163381 T4147 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
   [junit4]   2> 1163382 T4147 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
   [junit4]   2> 1163382 T4147 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 1163382 T4220 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 1163382 T4147 oasc.SolrCore.<init> [testOverseerCol_shard2_replica1] Opening new SolrCore at ./org.apache.solr.cloud.OverseerRolesTest-controljetty-1390342744102/testOverseerCol_shard2_replica1/, dataDir=./org.apache.solr.cloud.OverseerRolesTest-controljetty-1390342744102/testOverseerCol_shard2_replica1/data/
   [junit4]   2> 1163384 T4147 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@2c15fe95
   [junit4]   2> 1163382 T4164 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
   [junit4]   2> 1163385 T4164 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
   [junit4]   2> 1163386 T4164 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 1163386 T4164 oasc.SolrCore.<init> [testOverseerCol_shard3_replica2] Opening new SolrCore at ../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.OverseerRolesTest-jetty1-1390342744274/testOverseerCol_shard3_replica2/, dataDir=../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.OverseerRolesTest-jetty1-1390342744274/testOverseerCol_shard3_replica2/data/
   [junit4]   2> 1163387 T4164 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@2c15fe95
   [junit4]   2> 1163388 T4164 oasc.CachingDirectoryFactory.get return new directory for ../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.OverseerRolesTest-jetty1-1390342744274/testOverseerCol_shard3_replica2/data
   [junit4]   2> 1163385 T4147 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.OverseerRolesTest-controljetty-1390342744102/testOverseerCol_shard2_replica1/data
   [junit4]   2> 1163389 T4147 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.OverseerRolesTest-controljetty-1390342744102/testOverseerCol_shard2_replica1/data/index/
   [junit4]   2> 1163388 T4164 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.OverseerRolesTest-jetty1-1390342744274/testOverseerCol_shard3_replica2/data/index/
   [junit4]   2> 1163386 T4220 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 1163389 T4164 oasc.SolrCore.initIndex WARN [testOverseerCol_shard3_replica2] Solr index directory '../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.OverseerRolesTest-jetty1-1390342744274/testOverseerCol_shard3_replica2/data/index' doesn't exist. Creating new index...
   [junit4]   2> 1163389 T4147 oasc.SolrCore.initIndex WARN [testOverseerCol_shard2_replica1] Solr index directory './org.apache.solr.cloud.OverseerRolesTest-controljetty-1390342744102/testOverseerCol_shard2_replica1/data/index' doesn't exist. Creating new index...
   [junit4]   2> 1163391 T4164 oasc.CachingDirectoryFactory.get return new directory for ../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.OverseerRolesTest-jetty1-1390342744274/testOverseerCol_shard3_replica2/data/index
   [junit4]   2> 1163392 T4147 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.OverseerRolesTest-controljetty-1390342744102/testOverseerCol_shard2_replica1/data/index
   [junit4]   2> 1163390 T4220 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
   [junit4]   2> 1163392 T4147 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=29, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0]
   [junit4]   2> 1163392 T4164 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=29, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0]
   [junit4]   2> 1163393 T4220 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
   [junit4]   2> 1163394 T4147 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 		commit{dir=MockDirectoryWrapper(org.apache.lucene.store.RAMDirectory@4275e147 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@3c4095b1),segFN=segments_1,generation=1}
   [junit4]   2> 1163395 T4147 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 1163394 T4220 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 1163395 T4164 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 		commit{dir=MockDirectoryWrapper(org.apache.lucene.store.RAMDirectory@f2058b9 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@1781b091),segFN=segments_1,generation=1}
   [junit4]   2> 1163404 T4147 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "nodistrib"
   [junit4]   2> 1163403 T4220 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
   [junit4]   2> 1163411 T4147 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe"
   [junit4]   2> 1163412 T4147 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 1163412 T4147 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "stored_sig"
   [junit4]   2> 1163413 T4147 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 1163413 T4147 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
   [junit4]   2> 1163414 T4147 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 1163414 T4147 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 1163415 T4147 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 1163415 T4147 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardReques

[...truncated too long message...]

seerCol_shard3_replica1/data/index [CachedDir<<refCount=0;path=./org.apache.solr.cloud.OverseerRolesTest-jetty2-1390342745414/testOverseerCol_shard3_replica1/data/index;done=false>>]
   [junit4]   2> 1181448 T4183 oasc.CachingDirectoryFactory.close Closing directory: ./org.apache.solr.cloud.OverseerRolesTest-jetty2-1390342745414/testOverseerCol_shard3_replica1/data/index
   [junit4]   2> 1181454 T4183 oasc.CachingDirectoryFactory.closeCacheValue looking to close ./org.apache.solr.cloud.OverseerRolesTest-jetty2-1390342745414/testOverseerCol_shard3_replica1/data [CachedDir<<refCount=0;path=./org.apache.solr.cloud.OverseerRolesTest-jetty2-1390342745414/testOverseerCol_shard3_replica1/data;done=false>>]
   [junit4]   2> 1181454 T4183 oasc.CachingDirectoryFactory.close Closing directory: ./org.apache.solr.cloud.OverseerRolesTest-jetty2-1390342745414/testOverseerCol_shard3_replica1/data
   [junit4]   2> 1181454 T4183 oascc.SolrZkClient.makePath makePath: /collections/testOverseerCol/leaders/shard3
   [junit4]   2> 1181455 T4183 oasc.SolrException.log ERROR There was a problem trying to register as the leader:org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /collections
   [junit4]   2> 		at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
   [junit4]   2> 		at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
   [junit4]   2> 		at org.apache.zookeeper.ZooKeeper.exists(ZooKeeper.java:1041)
   [junit4]   2> 		at org.apache.solr.common.cloud.SolrZkClient$3.execute(SolrZkClient.java:205)
   [junit4]   2> 		at org.apache.solr.common.cloud.SolrZkClient$3.execute(SolrZkClient.java:202)
   [junit4]   2> 		at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:73)
   [junit4]   2> 		at org.apache.solr.common.cloud.SolrZkClient.exists(SolrZkClient.java:202)
   [junit4]   2> 		at org.apache.solr.common.cloud.SolrZkClient.makePath(SolrZkClient.java:413)
   [junit4]   2> 		at org.apache.solr.common.cloud.SolrZkClient.makePath(SolrZkClient.java:382)
   [junit4]   2> 		at org.apache.solr.common.cloud.SolrZkClient.makePath(SolrZkClient.java:369)
   [junit4]   2> 		at org.apache.solr.cloud.ShardLeaderElectionContextBase.runLeaderProcess(ElectionContext.java:112)
   [junit4]   2> 		at org.apache.solr.cloud.ShardLeaderElectionContext.runLeaderProcess(ElectionContext.java:273)
   [junit4]   2> 		at org.apache.solr.cloud.LeaderElector.runIamLeaderProcess(LeaderElector.java:164)
   [junit4]   2> 		at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:108)
   [junit4]   2> 		at org.apache.solr.cloud.LeaderElector.access$000(LeaderElector.java:55)
   [junit4]   2> 		at org.apache.solr.cloud.LeaderElector$1.process(LeaderElector.java:137)
   [junit4]   2> 		at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:519)
   [junit4]   2> 		at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:495)
   [junit4]   2> 	
   [junit4]   2> 1181455 T4183 oasc.LeaderElector$1.process WARN  org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /collections/testOverseerCol/leader_elect/shard3/election/91117502065934344-core_node2-n_0000000001
   [junit4]   2> 	at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
   [junit4]   2> 	at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
   [junit4]   2> 	at org.apache.zookeeper.ZooKeeper.delete(ZooKeeper.java:873)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$2.execute(SolrZkClient.java:173)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$2.execute(SolrZkClient.java:170)
   [junit4]   2> 	at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:73)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient.delete(SolrZkClient.java:170)
   [junit4]   2> 	at org.apache.solr.cloud.ElectionContext.cancelElection(ElectionContext.java:67)
   [junit4]   2> 	at org.apache.solr.cloud.ShardLeaderElectionContext.runLeaderProcess(ElectionContext.java:293)
   [junit4]   2> 	at org.apache.solr.cloud.LeaderElector.runIamLeaderProcess(LeaderElector.java:164)
   [junit4]   2> 	at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:108)
   [junit4]   2> 	at org.apache.solr.cloud.LeaderElector.access$000(LeaderElector.java:55)
   [junit4]   2> 	at org.apache.solr.cloud.LeaderElector$1.process(LeaderElector.java:137)
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:519)
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:495)
   [junit4]   2> 
   [junit4]   2> 1181456 T4183 oascc.ZkStateReader$3.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
   [junit4]   2> NOTE: test params are: codec=Lucene46: {}, docValues:{}, sim=RandomSimilarityProvider(queryNorm=false,coord=yes): {}, locale=en_SG, timezone=Pacific/Midway
   [junit4]   2> NOTE: Linux 3.8.0-35-generic amd64/Oracle Corporation 1.8.0-ea (64-bit)/cpus=8,threads=1,free=154381400,total=368992256
   [junit4]   2> NOTE: All tests run in this JVM: [TestJoin, TestSearchPerf, SolrIndexSplitterTest, HdfsSyncSliceTest, QueryElevationComponentTest, TestReversedWildcardFilterFactory, SuggestComponentTest, TestSurroundQueryParser, PrimitiveFieldTypeTest, AutoCommitTest, UpdateParamsTest, TestLMJelinekMercerSimilarityFactory, BasicFunctionalityTest, SuggesterFSTTest, HdfsBasicDistributedZkTest, WordBreakSolrSpellCheckerTest, LegacyHTMLStripCharFilterTest, SchemaVersionSpecificBehaviorTest, TestFaceting, CircularListTest, TestQueryUtils, TestAnalyzeInfixSuggestions, DateMathParserTest, TestLuceneMatchVersion, UpdateRequestProcessorFactoryTest, FunctionTest, DistributedQueryElevationComponentTest, SyncSliceTest, ScriptEngineTest, TestLMDirichletSimilarityFactory, TestReloadAndDeleteDocs, TestAddFieldRealTimeGet, DistributedTermsComponentTest, BinaryUpdateRequestHandlerTest, TestNRTOpen, HdfsRecoveryZkTest, TestCharFilters, TestCloudManagedSchema, ShardSplitTest, EchoParamsTest, SampleTest, TestConfig, SliceStateTest, TestPostingsSolrHighlighter, TestManagedSchema, MigrateRouteKeyTest, ChangedSchemaMergeTest, LeaderElectionTest, SynonymTokenizerTest, AddSchemaFieldsUpdateProcessorFactoryTest, TestCustomSort, RecoveryZkTest, DocumentBuilderTest, TestRemoteStreaming, TestSolrIndexConfig, DirectUpdateHandlerOptimizeTest, SoftAutoCommitTest, TestCollationFieldDocValues, TestMergePolicyConfig, TestJmxMonitoredMap, ZkControllerTest, ShardRoutingTest, PreAnalyzedUpdateProcessorTest, TestReplicationHandler, DOMUtilTest, SimpleFacetsTest, SystemInfoHandlerTest, TestLRUCache, TriLevelCompositeIdRoutingTest, DistributedSpellCheckComponentTest, BadCopyFieldTest, TestRealTimeGet, TestUniqueKeyFieldResource, MBeansHandlerTest, TestCoreContainer, TestValueSourceCache, PolyFieldTest, ResourceLoaderTest, RegexBoostProcessorTest, TestExtendedDismaxParser, FileBasedSpellCheckerTest, ShowFileRequestHandlerTest, URLClassifyProcessorTest, DistribCursorPagingTest, TestRecoveryHdfs, CSVRequestHandlerTest, DocValuesTest, QueryEqualityTest, ZkSolrClientTest, TestRandomDVFaceting, TestHashPartitioner, TestStressVersions, SolrCoreTest, TestGroupingSearch, TestFunctionQuery, TestBadConfig, OverseerCollectionProcessorTest, AnalysisAfterCoreReloadTest, CoreAdminHandlerTest, SuggesterTest, TestStressLucene, SpatialFilterTest, SolrCoreCheckLockOnStartupTest, TestUpdate, StatelessScriptUpdateProcessorFactoryTest, SolrInfoMBeanTest, CacheHeaderTest, StandardRequestHandlerTest, TestWriterPerf, TestQueryTypes, XmlUpdateRequestHandlerTest, MoreLikeThisHandlerTest, FieldAnalysisRequestHandlerTest, TestArbitraryIndexDir, LoggingHandlerTest, IndexSchemaTest, JSONWriterTest, QueryParsingTest, UniqFieldsUpdateProcessorFactoryTest, TestPHPSerializedResponseWriter, TestLFUCache, TestQuerySenderListener, CopyFieldTest, BadComponentTest, TestSolrDeletionPolicy2, TestDocSet, TestBinaryField, MinimalSchemaTest, TestElisionMultitermQuery, ExternalFileFieldSortTest, TestSolrCoreProperties, DirectSolrConnectionTest, TestPhraseSuggestions, SpellPossibilityIteratorTest, TestPerFieldSimilarity, TestBM25SimilarityFactory, TimeZoneUtilsTest, OpenExchangeRatesOrgProviderTest, PluginInfoTest, TestSolrXMLSerializer, TestSystemIdResolver, TestSuggestSpellingConverter, DateFieldTest, SpellingQueryConverterTest, ClusterStateTest, TestDocumentBuilder, UUIDFieldTest, CursorPagingTest, OverseerRolesTest]
   [junit4] Completed on J0 in 27.45s, 1 test, 1 error <<< FAILURES!

[...truncated 142 lines...]
BUILD FAILED
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/build.xml:453: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/build.xml:433: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/build.xml:39: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/extra-targets.xml:37: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build.xml:189: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/common-build.xml:491: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/common-build.xml:1304: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/common-build.xml:937: There were test failures: 364 suites, 1596 tests, 1 error, 34 ignored (6 assumptions)

Total time: 48 minutes 55 seconds
Build step 'Invoke Ant' marked build as failure
Description set: Java: 64bit/jdk1.8.0-ea-b123 -XX:-UseCompressedOops -XX:+UseConcMarkSweepGC
Archiving artifacts
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure