You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by Policeman Jenkins Server <je...@thetaphi.de> on 2014/02/06 12:55:51 UTC

[JENKINS] Lucene-Solr-4.x-MacOSX (64bit/jdk1.7.0) - Build # 1270 - Failure!

Build: http://jenkins.thetaphi.de/job/Lucene-Solr-4.x-MacOSX/1270/
Java: 64bit/jdk1.7.0 -XX:+UseCompressedOops -XX:+UseG1GC

1 tests failed.
REGRESSION:  org.apache.solr.cloud.OverseerRolesTest.testDistribSearch

Error Message:
Captured an uncaught exception in thread: Thread[id=392, name=Overseer-91205454901149710-127.0.0.1:52916_-n_0000000005, state=RUNNABLE, group=Overseer collection creation process.]

Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=392, name=Overseer-91205454901149710-127.0.0.1:52916_-n_0000000005, state=RUNNABLE, group=Overseer collection creation process.]
	at __randomizedtesting.SeedInfo.seed([55788483427CCF75:D49E0A9B3523AF49]:0)
Caused by: org.apache.solr.common.cloud.ZooKeeperException: 
	at __randomizedtesting.SeedInfo.seed([55788483427CCF75]:0)
	at org.apache.solr.cloud.OverseerCollectionProcessor.run(OverseerCollectionProcessor.java:209)
	at java.lang.Thread.run(Thread.java:744)
Caused by: org.apache.zookeeper.KeeperException$NoNodeException: KeeperErrorCode = NoNode for /overseer/collection-queue-work/qn-0000000004
	at org.apache.zookeeper.KeeperException.create(KeeperException.java:111)
	at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
	at org.apache.zookeeper.ZooKeeper.delete(ZooKeeper.java:873)
	at org.apache.solr.common.cloud.SolrZkClient$2.execute(SolrZkClient.java:174)
	at org.apache.solr.common.cloud.SolrZkClient$2.execute(SolrZkClient.java:171)
	at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:73)
	at org.apache.solr.common.cloud.SolrZkClient.delete(SolrZkClient.java:171)
	at org.apache.solr.cloud.DistributedQueue.remove(DistributedQueue.java:193)
	at org.apache.solr.cloud.OverseerCollectionProcessor.run(OverseerCollectionProcessor.java:200)
	... 1 more




Build Log:
[...truncated 10097 lines...]
   [junit4] Suite: org.apache.solr.cloud.OverseerRolesTest
   [junit4]   2> 81896 T236 oas.BaseDistributedSearchTestCase.initHostContext Setting hostContext system property: /
   [junit4]   2> 81910 T236 oas.SolrTestCaseJ4.setUp ###Starting testDistribSearch
   [junit4]   2> Creating dataDir: /Users/jenkins/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/./solrtest-OverseerRolesTest-1391684797675
   [junit4]   2> 81912 T236 oasc.ZkTestServer.run STARTING ZK TEST SERVER
   [junit4]   2> 81916 T237 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
   [junit4]   2> 81926 T236 oasc.ZkTestServer.run start zk server on port:52897
   [junit4]   2> 81929 T236 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 81934 T243 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@381ce812 name:ZooKeeperConnection Watcher:127.0.0.1:52897 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 81934 T236 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 81935 T236 oascc.SolrZkClient.makePath makePath: /solr
   [junit4]   2> 81994 T236 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 81998 T245 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@700352fe name:ZooKeeperConnection Watcher:127.0.0.1:52897/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 82002 T236 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 82005 T236 oascc.SolrZkClient.makePath makePath: /collections/collection1
   [junit4]   2> 82064 T236 oascc.SolrZkClient.makePath makePath: /collections/collection1/shards
   [junit4]   2> 82108 T236 oascc.SolrZkClient.makePath makePath: /collections/control_collection
   [junit4]   2> 82137 T236 oascc.SolrZkClient.makePath makePath: /collections/control_collection/shards
   [junit4]   2> 82164 T236 oasc.AbstractZkTestCase.putConfig put /Users/jenkins/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
   [junit4]   2> 82166 T236 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.xml
   [junit4]   2> 82218 T236 oasc.AbstractZkTestCase.putConfig put /Users/jenkins/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
   [junit4]   2> 82219 T236 oascc.SolrZkClient.makePath makePath: /configs/conf1/schema.xml
   [junit4]   2> 82231 T236 oasc.AbstractZkTestCase.putConfig put /Users/jenkins/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 82232 T236 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 82239 T236 oasc.AbstractZkTestCase.putConfig put /Users/jenkins/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
   [junit4]   2> 82240 T236 oascc.SolrZkClient.makePath makePath: /configs/conf1/stopwords.txt
   [junit4]   2> 82246 T236 oasc.AbstractZkTestCase.putConfig put /Users/jenkins/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
   [junit4]   2> 82247 T236 oascc.SolrZkClient.makePath makePath: /configs/conf1/protwords.txt
   [junit4]   2> 82260 T236 oasc.AbstractZkTestCase.putConfig put /Users/jenkins/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
   [junit4]   2> 82260 T236 oascc.SolrZkClient.makePath makePath: /configs/conf1/currency.xml
   [junit4]   2> 82269 T236 oasc.AbstractZkTestCase.putConfig put /Users/jenkins/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
   [junit4]   2> 82274 T236 oascc.SolrZkClient.makePath makePath: /configs/conf1/enumsConfig.xml
   [junit4]   2> 82283 T236 oasc.AbstractZkTestCase.putConfig put /Users/jenkins/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
   [junit4]   2> 82284 T236 oascc.SolrZkClient.makePath makePath: /configs/conf1/open-exchange-rates.json
   [junit4]   2> 82292 T236 oasc.AbstractZkTestCase.putConfig put /Users/jenkins/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 82292 T236 oascc.SolrZkClient.makePath makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 82300 T236 oasc.AbstractZkTestCase.putConfig put /Users/jenkins/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
   [junit4]   2> 82308 T236 oascc.SolrZkClient.makePath makePath: /configs/conf1/old_synonyms.txt
   [junit4]   2> 82325 T236 oasc.AbstractZkTestCase.putConfig put /Users/jenkins/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
   [junit4]   2> 82326 T236 oascc.SolrZkClient.makePath makePath: /configs/conf1/synonyms.txt
   [junit4]   2> 82742 T236 oejs.Server.doStart jetty-8.1.10.v20130312
   [junit4]   2> 82749 T236 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:52900
   [junit4]   2> 82749 T236 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
   [junit4]   2> 82750 T236 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 82751 T236 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ../../../../../../../../../Users/jenkins/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.OverseerRolesTest-controljetty-1391684798135
   [junit4]   2> 82751 T236 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../Users/jenkins/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.OverseerRolesTest-controljetty-1391684798135/'
   [junit4]   2> 82781 T236 oasc.ConfigSolr.fromFile Loading container configuration from /Users/jenkins/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/../../../../../../../../../Users/jenkins/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.OverseerRolesTest-controljetty-1391684798135/solr.xml
   [junit4]   2> 82859 T236 oasc.ConfigSolrXml.<init> Config-defined core root directory: 
   [junit4]   2> 82860 T236 oasc.CoreContainer.<init> New CoreContainer 815329945
   [junit4]   2> 82860 T236 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=../../../../../../../../../Users/jenkins/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.OverseerRolesTest-controljetty-1391684798135/]
   [junit4]   2> 82861 T236 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
   [junit4]   2> 82862 T236 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
   [junit4]   2> 82862 T236 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
   [junit4]   2> 82863 T236 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
   [junit4]   2> 82863 T236 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
   [junit4]   2> 82864 T236 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
   [junit4]   2> 82864 T236 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
   [junit4]   2> 82865 T236 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
   [junit4]   2> 82865 T236 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
   [junit4]   2> 82878 T236 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 82879 T236 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 82879 T236 oasc.CoreContainer.load Host Name: 127.0.0.1
   [junit4]   2> 82879 T236 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:52897/solr
   [junit4]   2> 82880 T236 oasc.ZkController.checkChrootPath zkHost includes chroot
   [junit4]   2> 82882 T236 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 82886 T256 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@95f33bc name:ZooKeeperConnection Watcher:127.0.0.1:52897 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 82887 T236 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 82895 T236 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 82901 T258 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@3fcd205c name:ZooKeeperConnection Watcher:127.0.0.1:52897/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 82901 T236 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 82907 T236 oascc.SolrZkClient.makePath makePath: /overseer/queue
   [junit4]   2> 82919 T236 oascc.SolrZkClient.makePath makePath: /overseer/collection-queue-work
   [junit4]   2> 82929 T236 oascc.SolrZkClient.makePath makePath: /live_nodes
   [junit4]   2> 82937 T236 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:52900_
   [junit4]   2> 82939 T236 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:52900_
   [junit4]   2> 82950 T236 oascc.SolrZkClient.makePath makePath: /overseer_elect
   [junit4]   2> 82956 T236 oascc.SolrZkClient.makePath makePath: /overseer_elect/election
   [junit4]   2> 82970 T236 oasc.OverseerElectionContext.runLeaderProcess I am going to be the leader 127.0.0.1:52900_
   [junit4]   2> 82970 T236 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
   [junit4]   2> 82977 T236 oasc.Overseer.start Overseer (id=91205454901149699-127.0.0.1:52900_-n_0000000000) starting
   [junit4]   2> 82987 T236 oascc.SolrZkClient.makePath makePath: /overseer/queue-work
   [junit4]   2> 82999 T260 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
   [junit4]   2> 83001 T236 oascc.SolrZkClient.makePath makePath: /clusterstate.json
   [junit4]   2> 83002 T260 oasc.OverseerCollectionProcessor.prioritizeOverseerNodes prioritizing overseer nodes
   [junit4]   2> 83012 T236 oascc.SolrZkClient.makePath makePath: /aliases.json
   [junit4]   2> 83018 T236 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 83027 T259 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
   [junit4]   2> 83032 T236 oasc.CorePropertiesLocator.discover Looking for core definitions underneath /Users/jenkins/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/../../../../../../../../../Users/jenkins/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.OverseerRolesTest-controljetty-1391684798135
   [junit4]   2> 83193 T236 oasc.CorePropertiesLocator.discoverUnder Found core conf in /Users/jenkins/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/../../../../../../../../../Users/jenkins/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.OverseerRolesTest-controljetty-1391684798135/conf/
   [junit4]   2> 83200 T236 oasc.CorePropertiesLocator.discover Found 1 core definitions
   [junit4]   2> 83202 T236 oass.SolrDispatchFilter.init user.dir=/Users/jenkins/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0
   [junit4]   2> 83202 T236 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
   [junit4]   2> 83276 T236 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 83280 T263 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@54848551 name:ZooKeeperConnection Watcher:127.0.0.1:52897/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 83280 T236 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 83287 T236 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 83304 T236 oasc.ChaosMonkey.monkeyLog monkey: init - expire sessions:false cause connection loss:false
   [junit4]   2> 83675 T236 oasc.AbstractFullDistribZkTestBase.createJettys create jetty 1
   [junit4]   2> 83676 T236 oejs.Server.doStart jetty-8.1.10.v20130312
   [junit4]   2> 83681 T236 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:52904
   [junit4]   2> 83683 T236 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
   [junit4]   2> 83683 T236 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 83684 T236 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ../../../../../../../../../Users/jenkins/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.OverseerRolesTest-jetty1-1391684799069
   [junit4]   2> 83684 T236 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../Users/jenkins/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.OverseerRolesTest-jetty1-1391684799069/'
   [junit4]   2> 83714 T236 oasc.ConfigSolr.fromFile Loading container configuration from /Users/jenkins/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/../../../../../../../../../Users/jenkins/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.OverseerRolesTest-jetty1-1391684799069/solr.xml
   [junit4]   2> 83786 T236 oasc.ConfigSolrXml.<init> Config-defined core root directory: 
   [junit4]   2> 83787 T236 oasc.CoreContainer.<init> New CoreContainer 1592798112
   [junit4]   2> 83787 T236 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=../../../../../../../../../Users/jenkins/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.OverseerRolesTest-jetty1-1391684799069/]
   [junit4]   2> 83789 T236 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
   [junit4]   2> 83789 T236 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
   [junit4]   2> 83789 T236 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
   [junit4]   2> 83790 T236 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
   [junit4]   2> 83790 T236 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
   [junit4]   2> 83791 T236 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
   [junit4]   2> 83791 T236 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
   [junit4]   2> 83791 T236 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
   [junit4]   2> 83792 T236 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
   [junit4]   2> 83802 T236 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 83803 T236 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 83803 T236 oasc.CoreContainer.load Host Name: 127.0.0.1
   [junit4]   2> 83803 T236 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:52897/solr
   [junit4]   2> 83804 T236 oasc.ZkController.checkChrootPath zkHost includes chroot
   [junit4]   2> 83805 T236 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 83810 T274 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@69c04611 name:ZooKeeperConnection Watcher:127.0.0.1:52897 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 83811 T236 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 83819 T236 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 83822 T276 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@62c7f808 name:ZooKeeperConnection Watcher:127.0.0.1:52897/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 83823 T236 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 83836 T236 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 84848 T236 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:52904_
   [junit4]   2> 84850 T236 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:52904_
   [junit4]   2> 84859 T258 oascc.ZkStateReader$3.process Updating live nodes... (2)
   [junit4]   2> 84860 T263 oascc.ZkStateReader$3.process Updating live nodes... (2)
   [junit4]   2> 84861 T276 oascc.ZkStateReader$3.process Updating live nodes... (2)
   [junit4]   2> 84876 T236 oasc.CorePropertiesLocator.discover Looking for core definitions underneath /Users/jenkins/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/../../../../../../../../../Users/jenkins/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.OverseerRolesTest-jetty1-1391684799069
   [junit4]   2> 85053 T236 oasc.CorePropertiesLocator.discoverUnder Found core conf in /Users/jenkins/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/../../../../../../../../../Users/jenkins/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.OverseerRolesTest-jetty1-1391684799069/conf/
   [junit4]   2> 85064 T236 oasc.CorePropertiesLocator.discover Found 1 core definitions
   [junit4]   2> 85066 T236 oass.SolrDispatchFilter.init user.dir=/Users/jenkins/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0
   [junit4]   2> 85066 T236 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
   [junit4]   2> 85430 T236 oasc.AbstractFullDistribZkTestBase.createJettys create jetty 2
   [junit4]   2> 85431 T236 oejs.Server.doStart jetty-8.1.10.v20130312
   [junit4]   2> 85445 T236 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:52907
   [junit4]   2> 85447 T236 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
   [junit4]   2> 85447 T236 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 85448 T236 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ./org.apache.solr.cloud.OverseerRolesTest-jetty2-1391684800831
   [junit4]   2> 85449 T236 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: './org.apache.solr.cloud.OverseerRolesTest-jetty2-1391684800831/'
   [junit4]   2> 85478 T236 oasc.ConfigSolr.fromFile Loading container configuration from /Users/jenkins/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.OverseerRolesTest-jetty2-1391684800831/solr.xml
   [junit4]   2> 85561 T236 oasc.ConfigSolrXml.<init> Config-defined core root directory: 
   [junit4]   2> 85562 T236 oasc.CoreContainer.<init> New CoreContainer 1632206107
   [junit4]   2> 85562 T236 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=./org.apache.solr.cloud.OverseerRolesTest-jetty2-1391684800831/]
   [junit4]   2> 85564 T236 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
   [junit4]   2> 85564 T236 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
   [junit4]   2> 85564 T236 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
   [junit4]   2> 85566 T236 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
   [junit4]   2> 85566 T236 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
   [junit4]   2> 85566 T236 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
   [junit4]   2> 85567 T236 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
   [junit4]   2> 85567 T236 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
   [junit4]   2> 85567 T236 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
   [junit4]   2> 85578 T236 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 85578 T236 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 85579 T236 oasc.CoreContainer.load Host Name: 127.0.0.1
   [junit4]   2> 85579 T236 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:52897/solr
   [junit4]   2> 85580 T236 oasc.ZkController.checkChrootPath zkHost includes chroot
   [junit4]   2> 85581 T236 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 85585 T288 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1fb4a983 name:ZooKeeperConnection Watcher:127.0.0.1:52897 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 85587 T236 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 85593 T236 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 85598 T290 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@50a2e9dc name:ZooKeeperConnection Watcher:127.0.0.1:52897/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 85598 T236 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 85614 T236 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 86625 T236 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:52907_
   [junit4]   2> 86628 T236 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:52907_
   [junit4]   2> 86637 T263 oascc.ZkStateReader$3.process Updating live nodes... (3)
   [junit4]   2> 86637 T276 oascc.ZkStateReader$3.process Updating live nodes... (3)
   [junit4]   2> 86637 T290 oascc.ZkStateReader$3.process Updating live nodes... (3)
   [junit4]   2> 86637 T258 oascc.ZkStateReader$3.process Updating live nodes... (3)
   [junit4]   2> 86649 T236 oasc.CorePropertiesLocator.discover Looking for core definitions underneath /Users/jenkins/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.OverseerRolesTest-jetty2-1391684800831
   [junit4]   2> 86755 T236 oasc.CorePropertiesLocator.discoverUnder Found core conf in /Users/jenkins/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.OverseerRolesTest-jetty2-1391684800831/conf/
   [junit4]   2> 86761 T236 oasc.CorePropertiesLocator.discover Found 1 core definitions
   [junit4]   2> 86762 T236 oass.SolrDispatchFilter.init user.dir=/Users/jenkins/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0
   [junit4]   2> 86763 T236 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
   [junit4]   2> 87127 T236 oasc.AbstractFullDistribZkTestBase.createJettys create jetty 3
   [junit4]   2> 87128 T236 oejs.Server.doStart jetty-8.1.10.v20130312
   [junit4]   2> 87133 T236 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:52910
   [junit4]   2> 87134 T236 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
   [junit4]   2> 87135 T236 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 87135 T236 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ./org.apache.solr.cloud.OverseerRolesTest-jetty3-1391684802528
   [junit4]   2> 87135 T236 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: './org.apache.solr.cloud.OverseerRolesTest-jetty3-1391684802528/'
   [junit4]   2> 87165 T236 oasc.ConfigSolr.fromFile Loading container configuration from /Users/jenkins/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.OverseerRolesTest-jetty3-1391684802528/solr.xml
   [junit4]   2> 87278 T236 oasc.ConfigSolrXml.<init> Config-defined core root directory: 
   [junit4]   2> 87279 T236 oasc.CoreContainer.<init> New CoreContainer 277239761
   [junit4]   2> 87279 T236 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=./org.apache.solr.cloud.OverseerRolesTest-jetty3-1391684802528/]
   [junit4]   2> 87281 T236 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
   [junit4]   2> 87281 T236 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
   [junit4]   2> 87281 T236 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
   [junit4]   2> 87282 T236 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
   [junit4]   2> 87282 T236 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
   [junit4]   2> 87282 T236 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
   [junit4]   2> 87283 T236 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
   [junit4]   2> 87283 T236 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
   [junit4]   2> 87284 T236 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
   [junit4]   2> 87294 T236 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 87294 T236 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 87295 T236 oasc.CoreContainer.load Host Name: 127.0.0.1
   [junit4]   2> 87295 T236 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:52897/solr
   [junit4]   2> 87295 T236 oasc.ZkController.checkChrootPath zkHost includes chroot
   [junit4]   2> 87298 T236 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 87302 T302 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@63ef62d0 name:ZooKeeperConnection Watcher:127.0.0.1:52897 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 87303 T236 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 87310 T236 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 87314 T304 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@547791fd name:ZooKeeperConnection Watcher:127.0.0.1:52897/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 87315 T236 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 87327 T236 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 88339 T236 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:52910_
   [junit4]   2> 88342 T236 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:52910_
   [junit4]   2> 88352 T263 oascc.ZkStateReader$3.process Updating live nodes... (4)
   [junit4]   2> 88352 T304 oascc.ZkStateReader$3.process Updating live nodes... (4)
   [junit4]   2> 88352 T258 oascc.ZkStateReader$3.process Updating live nodes... (4)
   [junit4]   2> 88354 T276 oascc.ZkStateReader$3.process Updating live nodes... (4)
   [junit4]   2> 88354 T290 oascc.ZkStateReader$3.process Updating live nodes... (4)
   [junit4]   2> 88371 T236 oasc.CorePropertiesLocator.discover Looking for core definitions underneath /Users/jenkins/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.OverseerRolesTest-jetty3-1391684802528
   [junit4]   2> 88478 T236 oasc.CorePropertiesLocator.discoverUnder Found core conf in /Users/jenkins/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.OverseerRolesTest-jetty3-1391684802528/conf/
   [junit4]   2> 88483 T236 oasc.CorePropertiesLocator.discover Found 1 core definitions
   [junit4]   2> 88484 T236 oass.SolrDispatchFilter.init user.dir=/Users/jenkins/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0
   [junit4]   2> 88485 T236 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
   [junit4]   2> 88858 T236 oasc.AbstractFullDistribZkTestBase.createJettys create jetty 4
   [junit4]   2> 88859 T236 oejs.Server.doStart jetty-8.1.10.v20130312
   [junit4]   2> 88865 T236 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:52913
   [junit4]   2> 88867 T236 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
   [junit4]   2> 88868 T236 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 88868 T236 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ./org.apache.solr.cloud.OverseerRolesTest-jetty4-1391684804250
   [junit4]   2> 88868 T236 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: './org.apache.solr.cloud.OverseerRolesTest-jetty4-1391684804250/'
   [junit4]   2> 88898 T236 oasc.ConfigSolr.fromFile Loading container configuration from /Users/jenkins/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.OverseerRolesTest-jetty4-1391684804250/solr.xml
   [junit4]   2> 89002 T236 oasc.ConfigSolrXml.<init> Config-defined core root directory: 
   [junit4]   2> 89002 T236 oasc.CoreContainer.<init> New CoreContainer 307548210
   [junit4]   2> 89003 T236 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=./org.apache.solr.cloud.OverseerRolesTest-jetty4-1391684804250/]
   [junit4]   2> 89005 T236 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
   [junit4]   2> 89005 T236 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
   [junit4]   2> 89006 T236 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
   [junit4]   2> 89006 T236 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
   [junit4]   2> 89006 T236 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
   [junit4]   2> 89007 T236 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
   [junit4]   2> 89007 T236 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
   [junit4]   2> 89007 T236 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
   [junit4]   2> 89008 T236 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
   [junit4]   2> 89019 T236 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 89019 T236 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 89020 T236 oasc.CoreContainer.load Host Name: 127.0.0.1
   [junit4]   2> 89020 T236 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:52897/solr
   [junit4]   2> 89020 T236 oasc.ZkController.checkChrootPath zkHost includes chroot
   [junit4]   2> 89023 T236 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 89038 T316 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@7e308c04 name:ZooKeeperConnection Watcher:127.0.0.1:52897 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 89039 T236 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 89079 T236 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 89085 T318 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@5ab4cfb2 name:ZooKeeperConnection Watcher:127.0.0.1:52897/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 89086 T236 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 89119 T236 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 90130 T236 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:52913_
   [junit4]   2> 90133 T236 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:52913_
   [junit4]   2> 90143 T258 oascc.ZkStateReader$3.process Updating live nodes... (5)
   [junit4]   2> 90143 T290 oascc.ZkStateReader$3.process Updating live nodes... (5)
   [junit4]   2> 90143 T318 oascc.ZkStateReader$3.process Updating live nodes... (5)
   [junit4]   2> 90143 T263 oascc.ZkStateReader$3.process Updating live nodes... (5)
   [junit4]   2> 90144 T276 oascc.ZkStateReader$3.process Updating live nodes... (5)
   [junit4]   2> 90144 T304 oascc.ZkStateReader$3.process Updating live nodes... (5)
   [junit4]   2> 90156 T236 oasc.CorePropertiesLocator.discover Looking for core definitions underneath /Users/jenkins/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.OverseerRolesTest-jetty4-1391684804250
   [junit4]   2> 90257 T236 oasc.CorePropertiesLocator.discoverUnder Found core conf in /Users/jenkins/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.OverseerRolesTest-jetty4-1391684804250/conf/
   [junit4]   2> 90262 T236 oasc.CorePropertiesLocator.discover Found 1 core definitions
   [junit4]   2> 90263 T236 oass.SolrDispatchFilter.init user.dir=/Users/jenkins/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0
   [junit4]   2> 90263 T236 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
   [junit4]   2> 90640 T236 oasc.AbstractFullDistribZkTestBase.createJettys create jetty 5
   [junit4]   2> 90640 T236 oejs.Server.doStart jetty-8.1.10.v20130312
   [junit4]   2> 90647 T236 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:52916
   [junit4]   2> 90649 T236 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
   [junit4]   2> 90650 T236 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 90650 T236 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ../../../../../../../../../Users/jenkins/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.OverseerRolesTest-jetty5-1391684806028
   [junit4]   2> 90651 T236 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../Users/jenkins/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.OverseerRolesTest-jetty5-1391684806028/'
   [junit4]   2> 90682 T236 oasc.ConfigSolr.fromFile Loading container configuration from /Users/jenkins/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/../../../../../../../../../Users/jenkins/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.OverseerRolesTest-jetty5-1391684806028/solr.xml
   [junit4]   2> 90772 T236 oasc.ConfigSolrXml.<init> Config-defined core root directory: 
   [junit4]   2> 90773 T236 oasc.CoreContainer.<init> New CoreContainer 1713107068
   [junit4]   2> 90773 T236 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=../../../../../../../../../Users/jenkins/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.OverseerRolesTest-jetty5-1391684806028/]
   [junit4]   2> 90775 T236 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
   [junit4]   2> 90776 T236 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
   [junit4]   2> 90776 T236 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
   [junit4]   2> 90777 T236 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
   [junit4]   2> 90777 T236 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
   [junit4]   2> 90778 T236 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
   [junit4]   2> 90778 T236 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
   [junit4]   2> 90779 T236 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
   [junit4]   2> 90779 T236 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
   [junit4]   2> 90792 T236 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 90793 T236 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 90793 T236 oasc.CoreContainer.load Host Name: 127.0.0.1
   [junit4]   2> 90794 T236 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:52897/solr
   [junit4]   2> 90794 T236 oasc.ZkController.checkChrootPath zkHost includes chroot
   [junit4]   2> 90795 T236 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 90804 T330 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@34499143 name:ZooKeeperConnection Watcher:127.0.0.1:52897 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 90805 T236 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 90811 T236 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 90814 T332 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@45312237 name:ZooKeeperConnection Watcher:127.0.0.1:52897/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 90815 T236 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 90827 T236 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 91839 T236 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:52916_
   [junit4]   2> 91842 T236 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:52916_
   [junit4]   2> 91854 T258 oascc.ZkStateReader$3.process Updating live nodes... (6)
   [junit4]   2> 91855 T276 oascc.ZkStateReader$3.process Updating live nodes... (6)
   [junit4]   2> 91855 T332 oascc.ZkStateReader$3.process Updating live nodes... (6)
   [junit4]   2> 91855 T318 oascc.ZkStateReader$3.process Updating live nodes... (6)
   [junit4]   2> 91856 T263 oascc.ZkStateReader$3.process Updating live nodes... (6)
   [junit4]   2> 91855 T290 oascc.ZkStateReader$3.process Updating live nodes... (6)
   [junit4]   2> 91855 T304 oascc.ZkStateReader$3.process Updating live nodes... (6)
   [junit4]   2> 91869 T236 oasc.CorePropertiesLocator.discover Looking for core definitions underneath /Users/jenkins/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/../../../../../../../../../Users/jenkins/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.OverseerRolesTest-jetty5-1391684806028
   [junit4]   2> 92036 T236 oasc.CorePropertiesLocator.discoverUnder Found core conf in /Users/jenkins/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/../../../../../../../../../Users/jenkins/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.OverseerRolesTest-jetty5-1391684806028/conf/
   [junit4]   2> 92046 T236 oasc.CorePropertiesLocator.discover Found 1 core definitions
   [junit4]   2> 92048 T236 oass.SolrDispatchFilter.init user.dir=/Users/jenkins/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0
   [junit4]   2> 92048 T236 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
   [junit4]   2> 92410 T236 oasc.AbstractFullDistribZkTestBase.createJettys create jetty 6
   [junit4]   2> 92410 T236 oejs.Server.doStart jetty-8.1.10.v20130312
   [junit4]   2> 92414 T236 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:52919
   [junit4]   2> 92416 T236 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
   [junit4]   2> 92417 T236 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 92417 T236 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ./org.apache.solr.cloud.OverseerRolesTest-jetty6-1391684807813
   [junit4]   2> 92418 T236 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: './org.apache.solr.cloud.OverseerRolesTest-jetty6-1391684807813/'
   [junit4]   2> 92461 T236 oasc.ConfigSolr.fromFile Loading container configuration from /Users/jenkins/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.OverseerRolesTest-jetty6-1391684807813/solr.xml
   [junit4]   2> 92546 T236 oasc.ConfigSolrXml.<init> Config-defined core root directory: 
   [junit4]   2> 92547 T236 oasc.CoreContainer.<init> New CoreContainer 867452659
   [junit4]   2> 92547 T236 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=./org.apache.solr.cloud.OverseerRolesTest-jetty6-1391684807813/]
   [junit4]   2> 92549 T236 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
   [junit4]   2> 92550 T236 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
   [junit4]   2> 92550 T236 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
   [junit4]   2> 92551 T236 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
   [junit4]   2> 92551 T236 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
   [junit4]   2> 92552 T236 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
   [junit4]   2> 92552 T236 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
   [junit4]   2> 92552 T236 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
   [junit4]   2> 92553 T236 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
   [junit4]   2> 92564 T236 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 92564 T236 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 92564 T236 oasc.CoreContainer.load Host Name: 127.0.0.1
   [junit4]   2> 92565 T236 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:52897/solr
   [junit4]   2> 92565 T236 oasc.ZkController.checkChrootPath zkHost includes chroot
   [junit4]   2> 92566 T236 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 92571 T344 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@9e160a name:ZooKeeperConnection Watcher:127.0.0.1:52897 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 92572 T236 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 92590 T236 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 92597 T346 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@6ce08c10 name:ZooKeeperConnection Watcher:127.0.0.1:52897/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 92598 T236 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 92609 T236 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 93622 T236 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:52919_
   [junit4]   2> 93626 T236 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:52919_
   [junit4]   2> 93644 T332 oascc.ZkStateReader$3.process Updating live nodes... (7)
   [junit4]   2> 93645 T276 oascc.ZkStateReader$3.process Updating live nodes... (7)
   [junit4]   2> 93645 T290 oascc.ZkStateReader$3.process Updating live nodes... (7)
   [junit4]   2> 93646 T346 oascc.ZkStateReader$3.process Updating live nodes... (7)
   [junit4]   2> 93646 T304 oascc.ZkStateReader$3.process Updating live nodes... (7)
   [junit4]   2> 93646 T263 oascc.ZkStateReader$3.process Updating live nodes... (7)
   [junit4]   2> 93646 T318 oascc.ZkStateReader$3.process Updating live nodes... (7)
   [junit4]   2> 93646 T258 oascc.ZkStateReader$3.process Updating live nodes... (7)
   [junit4]   2> 93661 T236 oasc.CorePropertiesLocator.discover Looking for core definitions underneath /Users/jenkins/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.OverseerRolesTest-jetty6-1391684807813
   [junit4]   2> 93769 T236 oasc.CorePropertiesLocator.discoverUnder Found core conf in /Users/jenkins/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.OverseerRolesTest-jetty6-1391684807813/conf/
   [junit4]   2> 93774 T236 oasc.CorePropertiesLocator.discover Found 1 core definitions
   [junit4]   2> 93775 T236 oass.SolrDispatchFilter.init user.dir=/Users/jenkins/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0
   [junit4]   2> 93775 T236 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
   [junit4]   2> 93778 T236 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 93782 T349 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1f0fee7c name:ZooKeeperConnection Watcher:127.0.0.1:52897/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 93783 T236 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 93789 T236 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 93815 T236 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 93820 T351 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1d6f9042 name:ZooKeeperConnection Watcher:127.0.0.1:52897/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 93820 T236 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 93824 T236 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 94669 T254 oasha.CollectionsHandler.handleCreateAction Creating Collection : version=2&numShards=4&action=CREATE&maxShardsPerNode=2&name=testOverseerCol&wt=javabin&replicationFactor=2
   [junit4]   2> 94674 T258 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/collection-queue-work state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 94683 T260 oasc.OverseerCollectionProcessor.run Overseer Collection Processor: Get the message id:/overseer/collection-queue-work/qn-0000000000 message:{
   [junit4]   2> 	  "operation":"createcollection",
   [junit4]   2> 	  "fromApi":"true",
   [junit4]   2> 	  "name":"testOverseerCol",
   [junit4]   2> 	  "replicationFactor":"2",
   [junit4]   2> 	  "numShards":"4",
   [junit4]   2> 	  "maxShardsPerNode":"2"}
   [junit4]   2> 94685 T260 oasc.OverseerCollectionProcessor.processMessage WARN OverseerCollectionProcessor.processMessage : createcollection , {
   [junit4]   2> 	  "operation":"createcollection",
   [junit4]   2> 	  "fromApi":"true",
   [junit4]   2> 	  "name":"testOverseerCol",
   [junit4]   2> 	  "replicationFactor":"2",
   [junit4]   2> 	  "numShards":"4",
   [junit4]   2> 	  "maxShardsPerNode":"2"}
   [junit4]   2> 94691 T260 oasc.OverseerCollectionProcessor.createConfNode Only one config set found in zk - using it:conf1
   [junit4]   2> 94692 T260 oasc.OverseerCollectionProcessor.createConfNode creating collections conf node /collections/testOverseerCol 
   [junit4]   2> 94693 T260 oascc.SolrZkClient.makePath makePath: /collections/testOverseerCol
   [junit4]   2> 94709 T258 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 94713 T259 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 94715 T259 oasc.Overseer$ClusterStateUpdater.buildCollection building a new collection: testOverseerCol
   [junit4]   2> 94716 T259 oasc.Overseer$ClusterStateUpdater.createCollection Create collection testOverseerCol with shards [shard1, shard2, shard3, shard4]
   [junit4]   2> 94731 T258 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 94735 T263 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 7)
   [junit4]   2> 94736 T332 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 7)
   [junit4]   2> 94737 T346 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 7)
   [junit4]   2> 94735 T304 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 7)
   [junit4]   2> 94737 T276 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 7)
   [junit4]   2> 94737 T258 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 7)
   [junit4]   2> 94737 T290 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 7)
   [junit4]   2> 94736 T349 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 7)
   [junit4]   2> 94736 T318 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 7)
   [junit4]   2> 94736 T351 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 7)
   [junit4]   2> 94812 T260 oasc.OverseerCollectionProcessor.createCollection going to create cores replicas shardNames [shard1, shard2, shard3, shard4] , repFactor : 2
   [junit4]   2> 94813 T260 oasc.OverseerCollectionProcessor.createCollection Creating shard testOverseerCol_shard1_replica1 as part of slice shard1 of collection testOverseerCol on 127.0.0.1:52919_
   [junit4]   2> 94825 T260 oasc.OverseerCollectionProcessor.createCollection Creating shard testOverseerCol_shard1_replica2 as part of slice shard1 of collection testOverseerCol on 127.0.0.1:52900_
   [junit4]   2> 94826 T260 oasc.OverseerCollectionProcessor.createCollection Creating shard testOverseerCol_shard2_replica1 as part of slice shard2 of collection testOverseerCol on 127.0.0.1:52907_
   [junit4]   2> 94836 T260 oasc.OverseerCollectionProcessor.createCollection Creating shard testOverseerCol_shard2_replica2 as part of slice shard2 of collection testOverseerCol on 127.0.0.1:52910_
   [junit4]   2> 94837 T260 oasc.OverseerCollectionProcessor.createCollection Creating shard testOverseerCol_shard3_replica1 as part of slice shard3 of collection testOverseerCol on 127.0.0.1:52913_
   [junit4]   2> 94838 T260 oasc.OverseerCollectionProcessor.createCollection Creating shard testOverseerCol_shard3_replica2 as part of slice shard3 of collection testOverseerCol on 127.0.0.1:52904_
   [junit4]   2> 94840 T260 oasc.OverseerCollectionProcessor.createCollection Creating shard testOverseerCol_shard4_replica1 as part of slice shard4 of collection testOverseerCol on 127.0.0.1:52916_
   [junit4]   2> 94841 T260 oasc.OverseerCollectionProcessor.createCollection Creating shard testOverseerCol_shard4_replica2 as part of slice shard4 of collection testOverseerCol on 127.0.0.1:52919_
   [junit4]   2> 94887 T311 oasc.ZkController.publish publishing core=testOverseerCol_shard3_replica1 state=down
   [junit4]   2> 94893 T250 oasc.ZkController.publish publishing core=testOverseerCol_shard1_replica2 state=down
   [junit4]   2> 94893 T269 oasc.ZkController.publish publishing core=testOverseerCol_shard3_replica2 state=down
   [junit4]   2> 94896 T286 oasc.ZkController.publish publishing core=testOverseerCol_shard2_replica1 state=down
   [junit4]   2> 94903 T337 oasc.ZkController.publish publishing core=testOverseerCol_shard4_replica2 state=down
   [junit4]   2> 94903 T326 oasc.ZkController.publish publishing core=testOverseerCol_shard4_replica1 state=down
   [junit4]   2> 94907 T258 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 94907 T250 oasc.ZkController.waitForCoreNodeName look for our core node name
   [junit4]   2> 94908 T311 oasc.ZkController.waitForCoreNodeName look for our core node name
   [junit4]   2> 94910 T269 oasc.ZkController.waitForCoreNodeName look for our core node name
   [junit4]   2> 94909 T296 oasc.ZkController.publish publishing core=testOverseerCol_shard2_replica2 state=down
   [junit4]   2> 94915 T338 oasc.ZkController.publish publishing core=testOverseerCol_shard1_replica1 state=down
   [junit4]   2> 94918 T326 oasc.ZkController.waitForCoreNodeName look for our core node name
   [junit4]   2> 94920 T286 oasc.ZkController.waitForCoreNodeName look for our core node name
   [junit4]   2> 94922 T258 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 94924 T337 oasc.ZkController.waitForCoreNodeName look for our core node name
   [junit4]   2> 94925 T296 oasc.ZkController.waitForCoreNodeName look for our core node name
   [junit4]   2> 94925 T338 oasc.ZkController.waitForCoreNodeName look for our core node name
   [junit4]   2> 94933 T259 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 94935 T259 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=4 message={
   [junit4]   2> 	  "operation":"state",
   [junit4]   2> 	  "state":"down",
   [junit4]   2> 	  "base_url":"http://127.0.0.1:52913",
   [junit4]   2> 	  "core":"testOverseerCol_shard3_replica1",
   [junit4]   2> 	  "roles":null,
   [junit4]   2> 	  "node_name":"127.0.0.1:52913_",
   [junit4]   2> 	  "shard":"shard3",
   [junit4]   2> 	  "shard_range":null,
   [junit4]   2> 	  "shard_state":"active",
   [junit4]   2> 	  "shard_parent":null,
   [junit4]   2> 	  "collection":"testOverseerCol",
   [junit4]   2> 	  "numShards":"4",
   [junit4]   2> 	  "core_node_name":null}
   [junit4]   2> 94959 T259 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=4 message={
   [junit4]   2> 	  "operation":"state",
   [junit4]   2> 	  "state":"down",
   [junit4]   2> 	  "base_url":"http://127.0.0.1:52904",
   [junit4]   2> 	  "core":"testOverseerCol_shard3_replica2",
   [junit4]   2> 	  "roles":null,
   [junit4]   2> 	  "node_name":"127.0.0.1:52904_",
   [junit4]   2> 	  "shard":"shard3",
   [junit4]   2> 	  "shard_range":null,
   [junit4]   2> 	  "shard_state":"active",
   [junit4]   2> 	  "shard_parent":null,
   [junit4]   2> 	  "collection":"testOverseerCol",
   [junit4]   2> 	  "numShards":"4",
   [junit4]   2> 	  "core_node_name":null}
   [junit4]   2> 94967 T258 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 94970 T259 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=4 message={
   [junit4]   2> 	  "operation":"state",
   [junit4]   2> 	  "state":"down",
   [junit4]   2> 	  "base_url":"http://127.0.0.1:52900",
   [junit4]   2> 	  "core":"testOverseerCol_shard1_replica2",
   [junit4]   2> 	  "roles":null,
   [junit4]   2> 	  "node_name":"127.0.0.1:52900_",
   [junit4]   2> 	  "shard":"shard1",
   [junit4]   2> 	  "shard_range":null,
   [junit4]   2> 	  "shard_state":"active",
   [junit4]   2> 	  "shard_parent":null,
   [junit4]   2> 	  "collection":"testOverseerCol",
   [junit4]   2> 	  "numShards":"4",
   [junit4]   2> 	  "core_node_name":null}
   [junit4]   2> 94977 T258 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 94979 T259 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=4 message={
   [junit4]   2> 	  "operation":"state",
   [junit4]   2> 	  "state":"down",
   [junit4]   2> 	  "base_url":"http://127.0.0.1:52916",
   [junit4]   2> 	  "core":"testOverseerCol_shard4_replica1",
   [junit4]   2> 	  "roles":null,
   [junit4]   2> 	  "node_name":"127.0.0.1:52916_",
   [junit4]   2> 	  "shard":"shard4",
   [junit4]   2> 	  "shard_range":null,
   [junit4]   2> 	  "shard_state":"active",
   [junit4]   2> 	  "shard_parent":null,
   [junit4]   2> 	  "collection":"testOverseerCol",
   [junit4]   2> 	  "numShards":"4",
   [junit4]   2> 	  "core_node_name":null}
   [junit4]   2> 94985 T258 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 94987 T259 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=4 message={
   [junit4]   2> 	  "operation":"state",
   [junit4]   2> 	  "state":"down",
   [junit4]   2> 	  "base_url":"http://127.0.0.1:52919",
   [junit4]   2> 	  "core":"testOverseerCol_shard4_replica2",
   [junit4]   2> 	  "roles":null,
   [junit4]   2> 	  "node_name":"127.0.0.1:52919_",
   [junit4]   2> 	  "shard":"shard4",
   [junit4]   2> 	  "shard_range":null,
   [junit4]   2> 	  "shard_state":"active",
   [junit4]   2> 	  "shard_parent":null,
   [junit4]   2> 	  "collection":"testOverseerCol",
   [junit4]   2> 	  "numShards":"4",
   [junit4]   2> 	  "core_node_name":null}
   [junit4]   2> 94994 T258 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 94996 T259 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=4 message={
   [junit4]   2> 	  "operation":"state",
   [junit4]   2> 	  "state":"down",
   [junit4]   2> 	  "base_url":"http://127.0.0.1:52907",
   [junit4]   2> 	  "core":"testOverseerCol_shard2_replica1",
   [junit4]   2> 	  "roles":null,
   [junit4]   2> 	  "node_name":"127.0.0.1:52907_",
   [junit4]   2> 	  "shard":"shard2",
   [junit4]   2> 	  "shard_range":null,
   [junit4]   2> 	  "shard_state":"active",
   [junit4]   2> 	  "shard_parent":null,
   [junit4]   2> 	  "collection":"testOverseerCol",
   [junit4]   2> 	  "numShards":"4",
   [junit4]   2> 	  "core_node_name":null}
   [junit4]   2> 95005 T258 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 95007 T259 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=4 message={
   [junit4]   2> 	  "operation":"state",
   [junit4]   2> 	  "state":"down",
   [junit4]   2> 	  "base_url":"http://127.0.0.1:52910",
   [junit4]   2> 	  "core":"testOverseerCol_shard2_replica2",
   [junit4]   2> 	  "roles":null,
   [junit4]   2> 	  "node_name":"127.0.0.1:52910_",
   [junit4]   2> 	  "shard":"shard2",
   [junit4]   2> 	  "shard_range":null,
   [junit4]   2> 	  "shard_state":"active",
   [junit4]   2> 	  "shard_parent":null,
   [junit4]   2> 	  "collection":"testOverseerCol",
   [junit4]   2> 	  "numShards":"4",
   [junit4]   2> 	  "core_node_name":null}
   [junit4]   2> 95015 T258 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 95019 T259 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=4 message={
   [junit4]   2> 	  "operation":"state",
   [junit4]   2> 	  "state":"down",
   [junit4]   2> 	  "base_url":"http://127.0.0.1:52919",
   [junit4]   2> 	  "core":"testOverseerCol_shard1_replica1",
   [junit4]   2> 	  "roles":null,
   [junit4]   2> 	  "node_name":"127.0.0.1:52919_",
   [junit4]   2> 	  "shard":"shard1",
   [junit4]   2> 	  "shard_range":null,
   [junit4]   2> 	  "shard_state":"active",
   [junit4]   2> 	  "shard_parent":null,
   [junit4]   2> 	  "collection":"testOverseerCol",
   [junit4]   2> 	  "numShards":"4",
   [junit4]   2> 	  "core_node_name":null}
   [junit4]   2> 95027 T258 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 95139 T351 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 7)
   [junit4]   2> 95140 T304 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 7)
   [junit4]   2> 95140 T276 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 7)
   [junit4]   2> 95140 T318 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 7)
   [junit4]   2> 95140 T346 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 7)
   [junit4]   2> 95141 T290 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 7)
   [junit4]   2> 95141 T332 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 7)
   [junit4]   2> 95141 T349 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 7)
   [junit4]   2> 95141 T263 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 7)
   [junit4]   2> 95150 T258 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 7)
   [junit4]   2> 95910 T250 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for testOverseerCol_shard1_replica2
   [junit4]   2> 95911 T269 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for testOverseerCol_shard3_replica2
   [junit4]   2> 95910 T311 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for testOverseerCol_shard3_replica1
   [junit4]   2> 95916 T269 oasc.CoreContainer.create Creating SolrCore 'testOverseerCol_shard3_replica2' using instanceDir: ../../../../../../../../../Users/jenkins/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.OverseerRolesTest-jetty1-1391684799069/testOverseerCol_shard3_replica2
   [junit4]   2> 95916 T269 oasc.ZkController.createCollectionZkNode Check for collection zkNode:testOverseerCol
   [junit4]   2> 95915 T250 oasc.CoreContainer.create Creating SolrCore 'testOverseerCol_shard1_replica2' using instanceDir: ../../../../../../../../../Users/jenkins/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.OverseerRolesTest-controljetty-1391684798135/testOverseerCol_shard1_replica2
   [junit4]   2> 95918 T250 oasc.ZkController.createCollectionZkNode Check for collection zkNode:testOverseerCol
   [junit4]   2> 95917 T311 oasc.CoreContainer.create Creating SolrCore 'testOverseerCol_shard3_replica1' using instanceDir: ./org.apache.solr.cloud.OverseerRolesTest-jetty4-1391684804250/testOverseerCol_shard3_replica1
   [junit4]   2> 95919 T311 oasc.ZkController.createCollectionZkNode Check for collection zkNode:testOverseerCol
   [junit4]   2> 95919 T269 oasc.ZkController.createCollectionZkNode Collection zkNode exists
   [junit4]   2> 95920 T269 oascc.ZkStateReader.readConfigName Load collection config from:/collections/testOverseerCol
   [junit4]   2> 95920 T250 oasc.ZkController.createCollectionZkNode Collection zkNode exists
   [junit4]   2> 95921 T250 oascc.ZkStateReader.readConfigName Load collection config from:/collections/testOverseerCol
   [junit4]   2> 95921 T326 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for testOverseerCol_shard4_replica1
   [junit4]   2> 95921 T311 oasc.ZkController.createCollectionZkNode Collection zkNode exists
   [junit4]   2> 95923 T311 oascc.ZkStateReader.readConfigName Load collection config from:/collections/testOverseerCol
   [junit4]   2> 95922 T286 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for testOverseerCol_shard2_replica1
   [junit4]   2> 95924 T326 oasc.CoreContainer.create Creating SolrCore 'testOverseerCol_shard4_replica1' using instanceDir: ../../../../../../../../../Users/jenkins/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.OverseerRolesTest-jetty5-1391684806028/testOverseerCol_shard4_replica1
   [junit4]   2> 95925 T326 oasc.ZkController.createCollectionZkNode Check for collection zkNode:testOverseerCol
   [junit4]   2> 95926 T337 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for testOverseerCol_shard4_replica2
   [junit4]   2> 95927 T338 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for testOverseerCol_shard1_replica1
   [junit4]   2> 95928 T296 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for testOverseerCol_shard2_replica2
   [junit4]   2> 95926 T286 oasc.CoreContainer.create Creating SolrCore 'testOverseerCol_shard2_replica1' using instanceDir: ./org.apache.solr.cloud.OverseerRolesTest-jetty2-1391684800831/testOverseerCol_shard2_replica1
   [junit4]   2> 95929 T286 oasc.ZkController.createCollectionZkNode Check for collection zkNode:testOverseerCol
   [junit4]   2> 95928 T338 oasc.CoreContainer.create Creating SolrCore 'testOverseerCol_shard1_replica1' using instanceDir: ./org.apache.solr.cloud.OverseerRolesTest-jetty6-1391684807813/testOverseerCol_shard1_replica1
   [junit4]   2> 95930 T338 oasc.ZkController.createCollectionZkNode Check for collection zkNode:testOverseerCol
   [junit4]   2> 95930 T296 oasc.CoreContainer.create Creating SolrCore 'testOverseerCol_shard2_replica2' using instanceDir: ./org.apache.solr.cloud.OverseerRolesTest-jetty3-1391684802528/testOverseerCol_shard2_replica2
   [junit4]   2> 95932 T296 oasc.ZkController.createCollectionZkNode Check for collection zkNode:testOverseerCol
   [junit4]   2> 95931 T337 oasc.CoreContainer.create Creating SolrCore 'testOverseerCol_shard4_replica2' using instanceDir: ./org.apache.solr.cloud.OverseerRolesTest-jetty6-1391684807813/testOverseerCol_shard4_replica2
   [junit4]   2> 95933 T337 oasc.ZkController.createCollectionZkNode Check for collection zkNode:testOverseerCol
   [junit4]   2> 95934 T326 oasc.ZkController.createCollectionZkNode Collection zkNode exists
   [junit4]   2> 95934 T326 oascc.ZkStateReader.readConfigName Load collection config from:/collections/testOverseerCol
   [junit4]   2> 95932 T269 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../Users/jenkins/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.OverseerRolesTest-jetty1-1391684799069/testOverseerCol_shard3_replica2/'
   [junit4]   2> 95934 T338 oasc.ZkController.createCollectionZkNode Collection zkNode exists
   [junit4]   2> 95936 T338 oascc.ZkStateReader.readConfigName Load collection config from:/collections/testOverseerCol
   [junit4]   2> 95934 T250 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../Users/jenkins/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.OverseerRolesTest-controljetty-1391684798135/testOverseerCol_shard1_replica2/'
   [junit4]   2> 95934 T286 oasc.ZkController.createCollectionZkNode Collection zkNode exists
   [junit4]   2> 95938 T286 oascc.ZkStateReader.readConfigName Load collection config from:/collections/testOverseerCol
   [junit4]   2> 95943 T296 oasc.ZkController.createCollectionZkNode Collection zkNode exists
   [junit4]   2> 95943 T296 oascc.ZkStateReader.readConfigName Load collection config from:/collections/testOverseerCol
   [junit4]   2> 95950 T337 oasc.ZkController.createCollectionZkNode Collection zkNode exists
   [junit4]   2> 95951 T337 oascc.ZkStateReader.readConfigName Load collection config from:/collections/testOverseerCol
   [junit4]   2> 95951 T311 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: './org.apache.solr.cloud.OverseerRolesTest-jetty4-1391684804250/testOverseerCol_shard3_replica1/'
   [junit4]   2> 95957 T326 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../Users/jenkins/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.OverseerRolesTest-jetty5-1391684806028/testOverseerCol_shard4_replica1/'
   [junit4]   2> 95998 T338 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: './org.apache.solr.cloud.OverseerRolesTest-jetty6-1391684807813/testOverseerCol_shard1_replica1/'
   [junit4]   2> 95999 T286 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: './org.apache.solr.cloud.OverseerRolesTest-jetty2-1391684800831/testOverseerCol_shard2_replica1/'
   [junit4]   2> 96001 T337 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: './org.apache.solr.cloud.OverseerRolesTest-jetty6-1391684807813/testOverseerCol_shard4_replica2/'
   [junit4]   2> 96004 T296 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: './org.apache.solr.cloud.OverseerRolesTest-jetty3-1391684802528/testOverseerCol_shard2_replica2/'
   [junit4]   2> 96185 T326 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_47
   [junit4]   2> 96226 T269 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_47
   [junit4]   2> 96254 T337 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_47
   [junit4]   2> 96270 T338 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_47
   [junit4]   2> 96396 T250 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_47
   [junit4]   2> 96401 T311 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_47
   [junit4]   2> 96401 T286 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_47
   [junit4]   2> 96415 T296 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_47
   [junit4]   2> 96672 T311 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 96676 T338 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 96694 T311 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
   [junit4]   2> 96706 T338 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
   [junit4]   2> 96715 T311 oass.IndexSchema.readSchema [testOverseerCol_shard3_replica1] Schema name=test
   [junit4]   2> 96718 T250 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 96721 T250 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
   [junit4]   2> 96743 T296 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 96746 T296 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
   [junit4]   2> 96750 T286 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 96754 T286 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
   [junit4]   2> 96758 T326 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 96754 T250 oass.IndexSchema.readSchema [testOverseerCol_shard1_replica2] Schema name=test
   [junit4]   2> 96763 T326 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
   [junit4]   2> 96766 T296 oass.IndexSchema.readSchema [testOverseerCol_shard2_replica2] Schema name=test
   [junit4]   2> 96779 T338 oass.IndexSchema.readSchema [testOverseerCol_shard1_replica1] Schema name=test
   [junit4]   2> 96784 T286 oass.IndexSchema.readSchema [testOverseerCol_shard2_replica1] Schema name=test
   [junit4]   2> 96815 T326 oass.IndexSchema.readSchema [testOverseerCol_shard4_replica1] Schema name=test
   [junit4]   2> 96872 T269 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 96881 T269 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
   [junit4]   2> 96894 T269 oass.IndexSchema.readSchema [testOverseerCol_shard3_replica2] Schema name=test
   [junit4]   2> 96928 T337 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 96936 T337 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
   [junit4]   2> 96951 T337 oass.IndexSchema.readSchema [testOverseerCol_shard4_replica2] Schema name=test
   [junit4]   2> 99417 T286 oass.ByteField.init WARN ByteField is deprecated and will be removed in 5.0. You should use TrieIntField instead.
   [junit4]   2> 99436 T286 oass.ShortField.init WARN ShortField is deprecated and will be removed in 5.0. You should use TrieIntField instead.
   [junit4]   2> 99437 T286 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 99455 T286 oass.IndexSchema.readSchema default search field in schema is text
   [junit4]   2> 99459 T286 oass.IndexSchema.readSchema unique key field: id
   [junit4]   2> 99646 T326 oass.ByteField.init WARN ByteField is deprecated and will be removed in 5.0. You should use TrieIntField instead.
   [junit4]   2> 99655 T326 oass.ShortField.init WARN ShortField is deprecated and will be removed in 5.0. You should use TrieIntField instead.
   [junit4]   2> 99656 T326 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 99679 T326 oass.IndexSchema.readSchema default search field in schema is text
   [junit4]   2> 99706 T326 oass.IndexSchema.readSchema unique key field: id
   [junit4]   2> 99713 T286 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 99724 T286 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 99741 T286 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 99767 T286 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
   [junit4]   2> 99767 T286 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
   [junit4]   2> 99767 T286 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 99782 T286 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
   [junit4]   2> 99783 T286 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
   [junit4]   2> 99783 T286 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 99784 T286 oasc.SolrCore.<init> [testOverseerCol_shard2_replica1] Opening new SolrCore at ./org.apache.solr.cloud.OverseerRolesTest-jetty2-1391684800831/testOverseerCol_shard2_replica1/, dataDir=./org.apache.solr.cloud.OverseerRolesTest-jetty2-1391684800831/testOverseerCol_shard2_replica1/data/
   [junit4]   2> 99791 T286 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@10ae830c
   [junit4]   2> 99818 T286 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.OverseerRolesTest-jetty2-1391684800831/testOverseerCol_shard2_replica1/data
   [junit4]   2> 99829 T286 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.OverseerRolesTest-jetty2-1391684800831/testOverseerCol_shard2_replica1/data/index/
   [junit4]   2> 99830 T286 oasc.SolrCore.initIndex WARN [testOverseerCol_shard2_replica1] Solr index directory './org.apache.solr.cloud.OverseerRolesTest-jetty2-1391684800831/testOverseerCol_shard2_replica1/data/index' doesn't exist. Creating new index...
   [junit4]   2> 99830 T286 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.OverseerRolesTest-jetty2-1391684800831/testOverseerCol_shard2_replica1/data/index
   [junit4]   2> 99831 T286 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=16, maxMergeAtOnceExplicit=36, maxMergedSegmentMB=46.3818359375, floorSegmentMB=0.390625, forceMergeDeletesPctAllowed=6.774691956688858, segmentsPerTier=33.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0
   [junit4]   2> 99841 T326 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 99849 T326 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 99856 T326 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 99860 T286 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 		commit{dir=MockDirectoryWrapper(RAMDirectory@122c00b3 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@2574efb4),segFN=segments_1,generation=1}
   [junit4]   2> 99861 T286 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 99866 T286 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "nodistrib"
   [junit4]   2> 99873 T326 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
   [junit4]   2> 99873 T326 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
   [junit4]   2> 99874 T326 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 99875 T326 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
   [junit4]   2> 99876 T326 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
   [junit4]   2> 99876 T326 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 99878 T326 oasc.SolrCore.<init> [testOverseerCol_shard4_replica1] Opening new SolrCore at ../../../../../../../../../Users/jenkins/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.OverseerRolesTest-jetty5-1391684806028/testOverseerCol_shard4_replica1/, dataDir=../../../../../../../../../Users/jenkins/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.OverseerRolesTest-jetty5-1391684806028/testOverseerCol_shard4_replica1/data/
   [junit4]   2> 99878 T326 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@10ae830c
   [junit4]   2> 99904 T250 oass.ByteField.init WARN ByteField is deprecated and will be removed in 5.0. You should use TrieIntField instead.
   [junit4]   2> 99905 T250 oass.ShortField.init WARN ShortField is deprecated and will be removed in 5.0. You should use TrieIntField instead.
   [junit4]   2> 99905 T250 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 99913 T326 oasc.CachingDirectoryFactory.get return new directory for ../../../../../../../../../Users/jenkins/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.OverseerRolesTest-jetty5-1391684806028/testOverseerCol_shard4_replica1/data
   [junit4]   2> 99914 T326 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=../../../../../../../../../Users/jenkins/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.OverseerRolesTest-jetty5-1391684806028/testOverseerCol_shard4_replica1/data/index/
   [junit4]   2> 99914 T326 oasc.SolrCore.initIndex WARN [testOverseerCol_shard4_replica1] Solr index directory '../../../../../../../../../Users/jenkins/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.OverseerRolesTest-jetty5-1391684806028/testOverseerCol_shard4_replica1/data/index' doesn't exist. Creating new index...
   [junit4]   2> 99915 T326 oasc.CachingDirectoryFactory.get return new directory for ../../../../../../../../../Users/jenkins/workspace/Lucene-Solr-4.x-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.OverseerRolesTest-jetty5-1391684806028/testOverseerCol_shard4_replica1/data/index
   [junit4]   2> 99915 T326 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=16, maxMergeAtOnceExplicit=36, maxMergedSegmentMB=46.3818359375, floorSegmentMB=0.390625, forceMergeDeletesPctAllowed=6.774691956688858, segmentsPerTier=33.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0
   [junit4]   2> 99917 T326 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 		commit{dir=MockDirectoryWrapper(RAMDirectory@3b11ed3a lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@1ead437b),segFN=segments_1,generation=1}
   [junit4]   2> 99917 T326 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 99923 T326 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "nodistrib"
   [junit4]   2> 99925 T250 oass.IndexSchema.readSchema default search field in schema is text
   [junit4]   2> 99983 T250 oass.IndexSchema.readSchema unique key field: id
   [junit4]   2> 99984 T326 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe"
   [junit4]   2> 99984 T326 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 99985 T326 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "stored_sig"
   [junit4]   2> 99985 T326 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 99986 T326 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
   [junit4]   2> 99986 T326 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 99987 T326 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 99987 T326 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 99988 T326 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
   [junit4]   2> 99989 T326 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
   [junit4]   2> 99989 T326 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
   [junit4]   2> 99989 T326 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
   [junit4]   2> 99990 T326 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
   [junit4]   2> 99991 T326 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
   [junit4]   2> 99992 T326 oasc.RequestHandlers.initHandlersFromConfig created /admin/file: solr.admin.ShowFileRequestHandler
   [junit4]   2> 99983 T286 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe"
   [junit4]   2> 99994 T286 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 99995 T286 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "stored_sig"
   [junit4]   2> 99995 T286 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 99996 T286 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
   [junit4]   2> 99996 T286 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 99997 T286 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 99997 T286 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 99998 T286 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
   [junit4]   2> 99999 T286 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
   [junit4]   2> 99999 T286 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
   [junit4]   2> 100000 T286 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
   [junit4]   2> 100001 T286 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
   [junit4]   2> 100002 T286 o

[...truncated too long message...]

g.apache.solr.common.cloud.SolrZkClient.delete(SolrZkClient.java:171)
   [junit4]   2> 	at org.apache.solr.cloud.ElectionContext.cancelElection(ElectionContext.java:67)
   [junit4]   2> 	at org.apache.solr.cloud.ShardLeaderElectionContext.runLeaderProcess(ElectionContext.java:293)
   [junit4]   2> 	at org.apache.solr.cloud.LeaderElector.runIamLeaderProcess(LeaderElector.java:164)
   [junit4]   2> 	at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:108)
   [junit4]   2> 	at org.apache.solr.cloud.LeaderElector.access$000(LeaderElector.java:55)
   [junit4]   2> 	at org.apache.solr.cloud.LeaderElector$1.process(LeaderElector.java:137)
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:519)
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:495)
   [junit4]   2> 
   [junit4]   2> 109489 T318 oascc.ZkStateReader$3.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
   [junit4]   2>  C23_STATE=coll:testOverseerCol core:testOverseerCol_shard2_replica2 props:{state=active, base_url=http://127.0.0.1:52910, core=testOverseerCol_shard2_replica2, node_name=127.0.0.1:52910_}
   [junit4]   2> 109564 T304 C23 P52910 oasc.SyncStrategy.sync WARN Closed, skipping sync up.
   [junit4]   2> 109565 T304 oasc.ShardLeaderElectionContext.runLeaderProcess We failed sync, but we have no versions - we can't sync in that case - we were active before, so become leader anyway
   [junit4]   2> 109565 T304 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:52910/testOverseerCol_shard2_replica2/ shard2
   [junit4]   2> 109566 T304 oasc.SolrCore.close [testOverseerCol_shard2_replica2]  CLOSING SolrCore org.apache.solr.core.SolrCore@1605bc16
   [junit4]   2> 109585 T304 oasu.DirectUpdateHandler2.close closing DirectUpdateHandler2{commits=0,autocommits=0,soft autocommits=0,optimizes=0,rollbacks=0,expungeDeletes=0,docsPending=0,adds=0,deletesById=0,deletesByQuery=0,errors=0,cumulative_adds=0,cumulative_deletesById=0,cumulative_deletesByQuery=0,cumulative_errors=0,transaction_logs_total_size=0,transaction_logs_total_number=0}
   [junit4]   2> 109586 T304 oasu.SolrCoreState.decrefSolrCoreState Closing SolrCoreState
   [junit4]   2> 109587 T304 oasu.DefaultSolrCoreState.closeIndexWriter SolrCoreState ref count has reached 0 - closing IndexWriter
   [junit4]   2> 109587 T304 oasu.DefaultSolrCoreState.closeIndexWriter closing IndexWriter with IndexWriterCloser
   [junit4]   2> 109589 T304 oasc.SolrCore.closeSearcher [testOverseerCol_shard2_replica2] Closing main searcher on request.
   [junit4]   2> 109589 T304 oasc.CachingDirectoryFactory.close Closing MockDirectoryFactory - 2 directories currently being tracked
   [junit4]   2> 109589 T304 oasc.CachingDirectoryFactory.closeCacheValue looking to close ./org.apache.solr.cloud.OverseerRolesTest-jetty3-1391684802528/testOverseerCol_shard2_replica2/data [CachedDir<<refCount=0;path=./org.apache.solr.cloud.OverseerRolesTest-jetty3-1391684802528/testOverseerCol_shard2_replica2/data;done=false>>]
   [junit4]   2> 109590 T304 oasc.CachingDirectoryFactory.close Closing directory: ./org.apache.solr.cloud.OverseerRolesTest-jetty3-1391684802528/testOverseerCol_shard2_replica2/data
   [junit4]   2> 109591 T304 oasc.CachingDirectoryFactory.closeCacheValue looking to close ./org.apache.solr.cloud.OverseerRolesTest-jetty3-1391684802528/testOverseerCol_shard2_replica2/data/index [CachedDir<<refCount=0;path=./org.apache.solr.cloud.OverseerRolesTest-jetty3-1391684802528/testOverseerCol_shard2_replica2/data/index;done=false>>]
   [junit4]   2> 109591 T304 oasc.CachingDirectoryFactory.close Closing directory: ./org.apache.solr.cloud.OverseerRolesTest-jetty3-1391684802528/testOverseerCol_shard2_replica2/data/index
   [junit4]   2> 109592 T304 oascc.SolrZkClient.makePath makePath: /collections/testOverseerCol/leaders/shard2
   [junit4]   2> 109592 T304 oasc.SolrException.log ERROR There was a problem trying to register as the leader:org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /collections
   [junit4]   2> 		at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
   [junit4]   2> 		at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
   [junit4]   2> 		at org.apache.zookeeper.ZooKeeper.exists(ZooKeeper.java:1041)
   [junit4]   2> 		at org.apache.solr.common.cloud.SolrZkClient$3.execute(SolrZkClient.java:206)
   [junit4]   2> 		at org.apache.solr.common.cloud.SolrZkClient$3.execute(SolrZkClient.java:203)
   [junit4]   2> 		at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:73)
   [junit4]   2> 		at org.apache.solr.common.cloud.SolrZkClient.exists(SolrZkClient.java:203)
   [junit4]   2> 		at org.apache.solr.common.cloud.SolrZkClient.makePath(SolrZkClient.java:414)
   [junit4]   2> 		at org.apache.solr.common.cloud.SolrZkClient.makePath(SolrZkClient.java:383)
   [junit4]   2> 		at org.apache.solr.common.cloud.SolrZkClient.makePath(SolrZkClient.java:370)
   [junit4]   2> 		at org.apache.solr.cloud.ShardLeaderElectionContextBase.runLeaderProcess(ElectionContext.java:112)
   [junit4]   2> 		at org.apache.solr.cloud.ShardLeaderElectionContext.runLeaderProcess(ElectionContext.java:273)
   [junit4]   2> 		at org.apache.solr.cloud.LeaderElector.runIamLeaderProcess(LeaderElector.java:164)
   [junit4]   2> 		at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:108)
   [junit4]   2> 		at org.apache.solr.cloud.LeaderElector.access$000(LeaderElector.java:55)
   [junit4]   2> 		at org.apache.solr.cloud.LeaderElector$1.process(LeaderElector.java:137)
   [junit4]   2> 		at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:519)
   [junit4]   2> 		at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:495)
   [junit4]   2> 	
   [junit4]   2> 109594 T304 oasc.LeaderElector$1.process WARN  org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /collections/testOverseerCol/leader_elect/shard2/election/91205454901149706-core_node7-n_0000000001
   [junit4]   2> 	at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
   [junit4]   2> 	at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
   [junit4]   2> 	at org.apache.zookeeper.ZooKeeper.delete(ZooKeeper.java:873)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$2.execute(SolrZkClient.java:174)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$2.execute(SolrZkClient.java:171)
   [junit4]   2> 	at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:73)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient.delete(SolrZkClient.java:171)
   [junit4]   2> 	at org.apache.solr.cloud.ElectionContext.cancelElection(ElectionContext.java:67)
   [junit4]   2> 	at org.apache.solr.cloud.ShardLeaderElectionContext.runLeaderProcess(ElectionContext.java:293)
   [junit4]   2> 	at org.apache.solr.cloud.LeaderElector.runIamLeaderProcess(LeaderElector.java:164)
   [junit4]   2> 	at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:108)
   [junit4]   2> 	at org.apache.solr.cloud.LeaderElector.access$000(LeaderElector.java:55)
   [junit4]   2> 	at org.apache.solr.cloud.LeaderElector$1.process(LeaderElector.java:137)
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:519)
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:495)
   [junit4]   2> 
   [junit4]   2> 109595 T304 oascc.ZkStateReader$3.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
   [junit4]   2> NOTE: test params are: codec=Asserting, sim=RandomSimilarityProvider(queryNorm=true,coord=no): {}, locale=cs_CZ, timezone=America/Araguaina
   [junit4]   2> NOTE: Mac OS X 10.8.5 x86_64/Oracle Corporation 1.7.0_51 (64-bit)/cpus=2,threads=5,free=256729896,total=417333248
   [junit4]   2> NOTE: All tests run in this JVM: [HdfsDirectoryTest, SpellingQueryConverterTest, TestCSVResponseWriter, RequestHandlersTest, TestDefaultSearchFieldResource, CoreAdminCreateDiscoverTest, TestLRUCache, TestNonNRTOpen, SuggesterTest, TestDocumentBuilder, TestIndexSearcher, PathHierarchyTokenizerFactoryTest, SpellCheckCollatorTest, TestFieldResource, TestCharFilters, ClusterStateUpdateTest, BadCopyFieldTest, DirectUpdateHandlerOptimizeTest, ZkControllerTest, HdfsBasicDistributedZk2Test, TestAnalyzedSuggestions, OverseerRolesTest]
   [junit4] Completed in 27.85s, 1 test, 1 error <<< FAILURES!

[...truncated 1028 lines...]
BUILD FAILED
/Users/jenkins/workspace/Lucene-Solr-4.x-MacOSX/build.xml:459: The following error occurred while executing this line:
/Users/jenkins/workspace/Lucene-Solr-4.x-MacOSX/build.xml:439: The following error occurred while executing this line:
/Users/jenkins/workspace/Lucene-Solr-4.x-MacOSX/build.xml:39: The following error occurred while executing this line:
/Users/jenkins/workspace/Lucene-Solr-4.x-MacOSX/extra-targets.xml:37: The following error occurred while executing this line:
/Users/jenkins/workspace/Lucene-Solr-4.x-MacOSX/solr/build.xml:189: The following error occurred while executing this line:
/Users/jenkins/workspace/Lucene-Solr-4.x-MacOSX/solr/common-build.xml:491: The following error occurred while executing this line:
/Users/jenkins/workspace/Lucene-Solr-4.x-MacOSX/lucene/common-build.xml:1308: The following error occurred while executing this line:
/Users/jenkins/workspace/Lucene-Solr-4.x-MacOSX/lucene/common-build.xml:941: There were test failures: 358 suites, 1554 tests, 1 error, 40 ignored (6 assumptions)

Total time: 83 minutes 36 seconds
Build step 'Invoke Ant' marked build as failure
Description set: Java: 64bit/jdk1.7.0 -XX:+UseCompressedOops -XX:+UseG1GC
Archiving artifacts
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure