You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by Policeman Jenkins Server <je...@thetaphi.de> on 2014/02/01 18:09:20 UTC

[JENKINS] Lucene-Solr-trunk-MacOSX (64bit/jdk1.7.0) - Build # 1284 - Still Failing!

Build: http://jenkins.thetaphi.de/job/Lucene-Solr-trunk-MacOSX/1284/
Java: 64bit/jdk1.7.0 -XX:+UseCompressedOops -XX:+UseParallelGC

1 tests failed.
FAILED:  org.apache.solr.cloud.OverseerRolesTest.testDistribSearch

Error Message:
Captured an uncaught exception in thread: Thread[id=3473, name=Overseer-91178394928414736-127.0.0.1:50320_ie_uaf%2Ft-n_0000000016, state=RUNNABLE, group=Overseer collection creation process.]

Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=3473, name=Overseer-91178394928414736-127.0.0.1:50320_ie_uaf%2Ft-n_0000000016, state=RUNNABLE, group=Overseer collection creation process.]
	at __randomizedtesting.SeedInfo.seed([BABFEB6317D026A2:3B59657B608F469E]:0)
Caused by: org.apache.solr.common.cloud.ZooKeeperException: 
	at __randomizedtesting.SeedInfo.seed([BABFEB6317D026A2]:0)
	at org.apache.solr.cloud.OverseerCollectionProcessor.run(OverseerCollectionProcessor.java:202)
	at java.lang.Thread.run(Thread.java:744)
Caused by: org.apache.zookeeper.KeeperException$NoNodeException: KeeperErrorCode = NoNode for /overseer/collection-queue-work/qn-0000000004
	at org.apache.zookeeper.KeeperException.create(KeeperException.java:111)
	at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
	at org.apache.zookeeper.ZooKeeper.delete(ZooKeeper.java:873)
	at org.apache.solr.common.cloud.SolrZkClient$2.execute(SolrZkClient.java:173)
	at org.apache.solr.common.cloud.SolrZkClient$2.execute(SolrZkClient.java:170)
	at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:73)
	at org.apache.solr.common.cloud.SolrZkClient.delete(SolrZkClient.java:170)
	at org.apache.solr.cloud.DistributedQueue.remove(DistributedQueue.java:193)
	at org.apache.solr.cloud.OverseerCollectionProcessor.run(OverseerCollectionProcessor.java:193)
	... 1 more




Build Log:
[...truncated 10048 lines...]
   [junit4] Suite: org.apache.solr.cloud.OverseerRolesTest
   [junit4]   2> 916436 T3306 oas.BaseDistributedSearchTestCase.initHostContext Setting hostContext system property: /ie_uaf/t
   [junit4]   2> 916445 T3306 oas.SolrTestCaseJ4.setUp ###Starting testDistribSearch
   [junit4]   2> Creating dataDir: /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./solrtest-OverseerRolesTest-1391271895261
   [junit4]   2> 916447 T3306 oasc.ZkTestServer.run STARTING ZK TEST SERVER
   [junit4]   2> 916448 T3307 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
   [junit4]   2> 916549 T3306 oasc.ZkTestServer.run start zk server on port:50298
   [junit4]   2> 916551 T3306 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 916557 T3313 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@30d21291 name:ZooKeeperConnection Watcher:127.0.0.1:50298 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 916558 T3306 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 916558 T3306 oascc.SolrZkClient.makePath makePath: /solr
   [junit4]   2> 916571 T3306 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 916576 T3315 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@17398876 name:ZooKeeperConnection Watcher:127.0.0.1:50298/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 916577 T3306 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 916578 T3306 oascc.SolrZkClient.makePath makePath: /collections/collection1
   [junit4]   2> 916588 T3306 oascc.SolrZkClient.makePath makePath: /collections/collection1/shards
   [junit4]   2> 916595 T3306 oascc.SolrZkClient.makePath makePath: /collections/control_collection
   [junit4]   2> 916601 T3306 oascc.SolrZkClient.makePath makePath: /collections/control_collection/shards
   [junit4]   2> 916608 T3306 oasc.AbstractZkTestCase.putConfig put /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
   [junit4]   2> 916609 T3306 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.xml
   [junit4]   2> 916623 T3306 oasc.AbstractZkTestCase.putConfig put /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
   [junit4]   2> 916624 T3306 oascc.SolrZkClient.makePath makePath: /configs/conf1/schema.xml
   [junit4]   2> 916632 T3306 oasc.AbstractZkTestCase.putConfig put /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 916632 T3306 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 916640 T3306 oasc.AbstractZkTestCase.putConfig put /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
   [junit4]   2> 916641 T3306 oascc.SolrZkClient.makePath makePath: /configs/conf1/stopwords.txt
   [junit4]   2> 916649 T3306 oasc.AbstractZkTestCase.putConfig put /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
   [junit4]   2> 916650 T3306 oascc.SolrZkClient.makePath makePath: /configs/conf1/protwords.txt
   [junit4]   2> 916659 T3306 oasc.AbstractZkTestCase.putConfig put /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
   [junit4]   2> 916660 T3306 oascc.SolrZkClient.makePath makePath: /configs/conf1/currency.xml
   [junit4]   2> 916667 T3306 oasc.AbstractZkTestCase.putConfig put /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
   [junit4]   2> 916669 T3306 oascc.SolrZkClient.makePath makePath: /configs/conf1/enumsConfig.xml
   [junit4]   2> 916678 T3306 oasc.AbstractZkTestCase.putConfig put /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
   [junit4]   2> 916679 T3306 oascc.SolrZkClient.makePath makePath: /configs/conf1/open-exchange-rates.json
   [junit4]   2> 916688 T3306 oasc.AbstractZkTestCase.putConfig put /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 916688 T3306 oascc.SolrZkClient.makePath makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 916697 T3306 oasc.AbstractZkTestCase.putConfig put /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
   [junit4]   2> 916698 T3306 oascc.SolrZkClient.makePath makePath: /configs/conf1/old_synonyms.txt
   [junit4]   2> 916706 T3306 oasc.AbstractZkTestCase.putConfig put /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
   [junit4]   2> 916707 T3306 oascc.SolrZkClient.makePath makePath: /configs/conf1/synonyms.txt
   [junit4]   2> 917077 T3306 oejs.Server.doStart jetty-8.1.10.v20130312
   [junit4]   2> 917084 T3306 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:50301
   [junit4]   2> 917084 T3306 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
   [junit4]   2> 917085 T3306 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 917085 T3306 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ./org.apache.solr.cloud.OverseerRolesTest-controljetty-1391271895534
   [junit4]   2> 917085 T3306 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: './org.apache.solr.cloud.OverseerRolesTest-controljetty-1391271895534/'
   [junit4]   2> 917120 T3306 oasc.ConfigSolr.fromFile Loading container configuration from /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.OverseerRolesTest-controljetty-1391271895534/solr.xml
   [junit4]   2> 917200 T3306 oasc.ConfigSolrXml.<init> Config-defined core root directory: 
   [junit4]   2> 917201 T3306 oasc.CoreContainer.<init> New CoreContainer 1083546251
   [junit4]   2> 917201 T3306 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=./org.apache.solr.cloud.OverseerRolesTest-controljetty-1391271895534/]
   [junit4]   2> 917203 T3306 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
   [junit4]   2> 917203 T3306 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
   [junit4]   2> 917203 T3306 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
   [junit4]   2> 917204 T3306 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
   [junit4]   2> 917204 T3306 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
   [junit4]   2> 917205 T3306 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
   [junit4]   2> 917205 T3306 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
   [junit4]   2> 917205 T3306 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
   [junit4]   2> 917205 T3306 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
   [junit4]   2> 917215 T3306 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 917216 T3306 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 917216 T3306 oasc.CoreContainer.load Host Name: 127.0.0.1
   [junit4]   2> 917216 T3306 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:50298/solr
   [junit4]   2> 917217 T3306 oasc.ZkController.checkChrootPath zkHost includes chroot
   [junit4]   2> 917218 T3306 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 917225 T3326 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1dd53ccd name:ZooKeeperConnection Watcher:127.0.0.1:50298 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 917226 T3306 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 917230 T3306 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 917240 T3328 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@284cf666 name:ZooKeeperConnection Watcher:127.0.0.1:50298/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 917240 T3306 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 917243 T3306 oascc.SolrZkClient.makePath makePath: /overseer/queue
   [junit4]   2> 917254 T3306 oascc.SolrZkClient.makePath makePath: /overseer/collection-queue-work
   [junit4]   2> 917261 T3306 oascc.SolrZkClient.makePath makePath: /live_nodes
   [junit4]   2> 917265 T3306 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:50301_ie_uaf%2Ft
   [junit4]   2> 917266 T3306 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:50301_ie_uaf%2Ft
   [junit4]   2> 917274 T3306 oascc.SolrZkClient.makePath makePath: /overseer_elect
   [junit4]   2> 917282 T3306 oascc.SolrZkClient.makePath makePath: /overseer_elect/election
   [junit4]   2> 917292 T3306 oasc.OverseerElectionContext.runLeaderProcess I am going to be the leader 127.0.0.1:50301_ie_uaf%2Ft
   [junit4]   2> 917293 T3306 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
   [junit4]   2> 917296 T3306 oasc.Overseer.start Overseer (id=91178394928414723-127.0.0.1:50301_ie_uaf%2Ft-n_0000000000) starting
   [junit4]   2> 917302 T3306 oascc.SolrZkClient.makePath makePath: /overseer/queue-work
   [junit4]   2> 917309 T3306 oascc.SolrZkClient.makePath makePath: /clusterstate.json
   [junit4]   2> 917311 T3330 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
   [junit4]   2> 917313 T3330 oasc.OverseerCollectionProcessor.prioritizeOverseerNodes prioritizing overseer nodes
   [junit4]   2> 917315 T3306 oascc.SolrZkClient.makePath makePath: /aliases.json
   [junit4]   2> 917320 T3306 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 917328 T3306 oasc.CorePropertiesLocator.discover Looking for core definitions underneath /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.OverseerRolesTest-controljetty-1391271895534
   [junit4]   2> 917367 T3329 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
   [junit4]   2> 917441 T3306 oasc.CorePropertiesLocator.discoverUnder Found core conf in /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.OverseerRolesTest-controljetty-1391271895534/conf/
   [junit4]   2> 917446 T3306 oasc.CorePropertiesLocator.discover Found 1 core definitions
   [junit4]   2> 917447 T3306 oass.SolrDispatchFilter.init user.dir=/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0
   [junit4]   2> 917447 T3306 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
   [junit4]   2> 917449 T3306 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 917453 T3333 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@ac98b56 name:ZooKeeperConnection Watcher:127.0.0.1:50298/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 917454 T3306 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 917459 T3306 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 917466 T3306 oasc.ChaosMonkey.monkeyLog monkey: init - expire sessions:false cause connection loss:false
   [junit4]   2> 917832 T3306 oasc.AbstractFullDistribZkTestBase.createJettys create jetty 1
   [junit4]   2> 917832 T3306 oejs.Server.doStart jetty-8.1.10.v20130312
   [junit4]   2> 917837 T3306 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:50305
   [junit4]   2> 917839 T3306 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
   [junit4]   2> 917839 T3306 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 917840 T3306 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ../../../../../../../../../Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.OverseerRolesTest-jetty1-1391271896281
   [junit4]   2> 917841 T3306 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.OverseerRolesTest-jetty1-1391271896281/'
   [junit4]   2> 917871 T3306 oasc.ConfigSolr.fromFile Loading container configuration from /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/../../../../../../../../../Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.OverseerRolesTest-jetty1-1391271896281/solr.xml
   [junit4]   2> 917960 T3306 oasc.ConfigSolrXml.<init> Config-defined core root directory: 
   [junit4]   2> 917962 T3306 oasc.CoreContainer.<init> New CoreContainer 1876494494
   [junit4]   2> 917962 T3306 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=../../../../../../../../../Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.OverseerRolesTest-jetty1-1391271896281/]
   [junit4]   2> 917964 T3306 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
   [junit4]   2> 917964 T3306 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
   [junit4]   2> 917964 T3306 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
   [junit4]   2> 917964 T3306 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
   [junit4]   2> 917965 T3306 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
   [junit4]   2> 917965 T3306 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
   [junit4]   2> 917965 T3306 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
   [junit4]   2> 917965 T3306 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
   [junit4]   2> 917966 T3306 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
   [junit4]   2> 917975 T3306 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 917976 T3306 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 917976 T3306 oasc.CoreContainer.load Host Name: 127.0.0.1
   [junit4]   2> 917976 T3306 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:50298/solr
   [junit4]   2> 917977 T3306 oasc.ZkController.checkChrootPath zkHost includes chroot
   [junit4]   2> 917978 T3306 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 917982 T3344 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@11c4f670 name:ZooKeeperConnection Watcher:127.0.0.1:50298 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 917983 T3306 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 917995 T3306 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 918001 T3346 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@6c73c1c0 name:ZooKeeperConnection Watcher:127.0.0.1:50298/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 918002 T3306 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 918014 T3306 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 919027 T3306 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:50305_ie_uaf%2Ft
   [junit4]   2> 919029 T3306 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:50305_ie_uaf%2Ft
   [junit4]   2> 919036 T3346 oascc.ZkStateReader$3.process Updating live nodes... (2)
   [junit4]   2> 919037 T3328 oascc.ZkStateReader$3.process Updating live nodes... (2)
   [junit4]   2> 919037 T3333 oascc.ZkStateReader$3.process Updating live nodes... (2)
   [junit4]   2> 919051 T3306 oasc.CorePropertiesLocator.discover Looking for core definitions underneath /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/../../../../../../../../../Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.OverseerRolesTest-jetty1-1391271896281
   [junit4]   2> 919192 T3306 oasc.CorePropertiesLocator.discoverUnder Found core conf in /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/../../../../../../../../../Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.OverseerRolesTest-jetty1-1391271896281/conf/
   [junit4]   2> 919199 T3306 oasc.CorePropertiesLocator.discover Found 1 core definitions
   [junit4]   2> 919200 T3306 oass.SolrDispatchFilter.init user.dir=/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0
   [junit4]   2> 919200 T3306 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
   [junit4]   2> 919551 T3306 oasc.AbstractFullDistribZkTestBase.createJettys create jetty 2
   [junit4]   2> 919552 T3306 oejs.Server.doStart jetty-8.1.10.v20130312
   [junit4]   2> 919557 T3306 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:50308
   [junit4]   2> 919559 T3306 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
   [junit4]   2> 919559 T3306 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 919559 T3306 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ./org.apache.solr.cloud.OverseerRolesTest-jetty2-1391271898016
   [junit4]   2> 919560 T3306 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: './org.apache.solr.cloud.OverseerRolesTest-jetty2-1391271898016/'
   [junit4]   2> 919591 T3306 oasc.ConfigSolr.fromFile Loading container configuration from /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.OverseerRolesTest-jetty2-1391271898016/solr.xml
   [junit4]   2> 919666 T3306 oasc.ConfigSolrXml.<init> Config-defined core root directory: 
   [junit4]   2> 919666 T3306 oasc.CoreContainer.<init> New CoreContainer 1063807477
   [junit4]   2> 919667 T3306 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=./org.apache.solr.cloud.OverseerRolesTest-jetty2-1391271898016/]
   [junit4]   2> 919668 T3306 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
   [junit4]   2> 919668 T3306 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
   [junit4]   2> 919668 T3306 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
   [junit4]   2> 919669 T3306 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
   [junit4]   2> 919669 T3306 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
   [junit4]   2> 919669 T3306 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
   [junit4]   2> 919669 T3306 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
   [junit4]   2> 919670 T3306 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
   [junit4]   2> 919670 T3306 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
   [junit4]   2> 919680 T3306 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 919680 T3306 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 919681 T3306 oasc.CoreContainer.load Host Name: 127.0.0.1
   [junit4]   2> 919681 T3306 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:50298/solr
   [junit4]   2> 919681 T3306 oasc.ZkController.checkChrootPath zkHost includes chroot
   [junit4]   2> 919683 T3306 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 919687 T3358 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@60047458 name:ZooKeeperConnection Watcher:127.0.0.1:50298 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 919689 T3306 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 919696 T3306 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 919701 T3360 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@241bfc05 name:ZooKeeperConnection Watcher:127.0.0.1:50298/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 919702 T3306 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 919722 T3306 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 920735 T3306 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:50308_ie_uaf%2Ft
   [junit4]   2> 920737 T3306 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:50308_ie_uaf%2Ft
   [junit4]   2> 920746 T3333 oascc.ZkStateReader$3.process Updating live nodes... (3)
   [junit4]   2> 920746 T3346 oascc.ZkStateReader$3.process Updating live nodes... (3)
   [junit4]   2> 920746 T3328 oascc.ZkStateReader$3.process Updating live nodes... (3)
   [junit4]   2> 920746 T3360 oascc.ZkStateReader$3.process Updating live nodes... (3)
   [junit4]   2> 920757 T3306 oasc.CorePropertiesLocator.discover Looking for core definitions underneath /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.OverseerRolesTest-jetty2-1391271898016
   [junit4]   2> 920854 T3306 oasc.CorePropertiesLocator.discoverUnder Found core conf in /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.OverseerRolesTest-jetty2-1391271898016/conf/
   [junit4]   2> 920859 T3306 oasc.CorePropertiesLocator.discover Found 1 core definitions
   [junit4]   2> 920860 T3306 oass.SolrDispatchFilter.init user.dir=/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0
   [junit4]   2> 920860 T3306 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
   [junit4]   2> 921212 T3306 oasc.AbstractFullDistribZkTestBase.createJettys create jetty 3
   [junit4]   2> 921214 T3306 oejs.Server.doStart jetty-8.1.10.v20130312
   [junit4]   2> 921218 T3306 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:50311
   [junit4]   2> 921221 T3306 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
   [junit4]   2> 921222 T3306 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 921222 T3306 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ../../../../../../../../../Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.OverseerRolesTest-jetty3-1391271899676
   [junit4]   2> 921223 T3306 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.OverseerRolesTest-jetty3-1391271899676/'
   [junit4]   2> 921249 T3306 oasc.ConfigSolr.fromFile Loading container configuration from /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/../../../../../../../../../Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.OverseerRolesTest-jetty3-1391271899676/solr.xml
   [junit4]   2> 921316 T3306 oasc.ConfigSolrXml.<init> Config-defined core root directory: 
   [junit4]   2> 921316 T3306 oasc.CoreContainer.<init> New CoreContainer 511120476
   [junit4]   2> 921316 T3306 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=../../../../../../../../../Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.OverseerRolesTest-jetty3-1391271899676/]
   [junit4]   2> 921318 T3306 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
   [junit4]   2> 921318 T3306 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
   [junit4]   2> 921318 T3306 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
   [junit4]   2> 921319 T3306 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
   [junit4]   2> 921319 T3306 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
   [junit4]   2> 921319 T3306 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
   [junit4]   2> 921319 T3306 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
   [junit4]   2> 921320 T3306 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
   [junit4]   2> 921321 T3306 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
   [junit4]   2> 921331 T3306 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 921331 T3306 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 921332 T3306 oasc.CoreContainer.load Host Name: 127.0.0.1
   [junit4]   2> 921332 T3306 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:50298/solr
   [junit4]   2> 921332 T3306 oasc.ZkController.checkChrootPath zkHost includes chroot
   [junit4]   2> 921334 T3306 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 921338 T3372 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1bd9b3a4 name:ZooKeeperConnection Watcher:127.0.0.1:50298 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 921341 T3306 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 921348 T3306 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 921352 T3374 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@2ab24ac3 name:ZooKeeperConnection Watcher:127.0.0.1:50298/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 921352 T3306 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 921365 T3306 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 922380 T3306 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:50311_ie_uaf%2Ft
   [junit4]   2> 922382 T3306 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:50311_ie_uaf%2Ft
   [junit4]   2> 922390 T3328 oascc.ZkStateReader$3.process Updating live nodes... (4)
   [junit4]   2> 922390 T3346 oascc.ZkStateReader$3.process Updating live nodes... (4)
   [junit4]   2> 922391 T3360 oascc.ZkStateReader$3.process Updating live nodes... (4)
   [junit4]   2> 922391 T3333 oascc.ZkStateReader$3.process Updating live nodes... (4)
   [junit4]   2> 922391 T3374 oascc.ZkStateReader$3.process Updating live nodes... (4)
   [junit4]   2> 922405 T3306 oasc.CorePropertiesLocator.discover Looking for core definitions underneath /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/../../../../../../../../../Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.OverseerRolesTest-jetty3-1391271899676
   [junit4]   2> 922562 T3306 oasc.CorePropertiesLocator.discoverUnder Found core conf in /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/../../../../../../../../../Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.OverseerRolesTest-jetty3-1391271899676/conf/
   [junit4]   2> 922573 T3306 oasc.CorePropertiesLocator.discover Found 1 core definitions
   [junit4]   2> 922575 T3306 oass.SolrDispatchFilter.init user.dir=/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0
   [junit4]   2> 922575 T3306 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
   [junit4]   2> 922931 T3306 oasc.AbstractFullDistribZkTestBase.createJettys create jetty 4
   [junit4]   2> 922931 T3306 oejs.Server.doStart jetty-8.1.10.v20130312
   [junit4]   2> 922937 T3306 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:50314
   [junit4]   2> 922939 T3306 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
   [junit4]   2> 922939 T3306 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 922939 T3306 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ./org.apache.solr.cloud.OverseerRolesTest-jetty4-1391271901391
   [junit4]   2> 922940 T3306 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: './org.apache.solr.cloud.OverseerRolesTest-jetty4-1391271901391/'
   [junit4]   2> 922969 T3306 oasc.ConfigSolr.fromFile Loading container configuration from /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.OverseerRolesTest-jetty4-1391271901391/solr.xml
   [junit4]   2> 923039 T3306 oasc.ConfigSolrXml.<init> Config-defined core root directory: 
   [junit4]   2> 923040 T3306 oasc.CoreContainer.<init> New CoreContainer 1603034028
   [junit4]   2> 923041 T3306 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=./org.apache.solr.cloud.OverseerRolesTest-jetty4-1391271901391/]
   [junit4]   2> 923042 T3306 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
   [junit4]   2> 923042 T3306 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
   [junit4]   2> 923042 T3306 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
   [junit4]   2> 923043 T3306 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
   [junit4]   2> 923043 T3306 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
   [junit4]   2> 923043 T3306 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
   [junit4]   2> 923044 T3306 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
   [junit4]   2> 923044 T3306 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
   [junit4]   2> 923045 T3306 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
   [junit4]   2> 923055 T3306 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 923055 T3306 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 923056 T3306 oasc.CoreContainer.load Host Name: 127.0.0.1
   [junit4]   2> 923056 T3306 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:50298/solr
   [junit4]   2> 923057 T3306 oasc.ZkController.checkChrootPath zkHost includes chroot
   [junit4]   2> 923058 T3306 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 923063 T3386 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@77d6aa35 name:ZooKeeperConnection Watcher:127.0.0.1:50298 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 923064 T3306 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 923075 T3306 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 923083 T3388 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@76252f01 name:ZooKeeperConnection Watcher:127.0.0.1:50298/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 923083 T3306 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 923090 T3306 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 924098 T3306 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:50314_ie_uaf%2Ft
   [junit4]   2> 924104 T3306 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:50314_ie_uaf%2Ft
   [junit4]   2> 924119 T3346 oascc.ZkStateReader$3.process Updating live nodes... (5)
   [junit4]   2> 924129 T3328 oascc.ZkStateReader$3.process Updating live nodes... (5)
   [junit4]   2> 924129 T3360 oascc.ZkStateReader$3.process Updating live nodes... (5)
   [junit4]   2> 924134 T3374 oascc.ZkStateReader$3.process Updating live nodes... (5)
   [junit4]   2> 924134 T3333 oascc.ZkStateReader$3.process Updating live nodes... (5)
   [junit4]   2> 924135 T3388 oascc.ZkStateReader$3.process Updating live nodes... (5)
   [junit4]   2> 924158 T3306 oasc.CorePropertiesLocator.discover Looking for core definitions underneath /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.OverseerRolesTest-jetty4-1391271901391
   [junit4]   2> 924277 T3306 oasc.CorePropertiesLocator.discoverUnder Found core conf in /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.OverseerRolesTest-jetty4-1391271901391/conf/
   [junit4]   2> 924284 T3306 oasc.CorePropertiesLocator.discover Found 1 core definitions
   [junit4]   2> 924286 T3306 oass.SolrDispatchFilter.init user.dir=/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0
   [junit4]   2> 924287 T3306 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
   [junit4]   2> 924699 T3306 oasc.AbstractFullDistribZkTestBase.createJettys create jetty 5
   [junit4]   2> 924700 T3306 oejs.Server.doStart jetty-8.1.10.v20130312
   [junit4]   2> 924706 T3306 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:50317
   [junit4]   2> 924707 T3306 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
   [junit4]   2> 924707 T3306 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 924707 T3306 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ../../../../../../../../../Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.OverseerRolesTest-jetty5-1391271903103
   [junit4]   2> 924708 T3306 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.OverseerRolesTest-jetty5-1391271903103/'
   [junit4]   2> 924746 T3306 oasc.ConfigSolr.fromFile Loading container configuration from /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/../../../../../../../../../Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.OverseerRolesTest-jetty5-1391271903103/solr.xml
   [junit4]   2> 924830 T3306 oasc.ConfigSolrXml.<init> Config-defined core root directory: 
   [junit4]   2> 924831 T3306 oasc.CoreContainer.<init> New CoreContainer 1580173652
   [junit4]   2> 924832 T3306 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=../../../../../../../../../Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.OverseerRolesTest-jetty5-1391271903103/]
   [junit4]   2> 924834 T3306 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
   [junit4]   2> 924834 T3306 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
   [junit4]   2> 924834 T3306 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
   [junit4]   2> 924835 T3306 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
   [junit4]   2> 924835 T3306 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
   [junit4]   2> 924835 T3306 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
   [junit4]   2> 924836 T3306 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
   [junit4]   2> 924836 T3306 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
   [junit4]   2> 924837 T3306 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
   [junit4]   2> 924849 T3306 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 924849 T3306 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 924850 T3306 oasc.CoreContainer.load Host Name: 127.0.0.1
   [junit4]   2> 924850 T3306 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:50298/solr
   [junit4]   2> 924850 T3306 oasc.ZkController.checkChrootPath zkHost includes chroot
   [junit4]   2> 924852 T3306 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 924856 T3400 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@2e11a1b name:ZooKeeperConnection Watcher:127.0.0.1:50298 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 924857 T3306 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 924863 T3306 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 924867 T3402 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@2b1a3ce6 name:ZooKeeperConnection Watcher:127.0.0.1:50298/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 924868 T3306 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 924878 T3306 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 925889 T3306 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:50317_ie_uaf%2Ft
   [junit4]   2> 925895 T3306 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:50317_ie_uaf%2Ft
   [junit4]   2> 925915 T3374 oascc.ZkStateReader$3.process Updating live nodes... (6)
   [junit4]   2> 925916 T3388 oascc.ZkStateReader$3.process Updating live nodes... (6)
   [junit4]   2> 925917 T3333 oascc.ZkStateReader$3.process Updating live nodes... (6)
   [junit4]   2> 925917 T3360 oascc.ZkStateReader$3.process Updating live nodes... (6)
   [junit4]   2> 925917 T3402 oascc.ZkStateReader$3.process Updating live nodes... (6)
   [junit4]   2> 925919 T3346 oascc.ZkStateReader$3.process Updating live nodes... (6)
   [junit4]   2> 925919 T3328 oascc.ZkStateReader$3.process Updating live nodes... (6)
   [junit4]   2> 925943 T3306 oasc.CorePropertiesLocator.discover Looking for core definitions underneath /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/../../../../../../../../../Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.OverseerRolesTest-jetty5-1391271903103
   [junit4]   2> 926109 T3306 oasc.CorePropertiesLocator.discoverUnder Found core conf in /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/../../../../../../../../../Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.OverseerRolesTest-jetty5-1391271903103/conf/
   [junit4]   2> 926117 T3306 oasc.CorePropertiesLocator.discover Found 1 core definitions
   [junit4]   2> 926118 T3306 oass.SolrDispatchFilter.init user.dir=/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0
   [junit4]   2> 926119 T3306 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
   [junit4]   2> 926462 T3306 oasc.AbstractFullDistribZkTestBase.createJettys create jetty 6
   [junit4]   2> 926463 T3306 oejs.Server.doStart jetty-8.1.10.v20130312
   [junit4]   2> 926467 T3306 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:50320
   [junit4]   2> 926469 T3306 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
   [junit4]   2> 926469 T3306 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 926470 T3306 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ./org.apache.solr.cloud.OverseerRolesTest-jetty6-1391271904934
   [junit4]   2> 926470 T3306 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: './org.apache.solr.cloud.OverseerRolesTest-jetty6-1391271904934/'
   [junit4]   2> 926506 T3306 oasc.ConfigSolr.fromFile Loading container configuration from /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.OverseerRolesTest-jetty6-1391271904934/solr.xml
   [junit4]   2> 926586 T3306 oasc.ConfigSolrXml.<init> Config-defined core root directory: 
   [junit4]   2> 926586 T3306 oasc.CoreContainer.<init> New CoreContainer 226246337
   [junit4]   2> 926587 T3306 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=./org.apache.solr.cloud.OverseerRolesTest-jetty6-1391271904934/]
   [junit4]   2> 926588 T3306 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
   [junit4]   2> 926589 T3306 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
   [junit4]   2> 926590 T3306 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
   [junit4]   2> 926590 T3306 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
   [junit4]   2> 926590 T3306 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
   [junit4]   2> 926591 T3306 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
   [junit4]   2> 926591 T3306 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
   [junit4]   2> 926592 T3306 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
   [junit4]   2> 926592 T3306 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
   [junit4]   2> 926602 T3306 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 926603 T3306 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 926603 T3306 oasc.CoreContainer.load Host Name: 127.0.0.1
   [junit4]   2> 926604 T3306 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:50298/solr
   [junit4]   2> 926604 T3306 oasc.ZkController.checkChrootPath zkHost includes chroot
   [junit4]   2> 926606 T3306 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 926610 T3414 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@57fbf7d1 name:ZooKeeperConnection Watcher:127.0.0.1:50298 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 926611 T3306 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 926618 T3306 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 926623 T3416 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@21c9cc0c name:ZooKeeperConnection Watcher:127.0.0.1:50298/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 926624 T3306 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 926634 T3306 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 927647 T3306 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:50320_ie_uaf%2Ft
   [junit4]   2> 927653 T3306 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:50320_ie_uaf%2Ft
   [junit4]   2> 927669 T3333 oascc.ZkStateReader$3.process Updating live nodes... (7)
   [junit4]   2> 927670 T3374 oascc.ZkStateReader$3.process Updating live nodes... (7)
   [junit4]   2> 927670 T3328 oascc.ZkStateReader$3.process Updating live nodes... (7)
   [junit4]   2> 927672 T3360 oascc.ZkStateReader$3.process Updating live nodes... (7)
   [junit4]   2> 927675 T3402 oascc.ZkStateReader$3.process Updating live nodes... (7)
   [junit4]   2> 927676 T3416 oascc.ZkStateReader$3.process Updating live nodes... (7)
   [junit4]   2> 927676 T3346 oascc.ZkStateReader$3.process Updating live nodes... (7)
   [junit4]   2> 927676 T3388 oascc.ZkStateReader$3.process Updating live nodes... (7)
   [junit4]   2> 927691 T3306 oasc.CorePropertiesLocator.discover Looking for core definitions underneath /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.OverseerRolesTest-jetty6-1391271904934
   [junit4]   2> 927785 T3306 oasc.CorePropertiesLocator.discoverUnder Found core conf in /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.OverseerRolesTest-jetty6-1391271904934/conf/
   [junit4]   2> 927792 T3306 oasc.CorePropertiesLocator.discover Found 1 core definitions
   [junit4]   2> 927793 T3306 oass.SolrDispatchFilter.init user.dir=/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0
   [junit4]   2> 927793 T3306 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
   [junit4]   2> 927796 T3306 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 927801 T3419 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@2e926d63 name:ZooKeeperConnection Watcher:127.0.0.1:50298/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 927801 T3306 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 927804 T3306 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 927813 T3306 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 927816 T3421 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@3420158 name:ZooKeeperConnection Watcher:127.0.0.1:50298/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 927817 T3306 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 927820 T3306 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 927829 T3337 oasha.CollectionsHandler.handleCreateAction Creating Collection : version=2&name=testOverseerCol&numShards=4&replicationFactor=2&action=CREATE&maxShardsPerNode=2&wt=javabin
   [junit4]   2> 927832 T3328 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/collection-queue-work state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 927836 T3330 oasc.OverseerCollectionProcessor.run Overseer Collection Processor: Get the message id:/overseer/collection-queue-work/qn-0000000000 message:{
   [junit4]   2> 	  "operation":"createcollection",
   [junit4]   2> 	  "fromApi":"true",
   [junit4]   2> 	  "name":"testOverseerCol",
   [junit4]   2> 	  "replicationFactor":"2",
   [junit4]   2> 	  "numShards":"4",
   [junit4]   2> 	  "maxShardsPerNode":"2"}
   [junit4]   2> 927836 T3330 oasc.OverseerCollectionProcessor.processMessage WARN OverseerCollectionProcessor.processMessage : createcollection , {
   [junit4]   2> 	  "operation":"createcollection",
   [junit4]   2> 	  "fromApi":"true",
   [junit4]   2> 	  "name":"testOverseerCol",
   [junit4]   2> 	  "replicationFactor":"2",
   [junit4]   2> 	  "numShards":"4",
   [junit4]   2> 	  "maxShardsPerNode":"2"}
   [junit4]   2> 927838 T3330 oasc.OverseerCollectionProcessor.createConfNode Only one config set found in zk - using it:conf1
   [junit4]   2> 927838 T3330 oasc.OverseerCollectionProcessor.createConfNode creating collections conf node /collections/testOverseerCol 
   [junit4]   2> 927838 T3330 oascc.SolrZkClient.makePath makePath: /collections/testOverseerCol
   [junit4]   2> 927853 T3328 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 927858 T3329 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 927860 T3329 oasc.Overseer$ClusterStateUpdater.buildCollection building a new collection: testOverseerCol
   [junit4]   2> 927860 T3329 oasc.Overseer$ClusterStateUpdater.createCollection Create collection testOverseerCol with shards [shard1, shard2, shard3, shard4]
   [junit4]   2> 927872 T3328 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 927876 T3374 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 7)
   [junit4]   2> 927877 T3333 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 7)
   [junit4]   2> 927877 T3402 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 7)
   [junit4]   2> 927877 T3360 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 7)
   [junit4]   2> 927877 T3328 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 7)
   [junit4]   2> 927877 T3388 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 7)
   [junit4]   2> 927877 T3419 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 7)
   [junit4]   2> 927877 T3346 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 7)
   [junit4]   2> 927877 T3416 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 7)
   [junit4]   2> 927877 T3421 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 7)
   [junit4]   2> 927954 T3330 oasc.OverseerCollectionProcessor.createCollection going to create cores replicas shardNames [shard1, shard2, shard3, shard4] , repFactor : 2
   [junit4]   2> 927955 T3330 oasc.OverseerCollectionProcessor.createCollection Creating shard testOverseerCol_shard1_replica1 as part of slice shard1 of collection testOverseerCol on 127.0.0.1:50314_ie_uaf%2Ft
   [junit4]   2> 927956 T3330 oasc.OverseerCollectionProcessor.createCollection Creating shard testOverseerCol_shard1_replica2 as part of slice shard1 of collection testOverseerCol on 127.0.0.1:50317_ie_uaf%2Ft
   [junit4]   2> 927959 T3330 oasc.OverseerCollectionProcessor.createCollection Creating shard testOverseerCol_shard2_replica1 as part of slice shard2 of collection testOverseerCol on 127.0.0.1:50308_ie_uaf%2Ft
   [junit4]   2> 927960 T3330 oasc.OverseerCollectionProcessor.createCollection Creating shard testOverseerCol_shard2_replica2 as part of slice shard2 of collection testOverseerCol on 127.0.0.1:50311_ie_uaf%2Ft
   [junit4]   2> 927961 T3330 oasc.OverseerCollectionProcessor.createCollection Creating shard testOverseerCol_shard3_replica1 as part of slice shard3 of collection testOverseerCol on 127.0.0.1:50320_ie_uaf%2Ft
   [junit4]   2> 927966 T3330 oasc.OverseerCollectionProcessor.createCollection Creating shard testOverseerCol_shard3_replica2 as part of slice shard3 of collection testOverseerCol on 127.0.0.1:50301_ie_uaf%2Ft
   [junit4]   2> 927970 T3330 oasc.OverseerCollectionProcessor.createCollection Creating shard testOverseerCol_shard4_replica1 as part of slice shard4 of collection testOverseerCol on 127.0.0.1:50305_ie_uaf%2Ft
   [junit4]   2> 927980 T3384 oasc.ZkController.publish publishing core=testOverseerCol_shard1_replica1 state=down
   [junit4]   2> 927982 T3330 oasc.OverseerCollectionProcessor.createCollection Creating shard testOverseerCol_shard4_replica2 as part of slice shard4 of collection testOverseerCol on 127.0.0.1:50314_ie_uaf%2Ft
   [junit4]   2> 927986 T3395 oasc.ZkController.publish publishing core=testOverseerCol_shard1_replica2 state=down
   [junit4]   2> 927984 T3410 oasc.ZkController.publish publishing core=testOverseerCol_shard3_replica1 state=down
   [junit4]   2> 927986 T3338 oasc.ZkController.publish publishing core=testOverseerCol_shard4_replica1 state=down
   [junit4]   2> 927987 T3369 oasc.ZkController.publish publishing core=testOverseerCol_shard2_replica2 state=down
   [junit4]   2> 927987 T3353 oasc.ZkController.publish publishing core=testOverseerCol_shard2_replica1 state=down
   [junit4]   2> 927991 T3384 oasc.ZkController.waitForCoreNodeName look for our core node name
   [junit4]   2> 927992 T3322 oasc.ZkController.publish publishing core=testOverseerCol_shard3_replica2 state=down
   [junit4]   2> 927992 T3410 oasc.ZkController.waitForCoreNodeName look for our core node name
   [junit4]   2> 927992 T3353 oasc.ZkController.waitForCoreNodeName look for our core node name
   [junit4]   2> 927993 T3328 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 927995 T3395 oasc.ZkController.waitForCoreNodeName look for our core node name
   [junit4]   2> 927995 T3338 oasc.ZkController.waitForCoreNodeName look for our core node name
   [junit4]   2> 927996 T3379 oasc.ZkController.publish publishing core=testOverseerCol_shard4_replica2 state=down
   [junit4]   2> 927996 T3369 oasc.ZkController.waitForCoreNodeName look for our core node name
   [junit4]   2> 927998 T3322 oasc.ZkController.waitForCoreNodeName look for our core node name
   [junit4]   2> 927999 T3328 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 927999 T3379 oasc.ZkController.waitForCoreNodeName look for our core node name
   [junit4]   2> 928002 T3329 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 928003 T3329 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=4 message={
   [junit4]   2> 	  "operation":"state",
   [junit4]   2> 	  "state":"down",
   [junit4]   2> 	  "base_url":"http://127.0.0.1:50314/ie_uaf/t",
   [junit4]   2> 	  "core":"testOverseerCol_shard1_replica1",
   [junit4]   2> 	  "roles":null,
   [junit4]   2> 	  "node_name":"127.0.0.1:50314_ie_uaf%2Ft",
   [junit4]   2> 	  "shard":"shard1",
   [junit4]   2> 	  "shard_range":null,
   [junit4]   2> 	  "shard_state":"active",
   [junit4]   2> 	  "shard_parent":null,
   [junit4]   2> 	  "collection":"testOverseerCol",
   [junit4]   2> 	  "numShards":"4",
   [junit4]   2> 	  "core_node_name":null}
   [junit4]   2> 928013 T3329 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=4 message={
   [junit4]   2> 	  "operation":"state",
   [junit4]   2> 	  "state":"down",
   [junit4]   2> 	  "base_url":"http://127.0.0.1:50308/ie_uaf/t",
   [junit4]   2> 	  "core":"testOverseerCol_shard2_replica1",
   [junit4]   2> 	  "roles":null,
   [junit4]   2> 	  "node_name":"127.0.0.1:50308_ie_uaf%2Ft",
   [junit4]   2> 	  "shard":"shard2",
   [junit4]   2> 	  "shard_range":null,
   [junit4]   2> 	  "shard_state":"active",
   [junit4]   2> 	  "shard_parent":null,
   [junit4]   2> 	  "collection":"testOverseerCol",
   [junit4]   2> 	  "numShards":"4",
   [junit4]   2> 	  "core_node_name":null}
   [junit4]   2> 928020 T3328 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 928024 T3329 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=4 message={
   [junit4]   2> 	  "operation":"state",
   [junit4]   2> 	  "state":"down",
   [junit4]   2> 	  "base_url":"http://127.0.0.1:50320/ie_uaf/t",
   [junit4]   2> 	  "core":"testOverseerCol_shard3_replica1",
   [junit4]   2> 	  "roles":null,
   [junit4]   2> 	  "node_name":"127.0.0.1:50320_ie_uaf%2Ft",
   [junit4]   2> 	  "shard":"shard3",
   [junit4]   2> 	  "shard_range":null,
   [junit4]   2> 	  "shard_state":"active",
   [junit4]   2> 	  "shard_parent":null,
   [junit4]   2> 	  "collection":"testOverseerCol",
   [junit4]   2> 	  "numShards":"4",
   [junit4]   2> 	  "core_node_name":null}
   [junit4]   2> 928033 T3328 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 928036 T3329 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=4 message={
   [junit4]   2> 	  "operation":"state",
   [junit4]   2> 	  "state":"down",
   [junit4]   2> 	  "base_url":"http://127.0.0.1:50317/ie_uaf/t",
   [junit4]   2> 	  "core":"testOverseerCol_shard1_replica2",
   [junit4]   2> 	  "roles":null,
   [junit4]   2> 	  "node_name":"127.0.0.1:50317_ie_uaf%2Ft",
   [junit4]   2> 	  "shard":"shard1",
   [junit4]   2> 	  "shard_range":null,
   [junit4]   2> 	  "shard_state":"active",
   [junit4]   2> 	  "shard_parent":null,
   [junit4]   2> 	  "collection":"testOverseerCol",
   [junit4]   2> 	  "numShards":"4",
   [junit4]   2> 	  "core_node_name":null}
   [junit4]   2> 928044 T3328 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 928048 T3329 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=4 message={
   [junit4]   2> 	  "operation":"state",
   [junit4]   2> 	  "state":"down",
   [junit4]   2> 	  "base_url":"http://127.0.0.1:50305/ie_uaf/t",
   [junit4]   2> 	  "core":"testOverseerCol_shard4_replica1",
   [junit4]   2> 	  "roles":null,
   [junit4]   2> 	  "node_name":"127.0.0.1:50305_ie_uaf%2Ft",
   [junit4]   2> 	  "shard":"shard4",
   [junit4]   2> 	  "shard_range":null,
   [junit4]   2> 	  "shard_state":"active",
   [junit4]   2> 	  "shard_parent":null,
   [junit4]   2> 	  "collection":"testOverseerCol",
   [junit4]   2> 	  "numShards":"4",
   [junit4]   2> 	  "core_node_name":null}
   [junit4]   2> 928055 T3328 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 928058 T3329 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=4 message={
   [junit4]   2> 	  "operation":"state",
   [junit4]   2> 	  "state":"down",
   [junit4]   2> 	  "base_url":"http://127.0.0.1:50311/ie_uaf/t",
   [junit4]   2> 	  "core":"testOverseerCol_shard2_replica2",
   [junit4]   2> 	  "roles":null,
   [junit4]   2> 	  "node_name":"127.0.0.1:50311_ie_uaf%2Ft",
   [junit4]   2> 	  "shard":"shard2",
   [junit4]   2> 	  "shard_range":null,
   [junit4]   2> 	  "shard_state":"active",
   [junit4]   2> 	  "shard_parent":null,
   [junit4]   2> 	  "collection":"testOverseerCol",
   [junit4]   2> 	  "numShards":"4",
   [junit4]   2> 	  "core_node_name":null}
   [junit4]   2> 928065 T3328 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 928068 T3329 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=4 message={
   [junit4]   2> 	  "operation":"state",
   [junit4]   2> 	  "state":"down",
   [junit4]   2> 	  "base_url":"http://127.0.0.1:50301/ie_uaf/t",
   [junit4]   2> 	  "core":"testOverseerCol_shard3_replica2",
   [junit4]   2> 	  "roles":null,
   [junit4]   2> 	  "node_name":"127.0.0.1:50301_ie_uaf%2Ft",
   [junit4]   2> 	  "shard":"shard3",
   [junit4]   2> 	  "shard_range":null,
   [junit4]   2> 	  "shard_state":"active",
   [junit4]   2> 	  "shard_parent":null,
   [junit4]   2> 	  "collection":"testOverseerCol",
   [junit4]   2> 	  "numShards":"4",
   [junit4]   2> 	  "core_node_name":null}
   [junit4]   2> 928078 T3328 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 928081 T3329 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=4 message={
   [junit4]   2> 	  "operation":"state",
   [junit4]   2> 	  "state":"down",
   [junit4]   2> 	  "base_url":"http://127.0.0.1:50314/ie_uaf/t",
   [junit4]   2> 	  "core":"testOverseerCol_shard4_replica2",
   [junit4]   2> 	  "roles":null,
   [junit4]   2> 	  "node_name":"127.0.0.1:50314_ie_uaf%2Ft",
   [junit4]   2> 	  "shard":"shard4",
   [junit4]   2> 	  "shard_range":null,
   [junit4]   2> 	  "shard_state":"active",
   [junit4]   2> 	  "shard_parent":null,
   [junit4]   2> 	  "collection":"testOverseerCol",
   [junit4]   2> 	  "numShards":"4",
   [junit4]   2> 	  "core_node_name":null}
   [junit4]   2> 928090 T3328 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 928202 T3360 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 7)
   [junit4]   2> 928203 T3388 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 7)
   [junit4]   2> 928204 T3421 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 7)
   [junit4]   2> 928204 T3419 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 7)
   [junit4]   2> 928204 T3402 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 7)
   [junit4]   2> 928206 T3346 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 7)
   [junit4]   2> 928204 T3328 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 7)
   [junit4]   2> 928205 T3416 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 7)
   [junit4]   2> 928205 T3374 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 7)
   [junit4]   2> 928204 T3333 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 7)
   [junit4]   2> 928992 T3384 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for testOverseerCol_shard1_replica1
   [junit4]   2> 928994 T3410 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for testOverseerCol_shard3_replica1
   [junit4]   2> 928997 T3395 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for testOverseerCol_shard1_replica2
   [junit4]   2> 928998 T3410 oasc.CoreContainer.create Creating SolrCore 'testOverseerCol_shard3_replica1' using instanceDir: ./org.apache.solr.cloud.OverseerRolesTest-jetty6-1391271904934/testOverseerCol_shard3_replica1
   [junit4]   2> 929000 T3410 oasc.ZkController.createCollectionZkNode Check for collection zkNode:testOverseerCol
   [junit4]   2> 929000 T3353 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for testOverseerCol_shard2_replica1
   [junit4]   2> 929003 T3369 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for testOverseerCol_shard2_replica2
   [junit4]   2> 929004 T3353 oasc.CoreContainer.create Creating SolrCore 'testOverseerCol_shard2_replica1' using instanceDir: ./org.apache.solr.cloud.OverseerRolesTest-jetty2-1391271898016/testOverseerCol_shard2_replica1
   [junit4]   2> 929004 T3353 oasc.ZkController.createCollectionZkNode Check for collection zkNode:testOverseerCol
   [junit4]   2> 929005 T3338 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for testOverseerCol_shard4_replica1
   [junit4]   2> 929007 T3369 oasc.CoreContainer.create Creating SolrCore 'testOverseerCol_shard2_replica2' using instanceDir: ../../../../../../../../../Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.OverseerRolesTest-jetty3-1391271899676/testOverseerCol_shard2_replica2
   [junit4]   2> 929008 T3369 oasc.ZkController.createCollectionZkNode Check for collection zkNode:testOverseerCol
   [junit4]   2> 929009 T3395 oasc.CoreContainer.create Creating SolrCore 'testOverseerCol_shard1_replica2' using instanceDir: ../../../../../../../../../Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.OverseerRolesTest-jetty5-1391271903103/testOverseerCol_shard1_replica2
   [junit4]   2> 929010 T3395 oasc.ZkController.createCollectionZkNode Check for collection zkNode:testOverseerCol
   [junit4]   2> 929011 T3338 oasc.CoreContainer.create Creating SolrCore 'testOverseerCol_shard4_replica1' using instanceDir: ../../../../../../../../../Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.OverseerRolesTest-jetty1-1391271896281/testOverseerCol_shard4_replica1
   [junit4]   2> 929012 T3338 oasc.ZkController.createCollectionZkNode Check for collection zkNode:testOverseerCol
   [junit4]   2> 929010 T3379 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for testOverseerCol_shard4_replica2
   [junit4]   2> 929010 T3322 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for testOverseerCol_shard3_replica2
   [junit4]   2> 929016 T3379 oasc.CoreContainer.create Creating SolrCore 'testOverseerCol_shard4_replica2' using instanceDir: ./org.apache.solr.cloud.OverseerRolesTest-jetty4-1391271901391/testOverseerCol_shard4_replica2
   [junit4]   2> 929016 T3379 oasc.ZkController.createCollectionZkNode Check for collection zkNode:testOverseerCol
   [junit4]   2> 929011 T3384 oasc.CoreContainer.create Creating SolrCore 'testOverseerCol_shard1_replica1' using instanceDir: ./org.apache.solr.cloud.OverseerRolesTest-jetty4-1391271901391/testOverseerCol_shard1_replica1
   [junit4]   2> 929020 T3384 oasc.ZkController.createCollectionZkNode Check for collection zkNode:testOverseerCol
   [junit4]   2> 929021 T3410 oasc.ZkController.createCollectionZkNode Collection zkNode exists
   [junit4]   2> 929022 T3410 oascc.ZkStateReader.readConfigName Load collection config from:/collections/testOverseerCol
   [junit4]   2> 929020 T3322 oasc.CoreContainer.create Creating SolrCore 'testOverseerCol_shard3_replica2' using instanceDir: ./org.apache.solr.cloud.OverseerRolesTest-controljetty-1391271895534/testOverseerCol_shard3_replica2
   [junit4]   2> 929023 T3322 oasc.ZkController.createCollectionZkNode Check for collection zkNode:testOverseerCol
   [junit4]   2> 929023 T3379 oasc.ZkController.createCollectionZkNode Collection zkNode exists
   [junit4]   2> 929024 T3379 oascc.ZkStateReader.readConfigName Load collection config from:/collections/testOverseerCol
   [junit4]   2> 929023 T3395 oasc.ZkController.createCollectionZkNode Collection zkNode exists
   [junit4]   2> 929024 T3395 oascc.ZkStateReader.readConfigName Load collection config from:/collections/testOverseerCol
   [junit4]   2> 929023 T3338 oasc.ZkController.createCollectionZkNode Collection zkNode exists
   [junit4]   2> 929025 T3338 oascc.ZkStateReader.readConfigName Load collection config from:/collections/testOverseerCol
   [junit4]   2> 929024 T3384 oasc.ZkController.createCollectionZkNode Collection zkNode exists
   [junit4]   2> 929026 T3384 oascc.ZkStateReader.readConfigName Load collection config from:/collections/testOverseerCol
   [junit4]   2> 929024 T3353 oasc.ZkController.createCollectionZkNode Collection zkNode exists
   [junit4]   2> 929026 T3353 oascc.ZkStateReader.readConfigName Load collection config from:/collections/testOverseerCol
   [junit4]   2> 929024 T3369 oasc.ZkController.createCollectionZkNode Collection zkNode exists
   [junit4]   2> 929027 T3369 oascc.ZkStateReader.readConfigName Load collection config from:/collections/testOverseerCol
   [junit4]   2> 929026 T3322 oasc.ZkController.createCollectionZkNode Collection zkNode exists
   [junit4]   2> 929028 T3322 oascc.ZkStateReader.readConfigName Load collection config from:/collections/testOverseerCol
   [junit4]   2> 929030 T3379 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: './org.apache.solr.cloud.OverseerRolesTest-jetty4-1391271901391/testOverseerCol_shard4_replica2/'
   [junit4]   2> 929031 T3395 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.OverseerRolesTest-jetty5-1391271903103/testOverseerCol_shard1_replica2/'
   [junit4]   2> 929031 T3410 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: './org.apache.solr.cloud.OverseerRolesTest-jetty6-1391271904934/testOverseerCol_shard3_replica1/'
   [junit4]   2> 929034 T3369 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.OverseerRolesTest-jetty3-1391271899676/testOverseerCol_shard2_replica2/'
   [junit4]   2> 929036 T3338 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.OverseerRolesTest-jetty1-1391271896281/testOverseerCol_shard4_replica1/'
   [junit4]   2> 929036 T3322 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: './org.apache.solr.cloud.OverseerRolesTest-controljetty-1391271895534/testOverseerCol_shard3_replica2/'
   [junit4]   2> 929037 T3384 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: './org.apache.solr.cloud.OverseerRolesTest-jetty4-1391271901391/testOverseerCol_shard1_replica1/'
   [junit4]   2> 929045 T3353 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: './org.apache.solr.cloud.OverseerRolesTest-jetty2-1391271898016/testOverseerCol_shard2_replica1/'
   [junit4]   2> 929265 T3395 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
   [junit4]   2> 929309 T3379 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
   [junit4]   2> 929309 T3322 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
   [junit4]   2> 929321 T3384 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
   [junit4]   2> 929335 T3369 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
   [junit4]   2> 929355 T3353 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
   [junit4]   2> 929368 T3410 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
   [junit4]   2> 929431 T3338 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
   [junit4]   2> 929503 T3395 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 929506 T3369 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 929512 T3395 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
   [junit4]   2> 929545 T3395 oass.IndexSchema.readSchema [testOverseerCol_shard1_replica2] Schema name=test
   [junit4]   2> 929578 T3384 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 929585 T3384 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
   [junit4]   2> 929590 T3369 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
   [junit4]   2> 929643 T3379 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 929648 T3322 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 929653 T3379 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
   [junit4]   2> 929670 T3322 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
   [junit4]   2> 929729 T3369 oass.IndexSchema.readSchema [testOverseerCol_shard2_replica2] Schema name=test
   [junit4]   2> 929743 T3384 oass.IndexSchema.readSchema [testOverseerCol_shard1_replica1] Schema name=test
   [junit4]   2> 929762 T3379 oass.IndexSchema.readSchema [testOverseerCol_shard4_replica2] Schema name=test
   [junit4]   2> 929765 T3322 oass.IndexSchema.readSchema [testOverseerCol_shard3_replica2] Schema name=test
   [junit4]   2> 929799 T3353 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 929820 T3353 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
   [junit4]   2> 929838 T3353 oass.IndexSchema.readSchema [testOverseerCol_shard2_replica1] Schema name=test
   [junit4]   2> 929941 T3410 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 929944 T3410 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
   [junit4]   2> 929992 T3410 oass.IndexSchema.readSchema [testOverseerCol_shard3_replica1] Schema name=test
   [junit4]   2> 930006 T3338 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 930016 T3338 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
   [junit4]   2> 930032 T3338 oass.IndexSchema.readSchema [testOverseerCol_shard4_replica1] Schema name=test
   [junit4]   2> 932309 T3338 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 932326 T3338 oass.IndexSchema.readSchema default search field in schema is text
   [junit4]   2> 932330 T3338 oass.IndexSchema.readSchema unique key field: id
   [junit4]   2> 932478 T3338 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 932493 T3338 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 932501 T3384 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 932507 T3353 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 932525 T3353 oass.IndexSchema.readSchema default search field in schema is text
   [junit4]   2> 932553 T3384 oass.IndexSchema.readSchema default search field in schema is text
   [junit4]   2> 932555 T3353 oass.IndexSchema.readSchema unique key field: id
   [junit4]   2> 932558 T3384 oass.IndexSchema.readSchema unique key field: id
   [junit4]   2> 932578 T3338 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 932580 T3322 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 932593 T3338 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
   [junit4]   2> 932593 T3338 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
   [junit4]   2> 932594 T3338 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 932601 T3338 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
   [junit4]   2> 932601 T3338 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
   [junit4]   2> 932602 T3338 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 932602 T3338 oasc.SolrCore.<init> [testOverseerCol_shard4_replica1] Opening new SolrCore at ../../../../../../../../../Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.OverseerRolesTest-jetty1-1391271896281/testOverseerCol_shard4_replica1/, dataDir=../../../../../../../../../Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.OverseerRolesTest-jetty1-1391271896281/testOverseerCol_shard4_replica1/data/
   [junit4]   2> 932602 T3338 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@2c8633e1
   [junit4]   2> 932610 T3322 oass.IndexSchema.readSchema default search field in schema is text
   [junit4]   2> 932612 T3338 oasc.CachingDirectoryFactory.get return new directory for ../../../../../../../../../Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.OverseerRolesTest-jetty1-1391271896281/testOverseerCol_shard4_replica1/data
   [junit4]   2> 932622 T3395 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 932623 T3338 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=../../../../../../../../../Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.OverseerRolesTest-jetty1-1391271896281/testOverseerCol_shard4_replica1/data/index/
   [junit4]   2> 932624 T3338 oasc.SolrCore.initIndex WARN [testOverseerCol_shard4_replica1] Solr index directory '../../../../../../../../../Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.OverseerRolesTest-jetty1-1391271896281/testOverseerCol_shard4_replica1/data/index' doesn't exist. Creating new index...
   [junit4]   2> 932635 T3322 oass.IndexSchema.readSchema unique key field: id
   [junit4]   2> 932636 T3395 oass.IndexSchema.readSchema default search field in schema is text
   [junit4]   2> 932639 T3338 oasc.CachingDirectoryFactory.get return new directory for ../../../../../../../../../Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.OverseerRolesTest-jetty1-1391271896281/testOverseerCol_shard4_replica1/data/index
   [junit4]   2> 932641 T3338 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=32, maxMergeAtOnceExplicit=50, maxMergedSegmentMB=54.408203125, floorSegmentMB=1.333984375, forceMergeDeletesPctAllowed=22.827411111393204, segmentsPerTier=37.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.43762038431474526
   [junit4]   2> 932643 T3395 oass.IndexSchema.readSchema unique key field: id
   [junit4]   2> 932671 T3379 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 932689 T3369 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 932713 T3338 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 		commit{dir=MockDirectoryWrapper(org.apache.lucene.store.RAMDirectory@777c6d5 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@30fe711f),segFN=segments_1,generation=1}
   [junit4]   2> 932713 T3338 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 932725 T3384 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 932726 T3338 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "nodistrib"
   [junit4]   2> 932731 T3369 oass.IndexSchema.readSchema default search field in schema is text
   [junit4]   2> 932730 T3410 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 932728 T3379 oass.IndexSchema.readSchema default search field in schema is text
   [junit4]   2> 932736 T3379 oass.IndexSchema.readSchema unique key field: id
   [junit4]   2> 932738 T3410 oass.IndexSchema.readSchema default search field in schema is text
   [junit4]   2> 932731 T3338 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe"
   [junit4]   2> 932742 T3338 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 932742 T3338 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "stored_sig"
   [junit4]   2> 932743 T3338 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 932743 T3338 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
   [junit4]   2> 932743 T3338 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 932744 T3338 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 932744 T3338 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 932745 T3338 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
   [junit4]   2> 932745 T3338 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
   [junit4]   2> 932745 T3338 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
   [junit4]   2> 932747 T3338 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
   [junit4]   2> 932751 T3338 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
   [junit4]   2> 932751 T3338 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
   [junit4]   2> 932752 T3338 oasc.RequestHandlers.initHandlersFromConfig created /admin/file: solr.admin.ShowFileRequestHandler
   [junit4]   2> 932752 T3338 oasc.RequestHandlers.initHandlersFromConfig created /admin/fileedit: solr.admin.EditFileRequestHandler
   [junit4]   2> 932755 T3353 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 932767 T3338 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 932779 T3384 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 932783 T3353 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 932791 T3369 oass.IndexSchema.readSchema unique key field: id
   [junit4]   2> 932801 T3338 oasu.CommitTracker.<init> Hard AutoCommit: disabled
   [junit4]   2> 932802 T3338 oasu.CommitTracker.<init> Soft AutoCommit: disabled
   [junit4]   2> 932803 T3384 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 932818 T3410 oass.IndexSchema.readSchema unique key field: id
   [junit4]   2> 932824 T3384 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
   [junit4]   2> 932825 T3384 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
   [junit4]   2> 932825 T3384 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 932831 T3353 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 932839 T3384 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
   [junit4]   2> 932839 T3384 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
   [junit4]   2> 932839 T3384 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 932840 T3384 oasc.SolrCore.<init> [testOverseerCol_shard1_replica1] Opening new SolrCore at ./org.apache.solr.cloud.OverseerRolesTest-jetty4-1391271901391/testOverseerCol_shard1_replica1/, dataDir=./org.apache.solr.cloud.OverseerRolesTest-jetty4-1391271901391/testOverseerCol_shard1_replica1/data/
   [junit4]   2> 932841 T3384 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@2c8633e1
   [junit4]   2> 932831 T3338 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=18, maxMergeAtOnceExplicit=17, maxMergedSegmentMB=30.4091796875, floorSegmentMB=1.

[...truncated too long message...]

st/J0/./org.apache.solr.cloud.OverseerRolesTest-jetty5-1391271903103/testOverseerCol_shard1_replica2/data/index
   [junit4]   2> 941833 T3402 oasc.CachingDirectoryFactory.closeCacheValue looking to close ../../../../../../../../../Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.OverseerRolesTest-jetty5-1391271903103/testOverseerCol_shard1_replica2/data [CachedDir<<refCount=0;path=../../../../../../../../../Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.OverseerRolesTest-jetty5-1391271903103/testOverseerCol_shard1_replica2/data;done=false>>]
   [junit4]   2> 941833 T3402 oasc.CachingDirectoryFactory.close Closing directory: ../../../../../../../../../Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.OverseerRolesTest-jetty5-1391271903103/testOverseerCol_shard1_replica2/data
   [junit4]   2> 941834 T3402 oascc.SolrZkClient.makePath makePath: /collections/testOverseerCol/leaders/shard1
   [junit4]   2> 941834 T3402 oasc.SolrException.log ERROR There was a problem trying to register as the leader:org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /collections
   [junit4]   2> 		at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
   [junit4]   2> 		at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
   [junit4]   2> 		at org.apache.zookeeper.ZooKeeper.exists(ZooKeeper.java:1041)
   [junit4]   2> 		at org.apache.solr.common.cloud.SolrZkClient$3.execute(SolrZkClient.java:205)
   [junit4]   2> 		at org.apache.solr.common.cloud.SolrZkClient$3.execute(SolrZkClient.java:202)
   [junit4]   2> 		at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:73)
   [junit4]   2> 		at org.apache.solr.common.cloud.SolrZkClient.exists(SolrZkClient.java:202)
   [junit4]   2> 		at org.apache.solr.common.cloud.SolrZkClient.makePath(SolrZkClient.java:413)
   [junit4]   2> 		at org.apache.solr.common.cloud.SolrZkClient.makePath(SolrZkClient.java:382)
   [junit4]   2> 		at org.apache.solr.common.cloud.SolrZkClient.makePath(SolrZkClient.java:369)
   [junit4]   2> 		at org.apache.solr.cloud.ShardLeaderElectionContextBase.runLeaderProcess(ElectionContext.java:112)
   [junit4]   2> 		at org.apache.solr.cloud.ShardLeaderElectionContext.runLeaderProcess(ElectionContext.java:273)
   [junit4]   2> 		at org.apache.solr.cloud.LeaderElector.runIamLeaderProcess(LeaderElector.java:164)
   [junit4]   2> 		at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:108)
   [junit4]   2> 		at org.apache.solr.cloud.LeaderElector.access$000(LeaderElector.java:55)
   [junit4]   2> 		at org.apache.solr.cloud.LeaderElector$1.process(LeaderElector.java:137)
   [junit4]   2> 		at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:519)
   [junit4]   2> 		at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:495)
   [junit4]   2> 	
   [junit4]   2> 941836 T3402 oasc.LeaderElector$1.process WARN  org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /collections/testOverseerCol/leader_elect/shard1/election/91178394928414734-core_node4-n_0000000001
   [junit4]   2> 	at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
   [junit4]   2> 	at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
   [junit4]   2> 	at org.apache.zookeeper.ZooKeeper.delete(ZooKeeper.java:873)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$2.execute(SolrZkClient.java:173)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$2.execute(SolrZkClient.java:170)
   [junit4]   2> 	at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:73)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient.delete(SolrZkClient.java:170)
   [junit4]   2> 	at org.apache.solr.cloud.ElectionContext.cancelElection(ElectionContext.java:67)
   [junit4]   2> 	at org.apache.solr.cloud.ShardLeaderElectionContext.runLeaderProcess(ElectionContext.java:293)
   [junit4]   2> 	at org.apache.solr.cloud.LeaderElector.runIamLeaderProcess(LeaderElector.java:164)
   [junit4]   2> 	at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:108)
   [junit4]   2> 	at org.apache.solr.cloud.LeaderElector.access$000(LeaderElector.java:55)
   [junit4]   2> 	at org.apache.solr.cloud.LeaderElector$1.process(LeaderElector.java:137)
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:519)
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:495)
   [junit4]   2> 
   [junit4]   2> 941836 T3402 oasc.LeaderElector$1.process WARN  org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /overseer_elect/election
   [junit4]   2> 	at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
   [junit4]   2> 	at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
   [junit4]   2> 	at org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:1468)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:256)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:253)
   [junit4]   2> 	at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:73)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient.getChildren(SolrZkClient.java:253)
   [junit4]   2> 	at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:92)
   [junit4]   2> 	at org.apache.solr.cloud.LeaderElector.access$000(LeaderElector.java:55)
   [junit4]   2> 	at org.apache.solr.cloud.LeaderElector$1.process(LeaderElector.java:137)
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:519)
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:495)
   [junit4]   2> 
   [junit4]   2> 941837 T3402 oascc.ZkStateReader$3.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
   [junit4]   2> NOTE: test params are: codec=Asserting, sim=DefaultSimilarity, locale=en_MT, timezone=Australia/Queensland
   [junit4]   2> NOTE: Mac OS X 10.8.5 x86_64/Oracle Corporation 1.7.0_51 (64-bit)/cpus=2,threads=2,free=150443448,total=339738624
   [junit4]   2> NOTE: All tests run in this JVM: [WordBreakSolrSpellCheckerTest, TestSchemaVersionResource, TestMultiCoreConfBootstrap, SuggesterWFSTTest, TestPHPSerializedResponseWriter, ConvertedLegacyTest, TestIndexSearcher, TestBM25SimilarityFactory, TestCSVResponseWriter, TestAnalyzeInfixSuggestions, TestCharFilters, PreAnalyzedFieldTest, DistributedTermsComponentTest, DeleteReplicaTest, TestRecoveryHdfs, SynonymTokenizerTest, TestRTGBase, HdfsRecoveryZkTest, IndexBasedSpellCheckerTest, TestExtendedDismaxParser, TestStressReorder, SolrCmdDistributorTest, TestPhraseSuggestions, TestDFRSimilarityFactory, TestRandomDVFaceting, TestUtils, TestIBSimilarityFactory, TestLRUCache, QueryFacetTest, StressHdfsTest, NotRequiredUniqueKeyTest, StandardRequestHandlerTest, NumericFieldsTest, ChaosMonkeyNothingIsSafeTest, SliceStateTest, TestRecovery, TestNonNRTOpen, RegexBoostProcessorTest, SuggesterTest, AnalysisAfterCoreReloadTest, SliceStateUpdateTest, QueryElevationComponentTest, TestSearchPerf, TestReload, AssignTest, TestCopyFieldCollectionResource, TestClassNameShortening, ExternalFileFieldSortTest, TestFastOutputStream, DocValuesTest, TestDynamicFieldResource, HdfsBasicDistributedZkTest, FunctionTest, TestSurroundQueryParser, TestUniqueKeyFieldResource, UpdateParamsTest, TestSystemIdResolver, TestQuerySenderListener, TestCSVLoader, TestCodecSupport, TestSweetSpotSimilarityFactory, TestSolrJ, TestValueSourceCache, ShardSplitTest, SuggestComponentTest, CursorPagingTest, TestFiltering, FieldFacetExtrasTest, TestSolrXmlPersistor, TriLevelCompositeIdRoutingTest, OutputWriterTest, SolrRequestParserTest, RAMDirectoryFactoryTest, StatelessScriptUpdateProcessorFactoryTest, SolrIndexSplitterTest, SoftAutoCommitTest, TestSimpleQParserPlugin, ChaosMonkeySafeLeaderTest, LukeRequestHandlerTest, RangeFacetTest, RequestHandlersTest, AddSchemaFieldsUpdateProcessorFactoryTest, ZkSolrClientTest, TestBinaryResponseWriter, TestWriterPerf, TestSolrXMLSerializer, TestConfig, TestRandomMergePolicy, TestJmxMonitoredMap, UpdateRequestProcessorFactoryTest, BJQParserTest, OverseerRolesTest]
   [junit4] Completed in 26.27s, 1 test, 1 error <<< FAILURES!

[...truncated 847 lines...]
BUILD FAILED
/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/build.xml:453: The following error occurred while executing this line:
/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/build.xml:433: The following error occurred while executing this line:
/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/build.xml:39: The following error occurred while executing this line:
/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/extra-targets.xml:37: The following error occurred while executing this line:
/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build.xml:189: The following error occurred while executing this line:
/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/common-build.xml:491: The following error occurred while executing this line:
/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/lucene/common-build.xml:1268: The following error occurred while executing this line:
/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/lucene/common-build.xml:901: There were test failures: 365 suites, 1590 tests, 1 error, 40 ignored (6 assumptions)

Total time: 89 minutes 15 seconds
Build step 'Invoke Ant' marked build as failure
Description set: Java: 64bit/jdk1.7.0 -XX:+UseCompressedOops -XX:+UseParallelGC
Archiving artifacts
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure



[JENKINS] Lucene-Solr-trunk-MacOSX (64bit/jdk1.7.0) - Build # 1285 - Still Failing!

Posted by Policeman Jenkins Server <je...@thetaphi.de>.
Build: http://jenkins.thetaphi.de/job/Lucene-Solr-trunk-MacOSX/1285/
Java: 64bit/jdk1.7.0 -XX:-UseCompressedOops -XX:+UseConcMarkSweepGC

All tests passed

Build Log:
[...truncated 10350 lines...]
   [junit4] JVM J0: stderr was not empty, see: /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/temp/junit4-J0-20140201_232742_828.syserr
   [junit4] >>> JVM J0: stderr (verbatim) ----
   [junit4] java(211,0x145004000) malloc: *** error for object 0x10000144ff3080: pointer being freed was not allocated
   [junit4] *** set a breakpoint in malloc_error_break to debug
   [junit4] <<< JVM J0: EOF ----

[...truncated 1 lines...]
   [junit4] ERROR: JVM J0 ended with an exception, command line: /Library/Java/JavaVirtualMachines/jdk1.7.0_51.jdk/Contents/Home/jre/bin/java -XX:-UseCompressedOops -XX:+UseConcMarkSweepGC -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/heapdumps -Dtests.prefix=tests -Dtests.seed=237A3ACAC48BA2B -Xmx512M -Dtests.iters= -Dtests.verbose=false -Dtests.infostream=false -Dtests.codec=random -Dtests.postingsformat=random -Dtests.docvaluesformat=random -Dtests.locale=random -Dtests.timezone=random -Dtests.directory=random -Dtests.linedocsfile=europarl.lines.txt.gz -Dtests.luceneMatchVersion=5.0 -Dtests.cleanthreads=perClass -Djava.util.logging.config.file=/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/lucene/tools/junit4/logging.properties -Dtests.nightly=false -Dtests.weekly=false -Dtests.slow=true -Dtests.asserts.gracious=false -Dtests.multiplier=1 -DtempDir=. -Djava.io.tmpdir=. -Djunit4.tempDir=/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/temp -Dclover.db.dir=/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/lucene/build/clover/db -Djava.security.manager=org.apache.lucene.util.TestSecurityManager -Djava.security.policy=/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/lucene/tools/junit4/tests.policy -Dlucene.version=5.0-SNAPSHOT -Djetty.testMode=1 -Djetty.insecurerandom=1 -Dsolr.directoryFactory=org.apache.solr.core.MockDirectoryFactory -Djava.awt.headless=true -Djdk.map.althashing.threshold=0 -Dtests.disableHdfs=true -Dfile.encoding=ISO-8859-1 -classpath /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/classes/test:/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-test-framework/classes/java:/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/test-framework/lib/junit4-ant-2.0.13.jar:/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test-files:/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/lucene/build/test-framework/classes/java:/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/lucene/build/codecs/classes/java:/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-solrj/classes/java:/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/classes/java:/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/lucene/build/analysis/common/lucene-analyzers-common-5.0-SNAPSHOT.jar:/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/lucene/build/analysis/kuromoji/lucene-analyzers-kuromoji-5.0-SNAPSHOT.jar:/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/lucene/build/analysis/phonetic/lucene-analyzers-phonetic-5.0-SNAPSHOT.jar:/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/lucene/build/codecs/lucene-codecs-5.0-SNAPSHOT.jar:/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/lucene/build/highlighter/lucene-highlighter-5.0-SNAPSHOT.jar:/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/lucene/build/memory/lucene-memory-5.0-SNAPSHOT.jar:/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/lucene/build/misc/lucene-misc-5.0-SNAPSHOT.jar:/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/lucene/build/spatial/lucene-spatial-5.0-SNAPSHOT.jar:/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/lucene/build/expressions/lucene-expressions-5.0-SNAPSHOT.jar:/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/lucene/build/suggest/lucene-suggest-5.0-SNAPSHOT.jar:/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/lucene/build/grouping/lucene-grouping-5.0-SNAPSHOT.jar:/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/lucene/build/queries/lucene-queries-5.0-SNAPSHOT.jar:/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/lucene/build/queryparser/lucene-queryparser-5.0-SNAPSHOT.jar:/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/lucene/build/join/lucene-join-5.0-SNAPSHOT.jar:/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/core/lib/antlr-runtime-3.5.jar:/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/core/lib/asm-4.1.jar:/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/core/lib/asm-commons-4.1.jar:/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/core/lib/commons-cli-1.2.jar:/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/core/lib/commons-codec-1.7.jar:/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/core/lib/commons-configuration-1.6.jar:/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/core/lib/commons-fileupload-1.2.1.jar:/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/core/lib/commons-lang-2.6.jar:/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/core/lib/concurrentlinkedhashmap-lru-1.2.jar:/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/core/lib/dom4j-1.6.1.jar:/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/core/lib/guava-14.0.1.jar:/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/core/lib/hadoop-annotations-2.2.0.jar:/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/core/lib/hadoop-auth-2.2.0.jar:/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/core/lib/hadoop-common-2.2.0.jar:/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/core/lib/hadoop-hdfs-2.2.0.jar:/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/core/lib/hppc-0.5.2.jar:/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/core/lib/joda-time-2.2.jar:/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/core/lib/org.restlet-2.1.1.jar:/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/core/lib/org.restlet.ext.servlet-2.1.1.jar:/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/core/lib/protobuf-java-2.5.0.jar:/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/core/lib/spatial4j-0.4.jar:/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/solrj/lib/commons-io-2.1.jar:/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/solrj/lib/httpclient-4.3.1.jar:/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/solrj/lib/httpcore-4.3.jar:/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/solrj/lib/httpmime-4.3.1.jar:/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/solrj/lib/jcl-over-slf4j-1.6.6.jar:/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/solrj/lib/jul-to-slf4j-1.6.6.jar:/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/solrj/lib/log4j-1.2.16.jar:/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/solrj/lib/noggit-0.5.jar:/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/solrj/lib/slf4j-api-1.6.6.jar:/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/solrj/lib/slf4j-log4j12-1.6.6.jar:/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/solrj/lib/wstx-asl-3.2.7.jar:/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/solrj/lib/zookeeper-3.4.5.jar:/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/example/lib/jetty-continuation-8.1.10.v20130312.jar:/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/example/lib/jetty-deploy-8.1.10.v20130312.jar:/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/example/lib/jetty-http-8.1.10.v20130312.jar:/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/example/lib/jetty-io-8.1.10.v20130312.jar:/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/example/lib/jetty-jmx-8.1.10.v20130312.jar:/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/example/lib/jetty-security-8.1.10.v20130312.jar:/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/example/lib/jetty-server-8.1.10.v20130312.jar:/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/example/lib/jetty-servlet-8.1.10.v20130312.jar:/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/example/lib/jetty-util-8.1.10.v20130312.jar:/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/example/lib/jetty-webapp-8.1.10.v20130312.jar:/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/example/lib/jetty-xml-8.1.10.v20130312.jar:/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/example/lib/servlet-api-3.0.jar:/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/example/example-DIH/solr/db/lib/derby-10.9.1.0.jar:/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/example/example-DIH/solr/db/lib/hsqldb-1.8.0.10.jar:/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/lucene/build/core/classes/java:/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/lucene/test-framework/lib/junit-4.10.jar:/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/lucene/test-framework/lib/randomizedtesting-runner-2.0.13.jar:/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/core/test-lib/antlr-runtime-3.5.jar:/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/core/test-lib/asm-4.1.jar:/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/core/test-lib/asm-commons-4.1.jar:/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/core/test-lib/cglib-nodep-2.2.jar:/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/core/test-lib/commons-collections-3.2.1.jar:/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/core/test-lib/dom4j-1.6.1.jar:/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/core/test-lib/easymock-3.0.jar:/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/core/test-lib/hadoop-common-2.2.0-tests.jar:/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/core/test-lib/hadoop-hdfs-2.2.0-tests.jar:/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/core/test-lib/hppc-0.5.2.jar:/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/core/test-lib/javax.servlet-api-3.0.1.jar:/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/core/test-lib/jersey-core-1.8.jar:/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/core/test-lib/jetty-6.1.26.jar:/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/core/test-lib/jetty-util-6.1.26.jar:/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/core/test-lib/objenesis-1.2.jar:/Users/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-launcher.jar:/Users/jenkins/.ant/lib/ivy-2.3.0.jar:/Users/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-antlr.jar:/Users/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-apache-bcel.jar:/Users/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-apache-bsf.jar:/Users/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-apache-log4j.jar:/Users/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-apache-oro.jar:/Users/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-apache-regexp.jar:/Users/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-apache-resolver.jar:/Users/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-apache-xalan2.jar:/Users/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-commons-logging.jar:/Users/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-commons-net.jar:/Users/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-jai.jar:/Users/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-javamail.jar:/Users/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-jdepend.jar:/Users/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-jmf.jar:/Users/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-jsch.jar:/Users/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-junit.jar:/Users/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-junit4.jar:/Users/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-netrexx.jar:/Users/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-swing.jar:/Users/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-testutil.jar:/Users/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant.jar:/Library/Java/JavaVirtualMachines/jdk1.7.0_51.jdk/Contents/Home/lib/tools.jar:/Users/jenkins/.ivy2/cache/com.carrotsearch.randomizedtesting/junit4-ant/jars/junit4-ant-2.0.13.jar -ea:org.apache.lucene... -ea:org.apache.solr... com.carrotsearch.ant.tasks.junit4.slave.SlaveMainSafe -flush -eventsfile /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/temp/junit4-J0-20140201_232742_828.events @/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/temp/junit4-J0-20140201_232742_828.suites
   [junit4] ERROR: JVM J0 ended with an exception: Forked process returned with error code: 134. Very likely a JVM crash.  Process output piped in logs above.
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.JUnit4.executeSlave(JUnit4.java:1373)
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.JUnit4.access$000(JUnit4.java:119)
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.JUnit4$2.call(JUnit4.java:911)
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.JUnit4$2.call(JUnit4.java:908)
   [junit4] 	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
   [junit4] 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
   [junit4] 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
   [junit4] 	at java.lang.Thread.run(Thread.java:744)

BUILD FAILED
/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/build.xml:453: The following error occurred while executing this line:
/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/build.xml:433: The following error occurred while executing this line:
/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/build.xml:39: The following error occurred while executing this line:
/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/extra-targets.xml:37: The following error occurred while executing this line:
/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build.xml:189: The following error occurred while executing this line:
/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/common-build.xml:491: The following error occurred while executing this line:
/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/lucene/common-build.xml:1268: The following error occurred while executing this line:
/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/lucene/common-build.xml:901: At least one slave process threw an exception, first: Forked process returned with error code: 134. Very likely a JVM crash.  Process output piped in logs above.

Total time: 68 minutes 50 seconds
Build step 'Invoke Ant' marked build as failure
Description set: Java: 64bit/jdk1.7.0 -XX:-UseCompressedOops -XX:+UseConcMarkSweepGC
Archiving artifacts
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure