You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by Policeman Jenkins Server <je...@thetaphi.de> on 2014/09/18 10:38:25 UTC

[JENKINS] Lucene-Solr-trunk-Linux (32bit/jdk1.8.0_20) - Build # 11279 - Failure!

Build: http://jenkins.thetaphi.de/job/Lucene-Solr-trunk-Linux/11279/
Java: 32bit/jdk1.8.0_20 -server -XX:+UseSerialGC

1 tests failed.
REGRESSION:  org.apache.solr.cloud.DeleteReplicaTest.testDistribSearch

Error Message:
java.lang.ClassCastException: org.apache.solr.common.params.CollectionParams$CollectionAction cannot be cast to java.lang.String

Stack Trace:
org.apache.solr.client.solrj.SolrServerException: java.lang.ClassCastException: org.apache.solr.common.params.CollectionParams$CollectionAction cannot be cast to java.lang.String
	at __randomizedtesting.SeedInfo.seed([2301E08B7645B5A6:A2E76E93011AD59A]:0)
	at org.apache.solr.client.solrj.impl.LBHttpSolrServer.doRequest(LBHttpSolrServer.java:381)
	at org.apache.solr.client.solrj.impl.LBHttpSolrServer.request(LBHttpSolrServer.java:304)
	at org.apache.solr.client.solrj.impl.CloudSolrServer.sendRequest(CloudSolrServer.java:874)
	at org.apache.solr.client.solrj.impl.CloudSolrServer.requestWithRetryOnStaleState(CloudSolrServer.java:658)
	at org.apache.solr.client.solrj.impl.CloudSolrServer.request(CloudSolrServer.java:601)
	at org.apache.solr.cloud.DeleteReplicaTest.tryToRemoveOnlyIfDown(DeleteReplicaTest.java:154)
	at org.apache.solr.cloud.DeleteReplicaTest.deleteLiveReplicaTest(DeleteReplicaTest.java:125)
	at org.apache.solr.cloud.DeleteReplicaTest.doTest(DeleteReplicaTest.java:88)
	at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:869)
	at sun.reflect.GeneratedMethodAccessor39.invoke(Unknown Source)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:483)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1618)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:827)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:877)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:836)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:738)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:772)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:783)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
	at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.ClassCastException: org.apache.solr.common.params.CollectionParams$CollectionAction cannot be cast to java.lang.String
	at org.apache.solr.common.params.MapSolrParams.getParams(MapSolrParams.java:43)
	at org.apache.solr.common.params.ModifiableSolrParams.add(ModifiableSolrParams.java:117)
	at org.apache.solr.common.params.ModifiableSolrParams.<init>(ModifiableSolrParams.java:56)
	at org.apache.solr.client.solrj.impl.HttpSolrServer.createMethod(HttpSolrServer.java:273)
	at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:210)
	at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:206)
	at org.apache.solr.client.solrj.impl.LBHttpSolrServer.doRequest(LBHttpSolrServer.java:343)
	... 47 more




Build Log:
[...truncated 11959 lines...]
   [junit4] Suite: org.apache.solr.cloud.DeleteReplicaTest
   [junit4]   2> Creating dataDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.DeleteReplicaTest-2301E08B7645B5A6-001/init-core-data-001
   [junit4]   2> 2134356 T6340 oas.SolrTestCaseJ4.buildSSLConfig Randomized ssl (true) and clientAuth (false)
   [junit4]   2> 2134356 T6340 oas.BaseDistributedSearchTestCase.initHostContext Setting hostContext system property: /b_y/tb
   [junit4]   2> 2134358 T6340 oas.SolrTestCaseJ4.setUp ###Starting testDistribSearch
   [junit4]   2> 2134359 T6340 oasc.ZkTestServer.run STARTING ZK TEST SERVER
   [junit4]   1> client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 2134359 T6341 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
   [junit4]   2> 2134459 T6340 oasc.ZkTestServer.run start zk server on port:46175
   [junit4]   2> 2134460 T6340 oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default ZkCredentialsProvider
   [junit4]   2> 2134461 T6340 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 2134462 T6347 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1c461fb name:ZooKeeperConnection Watcher:127.0.0.1:46175 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 2134463 T6340 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 2134463 T6340 oascc.SolrZkClient.createZkACLProvider Using default ZkACLProvider
   [junit4]   2> 2134463 T6340 oascc.SolrZkClient.makePath makePath: /solr
   [junit4]   2> 2134465 T6340 oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default ZkCredentialsProvider
   [junit4]   2> 2134466 T6340 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 2134467 T6349 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@185797f name:ZooKeeperConnection Watcher:127.0.0.1:46175/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 2134467 T6340 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 2134468 T6340 oascc.SolrZkClient.createZkACLProvider Using default ZkACLProvider
   [junit4]   2> 2134468 T6340 oascc.SolrZkClient.makePath makePath: /collections/collection1
   [junit4]   2> 2134469 T6340 oascc.SolrZkClient.makePath makePath: /collections/collection1/shards
   [junit4]   2> 2134470 T6340 oascc.SolrZkClient.makePath makePath: /collections/control_collection
   [junit4]   2> 2134471 T6340 oascc.SolrZkClient.makePath makePath: /collections/control_collection/shards
   [junit4]   2> 2134473 T6340 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
   [junit4]   2> 2134473 T6340 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.xml
   [junit4]   2> 2134475 T6340 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
   [junit4]   2> 2134475 T6340 oascc.SolrZkClient.makePath makePath: /configs/conf1/schema.xml
   [junit4]   2> 2134477 T6340 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 2134477 T6340 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 2134478 T6340 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
   [junit4]   2> 2134478 T6340 oascc.SolrZkClient.makePath makePath: /configs/conf1/stopwords.txt
   [junit4]   2> 2134480 T6340 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
   [junit4]   2> 2134480 T6340 oascc.SolrZkClient.makePath makePath: /configs/conf1/protwords.txt
   [junit4]   2> 2134481 T6340 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
   [junit4]   2> 2134481 T6340 oascc.SolrZkClient.makePath makePath: /configs/conf1/currency.xml
   [junit4]   2> 2134483 T6340 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
   [junit4]   2> 2134483 T6340 oascc.SolrZkClient.makePath makePath: /configs/conf1/enumsConfig.xml
   [junit4]   2> 2134484 T6340 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
   [junit4]   2> 2134485 T6340 oascc.SolrZkClient.makePath makePath: /configs/conf1/open-exchange-rates.json
   [junit4]   2> 2134486 T6340 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 2134486 T6340 oascc.SolrZkClient.makePath makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 2134487 T6340 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
   [junit4]   2> 2134488 T6340 oascc.SolrZkClient.makePath makePath: /configs/conf1/old_synonyms.txt
   [junit4]   2> 2134489 T6340 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
   [junit4]   2> 2134489 T6340 oascc.SolrZkClient.makePath makePath: /configs/conf1/synonyms.txt
   [junit4]   2> 2134491 T6340 oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default ZkCredentialsProvider
   [junit4]   2> 2134492 T6340 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 2134492 T6351 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@172cf96 name:ZooKeeperConnection Watcher:127.0.0.1:46175/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 2134492 T6340 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 2134493 T6340 oascc.SolrZkClient.createZkACLProvider Using default ZkACLProvider
   [junit4]   2> 2134753 T6340 oejs.Server.doStart jetty-8.1.10.v20130312
   [junit4]   2> 2134756 T6340 oejus.SslContextFactory.doStart Enabled Protocols [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2] of [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2]
   [junit4]   2> 2134760 T6340 oejs.AbstractConnector.doStart Started SslSelectChannelConnector@127.0.0.1:45141
   [junit4]   2> 2134762 T6340 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
   [junit4]   2> 2134762 T6340 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 2134762 T6340 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.DeleteReplicaTest-2301E08B7645B5A6-001/tempDir-002
   [junit4]   2> 2134763 T6340 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.DeleteReplicaTest-2301E08B7645B5A6-001/tempDir-002/'
   [junit4]   2> 2134774 T6340 oasc.ConfigSolr.fromFile Loading container configuration from /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.DeleteReplicaTest-2301E08B7645B5A6-001/tempDir-002/solr.xml
   [junit4]   2> 2134779 T6340 oasc.CorePropertiesLocator.<init> Config-defined core root directory: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.DeleteReplicaTest-2301E08B7645B5A6-001/tempDir-002
   [junit4]   2> 2134780 T6340 oasc.CoreContainer.<init> New CoreContainer 5474051
   [junit4]   2> 2134780 T6340 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.DeleteReplicaTest-2301E08B7645B5A6-001/tempDir-002/]
   [junit4]   2> 2134781 T6340 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
   [junit4]   2> 2134781 T6340 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: 
   [junit4]   2> 2134782 T6340 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
   [junit4]   2> 2134782 T6340 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
   [junit4]   2> 2134782 T6340 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
   [junit4]   2> 2134783 T6340 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
   [junit4]   2> 2134783 T6340 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
   [junit4]   2> 2134783 T6340 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
   [junit4]   2> 2134783 T6340 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
   [junit4]   2> 2134785 T6340 oasu.UpdateShardHandler.<init> Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=false
   [junit4]   2> 2134785 T6340 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 2134786 T6340 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 2134786 T6340 oasc.CoreContainer.load Host Name: 127.0.0.1
   [junit4]   2> 2134786 T6340 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:46175/solr
   [junit4]   2> 2134787 T6340 oasc.ZkController.checkChrootPath zkHost includes chroot
   [junit4]   2> 2134787 T6340 oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default ZkCredentialsProvider
   [junit4]   2> 2134788 T6340 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 2134788 T6362 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@14162d0 name:ZooKeeperConnection Watcher:127.0.0.1:46175 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 2134789 T6340 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 2134789 T6340 oascc.SolrZkClient.createZkACLProvider Using default ZkACLProvider
   [junit4]   2> 2134790 T6340 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 2134791 T6364 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@14aef6a name:ZooKeeperConnection Watcher:127.0.0.1:46175/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 2134791 T6340 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 2134792 T6340 oascc.SolrZkClient.makePath makePath: /overseer/queue
   [junit4]   2> 2134794 T6340 oascc.SolrZkClient.makePath makePath: /overseer/collection-queue-work
   [junit4]   2> 2134795 T6340 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-running
   [junit4]   2> 2134796 T6340 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-completed
   [junit4]   2> 2134798 T6340 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-failure
   [junit4]   2> 2134799 T6340 oascc.SolrZkClient.makePath makePath: /live_nodes
   [junit4]   2> 2134800 T6340 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:45141_b_y%2Ftb
   [junit4]   2> 2134801 T6340 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:45141_b_y%2Ftb
   [junit4]   2> 2134802 T6340 oascc.SolrZkClient.makePath makePath: /overseer_elect
   [junit4]   2> 2134803 T6340 oascc.SolrZkClient.makePath makePath: /overseer_elect/election
   [junit4]   2> 2134804 T6340 oasc.Overseer.close Overseer (id=null) closing
   [junit4]   2> 2134805 T6340 oasc.OverseerElectionContext.runLeaderProcess I am going to be the leader 127.0.0.1:45141_b_y%2Ftb
   [junit4]   2> 2134806 T6340 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
   [junit4]   2> 2134807 T6340 oasc.Overseer.start Overseer (id=92473192818212868-127.0.0.1:45141_b_y%2Ftb-n_0000000000) starting
   [junit4]   2> 2134808 T6340 oascc.SolrZkClient.makePath makePath: /overseer/queue-work
   [junit4]   2> 2134812 T6340 oasc.OverseerAutoReplicaFailoverThread.<init> Starting OverseerAutoReplicaFailoverThread autoReplicaFailoverWorkLoopDelay=10000 autoReplicaFailoverWaitAfterExpiration=10000 autoReplicaFailoverBadNodeExpiration=60000
   [junit4]   2> 2134812 T6366 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
   [junit4]   2> 2134813 T6365 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
   [junit4]   2> 2134814 T6340 oascc.SolrZkClient.makePath makePath: /clusterstate.json
   [junit4]   2> 2134815 T6340 oascc.SolrZkClient.makePath makePath: /aliases.json
   [junit4]   2> 2134820 T6340 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 2134823 T6340 oasc.CorePropertiesLocator.discover Looking for core definitions underneath /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.DeleteReplicaTest-2301E08B7645B5A6-001/tempDir-002
   [junit4]   2> 2134829 T6340 oasc.CorePropertiesLocator.discoverUnder Found core conf in /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.DeleteReplicaTest-2301E08B7645B5A6-001/tempDir-002/conf/
   [junit4]   2> 2134959 T6340 oasc.CorePropertiesLocator.discover Found 1 core definitions
   [junit4]   2> 2134960 T6340 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1
   [junit4]   2> 2134961 T6340 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
   [junit4]   2> 2134964 T6340 oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default ZkCredentialsProvider
   [junit4]   2> 2134970 T6340 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 2134970 T6370 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@15c1fc0 name:ZooKeeperConnection Watcher:127.0.0.1:46175/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 2134971 T6340 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 2134971 T6340 oascc.SolrZkClient.createZkACLProvider Using default ZkACLProvider
   [junit4]   2> 2134972 T6340 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 2134974 T6340 oasc.ChaosMonkey.monkeyLog monkey: init - expire sessions:false cause connection loss:false
   [junit4]   2> 2134974 T6340 oasc.AbstractFullDistribZkTestBase.createJettys Creating collection1 with stateFormat=2
   [junit4]   2> 2134974 T6340 oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default ZkCredentialsProvider
   [junit4]   2> 2134975 T6340 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 2134975 T6372 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@19fb5c9 name:ZooKeeperConnection Watcher:127.0.0.1:46175/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 2134976 T6340 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 2134976 T6340 oascc.SolrZkClient.createZkACLProvider Using default ZkACLProvider
   [junit4]   2> 2134977 T6373 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 2134979 T6365 oasc.Overseer$ClusterStateUpdater.buildCollection building a new collection: collection1
   [junit4]   2> 2134979 T6365 oasc.Overseer$ClusterStateUpdater.createCollection Create collection collection1 with shards [shard1, shard2]
   [junit4]   2> 2134979 T6365 oasc.Overseer$ClusterStateUpdater.createCollection state version collection1 2
   [junit4]   2> 2134980 T6365 oasc.Overseer$ClusterStateUpdater.updateZkStates going to create_collection /collections/collection1/state.json
   [junit4]   2> 2134981 T6373 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 2134981 T6374 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 2135272 T6340 oasc.AbstractFullDistribZkTestBase.createJettys create jetty 1
   [junit4]   2> 2135273 T6340 oejs.Server.doStart jetty-8.1.10.v20130312
   [junit4]   2> 2135276 T6340 oejus.SslContextFactory.doStart Enabled Protocols [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2] of [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2]
   [junit4]   2> 2135277 T6340 oejs.AbstractConnector.doStart Started SslSelectChannelConnector@127.0.0.1:37160
   [junit4]   2> 2135278 T6340 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
   [junit4]   2> 2135279 T6340 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 2135279 T6340 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.DeleteReplicaTest-2301E08B7645B5A6-001/tempDir-003
   [junit4]   2> 2135279 T6340 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.DeleteReplicaTest-2301E08B7645B5A6-001/tempDir-003/'
   [junit4]   2> 2135291 T6340 oasc.ConfigSolr.fromFile Loading container configuration from /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.DeleteReplicaTest-2301E08B7645B5A6-001/tempDir-003/solr.xml
   [junit4]   2> 2135297 T6340 oasc.CorePropertiesLocator.<init> Config-defined core root directory: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.DeleteReplicaTest-2301E08B7645B5A6-001/tempDir-003
   [junit4]   2> 2135297 T6340 oasc.CoreContainer.<init> New CoreContainer 2666391
   [junit4]   2> 2135298 T6340 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.DeleteReplicaTest-2301E08B7645B5A6-001/tempDir-003/]
   [junit4]   2> 2135298 T6340 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
   [junit4]   2> 2135299 T6340 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: 
   [junit4]   2> 2135299 T6340 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
   [junit4]   2> 2135299 T6340 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
   [junit4]   2> 2135300 T6340 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
   [junit4]   2> 2135300 T6340 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
   [junit4]   2> 2135300 T6340 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
   [junit4]   2> 2135301 T6340 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
   [junit4]   2> 2135301 T6340 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
   [junit4]   2> 2135303 T6340 oasu.UpdateShardHandler.<init> Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=false
   [junit4]   2> 2135303 T6340 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 2135304 T6340 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 2135304 T6340 oasc.CoreContainer.load Host Name: 127.0.0.1
   [junit4]   2> 2135305 T6340 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:46175/solr
   [junit4]   2> 2135305 T6340 oasc.ZkController.checkChrootPath zkHost includes chroot
   [junit4]   2> 2135305 T6340 oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default ZkCredentialsProvider
   [junit4]   2> 2135306 T6340 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 2135307 T6385 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@ed9821 name:ZooKeeperConnection Watcher:127.0.0.1:46175 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 2135307 T6340 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 2135307 T6340 oascc.SolrZkClient.createZkACLProvider Using default ZkACLProvider
   [junit4]   2> 2135309 T6340 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 2135311 T6387 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@166cea3 name:ZooKeeperConnection Watcher:127.0.0.1:46175/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 2135312 T6340 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 2135319 T6340 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 2136323 T6340 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:37160_b_y%2Ftb
   [junit4]   2> 2136324 T6340 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:37160_b_y%2Ftb
   [junit4]   2> 2136326 T6340 oasc.Overseer.close Overseer (id=null) closing
   [junit4]   2> 2136327 T6340 oasc.CorePropertiesLocator.discover Looking for core definitions underneath /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.DeleteReplicaTest-2301E08B7645B5A6-001/tempDir-003
   [junit4]   2> 2136333 T6340 oasc.CorePropertiesLocator.discoverUnder Found core conf in /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.DeleteReplicaTest-2301E08B7645B5A6-001/tempDir-003/conf/
   [junit4]   2> 2136456 T6340 oasc.CorePropertiesLocator.discover Found 1 core definitions
   [junit4]   2> 2136457 T6340 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1
   [junit4]   2> 2136457 T6340 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
   [junit4]   2> 2136717 T6340 oasc.AbstractFullDistribZkTestBase.createJettys create jetty 2
   [junit4]   2> 2136718 T6340 oejs.Server.doStart jetty-8.1.10.v20130312
   [junit4]   2> 2136721 T6340 oejus.SslContextFactory.doStart Enabled Protocols [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2] of [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2]
   [junit4]   2> 2136722 T6340 oejs.AbstractConnector.doStart Started SslSelectChannelConnector@127.0.0.1:51660
   [junit4]   2> 2136723 T6340 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
   [junit4]   2> 2136724 T6340 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 2136724 T6340 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.DeleteReplicaTest-2301E08B7645B5A6-001/tempDir-004
   [junit4]   2> 2136724 T6340 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.DeleteReplicaTest-2301E08B7645B5A6-001/tempDir-004/'
   [junit4]   2> 2136740 T6340 oasc.ConfigSolr.fromFile Loading container configuration from /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.DeleteReplicaTest-2301E08B7645B5A6-001/tempDir-004/solr.xml
   [junit4]   2> 2136746 T6340 oasc.CorePropertiesLocator.<init> Config-defined core root directory: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.DeleteReplicaTest-2301E08B7645B5A6-001/tempDir-004
   [junit4]   2> 2136747 T6340 oasc.CoreContainer.<init> New CoreContainer 6500733
   [junit4]   2> 2136747 T6340 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.DeleteReplicaTest-2301E08B7645B5A6-001/tempDir-004/]
   [junit4]   2> 2136748 T6340 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
   [junit4]   2> 2136748 T6340 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: 
   [junit4]   2> 2136748 T6340 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
   [junit4]   2> 2136749 T6340 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
   [junit4]   2> 2136749 T6340 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
   [junit4]   2> 2136749 T6340 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
   [junit4]   2> 2136750 T6340 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
   [junit4]   2> 2136750 T6340 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
   [junit4]   2> 2136750 T6340 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
   [junit4]   2> 2136751 T6340 oasu.UpdateShardHandler.<init> Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=false
   [junit4]   2> 2136752 T6340 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 2136753 T6340 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 2136753 T6340 oasc.CoreContainer.load Host Name: 127.0.0.1
   [junit4]   2> 2136753 T6340 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:46175/solr
   [junit4]   2> 2136754 T6340 oasc.ZkController.checkChrootPath zkHost includes chroot
   [junit4]   2> 2136754 T6340 oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default ZkCredentialsProvider
   [junit4]   2> 2136755 T6340 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 2136756 T6400 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@616b91 name:ZooKeeperConnection Watcher:127.0.0.1:46175 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 2136756 T6340 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 2136756 T6340 oascc.SolrZkClient.createZkACLProvider Using default ZkACLProvider
   [junit4]   2> 2136757 T6340 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 2136758 T6402 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@240bd5 name:ZooKeeperConnection Watcher:127.0.0.1:46175/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 2136758 T6340 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 2136761 T6340 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 2137764 T6340 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:51660_b_y%2Ftb
   [junit4]   2> 2137765 T6340 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:51660_b_y%2Ftb
   [junit4]   2> 2137767 T6340 oasc.Overseer.close Overseer (id=null) closing
   [junit4]   2> 2137769 T6340 oasc.CorePropertiesLocator.discover Looking for core definitions underneath /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.DeleteReplicaTest-2301E08B7645B5A6-001/tempDir-004
   [junit4]   2> 2137774 T6340 oasc.CorePropertiesLocator.discoverUnder Found core conf in /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.DeleteReplicaTest-2301E08B7645B5A6-001/tempDir-004/conf/
   [junit4]   2> 2137900 T6340 oasc.CorePropertiesLocator.discover Found 1 core definitions
   [junit4]   2> 2137901 T6340 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1
   [junit4]   2> 2137902 T6340 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
   [junit4]   2> 2138163 T6340 oasc.AbstractFullDistribZkTestBase.createJettys create jetty 3
   [junit4]   2> 2138164 T6340 oejs.Server.doStart jetty-8.1.10.v20130312
   [junit4]   2> 2138166 T6340 oejus.SslContextFactory.doStart Enabled Protocols [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2] of [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2]
   [junit4]   2> 2138167 T6340 oejs.AbstractConnector.doStart Started SslSelectChannelConnector@127.0.0.1:60980
   [junit4]   2> 2138169 T6340 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
   [junit4]   2> 2138169 T6340 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 2138170 T6340 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.DeleteReplicaTest-2301E08B7645B5A6-001/tempDir-005
   [junit4]   2> 2138170 T6340 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.DeleteReplicaTest-2301E08B7645B5A6-001/tempDir-005/'
   [junit4]   2> 2138181 T6340 oasc.ConfigSolr.fromFile Loading container configuration from /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.DeleteReplicaTest-2301E08B7645B5A6-001/tempDir-005/solr.xml
   [junit4]   2> 2138204 T6340 oasc.CorePropertiesLocator.<init> Config-defined core root directory: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.DeleteReplicaTest-2301E08B7645B5A6-001/tempDir-005
   [junit4]   2> 2138205 T6340 oasc.CoreContainer.<init> New CoreContainer 31173560
   [junit4]   2> 2138205 T6340 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.DeleteReplicaTest-2301E08B7645B5A6-001/tempDir-005/]
   [junit4]   2> 2138205 T6340 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
   [junit4]   2> 2138206 T6340 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: 
   [junit4]   2> 2138206 T6340 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
   [junit4]   2> 2138206 T6340 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
   [junit4]   2> 2138207 T6340 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
   [junit4]   2> 2138207 T6340 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
   [junit4]   2> 2138207 T6340 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
   [junit4]   2> 2138208 T6340 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
   [junit4]   2> 2138208 T6340 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
   [junit4]   2> 2138209 T6340 oasu.UpdateShardHandler.<init> Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=false
   [junit4]   2> 2138210 T6340 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 2138210 T6340 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 2138210 T6340 oasc.CoreContainer.load Host Name: 127.0.0.1
   [junit4]   2> 2138211 T6340 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:46175/solr
   [junit4]   2> 2138211 T6340 oasc.ZkController.checkChrootPath zkHost includes chroot
   [junit4]   2> 2138212 T6340 oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default ZkCredentialsProvider
   [junit4]   2> 2138212 T6340 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 2138213 T6415 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@5fdb48 name:ZooKeeperConnection Watcher:127.0.0.1:46175 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 2138213 T6340 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 2138213 T6340 oascc.SolrZkClient.createZkACLProvider Using default ZkACLProvider
   [junit4]   2> 2138215 T6340 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 2138215 T6417 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@bec75 name:ZooKeeperConnection Watcher:127.0.0.1:46175/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 2138215 T6340 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 2138218 T6340 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 2139221 T6340 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:60980_b_y%2Ftb
   [junit4]   2> 2139222 T6340 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:60980_b_y%2Ftb
   [junit4]   2> 2139225 T6340 oasc.Overseer.close Overseer (id=null) closing
   [junit4]   2> 2139227 T6340 oasc.CorePropertiesLocator.discover Looking for core definitions underneath /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.DeleteReplicaTest-2301E08B7645B5A6-001/tempDir-005
   [junit4]   2> 2139237 T6340 oasc.CorePropertiesLocator.discoverUnder Found core conf in /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.DeleteReplicaTest-2301E08B7645B5A6-001/tempDir-005/conf/
   [junit4]   2> 2139389 T6340 oasc.CorePropertiesLocator.discover Found 1 core definitions
   [junit4]   2> 2139390 T6340 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1
   [junit4]   2> 2139390 T6340 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
   [junit4]   2> 2139661 T6340 oasc.AbstractFullDistribZkTestBase.createJettys create jetty 4
   [junit4]   2> 2139662 T6340 oejs.Server.doStart jetty-8.1.10.v20130312
   [junit4]   2> 2139664 T6340 oejus.SslContextFactory.doStart Enabled Protocols [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2] of [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2]
   [junit4]   2> 2139665 T6340 oejs.AbstractConnector.doStart Started SslSelectChannelConnector@127.0.0.1:54998
   [junit4]   2> 2139667 T6340 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
   [junit4]   2> 2139667 T6340 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 2139668 T6340 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.DeleteReplicaTest-2301E08B7645B5A6-001/tempDir-006
   [junit4]   2> 2139668 T6340 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.DeleteReplicaTest-2301E08B7645B5A6-001/tempDir-006/'
   [junit4]   2> 2139679 T6340 oasc.ConfigSolr.fromFile Loading container configuration from /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.DeleteReplicaTest-2301E08B7645B5A6-001/tempDir-006/solr.xml
   [junit4]   2> 2139684 T6340 oasc.CorePropertiesLocator.<init> Config-defined core root directory: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.DeleteReplicaTest-2301E08B7645B5A6-001/tempDir-006
   [junit4]   2> 2139684 T6340 oasc.CoreContainer.<init> New CoreContainer 32630652
   [junit4]   2> 2139685 T6340 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.DeleteReplicaTest-2301E08B7645B5A6-001/tempDir-006/]
   [junit4]   2> 2139685 T6340 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
   [junit4]   2> 2139686 T6340 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: 
   [junit4]   2> 2139686 T6340 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
   [junit4]   2> 2139686 T6340 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
   [junit4]   2> 2139686 T6340 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
   [junit4]   2> 2139687 T6340 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
   [junit4]   2> 2139687 T6340 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
   [junit4]   2> 2139687 T6340 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
   [junit4]   2> 2139688 T6340 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
   [junit4]   2> 2139689 T6340 oasu.UpdateShardHandler.<init> Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=false
   [junit4]   2> 2139690 T6340 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 2139690 T6340 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 2139690 T6340 oasc.CoreContainer.load Host Name: 127.0.0.1
   [junit4]   2> 2139691 T6340 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:46175/solr
   [junit4]   2> 2139691 T6340 oasc.ZkController.checkChrootPath zkHost includes chroot
   [junit4]   2> 2139691 T6340 oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default ZkCredentialsProvider
   [junit4]   2> 2139692 T6340 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 2139693 T6430 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@5353b9 name:ZooKeeperConnection Watcher:127.0.0.1:46175 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 2139693 T6340 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 2139693 T6340 oascc.SolrZkClient.createZkACLProvider Using default ZkACLProvider
   [junit4]   2> 2139694 T6340 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 2139695 T6432 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@45a205 name:ZooKeeperConnection Watcher:127.0.0.1:46175/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 2139695 T6340 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 2139698 T6340 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 2140701 T6340 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:54998_b_y%2Ftb
   [junit4]   2> 2140702 T6340 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:54998_b_y%2Ftb
   [junit4]   2> 2140705 T6340 oasc.Overseer.close Overseer (id=null) closing
   [junit4]   2> 2140708 T6340 oasc.CorePropertiesLocator.discover Looking for core definitions underneath /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.DeleteReplicaTest-2301E08B7645B5A6-001/tempDir-006
   [junit4]   2> 2140718 T6340 oasc.CorePropertiesLocator.discoverUnder Found core conf in /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.DeleteReplicaTest-2301E08B7645B5A6-001/tempDir-006/conf/
   [junit4]   2> 2140924 T6340 oasc.CorePropertiesLocator.discover Found 1 core definitions
   [junit4]   2> 2140926 T6340 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1
   [junit4]   2> 2140926 T6340 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
   [junit4]   2> 2140930 T6340 oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default ZkCredentialsProvider
   [junit4]   2> 2140931 T6340 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 2140932 T6436 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1ef3a24 name:ZooKeeperConnection Watcher:127.0.0.1:46175/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 2140932 T6340 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 2140932 T6340 oascc.SolrZkClient.createZkACLProvider Using default ZkACLProvider
   [junit4]   2> 2140933 T6340 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 2140935 T6340 oasc.AbstractFullDistribZkTestBase.createCollection Creating collection with stateFormat=2: delLiveColl
   [junit4]   2> 2140935 T6340 oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default ZkCredentialsProvider
   [junit4]   2> 2140936 T6340 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 2140937 T6438 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@177c8c6 name:ZooKeeperConnection Watcher:127.0.0.1:46175/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 2140937 T6340 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 2140937 T6340 oascc.SolrZkClient.createZkACLProvider Using default ZkACLProvider
   [junit4]   2> 2140938 T6340 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 2140992 T6381 oasha.CollectionsHandler.handleCreateAction Creating Collection : replicationFactor=2&maxShardsPerNode=2&name=delLiveColl&action=CREATE&numShards=2&stateFormat=2&wt=javabin&version=2
   [junit4]   2> 2140993 T6373 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/collection-queue-work state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 2140993 T6366 oasc.OverseerCollectionProcessor.run Overseer Collection Processor: Get the message id:/overseer/collection-queue-work/qn-0000000000 message:{
   [junit4]   2> 	  "operation":"create",
   [junit4]   2> 	  "fromApi":"true",
   [junit4]   2> 	  "name":"delLiveColl",
   [junit4]   2> 	  "replicationFactor":"2",
   [junit4]   2> 	  "numShards":"2",
   [junit4]   2> 	  "maxShardsPerNode":"2",
   [junit4]   2> 	  "stateFormat":"2"}
   [junit4]   2> 2140995 T6439 oasc.OverseerCollectionProcessor.processMessage WARN OverseerCollectionProcessor.processMessage : create , {
   [junit4]   2> 	  "operation":"create",
   [junit4]   2> 	  "fromApi":"true",
   [junit4]   2> 	  "name":"delLiveColl",
   [junit4]   2> 	  "replicationFactor":"2",
   [junit4]   2> 	  "numShards":"2",
   [junit4]   2> 	  "maxShardsPerNode":"2",
   [junit4]   2> 	  "stateFormat":"2"}
   [junit4]   2> 2140996 T6439 oasc.OverseerCollectionProcessor.createConfNode Only one config set found in zk - using it:conf1
   [junit4]   2> 2140996 T6439 oasc.OverseerCollectionProcessor.createConfNode creating collections conf node /collections/delLiveColl 
   [junit4]   2> 2140997 T6439 oascc.SolrZkClient.makePath makePath: /collections/delLiveColl
   [junit4]   2> 2140998 T6373 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 2141000 T6365 oasc.Overseer$ClusterStateUpdater.buildCollection building a new collection: delLiveColl
   [junit4]   2> 2141000 T6365 oasc.Overseer$ClusterStateUpdater.createCollection Create collection delLiveColl with shards [shard1, shard2]
   [junit4]   2> 2141000 T6365 oasc.Overseer$ClusterStateUpdater.createCollection state version delLiveColl 2
   [junit4]   2> 2141002 T6365 oasc.Overseer$ClusterStateUpdater.updateZkStates going to create_collection /collections/delLiveColl/state.json
   [junit4]   2> 2141002 T6374 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
   [junit4]   2> 2141002 T6403 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
   [junit4]   2> 2141003 T6418 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
   [junit4]   2> 2141003 T6388 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
   [junit4]   2> 2141002 T6440 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
   [junit4]   2> 2141003 T6373 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
   [junit4]   2> 2141004 T6441 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
   [junit4]   2> 2141005 T6433 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
   [junit4]   2> 2141098 T6439 oasc.OverseerCollectionProcessor.createCollection Creating SolrCores for new collection delLiveColl, shardNames [shard1, shard2] , replicationFactor : 2
   [junit4]   2> 2141099 T6439 oasc.OverseerCollectionProcessor.createCollection Creating shard delLiveColl_shard1_replica1 as part of slice shard1 of collection delLiveColl on 127.0.0.1:54998_b_y%2Ftb
   [junit4]   2> 2141099 T6439 oasc.OverseerCollectionProcessor.createCollection Creating shard delLiveColl_shard1_replica2 as part of slice shard1 of collection delLiveColl on 127.0.0.1:45141_b_y%2Ftb
   [junit4]   2> 2141101 T6439 oasc.OverseerCollectionProcessor.createCollection Creating shard delLiveColl_shard2_replica1 as part of slice shard2 of collection delLiveColl on 127.0.0.1:60980_b_y%2Ftb
   [junit4]   2> 2141104 T6439 oasc.OverseerCollectionProcessor.createCollection Creating shard delLiveColl_shard2_replica2 as part of slice shard2 of collection delLiveColl on 127.0.0.1:51660_b_y%2Ftb
   [junit4]   2> 2141181 T6410 oasha.CoreAdminHandler.handleCreateAction core create command qt=/admin/cores&collection.configName=conf1&name=delLiveColl_shard2_replica1&action=CREATE&numShards=2&collection=delLiveColl&shard=shard2&wt=javabin&version=2
   [junit4]   2> 2141183 T6410 oasc.ZkController.publish publishing core=delLiveColl_shard2_replica1 state=down collection=delLiveColl
   [junit4]   2> 2141186 T6357 oasha.CoreAdminHandler.handleCreateAction core create command qt=/admin/cores&collection.configName=conf1&name=delLiveColl_shard1_replica2&action=CREATE&numShards=2&collection=delLiveColl&shard=shard1&wt=javabin&version=2
   [junit4]   2> 2141186 T6396 oasha.CoreAdminHandler.handleCreateAction core create command qt=/admin/cores&collection.configName=conf1&name=delLiveColl_shard2_replica2&action=CREATE&numShards=2&collection=delLiveColl&shard=shard2&wt=javabin&version=2
   [junit4]   2> 2141187 T6373 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 2141187 T6410 oasc.ZkController.preRegister Registering watch for external collection delLiveColl
   [junit4]   2> 2141188 T6410 oascc.ZkStateReader.addZkWatch addZkWatch delLiveColl
   [junit4]   2> 2141188 T6410 oascc.ZkStateReader.addZkWatch Updating collection state at /collections/delLiveColl/state.json from ZooKeeper... 
   [junit4]   2> 2141189 T6396 oasc.ZkController.publish publishing core=delLiveColl_shard2_replica2 state=down collection=delLiveColl
   [junit4]   2> 2141189 T6357 oasc.ZkController.publish publishing core=delLiveColl_shard1_replica2 state=down collection=delLiveColl
   [junit4]   2> 2141190 T6365 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
   [junit4]   2> 	  "core":"delLiveColl_shard2_replica1",
   [junit4]   2> 	  "roles":null,
   [junit4]   2> 	  "base_url":"https://127.0.0.1:60980/b_y/tb",
   [junit4]   2> 	  "node_name":"127.0.0.1:60980_b_y%2Ftb",
   [junit4]   2> 	  "numShards":"2",
   [junit4]   2> 	  "state":"down",
   [junit4]   2> 	  "shard":"shard2",
   [junit4]   2> 	  "collection":"delLiveColl",
   [junit4]   2> 	  "operation":"state"}
   [junit4]   2> 2141190 T6410 oascc.ZkStateReader.updateWatchedCollection Updating data for delLiveColl to ver 0 
   [junit4]   2> 2141191 T6410 oasc.ZkController.waitForCoreNodeName look for our core node name
   [junit4]   2> 2141191 T6428 oasha.CoreAdminHandler.handleCreateAction core create command qt=/admin/cores&collection.configName=conf1&name=delLiveColl_shard1_replica1&action=CREATE&numShards=2&collection=delLiveColl&shard=shard1&wt=javabin&version=2
   [junit4]   2> 2141191 T6396 oasc.ZkController.preRegister Registering watch for external collection delLiveColl
   [junit4]   2> 2141192 T6396 oascc.ZkStateReader.addZkWatch addZkWatch delLiveColl
   [junit4]   2> 2141192 T6396 oascc.ZkStateReader.addZkWatch Updating collection state at /collections/delLiveColl/state.json from ZooKeeper... 
   [junit4]   2> 2141192 T6357 oasc.ZkController.preRegister Registering watch for external collection delLiveColl
   [junit4]   2> 2141193 T6396 oascc.ZkStateReader.updateWatchedCollection Updating data for delLiveColl to ver 0 
   [junit4]   2> 2141194 T6396 oasc.ZkController.waitForCoreNodeName look for our core node name
   [junit4]   2> 2141194 T6428 oasc.ZkController.publish publishing core=delLiveColl_shard1_replica1 state=down collection=delLiveColl
   [junit4]   2> 2141196 T6428 oasc.ZkController.preRegister Registering watch for external collection delLiveColl
   [junit4]   2> 2141196 T6428 oascc.ZkStateReader.addZkWatch addZkWatch delLiveColl
   [junit4]   2> 2141196 T6428 oascc.ZkStateReader.addZkWatch Updating collection state at /collections/delLiveColl/state.json from ZooKeeper... 
   [junit4]   2> 2141196 T6365 oasc.Overseer$ClusterStateUpdater.updateZkStates going to update_collection /collections/delLiveColl/state.json
   [junit4]   2> 2141197 T6418 oascc.ZkStateReader$7.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/delLiveColl/state.json, has occurred - updating... 
   [junit4]   2> 2141198 T6433 oascc.ZkStateReader$7.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/delLiveColl/state.json, has occurred - updating... 
   [junit4]   2> 2141198 T6418 oascc.ZkStateReader.updateWatchedCollection Updating data for delLiveColl to ver 1 
   [junit4]   2> 2141198 T6403 oascc.ZkStateReader$7.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/delLiveColl/state.json, has occurred - updating... 
   [junit4]   2> 2141198 T6428 oascc.ZkStateReader.updateWatchedCollection Updating data for delLiveColl to ver 1 
   [junit4]   2> 2141199 T6357 oascc.ZkStateReader.addZkWatch addZkWatch delLiveColl
   [junit4]   2> 2141199 T6403 oascc.ZkStateReader.updateWatchedCollection Updating data for delLiveColl to ver 1 
   [junit4]   2> 2141199 T6428 oasc.ZkController.waitForCoreNodeName look for our core node name
   [junit4]   2> 2141199 T6357 oascc.ZkStateReader.addZkWatch Updating collection state at /collections/delLiveColl/state.json from ZooKeeper... 
   [junit4]   2> 2141199 T6433 oascc.ZkStateReader.updateWatchedCollection Updating data for delLiveColl to ver 1 
   [junit4]   2> 2141201 T6357 oascc.ZkStateReader.updateWatchedCollection Updating data for delLiveColl to ver 1 
   [junit4]   2> 2141201 T6357 oasc.ZkController.waitForCoreNodeName look for our core node name
   [junit4]   2> 2141202 T6365 oascc.ZkStateReader.updateWatchedCollection Updating data for delLiveColl to ver 1 
   [junit4]   2> 2141202 T6365 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
   [junit4]   2> 	  "core":"delLiveColl_shard2_replica2",
   [junit4]   2> 	  "roles":null,
   [junit4]   2> 	  "base_url":"https://127.0.0.1:51660/b_y/tb",
   [junit4]   2> 	  "node_name":"127.0.0.1:51660_b_y%2Ftb",
   [junit4]   2> 	  "numShards":"2",
   [junit4]   2> 	  "state":"down",
   [junit4]   2> 	  "shard":"shard2",
   [junit4]   2> 	  "collection":"delLiveColl",
   [junit4]   2> 	  "operation":"state"}
   [junit4]   2> 2141204 T6373 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 2141204 T6365 oasc.Overseer$ClusterStateUpdater.updateZkStates going to update_collection /collections/delLiveColl/state.json
   [junit4]   2> 2141204 T6418 oascc.ZkStateReader$7.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/delLiveColl/state.json, has occurred - updating... 
   [junit4]   2> 2141204 T6373 oascc.ZkStateReader$7.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/delLiveColl/state.json, has occurred - updating... 
   [junit4]   2> 2141204 T6433 oascc.ZkStateReader$7.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/delLiveColl/state.json, has occurred - updating... 
   [junit4]   2> 2141205 T6418 oascc.ZkStateReader.updateWatchedCollection Updating data for delLiveColl to ver 2 
   [junit4]   2> 2141205 T6433 oascc.ZkStateReader.updateWatchedCollection Updating data for delLiveColl to ver 2 
   [junit4]   2> 2141204 T6403 oascc.ZkStateReader$7.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/delLiveColl/state.json, has occurred - updating... 
   [junit4]   2> 2141206 T6403 oascc.ZkStateReader.updateWatchedCollection Updating data for delLiveColl to ver 2 
   [junit4]   2> 2141206 T6373 oascc.ZkStateReader.updateWatchedCollection Updating data for delLiveColl to ver 2 
   [junit4]   2> 2141208 T6365 oascc.ZkStateReader.updateWatchedCollection Updating data for delLiveColl to ver 2 
   [junit4]   2> 2141208 T6365 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
   [junit4]   2> 	  "core":"delLiveColl_shard1_replica2",
   [junit4]   2> 	  "roles":null,
   [junit4]   2> 	  "base_url":"https://127.0.0.1:45141/b_y/tb",
   [junit4]   2> 	  "node_name":"127.0.0.1:45141_b_y%2Ftb",
   [junit4]   2> 	  "numShards":"2",
   [junit4]   2> 	  "state":"down",
   [junit4]   2> 	  "shard":"shard1",
   [junit4]   2> 	  "collection":"delLiveColl",
   [junit4]   2> 	  "operation":"state"}
   [junit4]   2> 2141209 T6373 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 2141209 T6365 oasc.Overseer$ClusterStateUpdater.updateZkStates going to update_collection /collections/delLiveColl/state.json
   [junit4]   2> 2141209 T6418 oascc.ZkStateReader$7.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/delLiveColl/state.json, has occurred - updating... 
   [junit4]   2> 2141210 T6403 oascc.ZkStateReader$7.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/delLiveColl/state.json, has occurred - updating... 
   [junit4]   2> 2141210 T6373 oascc.ZkStateReader$7.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/delLiveColl/state.json, has occurred - updating... 
   [junit4]   2> 2141210 T6403 oascc.ZkStateReader.updateWatchedCollection Updating data for delLiveColl to ver 3 
   [junit4]   2> 2141209 T6433 oascc.ZkStateReader$7.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/delLiveColl/state.json, has occurred - updating... 
   [junit4]   2> 2141211 T6373 oascc.ZkStateReader.updateWatchedCollection Updating data for delLiveColl to ver 3 
   [junit4]   2> 2141211 T6433 oascc.ZkStateReader.updateWatchedCollection Updating data for delLiveColl to ver 3 
   [junit4]   2> 2141210 T6418 oascc.ZkStateReader.updateWatchedCollection Updating data for delLiveColl to ver 3 
   [junit4]   2> 2141213 T6365 oascc.ZkStateReader.updateWatchedCollection Updating data for delLiveColl to ver 3 
   [junit4]   2> 2141213 T6365 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
   [junit4]   2> 	  "core":"delLiveColl_shard1_replica1",
   [junit4]   2> 	  "roles":null,
   [junit4]   2> 	  "base_url":"https://127.0.0.1:54998/b_y/tb",
   [junit4]   2> 	  "node_name":"127.0.0.1:54998_b_y%2Ftb",
   [junit4]   2> 	  "numShards":"2",
   [junit4]   2> 	  "state":"down",
   [junit4]   2> 	  "shard":"shard1",
   [junit4]   2> 	  "collection":"delLiveColl",
   [junit4]   2> 	  "operation":"state"}
   [junit4]   2> 2141214 T6373 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 2141214 T6365 oasc.Overseer$ClusterStateUpdater.updateZkStates going to update_collection /collections/delLiveColl/state.json
   [junit4]   2> 2141215 T6418 oascc.ZkStateReader$7.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/delLiveColl/state.json, has occurred - updating... 
   [junit4]   2> 2141215 T6373 oascc.ZkStateReader$7.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/delLiveColl/state.json, has occurred - updating... 
   [junit4]   2> 2141215 T6403 oascc.ZkStateReader$7.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/delLiveColl/state.json, has occurred - updating... 
   [junit4]   2> 2141216 T6403 oascc.ZkStateReader.updateWatchedCollection Updating data for delLiveColl to ver 4 
   [junit4]   2> 2141215 T6433 oascc.ZkStateReader$7.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/delLiveColl/state.json, has occurred - updating... 
   [junit4]   2> 2141215 T6418 oascc.ZkStateReader.updateWatchedCollection Updating data for delLiveColl to ver 4 
   [junit4]   2> 2141217 T6433 oascc.ZkStateReader.updateWatchedCollection Updating data for delLiveColl to ver 4 
   [junit4]   2> 2141217 T6373 oascc.ZkStateReader.updateWatchedCollection Updating data for delLiveColl to ver 4 
   [junit4]   2> 2142191 T6410 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for delLiveColl_shard2_replica1
   [junit4]   2> 2142191 T6410 oasc.ZkController.createCollectionZkNode Check for collection zkNode:delLiveColl
   [junit4]   2> 2142192 T6410 oasc.ZkController.createCollectionZkNode Collection zkNode exists
   [junit4]   2> 2142192 T6410 oascc.ZkStateReader.readConfigName Load collection config from:/collections/delLiveColl
   [junit4]   2> 2142193 T6410 oascc.ZkStateReader.readConfigName path=/collections/delLiveColl configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 2142194 T6410 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.DeleteReplicaTest-2301E08B7645B5A6-001/tempDir-005/delLiveColl_shard2_replica1/'
   [junit4]   2> 2142194 T6396 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for delLiveColl_shard2_replica2
   [junit4]   2> 2142194 T6396 oasc.ZkController.createCollectionZkNode Check for collection zkNode:delLiveColl
   [junit4]   2> 2142197 T6396 oasc.ZkController.createCollectionZkNode Collection zkNode exists
   [junit4]   2> 2142197 T6396 oascc.ZkStateReader.readConfigName Load collection config from:/collections/delLiveColl
   [junit4]   2> 2142198 T6396 oascc.ZkStateReader.readConfigName path=/collections/delLiveColl configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 2142198 T6396 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.DeleteReplicaTest-2301E08B7645B5A6-001/tempDir-004/delLiveColl_shard2_replica2/'
   [junit4]   2> 2142200 T6428 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for delLiveColl_shard1_replica1
   [junit4]   2> 2142200 T6428 oasc.ZkController.createCollectionZkNode Check for collection zkNode:delLiveColl
   [junit4]   2> 2142201 T6428 oasc.ZkController.createCollectionZkNode Collection zkNode exists
   [junit4]   2> 2142201 T6428 oascc.ZkStateReader.readConfigName Load collection config from:/collections/delLiveColl
   [junit4]   2> 2142201 T6357 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for delLiveColl_shard1_replica2
   [junit4]   2> 2142202 T6357 oasc.ZkController.createCollectionZkNode Check for collection zkNode:delLiveColl
   [junit4]   2> 2142202 T6357 oasc.ZkController.createCollectionZkNode Collection zkNode exists
   [junit4]   2> 2142202 T6357 oascc.ZkStateReader.readConfigName Load collection config from:/collections/delLiveColl
   [junit4]   2> 2142203 T6428 oascc.ZkStateReader.readConfigName path=/collections/delLiveColl configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 2142203 T6428 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.DeleteReplicaTest-2301E08B7645B5A6-001/tempDir-006/delLiveColl_shard1_replica1/'
   [junit4]   2> 2142203 T6357 oascc.ZkStateReader.readConfigName path=/collections/delLiveColl configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 2142204 T6357 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.DeleteReplicaTest-2301E08B7645B5A6-001/tempDir-002/delLiveColl_shard1_replica2/'
   [junit4]   2> 2142229 T6410 oasc.SolrConfig.<init> Using Lucene MatchVersion: 5.0.0
   [junit4]   2> 2142254 T6410 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 2142257 T6410 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
   [junit4]   2> 2142258 T6357 oasc.SolrConfig.<init> Using Lucene MatchVersion: 5.0.0
   [junit4]   2> 2142259 T6396 oasc.SolrConfig.<init> Using Lucene MatchVersion: 5.0.0
   [junit4]   2> 2142263 T6428 oasc.SolrConfig.<init> Using Lucene MatchVersion: 5.0.0
   [junit4]   2> 2142267 T6410 oass.IndexSchema.readSchema [delLiveColl_shard2_replica1] Schema name=test
   [junit4]   2> 2142272 T6396 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 2142272 T6357 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 2142285 T6357 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
   [junit4]   2> 2142289 T6396 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
   [junit4]   2> 2142289 T6428 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 2142291 T6428 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
   [junit4]   2> 2142291 T6357 oass.IndexSchema.readSchema [delLiveColl_shard1_replica2] Schema name=test
   [junit4]   2> 2142298 T6396 oass.IndexSchema.readSchema [delLiveColl_shard2_replica2] Schema name=test
   [junit4]   2> 2142298 T6428 oass.IndexSchema.readSchema [delLiveColl_shard1_replica1] Schema name=test
   [junit4]   2> 2142452 T6410 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 2142457 T6410 oass.IndexSchema.readSchema default search field in schema is text
   [junit4]   2> 2142459 T6410 oass.IndexSchema.readSchema unique key field: id
   [junit4]   2> 2142464 T6428 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 2142466 T6410 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 2142468 T6410 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 2142469 T6428 oass.IndexSchema.readSchema default search field in schema is text
   [junit4]   2> 2142471 T6428 oass.IndexSchema.readSchema unique key field: id
   [junit4]   2> 2142472 T6410 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 2142473 T6410 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
   [junit4]   2> 2142474 T6410 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
   [junit4]   2> 2142475 T6410 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 2142475 T6410 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
   [junit4]   2> 2142476 T6410 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
   [junit4]   2> 2142476 T6410 oasc.CoreContainer.create Creating SolrCore 'delLiveColl_shard2_replica1' using configuration from collection delLiveColl
   [junit4]   2> 2142476 T6410 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 2142477 T6410 oasc.SolrCore.<init> [delLiveColl_shard2_replica1] Opening new SolrCore at /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.DeleteReplicaTest-2301E08B7645B5A6-001/tempDir-005/delLiveColl_shard2_replica1/, dataDir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.DeleteReplicaTest-2301E08B7645B5A6-001/tempDir-005/delLiveColl_shard2_replica1/data/
   [junit4]   2> 2142477 T6410 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@2049e8
   [junit4]   2> 2142478 T6410 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.DeleteReplicaTest-2301E08B7645B5A6-001/tempDir-005/delLiveColl_shard2_replica1/data
   [junit4]   2> 2142478 T6410 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.DeleteReplicaTest-2301E08B7645B5A6-001/tempDir-005/delLiveColl_shard2_replica1/data/index/
   [junit4]   2> 2142479 T6410 oasc.SolrCore.initIndex WARN [delLiveColl_shard2_replica1] Solr index directory '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.DeleteReplicaTest-2301E08B7645B5A6-001/tempDir-005/delLiveColl_shard2_replica1/data/index' doesn't exist. Creating new index...
   [junit4]   2> 2142480 T6410 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.DeleteReplicaTest-2301E08B7645B5A6-001/tempDir-005/delLiveColl_shard2_replica1/data/index
   [junit4]   2> 2142480 T6410 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=23, maxMergeAtOnceExplicit=49, maxMergedSegmentMB=2.9052734375, floorSegmentMB=0.55078125, forceMergeDeletesPctAllowed=18.294957546728785, segmentsPerTier=44.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.14684349874543712
   [junit4]   2> 2142481 T6410 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 		commit{dir=MockDirectoryWrapper(RAMDirectory(id=lucene-5d5088)),segFN=segments_1,generation=1}
   [junit4]   2> 2142481 T6410 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 2142484 T6410 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "nodistrib"
   [junit4]   2> 2142485 T6428 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 2142485 T6410 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe"
   [junit4]   2> 2142486 T6410 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 2142487 T6410 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "stored_sig"
   [junit4]   2> 2142487 T6410 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 2142498 T6410 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
   [junit4]   2> 2142498 T6410 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 2142499 T6410 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 2142499 T6410 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 2142499 T6428 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 2142500 T6410 oasc.RequestHandlers.initHandlersFromConfig created /update/csv: org.apache.solr.handler.UpdateRequestHandler
   [junit4]   2> 2142500 T6410 oasc.RequestHandlers.initHandlersFromConfig created /update/json: org.apache.solr.handler.UpdateRequestHandler
   [junit4]   2> 2142501 T6410 oasc.RequestHandlers.initHandlersFromConfig created /update/json/docs: org.apache.solr.handler.UpdateRequestHandler
   [junit4]   2> 2142501 T6410 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
   [junit4]   2> 2142502 T6410 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
   [junit4]   2> 2142502 T6410 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
   [junit4]   2> 2142502 T6428 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 2142503 T6410 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
   [junit4]   2> 2142504 T6410 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
   [junit4]   2> 2142504 T6428 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
   [junit4]   2> 2142505 T6396 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 2142505 T6428 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
   [junit4]   2> 2142504 T6410 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
   [junit4]   2> 2142506 T6428 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 2142507 T6410 oasc.RequestHandlers.initHandlersFromConfig created /admin/file: solr.admin.ShowFileRequestHandler
   [junit4]   2> 2142507 T6410 oasc.RequestHandlers.initHandlersFromConfig created /admin/fileedit: solr.admin.EditFileRequestHandler
   [junit4]   2> 2142508 T6428 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
   [junit4]   2> 2142508 T6428 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
   [junit4]   2> 2142509 T6428 oasc.CoreContainer.create Creating SolrCore 'delLiveColl_shard1_replica1' using configuration from collection delLiveColl
   [junit4]   2> 2142509 T6396 oass.IndexSchema.readSchema default search field in schema is text
   [junit4]   2> 2142510 T6410 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 2142510 T6428 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 2142511 T6428 oasc.SolrCore.<init> [delLiveColl_shard1_replica1] Opening new SolrCore at /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.DeleteReplicaTest-2301E08B7645B5A6-001/tempDir-006/delLiveColl_shard1_replica1/, dataDir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.DeleteReplicaTest-2301E08B7645B5A6-001/tempDir-006/delLiveColl_shard1_replica1/data/
   [junit4]   2> 2142511 T6410 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 2142512 T6428 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@2049e8
   [junit4]   2> 2142512 T6396 oass.IndexSchema.readSchema unique key field: id
   [junit4]   2> 2142513 T6410 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 2142513 T6428 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.DeleteReplicaTest-2301E08B7645B5A6-001/tempDir-006/delLiveColl_shard1_replica1/data
   [junit4]   2> 2142514 T6428 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.DeleteReplicaTest-2301E08B7645B5A6-001/tempDir-006/delLiveColl_shard1_replica1/data/index/
   [junit4]   2> 2142514 T6428 oasc.SolrCore.initIndex WARN [delLiveColl_shard1_replica1] Solr index directory '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.DeleteReplicaTest-2301E08B7645B5A6-001/tempDir-006/delLiveColl_shard1_replica1/data/index' doesn't exist. Creating new index...
   [junit4]   2> 2142515 T6410 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 2142515 T6428 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.DeleteReplicaTest-2301E08B7645B5A6-001/tempDir-006/delLiveColl_shard1_replica1/data/index
   [junit4]   2> 2142516 T6428 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=23, maxMergeAtOnceExplicit=49, maxMergedSegmentMB=2.9052734375, floorSegmentMB=0.55078125, forceMergeDeletesPctAllowed=18.294957546728785, segmentsPerTier=44.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.14684349874543712
   [junit4]   2> 2142516 T6428 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 		commit{dir=MockDirectoryWrapper(RAMDirectory(id=lucene-31cc74)),segFN=segments_1,generation=1}
   [junit4]   2> 2142517 T6428 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 2142518 T6357 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 2142520 T6428 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "nodistrib"
   [junit4]   2> 2142521 T6428 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe"
   [junit4]   2> 2142521 T6428 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 2142522 T6428 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "stored_sig"
   [junit4]   2> 2142522 T6428 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 2142522 T6357 oass.IndexSchema.readSchema default search field in schema is text
   [junit4]   2> 2142523 T6428 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
   [junit4]   2> 2142524 T6410 oasu.CommitTracker.<init> Hard AutoCommit: disabled
   [junit4]   2> 2142524 T6410 oasu.CommitTracker.<init> Soft AutoCommit: disabled
   [junit4]   2> 2142524 T6428 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 2142525 T6410 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=47, maxMergeAtOnceExplicit=33, maxMergedSegmentMB=72.443359375, floorSegmentMB=1.203125, forceMergeDeletesPctAllowed=2.339102877818436, segmentsPerTier=40.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0
   [junit4]   2> 2142525 T6428 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 2142525 T6428 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 2142525 T6357 oass.IndexSchema.readSchema unique key field: id
   [junit4]   2> 2142524 T6396 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 2142526 T6428 oasc.RequestHandlers.initHandlersFromConfig created /update/csv: org.apache.solr.handler.UpdateRequestHandler
   [junit4]   2> 2142527 T6428 oasc.RequestHandlers.initHandlersFromConfig created /update/json: org.apache.solr.handler.UpdateRequestHandler
   [junit4]   2> 2142527 T6428 oasc.RequestHandlers.initHandlersFromConfig created /update/json/docs: org.apache.solr.handler.UpdateRequestHandler
   [junit4]   2> 2142528 T6428 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
   [junit4]   2> 2142528 T6428 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
   [junit4]   2> 2142529 T6428 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
   [junit4]   2> 2142529 T6428 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
   [junit4]   2> 2142530 T6428 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
   [junit4]   2> 2142530 T6428 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
   [junit4]   2> 2142530 T6428 oasc.RequestHandlers.initHandlersFromConfig created /admin/file: solr.admin.ShowFileRequestHandler
   [junit4]   2> 2142531 T6428 oasc.RequestHandlers.initHandlersFromConfig created /admin/fileedit: solr.admin.EditFileRequestHandler
   [junit4]   2> 2142532 T6428 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 2142533 T6428 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 2142534 T6428 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 2142534 T6357 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 2142534 T6428 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 2142525 T6410 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 		commit{dir=MockDirectoryWrapper(RAMDirectory(id=lucene-5d5088)),segFN=segments_1,generation=1}
   [junit4]   2> 2142536 T6410 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 2142536 T6357 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 2142528 T6396 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 2142536 T6410 oass.SolrIndexSearcher.<init> Opening Searcher@1bc1bc4[delLiveColl_shard2_replica1] main
   [junit4]   2> 2142537 T6410 oascc.ZkStateReader.readConfigName Load collection config from:/collections/delLiveColl
   [junit4]   2> 2142537 T6428 oasu.CommitTracker.<init> Hard AutoCommit: disabled
   [junit4]   2> 2142538 T6428 oasu.CommitTracker.<init> Soft AutoCommit: disabled
   [junit4]   2> 2142538 T6357 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 2142538 T6410 oascc.ZkStateReader.readConfigName path=/collections/delLiveColl configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 2142540 T6410 oasr.ManagedResourceStorage.newStorageIO Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
   [junit4]   2> 2142540 T6357 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
   [junit4]   2> 2142541 T6357 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
   [junit4]   2> 2142539 T6428 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=47, maxMergeAtOnc

[...truncated too long message...]

er.java:383)
   [junit4]   2> 	at org.apache.solr.cloud.Overseer$ClusterStateUpdater.access$300(Overseer.java:126)
   [junit4]   2> 	at org.apache.solr.cloud.Overseer$ClusterStateUpdater$1.run(Overseer.java:333)
   [junit4]   2> 
   [junit4]   2> 2174408 T6340 oejsh.ContextHandler.doStop stopped o.e.j.s.ServletContextHandler{/b_y/tb,null}
   [junit4]   2> 2179398 T6340 oas.SolrTestCaseJ4.tearDown ###Ending testDistribSearch
   [junit4]   2> 2179399 T6340 oasc.ZkTestServer.send4LetterWord connecting to 127.0.0.1:46175 46175
   [junit4]   2> 2179968 T6341 oasc.ZkTestServer.send4LetterWord connecting to 127.0.0.1:46175 46175
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=DeleteReplicaTest -Dtests.method=testDistribSearch -Dtests.seed=2301E08B7645B5A6 -Dtests.multiplier=3 -Dtests.slow=true -Dtests.locale=ar_TN -Dtests.timezone=America/Asuncion -Dtests.file.encoding=UTF-8
   [junit4] ERROR   45.6s J1 | DeleteReplicaTest.testDistribSearch <<<
   [junit4]    > Throwable #1: org.apache.solr.client.solrj.SolrServerException: java.lang.ClassCastException: org.apache.solr.common.params.CollectionParams$CollectionAction cannot be cast to java.lang.String
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([2301E08B7645B5A6:A2E76E93011AD59A]:0)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.LBHttpSolrServer.doRequest(LBHttpSolrServer.java:381)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.LBHttpSolrServer.request(LBHttpSolrServer.java:304)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.CloudSolrServer.sendRequest(CloudSolrServer.java:874)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.CloudSolrServer.requestWithRetryOnStaleState(CloudSolrServer.java:658)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.CloudSolrServer.request(CloudSolrServer.java:601)
   [junit4]    > 	at org.apache.solr.cloud.DeleteReplicaTest.tryToRemoveOnlyIfDown(DeleteReplicaTest.java:154)
   [junit4]    > 	at org.apache.solr.cloud.DeleteReplicaTest.deleteLiveReplicaTest(DeleteReplicaTest.java:125)
   [junit4]    > 	at org.apache.solr.cloud.DeleteReplicaTest.doTest(DeleteReplicaTest.java:88)
   [junit4]    > 	at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:869)
   [junit4]    > 	at java.lang.Thread.run(Thread.java:745)
   [junit4]    > Caused by: java.lang.ClassCastException: org.apache.solr.common.params.CollectionParams$CollectionAction cannot be cast to java.lang.String
   [junit4]    > 	at org.apache.solr.common.params.MapSolrParams.getParams(MapSolrParams.java:43)
   [junit4]    > 	at org.apache.solr.common.params.ModifiableSolrParams.add(ModifiableSolrParams.java:117)
   [junit4]    > 	at org.apache.solr.common.params.ModifiableSolrParams.<init>(ModifiableSolrParams.java:56)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrServer.createMethod(HttpSolrServer.java:273)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:210)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:206)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.LBHttpSolrServer.doRequest(LBHttpSolrServer.java:343)
   [junit4]    > 	... 47 more
   [junit4]   2> 2179976 T6340 oas.SolrTestCaseJ4.deleteCore ###deleteCore
   [junit4]   2> NOTE: leaving temporary files on disk at: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.DeleteReplicaTest-2301E08B7645B5A6-001
   [junit4]   2> 45623 T6339 ccr.ThreadLeakControl.checkThreadLeaks WARNING Will linger awaiting termination of 1 leaked thread(s).
   [junit4]   2> NOTE: test params are: codec=Lucene410: {}, docValues:{}, sim=RandomSimilarityProvider(queryNorm=true,coord=crazy): {}, locale=ar_TN, timezone=America/Asuncion
   [junit4]   2> NOTE: Linux 3.13.0-35-generic i386/Oracle Corporation 1.8.0_20 (32-bit)/cpus=8,threads=1,free=99883032,total=302505984
   [junit4]   2> NOTE: All tests run in this JVM: [DocValuesMissingTest, HdfsBasicDistributedZk2Test, TestCollapseQParserPlugin, DistribCursorPagingTest, TestRealTimeGet, ConvertedLegacyTest, LoggingHandlerTest, IndexSchemaRuntimeFieldTest, TestMiniSolrCloudCluster, TestXIncludeConfig, CoreContainerCoreInitFailuresTest, TestInfoStreamLogging, TestRandomMergePolicy, MultiThreadedOCPTest, TestCSVResponseWriter, TestRecovery, TriLevelCompositeIdRoutingTest, OpenCloseCoreStressTest, CachingDirectoryFactoryTest, AnalyticsQueryTest, TestSolrQueryParserResource, UpdateRequestProcessorFactoryTest, LegacyHTMLStripCharFilterTest, DirectUpdateHandlerOptimizeTest, TestReversedWildcardFilterFactory, TestCloudManagedSchemaConcurrent, TestQueryUtils, TermVectorComponentDistributedTest, TestNonNRTOpen, TestFieldTypeResource, UniqFieldsUpdateProcessorFactoryTest, TermVectorComponentTest, TestDistributedMissingSort, QueryElevationComponentTest, TestFunctionQuery, SOLR749Test, SortByFunctionTest, URLClassifyProcessorTest, BasicDistributedZk2Test, SynonymTokenizerTest, BlockCacheTest, SimplePostToolTest, TestSolrQueryParser, TestAtomicUpdateErrorCases, SampleTest, ClusterStateTest, OverseerRolesTest, HardAutoCommitTest, TestFiltering, TestJoin, QueryParsingTest, PolyFieldTest, TestUpdate, DistanceFunctionTest, TestManagedStopFilterFactory, DistributedTermsComponentTest, TestSchemaSimilarityResource, ChangedSchemaMergeTest, TestElisionMultitermQuery, OutputWriterTest, ExternalCollectionsTest, DeleteInactiveReplicaTest, TestQueryTypes, TestSearcherReuse, ScriptEngineTest, FileBasedSpellCheckerTest, TestUtils, TestManagedSchema, TestPHPSerializedResponseWriter, TestRandomFaceting, LeaderElectionIntegrationTest, ZkSolrClientTest, SolrPluginUtilsTest, TestIndexSearcher, TestCharFilters, TestReRankQParserPlugin, LukeRequestHandlerTest, TestPhraseSuggestions, CursorMarkTest, RankQueryTest, CopyFieldTest, DeleteShardTest, TestSchemaResource, TestManagedSchemaFieldResource, DistributedFacetPivotSmallTest, DeleteLastCustomShardedReplicaTest, ResourceLoaderTest, TestBadConfig, ChaosMonkeySafeLeaderTest, TestNonDefinedSimilarityFactory, TestNoOpRegenerator, TestSimpleQParserPlugin, OverseerStatusTest, DistributedDebugComponentTest, HdfsUnloadDistributedZkTest, RAMDirectoryFactoryTest, PreAnalyzedUpdateProcessorTest, TestManagedResource, HdfsChaosMonkeySafeLeaderTest, TestSchemaVersionResource, SpatialFilterTest, TestStressLucene, SearchHandlerTest, TestFieldSortValues, UnloadDistributedZkTest, CollectionsAPIDistributedZkTest, LeaderElectionTest, TestZkChroot, ShardRoutingCustomTest, AutoCommitTest, TestDistributedGrouping, TestStressReorder, TestRangeQuery, TestSolr4Spatial, StatsComponentTest, PeerSyncTest, BasicFunctionalityTest, DirectUpdateHandlerTest, HighlighterTest, SoftAutoCommitTest, CurrencyFieldXmlFileTest, SolrIndexSplitterTest, TestCoreDiscovery, AnalysisAfterCoreReloadTest, TestExtendedDismaxParser, SuggesterFSTTest, SolrRequestParserTest, TestFoldingMultitermQuery, DocValuesTest, SuggesterTest, TestTrie, SuggesterWFSTTest, TestCSVLoader, WordBreakSolrSpellCheckerTest, SchemaVersionSpecificBehaviorTest, TestPseudoReturnFields, QueryEqualityTest, DefaultValueUpdateProcessorTest, StatelessScriptUpdateProcessorFactoryTest, XsltUpdateRequestHandlerTest, StandardRequestHandlerTest, TestWriterPerf, DirectSolrSpellCheckerTest, TestOmitPositions, RequestHandlersTest, TestValueSourceCache, TestIndexingPerformance, RequiredFieldsTest, RegexBoostProcessorTest, TestCollationField, JSONWriterTest, JsonLoaderTest, TestPartialUpdateDeduplication, CSVRequestHandlerTest, TestBinaryResponseWriter, TestQuerySenderListener, AlternateDirectoryTest, SolrIndexConfigTest, TestMergePolicyConfig, MultiTermTest, TestDocSet, TestBinaryField, TestSearchPerf, NumericFieldsTest, ExternalFileFieldSortTest, TestPostingsSolrHighlighter, DirectSolrConnectionTest, TestLuceneMatchVersion, SpellPossibilityIteratorTest, TestSweetSpotSimilarityFactory, TestPerFieldSimilarity, TestLMDirichletSimilarityFactory, TestLMJelinekMercerSimilarityFactory, TestIBSimilarityFactory, TestBM25SimilarityFactory, TestDefaultSimilarityFactory, TestFastWriter, TestJmxMonitoredMap, TestFastOutputStream, PluginInfoTest, DateMathParserTest, PreAnalyzedFieldTest, PrimUtilsTest, SpellingQueryConverterTest, TestSolrJ, TestDocumentBuilder, SliceStateTest, UUIDFieldTest, CircularListTest, CursorPagingTest, SolrTestCaseJ4Test, TestCursorMarkWithoutUniqueKey, TestHighlightDedupGrouping, TestTolerantSearch, AliasIntegrationTest, AsyncMigrateRouteKeyTest, DeleteReplicaTest]
   [junit4] Completed on J1 in 45.89s, 1 test, 1 error <<< FAILURES!

[...truncated 241 lines...]
BUILD FAILED
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/build.xml:491: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/build.xml:471: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/build.xml:60: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/extra-targets.xml:39: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build.xml:189: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/common-build.xml:496: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/common-build.xml:1358: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/common-build.xml:961: There were test failures: 429 suites, 1772 tests, 1 error, 45 ignored (23 assumptions)

Total time: 96 minutes 21 seconds
Build step 'Invoke Ant' marked build as failure
[description-setter] Description set: Java: 32bit/jdk1.8.0_20 -server -XX:+UseSerialGC
Archiving artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any



Re: [JENKINS] Lucene-Solr-trunk-Linux (32bit/jdk1.8.0_20) - Build # 11279 - Failure!

Posted by Shalin Shekhar Mangar <sh...@apache.org>.
My bad, I fixed it.

On Thu, Sep 18, 2014 at 2:08 PM, Policeman Jenkins Server <
jenkins@thetaphi.de> wrote:

> Build: http://jenkins.thetaphi.de/job/Lucene-Solr-trunk-Linux/11279/
> Java: 32bit/jdk1.8.0_20 -server -XX:+UseSerialGC
>
> 1 tests failed.
> REGRESSION:  org.apache.solr.cloud.DeleteReplicaTest.testDistribSearch
>
> Error Message:
> java.lang.ClassCastException:
> org.apache.solr.common.params.CollectionParams$CollectionAction cannot be
> cast to java.lang.String
>
> Stack Trace:
> org.apache.solr.client.solrj.SolrServerException:
> java.lang.ClassCastException:
> org.apache.solr.common.params.CollectionParams$CollectionAction cannot be
> cast to java.lang.String
>         at
> __randomizedtesting.SeedInfo.seed([2301E08B7645B5A6:A2E76E93011AD59A]:0)
>         at
> org.apache.solr.client.solrj.impl.LBHttpSolrServer.doRequest(LBHttpSolrServer.java:381)
>         at
> org.apache.solr.client.solrj.impl.LBHttpSolrServer.request(LBHttpSolrServer.java:304)
>         at
> org.apache.solr.client.solrj.impl.CloudSolrServer.sendRequest(CloudSolrServer.java:874)
>         at
> org.apache.solr.client.solrj.impl.CloudSolrServer.requestWithRetryOnStaleState(CloudSolrServer.java:658)
>         at
> org.apache.solr.client.solrj.impl.CloudSolrServer.request(CloudSolrServer.java:601)
>         at
> org.apache.solr.cloud.DeleteReplicaTest.tryToRemoveOnlyIfDown(DeleteReplicaTest.java:154)
>         at
> org.apache.solr.cloud.DeleteReplicaTest.deleteLiveReplicaTest(DeleteReplicaTest.java:125)
>         at
> org.apache.solr.cloud.DeleteReplicaTest.doTest(DeleteReplicaTest.java:88)
>         at
> org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:869)
>         at sun.reflect.GeneratedMethodAccessor39.invoke(Unknown Source)
>         at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>         at java.lang.reflect.Method.invoke(Method.java:483)
>         at
> com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1618)
>         at
> com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:827)
>         at
> com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
>         at
> com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:877)
>         at
> com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
>         at
> org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
>         at
> org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
>         at
> com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
>         at
> org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
>         at
> org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
>         at
> org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
>         at
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>         at
> com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
>         at
> com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798)
>         at
> com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458)
>         at
> com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:836)
>         at
> com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:738)
>         at
> com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:772)
>         at
> com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:783)
>         at
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>         at
> com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
>         at
> org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
>         at
> org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
>         at
> com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
>         at
> com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
>         at
> com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
>         at
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>         at
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>         at
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>         at
> org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
>         at
> org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
>         at
> org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
>         at
> org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
>         at
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>         at
> com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
>         at java.lang.Thread.run(Thread.java:745)
> Caused by: java.lang.ClassCastException:
> org.apache.solr.common.params.CollectionParams$CollectionAction cannot be
> cast to java.lang.String
>         at
> org.apache.solr.common.params.MapSolrParams.getParams(MapSolrParams.java:43)
>         at
> org.apache.solr.common.params.ModifiableSolrParams.add(ModifiableSolrParams.java:117)
>         at
> org.apache.solr.common.params.ModifiableSolrParams.<init>(ModifiableSolrParams.java:56)
>         at
> org.apache.solr.client.solrj.impl.HttpSolrServer.createMethod(HttpSolrServer.java:273)
>         at
> org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:210)
>         at
> org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:206)
>         at
> org.apache.solr.client.solrj.impl.LBHttpSolrServer.doRequest(LBHttpSolrServer.java:343)
>         ... 47 more
>
>
>
>
> Build Log:
> [...truncated 11959 lines...]
>    [junit4] Suite: org.apache.solr.cloud.DeleteReplicaTest
>    [junit4]   2> Creating dataDir:
> /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.DeleteReplicaTest-2301E08B7645B5A6-001/init-core-data-001
>    [junit4]   2> 2134356 T6340 oas.SolrTestCaseJ4.buildSSLConfig
> Randomized ssl (true) and clientAuth (false)
>    [junit4]   2> 2134356 T6340
> oas.BaseDistributedSearchTestCase.initHostContext Setting hostContext
> system property: /b_y/tb
>    [junit4]   2> 2134358 T6340 oas.SolrTestCaseJ4.setUp ###Starting
> testDistribSearch
>    [junit4]   2> 2134359 T6340 oasc.ZkTestServer.run STARTING ZK TEST
> SERVER
>    [junit4]   1> client port:0.0.0.0/0.0.0.0:0
>    [junit4]   2> 2134359 T6341
> oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
>    [junit4]   2> 2134459 T6340 oasc.ZkTestServer.run start zk server on
> port:46175
>    [junit4]   2> 2134460 T6340
> oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default
> ZkCredentialsProvider
>    [junit4]   2> 2134461 T6340 oascc.ConnectionManager.waitForConnected
> Waiting for client to connect to ZooKeeper
>    [junit4]   2> 2134462 T6347 oascc.ConnectionManager.process Watcher
> org.apache.solr.common.cloud.ConnectionManager@1c461fb
> name:ZooKeeperConnection Watcher:127.0.0.1:46175 got event WatchedEvent
> state:SyncConnected type:None path:null path:null type:None
>    [junit4]   2> 2134463 T6340 oascc.ConnectionManager.waitForConnected
> Client is connected to ZooKeeper
>    [junit4]   2> 2134463 T6340 oascc.SolrZkClient.createZkACLProvider
> Using default ZkACLProvider
>    [junit4]   2> 2134463 T6340 oascc.SolrZkClient.makePath makePath: /solr
>    [junit4]   2> 2134465 T6340
> oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default
> ZkCredentialsProvider
>    [junit4]   2> 2134466 T6340 oascc.ConnectionManager.waitForConnected
> Waiting for client to connect to ZooKeeper
>    [junit4]   2> 2134467 T6349 oascc.ConnectionManager.process Watcher
> org.apache.solr.common.cloud.ConnectionManager@185797f
> name:ZooKeeperConnection Watcher:127.0.0.1:46175/solr got event
> WatchedEvent state:SyncConnected type:None path:null path:null type:None
>    [junit4]   2> 2134467 T6340 oascc.ConnectionManager.waitForConnected
> Client is connected to ZooKeeper
>    [junit4]   2> 2134468 T6340 oascc.SolrZkClient.createZkACLProvider
> Using default ZkACLProvider
>    [junit4]   2> 2134468 T6340 oascc.SolrZkClient.makePath makePath:
> /collections/collection1
>    [junit4]   2> 2134469 T6340 oascc.SolrZkClient.makePath makePath:
> /collections/collection1/shards
>    [junit4]   2> 2134470 T6340 oascc.SolrZkClient.makePath makePath:
> /collections/control_collection
>    [junit4]   2> 2134471 T6340 oascc.SolrZkClient.makePath makePath:
> /collections/control_collection/shards
>    [junit4]   2> 2134473 T6340 oasc.AbstractZkTestCase.putConfig put
> /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml
> to /configs/conf1/solrconfig.xml
>    [junit4]   2> 2134473 T6340 oascc.SolrZkClient.makePath makePath:
> /configs/conf1/solrconfig.xml
>    [junit4]   2> 2134475 T6340 oasc.AbstractZkTestCase.putConfig put
> /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/schema.xml
> to /configs/conf1/schema.xml
>    [junit4]   2> 2134475 T6340 oascc.SolrZkClient.makePath makePath:
> /configs/conf1/schema.xml
>    [junit4]   2> 2134477 T6340 oasc.AbstractZkTestCase.putConfig put
> /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml
> to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
>    [junit4]   2> 2134477 T6340 oascc.SolrZkClient.makePath makePath:
> /configs/conf1/solrconfig.snippet.randomindexconfig.xml
>    [junit4]   2> 2134478 T6340 oasc.AbstractZkTestCase.putConfig put
> /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/stopwords.txt
> to /configs/conf1/stopwords.txt
>    [junit4]   2> 2134478 T6340 oascc.SolrZkClient.makePath makePath:
> /configs/conf1/stopwords.txt
>    [junit4]   2> 2134480 T6340 oasc.AbstractZkTestCase.putConfig put
> /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/protwords.txt
> to /configs/conf1/protwords.txt
>    [junit4]   2> 2134480 T6340 oascc.SolrZkClient.makePath makePath:
> /configs/conf1/protwords.txt
>    [junit4]   2> 2134481 T6340 oasc.AbstractZkTestCase.putConfig put
> /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/currency.xml
> to /configs/conf1/currency.xml
>    [junit4]   2> 2134481 T6340 oascc.SolrZkClient.makePath makePath:
> /configs/conf1/currency.xml
>    [junit4]   2> 2134483 T6340 oasc.AbstractZkTestCase.putConfig put
> /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml
> to /configs/conf1/enumsConfig.xml
>    [junit4]   2> 2134483 T6340 oascc.SolrZkClient.makePath makePath:
> /configs/conf1/enumsConfig.xml
>    [junit4]   2> 2134484 T6340 oasc.AbstractZkTestCase.putConfig put
> /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json
> to /configs/conf1/open-exchange-rates.json
>    [junit4]   2> 2134485 T6340 oascc.SolrZkClient.makePath makePath:
> /configs/conf1/open-exchange-rates.json
>    [junit4]   2> 2134486 T6340 oasc.AbstractZkTestCase.putConfig put
> /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt
> to /configs/conf1/mapping-ISOLatin1Accent.txt
>    [junit4]   2> 2134486 T6340 oascc.SolrZkClient.makePath makePath:
> /configs/conf1/mapping-ISOLatin1Accent.txt
>    [junit4]   2> 2134487 T6340 oasc.AbstractZkTestCase.putConfig put
> /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt
> to /configs/conf1/old_synonyms.txt
>    [junit4]   2> 2134488 T6340 oascc.SolrZkClient.makePath makePath:
> /configs/conf1/old_synonyms.txt
>    [junit4]   2> 2134489 T6340 oasc.AbstractZkTestCase.putConfig put
> /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/synonyms.txt
> to /configs/conf1/synonyms.txt
>    [junit4]   2> 2134489 T6340 oascc.SolrZkClient.makePath makePath:
> /configs/conf1/synonyms.txt
>    [junit4]   2> 2134491 T6340
> oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default
> ZkCredentialsProvider
>    [junit4]   2> 2134492 T6340 oascc.ConnectionManager.waitForConnected
> Waiting for client to connect to ZooKeeper
>    [junit4]   2> 2134492 T6351 oascc.ConnectionManager.process Watcher
> org.apache.solr.common.cloud.ConnectionManager@172cf96
> name:ZooKeeperConnection Watcher:127.0.0.1:46175/solr got event
> WatchedEvent state:SyncConnected type:None path:null path:null type:None
>    [junit4]   2> 2134492 T6340 oascc.ConnectionManager.waitForConnected
> Client is connected to ZooKeeper
>    [junit4]   2> 2134493 T6340 oascc.SolrZkClient.createZkACLProvider
> Using default ZkACLProvider
>    [junit4]   2> 2134753 T6340 oejs.Server.doStart jetty-8.1.10.v20130312
>    [junit4]   2> 2134756 T6340 oejus.SslContextFactory.doStart Enabled
> Protocols [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2] of [SSLv2Hello,
> SSLv3, TLSv1, TLSv1.1, TLSv1.2]
>    [junit4]   2> 2134760 T6340 oejs.AbstractConnector.doStart Started
> SslSelectChannelConnector@127.0.0.1:45141
>    [junit4]   2> 2134762 T6340 oass.SolrDispatchFilter.init
> SolrDispatchFilter.init()
>    [junit4]   2> 2134762 T6340 oasc.SolrResourceLoader.locateSolrHome JNDI
> not configured for solr (NoInitialContextEx)
>    [junit4]   2> 2134762 T6340 oasc.SolrResourceLoader.locateSolrHome
> using system property solr.solr.home:
> /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.DeleteReplicaTest-2301E08B7645B5A6-001/tempDir-002
>    [junit4]   2> 2134763 T6340 oasc.SolrResourceLoader.<init> new
> SolrResourceLoader for directory:
> '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.DeleteReplicaTest-2301E08B7645B5A6-001/tempDir-002/'
>    [junit4]   2> 2134774 T6340 oasc.ConfigSolr.fromFile Loading container
> configuration from
> /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.DeleteReplicaTest-2301E08B7645B5A6-001/tempDir-002/solr.xml
>    [junit4]   2> 2134779 T6340 oasc.CorePropertiesLocator.<init>
> Config-defined core root directory:
> /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.DeleteReplicaTest-2301E08B7645B5A6-001/tempDir-002
>    [junit4]   2> 2134780 T6340 oasc.CoreContainer.<init> New CoreContainer
> 5474051
>    [junit4]   2> 2134780 T6340 oasc.CoreContainer.load Loading cores into
> CoreContainer
> [instanceDir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.DeleteReplicaTest-2301E08B7645B5A6-001/tempDir-002/]
>    [junit4]   2> 2134781 T6340 oashc.HttpShardHandlerFactory.getParameter
> Setting socketTimeout to: 90000
>    [junit4]   2> 2134781 T6340 oashc.HttpShardHandlerFactory.getParameter
> Setting urlScheme to:
>    [junit4]   2> 2134782 T6340 oashc.HttpShardHandlerFactory.getParameter
> Setting connTimeout to: 15000
>    [junit4]   2> 2134782 T6340 oashc.HttpShardHandlerFactory.getParameter
> Setting maxConnectionsPerHost to: 20
>    [junit4]   2> 2134782 T6340 oashc.HttpShardHandlerFactory.getParameter
> Setting corePoolSize to: 0
>    [junit4]   2> 2134783 T6340 oashc.HttpShardHandlerFactory.getParameter
> Setting maximumPoolSize to: 2147483647
>    [junit4]   2> 2134783 T6340 oashc.HttpShardHandlerFactory.getParameter
> Setting maxThreadIdleTime to: 5
>    [junit4]   2> 2134783 T6340 oashc.HttpShardHandlerFactory.getParameter
> Setting sizeOfQueue to: -1
>    [junit4]   2> 2134783 T6340 oashc.HttpShardHandlerFactory.getParameter
> Setting fairnessPolicy to: false
>    [junit4]   2> 2134785 T6340 oasu.UpdateShardHandler.<init> Creating
> UpdateShardHandler HTTP client with params:
> socketTimeout=340000&connTimeout=45000&retry=false
>    [junit4]   2> 2134785 T6340 oasl.LogWatcher.createWatcher SLF4J impl is
> org.slf4j.impl.Log4jLoggerFactory
>    [junit4]   2> 2134786 T6340 oasl.LogWatcher.newRegisteredLogWatcher
> Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
>    [junit4]   2> 2134786 T6340 oasc.CoreContainer.load Host Name: 127.0.0.1
>    [junit4]   2> 2134786 T6340 oasc.ZkContainer.initZooKeeper Zookeeper
> client=127.0.0.1:46175/solr
>    [junit4]   2> 2134787 T6340 oasc.ZkController.checkChrootPath zkHost
> includes chroot
>    [junit4]   2> 2134787 T6340
> oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default
> ZkCredentialsProvider
>    [junit4]   2> 2134788 T6340 oascc.ConnectionManager.waitForConnected
> Waiting for client to connect to ZooKeeper
>    [junit4]   2> 2134788 T6362 oascc.ConnectionManager.process Watcher
> org.apache.solr.common.cloud.ConnectionManager@14162d0
> name:ZooKeeperConnection Watcher:127.0.0.1:46175 got event WatchedEvent
> state:SyncConnected type:None path:null path:null type:None
>    [junit4]   2> 2134789 T6340 oascc.ConnectionManager.waitForConnected
> Client is connected to ZooKeeper
>    [junit4]   2> 2134789 T6340 oascc.SolrZkClient.createZkACLProvider
> Using default ZkACLProvider
>    [junit4]   2> 2134790 T6340 oascc.ConnectionManager.waitForConnected
> Waiting for client to connect to ZooKeeper
>    [junit4]   2> 2134791 T6364 oascc.ConnectionManager.process Watcher
> org.apache.solr.common.cloud.ConnectionManager@14aef6a
> name:ZooKeeperConnection Watcher:127.0.0.1:46175/solr got event
> WatchedEvent state:SyncConnected type:None path:null path:null type:None
>    [junit4]   2> 2134791 T6340 oascc.ConnectionManager.waitForConnected
> Client is connected to ZooKeeper
>    [junit4]   2> 2134792 T6340 oascc.SolrZkClient.makePath makePath:
> /overseer/queue
>    [junit4]   2> 2134794 T6340 oascc.SolrZkClient.makePath makePath:
> /overseer/collection-queue-work
>    [junit4]   2> 2134795 T6340 oascc.SolrZkClient.makePath makePath:
> /overseer/collection-map-running
>    [junit4]   2> 2134796 T6340 oascc.SolrZkClient.makePath makePath:
> /overseer/collection-map-completed
>    [junit4]   2> 2134798 T6340 oascc.SolrZkClient.makePath makePath:
> /overseer/collection-map-failure
>    [junit4]   2> 2134799 T6340 oascc.SolrZkClient.makePath makePath:
> /live_nodes
>    [junit4]   2> 2134800 T6340 oasc.ZkController.createEphemeralLiveNode
> Register node as live in ZooKeeper:/live_nodes/127.0.0.1:45141_b_y%2Ftb
>    [junit4]   2> 2134801 T6340 oascc.SolrZkClient.makePath makePath:
> /live_nodes/127.0.0.1:45141_b_y%2Ftb
>    [junit4]   2> 2134802 T6340 oascc.SolrZkClient.makePath makePath:
> /overseer_elect
>    [junit4]   2> 2134803 T6340 oascc.SolrZkClient.makePath makePath:
> /overseer_elect/election
>    [junit4]   2> 2134804 T6340 oasc.Overseer.close Overseer (id=null)
> closing
>    [junit4]   2> 2134805 T6340
> oasc.OverseerElectionContext.runLeaderProcess I am going to be the leader
> 127.0.0.1:45141_b_y%2Ftb
>    [junit4]   2> 2134806 T6340 oascc.SolrZkClient.makePath makePath:
> /overseer_elect/leader
>    [junit4]   2> 2134807 T6340 oasc.Overseer.start Overseer
> (id=92473192818212868-127.0.0.1:45141_b_y%2Ftb-n_0000000000) starting
>    [junit4]   2> 2134808 T6340 oascc.SolrZkClient.makePath makePath:
> /overseer/queue-work
>    [junit4]   2> 2134812 T6340
> oasc.OverseerAutoReplicaFailoverThread.<init> Starting
> OverseerAutoReplicaFailoverThread autoReplicaFailoverWorkLoopDelay=10000
> autoReplicaFailoverWaitAfterExpiration=10000
> autoReplicaFailoverBadNodeExpiration=60000
>    [junit4]   2> 2134812 T6366 oasc.OverseerCollectionProcessor.run
> Process current queue of collection creations
>    [junit4]   2> 2134813 T6365 oasc.Overseer$ClusterStateUpdater.run
> Starting to work on the main queue
>    [junit4]   2> 2134814 T6340 oascc.SolrZkClient.makePath makePath:
> /clusterstate.json
>    [junit4]   2> 2134815 T6340 oascc.SolrZkClient.makePath makePath:
> /aliases.json
>    [junit4]   2> 2134820 T6340
> oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster
> state from ZooKeeper...
>    [junit4]   2> 2134823 T6340 oasc.CorePropertiesLocator.discover Looking
> for core definitions underneath
> /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.DeleteReplicaTest-2301E08B7645B5A6-001/tempDir-002
>    [junit4]   2> 2134829 T6340 oasc.CorePropertiesLocator.discoverUnder
> Found core conf in
> /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.DeleteReplicaTest-2301E08B7645B5A6-001/tempDir-002/conf/
>    [junit4]   2> 2134959 T6340 oasc.CorePropertiesLocator.discover Found 1
> core definitions
>    [junit4]   2> 2134960 T6340 oass.SolrDispatchFilter.init
> user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1
>    [junit4]   2> 2134961 T6340 oass.SolrDispatchFilter.init
> SolrDispatchFilter.init() done
>    [junit4]   2> 2134964 T6340
> oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default
> ZkCredentialsProvider
>    [junit4]   2> 2134970 T6340 oascc.ConnectionManager.waitForConnected
> Waiting for client to connect to ZooKeeper
>    [junit4]   2> 2134970 T6370 oascc.ConnectionManager.process Watcher
> org.apache.solr.common.cloud.ConnectionManager@15c1fc0
> name:ZooKeeperConnection Watcher:127.0.0.1:46175/solr got event
> WatchedEvent state:SyncConnected type:None path:null path:null type:None
>    [junit4]   2> 2134971 T6340 oascc.ConnectionManager.waitForConnected
> Client is connected to ZooKeeper
>    [junit4]   2> 2134971 T6340 oascc.SolrZkClient.createZkACLProvider
> Using default ZkACLProvider
>    [junit4]   2> 2134972 T6340
> oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster
> state from ZooKeeper...
>    [junit4]   2> 2134974 T6340 oasc.ChaosMonkey.monkeyLog monkey: init -
> expire sessions:false cause connection loss:false
>    [junit4]   2> 2134974 T6340
> oasc.AbstractFullDistribZkTestBase.createJettys Creating collection1 with
> stateFormat=2
>    [junit4]   2> 2134974 T6340
> oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default
> ZkCredentialsProvider
>    [junit4]   2> 2134975 T6340 oascc.ConnectionManager.waitForConnected
> Waiting for client to connect to ZooKeeper
>    [junit4]   2> 2134975 T6372 oascc.ConnectionManager.process Watcher
> org.apache.solr.common.cloud.ConnectionManager@19fb5c9
> name:ZooKeeperConnection Watcher:127.0.0.1:46175/solr got event
> WatchedEvent state:SyncConnected type:None path:null path:null type:None
>    [junit4]   2> 2134976 T6340 oascc.ConnectionManager.waitForConnected
> Client is connected to ZooKeeper
>    [junit4]   2> 2134976 T6340 oascc.SolrZkClient.createZkACLProvider
> Using default ZkACLProvider
>    [junit4]   2> 2134977 T6373
> oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on
> path: /overseer/queue state: SyncConnected type NodeChildrenChanged
>    [junit4]   2> 2134979 T6365
> oasc.Overseer$ClusterStateUpdater.buildCollection building a new
> collection: collection1
>    [junit4]   2> 2134979 T6365
> oasc.Overseer$ClusterStateUpdater.createCollection Create collection
> collection1 with shards [shard1, shard2]
>    [junit4]   2> 2134979 T6365
> oasc.Overseer$ClusterStateUpdater.createCollection state version
> collection1 2
>    [junit4]   2> 2134980 T6365
> oasc.Overseer$ClusterStateUpdater.updateZkStates going to create_collection
> /collections/collection1/state.json
>    [junit4]   2> 2134981 T6373 oascc.ZkStateReader$2.process A cluster
> state change: WatchedEvent state:SyncConnected type:NodeDataChanged
> path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
>    [junit4]   2> 2134981 T6374 oascc.ZkStateReader$2.process A cluster
> state change: WatchedEvent state:SyncConnected type:NodeDataChanged
> path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
>    [junit4]   2> 2135272 T6340
> oasc.AbstractFullDistribZkTestBase.createJettys create jetty 1
>    [junit4]   2> 2135273 T6340 oejs.Server.doStart jetty-8.1.10.v20130312
>    [junit4]   2> 2135276 T6340 oejus.SslContextFactory.doStart Enabled
> Protocols [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2] of [SSLv2Hello,
> SSLv3, TLSv1, TLSv1.1, TLSv1.2]
>    [junit4]   2> 2135277 T6340 oejs.AbstractConnector.doStart Started
> SslSelectChannelConnector@127.0.0.1:37160
>    [junit4]   2> 2135278 T6340 oass.SolrDispatchFilter.init
> SolrDispatchFilter.init()
>    [junit4]   2> 2135279 T6340 oasc.SolrResourceLoader.locateSolrHome JNDI
> not configured for solr (NoInitialContextEx)
>    [junit4]   2> 2135279 T6340 oasc.SolrResourceLoader.locateSolrHome
> using system property solr.solr.home:
> /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.DeleteReplicaTest-2301E08B7645B5A6-001/tempDir-003
>    [junit4]   2> 2135279 T6340 oasc.SolrResourceLoader.<init> new
> SolrResourceLoader for directory:
> '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.DeleteReplicaTest-2301E08B7645B5A6-001/tempDir-003/'
>    [junit4]   2> 2135291 T6340 oasc.ConfigSolr.fromFile Loading container
> configuration from
> /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.DeleteReplicaTest-2301E08B7645B5A6-001/tempDir-003/solr.xml
>    [junit4]   2> 2135297 T6340 oasc.CorePropertiesLocator.<init>
> Config-defined core root directory:
> /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.DeleteReplicaTest-2301E08B7645B5A6-001/tempDir-003
>    [junit4]   2> 2135297 T6340 oasc.CoreContainer.<init> New CoreContainer
> 2666391
>    [junit4]   2> 2135298 T6340 oasc.CoreContainer.load Loading cores into
> CoreContainer
> [instanceDir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.DeleteReplicaTest-2301E08B7645B5A6-001/tempDir-003/]
>    [junit4]   2> 2135298 T6340 oashc.HttpShardHandlerFactory.getParameter
> Setting socketTimeout to: 90000
>    [junit4]   2> 2135299 T6340 oashc.HttpShardHandlerFactory.getParameter
> Setting urlScheme to:
>    [junit4]   2> 2135299 T6340 oashc.HttpShardHandlerFactory.getParameter
> Setting connTimeout to: 15000
>    [junit4]   2> 2135299 T6340 oashc.HttpShardHandlerFactory.getParameter
> Setting maxConnectionsPerHost to: 20
>    [junit4]   2> 2135300 T6340 oashc.HttpShardHandlerFactory.getParameter
> Setting corePoolSize to: 0
>    [junit4]   2> 2135300 T6340 oashc.HttpShardHandlerFactory.getParameter
> Setting maximumPoolSize to: 2147483647
>    [junit4]   2> 2135300 T6340 oashc.HttpShardHandlerFactory.getParameter
> Setting maxThreadIdleTime to: 5
>    [junit4]   2> 2135301 T6340 oashc.HttpShardHandlerFactory.getParameter
> Setting sizeOfQueue to: -1
>    [junit4]   2> 2135301 T6340 oashc.HttpShardHandlerFactory.getParameter
> Setting fairnessPolicy to: false
>    [junit4]   2> 2135303 T6340 oasu.UpdateShardHandler.<init> Creating
> UpdateShardHandler HTTP client with params:
> socketTimeout=340000&connTimeout=45000&retry=false
>    [junit4]   2> 2135303 T6340 oasl.LogWatcher.createWatcher SLF4J impl is
> org.slf4j.impl.Log4jLoggerFactory
>    [junit4]   2> 2135304 T6340 oasl.LogWatcher.newRegisteredLogWatcher
> Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
>    [junit4]   2> 2135304 T6340 oasc.CoreContainer.load Host Name: 127.0.0.1
>    [junit4]   2> 2135305 T6340 oasc.ZkContainer.initZooKeeper Zookeeper
> client=127.0.0.1:46175/solr
>    [junit4]   2> 2135305 T6340 oasc.ZkController.checkChrootPath zkHost
> includes chroot
>    [junit4]   2> 2135305 T6340
> oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default
> ZkCredentialsProvider
>    [junit4]   2> 2135306 T6340 oascc.ConnectionManager.waitForConnected
> Waiting for client to connect to ZooKeeper
>    [junit4]   2> 2135307 T6385 oascc.ConnectionManager.process Watcher
> org.apache.solr.common.cloud.ConnectionManager@ed9821
> name:ZooKeeperConnection Watcher:127.0.0.1:46175 got event WatchedEvent
> state:SyncConnected type:None path:null path:null type:None
>    [junit4]   2> 2135307 T6340 oascc.ConnectionManager.waitForConnected
> Client is connected to ZooKeeper
>    [junit4]   2> 2135307 T6340 oascc.SolrZkClient.createZkACLProvider
> Using default ZkACLProvider
>    [junit4]   2> 2135309 T6340 oascc.ConnectionManager.waitForConnected
> Waiting for client to connect to ZooKeeper
>    [junit4]   2> 2135311 T6387 oascc.ConnectionManager.process Watcher
> org.apache.solr.common.cloud.ConnectionManager@166cea3
> name:ZooKeeperConnection Watcher:127.0.0.1:46175/solr got event
> WatchedEvent state:SyncConnected type:None path:null path:null type:None
>    [junit4]   2> 2135312 T6340 oascc.ConnectionManager.waitForConnected
> Client is connected to ZooKeeper
>    [junit4]   2> 2135319 T6340
> oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster
> state from ZooKeeper...
>    [junit4]   2> 2136323 T6340 oasc.ZkController.createEphemeralLiveNode
> Register node as live in ZooKeeper:/live_nodes/127.0.0.1:37160_b_y%2Ftb
>    [junit4]   2> 2136324 T6340 oascc.SolrZkClient.makePath makePath:
> /live_nodes/127.0.0.1:37160_b_y%2Ftb
>    [junit4]   2> 2136326 T6340 oasc.Overseer.close Overseer (id=null)
> closing
>    [junit4]   2> 2136327 T6340 oasc.CorePropertiesLocator.discover Looking
> for core definitions underneath
> /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.DeleteReplicaTest-2301E08B7645B5A6-001/tempDir-003
>    [junit4]   2> 2136333 T6340 oasc.CorePropertiesLocator.discoverUnder
> Found core conf in
> /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.DeleteReplicaTest-2301E08B7645B5A6-001/tempDir-003/conf/
>    [junit4]   2> 2136456 T6340 oasc.CorePropertiesLocator.discover Found 1
> core definitions
>    [junit4]   2> 2136457 T6340 oass.SolrDispatchFilter.init
> user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1
>    [junit4]   2> 2136457 T6340 oass.SolrDispatchFilter.init
> SolrDispatchFilter.init() done
>    [junit4]   2> 2136717 T6340
> oasc.AbstractFullDistribZkTestBase.createJettys create jetty 2
>    [junit4]   2> 2136718 T6340 oejs.Server.doStart jetty-8.1.10.v20130312
>    [junit4]   2> 2136721 T6340 oejus.SslContextFactory.doStart Enabled
> Protocols [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2] of [SSLv2Hello,
> SSLv3, TLSv1, TLSv1.1, TLSv1.2]
>    [junit4]   2> 2136722 T6340 oejs.AbstractConnector.doStart Started
> SslSelectChannelConnector@127.0.0.1:51660
>    [junit4]   2> 2136723 T6340 oass.SolrDispatchFilter.init
> SolrDispatchFilter.init()
>    [junit4]   2> 2136724 T6340 oasc.SolrResourceLoader.locateSolrHome JNDI
> not configured for solr (NoInitialContextEx)
>    [junit4]   2> 2136724 T6340 oasc.SolrResourceLoader.locateSolrHome
> using system property solr.solr.home:
> /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.DeleteReplicaTest-2301E08B7645B5A6-001/tempDir-004
>    [junit4]   2> 2136724 T6340 oasc.SolrResourceLoader.<init> new
> SolrResourceLoader for directory:
> '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.DeleteReplicaTest-2301E08B7645B5A6-001/tempDir-004/'
>    [junit4]   2> 2136740 T6340 oasc.ConfigSolr.fromFile Loading container
> configuration from
> /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.DeleteReplicaTest-2301E08B7645B5A6-001/tempDir-004/solr.xml
>    [junit4]   2> 2136746 T6340 oasc.CorePropertiesLocator.<init>
> Config-defined core root directory:
> /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.DeleteReplicaTest-2301E08B7645B5A6-001/tempDir-004
>    [junit4]   2> 2136747 T6340 oasc.CoreContainer.<init> New CoreContainer
> 6500733
>    [junit4]   2> 2136747 T6340 oasc.CoreContainer.load Loading cores into
> CoreContainer
> [instanceDir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.DeleteReplicaTest-2301E08B7645B5A6-001/tempDir-004/]
>    [junit4]   2> 2136748 T6340 oashc.HttpShardHandlerFactory.getParameter
> Setting socketTimeout to: 90000
>    [junit4]   2> 2136748 T6340 oashc.HttpShardHandlerFactory.getParameter
> Setting urlScheme to:
>    [junit4]   2> 2136748 T6340 oashc.HttpShardHandlerFactory.getParameter
> Setting connTimeout to: 15000
>    [junit4]   2> 2136749 T6340 oashc.HttpShardHandlerFactory.getParameter
> Setting maxConnectionsPerHost to: 20
>    [junit4]   2> 2136749 T6340 oashc.HttpShardHandlerFactory.getParameter
> Setting corePoolSize to: 0
>    [junit4]   2> 2136749 T6340 oashc.HttpShardHandlerFactory.getParameter
> Setting maximumPoolSize to: 2147483647
>    [junit4]   2> 2136750 T6340 oashc.HttpShardHandlerFactory.getParameter
> Setting maxThreadIdleTime to: 5
>    [junit4]   2> 2136750 T6340 oashc.HttpShardHandlerFactory.getParameter
> Setting sizeOfQueue to: -1
>    [junit4]   2> 2136750 T6340 oashc.HttpShardHandlerFactory.getParameter
> Setting fairnessPolicy to: false
>    [junit4]   2> 2136751 T6340 oasu.UpdateShardHandler.<init> Creating
> UpdateShardHandler HTTP client with params:
> socketTimeout=340000&connTimeout=45000&retry=false
>    [junit4]   2> 2136752 T6340 oasl.LogWatcher.createWatcher SLF4J impl is
> org.slf4j.impl.Log4jLoggerFactory
>    [junit4]   2> 2136753 T6340 oasl.LogWatcher.newRegisteredLogWatcher
> Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
>    [junit4]   2> 2136753 T6340 oasc.CoreContainer.load Host Name: 127.0.0.1
>    [junit4]   2> 2136753 T6340 oasc.ZkContainer.initZooKeeper Zookeeper
> client=127.0.0.1:46175/solr
>    [junit4]   2> 2136754 T6340 oasc.ZkController.checkChrootPath zkHost
> includes chroot
>    [junit4]   2> 2136754 T6340
> oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default
> ZkCredentialsProvider
>    [junit4]   2> 2136755 T6340 oascc.ConnectionManager.waitForConnected
> Waiting for client to connect to ZooKeeper
>    [junit4]   2> 2136756 T6400 oascc.ConnectionManager.process Watcher
> org.apache.solr.common.cloud.ConnectionManager@616b91
> name:ZooKeeperConnection Watcher:127.0.0.1:46175 got event WatchedEvent
> state:SyncConnected type:None path:null path:null type:None
>    [junit4]   2> 2136756 T6340 oascc.ConnectionManager.waitForConnected
> Client is connected to ZooKeeper
>    [junit4]   2> 2136756 T6340 oascc.SolrZkClient.createZkACLProvider
> Using default ZkACLProvider
>    [junit4]   2> 2136757 T6340 oascc.ConnectionManager.waitForConnected
> Waiting for client to connect to ZooKeeper
>    [junit4]   2> 2136758 T6402 oascc.ConnectionManager.process Watcher
> org.apache.solr.common.cloud.ConnectionManager@240bd5
> name:ZooKeeperConnection Watcher:127.0.0.1:46175/solr got event
> WatchedEvent state:SyncConnected type:None path:null path:null type:None
>    [junit4]   2> 2136758 T6340 oascc.ConnectionManager.waitForConnected
> Client is connected to ZooKeeper
>    [junit4]   2> 2136761 T6340
> oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster
> state from ZooKeeper...
>    [junit4]   2> 2137764 T6340 oasc.ZkController.createEphemeralLiveNode
> Register node as live in ZooKeeper:/live_nodes/127.0.0.1:51660_b_y%2Ftb
>    [junit4]   2> 2137765 T6340 oascc.SolrZkClient.makePath makePath:
> /live_nodes/127.0.0.1:51660_b_y%2Ftb
>    [junit4]   2> 2137767 T6340 oasc.Overseer.close Overseer (id=null)
> closing
>    [junit4]   2> 2137769 T6340 oasc.CorePropertiesLocator.discover Looking
> for core definitions underneath
> /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.DeleteReplicaTest-2301E08B7645B5A6-001/tempDir-004
>    [junit4]   2> 2137774 T6340 oasc.CorePropertiesLocator.discoverUnder
> Found core conf in
> /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.DeleteReplicaTest-2301E08B7645B5A6-001/tempDir-004/conf/
>    [junit4]   2> 2137900 T6340 oasc.CorePropertiesLocator.discover Found 1
> core definitions
>    [junit4]   2> 2137901 T6340 oass.SolrDispatchFilter.init
> user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1
>    [junit4]   2> 2137902 T6340 oass.SolrDispatchFilter.init
> SolrDispatchFilter.init() done
>    [junit4]   2> 2138163 T6340
> oasc.AbstractFullDistribZkTestBase.createJettys create jetty 3
>    [junit4]   2> 2138164 T6340 oejs.Server.doStart jetty-8.1.10.v20130312
>    [junit4]   2> 2138166 T6340 oejus.SslContextFactory.doStart Enabled
> Protocols [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2] of [SSLv2Hello,
> SSLv3, TLSv1, TLSv1.1, TLSv1.2]
>    [junit4]   2> 2138167 T6340 oejs.AbstractConnector.doStart Started
> SslSelectChannelConnector@127.0.0.1:60980
>    [junit4]   2> 2138169 T6340 oass.SolrDispatchFilter.init
> SolrDispatchFilter.init()
>    [junit4]   2> 2138169 T6340 oasc.SolrResourceLoader.locateSolrHome JNDI
> not configured for solr (NoInitialContextEx)
>    [junit4]   2> 2138170 T6340 oasc.SolrResourceLoader.locateSolrHome
> using system property solr.solr.home:
> /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.DeleteReplicaTest-2301E08B7645B5A6-001/tempDir-005
>    [junit4]   2> 2138170 T6340 oasc.SolrResourceLoader.<init> new
> SolrResourceLoader for directory:
> '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.DeleteReplicaTest-2301E08B7645B5A6-001/tempDir-005/'
>    [junit4]   2> 2138181 T6340 oasc.ConfigSolr.fromFile Loading container
> configuration from
> /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.DeleteReplicaTest-2301E08B7645B5A6-001/tempDir-005/solr.xml
>    [junit4]   2> 2138204 T6340 oasc.CorePropertiesLocator.<init>
> Config-defined core root directory:
> /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.DeleteReplicaTest-2301E08B7645B5A6-001/tempDir-005
>    [junit4]   2> 2138205 T6340 oasc.CoreContainer.<init> New CoreContainer
> 31173560
>    [junit4]   2> 2138205 T6340 oasc.CoreContainer.load Loading cores into
> CoreContainer
> [instanceDir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.DeleteReplicaTest-2301E08B7645B5A6-001/tempDir-005/]
>    [junit4]   2> 2138205 T6340 oashc.HttpShardHandlerFactory.getParameter
> Setting socketTimeout to: 90000
>    [junit4]   2> 2138206 T6340 oashc.HttpShardHandlerFactory.getParameter
> Setting urlScheme to:
>    [junit4]   2> 2138206 T6340 oashc.HttpShardHandlerFactory.getParameter
> Setting connTimeout to: 15000
>    [junit4]   2> 2138206 T6340 oashc.HttpShardHandlerFactory.getParameter
> Setting maxConnectionsPerHost to: 20
>    [junit4]   2> 2138207 T6340 oashc.HttpShardHandlerFactory.getParameter
> Setting corePoolSize to: 0
>    [junit4]   2> 2138207 T6340 oashc.HttpShardHandlerFactory.getParameter
> Setting maximumPoolSize to: 2147483647
>    [junit4]   2> 2138207 T6340 oashc.HttpShardHandlerFactory.getParameter
> Setting maxThreadIdleTime to: 5
>    [junit4]   2> 2138208 T6340 oashc.HttpShardHandlerFactory.getParameter
> Setting sizeOfQueue to: -1
>    [junit4]   2> 2138208 T6340 oashc.HttpShardHandlerFactory.getParameter
> Setting fairnessPolicy to: false
>    [junit4]   2> 2138209 T6340 oasu.UpdateShardHandler.<init> Creating
> UpdateShardHandler HTTP client with params:
> socketTimeout=340000&connTimeout=45000&retry=false
>    [junit4]   2> 2138210 T6340 oasl.LogWatcher.createWatcher SLF4J impl is
> org.slf4j.impl.Log4jLoggerFactory
>    [junit4]   2> 2138210 T6340 oasl.LogWatcher.newRegisteredLogWatcher
> Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
>    [junit4]   2> 2138210 T6340 oasc.CoreContainer.load Host Name: 127.0.0.1
>    [junit4]   2> 2138211 T6340 oasc.ZkContainer.initZooKeeper Zookeeper
> client=127.0.0.1:46175/solr
>    [junit4]   2> 2138211 T6340 oasc.ZkController.checkChrootPath zkHost
> includes chroot
>    [junit4]   2> 2138212 T6340
> oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default
> ZkCredentialsProvider
>    [junit4]   2> 2138212 T6340 oascc.ConnectionManager.waitForConnected
> Waiting for client to connect to ZooKeeper
>    [junit4]   2> 2138213 T6415 oascc.ConnectionManager.process Watcher
> org.apache.solr.common.cloud.ConnectionManager@5fdb48
> name:ZooKeeperConnection Watcher:127.0.0.1:46175 got event WatchedEvent
> state:SyncConnected type:None path:null path:null type:None
>    [junit4]   2> 2138213 T6340 oascc.ConnectionManager.waitForConnected
> Client is connected to ZooKeeper
>    [junit4]   2> 2138213 T6340 oascc.SolrZkClient.createZkACLProvider
> Using default ZkACLProvider
>    [junit4]   2> 2138215 T6340 oascc.ConnectionManager.waitForConnected
> Waiting for client to connect to ZooKeeper
>    [junit4]   2> 2138215 T6417 oascc.ConnectionManager.process Watcher
> org.apache.solr.common.cloud.ConnectionManager@bec75
> name:ZooKeeperConnection Watcher:127.0.0.1:46175/solr got event
> WatchedEvent state:SyncConnected type:None path:null path:null type:None
>    [junit4]   2> 2138215 T6340 oascc.ConnectionManager.waitForConnected
> Client is connected to ZooKeeper
>    [junit4]   2> 2138218 T6340
> oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster
> state from ZooKeeper...
>    [junit4]   2> 2139221 T6340 oasc.ZkController.createEphemeralLiveNode
> Register node as live in ZooKeeper:/live_nodes/127.0.0.1:60980_b_y%2Ftb
>    [junit4]   2> 2139222 T6340 oascc.SolrZkClient.makePath makePath:
> /live_nodes/127.0.0.1:60980_b_y%2Ftb
>    [junit4]   2> 2139225 T6340 oasc.Overseer.close Overseer (id=null)
> closing
>    [junit4]   2> 2139227 T6340 oasc.CorePropertiesLocator.discover Looking
> for core definitions underneath
> /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.DeleteReplicaTest-2301E08B7645B5A6-001/tempDir-005
>    [junit4]   2> 2139237 T6340 oasc.CorePropertiesLocator.discoverUnder
> Found core conf in
> /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.DeleteReplicaTest-2301E08B7645B5A6-001/tempDir-005/conf/
>    [junit4]   2> 2139389 T6340 oasc.CorePropertiesLocator.discover Found 1
> core definitions
>    [junit4]   2> 2139390 T6340 oass.SolrDispatchFilter.init
> user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1
>    [junit4]   2> 2139390 T6340 oass.SolrDispatchFilter.init
> SolrDispatchFilter.init() done
>    [junit4]   2> 2139661 T6340
> oasc.AbstractFullDistribZkTestBase.createJettys create jetty 4
>    [junit4]   2> 2139662 T6340 oejs.Server.doStart jetty-8.1.10.v20130312
>    [junit4]   2> 2139664 T6340 oejus.SslContextFactory.doStart Enabled
> Protocols [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2] of [SSLv2Hello,
> SSLv3, TLSv1, TLSv1.1, TLSv1.2]
>    [junit4]   2> 2139665 T6340 oejs.AbstractConnector.doStart Started
> SslSelectChannelConnector@127.0.0.1:54998
>    [junit4]   2> 2139667 T6340 oass.SolrDispatchFilter.init
> SolrDispatchFilter.init()
>    [junit4]   2> 2139667 T6340 oasc.SolrResourceLoader.locateSolrHome JNDI
> not configured for solr (NoInitialContextEx)
>    [junit4]   2> 2139668 T6340 oasc.SolrResourceLoader.locateSolrHome
> using system property solr.solr.home:
> /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.DeleteReplicaTest-2301E08B7645B5A6-001/tempDir-006
>    [junit4]   2> 2139668 T6340 oasc.SolrResourceLoader.<init> new
> SolrResourceLoader for directory:
> '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.DeleteReplicaTest-2301E08B7645B5A6-001/tempDir-006/'
>    [junit4]   2> 2139679 T6340 oasc.ConfigSolr.fromFile Loading container
> configuration from
> /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.DeleteReplicaTest-2301E08B7645B5A6-001/tempDir-006/solr.xml
>    [junit4]   2> 2139684 T6340 oasc.CorePropertiesLocator.<init>
> Config-defined core root directory:
> /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.DeleteReplicaTest-2301E08B7645B5A6-001/tempDir-006
>    [junit4]   2> 2139684 T6340 oasc.CoreContainer.<init> New CoreContainer
> 32630652
>    [junit4]   2> 2139685 T6340 oasc.CoreContainer.load Loading cores into
> CoreContainer
> [instanceDir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.DeleteReplicaTest-2301E08B7645B5A6-001/tempDir-006/]
>    [junit4]   2> 2139685 T6340 oashc.HttpShardHandlerFactory.getParameter
> Setting socketTimeout to: 90000
>    [junit4]   2> 2139686 T6340 oashc.HttpShardHandlerFactory.getParameter
> Setting urlScheme to:
>    [junit4]   2> 2139686 T6340 oashc.HttpShardHandlerFactory.getParameter
> Setting connTimeout to: 15000
>    [junit4]   2> 2139686 T6340 oashc.HttpShardHandlerFactory.getParameter
> Setting maxConnectionsPerHost to: 20
>    [junit4]   2> 2139686 T6340 oashc.HttpShardHandlerFactory.getParameter
> Setting corePoolSize to: 0
>    [junit4]   2> 2139687 T6340 oashc.HttpShardHandlerFactory.getParameter
> Setting maximumPoolSize to: 2147483647
>    [junit4]   2> 2139687 T6340 oashc.HttpShardHandlerFactory.getParameter
> Setting maxThreadIdleTime to: 5
>    [junit4]   2> 2139687 T6340 oashc.HttpShardHandlerFactory.getParameter
> Setting sizeOfQueue to: -1
>    [junit4]   2> 2139688 T6340 oashc.HttpShardHandlerFactory.getParameter
> Setting fairnessPolicy to: false
>    [junit4]   2> 2139689 T6340 oasu.UpdateShardHandler.<init> Creating
> UpdateShardHandler HTTP client with params:
> socketTimeout=340000&connTimeout=45000&retry=false
>    [junit4]   2> 2139690 T6340 oasl.LogWatcher.createWatcher SLF4J impl is
> org.slf4j.impl.Log4jLoggerFactory
>    [junit4]   2> 2139690 T6340 oasl.LogWatcher.newRegisteredLogWatcher
> Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
>    [junit4]   2> 2139690 T6340 oasc.CoreContainer.load Host Name: 127.0.0.1
>    [junit4]   2> 2139691 T6340 oasc.ZkContainer.initZooKeeper Zookeeper
> client=127.0.0.1:46175/solr
>    [junit4]   2> 2139691 T6340 oasc.ZkController.checkChrootPath zkHost
> includes chroot
>    [junit4]   2> 2139691 T6340
> oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default
> ZkCredentialsProvider
>    [junit4]   2> 2139692 T6340 oascc.ConnectionManager.waitForConnected
> Waiting for client to connect to ZooKeeper
>    [junit4]   2> 2139693 T6430 oascc.ConnectionManager.process Watcher
> org.apache.solr.common.cloud.ConnectionManager@5353b9
> name:ZooKeeperConnection Watcher:127.0.0.1:46175 got event WatchedEvent
> state:SyncConnected type:None path:null path:null type:None
>    [junit4]   2> 2139693 T6340 oascc.ConnectionManager.waitForConnected
> Client is connected to ZooKeeper
>    [junit4]   2> 2139693 T6340 oascc.SolrZkClient.createZkACLProvider
> Using default ZkACLProvider
>    [junit4]   2> 2139694 T6340 oascc.ConnectionManager.waitForConnected
> Waiting for client to connect to ZooKeeper
>    [junit4]   2> 2139695 T6432 oascc.ConnectionManager.process Watcher
> org.apache.solr.common.cloud.ConnectionManager@45a205
> name:ZooKeeperConnection Watcher:127.0.0.1:46175/solr got event
> WatchedEvent state:SyncConnected type:None path:null path:null type:None
>    [junit4]   2> 2139695 T6340 oascc.ConnectionManager.waitForConnected
> Client is connected to ZooKeeper
>    [junit4]   2> 2139698 T6340
> oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster
> state from ZooKeeper...
>    [junit4]   2> 2140701 T6340 oasc.ZkController.createEphemeralLiveNode
> Register node as live in ZooKeeper:/live_nodes/127.0.0.1:54998_b_y%2Ftb
>    [junit4]   2> 2140702 T6340 oascc.SolrZkClient.makePath makePath:
> /live_nodes/127.0.0.1:54998_b_y%2Ftb
>    [junit4]   2> 2140705 T6340 oasc.Overseer.close Overseer (id=null)
> closing
>    [junit4]   2> 2140708 T6340 oasc.CorePropertiesLocator.discover Looking
> for core definitions underneath
> /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.DeleteReplicaTest-2301E08B7645B5A6-001/tempDir-006
>    [junit4]   2> 2140718 T6340 oasc.CorePropertiesLocator.discoverUnder
> Found core conf in
> /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.DeleteReplicaTest-2301E08B7645B5A6-001/tempDir-006/conf/
>    [junit4]   2> 2140924 T6340 oasc.CorePropertiesLocator.discover Found 1
> core definitions
>    [junit4]   2> 2140926 T6340 oass.SolrDispatchFilter.init
> user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1
>    [junit4]   2> 2140926 T6340 oass.SolrDispatchFilter.init
> SolrDispatchFilter.init() done
>    [junit4]   2> 2140930 T6340
> oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default
> ZkCredentialsProvider
>    [junit4]   2> 2140931 T6340 oascc.ConnectionManager.waitForConnected
> Waiting for client to connect to ZooKeeper
>    [junit4]   2> 2140932 T6436 oascc.ConnectionManager.process Watcher
> org.apache.solr.common.cloud.ConnectionManager@1ef3a24
> name:ZooKeeperConnection Watcher:127.0.0.1:46175/solr got event
> WatchedEvent state:SyncConnected type:None path:null path:null type:None
>    [junit4]   2> 2140932 T6340 oascc.ConnectionManager.waitForConnected
> Client is connected to ZooKeeper
>    [junit4]   2> 2140932 T6340 oascc.SolrZkClient.createZkACLProvider
> Using default ZkACLProvider
>    [junit4]   2> 2140933 T6340
> oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster
> state from ZooKeeper...
>    [junit4]   2> 2140935 T6340
> oasc.AbstractFullDistribZkTestBase.createCollection Creating collection
> with stateFormat=2: delLiveColl
>    [junit4]   2> 2140935 T6340
> oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default
> ZkCredentialsProvider
>    [junit4]   2> 2140936 T6340 oascc.ConnectionManager.waitForConnected
> Waiting for client to connect to ZooKeeper
>    [junit4]   2> 2140937 T6438 oascc.ConnectionManager.process Watcher
> org.apache.solr.common.cloud.ConnectionManager@177c8c6
> name:ZooKeeperConnection Watcher:127.0.0.1:46175/solr got event
> WatchedEvent state:SyncConnected type:None path:null path:null type:None
>    [junit4]   2> 2140937 T6340 oascc.ConnectionManager.waitForConnected
> Client is connected to ZooKeeper
>    [junit4]   2> 2140937 T6340 oascc.SolrZkClient.createZkACLProvider
> Using default ZkACLProvider
>    [junit4]   2> 2140938 T6340
> oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster
> state from ZooKeeper...
>    [junit4]   2> 2140992 T6381 oasha.CollectionsHandler.handleCreateAction
> Creating Collection :
> replicationFactor=2&maxShardsPerNode=2&name=delLiveColl&action=CREATE&numShards=2&stateFormat=2&wt=javabin&version=2
>    [junit4]   2> 2140993 T6373
> oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on
> path: /overseer/collection-queue-work state: SyncConnected type
> NodeChildrenChanged
>    [junit4]   2> 2140993 T6366 oasc.OverseerCollectionProcessor.run
> Overseer Collection Processor: Get the message
> id:/overseer/collection-queue-work/qn-0000000000 message:{
>    [junit4]   2>          "operation":"create",
>    [junit4]   2>          "fromApi":"true",
>    [junit4]   2>          "name":"delLiveColl",
>    [junit4]   2>          "replicationFactor":"2",
>    [junit4]   2>          "numShards":"2",
>    [junit4]   2>          "maxShardsPerNode":"2",
>    [junit4]   2>          "stateFormat":"2"}
>    [junit4]   2> 2140995 T6439
> oasc.OverseerCollectionProcessor.processMessage WARN
> OverseerCollectionProcessor.processMessage : create , {
>    [junit4]   2>          "operation":"create",
>    [junit4]   2>          "fromApi":"true",
>    [junit4]   2>          "name":"delLiveColl",
>    [junit4]   2>          "replicationFactor":"2",
>    [junit4]   2>          "numShards":"2",
>    [junit4]   2>          "maxShardsPerNode":"2",
>    [junit4]   2>          "stateFormat":"2"}
>    [junit4]   2> 2140996 T6439
> oasc.OverseerCollectionProcessor.createConfNode Only one config set found
> in zk - using it:conf1
>    [junit4]   2> 2140996 T6439
> oasc.OverseerCollectionProcessor.createConfNode creating collections conf
> node /collections/delLiveColl
>    [junit4]   2> 2140997 T6439 oascc.SolrZkClient.makePath makePath:
> /collections/delLiveColl
>    [junit4]   2> 2140998 T6373
> oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on
> path: /overseer/queue state: SyncConnected type NodeChildrenChanged
>    [junit4]   2> 2141000 T6365
> oasc.Overseer$ClusterStateUpdater.buildCollection building a new
> collection: delLiveColl
>    [junit4]   2> 2141000 T6365
> oasc.Overseer$ClusterStateUpdater.createCollection Create collection
> delLiveColl with shards [shard1, shard2]
>    [junit4]   2> 2141000 T6365
> oasc.Overseer$ClusterStateUpdater.createCollection state version
> delLiveColl 2
>    [junit4]   2> 2141002 T6365
> oasc.Overseer$ClusterStateUpdater.updateZkStates going to create_collection
> /collections/delLiveColl/state.json
>    [junit4]   2> 2141002 T6374 oascc.ZkStateReader$2.process A cluster
> state change: WatchedEvent state:SyncConnected type:NodeDataChanged
> path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
>    [junit4]   2> 2141002 T6403 oascc.ZkStateReader$2.process A cluster
> state change: WatchedEvent state:SyncConnected type:NodeDataChanged
> path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
>    [junit4]   2> 2141003 T6418 oascc.ZkStateReader$2.process A cluster
> state change: WatchedEvent state:SyncConnected type:NodeDataChanged
> path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
>    [junit4]   2> 2141003 T6388 oascc.ZkStateReader$2.process A cluster
> state change: WatchedEvent state:SyncConnected type:NodeDataChanged
> path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
>    [junit4]   2> 2141002 T6440 oascc.ZkStateReader$2.process A cluster
> state change: WatchedEvent state:SyncConnected type:NodeDataChanged
> path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
>    [junit4]   2> 2141003 T6373 oascc.ZkStateReader$2.process A cluster
> state change: WatchedEvent state:SyncConnected type:NodeDataChanged
> path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
>    [junit4]   2> 2141004 T6441 oascc.ZkStateReader$2.process A cluster
> state change: WatchedEvent state:SyncConnected type:NodeDataChanged
> path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
>    [junit4]   2> 2141005 T6433 oascc.ZkStateReader$2.process A cluster
> state change: WatchedEvent state:SyncConnected type:NodeDataChanged
> path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
>    [junit4]   2> 2141098 T6439
> oasc.OverseerCollectionProcessor.createCollection Creating SolrCores for
> new collection delLiveColl, shardNames [shard1, shard2] , replicationFactor
> : 2
>    [junit4]   2> 2141099 T6439
> oasc.OverseerCollectionProcessor.createCollection Creating shard
> delLiveColl_shard1_replica1 as part of slice shard1 of collection
> delLiveColl on 127.0.0.1:54998_b_y%2Ftb
>    [junit4]   2> 2141099 T6439
> oasc.OverseerCollectionProcessor.createCollection Creating shard
> delLiveColl_shard1_replica2 as part of slice shard1 of collection
> delLiveColl on 127.0.0.1:45141_b_y%2Ftb
>    [junit4]   2> 2141101 T6439
> oasc.OverseerCollectionProcessor.createCollection Creating shard
> delLiveColl_shard2_replica1 as part of slice shard2 of collection
> delLiveColl on 127.0.0.1:60980_b_y%2Ftb
>    [junit4]   2> 2141104 T6439
> oasc.OverseerCollectionProcessor.createCollection Creating shard
> delLiveColl_shard2_replica2 as part of slice shard2 of collection
> delLiveColl on 127.0.0.1:51660_b_y%2Ftb
>    [junit4]   2> 2141181 T6410 oasha.CoreAdminHandler.handleCreateAction
> core create command
> qt=/admin/cores&collection.configName=conf1&name=delLiveColl_shard2_replica1&action=CREATE&numShards=2&collection=delLiveColl&shard=shard2&wt=javabin&version=2
>    [junit4]   2> 2141183 T6410 oasc.ZkController.publish publishing
> core=delLiveColl_shard2_replica1 state=down collection=delLiveColl
>    [junit4]   2> 2141186 T6357 oasha.CoreAdminHandler.handleCreateAction
> core create command
> qt=/admin/cores&collection.configName=conf1&name=delLiveColl_shard1_replica2&action=CREATE&numShards=2&collection=delLiveColl&shard=shard1&wt=javabin&version=2
>    [junit4]   2> 2141186 T6396 oasha.CoreAdminHandler.handleCreateAction
> core create command
> qt=/admin/cores&collection.configName=conf1&name=delLiveColl_shard2_replica2&action=CREATE&numShards=2&collection=delLiveColl&shard=shard2&wt=javabin&version=2
>    [junit4]   2> 2141187 T6373
> oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on
> path: /overseer/queue state: SyncConnected type NodeChildrenChanged
>    [junit4]   2> 2141187 T6410 oasc.ZkController.preRegister Registering
> watch for external collection delLiveColl
>    [junit4]   2> 2141188 T6410 oascc.ZkStateReader.addZkWatch addZkWatch
> delLiveColl
>    [junit4]   2> 2141188 T6410 oascc.ZkStateReader.addZkWatch Updating
> collection state at /collections/delLiveColl/state.json from ZooKeeper...
>    [junit4]   2> 2141189 T6396 oasc.ZkController.publish publishing
> core=delLiveColl_shard2_replica2 state=down collection=delLiveColl
>    [junit4]   2> 2141189 T6357 oasc.ZkController.publish publishing
> core=delLiveColl_shard1_replica2 state=down collection=delLiveColl
>    [junit4]   2> 2141190 T6365
> oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2
> message={
>    [junit4]   2>          "core":"delLiveColl_shard2_replica1",
>    [junit4]   2>          "roles":null,
>    [junit4]   2>          "base_url":"https://127.0.0.1:60980/b_y/tb",
>    [junit4]   2>          "node_name":"127.0.0.1:60980_b_y%2Ftb",
>    [junit4]   2>          "numShards":"2",
>    [junit4]   2>          "state":"down",
>    [junit4]   2>          "shard":"shard2",
>    [junit4]   2>          "collection":"delLiveColl",
>    [junit4]   2>          "operation":"state"}
>    [junit4]   2> 2141190 T6410 oascc.ZkStateReader.updateWatchedCollection
> Updating data for delLiveColl to ver 0
>    [junit4]   2> 2141191 T6410 oasc.ZkController.waitForCoreNodeName look
> for our core node name
>    [junit4]   2> 2141191 T6428 oasha.CoreAdminHandler.handleCreateAction
> core create command
> qt=/admin/cores&collection.configName=conf1&name=delLiveColl_shard1_replica1&action=CREATE&numShards=2&collection=delLiveColl&shard=shard1&wt=javabin&version=2
>    [junit4]   2> 2141191 T6396 oasc.ZkController.preRegister Registering
> watch for external collection delLiveColl
>    [junit4]   2> 2141192 T6396 oascc.ZkStateReader.addZkWatch addZkWatch
> delLiveColl
>    [junit4]   2> 2141192 T6396 oascc.ZkStateReader.addZkWatch Updating
> collection state at /collections/delLiveColl/state.json from ZooKeeper...
>    [junit4]   2> 2141192 T6357 oasc.ZkController.preRegister Registering
> watch for external collection delLiveColl
>    [junit4]   2> 2141193 T6396 oascc.ZkStateReader.updateWatchedCollection
> Updating data for delLiveColl to ver 0
>    [junit4]   2> 2141194 T6396 oasc.ZkController.waitForCoreNodeName look
> for our core node name
>    [junit4]   2> 2141194 T6428 oasc.ZkController.publish publishing
> core=delLiveColl_shard1_replica1 state=down collection=delLiveColl
>    [junit4]   2> 2141196 T6428 oasc.ZkController.preRegister Registering
> watch for external collection delLiveColl
>    [junit4]   2> 2141196 T6428 oascc.ZkStateReader.addZkWatch addZkWatch
> delLiveColl
>    [junit4]   2> 2141196 T6428 oascc.ZkStateReader.addZkWatch Updating
> collection state at /collections/delLiveColl/state.json from ZooKeeper...
>    [junit4]   2> 2141196 T6365
> oasc.Overseer$ClusterStateUpdater.updateZkStates going to update_collection
> /collections/delLiveColl/state.json
>    [junit4]   2> 2141197 T6418 oascc.ZkStateReader$7.process A cluster
> state change: WatchedEvent state:SyncConnected type:NodeDataChanged
> path:/collections/delLiveColl/state.json, has occurred - updating...
>    [junit4]   2> 2141198 T6433 oascc.ZkStateReader$7.process A cluster
> state change: WatchedEvent state:SyncConnected type:NodeDataChanged
> path:/collections/delLiveColl/state.json, has occurred - updating...
>    [junit4]   2> 2141198 T6418 oascc.ZkStateReader.updateWatchedCollection
> Updating data for delLiveColl to ver 1
>    [junit4]   2> 2141198 T6403 oascc.ZkStateReader$7.process A cluster
> state change: WatchedEvent state:SyncConnected type:NodeDataChanged
> path:/collections/delLiveColl/state.json, has occurred - updating...
>    [junit4]   2> 2141198 T6428 oascc.ZkStateReader.updateWatchedCollection
> Updating data for delLiveColl to ver 1
>    [junit4]   2> 2141199 T6357 oascc.ZkStateReader.addZkWatch addZkWatch
> delLiveColl
>    [junit4]   2> 2141199 T6403 oascc.ZkStateReader.updateWatchedCollection
> Updating data for delLiveColl to ver 1
>    [junit4]   2> 2141199 T6428 oasc.ZkController.waitForCoreNodeName look
> for our core node name
>    [junit4]   2> 2141199 T6357 oascc.ZkStateReader.addZkWatch Updating
> collection state at /collections/delLiveColl/state.json from ZooKeeper...
>    [junit4]   2> 2141199 T6433 oascc.ZkStateReader.updateWatchedCollection
> Updating data for delLiveColl to ver 1
>    [junit4]   2> 2141201 T6357 oascc.ZkStateReader.updateWatchedCollection
> Updating data for delLiveColl to ver 1
>    [junit4]   2> 2141201 T6357 oasc.ZkController.waitForCoreNodeName look
> for our core node name
>    [junit4]   2> 2141202 T6365 oascc.ZkStateReader.updateWatchedCollection
> Updating data for delLiveColl to ver 1
>    [junit4]   2> 2141202 T6365
> oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2
> message={
>    [junit4]   2>          "core":"delLiveColl_shard2_replica2",
>    [junit4]   2>          "roles":null,
>    [junit4]   2>          "base_url":"https://127.0.0.1:51660/b_y/tb",
>    [junit4]   2>          "node_name":"127.0.0.1:51660_b_y%2Ftb",
>    [junit4]   2>          "numShards":"2",
>    [junit4]   2>          "state":"down",
>    [junit4]   2>          "shard":"shard2",
>    [junit4]   2>          "collection":"delLiveColl",
>    [junit4]   2>          "operation":"state"}
>    [junit4]   2> 2141204 T6373
> oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on
> path: /overseer/queue state: SyncConnected type NodeChildrenChanged
>    [junit4]   2> 2141204 T6365
> oasc.Overseer$ClusterStateUpdater.updateZkStates going to update_collection
> /collections/delLiveColl/state.json
>    [junit4]   2> 2141204 T6418 oascc.ZkStateReader$7.process A cluster
> state change: WatchedEvent state:SyncConnected type:NodeDataChanged
> path:/collections/delLiveColl/state.json, has occurred - updating...
>    [junit4]   2> 2141204 T6373 oascc.ZkStateReader$7.process A cluster
> state change: WatchedEvent state:SyncConnected type:NodeDataChanged
> path:/collections/delLiveColl/state.json, has occurred - updating...
>    [junit4]   2> 2141204 T6433 oascc.ZkStateReader$7.process A cluster
> state change: WatchedEvent state:SyncConnected type:NodeDataChanged
> path:/collections/delLiveColl/state.json, has occurred - updating...
>    [junit4]   2> 2141205 T6418 oascc.ZkStateReader.updateWatchedCollection
> Updating data for delLiveColl to ver 2
>    [junit4]   2> 2141205 T6433 oascc.ZkStateReader.updateWatchedCollection
> Updating data for delLiveColl to ver 2
>    [junit4]   2> 2141204 T6403 oascc.ZkStateReader$7.process A cluster
> state change: WatchedEvent state:SyncConnected type:NodeDataChanged
> path:/collections/delLiveColl/state.json, has occurred - updating...
>    [junit4]   2> 2141206 T6403 oascc.ZkStateReader.updateWatchedCollection
> Updating data for delLiveColl to ver 2
>    [junit4]   2> 2141206 T6373 oascc.ZkStateReader.updateWatchedCollection
> Updating data for delLiveColl to ver 2
>    [junit4]   2> 2141208 T6365 oascc.ZkStateReader.updateWatchedCollection
> Updating data for delLiveColl to ver 2
>    [junit4]   2> 2141208 T6365
> oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2
> message={
>    [junit4]   2>          "core":"delLiveColl_shard1_replica2",
>    [junit4]   2>          "roles":null,
>    [junit4]   2>          "base_url":"https://127.0.0.1:45141/b_y/tb",
>    [junit4]   2>          "node_name":"127.0.0.1:45141_b_y%2Ftb",
>    [junit4]   2>          "numShards":"2",
>    [junit4]   2>          "state":"down",
>    [junit4]   2>          "shard":"shard1",
>    [junit4]   2>          "collection":"delLiveColl",
>    [junit4]   2>          "operation":"state"}
>    [junit4]   2> 2141209 T6373
> oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on
> path: /overseer/queue state: SyncConnected type NodeChildrenChanged
>    [junit4]   2> 2141209 T6365
> oasc.Overseer$ClusterStateUpdater.updateZkStates going to update_collection
> /collections/delLiveColl/state.json
>    [junit4]   2> 2141209 T6418 oascc.ZkStateReader$7.process A cluster
> state change: WatchedEvent state:SyncConnected type:NodeDataChanged
> path:/collections/delLiveColl/state.json, has occurred - updating...
>    [junit4]   2> 2141210 T6403 oascc.ZkStateReader$7.process A cluster
> state change: WatchedEvent state:SyncConnected type:NodeDataChanged
> path:/collections/delLiveColl/state.json, has occurred - updating...
>    [junit4]   2> 2141210 T6373 oascc.ZkStateReader$7.process A cluster
> state change: WatchedEvent state:SyncConnected type:NodeDataChanged
> path:/collections/delLiveColl/state.json, has occurred - updating...
>    [junit4]   2> 2141210 T6403 oascc.ZkStateReader.updateWatchedCollection
> Updating data for delLiveColl to ver 3
>    [junit4]   2> 2141209 T6433 oascc.ZkStateReader$7.process A cluster
> state change: WatchedEvent state:SyncConnected type:NodeDataChanged
> path:/collections/delLiveColl/state.json, has occurred - updating...
>    [junit4]   2> 2141211 T6373 oascc.ZkStateReader.updateWatchedCollection
> Updating data for delLiveColl to ver 3
>    [junit4]   2> 2141211 T6433 oascc.ZkStateReader.updateWatchedCollection
> Updating data for delLiveColl to ver 3
>    [junit4]   2> 2141210 T6418 oascc.ZkStateReader.updateWatchedCollection
> Updating data for delLiveColl to ver 3
>    [junit4]   2> 2141213 T6365 oascc.ZkStateReader.updateWatchedCollection
> Updating data for delLiveColl to ver 3
>    [junit4]   2> 2141213 T6365
> oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2
> message={
>    [junit4]   2>          "core":"delLiveColl_shard1_replica1",
>    [junit4]   2>          "roles":null,
>    [junit4]   2>          "base_url":"https://127.0.0.1:54998/b_y/tb",
>    [junit4]   2>          "node_name":"127.0.0.1:54998_b_y%2Ftb",
>    [junit4]   2>          "numShards":"2",
>    [junit4]   2>          "state":"down",
>    [junit4]   2>          "shard":"shard1",
>    [junit4]   2>          "collection":"delLiveColl",
>    [junit4]   2>          "operation":"state"}
>    [junit4]   2> 2141214 T6373
> oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on
> path: /overseer/queue state: SyncConnected type NodeChildrenChanged
>    [junit4]   2> 2141214 T6365
> oasc.Overseer$ClusterStateUpdater.updateZkStates going to update_collection
> /collections/delLiveColl/state.json
>    [junit4]   2> 2141215 T6418 oascc.ZkStateReader$7.process A cluster
> state change: WatchedEvent state:SyncConnected type:NodeDataChanged
> path:/collections/delLiveColl/state.json, has occurred - updating...
>    [junit4]   2> 2141215 T6373 oascc.ZkStateReader$7.process A cluster
> state change: WatchedEvent state:SyncConnected type:NodeDataChanged
> path:/collections/delLiveColl/state.json, has occurred - updating...
>    [junit4]   2> 2141215 T6403 oascc.ZkStateReader$7.process A cluster
> state change: WatchedEvent state:SyncConnected type:NodeDataChanged
> path:/collections/delLiveColl/state.json, has occurred - updating...
>    [junit4]   2> 2141216 T6403 oascc.ZkStateReader.updateWatchedCollection
> Updating data for delLiveColl to ver 4
>    [junit4]   2> 2141215 T6433 oascc.ZkStateReader$7.process A cluster
> state change: WatchedEvent state:SyncConnected type:NodeDataChanged
> path:/collections/delLiveColl/state.json, has occurred - updating...
>    [junit4]   2> 2141215 T6418 oascc.ZkStateReader.updateWatchedCollection
> Updating data for delLiveColl to ver 4
>    [junit4]   2> 2141217 T6433 oascc.ZkStateReader.updateWatchedCollection
> Updating data for delLiveColl to ver 4
>    [junit4]   2> 2141217 T6373 oascc.ZkStateReader.updateWatchedCollection
> Updating data for delLiveColl to ver 4
>    [junit4]   2> 2142191 T6410 oasc.ZkController.waitForShardId waiting to
> find shard id in clusterstate for delLiveColl_shard2_replica1
>    [junit4]   2> 2142191 T6410 oasc.ZkController.createCollectionZkNode
> Check for collection zkNode:delLiveColl
>    [junit4]   2> 2142192 T6410 oasc.ZkController.createCollectionZkNode
> Collection zkNode exists
>    [junit4]   2> 2142192 T6410 oascc.ZkStateReader.readConfigName Load
> collection config from:/collections/delLiveColl
>    [junit4]   2> 2142193 T6410 oascc.ZkStateReader.readConfigName
> path=/collections/delLiveColl configName=conf1 specified config exists in
> ZooKeeper
>    [junit4]   2> 2142194 T6410 oasc.SolrResourceLoader.<init> new
> SolrResourceLoader for directory:
> '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.DeleteReplicaTest-2301E08B7645B5A6-001/tempDir-005/delLiveColl_shard2_replica1/'
>    [junit4]   2> 2142194 T6396 oasc.ZkController.waitForShardId waiting to
> find shard id in clusterstate for delLiveColl_shard2_replica2
>    [junit4]   2> 2142194 T6396 oasc.ZkController.createCollectionZkNode
> Check for collection zkNode:delLiveColl
>    [junit4]   2> 2142197 T6396 oasc.ZkController.createCollectionZkNode
> Collection zkNode exists
>    [junit4]   2> 2142197 T6396 oascc.ZkStateReader.readConfigName Load
> collection config from:/collections/delLiveColl
>    [junit4]   2> 2142198 T6396 oascc.ZkStateReader.readConfigName
> path=/collections/delLiveColl configName=conf1 specified config exists in
> ZooKeeper
>    [junit4]   2> 2142198 T6396 oasc.SolrResourceLoader.<init> new
> SolrResourceLoader for directory:
> '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.DeleteReplicaTest-2301E08B7645B5A6-001/tempDir-004/delLiveColl_shard2_replica2/'
>    [junit4]   2> 2142200 T6428 oasc.ZkController.waitForShardId waiting to
> find shard id in clusterstate for delLiveColl_shard1_replica1
>    [junit4]   2> 2142200 T6428 oasc.ZkController.createCollectionZkNode
> Check for collection zkNode:delLiveColl
>    [junit4]   2> 2142201 T6428 oasc.ZkController.createCollectionZkNode
> Collection zkNode exists
>    [junit4]   2> 2142201 T6428 oascc.ZkStateReader.readConfigName Load
> collection config from:/collections/delLiveColl
>    [junit4]   2> 2142201 T6357 oasc.ZkController.waitForShardId waiting to
> find shard id in clusterstate for delLiveColl_shard1_replica2
>    [junit4]   2> 2142202 T6357 oasc.ZkController.createCollectionZkNode
> Check for collection zkNode:delLiveColl
>    [junit4]   2> 2142202 T6357 oasc.ZkController.createCollectionZkNode
> Collection zkNode exists
>    [junit4]   2> 2142202 T6357 oascc.ZkStateReader.readConfigName Load
> collection config from:/collections/delLiveColl
>    [junit4]   2> 2142203 T6428 oascc.ZkStateReader.readConfigName
> path=/collections/delLiveColl configName=conf1 specified config exists in
> ZooKeeper
>    [junit4]   2> 2142203 T6428 oasc.SolrResourceLoader.<init> new
> SolrResourceLoader for directory:
> '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.DeleteReplicaTest-2301E08B7645B5A6-001/tempDir-006/delLiveColl_shard1_replica1/'
>    [junit4]   2> 2142203 T6357 oascc.ZkStateReader.readConfigName
> path=/collections/delLiveColl configName=conf1 specified config exists in
> ZooKeeper
>    [junit4]   2> 2142204 T6357 oasc.SolrResourceLoader.<init> new
> SolrResourceLoader for directory:
> '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.DeleteReplicaTest-2301E08B7645B5A6-001/tempDir-002/delLiveColl_shard1_replica2/'
>    [junit4]   2> 2142229 T6410 oasc.SolrConfig.<init> Using Lucene
> MatchVersion: 5.0.0
>    [junit4]   2> 2142254 T6410 oasc.SolrConfig.<init> Loaded SolrConfig:
> solrconfig.xml
>    [junit4]   2> 2142257 T6410 oass.IndexSchema.readSchema Reading Solr
> Schema from schema.xml
>    [junit4]   2> 2142258 T6357 oasc.SolrConfig.<init> Using Lucene
> MatchVersion: 5.0.0
>    [junit4]   2> 2142259 T6396 oasc.SolrConfig.<init> Using Lucene
> MatchVersion: 5.0.0
>    [junit4]   2> 2142263 T6428 oasc.SolrConfig.<init> Using Lucene
> MatchVersion: 5.0.0
>    [junit4]   2> 2142267 T6410 oass.IndexSchema.readSchema
> [delLiveColl_shard2_replica1] Schema name=test
>    [junit4]   2> 2142272 T6396 oasc.SolrConfig.<init> Loaded SolrConfig:
> solrconfig.xml
>    [junit4]   2> 2142272 T6357 oasc.SolrConfig.<init> Loaded SolrConfig:
> solrconfig.xml
>    [junit4]   2> 2142285 T6357 oass.IndexSchema.readSchema Reading Solr
> Schema from schema.xml
>    [junit4]   2> 2142289 T6396 oass.IndexSchema.readSchema Reading Solr
> Schema from schema.xml
>    [junit4]   2> 2142289 T6428 oasc.SolrConfig.<init> Loaded SolrConfig:
> solrconfig.xml
>    [junit4]   2> 2142291 T6428 oass.IndexSchema.readSchema Reading Solr
> Schema from schema.xml
>    [junit4]   2> 2142291 T6357 oass.IndexSchema.readSchema
> [delLiveColl_shard1_replica2] Schema name=test
>    [junit4]   2> 2142298 T6396 oass.IndexSchema.readSchema
> [delLiveColl_shard2_replica2] Schema name=test
>    [junit4]   2> 2142298 T6428 oass.IndexSchema.readSchema
> [delLiveColl_shard1_replica1] Schema name=test
>    [junit4]   2> 2142452 T6410 oass.OpenExchangeRatesOrgProvider.init
> Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
>    [junit4]   2> 2142457 T6410 oass.IndexSchema.readSchema default search
> field in schema is text
>    [junit4]   2> 2142459 T6410 oass.IndexSchema.readSchema unique key
> field: id
>    [junit4]   2> 2142464 T6428 oass.OpenExchangeRatesOrgProvider.init
> Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
>    [junit4]   2> 2142466 T6410 oass.FileExchangeRateProvider.reload
> Reloading exchange rates from file currency.xml
>    [junit4]   2> 2142468 T6410 oass.FileExchangeRateProvider.reload
> Reloading exchange rates from file currency.xml
>    [junit4]   2> 2142469 T6428 oass.IndexSchema.readSchema default search
> field in schema is text
>    [junit4]   2> 2142471 T6428 oass.IndexSchema.readSchema unique key
> field: id
>    [junit4]   2> 2142472 T6410 oass.OpenExchangeRatesOrgProvider.reload
> Reloading exchange rates from open-exchange-rates.json
>    [junit4]   2> 2142473 T6410
> oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key
> IMPORTANT NOTE
>    [junit4]   2> 2142474 T6410
> oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected
> key, got STRING
>    [junit4]   2> 2142475 T6410 oass.OpenExchangeRatesOrgProvider.reload
> Reloading exchange rates from open-exchange-rates.json
>    [junit4]   2> 2142475 T6410
> oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key
> IMPORTANT NOTE
>    [junit4]   2> 2142476 T6410
> oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected
> key, got STRING
>    [junit4]   2> 2142476 T6410 oasc.CoreContainer.create Creating SolrCore
> 'delLiveColl_shard2_replica1' using configuration from collection
> delLiveColl
>    [junit4]   2> 2142476 T6410 oasc.SolrCore.initDirectoryFactory
> org.apache.solr.core.MockDirectoryFactory
>    [junit4]   2> 2142477 T6410 oasc.SolrCore.<init>
> [delLiveColl_shard2_replica1] Opening new SolrCore at
> /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.DeleteReplicaTest-2301E08B7645B5A6-001/tempDir-005/delLiveColl_shard2_replica1/,
> dataDir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.DeleteReplicaTest-2301E08B7645B5A6-001/tempDir-005/delLiveColl_shard2_replica1/data/
>    [junit4]   2> 2142477 T6410 oasc.JmxMonitoredMap.<init> JMX monitoring
> is enabled. Adding Solr mbeans to JMX Server:
> com.sun.jmx.mbeanserver.JmxMBeanServer@2049e8
>    [junit4]   2> 2142478 T6410 oasc.CachingDirectoryFactory.get return new
> directory for
> /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.DeleteReplicaTest-2301E08B7645B5A6-001/tempDir-005/delLiveColl_shard2_replica1/data
>    [junit4]   2> 2142478 T6410 oasc.SolrCore.getNewIndexDir New index
> directory detected: old=null
> new=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.DeleteReplicaTest-2301E08B7645B5A6-001/tempDir-005/delLiveColl_shard2_replica1/data/index/
>    [junit4]   2> 2142479 T6410 oasc.SolrCore.initIndex WARN
> [delLiveColl_shard2_replica1] Solr index directory
> '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.DeleteReplicaTest-2301E08B7645B5A6-001/tempDir-005/delLiveColl_shard2_replica1/data/index'
> doesn't exist. Creating new index...
>    [junit4]   2> 2142480 T6410 oasc.CachingDirectoryFactory.get return new
> directory for
> /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.DeleteReplicaTest-2301E08B7645B5A6-001/tempDir-005/delLiveColl_shard2_replica1/data/index
>    [junit4]   2> 2142480 T6410 oasu.RandomMergePolicy.<init>
> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy:
> [TieredMergePolicy: maxMergeAtOnce=23, maxMergeAtOnceExplicit=49,
> maxMergedSegmentMB=2.9052734375, floorSegmentMB=0.55078125,
> forceMergeDeletesPctAllowed=18.294957546728785, segmentsPerTier=44.0,
> maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.14684349874543712
>    [junit4]   2> 2142481 T6410 oasc.SolrDeletionPolicy.onCommit
> SolrDeletionPolicy.onCommit: commits: num=1
>    [junit4]   2>
> commit{dir=MockDirectoryWrapper(RAMDirectory(id=lucene-5d5088)),segFN=segments_1,generation=1}
>    [junit4]   2> 2142481 T6410 oasc.SolrDeletionPolicy.updateCommits
> newest commit generation = 1
>    [junit4]   2> 2142484 T6410 oasup.UpdateRequestProcessorChain.init
> creating updateRequestProcessorChain "nodistrib"
>    [junit4]   2> 2142485 T6428 oass.FileExchangeRateProvider.reload
> Reloading exchange rates from file currency.xml
>    [junit4]   2> 2142485 T6410 oasup.UpdateRequestProcessorChain.init
> creating updateRequestProcessorChain "dedupe"
>    [junit4]   2> 2142486 T6410 oasup.UpdateRequestProcessorChain.init
> inserting DistributedUpdateProcessorFactory into
> updateRequestProcessorChain "dedupe"
>    [junit4]   2> 2142487 T6410 oasup.UpdateRequestProcessorChain.init
> creating updateRequestProcessorChain "stored_sig"
>    [junit4]   2> 2142487 T6410 oasup.UpdateRequestProcessorChain.init
> inserting DistributedUpdateProcessorFactory into
> updateRequestProcessorChain "stored_sig"
>    [junit4]   2> 2142498 T6410 oasup.UpdateRequestProcessorChain.init
> creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
>    [junit4]   2> 2142498 T6410 oasup.UpdateRequestProcessorChain.init
> creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
>    [junit4]   2> 2142499 T6410 oasup.UpdateRequestProcessorChain.init
> inserting DistributedUpdateProcessorFactory into
> updateRequestProcessorChain "distrib-dup-test-chain-implicit"
>    [junit4]   2> 2142499 T6410 oasc.SolrCore.loadUpdateProcessorChains no
> updateRequestProcessorChain defined as default, creating implicit default
>    [junit4]   2> 2142499 T6428 oass.FileExchangeRateProvider.reload
> Reloading exchange rates from file currency.xml
>    [junit4]   2> 2142500 T6410 oasc.RequestHandlers.initHandlersFromConfig
> created /update/csv: org.apache.solr.handler.UpdateRequestHandler
>    [junit4]   2> 2142500 T6410 oasc.RequestHandlers.initHandlersFromConfig
> created /update/json: org.apache.solr.handler.UpdateRequestHandler
>    [junit4]   2> 2142501 T6410 oasc.RequestHandlers.initHandlersFromConfig
> created /update/json/docs: org.apache.solr.handler.UpdateRequestHandler
>    [junit4]   2> 2142501 T6410 oasc.RequestHandlers.initHandlersFromConfig
> created standard: solr.StandardRequestHandler
>    [junit4]   2> 2142502 T6410 oasc.RequestHandlers.initHandlersFromConfig
> created /get: solr.RealTimeGetHandler
>    [junit4]   2> 2142502 T6410 oasc.RequestHandlers.initHandlersFromConfig
> adding lazy requestHandler: solr.ReplicationHandler
>    [junit4]   2> 2142502 T6428 oass.OpenExchangeRatesOrgProvider.reload
> Reloading exchange rates from open-exchange-rates.json
>    [junit4]   2> 2142503 T6410 oasc.RequestHandlers.initHandlersFromConfig
> created /replication: solr.ReplicationHandler
>    [junit4]   2> 2142504 T6410 oasc.RequestHandlers.initHandlersFromConfig
> created /update: solr.UpdateRequestHandler
>    [junit4]   2> 2142504 T6428
> oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key
> IMPORTANT NOTE
>    [junit4]   2> 2142505 T6396 oass.OpenExchangeRatesOrgProvider.init
> Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
>    [junit4]   2> 2142505 T6428
> oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected
> key, got STRING
>    [junit4]   2> 2142504 T6410 oasc.RequestHandlers.initHandlersFromConfig
> created /admin/: org.apache.solr.handler.admin.AdminHandlers
>    [junit4]   2> 2142506 T6428 oass.OpenExchangeRatesOrgProvider.reload
> Reloading exchange rates from open-exchange-rates.json
>    [junit4]   2> 2142507 T6410 oasc.RequestHandlers.initHandlersFromConfig
> created /admin/file: solr.admin.ShowFileRequestHandler
>    [junit4]   2> 2142507 T6410 oasc.RequestHandlers.initHandlersFromConfig
> created /admin/fileedit: solr.admin.EditFileRequestHandler
>    [junit4]   2> 2142508 T6428
> oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key
> IMPORTANT NOTE
>    [junit4]   2> 2142508 T6428
> oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected
> key, got STRING
>    [junit4]   2> 2142509 T6428 oasc.CoreContainer.create Creating SolrCore
> 'delLiveColl_shard1_replica1' using configuration from collection
> delLiveColl
>    [junit4]   2> 2142509 T6396 oass.IndexSchema.readSchema default search
> field in schema is text
>    [junit4]   2> 2142510 T6410 oashl.XMLLoader.init
> xsltCacheLifetimeSeconds=60
>    [junit4]   2> 2142510 T6428 oasc.SolrCore.initDirectoryFactory
> org.apache.solr.core.MockDirectoryFactory
>    [junit4]   2> 2142511 T6428 oasc.SolrCore.<init>
> [delLiveColl_shard1_replica1] Opening new SolrCore at
> /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.DeleteReplicaTest-2301E08B7645B5A6-001/tempDir-006/delLiveColl_shard1_replica1/,
> dataDir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.DeleteReplicaTest-2301E08B7645B5A6-001/tempDir-006/delLiveColl_shard1_replica1/data/
>    [junit4]   2> 2142511 T6410 oashl.XMLLoader.init
> xsltCacheLifetimeSeconds=60
>    [junit4]   2> 2142512 T6428 oasc.JmxMonitoredMap.<init> JMX monitoring
> is enabled. Adding Solr mbeans to JMX Server:
> com.sun.jmx.mbeanserver.JmxMBeanServer@2049e8
>    [junit4]   2> 2142512 T6396 oass.IndexSchema.readSchema unique key
> field: id
>    [junit4]   2> 2142513 T6410 oashl.XMLLoader.init
> xsltCacheLifetimeSeconds=60
>    [junit4]   2> 2142513 T6428 oasc.CachingDirectoryFactory.get return new
> directory for
> /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.DeleteReplicaTest-2301E08B7645B5A6-001/tempDir-006/delLiveColl_shard1_replica1/data
>    [junit4]   2> 2142514 T6428 oasc.SolrCore.getNewIndexDir New index
> directory detected: old=null
> new=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.DeleteReplicaTest-2301E08B7645B5A6-001/tempDir-006/delLiveColl_shard1_replica1/data/index/
>    [junit4]   2> 2142514 T6428 oasc.SolrCore.initIndex WARN
> [delLiveColl_shard1_replica1] Solr index directory
> '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.DeleteReplicaTest-2301E08B7645B5A6-001/tempDir-006/delLiveColl_shard1_replica1/data/index'
> doesn't exist. Creating new index...
>    [junit4]   2> 2142515 T6410 oashl.XMLLoader.init
> xsltCacheLifetimeSeconds=60
>    [junit4]   2> 2142515 T6428 oasc.CachingDirectoryFactory.get return new
> directory for
> /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.DeleteReplicaTest-2301E08B7645B5A6-001/tempDir-006/delLiveColl_shard1_replica1/data/index
>    [junit4]   2> 2142516 T6428 oasu.RandomMergePolicy.<init>
> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy:
> [TieredMergePolicy: maxMergeAtOnce=23, maxMergeAtOnceExplicit=49,
> maxMergedSegmentMB=2.9052734375, floorSegmentMB=0.55078125,
> forceMergeDeletesPctAllowed=18.294957546728785, segmentsPerTier=44.0,
> maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.14684349874543712
>    [junit4]   2> 2142516 T6428 oasc.SolrDeletionPolicy.onCommit
> SolrDeletionPolicy.onCommit: commits: num=1
>    [junit4]   2>
> commit{dir=MockDirectoryWrapper(RAMDirectory(id=lucene-31cc74)),segFN=segments_1,generation=1}
>    [junit4]   2> 2142517 T6428 oasc.SolrDeletionPolicy.updateCommits
> newest commit generation = 1
>    [junit4]   2> 2142518 T6357 oass.OpenExchangeRatesOrgProvider.init
> Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
>    [junit4]   2> 2142520 T6428 oasup.UpdateRequestProcessorChain.init
> creating updateRequestProcessorChain "nodistrib"
>    [junit4]   2> 2142521 T6428 oasup.UpdateRequestProcessorChain.init
> creating updateRequestProcessorChain "dedupe"
>    [junit4]   2> 2142521 T6428 oasup.UpdateRequestProcessorChain.init
> inserting DistributedUpdateProcessorFactory into
> updateRequestProcessorChain "dedupe"
>    [junit4]   2> 2142522 T6428 oasup.UpdateRequestProcessorChain.init
> creating updateRequestProcessorChain "stored_sig"
>    [junit4]   2> 2142522 T6428 oasup.UpdateRequestProcessorChain.init
> inserting DistributedUpdateProcessorFactory into
> updateRequestProcessorChain "stored_sig"
>    [junit4]   2> 2142522 T6357 oass.IndexSchema.readSchema default search
> field in schema is text
>    [junit4]   2> 2142523 T6428 oasup.UpdateRequestProcessorChain.init
> creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
>    [junit4]   2> 2142524 T6410 oasu.CommitTracker.<init> Hard AutoCommit:
> disabled
>    [junit4]   2> 2142524 T6410 oasu.CommitTracker.<init> Soft AutoCommit:
> disabled
>    [junit4]   2> 2142524 T6428 oasup.UpdateRequestProcessorChain.init
> creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
>    [junit4]   2> 2142525 T6410 oasu.RandomMergePolicy.<init>
> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy:
> [TieredMergePolicy: maxMergeAtOnce=47, maxMergeAtOnceExplicit=33,
> maxMergedSegmentMB=72.443359375, floorSegmentMB=1.203125,
> forceMergeDeletesPctAllowed=2.339102877818436, segmentsPerTier=40.0,
> maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0
>    [junit4]   2> 2142525 T6428 oasup.UpdateRequestProcessorChain.init
> inserting DistributedUpdateProcessorFactory into
> updateRequestProcessorChain "distrib-dup-test-chain-implicit"
>    [junit4]   2> 2142525 T6428 oasc.SolrCore.loadUpdateProcessorChains no
> updateRequestProcessorChain defined as default, creating implicit default
>    [junit4]   2> 2142525 T6357 oass.IndexSchema.readSchema unique key
> field: id
>    [junit4]   2> 2142524 T6396 oass.FileExchangeRateProvider.reload
> Reloading exchange rates from file currency.xml
>    [junit4]   2> 2142526 T6428 oasc.RequestHandlers.initHandlersFromConfig
> created /update/csv: org.apache.solr.handler.UpdateRequestHandler
>    [junit4]   2> 2142527 T6428 oasc.RequestHandlers.initHandlersFromConfig
> created /update/json: org.apache.solr.handler.UpdateRequestHandler
>    [junit4]   2> 2142527 T6428 oasc.RequestHandlers.initHandlersFromConfig
> created /update/json/docs: org.apache.solr.handler.UpdateRequestHandler
>    [junit4]   2> 2142528 T6428 oasc.RequestHandlers.initHandlersFromConfig
> created standard: solr.StandardRequestHandler
>    [junit4]   2> 2142528 T6428 oasc.RequestHandlers.initHandlersFromConfig
> created /get: solr.RealTimeGetHandler
>    [junit4]   2> 2142529 T6428 oasc.RequestHandlers.initHandlersFromConfig
> adding lazy requestHandler: solr.ReplicationHandler
>    [junit4]   2> 2142529 T6428 oasc.RequestHandlers.initHandlersFromConfig
> created /replication: solr.ReplicationHandler
>    [junit4]   2> 2142530 T6428 oasc.RequestHandlers.initHandlersFromConfig
> created /update: solr.UpdateRequestHandler
>    [junit4]   2> 2142530 T6428 oasc.RequestHandlers.initHandlersFromConfig
> created /admin/: org.apache.solr.handler.admin.AdminHandlers
>    [junit4]   2> 2142530 T6428 oasc.RequestHandlers.initHandlersFromConfig
> created /admin/file: solr.admin.ShowFileRequestHandler
>    [junit4]   2> 2142531 T6428 oasc.RequestHandlers.initHandlersFromConfig
> created /admin/fileedit: solr.admin.EditFileRequestHandler
>    [junit4]   2> 2142532 T6428 oashl.XMLLoader.init
> xsltCacheLifetimeSeconds=60
>    [junit4]   2> 2142533 T6428 oashl.XMLLoader.init
> xsltCacheLifetimeSeconds=60
>    [junit4]   2> 2142534 T6428 oashl.XMLLoader.init
> xsltCacheLifetimeSeconds=60
>    [junit4]   2> 2142534 T6357 oass.FileExchangeRateProvider.reload
> Reloading exchange rates from file currency.xml
>    [junit4]   2> 2142534 T6428 oashl.XMLLoader.init
> xsltCacheLifetimeSeconds=60
>    [junit4]   2> 2142525 T6410 oasc.SolrDeletionPolicy.onInit
> SolrDeletionPolicy.onInit: commits: num=1
>    [junit4]   2>
> commit{dir=MockDirectoryWrapper(RAMDirectory(id=lucene-5d5088)),segFN=segments_1,generation=1}
>    [junit4]   2> 2142536 T6410 oasc.SolrDeletionPolicy.updateCommits
> newest commit generation = 1
>    [junit4]   2> 2142536 T6357 oass.FileExchangeRateProvider.reload
> Reloading exchange rates from file currency.xml
>    [junit4]   2> 2142528 T6396 oass.FileExchangeRateProvider.reload
> Reloading exchange rates from file currency.xml
>    [junit4]   2> 2142536 T6410 oass.SolrIndexSearcher.<init> Opening
> Searcher@1bc1bc4[delLiveColl_shard2_replica1] main
>    [junit4]   2> 2142537 T6410 oascc.ZkStateReader.readConfigName Load
> collection config from:/collections/delLiveColl
>    [junit4]   2> 2142537 T6428 oasu.CommitTracker.<init> Hard AutoCommit:
> disabled
>    [junit4]   2> 2142538 T6428 oasu.CommitTracker.<init> Soft AutoCommit:
> disabled
>    [junit4]   2> 2142538 T6357 oass.OpenExchangeRatesOrgProvider.reload
> Reloading exchange rates from open-exchange-rates.json
>    [junit4]   2> 2142538 T6410 oascc.ZkStateReader.readConfigName
> path=/collections/delLiveColl configName=conf1 specified config exists in
> ZooKeeper
>    [junit4]   2> 2142540 T6410 oasr.ManagedResourceStorage.newStorageIO
> Setting up ZooKeeper-based storage for the RestManager with znodeBase:
> /configs/conf1
>    [junit4]   2> 2142540 T6357
> oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key
> IMPORTANT NOTE
>    [junit4]   2> 2142541 T6357
> oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected
> key, got STRING
>    [junit4]   2> 2142539 T6428 oasu.RandomMergePolicy.<init>
> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy:
> [TieredMergePolicy: maxMergeAtOnce=47, maxMergeAtOnc
>
> [...truncated too long message...]
>
> er.java:383)
>    [junit4]   2>        at
> org.apache.solr.cloud.Overseer$ClusterStateUpdater.access$300(Overseer.java:126)
>    [junit4]   2>        at
> org.apache.solr.cloud.Overseer$ClusterStateUpdater$1.run(Overseer.java:333)
>    [junit4]   2>
>    [junit4]   2> 2174408 T6340 oejsh.ContextHandler.doStop stopped
> o.e.j.s.ServletContextHandler{/b_y/tb,null}
>    [junit4]   2> 2179398 T6340 oas.SolrTestCaseJ4.tearDown ###Ending
> testDistribSearch
>    [junit4]   2> 2179399 T6340 oasc.ZkTestServer.send4LetterWord
> connecting to 127.0.0.1:46175 46175
>    [junit4]   2> 2179968 T6341 oasc.ZkTestServer.send4LetterWord
> connecting to 127.0.0.1:46175 46175
>    [junit4]   2> NOTE: reproduce with: ant test
> -Dtestcase=DeleteReplicaTest -Dtests.method=testDistribSearch
> -Dtests.seed=2301E08B7645B5A6 -Dtests.multiplier=3 -Dtests.slow=true
> -Dtests.locale=ar_TN -Dtests.timezone=America/Asuncion
> -Dtests.file.encoding=UTF-8
>    [junit4] ERROR   45.6s J1 | DeleteReplicaTest.testDistribSearch <<<
>    [junit4]    > Throwable #1:
> org.apache.solr.client.solrj.SolrServerException:
> java.lang.ClassCastException:
> org.apache.solr.common.params.CollectionParams$CollectionAction cannot be
> cast to java.lang.String
>    [junit4]    >        at
> __randomizedtesting.SeedInfo.seed([2301E08B7645B5A6:A2E76E93011AD59A]:0)
>    [junit4]    >        at
> org.apache.solr.client.solrj.impl.LBHttpSolrServer.doRequest(LBHttpSolrServer.java:381)
>    [junit4]    >        at
> org.apache.solr.client.solrj.impl.LBHttpSolrServer.request(LBHttpSolrServer.java:304)
>    [junit4]    >        at
> org.apache.solr.client.solrj.impl.CloudSolrServer.sendRequest(CloudSolrServer.java:874)
>    [junit4]    >        at
> org.apache.solr.client.solrj.impl.CloudSolrServer.requestWithRetryOnStaleState(CloudSolrServer.java:658)
>    [junit4]    >        at
> org.apache.solr.client.solrj.impl.CloudSolrServer.request(CloudSolrServer.java:601)
>    [junit4]    >        at
> org.apache.solr.cloud.DeleteReplicaTest.tryToRemoveOnlyIfDown(DeleteReplicaTest.java:154)
>    [junit4]    >        at
> org.apache.solr.cloud.DeleteReplicaTest.deleteLiveReplicaTest(DeleteReplicaTest.java:125)
>    [junit4]    >        at
> org.apache.solr.cloud.DeleteReplicaTest.doTest(DeleteReplicaTest.java:88)
>    [junit4]    >        at
> org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:869)
>    [junit4]    >        at java.lang.Thread.run(Thread.java:745)
>    [junit4]    > Caused by: java.lang.ClassCastException:
> org.apache.solr.common.params.CollectionParams$CollectionAction cannot be
> cast to java.lang.String
>    [junit4]    >        at
> org.apache.solr.common.params.MapSolrParams.getParams(MapSolrParams.java:43)
>    [junit4]    >        at
> org.apache.solr.common.params.ModifiableSolrParams.add(ModifiableSolrParams.java:117)
>    [junit4]    >        at
> org.apache.solr.common.params.ModifiableSolrParams.<init>(ModifiableSolrParams.java:56)
>    [junit4]    >        at
> org.apache.solr.client.solrj.impl.HttpSolrServer.createMethod(HttpSolrServer.java:273)
>    [junit4]    >        at
> org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:210)
>    [junit4]    >        at
> org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:206)
>    [junit4]    >        at
> org.apache.solr.client.solrj.impl.LBHttpSolrServer.doRequest(LBHttpSolrServer.java:343)
>    [junit4]    >        ... 47 more
>    [junit4]   2> 2179976 T6340 oas.SolrTestCaseJ4.deleteCore ###deleteCore
>    [junit4]   2> NOTE: leaving temporary files on disk at:
> /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.DeleteReplicaTest-2301E08B7645B5A6-001
>    [junit4]   2> 45623 T6339 ccr.ThreadLeakControl.checkThreadLeaks
> WARNING Will linger awaiting termination of 1 leaked thread(s).
>    [junit4]   2> NOTE: test params are: codec=Lucene410: {}, docValues:{},
> sim=RandomSimilarityProvider(queryNorm=true,coord=crazy): {}, locale=ar_TN,
> timezone=America/Asuncion
>    [junit4]   2> NOTE: Linux 3.13.0-35-generic i386/Oracle Corporation
> 1.8.0_20 (32-bit)/cpus=8,threads=1,free=99883032,total=302505984
>    [junit4]   2> NOTE: All tests run in this JVM: [DocValuesMissingTest,
> HdfsBasicDistributedZk2Test, TestCollapseQParserPlugin,
> DistribCursorPagingTest, TestRealTimeGet, ConvertedLegacyTest,
> LoggingHandlerTest, IndexSchemaRuntimeFieldTest, TestMiniSolrCloudCluster,
> TestXIncludeConfig, CoreContainerCoreInitFailuresTest,
> TestInfoStreamLogging, TestRandomMergePolicy, MultiThreadedOCPTest,
> TestCSVResponseWriter, TestRecovery, TriLevelCompositeIdRoutingTest,
> OpenCloseCoreStressTest, CachingDirectoryFactoryTest, AnalyticsQueryTest,
> TestSolrQueryParserResource, UpdateRequestProcessorFactoryTest,
> LegacyHTMLStripCharFilterTest, DirectUpdateHandlerOptimizeTest,
> TestReversedWildcardFilterFactory, TestCloudManagedSchemaConcurrent,
> TestQueryUtils, TermVectorComponentDistributedTest, TestNonNRTOpen,
> TestFieldTypeResource, UniqFieldsUpdateProcessorFactoryTest,
> TermVectorComponentTest, TestDistributedMissingSort,
> QueryElevationComponentTest, TestFunctionQuery, SOLR749Test,
> SortByFunctionTest, URLClassifyProcessorTest, BasicDistributedZk2Test,
> SynonymTokenizerTest, BlockCacheTest, SimplePostToolTest,
> TestSolrQueryParser, TestAtomicUpdateErrorCases, SampleTest,
> ClusterStateTest, OverseerRolesTest, HardAutoCommitTest, TestFiltering,
> TestJoin, QueryParsingTest, PolyFieldTest, TestUpdate,
> DistanceFunctionTest, TestManagedStopFilterFactory,
> DistributedTermsComponentTest, TestSchemaSimilarityResource,
> ChangedSchemaMergeTest, TestElisionMultitermQuery, OutputWriterTest,
> ExternalCollectionsTest, DeleteInactiveReplicaTest, TestQueryTypes,
> TestSearcherReuse, ScriptEngineTest, FileBasedSpellCheckerTest, TestUtils,
> TestManagedSchema, TestPHPSerializedResponseWriter, TestRandomFaceting,
> LeaderElectionIntegrationTest, ZkSolrClientTest, SolrPluginUtilsTest,
> TestIndexSearcher, TestCharFilters, TestReRankQParserPlugin,
> LukeRequestHandlerTest, TestPhraseSuggestions, CursorMarkTest,
> RankQueryTest, CopyFieldTest, DeleteShardTest, TestSchemaResource,
> TestManagedSchemaFieldResource, DistributedFacetPivotSmallTest,
> DeleteLastCustomShardedReplicaTest, ResourceLoaderTest, TestBadConfig,
> ChaosMonkeySafeLeaderTest, TestNonDefinedSimilarityFactory,
> TestNoOpRegenerator, TestSimpleQParserPlugin, OverseerStatusTest,
> DistributedDebugComponentTest, HdfsUnloadDistributedZkTest,
> RAMDirectoryFactoryTest, PreAnalyzedUpdateProcessorTest,
> TestManagedResource, HdfsChaosMonkeySafeLeaderTest,
> TestSchemaVersionResource, SpatialFilterTest, TestStressLucene,
> SearchHandlerTest, TestFieldSortValues, UnloadDistributedZkTest,
> CollectionsAPIDistributedZkTest, LeaderElectionTest, TestZkChroot,
> ShardRoutingCustomTest, AutoCommitTest, TestDistributedGrouping,
> TestStressReorder, TestRangeQuery, TestSolr4Spatial, StatsComponentTest,
> PeerSyncTest, BasicFunctionalityTest, DirectUpdateHandlerTest,
> HighlighterTest, SoftAutoCommitTest, CurrencyFieldXmlFileTest,
> SolrIndexSplitterTest, TestCoreDiscovery, AnalysisAfterCoreReloadTest,
> TestExtendedDismaxParser, SuggesterFSTTest, SolrRequestParserTest,
> TestFoldingMultitermQuery, DocValuesTest, SuggesterTest, TestTrie,
> SuggesterWFSTTest, TestCSVLoader, WordBreakSolrSpellCheckerTest,
> SchemaVersionSpecificBehaviorTest, TestPseudoReturnFields,
> QueryEqualityTest, DefaultValueUpdateProcessorTest,
> StatelessScriptUpdateProcessorFactoryTest, XsltUpdateRequestHandlerTest,
> StandardRequestHandlerTest, TestWriterPerf, DirectSolrSpellCheckerTest,
> TestOmitPositions, RequestHandlersTest, TestValueSourceCache,
> TestIndexingPerformance, RequiredFieldsTest, RegexBoostProcessorTest,
> TestCollationField, JSONWriterTest, JsonLoaderTest,
> TestPartialUpdateDeduplication, CSVRequestHandlerTest,
> TestBinaryResponseWriter, TestQuerySenderListener, AlternateDirectoryTest,
> SolrIndexConfigTest, TestMergePolicyConfig, MultiTermTest, TestDocSet,
> TestBinaryField, TestSearchPerf, NumericFieldsTest,
> ExternalFileFieldSortTest, TestPostingsSolrHighlighter,
> DirectSolrConnectionTest, TestLuceneMatchVersion,
> SpellPossibilityIteratorTest, TestSweetSpotSimilarityFactory,
> TestPerFieldSimilarity, TestLMDirichletSimilarityFactory,
> TestLMJelinekMercerSimilarityFactory, TestIBSimilarityFactory,
> TestBM25SimilarityFactory, TestDefaultSimilarityFactory, TestFastWriter,
> TestJmxMonitoredMap, TestFastOutputStream, PluginInfoTest,
> DateMathParserTest, PreAnalyzedFieldTest, PrimUtilsTest,
> SpellingQueryConverterTest, TestSolrJ, TestDocumentBuilder, SliceStateTest,
> UUIDFieldTest, CircularListTest, CursorPagingTest, SolrTestCaseJ4Test,
> TestCursorMarkWithoutUniqueKey, TestHighlightDedupGrouping,
> TestTolerantSearch, AliasIntegrationTest, AsyncMigrateRouteKeyTest,
> DeleteReplicaTest]
>    [junit4] Completed on J1 in 45.89s, 1 test, 1 error <<< FAILURES!
>
> [...truncated 241 lines...]
> BUILD FAILED
> /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/build.xml:491: The
> following error occurred while executing this line:
> /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/build.xml:471: The
> following error occurred while executing this line:
> /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/build.xml:60: The
> following error occurred while executing this line:
> /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/extra-targets.xml:39:
> The following error occurred while executing this line:
> /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build.xml:189: The
> following error occurred while executing this line:
> /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/common-build.xml:496:
> The following error occurred while executing this line:
> /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/common-build.xml:1358:
> The following error occurred while executing this line:
> /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/common-build.xml:961:
> There were test failures: 429 suites, 1772 tests, 1 error, 45 ignored (23
> assumptions)
>
> Total time: 96 minutes 21 seconds
> Build step 'Invoke Ant' marked build as failure
> [description-setter] Description set: Java: 32bit/jdk1.8.0_20 -server
> -XX:+UseSerialGC
> Archiving artifacts
> Recording test results
> Email was triggered for: Failure - Any
> Sending email for trigger: Failure - Any
>
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: dev-help@lucene.apache.org
>



-- 
Regards,
Shalin Shekhar Mangar.

Re: [JENKINS] Lucene-Solr-trunk-Linux (64bit/ibm-j9-jdk7) - Build # 11281 - Still Failing!

Posted by Michael McCandless <lu...@mikemccandless.com>.
Uwe fixed this one ...

Mike McCandless

http://blog.mikemccandless.com


On Thu, Sep 18, 2014 at 10:41 AM, Policeman Jenkins Server
<je...@thetaphi.de> wrote:
> Build: http://jenkins.thetaphi.de/job/Lucene-Solr-trunk-Linux/11281/
> Java: 64bit/ibm-j9-jdk7 -Xjit:exclude={org/apache/lucene/util/fst/FST.pack(IIF)Lorg/apache/lucene/util/fst/FST;}
>
> 1 tests failed.
> REGRESSION:  org.apache.lucene.util.TestVersion.testParseExceptions
>
> Error Message:
>
>
> Stack Trace:
> java.lang.AssertionError
>         at __randomizedtesting.SeedInfo.seed([9253AB30E3046055:9503F9004509D07A]:0)
>         at org.junit.Assert.fail(Assert.java:92)
>         at org.junit.Assert.fail(Assert.java:100)
>         at org.apache.lucene.util.TestVersion.testParseExceptions(TestVersion.java:201)
>         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>         at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:94)
>         at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:55)
>         at java.lang.reflect.Method.invoke(Method.java:619)
>         at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1618)
>         at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:827)
>         at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
>         at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:877)
>         at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
>         at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
>         at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
>         at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
>         at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
>         at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
>         at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>         at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
>         at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798)
>         at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458)
>         at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:836)
>         at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:738)
>         at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:772)
>         at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:783)
>         at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
>         at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
>         at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
>         at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
>         at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
>         at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>         at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>         at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>         at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
>         at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
>         at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
>         at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
>         at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>         at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
>         at java.lang.Thread.run(Thread.java:853)
>
>
>
>
> Build Log:
> [...truncated 1543 lines...]
>    [junit4] Suite: org.apache.lucene.util.TestVersion
>    [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=TestVersion -Dtests.method=testParseExceptions -Dtests.seed=9253AB30E3046055 -Dtests.multiplier=3 -Dtests.slow=true -Dtests.locale=nb_NO -Dtests.timezone=Africa/Ndjamena -Dtests.file.encoding=ISO-8859-1
>    [junit4] FAILURE 0.02s J0 | TestVersion.testParseExceptions <<<
>    [junit4]    > Throwable #1: java.lang.AssertionError
>    [junit4]    >        at __randomizedtesting.SeedInfo.seed([9253AB30E3046055:9503F9004509D07A]:0)
>    [junit4]    >        at org.apache.lucene.util.TestVersion.testParseExceptions(TestVersion.java:201)
>    [junit4]    >        at java.lang.Thread.run(Thread.java:853)
>    [junit4]   2> NOTE: test params are: codec=Lucene410: {}, docValues:{}, sim=DefaultSimilarity, locale=nb_NO, timezone=Africa/Ndjamena
>    [junit4]   2> NOTE: Linux 3.13.0-35-generic amd64/IBM Corporation 1.7.0 (64-bit)/cpus=8,threads=1,free=74944848,total=257949696
>    [junit4]   2> NOTE: All tests run in this JVM: [TestMergedIterator, TestPersistentSnapshotDeletionPolicy, TestMultiFields, TestNewestSegment, TestQueryRescorer, TestTopFieldCollector, TestNoMergeScheduler, TestTermVectorsWriter, TestPrefixInBooleanQuery, TestSpanMultiTermQueryWrapper, TestPackedTokenAttributeImpl, Test2BPositions, TestStressDeletes, TestAttributeSource, TestRecyclingByteBlockAllocator, TestOmitTf, TestTransactions, TestNRTCachingDirectory, TestTermsEnum2, TestFSTs, TestDocumentsWriterDeleteQueue, TestMixedDocValuesUpdates, TestPrefixRandom, TestWorstCaseTestBehavior, TestRegexpQuery, TestFieldCacheRangeFilter, TestMinimize, TestAutomatonQuery, TestSortedNumericSortField, Nested1, Nested2, Nested3, TestDocTermOrdsRangeFilter, TestSimpleFSDirectory, TestDateSort, TestSloppyPhraseQuery2, TestFieldsReader, TestIndexWriterOutOfMemory, TestIndexWriterNRTIsCurrent, TestSpanExplanations, TestPayloadExplanations, TestTermScorer, TestNRTThreads, TestNGramPhraseQuery, TestDoubleBarrelLRUCache, TestDeterminizeLexicon, TestAddIndexes, TestSortedSetSortField, TestOrdinalMap, TestRateLimitedDirectoryWrapper, TestSloppyMath, TestTermRangeFilter, TestSort, TestMockAnalyzer, TestOpenBitSet, TestPrefixCodedTerms, TestTwoPhaseCommitTool, TestIndexWriterExceptions2, TestFilteredSearch, TestBufferedIndexInput, TestMultiPhraseQuery, Nested, TestDocInverterPerFieldErrorInfo, TestFieldReuse, TestNRTReaderWithThreads, TestRegexpRandom, TestStressIndexing2, TestPerFieldPostingsFormat2, TestPayloadSpans, TestPackedInts, TestCustomNorms, TestCodecHoldsOpenFiles, TestLongPostings, TestBufferedChecksum, TestConsistentFieldNumbers, TestRollback, TestSubScorerFreqs, TestSortedSetSelector, TestPhrasePrefixQuery, TestSentinelIntSet, TestSloppyPhraseQuery, Before3, Before3, TestIntBlockPool, TestMmapDirectory, TestIndexReaderClose, TestPriorityQueue, TestBroadWord, TestStressAdvance, TestIdentityHashSet, TestMixedCodecs, TestFilterAtomicReader, TestIndexWriter, TestTopScoreDocCollector, TestThreadedForceMerge, TestDateTools, TestSegmentTermDocs, TestNumericRangeQuery32, TestBytesRef, TestWildcard, TestLock, TestIndexWriterReader, TestCompressingTermVectorsFormat, TestBytesStore, TestGraphTokenizers, TestIndexWriterExceptions, TestMultiMMap, TestDeletionPolicy, TestBooleanOr, TestIndexWriterDelete, TestIndexWriterMergePolicy, TestDuelingCodecs, TestAtomicUpdate, TestTimeLimitingCollector, TestSearchAfter, TestSimpleExplanations, TestDirectoryReader, TestCompressingStoredFieldsFormat, TestLockFactory, TestSegmentMerger, TestPhraseQuery, TestCollectionUtil, TestBagOfPositions, TestOmitNorms, TestComplexExplanations, TestRegexpRandom2, TestSegmentReader, TestFastDecompressionMode, TestFieldCacheRewriteMethod, TestCrash, TestMultiThreadTermVectors, TestMultiDocValues, TestToken, TestCustomSearcherSort, TestPostingsFormat, TestSimilarityBase, TestBooleanMinShouldMatch, TestNumericUtils, TestIndexWriterOnDiskFull, TestPostingsOffsets, TestTransactionRollback, TestFlex, TestUTF32ToUTF8, TestFastCompressionMode, TestLazyProxSkipping, TestDeterminism, TestDocsAndPositions, TestMergeSchedulerExternal, TestBagOfPostings, TestCodecs, TestCharTermAttributeImpl, TestParallelAtomicReader, TestStressNRT, TestIndexInput, TestFilteredQuery, TestForceMergeForever, TestSameScoresWithThreads, TestQueryWrapperFilter, TestPayloadNearQuery, TestDocumentWriter, TestCrashCausesCorruptIndex, TestMultiTermConstantScore, TestLiveFieldValues, Nested, Nested, Nested, Nested, Nested, Nested, Nested, Nested, Nested, Nested, Nested, Nested, Nested, Nested, Nested, Nested, Nested, Nested, Nested, Nested, Nested, TestFuzzyQuery, TestPayloads, TestSimilarity2, TestExternalCodecs, TestSimpleExplanationsOfNonMatches, TestTermVectors, TestTermVectorsReader, TestSpansAdvanced2, TestPerSegmentDeletes, TestSegmentTermEnum, TestUniqueTermCount, TestComplexExplanationsOfNonMatches, TestUnicodeUtil, TestSortRandom, TestAllFilesHaveCodecHeader, TestParallelReaderEmptyIndex, TestDocument, TestMaxTermFrequency, TestFileSwitchDirectory, TestIndexableField, TestWindowsMMap, TestWildcardRandom, TestDirectory, TestMultiLevelSkipList, TestPayloadsOnVectors, TestFieldValueFilter, TestBooleanQueryVisitSubscorers, TestSearchForDuplicates, InBeforeClass, InAfterClass, InTestMethod, NonStringProperties, TestSizeBoundedForceMerge, TestIndexFileDeleter, TestCompiledAutomaton, TestConstantScoreQuery, TestParallelTermEnum, TestDocIdSet, TestIndexWriterConfig, TestConjunctions, TestMultiTermQueryRewrites, TestPositionIncrement, Nested, Nested, TestSimilarity, TestSpanExplanationsOfNonMatches, TestRollingBuffer, TestSmallFloat, TestSimilarityProvider, TestSearch, TestCachingTokenFilter, ThrowInUncaught, TestDateFilter, TestCharsRef, TestSameTokenSamePosition, TestDocCount, TestCachingCollector, TestAutomatonQueryUnicode, TestRamUsageEstimator, TestSpanFirstQuery, TestNamedSPILoader, TestDemo, TestExplanations, TestMockCharFilter, TestVersion]
>    [junit4] Completed on J0 in 0.07s, 11 tests, 1 failure <<< FAILURES!
>
> [...truncated 194 lines...]
> BUILD FAILED
> /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/build.xml:491: The following error occurred while executing this line:
> /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/build.xml:484: The following error occurred while executing this line:
> /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/build.xml:60: The following error occurred while executing this line:
> /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/extra-targets.xml:39: The following error occurred while executing this line:
> /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/build.xml:49: The following error occurred while executing this line:
> /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/common-build.xml:1358: The following error occurred while executing this line:
> /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/common-build.xml:961: There were test failures: 395 suites, 3129 tests, 1 failure, 61 ignored (50 assumptions)
>
> Total time: 24 minutes 34 seconds
> Build step 'Invoke Ant' marked build as failure
> [description-setter] Description set: Java: 64bit/ibm-j9-jdk7 -Xjit:exclude={org/apache/lucene/util/fst/FST.pack(IIF)Lorg/apache/lucene/util/fst/FST;}
> Archiving artifacts
> Recording test results
> Email was triggered for: Failure - Any
> Sending email for trigger: Failure - Any
>

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org


[JENKINS] Lucene-Solr-trunk-Linux (64bit/ibm-j9-jdk7) - Build # 11281 - Still Failing!

Posted by Policeman Jenkins Server <je...@thetaphi.de>.
Build: http://jenkins.thetaphi.de/job/Lucene-Solr-trunk-Linux/11281/
Java: 64bit/ibm-j9-jdk7 -Xjit:exclude={org/apache/lucene/util/fst/FST.pack(IIF)Lorg/apache/lucene/util/fst/FST;}

1 tests failed.
REGRESSION:  org.apache.lucene.util.TestVersion.testParseExceptions

Error Message:


Stack Trace:
java.lang.AssertionError
	at __randomizedtesting.SeedInfo.seed([9253AB30E3046055:9503F9004509D07A]:0)
	at org.junit.Assert.fail(Assert.java:92)
	at org.junit.Assert.fail(Assert.java:100)
	at org.apache.lucene.util.TestVersion.testParseExceptions(TestVersion.java:201)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:94)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:55)
	at java.lang.reflect.Method.invoke(Method.java:619)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1618)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:827)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:877)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:836)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:738)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:772)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:783)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
	at java.lang.Thread.run(Thread.java:853)




Build Log:
[...truncated 1543 lines...]
   [junit4] Suite: org.apache.lucene.util.TestVersion
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=TestVersion -Dtests.method=testParseExceptions -Dtests.seed=9253AB30E3046055 -Dtests.multiplier=3 -Dtests.slow=true -Dtests.locale=nb_NO -Dtests.timezone=Africa/Ndjamena -Dtests.file.encoding=ISO-8859-1
   [junit4] FAILURE 0.02s J0 | TestVersion.testParseExceptions <<<
   [junit4]    > Throwable #1: java.lang.AssertionError
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([9253AB30E3046055:9503F9004509D07A]:0)
   [junit4]    > 	at org.apache.lucene.util.TestVersion.testParseExceptions(TestVersion.java:201)
   [junit4]    > 	at java.lang.Thread.run(Thread.java:853)
   [junit4]   2> NOTE: test params are: codec=Lucene410: {}, docValues:{}, sim=DefaultSimilarity, locale=nb_NO, timezone=Africa/Ndjamena
   [junit4]   2> NOTE: Linux 3.13.0-35-generic amd64/IBM Corporation 1.7.0 (64-bit)/cpus=8,threads=1,free=74944848,total=257949696
   [junit4]   2> NOTE: All tests run in this JVM: [TestMergedIterator, TestPersistentSnapshotDeletionPolicy, TestMultiFields, TestNewestSegment, TestQueryRescorer, TestTopFieldCollector, TestNoMergeScheduler, TestTermVectorsWriter, TestPrefixInBooleanQuery, TestSpanMultiTermQueryWrapper, TestPackedTokenAttributeImpl, Test2BPositions, TestStressDeletes, TestAttributeSource, TestRecyclingByteBlockAllocator, TestOmitTf, TestTransactions, TestNRTCachingDirectory, TestTermsEnum2, TestFSTs, TestDocumentsWriterDeleteQueue, TestMixedDocValuesUpdates, TestPrefixRandom, TestWorstCaseTestBehavior, TestRegexpQuery, TestFieldCacheRangeFilter, TestMinimize, TestAutomatonQuery, TestSortedNumericSortField, Nested1, Nested2, Nested3, TestDocTermOrdsRangeFilter, TestSimpleFSDirectory, TestDateSort, TestSloppyPhraseQuery2, TestFieldsReader, TestIndexWriterOutOfMemory, TestIndexWriterNRTIsCurrent, TestSpanExplanations, TestPayloadExplanations, TestTermScorer, TestNRTThreads, TestNGramPhraseQuery, TestDoubleBarrelLRUCache, TestDeterminizeLexicon, TestAddIndexes, TestSortedSetSortField, TestOrdinalMap, TestRateLimitedDirectoryWrapper, TestSloppyMath, TestTermRangeFilter, TestSort, TestMockAnalyzer, TestOpenBitSet, TestPrefixCodedTerms, TestTwoPhaseCommitTool, TestIndexWriterExceptions2, TestFilteredSearch, TestBufferedIndexInput, TestMultiPhraseQuery, Nested, TestDocInverterPerFieldErrorInfo, TestFieldReuse, TestNRTReaderWithThreads, TestRegexpRandom, TestStressIndexing2, TestPerFieldPostingsFormat2, TestPayloadSpans, TestPackedInts, TestCustomNorms, TestCodecHoldsOpenFiles, TestLongPostings, TestBufferedChecksum, TestConsistentFieldNumbers, TestRollback, TestSubScorerFreqs, TestSortedSetSelector, TestPhrasePrefixQuery, TestSentinelIntSet, TestSloppyPhraseQuery, Before3, Before3, TestIntBlockPool, TestMmapDirectory, TestIndexReaderClose, TestPriorityQueue, TestBroadWord, TestStressAdvance, TestIdentityHashSet, TestMixedCodecs, TestFilterAtomicReader, TestIndexWriter, TestTopScoreDocCollector, TestThreadedForceMerge, TestDateTools, TestSegmentTermDocs, TestNumericRangeQuery32, TestBytesRef, TestWildcard, TestLock, TestIndexWriterReader, TestCompressingTermVectorsFormat, TestBytesStore, TestGraphTokenizers, TestIndexWriterExceptions, TestMultiMMap, TestDeletionPolicy, TestBooleanOr, TestIndexWriterDelete, TestIndexWriterMergePolicy, TestDuelingCodecs, TestAtomicUpdate, TestTimeLimitingCollector, TestSearchAfter, TestSimpleExplanations, TestDirectoryReader, TestCompressingStoredFieldsFormat, TestLockFactory, TestSegmentMerger, TestPhraseQuery, TestCollectionUtil, TestBagOfPositions, TestOmitNorms, TestComplexExplanations, TestRegexpRandom2, TestSegmentReader, TestFastDecompressionMode, TestFieldCacheRewriteMethod, TestCrash, TestMultiThreadTermVectors, TestMultiDocValues, TestToken, TestCustomSearcherSort, TestPostingsFormat, TestSimilarityBase, TestBooleanMinShouldMatch, TestNumericUtils, TestIndexWriterOnDiskFull, TestPostingsOffsets, TestTransactionRollback, TestFlex, TestUTF32ToUTF8, TestFastCompressionMode, TestLazyProxSkipping, TestDeterminism, TestDocsAndPositions, TestMergeSchedulerExternal, TestBagOfPostings, TestCodecs, TestCharTermAttributeImpl, TestParallelAtomicReader, TestStressNRT, TestIndexInput, TestFilteredQuery, TestForceMergeForever, TestSameScoresWithThreads, TestQueryWrapperFilter, TestPayloadNearQuery, TestDocumentWriter, TestCrashCausesCorruptIndex, TestMultiTermConstantScore, TestLiveFieldValues, Nested, Nested, Nested, Nested, Nested, Nested, Nested, Nested, Nested, Nested, Nested, Nested, Nested, Nested, Nested, Nested, Nested, Nested, Nested, Nested, Nested, TestFuzzyQuery, TestPayloads, TestSimilarity2, TestExternalCodecs, TestSimpleExplanationsOfNonMatches, TestTermVectors, TestTermVectorsReader, TestSpansAdvanced2, TestPerSegmentDeletes, TestSegmentTermEnum, TestUniqueTermCount, TestComplexExplanationsOfNonMatches, TestUnicodeUtil, TestSortRandom, TestAllFilesHaveCodecHeader, TestParallelReaderEmptyIndex, TestDocument, TestMaxTermFrequency, TestFileSwitchDirectory, TestIndexableField, TestWindowsMMap, TestWildcardRandom, TestDirectory, TestMultiLevelSkipList, TestPayloadsOnVectors, TestFieldValueFilter, TestBooleanQueryVisitSubscorers, TestSearchForDuplicates, InBeforeClass, InAfterClass, InTestMethod, NonStringProperties, TestSizeBoundedForceMerge, TestIndexFileDeleter, TestCompiledAutomaton, TestConstantScoreQuery, TestParallelTermEnum, TestDocIdSet, TestIndexWriterConfig, TestConjunctions, TestMultiTermQueryRewrites, TestPositionIncrement, Nested, Nested, TestSimilarity, TestSpanExplanationsOfNonMatches, TestRollingBuffer, TestSmallFloat, TestSimilarityProvider, TestSearch, TestCachingTokenFilter, ThrowInUncaught, TestDateFilter, TestCharsRef, TestSameTokenSamePosition, TestDocCount, TestCachingCollector, TestAutomatonQueryUnicode, TestRamUsageEstimator, TestSpanFirstQuery, TestNamedSPILoader, TestDemo, TestExplanations, TestMockCharFilter, TestVersion]
   [junit4] Completed on J0 in 0.07s, 11 tests, 1 failure <<< FAILURES!

[...truncated 194 lines...]
BUILD FAILED
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/build.xml:491: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/build.xml:484: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/build.xml:60: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/extra-targets.xml:39: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/build.xml:49: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/common-build.xml:1358: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/common-build.xml:961: There were test failures: 395 suites, 3129 tests, 1 failure, 61 ignored (50 assumptions)

Total time: 24 minutes 34 seconds
Build step 'Invoke Ant' marked build as failure
[description-setter] Description set: Java: 64bit/ibm-j9-jdk7 -Xjit:exclude={org/apache/lucene/util/fst/FST.pack(IIF)Lorg/apache/lucene/util/fst/FST;}
Archiving artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any



[JENKINS] Lucene-Solr-trunk-Linux (64bit/jdk1.8.0_20) - Build # 11280 - Still Failing!

Posted by Policeman Jenkins Server <je...@thetaphi.de>.
Build: http://jenkins.thetaphi.de/job/Lucene-Solr-trunk-Linux/11280/
Java: 64bit/jdk1.8.0_20 -XX:+UseCompressedOops -XX:+UseParallelGC

All tests passed

Build Log:
[...truncated 60177 lines...]
-documentation-lint:
    [jtidy] Checking for broken html (such as invalid tags)...
   [delete] Deleting directory /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/build/jtidy_tmp
     [echo] Checking for broken links...
     [exec] 
     [exec] Crawl/parse...
     [exec] 
     [exec] Verify...
     [echo] Checking for malformed docs...
     [exec] 
     [exec] /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/docs/solr-analytics/overview-summary.html
     [exec]   missing: org.apache.solr.handler.component
     [exec] 
     [exec] Missing javadocs were found!

BUILD FAILED
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/build.xml:491: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/build.xml:78: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build.xml:548: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build.xml:564: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/common-build.xml:2471: exec returned: 1

Total time: 111 minutes 19 seconds
Build step 'Invoke Ant' marked build as failure
[description-setter] Description set: Java: 64bit/jdk1.8.0_20 -XX:+UseCompressedOops -XX:+UseParallelGC
Archiving artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any