You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by Apache Jenkins Server <je...@builds.apache.org> on 2013/10/15 11:47:16 UTC

[JENKINS] Lucene-Solr-Tests-4.x-Java7 - Build # 1680 - Failure

Build: https://builds.apache.org/job/Lucene-Solr-Tests-4.x-Java7/1680/

1 tests failed.
REGRESSION:  org.apache.solr.cloud.SyncSliceTest.testDistribSearch

Error Message:
Test Setup Failure: shard1 should have just been set up to be inconsistent - but it's still consistent. Leader:http://127.0.0.1:33880/collection1 Dead Guy:http://127.0.0.1:16681/collection1skip list:[CloudJettyRunner [url=http://127.0.0.1:61551/collection1], CloudJettyRunner [url=http://127.0.0.1:61551/collection1]]

Stack Trace:
java.lang.AssertionError: Test Setup Failure: shard1 should have just been set up to be inconsistent - but it's still consistent. Leader:http://127.0.0.1:33880/collection1 Dead Guy:http://127.0.0.1:16681/collection1skip list:[CloudJettyRunner [url=http://127.0.0.1:61551/collection1], CloudJettyRunner [url=http://127.0.0.1:61551/collection1]]
	at __randomizedtesting.SeedInfo.seed([38EEFE0AB3312BCF:B9087012C46E4BF3]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.junit.Assert.assertTrue(Assert.java:43)
	at org.junit.Assert.assertNotNull(Assert.java:526)
	at org.apache.solr.cloud.SyncSliceTest.doTest(SyncSliceTest.java:216)
	at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:835)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1559)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.access$600(RandomizedRunner.java:79)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:737)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:773)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:787)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:782)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:442)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:746)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:648)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:682)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:693)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
	at java.lang.Thread.run(Thread.java:724)




Build Log:
[...truncated 10164 lines...]
   [junit4] Suite: org.apache.solr.cloud.SyncSliceTest
   [junit4]   2> 1336138 T2529 oas.BaseDistributedSearchTestCase.initHostContext Setting hostContext system property: /
   [junit4]   2> 1336145 T2529 oas.SolrTestCaseJ4.setUp ###Starting testDistribSearch
   [junit4]   2> Creating dataDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J0/./solrtest-SyncSliceTest-1381829698890
   [junit4]   2> 1336147 T2529 oasc.ZkTestServer.run STARTING ZK TEST SERVER
   [junit4]   2> 1336148 T2530 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
   [junit4]   2> 1336248 T2529 oasc.ZkTestServer.run start zk server on port:22135
   [junit4]   2> 1336248 T2529 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 1336253 T2536 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@22afeee0 name:ZooKeeperConnection Watcher:127.0.0.1:22135 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1336253 T2529 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 1336254 T2529 oascc.SolrZkClient.makePath makePath: /solr
   [junit4]   2> 1336268 T2529 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 1336270 T2538 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@52d042f9 name:ZooKeeperConnection Watcher:127.0.0.1:22135/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1336270 T2529 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 1336270 T2529 oascc.SolrZkClient.makePath makePath: /collections/collection1
   [junit4]   2> 1336275 T2529 oascc.SolrZkClient.makePath makePath: /collections/collection1/shards
   [junit4]   2> 1336284 T2529 oascc.SolrZkClient.makePath makePath: /collections/control_collection
   [junit4]   2> 1336286 T2529 oascc.SolrZkClient.makePath makePath: /collections/control_collection/shards
   [junit4]   2> 1336289 T2529 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
   [junit4]   2> 1336290 T2529 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.xml
   [junit4]   2> 1336300 T2529 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
   [junit4]   2> 1336301 T2529 oascc.SolrZkClient.makePath makePath: /configs/conf1/schema.xml
   [junit4]   2> 1336404 T2529 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 1336405 T2529 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 1336408 T2529 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
   [junit4]   2> 1336409 T2529 oascc.SolrZkClient.makePath makePath: /configs/conf1/stopwords.txt
   [junit4]   2> 1336419 T2529 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
   [junit4]   2> 1336419 T2529 oascc.SolrZkClient.makePath makePath: /configs/conf1/protwords.txt
   [junit4]   2> 1336423 T2529 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
   [junit4]   2> 1336423 T2529 oascc.SolrZkClient.makePath makePath: /configs/conf1/currency.xml
   [junit4]   2> 1336427 T2529 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
   [junit4]   2> 1336427 T2529 oascc.SolrZkClient.makePath makePath: /configs/conf1/open-exchange-rates.json
   [junit4]   2> 1336431 T2529 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 1336431 T2529 oascc.SolrZkClient.makePath makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 1336434 T2529 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
   [junit4]   2> 1336435 T2529 oascc.SolrZkClient.makePath makePath: /configs/conf1/old_synonyms.txt
   [junit4]   2> 1336438 T2529 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
   [junit4]   2> 1336439 T2529 oascc.SolrZkClient.makePath makePath: /configs/conf1/synonyms.txt
   [junit4]   2> 1336731 T2529 oejs.Server.doStart jetty-8.1.10.v20130312
   [junit4]   2> 1336735 T2529 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:63121
   [junit4]   2> 1336736 T2529 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
   [junit4]   2> 1336737 T2529 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 1336737 T2529 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ../../../../../../../../../../../usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-controljetty-1381829699193
   [junit4]   2> 1336738 T2529 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../../../usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-controljetty-1381829699193/'
   [junit4]   2> 1336779 T2529 oasc.ConfigSolr.fromFile Loading container configuration from /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J0/../../../../../../../../../../../usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-controljetty-1381829699193/solr.xml
   [junit4]   2> 1336900 T2529 oasc.CoreContainer.<init> New CoreContainer 416006945
   [junit4]   2> 1336901 T2529 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=../../../../../../../../../../../usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-controljetty-1381829699193/]
   [junit4]   2> 1336903 T2529 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
   [junit4]   2> 1336903 T2529 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
   [junit4]   2> 1336904 T2529 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
   [junit4]   2> 1336904 T2529 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
   [junit4]   2> 1336905 T2529 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
   [junit4]   2> 1336905 T2529 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
   [junit4]   2> 1336906 T2529 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
   [junit4]   2> 1336906 T2529 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
   [junit4]   2> 1336907 T2529 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
   [junit4]   2> 1336907 T2529 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
   [junit4]   2> 1336914 T2529 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 1336914 T2529 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 1336915 T2529 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:22135/solr
   [junit4]   2> 1336916 T2529 oasc.ZkController.checkChrootPath zkHost includes chroot
   [junit4]   2> 1336917 T2529 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 1336921 T2549 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@5b25fa1e name:ZooKeeperConnection Watcher:127.0.0.1:22135 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1336921 T2529 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 1336931 T2529 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
   [junit4]   2> 1336933 T2529 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 1336935 T2551 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@376bc47 name:ZooKeeperConnection Watcher:127.0.0.1:22135/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1336935 T2529 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 1336938 T2529 oascc.SolrZkClient.makePath makePath: /overseer/queue
   [junit4]   2> 1336950 T2529 oascc.SolrZkClient.makePath makePath: /overseer/collection-queue-work
   [junit4]   2> 1336953 T2529 oascc.SolrZkClient.makePath makePath: /live_nodes
   [junit4]   2> 1336955 T2529 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:63121_
   [junit4]   2> 1336962 T2529 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:63121_
   [junit4]   2> 1336965 T2529 oascc.SolrZkClient.makePath makePath: /overseer_elect/election
   [junit4]   2> 1336980 T2529 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
   [junit4]   2> 1336987 T2529 oasc.Overseer.start Overseer (id=90559591146848259-127.0.0.1:63121_-n_0000000000) starting
   [junit4]   2> 1336992 T2529 oascc.SolrZkClient.makePath makePath: /overseer/queue-work
   [junit4]   2> 1337004 T2553 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
   [junit4]   2> 1337005 T2529 oascc.SolrZkClient.makePath makePath: /clusterstate.json
   [junit4]   2> 1337008 T2529 oascc.SolrZkClient.makePath makePath: /aliases.json
   [junit4]   2> 1337010 T2529 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 1337012 T2552 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
   [junit4]   2> 1337018 T2554 oasc.ZkController.publish publishing core=collection1 state=down
   [junit4]   2> 1337018 T2554 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
   [junit4]   2> 1337020 T2554 oasc.ZkController.waitForCoreNodeName look for our core node name
   [junit4]   2> 1338516 T2552 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 1338517 T2552 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
   [junit4]   2> 	  "operation":"state",
   [junit4]   2> 	  "state":"down",
   [junit4]   2> 	  "base_url":"http://127.0.0.1:63121",
   [junit4]   2> 	  "core":"collection1",
   [junit4]   2> 	  "roles":null,
   [junit4]   2> 	  "node_name":"127.0.0.1:63121_",
   [junit4]   2> 	  "shard":null,
   [junit4]   2> 	  "shard_range":null,
   [junit4]   2> 	  "shard_state":"active",
   [junit4]   2> 	  "shard_parent":null,
   [junit4]   2> 	  "collection":"control_collection",
   [junit4]   2> 	  "numShards":"1",
   [junit4]   2> 	  "core_node_name":"core_node1"}
   [junit4]   2> 1338518 T2552 oasc.Overseer$ClusterStateUpdater.createCollection Create collection control_collection with shards [shard1]
   [junit4]   2> 1338518 T2552 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
   [junit4]   2> 1338531 T2551 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1339023 T2554 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
   [junit4]   2> 1339023 T2554 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: ../../../../../../../../../../../usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-controljetty-1381829699193/collection1
   [junit4]   2> 1339023 T2554 oasc.ZkController.createCollectionZkNode Check for collection zkNode:control_collection
   [junit4]   2> 1339024 T2554 oasc.ZkController.createCollectionZkNode Collection zkNode exists
   [junit4]   2> 1339025 T2554 oasc.ZkController.readConfigName Load collection config from:/collections/control_collection
   [junit4]   2> 1339026 T2554 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../../../usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-controljetty-1381829699193/collection1/'
   [junit4]   2> 1339028 T2554 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-controljetty-1381829699193/collection1/lib/README' to classloader
   [junit4]   2> 1339029 T2554 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-controljetty-1381829699193/collection1/lib/classes/' to classloader
   [junit4]   2> 1339095 T2554 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_46
   [junit4]   2> 1339160 T2554 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 1339262 T2554 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
   [junit4]   2> 1339274 T2554 oass.IndexSchema.readSchema [collection1] Schema name=test
   [junit4]   2> 1339890 T2554 oass.ByteField.init WARN ByteField is deprecated and will be removed in 5.0. You should use TrieIntField instead.
   [junit4]   2> 1339892 T2554 oass.ShortField.init WARN ShortField is deprecated and will be removed in 5.0. You should use TrieIntField instead.
   [junit4]   2> 1339892 T2554 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 1339901 T2554 oass.IndexSchema.readSchema default search field in schema is text
   [junit4]   2> 1339904 T2554 oass.IndexSchema.readSchema unique key field: id
   [junit4]   2> 1339916 T2554 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 1339920 T2554 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 1339923 T2554 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 1339924 T2554 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
   [junit4]   2> 1339924 T2554 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
   [junit4]   2> 1339925 T2554 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 1339925 T2554 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
   [junit4]   2> 1339926 T2554 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
   [junit4]   2> 1339926 T2554 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 1339926 T2554 oasc.SolrCore.<init> [collection1] Opening new SolrCore at ../../../../../../../../../../../usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-controljetty-1381829699193/collection1/, dataDir=./org.apache.solr.cloud.SyncSliceTest-1381829698890/control/data/
   [junit4]   2> 1339926 T2554 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@4c4abadd
   [junit4]   2> 1339927 T2554 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.SyncSliceTest-1381829698890/control/data
   [junit4]   2> 1339927 T2554 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.SyncSliceTest-1381829698890/control/data/index/
   [junit4]   2> 1339928 T2554 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.SyncSliceTest-1381829698890/control/data/index' doesn't exist. Creating new index...
   [junit4]   2> 1339928 T2554 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.SyncSliceTest-1381829698890/control/data/index
   [junit4]   2> 1339928 T2554 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=50, maxMergeAtOnceExplicit=48, maxMergedSegmentMB=47.220703125, floorSegmentMB=0.458984375, forceMergeDeletesPctAllowed=7.0902598619911705, segmentsPerTier=40.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0
   [junit4]   2> 1339929 T2554 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 		commit{dir=MockDirectoryWrapper(org.apache.lucene.store.RAMDirectory@29d15b0f lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@72f649ca),segFN=segments_1,generation=1}
   [junit4]   2> 1339929 T2554 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 1339932 T2554 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "nodistrib"
   [junit4]   2> 1339933 T2554 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe"
   [junit4]   2> 1339933 T2554 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 1339933 T2554 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "stored_sig"
   [junit4]   2> 1339933 T2554 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 1339934 T2554 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
   [junit4]   2> 1339934 T2554 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 1339934 T2554 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 1339935 T2554 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 1339935 T2554 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
   [junit4]   2> 1339935 T2554 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
   [junit4]   2> 1339936 T2554 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
   [junit4]   2> 1339936 T2554 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
   [junit4]   2> 1339936 T2554 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
   [junit4]   2> 1339937 T2554 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
   [junit4]   2> 1339947 T2554 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 1339952 T2554 oasu.CommitTracker.<init> Hard AutoCommit: disabled
   [junit4]   2> 1339952 T2554 oasu.CommitTracker.<init> Soft AutoCommit: disabled
   [junit4]   2> 1339952 T2554 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=17, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0]
   [junit4]   2> 1339953 T2554 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 		commit{dir=MockDirectoryWrapper(org.apache.lucene.store.RAMDirectory@29d15b0f lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@72f649ca),segFN=segments_1,generation=1}
   [junit4]   2> 1339953 T2554 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 1339954 T2554 oass.SolrIndexSearcher.<init> Opening Searcher@4ed1bfbb main
   [junit4]   2> 1339956 T2555 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@4ed1bfbb main{StandardDirectoryReader(segments_1:1:nrt)}
   [junit4]   2> 1339958 T2554 oasc.CoreContainer.registerCore registering core: collection1
   [junit4]   2> 1339958 T2554 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:63121 collection:control_collection shard:shard1
   [junit4]   2> 1339959 T2554 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=10000&maxConnectionsPerHost=20&connTimeout=15000&socketTimeout=30000&retry=false
   [junit4]   2> 1339960 T2554 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leader_elect/shard1/election
   [junit4]   2> 1339985 T2554 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard1
   [junit4]   2> 1339996 T2554 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
   [junit4]   2> 1339996 T2554 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
   [junit4]   2> 1339996 T2554 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:63121/collection1/
   [junit4]   2> 1339996 T2554 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
   [junit4]   2> 1339997 T2554 oasc.SyncStrategy.syncToMe http://127.0.0.1:63121/collection1/ has no replicas
   [junit4]   2> 1339997 T2554 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:63121/collection1/ shard1
   [junit4]   2> 1339997 T2554 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leaders/shard1
   [junit4]   2> 1340035 T2552 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 1340048 T2551 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1340061 T2554 oasc.ZkController.register We are http://127.0.0.1:63121/collection1/ and leader is http://127.0.0.1:63121/collection1/
   [junit4]   2> 1340061 T2554 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:63121
   [junit4]   2> 1340061 T2554 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
   [junit4]   2> 1340061 T2554 oasc.ZkController.publish publishing core=collection1 state=active
   [junit4]   2> 1340062 T2554 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
   [junit4]   2> 1340063 T2554 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 1340064 T2529 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J0
   [junit4]   2> 1340065 T2529 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
   [junit4]   2> 1340065 T2529 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
   [junit4]   2> 1340066 T2529 oascsi.HttpClientUtil.createClient Creating new http client, config:
   [junit4]   2> 1340068 T2529 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 1340070 T2558 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@fdc8230 name:ZooKeeperConnection Watcher:127.0.0.1:22135/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1340070 T2529 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 1340072 T2529 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 1340074 T2529 oasc.ChaosMonkey.monkeyLog monkey: init - expire sessions:true cause connection loss:true
   [junit4]   2> 1340353 T2529 oejs.Server.doStart jetty-8.1.10.v20130312
   [junit4]   2> 1340357 T2529 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:16681
   [junit4]   2> 1340357 T2529 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
   [junit4]   2> 1340358 T2529 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 1340358 T2529 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ../../../../../../../../../../../usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty1-1381829702818
   [junit4]   2> 1340359 T2529 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../../../usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty1-1381829702818/'
   [junit4]   2> 1340400 T2529 oasc.ConfigSolr.fromFile Loading container configuration from /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J0/../../../../../../../../../../../usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty1-1381829702818/solr.xml
   [junit4]   2> 1340492 T2529 oasc.CoreContainer.<init> New CoreContainer 1502210091
   [junit4]   2> 1340493 T2529 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=../../../../../../../../../../../usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty1-1381829702818/]
   [junit4]   2> 1340495 T2529 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
   [junit4]   2> 1340495 T2529 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
   [junit4]   2> 1340496 T2529 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
   [junit4]   2> 1340496 T2529 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
   [junit4]   2> 1340497 T2529 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
   [junit4]   2> 1340497 T2529 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
   [junit4]   2> 1340498 T2529 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
   [junit4]   2> 1340498 T2529 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
   [junit4]   2> 1340499 T2529 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
   [junit4]   2> 1340499 T2529 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
   [junit4]   2> 1340506 T2529 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 1340507 T2529 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 1340507 T2529 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:22135/solr
   [junit4]   2> 1340508 T2529 oasc.ZkController.checkChrootPath zkHost includes chroot
   [junit4]   2> 1340509 T2529 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 1340512 T2569 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@68479aa6 name:ZooKeeperConnection Watcher:127.0.0.1:22135 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1340513 T2529 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 1340528 T2529 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
   [junit4]   2> 1340529 T2529 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 1340531 T2571 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@573aed00 name:ZooKeeperConnection Watcher:127.0.0.1:22135/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1340531 T2529 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 1340541 T2529 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 1341544 T2529 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:16681_
   [junit4]   2> 1341547 T2529 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:16681_
   [junit4]   2> 1341550 T2551 oascc.ZkStateReader$3.process Updating live nodes... (2)
   [junit4]   2> 1341550 T2571 oascc.ZkStateReader$3.process Updating live nodes... (2)
   [junit4]   2> 1341550 T2558 oascc.ZkStateReader$3.process Updating live nodes... (2)
   [junit4]   2> 1341561 T2552 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 1341562 T2552 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
   [junit4]   2> 	  "operation":"state",
   [junit4]   2> 	  "state":"active",
   [junit4]   2> 	  "base_url":"http://127.0.0.1:63121",
   [junit4]   2> 	  "core":"collection1",
   [junit4]   2> 	  "roles":null,
   [junit4]   2> 	  "node_name":"127.0.0.1:63121_",
   [junit4]   2> 	  "shard":"shard1",
   [junit4]   2> 	  "shard_range":null,
   [junit4]   2> 	  "shard_state":"active",
   [junit4]   2> 	  "shard_parent":null,
   [junit4]   2> 	  "collection":"control_collection",
   [junit4]   2> 	  "numShards":"1",
   [junit4]   2> 	  "core_node_name":"core_node1"}
   [junit4]   2> 1341567 T2572 oasc.ZkController.publish publishing core=collection1 state=down
   [junit4]   2> 1341567 T2572 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
   [junit4]   2> 1341571 T2572 oasc.ZkController.waitForCoreNodeName look for our core node name
   [junit4]   2> 1341572 T2552 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
   [junit4]   2> 	  "operation":"state",
   [junit4]   2> 	  "state":"down",
   [junit4]   2> 	  "base_url":"http://127.0.0.1:16681",
   [junit4]   2> 	  "core":"collection1",
   [junit4]   2> 	  "roles":null,
   [junit4]   2> 	  "node_name":"127.0.0.1:16681_",
   [junit4]   2> 	  "shard":null,
   [junit4]   2> 	  "shard_range":null,
   [junit4]   2> 	  "shard_state":"active",
   [junit4]   2> 	  "shard_parent":null,
   [junit4]   2> 	  "collection":"collection1",
   [junit4]   2> 	  "numShards":"1",
   [junit4]   2> 	  "core_node_name":"core_node1"}
   [junit4]   2> 1341572 T2552 oasc.Overseer$ClusterStateUpdater.createCollection Create collection collection1 with shards [shard1]
   [junit4]   2> 1341572 T2552 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
   [junit4]   2> 1341583 T2571 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 1341583 T2551 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 1341583 T2558 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 1342572 T2572 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
   [junit4]   2> 1342572 T2572 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: ../../../../../../../../../../../usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty1-1381829702818/collection1
   [junit4]   2> 1342573 T2572 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
   [junit4]   2> 1342573 T2572 oasc.ZkController.createCollectionZkNode Collection zkNode exists
   [junit4]   2> 1342574 T2572 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
   [junit4]   2> 1342575 T2572 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../../../usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty1-1381829702818/collection1/'
   [junit4]   2> 1342577 T2572 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-jetty1-1381829702818/collection1/lib/README' to classloader
   [junit4]   2> 1342578 T2572 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-jetty1-1381829702818/collection1/lib/classes/' to classloader
   [junit4]   2> 1342645 T2572 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_46
   [junit4]   2> 1342709 T2572 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 1342811 T2572 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
   [junit4]   2> 1342823 T2572 oass.IndexSchema.readSchema [collection1] Schema name=test
   [junit4]   2> 1343485 T2572 oass.ByteField.init WARN ByteField is deprecated and will be removed in 5.0. You should use TrieIntField instead.
   [junit4]   2> 1343486 T2572 oass.ShortField.init WARN ShortField is deprecated and will be removed in 5.0. You should use TrieIntField instead.
   [junit4]   2> 1343487 T2572 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 1343499 T2572 oass.IndexSchema.readSchema default search field in schema is text
   [junit4]   2> 1343503 T2572 oass.IndexSchema.readSchema unique key field: id
   [junit4]   2> 1343517 T2572 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 1343521 T2572 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 1343526 T2572 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 1343527 T2572 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
   [junit4]   2> 1343527 T2572 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
   [junit4]   2> 1343527 T2572 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 1343528 T2572 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
   [junit4]   2> 1343529 T2572 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
   [junit4]   2> 1343529 T2572 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 1343529 T2572 oasc.SolrCore.<init> [collection1] Opening new SolrCore at ../../../../../../../../../../../usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty1-1381829702818/collection1/, dataDir=./org.apache.solr.cloud.SyncSliceTest-1381829698890/jetty1/
   [junit4]   2> 1343530 T2572 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@4c4abadd
   [junit4]   2> 1343531 T2572 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.SyncSliceTest-1381829698890/jetty1
   [junit4]   2> 1343531 T2572 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.SyncSliceTest-1381829698890/jetty1/index/
   [junit4]   2> 1343531 T2572 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.SyncSliceTest-1381829698890/jetty1/index' doesn't exist. Creating new index...
   [junit4]   2> 1343532 T2572 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.SyncSliceTest-1381829698890/jetty1/index
   [junit4]   2> 1343532 T2572 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=50, maxMergeAtOnceExplicit=48, maxMergedSegmentMB=47.220703125, floorSegmentMB=0.458984375, forceMergeDeletesPctAllowed=7.0902598619911705, segmentsPerTier=40.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0
   [junit4]   2> 1343533 T2572 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 		commit{dir=MockDirectoryWrapper(org.apache.lucene.store.RAMDirectory@2ce6c028 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@1739408f),segFN=segments_1,generation=1}
   [junit4]   2> 1343534 T2572 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 1343538 T2572 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "nodistrib"
   [junit4]   2> 1343539 T2572 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe"
   [junit4]   2> 1343539 T2572 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 1343539 T2572 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "stored_sig"
   [junit4]   2> 1343540 T2572 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 1343540 T2572 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
   [junit4]   2> 1343541 T2572 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 1343541 T2572 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 1343541 T2572 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 1343542 T2572 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
   [junit4]   2> 1343543 T2572 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
   [junit4]   2> 1343543 T2572 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
   [junit4]   2> 1343544 T2572 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
   [junit4]   2> 1343544 T2572 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
   [junit4]   2> 1343545 T2572 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
   [junit4]   2> 1343560 T2572 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 1343565 T2572 oasu.CommitTracker.<init> Hard AutoCommit: disabled
   [junit4]   2> 1343565 T2572 oasu.CommitTracker.<init> Soft AutoCommit: disabled
   [junit4]   2> 1343566 T2572 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=17, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0]
   [junit4]   2> 1343567 T2572 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 		commit{dir=MockDirectoryWrapper(org.apache.lucene.store.RAMDirectory@2ce6c028 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@1739408f),segFN=segments_1,generation=1}
   [junit4]   2> 1343567 T2572 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 1343568 T2572 oass.SolrIndexSearcher.<init> Opening Searcher@21235d04 main
   [junit4]   2> 1343573 T2573 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@21235d04 main{StandardDirectoryReader(segments_1:1:nrt)}
   [junit4]   2> 1343575 T2572 oasc.CoreContainer.registerCore registering core: collection1
   [junit4]   2> 1343575 T2572 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:16681 collection:collection1 shard:shard1
   [junit4]   2> 1343576 T2572 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=10000&maxConnectionsPerHost=20&connTimeout=15000&socketTimeout=30000&retry=false
   [junit4]   2> 1343577 T2572 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
   [junit4]   2> 1343619 T2572 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard1
   [junit4]   2> 1343628 T2572 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
   [junit4]   2> 1343629 T2572 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
   [junit4]   2> 1343629 T2572 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:16681/collection1/
   [junit4]   2> 1343629 T2572 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
   [junit4]   2> 1343629 T2572 oasc.SyncStrategy.syncToMe http://127.0.0.1:16681/collection1/ has no replicas
   [junit4]   2> 1343630 T2572 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:16681/collection1/ shard1
   [junit4]   2> 1343630 T2572 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
   [junit4]   2> 1344597 T2552 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 1344642 T2558 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 1344642 T2571 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 1344642 T2551 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 1344664 T2572 oasc.ZkController.register We are http://127.0.0.1:16681/collection1/ and leader is http://127.0.0.1:16681/collection1/
   [junit4]   2> 1344664 T2572 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:16681
   [junit4]   2> 1344664 T2572 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
   [junit4]   2> 1344664 T2572 oasc.ZkController.publish publishing core=collection1 state=active
   [junit4]   2> 1344665 T2572 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
   [junit4]   2> 1344666 T2572 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 1344668 T2529 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J0
   [junit4]   2> 1344668 T2529 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
   [junit4]   2> 1344669 T2529 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
   [junit4]   2> 1344950 T2529 oejs.Server.doStart jetty-8.1.10.v20130312
   [junit4]   2> 1344954 T2529 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:61551
   [junit4]   2> 1344954 T2529 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
   [junit4]   2> 1344955 T2529 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 1344956 T2529 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ./org.apache.solr.cloud.SyncSliceTest-jetty2-1381829707414
   [junit4]   2> 1344956 T2529 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: './org.apache.solr.cloud.SyncSliceTest-jetty2-1381829707414/'
   [junit4]   2> 1344997 T2529 oasc.ConfigSolr.fromFile Loading container configuration from /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty2-1381829707414/solr.xml
   [junit4]   2> 1345088 T2529 oasc.CoreContainer.<init> New CoreContainer 992913037
   [junit4]   2> 1345089 T2529 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=./org.apache.solr.cloud.SyncSliceTest-jetty2-1381829707414/]
   [junit4]   2> 1345091 T2529 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
   [junit4]   2> 1345092 T2529 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
   [junit4]   2> 1345092 T2529 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
   [junit4]   2> 1345093 T2529 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
   [junit4]   2> 1345093 T2529 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
   [junit4]   2> 1345094 T2529 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
   [junit4]   2> 1345094 T2529 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
   [junit4]   2> 1345095 T2529 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
   [junit4]   2> 1345095 T2529 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
   [junit4]   2> 1345096 T2529 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
   [junit4]   2> 1345103 T2529 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 1345103 T2529 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 1345104 T2529 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:22135/solr
   [junit4]   2> 1345104 T2529 oasc.ZkController.checkChrootPath zkHost includes chroot
   [junit4]   2> 1345105 T2529 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 1345109 T2585 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1df4d30c name:ZooKeeperConnection Watcher:127.0.0.1:22135 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1345109 T2529 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 1345126 T2529 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
   [junit4]   2> 1345128 T2529 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 1345130 T2587 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@6f78f55 name:ZooKeeperConnection Watcher:127.0.0.1:22135/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1345130 T2529 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 1345140 T2529 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 1346143 T2529 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:61551_
   [junit4]   2> 1346154 T2529 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:61551_
   [junit4]   2> 1346156 T2552 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 1346158 T2552 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
   [junit4]   2> 	  "operation":"state",
   [junit4]   2> 	  "state":"active",
   [junit4]   2> 	  "base_url":"http://127.0.0.1:16681",
   [junit4]   2> 	  "core":"collection1",
   [junit4]   2> 	  "roles":null,
   [junit4]   2> 	  "node_name":"127.0.0.1:16681_",
   [junit4]   2> 	  "shard":"shard1",
   [junit4]   2> 	  "shard_range":null,
   [junit4]   2> 	  "shard_state":"active",
   [junit4]   2> 	  "shard_parent":null,
   [junit4]   2> 	  "collection":"collection1",
   [junit4]   2> 	  "numShards":"1",
   [junit4]   2> 	  "core_node_name":"core_node1"}
   [junit4]   2> 1346158 T2587 oascc.ZkStateReader$3.process Updating live nodes... (3)
   [junit4]   2> 1346158 T2571 oascc.ZkStateReader$3.process Updating live nodes... (3)
   [junit4]   2> 1346158 T2558 oascc.ZkStateReader$3.process Updating live nodes... (3)
   [junit4]   2> 1346174 T2588 oasc.ZkController.publish publishing core=collection1 state=down
   [junit4]   2> 1346174 T2588 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
   [junit4]   2> 1346177 T2571 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
   [junit4]   2> 1346177 T2587 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
   [junit4]   2> 1346177 T2558 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
   [junit4]   2> 1346178 T2588 oasc.ZkController.waitForCoreNodeName look for our core node name
   [junit4]   2> 1346186 T2551 oascc.ZkStateReader$3.process Updating live nodes... (3)
   [junit4]   2> 1346186 T2551 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
   [junit4]   2> 1347687 T2552 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 1347688 T2552 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
   [junit4]   2> 	  "operation":"state",
   [junit4]   2> 	  "state":"down",
   [junit4]   2> 	  "base_url":"http://127.0.0.1:61551",
   [junit4]   2> 	  "core":"collection1",
   [junit4]   2> 	  "roles":null,
   [junit4]   2> 	  "node_name":"127.0.0.1:61551_",
   [junit4]   2> 	  "shard":null,
   [junit4]   2> 	  "shard_range":null,
   [junit4]   2> 	  "shard_state":"active",
   [junit4]   2> 	  "shard_parent":null,
   [junit4]   2> 	  "collection":"collection1",
   [junit4]   2> 	  "numShards":"1",
   [junit4]   2> 	  "core_node_name":"core_node2"}
   [junit4]   2> 1347689 T2552 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=1
   [junit4]   2> 1347689 T2552 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
   [junit4]   2> 1347704 T2558 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
   [junit4]   2> 1347704 T2551 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
   [junit4]   2> 1347704 T2571 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
   [junit4]   2> 1347704 T2587 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
   [junit4]   2> 1348180 T2588 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
   [junit4]   2> 1348180 T2588 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: ./org.apache.solr.cloud.SyncSliceTest-jetty2-1381829707414/collection1
   [junit4]   2> 1348181 T2588 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
   [junit4]   2> 1348182 T2588 oasc.ZkController.createCollectionZkNode Collection zkNode exists
   [junit4]   2> 1348182 T2588 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
   [junit4]   2> 1348183 T2588 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: './org.apache.solr.cloud.SyncSliceTest-jetty2-1381829707414/collection1/'
   [junit4]   2> 1348184 T2588 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-jetty2-1381829707414/collection1/lib/classes/' to classloader
   [junit4]   2> 1348185 T2588 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-jetty2-1381829707414/collection1/lib/README' to classloader
   [junit4]   2> 1348252 T2588 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_46
   [junit4]   2> 1348317 T2588 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 1348418 T2588 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
   [junit4]   2> 1348430 T2588 oass.IndexSchema.readSchema [collection1] Schema name=test
   [junit4]   2> 1349099 T2588 oass.ByteField.init WARN ByteField is deprecated and will be removed in 5.0. You should use TrieIntField instead.
   [junit4]   2> 1349099 T2588 oass.ShortField.init WARN ShortField is deprecated and will be removed in 5.0. You should use TrieIntField instead.
   [junit4]   2> 1349100 T2588 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 1349113 T2588 oass.IndexSchema.readSchema default search field in schema is text
   [junit4]   2> 1349117 T2588 oass.IndexSchema.readSchema unique key field: id
   [junit4]   2> 1349132 T2588 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 1349137 T2588 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 1349141 T2588 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 1349142 T2588 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
   [junit4]   2> 1349143 T2588 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
   [junit4]   2> 1349143 T2588 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 1349144 T2588 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
   [junit4]   2> 1349144 T2588 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
   [junit4]   2> 1349145 T2588 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 1349145 T2588 oasc.SolrCore.<init> [collection1] Opening new SolrCore at ./org.apache.solr.cloud.SyncSliceTest-jetty2-1381829707414/collection1/, dataDir=./org.apache.solr.cloud.SyncSliceTest-1381829698890/jetty2/
   [junit4]   2> 1349146 T2588 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@4c4abadd
   [junit4]   2> 1349147 T2588 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.SyncSliceTest-1381829698890/jetty2
   [junit4]   2> 1349147 T2588 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.SyncSliceTest-1381829698890/jetty2/index/
   [junit4]   2> 1349147 T2588 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.SyncSliceTest-1381829698890/jetty2/index' doesn't exist. Creating new index...
   [junit4]   2> 1349148 T2588 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.SyncSliceTest-1381829698890/jetty2/index
   [junit4]   2> 1349148 T2588 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=50, maxMergeAtOnceExplicit=48, maxMergedSegmentMB=47.220703125, floorSegmentMB=0.458984375, forceMergeDeletesPctAllowed=7.0902598619911705, segmentsPerTier=40.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0
   [junit4]   2> 1349149 T2588 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 		commit{dir=MockDirectoryWrapper(org.apache.lucene.store.RAMDirectory@3da007e lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@591a1a0c),segFN=segments_1,generation=1}
   [junit4]   2> 1349149 T2588 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 1349154 T2588 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "nodistrib"
   [junit4]   2> 1349155 T2588 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe"
   [junit4]   2> 1349155 T2588 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 1349156 T2588 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "stored_sig"
   [junit4]   2> 1349156 T2588 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 1349156 T2588 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
   [junit4]   2> 1349157 T2588 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 1349157 T2588 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 1349158 T2588 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 1349158 T2588 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
   [junit4]   2> 1349159 T2588 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
   [junit4]   2> 1349160 T2588 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
   [junit4]   2> 1349160 T2588 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
   [junit4]   2> 1349161 T2588 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
   [junit4]   2> 1349161 T2588 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
   [junit4]   2> 1349178 T2588 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 1349183 T2588 oasu.CommitTracker.<init> Hard AutoCommit: disabled
   [junit4]   2> 1349184 T2588 oasu.CommitTracker.<init> Soft AutoCommit: disabled
   [junit4]   2> 1349185 T2588 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=17, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0]
   [junit4]   2> 1349185 T2588 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 		commit{dir=MockDirectoryWrapper(org.apache.lucene.store.RAMDirectory@3da007e lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@591a1a0c),segFN=segments_1,generation=1}
   [junit4]   2> 1349186 T2588 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 1349186 T2588 oass.SolrIndexSearcher.<init> Opening Searcher@2fd85f7a main
   [junit4]   2> 1349191 T2589 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@2fd85f7a main{StandardDirectoryReader(segments_1:1:nrt)}
   [junit4]   2> 1349194 T2588 oasc.CoreContainer.registerCore registering core: collection1
   [junit4]   2> 1349194 T2588 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:61551 collection:collection1 shard:shard1
   [junit4]   2> 1349194 T2588 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=10000&maxConnectionsPerHost=20&connTimeout=15000&socketTimeout=30000&retry=false
   [junit4]   2> 1349224 T2588 oasc.ZkController.register We are http://127.0.0.1:61551/collection1/ and leader is http://127.0.0.1:16681/collection1/
   [junit4]   2> 1349225 T2588 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:61551
   [junit4]   2> 1349225 T2588 oasc.ZkController.checkRecovery Core needs to recover:collection1
   [junit4]   2> 1349225 T2588 oasu.DefaultSolrCoreState.doRecovery Running recovery - first canceling any ongoing recovery
   [junit4]   2> ASYNC  NEW_CORE C1374 name=collection1 org.apache.solr.core.SolrCore@4d25469e url=http://127.0.0.1:61551/collection1 node=127.0.0.1:61551_ C1374_STATE=coll:collection1 core:collection1 props:{state=down, base_url=http://127.0.0.1:61551, core=collection1, node_name=127.0.0.1:61551_}
   [junit4]   2> 1349226 T2590 C1374 P61551 oasc.RecoveryStrategy.run Starting recovery process.  core=collection1 recoveringAfterStartup=true
   [junit4]   2> 1349226 T2588 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 1349227 T2590 C1374 P61551 oasc.RecoveryStrategy.doRecovery ###### startupVersions=[]
   [junit4]   2> 1349227 T2590 C1374 P61551 oasc.ZkController.publish publishing core=collection1 state=recovering
   [junit4]   2> 1349228 T2590 C1374 P61551 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
   [junit4]   2> 1349228 T2529 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J0
   [junit4]   2> 1349229 T2529 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
   [junit4]   2> 1349229 T2590 C1374 P61551 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
   [junit4]   2> 1349229 T2529 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
   [junit4]   2> 1349232 T2561 oasha.CoreAdminHandler.handleWaitForStateAction Going to wait for coreNodeName: core_node2, state: recovering, checkLive: true, onlyIfLeader: true
   [junit4]   2> 1349513 T2529 oejs.Server.doStart jetty-8.1.10.v20130312
   [junit4]   2> 1349517 T2529 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:33880
   [junit4]   2> 1349517 T2529 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
   [junit4]   2> 1349518 T2529 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 1349518 T2529 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ./org.apache.solr.cloud.SyncSliceTest-jetty3-1381829711975
   [junit4]   2> 1349519 T2529 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: './org.apache.solr.cloud.SyncSliceTest-jetty3-1381829711975/'
   [junit4]   2> 1349562 T2529 oasc.ConfigSolr.fromFile Loading container configuration from /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty3-1381829711975/solr.xml
   [junit4]   2> 1349655 T2529 oasc.CoreContainer.<init> New CoreContainer 2127279082
   [junit4]   2> 1349656 T2529 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=./org.apache.solr.cloud.SyncSliceTest-jetty3-1381829711975/]
   [junit4]   2> 1349658 T2529 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
   [junit4]   2> 1349658 T2529 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
   [junit4]   2> 1349659 T2529 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
   [junit4]   2> 1349659 T2529 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
   [junit4]   2> 1349660 T2529 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
   [junit4]   2> 1349660 T2529 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
   [junit4]   2> 1349661 T2529 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
   [junit4]   2> 1349661 T2529 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
   [junit4]   2> 1349662 T2529 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
   [junit4]   2> 1349662 T2529 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
   [junit4]   2> 1349669 T2529 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 1349670 T2529 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 1349670 T2529 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:22135/solr
   [junit4]   2> 1349671 T2529 oasc.ZkController.checkChrootPath zkHost includes chroot
   [junit4]   2> 1349672 T2529 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 1349676 T2602 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@13a172be name:ZooKeeperConnection Watcher:127.0.0.1:22135 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1349676 T2529 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 1349692 T2529 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
   [junit4]   2> 1349694 T2529 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 1349696 T2604 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@3f098c72 name:ZooKeeperConnection Watcher:127.0.0.1:22135/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1349697 T2529 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 1349708 T2529 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 1350712 T2529 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:33880_
   [junit4]   2> 1350717 T2529 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:33880_
   [junit4]   2> 1350721 T2558 oascc.ZkStateReader$3.process Updating live nodes... (4)
   [junit4]   2> 1350721 T2587 oascc.ZkStateReader$3.process Updating live nodes... (4)
   [junit4]   2> 1350721 T2604 oascc.ZkStateReader$3.process Updating live nodes... (4)
   [junit4]   2> 1350721 T2551 oascc.ZkStateReader$3.process Updating live nodes... (4)
   [junit4]   2> 1350721 T2571 oascc.ZkStateReader$3.process Updating live nodes... (4)
   [junit4]   2> 1350726 T2552 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 1350727 T2552 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
   [junit4]   2> 	  "operation":"state",
   [junit4]   2> 	  "state":"recovering",
   [junit4]   2> 	  "base_url":"http://127.0.0.1:61551",
   [junit4]   2> 	  "core":"collection1",
   [junit4]   2> 	  "roles":null,
   [junit4]   2> 	  "node_name":"127.0.0.1:61551_",
   [junit4]   2> 	  "shard":"shard1",
   [junit4]   2> 	  "shard_range":null,
   [junit4]   2> 	  "shard_state":"active",
   [junit4]   2> 	  "shard_parent":null,
   [junit4]   2> 	  "collection":"collection1",
   [junit4]   2> 	  "numShards":"1",
   [junit4]   2> 	  "core_node_name":"core_node2"}
   [junit4]   2> 1350730 T2605 oasc.ZkController.publish publishing core=collection1 state=down
   [junit4]   2> 1350730 T2605 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
   [junit4]   2> 1350737 T2605 oasc.ZkController.waitForCoreNodeName look for our core node name
   [junit4]   2> 1350737 T2552 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
   [junit4]   2> 	  "operation":"state",
   [junit4]   2> 	  "state":"down",
   [junit4]   2> 	  "base_url":"http://127.0.0.1:33880",
   [junit4]   2> 	  "core":"collection1",
   [junit4]   2> 	  "roles":null,
   [junit4]   2> 	  "node_name":"127.0.0.1:33880_",
   [junit4]   2> 	  "shard":null,
   [junit4]   2> 	  "shard_range":null,
   [junit4]   2> 	  "shard_state":"active",
   [junit4]   2> 	  "shard_parent":null,
   [junit4]   2> 	  "collection":"collection1",
   [junit4]   2> 	  "numShards":"1",
   [junit4]   2> 	  "core_node_name":"core_node3"}
   [junit4]   2> 1350737 T2552 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=1
   [junit4]   2> 1350738 T2552 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
   [junit4]   2> 1350747 T2558 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
   [junit4]   2> 1350747 T2604 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
   [junit4]   2> 1350748 T2587 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
   [junit4]   2> 1350747 T2551 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
   [junit4]   2> 1350747 T2571 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
   [junit4]   2> 1351234 T2561 oasha.CoreAdminHandler.handleWaitForStateAction Waited coreNodeName: core_node2, state: recovering, checkLive: true, onlyIfLeader: true for: 2 seconds.
   [junit4]   2> 1351234 T2561 oass.SolrDispatchFilter.handleAdminRequest [admin] webapp=null path=/admin/cores params={coreNodeName=core_node2&state=recovering&nodeName=127.0.0.1:61551_&action=PREPRECOVERY&checkLive=true&core=collection1&wt=javabin&onlyIfLeader=true&version=2} status=0 QTime=2002 
   [junit4]   2> 1351738 T2605 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
   [junit4]   2> 1351738 T2605 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: ./org.apache.solr.cloud.SyncSliceTest-jetty3-1381829711975/collection1
   [junit4]   2> 1351738 T2605 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
   [junit4]   2> 1351739 T2605 oasc.ZkController.createCollectionZkNode Collection zkNode exists
   [junit4]   2> 1351739 T2605 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
   [junit4]   2> 1351741 T2605 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: './org.apache.solr.cloud.SyncSliceTest-jetty3-1381829711975/collection1/'
   [junit4]   2> 1351742 T2605 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-jetty3-1381829711975/collection1/lib/classes/' to classloader
   [junit4]   2> 1351743 T2605 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-jetty3-1381829711975/collection1/lib/README' to classloader
   [junit4]   2> 1351809 T2605 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_46
   [junit4]   2> 1351874 T2605 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 1351976 T2605 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
   [junit4]   2> 1351988 T2605 oass.IndexSchema.readSchema [collection1] Schema name=test
   [junit4]   2> 1352656 T2605 oass.ByteField.init WARN ByteField is deprecated and will be removed in 5.0. You should use TrieIntField instead.
   [junit4]   2> 1352657 T2605 oass.ShortField.init WARN ShortField is deprecated and will be removed in 5.0. You should use TrieIntField instead.
   [junit4]   2> 1352657 T2605 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 1352670 T2605 oass.IndexSchema.readSchema default search field in schema is text
   [junit4]   2> 1352674 T2605 oass.IndexSchema.readSchema unique key field: id
   [junit4]   2> 1352688 T2605 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 1352694 T2605 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 1352698 T2605 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 1352699 T2605 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
   [junit4]   2> 1352700 T2605 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
   [junit4]   2> 1352700 T2605 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 1352701 T2605 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
   [junit4]   2> 1352701 T2605 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
   [junit4]   2> 1352702 T2605 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 1352702 T2605 oasc.SolrCore.<init> [collection1] Opening new SolrCore at ./org.apache.solr.cloud.SyncSliceTest-jetty3-1381829711975/collection1/, dataDir=./org.apache.solr.cloud.SyncSliceTest-1381829698890/jetty3/
   [junit4]   2> 1352702 T2605 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@4c4abadd
   [junit4]   2> 1352703 T2605 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.SyncSliceTest-1381829698890/jetty3
   [junit4]   2> 1352704 T2605 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.SyncSliceTest-1381829698890/jetty3/index/
   [junit4]   2> 1352704 T2605 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.SyncSliceTest-1381829698890/jetty3/index' doesn't exist. Creating new index...
   [junit4]   2> 1352704 T2605 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.SyncSliceTest-1381829698890/jetty3/index
   [junit4]   2> 1352705 T2605 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=50, maxMergeAtOnceExplicit=48, maxMergedSegmentMB=47.220703125, floorSegmentMB=0.458984375, forceMergeDeletesPctAllowed=7.0902598619911705, segmentsPerTier=40.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0
   [junit4]   2> 1352706 T2605 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 		commit{dir=MockDirectoryWrapper(org.apache.lucene.store.RAMDirectory@7c54e01b lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@1bc32c35),segFN=segments_1,generation=1}
   [junit4]   2> 1352706 T2605 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 1352711 T2605 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "nodistrib"
   [junit4]   2> 1352711 T2605 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe"
   [junit4]   2> 1352712 T2605 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 1352712 T2605 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "stored_sig"
   [junit4]   2> 1352713 T2605 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 1352713 T2605 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
   [junit4]   2> 1352714 T2605 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 1352714 T2605 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 1352714 T2605 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 1352715 T2605 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
   [junit4]   2> 1352716 T2605 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
   [junit4]   2> 1352716 T2605 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
   [junit4]   2> 1352717 T2605 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
   [junit4]   2> 1352717 T2605 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
   [junit4]   2> 1352718 T2605 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
   [junit4]   2> 1352735 T2605 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 1352740 T2605 oasu.CommitTracker.<init> Hard AutoCommit: disabled
   [junit4]   2> 1352740 T2605 oasu.CommitTracker.<init> Soft AutoCommit: disabled
   [junit4]   2> 1352741 T2605 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=17, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0]
   [junit4]   2> 1352742 T2605 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 		commit{dir=MockDirectoryWrapper(org.apache.lucene.store.RAMDirectory@7c54e01b lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@1bc32c35),segFN=segments_1,generation=1}
   [junit4]   2> 1352742 T2605 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 1352743 T2605 oass.SolrIndexSearcher.<init> Opening Searcher@6a921ae0 main
   [junit4]   2> 1352748 T2606 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@6a921ae0 main{StandardDirectoryReader(segments_1:1:nrt)}
   [junit4]   2> 1352751 T2605 oasc.CoreContainer.registerCore registering core: collection1
   [junit4]   2> 1352751 T2605 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:33880 collection:collection1 shard:shard1
   [junit4]   2> 1352751 T2605 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=10000&maxConnectionsPerHost=20&connTimeout=15000&socketTimeout=30000&retry=false
   [junit4]   2> 1352755 T2605 oasc.ZkController.register We are http://127.0.0.1:33880/collection1/ and leader is http://127.0.0.1:16681/collection1/
   [junit4]   2> 1352756 T2605 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:33880
   [junit4]   2> 1352756 T2605 oasc.ZkController.checkRecovery Core needs to recover:collection1
   [junit4]   2> 1352756 T2605 oasu.DefaultSolrCoreState.doRecovery Running recovery - first canceling any ongoing recovery
   [junit4]   2> ASYNC  NEW_CORE C1375 name=collection1 org.apache.solr.core.SolrCore@24a113eb url=http://127.0.0.1:33880/collection1 node=127.0.0.1:33880_ C1375_STATE=coll:collection1 core:collection1 props:{state=down, base_url=http://127.0.0.1:33880, core=collection1, node_name=127.0.0.1:33880_}
   [junit4]   2> 1352757 T2607 C1375 P33880 oasc.RecoveryStrategy.run Starting recovery process.  core=collection1 recoveringAfterStartup=true
   [junit4]   2> 1352758 T2605 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 1352758 T2607 C1375 P33880 oasc.RecoveryStrategy.doRecovery ###### startupVersions=[]
   [junit4]   2> 1352758 T2607 C1375 P33880 oasc.ZkController.publish publishing core=collection1 state=recovering
   [junit4]   2> 1352759 T2607 C1375 P33880 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
   [junit4]   2> 1352759 T2529 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J0
   [junit4]   2> 1352760 T2529 oass.SolrDis

[...truncated too long message...]

              "state":"active",
   [junit4]   1>                   "base_url":"http://127.0.0.1:63121",
   [junit4]   1>                   "core":"collection1",
   [junit4]   1>                   "node_name":"127.0.0.1:63121_",
   [junit4]   1>                   "leader":"true"}}}},
   [junit4]   1>           "maxShardsPerNode":"1",
   [junit4]   1>           "router":{"name":"compositeId"},
   [junit4]   1>           "replicationFactor":"1"}}
   [junit4]   1>   /solr/aliases.json (0)
   [junit4]   1>   /solr/live_nodes (5)
   [junit4]   1>    /solr/live_nodes/127.0.0.1:16681_ (0)
   [junit4]   1>    /solr/live_nodes/127.0.0.1:63121_ (0)
   [junit4]   1>    /solr/live_nodes/127.0.0.1:33880_ (0)
   [junit4]   1>    /solr/live_nodes/127.0.0.1:61551_ (0)
   [junit4]   1>    /solr/live_nodes/127.0.0.1:54720_ (0)
   [junit4]   1>   /solr/overseer (3)
   [junit4]   1>   DATA:
   [junit4]   1>       
   [junit4]   1>    /solr/overseer/queue (0)
   [junit4]   1>    /solr/overseer/queue-work (0)
   [junit4]   1>    /solr/overseer/collection-queue-work (0)
   [junit4]   1>   /solr/collections (2)
   [junit4]   1>    /solr/collections/collection1 (3)
   [junit4]   1>    DATA:
   [junit4]   1>        {"configName":"conf1"}
   [junit4]   1>     /solr/collections/collection1/shards (0)
   [junit4]   1>     /solr/collections/collection1/leader_elect (1)
   [junit4]   1>      /solr/collections/collection1/leader_elect/shard1 (1)
   [junit4]   1>       /solr/collections/collection1/leader_elect/shard1/election (4)
   [junit4]   1>        /solr/collections/collection1/leader_elect/shard1/election/90559591146848268-core_node4-n_0000000003 (0)
   [junit4]   1>        /solr/collections/collection1/leader_elect/shard1/election/90559591146848264-core_node2-n_0000000004 (0)
   [junit4]   1>        /solr/collections/collection1/leader_elect/shard1/election/90559591146848266-core_node3-n_0000000002 (0)
   [junit4]   1>        /solr/collections/collection1/leader_elect/shard1/election/90559591146848270-core_node1-n_0000000005 (0)
   [junit4]   1>     /solr/collections/collection1/leaders (1)
   [junit4]   1>      /solr/collections/collection1/leaders/shard1 (0)
   [junit4]   1>      DATA:
   [junit4]   1>          {
   [junit4]   1>            "core":"collection1",
   [junit4]   1>            "node_name":"127.0.0.1:33880_",
   [junit4]   1>            "base_url":"http://127.0.0.1:33880"}
   [junit4]   1>    /solr/collections/control_collection (3)
   [junit4]   1>    DATA:
   [junit4]   1>        {"configName":"conf1"}
   [junit4]   1>     /solr/collections/control_collection/shards (0)
   [junit4]   1>     /solr/collections/control_collection/leader_elect (1)
   [junit4]   1>      /solr/collections/control_collection/leader_elect/shard1 (1)
   [junit4]   1>       /solr/collections/control_collection/leader_elect/shard1/election (1)
   [junit4]   1>        /solr/collections/control_collection/leader_elect/shard1/election/90559591146848259-core_node1-n_0000000000 (0)
   [junit4]   1>     /solr/collections/control_collection/leaders (1)
   [junit4]   1>      /solr/collections/control_collection/leaders/shard1 (0)
   [junit4]   1>      DATA:
   [junit4]   1>          {
   [junit4]   1>            "core":"collection1",
   [junit4]   1>            "node_name":"127.0.0.1:63121_",
   [junit4]   1>            "base_url":"http://127.0.0.1:63121"}
   [junit4]   1>   /solr/overseer_elect (2)
   [junit4]   1>    /solr/overseer_elect/election (5)
   [junit4]   1>     /solr/overseer_elect/election/90559591146848270-127.0.0.1:16681_-n_0000000005 (0)
   [junit4]   1>     /solr/overseer_elect/election/90559591146848264-127.0.0.1:61551_-n_0000000002 (0)
   [junit4]   1>     /solr/overseer_elect/election/90559591146848259-127.0.0.1:63121_-n_0000000000 (0)
   [junit4]   1>     /solr/overseer_elect/election/90559591146848268-127.0.0.1:54720_-n_0000000004 (0)
   [junit4]   1>     /solr/overseer_elect/election/90559591146848266-127.0.0.1:33880_-n_0000000003 (0)
   [junit4]   1>    /solr/overseer_elect/leader (0)
   [junit4]   1>    DATA:
   [junit4]   1>        {"id":"90559591146848259-127.0.0.1:63121_-n_0000000000"}
   [junit4]   1>  /zookeeper (1)
   [junit4]   1>  DATA:
   [junit4]   1>      
   [junit4]   1> 
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=SyncSliceTest -Dtests.method=testDistribSearch -Dtests.seed=38EEFE0AB3312BCF -Dtests.multiplier=3 -Dtests.slow=true -Dtests.locale=fi_FI -Dtests.timezone=Canada/Pacific -Dtests.file.encoding=US-ASCII
   [junit4] FAILURE  111s J0 | SyncSliceTest.testDistribSearch <<<
   [junit4]    > Throwable #1: java.lang.AssertionError: Test Setup Failure: shard1 should have just been set up to be inconsistent - but it's still consistent. Leader:http://127.0.0.1:33880/collection1 Dead Guy:http://127.0.0.1:16681/collection1skip list:[CloudJettyRunner [url=http://127.0.0.1:61551/collection1], CloudJettyRunner [url=http://127.0.0.1:61551/collection1]]
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([38EEFE0AB3312BCF:B9087012C46E4BF3]:0)
   [junit4]    > 	at org.apache.solr.cloud.SyncSliceTest.doTest(SyncSliceTest.java:216)
   [junit4]    > 	at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:835)
   [junit4]    > 	at java.lang.Thread.run(Thread.java:724)
   [junit4]   2> 1447156 T2529 oas.SolrTestCaseJ4.deleteCore ###deleteCore
   [junit4]   2> 111023 T2528 ccr.ThreadLeakControl.checkThreadLeaks WARNING Will linger awaiting termination of 3 leaked thread(s).
   [junit4]   2> 1447225 T2604 oascc.ZkStateReader$3.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
   [junit4]   2> 1448318 T2621 oascc.ZkStateReader$3.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
   [junit4]   2> NOTE: test params are: codec=Lucene45: {range_facet_l=PostingsFormat(name=Direct), text=Pulsing41(freqCutoff=18 minBlockSize=35 maxBlockSize=160), _version_=Pulsing41(freqCutoff=13 minBlockSize=35 maxBlockSize=160), rnd_b=MockFixedIntBlock(blockSize=766), intDefault=Pulsing41(freqCutoff=13 minBlockSize=35 maxBlockSize=160), timestamp=PostingsFormat(name=Direct), id=MockFixedIntBlock(blockSize=766), a_t=PostingsFormat(name=Direct), range_facet_sl=MockFixedIntBlock(blockSize=766), range_facet_si=PostingsFormat(name=Direct), other_tl1=Pulsing41(freqCutoff=13 minBlockSize=35 maxBlockSize=160), multiDefault=PostingsFormat(name=Direct), a_si=Pulsing41(freqCutoff=13 minBlockSize=35 maxBlockSize=160)}, docValues:{}, sim=RandomSimilarityProvider(queryNorm=true,coord=yes): {}, locale=fi_FI, timezone=Canada/Pacific
   [junit4]   2> NOTE: FreeBSD 9.1-RELEASE-p3 amd64/Oracle Corporation 1.7.0_25 (64-bit)/cpus=16,threads=1,free=159514624,total=536870912
   [junit4]   2> NOTE: All tests run in this JVM: [SolrIndexSplitterTest, SampleTest, TestSolrCoreProperties, CopyFieldTest, TestFastLRUCache, DocValuesTest, TestBinaryResponseWriter, TestCloudManagedSchemaAddField, HdfsBasicDistributedZk2Test, TestSolrIndexConfig, BasicFunctionalityTest, TestLazyCores, TestSystemIdResolver, RecoveryZkTest, JsonLoaderTest, TestSolrQueryParserDefaultOperatorResource, TestSolrQueryParser, TestLFUCache, TestGroupingSearch, BasicZkTest, SimplePostToolTest, TestUniqueKeyFieldResource, CoreAdminCreateDiscoverTest, DistributedQueryElevationComponentTest, TestMergePolicyConfig, TestFunctionQuery, DeleteShardTest, HdfsDirectoryTest, SolrCmdDistributorTest, TestBadConfig, TestRandomFaceting, ChaosMonkeySafeLeaderTest, HighlighterTest, TestRTGBase, FieldMutatingUpdateProcessorTest, SynonymTokenizerTest, TestRandomMergePolicy, ConvertedLegacyTest, TestNRTOpen, FullSolrCloudDistribCmdsTest, TestDFRSimilarityFactory, LegacyHTMLStripCharFilterTest, BinaryUpdateRequestHandlerTest, TestFieldTypeResource, TestDefaultSimilarityFactory, TestXIncludeConfig, DocValuesMissingTest, CachingDirectoryFactoryTest, TestCopyFieldCollectionResource, TestJmxIntegration, TestQueryTypes, TestStressReorder, DistributedSpellCheckComponentTest, TestPartialUpdateDeduplication, DocValuesMultiTest, TestRangeQuery, AlternateDirectoryTest, TestAddFieldRealTimeGet, MinimalSchemaTest, SortByFunctionTest, DistributedTermsComponentTest, TestLMDirichletSimilarityFactory, BasicDistributedZkTest, FastVectorHighlighterTest, TestFieldCollectionResource, TestStressRecovery, TestWordDelimiterFilterFactory, TestSchemaSimilarityResource, LeaderElectionIntegrationTest, TestValueSourceCache, TestBinaryField, TestExtendedDismaxParser, SearchHandlerTest, DocumentBuilderTest, TestLuceneMatchVersion, QueryParsingTest, DirectUpdateHandlerOptimizeTest, TestRemoteStreaming, HdfsSyncSliceTest, SyncSliceTest]
   [junit4] Completed on J0 in 112.31s, 1 test, 1 failure <<< FAILURES!

[...truncated 500 lines...]
BUILD FAILED
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/build.xml:428: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/build.xml:408: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/build.xml:39: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build.xml:189: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/common-build.xml:483: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/lucene/common-build.xml:1263: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/lucene/common-build.xml:906: There were test failures: 327 suites, 1453 tests, 1 failure, 35 ignored (6 assumptions)

Total time: 57 minutes 55 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure