You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by Policeman Jenkins Server <je...@thetaphi.de> on 2013/08/13 13:27:05 UTC
[JENKINS] Lucene-Solr-trunk-Linux (64bit/jdk1.7.0_25) - Build #
6996 - Still Failing!
Build: http://jenkins.thetaphi.de/job/Lucene-Solr-trunk-Linux/6996/
Java: 64bit/jdk1.7.0_25 -XX:-UseCompressedOops -XX:+UseParallelGC
1 tests failed.
FAILED: org.apache.solr.cloud.FullSolrCloudDistribCmdsTest.testDistribSearch
Error Message:
Server at http://127.0.0.1:34548/ov_v/collection1 returned non ok status:500, message:Server Error
Stack Trace:
org.apache.solr.client.solrj.impl.HttpSolrServer$RemoteSolrException: Server at http://127.0.0.1:34548/ov_v/collection1 returned non ok status:500, message:Server Error
at __randomizedtesting.SeedInfo.seed([AC2C812CF5D193D:8B24460AB8027901]:0)
at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:385)
at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:180)
at org.apache.solr.client.solrj.request.AbstractUpdateRequest.process(AbstractUpdateRequest.java:117)
at org.apache.solr.cloud.FullSolrCloudDistribCmdsTest.testIndexQueryDeleteHierarchical(FullSolrCloudDistribCmdsTest.java:259)
at org.apache.solr.cloud.FullSolrCloudDistribCmdsTest.doTest(FullSolrCloudDistribCmdsTest.java:124)
at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:835)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1559)
at com.carrotsearch.randomizedtesting.RandomizedRunner.access$600(RandomizedRunner.java:79)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:773)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:787)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:782)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:442)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:746)
at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:648)
at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:682)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:693)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
at java.lang.Thread.run(Thread.java:724)
Build Log:
[...truncated 9861 lines...]
[junit4] Suite: org.apache.solr.cloud.FullSolrCloudDistribCmdsTest
[junit4] 2> 924844 T3063 oas.BaseDistributedSearchTestCase.initHostContext Setting hostContext system property: /ov_v/
[junit4] 2> 924848 T3063 oas.SolrTestCaseJ4.setUp ###Starting testDistribSearch
[junit4] 2> Creating dataDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./solrtest-FullSolrCloudDistribCmdsTest-1376392776073
[junit4] 2> 924848 T3063 oasc.ZkTestServer.run STARTING ZK TEST SERVER
[junit4] 2> 924849 T3064 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
[junit4] 2> 924949 T3063 oasc.ZkTestServer.run start zk server on port:36230
[junit4] 2> 924950 T3063 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 925149 T3070 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@39ce02f7 name:ZooKeeperConnection Watcher:127.0.0.1:36230 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 925150 T3063 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 925150 T3063 oascc.SolrZkClient.makePath makePath: /solr
[junit4] 2> 925154 T3063 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 925155 T3072 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@22a29f97 name:ZooKeeperConnection Watcher:127.0.0.1:36230/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 925155 T3063 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 925155 T3063 oascc.SolrZkClient.makePath makePath: /collections/collection1
[junit4] 2> 925157 T3063 oascc.SolrZkClient.makePath makePath: /collections/collection1/shards
[junit4] 2> 925158 T3063 oascc.SolrZkClient.makePath makePath: /collections/control_collection
[junit4] 2> 925159 T3063 oascc.SolrZkClient.makePath makePath: /collections/control_collection/shards
[junit4] 2> 925161 T3063 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
[junit4] 2> 925161 T3063 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.xml
[junit4] 2> 925163 T3063 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
[junit4] 2> 925164 T3063 oascc.SolrZkClient.makePath makePath: /configs/conf1/schema.xml
[junit4] 2> 925165 T3063 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
[junit4] 2> 925166 T3063 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.snippet.randomindexconfig.xml
[junit4] 2> 925167 T3063 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
[junit4] 2> 925167 T3063 oascc.SolrZkClient.makePath makePath: /configs/conf1/stopwords.txt
[junit4] 2> 925169 T3063 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
[junit4] 2> 925169 T3063 oascc.SolrZkClient.makePath makePath: /configs/conf1/protwords.txt
[junit4] 2> 925171 T3063 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
[junit4] 2> 925171 T3063 oascc.SolrZkClient.makePath makePath: /configs/conf1/currency.xml
[junit4] 2> 925172 T3063 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
[junit4] 2> 925173 T3063 oascc.SolrZkClient.makePath makePath: /configs/conf1/open-exchange-rates.json
[junit4] 2> 925174 T3063 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4] 2> 925174 T3063 oascc.SolrZkClient.makePath makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4] 2> 925176 T3063 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
[junit4] 2> 925176 T3063 oascc.SolrZkClient.makePath makePath: /configs/conf1/old_synonyms.txt
[junit4] 2> 925178 T3063 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
[junit4] 2> 925178 T3063 oascc.SolrZkClient.makePath makePath: /configs/conf1/synonyms.txt
[junit4] 2> 925240 T3063 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4] 2> 925254 T3063 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:34548
[junit4] 2> 925255 T3063 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4] 2> 925255 T3063 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4] 2> 925255 T3063 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-controljetty-1376392776405
[junit4] 2> 925255 T3063 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-controljetty-1376392776405/'
[junit4] 2> 925273 T3063 oasc.ConfigSolr.fromFile Loading container configuration from /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-controljetty-1376392776405/solr.xml
[junit4] 2> 925303 T3063 oasc.CoreContainer.<init> New CoreContainer 146594956
[junit4] 2> 925304 T3063 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-controljetty-1376392776405/]
[junit4] 2> 925305 T3063 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4] 2> 925305 T3063 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4] 2> 925305 T3063 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4] 2> 925305 T3063 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4] 2> 925306 T3063 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4] 2> 925306 T3063 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4] 2> 925306 T3063 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4] 2> 925306 T3063 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4] 2> 925306 T3063 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4] 2> 925307 T3063 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4] 2> 925310 T3063 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:36230/solr
[junit4] 2> 925311 T3063 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4] 2> 925311 T3063 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 925313 T3083 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@7af331e4 name:ZooKeeperConnection Watcher:127.0.0.1:36230 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 925313 T3063 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 925314 T3063 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4] 2> 925316 T3063 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 925317 T3085 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@2c3ccd9 name:ZooKeeperConnection Watcher:127.0.0.1:36230/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 925317 T3063 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 925318 T3063 oascc.SolrZkClient.makePath makePath: /overseer/queue
[junit4] 2> 925321 T3063 oascc.SolrZkClient.makePath makePath: /overseer/collection-queue-work
[junit4] 2> 925322 T3063 oascc.SolrZkClient.makePath makePath: /live_nodes
[junit4] 2> 925323 T3063 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:34548_ov_v
[junit4] 2> 925324 T3063 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:34548_ov_v
[junit4] 2> 925325 T3063 oascc.SolrZkClient.makePath makePath: /overseer_elect/election
[junit4] 2> 925328 T3063 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
[junit4] 2> 925329 T3063 oasc.Overseer.start Overseer (id=90203276972916739-127.0.0.1:34548_ov_v-n_0000000000) starting
[junit4] 2> 925330 T3063 oascc.SolrZkClient.makePath makePath: /overseer/queue-work
[junit4] 2> 925332 T3087 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
[junit4] 2> 925333 T3063 oascc.SolrZkClient.makePath makePath: /clusterstate.json
[junit4] 2> 925334 T3063 oascc.SolrZkClient.makePath makePath: /aliases.json
[junit4] 2> 925335 T3063 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 925336 T3086 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
[junit4] 2> 925339 T3088 oasc.ZkController.publish publishing core=collection1 state=down
[junit4] 2> 925339 T3088 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 925340 T3088 oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4] 2> 926838 T3086 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 926839 T3086 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4] 2> "operation":"state",
[junit4] 2> "core_node_name":"core_node1",
[junit4] 2> "numShards":"1",
[junit4] 2> "shard":null,
[junit4] 2> "roles":null,
[junit4] 2> "state":"down",
[junit4] 2> "shard_state":"active",
[junit4] 2> "core":"collection1",
[junit4] 2> "collection":"control_collection",
[junit4] 2> "shard_range":null,
[junit4] 2> "node_name":"127.0.0.1:34548_ov_v",
[junit4] 2> "base_url":"http://127.0.0.1:34548/ov_v"}
[junit4] 2> 926839 T3086 oasc.Overseer$ClusterStateUpdater.createCollection Create collection control_collection with shards [shard1]
[junit4] 2> 926839 T3086 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4] 2> 926843 T3085 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 927340 T3088 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
[junit4] 2> 927341 T3088 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-controljetty-1376392776405/collection1
[junit4] 2> 927341 T3088 oasc.ZkController.createCollectionZkNode Check for collection zkNode:control_collection
[junit4] 2> 927341 T3088 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4] 2> 927341 T3088 oasc.ZkController.readConfigName Load collection config from:/collections/control_collection
[junit4] 2> 927342 T3088 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-controljetty-1376392776405/collection1/'
[junit4] 2> 927343 T3088 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-controljetty-1376392776405/collection1/lib/README' to classloader
[junit4] 2> 927343 T3088 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-controljetty-1376392776405/collection1/lib/classes/' to classloader
[junit4] 2> 927368 T3088 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4] 2> 927398 T3088 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4] 2> 927399 T3088 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4] 2> 927404 T3088 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4] 2> 927705 T3088 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4] 2> 927709 T3088 oass.IndexSchema.readSchema default search field in schema is text
[junit4] 2> 927711 T3088 oass.IndexSchema.readSchema unique key field: id
[junit4] 2> 927719 T3088 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 927721 T3088 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 927724 T3088 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4] 2> 927724 T3088 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4] 2> 927724 T3088 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4] 2> 927725 T3088 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4] 2> 927725 T3088 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4] 2> 927725 T3088 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4] 2> 927725 T3088 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4] 2> 927726 T3088 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-controljetty-1376392776405/collection1/, dataDir=./org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-1376392776073/control/data/
[junit4] 2> 927726 T3088 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@7d9acbbd
[junit4] 2> 927726 T3088 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-1376392776073/control/data
[junit4] 2> 927727 T3088 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-1376392776073/control/data/index/
[junit4] 2> 927727 T3088 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-1376392776073/control/data/index' doesn't exist. Creating new index...
[junit4] 2> 927727 T3088 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-1376392776073/control/data/index
[junit4] 2> 927728 T3088 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@29b9afde lockFactory=org.apache.lucene.store.NativeFSLockFactory@37c85ee6),segFN=segments_1,generation=1}
[junit4] 2> 927729 T3088 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 927731 T3088 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4] 2> 927731 T3088 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4] 2> 927732 T3088 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 927732 T3088 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4] 2> 927732 T3088 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4] 2> 927733 T3088 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4] 2> 927733 T3088 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4] 2> 927733 T3088 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4] 2> 927733 T3088 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4] 2> 927735 T3088 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 927737 T3088 oass.SolrIndexSearcher.<init> Opening Searcher@2401ffbc main
[junit4] 2> 927738 T3088 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4] 2> 927738 T3088 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4] 2> 927740 T3089 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@2401ffbc main{StandardDirectoryReader(segments_1:1)}
[junit4] 2> 927741 T3088 oasc.CoreContainer.registerCore registering core: collection1
[junit4] 2> 927741 T3088 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:34548/ov_v collection:control_collection shard:shard1
[junit4] 2> 927741 T3088 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=10000&maxConnectionsPerHost=20&connTimeout=15000&socketTimeout=30000&retry=false
[junit4] 2> 927744 T3088 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leader_elect/shard1/election
[junit4] 2> 927748 T3088 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard1
[junit4] 2> 927750 T3088 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4] 2> 927750 T3088 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4] 2> 927750 T3088 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:34548/ov_v/collection1/
[junit4] 2> 927750 T3088 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4] 2> 927750 T3088 oasc.SyncStrategy.syncToMe http://127.0.0.1:34548/ov_v/collection1/ has no replicas
[junit4] 2> 927751 T3088 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:34548/ov_v/collection1/ shard1
[junit4] 2> 927751 T3088 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leaders/shard1
[junit4] 2> 928345 T3086 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 928349 T3085 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 928356 T3088 oasc.ZkController.register We are http://127.0.0.1:34548/ov_v/collection1/ and leader is http://127.0.0.1:34548/ov_v/collection1/
[junit4] 2> 928356 T3088 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:34548/ov_v
[junit4] 2> 928356 T3088 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4] 2> 928356 T3088 oasc.ZkController.publish publishing core=collection1 state=active
[junit4] 2> 928356 T3088 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 928357 T3088 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 928358 T3063 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1
[junit4] 2> 928358 T3063 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4] 2> 928359 T3063 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4] 2> 928361 T3063 oascsi.HttpClientUtil.createClient Creating new http client, config:
[junit4] 2> 928362 T3063 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 928363 T3092 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@5fe4fa2 name:ZooKeeperConnection Watcher:127.0.0.1:36230/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 928363 T3063 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 928364 T3063 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 928365 T3063 oasc.ChaosMonkey.monkeyLog monkey: init - expire sessions:true cause connection loss:true
[junit4] 2> 928430 T3063 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4] 2> 928432 T3063 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:59191
[junit4] 2> 928433 T3063 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4] 2> 928433 T3063 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4] 2> 928433 T3063 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-jetty1-1376392779590
[junit4] 2> 928434 T3063 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-jetty1-1376392779590/'
[junit4] 2> 928443 T3063 oasc.ConfigSolr.fromFile Loading container configuration from /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-jetty1-1376392779590/solr.xml
[junit4] 2> 928479 T3063 oasc.CoreContainer.<init> New CoreContainer 1999633734
[junit4] 2> 928480 T3063 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-jetty1-1376392779590/]
[junit4] 2> 928481 T3063 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4] 2> 928481 T3063 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4] 2> 928481 T3063 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4] 2> 928482 T3063 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4] 2> 928482 T3063 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4] 2> 928482 T3063 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4] 2> 928482 T3063 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4] 2> 928482 T3063 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4] 2> 928483 T3063 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4] 2> 928483 T3063 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4] 2> 928486 T3063 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:36230/solr
[junit4] 2> 928486 T3063 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4] 2> 928487 T3063 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 928488 T3103 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@6cf1bdd9 name:ZooKeeperConnection Watcher:127.0.0.1:36230 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 928488 T3063 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 928489 T3063 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4] 2> 928492 T3063 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 928493 T3105 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@3baf56f2 name:ZooKeeperConnection Watcher:127.0.0.1:36230/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 928493 T3063 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 928495 T3063 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 929497 T3063 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:59191_ov_v
[junit4] 2> 929498 T3063 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:59191_ov_v
[junit4] 2> 929501 T3085 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4] 2> 929501 T3105 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4] 2> 929501 T3092 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4] 2> 929501 T3085 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
[junit4] 2> 929505 T3106 oasc.ZkController.publish publishing core=collection1 state=down
[junit4] 2> 929506 T3106 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 929507 T3106 oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4] 2> 929855 T3086 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 929856 T3086 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4] 2> "operation":"state",
[junit4] 2> "core_node_name":"core_node1",
[junit4] 2> "numShards":"1",
[junit4] 2> "shard":"shard1",
[junit4] 2> "roles":null,
[junit4] 2> "state":"active",
[junit4] 2> "shard_state":"active",
[junit4] 2> "core":"collection1",
[junit4] 2> "collection":"control_collection",
[junit4] 2> "shard_range":null,
[junit4] 2> "node_name":"127.0.0.1:34548_ov_v",
[junit4] 2> "base_url":"http://127.0.0.1:34548/ov_v"}
[junit4] 2> 929860 T3086 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4] 2> "operation":"state",
[junit4] 2> "core_node_name":"core_node1",
[junit4] 2> "numShards":"2",
[junit4] 2> "shard":null,
[junit4] 2> "roles":null,
[junit4] 2> "state":"down",
[junit4] 2> "shard_state":"active",
[junit4] 2> "core":"collection1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "shard_range":null,
[junit4] 2> "node_name":"127.0.0.1:59191_ov_v",
[junit4] 2> "base_url":"http://127.0.0.1:59191/ov_v"}
[junit4] 2> 929860 T3086 oasc.Overseer$ClusterStateUpdater.createCollection Create collection collection1 with shards [shard1, shard2]
[junit4] 2> 929860 T3086 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4] 2> 929863 T3105 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 929863 T3092 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 929863 T3085 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 930507 T3106 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
[junit4] 2> 930507 T3106 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-jetty1-1376392779590/collection1
[junit4] 2> 930507 T3106 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4] 2> 930508 T3106 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4] 2> 930508 T3106 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4] 2> 930509 T3106 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-jetty1-1376392779590/collection1/'
[junit4] 2> 930509 T3106 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-jetty1-1376392779590/collection1/lib/README' to classloader
[junit4] 2> 930509 T3106 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-jetty1-1376392779590/collection1/lib/classes/' to classloader
[junit4] 2> 930534 T3106 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4] 2> 930563 T3106 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4] 2> 930564 T3106 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4] 2> 930570 T3106 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4] 2> 930877 T3106 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4] 2> 930882 T3106 oass.IndexSchema.readSchema default search field in schema is text
[junit4] 2> 930884 T3106 oass.IndexSchema.readSchema unique key field: id
[junit4] 2> 930892 T3106 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 930895 T3106 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 930897 T3106 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4] 2> 930897 T3106 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4] 2> 930897 T3106 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4] 2> 930898 T3106 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4] 2> 930898 T3106 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4] 2> 930898 T3106 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4] 2> 930898 T3106 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4] 2> 930899 T3106 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-jetty1-1376392779590/collection1/, dataDir=./org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-1376392776073/jetty1/
[junit4] 2> 930899 T3106 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@7d9acbbd
[junit4] 2> 930899 T3106 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-1376392776073/jetty1
[junit4] 2> 930900 T3106 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-1376392776073/jetty1/index/
[junit4] 2> 930900 T3106 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-1376392776073/jetty1/index' doesn't exist. Creating new index...
[junit4] 2> 930900 T3106 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-1376392776073/jetty1/index
[junit4] 2> 930901 T3106 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@617e6614 lockFactory=org.apache.lucene.store.NativeFSLockFactory@52b0a9e0),segFN=segments_1,generation=1}
[junit4] 2> 930901 T3106 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 930904 T3106 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4] 2> 930904 T3106 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4] 2> 930905 T3106 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 930905 T3106 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4] 2> 930906 T3106 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4] 2> 930906 T3106 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4] 2> 930906 T3106 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4] 2> 930906 T3106 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4] 2> 930907 T3106 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4] 2> 930909 T3106 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 930911 T3106 oass.SolrIndexSearcher.<init> Opening Searcher@4e317904 main
[junit4] 2> 930911 T3106 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4] 2> 930911 T3106 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4] 2> 930915 T3107 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@4e317904 main{StandardDirectoryReader(segments_1:1)}
[junit4] 2> 930916 T3106 oasc.CoreContainer.registerCore registering core: collection1
[junit4] 2> 930916 T3106 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:59191/ov_v collection:collection1 shard:shard1
[junit4] 2> 930916 T3106 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=10000&maxConnectionsPerHost=20&connTimeout=15000&socketTimeout=30000&retry=false
[junit4] 2> 930919 T3106 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
[junit4] 2> 930923 T3106 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard1
[junit4] 2> 930924 T3106 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4] 2> 930924 T3106 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4] 2> 930924 T3106 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:59191/ov_v/collection1/
[junit4] 2> 930924 T3106 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4] 2> 930925 T3106 oasc.SyncStrategy.syncToMe http://127.0.0.1:59191/ov_v/collection1/ has no replicas
[junit4] 2> 930925 T3106 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:59191/ov_v/collection1/ shard1
[junit4] 2> 930925 T3106 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
[junit4] 2> 931367 T3086 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 931372 T3092 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 931372 T3085 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 931372 T3105 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 931379 T3106 oasc.ZkController.register We are http://127.0.0.1:59191/ov_v/collection1/ and leader is http://127.0.0.1:59191/ov_v/collection1/
[junit4] 2> 931380 T3106 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:59191/ov_v
[junit4] 2> 931380 T3106 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4] 2> 931380 T3106 oasc.ZkController.publish publishing core=collection1 state=active
[junit4] 2> 931380 T3106 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 931382 T3106 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 931383 T3063 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1
[junit4] 2> 931383 T3063 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4] 2> 931383 T3063 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4] 2> 931446 T3063 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4] 2> 931447 T3063 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:42347
[junit4] 2> 931449 T3063 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4] 2> 931449 T3063 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4] 2> 931449 T3063 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-jetty2-1376392782611
[junit4] 2> 931450 T3063 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-jetty2-1376392782611/'
[junit4] 2> 931459 T3063 oasc.ConfigSolr.fromFile Loading container configuration from /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-jetty2-1376392782611/solr.xml
[junit4] 2> 931488 T3063 oasc.CoreContainer.<init> New CoreContainer 1334927866
[junit4] 2> 931489 T3063 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-jetty2-1376392782611/]
[junit4] 2> 931489 T3063 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4] 2> 931490 T3063 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4] 2> 931490 T3063 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4] 2> 931490 T3063 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4] 2> 931490 T3063 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4] 2> 931491 T3063 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4] 2> 931491 T3063 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4] 2> 931491 T3063 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4] 2> 931491 T3063 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4] 2> 931491 T3063 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4] 2> 931494 T3063 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:36230/solr
[junit4] 2> 931495 T3063 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4] 2> 931495 T3063 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 931496 T3119 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@25e53bb2 name:ZooKeeperConnection Watcher:127.0.0.1:36230 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 931497 T3063 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 931497 T3063 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4] 2> 931500 T3063 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 931501 T3121 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@5d61931d name:ZooKeeperConnection Watcher:127.0.0.1:36230/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 931501 T3063 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 931504 T3063 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 932506 T3063 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:42347_ov_v
[junit4] 2> 932507 T3063 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:42347_ov_v
[junit4] 2> 932508 T3092 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
[junit4] 2> 932508 T3121 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4] 2> 932509 T3085 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4] 2> 932508 T3105 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4] 2> 932509 T3092 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4] 2> 932509 T3105 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4] 2> 932509 T3085 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4] 2> 932513 T3122 oasc.ZkController.publish publishing core=collection1 state=down
[junit4] 2> 932513 T3122 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 932514 T3122 oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4] 2> 932876 T3086 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 932877 T3086 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4] 2> "operation":"state",
[junit4] 2> "core_node_name":"core_node1",
[junit4] 2> "numShards":"2",
[junit4] 2> "shard":"shard1",
[junit4] 2> "roles":null,
[junit4] 2> "state":"active",
[junit4] 2> "shard_state":"active",
[junit4] 2> "core":"collection1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "shard_range":null,
[junit4] 2> "node_name":"127.0.0.1:59191_ov_v",
[junit4] 2> "base_url":"http://127.0.0.1:59191/ov_v"}
[junit4] 2> 932881 T3086 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4] 2> "operation":"state",
[junit4] 2> "core_node_name":"core_node2",
[junit4] 2> "numShards":"2",
[junit4] 2> "shard":null,
[junit4] 2> "roles":null,
[junit4] 2> "state":"down",
[junit4] 2> "shard_state":"active",
[junit4] 2> "core":"collection1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "shard_range":null,
[junit4] 2> "node_name":"127.0.0.1:42347_ov_v",
[junit4] 2> "base_url":"http://127.0.0.1:42347/ov_v"}
[junit4] 2> 932881 T3086 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=2
[junit4] 2> 932881 T3086 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard2
[junit4] 2> 932883 T3105 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 932883 T3121 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 932883 T3085 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 932883 T3092 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 933514 T3122 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
[junit4] 2> 933515 T3122 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-jetty2-1376392782611/collection1
[junit4] 2> 933515 T3122 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4] 2> 933515 T3122 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4] 2> 933516 T3122 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4] 2> 933516 T3122 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-jetty2-1376392782611/collection1/'
[junit4] 2> 933517 T3122 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-jetty2-1376392782611/collection1/lib/README' to classloader
[junit4] 2> 933517 T3122 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-jetty2-1376392782611/collection1/lib/classes/' to classloader
[junit4] 2> 933548 T3122 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4] 2> 933577 T3122 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4] 2> 933578 T3122 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4] 2> 933584 T3122 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4] 2> 933878 T3122 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4] 2> 933883 T3122 oass.IndexSchema.readSchema default search field in schema is text
[junit4] 2> 933885 T3122 oass.IndexSchema.readSchema unique key field: id
[junit4] 2> 933893 T3122 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 933895 T3122 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 933897 T3122 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4] 2> 933898 T3122 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4] 2> 933898 T3122 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4] 2> 933899 T3122 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4] 2> 933899 T3122 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4] 2> 933899 T3122 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4] 2> 933900 T3122 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4] 2> 933900 T3122 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-jetty2-1376392782611/collection1/, dataDir=./org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-1376392776073/jetty2/
[junit4] 2> 933900 T3122 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@7d9acbbd
[junit4] 2> 933901 T3122 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-1376392776073/jetty2
[junit4] 2> 933901 T3122 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-1376392776073/jetty2/index/
[junit4] 2> 933901 T3122 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-1376392776073/jetty2/index' doesn't exist. Creating new index...
[junit4] 2> 933901 T3122 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-1376392776073/jetty2/index
[junit4] 2> 933902 T3122 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@4520e6f1 lockFactory=org.apache.lucene.store.NativeFSLockFactory@5e908de1),segFN=segments_1,generation=1}
[junit4] 2> 933902 T3122 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 933905 T3122 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4] 2> 933906 T3122 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4] 2> 933906 T3122 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 933907 T3122 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4] 2> 933907 T3122 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4] 2> 933907 T3122 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4] 2> 933907 T3122 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4] 2> 933908 T3122 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4] 2> 933908 T3122 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4] 2> 933910 T3122 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 933912 T3122 oass.SolrIndexSearcher.<init> Opening Searcher@36f3669d main
[junit4] 2> 933913 T3122 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4] 2> 933913 T3122 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4] 2> 933916 T3123 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@36f3669d main{StandardDirectoryReader(segments_1:1)}
[junit4] 2> 933917 T3122 oasc.CoreContainer.registerCore registering core: collection1
[junit4] 2> 933917 T3122 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:42347/ov_v collection:collection1 shard:shard2
[junit4] 2> 933917 T3122 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=10000&maxConnectionsPerHost=20&connTimeout=15000&socketTimeout=30000&retry=false
[junit4] 2> 933920 T3122 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard2/election
[junit4] 2> 933924 T3122 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard2
[junit4] 2> 933925 T3122 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4] 2> 933926 T3122 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4] 2> 933926 T3122 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:42347/ov_v/collection1/
[junit4] 2> 933926 T3122 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4] 2> 933926 T3122 oasc.SyncStrategy.syncToMe http://127.0.0.1:42347/ov_v/collection1/ has no replicas
[junit4] 2> 933926 T3122 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:42347/ov_v/collection1/ shard2
[junit4] 2> 933926 T3122 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard2
[junit4] 2> 934388 T3086 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 934393 T3085 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 934393 T3092 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 934394 T3121 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 934393 T3105 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 934434 T3122 oasc.ZkController.register We are http://127.0.0.1:42347/ov_v/collection1/ and leader is http://127.0.0.1:42347/ov_v/collection1/
[junit4] 2> 934434 T3122 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:42347/ov_v
[junit4] 2> 934434 T3122 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4] 2> 934434 T3122 oasc.ZkController.publish publishing core=collection1 state=active
[junit4] 2> 934434 T3122 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 934436 T3122 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 934437 T3063 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1
[junit4] 2> 934437 T3063 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4] 2> 934437 T3063 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4] 2> 934499 T3063 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4] 2> 934502 T3063 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:60930
[junit4] 2> 934502 T3063 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4] 2> 934502 T3063 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4] 2> 934503 T3063 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-jetty3-1376392785665
[junit4] 2> 934503 T3063 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-jetty3-1376392785665/'
[junit4] 2> 934512 T3063 oasc.ConfigSolr.fromFile Loading container configuration from /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-jetty3-1376392785665/solr.xml
[junit4] 2> 934541 T3063 oasc.CoreContainer.<init> New CoreContainer 1750956191
[junit4] 2> 934541 T3063 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-jetty3-1376392785665/]
[junit4] 2> 934542 T3063 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4] 2> 934543 T3063 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4] 2> 934543 T3063 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4] 2> 934543 T3063 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4] 2> 934543 T3063 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4] 2> 934543 T3063 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4] 2> 934544 T3063 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4] 2> 934544 T3063 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4] 2> 934544 T3063 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4] 2> 934544 T3063 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4] 2> 934547 T3063 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:36230/solr
[junit4] 2> 934547 T3063 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4] 2> 934548 T3063 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 934550 T3135 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@6ccaefcf name:ZooKeeperConnection Watcher:127.0.0.1:36230 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 934550 T3063 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 934551 T3063 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4] 2> 934553 T3063 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 934554 T3137 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@7b642f48 name:ZooKeeperConnection Watcher:127.0.0.1:36230/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 934555 T3063 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 934557 T3063 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 935559 T3063 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:60930_ov_v
[junit4] 2> 935561 T3063 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:60930_ov_v
[junit4] 2> 935562 T3092 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4] 2> 935563 T3085 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4] 2> 935563 T3105 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4] 2> 935563 T3085 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4] 2> 935563 T3121 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4] 2> 935563 T3137 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4] 2> 935563 T3105 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4] 2> 935564 T3092 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4] 2> 935563 T3121 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4] 2> 935568 T3138 oasc.ZkController.publish publishing core=collection1 state=down
[junit4] 2> 935568 T3138 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 935569 T3138 oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4] 2> 935897 T3086 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 935898 T3086 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4] 2> "operation":"state",
[junit4] 2> "core_node_name":"core_node2",
[junit4] 2> "numShards":"2",
[junit4] 2> "shard":"shard2",
[junit4] 2> "roles":null,
[junit4] 2> "state":"active",
[junit4] 2> "shard_state":"active",
[junit4] 2> "core":"collection1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "shard_range":null,
[junit4] 2> "node_name":"127.0.0.1:42347_ov_v",
[junit4] 2> "base_url":"http://127.0.0.1:42347/ov_v"}
[junit4] 2> 935901 T3086 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4] 2> "operation":"state",
[junit4] 2> "core_node_name":"core_node3",
[junit4] 2> "numShards":"2",
[junit4] 2> "shard":null,
[junit4] 2> "roles":null,
[junit4] 2> "state":"down",
[junit4] 2> "shard_state":"active",
[junit4] 2> "core":"collection1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "shard_range":null,
[junit4] 2> "node_name":"127.0.0.1:60930_ov_v",
[junit4] 2> "base_url":"http://127.0.0.1:60930/ov_v"}
[junit4] 2> 935901 T3086 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=2
[junit4] 2> 935901 T3086 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4] 2> 935905 T3092 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4] 2> 935905 T3121 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4] 2> 935905 T3137 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4] 2> 935905 T3105 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4] 2> 935905 T3085 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4] 2> 936570 T3138 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
[junit4] 2> 936570 T3138 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-jetty3-1376392785665/collection1
[junit4] 2> 936570 T3138 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4] 2> 936571 T3138 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4] 2> 936571 T3138 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4] 2> 936572 T3138 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-jetty3-1376392785665/collection1/'
[junit4] 2> 936572 T3138 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-jetty3-1376392785665/collection1/lib/README' to classloader
[junit4] 2> 936572 T3138 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-jetty3-1376392785665/collection1/lib/classes/' to classloader
[junit4] 2> 936597 T3138 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4] 2> 936626 T3138 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4] 2> 936627 T3138 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4] 2> 936633 T3138 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4] 2> 936925 T3138 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4] 2> 936930 T3138 oass.IndexSchema.readSchema default search field in schema is text
[junit4] 2> 936932 T3138 oass.IndexSchema.readSchema unique key field: id
[junit4] 2> 936941 T3138 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 936944 T3138 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 936946 T3138 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4] 2> 936946 T3138 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4] 2> 936946 T3138 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4] 2> 936947 T3138 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4] 2> 936947 T3138 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4] 2> 936947 T3138 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4] 2> 936948 T3138 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4] 2> 936948 T3138 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-jetty3-1376392785665/collection1/, dataDir=./org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-1376392776073/jetty3/
[junit4] 2> 936948 T3138 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@7d9acbbd
[junit4] 2> 936949 T3138 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-1376392776073/jetty3
[junit4] 2> 936949 T3138 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-1376392776073/jetty3/index/
[junit4] 2> 936949 T3138 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-1376392776073/jetty3/index' doesn't exist. Creating new index...
[junit4] 2> 936950 T3138 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-1376392776073/jetty3/index
[junit4] 2> 936951 T3138 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@4faf5dfb lockFactory=org.apache.lucene.store.NativeFSLockFactory@696e2c92),segFN=segments_1,generation=1}
[junit4] 2> 936951 T3138 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 936955 T3138 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4] 2> 936955 T3138 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4] 2> 936956 T3138 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 936956 T3138 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4] 2> 936957 T3138 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4] 2> 936957 T3138 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4] 2> 936957 T3138 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4] 2> 936958 T3138 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4] 2> 936958 T3138 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4] 2> 936961 T3138 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 936964 T3138 oass.SolrIndexSearcher.<init> Opening Searcher@25e4f36e main
[junit4] 2> 936964 T3138 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4] 2> 936964 T3138 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4] 2> 936973 T3139 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@25e4f36e main{StandardDirectoryReader(segments_1:1)}
[junit4] 2> 936974 T3138 oasc.CoreContainer.registerCore registering core: collection1
[junit4] 2> 936975 T3138 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:60930/ov_v collection:collection1 shard:shard1
[junit4] 2> 936975 T3138 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=10000&maxConnectionsPerHost=20&connTimeout=15000&socketTimeout=30000&retry=false
[junit4] 2> 936979 T3138 oasc.ZkController.register We are http://127.0.0.1:60930/ov_v/collection1/ and leader is http://127.0.0.1:59191/ov_v/collection1/
[junit4] 2> 936979 T3138 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:60930/ov_v
[junit4] 2> 936980 T3138 oasc.ZkController.checkRecovery Core needs to recover:collection1
[junit4] 2> 936980 T3138 oasu.DefaultSolrCoreState.doRecovery Running recovery - first canceling any ongoing recovery
[junit4] 2> ASYNC NEW_CORE C2307 name=collection1 org.apache.solr.core.SolrCore@5ddfa912 url=http://127.0.0.1:60930/ov_v/collection1 node=127.0.0.1:60930_ov_v C2307_STATE=coll:collection1 core:collection1 props:{state=down, core=collection1, node_name=127.0.0.1:60930_ov_v, base_url=http://127.0.0.1:60930/ov_v}
[junit4] 2> 936980 T3140 C2307 P60930 oasc.RecoveryStrategy.run Starting recovery process. core=collection1 recoveringAfterStartup=true
[junit4] 2> 936980 T3138 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 936980 T3140 C2307 P60930 oasc.RecoveryStrategy.doRecovery ###### startupVersions=[]
[junit4] 2> 936981 T3140 C2307 P60930 oasc.ZkController.publish publishing core=collection1 state=recovering
[junit4] 2> 936981 T3140 C2307 P60930 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 936981 T3063 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1
[junit4] 2> 936981 T3063 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4] 2> 936981 T3140 C2307 P60930 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4] 2> 936982 T3063 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4] 2> 936986 T3101 oasha.CoreAdminHandler.handleWaitForStateAction Going to wait for coreNodeName: core_node3, state: recovering, checkLive: true, onlyIfLeader: true
[junit4] 2> 937045 T3063 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4] 2> 937047 T3063 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:33287
[junit4] 2> 937047 T3063 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4] 2> 937047 T3063 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4] 2> 937048 T3063 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-jetty4-1376392788210
[junit4] 2> 937048 T3063 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-jetty4-1376392788210/'
[junit4] 2> 937057 T3063 oasc.ConfigSolr.fromFile Loading container configuration from /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-jetty4-1376392788210/solr.xml
[junit4] 2> 937087 T3063 oasc.CoreContainer.<init> New CoreContainer 1887569806
[junit4] 2> 937088 T3063 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-jetty4-1376392788210/]
[junit4] 2> 937088 T3063 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4] 2> 937089 T3063 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4] 2> 937089 T3063 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4] 2> 937089 T3063 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4] 2> 937089 T3063 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4] 2> 937090 T3063 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4] 2> 937090 T3063 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4] 2> 937090 T3063 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4] 2> 937090 T3063 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4] 2> 937091 T3063 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4] 2> 937094 T3063 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:36230/solr
[junit4] 2> 937094 T3063 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4] 2> 937095 T3063 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 937096 T3152 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@44aff348 name:ZooKeeperConnection Watcher:127.0.0.1:36230 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 937096 T3063 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 937097 T3063 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4] 2> 937100 T3063 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 937101 T3154 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@b72ae02 name:ZooKeeperConnection Watcher:127.0.0.1:36230/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 937101 T3063 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 937103 T3063 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 937409 T3086 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 937409 T3086 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4] 2> "operation":"state",
[junit4] 2> "core_node_name":"core_node3",
[junit4] 2> "numShards":"2",
[junit4] 2> "shard":"shard1",
[junit4] 2> "roles":null,
[junit4] 2> "state":"recovering",
[junit4] 2> "shard_state":"active",
[junit4] 2> "core":"collection1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "shard_range":null,
[junit4] 2> "node_name":"127.0.0.1:60930_ov_v",
[junit4] 2> "base_url":"http://127.0.0.1:60930/ov_v"}
[junit4] 2> 937412 T3105 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4] 2> 937412 T3154 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4] 2> 937413 T3121 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4] 2> 937412 T3092 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4] 2> 937413 T3137 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4] 2> 937412 T3085 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4] 2> 937986 T3101 oasha.CoreAdminHandler.handleWaitForStateAction Waited coreNodeName: core_node3, state: recovering, checkLive: true, onlyIfLeader: true for: 1 seconds.
[junit4] 2> 937986 T3101 oass.SolrDispatchFilter.handleAdminRequest [admin] webapp=null path=/admin/cores params={coreNodeName=core_node3&state=recovering&nodeName=127.0.0.1:60930_ov_v&action=PREPRECOVERY&checkLive=true&core=collection1&wt=javabin&onlyIfLeader=true&version=2} status=0 QTime=1001
[junit4] 2> 938105 T3063 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:33287_ov_v
[junit4] 2> 938106 T3063 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:33287_ov_v
[junit4] 2> 938107 T3137 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4] 2> 938107 T3085 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4] 2> 938107 T3092 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4] 2> 938107 T3085 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4] 2> 938107 T3105 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4] 2> 938108 T3105 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4] 2> 938107 T3121 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4] 2> 938108 T3137 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4] 2> 938107 T3154 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4] 2> 938108 T3092 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4] 2> 938108 T3121 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_
[...truncated too long message...]
lockSize=73 maxBlockSize=236), 59grand53_s=Lucene41(blocksize=128), 73grand9_s=Pulsing41(freqCutoff=11 minBlockSize=73 maxBlockSize=236), 55grand42_s=Lucene41(blocksize=128), 37grand4_s=PostingsFormat(name=Asserting), 71grand0_s=Lucene41(blocksize=128), 57grand43_s=Pulsing41(freqCutoff=11 minBlockSize=73 maxBlockSize=236), 39grand7_s=PostingsFormat(name=Direct), 13child2_s=PostingsFormat(name=Asserting), 70grand58_s=Lucene41(blocksize=128), 14grand9_s=PostingsFormat(name=Asserting), 61grand3_s=Lucene41(blocksize=128), 50grand18_s=PostingsFormat(name=Asserting), 67grand15_s=Pulsing41(freqCutoff=11 minBlockSize=73 maxBlockSize=236), 71grand25_s=Pulsing41(freqCutoff=11 minBlockSize=73 maxBlockSize=236), 55grand21_s=PostingsFormat(name=Direct), 22child12_s=PostingsFormat(name=Direct), 30grand29_s=Lucene41(blocksize=128), 43grand17_s=Pulsing41(freqCutoff=11 minBlockSize=73 maxBlockSize=236), 74grand63_s=PostingsFormat(name=Asserting), 58grand50_s=PostingsFormat(name=Asserting), 53grand27_s=PostingsFormat(name=Direct), 17grand15_s=Lucene41(blocksize=128), 33grand6_s=Lucene41(blocksize=128), 75grand25_s=Pulsing41(freqCutoff=11 minBlockSize=73 maxBlockSize=236), 32grand16_s=Lucene41(blocksize=128), 24grand9_s=Pulsing41(freqCutoff=11 minBlockSize=73 maxBlockSize=236), 53grand38_s=PostingsFormat(name=Direct), 0parent_f1_s=PostingsFormat(name=Asserting), 34grand26_s=Pulsing41(freqCutoff=11 minBlockSize=73 maxBlockSize=236), 37child10_s=PostingsFormat(name=Direct), 68grand40_s=Lucene41(blocksize=128), 59grand0_s=PostingsFormat(name=Asserting), 63grand24_s=PostingsFormat(name=Direct), 52grand18_s=Lucene41(blocksize=128), 64grand7_s=PostingsFormat(name=Direct), 9grand2_s=PostingsFormat(name=Direct), 20grand5_s=Pulsing41(freqCutoff=11 minBlockSize=73 maxBlockSize=236), 23child6_s=Pulsing41(freqCutoff=11 minBlockSize=73 maxBlockSize=236), 29child18_s=PostingsFormat(name=Asserting), 72grand65_s=PostingsFormat(name=Asserting), 38child33_s=PostingsFormat(name=Direct), 40grand18_s=Pulsing41(freqCutoff=11 minBlockSize=73 maxBlockSize=236), 71grand19_s=Lucene41(blocksize=128), 65grand36_s=PostingsFormat(name=Asserting), 34child16_s=PostingsFormat(name=Asserting), 73grand40_s=PostingsFormat(name=Asserting), 64grand48_s=Lucene41(blocksize=128), 26child13_s=Lucene41(blocksize=128), 35grand34_s=Pulsing41(freqCutoff=11 minBlockSize=73 maxBlockSize=236), 65grand42_s=Pulsing41(freqCutoff=11 minBlockSize=73 maxBlockSize=236), 64grand53_s=PostingsFormat(name=Asserting), 39grand10_s=PostingsFormat(name=Direct), 18grand16_s=Lucene41(blocksize=128), 69grand44_s=PostingsFormat(name=Direct), 34grand20_s=PostingsFormat(name=Direct), 5grand0_s=PostingsFormat(name=Direct), 71grand14_s=Pulsing41(freqCutoff=11 minBlockSize=73 maxBlockSize=236), 7grand2_s=PostingsFormat(name=Direct), 75grand63_s=PostingsFormat(name=Direct), 65grand7_s=Lucene41(blocksize=128), 37grand23_s=PostingsFormat(name=Direct), 57grand54_s=Pulsing41(freqCutoff=11 minBlockSize=73 maxBlockSize=236), 31grand3_s=Pulsing41(freqCutoff=11 minBlockSize=73 maxBlockSize=236), 20child5_s=Pulsing41(freqCutoff=11 minBlockSize=73 maxBlockSize=236), 59grand36_s=Pulsing41(freqCutoff=11 minBlockSize=73 maxBlockSize=236), 35child9_s=Pulsing41(freqCutoff=11 minBlockSize=73 maxBlockSize=236), 34grand15_s=Pulsing41(freqCutoff=11 minBlockSize=73 maxBlockSize=236), 28child7_s=PostingsFormat(name=Direct), 33child32_s=PostingsFormat(name=Direct), 38child0_s=Pulsing41(freqCutoff=11 minBlockSize=73 maxBlockSize=236), 62grand1_s=PostingsFormat(name=Direct), 70grand6_s=PostingsFormat(name=Direct), 45grand21_s=Lucene41(blocksize=128), 58grand54_s=PostingsFormat(name=Asserting), 16grand11_s=Pulsing41(freqCutoff=11 minBlockSize=73 maxBlockSize=236), 37child27_s=Pulsing41(freqCutoff=11 minBlockSize=73 maxBlockSize=236), 29child24_s=PostingsFormat(name=Direct), 36grand14_s=PostingsFormat(name=Asserting), 41grand22_s=Pulsing41(freqCutoff=11 minBlockSize=73 maxBlockSize=236), 56grand22_s=PostingsFormat(name=Direct), 18child14_s=PostingsFormat(name=Asserting), 49grand30_s=PostingsFormat(name=Asserting), 52grand7_s=PostingsFormat(name=Asserting), 62grand55_s=PostingsFormat(name=Asserting), 38grand13_s=PostingsFormat(name=Direct), 65grand20_s=Pulsing41(freqCutoff=11 minBlockSize=73 maxBlockSize=236), 38child23_s=PostingsFormat(name=Asserting), 14child5_s=PostingsFormat(name=Asserting), 28grand20_s=PostingsFormat(name=Asserting), 39grand18_s=PostingsFormat(name=Direct), 26child20_s=Lucene41(blocksize=128), 59grand41_s=PostingsFormat(name=Direct), 31child23_s=PostingsFormat(name=Direct), 72grand55_s=Pulsing41(freqCutoff=11 minBlockSize=73 maxBlockSize=236), 22grand14_s=Pulsing41(freqCutoff=11 minBlockSize=73 maxBlockSize=236), 74grand14_s=Lucene41(blocksize=128), 54grand1_s=PostingsFormat(name=Asserting), 20grand12_s=Pulsing41(freqCutoff=11 minBlockSize=73 maxBlockSize=236), 15child5_s=PostingsFormat(name=Direct), 33grand14_s=PostingsFormat(name=Direct), 33child0_s=PostingsFormat(name=Asserting), 37child30_s=Pulsing41(freqCutoff=11 minBlockSize=73 maxBlockSize=236), 40grand22_s=PostingsFormat(name=Asserting), 31grand29_s=Pulsing41(freqCutoff=11 minBlockSize=73 maxBlockSize=236), 45grand5_s=Lucene41(blocksize=128), 24child19_s=Lucene41(blocksize=128), 58grand7_s=Lucene41(blocksize=128), 55grand36_s=PostingsFormat(name=Direct), 68grand35_s=PostingsFormat(name=Asserting), 9child0_s=Pulsing41(freqCutoff=11 minBlockSize=73 maxBlockSize=236), 52grand51_s=PostingsFormat(name=Direct), 61grand44_s=PostingsFormat(name=Direct), 68grand62_s=Lucene41(blocksize=128), 58grand24_s=Pulsing41(freqCutoff=11 minBlockSize=73 maxBlockSize=236), 58grand35_s=Pulsing41(freqCutoff=11 minBlockSize=73 maxBlockSize=236), 19child17_s=Lucene41(blocksize=128), 70grand64_s=PostingsFormat(name=Direct), 33child8_s=PostingsFormat(name=Asserting), 73grand47_s=PostingsFormat(name=Direct), 36child27_s=Lucene41(blocksize=128), 49grand39_s=Pulsing41(freqCutoff=11 minBlockSize=73 maxBlockSize=236), 28grand14_s=Pulsing41(freqCutoff=11 minBlockSize=73 maxBlockSize=236), 57grand33_s=Lucene41(blocksize=128), 12child7_s=PostingsFormat(name=Asserting), 51grand50_s=PostingsFormat(name=Direct), 32child3_s=Lucene41(blocksize=128), 35child34_s=PostingsFormat(name=Direct), 40grand16_s=PostingsFormat(name=Direct), 31grand19_s=Lucene41(blocksize=128), 51grand0_s=PostingsFormat(name=Asserting), 18child7_s=Lucene41(blocksize=128), 28child20_s=PostingsFormat(name=Asserting), 14child12_s=Lucene41(blocksize=128), 65grand50_s=PostingsFormat(name=Asserting)}, docValues:{timestamp=DocValuesFormat(name=SimpleText)}, sim=RandomSimilarityProvider(queryNorm=true,coord=no): {}, locale=fr_CH, timezone=Europe/Vienna
[junit4] 2> NOTE: Linux 3.2.0-51-generic amd64/Oracle Corporation 1.7.0_25 (64-bit)/cpus=8,threads=1,free=92833808,total=454033408
[junit4] 2> NOTE: All tests run in this JVM: [UUIDFieldTest, ChaosMonkeySafeLeaderTest, SuggesterWFSTTest, TestOmitPositions, TestQuerySenderListener, TestStressReorder, TestDistributedGrouping, CurrencyFieldOpenExchangeTest, TestRandomFaceting, TestNumberUtils, TestSolrCoreProperties, TestComponentsName, BasicZkTest, TestRecovery, TestQueryTypes, FieldAnalysisRequestHandlerTest, DefaultValueUpdateProcessorTest, NoCacheHeaderTest, HighlighterConfigTest, HighlighterTest, OpenExchangeRatesOrgProviderTest, BadIndexSchemaTest, TestBinaryField, StatsComponentTest, XmlUpdateRequestHandlerTest, PluginInfoTest, SimplePostToolTest, TestAddFieldRealTimeGet, TestSchemaVersionResource, ConvertedLegacyTest, SpellCheckComponentTest, BasicDistributedZk2Test, OutputWriterTest, NumericFieldsTest, TestUpdate, TestPartialUpdateDeduplication, CollectionsAPIDistributedZkTest, SortByFunctionTest, HdfsBasicDistributedZk2Test, TestSchemaNameResource, PathHierarchyTokenizerFactoryTest, DirectSolrSpellCheckerTest, RequestHandlersTest, TestSolr4Spatial, XsltUpdateRequestHandlerTest, SpellingQueryConverterTest, TestCopyFieldCollectionResource, TestArbitraryIndexDir, ZkSolrClientTest, TestFiltering, ClusterStateTest, TestStressLucene, TestMultiCoreConfBootstrap, TestCodecSupport, TestCollationField, DocValuesMultiTest, TestCSVLoader, TestAtomicUpdateErrorCases, SuggesterTSTTest, TestQuerySenderNoQuery, LeaderElectionIntegrationTest, ShardRoutingCustomTest, TestCloudManagedSchema, TestRangeQuery, TestLazyCores, NotRequiredUniqueKeyTest, SpellCheckCollatorTest, TestCSVResponseWriter, SignatureUpdateProcessorFactoryTest, TestDefaultSimilarityFactory, AlternateDirectoryTest, DebugComponentTest, TestUtils, RAMDirectoryFactoryTest, TestAnalyzedSuggestions, DirectUpdateHandlerTest, TestFastWriter, SyncSliceTest, OpenCloseCoreStressTest, OverseerTest, FullSolrCloudDistribCmdsTest]
[junit4] Completed on J1 in 127.22s, 1 test, 1 error <<< FAILURES!
[...truncated 471 lines...]
BUILD FAILED
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/build.xml:389: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/build.xml:369: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/build.xml:39: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build.xml:181: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/common-build.xml:450: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/common-build.xml:1230: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/common-build.xml:873: There were test failures: 320 suites, 1360 tests, 1 error, 35 ignored (6 assumptions)
Total time: 44 minutes 3 seconds
Build step 'Invoke Ant' marked build as failure
Description set: Java: 64bit/jdk1.7.0_25 -XX:-UseCompressedOops -XX:+UseParallelGC
Archiving artifacts
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure
[JENKINS] Lucene-Solr-trunk-Linux (32bit/ibm-j9-jdk7) - Build #
6999 - Still Failing!
Posted by Policeman Jenkins Server <je...@thetaphi.de>.
Build: http://jenkins.thetaphi.de/job/Lucene-Solr-trunk-Linux/6999/
Java: 32bit/ibm-j9-jdk7 -Xjit:exclude={org/apache/lucene/util/fst/FST.pack(IIF)Lorg/apache/lucene/util/fst/FST;}
24 tests failed.
FAILED: junit.framework.TestSuite.org.apache.solr.cloud.FullSolrCloudDistribCmdsTest
Error Message:
Suite timeout exceeded (>= 7200000 msec).
Stack Trace:
java.lang.Exception: Suite timeout exceeded (>= 7200000 msec).
at __randomizedtesting.SeedInfo.seed([93CD8FAF60B2976]:0)
FAILED: junit.framework.TestSuite.org.apache.solr.cloud.FullSolrCloudDistribCmdsTest
Error Message:
Captured an uncaught exception in thread: Thread[id=1008, name=HashSessionScavenger-16, state=RUNNABLE, group=TGRP-FullSolrCloudDistribCmdsTest]
Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=1008, name=HashSessionScavenger-16, state=RUNNABLE, group=TGRP-FullSolrCloudDistribCmdsTest]
Caused by: java.lang.OutOfMemoryError: Java heap space
at __randomizedtesting.SeedInfo.seed([93CD8FAF60B2976]:0)
at java.lang.Throwable.fillInStackTrace(Native Method)
at java.lang.Throwable.<init>(Throwable.java:67)
at java.lang.Throwable.<init>(Throwable.java:78)
at java.lang.Error.<init>(Error.java:81)
at java.lang.VirtualMachineError.<init>(VirtualMachineError.java:63)
at java.lang.OutOfMemoryError.<init>(OutOfMemoryError.java:69)
at java.security.AccessController.getProtectionDomains(Native Method)
at java.security.AccessController.checkPermission(AccessController.java:88)
at java.lang.SecurityManager.checkPermission(SecurityManager.java:560)
at java.lang.Thread.setContextClassLoader(Thread.java:797)
at org.eclipse.jetty.server.session.HashSessionManager.scavenge(HashSessionManager.java:308)
at org.eclipse.jetty.server.session.HashSessionManager$2.run(HashSessionManager.java:284)
at java.util.Timer$TimerImpl.run(Timer.java:296)
FAILED: junit.framework.TestSuite.org.apache.solr.cloud.FullSolrCloudDistribCmdsTest
Error Message:
Captured an uncaught exception in thread: Thread[id=994, name=qtp220461705-994, state=RUNNABLE, group=TGRP-FullSolrCloudDistribCmdsTest]
Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=994, name=qtp220461705-994, state=RUNNABLE, group=TGRP-FullSolrCloudDistribCmdsTest]
Caused by: java.lang.OutOfMemoryError: Java heap space
at __randomizedtesting.SeedInfo.seed([93CD8FAF60B2976]:0)
at java.util.HashMap$1.iterator(HashMap.java:672)
at java.util.HashSet.iterator(HashSet.java:173)
at sun.nio.ch.Util$2.iterator(Util.java:315)
at org.eclipse.jetty.io.nio.SelectorManager$SelectSet.doSelect(SelectorManager.java:600)
at org.eclipse.jetty.io.nio.SelectorManager$1.run(SelectorManager.java:290)
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
at java.lang.Thread.run(Thread.java:780)
FAILED: junit.framework.TestSuite.org.apache.solr.cloud.FullSolrCloudDistribCmdsTest
Error Message:
Captured an uncaught exception in thread: Thread[id=966, name=SyncThread:0, state=RUNNABLE, group=TGRP-FullSolrCloudDistribCmdsTest]
Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=966, name=SyncThread:0, state=RUNNABLE, group=TGRP-FullSolrCloudDistribCmdsTest]
Caused by: java.lang.OutOfMemoryError: Java heap space
at org.apache.log4j.spi.LoggingEvent.<init>(LoggingEvent.java:165)
at org.apache.log4j.Category.forcedLog(Category.java:391)
at org.apache.log4j.Category.log(Category.java:856)
at org.slf4j.impl.Log4jLoggerAdapter.error(Log4jLoggerAdapter.java:576)
at org.apache.zookeeper.server.SyncRequestProcessor.run(SyncRequestProcessor.java:151)
FAILED: junit.framework.TestSuite.org.apache.solr.cloud.FullSolrCloudDistribCmdsTest
Error Message:
Captured an uncaught exception in thread: Thread[id=1026, name=qtp-1726714643-1026, state=RUNNABLE, group=TGRP-FullSolrCloudDistribCmdsTest]
Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=1026, name=qtp-1726714643-1026, state=RUNNABLE, group=TGRP-FullSolrCloudDistribCmdsTest]
Caused by: java.lang.OutOfMemoryError: Java heap space
at __randomizedtesting.SeedInfo.seed([93CD8FAF60B2976]:0)
at java.util.HashMap$1.iterator(HashMap.java:672)
at java.util.HashSet.iterator(HashSet.java:173)
at sun.nio.ch.Util$2.iterator(Util.java:315)
at org.eclipse.jetty.io.nio.SelectorManager$SelectSet.doSelect(SelectorManager.java:600)
at org.eclipse.jetty.io.nio.SelectorManager$1.run(SelectorManager.java:290)
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
at java.lang.Thread.run(Thread.java:780)
FAILED: junit.framework.TestSuite.org.apache.solr.cloud.FullSolrCloudDistribCmdsTest
Error Message:
Captured an uncaught exception in thread: Thread[id=992, name=HashSessionScavenger-15, state=RUNNABLE, group=TGRP-FullSolrCloudDistribCmdsTest]
Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=992, name=HashSessionScavenger-15, state=RUNNABLE, group=TGRP-FullSolrCloudDistribCmdsTest]
Caused by: java.lang.OutOfMemoryError: Java heap space
at __randomizedtesting.SeedInfo.seed([93CD8FAF60B2976]:0)
at java.lang.Throwable.fillInStackTrace(Native Method)
at java.lang.Throwable.<init>(Throwable.java:67)
at java.lang.Throwable.<init>(Throwable.java:78)
at java.lang.Error.<init>(Error.java:81)
at java.lang.VirtualMachineError.<init>(VirtualMachineError.java:63)
at java.lang.OutOfMemoryError.<init>(OutOfMemoryError.java:69)
at java.security.AccessController.toArrayOfProtectionDomains(AccessController.java:184)
at java.security.AccessController.checkPermission(AccessController.java:102)
at java.lang.SecurityManager.checkPermission(SecurityManager.java:560)
at java.lang.Thread.setContextClassLoader(Thread.java:797)
at org.eclipse.jetty.server.session.HashSessionManager.scavenge(HashSessionManager.java:308)
at org.eclipse.jetty.server.session.HashSessionManager$2.run(HashSessionManager.java:284)
at java.util.Timer$TimerImpl.run(Timer.java:296)
FAILED: junit.framework.TestSuite.org.apache.solr.cloud.FullSolrCloudDistribCmdsTest
Error Message:
Captured an uncaught exception in thread: Thread[id=993, name=qtp220461705-993, state=RUNNABLE, group=TGRP-FullSolrCloudDistribCmdsTest]
Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=993, name=qtp220461705-993, state=RUNNABLE, group=TGRP-FullSolrCloudDistribCmdsTest]
Caused by: java.lang.OutOfMemoryError: Java heap space
at java.lang.Throwable.fillInStackTrace(Native Method)
at java.lang.Throwable.<init>(Throwable.java:67)
at java.lang.Throwable.<init>(Throwable.java:78)
at java.lang.Error.<init>(Error.java:81)
at java.lang.VirtualMachineError.<init>(VirtualMachineError.java:63)
at java.lang.OutOfMemoryError.<init>(OutOfMemoryError.java:69)
at org.eclipse.jetty.io.nio.SelectorManager$SelectSet.doSelect(SelectorManager.java:708)
at org.eclipse.jetty.io.nio.SelectorManager$1.run(SelectorManager.java:290)
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
at java.lang.Thread.run(Thread.java:780)
FAILED: junit.framework.TestSuite.org.apache.solr.cloud.FullSolrCloudDistribCmdsTest
Error Message:
Captured an uncaught exception in thread: Thread[id=1029, name=qtp-1726714643-1029, state=RUNNABLE, group=TGRP-FullSolrCloudDistribCmdsTest]
Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=1029, name=qtp-1726714643-1029, state=RUNNABLE, group=TGRP-FullSolrCloudDistribCmdsTest]
Caused by: java.lang.OutOfMemoryError: Java heap space
at __randomizedtesting.SeedInfo.seed([93CD8FAF60B2976]:0)
at java.util.concurrent.ConcurrentHashMap$KeySet.iterator(ConcurrentHashMap.java:1273)
at org.eclipse.jetty.io.nio.SelectorManager$SelectSet$1.run(SelectorManager.java:712)
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
at java.lang.Thread.run(Thread.java:780)
FAILED: junit.framework.TestSuite.org.apache.solr.cloud.FullSolrCloudDistribCmdsTest
Error Message:
Captured an uncaught exception in thread: Thread[id=1042, name=qtp1556593299-1042, state=RUNNABLE, group=TGRP-FullSolrCloudDistribCmdsTest]
Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=1042, name=qtp1556593299-1042, state=RUNNABLE, group=TGRP-FullSolrCloudDistribCmdsTest]
Caused by: java.lang.OutOfMemoryError: Java heap space
at __randomizedtesting.SeedInfo.seed([93CD8FAF60B2976]:0)
at java.util.concurrent.locks.AbstractQueuedSynchronizer.addWaiter(AbstractQueuedSynchronizer.java:617)
at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquire(AbstractQueuedSynchronizer.java:1208)
at java.util.concurrent.locks.ReentrantLock$NonfairSync.lock(ReentrantLock.java:225)
at java.util.concurrent.locks.ReentrantLock.lock(ReentrantLock.java:301)
at org.eclipse.jetty.util.BlockingArrayQueue.offer(BlockingArrayQueue.java:224)
at org.eclipse.jetty.util.thread.QueuedThreadPool.dispatch(QueuedThreadPool.java:359)
at org.eclipse.jetty.server.nio.SelectChannelConnector$ConnectorSelectorManager.dispatch(SelectChannelConnector.java:300)
at org.eclipse.jetty.io.nio.SelectorManager$SelectSet.doSelect(SelectorManager.java:708)
at org.eclipse.jetty.io.nio.SelectorManager$1.run(SelectorManager.java:290)
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
at java.lang.Thread.run(Thread.java:780)
FAILED: junit.framework.TestSuite.org.apache.solr.cloud.FullSolrCloudDistribCmdsTest
Error Message:
Captured an uncaught exception in thread: Thread[id=1048, name=qtp1556593299-1048, state=RUNNABLE, group=TGRP-FullSolrCloudDistribCmdsTest]
Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=1048, name=qtp1556593299-1048, state=RUNNABLE, group=TGRP-FullSolrCloudDistribCmdsTest]
Caused by: java.lang.OutOfMemoryError: Java heap space
at __randomizedtesting.SeedInfo.seed([93CD8FAF60B2976]:0)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.addConditionWaiter(AbstractQueuedSynchronizer.java:1868)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2084)
at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:342)
at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:526)
at org.eclipse.jetty.util.thread.QueuedThreadPool.access$600(QueuedThreadPool.java:44)
at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:572)
at java.lang.Thread.run(Thread.java:780)
FAILED: junit.framework.TestSuite.org.apache.solr.cloud.FullSolrCloudDistribCmdsTest
Error Message:
Captured an uncaught exception in thread: Thread[id=1210, name=qtp220461705-1210, state=RUNNABLE, group=TGRP-FullSolrCloudDistribCmdsTest]
Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=1210, name=qtp220461705-1210, state=RUNNABLE, group=TGRP-FullSolrCloudDistribCmdsTest]
Caused by: java.lang.OutOfMemoryError: Java heap space
at __randomizedtesting.SeedInfo.seed([93CD8FAF60B2976]:0)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.addConditionWaiter(AbstractQueuedSynchronizer.java:1868)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2084)
at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:342)
at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:526)
at org.eclipse.jetty.util.thread.QueuedThreadPool.access$600(QueuedThreadPool.java:44)
at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:572)
at java.lang.Thread.run(Thread.java:780)
FAILED: junit.framework.TestSuite.org.apache.solr.cloud.FullSolrCloudDistribCmdsTest
Error Message:
Captured an uncaught exception in thread: Thread[id=1109, name=qtp1556593299-1109, state=RUNNABLE, group=TGRP-FullSolrCloudDistribCmdsTest]
Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=1109, name=qtp1556593299-1109, state=RUNNABLE, group=TGRP-FullSolrCloudDistribCmdsTest]
Caused by: java.lang.OutOfMemoryError: Java heap space
at __randomizedtesting.SeedInfo.seed([93CD8FAF60B2976]:0)
at java.util.concurrent.ConcurrentHashMap$KeySet.iterator(ConcurrentHashMap.java:1273)
at org.eclipse.jetty.io.nio.SelectorManager$SelectSet$1.run(SelectorManager.java:712)
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
at java.lang.Thread.run(Thread.java:780)
FAILED: junit.framework.TestSuite.org.apache.solr.cloud.FullSolrCloudDistribCmdsTest
Error Message:
Captured an uncaught exception in thread: Thread[id=1106, name=qtp220461705-1106, state=RUNNABLE, group=TGRP-FullSolrCloudDistribCmdsTest]
Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=1106, name=qtp220461705-1106, state=RUNNABLE, group=TGRP-FullSolrCloudDistribCmdsTest]
Caused by: java.lang.OutOfMemoryError: Java heap space
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.addConditionWaiter(AbstractQueuedSynchronizer.java:1868)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2084)
at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:342)
at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:526)
at org.eclipse.jetty.util.thread.QueuedThreadPool.access$600(QueuedThreadPool.java:44)
at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:572)
at java.lang.Thread.run(Thread.java:780)
FAILED: junit.framework.TestSuite.org.apache.solr.cloud.FullSolrCloudDistribCmdsTest
Error Message:
Captured an uncaught exception in thread: Thread[id=1043, name=qtp1556593299-1043, state=RUNNABLE, group=TGRP-FullSolrCloudDistribCmdsTest]
Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=1043, name=qtp1556593299-1043, state=RUNNABLE, group=TGRP-FullSolrCloudDistribCmdsTest]
Caused by: java.lang.OutOfMemoryError: Java heap space
at java.lang.Throwable.fillInStackTrace(Native Method)
at java.lang.Throwable.<init>(Throwable.java:67)
at java.lang.Throwable.<init>(Throwable.java:78)
at java.lang.Error.<init>(Error.java:81)
at java.lang.VirtualMachineError.<init>(VirtualMachineError.java:63)
at java.lang.OutOfMemoryError.<init>(OutOfMemoryError.java:69)
at java.util.HashMap$1.iterator(HashMap.java:672)
at java.util.HashSet.iterator(HashSet.java:173)
at sun.nio.ch.Util$2.iterator(Util.java:315)
at org.eclipse.jetty.io.nio.SelectorManager$SelectSet.doSelect(SelectorManager.java:600)
at org.eclipse.jetty.io.nio.SelectorManager$1.run(SelectorManager.java:290)
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
at java.lang.Thread.run(Thread.java:780)
FAILED: junit.framework.TestSuite.org.apache.solr.cloud.FullSolrCloudDistribCmdsTest
Error Message:
Captured an uncaught exception in thread: Thread[id=1024, name=HashSessionScavenger-17, state=RUNNABLE, group=TGRP-FullSolrCloudDistribCmdsTest]
Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=1024, name=HashSessionScavenger-17, state=RUNNABLE, group=TGRP-FullSolrCloudDistribCmdsTest]
Caused by: java.lang.OutOfMemoryError: Java heap space
at __randomizedtesting.SeedInfo.seed([93CD8FAF60B2976]:0)
at java.security.AccessController.toArrayOfProtectionDomains(AccessController.java:184)
at java.security.AccessController.checkPermission(AccessController.java:102)
at java.lang.SecurityManager.checkPermission(SecurityManager.java:560)
at java.lang.Thread.setContextClassLoader(Thread.java:797)
at org.eclipse.jetty.server.session.HashSessionManager.scavenge(HashSessionManager.java:344)
at org.eclipse.jetty.server.session.HashSessionManager$2.run(HashSessionManager.java:284)
at java.util.Timer$TimerImpl.run(Timer.java:296)
FAILED: junit.framework.TestSuite.org.apache.solr.cloud.FullSolrCloudDistribCmdsTest
Error Message:
Captured an uncaught exception in thread: Thread[id=964, name=NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0, state=RUNNABLE, group=TGRP-FullSolrCloudDistribCmdsTest]
Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=964, name=NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0, state=RUNNABLE, group=TGRP-FullSolrCloudDistribCmdsTest]
Caused by: java.lang.OutOfMemoryError: Java heap space
at __randomizedtesting.SeedInfo.seed([93CD8FAF60B2976]:0)
at java.util.ArrayList$SubList.listIterator(ArrayList.java:1024)
at java.util.AbstractList.listIterator(AbstractList.java:629)
at java.util.ArrayList$SubList.iterator(ArrayList.java:1016)
at java.util.AbstractCollection.toArray(AbstractCollection.java:363)
at java.util.regex.Pattern.split(Pattern.java:1238)
at java.util.regex.Pattern.split(Pattern.java:1270)
at java.lang.String.split(String.java:1749)
at org.apache.zookeeper.jmx.MBeanRegistry.tokenize(MBeanRegistry.java:182)
at org.apache.zookeeper.jmx.MBeanRegistry.makeObjectName(MBeanRegistry.java:204)
at org.apache.zookeeper.jmx.MBeanRegistry.unregister(MBeanRegistry.java:115)
at org.apache.zookeeper.jmx.MBeanRegistry.unregister(MBeanRegistry.java:132)
at org.apache.zookeeper.server.ServerCnxnFactory.unregisterConnection(ServerCnxnFactory.java:137)
at org.apache.zookeeper.server.NIOServerCnxn.close(NIOServerCnxn.java:972)
at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:352)
at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208)
at java.lang.Thread.run(Thread.java:780)
FAILED: junit.framework.TestSuite.org.apache.solr.cloud.FullSolrCloudDistribCmdsTest
Error Message:
Captured an uncaught exception in thread: Thread[id=1000, name=qtp220461705-1000, state=RUNNABLE, group=TGRP-FullSolrCloudDistribCmdsTest]
Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=1000, name=qtp220461705-1000, state=RUNNABLE, group=TGRP-FullSolrCloudDistribCmdsTest]
Caused by: java.lang.OutOfMemoryError: Java heap space
at __randomizedtesting.SeedInfo.seed([93CD8FAF60B2976]:0)
at java.util.concurrent.ConcurrentHashMap$KeySet.iterator(ConcurrentHashMap.java:1273)
at org.eclipse.jetty.io.nio.SelectorManager$SelectSet$1.run(SelectorManager.java:712)
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
at java.lang.Thread.run(Thread.java:780)
FAILED: junit.framework.TestSuite.org.apache.solr.cloud.FullSolrCloudDistribCmdsTest
Error Message:
Captured an uncaught exception in thread: Thread[id=1096, name=qtp1639013060-1096, state=RUNNABLE, group=TGRP-FullSolrCloudDistribCmdsTest]
Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=1096, name=qtp1639013060-1096, state=RUNNABLE, group=TGRP-FullSolrCloudDistribCmdsTest]
Caused by: java.lang.OutOfMemoryError: Java heap space
at __randomizedtesting.SeedInfo.seed([93CD8FAF60B2976]:0)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.addConditionWaiter(AbstractQueuedSynchronizer.java:1868)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2084)
at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:342)
at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:526)
at org.eclipse.jetty.util.thread.QueuedThreadPool.access$600(QueuedThreadPool.java:44)
at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:572)
at java.lang.Thread.run(Thread.java:780)
FAILED: junit.framework.TestSuite.org.apache.solr.cloud.FullSolrCloudDistribCmdsTest
Error Message:
Captured an uncaught exception in thread: Thread[id=1009, name=qtp1639013060-1009 Selector0, state=RUNNABLE, group=TGRP-FullSolrCloudDistribCmdsTest]
Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=1009, name=qtp1639013060-1009 Selector0, state=RUNNABLE, group=TGRP-FullSolrCloudDistribCmdsTest]
Caused by: java.lang.OutOfMemoryError: Java heap space
at __randomizedtesting.SeedInfo.seed([93CD8FAF60B2976]:0)
at org.eclipse.jetty.io.nio.SelectorManager$1.run(SelectorManager.java:304)
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
at java.lang.Thread.run(Thread.java:780)
FAILED: junit.framework.TestSuite.org.apache.solr.cloud.FullSolrCloudDistribCmdsTest
Error Message:
Captured an uncaught exception in thread: Thread[id=1025, name=qtp-1726714643-1025, state=RUNNABLE, group=TGRP-FullSolrCloudDistribCmdsTest]
Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=1025, name=qtp-1726714643-1025, state=RUNNABLE, group=TGRP-FullSolrCloudDistribCmdsTest]
Caused by: java.lang.OutOfMemoryError: Java heap space
at __randomizedtesting.SeedInfo.seed([93CD8FAF60B2976]:0)
at java.util.HashMap$1.iterator(HashMap.java:672)
at java.util.HashSet.iterator(HashSet.java:173)
at sun.nio.ch.Util$2.iterator(Util.java:315)
at org.eclipse.jetty.io.nio.SelectorManager$SelectSet.doSelect(SelectorManager.java:600)
at org.eclipse.jetty.io.nio.SelectorManager$1.run(SelectorManager.java:290)
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
at java.lang.Thread.run(Thread.java:780)
FAILED: junit.framework.TestSuite.org.apache.solr.cloud.FullSolrCloudDistribCmdsTest
Error Message:
Captured an uncaught exception in thread: Thread[id=1209, name=qtp1556593299-1209, state=RUNNABLE, group=TGRP-FullSolrCloudDistribCmdsTest]
Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=1209, name=qtp1556593299-1209, state=RUNNABLE, group=TGRP-FullSolrCloudDistribCmdsTest]
Caused by: java.lang.OutOfMemoryError: Java heap space
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.addConditionWaiter(AbstractQueuedSynchronizer.java:1868)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2084)
at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:342)
at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:526)
at org.eclipse.jetty.util.thread.QueuedThreadPool.access$600(QueuedThreadPool.java:44)
at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:572)
at java.lang.Thread.run(Thread.java:780)
FAILED: junit.framework.TestSuite.org.apache.solr.cloud.FullSolrCloudDistribCmdsTest
Error Message:
Captured an uncaught exception in thread: Thread[id=1041, name=HashSessionScavenger-18, state=RUNNABLE, group=TGRP-FullSolrCloudDistribCmdsTest]
Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=1041, name=HashSessionScavenger-18, state=RUNNABLE, group=TGRP-FullSolrCloudDistribCmdsTest]
Caused by: java.lang.OutOfMemoryError: Java heap space
at __randomizedtesting.SeedInfo.seed([93CD8FAF60B2976]:0)
at java.security.AccessController.getProtectionDomains(Native Method)
at java.security.AccessController.checkPermission(AccessController.java:88)
at java.lang.SecurityManager.checkPermission(SecurityManager.java:560)
at java.lang.Thread.setContextClassLoader(Thread.java:797)
at org.eclipse.jetty.server.session.HashSessionManager.scavenge(HashSessionManager.java:344)
at org.eclipse.jetty.server.session.HashSessionManager$2.run(HashSessionManager.java:284)
at java.util.Timer$TimerImpl.run(Timer.java:296)
FAILED: org.apache.solr.cloud.BasicDistributedZk2Test.testDistribSearch
Error Message:
Server at http://127.0.0.1:38771/n_b/m/onenodecollectioncore returned non ok status:404, message:Can not find: /n_b/m/onenodecollectioncore/update
Stack Trace:
org.apache.solr.client.solrj.impl.HttpSolrServer$RemoteSolrException: Server at http://127.0.0.1:38771/n_b/m/onenodecollectioncore returned non ok status:404, message:Can not find: /n_b/m/onenodecollectioncore/update
at __randomizedtesting.SeedInfo.seed([93CD8FAF60B2976:88DA56E28154494A]:0)
at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:385)
at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:180)
at org.apache.solr.client.solrj.request.AbstractUpdateRequest.process(AbstractUpdateRequest.java:117)
at org.apache.solr.client.solrj.SolrServer.add(SolrServer.java:116)
at org.apache.solr.client.solrj.SolrServer.add(SolrServer.java:102)
at org.apache.solr.cloud.BasicDistributedZk2Test.testNodeWithoutCollectionForwarding(BasicDistributedZk2Test.java:196)
at org.apache.solr.cloud.BasicDistributedZk2Test.doTest(BasicDistributedZk2Test.java:88)
at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:835)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:88)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:55)
at java.lang.reflect.Method.invoke(Method.java:613)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1559)
at com.carrotsearch.randomizedtesting.RandomizedRunner.access$600(RandomizedRunner.java:79)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:773)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:787)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:782)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:442)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:746)
at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:648)
at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:682)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:693)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
at java.lang.Thread.run(Thread.java:780)
FAILED: org.apache.solr.cloud.FullSolrCloudDistribCmdsTest.testDistribSearch
Error Message:
Test abandoned because suite timeout was reached.
Stack Trace:
java.lang.Exception: Test abandoned because suite timeout was reached.
at __randomizedtesting.SeedInfo.seed([93CD8FAF60B2976]:0)
Build Log:
[...truncated 9663 lines...]
[junit4] Suite: org.apache.solr.cloud.BasicDistributedZk2Test
[junit4] 2> 1279686 T3618 oas.BaseDistributedSearchTestCase.initHostContext Setting hostContext system property: /n_b/m
[junit4] 2> 1279690 T3618 oas.SolrTestCaseJ4.setUp ###Starting testDistribSearch
[junit4] 2> Creating dataDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./solrtest-BasicDistributedZk2Test-1376429262138
[junit4] 2> 1279691 T3618 oasc.ZkTestServer.run STARTING ZK TEST SERVER
[junit4] 2> 1279693 T3619 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
[junit4] 2> 1279793 T3618 oasc.ZkTestServer.run start zk server on port:56032
[junit4] 2> 1279795 T3618 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1280075 T3625 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@3afff252 name:ZooKeeperConnection Watcher:127.0.0.1:56032 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1280075 T3618 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1280076 T3618 oascc.SolrZkClient.makePath makePath: /solr
[junit4] 2> 1280083 T3618 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1280084 T3627 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@ef341f0e name:ZooKeeperConnection Watcher:127.0.0.1:56032/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1280085 T3618 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1280085 T3618 oascc.SolrZkClient.makePath makePath: /collections/collection1
[junit4] 2> 1280088 T3618 oascc.SolrZkClient.makePath makePath: /collections/collection1/shards
[junit4] 2> 1280090 T3618 oascc.SolrZkClient.makePath makePath: /collections/control_collection
[junit4] 2> 1280092 T3618 oascc.SolrZkClient.makePath makePath: /collections/control_collection/shards
[junit4] 2> 1280094 T3618 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
[junit4] 2> 1280095 T3618 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.xml
[junit4] 2> 1280098 T3618 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
[junit4] 2> 1280099 T3618 oascc.SolrZkClient.makePath makePath: /configs/conf1/schema.xml
[junit4] 2> 1280101 T3618 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
[junit4] 2> 1280102 T3618 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.snippet.randomindexconfig.xml
[junit4] 2> 1280104 T3618 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
[junit4] 2> 1280105 T3618 oascc.SolrZkClient.makePath makePath: /configs/conf1/stopwords.txt
[junit4] 2> 1280107 T3618 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
[junit4] 2> 1280108 T3618 oascc.SolrZkClient.makePath makePath: /configs/conf1/protwords.txt
[junit4] 2> 1280110 T3618 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
[junit4] 2> 1280110 T3618 oascc.SolrZkClient.makePath makePath: /configs/conf1/currency.xml
[junit4] 2> 1280112 T3618 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
[junit4] 2> 1280113 T3618 oascc.SolrZkClient.makePath makePath: /configs/conf1/open-exchange-rates.json
[junit4] 2> 1280115 T3618 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4] 2> 1280115 T3618 oascc.SolrZkClient.makePath makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4] 2> 1280118 T3618 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
[junit4] 2> 1280118 T3618 oascc.SolrZkClient.makePath makePath: /configs/conf1/old_synonyms.txt
[junit4] 2> 1280120 T3618 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
[junit4] 2> 1280121 T3618 oascc.SolrZkClient.makePath makePath: /configs/conf1/synonyms.txt
[junit4] 2> 1280251 T3618 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4] 2> 1280293 T3618 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:56324
[junit4] 2> 1280294 T3618 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4] 2> 1280297 T3618 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4] 2> 1280298 T3618 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-controljetty-1376429262571
[junit4] 2> 1280299 T3618 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-controljetty-1376429262571/'
[junit4] 2> 1280317 T3618 oasc.ConfigSolr.fromFile Loading container configuration from /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-controljetty-1376429262571/solr.xml
[junit4] 2> 1280360 T3618 oasc.CoreContainer.<init> New CoreContainer 2074598383
[junit4] 2> 1280361 T3618 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-controljetty-1376429262571/]
[junit4] 2> 1280363 T3618 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4] 2> 1280364 T3618 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4] 2> 1280365 T3618 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4] 2> 1280366 T3618 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4] 2> 1280367 T3618 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4] 2> 1280368 T3618 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4] 2> 1280368 T3618 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4] 2> 1280369 T3618 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4] 2> 1280370 T3618 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4] 2> 1280371 T3618 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4] 2> 1280382 T3618 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:56032/solr
[junit4] 2> 1280383 T3618 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4] 2> 1280385 T3618 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1280387 T3638 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@72ebed10 name:ZooKeeperConnection Watcher:127.0.0.1:56032 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1280387 T3618 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1280389 T3618 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4] 2> 1280397 T3618 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1280399 T3640 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@76ca6e5b name:ZooKeeperConnection Watcher:127.0.0.1:56032/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1280399 T3618 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1280401 T3618 oascc.SolrZkClient.makePath makePath: /overseer/queue
[junit4] 2> 1280404 T3618 oascc.SolrZkClient.makePath makePath: /overseer/collection-queue-work
[junit4] 2> 1280408 T3618 oascc.SolrZkClient.makePath makePath: /live_nodes
[junit4] 2> 1280410 T3618 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:56324_n_b%2Fm
[junit4] 2> 1280411 T3618 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:56324_n_b%2Fm
[junit4] 2> 1280414 T3618 oascc.SolrZkClient.makePath makePath: /overseer_elect/election
[junit4] 2> 1280418 T3618 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
[junit4] 2> 1280419 T3618 oasc.Overseer.start Overseer (id=90205668124721155-127.0.0.1:56324_n_b%2Fm-n_0000000000) starting
[junit4] 2> 1280422 T3618 oascc.SolrZkClient.makePath makePath: /overseer/queue-work
[junit4] 2> 1280425 T3642 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
[junit4] 2> 1280426 T3618 oascc.SolrZkClient.makePath makePath: /clusterstate.json
[junit4] 2> 1280429 T3618 oascc.SolrZkClient.makePath makePath: /aliases.json
[junit4] 2> 1280431 T3618 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 1280434 T3641 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
[junit4] 2> 1280437 T3643 oasc.ZkController.publish publishing core=collection1 state=down
[junit4] 2> 1280437 T3643 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 1280438 T3643 oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4] 2> 1281944 T3641 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 1281945 T3641 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4] 2> "shard_state":"active",
[junit4] 2> "base_url":"http://127.0.0.1:56324/n_b/m",
[junit4] 2> "state":"down",
[junit4] 2> "operation":"state",
[junit4] 2> "core_node_name":"core_node1",
[junit4] 2> "node_name":"127.0.0.1:56324_n_b%2Fm",
[junit4] 2> "numShards":"1",
[junit4] 2> "shard_range":null,
[junit4] 2> "roles":null,
[junit4] 2> "collection":"control_collection",
[junit4] 2> "shard":null,
[junit4] 2> "core":"collection1"}
[junit4] 2> 1281946 T3641 oasc.Overseer$ClusterStateUpdater.createCollection Create collection control_collection with shards [shard1]
[junit4] 2> 1281946 T3641 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4] 2> 1281952 T3640 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 1282442 T3643 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
[junit4] 2> 1282442 T3643 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-controljetty-1376429262571/collection1
[junit4] 2> 1282443 T3643 oasc.ZkController.createCollectionZkNode Check for collection zkNode:control_collection
[junit4] 2> 1282444 T3643 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4] 2> 1282444 T3643 oasc.ZkController.readConfigName Load collection config from:/collections/control_collection
[junit4] 2> 1282445 T3643 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-controljetty-1376429262571/collection1/'
[junit4] 2> 1282447 T3643 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZk2Test-controljetty-1376429262571/collection1/lib/README' to classloader
[junit4] 2> 1282447 T3643 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZk2Test-controljetty-1376429262571/collection1/lib/classes/' to classloader
[junit4] 2> 1282505 T3643 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4] 2> 1282556 T3643 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4] 2> 1282558 T3643 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4] 2> 1282571 T3643 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4] 2> 1283040 T3643 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4] 2> 1283046 T3643 oass.IndexSchema.readSchema default search field in schema is text
[junit4] 2> 1283050 T3643 oass.IndexSchema.readSchema unique key field: id
[junit4] 2> 1283103 T3643 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 1283108 T3643 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 1283114 T3643 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4] 2> 1283118 T3643 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4] 2> 1283119 T3643 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4] 2> 1283119 T3643 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4] 2> 1283129 T3643 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4] 2> 1283129 T3643 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4] 2> 1283130 T3643 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4] 2> 1283131 T3643 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-controljetty-1376429262571/collection1/, dataDir=./org.apache.solr.cloud.BasicDistributedZk2Test-1376429262138/control/data/
[junit4] 2> 1283131 T3643 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@2f99c27e
[junit4] 2> 1283132 T3643 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.BasicDistributedZk2Test-1376429262138/control/data
[junit4] 2> 1283133 T3643 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.BasicDistributedZk2Test-1376429262138/control/data/index/
[junit4] 2> 1283134 T3643 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.BasicDistributedZk2Test-1376429262138/control/data/index' doesn't exist. Creating new index...
[junit4] 2> 1283137 T3643 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.BasicDistributedZk2Test-1376429262138/control/data/index
[junit4] 2> 1283139 T3643 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@bad203ce lockFactory=org.apache.lucene.store.NativeFSLockFactory@e554df6d),segFN=segments_1,generation=1}
[junit4] 2> 1283140 T3643 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 1283144 T3643 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4] 2> 1283145 T3643 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4] 2> 1283145 T3643 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 1283147 T3643 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4] 2> 1283147 T3643 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4] 2> 1283149 T3643 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4] 2> 1283149 T3643 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4] 2> 1283166 T3643 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4] 2> 1283167 T3643 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4] 2> 1283170 T3643 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 1283186 T3643 oass.SolrIndexSearcher.<init> Opening Searcher@5bf3ca55 main
[junit4] 2> 1283189 T3643 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4] 2> 1283190 T3643 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4] 2> 1283193 T3644 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@5bf3ca55 main{StandardDirectoryReader(segments_1:1)}
[junit4] 2> 1283194 T3643 oasc.CoreContainer.registerCore registering core: collection1
[junit4] 2> 1283195 T3643 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:56324/n_b/m collection:control_collection shard:shard1
[junit4] 2> 1283195 T3643 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=10000&maxConnectionsPerHost=20&connTimeout=15000&socketTimeout=30000&retry=false
[junit4] 2> 1283204 T3643 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leader_elect/shard1/election
[junit4] 2> 1283209 T3643 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard1
[junit4] 2> 1283212 T3643 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4] 2> 1283212 T3643 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4] 2> 1283212 T3643 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:56324/n_b/m/collection1/
[junit4] 2> 1283213 T3643 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4] 2> 1283213 T3643 oasc.SyncStrategy.syncToMe http://127.0.0.1:56324/n_b/m/collection1/ has no replicas
[junit4] 2> 1283214 T3643 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:56324/n_b/m/collection1/ shard1
[junit4] 2> 1283214 T3643 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leaders/shard1
[junit4] 2> 1283456 T3641 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 1283465 T3640 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 1283520 T3643 oasc.ZkController.register We are http://127.0.0.1:56324/n_b/m/collection1/ and leader is http://127.0.0.1:56324/n_b/m/collection1/
[junit4] 2> 1283520 T3643 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:56324/n_b/m
[junit4] 2> 1283521 T3643 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4] 2> 1283521 T3643 oasc.ZkController.publish publishing core=collection1 state=active
[junit4] 2> 1283522 T3643 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 1283523 T3643 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 1283524 T3618 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0
[junit4] 2> 1283525 T3618 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4] 2> 1283526 T3618 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4] 2> 1283533 T3618 oascsi.HttpClientUtil.createClient Creating new http client, config:
[junit4] 2> 1283537 T3618 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1283538 T3647 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@55c058e0 name:ZooKeeperConnection Watcher:127.0.0.1:56032/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1283538 T3618 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1283540 T3618 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 1283542 T3618 oasc.ChaosMonkey.monkeyLog monkey: init - expire sessions:true cause connection loss:true
[junit4] 2> 1283648 T3618 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4] 2> 1283665 T3618 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:59579
[junit4] 2> 1283666 T3618 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4] 2> 1283667 T3618 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4] 2> 1283668 T3618 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty1-1376429265990
[junit4] 2> 1283668 T3618 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty1-1376429265990/'
[junit4] 2> 1283677 T3618 oasc.ConfigSolr.fromFile Loading container configuration from /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty1-1376429265990/solr.xml
[junit4] 2> 1283746 T3618 oasc.CoreContainer.<init> New CoreContainer 712316351
[junit4] 2> 1283747 T3618 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty1-1376429265990/]
[junit4] 2> 1283749 T3618 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4] 2> 1283750 T3618 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4] 2> 1283751 T3618 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4] 2> 1283752 T3618 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4] 2> 1283752 T3618 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4] 2> 1283753 T3618 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4] 2> 1283754 T3618 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4] 2> 1283755 T3618 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4] 2> 1283756 T3618 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4] 2> 1283757 T3618 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4] 2> 1283765 T3618 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:56032/solr
[junit4] 2> 1283766 T3618 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4] 2> 1283769 T3618 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1283770 T3658 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@7e737e00 name:ZooKeeperConnection Watcher:127.0.0.1:56032 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1283771 T3618 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1283782 T3618 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4] 2> 1283792 T3618 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1283816 T3660 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@ae959547 name:ZooKeeperConnection Watcher:127.0.0.1:56032/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1283817 T3618 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1283821 T3618 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 1284824 T3618 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:59579_n_b%2Fm
[junit4] 2> 1284827 T3618 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:59579_n_b%2Fm
[junit4] 2> 1284830 T3640 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
[junit4] 2> 1284830 T3647 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4] 2> 1284831 T3660 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4] 2> 1284833 T3640 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4] 2> 1284842 T3661 oasc.ZkController.publish publishing core=collection1 state=down
[junit4] 2> 1284842 T3661 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 1284846 T3661 oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4] 2> 1284988 T3641 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 1284989 T3641 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4] 2> "shard_state":"active",
[junit4] 2> "base_url":"http://127.0.0.1:56324/n_b/m",
[junit4] 2> "state":"active",
[junit4] 2> "operation":"state",
[junit4] 2> "core_node_name":"core_node1",
[junit4] 2> "node_name":"127.0.0.1:56324_n_b%2Fm",
[junit4] 2> "numShards":"1",
[junit4] 2> "shard_range":null,
[junit4] 2> "roles":null,
[junit4] 2> "collection":"control_collection",
[junit4] 2> "shard":"shard1",
[junit4] 2> "core":"collection1"}
[junit4] 2> 1284995 T3641 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4] 2> "shard_state":"active",
[junit4] 2> "base_url":"http://127.0.0.1:59579/n_b/m",
[junit4] 2> "state":"down",
[junit4] 2> "operation":"state",
[junit4] 2> "core_node_name":"core_node1",
[junit4] 2> "node_name":"127.0.0.1:59579_n_b%2Fm",
[junit4] 2> "numShards":"2",
[junit4] 2> "shard_range":null,
[junit4] 2> "roles":null,
[junit4] 2> "collection":"collection1",
[junit4] 2> "shard":null,
[junit4] 2> "core":"collection1"}
[junit4] 2> 1284996 T3641 oasc.Overseer$ClusterStateUpdater.createCollection Create collection collection1 with shards [shard1, shard2]
[junit4] 2> 1284996 T3641 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4] 2> 1285000 T3647 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 1285000 T3660 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 1285000 T3640 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 1285846 T3661 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
[junit4] 2> 1285847 T3661 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty1-1376429265990/collection1
[junit4] 2> 1285848 T3661 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4] 2> 1285849 T3661 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4] 2> 1285849 T3661 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4] 2> 1285850 T3661 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty1-1376429265990/collection1/'
[junit4] 2> 1285852 T3661 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZk2Test-jetty1-1376429265990/collection1/lib/README' to classloader
[junit4] 2> 1285852 T3661 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZk2Test-jetty1-1376429265990/collection1/lib/classes/' to classloader
[junit4] 2> 1285893 T3661 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4] 2> 1285941 T3661 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4] 2> 1285945 T3661 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4] 2> 1285954 T3661 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4] 2> 1286433 T3661 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4] 2> 1286439 T3661 oass.IndexSchema.readSchema default search field in schema is text
[junit4] 2> 1286442 T3661 oass.IndexSchema.readSchema unique key field: id
[junit4] 2> 1286453 T3661 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 1286457 T3661 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 1286461 T3661 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4] 2> 1286463 T3661 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4] 2> 1286463 T3661 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4] 2> 1286463 T3661 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4] 2> 1286464 T3661 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4] 2> 1286465 T3661 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4] 2> 1286465 T3661 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4] 2> 1286465 T3661 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty1-1376429265990/collection1/, dataDir=./org.apache.solr.cloud.BasicDistributedZk2Test-1376429262138/jetty1/
[junit4] 2> 1286466 T3661 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@2f99c27e
[junit4] 2> 1286467 T3661 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.BasicDistributedZk2Test-1376429262138/jetty1
[junit4] 2> 1286467 T3661 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.BasicDistributedZk2Test-1376429262138/jetty1/index/
[junit4] 2> 1286467 T3661 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.BasicDistributedZk2Test-1376429262138/jetty1/index' doesn't exist. Creating new index...
[junit4] 2> 1286468 T3661 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.BasicDistributedZk2Test-1376429262138/jetty1/index
[junit4] 2> 1286469 T3661 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@ebdce726 lockFactory=org.apache.lucene.store.NativeFSLockFactory@56c9cf76),segFN=segments_1,generation=1}
[junit4] 2> 1286470 T3661 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 1286472 T3661 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4] 2> 1286472 T3661 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4] 2> 1286475 T3661 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 1286476 T3661 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4] 2> 1286477 T3661 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4] 2> 1286478 T3661 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4] 2> 1286478 T3661 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4] 2> 1286479 T3661 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4] 2> 1286480 T3661 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4] 2> 1286482 T3661 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 1286486 T3661 oass.SolrIndexSearcher.<init> Opening Searcher@13a1840f main
[junit4] 2> 1286489 T3661 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4] 2> 1286489 T3661 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4] 2> 1286499 T3661 oasc.CoreContainer.registerCore registering core: collection1
[junit4] 2> 1286499 T3661 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:59579/n_b/m collection:collection1 shard:shard1
[junit4] 2> 1286500 T3661 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=10000&maxConnectionsPerHost=20&connTimeout=15000&socketTimeout=30000&retry=false
[junit4] 2> 1286502 T3662 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@13a1840f main{StandardDirectoryReader(segments_1:1)}
[junit4] 2> 1286522 T3661 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
[junit4] 2> 1286528 T3661 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard1
[junit4] 2> 1286531 T3661 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4] 2> 1286532 T3661 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4] 2> 1286532 T3661 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:59579/n_b/m/collection1/
[junit4] 2> 1286533 T3661 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4] 2> 1286533 T3661 oasc.SyncStrategy.syncToMe http://127.0.0.1:59579/n_b/m/collection1/ has no replicas
[junit4] 2> 1286534 T3661 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:59579/n_b/m/collection1/ shard1
[junit4] 2> 1286534 T3661 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
[junit4] 2> 1288013 T3641 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 1288025 T3660 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 1288025 T3647 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 1288029 T3640 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 1288056 T3661 oasc.ZkController.register We are http://127.0.0.1:59579/n_b/m/collection1/ and leader is http://127.0.0.1:59579/n_b/m/collection1/
[junit4] 2> 1288056 T3661 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:59579/n_b/m
[junit4] 2> 1288057 T3661 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4] 2> 1288057 T3661 oasc.ZkController.publish publishing core=collection1 state=active
[junit4] 2> 1288057 T3661 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 1288059 T3661 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 1288060 T3618 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0
[junit4] 2> 1288061 T3618 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4] 2> 1288061 T3618 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4] 2> 1288159 T3618 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4] 2> 1288166 T3618 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:38771
[junit4] 2> 1288167 T3618 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4] 2> 1288168 T3618 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4] 2> 1288169 T3618 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty2-1376429270515
[junit4] 2> 1288170 T3618 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty2-1376429270515/'
[junit4] 2> 1288195 T3618 oasc.ConfigSolr.fromFile Loading container configuration from /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty2-1376429270515/solr.xml
[junit4] 2> 1288243 T3618 oasc.CoreContainer.<init> New CoreContainer -648773389
[junit4] 2> 1288244 T3618 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty2-1376429270515/]
[junit4] 2> 1288246 T3618 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4] 2> 1288247 T3618 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4] 2> 1288248 T3618 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4] 2> 1288248 T3618 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4] 2> 1288249 T3618 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4] 2> 1288250 T3618 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4] 2> 1288252 T3618 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4] 2> 1288253 T3618 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4] 2> 1288254 T3618 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4] 2> 1288255 T3618 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4] 2> 1288274 T3618 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:56032/solr
[junit4] 2> 1288275 T3618 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4] 2> 1288281 T3618 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1288283 T3678 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@90e2423d name:ZooKeeperConnection Watcher:127.0.0.1:56032 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1288283 T3618 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1288284 T3618 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4] 2> 1288291 T3618 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1288300 T3680 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@758e0c99 name:ZooKeeperConnection Watcher:127.0.0.1:56032/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1288301 T3618 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1288306 T3618 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 1289310 T3618 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:38771_n_b%2Fm
[junit4] 2> 1289312 T3618 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:38771_n_b%2Fm
[junit4] 2> 1289315 T3640 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
[junit4] 2> 1289315 T3647 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4] 2> 1289316 T3680 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4] 2> 1289316 T3647 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4] 2> 1289316 T3660 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4] 2> 1289317 T3660 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4] 2> 1289317 T3640 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4] 2> 1289342 T3681 oasc.ZkController.publish publishing core=collection1 state=down
[junit4] 2> 1289342 T3681 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 1289344 T3681 oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4] 2> 1289539 T3641 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 1289540 T3641 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4] 2> "shard_state":"active",
[junit4] 2> "base_url":"http://127.0.0.1:59579/n_b/m",
[junit4] 2> "state":"active",
[junit4] 2> "operation":"state",
[junit4] 2> "core_node_name":"core_node1",
[junit4] 2> "node_name":"127.0.0.1:59579_n_b%2Fm",
[junit4] 2> "numShards":"2",
[junit4] 2> "shard_range":null,
[junit4] 2> "roles":null,
[junit4] 2> "collection":"collection1",
[junit4] 2> "shard":"shard1",
[junit4] 2> "core":"collection1"}
[junit4] 2> 1289545 T3641 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4] 2> "shard_state":"active",
[junit4] 2> "base_url":"http://127.0.0.1:38771/n_b/m",
[junit4] 2> "state":"down",
[junit4] 2> "operation":"state",
[junit4] 2> "core_node_name":"core_node2",
[junit4] 2> "node_name":"127.0.0.1:38771_n_b%2Fm",
[junit4] 2> "numShards":"2",
[junit4] 2> "shard_range":null,
[junit4] 2> "roles":null,
[junit4] 2> "collection":"collection1",
[junit4] 2> "shard":null,
[junit4] 2> "core":"collection1"}
[junit4] 2> 1289546 T3641 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=2
[junit4] 2> 1289546 T3641 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard2
[junit4] 2> 1289550 T3660 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 1289550 T3680 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 1289550 T3647 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 1289550 T3640 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 1290344 T3681 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
[junit4] 2> 1290345 T3681 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty2-1376429270515/collection1
[junit4] 2> 1290345 T3681 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4] 2> 1290346 T3681 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4] 2> 1290347 T3681 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4] 2> 1290348 T3681 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty2-1376429270515/collection1/'
[junit4] 2> 1290349 T3681 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZk2Test-jetty2-1376429270515/collection1/lib/README' to classloader
[junit4] 2> 1290350 T3681 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZk2Test-jetty2-1376429270515/collection1/lib/classes/' to classloader
[junit4] 2> 1290399 T3681 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4] 2> 1290449 T3681 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4] 2> 1290451 T3681 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4] 2> 1290460 T3681 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4] 2> 1290959 T3681 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4] 2> 1290965 T3681 oass.IndexSchema.readSchema default search field in schema is text
[junit4] 2> 1290967 T3681 oass.IndexSchema.readSchema unique key field: id
[junit4] 2> 1290978 T3681 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 1290984 T3681 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 1290995 T3681 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4] 2> 1291001 T3681 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4] 2> 1291001 T3681 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4] 2> 1291003 T3681 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4] 2> 1291006 T3681 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4] 2> 1291007 T3681 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4] 2> 1291008 T3681 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4] 2> 1291009 T3681 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty2-1376429270515/collection1/, dataDir=./org.apache.solr.cloud.BasicDistributedZk2Test-1376429262138/jetty2/
[junit4] 2> 1291010 T3681 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@2f99c27e
[junit4] 2> 1291015 T3681 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.BasicDistributedZk2Test-1376429262138/jetty2
[junit4] 2> 1291015 T3681 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.BasicDistributedZk2Test-1376429262138/jetty2/index/
[junit4] 2> 1291016 T3681 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.BasicDistributedZk2Test-1376429262138/jetty2/index' doesn't exist. Creating new index...
[junit4] 2> 1291017 T3681 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.BasicDistributedZk2Test-1376429262138/jetty2/index
[junit4] 2> 1291021 T3681 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@a8d77bfa lockFactory=org.apache.lucene.store.NativeFSLockFactory@c4f0017),segFN=segments_1,generation=1}
[junit4] 2> 1291021 T3681 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 1291033 T3681 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4] 2> 1291033 T3681 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4] 2> 1291035 T3681 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 1291036 T3681 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4] 2> 1291037 T3681 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4] 2> 1291038 T3681 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4] 2> 1291039 T3681 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4] 2> 1291040 T3681 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4] 2> 1291040 T3681 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4] 2> 1291042 T3681 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 1291046 T3681 oass.SolrIndexSearcher.<init> Opening Searcher@3afcff90 main
[junit4] 2> 1291049 T3681 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4] 2> 1291049 T3681 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4] 2> 1291055 T3682 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@3afcff90 main{StandardDirectoryReader(segments_1:1)}
[junit4] 2> 1291057 T3681 oasc.CoreContainer.registerCore registering core: collection1
[junit4] 2> 1291057 T3681 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:38771/n_b/m collection:collection1 shard:shard2
[junit4] 2> 1291058 T3681 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=10000&maxConnectionsPerHost=20&connTimeout=15000&socketTimeout=30000&retry=false
[junit4] 2> 1291065 T3681 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard2/election
[junit4] 2> 1291073 T3681 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard2
[junit4] 2> 1291076 T3681 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4] 2> 1291076 T3681 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4] 2> 1291076 T3681 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:38771/n_b/m/collection1/
[junit4] 2> 1291077 T3681 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4] 2> 1291077 T3681 oasc.SyncStrategy.syncToMe http://127.0.0.1:38771/n_b/m/collection1/ has no replicas
[junit4] 2> 1291078 T3681 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:38771/n_b/m/collection1/ shard2
[junit4] 2> 1291078 T3681 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard2
[junit4] 2> 1292558 T3641 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 1292564 T3680 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 1292564 T3660 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 1292564 T3640 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 1292564 T3647 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 1292607 T3681 oasc.ZkController.register We are http://127.0.0.1:38771/n_b/m/collection1/ and leader is http://127.0.0.1:38771/n_b/m/collection1/
[junit4] 2> 1292607 T3681 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:38771/n_b/m
[junit4] 2> 1292607 T3681 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4] 2> 1292608 T3681 oasc.ZkController.publish publishing core=collection1 state=active
[junit4] 2> 1292608 T3681 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 1292610 T3681 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 1292611 T3618 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0
[junit4] 2> 1292612 T3618 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4] 2> 1292613 T3618 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4] 2> 1292723 T3618 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4] 2> 1292729 T3618 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:57465
[junit4] 2> 1292731 T3618 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4] 2> 1292732 T3618 oasc.SolrResourceLoader.locateSolrHome JNDI
[...truncated too long message...]
un(QueuedThreadPool.java:543)
[junit4] > at java.lang.Thread.run(Thread.java:780)Throwable #15: com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=1024, name=HashSessionScavenger-17, state=RUNNABLE, group=TGRP-FullSolrCloudDistribCmdsTest]
[junit4] > Caused by: java.lang.OutOfMemoryError: Java heap space
[junit4] > at __randomizedtesting.SeedInfo.seed([93CD8FAF60B2976]:0)
[junit4] > at java.security.AccessController.toArrayOfProtectionDomains(AccessController.java:184)
[junit4] > at java.security.AccessController.checkPermission(AccessController.java:102)
[junit4] > at java.lang.SecurityManager.checkPermission(SecurityManager.java:560)
[junit4] > at java.lang.Thread.setContextClassLoader(Thread.java:797)
[junit4] > at org.eclipse.jetty.server.session.HashSessionManager.scavenge(HashSessionManager.java:344)
[junit4] > at org.eclipse.jetty.server.session.HashSessionManager$2.run(HashSessionManager.java:284)
[junit4] > at java.util.Timer$TimerImpl.run(Timer.java:296)Throwable #16: com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=964, name=NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0, state=RUNNABLE, group=TGRP-FullSolrCloudDistribCmdsTest]
[junit4] > Caused by: java.lang.OutOfMemoryError: Java heap space
[junit4] > at __randomizedtesting.SeedInfo.seed([93CD8FAF60B2976]:0)
[junit4] > at java.util.ArrayList$SubList.listIterator(ArrayList.java:1024)
[junit4] > at java.util.AbstractList.listIterator(AbstractList.java:629)
[junit4] > at java.util.ArrayList$SubList.iterator(ArrayList.java:1016)
[junit4] > at java.util.AbstractCollection.toArray(AbstractCollection.java:363)
[junit4] > at java.util.regex.Pattern.split(Pattern.java:1238)
[junit4] > at java.util.regex.Pattern.split(Pattern.java:1270)
[junit4] > at java.lang.String.split(String.java:1749)
[junit4] > at org.apache.zookeeper.jmx.MBeanRegistry.tokenize(MBeanRegistry.java:182)
[junit4] > at org.apache.zookeeper.jmx.MBeanRegistry.makeObjectName(MBeanRegistry.java:204)
[junit4] > at org.apache.zookeeper.jmx.MBeanRegistry.unregister(MBeanRegistry.java:115)
[junit4] > at org.apache.zookeeper.jmx.MBeanRegistry.unregister(MBeanRegistry.java:132)
[junit4] > at org.apache.zookeeper.server.ServerCnxnFactory.unregisterConnection(ServerCnxnFactory.java:137)
[junit4] > at org.apache.zookeeper.server.NIOServerCnxn.close(NIOServerCnxn.java:972)
[junit4] > at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:352)
[junit4] > at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208)
[junit4] > at java.lang.Thread.run(Thread.java:780)Throwable #17: com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=1000, name=qtp220461705-1000, state=RUNNABLE, group=TGRP-FullSolrCloudDistribCmdsTest]
[junit4] > Caused by: java.lang.OutOfMemoryError: Java heap space
[junit4] > at __randomizedtesting.SeedInfo.seed([93CD8FAF60B2976]:0)
[junit4] > at java.util.concurrent.ConcurrentHashMap$KeySet.iterator(ConcurrentHashMap.java:1273)
[junit4] > at org.eclipse.jetty.io.nio.SelectorManager$SelectSet$1.run(SelectorManager.java:712)
[junit4] > at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
[junit4] > at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
[junit4] > at java.lang.Thread.run(Thread.java:780)Throwable #18: com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=1096, name=qtp1639013060-1096, state=RUNNABLE, group=TGRP-FullSolrCloudDistribCmdsTest]
[junit4] > Caused by: java.lang.OutOfMemoryError: Java heap space
[junit4] > at __randomizedtesting.SeedInfo.seed([93CD8FAF60B2976]:0)
[junit4] > at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.addConditionWaiter(AbstractQueuedSynchronizer.java:1868)
[junit4] > at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2084)
[junit4] > at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:342)
[junit4] > at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:526)
[junit4] > at org.eclipse.jetty.util.thread.QueuedThreadPool.access$600(QueuedThreadPool.java:44)
[junit4] > at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:572)
[junit4] > at java.lang.Thread.run(Thread.java:780)Throwable #19: com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=1009, name=qtp1639013060-1009 Selector0, state=RUNNABLE, group=TGRP-FullSolrCloudDistribCmdsTest]
[junit4] > Caused by: java.lang.OutOfMemoryError: Java heap space
[junit4] > at __randomizedtesting.SeedInfo.seed([93CD8FAF60B2976]:0)
[junit4] > at org.eclipse.jetty.io.nio.SelectorManager$1.run(SelectorManager.java:304)
[junit4] > at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
[junit4] > at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
[junit4] > at java.lang.Thread.run(Thread.java:780)Throwable #20: com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=1025, name=qtp-1726714643-1025, state=RUNNABLE, group=TGRP-FullSolrCloudDistribCmdsTest]
[junit4] > Caused by: java.lang.OutOfMemoryError: Java heap space
[junit4] > at __randomizedtesting.SeedInfo.seed([93CD8FAF60B2976]:0)
[junit4] > at java.util.HashMap$1.iterator(HashMap.java:672)
[junit4] > at java.util.HashSet.iterator(HashSet.java:173)
[junit4] > at sun.nio.ch.Util$2.iterator(Util.java:315)
[junit4] > at org.eclipse.jetty.io.nio.SelectorManager$SelectSet.doSelect(SelectorManager.java:600)
[junit4] > at org.eclipse.jetty.io.nio.SelectorManager$1.run(SelectorManager.java:290)
[junit4] > at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
[junit4] > at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
[junit4] > at java.lang.Thread.run(Thread.java:780)Throwable #21: com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=1209, name=qtp1556593299-1209, state=RUNNABLE, group=TGRP-FullSolrCloudDistribCmdsTest]
[junit4] > Caused by: java.lang.OutOfMemoryError: Java heap space
[junit4] > at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.addConditionWaiter(AbstractQueuedSynchronizer.java:1868)
[junit4] > at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2084)
[junit4] > at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:342)
[junit4] > at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:526)
[junit4] > at org.eclipse.jetty.util.thread.QueuedThreadPool.access$600(QueuedThreadPool.java:44)
[junit4] > at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:572)
[junit4] > at java.lang.Thread.run(Thread.java:780)Throwable #22: com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=1041, name=HashSessionScavenger-18, state=RUNNABLE, group=TGRP-FullSolrCloudDistribCmdsTest]
[junit4] > Caused by: java.lang.OutOfMemoryError: Java heap space
[junit4] > at __randomizedtesting.SeedInfo.seed([93CD8FAF60B2976]:0)
[junit4] > at java.security.AccessController.getProtectionDomains(Native Method)
[junit4] > at java.security.AccessController.checkPermission(AccessController.java:88)
[junit4] > at java.lang.SecurityManager.checkPermission(SecurityManager.java:560)
[junit4] > at java.lang.Thread.setContextClassLoader(Thread.java:797)
[junit4] > at org.eclipse.jetty.server.session.HashSessionManager.scavenge(HashSessionManager.java:344)
[junit4] > at org.eclipse.jetty.server.session.HashSessionManager$2.run(HashSessionManager.java:284)
[junit4] > at java.util.Timer$TimerImpl.run(Timer.java:296)
[junit4] Completed on J1 in 7226.21s, 1 test, 23 errors <<< FAILURES!
[...truncated 229 lines...]
BUILD FAILED
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/build.xml:389: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/build.xml:382: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/build.xml:39: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build.xml:181: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/common-build.xml:450: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/common-build.xml:1230: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/common-build.xml:873: There were test failures: 320 suites, 1360 tests, 22 suite-level errors, 2 errors, 215 ignored (5 assumptions)
Total time: 151 minutes 8 seconds
Build step 'Invoke Ant' marked build as failure
Description set: Java: 32bit/ibm-j9-jdk7 -Xjit:exclude={org/apache/lucene/util/fst/FST.pack(IIF)Lorg/apache/lucene/util/fst/FST;}
Archiving artifacts
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure
[JENKINS] Lucene-Solr-trunk-Linux (64bit/jdk1.8.0-ea-b102) - Build
# 6998 - Still Failing!
Posted by Policeman Jenkins Server <je...@thetaphi.de>.
Build: http://jenkins.thetaphi.de/job/Lucene-Solr-trunk-Linux/6998/
Java: 64bit/jdk1.8.0-ea-b102 -XX:+UseCompressedOops -XX:+UseG1GC
12 tests failed.
FAILED: junit.framework.TestSuite.org.apache.solr.cloud.FullSolrCloudDistribCmdsTest
Error Message:
Suite timeout exceeded (>= 7200000 msec).
Stack Trace:
java.lang.Exception: Suite timeout exceeded (>= 7200000 msec).
at __randomizedtesting.SeedInfo.seed([CA626772DFF8AF14]:0)
FAILED: junit.framework.TestSuite.org.apache.solr.cloud.FullSolrCloudDistribCmdsTest
Error Message:
Captured an uncaught exception in thread: Thread[id=1427, name=qtp2077582201-1427, state=RUNNABLE, group=TGRP-FullSolrCloudDistribCmdsTest]
Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=1427, name=qtp2077582201-1427, state=RUNNABLE, group=TGRP-FullSolrCloudDistribCmdsTest]
Caused by: java.lang.OutOfMemoryError: Java heap space
FAILED: junit.framework.TestSuite.org.apache.solr.cloud.FullSolrCloudDistribCmdsTest
Error Message:
Captured an uncaught exception in thread: Thread[id=1381, name=NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0, state=RUNNABLE, group=TGRP-FullSolrCloudDistribCmdsTest]
Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=1381, name=NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0, state=RUNNABLE, group=TGRP-FullSolrCloudDistribCmdsTest]
Caused by: java.lang.OutOfMemoryError: Java heap space
FAILED: junit.framework.TestSuite.org.apache.solr.cloud.FullSolrCloudDistribCmdsTest
Error Message:
Captured an uncaught exception in thread: Thread[id=1460, name=qtp2038018135-1460, state=RUNNABLE, group=TGRP-FullSolrCloudDistribCmdsTest]
Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=1460, name=qtp2038018135-1460, state=RUNNABLE, group=TGRP-FullSolrCloudDistribCmdsTest]
Caused by: java.lang.OutOfMemoryError: Java heap space
FAILED: junit.framework.TestSuite.org.apache.solr.cloud.FullSolrCloudDistribCmdsTest
Error Message:
Captured an uncaught exception in thread: Thread[id=1458, name=HashSessionScavenger-49, state=RUNNABLE, group=TGRP-FullSolrCloudDistribCmdsTest]
Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=1458, name=HashSessionScavenger-49, state=RUNNABLE, group=TGRP-FullSolrCloudDistribCmdsTest]
Caused by: java.lang.OutOfMemoryError: Java heap space
FAILED: junit.framework.TestSuite.org.apache.solr.cloud.FullSolrCloudDistribCmdsTest
Error Message:
Captured an uncaught exception in thread: Thread[id=1503, name=Thread-711, state=RUNNABLE, group=Overseer state updater.]
Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=1503, name=Thread-711, state=RUNNABLE, group=Overseer state updater.]
Caused by: java.lang.OutOfMemoryError: Java heap space
FAILED: junit.framework.TestSuite.org.apache.solr.cloud.FullSolrCloudDistribCmdsTest
Error Message:
Captured an uncaught exception in thread: Thread[id=1426, name=qtp2077582201-1426, state=RUNNABLE, group=TGRP-FullSolrCloudDistribCmdsTest]
Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=1426, name=qtp2077582201-1426, state=RUNNABLE, group=TGRP-FullSolrCloudDistribCmdsTest]
Caused by: java.lang.OutOfMemoryError: Java heap space
FAILED: junit.framework.TestSuite.org.apache.solr.cloud.FullSolrCloudDistribCmdsTest
Error Message:
Captured an uncaught exception in thread: Thread[id=1382, name=SessionTracker, state=RUNNABLE, group=TGRP-FullSolrCloudDistribCmdsTest]
Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=1382, name=SessionTracker, state=RUNNABLE, group=TGRP-FullSolrCloudDistribCmdsTest]
Caused by: java.lang.OutOfMemoryError: Java heap space
FAILED: junit.framework.TestSuite.org.apache.solr.cloud.FullSolrCloudDistribCmdsTest
Error Message:
Captured an uncaught exception in thread: Thread[id=1411, name=qtp451236801-1411, state=RUNNABLE, group=TGRP-FullSolrCloudDistribCmdsTest]
Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=1411, name=qtp451236801-1411, state=RUNNABLE, group=TGRP-FullSolrCloudDistribCmdsTest]
Caused by: java.lang.OutOfMemoryError: Java heap space
at __randomizedtesting.SeedInfo.seed([CA626772DFF8AF14]:0)
at java.util.HashMap.newKeyIterator(HashMap.java:2472)
at java.util.HashMap$KeySet.iterator(HashMap.java:2506)
at java.util.HashSet.iterator(HashSet.java:170)
at sun.nio.ch.Util$2.iterator(Util.java:243)
at org.eclipse.jetty.io.nio.SelectorManager$SelectSet.doSelect(SelectorManager.java:600)
at org.eclipse.jetty.io.nio.SelectorManager$1.run(SelectorManager.java:290)
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
at java.lang.Thread.run(Thread.java:724)
FAILED: junit.framework.TestSuite.org.apache.solr.cloud.FullSolrCloudDistribCmdsTest
Error Message:
Captured an uncaught exception in thread: Thread[id=1443, name=qtp1634913681-1443, state=RUNNABLE, group=TGRP-FullSolrCloudDistribCmdsTest]
Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=1443, name=qtp1634913681-1443, state=RUNNABLE, group=TGRP-FullSolrCloudDistribCmdsTest]
Caused by: java.lang.OutOfMemoryError: Java heap space
at __randomizedtesting.SeedInfo.seed([CA626772DFF8AF14]:0)
at java.util.HashMap.newKeyIterator(HashMap.java:2472)
at java.util.HashMap$KeySet.iterator(HashMap.java:2506)
at java.util.HashSet.iterator(HashSet.java:170)
at sun.nio.ch.Util$2.iterator(Util.java:243)
at org.eclipse.jetty.io.nio.SelectorManager$SelectSet.doSelect(SelectorManager.java:600)
at org.eclipse.jetty.io.nio.SelectorManager$1.run(SelectorManager.java:290)
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
at java.lang.Thread.run(Thread.java:724)
FAILED: junit.framework.TestSuite.org.apache.solr.cloud.FullSolrCloudDistribCmdsTest
Error Message:
Captured an uncaught exception in thread: Thread[id=1442, name=qtp1634913681-1442 Selector0, state=RUNNABLE, group=TGRP-FullSolrCloudDistribCmdsTest]
Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=1442, name=qtp1634913681-1442 Selector0, state=RUNNABLE, group=TGRP-FullSolrCloudDistribCmdsTest]
Caused by: java.lang.OutOfMemoryError: Java heap space
FAILED: org.apache.solr.cloud.FullSolrCloudDistribCmdsTest.testDistribSearch
Error Message:
Test abandoned because suite timeout was reached.
Stack Trace:
java.lang.Exception: Test abandoned because suite timeout was reached.
at __randomizedtesting.SeedInfo.seed([CA626772DFF8AF14]:0)
Build Log:
[...truncated 10368 lines...]
[junit4] Suite: org.apache.solr.cloud.FullSolrCloudDistribCmdsTest
[junit4] 2> 322856 T1379 oas.BaseDistributedSearchTestCase.initHostContext Setting hostContext system property: /
[junit4] 2> 322859 T1379 oas.SolrTestCaseJ4.setUp ###Starting testDistribSearch
[junit4] 2> Creating dataDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./solrtest-FullSolrCloudDistribCmdsTest-1376409781697
[junit4] 2> 322860 T1379 oasc.ZkTestServer.run STARTING ZK TEST SERVER
[junit4] 2> 322861 T1380 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
[junit4] 2> 322961 T1379 oasc.ZkTestServer.run start zk server on port:40259
[junit4] 2> 322962 T1379 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 322996 T1386 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@714f8225 name:ZooKeeperConnection Watcher:127.0.0.1:40259 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 322996 T1379 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 322997 T1379 oascc.SolrZkClient.makePath makePath: /solr
[junit4] 2> 323016 T1379 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 323017 T1388 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@7f88727c name:ZooKeeperConnection Watcher:127.0.0.1:40259/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 323018 T1379 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 323019 T1379 oascc.SolrZkClient.makePath makePath: /collections/collection1
[junit4] 2> 323021 T1379 oascc.SolrZkClient.makePath makePath: /collections/collection1/shards
[junit4] 2> 323023 T1379 oascc.SolrZkClient.makePath makePath: /collections/control_collection
[junit4] 2> 323024 T1379 oascc.SolrZkClient.makePath makePath: /collections/control_collection/shards
[junit4] 2> 323025 T1379 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
[junit4] 2> 323026 T1379 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.xml
[junit4] 2> 323028 T1379 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
[junit4] 2> 323028 T1379 oascc.SolrZkClient.makePath makePath: /configs/conf1/schema.xml
[junit4] 2> 323031 T1379 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
[junit4] 2> 323031 T1379 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.snippet.randomindexconfig.xml
[junit4] 2> 323040 T1379 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
[junit4] 2> 323041 T1379 oascc.SolrZkClient.makePath makePath: /configs/conf1/stopwords.txt
[junit4] 2> 323043 T1379 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
[junit4] 2> 323044 T1379 oascc.SolrZkClient.makePath makePath: /configs/conf1/protwords.txt
[junit4] 2> 323049 T1379 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
[junit4] 2> 323049 T1379 oascc.SolrZkClient.makePath makePath: /configs/conf1/currency.xml
[junit4] 2> 323051 T1379 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
[junit4] 2> 323051 T1379 oascc.SolrZkClient.makePath makePath: /configs/conf1/open-exchange-rates.json
[junit4] 2> 323053 T1379 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4] 2> 323053 T1379 oascc.SolrZkClient.makePath makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4] 2> 323055 T1379 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
[junit4] 2> 323055 T1379 oascc.SolrZkClient.makePath makePath: /configs/conf1/old_synonyms.txt
[junit4] 2> 323056 T1379 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
[junit4] 2> 323057 T1379 oascc.SolrZkClient.makePath makePath: /configs/conf1/synonyms.txt
[junit4] 2> 323134 T1379 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4] 2> 323137 T1379 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:41202
[junit4] 2> 323138 T1379 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4] 2> 323139 T1379 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4] 2> 323139 T1379 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-controljetty-1376409781897
[junit4] 2> 323139 T1379 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-controljetty-1376409781897/'
[junit4] 2> 323150 T1379 oasc.ConfigSolr.fromFile Loading container configuration from /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-controljetty-1376409781897/solr.xml
[junit4] 2> 323159 T1379 oasc.CoreContainer.<init> New CoreContainer 90753564
[junit4] 2> 323159 T1379 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-controljetty-1376409781897/]
[junit4] 2> 323160 T1379 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4] 2> 323161 T1379 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4] 2> 323161 T1379 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4] 2> 323161 T1379 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4] 2> 323162 T1379 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4] 2> 323162 T1379 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4] 2> 323162 T1379 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4] 2> 323163 T1379 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4] 2> 323163 T1379 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4] 2> 323163 T1379 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4] 2> 323168 T1379 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:40259/solr
[junit4] 2> 323168 T1379 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4] 2> 323169 T1379 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 323171 T1399 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@7473a8d3 name:ZooKeeperConnection Watcher:127.0.0.1:40259 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 323171 T1379 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 323173 T1379 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4] 2> 323177 T1379 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 323178 T1401 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@48a5240a name:ZooKeeperConnection Watcher:127.0.0.1:40259/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 323179 T1379 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 323180 T1379 oascc.SolrZkClient.makePath makePath: /overseer/queue
[junit4] 2> 323183 T1379 oascc.SolrZkClient.makePath makePath: /overseer/collection-queue-work
[junit4] 2> 323185 T1379 oascc.SolrZkClient.makePath makePath: /live_nodes
[junit4] 2> 323187 T1379 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:41202_
[junit4] 2> 323188 T1379 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:41202_
[junit4] 2> 323190 T1379 oascc.SolrZkClient.makePath makePath: /overseer_elect/election
[junit4] 2> 323194 T1379 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
[junit4] 2> 323197 T1379 oasc.Overseer.start Overseer (id=90204391453491203-127.0.0.1:41202_-n_0000000000) starting
[junit4] 2> 323199 T1379 oascc.SolrZkClient.makePath makePath: /overseer/queue-work
[junit4] 2> 323201 T1379 oascc.SolrZkClient.makePath makePath: /clusterstate.json
[junit4] 2> 323202 T1403 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
[junit4] 2> 323203 T1379 oascc.SolrZkClient.makePath makePath: /aliases.json
[junit4] 2> 323204 T1379 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 323206 T1402 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
[junit4] 2> 323207 T1404 oasc.ZkController.publish publishing core=collection1 state=down
[junit4] 2> 323207 T1404 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 323209 T1404 oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4] 2> 324709 T1402 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 324710 T1402 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4] 2> "operation":"state",
[junit4] 2> "core_node_name":"core_node1",
[junit4] 2> "numShards":"1",
[junit4] 2> "shard":null,
[junit4] 2> "roles":null,
[junit4] 2> "state":"down",
[junit4] 2> "shard_state":"active",
[junit4] 2> "core":"collection1",
[junit4] 2> "collection":"control_collection",
[junit4] 2> "shard_range":null,
[junit4] 2> "node_name":"127.0.0.1:41202_",
[junit4] 2> "base_url":"http://127.0.0.1:41202"}
[junit4] 2> 324710 T1402 oasc.Overseer$ClusterStateUpdater.createCollection Create collection control_collection with shards [shard1]
[junit4] 2> 324710 T1402 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4] 2> 324715 T1401 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 325210 T1404 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
[junit4] 2> 325210 T1404 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-controljetty-1376409781897/collection1
[junit4] 2> 325210 T1404 oasc.ZkController.createCollectionZkNode Check for collection zkNode:control_collection
[junit4] 2> 325211 T1404 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4] 2> 325211 T1404 oasc.ZkController.readConfigName Load collection config from:/collections/control_collection
[junit4] 2> 325212 T1404 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-controljetty-1376409781897/collection1/'
[junit4] 2> 325212 T1404 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-controljetty-1376409781897/collection1/lib/README' to classloader
[junit4] 2> 325212 T1404 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-controljetty-1376409781897/collection1/lib/classes/' to classloader
[junit4] 2> 325226 T1404 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4] 2> 325233 T1404 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4] 2> 325234 T1404 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4] 2> 325239 T1404 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4] 2> 325347 T1404 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4] 2> 325352 T1404 oass.IndexSchema.readSchema default search field in schema is text
[junit4] 2> 325353 T1404 oass.IndexSchema.readSchema unique key field: id
[junit4] 2> 325362 T1404 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 325364 T1404 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 325366 T1404 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4] 2> 325366 T1404 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4] 2> 325367 T1404 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4] 2> 325367 T1404 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4] 2> 325367 T1404 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4] 2> 325367 T1404 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4] 2> 325368 T1404 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4] 2> 325368 T1404 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-controljetty-1376409781897/collection1/, dataDir=./org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-1376409781697/control/data/
[junit4] 2> 325368 T1404 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@5d346920
[junit4] 2> 325368 T1404 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-1376409781697/control/data
[junit4] 2> 325368 T1404 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-1376409781697/control/data/index/
[junit4] 2> 325369 T1404 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-1376409781697/control/data/index' doesn't exist. Creating new index...
[junit4] 2> 325369 T1404 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-1376409781697/control/data/index
[junit4] 2> 325370 T1404 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@6e7eeec3 lockFactory=org.apache.lucene.store.NativeFSLockFactory@4a8eb53),segFN=segments_1,generation=1}
[junit4] 2> 325370 T1404 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 325372 T1404 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4] 2> 325372 T1404 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4] 2> 325373 T1404 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 325373 T1404 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4] 2> 325373 T1404 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4] 2> 325373 T1404 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4] 2> 325374 T1404 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4] 2> 325374 T1404 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4] 2> 325374 T1404 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4] 2> 325375 T1404 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 325376 T1404 oass.SolrIndexSearcher.<init> Opening Searcher@16c9783c main
[junit4] 2> 325377 T1404 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4] 2> 325377 T1404 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4] 2> 325378 T1405 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@16c9783c main{StandardDirectoryReader(segments_1:1)}
[junit4] 2> 325378 T1404 oasc.CoreContainer.registerCore registering core: collection1
[junit4] 2> 325378 T1404 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:41202 collection:control_collection shard:shard1
[junit4] 2> 325379 T1404 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=10000&maxConnectionsPerHost=20&connTimeout=15000&socketTimeout=30000&retry=false
[junit4] 2> 325381 T1404 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leader_elect/shard1/election
[junit4] 2> 325385 T1404 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard1
[junit4] 2> 325387 T1404 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4] 2> 325387 T1404 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4] 2> 325387 T1404 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:41202/collection1/
[junit4] 2> 325387 T1404 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4] 2> 325387 T1404 oasc.SyncStrategy.syncToMe http://127.0.0.1:41202/collection1/ has no replicas
[junit4] 2> 325387 T1404 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:41202/collection1/ shard1
[junit4] 2> 325388 T1404 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leaders/shard1
[junit4] 2> 326218 T1402 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 326223 T1401 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 326248 T1404 oasc.ZkController.register We are http://127.0.0.1:41202/collection1/ and leader is http://127.0.0.1:41202/collection1/
[junit4] 2> 326248 T1404 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:41202
[junit4] 2> 326249 T1404 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4] 2> 326249 T1404 oasc.ZkController.publish publishing core=collection1 state=active
[junit4] 2> 326249 T1404 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 326250 T1404 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 326251 T1379 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1
[junit4] 2> 326251 T1379 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4] 2> 326251 T1379 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4] 2> 326253 T1379 oascsi.HttpClientUtil.createClient Creating new http client, config:
[junit4] 2> 326254 T1379 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 326258 T1408 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@5381376e name:ZooKeeperConnection Watcher:127.0.0.1:40259/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 326258 T1379 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 326259 T1379 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 326260 T1379 oasc.ChaosMonkey.monkeyLog monkey: init - expire sessions:true cause connection loss:true
[junit4] 2> 326325 T1379 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4] 2> 326327 T1379 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:45314
[junit4] 2> 326328 T1379 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4] 2> 326328 T1379 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4] 2> 326328 T1379 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-jetty1-1376409785097
[junit4] 2> 326328 T1379 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-jetty1-1376409785097/'
[junit4] 2> 326338 T1379 oasc.ConfigSolr.fromFile Loading container configuration from /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-jetty1-1376409785097/solr.xml
[junit4] 2> 326345 T1379 oasc.CoreContainer.<init> New CoreContainer 99983877
[junit4] 2> 326345 T1379 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-jetty1-1376409785097/]
[junit4] 2> 326346 T1379 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4] 2> 326346 T1379 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4] 2> 326346 T1379 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4] 2> 326346 T1379 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4] 2> 326346 T1379 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4] 2> 326347 T1379 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4] 2> 326347 T1379 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4] 2> 326347 T1379 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4] 2> 326347 T1379 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4] 2> 326347 T1379 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4] 2> 326350 T1379 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:40259/solr
[junit4] 2> 326350 T1379 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4] 2> 326351 T1379 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 326353 T1419 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@521ef3c1 name:ZooKeeperConnection Watcher:127.0.0.1:40259 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 326353 T1379 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 326354 T1379 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4] 2> 326357 T1379 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 326358 T1421 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@54893882 name:ZooKeeperConnection Watcher:127.0.0.1:40259/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 326358 T1379 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 326360 T1379 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 327362 T1379 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:45314_
[junit4] 2> 327363 T1379 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:45314_
[junit4] 2> 327366 T1401 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
[junit4] 2> 327367 T1408 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4] 2> 327367 T1421 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4] 2> 327367 T1401 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4] 2> 327370 T1422 oasc.ZkController.publish publishing core=collection1 state=down
[junit4] 2> 327370 T1422 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 327371 T1422 oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4] 2> 327726 T1402 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 327727 T1402 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4] 2> "operation":"state",
[junit4] 2> "core_node_name":"core_node1",
[junit4] 2> "numShards":"1",
[junit4] 2> "shard":"shard1",
[junit4] 2> "roles":null,
[junit4] 2> "state":"active",
[junit4] 2> "shard_state":"active",
[junit4] 2> "core":"collection1",
[junit4] 2> "collection":"control_collection",
[junit4] 2> "shard_range":null,
[junit4] 2> "node_name":"127.0.0.1:41202_",
[junit4] 2> "base_url":"http://127.0.0.1:41202"}
[junit4] 2> 327729 T1402 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4] 2> "operation":"state",
[junit4] 2> "core_node_name":"core_node1",
[junit4] 2> "numShards":"2",
[junit4] 2> "shard":null,
[junit4] 2> "roles":null,
[junit4] 2> "state":"down",
[junit4] 2> "shard_state":"active",
[junit4] 2> "core":"collection1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "shard_range":null,
[junit4] 2> "node_name":"127.0.0.1:45314_",
[junit4] 2> "base_url":"http://127.0.0.1:45314"}
[junit4] 2> 327730 T1402 oasc.Overseer$ClusterStateUpdater.createCollection Create collection collection1 with shards [shard1, shard2]
[junit4] 2> 327730 T1402 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4] 2> 327732 T1421 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 327732 T1408 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 327732 T1401 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 328371 T1422 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
[junit4] 2> 328371 T1422 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-jetty1-1376409785097/collection1
[junit4] 2> 328372 T1422 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4] 2> 328372 T1422 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4] 2> 328372 T1422 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4] 2> 328373 T1422 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-jetty1-1376409785097/collection1/'
[junit4] 2> 328373 T1422 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-jetty1-1376409785097/collection1/lib/README' to classloader
[junit4] 2> 328374 T1422 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-jetty1-1376409785097/collection1/lib/classes/' to classloader
[junit4] 2> 328387 T1422 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4] 2> 328394 T1422 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4] 2> 328395 T1422 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4] 2> 328401 T1422 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4] 2> 328504 T1422 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4] 2> 328507 T1422 oass.IndexSchema.readSchema default search field in schema is text
[junit4] 2> 328508 T1422 oass.IndexSchema.readSchema unique key field: id
[junit4] 2> 328515 T1422 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 328517 T1422 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 328519 T1422 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4] 2> 328519 T1422 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4] 2> 328519 T1422 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4] 2> 328520 T1422 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4] 2> 328520 T1422 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4] 2> 328520 T1422 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4] 2> 328520 T1422 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4] 2> 328521 T1422 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-jetty1-1376409785097/collection1/, dataDir=./org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-1376409781697/jetty1/
[junit4] 2> 328521 T1422 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@5d346920
[junit4] 2> 328521 T1422 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-1376409781697/jetty1
[junit4] 2> 328522 T1422 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-1376409781697/jetty1/index/
[junit4] 2> 328522 T1422 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-1376409781697/jetty1/index' doesn't exist. Creating new index...
[junit4] 2> 328522 T1422 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-1376409781697/jetty1/index
[junit4] 2> 328523 T1422 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@5e45441 lockFactory=org.apache.lucene.store.NativeFSLockFactory@5b9d2702),segFN=segments_1,generation=1}
[junit4] 2> 328523 T1422 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 328525 T1422 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4] 2> 328525 T1422 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4] 2> 328526 T1422 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 328526 T1422 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4] 2> 328526 T1422 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4] 2> 328527 T1422 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4] 2> 328527 T1422 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4] 2> 328527 T1422 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4] 2> 328527 T1422 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4] 2> 328529 T1422 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 328529 T1422 oass.SolrIndexSearcher.<init> Opening Searcher@443e3e24 main
[junit4] 2> 328530 T1422 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4] 2> 328530 T1422 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4] 2> 328532 T1423 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@443e3e24 main{StandardDirectoryReader(segments_1:1)}
[junit4] 2> 328533 T1422 oasc.CoreContainer.registerCore registering core: collection1
[junit4] 2> 328533 T1422 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:45314 collection:collection1 shard:shard1
[junit4] 2> 328534 T1422 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=10000&maxConnectionsPerHost=20&connTimeout=15000&socketTimeout=30000&retry=false
[junit4] 2> 328537 T1422 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
[junit4] 2> 328540 T1422 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard1
[junit4] 2> 328542 T1422 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4] 2> 328542 T1422 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4] 2> 328542 T1422 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:45314/collection1/
[junit4] 2> 328542 T1422 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4] 2> 328542 T1422 oasc.SyncStrategy.syncToMe http://127.0.0.1:45314/collection1/ has no replicas
[junit4] 2> 328543 T1422 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:45314/collection1/ shard1
[junit4] 2> 328543 T1422 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
[junit4] 2> 329236 T1402 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 329242 T1408 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 329242 T1421 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 329242 T1401 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 329248 T1422 oasc.ZkController.register We are http://127.0.0.1:45314/collection1/ and leader is http://127.0.0.1:45314/collection1/
[junit4] 2> 329249 T1422 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:45314
[junit4] 2> 329249 T1422 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4] 2> 329249 T1422 oasc.ZkController.publish publishing core=collection1 state=active
[junit4] 2> 329249 T1422 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 329251 T1422 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 329252 T1379 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1
[junit4] 2> 329252 T1379 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4] 2> 329252 T1379 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4] 2> 329313 T1379 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4] 2> 329314 T1379 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:49936
[junit4] 2> 329315 T1379 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4] 2> 329315 T1379 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4] 2> 329316 T1379 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-jetty2-1376409788091
[junit4] 2> 329316 T1379 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-jetty2-1376409788091/'
[junit4] 2> 329326 T1379 oasc.ConfigSolr.fromFile Loading container configuration from /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-jetty2-1376409788091/solr.xml
[junit4] 2> 329334 T1379 oasc.CoreContainer.<init> New CoreContainer 1680629742
[junit4] 2> 329334 T1379 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-jetty2-1376409788091/]
[junit4] 2> 329335 T1379 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4] 2> 329335 T1379 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4] 2> 329335 T1379 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4] 2> 329335 T1379 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4] 2> 329336 T1379 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4] 2> 329336 T1379 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4] 2> 329336 T1379 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4] 2> 329336 T1379 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4] 2> 329337 T1379 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4] 2> 329337 T1379 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4] 2> 329340 T1379 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:40259/solr
[junit4] 2> 329340 T1379 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4] 2> 329341 T1379 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 329344 T1435 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@72697b8 name:ZooKeeperConnection Watcher:127.0.0.1:40259 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 329344 T1379 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 329348 T1379 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4] 2> 329351 T1379 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 329353 T1437 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@6059bc00 name:ZooKeeperConnection Watcher:127.0.0.1:40259/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 329353 T1379 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 329365 T1379 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 330368 T1379 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:49936_
[junit4] 2> 330369 T1379 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:49936_
[junit4] 2> 330371 T1401 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
[junit4] 2> 330372 T1437 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4] 2> 330372 T1421 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4] 2> 330372 T1421 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4] 2> 330372 T1408 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4] 2> 330372 T1408 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4] 2> 330373 T1401 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4] 2> 330374 T1438 oasc.ZkController.publish publishing core=collection1 state=down
[junit4] 2> 330374 T1438 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 330375 T1438 oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4] 2> 330747 T1402 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 330748 T1402 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4] 2> "operation":"state",
[junit4] 2> "core_node_name":"core_node1",
[junit4] 2> "numShards":"2",
[junit4] 2> "shard":"shard1",
[junit4] 2> "roles":null,
[junit4] 2> "state":"active",
[junit4] 2> "shard_state":"active",
[junit4] 2> "core":"collection1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "shard_range":null,
[junit4] 2> "node_name":"127.0.0.1:45314_",
[junit4] 2> "base_url":"http://127.0.0.1:45314"}
[junit4] 2> 330750 T1402 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4] 2> "operation":"state",
[junit4] 2> "core_node_name":"core_node2",
[junit4] 2> "numShards":"2",
[junit4] 2> "shard":null,
[junit4] 2> "roles":null,
[junit4] 2> "state":"down",
[junit4] 2> "shard_state":"active",
[junit4] 2> "core":"collection1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "shard_range":null,
[junit4] 2> "node_name":"127.0.0.1:49936_",
[junit4] 2> "base_url":"http://127.0.0.1:49936"}
[junit4] 2> 330750 T1402 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=2
[junit4] 2> 330750 T1402 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard2
[junit4] 2> 330756 T1408 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 330756 T1401 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 330756 T1421 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 330756 T1437 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 331375 T1438 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
[junit4] 2> 331376 T1438 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-jetty2-1376409788091/collection1
[junit4] 2> 331376 T1438 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4] 2> 331377 T1438 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4] 2> 331377 T1438 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4] 2> 331377 T1438 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-jetty2-1376409788091/collection1/'
[junit4] 2> 331378 T1438 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-jetty2-1376409788091/collection1/lib/README' to classloader
[junit4] 2> 331378 T1438 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-jetty2-1376409788091/collection1/lib/classes/' to classloader
[junit4] 2> 331391 T1438 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4] 2> 331398 T1438 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4] 2> 331399 T1438 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4] 2> 331404 T1438 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4] 2> 331507 T1438 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4] 2> 331510 T1438 oass.IndexSchema.readSchema default search field in schema is text
[junit4] 2> 331512 T1438 oass.IndexSchema.readSchema unique key field: id
[junit4] 2> 331519 T1438 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 331520 T1438 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 331522 T1438 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4] 2> 331523 T1438 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4] 2> 331523 T1438 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4] 2> 331523 T1438 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4] 2> 331523 T1438 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4] 2> 331524 T1438 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4] 2> 331524 T1438 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4] 2> 331524 T1438 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-jetty2-1376409788091/collection1/, dataDir=./org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-1376409781697/jetty2/
[junit4] 2> 331524 T1438 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@5d346920
[junit4] 2> 331525 T1438 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-1376409781697/jetty2
[junit4] 2> 331525 T1438 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-1376409781697/jetty2/index/
[junit4] 2> 331525 T1438 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-1376409781697/jetty2/index' doesn't exist. Creating new index...
[junit4] 2> 331525 T1438 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-1376409781697/jetty2/index
[junit4] 2> 331526 T1438 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@6948c532 lockFactory=org.apache.lucene.store.NativeFSLockFactory@26456fa3),segFN=segments_1,generation=1}
[junit4] 2> 331526 T1438 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 331528 T1438 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4] 2> 331529 T1438 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4] 2> 331529 T1438 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 331529 T1438 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4] 2> 331530 T1438 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4] 2> 331530 T1438 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4] 2> 331530 T1438 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4] 2> 331530 T1438 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4] 2> 331530 T1438 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4] 2> 331532 T1438 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 331533 T1438 oass.SolrIndexSearcher.<init> Opening Searcher@1600c78d main
[junit4] 2> 331533 T1438 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4] 2> 331533 T1438 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4] 2> 331535 T1439 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@1600c78d main{StandardDirectoryReader(segments_1:1)}
[junit4] 2> 331535 T1438 oasc.CoreContainer.registerCore registering core: collection1
[junit4] 2> 331535 T1438 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:49936 collection:collection1 shard:shard2
[junit4] 2> 331536 T1438 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=10000&maxConnectionsPerHost=20&connTimeout=15000&socketTimeout=30000&retry=false
[junit4] 2> 331538 T1438 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard2/election
[junit4] 2> 331542 T1438 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard2
[junit4] 2> 331543 T1438 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4] 2> 331543 T1438 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4] 2> 331543 T1438 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:49936/collection1/
[junit4] 2> 331544 T1438 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4] 2> 331544 T1438 oasc.SyncStrategy.syncToMe http://127.0.0.1:49936/collection1/ has no replicas
[junit4] 2> 331544 T1438 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:49936/collection1/ shard2
[junit4] 2> 331544 T1438 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard2
[junit4] 2> 332259 T1402 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 332264 T1408 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 332264 T1421 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 332265 T1401 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 332265 T1437 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 332298 T1438 oasc.ZkController.register We are http://127.0.0.1:49936/collection1/ and leader is http://127.0.0.1:49936/collection1/
[junit4] 2> 332299 T1438 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:49936
[junit4] 2> 332299 T1438 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4] 2> 332299 T1438 oasc.ZkController.publish publishing core=collection1 state=active
[junit4] 2> 332299 T1438 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 332300 T1438 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 332301 T1379 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1
[junit4] 2> 332301 T1379 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4] 2> 332302 T1379 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4] 2> 332374 T1379 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4] 2> 332376 T1379 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:57730
[junit4] 2> 332376 T1379 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4] 2> 332377 T1379 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4] 2> 332377 T1379 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-jetty3-1376409791141
[junit4] 2> 332377 T1379 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-jetty3-1376409791141/'
[junit4] 2> 332387 T1379 oasc.ConfigSolr.fromFile Loading container configuration from /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-jetty3-1376409791141/solr.xml
[junit4] 2> 332394 T1379 oasc.CoreContainer.<init> New CoreContainer 40941163
[junit4] 2> 332395 T1379 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-jetty3-1376409791141/]
[junit4] 2> 332395 T1379 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4] 2> 332395 T1379 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4] 2> 332396 T1379 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4] 2> 332396 T1379 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4] 2> 332396 T1379 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4] 2> 332396 T1379 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4] 2> 332397 T1379 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4] 2> 332397 T1379 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4] 2> 332397 T1379 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4] 2> 332397 T1379 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4] 2> 332400 T1379 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:40259/solr
[junit4] 2> 332400 T1379 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4] 2> 332401 T1379 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 332403 T1451 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@75b5f50e name:ZooKeeperConnection Watcher:127.0.0.1:40259 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 332403 T1379 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 332404 T1379 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4] 2> 332407 T1379 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 332408 T1453 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@8d3a725 name:ZooKeeperConnection Watcher:127.0.0.1:40259/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 332409 T1379 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 332411 T1379 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 333414 T1379 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:57730_
[junit4] 2> 333415 T1379 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:57730_
[junit4] 2> 333417 T1401 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4] 2> 333417 T1408 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4] 2> 333418 T1408 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4] 2> 333418 T1421 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4] 2> 333418 T1453 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4] 2> 333418 T1421 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4] 2> 333418 T1437 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4] 2> 333419 T1437 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4] 2> 333419 T1401 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4] 2> 333421 T1454 oasc.ZkController.publish publishing core=collection1 state=down
[junit4] 2> 333421 T1454 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 333422 T1454 oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4] 2> 333769 T1402 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 333770 T1402 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4] 2> "operation":"state",
[junit4] 2> "core_node_name":"core_node2",
[junit4] 2> "numShards":"2",
[junit4] 2> "shard":"shard2",
[junit4] 2> "roles":null,
[junit4] 2> "state":"active",
[junit4] 2> "shard_state":"active",
[junit4] 2> "core":"collection1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "shard_range":null,
[junit4] 2> "node_name":"127.0.0.1:49936_",
[junit4] 2> "base_url":"http://127.0.0.1:49936"}
[junit4] 2> 333772 T1402 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4] 2> "operation":"state",
[junit4] 2> "core_node_name":"core_node3",
[junit4] 2> "numShards":"2",
[junit4] 2> "shard":null,
[junit4] 2> "roles":null,
[junit4] 2> "state":"down",
[junit4] 2> "shard_state":"active",
[junit4] 2> "core":"collection1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "shard_range":null,
[junit4] 2> "node_name":"127.0.0.1:57730_",
[junit4] 2> "base_url":"http://127.0.0.1:57730"}
[junit4] 2> 333773 T1402 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=2
[junit4] 2> 333773 T1402 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4] 2> 333775 T1408 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4] 2> 333775 T1437 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4] 2> 333776 T1401 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4] 2> 333775 T1453 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4] 2> 333775 T1421 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4] 2> 334422 T1454 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
[junit4] 2> 334422 T1454 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-jetty3-1376409791141/collection1
[junit4] 2> 334423 T1454 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4] 2> 334424 T1454 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4] 2> 334424 T1454 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4] 2> 334425 T1454 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-jetty3-1376409791141/collection1/'
[junit4] 2> 334426 T1454 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-jetty3-1376409791141/collection1/lib/README' to classloader
[junit4] 2> 334427 T1454 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-jetty3-1376409791141/collection1/lib/classes/' to classloader
[junit4] 2> 334444 T1454 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4] 2> 334451 T1454 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4] 2> 334453 T1454 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4] 2> 334457 T1454 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4] 2> 334545 T1454 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4] 2> 334549 T1454 oass.IndexSchema.readSchema default search field in schema is text
[junit4] 2> 334550 T1454 oass.IndexSchema.readSchema unique key field: id
[junit4] 2> 334557 T1454 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 334559 T1454 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 334585 T1454 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4] 2> 334586 T1454 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4] 2> 334586 T1454 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4] 2> 334587 T1454 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4] 2> 334587 T1454 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4] 2> 334587 T1454 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4] 2> 334587 T1454 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4] 2> 334588 T1454 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-jetty3-1376409791141/collection1/, dataDir=./org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-1376409781697/jetty3/
[junit4] 2> 334588 T1454 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@5d346920
[junit4] 2> 334588 T1454 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-1376409781697/jetty3
[junit4] 2> 334588 T1454 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-1376409781697/jetty3/index/
[junit4] 2> 334589 T1454 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-1376409781697/jetty3/index' doesn't exist. Creating new index...
[junit4] 2> 334589 T1454 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-1376409781697/jetty3/index
[junit4] 2> 334590 T1454 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@43409f14 lockFactory=org.apache.lucene.store.NativeFSLockFactory@57f90120),segFN=segments_1,generation=1}
[junit4] 2> 334590 T1454 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 334592 T1454 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4] 2> 334592 T1454 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4] 2> 334593 T1454 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 334593 T1454 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4] 2> 334593 T1454 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4] 2> 334594 T1454 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4] 2> 334594 T1454 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4] 2> 334594 T1454 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4] 2> 334594 T1454 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4] 2> 334596 T1454 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 334596 T1454 oass.SolrIndexSearcher.<init> Opening Searcher@5504ff1c main
[junit4] 2> 334597 T1454 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4] 2> 334597 T1454 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4] 2> 334599 T1455 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@5504ff1c main{StandardDirectoryReader(segments_1:1)}
[junit4] 2> 334600 T1454 oasc.CoreContainer.registerCore registering core: collection1
[junit4] 2> 334600 T1454 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:57730 collection:collection1 shard:shard1
[junit4] 2> 334600 T1454 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=10000&maxConnectionsPerHost=20&connTimeout=15000&socketTimeout=30000&retry=false
[junit4] 2> 334605 T1454 oasc.ZkController.register We are http://127.0.0.1:57730/collection1/ and leader is http://127.0.0.1:45314/collection1/
[junit4] 2> 334605 T1454 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:57730
[junit4] 2> 334605 T1454 oasc.ZkController.checkRecovery Core needs to recover:collection1
[junit4] 2> 334606 T1454 oasu.DefaultSolrCoreState.doRecovery Running recovery - first canceling any ongoing recovery
[junit4] 2> ASYNC NEW_CORE C387 name=collection1 org.apache.solr.core.SolrCore@455d491c url=http://127.0.0.1:57730/collection1 node=127.0.0.1:57730_ C387_STATE=coll:collection1 core:collection1 props:{state=down, core=collection1, node_name=127.0.0.1:57730_, base_url=http://127.0.0.1:57730}
[junit4] 2> 334606 T1456 C387 P57730 oasc.RecoveryStrategy.run Starting recovery process. core=collection1 recoveringAfterStartup=true
[junit4] 2> 334606 T1454 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 334607 T1456 C387 P57730 oasc.RecoveryStrategy.doRecovery ###### startupVersions=[]
[junit4] 2> 334607 T1456 C387 P57730 oasc.ZkController.publish publishing core=collection1 state=recovering
[junit4] 2> 334607 T1456 C387 P57730 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 334607 T1379 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1
[junit4] 2> 334608 T1379 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4] 2> 334608 T1456 C387 P57730 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4] 2> 334608 T1379 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4] 2> 334612 T1416 oasha.CoreAdminHandler.handleWaitForStateAction Going to wait for coreNodeName: core_node3, state: recovering, checkLive: true, onlyIfLeader: true
[junit4] 2> 334681 T1379 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4] 2> 334684 T1379 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:39994
[junit4] 2> 334684 T1379 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4] 2> 334685 T1379 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4] 2> 334685 T1379 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-jetty4-1376409793447
[junit4] 2> 334685 T1379 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-jetty4-1376409793447/'
[junit4] 2> 334699 T1379 oasc.ConfigSolr.fromFile Loading container configuration from /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-jetty4-1376409793447/solr.xml
[junit4] 2> 334706 T1379 oasc.CoreContainer.<init> New CoreContainer 1997155242
[junit4] 2> 334707 T1379 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-jetty4-1376409793447/]
[junit4] 2> 334707 T1379 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4] 2> 334707 T1379 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4] 2> 334708 T1379 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4] 2> 334708 T1379 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4] 2> 334708 T1379 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4] 2> 334708 T1379 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4] 2> 334708 T1379 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4] 2> 334709 T1379 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4] 2> 334709 T1379 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4] 2> 334709 T1379 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4] 2> 334712 T1379 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:40259/solr
[junit4] 2> 334712 T1379 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4] 2> 334713 T1379 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 334714 T1468 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@d4d6e2d name:ZooKeeperConnection Watcher:127.0.0.1:40259 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 334715 T1379 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 334716 T1379 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4] 2> 334718 T1379 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 334720 T1470 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@9bbc4e7 name:ZooKeeperConnection Watcher:127.0.0.1:40259/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 334720 T1379 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 334722 T1379 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 335279 T1402 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 335280 T1402 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4] 2> "operation":"state",
[junit4] 2> "core_node_name":"core_node3",
[junit4] 2> "numShards":"2",
[junit4] 2> "shard":"shard1",
[junit4] 2> "roles":null,
[junit4] 2> "state":"recovering",
[junit4] 2> "shard_state":"active",
[junit4] 2> "core":"collection1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "shard_range":null,
[junit4] 2> "node_name":"127.0.0.1:57730_",
[junit4] 2> "base_url":"http://127.0.0.1:57730"}
[junit4] 2> 335283 T1408 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4] 2> 335283 T1470 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4] 2> 335284 T1401 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4] 2> 335283 T1421 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4] 2> 335283 T1437 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4] 2> 335284 T1453 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4] 2> 335612 T1416 oasha.CoreAdminHandler.handleWaitForStateAction Waited coreNodeName: core_node3, state: recovering, checkLive: true, onlyIfLeader: true for: 1 seconds.
[junit4] 2> 335613 T1416 oass.SolrDispatchFilter.handleAdminRequest [admin] webapp=null path=/admin/cores params={coreNodeName=core_node3&state=recovering&nodeName=127.0.0.1:57730_&action=PREPRECOVERY&checkLive=true&core=collection1&wt=javabin&onlyIfLeader=true&version=2} status=0 QTi
[...truncated too long message...]
ipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
[junit4] 2> at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
[junit4] 2> at java.lang.Thread.run(Thread.java:724)
[junit4] 2> 8) Thread[id=1441, name=HashSessionScavenger-48, state=TIMED_WAITING, group=TGRP-FullSolrCloudDistribCmdsTest]
[junit4] 2> at java.lang.Object.wait(Native Method)
[junit4] 2> at java.util.TimerThread.mainLoop(Timer.java:552)
[junit4] 2> at java.util.TimerThread.run(Timer.java:505)
[junit4] 2> 9) Thread[id=1473, name=searcherExecutor-1030-thread-1, state=WAITING, group=TGRP-FullSolrCloudDistribCmdsTest]
[junit4] 2> at sun.misc.Unsafe.park(Native Method)
[junit4] 2> at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
[junit4] 2> at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2037)
[junit4] 2> at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
[junit4] 2> at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)
[junit4] 2> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
[junit4] 2> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
[junit4] 2> at java.lang.Thread.run(Thread.java:724)
[junit4] 2> 10) Thread[id=1462, name=qtp2038018135-1462 Acceptor1 SelectChannelConnector@127.0.0.1:39994, state=RUNNABLE, group=TGRP-FullSolrCloudDistribCmdsTest]
[junit4] 2> at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:102)
[junit4] 2> at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
[junit4] 2> at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
[junit4] 2> at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
[junit4] 2> at java.lang.Thread.run(Thread.java:724)
[junit4] 2> NOTE: test params are: codec=Asserting, sim=RandomSimilarityProvider(queryNorm=true,coord=no): {}, locale=es_BO, timezone=Atlantic/Cape_Verde
[junit4] 2> NOTE: Linux 3.2.0-51-generic amd64/Oracle Corporation 1.8.0-ea (64-bit)/cpus=8,threads=11,free=207176640,total=536870912
[junit4] 2> NOTE: All tests run in this JVM: [XmlUpdateRequestHandlerTest, BadCopyFieldTest, SolrPluginUtilsTest, TestLazyCores, TestBinaryResponseWriter, BasicZkTest, RequestHandlersTest, TestDynamicFieldResource, ConvertedLegacyTest, TestMultiCoreConfBootstrap, ChaosMonkeySafeLeaderTest, OutputWriterTest, PrimUtilsTest, StatsComponentTest, UniqFieldsUpdateProcessorFactoryTest, UUIDFieldTest, TestFastOutputStream, TestCloudManagedSchema, TestSolr4Spatial, DisMaxRequestHandlerTest, PingRequestHandlerTest, NoCacheHeaderTest, TestSchemaSimilarityResource, HdfsCollectionsAPIDistributedZkTest, TestManagedSchema, TestBinaryField, DeleteShardTest, TestDistributedSearch, DateMathParserTest, TestFastWriter, DirectSolrSpellCheckerTest, TestUpdate, TestDistributedGrouping, RAMDirectoryFactoryTest, TestQueryTypes, MinimalSchemaTest, TestDFRSimilarityFactory, TestAddFieldRealTimeGet, TestRandomFaceting, PreAnalyzedFieldTest, TestSolrCoreProperties, TestInfoStreamLogging, TestWordDelimiterFilterFactory, TestRandomMergePolicy, TestSuggestSpellingConverter, TestRecovery, HdfsBasicDistributedZk2Test, HighlighterConfigTest, FullSolrCloudDistribCmdsTest]
[junit4] 2> NOTE: reproduce with: ant test -Dtestcase=FullSolrCloudDistribCmdsTest -Dtests.seed=CA626772DFF8AF14 -Dtests.multiplier=3 -Dtests.slow=true -Dtests.locale=es_BO -Dtests.timezone=Atlantic/Cape_Verde -Dtests.file.encoding=ISO-8859-1
[junit4] ERROR 0.00s J1 | FullSolrCloudDistribCmdsTest (suite) <<<
[junit4] > Throwable #1: java.lang.Exception: Suite timeout exceeded (>= 7200000 msec).
[junit4] > at __randomizedtesting.SeedInfo.seed([CA626772DFF8AF14]:0)Throwable #2: com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=1427, name=qtp2077582201-1427, state=RUNNABLE, group=TGRP-FullSolrCloudDistribCmdsTest]
[junit4] > Caused by: java.lang.OutOfMemoryError: Java heap spaceThrowable #3: com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=1381, name=NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0, state=RUNNABLE, group=TGRP-FullSolrCloudDistribCmdsTest]
[junit4] > Caused by: java.lang.OutOfMemoryError: Java heap spaceThrowable #4: com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=1460, name=qtp2038018135-1460, state=RUNNABLE, group=TGRP-FullSolrCloudDistribCmdsTest]
[junit4] > Caused by: java.lang.OutOfMemoryError: Java heap spaceThrowable #5: com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=1458, name=HashSessionScavenger-49, state=RUNNABLE, group=TGRP-FullSolrCloudDistribCmdsTest]
[junit4] > Caused by: java.lang.OutOfMemoryError: Java heap spaceThrowable #6: com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=1503, name=Thread-711, state=RUNNABLE, group=Overseer state updater.]
[junit4] > Caused by: java.lang.OutOfMemoryError: Java heap spaceThrowable #7: com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=1426, name=qtp2077582201-1426, state=RUNNABLE, group=TGRP-FullSolrCloudDistribCmdsTest]
[junit4] > Caused by: java.lang.OutOfMemoryError: Java heap spaceThrowable #8: com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=1382, name=SessionTracker, state=RUNNABLE, group=TGRP-FullSolrCloudDistribCmdsTest]
[junit4] > Caused by: java.lang.OutOfMemoryError: Java heap spaceThrowable #9: com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=1411, name=qtp451236801-1411, state=RUNNABLE, group=TGRP-FullSolrCloudDistribCmdsTest]
[junit4] > Caused by: java.lang.OutOfMemoryError: Java heap space
[junit4] > at __randomizedtesting.SeedInfo.seed([CA626772DFF8AF14]:0)
[junit4] > at java.util.HashMap.newKeyIterator(HashMap.java:2472)
[junit4] > at java.util.HashMap$KeySet.iterator(HashMap.java:2506)
[junit4] > at java.util.HashSet.iterator(HashSet.java:170)
[junit4] > at sun.nio.ch.Util$2.iterator(Util.java:243)
[junit4] > at org.eclipse.jetty.io.nio.SelectorManager$SelectSet.doSelect(SelectorManager.java:600)
[junit4] > at org.eclipse.jetty.io.nio.SelectorManager$1.run(SelectorManager.java:290)
[junit4] > at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
[junit4] > at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
[junit4] > at java.lang.Thread.run(Thread.java:724)Throwable #10: com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=1443, name=qtp1634913681-1443, state=RUNNABLE, group=TGRP-FullSolrCloudDistribCmdsTest]
[junit4] > Caused by: java.lang.OutOfMemoryError: Java heap space
[junit4] > at __randomizedtesting.SeedInfo.seed([CA626772DFF8AF14]:0)
[junit4] > at java.util.HashMap.newKeyIterator(HashMap.java:2472)
[junit4] > at java.util.HashMap$KeySet.iterator(HashMap.java:2506)
[junit4] > at java.util.HashSet.iterator(HashSet.java:170)
[junit4] > at sun.nio.ch.Util$2.iterator(Util.java:243)
[junit4] > at org.eclipse.jetty.io.nio.SelectorManager$SelectSet.doSelect(SelectorManager.java:600)
[junit4] > at org.eclipse.jetty.io.nio.SelectorManager$1.run(SelectorManager.java:290)
[junit4] > at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
[junit4] > at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
[junit4] > at java.lang.Thread.run(Thread.java:724)Throwable #11: com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=1442, name=qtp1634913681-1442 Selector0, state=RUNNABLE, group=TGRP-FullSolrCloudDistribCmdsTest]
[junit4] > Caused by: java.lang.OutOfMemoryError: Java heap space
[junit4] Completed on J1 in 7223.55s, 1 test, 12 errors <<< FAILURES!
[...truncated 111 lines...]
BUILD FAILED
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/build.xml:389: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/build.xml:369: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/build.xml:39: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build.xml:181: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/common-build.xml:450: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/common-build.xml:1230: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/common-build.xml:873: There were test failures: 320 suites, 1360 tests, 11 suite-level errors, 1 error, 111 ignored (6 assumptions)
Total time: 147 minutes 29 seconds
Build step 'Invoke Ant' marked build as failure
Description set: Java: 64bit/jdk1.8.0-ea-b102 -XX:+UseCompressedOops -XX:+UseG1GC
Archiving artifacts
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure
[JENKINS] Lucene-Solr-trunk-Linux (64bit/jdk1.7.0_25) - Build #
6997 - Still Failing!
Posted by Policeman Jenkins Server <je...@thetaphi.de>.
Build: http://jenkins.thetaphi.de/job/Lucene-Solr-trunk-Linux/6997/
Java: 64bit/jdk1.7.0_25 -XX:-UseCompressedOops -XX:+UseParallelGC
4 tests failed.
FAILED: junit.framework.TestSuite.org.apache.solr.cloud.FullSolrCloudDistribCmdsTest
Error Message:
Suite timeout exceeded (>= 7200000 msec).
Stack Trace:
java.lang.Exception: Suite timeout exceeded (>= 7200000 msec).
at __randomizedtesting.SeedInfo.seed([83718ADB2B906D29]:0)
FAILED: junit.framework.TestSuite.org.apache.solr.cloud.FullSolrCloudDistribCmdsTest
Error Message:
Captured an uncaught exception in thread: Thread[id=1640, name=NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0, state=RUNNABLE, group=TGRP-FullSolrCloudDistribCmdsTest]
Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=1640, name=NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0, state=RUNNABLE, group=TGRP-FullSolrCloudDistribCmdsTest]
Caused by: java.lang.OutOfMemoryError: GC overhead limit exceeded
at __randomizedtesting.SeedInfo.seed([83718ADB2B906D29]:0)
at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:183)
at java.lang.Thread.run(Thread.java:724)
FAILED: junit.framework.TestSuite.org.apache.solr.cloud.FullSolrCloudDistribCmdsTest
Error Message:
Captured an uncaught exception in thread: Thread[id=1649, name=qtp1079735746-1649, state=RUNNABLE, group=TGRP-FullSolrCloudDistribCmdsTest]
Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=1649, name=qtp1079735746-1649, state=RUNNABLE, group=TGRP-FullSolrCloudDistribCmdsTest]
Caused by: java.lang.OutOfMemoryError: GC overhead limit exceeded
FAILED: org.apache.solr.cloud.FullSolrCloudDistribCmdsTest.testDistribSearch
Error Message:
Test abandoned because suite timeout was reached.
Stack Trace:
java.lang.Exception: Test abandoned because suite timeout was reached.
at __randomizedtesting.SeedInfo.seed([83718ADB2B906D29]:0)
Build Log:
[...truncated 10464 lines...]
[junit4] Suite: org.apache.solr.cloud.FullSolrCloudDistribCmdsTest
[junit4] 2> 381549 T1638 oas.BaseDistributedSearchTestCase.initHostContext Setting hostContext system property: /
[junit4] 2> 381552 T1638 oas.SolrTestCaseJ4.setUp ###Starting testDistribSearch
[junit4] 2> Creating dataDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./solrtest-FullSolrCloudDistribCmdsTest-1376397621299
[junit4] 2> 381553 T1638 oasc.ZkTestServer.run STARTING ZK TEST SERVER
[junit4] 2> 381553 T1639 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
[junit4] 2> 381653 T1638 oasc.ZkTestServer.run start zk server on port:60829
[junit4] 2> 381654 T1638 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 381702 T1645 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@2cf034ac name:ZooKeeperConnection Watcher:127.0.0.1:60829 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 381702 T1638 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 381702 T1638 oascc.SolrZkClient.makePath makePath: /solr
[junit4] 2> 381707 T1638 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 381708 T1647 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@5b795762 name:ZooKeeperConnection Watcher:127.0.0.1:60829/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 381708 T1638 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 381708 T1638 oascc.SolrZkClient.makePath makePath: /collections/collection1
[junit4] 2> 381711 T1638 oascc.SolrZkClient.makePath makePath: /collections/collection1/shards
[junit4] 2> 381713 T1638 oascc.SolrZkClient.makePath makePath: /collections/control_collection
[junit4] 2> 381714 T1638 oascc.SolrZkClient.makePath makePath: /collections/control_collection/shards
[junit4] 2> 381716 T1638 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
[junit4] 2> 381716 T1638 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.xml
[junit4] 2> 381719 T1638 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
[junit4] 2> 381720 T1638 oascc.SolrZkClient.makePath makePath: /configs/conf1/schema.xml
[junit4] 2> 381722 T1638 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
[junit4] 2> 381723 T1638 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.snippet.randomindexconfig.xml
[junit4] 2> 381724 T1638 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
[junit4] 2> 381725 T1638 oascc.SolrZkClient.makePath makePath: /configs/conf1/stopwords.txt
[junit4] 2> 381727 T1638 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
[junit4] 2> 381727 T1638 oascc.SolrZkClient.makePath makePath: /configs/conf1/protwords.txt
[junit4] 2> 381729 T1638 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
[junit4] 2> 381729 T1638 oascc.SolrZkClient.makePath makePath: /configs/conf1/currency.xml
[junit4] 2> 381731 T1638 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
[junit4] 2> 381731 T1638 oascc.SolrZkClient.makePath makePath: /configs/conf1/open-exchange-rates.json
[junit4] 2> 381733 T1638 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4] 2> 381733 T1638 oascc.SolrZkClient.makePath makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4] 2> 381735 T1638 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
[junit4] 2> 381735 T1638 oascc.SolrZkClient.makePath makePath: /configs/conf1/old_synonyms.txt
[junit4] 2> 381737 T1638 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
[junit4] 2> 381737 T1638 oascc.SolrZkClient.makePath makePath: /configs/conf1/synonyms.txt
[junit4] 2> 381817 T1638 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4] 2> 381820 T1638 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:37631
[junit4] 2> 381820 T1638 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4] 2> 381820 T1638 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4] 2> 381821 T1638 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-controljetty-1376397621486
[junit4] 2> 381821 T1638 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-controljetty-1376397621486/'
[junit4] 2> 381830 T1638 oasc.ConfigSolr.fromFile Loading container configuration from /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-controljetty-1376397621486/solr.xml
[junit4] 2> 381872 T1638 oasc.CoreContainer.<init> New CoreContainer 1127103805
[junit4] 2> 381872 T1638 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-controljetty-1376397621486/]
[junit4] 2> 381873 T1638 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4] 2> 381873 T1638 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4] 2> 381874 T1638 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4] 2> 381874 T1638 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4] 2> 381874 T1638 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4] 2> 381874 T1638 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4] 2> 381874 T1638 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4] 2> 381875 T1638 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4] 2> 381875 T1638 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4] 2> 381875 T1638 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4] 2> 381878 T1638 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:60829/solr
[junit4] 2> 381879 T1638 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4] 2> 381879 T1638 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 381881 T1658 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@5cb4f976 name:ZooKeeperConnection Watcher:127.0.0.1:60829 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 381882 T1638 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 381883 T1638 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4] 2> 381886 T1638 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 381887 T1660 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@4228abeb name:ZooKeeperConnection Watcher:127.0.0.1:60829/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 381887 T1638 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 381888 T1638 oascc.SolrZkClient.makePath makePath: /overseer/queue
[junit4] 2> 381893 T1638 oascc.SolrZkClient.makePath makePath: /overseer/collection-queue-work
[junit4] 2> 381895 T1638 oascc.SolrZkClient.makePath makePath: /live_nodes
[junit4] 2> 381896 T1638 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:37631_
[junit4] 2> 381897 T1638 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:37631_
[junit4] 2> 381899 T1638 oascc.SolrZkClient.makePath makePath: /overseer_elect/election
[junit4] 2> 381902 T1638 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
[junit4] 2> 381903 T1638 oasc.Overseer.start Overseer (id=90203594509582339-127.0.0.1:37631_-n_0000000000) starting
[junit4] 2> 381905 T1638 oascc.SolrZkClient.makePath makePath: /overseer/queue-work
[junit4] 2> 381908 T1662 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
[junit4] 2> 381911 T1638 oascc.SolrZkClient.makePath makePath: /clusterstate.json
[junit4] 2> 381916 T1638 oascc.SolrZkClient.makePath makePath: /aliases.json
[junit4] 2> 381918 T1638 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 381920 T1661 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
[junit4] 2> 381924 T1663 oasc.ZkController.publish publishing core=collection1 state=down
[junit4] 2> 381924 T1663 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 381925 T1663 oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4] 2> 383423 T1661 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 383424 T1661 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4] 2> "operation":"state",
[junit4] 2> "core_node_name":"core_node1",
[junit4] 2> "numShards":"1",
[junit4] 2> "shard":null,
[junit4] 2> "roles":null,
[junit4] 2> "state":"down",
[junit4] 2> "shard_state":"active",
[junit4] 2> "core":"collection1",
[junit4] 2> "collection":"control_collection",
[junit4] 2> "shard_range":null,
[junit4] 2> "node_name":"127.0.0.1:37631_",
[junit4] 2> "base_url":"http://127.0.0.1:37631"}
[junit4] 2> 383424 T1661 oasc.Overseer$ClusterStateUpdater.createCollection Create collection control_collection with shards [shard1]
[junit4] 2> 383424 T1661 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4] 2> 383429 T1660 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 383925 T1663 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
[junit4] 2> 383925 T1663 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-controljetty-1376397621486/collection1
[junit4] 2> 383925 T1663 oasc.ZkController.createCollectionZkNode Check for collection zkNode:control_collection
[junit4] 2> 383926 T1663 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4] 2> 383926 T1663 oasc.ZkController.readConfigName Load collection config from:/collections/control_collection
[junit4] 2> 383927 T1663 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-controljetty-1376397621486/collection1/'
[junit4] 2> 383928 T1663 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-controljetty-1376397621486/collection1/lib/README' to classloader
[junit4] 2> 383928 T1663 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-controljetty-1376397621486/collection1/lib/classes/' to classloader
[junit4] 2> 383964 T1663 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4] 2> 384001 T1663 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4] 2> 384003 T1663 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4] 2> 384009 T1663 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4] 2> 384389 T1663 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4] 2> 384394 T1663 oass.IndexSchema.readSchema default search field in schema is text
[junit4] 2> 384397 T1663 oass.IndexSchema.readSchema unique key field: id
[junit4] 2> 384407 T1663 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 384410 T1663 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 384413 T1663 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4] 2> 384414 T1663 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4] 2> 384414 T1663 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4] 2> 384414 T1663 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4] 2> 384415 T1663 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4] 2> 384415 T1663 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4] 2> 384415 T1663 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4] 2> 384416 T1663 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-controljetty-1376397621486/collection1/, dataDir=./org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-1376397621299/control/data/
[junit4] 2> 384416 T1663 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@58ef38f0
[junit4] 2> 384416 T1663 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-1376397621299/control/data
[junit4] 2> 384417 T1663 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-1376397621299/control/data/index/
[junit4] 2> 384417 T1663 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-1376397621299/control/data/index' doesn't exist. Creating new index...
[junit4] 2> 384417 T1663 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-1376397621299/control/data/index
[junit4] 2> 384418 T1663 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@5d5489a9 lockFactory=org.apache.lucene.store.NativeFSLockFactory@777a9064),segFN=segments_1,generation=1}
[junit4] 2> 384419 T1663 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 384421 T1663 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4] 2> 384421 T1663 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4] 2> 384422 T1663 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 384422 T1663 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4] 2> 384422 T1663 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4] 2> 384422 T1663 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4] 2> 384423 T1663 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4] 2> 384423 T1663 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4] 2> 384423 T1663 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4] 2> 384425 T1663 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 384428 T1663 oass.SolrIndexSearcher.<init> Opening Searcher@64f9644e main
[junit4] 2> 384428 T1663 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4] 2> 384429 T1663 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4] 2> 384431 T1664 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@64f9644e main{StandardDirectoryReader(segments_1:1)}
[junit4] 2> 384431 T1663 oasc.CoreContainer.registerCore registering core: collection1
[junit4] 2> 384432 T1663 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:37631 collection:control_collection shard:shard1
[junit4] 2> 384432 T1663 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=10000&maxConnectionsPerHost=20&connTimeout=15000&socketTimeout=30000&retry=false
[junit4] 2> 384435 T1663 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leader_elect/shard1/election
[junit4] 2> 384439 T1663 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard1
[junit4] 2> 384441 T1663 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4] 2> 384442 T1663 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4] 2> 384442 T1663 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:37631/collection1/
[junit4] 2> 384442 T1663 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4] 2> 384442 T1663 oasc.SyncStrategy.syncToMe http://127.0.0.1:37631/collection1/ has no replicas
[junit4] 2> 384442 T1663 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:37631/collection1/ shard1
[junit4] 2> 384442 T1663 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leaders/shard1
[junit4] 2> 384932 T1661 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 384937 T1660 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 384948 T1663 oasc.ZkController.register We are http://127.0.0.1:37631/collection1/ and leader is http://127.0.0.1:37631/collection1/
[junit4] 2> 384949 T1663 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:37631
[junit4] 2> 384949 T1663 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4] 2> 384949 T1663 oasc.ZkController.publish publishing core=collection1 state=active
[junit4] 2> 384949 T1663 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 384951 T1663 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 384952 T1638 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1
[junit4] 2> 384952 T1638 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4] 2> 384953 T1638 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4] 2> 384956 T1638 oascsi.HttpClientUtil.createClient Creating new http client, config:
[junit4] 2> 384957 T1638 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 384959 T1667 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@3859adee name:ZooKeeperConnection Watcher:127.0.0.1:60829/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 384959 T1638 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 384960 T1638 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 384962 T1638 oasc.ChaosMonkey.monkeyLog monkey: init - expire sessions:true cause connection loss:true
[junit4] 2> 385057 T1638 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4] 2> 385059 T1638 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:39593
[junit4] 2> 385060 T1638 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4] 2> 385060 T1638 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4] 2> 385060 T1638 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-jetty1-1376397624709
[junit4] 2> 385061 T1638 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-jetty1-1376397624709/'
[junit4] 2> 385075 T1638 oasc.ConfigSolr.fromFile Loading container configuration from /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-jetty1-1376397624709/solr.xml
[junit4] 2> 385118 T1638 oasc.CoreContainer.<init> New CoreContainer 1023998742
[junit4] 2> 385118 T1638 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-jetty1-1376397624709/]
[junit4] 2> 385120 T1638 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4] 2> 385120 T1638 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4] 2> 385120 T1638 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4] 2> 385120 T1638 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4] 2> 385121 T1638 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4] 2> 385121 T1638 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4] 2> 385121 T1638 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4] 2> 385121 T1638 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4] 2> 385122 T1638 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4] 2> 385122 T1638 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4] 2> 385125 T1638 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:60829/solr
[junit4] 2> 385125 T1638 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4] 2> 385126 T1638 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 385127 T1678 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@3027ef39 name:ZooKeeperConnection Watcher:127.0.0.1:60829 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 385127 T1638 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 385129 T1638 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4] 2> 385132 T1638 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 385133 T1680 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@7e9d2736 name:ZooKeeperConnection Watcher:127.0.0.1:60829/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 385133 T1638 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 385137 T1638 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 386139 T1638 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:39593_
[junit4] 2> 386140 T1638 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:39593_
[junit4] 2> 386143 T1667 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4] 2> 386144 T1660 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4] 2> 386144 T1660 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
[junit4] 2> 386144 T1680 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4] 2> 386150 T1681 oasc.ZkController.publish publishing core=collection1 state=down
[junit4] 2> 386150 T1681 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 386151 T1681 oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4] 2> 386442 T1661 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 386443 T1661 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4] 2> "operation":"state",
[junit4] 2> "core_node_name":"core_node1",
[junit4] 2> "numShards":"1",
[junit4] 2> "shard":"shard1",
[junit4] 2> "roles":null,
[junit4] 2> "state":"active",
[junit4] 2> "shard_state":"active",
[junit4] 2> "core":"collection1",
[junit4] 2> "collection":"control_collection",
[junit4] 2> "shard_range":null,
[junit4] 2> "node_name":"127.0.0.1:37631_",
[junit4] 2> "base_url":"http://127.0.0.1:37631"}
[junit4] 2> 386446 T1661 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4] 2> "operation":"state",
[junit4] 2> "core_node_name":"core_node1",
[junit4] 2> "numShards":"2",
[junit4] 2> "shard":null,
[junit4] 2> "roles":null,
[junit4] 2> "state":"down",
[junit4] 2> "shard_state":"active",
[junit4] 2> "core":"collection1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "shard_range":null,
[junit4] 2> "node_name":"127.0.0.1:39593_",
[junit4] 2> "base_url":"http://127.0.0.1:39593"}
[junit4] 2> 386446 T1661 oasc.Overseer$ClusterStateUpdater.createCollection Create collection collection1 with shards [shard1, shard2]
[junit4] 2> 386446 T1661 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4] 2> 386449 T1667 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 386449 T1660 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 386449 T1680 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 387151 T1681 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
[junit4] 2> 387152 T1681 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-jetty1-1376397624709/collection1
[junit4] 2> 387152 T1681 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4] 2> 387153 T1681 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4] 2> 387153 T1681 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4] 2> 387154 T1681 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-jetty1-1376397624709/collection1/'
[junit4] 2> 387154 T1681 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-jetty1-1376397624709/collection1/lib/README' to classloader
[junit4] 2> 387155 T1681 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-jetty1-1376397624709/collection1/lib/classes/' to classloader
[junit4] 2> 387179 T1681 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4] 2> 387209 T1681 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4] 2> 387210 T1681 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4] 2> 387216 T1681 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4] 2> 387534 T1681 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4] 2> 387543 T1681 oass.IndexSchema.readSchema default search field in schema is text
[junit4] 2> 387546 T1681 oass.IndexSchema.readSchema unique key field: id
[junit4] 2> 387558 T1681 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 387562 T1681 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 387565 T1681 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4] 2> 387567 T1681 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4] 2> 387567 T1681 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4] 2> 387567 T1681 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4] 2> 387568 T1681 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4] 2> 387568 T1681 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4] 2> 387569 T1681 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4] 2> 387569 T1681 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-jetty1-1376397624709/collection1/, dataDir=./org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-1376397621299/jetty1/
[junit4] 2> 387569 T1681 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@58ef38f0
[junit4] 2> 387570 T1681 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-1376397621299/jetty1
[junit4] 2> 387570 T1681 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-1376397621299/jetty1/index/
[junit4] 2> 387571 T1681 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-1376397621299/jetty1/index' doesn't exist. Creating new index...
[junit4] 2> 387571 T1681 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-1376397621299/jetty1/index
[junit4] 2> 387573 T1681 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@7bc2a76a lockFactory=org.apache.lucene.store.NativeFSLockFactory@7ac4b5d4),segFN=segments_1,generation=1}
[junit4] 2> 387573 T1681 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 387578 T1681 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4] 2> 387578 T1681 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4] 2> 387579 T1681 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 387580 T1681 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4] 2> 387580 T1681 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4] 2> 387580 T1681 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4] 2> 387581 T1681 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4] 2> 387581 T1681 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4] 2> 387582 T1681 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4] 2> 387586 T1681 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 387589 T1681 oass.SolrIndexSearcher.<init> Opening Searcher@52ade616 main
[junit4] 2> 387590 T1681 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4] 2> 387590 T1681 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4] 2> 387594 T1682 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@52ade616 main{StandardDirectoryReader(segments_1:1)}
[junit4] 2> 387595 T1681 oasc.CoreContainer.registerCore registering core: collection1
[junit4] 2> 387595 T1681 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:39593 collection:collection1 shard:shard1
[junit4] 2> 387596 T1681 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=10000&maxConnectionsPerHost=20&connTimeout=15000&socketTimeout=30000&retry=false
[junit4] 2> 387599 T1681 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
[junit4] 2> 387604 T1681 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard1
[junit4] 2> 387606 T1681 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4] 2> 387606 T1681 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4] 2> 387606 T1681 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:39593/collection1/
[junit4] 2> 387606 T1681 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4] 2> 387606 T1681 oasc.SyncStrategy.syncToMe http://127.0.0.1:39593/collection1/ has no replicas
[junit4] 2> 387607 T1681 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:39593/collection1/ shard1
[junit4] 2> 387607 T1681 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
[junit4] 2> 387954 T1661 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 387963 T1667 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 387963 T1680 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 387963 T1660 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 387981 T1681 oasc.ZkController.register We are http://127.0.0.1:39593/collection1/ and leader is http://127.0.0.1:39593/collection1/
[junit4] 2> 387982 T1681 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:39593
[junit4] 2> 387982 T1681 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4] 2> 387982 T1681 oasc.ZkController.publish publishing core=collection1 state=active
[junit4] 2> 387982 T1681 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 387984 T1681 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 387985 T1638 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1
[junit4] 2> 387986 T1638 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4] 2> 387986 T1638 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4] 2> 388085 T1638 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4] 2> 388088 T1638 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:37559
[junit4] 2> 388088 T1638 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4] 2> 388089 T1638 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4] 2> 388089 T1638 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-jetty2-1376397627734
[junit4] 2> 388090 T1638 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-jetty2-1376397627734/'
[junit4] 2> 388104 T1638 oasc.ConfigSolr.fromFile Loading container configuration from /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-jetty2-1376397627734/solr.xml
[junit4] 2> 388144 T1638 oasc.CoreContainer.<init> New CoreContainer 1182590131
[junit4] 2> 388145 T1638 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-jetty2-1376397627734/]
[junit4] 2> 388146 T1638 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4] 2> 388146 T1638 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4] 2> 388146 T1638 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4] 2> 388147 T1638 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4] 2> 388147 T1638 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4] 2> 388147 T1638 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4] 2> 388147 T1638 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4] 2> 388148 T1638 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4] 2> 388148 T1638 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4] 2> 388148 T1638 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4] 2> 388151 T1638 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:60829/solr
[junit4] 2> 388152 T1638 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4] 2> 388152 T1638 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 388154 T1694 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@7abdf0ff name:ZooKeeperConnection Watcher:127.0.0.1:60829 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 388154 T1638 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 388156 T1638 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4] 2> 388159 T1638 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 388160 T1696 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@4ab38c3 name:ZooKeeperConnection Watcher:127.0.0.1:60829/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 388160 T1638 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 388164 T1638 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 389167 T1638 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:37559_
[junit4] 2> 389168 T1638 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:37559_
[junit4] 2> 389170 T1667 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
[junit4] 2> 389171 T1660 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4] 2> 389171 T1660 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4] 2> 389171 T1696 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4] 2> 389171 T1680 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4] 2> 389172 T1680 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4] 2> 389173 T1667 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4] 2> 389177 T1697 oasc.ZkController.publish publishing core=collection1 state=down
[junit4] 2> 389177 T1697 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 389178 T1697 oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4] 2> 389467 T1661 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 389468 T1661 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4] 2> "operation":"state",
[junit4] 2> "core_node_name":"core_node1",
[junit4] 2> "numShards":"2",
[junit4] 2> "shard":"shard1",
[junit4] 2> "roles":null,
[junit4] 2> "state":"active",
[junit4] 2> "shard_state":"active",
[junit4] 2> "core":"collection1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "shard_range":null,
[junit4] 2> "node_name":"127.0.0.1:39593_",
[junit4] 2> "base_url":"http://127.0.0.1:39593"}
[junit4] 2> 389471 T1661 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4] 2> "operation":"state",
[junit4] 2> "core_node_name":"core_node2",
[junit4] 2> "numShards":"2",
[junit4] 2> "shard":null,
[junit4] 2> "roles":null,
[junit4] 2> "state":"down",
[junit4] 2> "shard_state":"active",
[junit4] 2> "core":"collection1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "shard_range":null,
[junit4] 2> "node_name":"127.0.0.1:37559_",
[junit4] 2> "base_url":"http://127.0.0.1:37559"}
[junit4] 2> 389471 T1661 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=2
[junit4] 2> 389471 T1661 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard2
[junit4] 2> 389534 T1667 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 389534 T1680 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 389534 T1696 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 389534 T1660 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 390178 T1697 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
[junit4] 2> 390179 T1697 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-jetty2-1376397627734/collection1
[junit4] 2> 390179 T1697 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4] 2> 390179 T1697 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4] 2> 390180 T1697 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4] 2> 390180 T1697 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-jetty2-1376397627734/collection1/'
[junit4] 2> 390181 T1697 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-jetty2-1376397627734/collection1/lib/README' to classloader
[junit4] 2> 390181 T1697 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-jetty2-1376397627734/collection1/lib/classes/' to classloader
[junit4] 2> 390207 T1697 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4] 2> 390239 T1697 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4] 2> 390240 T1697 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4] 2> 390247 T1697 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4] 2> 390563 T1697 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4] 2> 390569 T1697 oass.IndexSchema.readSchema default search field in schema is text
[junit4] 2> 390571 T1697 oass.IndexSchema.readSchema unique key field: id
[junit4] 2> 390580 T1697 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 390583 T1697 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 390586 T1697 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4] 2> 390587 T1697 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4] 2> 390587 T1697 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4] 2> 390587 T1697 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4] 2> 390588 T1697 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4] 2> 390588 T1697 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4] 2> 390588 T1697 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4] 2> 390588 T1697 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-jetty2-1376397627734/collection1/, dataDir=./org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-1376397621299/jetty2/
[junit4] 2> 390588 T1697 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@58ef38f0
[junit4] 2> 390589 T1697 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-1376397621299/jetty2
[junit4] 2> 390589 T1697 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-1376397621299/jetty2/index/
[junit4] 2> 390589 T1697 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-1376397621299/jetty2/index' doesn't exist. Creating new index...
[junit4] 2> 390590 T1697 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-1376397621299/jetty2/index
[junit4] 2> 390591 T1697 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@400d7fbb lockFactory=org.apache.lucene.store.NativeFSLockFactory@7c55f31a),segFN=segments_1,generation=1}
[junit4] 2> 390591 T1697 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 390594 T1697 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4] 2> 390594 T1697 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4] 2> 390595 T1697 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 390595 T1697 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4] 2> 390596 T1697 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4] 2> 390596 T1697 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4] 2> 390596 T1697 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4] 2> 390597 T1697 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4] 2> 390597 T1697 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4] 2> 390599 T1697 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 390601 T1697 oass.SolrIndexSearcher.<init> Opening Searcher@13dc4803 main
[junit4] 2> 390602 T1697 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4] 2> 390602 T1697 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4] 2> 390605 T1698 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@13dc4803 main{StandardDirectoryReader(segments_1:1)}
[junit4] 2> 390607 T1697 oasc.CoreContainer.registerCore registering core: collection1
[junit4] 2> 390607 T1697 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:37559 collection:collection1 shard:shard2
[junit4] 2> 390607 T1697 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=10000&maxConnectionsPerHost=20&connTimeout=15000&socketTimeout=30000&retry=false
[junit4] 2> 390610 T1697 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard2/election
[junit4] 2> 390615 T1697 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard2
[junit4] 2> 390617 T1697 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4] 2> 390617 T1697 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4] 2> 390617 T1697 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:37559/collection1/
[junit4] 2> 390617 T1697 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4] 2> 390617 T1697 oasc.SyncStrategy.syncToMe http://127.0.0.1:37559/collection1/ has no replicas
[junit4] 2> 390617 T1697 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:37559/collection1/ shard2
[junit4] 2> 390617 T1697 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard2
[junit4] 2> 391038 T1661 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 391045 T1667 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 391045 T1660 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 391045 T1680 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 391045 T1696 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 391072 T1697 oasc.ZkController.register We are http://127.0.0.1:37559/collection1/ and leader is http://127.0.0.1:37559/collection1/
[junit4] 2> 391072 T1697 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:37559
[junit4] 2> 391072 T1697 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4] 2> 391073 T1697 oasc.ZkController.publish publishing core=collection1 state=active
[junit4] 2> 391073 T1697 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 391074 T1697 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 391075 T1638 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1
[junit4] 2> 391076 T1638 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4] 2> 391076 T1638 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4] 2> 391158 T1638 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4] 2> 391160 T1638 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:53531
[junit4] 2> 391160 T1638 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4] 2> 391161 T1638 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4] 2> 391161 T1638 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-jetty3-1376397630825
[junit4] 2> 391161 T1638 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-jetty3-1376397630825/'
[junit4] 2> 391171 T1638 oasc.ConfigSolr.fromFile Loading container configuration from /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-jetty3-1376397630825/solr.xml
[junit4] 2> 391200 T1638 oasc.CoreContainer.<init> New CoreContainer 1923809468
[junit4] 2> 391200 T1638 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-jetty3-1376397630825/]
[junit4] 2> 391201 T1638 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4] 2> 391201 T1638 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4] 2> 391202 T1638 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4] 2> 391202 T1638 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4] 2> 391202 T1638 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4] 2> 391202 T1638 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4] 2> 391203 T1638 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4] 2> 391203 T1638 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4] 2> 391203 T1638 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4] 2> 391203 T1638 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4] 2> 391206 T1638 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:60829/solr
[junit4] 2> 391206 T1638 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4] 2> 391207 T1638 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 391208 T1710 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@747abb3c name:ZooKeeperConnection Watcher:127.0.0.1:60829 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 391209 T1638 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 391211 T1638 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4] 2> 391214 T1638 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 391215 T1712 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@d02ce83 name:ZooKeeperConnection Watcher:127.0.0.1:60829/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 391215 T1638 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 391218 T1638 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 392221 T1638 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:53531_
[junit4] 2> 392222 T1638 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:53531_
[junit4] 2> 392223 T1667 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4] 2> 392224 T1696 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4] 2> 392224 T1660 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4] 2> 392224 T1680 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4] 2> 392225 T1712 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4] 2> 392225 T1660 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4] 2> 392225 T1680 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4] 2> 392226 T1696 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4] 2> 392226 T1667 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4] 2> 392230 T1713 oasc.ZkController.publish publishing core=collection1 state=down
[junit4] 2> 392231 T1713 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 392232 T1713 oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4] 2> 392551 T1661 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 392553 T1661 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4] 2> "operation":"state",
[junit4] 2> "core_node_name":"core_node2",
[junit4] 2> "numShards":"2",
[junit4] 2> "shard":"shard2",
[junit4] 2> "roles":null,
[junit4] 2> "state":"active",
[junit4] 2> "shard_state":"active",
[junit4] 2> "core":"collection1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "shard_range":null,
[junit4] 2> "node_name":"127.0.0.1:37559_",
[junit4] 2> "base_url":"http://127.0.0.1:37559"}
[junit4] 2> 392556 T1661 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4] 2> "operation":"state",
[junit4] 2> "core_node_name":"core_node3",
[junit4] 2> "numShards":"2",
[junit4] 2> "shard":null,
[junit4] 2> "roles":null,
[junit4] 2> "state":"down",
[junit4] 2> "shard_state":"active",
[junit4] 2> "core":"collection1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "shard_range":null,
[junit4] 2> "node_name":"127.0.0.1:53531_",
[junit4] 2> "base_url":"http://127.0.0.1:53531"}
[junit4] 2> 392556 T1661 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=2
[junit4] 2> 392556 T1661 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4] 2> 392559 T1667 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4] 2> 392559 T1696 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4] 2> 392559 T1660 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4] 2> 392559 T1712 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4] 2> 392559 T1680 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4] 2> 393232 T1713 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
[junit4] 2> 393232 T1713 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-jetty3-1376397630825/collection1
[junit4] 2> 393232 T1713 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4] 2> 393233 T1713 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4] 2> 393233 T1713 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4] 2> 393234 T1713 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-jetty3-1376397630825/collection1/'
[junit4] 2> 393235 T1713 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-jetty3-1376397630825/collection1/lib/README' to classloader
[junit4] 2> 393235 T1713 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-jetty3-1376397630825/collection1/lib/classes/' to classloader
[junit4] 2> 393260 T1713 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4] 2> 393288 T1713 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4] 2> 393289 T1713 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4] 2> 393295 T1713 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4] 2> 393587 T1713 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4] 2> 393592 T1713 oass.IndexSchema.readSchema default search field in schema is text
[junit4] 2> 393594 T1713 oass.IndexSchema.readSchema unique key field: id
[junit4] 2> 393604 T1713 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 393607 T1713 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 393610 T1713 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4] 2> 393610 T1713 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4] 2> 393611 T1713 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4] 2> 393611 T1713 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4] 2> 393611 T1713 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4] 2> 393612 T1713 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4] 2> 393612 T1713 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4] 2> 393612 T1713 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-jetty3-1376397630825/collection1/, dataDir=./org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-1376397621299/jetty3/
[junit4] 2> 393612 T1713 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@58ef38f0
[junit4] 2> 393613 T1713 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-1376397621299/jetty3
[junit4] 2> 393613 T1713 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-1376397621299/jetty3/index/
[junit4] 2> 393613 T1713 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-1376397621299/jetty3/index' doesn't exist. Creating new index...
[junit4] 2> 393613 T1713 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-1376397621299/jetty3/index
[junit4] 2> 393615 T1713 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@35006fa3 lockFactory=org.apache.lucene.store.NativeFSLockFactory@4df18717),segFN=segments_1,generation=1}
[junit4] 2> 393615 T1713 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 393618 T1713 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4] 2> 393618 T1713 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4] 2> 393618 T1713 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 393619 T1713 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4] 2> 393619 T1713 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4] 2> 393619 T1713 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4] 2> 393619 T1713 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4] 2> 393620 T1713 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4] 2> 393620 T1713 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4] 2> 393622 T1713 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 393624 T1713 oass.SolrIndexSearcher.<init> Opening Searcher@bef0e4f main
[junit4] 2> 393625 T1713 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4] 2> 393625 T1713 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4] 2> 393628 T1714 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@bef0e4f main{StandardDirectoryReader(segments_1:1)}
[junit4] 2> 393630 T1713 oasc.CoreContainer.registerCore registering core: collection1
[junit4] 2> 393630 T1713 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:53531 collection:collection1 shard:shard1
[junit4] 2> 393630 T1713 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=10000&maxConnectionsPerHost=20&connTimeout=15000&socketTimeout=30000&retry=false
[junit4] 2> 393635 T1713 oasc.ZkController.register We are http://127.0.0.1:53531/collection1/ and leader is http://127.0.0.1:39593/collection1/
[junit4] 2> 393635 T1713 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:53531
[junit4] 2> 393635 T1713 oasc.ZkController.checkRecovery Core needs to recover:collection1
[junit4] 2> 393635 T1713 oasu.DefaultSolrCoreState.doRecovery Running recovery - first canceling any ongoing recovery
[junit4] 2> ASYNC NEW_CORE C523 name=collection1 org.apache.solr.core.SolrCore@132e86ed url=http://127.0.0.1:53531/collection1 node=127.0.0.1:53531_ C523_STATE=coll:collection1 core:collection1 props:{state=down, core=collection1, node_name=127.0.0.1:53531_, base_url=http://127.0.0.1:53531}
[junit4] 2> 393636 T1715 C523 P53531 oasc.RecoveryStrategy.run Starting recovery process. core=collection1 recoveringAfterStartup=true
[junit4] 2> 393636 T1713 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 393636 T1715 C523 P53531 oasc.RecoveryStrategy.doRecovery ###### startupVersions=[]
[junit4] 2> 393637 T1715 C523 P53531 oasc.ZkController.publish publishing core=collection1 state=recovering
[junit4] 2> 393637 T1715 C523 P53531 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 393637 T1638 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1
[junit4] 2> 393637 T1638 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4] 2> 393638 T1715 C523 P53531 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4] 2> 393638 T1638 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4] 2> 393642 T1674 oasha.CoreAdminHandler.handleWaitForStateAction Going to wait for coreNodeName: core_node3, state: recovering, checkLive: true, onlyIfLeader: true
[junit4] 2> 393703 T1638 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4] 2> 393705 T1638 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:43935
[junit4] 2> 393705 T1638 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4] 2> 393705 T1638 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4] 2> 393706 T1638 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-jetty4-1376397633387
[junit4] 2> 393706 T1638 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-jetty4-1376397633387/'
[junit4] 2> 393715 T1638 oasc.ConfigSolr.fromFile Loading container configuration from /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-jetty4-1376397633387/solr.xml
[junit4] 2> 393745 T1638 oasc.CoreContainer.<init> New CoreContainer 1995361181
[junit4] 2> 393745 T1638 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-jetty4-1376397633387/]
[junit4] 2> 393746 T1638 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4] 2> 393746 T1638 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4] 2> 393746 T1638 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4] 2> 393747 T1638 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4] 2> 393747 T1638 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4] 2> 393747 T1638 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4] 2> 393747 T1638 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4] 2> 393748 T1638 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4] 2> 393748 T1638 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4] 2> 393748 T1638 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4] 2> 393751 T1638 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:60829/solr
[junit4] 2> 393751 T1638 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4] 2> 393752 T1638 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 393753 T1727 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@850d6cd name:ZooKeeperConnection Watcher:127.0.0.1:60829 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 393753 T1638 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 393755 T1638 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4] 2> 393759 T1638 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 393760 T1729 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@51e4992e name:ZooKeeperConnection Watcher:127.0.0.1:60829/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 393760 T1638 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 393763 T1638 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 394066 T1661 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 394067 T1661 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4] 2> "operation":"state",
[junit4] 2> "core_node_name":"core_node3",
[junit4] 2> "numShards":"2",
[junit4] 2> "shard":"shard1",
[junit4] 2> "roles":null,
[junit4] 2> "state":"recovering",
[junit4] 2> "shard_state":"active",
[junit4] 2> "core":"collection1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "shard_range":null,
[junit4] 2> "node_name":"127.0.0.1:53531_",
[junit4] 2> "base_url":"http://127.0.0.1:53531"}
[junit4] 2> 394070 T1712 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4] 2> 394070 T1696 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4] 2> 394070 T1667 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4] 2> 394070 T1660 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4] 2> 394070 T1729 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4] 2> 394070 T1680 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4] 2> 394642 T1674 oasha.CoreAdminHandler.handleWaitForStateAction Waited coreNodeName: core_node3, state: recovering, checkLive: true, onlyIfLeader: true for: 1 seconds.
[junit4] 2> 394643 T1674 oass.SolrDispatchFilter.handleAdminRequest [admin] webapp=null path=/admin/cores params={coreNodeName=core_node3&state=recovering&nodeName=127.0.0.1:53531_&action=PREPRECOVERY&checkLive=true&core=collection1&wt=javabin&onlyIfLeader=true&version=2} status=0 QTime=1001
[junit4] 2> 394766 T1638 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:43935_
[junit4] 2> 394767 T1638 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:43935_
[junit4] 2> 394769 T1667 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4] 2> 394769 T1729 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4] 2> 394769 T1696 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4] 2> 394769 T1712 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4] 2> 394769 T1660 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4] 2> 394769 T1712 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4] 2> 394770 T1680 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4] 2> 394770 T1680 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4] 2> 394770 T1660 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4] 2> 394771 T1696 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4] 2> 394771 T1667 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4] 2> 394771 T1729 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4] 2> 394776 T1730 oasc.ZkController.publish publishing core=collection1 state=down
[junit4] 2> 394776 T1730 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 394777 T1730 oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4] 2> 395576 T1661 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 395578 T1661 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4] 2> "operation":"state",
[junit4] 2> "core_node_name":"core_node4",
[junit4] 2> "numShards":"2",
[junit4] 2> "shard":null,
[junit4] 2> "roles":null,
[junit4] 2> "state":"down",
[junit4] 2> "shard_state":"active",
[junit4] 2> "core":"collection1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "shard_range":null,
[junit4] 2> "node_name":"127.0.0.1:43935_",
[junit4] 2> "base_url":"http://127.0.0.1:43935"}
[junit4] 2> 395578 T1661 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=2
[junit4] 2> 395578 T1661 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard2
[junit4] 2> 395582 T1712 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4] 2> 395582 T1680 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4] 2> 395582 T1660 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4] 2> 395582 T1729 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4] 2> 395582 T1696 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4] 2> 395582 T1667 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4] 2> 395777 T1730 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
[junit4] 2> 395777 T1730 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.FullSolrCloudDistribCmdsTest-jetty4-1376397633387/collection1
[junit4] 2> 395777 T1730 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4] 2> 395778 T1730 oasc.ZkController.createCollectionZkNode Collection zk
[...truncated too long message...]
il.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
[junit4] 2> at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
[junit4] 2> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
[junit4] 2> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
[junit4] 2> at java.lang.Thread.run(Thread.java:724)
[junit4] 2> 36) Thread[id=1684, name=HashSessionScavenger-59, state=TIMED_WAITING, group=TGRP-FullSolrCloudDistribCmdsTest]
[junit4] 2> at java.lang.Object.wait(Native Method)
[junit4] 2> at java.util.TimerThread.mainLoop(Timer.java:552)
[junit4] 2> at java.util.TimerThread.run(Timer.java:505)
[junit4] 2> 37) Thread[id=1700, name=HashSessionScavenger-60, state=TIMED_WAITING, group=TGRP-FullSolrCloudDistribCmdsTest]
[junit4] 2> at java.lang.Object.wait(Native Method)
[junit4] 2> at java.util.TimerThread.mainLoop(Timer.java:552)
[junit4] 2> at java.util.TimerThread.run(Timer.java:505)
[junit4] 2> 38) Thread[id=1704, name=qtp1361015859-1704 Acceptor1 SelectChannelConnector@127.0.0.1:53531, state=RUNNABLE, group=TGRP-FullSolrCloudDistribCmdsTest]
[junit4] 2> at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:956)
[junit4] 2> at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
[junit4] 2> at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
[junit4] 2> at java.lang.Thread.run(Thread.java:724)
[junit4] 2> 39) Thread[id=1659, name=TEST-FullSolrCloudDistribCmdsTest.testDistribSearch-seed#[83718ADB2B906D29]-SendThread(localhost.localdomain:60829), state=TIMED_WAITING, group=TGRP-FullSolrCloudDistribCmdsTest]
[junit4] 2> at java.lang.Thread.sleep(Native Method)
[junit4] 2> at org.apache.zookeeper.client.StaticHostProvider.next(StaticHostProvider.java:86)
[junit4] 2> at org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:937)
[junit4] 2> at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:993)
[junit4] 2> 40) Thread[id=1672, name=qtp290512899-1672 Acceptor1 SelectChannelConnector@127.0.0.1:39593, state=RUNNABLE, group=TGRP-FullSolrCloudDistribCmdsTest]
[junit4] 2> at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:956)
[junit4] 2> at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
[junit4] 2> at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
[junit4] 2> at java.lang.Thread.run(Thread.java:724)
[junit4] 2> 41) Thread[id=1670, name=qtp290512899-1670 Selector1, state=RUNNABLE, group=TGRP-FullSolrCloudDistribCmdsTest]
[junit4] 2> at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
[junit4] 2> at sun.nio.ch.SelectorImpl.selectNow(SelectorImpl.java:106)
[junit4] 2> at org.eclipse.jetty.io.nio.SelectorManager$SelectSet.doSelect(SelectorManager.java:536)
[junit4] 2> at org.eclipse.jetty.io.nio.SelectorManager$1.run(SelectorManager.java:290)
[junit4] 2> at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
[junit4] 2> at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
[junit4] 2> at java.lang.Thread.run(Thread.java:724)
[junit4] 2> 7606968 T1659 oaz.ClientCnxn$SendThread.run WARN Session 0x14077b0f8350003 for server null, unexpected error, closing socket connection and attempting reconnect java.net.ConnectException: Connection refused
[junit4] 2> at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
[junit4] 2> at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:708)
[junit4] 2> at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:350)
[junit4] 2> at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1068)
[junit4] 2>
[junit4] 2> NOTE: test params are: codec=Lucene42: {range_facet_l=PostingsFormat(name=Memory doPackFST= true), n_ti1=Lucene41(blocksize=128), n_f1=PostingsFormat(name=SimpleText), text=Pulsing41(freqCutoff=11 minBlockSize=37 maxBlockSize=125), _version_=PostingsFormat(name=SimpleText), n_tl1=PostingsFormat(name=SimpleText), n_d1=PostingsFormat(name=Memory doPackFST= true), rnd_b=Lucene41(blocksize=128), intDefault=PostingsFormat(name=SimpleText), n_td1=PostingsFormat(name=SimpleText), timestamp=PostingsFormat(name=Memory doPackFST= true), id=Lucene41(blocksize=128), range_facet_sl=Lucene41(blocksize=128), a_t=PostingsFormat(name=Memory doPackFST= true), n_tdt1=PostingsFormat(name=SimpleText), range_facet_si=PostingsFormat(name=Memory doPackFST= true), n_l1=PostingsFormat(name=Memory doPackFST= true), multiDefault=PostingsFormat(name=Memory doPackFST= true), n_tf1=PostingsFormat(name=Memory doPackFST= true), n_dt1=PostingsFormat(name=SimpleText)}, docValues:{timestamp=DocValuesFormat(name=Lucene42)}, sim=DefaultSimilarity, locale=da, timezone=America/Montreal
[junit4] 2> NOTE: Linux 3.2.0-51-generic amd64/Oracle Corporation 1.7.0_25 (64-bit)/cpus=8,threads=48,free=72737424,total=477233152
[junit4] 2> NOTE: All tests run in this JVM: [TestCSVResponseWriter, TestIndexingPerformance, StandardRequestHandlerTest, PluginInfoTest, SpellingQueryConverterTest, ClusterStateTest, TestDynamicFieldResource, TestSchemaVersionResource, UUIDFieldTest, TestAtomicUpdateErrorCases, TestUpdate, DirectSolrSpellCheckerTest, TestComponentsName, IndexBasedSpellCheckerTest, TestZkChroot, DisMaxRequestHandlerTest, TestSolrQueryParserResource, ResourceLoaderTest, TestPartialUpdateDeduplication, BasicZkTest, TestClassNameShortening, SuggesterTSTTest, CoreContainerCoreInitFailuresTest, TestCloudManagedSchema, SuggesterWFSTTest, OutputWriterTest, SignatureUpdateProcessorFactoryTest, SpellCheckCollatorTest, TestManagedSchema, CustomCollectionTest, TestIndexSearcher, HdfsBasicDistributedZk2Test, PreAnalyzedUpdateProcessorTest, IndexSchemaRuntimeFieldTest, DefaultValueUpdateProcessorTest, NotRequiredUniqueKeyTest, SolrPluginUtilsTest, TestCollationField, RequestHandlersTest, FieldAnalysisRequestHandlerTest, TestStressRecovery, TestBinaryResponseWriter, TestJmxIntegration, XmlUpdateRequestHandlerTest, XsltUpdateRequestHandlerTest, TestCSVLoader, TestSchemaSimilarityResource, TestCopyFieldCollectionResource, TestFiltering, TestFastWriter, TestDistributedSearch, SpellCheckComponentTest, TestIBSimilarityFactory, TestCodecSupport, TestRandomMergePolicy, TestInfoStreamLogging, SolrCmdDistributorTest, TestRangeQuery, TermVectorComponentDistributedTest, TestOmitPositions, HighlighterConfigTest, BadCopyFieldTest, TestNumberUtils, TestXIncludeConfig, TestFastOutputStream, TestPluginEnable, DirectSolrConnectionTest, DebugComponentTest, BadIndexSchemaTest, TestDistributedGrouping, TestManagedSchemaFieldResource, PreAnalyzedFieldTest, TestUtils, HighlighterTest, FullSolrCloudDistribCmdsTest]
[junit4] 2> NOTE: reproduce with: ant test -Dtestcase=FullSolrCloudDistribCmdsTest -Dtests.seed=83718ADB2B906D29 -Dtests.multiplier=3 -Dtests.slow=true -Dtests.locale=da -Dtests.timezone=America/Montreal -Dtests.file.encoding=UTF-8
[junit4] ERROR 0.00s J1 | FullSolrCloudDistribCmdsTest (suite) <<<
[junit4] > Throwable #1: java.lang.Exception: Suite timeout exceeded (>= 7200000 msec).
[junit4] > at __randomizedtesting.SeedInfo.seed([83718ADB2B906D29]:0)Throwable #2: com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=1640, name=NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0, state=RUNNABLE, group=TGRP-FullSolrCloudDistribCmdsTest]
[junit4] > Caused by: java.lang.OutOfMemoryError: GC overhead limit exceeded
[junit4] > at __randomizedtesting.SeedInfo.seed([83718ADB2B906D29]:0)
[junit4] > at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:183)
[junit4] > at java.lang.Thread.run(Thread.java:724)Throwable #3: com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=1649, name=qtp1079735746-1649, state=RUNNABLE, group=TGRP-FullSolrCloudDistribCmdsTest]
[junit4] > Caused by: java.lang.OutOfMemoryError: GC overhead limit exceeded
[junit4] Completed on J1 in 7225.45s, 1 test, 4 errors <<< FAILURES!
[...truncated 29 lines...]
BUILD FAILED
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/build.xml:389: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/build.xml:369: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/build.xml:39: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build.xml:181: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/common-build.xml:450: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/common-build.xml:1230: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/common-build.xml:873: There were test failures: 320 suites, 1360 tests, 3 suite-level errors, 1 error, 40 ignored (6 assumptions)
Total time: 144 minutes 57 seconds
Build step 'Invoke Ant' marked build as failure
Description set: Java: 64bit/jdk1.7.0_25 -XX:-UseCompressedOops -XX:+UseParallelGC
Archiving artifacts
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure