You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by Policeman Jenkins Server <je...@thetaphi.de> on 2013/07/14 21:26:05 UTC
[JENKINS] Lucene-Solr-4.x-Linux (32bit/ibm-j9-jdk7) - Build # 6487
- Failure!
Build: http://jenkins.thetaphi.de/job/Lucene-Solr-4.x-Linux/6487/
Java: 32bit/ibm-j9-jdk7 -Xjit:exclude={org/apache/lucene/util/fst/FST.pack(IIF)Lorg/apache/lucene/util/fst/FST;}
1 tests failed.
REGRESSION: org.apache.solr.cloud.BasicDistributedZk2Test.testDistribSearch
Error Message:
Server at http://127.0.0.1:47766/onenodecollectioncore returned non ok status:404, message:Can not find: /onenodecollectioncore/update
Stack Trace:
org.apache.solr.client.solrj.impl.HttpSolrServer$RemoteSolrException: Server at http://127.0.0.1:47766/onenodecollectioncore returned non ok status:404, message:Can not find: /onenodecollectioncore/update
at __randomizedtesting.SeedInfo.seed([119935A2FA79F23E:907FBBBA8D269202]:0)
at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:385)
at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:180)
at org.apache.solr.client.solrj.request.AbstractUpdateRequest.process(AbstractUpdateRequest.java:117)
at org.apache.solr.client.solrj.SolrServer.add(SolrServer.java:116)
at org.apache.solr.client.solrj.SolrServer.add(SolrServer.java:102)
at org.apache.solr.cloud.BasicDistributedZk2Test.testNodeWithoutCollectionForwarding(BasicDistributedZk2Test.java:196)
at org.apache.solr.cloud.BasicDistributedZk2Test.doTest(BasicDistributedZk2Test.java:88)
at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:835)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:88)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:55)
at java.lang.reflect.Method.invoke(Method.java:613)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1559)
at com.carrotsearch.randomizedtesting.RandomizedRunner.access$600(RandomizedRunner.java:79)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:773)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:787)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:782)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:442)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:746)
at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:648)
at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:682)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:693)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
at java.lang.Thread.run(Thread.java:780)
Build Log:
[...truncated 9567 lines...]
[junit4] Suite: org.apache.solr.cloud.BasicDistributedZk2Test
[junit4] 2> 179250 T650 oas.BaseDistributedSearchTestCase.initHostContext Setting hostContext system property: /
[junit4] 2> 179255 T650 oas.SolrTestCaseJ4.setUp ###Starting testDistribSearch
[junit4] 2> Creating dataDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./solrtest-BasicDistributedZk2Test-1373828732652
[junit4] 2> 179256 T650 oasc.ZkTestServer.run STARTING ZK TEST SERVER
[junit4] 2> 179256 T651 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
[junit4] 2> 179356 T650 oasc.ZkTestServer.run start zk server on port:58961
[junit4] 2> 179357 T650 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 179450 T657 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@4fe1ad62 name:ZooKeeperConnection Watcher:127.0.0.1:58961 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 179450 T650 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 179451 T650 oascc.SolrZkClient.makePath makePath: /solr
[junit4] 2> 179458 T650 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 179460 T659 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@e395c235 name:ZooKeeperConnection Watcher:127.0.0.1:58961/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 179461 T650 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 179462 T650 oascc.SolrZkClient.makePath makePath: /collections/collection1
[junit4] 2> 179467 T650 oascc.SolrZkClient.makePath makePath: /collections/collection1/shards
[junit4] 2> 179471 T650 oascc.SolrZkClient.makePath makePath: /collections/control_collection
[junit4] 2> 179473 T650 oascc.SolrZkClient.makePath makePath: /collections/control_collection/shards
[junit4] 2> 179476 T650 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
[junit4] 2> 179476 T650 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.xml
[junit4] 2> 179480 T650 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
[junit4] 2> 179481 T650 oascc.SolrZkClient.makePath makePath: /configs/conf1/schema.xml
[junit4] 2> 179483 T650 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
[junit4] 2> 179484 T650 oascc.SolrZkClient.makePath makePath: /configs/conf1/stopwords.txt
[junit4] 2> 179486 T650 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
[junit4] 2> 179487 T650 oascc.SolrZkClient.makePath makePath: /configs/conf1/protwords.txt
[junit4] 2> 179489 T650 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
[junit4] 2> 179490 T650 oascc.SolrZkClient.makePath makePath: /configs/conf1/currency.xml
[junit4] 2> 179492 T650 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
[junit4] 2> 179493 T650 oascc.SolrZkClient.makePath makePath: /configs/conf1/open-exchange-rates.json
[junit4] 2> 179495 T650 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4] 2> 179496 T650 oascc.SolrZkClient.makePath makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4] 2> 179499 T650 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
[junit4] 2> 179499 T650 oascc.SolrZkClient.makePath makePath: /configs/conf1/old_synonyms.txt
[junit4] 2> 179502 T650 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
[junit4] 2> 179502 T650 oascc.SolrZkClient.makePath makePath: /configs/conf1/synonyms.txt
[junit4] 2> 179586 T650 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4] 2> 179590 T650 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:55788
[junit4] 2> 179590 T650 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4] 2> 179591 T650 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4] 2> 179592 T650 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-controljetty-1373828732902
[junit4] 2> 179592 T650 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-controljetty-1373828732902/'
[junit4] 2> 179600 T650 oasc.ConfigSolr.fromFile Loading container configuration from /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-controljetty-1373828732902/solr.xml
[junit4] 2> 179631 T650 oasc.CoreContainer.<init> New CoreContainer 1111087049
[junit4] 2> 179632 T650 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-controljetty-1373828732902/]
[junit4] 2> 179633 T650 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4] 2> 179634 T650 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4] 2> 179634 T650 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4] 2> 179635 T650 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4] 2> 179635 T650 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4] 2> 179636 T650 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4] 2> 179637 T650 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4] 2> 179637 T650 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4] 2> 179638 T650 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4] 2> 179638 T650 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4] 2> 179643 T650 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
[junit4] 2> 179643 T650 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:58961/solr
[junit4] 2> 179644 T650 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4] 2> 179645 T650 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 179646 T670 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@9f040e31 name:ZooKeeperConnection Watcher:127.0.0.1:58961 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 179647 T650 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 179648 T650 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4] 2> 179653 T650 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 179655 T672 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@b949e0ec name:ZooKeeperConnection Watcher:127.0.0.1:58961/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 179655 T650 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 179656 T650 oascc.SolrZkClient.makePath makePath: /overseer/queue
[junit4] 2> 179659 T650 oascc.SolrZkClient.makePath makePath: /overseer/collection-queue-work
[junit4] 2> 179662 T650 oascc.SolrZkClient.makePath makePath: /live_nodes
[junit4] 2> 179664 T650 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:55788_
[junit4] 2> 179665 T650 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:55788_
[junit4] 2> 179668 T650 oascc.SolrZkClient.makePath makePath: /overseer_elect/election
[junit4] 2> 179672 T650 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
[junit4] 2> 179674 T650 oasc.Overseer.start Overseer (id=90035239823278083-127.0.0.1:55788_-n_0000000000) starting
[junit4] 2> 179677 T650 oascc.SolrZkClient.makePath makePath: /overseer/queue-work
[junit4] 2> 179680 T674 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
[junit4] 2> 179681 T650 oascc.SolrZkClient.makePath makePath: /clusterstate.json
[junit4] 2> 179683 T650 oascc.SolrZkClient.makePath makePath: /aliases.json
[junit4] 2> 179684 T650 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 179687 T673 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
[junit4] 2> 179692 T675 oasc.ZkController.publish publishing core=collection1 state=down
[junit4] 2> 179692 T675 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 179694 T675 oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4] 2> 181191 T673 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 181192 T673 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4] 2> "shard_state":"active",
[junit4] 2> "base_url":"http://127.0.0.1:55788",
[junit4] 2> "state":"down",
[junit4] 2> "operation":"state",
[junit4] 2> "core_node_name":"core_node1",
[junit4] 2> "node_name":"127.0.0.1:55788_",
[junit4] 2> "numShards":"1",
[junit4] 2> "shard_range":null,
[junit4] 2> "roles":null,
[junit4] 2> "collection":"control_collection",
[junit4] 2> "shard":null,
[junit4] 2> "core":"collection1"}
[junit4] 2> 181193 T673 oasc.Overseer$ClusterStateUpdater.createCollection Create collection control_collection with numShards 1
[junit4] 2> 181193 T673 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4] 2> 181197 T672 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 181694 T675 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
[junit4] 2> 181695 T675 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-controljetty-1373828732902/collection1
[junit4] 2> 181695 T675 oasc.ZkController.createCollectionZkNode Check for collection zkNode:control_collection
[junit4] 2> 181696 T675 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4] 2> 181696 T675 oasc.ZkController.readConfigName Load collection config from:/collections/control_collection
[junit4] 2> 181698 T675 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-controljetty-1373828732902/collection1/'
[junit4] 2> 181698 T675 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.BasicDistributedZk2Test-controljetty-1373828732902/collection1/lib/README' to classloader
[junit4] 2> 181699 T675 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.BasicDistributedZk2Test-controljetty-1373828732902/collection1/lib/classes/' to classloader
[junit4] 2> 181724 T675 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_45
[junit4] 2> 181758 T675 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4] 2> 181759 T675 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4] 2> 181769 T675 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4] 2> 182102 T675 oass.ByteField.init WARN ByteField is deprecated and will be removed in 5.0. You should use TrieIntField instead.
[junit4] 2> 182102 T675 oass.ShortField.init WARN ShortField is deprecated and will be removed in 5.0. You should use TrieIntField instead.
[junit4] 2> 182103 T675 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4] 2> 182109 T675 oass.IndexSchema.readSchema default search field in schema is text
[junit4] 2> 182112 T675 oass.IndexSchema.readSchema unique key field: id
[junit4] 2> 182124 T675 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 182128 T675 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 182132 T675 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4] 2> 182133 T675 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4] 2> 182133 T675 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4] 2> 182133 T675 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4] 2> 182134 T675 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4] 2> 182135 T675 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4] 2> 182135 T675 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4] 2> 182135 T675 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-controljetty-1373828732902/collection1/, dataDir=./org.apache.solr.cloud.BasicDistributedZk2Test-1373828732652/control/data/
[junit4] 2> 182136 T675 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@c70432b2
[junit4] 2> 182136 T675 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.BasicDistributedZk2Test-1373828732652/control/data
[junit4] 2> 182137 T675 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.BasicDistributedZk2Test-1373828732652/control/data/index/
[junit4] 2> 182137 T675 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.BasicDistributedZk2Test-1373828732652/control/data/index' doesn't exist. Creating new index...
[junit4] 2> 182138 T675 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.BasicDistributedZk2Test-1373828732652/control/data/index
[junit4] 2> 182139 T675 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@2838b76e lockFactory=org.apache.lucene.store.NativeFSLockFactory@2acf7411),segFN=segments_1,generation=1}
[junit4] 2> 182139 T675 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 182142 T675 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4] 2> 182142 T675 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4] 2> 182143 T675 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 182143 T675 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4] 2> 182144 T675 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4] 2> 182144 T675 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4] 2> 182144 T675 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4] 2> 182145 T675 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4] 2> 182145 T675 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4] 2> 182147 T675 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 182149 T675 oass.SolrIndexSearcher.<init> Opening Searcher@d96fc208 main
[junit4] 2> 182149 T675 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4] 2> 182150 T675 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4] 2> 182154 T676 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@d96fc208 main{StandardDirectoryReader(segments_1:1)}
[junit4] 2> 182155 T675 oasc.CoreContainer.registerCore registering core: collection1
[junit4] 2> 182156 T675 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:55788 collection:control_collection shard:shard1
[junit4] 2> 182156 T675 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=10000&maxConnectionsPerHost=20&connTimeout=30000&socketTimeout=30000&retry=false
[junit4] 2> 182161 T675 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leader_elect/shard1/election
[junit4] 2> 182167 T675 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard1
[junit4] 2> 182169 T675 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4] 2> 182169 T675 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4] 2> 182169 T675 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:55788/collection1/
[junit4] 2> 182170 T675 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4] 2> 182170 T675 oasc.SyncStrategy.syncToMe http://127.0.0.1:55788/collection1/ has no replicas
[junit4] 2> 182170 T675 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:55788/collection1/ shard1
[junit4] 2> 182171 T675 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leaders/shard1
[junit4] 2> 182702 T673 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 182709 T672 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 182728 T675 oasc.ZkController.register We are http://127.0.0.1:55788/collection1/ and leader is http://127.0.0.1:55788/collection1/
[junit4] 2> 182728 T675 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4] 2> 182728 T675 oasc.ZkController.publish publishing core=collection1 state=active
[junit4] 2> 182729 T675 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 182730 T675 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 182732 T650 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1
[junit4] 2> 182732 T650 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4] 2> 182733 T650 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4] 2> 182738 T650 oascsi.HttpClientUtil.createClient Creating new http client, config:
[junit4] 2> 182739 T650 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 182740 T679 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@8d3f074c name:ZooKeeperConnection Watcher:127.0.0.1:58961/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 182741 T650 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 182742 T650 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 182744 T650 oasc.ChaosMonkey.monkeyLog monkey: init - expire sessions:true cause connection loss:true
[junit4] 2> 182824 T650 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4] 2> 182827 T650 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:37394
[junit4] 2> 182827 T650 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4] 2> 182828 T650 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4] 2> 182828 T650 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty1-1373828736141
[junit4] 2> 182829 T650 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty1-1373828736141/'
[junit4] 2> 182837 T650 oasc.ConfigSolr.fromFile Loading container configuration from /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty1-1373828736141/solr.xml
[junit4] 2> 182869 T650 oasc.CoreContainer.<init> New CoreContainer 72517398
[junit4] 2> 182870 T650 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty1-1373828736141/]
[junit4] 2> 182871 T650 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4] 2> 182872 T650 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4] 2> 182872 T650 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4] 2> 182873 T650 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4] 2> 182873 T650 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4] 2> 182874 T650 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4] 2> 182875 T650 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4] 2> 182875 T650 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4] 2> 182876 T650 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4] 2> 182876 T650 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4] 2> 182881 T650 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
[junit4] 2> 182882 T650 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:58961/solr
[junit4] 2> 182882 T650 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4] 2> 182883 T650 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 182885 T690 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@58b75aa7 name:ZooKeeperConnection Watcher:127.0.0.1:58961 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 182885 T650 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 182886 T650 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4] 2> 182891 T650 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 182893 T692 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@7b4d928c name:ZooKeeperConnection Watcher:127.0.0.1:58961/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 182893 T650 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 182896 T650 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 183900 T650 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:37394_
[junit4] 2> 183901 T650 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:37394_
[junit4] 2> 183904 T679 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4] 2> 183904 T672 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4] 2> 183905 T692 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4] 2> 183905 T672 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
[junit4] 2> 183911 T693 oasc.ZkController.publish publishing core=collection1 state=down
[junit4] 2> 183911 T693 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 183912 T693 oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4] 2> 184214 T673 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 184216 T673 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4] 2> "shard_state":"active",
[junit4] 2> "base_url":"http://127.0.0.1:55788",
[junit4] 2> "state":"active",
[junit4] 2> "operation":"state",
[junit4] 2> "core_node_name":"core_node1",
[junit4] 2> "node_name":"127.0.0.1:55788_",
[junit4] 2> "numShards":"1",
[junit4] 2> "shard_range":null,
[junit4] 2> "roles":null,
[junit4] 2> "collection":"control_collection",
[junit4] 2> "shard":"shard1",
[junit4] 2> "core":"collection1"}
[junit4] 2> 184220 T673 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4] 2> "shard_state":"active",
[junit4] 2> "base_url":"http://127.0.0.1:37394",
[junit4] 2> "state":"down",
[junit4] 2> "operation":"state",
[junit4] 2> "core_node_name":"core_node1",
[junit4] 2> "node_name":"127.0.0.1:37394_",
[junit4] 2> "numShards":"2",
[junit4] 2> "shard_range":null,
[junit4] 2> "roles":null,
[junit4] 2> "collection":"collection1",
[junit4] 2> "shard":null,
[junit4] 2> "core":"collection1"}
[junit4] 2> 184220 T673 oasc.Overseer$ClusterStateUpdater.createCollection Create collection collection1 with numShards 2
[junit4] 2> 184221 T673 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4] 2> 184225 T692 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 184225 T672 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 184225 T679 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 184913 T693 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
[junit4] 2> 184913 T693 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty1-1373828736141/collection1
[junit4] 2> 184914 T693 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4] 2> 184915 T693 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4] 2> 184915 T693 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4] 2> 184916 T693 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty1-1373828736141/collection1/'
[junit4] 2> 184917 T693 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.BasicDistributedZk2Test-jetty1-1373828736141/collection1/lib/README' to classloader
[junit4] 2> 184918 T693 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.BasicDistributedZk2Test-jetty1-1373828736141/collection1/lib/classes/' to classloader
[junit4] 2> 184943 T693 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_45
[junit4] 2> 184973 T693 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4] 2> 184975 T693 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4] 2> 184981 T693 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4] 2> 185288 T693 oass.ByteField.init WARN ByteField is deprecated and will be removed in 5.0. You should use TrieIntField instead.
[junit4] 2> 185289 T693 oass.ShortField.init WARN ShortField is deprecated and will be removed in 5.0. You should use TrieIntField instead.
[junit4] 2> 185289 T693 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4] 2> 185295 T693 oass.IndexSchema.readSchema default search field in schema is text
[junit4] 2> 185297 T693 oass.IndexSchema.readSchema unique key field: id
[junit4] 2> 185311 T693 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 185316 T693 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 185320 T693 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4] 2> 185321 T693 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4] 2> 185322 T693 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4] 2> 185322 T693 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4] 2> 185323 T693 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4] 2> 185324 T693 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4] 2> 185324 T693 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4] 2> 185324 T693 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty1-1373828736141/collection1/, dataDir=./org.apache.solr.cloud.BasicDistributedZk2Test-1373828732652/jetty1/
[junit4] 2> 185325 T693 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@c70432b2
[junit4] 2> 185326 T693 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.BasicDistributedZk2Test-1373828732652/jetty1
[junit4] 2> 185326 T693 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.BasicDistributedZk2Test-1373828732652/jetty1/index/
[junit4] 2> 185326 T693 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.BasicDistributedZk2Test-1373828732652/jetty1/index' doesn't exist. Creating new index...
[junit4] 2> 185327 T693 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.BasicDistributedZk2Test-1373828732652/jetty1/index
[junit4] 2> 185328 T693 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@1263b426 lockFactory=org.apache.lucene.store.NativeFSLockFactory@1a361fee),segFN=segments_1,generation=1}
[junit4] 2> 185328 T693 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 185331 T693 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4] 2> 185332 T693 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4] 2> 185332 T693 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 185333 T693 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4] 2> 185334 T693 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4] 2> 185334 T693 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4] 2> 185334 T693 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4] 2> 185335 T693 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4] 2> 185335 T693 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4] 2> 185336 T693 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 185338 T693 oass.SolrIndexSearcher.<init> Opening Searcher@de9e519e main
[junit4] 2> 185339 T693 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4] 2> 185339 T693 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4] 2> 185345 T694 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@de9e519e main{StandardDirectoryReader(segments_1:1)}
[junit4] 2> 185347 T693 oasc.CoreContainer.registerCore registering core: collection1
[junit4] 2> 185348 T693 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:37394 collection:collection1 shard:shard1
[junit4] 2> 185348 T693 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=10000&maxConnectionsPerHost=20&connTimeout=30000&socketTimeout=30000&retry=false
[junit4] 2> 185354 T693 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
[junit4] 2> 185361 T693 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard1
[junit4] 2> 185364 T693 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4] 2> 185365 T693 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4] 2> 185365 T693 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:37394/collection1/
[junit4] 2> 185365 T693 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4] 2> 185366 T693 oasc.SyncStrategy.syncToMe http://127.0.0.1:37394/collection1/ has no replicas
[junit4] 2> 185366 T693 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:37394/collection1/ shard1
[junit4] 2> 185366 T693 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
[junit4] 2> 185731 T673 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 185738 T692 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 185738 T679 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 185739 T672 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 185773 T693 oasc.ZkController.register We are http://127.0.0.1:37394/collection1/ and leader is http://127.0.0.1:37394/collection1/
[junit4] 2> 185774 T693 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4] 2> 185774 T693 oasc.ZkController.publish publishing core=collection1 state=active
[junit4] 2> 185774 T693 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 185777 T693 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 185778 T650 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1
[junit4] 2> 185779 T650 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4] 2> 185780 T650 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4] 2> 185878 T650 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4] 2> 185881 T650 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:47766
[junit4] 2> 185882 T650 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4] 2> 185883 T650 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4] 2> 185884 T650 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty2-1373828739184
[junit4] 2> 185885 T650 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty2-1373828739184/'
[junit4] 2> 185895 T650 oasc.ConfigSolr.fromFile Loading container configuration from /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty2-1373828739184/solr.xml
[junit4] 2> 185926 T650 oasc.CoreContainer.<init> New CoreContainer 351591603
[junit4] 2> 185926 T650 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty2-1373828739184/]
[junit4] 2> 185927 T650 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4] 2> 185928 T650 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4] 2> 185929 T650 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4] 2> 185929 T650 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4] 2> 185930 T650 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4] 2> 185930 T650 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4] 2> 185931 T650 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4] 2> 185931 T650 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4] 2> 185932 T650 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4] 2> 185933 T650 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4] 2> 185938 T650 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
[junit4] 2> 185938 T650 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:58961/solr
[junit4] 2> 185939 T650 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4] 2> 185940 T650 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 185943 T706 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@49f98350 name:ZooKeeperConnection Watcher:127.0.0.1:58961 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 185943 T650 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 185945 T650 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4] 2> 185950 T650 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 185953 T708 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@149121b1 name:ZooKeeperConnection Watcher:127.0.0.1:58961/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 185953 T650 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 185957 T650 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 186962 T650 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:47766_
[junit4] 2> 186963 T650 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:47766_
[junit4] 2> 186966 T679 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
[junit4] 2> 186966 T692 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4] 2> 186966 T672 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4] 2> 186967 T692 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4] 2> 186967 T672 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4] 2> 186967 T708 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4] 2> 186968 T679 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4] 2> 186973 T709 oasc.ZkController.publish publishing core=collection1 state=down
[junit4] 2> 186973 T709 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 186974 T709 oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4] 2> 187247 T673 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 187248 T673 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4] 2> "shard_state":"active",
[junit4] 2> "base_url":"http://127.0.0.1:37394",
[junit4] 2> "state":"active",
[junit4] 2> "operation":"state",
[junit4] 2> "core_node_name":"core_node1",
[junit4] 2> "node_name":"127.0.0.1:37394_",
[junit4] 2> "numShards":"2",
[junit4] 2> "shard_range":null,
[junit4] 2> "roles":null,
[junit4] 2> "collection":"collection1",
[junit4] 2> "shard":"shard1",
[junit4] 2> "core":"collection1"}
[junit4] 2> 187252 T673 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4] 2> "shard_state":"active",
[junit4] 2> "base_url":"http://127.0.0.1:47766",
[junit4] 2> "state":"down",
[junit4] 2> "operation":"state",
[junit4] 2> "core_node_name":"core_node2",
[junit4] 2> "node_name":"127.0.0.1:47766_",
[junit4] 2> "numShards":"2",
[junit4] 2> "shard_range":null,
[junit4] 2> "roles":null,
[junit4] 2> "collection":"collection1",
[junit4] 2> "shard":null,
[junit4] 2> "core":"collection1"}
[junit4] 2> 187253 T673 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=2
[junit4] 2> 187253 T673 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard2
[junit4] 2> 187256 T672 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 187257 T679 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 187256 T692 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 187256 T708 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 187975 T709 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
[junit4] 2> 187975 T709 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty2-1373828739184/collection1
[junit4] 2> 187975 T709 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4] 2> 187976 T709 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4] 2> 187977 T709 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4] 2> 187978 T709 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty2-1373828739184/collection1/'
[junit4] 2> 187979 T709 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.BasicDistributedZk2Test-jetty2-1373828739184/collection1/lib/README' to classloader
[junit4] 2> 187979 T709 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.BasicDistributedZk2Test-jetty2-1373828739184/collection1/lib/classes/' to classloader
[junit4] 2> 188005 T709 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_45
[junit4] 2> 188036 T709 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4] 2> 188037 T709 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4] 2> 188044 T709 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4] 2> 188357 T709 oass.ByteField.init WARN ByteField is deprecated and will be removed in 5.0. You should use TrieIntField instead.
[junit4] 2> 188357 T709 oass.ShortField.init WARN ShortField is deprecated and will be removed in 5.0. You should use TrieIntField instead.
[junit4] 2> 188358 T709 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4] 2> 188363 T709 oass.IndexSchema.readSchema default search field in schema is text
[junit4] 2> 188366 T709 oass.IndexSchema.readSchema unique key field: id
[junit4] 2> 188379 T709 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 188385 T709 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 188391 T709 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4] 2> 188392 T709 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4] 2> 188393 T709 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4] 2> 188393 T709 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4] 2> 188395 T709 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4] 2> 188395 T709 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4] 2> 188396 T709 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4] 2> 188396 T709 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty2-1373828739184/collection1/, dataDir=./org.apache.solr.cloud.BasicDistributedZk2Test-1373828732652/jetty2/
[junit4] 2> 188397 T709 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@c70432b2
[junit4] 2> 188398 T709 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.BasicDistributedZk2Test-1373828732652/jetty2
[junit4] 2> 188398 T709 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.BasicDistributedZk2Test-1373828732652/jetty2/index/
[junit4] 2> 188399 T709 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.BasicDistributedZk2Test-1373828732652/jetty2/index' doesn't exist. Creating new index...
[junit4] 2> 188399 T709 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.BasicDistributedZk2Test-1373828732652/jetty2/index
[junit4] 2> 188401 T709 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@d5bf3b93 lockFactory=org.apache.lucene.store.NativeFSLockFactory@6ef8d267),segFN=segments_1,generation=1}
[junit4] 2> 188402 T709 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 188407 T709 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4] 2> 188407 T709 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4] 2> 188408 T709 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 188409 T709 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4] 2> 188410 T709 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4] 2> 188410 T709 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4] 2> 188411 T709 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4] 2> 188412 T709 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4] 2> 188412 T709 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4] 2> 188413 T709 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 188415 T709 oass.SolrIndexSearcher.<init> Opening Searcher@c9e8563f main
[junit4] 2> 188416 T709 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4] 2> 188417 T709 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4] 2> 188421 T710 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@c9e8563f main{StandardDirectoryReader(segments_1:1)}
[junit4] 2> 188423 T709 oasc.CoreContainer.registerCore registering core: collection1
[junit4] 2> 188423 T709 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:47766 collection:collection1 shard:shard2
[junit4] 2> 188423 T709 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=10000&maxConnectionsPerHost=20&connTimeout=30000&socketTimeout=30000&retry=false
[junit4] 2> 188429 T709 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard2/election
[junit4] 2> 188511 T709 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard2
[junit4] 2> 188513 T709 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4] 2> 188514 T709 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4] 2> 188514 T709 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:47766/collection1/
[junit4] 2> 188514 T709 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4] 2> 188515 T709 oasc.SyncStrategy.syncToMe http://127.0.0.1:47766/collection1/ has no replicas
[junit4] 2> 188515 T709 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:47766/collection1/ shard2
[junit4] 2> 188515 T709 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard2
[junit4] 2> 188764 T673 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 188773 T679 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 188773 T692 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 188773 T708 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 188773 T672 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 188821 T709 oasc.ZkController.register We are http://127.0.0.1:47766/collection1/ and leader is http://127.0.0.1:47766/collection1/
[junit4] 2> 188821 T709 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4] 2> 188822 T709 oasc.ZkController.publish publishing core=collection1 state=active
[junit4] 2> 188822 T709 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 188824 T709 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 188826 T650 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1
[junit4] 2> 188827 T650 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4] 2> 188827 T650 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4] 2> 188934 T650 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4] 2> 188937 T650 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:43401
[junit4] 2> 188938 T650 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4] 2> 188938 T650 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4] 2> 188939 T650 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty3-1373828742229
[junit4] 2> 188939 T650 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty3-1373828742229/'
[junit4] 2> 188947 T650 oasc.ConfigSolr.fromFile Loading container configuration from /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty3-1373828742229/solr.xml
[junit4] 2> 188991 T650 oasc.CoreContainer.<init> New CoreContainer -1108876977
[junit4] 2> 188992 T650 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty3-1373828742229/]
[junit4] 2> 188994 T650 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4] 2> 188995 T650 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4] 2> 188996 T650 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4] 2> 188996 T650 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4] 2> 188997 T650 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4] 2> 188998 T650 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4] 2> 188999 T650 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4] 2> 189000 T650 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4] 2> 189001 T650 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4] 2> 189002 T650 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4] 2> 189009 T650 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
[junit4] 2> 189010 T650 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:58961/solr
[junit4] 2> 189011 T650 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4] 2> 189013 T650 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 189015 T722 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@4b372935 name:ZooKeeperConnection Watcher:127.0.0.1:58961 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 189015 T650 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 189018 T650 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4] 2> 189026 T650 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 189028 T724 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@d2c92ed5 name:ZooKeeperConnection Watcher:127.0.0.1:58961/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 189028 T650 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 189035 T650 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 190039 T650 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:43401_
[junit4] 2> 190041 T650 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:43401_
[junit4] 2> 190043 T679 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4] 2> 190043 T724 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4] 2> 190043 T708 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4] 2> 190043 T672 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4] 2> 190043 T692 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4] 2> 190044 T672 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4] 2> 190044 T692 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4] 2> 190044 T708 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4] 2> 190046 T679 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4] 2> 190054 T725 oasc.ZkController.publish publishing core=collection1 state=down
[junit4] 2> 190054 T725 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 190149 T725 oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4] 2> 190281 T673 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 190282 T673 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4] 2> "shard_state":"active",
[junit4] 2> "base_url":"http://127.0.0.1:47766",
[junit4] 2> "state":"active",
[junit4] 2> "operation":"state",
[junit4] 2> "core_node_name":"core_node2",
[junit4] 2> "node_name":"127.0.0.1:47766_",
[junit4] 2> "numShards":"2",
[junit4] 2> "shard_range":null,
[junit4] 2> "roles":null,
[junit4] 2> "collection":"collection1",
[junit4] 2> "shard":"shard2",
[junit4] 2> "core":"collection1"}
[junit4] 2> 190288 T673 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4] 2> "shard_state":"active",
[junit4] 2> "base_url":"http://127.0.0.1:43401",
[junit4] 2> "state":"down",
[junit4] 2> "operation":"state",
[junit4] 2> "core_node_name":"core_node3",
[junit4] 2> "node_name":"127.0.0.1:43401_",
[junit4] 2> "numShards":"2",
[junit4] 2> "shard_range":null,
[junit4] 2> "roles":null,
[junit4] 2> "collection":"collection1",
[junit4] 2> "shard":null,
[junit4] 2> "core":"collection1"}
[junit4] 2> 190288 T673 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=2
[junit4] 2> 190288 T673 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4] 2> 190292 T724 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4] 2> 190293 T679 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4] 2> 190293 T708 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4] 2> 190293 T672 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4] 2> 190293 T692 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4] 2> 191150 T725 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
[junit4] 2> 191151 T725 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty3-1373828742229/collection1
[junit4] 2> 191151 T725 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4] 2> 191152 T725 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4] 2> 191153 T725 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4] 2> 191154 T725 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty3-1373828742229/collection1/'
[junit4] 2> 191155 T725 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.BasicDistributedZk2Test-jetty3-1373828742229/collection1/lib/README' to classloader
[junit4] 2> 191156 T725 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.BasicDistributedZk2Test-jetty3-1373828742229/collection1/lib/classes/' to classloader
[junit4] 2> 191197 T725 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_45
[junit4] 2> 191246 T725 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4] 2> 191250 T725 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4] 2> 191262 T725 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4] 2> 191710 T725 oass.ByteField.init WARN ByteField is deprecated and will be removed in 5.0. You should use TrieIntField instead.
[junit4] 2> 191711 T725 oass.ShortField.init WARN ShortField is deprecated and will be removed in 5.0. You should use TrieIntField instead.
[junit4] 2> 191711 T725 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4] 2> 191716 T725 oass.IndexSchema.readSchema default search field in schema is text
[junit4] 2> 191719 T725 oass.IndexSchema.readSchema unique key field: id
[junit4] 2> 191733 T725 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 191740 T725 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 191746 T725 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4] 2> 191748 T725 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4] 2> 191749 T725 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4] 2> 191749 T725 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4] 2> 191752 T725 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4] 2> 191752 T725 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4] 2> 191752 T725 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4] 2> 191753 T725 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty3-1373828742229/collection1/, dataDir=./org.apache.solr.cloud.BasicDistributedZk2Test-1373828732652/jetty3/
[junit4] 2> 191753 T725 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@c70432b2
[junit4] 2> 191754 T725 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.BasicDistributedZk2Test-1373828732652/jetty3
[junit4] 2> 191754 T725 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.BasicDistributedZk2Test-1373828732652/jetty3/index/
[junit4] 2> 191755 T725 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.BasicDistributedZk2Test-1373828732652/jetty3/index' doesn't exist. Creating new index...
[junit4] 2> 191755 T725 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.BasicDistributedZk2Test-1373828732652/jetty3/index
[junit4] 2> 191757 T725 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@5c8bf38d lockFactory=org.apache.lucene.store.NativeFSLockFactory@4be64ea8),segFN=segments_1,generation=1}
[junit4] 2> 191757 T725 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 191760 T725 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4] 2> 191761 T725 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4] 2> 191761 T725 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 191762 T725 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4] 2> 191763 T725 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4] 2> 191763 T725 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4] 2> 191763 T725 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4] 2> 191764 T725 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4] 2> 191765 T725 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4] 2> 191766 T725 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 191768 T725 oass.SolrIndexSearcher.<init> Opening Searcher@2b405435 main
[junit4] 2> 191769 T725 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4] 2> 191770 T725 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4] 2> 191775 T726 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@2b405435 main{StandardDirectoryReader(segments_1:1)}
[junit4] 2> 191777 T725 oasc.CoreContainer.registerCore registering core: collection1
[junit4] 2> 191777 T725 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:43401 collection:collection1 shard:shard1
[junit4] 2> 191777 T725 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=10000&maxConnectionsPerHost=20&connTimeout=30000&socketTimeout=30000&retry=false
[junit4] 2> 191784 T725 oasc.ZkController.register We are http://127.0.0.1:43401/collection1/ and leader is http://127.0.0.1:37394/collection1/
[junit4] 2> 191785 T725 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:43401
[junit4] 2> 191785 T725 oasc.ZkController.checkRecovery Core needs to recover:collection1
[junit4] 2> 191786 T725 oasu.DefaultSolrCoreState.doRecovery Running recovery - first canceling any ongoing recovery
[junit4] 2> ASYNC NEW_CORE C277 name=collection1 org.apache.solr.core.SolrCore@a3f8ae6f url=http://127.0.0.1:43401/collection1 node=127.0.0.1:43401_ C277_STATE=coll:collection1 core:collection1 props:{base_url=http://127.0.0.1:43401, state=down, node_name=127.0.0.1:43401_, core=collection1}
[junit4] 2> 191786 T727 C277 P43401 oasc.RecoveryStrategy.run Starting recovery process. core=collection1 recoveringAfterStartup=true
[junit4] 2> 191787 T725 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 191787 T727 C277 P43401 oasc.RecoveryStrategy.doRecovery ###### startupVersions=[]
[junit4] 2> 191787 T727 C277 P43401 oasc.ZkController.publish publishing core=collection1 state=recovering
[junit4] 2> 191788 T727 C277 P43401 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 191788 T650 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1
[junit4] 2> 191789 T727 C277 P43401 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4] 2> 191789 T650 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4] 2> 191790 T650 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4] 2> 191795 T686 oasha.CoreAdminHandler.handleWaitForStateAction Going to wait for coreNodeName: core_node3, state: recovering, checkLive: true, onlyIfLeader: true
[junit4] 2> 191801 T673 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 191803 T673 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4] 2> "shard_state":"active",
[junit4] 2> "base_url":"http://127.0.0.1:43401",
[junit4] 2> "state":"recovering",
[junit4] 2> "operation":"state",
[junit4] 2> "core_node_name":"core_node3",
[junit4] 2> "node_name":"127.0.0.1:43401_",
[junit4] 2> "numShards":"2",
[junit4] 2> "shard_range":null,
[junit4] 2> "roles":null,
[junit4] 2> "collection":"collection1",
[junit4] 2> "shard":"shard1",
[junit4] 2> "core":"collection1"}
[junit4] 2> 191810 T672 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4] 2> 191810 T724 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4] 2> 191810 T708 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4] 2> 191810 T692 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4] 2> 191810 T679 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4] 2> 191886 T650 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4] 2> 191888 T650 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:48446
[junit4] 2> 191889 T650 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4] 2> 191890 T650 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4] 2> 191891 T650 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty4-1373828745195
[junit4] 2> 191891 T650 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty4-1373828745195/'
[junit4] 2> 191900 T650 oasc.ConfigSolr.fromFile Loading container configuration from /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty4-1373828745195/solr.xml
[junit4] 2> 191937 T650 oasc.CoreContainer.<init> New CoreContainer -1082656268
[junit4] 2> 191937 T650 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty4-1373828745195/]
[junit4] 2> 191939 T650 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4] 2> 191940 T650 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4] 2> 191940 T650 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4] 2> 191941 T650 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4] 2> 191942 T650 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4] 2> 191942 T650 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4] 2> 191943 T650 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4] 2> 191944 T650 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4] 2> 191944 T650 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4] 2> 191945 T650 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4] 2> 191950 T650 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
[junit4] 2> 191951 T650 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:58961/solr
[junit4] 2> 191951 T650 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4] 2> 191953 T650 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 191954 T739 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@2f5238ca name:ZooKeeperConnection Watcher:127.0.0.1:58961 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 191954 T650 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 191957 T650 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4] 2> 191962 T650 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 191963 T741 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@5b5e837d name:ZooKeeperConnection Watcher:127.0.0.1:58961/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 191964 T650 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 191967 T650 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 192796 T686 oasha.CoreAdminHandler.handleWaitForStateAction Waited coreNodeName: core_node3, state: recovering, checkLive: true, onlyIfLeader: true for: 1 seconds.
[junit4] 2> 192796 T686 oass.SolrDispatchFilter.handleAdminRequest [admin] webapp=null path=/admin/cores params={state=recovering&onlyIfLeader=true&checkLive=true&action=PREPRECOVERY&version=2&coreNodeName=core_node3&wt=javabin&nodeName=127.0.0.1:43401_&core=collection1} status=0 QTime=1001
[junit4] 2> 192970 T650 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:48446_
[junit4] 2> 192972 T650 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:48446_
[junit4] 2> 192975 T724 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4] 2> 192975 T741 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4] 2> 192976 T679 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4] 2> 192975 T672 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4] 2> 192976 T708 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4] 2> 192977 T708 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4] 2> 192977 T672 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4] 2> 192980 T679 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4] 2> 192981 T724 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4] 2> 192977 T692 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4] 2> 192983 T692 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4] 2> 192986 T742 oasc.ZkController.publish publishing core=collection1 state=down
[junit4] 2> 192987 T742 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 192990 T742 oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4] 2> 193317 T673 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 193319 T673 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4] 2> "shard_state":"active",
[junit4] 2> "base_url":"http://127.0.0.1:48446",
[junit4] 2> "state":"down",
[junit4] 2> "operation":"state",
[junit4] 2> "core_node_name":"core_node4",
[junit4] 2> "node_name":"127.0.0.1:48446_",
[junit4] 2> "numShards":"2",
[junit4] 2> "shard_range":null,
[junit4] 2> "roles":null,
[junit4] 2> "collection":"collection1",
[junit4] 2> "shard":null,
[junit4] 2> "core":"collection1"}
[junit4] 2> 193319 T673 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=2
[junit4] 2> 193319 T673 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard2
[junit4] 2> 193324 T672 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4] 2> 193324 T741 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4] 2> 193324 T679 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4] 2> 193324 T724 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4] 2> 193327 T708 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4] 2> 193328 T692 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4] 2> 193990 T742 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
[junit4] 2> 193991 T742 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty4-1373828745195/collection1
[junit4] 2> 193991 T742 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4] 2> 193992 T742 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4] 2> 193992 T742 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4] 2> 193993 T742 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty4-1373828745195/collection1/'
[junit4] 2> 193994 T742 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.BasicDistributedZk2Test-jetty4-1373828745195/collection1/lib/README' to classloader
[junit4] 2> 193995 T742 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.BasicDistributedZk2Test-jetty4-1373828745195/collection1/lib/classes/' to classloader
[junit4] 2> 194042 T742 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_45
[junit4] 2> 194075 T742 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4] 2> 194076 T742 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4] 2> 194082 T742 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4] 2> 194423 T742 oass.ByteField.init WARN ByteField is deprecated and will be removed in 5.0. You should use TrieIntField instead.
[junit4] 2> 194423 T742 oass.ShortField.init WARN ShortField is deprecated and will be removed in 5.0. You should use TrieIntField instead.
[junit4] 2> 194424 T742 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4] 2> 194430 T742 oass.IndexSchema.readSchema default search field in schema is text
[junit4] 2> 194433 T742 oass.IndexSchema.readSchema unique key field: id
[junit4] 2> 194449 T742 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 194453 T742 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 194458 T742 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4] 2> 194459 T742 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4] 2> 194460 T742 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4] 2> 194460 T742 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4] 2> 194461 T742 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4] 2> 194462 T742 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4] 2> 194462 T742 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4] 2> 194462 T742 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty4-1373828745195/collection1/, dataDir=./org.apache.solr.cloud.BasicDistributedZk2Test-1373828732652/jetty4/
[junit4] 2> 194463 T742 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@c70432b2
[junit4] 2> 194464 T742 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.BasicDistributedZk2Test-1373828732652/jetty4
[junit4] 2> 194464 T742 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.BasicDistributedZk2Test-1373828732652/jetty4/index/
[junit4] 2> 194464 T742 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.BasicDistributedZk2Test-1373828732652/jetty4/index' doesn't exist. Creating new index...
[junit4] 2> 194465 T742 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.BasicDistributedZk2Test-1373828732652/jetty4/index
[junit4] 2> 194466 T742 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@850b6e93 lockFactory=org.apache.lucene.store.NativeFSLockFactory@872798d3),segFN=segments_1,generation=1}
[junit4] 2> 194466 T742 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 194470 T742 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4] 2> 194470 T742 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4] 2> 194471 T742 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 194471 T742 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4] 2> 194472 T742 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4] 2> 194472 T742 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4] 2> 194473 T742 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4] 2> 194473 T742 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4] 2> 194474 T742 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4] 2> 194475 T742 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 194477 T742 oass.SolrIndexSearcher.<init> Opening Searcher@895b3187 main
[junit4] 2> 194478 T742 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4] 2> 194478 T742 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4] 2> 194482 T743 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@895b3187 main{StandardDirectoryReader(segments_1:1)}
[junit4] 2> 194484 T742 oasc.CoreContainer.registerCore registering core: collection1
[junit4] 2> 194484 T742 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:48446 collection:collection1 shard:shard2
[junit4] 2> 194485 T742 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=10000&maxConnectionsPerHost=20&connTimeout=30000&socketTimeout=30000&retry=false
[junit4] 2> 194492 T742 oasc.ZkController.register We are http://127.0.0.1:48446/collection1/ and leader is http://127.0.0.1:47766/collection1/
[junit4] 2> 194492 T742 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:48446
[junit4] 2> 194492 T742 oasc.ZkController.checkRecovery Core needs to recover:collection1
[junit4] 2> 194493 T742 oasu.DefaultSolrCoreState.doRecovery Running recovery - first canceling any ongoing recovery
[junit4] 2> ASYNC NEW_CORE C278 name=collection1 org.apache.solr.core.SolrCore@e03e5a3b url=http://127.0.0.1:48446/collection1 node=127.0.0.1:48446_ C278_STATE=coll:collection1 core:collection1 props:{base_url=http://127.0.0.1:48446, state=down, node_name=127.0.0.1:48446_, core=collection1}
[junit4] 2> 194493 T744 C278 P48446 oasc.RecoveryStrategy.run Starting recovery process. core=collection1 recoveringAfterStartup=true
[junit4] 2> 194494 T742 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 194494 T744 C278 P48446 oasc.RecoveryStrategy.doRecovery ###### startupVersions=[]
[junit4] 2> 194495 T744 C278 P48446 oasc.ZkController.publish publishing core=collection1 state=recovering
[junit4] 2> 194495 T744 C278 P48446 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 194496 T650 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1
[junit4] 2> 194497 T650 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4] 2> 194497 T650 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4] 2> 194498 T744 C278 P48446 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4] 2> 194503 T650 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 194505 T650 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4] 2> 194508 T704 oasha.CoreAdminHandler.handleWaitForStateAction Going to wait for coreNodeName: core_node4, state: recovering, checkLive: true, onlyIfLeader: true
[junit4] 2> 194601 T688 oasc.ZkController.publish publishing core=onenodecollectioncore state=down
[junit4] 2> 194603 T688 oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4] 2> ASYNC NEW_CORE C279 name=collection1 org.apache.solr.core.SolrCore@a3f8ae6f url=http://127.0.0.1:43401/collection1 node=127.0.0.1:43401_ C279_STATE=coll:collection1 core:collection1 props:{base_url=http://127.0.0.1:43401, state=recovering, node_name=127.0.0.1:43401_, core=collection1}
[junit4] 2> 194798 T727 C279 P43401 oasc.RecoveryStrategy.doRecovery Attempting to PeerSync from http://127.0.0.1:37394/collection1/ core=collection1 - recoveringAfterStartup=true
[junit4] 2> 194798 T727 C279 P43401 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:43401 START replicas=[http://127.0.0.1:37394/collection1/] nUpdates=100
[junit4] 2> 194799 T727 C279 P43401 oasu.PeerSync.sync WARN no frame of reference to tell of we've missed updates
[junit4] 2> 194799 T727 C279 P43401 oasc.RecoveryStrategy.doRecovery PeerSync Recovery was not successful - trying replication. core=collection1
[junit4] 2> 194799 T727 C279 P43401 oasc.RecoveryStrategy.doRecovery Starting Replication Recovery. core=collection1
[junit4] 2> 194800 T727 C279 P43401 oasc.RecoveryStrategy.doRecovery Begin buffering updates. core=collection1
[junit4] 2> 194800 T727 C279 P43401 oasu.UpdateLog.bufferUpdates Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null}
[junit4] 2> 194800 T727 C279 P43401 oasc.RecoveryStrategy.replicate Attempting to replicate from http://127.0.0.1:37394/collection1/. core=collection1
[junit4] 2> 194801 T727 C279 P43401 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4] 2> ASYNC NEW_CORE C280 name=collection1 org.apache.solr.core.SolrCore@364a9e90 url=http://127.0.0.1:37394/collection1 node=127.0.0.1:37394_ C280_STATE=coll:collection1 core:collection1 props:{base_url=http://127.0.0.1:37394, state=active, node_name=127.0.0.1:37394_, core=collection1, leader=true}
[junit4] 2> 194807 T685 C280 P37394 oasc.SolrCore.execute [collection1] webapp= path=/get params={getVersions=100&qt=/get&distrib=false&version=2&wt=javabin} status=0 QTime=0
[junit4] 2> 194809 T687 C280 P37394 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4] 2> 194811 T687 C280 P37394 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@1263b426 lockFactory=org.apache.lucene.store.NativeFSLockFactory@1a361fee),segFN=segments_1,generation=1}
[junit4] 2> 194812 T687 C280 P37394 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 194813 T687 C280 P37394 oasu.DirectUpdateHandler2.commit No uncommitted changes. Skipping IW.commit.
[junit4] 2> 194814 T687 C280 P37394 oass.SolrIndexSearcher.<init> Opening Searcher@9e26143d realtime
[junit4] 2> 194814 T687 C280 P37394 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4] 2> 194815 T687 C280 P37394 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={softCommit=false&commit_end_point=true&commit=true&version=2&waitSearcher=true&wt=javabin&openSearcher=false} {commit=} 0 6
[junit4] 2> 194817 T727 C279 P43401 oash.ReplicationHandler.inform Commits will be reserved for 10000
[junit4] 2> 194817 T727 C279 P43401 oash.SnapPuller.<init> No value set for 'pollInterval'. Timer Task not started.
[junit4] 2> 194821 T687 C280 P37394 oash.ReplicationHandler.inform Commits will be reserved for 10000
[junit4] 2> 194822 T687 C280 P37394 oasc.SolrCore.execute [collection1] webapp= path=/replication params={qt=/replication&version=2&command=indexversion&wt=javabin} status=0 QTime=2
[junit4] 2> 194823 T727 C279 P43401 oasc.RecoveryStrategy.replay No replay needed. core=collection1
[junit4] 2> 194824 T727 C279 P43401 oasc.RecoveryStrategy.doRecovery Replication Recovery was successful - registering as Active. core=collection1
[junit4] 2> 194824 T727 C279 P43401 oasc.ZkController.publish publishing core=collection1 state=active
[junit4] 2> 194824 T727 C279 P43401 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 194826 T727 C279 P43401 oasc.RecoveryStrategy.doRecovery Finished recovery process. core=collection1
[junit4] 2> 194830 T673 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 194831 T673 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4] 2> "shard_state":"active",
[junit4] 2> "base_url":"http://127.0.0.1:48446",
[junit4] 2> "state":"recovering",
[junit4] 2> "operation":"state",
[junit4] 2> "core_node_name":"core_node4",
[junit4] 2> "node_name":"127.0.0.1:48446_",
[junit4] 2> "numShards":"2",
[junit4] 2> "shard_range":null,
[junit4] 2> "roles":null,
[junit4] 2> "collection":"collection1",
[junit4] 2> "shard":"shard2",
[junit4] 2> "core":"collection1"}
[junit4] 2> 194835 T673 oasc.Overseer$ClusterStateUpdater.updateState WARN Could not find collection node for onenodecollection, skipping publish state
[junit4] 2> 194835 T673 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4] 2> "shard_state":"active",
[junit4] 2> "base_url":"http://127.0.0.1:37394",
[junit4] 2> "state":"down",
[junit4] 2> "operation":"state",
[junit4] 2> "core_node_name":"core_node1",
[junit4] 2> "node_name":"127.0.0.1:37394_",
[junit4] 2> "numShards":"1",
[junit4] 2> "shard_range":null,
[junit4] 2> "roles":"none",
[junit4] 2> "collection":"onenodecollection",
[junit4] 2> "shard":null,
[junit4] 2> "core":"onenodecollectioncore"}
[junit4] 2> 194836 T673 oasc.Overseer$ClusterStateUpdater.createCollection Create collection onenodecollection with numShards 1
[junit4] 2> 194836 T673 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4] 2> 194839 T673 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4] 2> "shard_state":"active",
[junit4] 2> "base_url":"http://127.0.0.1:43401",
[junit4] 2> "state":"active",
[junit4] 2> "operation":"state",
[junit4] 2> "core_node_name":"core_node3",
[junit4] 2> "node_name":"127.0.0.1:43401_",
[junit4] 2> "numShards":"2",
[junit4] 2> "shard_range":null,
[junit4] 2> "roles":null,
[junit4] 2> "collection":"collection1",
[junit4] 2> "shard":"shard1",
[junit4] 2> "core":"collection1"}
[junit4] 2> 194843 T692 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4] 2> 194843 T724 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4] 2> 194843 T679 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4] 2> 194844 T672 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4] 2> 194843 T741 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4] 2> 194844 T708 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4] 2> 195509 T704 oasha.CoreAdminHandler.handleWaitForStateAction Waited coreNodeName: core_node4, state: recovering, checkLive: true, onlyIfLeader: true for: 1 seconds.
[junit4] 2> 195510 T704 oass.SolrDispatchFilter.handleAdminRequest [admin] webapp=null path=/admin/cores params={state=recovering&onlyIfLeader=true&checkLive=true&action=PREPRECOVERY&version=2&coreNodeName=core_node4&wt=javabin&nodeName=127.0.0.1:48446_&core=collection1} status=0 QTime=1002
[junit4] 2> 195604 T688 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for onenodecollectioncore
[junit4] 2> 195604 T688 oasc.CoreContainer.create Creating SolrCore 'onenodecollectioncore' using instanceDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty1-1373828736141/onenodecollectioncore
[junit4] 2> 195605 T688 oasc.ZkController.createCollectionZkNode Check for collection zkNode:onenodecollection
[junit4] 2> 195606 T688 oasc.ZkController.createCollectionZkNode Creating collection in ZooKeeper:onenodecollection
[junit4] 2> 195606 T688 oasc.ZkController.getConfName Looking for collection configName
[junit4] 2> 195607 T688 oasc.ZkController.getConfName Only one config set found in zk - using it:conf1
[junit4] 2> 195608 T688 oascc.SolrZkClient.makePath makePath: /collections/onenodecollection
[junit4] 2> 195610 T688 oasc.ZkController.readConfigName Load collection config from:/collections/onenodecollection
[junit4] 2> 195612 T688 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty1-1373828736141/onenodecollectioncore/'
[junit4] 2> 195646 T688 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_45
[junit4] 2> 195680 T688 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4] 2> 195682 T688 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4] 2> 195689 T688 oass.IndexSchema.readSchema [onenodecollectioncore] Schema name=test
[junit4] 2> 196044 T688 oass.ByteField.init WARN ByteField is deprecated and will be removed in 5.0. You should use TrieIntField instead.
[junit4] 2> 196045 T688 oass.ShortField.init WARN ShortField is deprecated and will be removed in 5.0. You should use TrieIntField instead.
[junit4] 2> 196045 T688 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4] 2> 196052 T688 oass.IndexSchema.readSchema default search field in schema is text
[junit4] 2> 196055 T688 oass.IndexSchema.readSchema unique key field: id
[junit4] 2> 196067 T688 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 196071 T688 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 196075 T688 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4] 2> 196077 T688 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4] 2> 196077 T688 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4] 2> 196078 T688 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4] 2> 196079 T688 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4] 2> 196079 T688 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4] 2> 196080 T688 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4] 2> 196080 T688 oasc.SolrCore.<init> [onenodecollectioncore] Opening new SolrCore at /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty1-1373828736141/onenodecollectioncore/, dataDir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./solrtest-BasicDistributedZk2Test-1373828732652/onenodecollection/
[junit4] 2> 196081 T688 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@c70432b2
[junit4] 2> 196082 T688 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./solrtest-BasicDistributedZk2Test-1373828732652/onenodecollection
[junit4] 2> 196082 T688 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./solrtest-BasicDistributedZk2Test-1373828732652/onenodecollection/index/
[junit4] 2> 196083 T688 oasc.SolrCore.initIndex WARN [onenodecollectioncore] Solr index directory '/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./solrtest-BasicDistributedZk2Test-1373828732652/onenodecollection/index' doesn't exist. Creating new index...
[junit4] 2> 196084 T688 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./solrtest-BasicDistributedZk2Test-1373828732652/onenodecollection/index
[junit4] 2> 196085 T688 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@d4f04c68 lockFactory=org.apache.lucene.store.NativeFSLockFactory@7b93a78b),segFN=segments_1,generation=1}
[junit4] 2> 196086 T688 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 196088 T688 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4] 2> 196089 T688 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4] 2> 196090 T688 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 196090 T688 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4] 2> 196091 T688 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4] 2> 196092 T688 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4] 2> 196092 T688 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4] 2> 196093 T688 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4] 2> 196093 T688 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4] 2> 196095 T688 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 196097 T688 oass.SolrIndexSearcher.<init> Opening Searcher@43d6bb09 main
[junit4] 2> 196098 T688 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4] 2> 196098 T688 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4] 2> 196102 T747 oasc.SolrCore.registerSearcher [onenodecollectioncore] Registered new searcher Searcher@43d6bb09 main{StandardDirectoryReader(segments_1:1)}
[junit4] 2> 196103 T688 oasc.CoreContainer.registerCore registering core: onenodecollectioncore
[junit4] 2> 196103 T688 oasc.ZkController.register Register replica - core:onenodecollectioncore address:http://127.0.0.1:37394 collection:onenodecollection shard:shard1
[junit4] 2> 196104 T688 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=10000&maxConnectionsPerHost=20&connTimeout=30000&socketTimeout=30000&retry=false
[junit4] 2> 196108 T688 oascc.SolrZkClient.makePath makePath: /collections/onenodecollection/leader_elect/shard1/election
[junit4] 2> 196127 T688 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard1
[junit4] 2> 196133 T688 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4] 2> 196134 T688 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4] 2> 196134 T688 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:37394/onenodecollectioncore/
[junit4] 2> 196135 T688 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4] 2> 196137 T688 oasc.SyncStrategy.syncToMe http://127.0.0.1:37394/onenodecollectioncore/ has no replicas
[junit4] 2> 196137 T688 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:37394/onenodecollectioncore/ shard1
[junit4] 2> 196139 T688 oascc.SolrZkClient.makePath makePath: /collections/onenodecollection/leaders/shard1
[junit4] 2> 196354 T673 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 196363 T672 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4] 2> 196363 T679 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4] 2> 196363 T708 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4] 2> 196363 T741 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4] 2> 196363 T724 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4] 2> 196365 T692 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4] 2> 196400 T688 oasc.ZkController.register We are http://127.0.0.1:37394/onenodecollectioncore/ and leader is http://127.0.0.1:37394/onenodecollectioncore/
[junit4] 2> 196401 T688 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4] 2> 196401 T688 oasc.ZkController.publish publishing core=onenodecollectioncore state=active
[junit4] 2> 196404 T688 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 196405 T688 oass.SolrDispatchFilter.handleAdminRequest [admin] webapp=null path=/admin/cores params={dataDir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./solrtest-BasicDistributedZk2Test-1373828732652/onenodecollection&action=CREATE&version=2&numShards=1&name=onenodecollectioncore&wt=javabin&roles=none&collection=onenodecollection} status=0 QTime=1891
[junit4] 2> 196406 T650 oasc.AbstractDistribZkTestBase.waitForRecoveriesToFinish Wait for recoveries to finish - collection: onenodecollection failOnTimeout:true timeout (sec):330
[junit4] 2> 196408 T650 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 197410 T650 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> ASYNC NEW_CORE C281 name=collection1 org.apache.solr.core.SolrCore@e03e5a3b url=http://127.0.0.1:48446/collection1 node=127.0.0.1:48446_ C281_STATE=coll:collection1 core:collection1 props:{base_url=http://127.0.0.1:48446, state=recovering, node_name=127.0.0.1:48446_, core=collection1}
[junit4] 2> 197511 T744 C281 P48446 oasc.RecoveryStrategy.doRecovery Attempting to PeerSync from http://127.0.0.1:47766/collection1/ core=collection1 - recoveringAfterStartup=true
[junit4] 2> 197512 T744 C281 P48446 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:48446 START replicas=[http://127.0.0.1:47766/collection1/] nUpdates=100
[junit4] 2> 197512 T744 C281 P48446 oasu.PeerSync.sync WARN no frame of reference to tell of we've missed updates
[junit4] 2> 197512 T744 C281 P48446 oasc.RecoveryStrategy.doRecovery PeerSync Recovery was not successful - trying replication. core=collection1
[junit4] 2> 197513 T744 C281 P48446 oasc.RecoveryStrategy.doRecovery Starting Replication Recovery. core=collection1
[junit4] 2> 197513 T744 C281 P48446 oasc.RecoveryStrategy.doRecovery Begin buffering updates. core=collection1
[junit4] 2> 197514 T744 C281 P48446 oasu.UpdateLog.bufferUpdates Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null}
[junit4] 2> 197514 T744 C281 P48446 oasc.RecoveryStrategy.replicate Attempting to replicate from http://127.0.0.1:47766/collection1/. core=collection1
[junit4] 2> 197514 T744 C281 P48446 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4] 2> ASYNC NEW_CORE C282 name=collection1 org.apache.solr.core.SolrCore@9595f5cf url=http://127.0.0.1:47766/collection1 node=127.0.0.1:47766_ C282_STATE=coll:collection1 core:collection1 props:{base_url=http://127.0.0.1:47766, state=active, node_name=127.0.0.1:47766_, core=collection1, leader=true}
[junit4] 2> 197522 T703 C282 P47766 oasc.SolrCore.execute [collection1] webapp= path=/get params={getVersions=100&qt=/get&distrib=false&version=2&wt=javabin} status=0 QTime=0
[junit4] 2> 197526 T701 C282 P47766 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4] 2> 197527 T701 C282 P47766 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@d5bf3b93 lockFactory=org.apache.lucene.store.NativeFSLockFactory@6ef8d267),segFN=segments_1,generation=1}
[junit4] 2> 197528 T701 C282 P47766 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 197528 T701 C282 P47766 oasu.DirectUpdateHandler2.commit No uncommitted changes. Skipping IW.commit.
[junit4] 2> 197529 T701 C282 P47766 oass.SolrIndexSearcher.<init> Opening Searcher@a3a96a4b realtime
[junit4] 2> 197529 T701 C282 P47766 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4] 2> 197530 T701 C282 P47766 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={softCommit=false&commit_end_point=true&commit=true&version=2&waitSearcher=true&wt=javabin&openSearcher=false} {commit=} 0 4
[junit4] 2> 197531 T744 C281 P48446 oash.ReplicationHandler.inform Commits will be reserved for 10000
[junit4] 2> 197531 T744 C281 P48446 oash.SnapPuller.<init> No value set for 'pollInterval'. Timer Task not started.
[junit4] 2> 197534 T703 C282 P47766 oash.ReplicationHandler.inform Commits will be reserved for 10000
[junit4] 2> 197534 T703 C282 P47766 oasc.SolrCore.execute [collection1] webapp= path=/replication params={qt=/replication&version=2&command=indexversion&wt=javabin} status=0 QTime=1
[junit4] 2> 197535 T744 C281 P48446 oasc.RecoveryStrategy.replay No replay needed. core=collection1
[junit4] 2> 197536 T744 C281 P48446 oasc.RecoveryStrategy.doRecovery Replication Recovery was successful - registering as Active. core=collection1
[junit4] 2> 197536 T744 C281 P48446 oasc.ZkController.publish publishing core=collection1 state=active
[junit4] 2> 197537 T744 C281 P48446 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 197539 T744 C281 P48446 oasc.RecoveryStrategy.doRecovery Finished recovery process. core=collection1
[junit4] 2> 197871 T673 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 197873 T673 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4] 2> "shard_state":"active",
[junit4] 2> "base_url":"http://127.0.0.1:37394",
[junit4] 2> "state":"active",
[junit4] 2> "operation":"state",
[junit4] 2> "core_node_name":"core_node1",
[junit4] 2> "node_name":"127.0.0.1:37394_",
[junit4] 2> "numShards":"1",
[junit4] 2> "shard_range":null,
[junit4] 2> "roles":"none",
[junit4] 2> "collection":"onenodecollection",
[junit4] 2> "shard":"shard1",
[junit4] 2> "core":"onenodecollectioncore"}
[junit4] 2> 197877 T673 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4] 2> "shard_state":"active",
[junit4] 2> "base_url":"http://127.0.0.1:48446",
[junit4] 2> "state":"active",
[junit4] 2> "operation":"state",
[junit4] 2> "core_node_name":"core_node4",
[junit4] 2> "node_name":"127.0.0.1:48446_",
[junit4] 2> "numShards":"2",
[junit4] 2> "shard_range":null,
[junit4] 2> "roles":null,
[junit4] 2> "collection":"collection1",
[junit4] 2> "shard":"shard2",
[junit4] 2> "core":"collection1"}
[junit4] 2> 197883 T692 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4] 2> 197883 T741 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4] 2> 197884 T679 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4] 2> 197884 T708 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4] 2> 197886 T672 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4] 2> 197884 T724 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4] 2> 198412 T650 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 198414 T650 oasc.AbstractDistribZkTestBase.waitForRecoveriesToFinish Recoveries finished - collection: onenodecollection
[junit4] 2> 198415 T650 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4] 2> 198429 T650 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 198431 T750 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@38114397 name:ZooKeeperConnection Watcher:127.0.0.1:58961 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 198431 T650 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 198494 T650 oasc.ChaosMonkey.monkeyLog monkey: stop shard! 55788
[junit4] 2> 198495 T650 oasc.CoreContainer.shutdown Shutting down CoreContainer instance=1111087049
[junit4] 2> 199395 T673 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 199397 T673 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=null message={
[junit4] 2> "base_url":"http://127.0.0.1:55788",
[junit4] 2> "state":"down",
[junit4] 2> "operation":"state",
[junit4] 2> "core_node_name":"core_node1",
[junit4] 2> "node_name":"127.0.0.1:55788_",
[junit4] 2> "roles":null,
[junit4] 2> "collection":"control_collection",
[junit4] 2> "shard":null,
[junit4] 2> "core":"collection1"}
[junit4] 2> 199397 T673 oasc.Overseer$ClusterStateUpdater.updateState shard=shard1 is already registered
[junit4] 2> 199401 T692 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4] 2> 199401 T708 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4] 2> 199401 T724 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4] 2> 199401 T672 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4] 2> 199401 T741 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4] 2> 200497 T650 oasc.ZkController.publish publishing core=collection1 state=down
[junit4] 2> 200498 T650 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 200500 T650 oasc.SolrCore.close [collection1] CLOSING SolrCore org.apache.solr.core.SolrCore@f126dfc2
[junit4] 2> 200506 T650 oasu.DirectUpdateHandler2.close closing DirectUpdateHandler2{commits=0,autocommits=0,soft autocommits=0,optimizes=0,rollbacks=0,expungeDeletes=0,docsPending=0,adds=0,deletesById=0,deletesByQuery=0,errors=0,cumulative_adds=0,cumulative_deletesById=0,cumulative_deletesByQuery=0,cumulative_errors=0}
[junit4] 2> 200507 T650 oasu.SolrCoreState.decrefSolrCoreState Closing SolrCoreState
[junit4] 2> 200508 T650 oasu.DefaultSolrCoreState.closeIndexWriter SolrCoreState ref count has reached 0 - closing IndexWriter
[junit4] 2> 200508 T650 oasu.DefaultSolrCoreState.closeIndexWriter closing IndexWriter with IndexWriterCloser
[junit4] 2> 200509 T650 oasc.SolrCore.closeSearcher [collection1] Closing main searcher on request.
[junit4] 2> 200509 T650 oasc.CachingDirectoryFactory.close Closing MockDirectoryFactory - 2 directories currently being tracked
[junit4] 2> 200510 T650 oasc.CachingDirectoryFactory.closeCacheValue looking to close ./org.apache.solr.cloud.BasicDistributedZk2Test-1373828732652/control/data [CachedDir<<refCount=0;path=./org.apache.solr.cloud.BasicDistributedZk2Test-1373828732652/control/data;done=false>>]
[junit4] 2> 200511 T650 oasc.CachingDirectoryFactory.close Closing directory: ./org.apache.solr.cloud.BasicDistributedZk2Test-1373828732652/control/data
[junit4] 2> 200511 T650 oasc.CachingDirectoryFactory.closeCacheValue looking to close ./org.apache.solr.cloud.BasicDistributedZk2Test-1373828732652/control/data/index [CachedDir<<refCount=0;path=./org.apache.solr.cloud.BasicDistributedZk2Test-1373828732652/control/data/index;done=false>>]
[junit4] 2> 200512 T650 oasc.CachingDirectoryFactory.close Closing directory: ./org.apache.solr.cloud.BasicDistributedZk2Test-1373828732652/control/data/index
[junit4] 2> 200513 T673 oasc.Overseer$ClusterStateUpdater.amILeader According to ZK I (id=90035239823278083-127.0.0.1:55788_-n_0000000000) am no longer a leader.
[junit4] 2> 200514 T741 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4] 2> 200514 T672 oascc.ZkStateReader$3.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4] 2> 200514 T724 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4] 2> 200515 T672 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4] 2> 200514 T708 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4] 2> 200515 T672 oascc.ZkStateReader$2.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4] 2> 200515 T692 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
[junit4] 2> 200516 T708 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4] 2> 200517 T724 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4] 2> 200519 T741 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4] 2> 200520 T692 oasc.Overseer.start Overseer (id=90035239823278086-127.0.0.1:37394_-n_0000000001) starting
[junit4] 2> 200524 T752 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
[junit4] 2> 200525 T692 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4] 2> 200525 T692 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4] 2> 200526 T751 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
[junit4] 2> 200530 T751 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 200531 T751 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4] 2> "shard_state":"active",
[junit4] 2> "base_url":"http://127.0.0.1:55788",
[junit4] 2> "state":"down",
[junit4] 2> "operation":"state",
[junit4] 2> "core_node_name":"core_node1",
[junit4] 2> "node_name":"127.0.0.1:55788_",
[junit4] 2> "numShards":"2",
[junit4] 2> "shard_range":null,
[junit4] 2> "roles":null,
[junit4] 2> "collection":"control_collection",
[junit4] 2> "shard":"shard1",
[junit4] 2> "core":"collection1"}
[junit4] 2> 200535 T708 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4] 2> 200535 T724 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4] 2> 200535 T692 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4] 2> 200535 T741 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4] 2> 200538 T650 oejsh.ContextHandler.doStop stopped o.e.j.s.ServletContextHandler{/,null}
[junit4] 2> 200593 T650 oasc.ChaosMonkey.monkeyLog monkey: stop shard! 37394
[junit4] 2> 200594 T650 oasc.CoreContainer.shutdown Shutting down CoreContainer instance=72517398
[junit4] 2> 202042 T751 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 202043 T751 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=null message={
[junit4] 2> "base_url":"http://127.0.0.1:37394",
[junit4] 2> "state":"down",
[junit4] 2> "operation":"state",
[junit4] 2> "core_node_name":"core_node1",
[junit4] 2> "node_name":"127.0.0.1:37394_",
[junit4] 2> "roles":"none",
[junit4] 2> "collection":"onenodecollection",
[junit4] 2> "shard":null,
[junit4] 2> "core":"onenodecollectioncore"}
[junit4] 2> 202044 T751 oasc.Overseer$ClusterStateUpdater.updateState shard=shard1 is already registered
[junit4] 2> 202048 T751 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=null message={
[junit4] 2> "base_url":"http://127.0.0.1:37394",
[junit4] 2> "state":"down",
[junit4] 2> "operation":"state",
[junit4] 2> "core_node_name":"core_node1",
[junit4] 2> "node_name":"127.0.0.1:37394_",
[junit4] 2> "roles":null,
[junit4] 2> "collection":"collection1",
[junit4] 2> "shard":null,
[junit4] 2> "core":"collection1"}
[junit4] 2> 202048 T751 oasc.Overseer$ClusterStateUpdater.updateState shard=shard1 is already registered
[junit4] 2> 202053 T708 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4] 2> 202053 T724 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4] 2> 202053 T741 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4] 2> 202054 T692 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4] 2> 203597 T650 oasc.ZkController.publish publishing core=collection1 state=down
[junit4] 2> 203597 T650 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 203599 T650 oasc.ZkController.publish publishing core=onenodecollectioncore state=down
[junit4] 2> 203601 T650 oasc.SolrCore.close [collection1] CLOSING SolrCore org.apache.solr.core.SolrCore@364a9e90
[junit4] 2> 203606 T650 oasu.DirectUpdateHandler2.close closing DirectUpdateHandler2{commits=1,autocommits=0,soft autocommits=0,optimizes=0,rollbacks=0,expungeDeletes=0,docsPending=0,adds=0,deletesById=0,deletesByQuery=0,errors=0,cumulative_adds=0,cumulative_deletesById=0,cumulative_deletesByQuery=0,cumulative_errors=0}
[junit4] 2> 203606 T650 oasu.SolrCoreState.decrefSolrCoreState Closing SolrCoreState
[junit4] 2> 203607 T650 oasu.DefaultSolrCoreState.closeIndexWriter SolrCoreState ref count has reached 0 - closing IndexWriter
[junit4] 2> 203607 T650 oasu.DefaultSolrCoreState.closeIndexWriter closing IndexWriter with IndexWriterCloser
[junit4] 2> 203608 T650 oasc.SolrCore.closeSearcher [collection1] Closing main searcher on request.
[junit4] 2> 203609 T650 oasc.CachingDirectoryFactory.close Closing MockDirectoryFactory - 2 directories currently being tracked
[junit4] 2> 203609 T650 oasc.CachingDirectoryFactory.closeCacheValue looking to close ./org.apache.solr.cloud.BasicDistributedZk2Test-1373828732652/jetty1/index [CachedDir<<refCount=0;path=./org.apache.solr.cloud.BasicDistributedZk2Test-1373828732652/jetty1/index;done=false>>]
[junit4] 2> 203610 T650 oasc.CachingDirectoryFactory.close Closing directory: ./org.apache.solr.cloud.BasicDistributedZk2Test-1373828732652/jetty1/index
[junit4] 2> 203610 T650 oasc.CachingDirectoryFactory.closeCacheValue looking to close ./org.apache.solr.cloud.BasicDistributedZk2Test-1373828732652/jetty1 [CachedDir<<refCount=0;path=./org.apache.solr.cloud.BasicDistributedZk2Test-1373828732652/jetty1;done=false>>]
[junit4] 2> 203611 T650 oasc.CachingDirectoryFactory.close Closing directory: ./org.apache.solr.cloud.BasicDistributedZk2Test-1373828732652/jetty1
[junit4] 2> 203611 T650 oasc.SolrCore.close [onenodecollectioncore] CLOSING SolrCore org.apache.solr.core.SolrCore@1076208d
[junit4] 2> 203619 T650 oasu.DirectUpdateHandler2.close closing DirectUpdateHandler2{commits=0,autocommits=0,soft autocommits=0,optimizes=0,rollbacks=0,expungeDeletes=0,docsPending=0,adds=0,deletesById=0,deletesByQuery=0,errors=0,cumulative_adds=0,cumulative_deletesById=0,cumulative_deletesByQuery=0,cumulative_errors=0}
[junit4] 2> 203620 T650 oasu.SolrCoreState.decrefSolrCoreState Closing SolrCoreState
[junit4] 2> 203620 T650 oasu.DefaultSolrCoreState.closeIndexWriter SolrCoreState ref count has reached 0 - closing IndexWriter
[junit4] 2> 203621 T650 oasu.DefaultSolrCoreState.closeIndexWriter closing IndexWriter with IndexWriterCloser
[junit4] 2> 203622 T650 oasc.SolrCore.closeSearcher [onenodecollectioncore] Closing main searcher on request.
[junit4] 2> 203622 T650 oasc.CachingDirectoryFactory.close Closing MockDirectoryFactory - 2 directories currently being tracked
[junit4] 2> 203623 T650 oasc.CachingDirectoryFactory.closeCacheValue looking to close /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./solrtest-BasicDistributedZk2Test-1373828732652/onenodecollection [CachedDir<<refCount=0;path=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./solrtest-BasicDistributedZk2Test-1373828732652/onenodecollection;done=false>>]
[junit4] 2> 203623 T650 oasc.CachingDirectoryFactory.close Closing directory: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./solrtest-BasicDistributedZk2Test-1373828732652/onenodecollection
[junit4] 2> 203624 T650 oasc.CachingDirectoryFactory.closeCacheValue looking to close /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./solrtest-BasicDistributedZk2Test-1373828732652/onenodecollection/index [CachedDir<<refCount=0;path=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./solrtest-BasicDistributedZk2Test-1373828732652/onenodecollection/index;done=false>>]
[junit4] 2> 203624 T650 oasc.CachingDirectoryFactory.close Closing directory: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./solrtest-BasicDistributedZk2Test-1373828732652/onenodecollection/index
[junit4] 2> 203626 T751 oasc.Overseer$ClusterStateUpdater.amILeader According to ZK I (id=90035239823278086-127.0.0.1:37394_-n_0000000001) am no longer a leader.
[junit4] 2> 203627 T741 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4] 2> 203627 T724 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4] 2> 203628 T708 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4] 2> 203629 T708 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4] 2> 203630 T741 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4] 2> 203631 T724 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4] 2> 203632 T724 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard1
[junit4] 2> 203633 T708 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
[junit4] 2> 203635 T724 oasc.ShardLeaderElectionContext.shouldIBeLeader Checking if I should try and be the leader.
[junit4] 2> 203635 T724 oasc.ShardLeaderElectionContext.shouldIBeLeader My last published State was Active, it's okay to be the leader.
[junit4] 2> 203635 T724 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4] 2> 203636 T724 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:43401/collection1/
[junit4] 2> 203636 T724 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4] 2> 203636 T724 oasc.SyncStrategy.syncToMe http://127.0.0.1:43401/collection1/ has no replicas
[junit4] 2> 203637 T724 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:43401/collection1/ shard1
[junit4] 2> 203637 T724 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
[junit4] 2> 203638 T708 oasc.Overseer.start Overseer (id=90035239823278088-127.0.0.1:47766_-n_0000000002) starting
[junit4] 2> 203643 T754 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
[junit4] 2> 203645 T753 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
[junit4] 2> 203646 T753 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 203648 T753 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4] 2> "shard_state":"active",
[junit4] 2> "base_url":"http://127.0.0.1:37394",
[junit4] 2> "state":"down",
[junit4] 2> "operation":"state",
[junit4] 2> "core_node_name":"core_node1",
[junit4] 2> "node_name":"127.0.0.1:37394_",
[junit4] 2> "numShards":"2",
[junit4] 2> "shard_range":null,
[junit4] 2> "roles":null,
[junit4] 2> "collection":"collection1",
[junit4] 2> "shard":"shard1",
[junit4] 2> "core":"collection1"}
[junit4] 2> 203649 T650 oejsh.ContextHandler.doStop stopped o.e.j.s.ServletContextHandler{/,null}
[junit4] 2> 203651 T753 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4] 2> "shard_state":"active",
[junit4] 2> "base_url":"http://127.0.0.1:37394",
[junit4] 2> "state":"down",
[junit4] 2> "operation":"state",
[junit4] 2> "core_node_name":"core_node1",
[junit4] 2> "node_name":"127.0.0.1:37394_",
[junit4] 2> "numShards":"1",
[junit4] 2> "shard_range":null,
[junit4] 2> "roles":"none",
[junit4] 2> "collection":"onenodecollection",
[junit4] 2> "shard":"shard1",
[junit4] 2> "core":"onenodecollectioncore"}
[junit4] 2> 203664 T724 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 203664 T741 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 203665 T708 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 203700 T650 oasc.ChaosMonkey.monkeyLog monkey: stop shard! 47766
[junit4] 2> 203701 T650 oasc.CoreContainer.shutdown Shutting down CoreContainer instance=351591603
[junit4] 2> 204704 T650 oasc.ZkController.publish publishing core=collection1 state=down
[junit4] 2> 204705 T650 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 204706 T650 oasc.SolrCore.close [collection1] CLOSING SolrCore org.apache.solr.core.SolrCore@9595f5cf
[junit4] 2> 204711 T650 oasu.DirectUpdateHandler2.close closing DirectUpdateHandler2{commits=1,autocommits=0,soft autocommits=0,optimizes=0,rollbacks=0,expungeDeletes=0,docsPending=0,adds=0,deletesById=0,deletesByQuery=0,errors=0,cumulative_adds=0,cumulative_deletesById=0,cumulative_deletesByQuery=0,cumulative_errors=0}
[junit4] 2> 204712 T650 oasu.SolrCoreState.decrefSolrCoreState Closing SolrCoreState
[junit4] 2> 204712 T650 oasu.DefaultSolrCoreState.closeIndexWriter SolrCoreState ref count has reached 0 - closing IndexWriter
[junit4] 2> 204713 T650 oasu.DefaultSolrCoreState.closeIndexWriter closing IndexWriter with IndexWriterCloser
[junit4] 2> 204714 T650 oasc.SolrCore.closeSearcher [collection1] Closing main searcher on request.
[junit4] 2> 204714 T650 oasc.CachingDirectoryFactory.close Closing MockDirectoryFactory - 2 directories currently being tracked
[junit4] 2> 204715 T650 oasc.CachingDirectoryFactory.closeCacheValue looking to close ./org.apache.solr.cloud.BasicDistributedZk2Test-1373828732652/jetty2 [CachedDir<<refCount=0;path=./org.apache.solr.cloud.BasicDistributedZk2Test-1373828732652/jetty2;done=false>>]
[junit4] 2> 204715 T650 oasc.CachingDirectoryFactory.close Closing directory: ./org.apache.solr.cloud.BasicDistributedZk2Test-1373828732652/jetty2
[junit4] 2> 204716 T650 oasc.CachingDirectoryFactory.closeCacheValue looking to close ./org.apache.solr.cloud.BasicDistributedZk2Test-1373828732652/jetty2/index [CachedDir<<refCount=0;path=./org.apache.solr.cloud.BasicDistributedZk2Test-1373828732652/jetty2/index;done=false>>]
[junit4] 2> 204716 T650 oasc.CachingDirectoryFactory.close Closing directory: ./org.apache.solr.cloud.BasicDistributedZk2Test-1373828732652/jetty2/index
[junit4] 2> 204718 T753 oasc.Overseer$ClusterStateUpdater.amILeader According to ZK I (id=90035239823278088-127.0.0.1:47766_-n_0000000002) am no longer a leader.
[junit4] 2> 204718 T724 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4] 2> 204718 T741 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4] 2> 204721 T724 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4] 2> 204721 T741 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4] 2> 204722 T724 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
[junit4] 2> 204723 T741 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard2
[junit4] 2> 204725 T741 oasc.ShardLeaderElectionContext.shouldIBeLeader Checking if I should try and be the leader.
[junit4] 2> 204725 T724 oasc.Overseer.start Overseer (id=90035239823278090-127.0.0.1:43401_-n_0000000003) starting
[junit4] 2> 204725 T741 oasc.ShardLeaderElectionContext.shouldIBeLeader My last published State was Active, it's okay to be the leader.
[junit4] 2> 204726 T741 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4] 2> 204726 T741 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:48446/collection1/
[junit4] 2> 204726 T741 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4] 2> 204726 T741 oasc.SyncStrategy.syncToMe http://127.0.0.1:48446/collection1/ has no replicas
[junit4] 2> 204727 T741 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:48446/collection1/ shard2
[junit4] 2> 204727 T741 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard2
[junit4] 2> 204731 T756 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
[junit4] 2> 204732 T755 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
[junit4] 2> 204734 T755 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 204735 T755 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=null message={
[junit4] 2> "base_url":"http://127.0.0.1:47766",
[junit4] 2> "state":"down",
[junit4] 2> "operation":"state",
[junit4] 2> "core_node_name":"core_node2",
[junit4] 2> "node_name":"127.0.0.1:47766_",
[junit4] 2> "roles":null,
[junit4] 2> "collection":"collection1",
[junit4] 2> "shard":null,
[junit4] 2> "core":"collection1"}
[junit4] 2> 204735 T755 oasc.Overseer$ClusterStateUpdater.updateState shard=shard2 is already registered
[junit4] 2> 204739 T755 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4] 2> "shard_state":"active",
[junit4] 2> "base_url":"http://127.0.0.1:47766",
[junit4] 2> "state":"down",
[junit4] 2> "operation":"state",
[junit4] 2> "core_node_name":"core_node2",
[junit4] 2> "node_name":"127.0.0.1:47766_",
[junit4] 2> "numShards":"2",
[junit4] 2> "shard_range":null,
[junit4] 2> "roles":null,
[junit4] 2> "collection":"collection1",
[junit4] 2> "shard":"shard2",
[junit4] 2> "core":"collection1"}
[junit4] 2> 204741 T650 oejsh.ContextHandler.doStop stopped o.e.j.s.ServletContextHandler{/,null}
[junit4] 2> 204747 T741 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 204748 T724 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 204793 T650 oasc.ChaosMonkey.monkeyLog monkey: stop shard! 43401
[junit4] 2> 204793 T650 oasc.CoreContainer.shutdown Shutting down CoreContainer instance=-1108876977
[junit4] 2> 204927 T692 oascc.ZkStateReader$3.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4] 2> 204928 T692 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4] 2> 204928 T692 oascc.ZkStateReader$2.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4] 2> 205795 T650 oasc.ZkController.publish publishing core=collection1 state=down
[junit4] 2> 205796 T650 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 205797 T650 oasc.SolrCore.close [collection1] CLOSING SolrCore org.apache.solr.core.SolrCore@a3f8ae6f
[junit4] 2> 205802 T650 oasu.DirectUpdateHandler2.close closing DirectUpdateHandler2{commits=0,autocommits=0,soft autocommits=0,optimizes=0,rollbacks=0,expungeDeletes=0,docsPending=0,adds=0,deletesById=0,deletesByQuery=0,errors=0,cumulative_adds=0,cumulative_deletesById=0,cumulative_deletesByQuery=0,cumulative_errors=0}
[junit4] 2> 205803 T650 oasu.SolrCoreState.decrefSolrCoreState Closing SolrCoreState
[junit4] 2> 205803 T650 oasu.DefaultSolrCoreState.closeIndexWriter SolrCoreState ref count has reached 0 - closing IndexWriter
[junit4] 2> 205804 T650 oasu.DefaultSolrCoreState.closeIndexWriter closing IndexWriter with IndexWriterCloser
[junit4] 2> 205804 T650 oasc.SolrCore.closeSearcher [collection1] Closing main searcher on request.
[junit4] 2> 205805 T650 oasc.CachingDirectoryFactory.close Closing MockDirectoryFactory - 2 directories currently being tracked
[junit4] 2> 205805 T650 oasc.CachingDirectoryFactory.closeCacheValue looking to close ./org.apache.solr.cloud.BasicDistributedZk2Test-1373828732652/jetty3/index [CachedDir<<refCount=0;path=./org.apache.solr.cloud.BasicDistributedZk2Test-1373828732652/jetty3/index;done=false>>]
[junit4] 2> 205806 T650 oasc.CachingDirectoryFactory.close Closing directory: ./org.apache.solr.cloud.BasicDistributedZk2Test-1373828732652/jetty3/index
[junit4] 2> 205806 T650 oasc.CachingDirectoryFactory.closeCacheValue looking to close ./org.apache.solr.cloud.BasicDistributedZk2Test-1373828732652/jetty3 [CachedDir<<refCount=0;path=./org.apache.solr.cloud.BasicDistributedZk2Test-1373828732652/jetty3;done=false>>]
[junit4] 2> 205807 T650 oasc.CachingDirectoryFactory.close Closing directory: ./org.apache.solr.cloud.BasicDistributedZk2Test-1373828732652/jetty3
[junit4] 2> 205808 T755 oasc.Overseer$ClusterStateUpdater.amILeader According to ZK I (id=90035239823278090-127.0.0.1:43401_-n_0000000003) am no longer a leader.
[junit4] 2> 205808 T724 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
[junit4] 2> 205809 T741 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
[junit4] 2> 205811 T741 oasc.Overseer.start Overseer (id=90035239823278092-127.0.0.1:48446_-n_0000000004) starting
[junit4] 2> 205815 T741 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
[junit4] 2> 205815 T758 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
[junit4] 2> 205819 T741 oascc.ZkStateReader$3.process Updating live nodes... (1)
[junit4] 2> 205820 T757 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
[junit4] 2> 205821 T757 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 205822 T757 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=null message={
[junit4] 2> "base_url":"http://127.0.0.1:43401",
[junit4] 2> "state":"down",
[junit4] 2> "operation":"state",
[junit4] 2> "core_node_name":"core_node3",
[junit4] 2> "node_name":"127.0.0.1:43401_",
[junit4] 2> "roles":null,
[junit4] 2> "collection":"collection1",
[junit4] 2> "shard":null,
[junit4] 2> "core":"collection1"}
[junit4] 2> 205823 T757 oasc.Overseer$ClusterStateUpdater.updateState shard=shard1 is already registered
[junit4] 2> 205827 T757 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4] 2> "shard_state":"active",
[junit4] 2> "base_url":"http://127.0.0.1:43401",
[junit4] 2> "state":"down",
[junit4] 2> "operation":"state",
[junit4] 2> "core_node_name":"core_node3",
[junit4] 2> "node_name":"127.0.0.1:43401_",
[junit4] 2> "numShards":"2",
[junit4] 2> "shard_range":null,
[junit4] 2> "roles":null,
[junit4] 2> "collection":"collection1",
[junit4] 2> "shard":"shard1",
[junit4] 2> "core":"collection1"}
[junit4] 2> 205829 T650 oejsh.ContextHandler.doStop stopped o.e.j.s.ServletContextHandler{/,null}
[junit4] 2> 205831 T741 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 205881 T650 oasc.ChaosMonkey.monkeyLog monkey: stop shard! 48446
[junit4] 2> 205882 T650 oasc.CoreContainer.shutdown Shutting down CoreContainer instance=-1082656268
[junit4] 2> 206019 T708 oascc.ZkStateReader$3.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4] 2> 206019 T708 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4] 2> 206019 T708 oascc.ZkStateReader$2.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4] 2> 206883 T650 oasc.ZkController.publish publishing core=collection1 state=down
[junit4] 2> 206884 T650 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 206886 T650 oasc.SolrCore.close [collection1] CLOSING SolrCore org.apache.solr.core.SolrCore@e03e5a3b
[junit4] 2> 206894 T650 oasu.DirectUpdateHandler2.close closing DirectUpdateHandler2{commits=0,autocommits=0,soft autocommits=0,optimizes=0,rollbacks=0,expungeDeletes=0,docsPending=0,adds=0,deletesById=0,deletesByQuery=0,errors=0,cumulative_adds=0,cumulative_deletesById=0,cumulative_deletesByQuery=0,cumulative_errors=0}
[junit4] 2> 206894 T650 oasu.SolrCoreState.decrefSolrCoreState Closing SolrCoreState
[junit4] 2> 206895 T650 oasu.DefaultSolrCoreState.closeIndexWriter SolrCoreState ref count has reached 0 - closing IndexWriter
[junit4] 2> 206895 T650 oasu.DefaultSolrCoreState.closeIndexWriter closing IndexWriter with IndexWriterCloser
[junit4] 2> 206896 T650 oasc.SolrCore.closeSearcher [collection1] Closing main searcher on request.
[junit4] 2> 206897 T650 oasc.CachingDirectoryFactory.close Closing MockDirectoryFactory - 2 directories currently being tracked
[junit4] 2> 206897 T650 oasc.CachingDirectoryFactory.closeCacheValue looking to close ./org.apache.solr.cloud.BasicDistributedZk2Test-1373828732652/jetty4 [CachedDir<<refCount=0;path=./org.apache.solr.cloud.BasicDistributedZk2Test-1373828732652/jetty4;done=false>>]
[junit4] 2> 206898 T650 oasc.CachingDirectoryFactory.close Closing directory: ./org.apache.solr.cloud.BasicDistributedZk2Test-1373828732652/jetty4
[junit4] 2> 206898 T650 oasc.CachingDirectoryFactory.closeCacheValue looking to close ./org.apache.solr.cloud.BasicDistributedZk2Test-1373828732652/jetty4/index [CachedDir<<refCount=0;path=./org.apache.solr.cloud.BasicDistributedZk2Test-1373828732652/jetty4/index;done=false>>]
[junit4] 2> 206899 T650 oasc.CachingDirectoryFactory.close Closing directory: ./org.apache.solr.cloud.BasicDistributedZk2Test-1373828732652/jetty4/index
[junit4] 2> 206900 T757 oasc.Overseer$ClusterStateUpdater.amILeader According to ZK I (id=90035239823278092-127.0.0.1:48446_-n_0000000004) am no longer a leader.
[junit4] 2> 206901 T741 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
[junit4] 2> 206902 T741 oascc.ZkStateReader$2.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4] 2> 206902 T741 oascc.ZkStateReader$3.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4] 2> 206922 T650 oejsh.ContextHandler.doStop stopped o.e.j.s.ServletContextHandler{/,null}
[junit4] 2> 206978 T650 oas.SolrTestCaseJ4.tearDown ###Ending testDistribSearch
[junit4] 2> 206980 T650 oasc.ZkTestServer.send4LetterWord connecting to 127.0.0.1:58961 58961
[junit4] 1> client port:0.0.0.0/0.0.0.0:0
[junit4] 1> / (2)
[junit4] 1> DATA:
[junit4] 1>
[junit4] 1> /solr (7)
[junit4] 1> /solr/configs (1)
[junit4] 1> /solr/configs/conf1 (9)
[junit4] 1> /solr/configs/conf1/old_synonyms.txt (0)
[junit4] 1> DATA: ...supressed...
[junit4] 1> /solr/configs/conf1/protwords.txt (0)
[junit4] 1> DATA: ...supressed...
[junit4] 1> /solr/configs/conf1/stopwords.txt (0)
[junit4] 1> DATA: ...supressed...
[junit4] 1> /solr/configs/conf1/synonyms.txt (0)
[junit4] 1> DATA: ...supressed...
[junit4] 1> /solr/configs/conf1/schema.xml (0)
[junit4] 1> DATA: ...supressed...
[junit4] 1> /solr/configs/conf1/currency.xml (0)
[junit4] 1> DATA: ...supressed...
[junit4] 1> /solr/configs/conf1/open-exchange-rates.json (0)
[junit4] 1> DATA:
[junit4] 1> {
[junit4] 1> "disclaimer": "This data is not real, it was synthetically created to match currency.xml. It is modeled after the data format available from openexchangerates.org. See https://openexchangerates.org/documentation for details",
[junit4] 1> "license": "http://www.apache.org/licenses/LICENSE-2.0",
[junit4] 1> "timestamp": 1332070464,
[junit4] 1>
[junit4] 1>
[junit4] 1> "IMPORTANT NOTE": "In order for tests to work, this data must be kept in sync with ./currency.xml",
[junit4] 1>
[junit4] 1>
[junit4] 1> "base": "USD",
[junit4] 1> "rates": {
[junit4] 1> "USD": 1,
[junit4] 1> "JPY": 81.29,
[junit4] 1> "EUR": 2.5,
[junit4] 1> "GBP": 0.5,
[junit4] 1> "MXN": 2.0
[junit4] 1> }
[junit4] 1> }
[junit4] 1>
[junit4] 1> /solr/configs/conf1/solrconfig.xml (0)
[junit4] 1> DATA: ...supressed...
[junit4] 1> /solr/configs/conf1/mapping-ISOLatin1Accent.txt (0)
[junit4] 1> DATA: ...supressed...
[junit4] 1> /solr/overseer (3)
[junit4] 1> DATA:
[junit4] 1>
[junit4] 1> /solr/overseer/queue (0)
[junit4] 1> /solr/overseer/collection-queue-work (0)
[junit4] 1> /solr/overseer/queue-work (0)
[junit4] 1> /solr/overseer_elect (2)
[junit4] 1> /solr/overseer_elect/leader (0)
[junit4] 1> DATA:
[junit4] 1> {"id":"90035239823278083-127.0.0.1:55788_-n_0000000000"}
[junit4] 1> /solr/overseer_elect/election (5)
[junit4] 1> /solr/overseer_elect/election/90035239823278083-127.0.0.1:55788_-n_0000000000 (0)
[junit4] 1> /solr/overseer_elect/election/90035239823278088-127.0.0.1:47766_-n_0000000002 (0)
[junit4] 1> /solr/overseer_elect/election/90035239823278092-127.0.0.1:48446_-n_0000000004 (0)
[junit4] 1> /solr/overseer_elect/election/90035239823278090-127.0.0.1:43401_-n_0000000003 (0)
[junit4] 1> /solr/overseer_elect/election/90035239823278086-127.0.0.1:37394_-n_0000000001 (0)
[junit4] 1> /solr/collections (3)
[junit4] 1> /solr/collections/control_collection (3)
[junit4] 1> DATA:
[junit4] 1> {"configName":"conf1"}
[junit4] 1> /solr/collections/control_collection/leader_elect (1)
[junit4] 1> /solr/collections/control_collection/leader_elect/shard1 (1)
[junit4] 1> /solr/collections/control_collection/leader_elect/shard1/election (1)
[junit4] 1> /solr/collections/control_collection/leader_elect/shard1/election/90035239823278083-core_node1-n_0000000000 (0)
[junit4] 1> /solr/collections/control_collection/leaders (1)
[junit4] 1> /solr/collections/control_collection/leaders/shard1 (0)
[junit4] 1> DATA:
[junit4] 1> {
[junit4] 1> "base_url":"http://127.0.0.1:55788",
[junit4] 1> "node_name":"127.0.0.1:55788_",
[junit4] 1> "core":"collection1"}
[junit4] 1> /solr/collections/control_collection/shards (0)
[junit4] 1> /solr/collections/collection1 (3)
[junit4] 1> DATA:
[junit4] 1> {"configName":"conf1"}
[junit4] 1> /solr/collections/collection1/leader_elect (2)
[junit4] 1> /solr/collections/collection1/leader_elect/shard1 (1)
[junit4] 1> /solr/collections/collection1/leader_elect/shard1/election (2)
[junit4] 1> /solr/collections/collection1/leader_elect/shard1/election/90035239823278090-core_node3-n_0000000001 (0)
[junit4] 1> /solr/collections/collection1/leader_elect/shard1/election/90035239823278086-core_node1-n_0000000000 (0)
[junit4] 1> /solr/collections/collection1/leader_elect/shard2 (1)
[junit4] 1> /solr/collections/collection1/leader_elect/shard2/election (2)
[junit4] 1> /solr/collections/collection1/leader_elect/shard2/election/90035239823278092-core_node4-n_0000000001 (0)
[junit4] 1> /solr/collections/collection1/leader_elect/shard2/election/90035239823278088-core_node2-n_0000000000 (0)
[junit4] 1> /solr/collections/collection1/leaders (2)
[junit4] 1> /solr/collections/collection1/leaders/shard1 (0)
[junit4] 1> DATA:
[junit4] 1> {
[junit4] 1> "base_url":"http://127.0.0.1:37394",
[junit4] 1> "node_name":"127.0.0.1:37394_",
[junit4] 1> "core":"collection1"}
[junit4] 1> /solr/collections/collection1/leaders/shard2 (0)
[junit4] 1> DATA:
[junit4] 1> {
[junit4] 1> "base_url":"http://127.0.0.1:47766",
[junit4] 1> "node_name":"127.0.0.1:47766_",
[junit4] 1> "core":"collection1"}
[junit4] 1> /solr/collections/collection1/shards (0)
[junit4] 1> /solr/collections/onenodecollection (2)
[junit4] 1> DATA:
[junit4] 1> {
[junit4] 1> "router":"compositeId",
[junit4] 1> "configName":"conf1"}
[junit4] 1> /solr/collections/onenodecollection/leader_elect (1)
[junit4] 1> /solr/collections/onenodecollection/leader_elect/shard1 (1)
[junit4] 1> /solr/collections/onenodecollection/leader_elect/shard1/election (1)
[junit4] 1> /solr/collections/onenodecollection/leader_elect/shard1/election/90035239823278086-core_node1-n_0000000000 (0)
[junit4] 1> /solr/collections/onenodecollection/leaders (1)
[junit4] 1> /solr/collections/onenodecollection/leaders/shard1 (0)
[junit4] 1> DATA:
[junit4] 1> {
[junit4] 1> "base_url":"http://127.0.0.1:37394",
[junit4] 1> "node_name":"127.0.0.1:37394_",
[junit4] 1> "core":"onenodecollectioncore"}
[junit4] 1> /solr/aliases.json (0)
[junit4] 1> /solr/live_nodes (5)
[junit4] 1> /solr/live_nodes/127.0.0.1:47766_ (0)
[junit4] 1> /solr/live_nodes/127.0.0.1:48446_ (0)
[junit4] 1> /solr/live_nodes/127.0.0.1:37394_ (0)
[junit4] 1> /solr/live_nodes/127.0.0.1:55788_ (0)
[junit4] 1> /solr/live_nodes/127.0.0.1:43401_ (0)
[junit4] 1> /solr/clusterstate.json (0)
[junit4] 1> DATA:
[junit4] 1> {
[junit4] 1> "control_collection":{
[junit4] 1> "shards":{"shard1":{
[junit4] 1> "range":"80000000-7fffffff",
[junit4] 1> "state":"active",
[junit4] 1> "replicas":{"core_node1":{
[junit4] 1> "base_url":"http://127.0.0.1:55788",
[junit4] 1> "state":"active",
[junit4] 1> "node_name":"127.0.0.1:55788_",
[junit4] 1> "core":"collection1",
[junit4] 1> "leader":"true"}}}},
[junit4] 1> "router":"compositeId"},
[junit4] 1> "onenodecollection":{
[junit4] 1> "shards":{"shard1":{
[junit4] 1> "range":"80000000-7fffffff",
[junit4] 1> "state":"active",
[junit4] 1> "replicas":{"core_node1":{
[junit4] 1> "base_url":"http://127.0.0.1:37394",
[junit4] 1> "state":"active",
[junit4] 1> "node_name":"127.0.0.1:37394_",
[junit4] 1> "roles":"none",
[junit4] 1> "core":"onenodecollectioncore",
[junit4] 1> "leader":"true"}}}},
[junit4] 1> "router":"compositeId"},
[junit4] 1> "collection1":{
[junit4] 1> "shards":{
[junit4] 1> "shard1":{
[junit4] 1> "range":"80000000-ffffffff",
[junit4] 1> "state":"active",
[junit4] 1> "replicas":{
[junit4] 1> "core_node1":{
[junit4] 1> "base_url":"http://127.0.0.1:37394",
[junit4] 1> "state":"active",
[junit4] 1> "node_name":"127.0.0.1:37394_",
[junit4] 1> "core":"collection1",
[junit4] 1> "leader":"true"},
[junit4] 1> "core_node3":{
[junit4] 1> "base_url":"http://127.0.0.1:43401",
[junit4] 1> "state":"active",
[junit4] 1> "node_name":"127.0.0.1:43401_",
[junit4] 1> "core":"collection1"}}},
[junit4] 1> "shard2":{
[junit4] 1> "range":"0-7fffffff",
[junit4] 1> "state":"active",
[junit4] 1> "replicas":{
[junit4] 1> "core_node2":{
[junit4] 1> "base_url":"http://127.0.0.1:47766",
[junit4] 1> "state":"active",
[junit4] 1> "node_name":"127.0.0.1:47766_",
[junit4] 1> "core":"collection1",
[junit4] 1> "leader":"true"},
[junit4] 1> "core_node4":{
[junit4] 1> "base_url":"http://127.0.0.1:48446",
[junit4] 1> "state":"active",
[junit4] 1> "node_name":"127.0.0.1:48446_",
[junit4] 1> "core":"collection1"}}}},
[junit4] 1> "router":"compositeId"}}
[junit4] 1> /zookeeper (1)
[junit4] 1> DATA:
[junit4] 1>
[junit4] 1>
[junit4] 2> NOTE: reproduce with: ant test -Dtestcase=BasicDistributedZk2Test -Dtests.method=testDistribSearch -Dtests.seed=119935A2FA79F23E -Dtests.multiplier=3 -Dtests.slow=true -Dtests.locale=sr_RS_#Latn -Dtests.timezone=Australia/Currie -Dtests.file.encoding=UTF-8
[junit4] ERROR 27.8s J1 | BasicDistributedZk2Test.testDistribSearch <<<
[junit4] > Throwable #1: org.apache.solr.client.solrj.impl.HttpSolrServer$RemoteSolrException: Server at http://127.0.0.1:47766/onenodecollectioncore returned non ok status:404, message:Can not find: /onenodecollectioncore/update
[junit4] > at __randomizedtesting.SeedInfo.seed([119935A2FA79F23E:907FBBBA8D269202]:0)
[junit4] > at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:385)
[junit4] > at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:180)
[junit4] > at org.apache.solr.client.solrj.request.AbstractUpdateRequest.process(AbstractUpdateRequest.java:117)
[junit4] > at org.apache.solr.client.solrj.SolrServer.add(SolrServer.java:116)
[junit4] > at org.apache.solr.client.solrj.SolrServer.add(SolrServer.java:102)
[junit4] > at org.apache.solr.cloud.BasicDistributedZk2Test.testNodeWithoutCollectionForwarding(BasicDistributedZk2Test.java:196)
[junit4] > at org.apache.solr.cloud.BasicDistributedZk2Test.doTest(BasicDistributedZk2Test.java:88)
[junit4] > at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:835)
[junit4] > at java.lang.Thread.run(Thread.java:780)
[junit4] 2> 207005 T650 oas.SolrTestCaseJ4.deleteCore ###deleteCore
[junit4] 2> 27761 T649 ccr.ThreadLeakControl.checkThreadLeaks WARNING Will linger awaiting termination of 2 leaked thread(s).
[junit4] 2> 207109 T724 oascc.ZkStateReader$2.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4] 2> 207109 T724 oascc.ZkStateReader$3.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4] 2> NOTE: test params are: codec=Lucene42: {}, docValues:{}, sim=RandomSimilarityProvider(queryNorm=false,coord=yes): {}, locale=sr_RS_#Latn, timezone=Australia/Currie
[junit4] 2> NOTE: Linux 3.2.0-49-generic x86/IBM Corporation 1.7.0 (32-bit)/cpus=8,threads=1,free=22653720,total=157024256
[junit4] 2> NOTE: All tests run in this JVM: [TestDynamicFieldCollectionResource, TestFiltering, TestRecovery, HardAutoCommitTest, JSONWriterTest, SolrCoreCheckLockOnStartupTest, DocumentAnalysisRequestHandlerTest, TestCollationField, NumericFieldsTest, SpellPossibilityIteratorTest, TestLuceneMatchVersion, DeleteShardTest, MultiTermTest, TestUpdate, DistanceFunctionTest, TestCoreDiscovery, TestPartialUpdateDeduplication, CachingDirectoryFactoryTest, SpellCheckComponentTest, TestSolrQueryParserResource, SolrIndexConfigTest, SolrCmdDistributorTest, TestCloudManagedSchema, TestPropInject, AssignTest, TestClassNameShortening, SolrRequestParserTest, OutputWriterTest, TestFieldTypeCollectionResource, IndexSchemaRuntimeFieldTest, TestBinaryField, InfoHandlerTest, CurrencyFieldOpenExchangeTest, NoCacheHeaderTest, BasicDistributedZk2Test]
[junit4] Completed on J1 in 28.54s, 1 test, 1 error <<< FAILURES!
[...truncated 785 lines...]
BUILD FAILED
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/build.xml:395: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/build.xml:388: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/build.xml:39: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build.xml:181: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/common-build.xml:449: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/lucene/common-build.xml:1250: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/lucene/common-build.xml:893: There were test failures: 319 suites, 1345 tests, 1 error, 35 ignored (7 assumptions)
Total time: 48 minutes 44 seconds
Build step 'Invoke Ant' marked build as failure
Description set: Java: 32bit/ibm-j9-jdk7 -Xjit:exclude={org/apache/lucene/util/fst/FST.pack(IIF)Lorg/apache/lucene/util/fst/FST;}
Archiving artifacts
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure
[JENKINS] Lucene-Solr-4.x-Linux (32bit/ibm-j9-jdk7) - Build # 6488
- Still Failing!
Posted by Policeman Jenkins Server <je...@thetaphi.de>.
Build: http://jenkins.thetaphi.de/job/Lucene-Solr-4.x-Linux/6488/
Java: 32bit/ibm-j9-jdk7 -Xjit:exclude={org/apache/lucene/util/fst/FST.pack(IIF)Lorg/apache/lucene/util/fst/FST;}
1 tests failed.
FAILED: org.apache.solr.cloud.BasicDistributedZk2Test.testDistribSearch
Error Message:
Server at http://127.0.0.1:55399/_eg/onenodecollectioncore returned non ok status:404, message:Can not find: /_eg/onenodecollectioncore/update
Stack Trace:
org.apache.solr.client.solrj.impl.HttpSolrServer$RemoteSolrException: Server at http://127.0.0.1:55399/_eg/onenodecollectioncore returned non ok status:404, message:Can not find: /_eg/onenodecollectioncore/update
at __randomizedtesting.SeedInfo.seed([44B6599674515A8F:C550D78E030E3AB3]:0)
at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:385)
at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:180)
at org.apache.solr.client.solrj.request.AbstractUpdateRequest.process(AbstractUpdateRequest.java:117)
at org.apache.solr.client.solrj.SolrServer.add(SolrServer.java:116)
at org.apache.solr.client.solrj.SolrServer.add(SolrServer.java:102)
at org.apache.solr.cloud.BasicDistributedZk2Test.testNodeWithoutCollectionForwarding(BasicDistributedZk2Test.java:196)
at org.apache.solr.cloud.BasicDistributedZk2Test.doTest(BasicDistributedZk2Test.java:88)
at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:835)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:88)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:55)
at java.lang.reflect.Method.invoke(Method.java:613)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1559)
at com.carrotsearch.randomizedtesting.RandomizedRunner.access$600(RandomizedRunner.java:79)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:773)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:787)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:782)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:442)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:746)
at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:648)
at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:682)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:693)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
at java.lang.Thread.run(Thread.java:780)
Build Log:
[...truncated 9765 lines...]
[junit4] Suite: org.apache.solr.cloud.BasicDistributedZk2Test
[junit4] 2> 627388 T2661 oas.BaseDistributedSearchTestCase.initHostContext Setting hostContext system property: /_eg/
[junit4] 2> 627392 T2661 oas.SolrTestCaseJ4.setUp ###Starting testDistribSearch
[junit4] 2> Creating dataDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./solrtest-BasicDistributedZk2Test-1373835032114
[junit4] 2> 627393 T2661 oasc.ZkTestServer.run STARTING ZK TEST SERVER
[junit4] 2> 627393 T2662 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
[junit4] 2> 627493 T2661 oasc.ZkTestServer.run start zk server on port:54515
[junit4] 2> 627495 T2661 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 627572 T2668 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@fd7e3725 name:ZooKeeperConnection Watcher:127.0.0.1:54515 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 627572 T2661 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 627573 T2661 oascc.SolrZkClient.makePath makePath: /solr
[junit4] 2> 627577 T2661 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 627578 T2670 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@6b14a6 name:ZooKeeperConnection Watcher:127.0.0.1:54515/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 627578 T2661 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 627579 T2661 oascc.SolrZkClient.makePath makePath: /collections/collection1
[junit4] 2> 627581 T2661 oascc.SolrZkClient.makePath makePath: /collections/collection1/shards
[junit4] 2> 627583 T2661 oascc.SolrZkClient.makePath makePath: /collections/control_collection
[junit4] 2> 627584 T2661 oascc.SolrZkClient.makePath makePath: /collections/control_collection/shards
[junit4] 2> 627586 T2661 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
[junit4] 2> 627587 T2661 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.xml
[junit4] 2> 627590 T2661 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
[junit4] 2> 627591 T2661 oascc.SolrZkClient.makePath makePath: /configs/conf1/schema.xml
[junit4] 2> 627595 T2661 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
[junit4] 2> 627596 T2661 oascc.SolrZkClient.makePath makePath: /configs/conf1/stopwords.txt
[junit4] 2> 627598 T2661 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
[junit4] 2> 627598 T2661 oascc.SolrZkClient.makePath makePath: /configs/conf1/protwords.txt
[junit4] 2> 627601 T2661 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
[junit4] 2> 627601 T2661 oascc.SolrZkClient.makePath makePath: /configs/conf1/currency.xml
[junit4] 2> 627603 T2661 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
[junit4] 2> 627604 T2661 oascc.SolrZkClient.makePath makePath: /configs/conf1/open-exchange-rates.json
[junit4] 2> 627606 T2661 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4] 2> 627606 T2661 oascc.SolrZkClient.makePath makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4] 2> 627608 T2661 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
[junit4] 2> 627609 T2661 oascc.SolrZkClient.makePath makePath: /configs/conf1/old_synonyms.txt
[junit4] 2> 627611 T2661 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
[junit4] 2> 627611 T2661 oascc.SolrZkClient.makePath makePath: /configs/conf1/synonyms.txt
[junit4] 2> 627682 T2661 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4] 2> 627685 T2661 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:53363
[junit4] 2> 627686 T2661 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4] 2> 627687 T2661 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4] 2> 627687 T2661 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-controljetty-1373835032335
[junit4] 2> 627688 T2661 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-controljetty-1373835032335/'
[junit4] 2> 627696 T2661 oasc.ConfigSolr.fromFile Loading container configuration from /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-controljetty-1373835032335/solr.xml
[junit4] 2> 627737 T2661 oasc.CoreContainer.<init> New CoreContainer 1626852406
[junit4] 2> 627738 T2661 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-controljetty-1373835032335/]
[junit4] 2> 627740 T2661 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4] 2> 627741 T2661 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4] 2> 627742 T2661 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4] 2> 627743 T2661 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4] 2> 627744 T2661 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4] 2> 627745 T2661 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4] 2> 627746 T2661 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4] 2> 627747 T2661 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4] 2> 627748 T2661 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4] 2> 627749 T2661 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4] 2> 627756 T2661 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
[junit4] 2> 627757 T2661 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:54515/solr
[junit4] 2> 627757 T2661 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4] 2> 627758 T2661 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 627760 T2681 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@a6743c36 name:ZooKeeperConnection Watcher:127.0.0.1:54515 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 627760 T2661 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 627762 T2661 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4] 2> 627768 T2661 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 627769 T2683 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@fa4882df name:ZooKeeperConnection Watcher:127.0.0.1:54515/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 627769 T2661 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 627771 T2661 oascc.SolrZkClient.makePath makePath: /overseer/queue
[junit4] 2> 627774 T2661 oascc.SolrZkClient.makePath makePath: /overseer/collection-queue-work
[junit4] 2> 627777 T2661 oascc.SolrZkClient.makePath makePath: /live_nodes
[junit4] 2> 627779 T2661 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:53363__eg
[junit4] 2> 627780 T2661 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:53363__eg
[junit4] 2> 627784 T2661 oascc.SolrZkClient.makePath makePath: /overseer_elect/election
[junit4] 2> 627788 T2661 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
[junit4] 2> 627790 T2661 oasc.Overseer.start Overseer (id=90035652664754179-127.0.0.1:53363__eg-n_0000000000) starting
[junit4] 2> 627797 T2661 oascc.SolrZkClient.makePath makePath: /overseer/queue-work
[junit4] 2> 627800 T2685 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
[junit4] 2> 627801 T2661 oascc.SolrZkClient.makePath makePath: /clusterstate.json
[junit4] 2> 627803 T2661 oascc.SolrZkClient.makePath makePath: /aliases.json
[junit4] 2> 627805 T2661 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 627807 T2684 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
[junit4] 2> 627810 T2686 oasc.ZkController.publish publishing core=collection1 state=down
[junit4] 2> 627810 T2686 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 627811 T2686 oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4] 2> 629309 T2684 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 629310 T2684 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4] 2> "shard_state":"active",
[junit4] 2> "base_url":"http://127.0.0.1:53363/_eg",
[junit4] 2> "state":"down",
[junit4] 2> "operation":"state",
[junit4] 2> "core_node_name":"core_node1",
[junit4] 2> "node_name":"127.0.0.1:53363__eg",
[junit4] 2> "numShards":"1",
[junit4] 2> "shard_range":null,
[junit4] 2> "roles":null,
[junit4] 2> "collection":"control_collection",
[junit4] 2> "shard":null,
[junit4] 2> "core":"collection1"}
[junit4] 2> 629311 T2684 oasc.Overseer$ClusterStateUpdater.createCollection Create collection control_collection with numShards 1
[junit4] 2> 629311 T2684 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4] 2> 629314 T2683 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 629812 T2686 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
[junit4] 2> 629812 T2686 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-controljetty-1373835032335/collection1
[junit4] 2> 629813 T2686 oasc.ZkController.createCollectionZkNode Check for collection zkNode:control_collection
[junit4] 2> 629814 T2686 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4] 2> 629814 T2686 oasc.ZkController.readConfigName Load collection config from:/collections/control_collection
[junit4] 2> 629815 T2686 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-controljetty-1373835032335/collection1/'
[junit4] 2> 629816 T2686 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.BasicDistributedZk2Test-controljetty-1373835032335/collection1/lib/README' to classloader
[junit4] 2> 629816 T2686 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.BasicDistributedZk2Test-controljetty-1373835032335/collection1/lib/classes/' to classloader
[junit4] 2> 629844 T2686 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_45
[junit4] 2> 629877 T2686 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4] 2> 629878 T2686 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4] 2> 629885 T2686 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4] 2> 630204 T2686 oass.ByteField.init WARN ByteField is deprecated and will be removed in 5.0. You should use TrieIntField instead.
[junit4] 2> 630205 T2686 oass.ShortField.init WARN ShortField is deprecated and will be removed in 5.0. You should use TrieIntField instead.
[junit4] 2> 630206 T2686 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4] 2> 630219 T2686 oass.IndexSchema.readSchema default search field in schema is text
[junit4] 2> 630222 T2686 oass.IndexSchema.readSchema unique key field: id
[junit4] 2> 630233 T2686 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 630238 T2686 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 630242 T2686 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4] 2> 630243 T2686 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4] 2> 630243 T2686 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4] 2> 630244 T2686 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4] 2> 630245 T2686 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4] 2> 630246 T2686 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4] 2> 630246 T2686 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4] 2> 630247 T2686 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-controljetty-1373835032335/collection1/, dataDir=./org.apache.solr.cloud.BasicDistributedZk2Test-1373835032113/control/data/
[junit4] 2> 630247 T2686 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@d887bb70
[junit4] 2> 630249 T2686 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.BasicDistributedZk2Test-1373835032113/control/data
[junit4] 2> 630250 T2686 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.BasicDistributedZk2Test-1373835032113/control/data/index/
[junit4] 2> 630250 T2686 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.BasicDistributedZk2Test-1373835032113/control/data/index' doesn't exist. Creating new index...
[junit4] 2> 630251 T2686 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.BasicDistributedZk2Test-1373835032113/control/data/index
[junit4] 2> 630253 T2686 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@b9f8cb82 lockFactory=org.apache.lucene.store.NativeFSLockFactory@af28aede),segFN=segments_1,generation=1}
[junit4] 2> 630254 T2686 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 630256 T2686 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4] 2> 630257 T2686 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4] 2> 630258 T2686 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 630258 T2686 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4] 2> 630259 T2686 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4] 2> 630259 T2686 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4] 2> 630260 T2686 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4] 2> 630260 T2686 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4] 2> 630261 T2686 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4] 2> 630262 T2686 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 630264 T2686 oass.SolrIndexSearcher.<init> Opening Searcher@c3d6101a main
[junit4] 2> 630265 T2686 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4] 2> 630266 T2686 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4] 2> 630269 T2687 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@c3d6101a main{StandardDirectoryReader(segments_1:1)}
[junit4] 2> 630269 T2686 oasc.CoreContainer.registerCore registering core: collection1
[junit4] 2> 630270 T2686 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:53363/_eg collection:control_collection shard:shard1
[junit4] 2> 630271 T2686 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=10000&maxConnectionsPerHost=20&connTimeout=30000&socketTimeout=30000&retry=false
[junit4] 2> 630277 T2686 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leader_elect/shard1/election
[junit4] 2> 630281 T2686 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard1
[junit4] 2> 630283 T2686 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4] 2> 630283 T2686 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4] 2> 630284 T2686 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:53363/_eg/collection1/
[junit4] 2> 630284 T2686 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4] 2> 630284 T2686 oasc.SyncStrategy.syncToMe http://127.0.0.1:53363/_eg/collection1/ has no replicas
[junit4] 2> 630285 T2686 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:53363/_eg/collection1/ shard1
[junit4] 2> 630285 T2686 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leaders/shard1
[junit4] 2> 630817 T2684 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 630825 T2683 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 630841 T2686 oasc.ZkController.register We are http://127.0.0.1:53363/_eg/collection1/ and leader is http://127.0.0.1:53363/_eg/collection1/
[junit4] 2> 630841 T2686 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4] 2> 630841 T2686 oasc.ZkController.publish publishing core=collection1 state=active
[junit4] 2> 630841 T2686 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 630843 T2686 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 630844 T2661 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1
[junit4] 2> 630845 T2661 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4] 2> 630846 T2661 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4] 2> 630850 T2661 oascsi.HttpClientUtil.createClient Creating new http client, config:
[junit4] 2> 630851 T2661 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 630852 T2690 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@29772ebc name:ZooKeeperConnection Watcher:127.0.0.1:54515/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 630853 T2661 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 630854 T2661 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 630857 T2661 oasc.ChaosMonkey.monkeyLog monkey: init - expire sessions:true cause connection loss:true
[junit4] 2> 630921 T2661 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4] 2> 630923 T2661 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:50555
[junit4] 2> 630924 T2661 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4] 2> 630924 T2661 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4] 2> 630925 T2661 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty1-1373835035578
[junit4] 2> 630926 T2661 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty1-1373835035578/'
[junit4] 2> 630933 T2661 oasc.ConfigSolr.fromFile Loading container configuration from /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty1-1373835035578/solr.xml
[junit4] 2> 630964 T2661 oasc.CoreContainer.<init> New CoreContainer 570463034
[junit4] 2> 630965 T2661 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty1-1373835035578/]
[junit4] 2> 630966 T2661 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4] 2> 630966 T2661 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4] 2> 630967 T2661 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4] 2> 630968 T2661 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4] 2> 630968 T2661 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4] 2> 630969 T2661 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4] 2> 630969 T2661 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4] 2> 630970 T2661 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4] 2> 630970 T2661 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4] 2> 630971 T2661 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4] 2> 630975 T2661 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
[junit4] 2> 630976 T2661 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:54515/solr
[junit4] 2> 630977 T2661 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4] 2> 630978 T2661 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 630979 T2701 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@3ba38d99 name:ZooKeeperConnection Watcher:127.0.0.1:54515 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 630979 T2661 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 630981 T2661 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4] 2> 630986 T2661 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 630987 T2703 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@7ddc1b99 name:ZooKeeperConnection Watcher:127.0.0.1:54515/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 630987 T2661 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 630990 T2661 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 631993 T2661 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:50555__eg
[junit4] 2> 631995 T2661 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:50555__eg
[junit4] 2> 631998 T2690 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4] 2> 631998 T2683 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4] 2> 631998 T2703 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4] 2> 631998 T2683 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
[junit4] 2> 632005 T2704 oasc.ZkController.publish publishing core=collection1 state=down
[junit4] 2> 632006 T2704 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 632007 T2704 oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4] 2> 632330 T2684 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 632331 T2684 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4] 2> "shard_state":"active",
[junit4] 2> "base_url":"http://127.0.0.1:53363/_eg",
[junit4] 2> "state":"active",
[junit4] 2> "operation":"state",
[junit4] 2> "core_node_name":"core_node1",
[junit4] 2> "node_name":"127.0.0.1:53363__eg",
[junit4] 2> "numShards":"1",
[junit4] 2> "shard_range":null,
[junit4] 2> "roles":null,
[junit4] 2> "collection":"control_collection",
[junit4] 2> "shard":"shard1",
[junit4] 2> "core":"collection1"}
[junit4] 2> 632334 T2684 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4] 2> "shard_state":"active",
[junit4] 2> "base_url":"http://127.0.0.1:50555/_eg",
[junit4] 2> "state":"down",
[junit4] 2> "operation":"state",
[junit4] 2> "core_node_name":"core_node1",
[junit4] 2> "node_name":"127.0.0.1:50555__eg",
[junit4] 2> "numShards":"2",
[junit4] 2> "shard_range":null,
[junit4] 2> "roles":null,
[junit4] 2> "collection":"collection1",
[junit4] 2> "shard":null,
[junit4] 2> "core":"collection1"}
[junit4] 2> 632334 T2684 oasc.Overseer$ClusterStateUpdater.createCollection Create collection collection1 with numShards 2
[junit4] 2> 632334 T2684 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4] 2> 632337 T2690 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 632337 T2683 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 632337 T2703 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 633007 T2704 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
[junit4] 2> 633008 T2704 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty1-1373835035578/collection1
[junit4] 2> 633008 T2704 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4] 2> 633009 T2704 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4] 2> 633009 T2704 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4] 2> 633010 T2704 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty1-1373835035578/collection1/'
[junit4] 2> 633011 T2704 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.BasicDistributedZk2Test-jetty1-1373835035578/collection1/lib/README' to classloader
[junit4] 2> 633012 T2704 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.BasicDistributedZk2Test-jetty1-1373835035578/collection1/lib/classes/' to classloader
[junit4] 2> 633048 T2704 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_45
[junit4] 2> 633095 T2704 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4] 2> 633096 T2704 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4] 2> 633103 T2704 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4] 2> 633447 T2704 oass.ByteField.init WARN ByteField is deprecated and will be removed in 5.0. You should use TrieIntField instead.
[junit4] 2> 633448 T2704 oass.ShortField.init WARN ShortField is deprecated and will be removed in 5.0. You should use TrieIntField instead.
[junit4] 2> 633449 T2704 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4] 2> 633454 T2704 oass.IndexSchema.readSchema default search field in schema is text
[junit4] 2> 633457 T2704 oass.IndexSchema.readSchema unique key field: id
[junit4] 2> 633467 T2704 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 633471 T2704 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 633475 T2704 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4] 2> 633476 T2704 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4] 2> 633477 T2704 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4] 2> 633477 T2704 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4] 2> 633478 T2704 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4] 2> 633479 T2704 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4] 2> 633479 T2704 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4] 2> 633479 T2704 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty1-1373835035578/collection1/, dataDir=./org.apache.solr.cloud.BasicDistributedZk2Test-1373835032113/jetty1/
[junit4] 2> 633480 T2704 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@d887bb70
[junit4] 2> 633481 T2704 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.BasicDistributedZk2Test-1373835032113/jetty1
[junit4] 2> 633482 T2704 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.BasicDistributedZk2Test-1373835032113/jetty1/index/
[junit4] 2> 633482 T2704 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.BasicDistributedZk2Test-1373835032113/jetty1/index' doesn't exist. Creating new index...
[junit4] 2> 633483 T2704 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.BasicDistributedZk2Test-1373835032113/jetty1/index
[junit4] 2> 633484 T2704 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@c934cb89 lockFactory=org.apache.lucene.store.NativeFSLockFactory@d9f31d91),segFN=segments_1,generation=1}
[junit4] 2> 633484 T2704 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 633487 T2704 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4] 2> 633487 T2704 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4] 2> 633488 T2704 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 633488 T2704 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4] 2> 633489 T2704 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4] 2> 633489 T2704 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4] 2> 633489 T2704 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4] 2> 633490 T2704 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4] 2> 633490 T2704 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4] 2> 633492 T2704 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 633494 T2704 oass.SolrIndexSearcher.<init> Opening Searcher@83550cc8 main
[junit4] 2> 633495 T2704 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4] 2> 633495 T2704 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4] 2> 633499 T2705 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@83550cc8 main{StandardDirectoryReader(segments_1:1)}
[junit4] 2> 633500 T2704 oasc.CoreContainer.registerCore registering core: collection1
[junit4] 2> 633501 T2704 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:50555/_eg collection:collection1 shard:shard1
[junit4] 2> 633501 T2704 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=10000&maxConnectionsPerHost=20&connTimeout=30000&socketTimeout=30000&retry=false
[junit4] 2> 633507 T2704 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
[junit4] 2> 633513 T2704 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard1
[junit4] 2> 633515 T2704 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4] 2> 633515 T2704 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4] 2> 633516 T2704 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:50555/_eg/collection1/
[junit4] 2> 633516 T2704 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4] 2> 633517 T2704 oasc.SyncStrategy.syncToMe http://127.0.0.1:50555/_eg/collection1/ has no replicas
[junit4] 2> 633517 T2704 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:50555/_eg/collection1/ shard1
[junit4] 2> 633517 T2704 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
[junit4] 2> 633841 T2684 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 633847 T2690 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 633847 T2683 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 633847 T2703 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 633875 T2704 oasc.ZkController.register We are http://127.0.0.1:50555/_eg/collection1/ and leader is http://127.0.0.1:50555/_eg/collection1/
[junit4] 2> 633876 T2704 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4] 2> 633876 T2704 oasc.ZkController.publish publishing core=collection1 state=active
[junit4] 2> 633877 T2704 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 633878 T2704 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 633879 T2661 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1
[junit4] 2> 633880 T2661 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4] 2> 633880 T2661 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4] 2> 633962 T2661 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4] 2> 633964 T2661 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:55399
[junit4] 2> 633965 T2661 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4] 2> 633966 T2661 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4] 2> 633966 T2661 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty2-1373835038606
[junit4] 2> 633967 T2661 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty2-1373835038606/'
[junit4] 2> 633975 T2661 oasc.ConfigSolr.fromFile Loading container configuration from /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty2-1373835038606/solr.xml
[junit4] 2> 634011 T2661 oasc.CoreContainer.<init> New CoreContainer 348014057
[junit4] 2> 634012 T2661 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty2-1373835038606/]
[junit4] 2> 634014 T2661 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4] 2> 634014 T2661 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4] 2> 634015 T2661 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4] 2> 634016 T2661 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4] 2> 634017 T2661 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4] 2> 634017 T2661 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4] 2> 634018 T2661 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4] 2> 634019 T2661 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4] 2> 634020 T2661 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4] 2> 634021 T2661 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4] 2> 634026 T2661 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
[junit4] 2> 634027 T2661 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:54515/solr
[junit4] 2> 634028 T2661 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4] 2> 634029 T2661 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 634031 T2717 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@423f184c name:ZooKeeperConnection Watcher:127.0.0.1:54515 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 634031 T2661 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 634032 T2661 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4] 2> 634037 T2661 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 634039 T2719 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@e7eebc6d name:ZooKeeperConnection Watcher:127.0.0.1:54515/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 634039 T2661 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 634042 T2661 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 635044 T2661 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:55399__eg
[junit4] 2> 635046 T2661 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:55399__eg
[junit4] 2> 635048 T2690 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4] 2> 635049 T2683 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4] 2> 635049 T2690 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4] 2> 635049 T2719 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4] 2> 635049 T2683 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4] 2> 635049 T2703 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4] 2> 635051 T2703 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4] 2> 635056 T2720 oasc.ZkController.publish publishing core=collection1 state=down
[junit4] 2> 635057 T2720 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 635058 T2720 oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4] 2> 635353 T2684 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 635354 T2684 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4] 2> "shard_state":"active",
[junit4] 2> "base_url":"http://127.0.0.1:50555/_eg",
[junit4] 2> "state":"active",
[junit4] 2> "operation":"state",
[junit4] 2> "core_node_name":"core_node1",
[junit4] 2> "node_name":"127.0.0.1:50555__eg",
[junit4] 2> "numShards":"2",
[junit4] 2> "shard_range":null,
[junit4] 2> "roles":null,
[junit4] 2> "collection":"collection1",
[junit4] 2> "shard":"shard1",
[junit4] 2> "core":"collection1"}
[junit4] 2> 635357 T2684 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4] 2> "shard_state":"active",
[junit4] 2> "base_url":"http://127.0.0.1:55399/_eg",
[junit4] 2> "state":"down",
[junit4] 2> "operation":"state",
[junit4] 2> "core_node_name":"core_node2",
[junit4] 2> "node_name":"127.0.0.1:55399__eg",
[junit4] 2> "numShards":"2",
[junit4] 2> "shard_range":null,
[junit4] 2> "roles":null,
[junit4] 2> "collection":"collection1",
[junit4] 2> "shard":null,
[junit4] 2> "core":"collection1"}
[junit4] 2> 635357 T2684 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=2
[junit4] 2> 635358 T2684 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard2
[junit4] 2> 635360 T2690 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 635361 T2683 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 635361 T2703 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 635361 T2719 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 636058 T2720 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
[junit4] 2> 636059 T2720 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty2-1373835038606/collection1
[junit4] 2> 636059 T2720 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4] 2> 636060 T2720 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4] 2> 636060 T2720 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4] 2> 636061 T2720 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty2-1373835038606/collection1/'
[junit4] 2> 636062 T2720 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.BasicDistributedZk2Test-jetty2-1373835038606/collection1/lib/README' to classloader
[junit4] 2> 636062 T2720 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.BasicDistributedZk2Test-jetty2-1373835038606/collection1/lib/classes/' to classloader
[junit4] 2> 636089 T2720 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_45
[junit4] 2> 636122 T2720 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4] 2> 636123 T2720 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4] 2> 636129 T2720 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4] 2> 636451 T2720 oass.ByteField.init WARN ByteField is deprecated and will be removed in 5.0. You should use TrieIntField instead.
[junit4] 2> 636452 T2720 oass.ShortField.init WARN ShortField is deprecated and will be removed in 5.0. You should use TrieIntField instead.
[junit4] 2> 636453 T2720 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4] 2> 636459 T2720 oass.IndexSchema.readSchema default search field in schema is text
[junit4] 2> 636462 T2720 oass.IndexSchema.readSchema unique key field: id
[junit4] 2> 636473 T2720 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 636477 T2720 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 636481 T2720 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4] 2> 636482 T2720 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4] 2> 636482 T2720 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4] 2> 636483 T2720 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4] 2> 636484 T2720 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4] 2> 636484 T2720 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4] 2> 636485 T2720 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4] 2> 636485 T2720 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty2-1373835038606/collection1/, dataDir=./org.apache.solr.cloud.BasicDistributedZk2Test-1373835032113/jetty2/
[junit4] 2> 636485 T2720 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@d887bb70
[junit4] 2> 636487 T2720 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.BasicDistributedZk2Test-1373835032113/jetty2
[junit4] 2> 636487 T2720 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.BasicDistributedZk2Test-1373835032113/jetty2/index/
[junit4] 2> 636487 T2720 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.BasicDistributedZk2Test-1373835032113/jetty2/index' doesn't exist. Creating new index...
[junit4] 2> 636488 T2720 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.BasicDistributedZk2Test-1373835032113/jetty2/index
[junit4] 2> 636489 T2720 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@9dbe14dc lockFactory=org.apache.lucene.store.NativeFSLockFactory@89ff38bf),segFN=segments_1,generation=1}
[junit4] 2> 636489 T2720 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 636492 T2720 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4] 2> 636492 T2720 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4] 2> 636493 T2720 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 636493 T2720 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4] 2> 636494 T2720 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4] 2> 636494 T2720 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4] 2> 636495 T2720 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4] 2> 636495 T2720 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4] 2> 636496 T2720 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4] 2> 636497 T2720 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 636499 T2720 oass.SolrIndexSearcher.<init> Opening Searcher@d32c70a main
[junit4] 2> 636500 T2720 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4] 2> 636500 T2720 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4] 2> 636503 T2721 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@d32c70a main{StandardDirectoryReader(segments_1:1)}
[junit4] 2> 636505 T2720 oasc.CoreContainer.registerCore registering core: collection1
[junit4] 2> 636505 T2720 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:55399/_eg collection:collection1 shard:shard2
[junit4] 2> 636505 T2720 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=10000&maxConnectionsPerHost=20&connTimeout=30000&socketTimeout=30000&retry=false
[junit4] 2> 636509 T2720 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard2/election
[junit4] 2> 636515 T2720 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard2
[junit4] 2> 636517 T2720 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4] 2> 636517 T2720 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4] 2> 636517 T2720 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:55399/_eg/collection1/
[junit4] 2> 636518 T2720 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4] 2> 636518 T2720 oasc.SyncStrategy.syncToMe http://127.0.0.1:55399/_eg/collection1/ has no replicas
[junit4] 2> 636518 T2720 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:55399/_eg/collection1/ shard2
[junit4] 2> 636519 T2720 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard2
[junit4] 2> 636866 T2684 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 636871 T2690 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 636872 T2703 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 636871 T2719 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 636872 T2683 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 636877 T2720 oasc.ZkController.register We are http://127.0.0.1:55399/_eg/collection1/ and leader is http://127.0.0.1:55399/_eg/collection1/
[junit4] 2> 636877 T2720 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4] 2> 636877 T2720 oasc.ZkController.publish publishing core=collection1 state=active
[junit4] 2> 636877 T2720 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 636879 T2720 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 636880 T2661 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1
[junit4] 2> 636881 T2661 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4] 2> 636881 T2661 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4] 2> 636957 T2661 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4] 2> 636960 T2661 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:57404
[junit4] 2> 636961 T2661 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4] 2> 636962 T2661 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4] 2> 636963 T2661 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty3-1373835041608
[junit4] 2> 636964 T2661 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty3-1373835041608/'
[junit4] 2> 636978 T2661 oasc.ConfigSolr.fromFile Loading container configuration from /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty3-1373835041608/solr.xml
[junit4] 2> 637023 T2661 oasc.CoreContainer.<init> New CoreContainer -459037688
[junit4] 2> 637024 T2661 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty3-1373835041608/]
[junit4] 2> 637026 T2661 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4] 2> 637026 T2661 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4] 2> 637027 T2661 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4] 2> 637028 T2661 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4] 2> 637028 T2661 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4] 2> 637029 T2661 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4] 2> 637029 T2661 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4] 2> 637030 T2661 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4] 2> 637031 T2661 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4] 2> 637031 T2661 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4] 2> 637036 T2661 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
[junit4] 2> 637037 T2661 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:54515/solr
[junit4] 2> 637038 T2661 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4] 2> 637039 T2661 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 637040 T2733 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@754a9872 name:ZooKeeperConnection Watcher:127.0.0.1:54515 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 637041 T2661 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 637042 T2661 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4] 2> 637048 T2661 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 637049 T2735 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@8c484e4c name:ZooKeeperConnection Watcher:127.0.0.1:54515/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 637049 T2661 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 637052 T2661 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 638055 T2661 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:57404__eg
[junit4] 2> 638057 T2661 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:57404__eg
[junit4] 2> 638059 T2719 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4] 2> 638059 T2703 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4] 2> 638060 T2703 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4] 2> 638059 T2683 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4] 2> 638059 T2690 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4] 2> 638061 T2683 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4] 2> 638060 T2735 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4] 2> 638061 T2690 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4] 2> 638061 T2719 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4] 2> 638067 T2736 oasc.ZkController.publish publishing core=collection1 state=down
[junit4] 2> 638068 T2736 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 638070 T2736 oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4] 2> 638376 T2684 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 638377 T2684 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4] 2> "shard_state":"active",
[junit4] 2> "base_url":"http://127.0.0.1:55399/_eg",
[junit4] 2> "state":"active",
[junit4] 2> "operation":"state",
[junit4] 2> "core_node_name":"core_node2",
[junit4] 2> "node_name":"127.0.0.1:55399__eg",
[junit4] 2> "numShards":"2",
[junit4] 2> "shard_range":null,
[junit4] 2> "roles":null,
[junit4] 2> "collection":"collection1",
[junit4] 2> "shard":"shard2",
[junit4] 2> "core":"collection1"}
[junit4] 2> 638381 T2684 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4] 2> "shard_state":"active",
[junit4] 2> "base_url":"http://127.0.0.1:57404/_eg",
[junit4] 2> "state":"down",
[junit4] 2> "operation":"state",
[junit4] 2> "core_node_name":"core_node3",
[junit4] 2> "node_name":"127.0.0.1:57404__eg",
[junit4] 2> "numShards":"2",
[junit4] 2> "shard_range":null,
[junit4] 2> "roles":null,
[junit4] 2> "collection":"collection1",
[junit4] 2> "shard":null,
[junit4] 2> "core":"collection1"}
[junit4] 2> 638381 T2684 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=2
[junit4] 2> 638382 T2684 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4] 2> 638385 T2690 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4] 2> 638385 T2735 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4] 2> 638385 T2703 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4] 2> 638385 T2719 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4] 2> 638385 T2683 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4] 2> 639070 T2736 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
[junit4] 2> 639071 T2736 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty3-1373835041608/collection1
[junit4] 2> 639071 T2736 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4] 2> 639072 T2736 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4] 2> 639072 T2736 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4] 2> 639073 T2736 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty3-1373835041608/collection1/'
[junit4] 2> 639074 T2736 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.BasicDistributedZk2Test-jetty3-1373835041608/collection1/lib/README' to classloader
[junit4] 2> 639075 T2736 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.BasicDistributedZk2Test-jetty3-1373835041608/collection1/lib/classes/' to classloader
[junit4] 2> 639101 T2736 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_45
[junit4] 2> 639133 T2736 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4] 2> 639134 T2736 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4] 2> 639140 T2736 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4] 2> 639458 T2736 oass.ByteField.init WARN ByteField is deprecated and will be removed in 5.0. You should use TrieIntField instead.
[junit4] 2> 639458 T2736 oass.ShortField.init WARN ShortField is deprecated and will be removed in 5.0. You should use TrieIntField instead.
[junit4] 2> 639459 T2736 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4] 2> 639464 T2736 oass.IndexSchema.readSchema default search field in schema is text
[junit4] 2> 639467 T2736 oass.IndexSchema.readSchema unique key field: id
[junit4] 2> 639476 T2736 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 639480 T2736 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 639483 T2736 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4] 2> 639485 T2736 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4] 2> 639485 T2736 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4] 2> 639485 T2736 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4] 2> 639486 T2736 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4] 2> 639487 T2736 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4] 2> 639487 T2736 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4] 2> 639487 T2736 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty3-1373835041608/collection1/, dataDir=./org.apache.solr.cloud.BasicDistributedZk2Test-1373835032113/jetty3/
[junit4] 2> 639488 T2736 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@d887bb70
[junit4] 2> 639489 T2736 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.BasicDistributedZk2Test-1373835032113/jetty3
[junit4] 2> 639490 T2736 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.BasicDistributedZk2Test-1373835032113/jetty3/index/
[junit4] 2> 639490 T2736 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.BasicDistributedZk2Test-1373835032113/jetty3/index' doesn't exist. Creating new index...
[junit4] 2> 639490 T2736 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.BasicDistributedZk2Test-1373835032113/jetty3/index
[junit4] 2> 639492 T2736 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@cb85bb37 lockFactory=org.apache.lucene.store.NativeFSLockFactory@bf5d0e04),segFN=segments_1,generation=1}
[junit4] 2> 639492 T2736 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 639494 T2736 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4] 2> 639495 T2736 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4] 2> 639495 T2736 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 639496 T2736 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4] 2> 639496 T2736 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4] 2> 639497 T2736 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4] 2> 639497 T2736 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4] 2> 639498 T2736 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4] 2> 639498 T2736 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4] 2> 639499 T2736 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 639501 T2736 oass.SolrIndexSearcher.<init> Opening Searcher@4c0c63d0 main
[junit4] 2> 639502 T2736 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4] 2> 639502 T2736 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4] 2> 639506 T2737 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@4c0c63d0 main{StandardDirectoryReader(segments_1:1)}
[junit4] 2> 639507 T2736 oasc.CoreContainer.registerCore registering core: collection1
[junit4] 2> 639508 T2736 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:57404/_eg collection:collection1 shard:shard1
[junit4] 2> 639508 T2736 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=10000&maxConnectionsPerHost=20&connTimeout=30000&socketTimeout=30000&retry=false
[junit4] 2> 639514 T2736 oasc.ZkController.register We are http://127.0.0.1:57404/_eg/collection1/ and leader is http://127.0.0.1:50555/_eg/collection1/
[junit4] 2> 639515 T2736 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:57404/_eg
[junit4] 2> 639515 T2736 oasc.ZkController.checkRecovery Core needs to recover:collection1
[junit4] 2> 639515 T2736 oasu.DefaultSolrCoreState.doRecovery Running recovery - first canceling any ongoing recovery
[junit4] 2> ASYNC NEW_CORE C745 name=collection1 org.apache.solr.core.SolrCore@c5d87327 url=http://127.0.0.1:57404/_eg/collection1 node=127.0.0.1:57404__eg C745_STATE=coll:collection1 core:collection1 props:{base_url=http://127.0.0.1:57404/_eg, state=down, node_name=127.0.0.1:57404__eg, core=collection1}
[junit4] 2> 639516 T2738 C745 P57404 oasc.RecoveryStrategy.run Starting recovery process. core=collection1 recoveringAfterStartup=true
[junit4] 2> 639516 T2736 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 639516 T2738 C745 P57404 oasc.RecoveryStrategy.doRecovery ###### startupVersions=[]
[junit4] 2> 639517 T2738 C745 P57404 oasc.ZkController.publish publishing core=collection1 state=recovering
[junit4] 2> 639517 T2738 C745 P57404 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 639517 T2661 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1
[junit4] 2> 639518 T2661 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4] 2> 639519 T2738 C745 P57404 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4] 2> 639519 T2661 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4] 2> 639524 T2697 oasha.CoreAdminHandler.handleWaitForStateAction Going to wait for coreNodeName: core_node3, state: recovering, checkLive: true, onlyIfLeader: true
[junit4] 2> 639623 T2661 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4] 2> 639625 T2661 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:33603
[junit4] 2> 639626 T2661 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4] 2> 639627 T2661 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4] 2> 639627 T2661 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty4-1373835044247
[junit4] 2> 639628 T2661 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty4-1373835044247/'
[junit4] 2> 639636 T2661 oasc.ConfigSolr.fromFile Loading container configuration from /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty4-1373835044247/solr.xml
[junit4] 2> 639671 T2661 oasc.CoreContainer.<init> New CoreContainer 1340836413
[junit4] 2> 639672 T2661 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty4-1373835044247/]
[junit4] 2> 639673 T2661 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4] 2> 639674 T2661 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4] 2> 639674 T2661 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4] 2> 639675 T2661 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4] 2> 639675 T2661 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4] 2> 639676 T2661 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4] 2> 639677 T2661 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4] 2> 639686 T2661 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4] 2> 639687 T2661 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4] 2> 639688 T2661 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4] 2> 639695 T2661 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
[junit4] 2> 639696 T2661 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:54515/solr
[junit4] 2> 639697 T2661 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4] 2> 639699 T2661 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 639700 T2750 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@cb223fc5 name:ZooKeeperConnection Watcher:127.0.0.1:54515 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 639701 T2661 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 639707 T2661 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4] 2> 639714 T2661 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 639716 T2752 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@d3ec097c name:ZooKeeperConnection Watcher:127.0.0.1:54515/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 639716 T2661 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 639720 T2661 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 639890 T2684 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 639891 T2684 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4] 2> "shard_state":"active",
[junit4] 2> "base_url":"http://127.0.0.1:57404/_eg",
[junit4] 2> "state":"recovering",
[junit4] 2> "operation":"state",
[junit4] 2> "core_node_name":"core_node3",
[junit4] 2> "node_name":"127.0.0.1:57404__eg",
[junit4] 2> "numShards":"2",
[junit4] 2> "shard_range":null,
[junit4] 2> "roles":null,
[junit4] 2> "collection":"collection1",
[junit4] 2> "shard":"shard1",
[junit4] 2> "core":"collection1"}
[junit4] 2> 639896 T2735 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4] 2> 639896 T2719 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4] 2> 639897 T2703 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4] 2> 639896 T2690 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4] 2> 639897 T2752 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4] 2> 639896 T2683 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4] 2> 640525 T2697 oasha.CoreAdminHandler.handleWaitForStateAction Waited coreNodeName: core_node3, state: recovering, checkLive: true, onlyIfLeader: true for: 1 seconds.
[junit4] 2> 640526 T2697 oass.SolrDispatchFilter.handleAdminRequest [admin] webapp=null path=/admin/cores params={state=recovering&onlyIfLeader=true&checkLive=true&action=PREPRECOVERY&version=2&coreNodeName=core_node3&wt=javabin&nodeName=127.0.0.1:57404__eg&core=collection1} status=0 QTime=1002
[junit4] 2> 640724 T2661 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:33603__eg
[junit4] 2> 640726 T2661 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:33603__eg
[junit4] 2> 640728 T2719 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4] 2> 640728 T2703 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4] 2> 640729 T2683 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4] 2> 640728 T2735 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4] 2> 640729 T2752 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4] 2> 640729 T2703 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4] 2> 640730 T2735 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4] 2> 640730 T2719 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4] 2> 640730 T2752 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4] 2> 640728 T2690 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4] 2> 640729 T2683 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4] 2> 640732 T2690 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4] 2> 640736 T2753 oasc.ZkController.publish publishing core=collection1 state=down
[junit4] 2> 640736 T2753 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 640737 T2753 oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4] 2> 641401 T2684 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 641402 T2684 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4] 2> "shard_state":"active",
[junit4] 2> "base_url":"http://127.0.0.1:33603/_eg",
[junit4] 2> "state":"down",
[junit4] 2> "operation":"state",
[junit4] 2> "core_node_name":"core_node4",
[junit4] 2> "node_name":"127.0.0.1:33603__eg",
[junit4] 2> "numShards":"2",
[junit4] 2> "shard_range":null,
[junit4] 2> "roles":null,
[junit4] 2> "collection":"collection1",
[junit4] 2> "shard":null,
[junit4] 2> "core":"collection1"}
[junit4] 2> 641403 T2684 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=2
[junit4] 2> 641403 T2684 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard2
[junit4] 2> 641406 T2719 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4] 2> 641406 T2752 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4] 2> 641406 T2690 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4] 2> 641406 T2703 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4] 2> 641406 T2735 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4] 2> 641406 T2683 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4] 2> 641737 T2753 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
[junit4] 2> 641738 T2753 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty4-1373835044247/collection1
[junit4] 2> 641738 T2753 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4] 2> 641739 T2753 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4] 2> 641739 T2753 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4] 2> 641741 T2753 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty4-1373835044247/collection1/'
[junit4] 2> 641741 T2753 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.BasicDistributedZk2Test-jetty4-1373835044247/collection1/lib/README' to classloader
[junit4] 2> 641742 T2753 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.BasicDistributedZk2Test-jetty4-1373835044247/collection1/lib/classes/' to classloader
[junit4] 2> 641768 T2753 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_45
[junit4] 2> 641799 T2753 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4] 2> 641800 T2753 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4] 2> 641808 T2753 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4] 2> 642130 T2753 oass.ByteField.init WARN ByteField is deprecated and will be removed in 5.0. You should use TrieIntField instead.
[junit4] 2> 642131 T2753 oass.ShortField.init WARN ShortField is deprecated and will be removed in 5.0. You should use TrieIntField instead.
[junit4] 2> 642131 T2753 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4] 2> 642137 T2753 oass.IndexSchema.readSchema default search field in schema is text
[junit4] 2> 642140 T2753 oass.IndexSchema.readSchema unique key field: id
[junit4] 2> 642150 T2753 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 642154 T2753 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 642158 T2753 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4] 2> 642159 T2753 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4] 2> 642160 T2753 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4] 2> 642160 T2753 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4] 2> 642161 T2753 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4] 2> 642161 T2753 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4] 2> 642162 T2753 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4] 2> 642162 T2753 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty4-1373835044247/collection1/, dataDir=./org.apache.solr.cloud.BasicDistributedZk2Test-1373835032113/jetty4/
[junit4] 2> 642163 T2753 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@d887bb70
[junit4] 2> 642164 T2753 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.BasicDistributedZk2Test-1373835032113/jetty4
[junit4] 2> 642165 T2753 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.BasicDistributedZk2Test-1373835032113/jetty4/index/
[junit4] 2> 642165 T2753 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.BasicDistributedZk2Test-1373835032113/jetty4/index' doesn't exist. Creating new index...
[junit4] 2> 642165 T2753 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.BasicDistributedZk2Test-1373835032113/jetty4/index
[junit4] 2> 642167 T2753 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@3c26ac85 lockFactory=org.apache.lucene.store.NativeFSLockFactory@498ce002),segFN=segments_1,generation=1}
[junit4] 2> 642167 T2753 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 642170 T2753 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4] 2> 642170 T2753 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4] 2> 642171 T2753 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 642171 T2753 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4] 2> 642172 T2753 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4] 2> 642172 T2753 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4] 2> 642172 T2753 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4] 2> 642173 T2753 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4] 2> 642173 T2753 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4] 2> 642175 T2753 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 642177 T2753 oass.SolrIndexSearcher.<init> Opening Searcher@d6d17a81 main
[junit4] 2> 642177 T2753 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4] 2> 642178 T2753 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4] 2> 642181 T2754 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@d6d17a81 main{StandardDirectoryReader(segments_1:1)}
[junit4] 2> 642183 T2753 oasc.CoreContainer.registerCore registering core: collection1
[junit4] 2> 642183 T2753 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:33603/_eg collection:collection1 shard:shard2
[junit4] 2> 642183 T2753 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=10000&maxConnectionsPerHost=20&connTimeout=30000&socketTimeout=30000&retry=false
[junit4] 2> 642189 T2753 oasc.ZkController.register We are http://127.0.0.1:33603/_eg/collection1/ and leader is http://127.0.0.1:55399/_eg/collection1/
[junit4] 2> 642189 T2753 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:33603/_eg
[junit4] 2> 642190 T2753 oasc.ZkController.checkRecovery Core needs to recover:collection1
[junit4] 2> 642190 T2753 oasu.DefaultSolrCoreState.doRecovery Running recovery - first canceling any ongoing recovery
[junit4] 2> ASYNC NEW_CORE C746 name=collection1 org.apache.solr.core.SolrCore@831b5cd7 url=http://127.0.0.1:33603/_eg/collection1 node=127.0.0.1:33603__eg C746_STATE=coll:collection1 core:collection1 props:{base_url=http://127.0.0.1:33603/_eg, state=down, node_name=127.0.0.1:33603__eg, core=collection1}
[junit4] 2> 642190 T2755 C746 P33603 oasc.RecoveryStrategy.run Starting recovery process. core=collection1 recoveringAfterStartup=true
[junit4] 2> 642191 T2753 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 642191 T2755 C746 P33603 oasc.RecoveryStrategy.doRecovery ###### startupVersions=[]
[junit4] 2> 642192 T2755 C746 P33603 oasc.ZkController.publish publishing core=collection1 state=recovering
[junit4] 2> 642192 T2755 C746 P33603 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 642192 T2661 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1
[junit4] 2> 642193 T2661 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4] 2> 642193 T2755 C746 P33603 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4] 2> 642194 T2661 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4] 2> 642199 T2713 oasha.CoreAdminHandler.handleWaitForStateAction Going to wait for coreNodeName: core_node4, state: recovering, checkLive: true, onlyIfLeader: true
[junit4] 2> 642201 T2661 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 642203 T2661 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4] 2> 642211 T2698 oasc.ZkController.publish publishing core=onenodecollectioncore state=down
[junit4] 2> 642213 T2698 oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4] 2> ASYNC NEW_CORE C747 name=collection1 org.apache.solr.core.SolrCore@c5d87327 url=http://127.0.0.1:57404/_eg/collection1 node=127.0.0.1:57404__eg C747_STATE=coll:collection1 core:collection1 props:{base_url=http://127.0.0.1:57404/_eg, state=recovering, node_name=127.0.0.1:57404__eg, core=collection1}
[junit4] 2> 642527 T2738 C747 P57404 oasc.RecoveryStrategy.doRecovery Attempting to PeerSync from http://127.0.0.1:50555/_eg/collection1/ core=collection1 - recoveringAfterStartup=true
[junit4] 2> 642527 T2738 C747 P57404 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:57404/_eg START replicas=[http://127.0.0.1:50555/_eg/collection1/] nUpdates=100
[junit4] 2> 642528 T2738 C747 P57404 oasu.PeerSync.sync WARN no frame of reference to tell of we've missed updates
[junit4] 2> 642528 T2738 C747 P57404 oasc.RecoveryStrategy.doRecovery PeerSync Recovery was not successful - trying replication. core=collection1
[junit4] 2> 642529 T2738 C747 P57404 oasc.RecoveryStrategy.doRecovery Starting Replication Recovery. core=collection1
[junit4] 2> 642529 T2738 C747 P57404 oasc.RecoveryStrategy.doRecovery Begin buffering updates. core=collection1
[junit4] 2> 642529 T2738 C747 P57404 oasu.UpdateLog.bufferUpdates Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null}
[junit4] 2> 642529 T2738 C747 P57404 oasc.RecoveryStrategy.replicate Attempting to replicate from http://127.0.0.1:50555/_eg/collection1/. core=collection1
[junit4] 2> 642529 T2738 C747 P57404 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4] 2> ASYNC NEW_CORE C748 name=collection1 org.apache.solr.core.SolrCore@eb7dc2cf url=http://127.0.0.1:50555/_eg/collection1 node=127.0.0.1:50555__eg C748_STATE=coll:collection1 core:collection1 props:{base_url=http://127.0.0.1:50555/_eg, state=active, node_name=127.0.0.1:50555__eg, core=collection1, leader=true}
[junit4] 2> 642532 T2699 C748 P50555 oasc.SolrCore.execute [collection1] webapp=/_eg path=/get params={getVersions=100&qt=/get&distrib=false&version=2&wt=javabin} status=0 QTime=0
[junit4] 2> 642535 T2697 C748 P50555 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4] 2> 642536 T2697 C748 P50555 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@c934cb89 lockFactory=org.apache.lucene.store.NativeFSLockFactory@d9f31d91),segFN=segments_1,generation=1}
[junit4] 2> 642537 T2697 C748 P50555 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 642538 T2697 C748 P50555 oasu.DirectUpdateHandler2.commit No uncommitted changes. Skipping IW.commit.
[junit4] 2> 642538 T2697 C748 P50555 oass.SolrIndexSearcher.<init> Opening Searcher@609e4eec realtime
[junit4] 2> 642539 T2697 C748 P50555 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4] 2> 642540 T2697 C748 P50555 oasup.LogUpdateProcessor.finish [collection1] webapp=/_eg path=/update params={softCommit=false&commit_end_point=true&commit=true&version=2&waitSearcher=true&wt=javabin&openSearcher=false} {commit=} 0 6
[junit4] 2> 642541 T2738 C747 P57404 oash.ReplicationHandler.inform Commits will be reserved for 10000
[junit4] 2> 642541 T2738 C747 P57404 oash.SnapPuller.<init> No value set for 'pollInterval'. Timer Task not started.
[junit4] 2> 642543 T2696 C748 P50555 oash.ReplicationHandler.inform Commits will be reserved for 10000
[junit4] 2> 642544 T2696 C748 P50555 oasc.SolrCore.execute [collection1] webapp=/_eg path=/replication params={qt=/replication&version=2&command=indexversion&wt=javabin} status=0 QTime=2
[junit4] 2> 642545 T2738 C747 P57404 oasc.RecoveryStrategy.replay No replay needed. core=collection1
[junit4] 2> 642545 T2738 C747 P57404 oasc.RecoveryStrategy.doRecovery Replication Recovery was successful - registering as Active. core=collection1
[junit4] 2> 642545 T2738 C747 P57404 oasc.ZkController.publish publishing core=collection1 state=active
[junit4] 2> 642546 T2738 C747 P57404 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 642547 T2738 C747 P57404 oasc.RecoveryStrategy.doRecovery Finished recovery process. core=collection1
[junit4] 2> 642910 T2684 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 642912 T2684 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4] 2> "shard_state":"active",
[junit4] 2> "base_url":"http://127.0.0.1:33603/_eg",
[junit4] 2> "state":"recovering",
[junit4] 2> "operation":"state",
[junit4] 2> "core_node_name":"core_node4",
[junit4] 2> "node_name":"127.0.0.1:33603__eg",
[junit4] 2> "numShards":"2",
[junit4] 2> "shard_range":null,
[junit4] 2> "roles":null,
[junit4] 2> "collection":"collection1",
[junit4] 2> "shard":"shard2",
[junit4] 2> "core":"collection1"}
[junit4] 2> 642915 T2684 oasc.Overseer$ClusterStateUpdater.updateState WARN Could not find collection node for onenodecollection, skipping publish state
[junit4] 2> 642915 T2684 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4] 2> "shard_state":"active",
[junit4] 2> "base_url":"http://127.0.0.1:50555/_eg",
[junit4] 2> "state":"down",
[junit4] 2> "operation":"state",
[junit4] 2> "core_node_name":"core_node1",
[junit4] 2> "node_name":"127.0.0.1:50555__eg",
[junit4] 2> "numShards":"1",
[junit4] 2> "shard_range":null,
[junit4] 2> "roles":"none",
[junit4] 2> "collection":"onenodecollection",
[junit4] 2> "shard":null,
[junit4] 2> "core":"onenodecollectioncore"}
[junit4] 2> 642916 T2684 oasc.Overseer$ClusterStateUpdater.createCollection Create collection onenodecollection with numShards 1
[junit4] 2> 642916 T2684 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4] 2> 642919 T2684 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4] 2> "shard_state":"active",
[junit4] 2> "base_url":"http://127.0.0.1:57404/_eg",
[junit4] 2> "state":"active",
[junit4] 2> "operation":"state",
[junit4] 2> "core_node_name":"core_node3",
[junit4] 2> "node_name":"127.0.0.1:57404__eg",
[junit4] 2> "numShards":"2",
[junit4] 2> "shard_range":null,
[junit4] 2> "roles":null,
[junit4] 2> "collection":"collection1",
[junit4] 2> "shard":"shard1",
[junit4] 2> "core":"collection1"}
[junit4] 2> 642923 T2719 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4] 2> 642924 T2735 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4] 2> 642923 T2690 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4] 2> 642924 T2703 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4] 2> 642924 T2752 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4] 2> 642924 T2683 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4] 2> 643200 T2713 oasha.CoreAdminHandler.handleWaitForStateAction Waited coreNodeName: core_node4, state: recovering, checkLive: true, onlyIfLeader: true for: 1 seconds.
[junit4] 2> 643201 T2713 oass.SolrDispatchFilter.handleAdminRequest [admin] webapp=null path=/admin/cores params={state=recovering&onlyIfLeader=true&checkLive=true&action=PREPRECOVERY&version=2&coreNodeName=core_node4&wt=javabin&nodeName=127.0.0.1:33603__eg&core=collection1} status=0 QTime=1002
[junit4] 2> 643214 T2698 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for onenodecollectioncore
[junit4] 2> 643216 T2698 oasc.CoreContainer.create Creating SolrCore 'onenodecollectioncore' using instanceDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty1-1373835035578/onenodecollectioncore
[junit4] 2> 643217 T2698 oasc.ZkController.createCollectionZkNode Check for collection zkNode:onenodecollection
[junit4] 2> 643218 T2698 oasc.ZkController.createCollectionZkNode Creating collection in ZooKeeper:onenodecollection
[junit4] 2> 643219 T2698 oasc.ZkController.getConfName Looking for collection configName
[junit4] 2> 643221 T2698 oasc.ZkController.getConfName Only one config set found in zk - using it:conf1
[junit4] 2> 643222 T2698 oascc.SolrZkClient.makePath makePath: /collections/onenodecollection
[junit4] 2> 643225 T2698 oasc.ZkController.readConfigName Load collection config from:/collections/onenodecollection
[junit4] 2> 643227 T2698 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty1-1373835035578/onenodecollectioncore/'
[junit4] 2> 643270 T2698 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_45
[junit4] 2> 643317 T2698 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4] 2> 643319 T2698 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4] 2> 643326 T2698 oass.IndexSchema.readSchema [onenodecollectioncore] Schema name=test
[junit4] 2> 643734 T2698 oass.ByteField.init WARN ByteField is deprecated and will be removed in 5.0. You should use TrieIntField instead.
[junit4] 2> 643734 T2698 oass.ShortField.init WARN ShortField is deprecated and will be removed in 5.0. You should use TrieIntField instead.
[junit4] 2> 643735 T2698 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4] 2> 643741 T2698 oass.IndexSchema.readSchema default search field in schema is text
[junit4] 2> 643744 T2698 oass.IndexSchema.readSchema unique key field: id
[junit4] 2> 643758 T2698 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 643762 T2698 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 643767 T2698 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4] 2> 643768 T2698 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4] 2> 643769 T2698 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4] 2> 643769 T2698 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4] 2> 643771 T2698 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4] 2> 643771 T2698 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4] 2> 643772 T2698 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4] 2> 643773 T2698 oasc.SolrCore.<init> [onenodecollectioncore] Opening new SolrCore at /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty1-1373835035578/onenodecollectioncore/, dataDir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./solrtest-BasicDistributedZk2Test-1373835032114/onenodecollection/
[junit4] 2> 643774 T2698 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@d887bb70
[junit4] 2> 643776 T2698 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./solrtest-BasicDistributedZk2Test-1373835032114/onenodecollection
[junit4] 2> 643777 T2698 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./solrtest-BasicDistributedZk2Test-1373835032114/onenodecollection/index/
[junit4] 2> 643777 T2698 oasc.SolrCore.initIndex WARN [onenodecollectioncore] Solr index directory '/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./solrtest-BasicDistributedZk2Test-1373835032114/onenodecollection/index' doesn't exist. Creating new index...
[junit4] 2> 643778 T2698 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./solrtest-BasicDistributedZk2Test-1373835032114/onenodecollection/index
[junit4] 2> 643780 T2698 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@d44c0464 lockFactory=org.apache.lucene.store.NativeFSLockFactory@86d19010),segFN=segments_1,generation=1}
[junit4] 2> 643781 T2698 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 643784 T2698 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4] 2> 643785 T2698 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4] 2> 643786 T2698 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 643787 T2698 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4] 2> 643787 T2698 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4] 2> 643788 T2698 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4] 2> 643789 T2698 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4] 2> 643790 T2698 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4] 2> 643790 T2698 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4] 2> 643792 T2698 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 643794 T2698 oass.SolrIndexSearcher.<init> Opening Searcher@8bdc08d8 main
[junit4] 2> 643796 T2698 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4] 2> 643796 T2698 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4] 2> 643800 T2758 oasc.SolrCore.registerSearcher [onenodecollectioncore] Registered new searcher Searcher@8bdc08d8 main{StandardDirectoryReader(segments_1:1)}
[junit4] 2> 643801 T2698 oasc.CoreContainer.registerCore registering core: onenodecollectioncore
[junit4] 2> 643802 T2698 oasc.ZkController.register Register replica - core:onenodecollectioncore address:http://127.0.0.1:50555/_eg collection:onenodecollection shard:shard1
[junit4] 2> 643803 T2698 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=10000&maxConnectionsPerHost=20&connTimeout=30000&socketTimeout=30000&retry=false
[junit4] 2> 643808 T2698 oascc.SolrZkClient.makePath makePath: /collections/onenodecollection/leader_elect/shard1/election
[junit4] 2> 643815 T2698 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard1
[junit4] 2> 643818 T2698 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4] 2> 643818 T2698 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4] 2> 643819 T2698 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:50555/_eg/onenodecollectioncore/
[junit4] 2> 643819 T2698 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4] 2> 643820 T2698 oasc.SyncStrategy.syncToMe http://127.0.0.1:50555/_eg/onenodecollectioncore/ has no replicas
[junit4] 2> 643821 T2698 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:50555/_eg/onenodecollectioncore/ shard1
[junit4] 2> 643821 T2698 oascc.SolrZkClient.makePath makePath: /collections/onenodecollection/leaders/shard1
[junit4] 2> 644430 T2684 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 644436 T2752 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4] 2> 644436 T2719 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4] 2> 644436 T2703 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4] 2> 644436 T2690 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4] 2> 644436 T2683 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4] 2> 644436 T2735 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4] 2> 644479 T2698 oasc.ZkController.register We are http://127.0.0.1:50555/_eg/onenodecollectioncore/ and leader is http://127.0.0.1:50555/_eg/onenodecollectioncore/
[junit4] 2> 644479 T2698 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4] 2> 644480 T2698 oasc.ZkController.publish publishing core=onenodecollectioncore state=active
[junit4] 2> 644482 T2698 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 644484 T2698 oass.SolrDispatchFilter.handleAdminRequest [admin] webapp=null path=/admin/cores params={dataDir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./solrtest-BasicDistributedZk2Test-1373835032114/onenodecollection&action=CREATE&version=2&numShards=1&name=onenodecollectioncore&wt=javabin&roles=none&collection=onenodecollection} status=0 QTime=2272
[junit4] 2> 644484 T2661 oasc.AbstractDistribZkTestBase.waitForRecoveriesToFinish Wait for recoveries to finish - collection: onenodecollection failOnTimeout:true timeout (sec):330
[junit4] 2> 644486 T2661 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> ASYNC NEW_CORE C749 name=collection1 org.apache.solr.core.SolrCore@831b5cd7 url=http://127.0.0.1:33603/_eg/collection1 node=127.0.0.1:33603__eg C749_STATE=coll:collection1 core:collection1 props:{base_url=http://127.0.0.1:33603/_eg, state=recovering, node_name=127.0.0.1:33603__eg, core=collection1}
[junit4] 2> 645203 T2755 C749 P33603 oasc.RecoveryStrategy.doRecovery Attempting to PeerSync from http://127.0.0.1:55399/_eg/collection1/ core=collection1 - recoveringAfterStartup=true
[junit4] 2> 645203 T2755 C749 P33603 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:33603/_eg START replicas=[http://127.0.0.1:55399/_eg/collection1/] nUpdates=100
[junit4] 2> 645204 T2755 C749 P33603 oasu.PeerSync.sync WARN no frame of reference to tell of we've missed updates
[junit4] 2> 645204 T2755 C749 P33603 oasc.RecoveryStrategy.doRecovery PeerSync Recovery was not successful - trying replication. core=collection1
[junit4] 2> 645205 T2755 C749 P33603 oasc.RecoveryStrategy.doRecovery Starting Replication Recovery. core=collection1
[junit4] 2> 645205 T2755 C749 P33603 oasc.RecoveryStrategy.doRecovery Begin buffering updates. core=collection1
[junit4] 2> 645205 T2755 C749 P33603 oasu.UpdateLog.bufferUpdates Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null}
[junit4] 2> 645206 T2755 C749 P33603 oasc.RecoveryStrategy.replicate Attempting to replicate from http://127.0.0.1:55399/_eg/collection1/. core=collection1
[junit4] 2> 645206 T2755 C749 P33603 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4] 2> ASYNC NEW_CORE C750 name=collection1 org.apache.solr.core.SolrCore@39463499 url=http://127.0.0.1:55399/_eg/collection1 node=127.0.0.1:55399__eg C750_STATE=coll:collection1 core:collection1 props:{base_url=http://127.0.0.1:55399/_eg, state=active, node_name=127.0.0.1:55399__eg, core=collection1, leader=true}
[junit4] 2> 645212 T2713 C750 P55399 oasc.SolrCore.execute [collection1] webapp=/_eg path=/get params={getVersions=100&qt=/get&distrib=false&version=2&wt=javabin} status=0 QTime=0
[junit4] 2> 645213 T2715 C750 P55399 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4] 2> 645215 T2715 C750 P55399 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@9dbe14dc lockFactory=org.apache.lucene.store.NativeFSLockFactory@89ff38bf),segFN=segments_1,generation=1}
[junit4] 2> 645216 T2715 C750 P55399 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 645217 T2715 C750 P55399 oasu.DirectUpdateHandler2.commit No uncommitted changes. Skipping IW.commit.
[junit4] 2> 645219 T2715 C750 P55399 oass.SolrIndexSearcher.<init> Opening Searcher@341dd655 realtime
[junit4] 2> 645220 T2715 C750 P55399 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4] 2> 645221 T2715 C750 P55399 oasup.LogUpdateProcessor.finish [collection1] webapp=/_eg path=/update params={softCommit=false&commit_end_point=true&commit=true&version=2&waitSearcher=true&wt=javabin&openSearcher=false} {commit=} 0 8
[junit4] 2> 645223 T2755 C749 P33603 oash.ReplicationHandler.inform Commits will be reserved for 10000
[junit4] 2> 645224 T2755 C749 P33603 oash.SnapPuller.<init> No value set for 'pollInterval'. Timer Task not started.
[junit4] 2> 645227 T2714 C750 P55399 oash.ReplicationHandler.inform Commits will be reserved for 10000
[junit4] 2> 645228 T2714 C750 P55399 oasc.SolrCore.execute [collection1] webapp=/_eg path=/replication params={qt=/replication&version=2&command=indexversion&wt=javabin} status=0 QTime=2
[junit4] 2> 645229 T2755 C749 P33603 oasc.RecoveryStrategy.replay No replay needed. core=collection1
[junit4] 2> 645229 T2755 C749 P33603 oasc.RecoveryStrategy.doRecovery Replication Recovery was successful - registering as Active. core=collection1
[junit4] 2> 645230 T2755 C749 P33603 oasc.ZkController.publish publishing core=collection1 state=active
[junit4] 2> 645230 T2755 C749 P33603 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 645234 T2755 C749 P33603 oasc.RecoveryStrategy.doRecovery Finished recovery process. core=collection1
[junit4] 2> 645488 T2661 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 645942 T2684 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 645943 T2684 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4] 2> "shard_state":"active",
[junit4] 2> "base_url":"http://127.0.0.1:50555/_eg",
[junit4] 2> "state":"active",
[junit4] 2> "operation":"state",
[junit4] 2> "core_node_name":"core_node1",
[junit4] 2> "node_name":"127.0.0.1:50555__eg",
[junit4] 2> "numShards":"1",
[junit4] 2> "shard_range":null,
[junit4] 2> "roles":"none",
[junit4] 2> "collection":"onenodecollection",
[junit4] 2> "shard":"shard1",
[junit4] 2> "core":"onenodecollectioncore"}
[junit4] 2> 645946 T2684 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4] 2> "shard_state":"active",
[junit4] 2> "base_url":"http://127.0.0.1:33603/_eg",
[junit4] 2> "state":"active",
[junit4] 2> "operation":"state",
[junit4] 2> "core_node_name":"core_node4",
[junit4] 2> "node_name":"127.0.0.1:33603__eg",
[junit4] 2> "numShards":"2",
[junit4] 2> "shard_range":null,
[junit4] 2> "roles":null,
[junit4] 2> "collection":"collection1",
[junit4] 2> "shard":"shard2",
[junit4] 2> "core":"collection1"}
[junit4] 2> 645949 T2752 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4] 2> 645950 T2683 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4] 2> 645950 T2719 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4] 2> 645950 T2703 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4] 2> 645949 T2735 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4] 2> 645950 T2690 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4] 2> 646490 T2661 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 646491 T2661 oasc.AbstractDistribZkTestBase.waitForRecoveriesToFinish Recoveries finished - collection: onenodecollection
[junit4] 2> 646492 T2661 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4] 2> 646501 T2661 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 646503 T2761 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@715c8fb name:ZooKeeperConnection Watcher:127.0.0.1:54515 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 646504 T2661 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 646540 T2661 oasc.ChaosMonkey.monkeyLog monkey: stop shard! 53363
[junit4] 2> 646540 T2661 oasc.CoreContainer.shutdown Shutting down CoreContainer instance=1626852406
[junit4] 2> 647456 T2684 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 647457 T2684 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=null message={
[junit4] 2> "base_url":"http://127.0.0.1:53363/_eg",
[junit4] 2> "state":"down",
[junit4] 2> "operation":"state",
[junit4] 2> "core_node_name":"core_node1",
[junit4] 2> "node_name":"127.0.0.1:53363__eg",
[junit4] 2> "roles":null,
[junit4] 2> "collection":"control_collection",
[junit4] 2> "shard":null,
[junit4] 2> "core":"collection1"}
[junit4] 2> 647458 T2684 oasc.Overseer$ClusterStateUpdater.updateState shard=shard1 is already registered
[junit4] 2> 647461 T2752 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4] 2> 647461 T2719 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4] 2> 647461 T2703 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4] 2> 647461 T2735 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4] 2> 647461 T2683 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4] 2> 648542 T2661 oasc.ZkController.publish publishing core=collection1 state=down
[junit4] 2> 648543 T2661 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 648544 T2661 oasc.SolrCore.close [collection1] CLOSING SolrCore org.apache.solr.core.SolrCore@bb7d8426
[junit4] 2> 648548 T2661 oasu.DirectUpdateHandler2.close closing DirectUpdateHandler2{commits=0,autocommits=0,soft autocommits=0,optimizes=0,rollbacks=0,expungeDeletes=0,docsPending=0,adds=0,deletesById=0,deletesByQuery=0,errors=0,cumulative_adds=0,cumulative_deletesById=0,cumulative_deletesByQuery=0,cumulative_errors=0}
[junit4] 2> 648548 T2661 oasu.SolrCoreState.decrefSolrCoreState Closing SolrCoreState
[junit4] 2> 648549 T2661 oasu.DefaultSolrCoreState.closeIndexWriter SolrCoreState ref count has reached 0 - closing IndexWriter
[junit4] 2> 648549 T2661 oasu.DefaultSolrCoreState.closeIndexWriter closing IndexWriter with IndexWriterCloser
[junit4] 2> 648550 T2661 oasc.SolrCore.closeSearcher [collection1] Closing main searcher on request.
[junit4] 2> 648550 T2661 oasc.CachingDirectoryFactory.close Closing MockDirectoryFactory - 2 directories currently being tracked
[junit4] 2> 648551 T2661 oasc.CachingDirectoryFactory.closeCacheValue looking to close ./org.apache.solr.cloud.BasicDistributedZk2Test-1373835032113/control/data/index [CachedDir<<refCount=0;path=./org.apache.solr.cloud.BasicDistributedZk2Test-1373835032113/control/data/index;done=false>>]
[junit4] 2> 648551 T2661 oasc.CachingDirectoryFactory.close Closing directory: ./org.apache.solr.cloud.BasicDistributedZk2Test-1373835032113/control/data/index
[junit4] 2> 648552 T2661 oasc.CachingDirectoryFactory.closeCacheValue looking to close ./org.apache.solr.cloud.BasicDistributedZk2Test-1373835032113/control/data [CachedDir<<refCount=0;path=./org.apache.solr.cloud.BasicDistributedZk2Test-1373835032113/control/data;done=false>>]
[junit4] 2> 648552 T2661 oasc.CachingDirectoryFactory.close Closing directory: ./org.apache.solr.cloud.BasicDistributedZk2Test-1373835032113/control/data
[junit4] 2> 648554 T2684 oasc.Overseer$ClusterStateUpdater.amILeader According to ZK I (id=90035652664754179-127.0.0.1:53363__eg-n_0000000000) am no longer a leader.
[junit4] 2> 648554 T2719 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4] 2> 648555 T2752 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4] 2> 648555 T2752 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4] 2> 648556 T2735 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4] 2> 648556 T2735 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4] 2> 648557 T2703 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
[junit4] 2> 648557 T2719 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4] 2> 648559 T2703 oasc.Overseer.start Overseer (id=90035652664754182-127.0.0.1:50555__eg-n_0000000001) starting
[junit4] 2> 648563 T2703 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4] 2> 648563 T2763 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
[junit4] 2> 648563 T2703 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4] 2> 648564 T2762 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
[junit4] 2> 648566 T2762 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 648567 T2762 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4] 2> "shard_state":"active",
[junit4] 2> "base_url":"http://127.0.0.1:53363/_eg",
[junit4] 2> "state":"down",
[junit4] 2> "operation":"state",
[junit4] 2> "core_node_name":"core_node1",
[junit4] 2> "node_name":"127.0.0.1:53363__eg",
[junit4] 2> "numShards":"2",
[junit4] 2> "shard_range":null,
[junit4] 2> "roles":null,
[junit4] 2> "collection":"control_collection",
[junit4] 2> "shard":"shard1",
[junit4] 2> "core":"collection1"}
[junit4] 2> 648570 T2752 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4] 2> 648570 T2735 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4] 2> 648570 T2719 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4] 2> 648570 T2703 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4] 2> 648577 T2661 oejsh.ContextHandler.doStop stopped o.e.j.s.ServletContextHandler{/_eg,null}
[junit4] 2> 648630 T2661 oasc.ChaosMonkey.monkeyLog monkey: stop shard! 50555
[junit4] 2> 648631 T2661 oasc.CoreContainer.shutdown Shutting down CoreContainer instance=570463034
[junit4] 2> 649855 T2683 oascc.ZkStateReader$3.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4] 2> 649856 T2683 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4] 2> 649856 T2683 oascc.ZkStateReader$2.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4] 2> 650075 T2762 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 650076 T2762 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=null message={
[junit4] 2> "base_url":"http://127.0.0.1:50555/_eg",
[junit4] 2> "state":"down",
[junit4] 2> "operation":"state",
[junit4] 2> "core_node_name":"core_node1",
[junit4] 2> "node_name":"127.0.0.1:50555__eg",
[junit4] 2> "roles":"none",
[junit4] 2> "collection":"onenodecollection",
[junit4] 2> "shard":null,
[junit4] 2> "core":"onenodecollectioncore"}
[junit4] 2> 650076 T2762 oasc.Overseer$ClusterStateUpdater.updateState shard=shard1 is already registered
[junit4] 2> 650079 T2762 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=null message={
[junit4] 2> "base_url":"http://127.0.0.1:50555/_eg",
[junit4] 2> "state":"down",
[junit4] 2> "operation":"state",
[junit4] 2> "core_node_name":"core_node1",
[junit4] 2> "node_name":"127.0.0.1:50555__eg",
[junit4] 2> "roles":null,
[junit4] 2> "collection":"collection1",
[junit4] 2> "shard":null,
[junit4] 2> "core":"collection1"}
[junit4] 2> 650079 T2762 oasc.Overseer$ClusterStateUpdater.updateState shard=shard1 is already registered
[junit4] 2> 650082 T2752 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4] 2> 650082 T2735 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4] 2> 650082 T2719 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4] 2> 650082 T2703 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4] 2> 651634 T2661 oasc.ZkController.publish publishing core=collection1 state=down
[junit4] 2> 651635 T2661 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 651636 T2661 oasc.ZkController.publish publishing core=onenodecollectioncore state=down
[junit4] 2> 651638 T2661 oasc.SolrCore.close [collection1] CLOSING SolrCore org.apache.solr.core.SolrCore@eb7dc2cf
[junit4] 2> 651641 T2661 oasu.DirectUpdateHandler2.close closing DirectUpdateHandler2{commits=1,autocommits=0,soft autocommits=0,optimizes=0,rollbacks=0,expungeDeletes=0,docsPending=0,adds=0,deletesById=0,deletesByQuery=0,errors=0,cumulative_adds=0,cumulative_deletesById=0,cumulative_deletesByQuery=0,cumulative_errors=0}
[junit4] 2> 651642 T2661 oasu.SolrCoreState.decrefSolrCoreState Closing SolrCoreState
[junit4] 2> 651642 T2661 oasu.DefaultSolrCoreState.closeIndexWriter SolrCoreState ref count has reached 0 - closing IndexWriter
[junit4] 2> 651643 T2661 oasu.DefaultSolrCoreState.closeIndexWriter closing IndexWriter with IndexWriterCloser
[junit4] 2> 651644 T2661 oasc.SolrCore.closeSearcher [collection1] Closing main searcher on request.
[junit4] 2> 651644 T2661 oasc.CachingDirectoryFactory.close Closing MockDirectoryFactory - 2 directories currently being tracked
[junit4] 2> 651645 T2661 oasc.CachingDirectoryFactory.closeCacheValue looking to close ./org.apache.solr.cloud.BasicDistributedZk2Test-1373835032113/jetty1 [CachedDir<<refCount=0;path=./org.apache.solr.cloud.BasicDistributedZk2Test-1373835032113/jetty1;done=false>>]
[junit4] 2> 651645 T2661 oasc.CachingDirectoryFactory.close Closing directory: ./org.apache.solr.cloud.BasicDistributedZk2Test-1373835032113/jetty1
[junit4] 2> 651646 T2661 oasc.CachingDirectoryFactory.closeCacheValue looking to close ./org.apache.solr.cloud.BasicDistributedZk2Test-1373835032113/jetty1/index [CachedDir<<refCount=0;path=./org.apache.solr.cloud.BasicDistributedZk2Test-1373835032113/jetty1/index;done=false>>]
[junit4] 2> 651646 T2661 oasc.CachingDirectoryFactory.close Closing directory: ./org.apache.solr.cloud.BasicDistributedZk2Test-1373835032113/jetty1/index
[junit4] 2> 651647 T2661 oasc.SolrCore.close [onenodecollectioncore] CLOSING SolrCore org.apache.solr.core.SolrCore@72e7c462
[junit4] 2> 651652 T2661 oasu.DirectUpdateHandler2.close closing DirectUpdateHandler2{commits=0,autocommits=0,soft autocommits=0,optimizes=0,rollbacks=0,expungeDeletes=0,docsPending=0,adds=0,deletesById=0,deletesByQuery=0,errors=0,cumulative_adds=0,cumulative_deletesById=0,cumulative_deletesByQuery=0,cumulative_errors=0}
[junit4] 2> 651652 T2661 oasu.SolrCoreState.decrefSolrCoreState Closing SolrCoreState
[junit4] 2> 651653 T2661 oasu.DefaultSolrCoreState.closeIndexWriter SolrCoreState ref count has reached 0 - closing IndexWriter
[junit4] 2> 651653 T2661 oasu.DefaultSolrCoreState.closeIndexWriter closing IndexWriter with IndexWriterCloser
[junit4] 2> 651654 T2661 oasc.SolrCore.closeSearcher [onenodecollectioncore] Closing main searcher on request.
[junit4] 2> 651655 T2661 oasc.CachingDirectoryFactory.close Closing MockDirectoryFactory - 2 directories currently being tracked
[junit4] 2> 651655 T2661 oasc.CachingDirectoryFactory.closeCacheValue looking to close /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./solrtest-BasicDistributedZk2Test-1373835032114/onenodecollection [CachedDir<<refCount=0;path=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./solrtest-BasicDistributedZk2Test-1373835032114/onenodecollection;done=false>>]
[junit4] 2> 651656 T2661 oasc.CachingDirectoryFactory.close Closing directory: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./solrtest-BasicDistributedZk2Test-1373835032114/onenodecollection
[junit4] 2> 651656 T2661 oasc.CachingDirectoryFactory.closeCacheValue looking to close /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./solrtest-BasicDistributedZk2Test-1373835032114/onenodecollection/index [CachedDir<<refCount=0;path=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./solrtest-BasicDistributedZk2Test-1373835032114/onenodecollection/index;done=false>>]
[junit4] 2> 651657 T2661 oasc.CachingDirectoryFactory.close Closing directory: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./solrtest-BasicDistributedZk2Test-1373835032114/onenodecollection/index
[junit4] 2> 651658 T2762 oasc.Overseer$ClusterStateUpdater.amILeader According to ZK I (id=90035652664754182-127.0.0.1:50555__eg-n_0000000001) am no longer a leader.
[junit4] 2> 651659 T2752 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4] 2> 651659 T2752 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4] 2> 651660 T2735 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard1
[junit4] 2> 651660 T2719 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
[junit4] 2> 651662 T2735 oasc.ShardLeaderElectionContext.shouldIBeLeader Checking if I should try and be the leader.
[junit4] 2> 651662 T2735 oasc.ShardLeaderElectionContext.shouldIBeLeader My last published State was Active, it's okay to be the leader.
[junit4] 2> 651662 T2735 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4] 2> 651662 T2719 oasc.Overseer.start Overseer (id=90035652664754184-127.0.0.1:55399__eg-n_0000000002) starting
[junit4] 2> 651663 T2735 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:57404/_eg/collection1/
[junit4] 2> 651663 T2735 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:57404/_eg START replicas=[http://127.0.0.1:50555/_eg/collection1/] nUpdates=100
[junit4] 2> 651664 T2735 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:57404/_eg DONE. We have no versions. sync failed.
[junit4] 2> 651664 T2735 oasc.SyncStrategy.syncReplicas Leader's attempt to sync with shard failed, moving to the next candidate
[junit4] 2> 651665 T2735 oasc.ShardLeaderElectionContext.runLeaderProcess We failed sync, but we have no versions - we can't sync in that case - we were active before, so become leader anyway
[junit4] 2> 651665 T2735 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:57404/_eg/collection1/ shard1
[junit4] 2> 651665 T2735 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
[junit4] 2> 651667 T2719 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4] 2> 651667 T2766 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
[junit4] 2> 651669 T2719 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4] 2> 651669 T2765 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
[junit4] 2> 651669 T2735 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4] 2> 651670 T2735 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4] 2> 651671 T2765 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 651672 T2765 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4] 2> "shard_state":"active",
[junit4] 2> "base_url":"http://127.0.0.1:50555/_eg",
[junit4] 2> "state":"down",
[junit4] 2> "operation":"state",
[junit4] 2> "core_node_name":"core_node1",
[junit4] 2> "node_name":"127.0.0.1:50555__eg",
[junit4] 2> "numShards":"2",
[junit4] 2> "shard_range":null,
[junit4] 2> "roles":null,
[junit4] 2> "collection":"collection1",
[junit4] 2> "shard":"shard1",
[junit4] 2> "core":"collection1"}
[junit4] 2> 651677 T2765 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4] 2> "shard_state":"active",
[junit4] 2> "base_url":"http://127.0.0.1:50555/_eg",
[junit4] 2> "state":"down",
[junit4] 2> "operation":"state",
[junit4] 2> "core_node_name":"core_node1",
[junit4] 2> "node_name":"127.0.0.1:50555__eg",
[junit4] 2> "numShards":"1",
[junit4] 2> "shard_range":null,
[junit4] 2> "roles":"none",
[junit4] 2> "collection":"onenodecollection",
[junit4] 2> "shard":"shard1",
[junit4] 2> "core":"onenodecollectioncore"}
[junit4] 2> 651680 T2661 oejsh.ContextHandler.doStop stopped o.e.j.s.ServletContextHandler{/_eg,null}
[junit4] 2> 651686 T2752 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 651687 T2735 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 651687 T2719 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 651732 T2661 oasc.ChaosMonkey.monkeyLog monkey: stop shard! 55399
[junit4] 2> 651732 T2661 oasc.CoreContainer.shutdown Shutting down CoreContainer instance=348014057
[junit4] 2> 652734 T2661 oasc.ZkController.publish publishing core=collection1 state=down
[junit4] 2> 652735 T2661 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 652737 T2661 oasc.SolrCore.close [collection1] CLOSING SolrCore org.apache.solr.core.SolrCore@39463499
[junit4] 2> 652740 T2661 oasu.DirectUpdateHandler2.close closing DirectUpdateHandler2{commits=1,autocommits=0,soft autocommits=0,optimizes=0,rollbacks=0,expungeDeletes=0,docsPending=0,adds=0,deletesById=0,deletesByQuery=0,errors=0,cumulative_adds=0,cumulative_deletesById=0,cumulative_deletesByQuery=0,cumulative_errors=0}
[junit4] 2> 652740 T2661 oasu.SolrCoreState.decrefSolrCoreState Closing SolrCoreState
[junit4] 2> 652741 T2661 oasu.DefaultSolrCoreState.closeIndexWriter SolrCoreState ref count has reached 0 - closing IndexWriter
[junit4] 2> 652741 T2661 oasu.DefaultSolrCoreState.closeIndexWriter closing IndexWriter with IndexWriterCloser
[junit4] 2> 652743 T2661 oasc.SolrCore.closeSearcher [collection1] Closing main searcher on request.
[junit4] 2> 652743 T2661 oasc.CachingDirectoryFactory.close Closing MockDirectoryFactory - 2 directories currently being tracked
[junit4] 2> 652744 T2661 oasc.CachingDirectoryFactory.closeCacheValue looking to close ./org.apache.solr.cloud.BasicDistributedZk2Test-1373835032113/jetty2/index [CachedDir<<refCount=0;path=./org.apache.solr.cloud.BasicDistributedZk2Test-1373835032113/jetty2/index;done=false>>]
[junit4] 2> 652744 T2661 oasc.CachingDirectoryFactory.close Closing directory: ./org.apache.solr.cloud.BasicDistributedZk2Test-1373835032113/jetty2/index
[junit4] 2> 652745 T2661 oasc.CachingDirectoryFactory.closeCacheValue looking to close ./org.apache.solr.cloud.BasicDistributedZk2Test-1373835032113/jetty2 [CachedDir<<refCount=0;path=./org.apache.solr.cloud.BasicDistributedZk2Test-1373835032113/jetty2;done=false>>]
[junit4] 2> 652745 T2661 oasc.CachingDirectoryFactory.close Closing directory: ./org.apache.solr.cloud.BasicDistributedZk2Test-1373835032113/jetty2
[junit4] 2> 652746 T2765 oasc.Overseer$ClusterStateUpdater.amILeader According to ZK I (id=90035652664754184-127.0.0.1:55399__eg-n_0000000002) am no longer a leader.
[junit4] 2> 652747 T2719 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4] 2> 652747 T2735 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4] 2> 652747 T2752 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4] 2> 652748 T2735 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
[junit4] 2> 652748 T2752 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
[junit4] 2> 652750 T2735 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
[junit4] 2> 652750 T2752 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard2
[junit4] 2> 652752 T2752 oasc.ShardLeaderElectionContext.shouldIBeLeader Checking if I should try and be the leader.
[junit4] 2> 652753 T2752 oasc.ShardLeaderElectionContext.shouldIBeLeader My last published State was Active, it's okay to be the leader.
[junit4] 2> 652753 T2752 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4] 2> 652752 T2735 oasc.Overseer.start Overseer (id=90035652664754186-127.0.0.1:57404__eg-n_0000000003) starting
[junit4] 2> 652753 T2752 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:33603/_eg/collection1/
[junit4] 2> 652754 T2752 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4] 2> 652754 T2752 oasc.SyncStrategy.syncToMe http://127.0.0.1:33603/_eg/collection1/ has no replicas
[junit4] 2> 652754 T2752 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:33603/_eg/collection1/ shard2
[junit4] 2> 652755 T2752 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard2
[junit4] 2> 652757 T2768 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
[junit4] 2> 652758 T2767 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
[junit4] 2> 652760 T2767 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 652761 T2767 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=null message={
[junit4] 2> "base_url":"http://127.0.0.1:55399/_eg",
[junit4] 2> "state":"down",
[junit4] 2> "operation":"state",
[junit4] 2> "core_node_name":"core_node2",
[junit4] 2> "node_name":"127.0.0.1:55399__eg",
[junit4] 2> "roles":null,
[junit4] 2> "collection":"collection1",
[junit4] 2> "shard":null,
[junit4] 2> "core":"collection1"}
[junit4] 2> 652761 T2767 oasc.Overseer$ClusterStateUpdater.updateState shard=shard2 is already registered
[junit4] 2> 652764 T2767 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4] 2> "shard_state":"active",
[junit4] 2> "base_url":"http://127.0.0.1:55399/_eg",
[junit4] 2> "state":"down",
[junit4] 2> "operation":"state",
[junit4] 2> "core_node_name":"core_node2",
[junit4] 2> "node_name":"127.0.0.1:55399__eg",
[junit4] 2> "numShards":"2",
[junit4] 2> "shard_range":null,
[junit4] 2> "roles":null,
[junit4] 2> "collection":"collection1",
[junit4] 2> "shard":"shard2",
[junit4] 2> "core":"collection1"}
[junit4] 2> 652768 T2661 oejsh.ContextHandler.doStop stopped o.e.j.s.ServletContextHandler{/_eg,null}
[junit4] 2> 652770 T2752 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 652771 T2735 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 652820 T2661 oasc.ChaosMonkey.monkeyLog monkey: stop shard! 57404
[junit4] 2> 652821 T2661 oasc.CoreContainer.shutdown Shutting down CoreContainer instance=-459037688
[junit4] 2> 652959 T2703 oascc.ZkStateReader$3.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4] 2> 652959 T2703 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4] 2> 652959 T2703 oascc.ZkStateReader$2.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4] 2> 653823 T2661 oasc.ZkController.publish publishing core=collection1 state=down
[junit4] 2> 653824 T2661 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 653826 T2661 oasc.SolrCore.close [collection1] CLOSING SolrCore org.apache.solr.core.SolrCore@c5d87327
[junit4] 2> 653829 T2661 oasu.DirectUpdateHandler2.close closing DirectUpdateHandler2{commits=0,autocommits=0,soft autocommits=0,optimizes=0,rollbacks=0,expungeDeletes=0,docsPending=0,adds=0,deletesById=0,deletesByQuery=0,errors=0,cumulative_adds=0,cumulative_deletesById=0,cumulative_deletesByQuery=0,cumulative_errors=0}
[junit4] 2> 653830 T2661 oasu.SolrCoreState.decrefSolrCoreState Closing SolrCoreState
[junit4] 2> 653830 T2661 oasu.DefaultSolrCoreState.closeIndexWriter SolrCoreState ref count has reached 0 - closing IndexWriter
[junit4] 2> 653831 T2661 oasu.DefaultSolrCoreState.closeIndexWriter closing IndexWriter with IndexWriterCloser
[junit4] 2> 653832 T2661 oasc.SolrCore.closeSearcher [collection1] Closing main searcher on request.
[junit4] 2> 653832 T2661 oasc.CachingDirectoryFactory.close Closing MockDirectoryFactory - 2 directories currently being tracked
[junit4] 2> 653833 T2661 oasc.CachingDirectoryFactory.closeCacheValue looking to close ./org.apache.solr.cloud.BasicDistributedZk2Test-1373835032113/jetty3/index [CachedDir<<refCount=0;path=./org.apache.solr.cloud.BasicDistributedZk2Test-1373835032113/jetty3/index;done=false>>]
[junit4] 2> 653833 T2661 oasc.CachingDirectoryFactory.close Closing directory: ./org.apache.solr.cloud.BasicDistributedZk2Test-1373835032113/jetty3/index
[junit4] 2> 653834 T2661 oasc.CachingDirectoryFactory.closeCacheValue looking to close ./org.apache.solr.cloud.BasicDistributedZk2Test-1373835032113/jetty3 [CachedDir<<refCount=0;path=./org.apache.solr.cloud.BasicDistributedZk2Test-1373835032113/jetty3;done=false>>]
[junit4] 2> 653834 T2661 oasc.CachingDirectoryFactory.close Closing directory: ./org.apache.solr.cloud.BasicDistributedZk2Test-1373835032113/jetty3
[junit4] 2> 653835 T2767 oasc.Overseer$ClusterStateUpdater.amILeader According to ZK I (id=90035652664754186-127.0.0.1:57404__eg-n_0000000003) am no longer a leader.
[junit4] 2> 653837 T2752 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
[junit4] 2> 653839 T2752 oasc.Overseer.start Overseer (id=90035652664754188-127.0.0.1:33603__eg-n_0000000004) starting
[junit4] 2> 653842 T2770 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
[junit4] 2> 653843 T2752 oascc.ZkStateReader$3.process Updating live nodes... (1)
[junit4] 2> 653843 T2752 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
[junit4] 2> 653843 T2769 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
[junit4] 2> 653845 T2769 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 653846 T2769 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=null message={
[junit4] 2> "base_url":"http://127.0.0.1:57404/_eg",
[junit4] 2> "state":"down",
[junit4] 2> "operation":"state",
[junit4] 2> "core_node_name":"core_node3",
[junit4] 2> "node_name":"127.0.0.1:57404__eg",
[junit4] 2> "roles":null,
[junit4] 2> "collection":"collection1",
[junit4] 2> "shard":null,
[junit4] 2> "core":"collection1"}
[junit4] 2> 653846 T2769 oasc.Overseer$ClusterStateUpdater.updateState shard=shard1 is already registered
[junit4] 2> 653849 T2769 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4] 2> "shard_state":"active",
[junit4] 2> "base_url":"http://127.0.0.1:57404/_eg",
[junit4] 2> "state":"down",
[junit4] 2> "operation":"state",
[junit4] 2> "core_node_name":"core_node3",
[junit4] 2> "node_name":"127.0.0.1:57404__eg",
[junit4] 2> "numShards":"2",
[junit4] 2> "shard_range":null,
[junit4] 2> "roles":null,
[junit4] 2> "collection":"collection1",
[junit4] 2> "shard":"shard1",
[junit4] 2> "core":"collection1"}
[junit4] 2> 653851 T2752 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 653857 T2661 oejsh.ContextHandler.doStop stopped o.e.j.s.ServletContextHandler{/_eg,null}
[junit4] 2> 653909 T2661 oasc.ChaosMonkey.monkeyLog monkey: stop shard! 33603
[junit4] 2> 653910 T2661 oasc.CoreContainer.shutdown Shutting down CoreContainer instance=1340836413
[junit4] 2> 654047 T2719 oascc.ZkStateReader$2.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4] 2> 654048 T2719 oascc.ZkStateReader$3.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4] 2> 654912 T2661 oasc.ZkController.publish publishing core=collection1 state=down
[junit4] 2> 654912 T2661 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 654915 T2661 oasc.SolrCore.close [collection1] CLOSING SolrCore org.apache.solr.core.SolrCore@831b5cd7
[junit4] 2> 654921 T2661 oasu.DirectUpdateHandler2.close closing DirectUpdateHandler2{commits=0,autocommits=0,soft autocommits=0,optimizes=0,rollbacks=0,expungeDeletes=0,docsPending=0,adds=0,deletesById=0,deletesByQuery=0,errors=0,cumulative_adds=0,cumulative_deletesById=0,cumulative_deletesByQuery=0,cumulative_errors=0}
[junit4] 2> 654921 T2661 oasu.SolrCoreState.decrefSolrCoreState Closing SolrCoreState
[junit4] 2> 654922 T2661 oasu.DefaultSolrCoreState.closeIndexWriter SolrCoreState ref count has reached 0 - closing IndexWriter
[junit4] 2> 654922 T2661 oasu.DefaultSolrCoreState.closeIndexWriter closing IndexWriter with IndexWriterCloser
[junit4] 2> 654923 T2661 oasc.SolrCore.closeSearcher [collection1] Closing main searcher on request.
[junit4] 2> 654924 T2661 oasc.CachingDirectoryFactory.close Closing MockDirectoryFactory - 2 directories currently being tracked
[junit4] 2> 654924 T2661 oasc.CachingDirectoryFactory.closeCacheValue looking to close ./org.apache.solr.cloud.BasicDistributedZk2Test-1373835032113/jetty4/index [CachedDir<<refCount=0;path=./org.apache.solr.cloud.BasicDistributedZk2Test-1373835032113/jetty4/index;done=false>>]
[junit4] 2> 654925 T2661 oasc.CachingDirectoryFactory.close Closing directory: ./org.apache.solr.cloud.BasicDistributedZk2Test-1373835032113/jetty4/index
[junit4] 2> 654925 T2661 oasc.CachingDirectoryFactory.closeCacheValue looking to close ./org.apache.solr.cloud.BasicDistributedZk2Test-1373835032113/jetty4 [CachedDir<<refCount=0;path=./org.apache.solr.cloud.BasicDistributedZk2Test-1373835032113/jetty4;done=false>>]
[junit4] 2> 654926 T2661 oasc.CachingDirectoryFactory.close Closing directory: ./org.apache.solr.cloud.BasicDistributedZk2Test-1373835032113/jetty4
[junit4] 2> 654926 T2769 oasc.Overseer$ClusterStateUpdater.amILeader According to ZK I (id=90035652664754188-127.0.0.1:33603__eg-n_0000000004) am no longer a leader.
[junit4] 2> 654948 T2661 oejsh.ContextHandler.doStop stopped o.e.j.s.ServletContextHandler{/_eg,null}
[junit4] 2> 655004 T2661 oas.SolrTestCaseJ4.tearDown ###Ending testDistribSearch
[junit4] 2> 655005 T2661 oasc.ZkTestServer.send4LetterWord connecting to 127.0.0.1:54515 54515
[junit4] 1> client port:0.0.0.0/0.0.0.0:0
[junit4] 1> / (2)
[junit4] 1> DATA:
[junit4] 1>
[junit4] 1> /solr (7)
[junit4] 1> /solr/configs (1)
[junit4] 1> /solr/configs/conf1 (9)
[junit4] 1> /solr/configs/conf1/old_synonyms.txt (0)
[junit4] 1> DATA: ...supressed...
[junit4] 1> /solr/configs/conf1/protwords.txt (0)
[junit4] 1> DATA: ...supressed...
[junit4] 1> /solr/configs/conf1/stopwords.txt (0)
[junit4] 1> DATA: ...supressed...
[junit4] 1> /solr/configs/conf1/synonyms.txt (0)
[junit4] 1> DATA: ...supressed...
[junit4] 1> /solr/configs/conf1/schema.xml (0)
[junit4] 1> DATA: ...supressed...
[junit4] 1> /solr/configs/conf1/currency.xml (0)
[junit4] 1> DATA: ...supressed...
[junit4] 1> /solr/configs/conf1/open-exchange-rates.json (0)
[junit4] 1> DATA:
[junit4] 1> {
[junit4] 1> "disclaimer": "This data is not real, it was synthetically created to match currency.xml. It is modeled after the data format available from openexchangerates.org. See https://openexchangerates.org/documentation for details",
[junit4] 1> "license": "http://www.apache.org/licenses/LICENSE-2.0",
[junit4] 1> "timestamp": 1332070464,
[junit4] 1>
[junit4] 1>
[junit4] 1> "IMPORTANT NOTE": "In order for tests to work, this data must be kept in sync with ./currency.xml",
[junit4] 1>
[junit4] 1>
[junit4] 1> "base": "USD",
[junit4] 1> "rates": {
[junit4] 1> "USD": 1,
[junit4] 1> "JPY": 81.29,
[junit4] 1> "EUR": 2.5,
[junit4] 1> "GBP": 0.5,
[junit4] 1> "MXN": 2.0
[junit4] 1> }
[junit4] 1> }
[junit4] 1>
[junit4] 1> /solr/configs/conf1/solrconfig.xml (0)
[junit4] 1> DATA: ...supressed...
[junit4] 1> /solr/configs/conf1/mapping-ISOLatin1Accent.txt (0)
[junit4] 1> DATA: ...supressed...
[junit4] 1> /solr/overseer (3)
[junit4] 1> DATA:
[junit4] 1>
[junit4] 1> /solr/overseer/queue (0)
[junit4] 1> /solr/overseer/collection-queue-work (0)
[junit4] 1> /solr/overseer/queue-work (0)
[junit4] 1> /solr/overseer_elect (2)
[junit4] 1> /solr/overseer_elect/leader (0)
[junit4] 1> DATA:
[junit4] 1> {"id":"90035652664754179-127.0.0.1:53363__eg-n_0000000000"}
[junit4] 1> /solr/overseer_elect/election (5)
[junit4] 1> /solr/overseer_elect/election/90035652664754188-127.0.0.1:33603__eg-n_0000000004 (0)
[junit4] 1> /solr/overseer_elect/election/90035652664754184-127.0.0.1:55399__eg-n_0000000002 (0)
[junit4] 1> /solr/overseer_elect/election/90035652664754179-127.0.0.1:53363__eg-n_0000000000 (0)
[junit4] 1> /solr/overseer_elect/election/90035652664754182-127.0.0.1:50555__eg-n_0000000001 (0)
[junit4] 1> /solr/overseer_elect/election/90035652664754186-127.0.0.1:57404__eg-n_0000000003 (0)
[junit4] 1> /solr/collections (3)
[junit4] 1> /solr/collections/control_collection (3)
[junit4] 1> DATA:
[junit4] 1> {"configName":"conf1"}
[junit4] 1> /solr/collections/control_collection/leader_elect (1)
[junit4] 1> /solr/collections/control_collection/leader_elect/shard1 (1)
[junit4] 1> /solr/collections/control_collection/leader_elect/shard1/election (1)
[junit4] 1> /solr/collections/control_collection/leader_elect/shard1/election/90035652664754179-core_node1-n_0000000000 (0)
[junit4] 1> /solr/collections/control_collection/leaders (1)
[junit4] 1> /solr/collections/control_collection/leaders/shard1 (0)
[junit4] 1> DATA:
[junit4] 1> {
[junit4] 1> "base_url":"http://127.0.0.1:53363/_eg",
[junit4] 1> "node_name":"127.0.0.1:53363__eg",
[junit4] 1> "core":"collection1"}
[junit4] 1> /solr/collections/control_collection/shards (0)
[junit4] 1> /solr/collections/collection1 (3)
[junit4] 1> DATA:
[junit4] 1> {"configName":"conf1"}
[junit4] 1> /solr/collections/collection1/leader_elect (2)
[junit4] 1> /solr/collections/collection1/leader_elect/shard1 (1)
[junit4] 1> /solr/collections/collection1/leader_elect/shard1/election (2)
[junit4] 1> /solr/collections/collection1/leader_elect/shard1/election/90035652664754182-core_node1-n_0000000000 (0)
[junit4] 1> /solr/collections/collection1/leader_elect/shard1/election/90035652664754186-core_node3-n_0000000001 (0)
[junit4] 1> /solr/collections/collection1/leader_elect/shard2 (1)
[junit4] 1> /solr/collections/collection1/leader_elect/shard2/election (2)
[junit4] 1> /solr/collections/collection1/leader_elect/shard2/election/90035652664754184-core_node2-n_0000000000 (0)
[junit4] 1> /solr/collections/collection1/leader_elect/shard2/election/90035652664754188-core_node4-n_0000000001 (0)
[junit4] 1> /solr/collections/collection1/leaders (2)
[junit4] 1> /solr/collections/collection1/leaders/shard1 (0)
[junit4] 1> DATA:
[junit4] 1> {
[junit4] 1> "base_url":"http://127.0.0.1:50555/_eg",
[junit4] 1> "node_name":"127.0.0.1:50555__eg",
[junit4] 1> "core":"collection1"}
[junit4] 1> /solr/collections/collection1/leaders/shard2 (0)
[junit4] 1> DATA:
[junit4] 1> {
[junit4] 1> "base_url":"http://127.0.0.1:55399/_eg",
[junit4] 1> "node_name":"127.0.0.1:55399__eg",
[junit4] 1> "core":"collection1"}
[junit4] 1> /solr/collections/collection1/shards (0)
[junit4] 1> /solr/collections/onenodecollection (2)
[junit4] 1> DATA:
[junit4] 1> {
[junit4] 1> "router":"compositeId",
[junit4] 1> "configName":"conf1"}
[junit4] 1> /solr/collections/onenodecollection/leader_elect (1)
[junit4] 1> /solr/collections/onenodecollection/leader_elect/shard1 (1)
[junit4] 1> /solr/collections/onenodecollection/leader_elect/shard1/election (1)
[junit4] 1> /solr/collections/onenodecollection/leader_elect/shard1/election/90035652664754182-core_node1-n_0000000000 (0)
[junit4] 1> /solr/collections/onenodecollection/leaders (1)
[junit4] 1> /solr/collections/onenodecollection/leaders/shard1 (0)
[junit4] 1> DATA:
[junit4] 1> {
[junit4] 1> "base_url":"http://127.0.0.1:50555/_eg",
[junit4] 1> "node_name":"127.0.0.1:50555__eg",
[junit4] 1> "core":"onenodecollectioncore"}
[junit4] 1> /solr/aliases.json (0)
[junit4] 1> /solr/live_nodes (5)
[junit4] 1> /solr/live_nodes/127.0.0.1:55399__eg (0)
[junit4] 1> /solr/live_nodes/127.0.0.1:57404__eg (0)
[junit4] 1> /solr/live_nodes/127.0.0.1:33603__eg (0)
[junit4] 1> /solr/live_nodes/127.0.0.1:50555__eg (0)
[junit4] 1> /solr/live_nodes/127.0.0.1:53363__eg (0)
[junit4] 1> /solr/clusterstate.json (0)
[junit4] 1> DATA:
[junit4] 1> {
[junit4] 1> "control_collection":{
[junit4] 1> "shards":{"shard1":{
[junit4] 1> "range":"80000000-7fffffff",
[junit4] 1> "state":"active",
[junit4] 1> "replicas":{"core_node1":{
[junit4] 1> "base_url":"http://127.0.0.1:53363/_eg",
[junit4] 1> "state":"active",
[junit4] 1> "node_name":"127.0.0.1:53363__eg",
[junit4] 1> "core":"collection1",
[junit4] 1> "leader":"true"}}}},
[junit4] 1> "router":"compositeId"},
[junit4] 1> "onenodecollection":{
[junit4] 1> "shards":{"shard1":{
[junit4] 1> "range":"80000000-7fffffff",
[junit4] 1> "state":"active",
[junit4] 1> "replicas":{"core_node1":{
[junit4] 1> "base_url":"http://127.0.0.1:50555/_eg",
[junit4] 1> "state":"active",
[junit4] 1> "node_name":"127.0.0.1:50555__eg",
[junit4] 1> "roles":"none",
[junit4] 1> "core":"onenodecollectioncore",
[junit4] 1> "leader":"true"}}}},
[junit4] 1> "router":"compositeId"},
[junit4] 1> "collection1":{
[junit4] 1> "shards":{
[junit4] 1> "shard1":{
[junit4] 1> "range":"80000000-ffffffff",
[junit4] 1> "state":"active",
[junit4] 1> "replicas":{
[junit4] 1> "core_node1":{
[junit4] 1> "base_url":"http://127.0.0.1:50555/_eg",
[junit4] 1> "state":"active",
[junit4] 1> "node_name":"127.0.0.1:50555__eg",
[junit4] 1> "core":"collection1",
[junit4] 1> "leader":"true"},
[junit4] 1> "core_node3":{
[junit4] 1> "base_url":"http://127.0.0.1:57404/_eg",
[junit4] 1> "state":"active",
[junit4] 1> "node_name":"127.0.0.1:57404__eg",
[junit4] 1> "core":"collection1"}}},
[junit4] 1> "shard2":{
[junit4] 1> "range":"0-7fffffff",
[junit4] 1> "state":"active",
[junit4] 1> "replicas":{
[junit4] 1> "core_node2":{
[junit4] 1> "base_url":"http://127.0.0.1:55399/_eg",
[junit4] 1> "state":"active",
[junit4] 1> "node_name":"127.0.0.1:55399__eg",
[junit4] 1> "core":"collection1",
[junit4] 1> "leader":"true"},
[junit4] 1> "core_node4":{
[junit4] 1> "base_url":"http://127.0.0.1:33603/_eg",
[junit4] 1> "state":"active",
[junit4] 1> "node_name":"127.0.0.1:33603__eg",
[junit4] 1> "core":"collection1"}}}},
[junit4] 1> "router":"compositeId"}}
[junit4] 1> /zookeeper (1)
[junit4] 1> DATA:
[junit4] 1>
[junit4] 1>
[junit4] 2> NOTE: reproduce with: ant test -Dtestcase=BasicDistributedZk2Test -Dtests.method=testDistribSearch -Dtests.seed=44B6599674515A8F -Dtests.multiplier=3 -Dtests.slow=true -Dtests.locale=en_NZ -Dtests.timezone=America/Eirunepe -Dtests.file.encoding=UTF-8
[junit4] ERROR 27.6s J1 | BasicDistributedZk2Test.testDistribSearch <<<
[junit4] > Throwable #1: org.apache.solr.client.solrj.impl.HttpSolrServer$RemoteSolrException: Server at http://127.0.0.1:55399/_eg/onenodecollectioncore returned non ok status:404, message:Can not find: /_eg/onenodecollectioncore/update
[junit4] > at __randomizedtesting.SeedInfo.seed([44B6599674515A8F:C550D78E030E3AB3]:0)
[junit4] > at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:385)
[junit4] > at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:180)
[junit4] > at org.apache.solr.client.solrj.request.AbstractUpdateRequest.process(AbstractUpdateRequest.java:117)
[junit4] > at org.apache.solr.client.solrj.SolrServer.add(SolrServer.java:116)
[junit4] > at org.apache.solr.client.solrj.SolrServer.add(SolrServer.java:102)
[junit4] > at org.apache.solr.cloud.BasicDistributedZk2Test.testNodeWithoutCollectionForwarding(BasicDistributedZk2Test.java:196)
[junit4] > at org.apache.solr.cloud.BasicDistributedZk2Test.doTest(BasicDistributedZk2Test.java:88)
[junit4] > at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:835)
[junit4] > at java.lang.Thread.run(Thread.java:780)
[junit4] 2> 655014 T2661 oas.SolrTestCaseJ4.deleteCore ###deleteCore
[junit4] 2> 27631 T2660 ccr.ThreadLeakControl.checkThreadLeaks WARNING Will linger awaiting termination of 4 leaked thread(s).
[junit4] 2> 655136 T2735 oascc.ZkStateReader$3.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4] 2> 655137 T2735 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
[junit4] 2> 655137 T2735 oascc.ZkStateReader$2.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4] 2> 656228 T2752 oascc.ZkStateReader$3.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4] 2> 656228 T2752 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
[junit4] 2> 656228 T2752 oascc.ZkStateReader$2.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4] 2> NOTE: test params are: codec=Lucene42: {}, docValues:{}, sim=RandomSimilarityProvider(queryNorm=false,coord=no): {}, locale=en_NZ, timezone=America/Eirunepe
[junit4] 2> NOTE: Linux 3.2.0-49-generic x86/IBM Corporation 1.7.0 (32-bit)/cpus=8,threads=2,free=10509800,total=239927296
[junit4] 2> NOTE: All tests run in this JVM: [TestRecoveryHdfs, TestBinaryField, AssignTest, TestMergePolicyConfig, TestHashPartitioner, StatsComponentTest, InfoHandlerTest, DOMUtilTest, TestDistributedGrouping, TestFoldingMultitermQuery, ClusterStateTest, CoreContainerCoreInitFailuresTest, TestAnalyzedSuggestions, TestDefaultSimilarityFactory, CollectionsAPIDistributedZkTest, SuggesterWFSTTest, FullSolrCloudDistribCmdsTest, TestWordDelimiterFilterFactory, TestAtomicUpdateErrorCases, EchoParamsTest, HdfsBasicDistributedZk2Test, HdfsRecoveryZkTest, TestCodecSupport, TestBinaryResponseWriter, TestCloudManagedSchema, LeaderElectionIntegrationTest, HdfsUnloadDistributedZkTest, TermVectorComponentDistributedTest, TestNumberUtils, ChaosMonkeySafeLeaderTest, IndexReaderFactoryTest, TestArbitraryIndexDir, TestCloudManagedSchemaAddField, TestAddFieldRealTimeGet, SignatureUpdateProcessorFactoryTest, NoCacheHeaderTest, TestLMJelinekMercerSimilarityFactory, URLClassifyProcessorTest, RequestHandlersTest, DateMathParserTest, DirectSolrSpellCheckerTest, TestDefaultSearchFieldResource, TestStressReorder, CoreAdminCreateDiscoverTest, TestLazyCores, SolrPluginUtilsTest, XmlUpdateRequestHandlerTest, SpellCheckCollatorTest, SampleTest, TestPartialUpdateDeduplication, RequiredFieldsTest, TestSuggestSpellingConverter, TestOmitPositions, TestDistributedSearch, DocValuesMultiTest, OpenExchangeRatesOrgProviderTest, TestFieldCollectionResource, TestSolr4Spatial, TestLuceneMatchVersion, HdfsSyncSliceTest, DisMaxRequestHandlerTest, ZkNodePropsTest, DocumentAnalysisRequestHandlerTest, FileUtilsTest, TestUpdate, CurrencyFieldXmlFileTest, SolrCmdDistributorTest, TestFastOutputStream, HighlighterTest, PingRequestHandlerTest, TestZkChroot, CachingDirectoryFactoryTest, ZkSolrClientTest, SortByFunctionTest, TestCollationField, BasicDistributedZk2Test]
[junit4] Completed on J1 in 28.93s, 1 test, 1 error <<< FAILURES!
[...truncated 538 lines...]
BUILD FAILED
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/build.xml:395: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/build.xml:388: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/build.xml:39: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build.xml:181: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/common-build.xml:449: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/lucene/common-build.xml:1250: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/lucene/common-build.xml:893: There were test failures: 319 suites, 1345 tests, 1 error, 35 ignored (7 assumptions)
Total time: 47 minutes 36 seconds
Build step 'Invoke Ant' marked build as failure
Description set: Java: 32bit/ibm-j9-jdk7 -Xjit:exclude={org/apache/lucene/util/fst/FST.pack(IIF)Lorg/apache/lucene/util/fst/FST;}
Archiving artifacts
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure