You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by Policeman Jenkins Server <je...@thetaphi.de> on 2013/06/04 12:43:20 UTC
[JENKINS] Lucene-Solr-4.x-Linux (32bit/ibm-j9-jdk6) - Build # 5914
- Failure!
Build: http://jenkins.thetaphi.de/job/Lucene-Solr-4.x-Linux/5914/
Java: 32bit/ibm-j9-jdk6 -Xjit:exclude={org/apache/lucene/util/fst/FST.pack(IIF)Lorg/apache/lucene/util/fst/FST;}
1 tests failed.
FAILED: org.apache.solr.cloud.BasicDistributedZk2Test.testDistribSearch
Error Message:
Server at http://127.0.0.1:37412/onenodecollectioncore returned non ok status:404, message:Can not find: /onenodecollectioncore/update
Stack Trace:
org.apache.solr.client.solrj.impl.HttpSolrServer$RemoteSolrException: Server at http://127.0.0.1:37412/onenodecollectioncore returned non ok status:404, message:Can not find: /onenodecollectioncore/update
at __randomizedtesting.SeedInfo.seed([7B13816205D9C92F:FAF50F7A7286A913]:0)
at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:385)
at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:180)
at org.apache.solr.client.solrj.request.AbstractUpdateRequest.process(AbstractUpdateRequest.java:117)
at org.apache.solr.client.solrj.SolrServer.add(SolrServer.java:116)
at org.apache.solr.client.solrj.SolrServer.add(SolrServer.java:102)
at org.apache.solr.cloud.BasicDistributedZk2Test.testNodeWithoutCollectionForwarding(BasicDistributedZk2Test.java:198)
at org.apache.solr.cloud.BasicDistributedZk2Test.doTest(BasicDistributedZk2Test.java:90)
at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:815)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:60)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:37)
at java.lang.reflect.Method.invoke(Method.java:611)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1559)
at com.carrotsearch.randomizedtesting.RandomizedRunner.access$600(RandomizedRunner.java:79)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:773)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:787)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:782)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:442)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:746)
at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:648)
at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:682)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:693)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
at java.lang.Thread.run(Thread.java:738)
Build Log:
[...truncated 9089 lines...]
[junit4:junit4] Suite: org.apache.solr.cloud.BasicDistributedZk2Test
[junit4:junit4] 2> 508076 T1655 oas.BaseDistributedSearchTestCase.initHostContext Setting hostContext system property: /
[junit4:junit4] 2> 508079 T1655 oas.SolrTestCaseJ4.setUp ###Starting testDistribSearch
[junit4:junit4] 2> Creating dataDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./solrtest-BasicDistributedZk2Test-1370341553455
[junit4:junit4] 2> 508080 T1655 oasc.ZkTestServer.run STARTING ZK TEST SERVER
[junit4:junit4] 2> 508081 T1656 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
[junit4:junit4] 2> 508181 T1655 oasc.ZkTestServer.run start zk server on port:56712
[junit4:junit4] 2> 508182 T1655 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 508618 T1662 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@53005300 name:ZooKeeperConnection Watcher:127.0.0.1:56712 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 508619 T1655 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 508620 T1655 oascc.SolrZkClient.makePath makePath: /solr
[junit4:junit4] 2> 508630 T1655 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 508632 T1664 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1d7b1d7b name:ZooKeeperConnection Watcher:127.0.0.1:56712/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 508633 T1655 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 508635 T1655 oascc.SolrZkClient.makePath makePath: /collections/collection1
[junit4:junit4] 2> 508643 T1655 oascc.SolrZkClient.makePath makePath: /collections/collection1/shards
[junit4:junit4] 2> 508652 T1655 oascc.SolrZkClient.makePath makePath: /collections/control_collection
[junit4:junit4] 2> 508659 T1655 oascc.SolrZkClient.makePath makePath: /collections/control_collection/shards
[junit4:junit4] 2> 508667 T1655 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
[junit4:junit4] 2> 508670 T1655 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.xml
[junit4:junit4] 2> 508680 T1655 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
[junit4:junit4] 2> 508681 T1655 oascc.SolrZkClient.makePath makePath: /configs/conf1/schema.xml
[junit4:junit4] 2> 508686 T1655 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
[junit4:junit4] 2> 508687 T1655 oascc.SolrZkClient.makePath makePath: /configs/conf1/stopwords.txt
[junit4:junit4] 2> 508692 T1655 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
[junit4:junit4] 2> 508693 T1655 oascc.SolrZkClient.makePath makePath: /configs/conf1/protwords.txt
[junit4:junit4] 2> 508697 T1655 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
[junit4:junit4] 2> 508698 T1655 oascc.SolrZkClient.makePath makePath: /configs/conf1/currency.xml
[junit4:junit4] 2> 508704 T1655 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
[junit4:junit4] 2> 508705 T1655 oascc.SolrZkClient.makePath makePath: /configs/conf1/open-exchange-rates.json
[junit4:junit4] 2> 508710 T1655 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4:junit4] 2> 508711 T1655 oascc.SolrZkClient.makePath makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4:junit4] 2> 508715 T1655 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
[junit4:junit4] 2> 508716 T1655 oascc.SolrZkClient.makePath makePath: /configs/conf1/old_synonyms.txt
[junit4:junit4] 2> 508721 T1655 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
[junit4:junit4] 2> 508722 T1655 oascc.SolrZkClient.makePath makePath: /configs/conf1/synonyms.txt
[junit4:junit4] 2> 508891 T1655 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4] 2> 508894 T1655 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:39584
[junit4:junit4] 2> 508894 T1655 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 508895 T1655 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 508896 T1655 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-controljetty-1370341554102
[junit4:junit4] 2> 508897 T1655 oasc.CoreContainer$Initializer.initialize looking for solr config file: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-controljetty-1370341554102/solr.xml
[junit4:junit4] 2> 508898 T1655 oasc.CoreContainer.<init> New CoreContainer 22282580
[junit4:junit4] 2> 508899 T1655 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-controljetty-1370341554102/'
[junit4:junit4] 2> 508899 T1655 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-controljetty-1370341554102/'
[junit4:junit4] 2> 508947 T1655 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 508947 T1655 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 508948 T1655 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 508949 T1655 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 508950 T1655 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 508950 T1655 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 508951 T1655 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 508952 T1655 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 508953 T1655 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 508954 T1655 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 508960 T1655 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4] 2> 508962 T1655 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:56712/solr
[junit4:junit4] 2> 508963 T1655 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 508964 T1655 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 508966 T1675 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@a400a4 name:ZooKeeperConnection Watcher:127.0.0.1:56712 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 508967 T1655 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 508970 T1655 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 508977 T1655 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 508978 T1677 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@64976497 name:ZooKeeperConnection Watcher:127.0.0.1:56712/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 508979 T1655 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 508982 T1655 oascc.SolrZkClient.makePath makePath: /overseer/queue
[junit4:junit4] 2> 508988 T1655 oascc.SolrZkClient.makePath makePath: /overseer/collection-queue-work
[junit4:junit4] 2> 508993 T1655 oascc.SolrZkClient.makePath makePath: /live_nodes
[junit4:junit4] 2> 508997 T1655 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:39584_
[junit4:junit4] 2> 508998 T1655 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:39584_
[junit4:junit4] 2> 509004 T1655 oascc.SolrZkClient.makePath makePath: /overseer_elect/election
[junit4:junit4] 2> 509012 T1655 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
[junit4:junit4] 2> 509016 T1655 oasc.Overseer.start Overseer (id=89806704047423491-127.0.0.1:39584_-n_0000000000) starting
[junit4:junit4] 2> 509021 T1655 oascc.SolrZkClient.makePath makePath: /overseer/queue-work
[junit4:junit4] 2> 509028 T1679 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
[junit4:junit4] 2> 509029 T1655 oascc.SolrZkClient.makePath makePath: /clusterstate.json
[junit4:junit4] 2> 509033 T1655 oascc.SolrZkClient.makePath makePath: /aliases.json
[junit4:junit4] 2> 509036 T1655 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 509041 T1678 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
[junit4:junit4] 2> 509043 T1680 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-controljetty-1370341554102/collection1
[junit4:junit4] 2> 509044 T1680 oasc.ZkController.createCollectionZkNode Check for collection zkNode:control_collection
[junit4:junit4] 2> 509045 T1680 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 509045 T1680 oasc.ZkController.readConfigName Load collection config from:/collections/control_collection
[junit4:junit4] 2> 509047 T1680 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-controljetty-1370341554102/collection1/'
[junit4:junit4] 2> 509049 T1680 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZk2Test-controljetty-1370341554102/collection1/lib/README' to classloader
[junit4:junit4] 2> 509049 T1680 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZk2Test-controljetty-1370341554102/collection1/lib/classes/' to classloader
[junit4:junit4] 2> 509083 T1680 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_44
[junit4:junit4] 2> 509127 T1680 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 509130 T1680 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 509137 T1680 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4:junit4] 2> 509522 T1680 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 2> 509527 T1680 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 509530 T1680 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 509549 T1680 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 509555 T1680 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 509560 T1680 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 509562 T1680 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 509563 T1680 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 509563 T1680 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 509565 T1680 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 509566 T1680 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 509566 T1680 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 509567 T1680 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-controljetty-1370341554102/collection1/, dataDir=./org.apache.solr.cloud.BasicDistributedZk2Test-1370341553455/control/data/
[junit4:junit4] 2> 509568 T1680 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@6a616a61
[junit4:junit4] 2> 509568 T1680 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 509593 T1680 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.BasicDistributedZk2Test-1370341553455/control/data
[junit4:junit4] 2> 509594 T1680 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.BasicDistributedZk2Test-1370341553455/control/data/index/
[junit4:junit4] 2> 509595 T1680 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.BasicDistributedZk2Test-1370341553455/control/data/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 509595 T1680 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.BasicDistributedZk2Test-1370341553455/control/data/index
[junit4:junit4] 2> 509599 T1680 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@18b018b0 lockFactory=org.apache.lucene.store.NativeFSLockFactory@12ec12ec),segFN=segments_1,generation=1}
[junit4:junit4] 2> 509600 T1680 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 509603 T1680 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 509604 T1680 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 509605 T1680 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 509606 T1680 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 509607 T1680 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 509608 T1680 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 509609 T1680 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 509610 T1680 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 509611 T1680 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 509613 T1680 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 509616 T1680 oass.SolrIndexSearcher.<init> Opening Searcher@5ffa5ffa main
[junit4:junit4] 2> 509616 T1680 oasu.UpdateHandler.clearLog Clearing tlog files, tlogDir=./org.apache.solr.cloud.BasicDistributedZk2Test-1370341553455/control/data/tlog
[junit4:junit4] 2> 509617 T1680 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 509618 T1680 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 509627 T1681 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@5ffa5ffa main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 509633 T1680 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 509634 T1680 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 510547 T1678 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 510549 T1678 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:39584",
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"1",
[junit4:junit4] 2> "node_name":"127.0.0.1:39584_",
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "collection":"control_collection",
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "core":"collection1"}
[junit4:junit4] 2> 510549 T1678 oasc.Overseer$ClusterStateUpdater.createCollection Create collection control_collection with numShards 1
[junit4:junit4] 2> 510550 T1678 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard=shard1
[junit4:junit4] 2> 510556 T1677 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 510636 T1680 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4] 2> 510636 T1680 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:39584 collection:control_collection shard:shard1
[junit4:junit4] 2> 510638 T1680 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leader_elect/shard1/election
[junit4:junit4] 2> 510650 T1680 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
[junit4:junit4] 2> 510654 T1680 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4] 2> 510654 T1680 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4] 2> 510655 T1680 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:39584/collection1/
[junit4:junit4] 2> 510655 T1680 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4] 2> 510656 T1680 oasc.SyncStrategy.syncToMe http://127.0.0.1:39584/collection1/ has no replicas
[junit4:junit4] 2> 510656 T1680 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:39584/collection1/
[junit4:junit4] 2> 510657 T1680 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leaders/shard1
[junit4:junit4] 2> 512065 T1678 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 512082 T1677 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 512138 T1680 oasc.ZkController.register We are http://127.0.0.1:39584/collection1/ and leader is http://127.0.0.1:39584/collection1/
[junit4:junit4] 2> 512139 T1680 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:39584
[junit4:junit4] 2> 512139 T1680 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4] 2> 512140 T1680 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4] 2> 512140 T1680 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 512143 T1680 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 512146 T1655 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0
[junit4:junit4] 2> 512146 T1655 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 512147 T1655 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 512153 T1655 oascsi.HttpClientUtil.createClient Creating new http client, config:
[junit4:junit4] 2> 512154 T1655 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 512156 T1684 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@55675567 name:ZooKeeperConnection Watcher:127.0.0.1:56712/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 512156 T1655 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 512158 T1655 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 512162 T1655 oasc.ChaosMonkey.monkeyLog monkey: init - expire sessions:true cause connection loss:true
[junit4:junit4] 2> 512293 T1655 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4] 2> 512295 T1655 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:35035
[junit4:junit4] 2> 512296 T1655 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 512296 T1655 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 512297 T1655 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty1-1370341557538
[junit4:junit4] 2> 512298 T1655 oasc.CoreContainer$Initializer.initialize looking for solr config file: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty1-1370341557538/solr.xml
[junit4:junit4] 2> 512299 T1655 oasc.CoreContainer.<init> New CoreContainer 1114391148
[junit4:junit4] 2> 512300 T1655 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty1-1370341557538/'
[junit4:junit4] 2> 512300 T1655 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty1-1370341557538/'
[junit4:junit4] 2> 512347 T1655 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 512348 T1655 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 512348 T1655 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 512349 T1655 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 512350 T1655 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 512350 T1655 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 512351 T1655 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 512352 T1655 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 512353 T1655 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 512353 T1655 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 512359 T1655 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4] 2> 512361 T1655 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:56712/solr
[junit4:junit4] 2> 512362 T1655 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 512363 T1655 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 512365 T1695 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@46a346a3 name:ZooKeeperConnection Watcher:127.0.0.1:56712 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 512365 T1655 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 512368 T1655 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 512375 T1655 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 512377 T1697 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@14801480 name:ZooKeeperConnection Watcher:127.0.0.1:56712/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 512377 T1655 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 512384 T1655 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 513390 T1655 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:35035_
[junit4:junit4] 2> 513392 T1655 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:35035_
[junit4:junit4] 2> 513406 T1684 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4] 2> 513406 T1677 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4] 2> 513407 T1697 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4] 2> 513407 T1677 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 513414 T1698 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty1-1370341557538/collection1
[junit4:junit4] 2> 513414 T1698 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4] 2> 513416 T1698 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 513416 T1698 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4] 2> 513418 T1698 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty1-1370341557538/collection1/'
[junit4:junit4] 2> 513420 T1698 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZk2Test-jetty1-1370341557538/collection1/lib/README' to classloader
[junit4:junit4] 2> 513420 T1698 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZk2Test-jetty1-1370341557538/collection1/lib/classes/' to classloader
[junit4:junit4] 2> 513450 T1698 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_44
[junit4:junit4] 2> 513484 T1698 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 513486 T1698 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 513491 T1698 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4:junit4] 2> 513597 T1678 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 513599 T1678 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:39584",
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"1",
[junit4:junit4] 2> "node_name":"127.0.0.1:39584_",
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "collection":"control_collection",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "core":"collection1"}
[junit4:junit4] 2> 513607 T1697 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 513607 T1684 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 513607 T1677 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 513882 T1698 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 2> 513888 T1698 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 513891 T1698 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 513909 T1698 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 513913 T1698 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 513918 T1698 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 513920 T1698 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 513921 T1698 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 513921 T1698 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 513923 T1698 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 513924 T1698 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 513924 T1698 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 513925 T1698 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty1-1370341557538/collection1/, dataDir=./org.apache.solr.cloud.BasicDistributedZk2Test-1370341553455/jetty1/
[junit4:junit4] 2> 513926 T1698 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@6a616a61
[junit4:junit4] 2> 513927 T1698 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 513928 T1698 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.BasicDistributedZk2Test-1370341553455/jetty1
[junit4:junit4] 2> 513929 T1698 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.BasicDistributedZk2Test-1370341553455/jetty1/index/
[junit4:junit4] 2> 513930 T1698 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.BasicDistributedZk2Test-1370341553455/jetty1/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 513931 T1698 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.BasicDistributedZk2Test-1370341553455/jetty1/index
[junit4:junit4] 2> 513934 T1698 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@cd80cd8 lockFactory=org.apache.lucene.store.NativeFSLockFactory@cff0cff),segFN=segments_1,generation=1}
[junit4:junit4] 2> 513935 T1698 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 513937 T1698 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 513938 T1698 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 513939 T1698 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 513940 T1698 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 513941 T1698 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 513941 T1698 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 513942 T1698 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 513943 T1698 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 513943 T1698 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 513945 T1698 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 513947 T1698 oass.SolrIndexSearcher.<init> Opening Searcher@12381238 main
[junit4:junit4] 2> 513948 T1698 oasu.UpdateHandler.clearLog Clearing tlog files, tlogDir=./org.apache.solr.cloud.BasicDistributedZk2Test-1370341553455/jetty1/tlog
[junit4:junit4] 2> 513949 T1698 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 513949 T1698 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 513958 T1699 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@12381238 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 513963 T1698 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 513964 T1698 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 515116 T1678 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 515118 T1678 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:35035",
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"2",
[junit4:junit4] 2> "node_name":"127.0.0.1:35035_",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "core":"collection1"}
[junit4:junit4] 2> 515118 T1678 oasc.Overseer$ClusterStateUpdater.createCollection Create collection collection1 with numShards 2
[junit4:junit4] 2> 515119 T1678 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard=shard1
[junit4:junit4] 2> 515126 T1684 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 515126 T1697 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 515126 T1677 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 515966 T1698 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4] 2> 515967 T1698 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:35035 collection:collection1 shard:shard1
[junit4:junit4] 2> 515968 T1698 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
[junit4:junit4] 2> 515979 T1698 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
[junit4:junit4] 2> 515983 T1698 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4] 2> 515984 T1698 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4] 2> 515984 T1698 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:35035/collection1/
[junit4:junit4] 2> 515985 T1698 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4] 2> 515985 T1698 oasc.SyncStrategy.syncToMe http://127.0.0.1:35035/collection1/ has no replicas
[junit4:junit4] 2> 515986 T1698 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:35035/collection1/
[junit4:junit4] 2> 515986 T1698 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
[junit4:junit4] 2> 516633 T1678 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 516646 T1684 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 516646 T1697 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 516646 T1677 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 516697 T1698 oasc.ZkController.register We are http://127.0.0.1:35035/collection1/ and leader is http://127.0.0.1:35035/collection1/
[junit4:junit4] 2> 516698 T1698 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:35035
[junit4:junit4] 2> 516699 T1698 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4] 2> 516699 T1698 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4] 2> 516700 T1698 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 516707 T1698 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 516709 T1655 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0
[junit4:junit4] 2> 516709 T1655 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 516710 T1655 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 516853 T1655 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4] 2> 516855 T1655 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:37412
[junit4:junit4] 2> 516855 T1655 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 516856 T1655 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 516857 T1655 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty2-1370341562091
[junit4:junit4] 2> 516858 T1655 oasc.CoreContainer$Initializer.initialize looking for solr config file: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty2-1370341562091/solr.xml
[junit4:junit4] 2> 516858 T1655 oasc.CoreContainer.<init> New CoreContainer 1475500018
[junit4:junit4] 2> 516859 T1655 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty2-1370341562091/'
[junit4:junit4] 2> 516860 T1655 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty2-1370341562091/'
[junit4:junit4] 2> 516905 T1655 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 516906 T1655 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 516906 T1655 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 516907 T1655 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 516908 T1655 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 516908 T1655 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 516909 T1655 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 516910 T1655 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 516911 T1655 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 516911 T1655 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 516917 T1655 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4] 2> 516919 T1655 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:56712/solr
[junit4:junit4] 2> 516920 T1655 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 516921 T1655 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 516923 T1711 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@3a4b3a4b name:ZooKeeperConnection Watcher:127.0.0.1:56712 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 516923 T1655 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 516926 T1655 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 516934 T1655 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 516935 T1713 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@3e993e99 name:ZooKeeperConnection Watcher:127.0.0.1:56712/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 516936 T1655 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 516943 T1655 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 517948 T1655 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:37412_
[junit4:junit4] 2> 517950 T1655 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:37412_
[junit4:junit4] 2> 517955 T1684 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4] 2> 517955 T1677 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4] 2> 517955 T1684 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 517956 T1713 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4] 2> 517956 T1677 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 517955 T1697 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4] 2> 517958 T1697 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 517964 T1714 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty2-1370341562091/collection1
[junit4:junit4] 2> 517965 T1714 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4] 2> 517966 T1714 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 517966 T1714 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4] 2> 517969 T1714 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty2-1370341562091/collection1/'
[junit4:junit4] 2> 517970 T1714 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZk2Test-jetty2-1370341562091/collection1/lib/README' to classloader
[junit4:junit4] 2> 517970 T1714 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZk2Test-jetty2-1370341562091/collection1/lib/classes/' to classloader
[junit4:junit4] 2> 518000 T1714 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_44
[junit4:junit4] 2> 518034 T1714 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 518037 T1714 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 518041 T1714 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4:junit4] 2> 518157 T1678 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 518159 T1678 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:35035",
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"2",
[junit4:junit4] 2> "node_name":"127.0.0.1:35035_",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "core":"collection1"}
[junit4:junit4] 2> 518165 T1684 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 518165 T1697 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 518165 T1677 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 518165 T1713 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 518435 T1714 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 2> 518441 T1714 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 518444 T1714 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 518466 T1714 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 518471 T1714 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 518476 T1714 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 518478 T1714 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 518478 T1714 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 518479 T1714 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 518481 T1714 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 518481 T1714 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 518482 T1714 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 518482 T1714 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty2-1370341562091/collection1/, dataDir=./org.apache.solr.cloud.BasicDistributedZk2Test-1370341553455/jetty2/
[junit4:junit4] 2> 518483 T1714 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@6a616a61
[junit4:junit4] 2> 518484 T1714 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 518485 T1714 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.BasicDistributedZk2Test-1370341553455/jetty2
[junit4:junit4] 2> 518485 T1714 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.BasicDistributedZk2Test-1370341553455/jetty2/index/
[junit4:junit4] 2> 518486 T1714 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.BasicDistributedZk2Test-1370341553455/jetty2/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 518486 T1714 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.BasicDistributedZk2Test-1370341553455/jetty2/index
[junit4:junit4] 2> 518488 T1714 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@54765476 lockFactory=org.apache.lucene.store.NativeFSLockFactory@30d230d2),segFN=segments_1,generation=1}
[junit4:junit4] 2> 518489 T1714 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 518492 T1714 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 518492 T1714 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 518493 T1714 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 518494 T1714 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 518495 T1714 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 518495 T1714 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 518496 T1714 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 518497 T1714 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 518498 T1714 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 518499 T1714 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 518501 T1714 oass.SolrIndexSearcher.<init> Opening Searcher@4bc24bc2 main
[junit4:junit4] 2> 518502 T1714 oasu.UpdateHandler.clearLog Clearing tlog files, tlogDir=./org.apache.solr.cloud.BasicDistributedZk2Test-1370341553455/jetty2/tlog
[junit4:junit4] 2> 518503 T1714 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 518503 T1714 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 518511 T1715 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@4bc24bc2 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 518515 T1714 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 518515 T1714 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 519674 T1678 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 519675 T1678 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:37412",
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"3",
[junit4:junit4] 2> "node_name":"127.0.0.1:37412_",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "core":"collection1"}
[junit4:junit4] 2> 519676 T1678 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=2
[junit4:junit4] 2> 519676 T1678 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard=shard2
[junit4:junit4] 2> 519685 T1684 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 519685 T1697 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 519685 T1677 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 519685 T1713 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 520517 T1714 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4] 2> 520518 T1714 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:37412 collection:collection1 shard:shard2
[junit4:junit4] 2> 520519 T1714 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard2/election
[junit4:junit4] 2> 520529 T1714 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
[junit4:junit4] 2> 520533 T1714 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4] 2> 520534 T1714 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4] 2> 520534 T1714 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:37412/collection1/
[junit4:junit4] 2> 520535 T1714 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4] 2> 520535 T1714 oasc.SyncStrategy.syncToMe http://127.0.0.1:37412/collection1/ has no replicas
[junit4:junit4] 2> 520536 T1714 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:37412/collection1/
[junit4:junit4] 2> 520536 T1714 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard2
[junit4:junit4] 2> 521193 T1678 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 521207 T1684 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 521207 T1713 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 521207 T1677 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 521208 T1697 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 521246 T1714 oasc.ZkController.register We are http://127.0.0.1:37412/collection1/ and leader is http://127.0.0.1:37412/collection1/
[junit4:junit4] 2> 521247 T1714 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:37412
[junit4:junit4] 2> 521247 T1714 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4] 2> 521248 T1714 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4] 2> 521248 T1714 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 521251 T1714 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 521252 T1655 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0
[junit4:junit4] 2> 521253 T1655 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 521254 T1655 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 521410 T1655 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4] 2> 521412 T1655 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:45714
[junit4:junit4] 2> 521413 T1655 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 521414 T1655 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 521415 T1655 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty3-1370341566635
[junit4:junit4] 2> 521415 T1655 oasc.CoreContainer$Initializer.initialize looking for solr config file: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty3-1370341566635/solr.xml
[junit4:junit4] 2> 521416 T1655 oasc.CoreContainer.<init> New CoreContainer 420026633
[junit4:junit4] 2> 521417 T1655 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty3-1370341566635/'
[junit4:junit4] 2> 521418 T1655 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty3-1370341566635/'
[junit4:junit4] 2> 521463 T1655 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 521463 T1655 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 521464 T1655 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 521465 T1655 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 521466 T1655 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 521466 T1655 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 521467 T1655 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 521468 T1655 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 521469 T1655 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 521469 T1655 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 521475 T1655 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4] 2> 521477 T1655 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:56712/solr
[junit4:junit4] 2> 521478 T1655 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 521479 T1655 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 521481 T1727 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@18ce18ce name:ZooKeeperConnection Watcher:127.0.0.1:56712 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 521481 T1655 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 521484 T1655 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 521490 T1655 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 521492 T1729 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@699e699e name:ZooKeeperConnection Watcher:127.0.0.1:56712/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 521492 T1655 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 521498 T1655 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 522504 T1655 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:45714_
[junit4:junit4] 2> 522507 T1655 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:45714_
[junit4:junit4] 2> 522511 T1684 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 522511 T1697 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 522512 T1684 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 522512 T1697 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 522511 T1729 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 522513 T1677 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 522515 T1677 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 522514 T1713 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 522519 T1713 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 522520 T1730 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty3-1370341566635/collection1
[junit4:junit4] 2> 522521 T1730 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4] 2> 522522 T1730 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 522523 T1730 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4] 2> 522525 T1730 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty3-1370341566635/collection1/'
[junit4:junit4] 2> 522526 T1730 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZk2Test-jetty3-1370341566635/collection1/lib/README' to classloader
[junit4:junit4] 2> 522527 T1730 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZk2Test-jetty3-1370341566635/collection1/lib/classes/' to classloader
[junit4:junit4] 2> 522556 T1730 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_44
[junit4:junit4] 2> 522590 T1730 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 522592 T1730 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 522597 T1730 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4:junit4] 2> 522720 T1678 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 522721 T1678 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:37412",
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"3",
[junit4:junit4] 2> "node_name":"127.0.0.1:37412_",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard":"shard2",
[junit4:junit4] 2> "core":"collection1"}
[junit4:junit4] 2> 522726 T1729 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 522727 T1713 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 522727 T1697 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 522727 T1684 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 522727 T1677 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 522981 T1730 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 2> 522986 T1730 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 522989 T1730 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 523008 T1730 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 523013 T1730 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 523018 T1730 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 523020 T1730 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 523021 T1730 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 523021 T1730 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 523023 T1730 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 523024 T1730 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 523024 T1730 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 523025 T1730 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty3-1370341566635/collection1/, dataDir=./org.apache.solr.cloud.BasicDistributedZk2Test-1370341553455/jetty3/
[junit4:junit4] 2> 523025 T1730 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@6a616a61
[junit4:junit4] 2> 523026 T1730 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 523027 T1730 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.BasicDistributedZk2Test-1370341553455/jetty3
[junit4:junit4] 2> 523028 T1730 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.BasicDistributedZk2Test-1370341553455/jetty3/index/
[junit4:junit4] 2> 523028 T1730 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.BasicDistributedZk2Test-1370341553455/jetty3/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 523029 T1730 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.BasicDistributedZk2Test-1370341553455/jetty3/index
[junit4:junit4] 2> 523031 T1730 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@1a7a1a7a lockFactory=org.apache.lucene.store.NativeFSLockFactory@13d113d1),segFN=segments_1,generation=1}
[junit4:junit4] 2> 523031 T1730 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 523034 T1730 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 523035 T1730 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 523036 T1730 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 523036 T1730 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 523037 T1730 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 523038 T1730 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 523039 T1730 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 523039 T1730 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 523040 T1730 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 523042 T1730 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 523044 T1730 oass.SolrIndexSearcher.<init> Opening Searcher@19021902 main
[junit4:junit4] 2> 523045 T1730 oasu.UpdateHandler.clearLog Clearing tlog files, tlogDir=./org.apache.solr.cloud.BasicDistributedZk2Test-1370341553455/jetty3/tlog
[junit4:junit4] 2> 523045 T1730 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 523046 T1730 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 523055 T1731 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@19021902 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 523059 T1730 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 523059 T1730 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 524237 T1678 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 524239 T1678 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:45714",
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"4",
[junit4:junit4] 2> "node_name":"127.0.0.1:45714_",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "core":"collection1"}
[junit4:junit4] 2> 524240 T1678 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=2
[junit4:junit4] 2> 524240 T1678 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard=shard1
[junit4:junit4] 2> 524249 T1697 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 524250 T1713 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 524250 T1677 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 524250 T1729 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 524249 T1684 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 525061 T1730 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4] 2> 525061 T1730 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:45714 collection:collection1 shard:shard1
[junit4:junit4] 2> 525066 T1730 oasc.ZkController.register We are http://127.0.0.1:45714/collection1/ and leader is http://127.0.0.1:35035/collection1/
[junit4:junit4] 2> 525067 T1730 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:45714
[junit4:junit4] 2> 525067 T1730 oasc.ZkController.checkRecovery Core needs to recover:collection1
[junit4:junit4] 2> 525068 T1730 oasu.DefaultSolrCoreState.doRecovery Running recovery - first canceling any ongoing recovery
[junit4:junit4] 2> ASYNC NEW_CORE C1891 name=collection1 org.apache.solr.core.SolrCore@7d317d31 url=http://127.0.0.1:45714/collection1 node=127.0.0.1:45714_ C1891_STATE=coll:collection1 core:collection1 props:{base_url=http://127.0.0.1:45714, state=down, node_name=127.0.0.1:45714_, collection=collection1, core=collection1}
[junit4:junit4] 2> 525068 T1732 C1891 P45714 oasc.RecoveryStrategy.run Starting recovery process. core=collection1 recoveringAfterStartup=true
[junit4:junit4] 2> 525069 T1732 C1891 P45714 oasc.RecoveryStrategy.doRecovery ###### startupVersions=[]
[junit4:junit4] 2> 525069 T1730 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 525070 T1732 C1891 P45714 oasc.ZkController.publish publishing core=collection1 state=recovering
[junit4:junit4] 2> 525071 T1732 C1891 P45714 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 525072 T1655 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0
[junit4:junit4] 2> 525074 T1655 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 525074 T1732 C1891 P45714 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 525075 T1655 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 525081 T1690 oasha.CoreAdminHandler.handleWaitForStateAction Going to wait for coreNodeName: 4, state: recovering, checkLive: true, onlyIfLeader: true
[junit4:junit4] 2> 525215 T1655 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4] 2> 525217 T1655 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:55099
[junit4:junit4] 2> 525217 T1655 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 525218 T1655 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 525219 T1655 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty4-1370341570455
[junit4:junit4] 2> 525220 T1655 oasc.CoreContainer$Initializer.initialize looking for solr config file: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty4-1370341570455/solr.xml
[junit4:junit4] 2> 525220 T1655 oasc.CoreContainer.<init> New CoreContainer 1868787555
[junit4:junit4] 2> 525221 T1655 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty4-1370341570455/'
[junit4:junit4] 2> 525222 T1655 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty4-1370341570455/'
[junit4:junit4] 2> 525266 T1655 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 525267 T1655 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 525267 T1655 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 525268 T1655 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 525269 T1655 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 525269 T1655 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 525270 T1655 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 525271 T1655 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 525272 T1655 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 525272 T1655 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 525278 T1655 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4] 2> 525280 T1655 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:56712/solr
[junit4:junit4] 2> 525281 T1655 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 525282 T1655 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 525284 T1744 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@13c613c6 name:ZooKeeperConnection Watcher:127.0.0.1:56712 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 525284 T1655 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 525287 T1655 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 525294 T1655 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 525296 T1746 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@7a637a63 name:ZooKeeperConnection Watcher:127.0.0.1:56712/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 525297 T1655 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 525303 T1655 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 525759 T1678 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 525760 T1678 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:45714",
[junit4:junit4] 2> "state":"recovering",
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"4",
[junit4:junit4] 2> "node_name":"127.0.0.1:45714_",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "core":"collection1"}
[junit4:junit4] 2> 525770 T1729 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 525770 T1713 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 525770 T1697 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 525772 T1684 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 525770 T1746 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 525771 T1677 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 526083 T1690 oasha.CoreAdminHandler.handleWaitForStateAction Waited coreNodeName: 4, state: recovering, checkLive: true, onlyIfLeader: true for: 1 seconds.
[junit4:junit4] 2> 526084 T1690 oass.SolrDispatchFilter.handleAdminRequest [admin] webapp=null path=/admin/cores params={state=recovering&onlyIfLeader=true&checkLive=true&action=PREPRECOVERY&version=2&coreNodeName=4&wt=javabin&nodeName=127.0.0.1:45714_&core=collection1} status=0 QTime=1003
[junit4:junit4] 2> 526309 T1655 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:55099_
[junit4:junit4] 2> 526311 T1655 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:55099_
[junit4:junit4] 2> 526315 T1729 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 526315 T1713 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 526316 T1746 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 526317 T1697 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 526318 T1684 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 526318 T1697 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 526318 T1677 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 526319 T1684 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 526320 T1713 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 526321 T1729 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 526320 T1677 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 526363 T1746 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 526368 T1747 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty4-1370341570455/collection1
[junit4:junit4] 2> 526369 T1747 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4] 2> 526371 T1747 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 526372 T1747 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4] 2> 526374 T1747 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty4-1370341570455/collection1/'
[junit4:junit4] 2> 526376 T1747 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZk2Test-jetty4-1370341570455/collection1/lib/README' to classloader
[junit4:junit4] 2> 526377 T1747 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZk2Test-jetty4-1370341570455/collection1/lib/classes/' to classloader
[junit4:junit4] 2> 526421 T1747 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_44
[junit4:junit4] 2> 526456 T1747 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 526460 T1747 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 526466 T1747 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4:junit4] 2> 526878 T1747 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 2> 526885 T1747 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 526888 T1747 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 526907 T1747 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 526912 T1747 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 526948 T1747 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 526951 T1747 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 526952 T1747 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 526952 T1747 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 526954 T1747 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 526955 T1747 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 526956 T1747 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 526956 T1747 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty4-1370341570455/collection1/, dataDir=./org.apache.solr.cloud.BasicDistributedZk2Test-1370341553455/jetty4/
[junit4:junit4] 2> 526957 T1747 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@6a616a61
[junit4:junit4] 2> 526958 T1747 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 526959 T1747 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.BasicDistributedZk2Test-1370341553455/jetty4
[junit4:junit4] 2> 526959 T1747 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.BasicDistributedZk2Test-1370341553455/jetty4/index/
[junit4:junit4] 2> 526960 T1747 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.BasicDistributedZk2Test-1370341553455/jetty4/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 526961 T1747 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.BasicDistributedZk2Test-1370341553455/jetty4/index
[junit4:junit4] 2> 526963 T1747 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@2d042d04 lockFactory=org.apache.lucene.store.NativeFSLockFactory@5d165d16),segFN=segments_1,generation=1}
[junit4:junit4] 2> 526964 T1747 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 526966 T1747 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 526967 T1747 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 526968 T1747 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 526969 T1747 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 526970 T1747 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 526970 T1747 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 526971 T1747 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 526972 T1747 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 526972 T1747 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 526974 T1747 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 526976 T1747 oass.SolrIndexSearcher.<init> Opening Searcher@5d385d38 main
[junit4:junit4] 2> 526977 T1747 oasu.UpdateHandler.clearLog Clearing tlog files, tlogDir=./org.apache.solr.cloud.BasicDistributedZk2Test-1370341553455/jetty4/tlog
[junit4:junit4] 2> 526978 T1747 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 526978 T1747 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 526985 T1748 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@5d385d38 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 526989 T1747 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 526989 T1747 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 527283 T1678 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 527284 T1678 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:55099",
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"5",
[junit4:junit4] 2> "node_name":"127.0.0.1:55099_",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "core":"collection1"}
[junit4:junit4] 2> 527284 T1678 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=2
[junit4:junit4] 2> 527285 T1678 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard=shard2
[junit4:junit4] 2> 527291 T1729 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 527291 T1713 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 527291 T1746 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 527291 T1684 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 527291 T1697 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 527291 T1677 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 527992 T1747 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4] 2> 527992 T1747 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:55099 collection:collection1 shard:shard2
[junit4:junit4] 2> 527998 T1747 oasc.ZkController.register We are http://127.0.0.1:55099/collection1/ and leader is http://127.0.0.1:37412/collection1/
[junit4:junit4] 2> 527999 T1747 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:55099
[junit4:junit4] 2> 527999 T1747 oasc.ZkController.checkRecovery Core needs to recover:collection1
[junit4:junit4] 2> 528000 T1747 oasu.DefaultSolrCoreState.doRecovery Running recovery - first canceling any ongoing recovery
[junit4:junit4] 2> ASYNC NEW_CORE C1892 name=collection1 org.apache.solr.core.SolrCore@466a466a url=http://127.0.0.1:55099/collection1 node=127.0.0.1:55099_ C1892_STATE=coll:collection1 core:collection1 props:{base_url=http://127.0.0.1:55099, state=down, node_name=127.0.0.1:55099_, collection=collection1, core=collection1}
[junit4:junit4] 2> 528001 T1749 C1892 P55099 oasc.RecoveryStrategy.run Starting recovery process. core=collection1 recoveringAfterStartup=true
[junit4:junit4] 2> 528001 T1749 C1892 P55099 oasc.RecoveryStrategy.doRecovery ###### startupVersions=[]
[junit4:junit4] 2> 528001 T1747 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 528002 T1749 C1892 P55099 oasc.ZkController.publish publishing core=collection1 state=recovering
[junit4:junit4] 2> 528003 T1749 C1892 P55099 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 528004 T1655 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0
[junit4:junit4] 2> 528005 T1749 C1892 P55099 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 528005 T1655 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 528007 T1655 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 528015 T1708 oasha.CoreAdminHandler.handleWaitForStateAction Going to wait for coreNodeName: 5, state: recovering, checkLive: true, onlyIfLeader: true
[junit4:junit4] 2> 528016 T1655 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 528018 T1655 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 528028 T1691 oasc.CoreContainer.create Creating SolrCore 'onenodecollectioncore' using instanceDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty1-1370341557538/onenodecollectioncore
[junit4:junit4] 2> 528029 T1691 oasc.ZkController.createCollectionZkNode Check for collection zkNode:onenodecollection
[junit4:junit4] 2> 528031 T1691 oasc.ZkController.createCollectionZkNode Creating collection in ZooKeeper:onenodecollection
[junit4:junit4] 2> 528032 T1691 oasc.ZkController.getConfName Looking for collection configName
[junit4:junit4] 2> 528035 T1691 oasc.ZkController.getConfName Only one config set found in zk - using it:conf1
[junit4:junit4] 2> 528036 T1691 oascc.SolrZkClient.makePath makePath: /collections/onenodecollection
[junit4:junit4] 2> 528041 T1691 oasc.ZkController.readConfigName Load collection config from:/collections/onenodecollection
[junit4:junit4] 2> 528043 T1691 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty1-1370341557538/onenodecollectioncore/'
[junit4:junit4] 2> 528085 T1691 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_44
[junit4:junit4] 2> ASYNC NEW_CORE C1893 name=collection1 org.apache.solr.core.SolrCore@7d317d31 url=http://127.0.0.1:45714/collection1 node=127.0.0.1:45714_ C1893_STATE=coll:collection1 core:collection1 props:{base_url=http://127.0.0.1:45714, state=recovering, node_name=127.0.0.1:45714_, collection=collection1, shard=shard1, core=collection1}
[junit4:junit4] 2> 528085 T1732 C1893 P45714 oasc.RecoveryStrategy.doRecovery Attempting to PeerSync from http://127.0.0.1:35035/collection1/ core=collection1 - recoveringAfterStartup=true
[junit4:junit4] 2> 528086 T1732 C1893 P45714 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:45714 START replicas=[http://127.0.0.1:35035/collection1/] nUpdates=100
[junit4:junit4] 2> 528087 T1732 C1893 P45714 oasu.PeerSync.sync WARN no frame of reference to tell of we've missed updates
[junit4:junit4] 2> 528088 T1732 C1893 P45714 oasc.RecoveryStrategy.doRecovery PeerSync Recovery was not successful - trying replication. core=collection1
[junit4:junit4] 2> 528088 T1732 C1893 P45714 oasc.RecoveryStrategy.doRecovery Starting Replication Recovery. core=collection1
[junit4:junit4] 2> 528089 T1732 C1893 P45714 oasc.RecoveryStrategy.doRecovery Begin buffering updates. core=collection1
[junit4:junit4] 2> 528090 T1732 C1893 P45714 oasu.UpdateLog.bufferUpdates Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null}
[junit4:junit4] 2> 528090 T1732 C1893 P45714 oasc.RecoveryStrategy.replicate Attempting to replicate from http://127.0.0.1:35035/collection1/. core=collection1
[junit4:junit4] 2> 528091 T1732 C1893 P45714 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> ASYNC NEW_CORE C1894 name=collection1 org.apache.solr.core.SolrCore@76677667 url=http://127.0.0.1:35035/collection1 node=127.0.0.1:35035_ C1894_STATE=coll:collection1 core:collection1 props:{base_url=http://127.0.0.1:35035, state=active, node_name=127.0.0.1:35035_, collection=collection1, shard=shard1, core=collection1, leader=true}
[junit4:junit4] 2> 528093 T1690 C1894 P35035 oasc.SolrCore.execute [collection1] webapp= path=/get params={getVersions=100&qt=/get&distrib=false&version=2&wt=javabin} status=0 QTime=0
[junit4:junit4] 2> 528106 T1692 C1894 P35035 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 528108 T1692 C1894 P35035 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@cd80cd8 lockFactory=org.apache.lucene.store.NativeFSLockFactory@cff0cff),segFN=segments_1,generation=1}
[junit4:junit4] 2> 528109 T1692 C1894 P35035 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 528110 T1692 C1894 P35035 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@cd80cd8 lockFactory=org.apache.lucene.store.NativeFSLockFactory@cff0cff),segFN=segments_1,generation=1}
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@cd80cd8 lockFactory=org.apache.lucene.store.NativeFSLockFactory@cff0cff),segFN=segments_2,generation=2}
[junit4:junit4] 2> 528111 T1692 C1894 P35035 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 2
[junit4:junit4] 2> 528112 T1692 C1894 P35035 oass.SolrIndexSearcher.<init> Opening Searcher@1a031a03 realtime
[junit4:junit4] 2> 528113 T1692 C1894 P35035 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 528114 T1692 C1894 P35035 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={softCommit=false&commit_end_point=true&commit=true&version=2&waitSearcher=true&wt=javabin&openSearcher=false} {commit=} 0 8
[junit4:junit4] 2> 528116 T1732 C1893 P45714 oash.ReplicationHandler.inform Commits will be reserved for 10000
[junit4:junit4] 2> 528116 T1732 C1893 P45714 oash.SnapPuller.<init> No value set for 'pollInterval'. Timer Task not started.
[junit4:junit4] 2> 528119 T1690 C1894 P35035 oash.ReplicationHandler.inform Commits will be reserved for 10000
[junit4:junit4] 2> 528120 T1690 C1894 P35035 oasc.SolrCore.execute [collection1] webapp= path=/replication params={qt=/replication&version=2&command=indexversion&wt=javabin} status=0 QTime=2
[junit4:junit4] 2> 528121 T1732 C1893 P45714 oash.SnapPuller.fetchLatestIndex Master's generation: 2
[junit4:junit4] 2> 528121 T1732 C1893 P45714 oash.SnapPuller.fetchLatestIndex Slave's generation: 1
[junit4:junit4] 2> 528122 T1732 C1893 P45714 oash.SnapPuller.fetchLatestIndex Starting replication process
[junit4:junit4] 2> 528124 T1692 C1894 P35035 oasc.SolrCore.execute [collection1] webapp= path=/replication params={qt=/replication&version=2&generation=2&command=filelist&wt=javabin} status=0 QTime=0
[junit4:junit4] 2> ASYNC NEW_CORE C1895 name=collection1 org.apache.solr.core.SolrCore@7d317d31 url=http://127.0.0.1:45714/collection1 node=127.0.0.1:45714_ C1895_STATE=coll:collection1 core:collection1 props:{base_url=http://127.0.0.1:45714, state=recovering, node_name=127.0.0.1:45714_, collection=collection1, shard=shard1, core=collection1}
[junit4:junit4] 2> 528137 T1732 C1895 P45714 oash.SnapPuller.fetchLatestIndex Number of files in latest index in master: 1
[junit4:junit4] 2> 528138 T1732 C1895 P45714 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.BasicDistributedZk2Test-1370341553455/jetty3/index.20130604052613513
[junit4:junit4] 2> 528139 T1732 C1895 P45714 oash.SnapPuller.fetchLatestIndex Starting download to MockDirWrapper(org.apache.lucene.store.RAMDirectory@4af44af4 lockFactory=org.apache.lucene.store.NativeFSLockFactory@6a886a88) fullCopy=false
[junit4:junit4] 2> ASYNC NEW_CORE C1896 name=collection1 org.apache.solr.core.SolrCore@76677667 url=http://127.0.0.1:35035/collection1 node=127.0.0.1:35035_ C1896_STATE=coll:collection1 core:collection1 props:{base_url=http://127.0.0.1:35035, state=active, node_name=127.0.0.1:35035_, collection=collection1, shard=shard1, core=collection1, leader=true}
[junit4:junit4] 2> 528142 T1693 C1896 P35035 oasc.SolrCore.execute [collection1] webapp= path=/replication params={checksum=true&qt=/replication&generation=2&command=filecontent&file=segments_2&wt=filestream} status=0 QTime=0
[junit4:junit4] 2> 528144 T1732 C1895 P45714 oash.SnapPuller.fetchLatestIndex Total time taken for download : 0 secs
[junit4:junit4] 2> 528145 T1732 C1895 P45714 oasu.DefaultSolrCoreState.newIndexWriter Creating new IndexWriter...
[junit4:junit4] 2> 528146 T1732 C1895 P45714 oasu.DefaultSolrCoreState.newIndexWriter Waiting until IndexWriter is unused... core=collection1
[junit4:junit4] 2> 528147 T1732 C1895 P45714 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@1a7a1a7a lockFactory=org.apache.lucene.store.NativeFSLockFactory@13d113d1),segFN=segments_1,generation=1}
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@1a7a1a7a lockFactory=org.apache.lucene.store.NativeFSLockFactory@13d113d1),segFN=segments_2,generation=2}
[junit4:junit4] 2> 528148 T1732 C1895 P45714 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 2
[junit4:junit4] 2> 528149 T1732 C1895 P45714 oasu.DefaultSolrCoreState.newIndexWriter New IndexWriter is ready to be used.
[junit4:junit4] 2> 528149 T1732 C1895 P45714 oass.SolrIndexSearcher.<init> Opening Searcher@2cf22cf2 main
[junit4:junit4] 2> 528151 T1691 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 528151 T1731 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@2cf22cf2 main{StandardDirectoryReader(segments_2:1:nrt)}
[junit4:junit4] 2> 528154 T1732 C1895 P45714 oasc.CachingDirectoryFactory.closeCacheValue looking to close ./org.apache.solr.cloud.BasicDistributedZk2Test-1370341553455/jetty3/index.20130604052613513 [CachedDir<<refCount=0;path=./org.apache.solr.cloud.BasicDistributedZk2Test-1370341553455/jetty3/index.20130604052613513;done=true>>]
[junit4:junit4] 2> 528154 T1732 C1895 P45714 oasc.CachingDirectoryFactory.close Closing directory: ./org.apache.solr.cloud.BasicDistributedZk2Test-1370341553455/jetty3/index.20130604052613513
[junit4:junit4] 2> 528155 T1732 C1895 P45714 oasc.CachingDirectoryFactory.closeCacheValue Removing directory before core close: ./org.apache.solr.cloud.BasicDistributedZk2Test-1370341553455/jetty3/index.20130604052613513
[junit4:junit4] 2> 528155 T1732 C1895 P45714 oasc.RecoveryStrategy.replay No replay needed. core=collection1
[junit4:junit4] 2> 528156 T1732 C1895 P45714 oasc.RecoveryStrategy.doRecovery Replication Recovery was successful - registering as Active. core=collection1
[junit4:junit4] 2> 528157 T1732 C1895 P45714 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4] 2> 528157 T1732 C1895 P45714 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 528158 T1691 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 528159 T1732 C1895 P45714 oasc.RecoveryStrategy.doRecovery Finished recovery process. core=collection1
[junit4:junit4] 2> 528165 T1691 oass.IndexSchema.readSchema [onenodecollectioncore] Schema name=test
[junit4:junit4] 2> 528630 T1691 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 2> 528636 T1691 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 528639 T1691 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 528671 T1691 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 528677 T1691 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 528682 T1691 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 528685 T1691 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 528686 T1691 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 528687 T1691 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 528689 T1691 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 528690 T1691 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 528691 T1691 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 528691 T1691 oasc.SolrCore.<init> [onenodecollectioncore] Opening new SolrCore at /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty1-1370341557538/onenodecollectioncore/, dataDir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./solrtest-BasicDistributedZk2Test-1370341553455/onenodecollection/
[junit4:junit4] 2> 528692 T1691 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@6a616a61
[junit4:junit4] 2> 528693 T1691 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 528695 T1691 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./solrtest-BasicDistributedZk2Test-1370341553455/onenodecollection
[junit4:junit4] 2> 528696 T1691 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./solrtest-BasicDistributedZk2Test-1370341553455/onenodecollection/index/
[junit4:junit4] 2> 528697 T1691 oasc.SolrCore.initIndex WARN [onenodecollectioncore] Solr index directory '/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./solrtest-BasicDistributedZk2Test-1370341553455/onenodecollection/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 528698 T1691 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./solrtest-BasicDistributedZk2Test-1370341553455/onenodecollection/index
[junit4:junit4] 2> 528701 T1691 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@47c647c6 lockFactory=org.apache.lucene.store.NativeFSLockFactory@6e1a6e1a),segFN=segments_1,generation=1}
[junit4:junit4] 2> 528702 T1691 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 528705 T1691 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 528706 T1691 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 528707 T1691 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 528708 T1691 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 528709 T1691 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 528710 T1691 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 528711 T1691 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 528712 T1691 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 528713 T1691 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 528715 T1691 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 528718 T1691 oass.SolrIndexSearcher.<init> Opening Searcher@146d146d main
[junit4:junit4] 2> 528719 T1691 oasu.UpdateHandler.clearLog Clearing tlog files, tlogDir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./solrtest-BasicDistributedZk2Test-1370341553455/onenodecollection/tlog
[junit4:junit4] 2> 528720 T1691 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 528721 T1691 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 528729 T1753 oasc.SolrCore.registerSearcher [onenodecollectioncore] Registered new searcher Searcher@146d146d main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 528734 T1691 oasc.ZkController.publish publishing core=onenodecollectioncore state=down
[junit4:junit4] 2> 528800 T1678 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 528802 T1678 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:55099",
[junit4:junit4] 2> "state":"recovering",
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"5",
[junit4:junit4] 2> "node_name":"127.0.0.1:55099_",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard":"shard2",
[junit4:junit4] 2> "core":"collection1"}
[junit4:junit4] 2> 528814 T1678 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:45714",
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"4",
[junit4:junit4] 2> "node_name":"127.0.0.1:45714_",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "core":"collection1"}
[junit4:junit4] 2> 528821 T1678 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:35035",
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:35035_",
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "roles":"none",
[junit4:junit4] 2> "collection":"onenodecollection",
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "core":"onenodecollectioncore"}
[junit4:junit4] 2> 528822 T1678 oasc.Overseer$ClusterStateUpdater.createCollection Create collection onenodecollection with numShards 1
[junit4:junit4] 2> 528823 T1678 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard=shard1
[junit4:junit4] 2> 528831 T1697 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 528831 T1729 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 528831 T1746 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 528831 T1684 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 528831 T1713 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 528831 T1677 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 529016 T1708 oasha.CoreAdminHandler.handleWaitForStateAction Waited coreNodeName: 5, state: recovering, checkLive: true, onlyIfLeader: true for: 1 seconds.
[junit4:junit4] 2> 529017 T1708 oass.SolrDispatchFilter.handleAdminRequest [admin] webapp=null path=/admin/cores params={state=recovering&onlyIfLeader=true&checkLive=true&action=PREPRECOVERY&version=2&coreNodeName=5&wt=javabin&nodeName=127.0.0.1:55099_&core=collection1} status=0 QTime=1003
[junit4:junit4] 2> 529750 T1691 oasc.CoreContainer.registerCore registering core: onenodecollectioncore
[junit4:junit4] 2> 529752 T1691 oasc.ZkController.register Register replica - core:onenodecollectioncore address:http://127.0.0.1:35035 collection:onenodecollection shard:shard1
[junit4:junit4] 2> 529755 T1691 oascc.SolrZkClient.makePath makePath: /collections/onenodecollection/leader_elect/shard1/election
[junit4:junit4] 2> 529770 T1691 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
[junit4:junit4] 2> 529775 T1691 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4] 2> 529777 T1691 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4] 2> 529779 T1691 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:35035/onenodecollectioncore/
[junit4:junit4] 2> 529780 T1691 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4] 2> 529781 T1691 oasc.SyncStrategy.syncToMe http://127.0.0.1:35035/onenodecollectioncore/ has no replicas
[junit4:junit4] 2> 529783 T1691 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:35035/onenodecollectioncore/
[junit4:junit4] 2> 529784 T1691 oascc.SolrZkClient.makePath makePath: /collections/onenodecollection/leaders/shard1
[junit4:junit4] 2> 530346 T1678 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 530358 T1697 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 530358 T1713 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 530359 T1684 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 530358 T1729 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 530358 T1746 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 530358 T1677 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 530398 T1691 oasc.ZkController.register We are http://127.0.0.1:35035/onenodecollectioncore/ and leader is http://127.0.0.1:35035/onenodecollectioncore/
[junit4:junit4] 2> 530399 T1691 oasc.ZkController.register No LogReplay needed for core=onenodecollectioncore baseURL=http://127.0.0.1:35035
[junit4:junit4] 2> 530400 T1691 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4] 2> 530401 T1691 oasc.ZkController.publish publishing core=onenodecollectioncore state=active
[junit4:junit4] 2> 530404 T1691 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 530406 T1691 oass.SolrDispatchFilter.handleAdminRequest [admin] webapp=null path=/admin/cores params={dataDir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./solrtest-BasicDistributedZk2Test-1370341553455/onenodecollection&action=CREATE&version=2&numShards=1&name=onenodecollectioncore&wt=javabin&roles=none&collection=onenodecollection} status=0 QTime=2378
[junit4:junit4] 2> 530407 T1655 oasc.AbstractDistribZkTestBase.waitForRecoveriesToFinish Wait for recoveries to finish - collection: onenodecollection failOnTimeout:true timeout (sec):230
[junit4:junit4] 2> 530408 T1655 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> ASYNC NEW_CORE C1897 name=collection1 org.apache.solr.core.SolrCore@466a466a url=http://127.0.0.1:55099/collection1 node=127.0.0.1:55099_ C1897_STATE=coll:collection1 core:collection1 props:{base_url=http://127.0.0.1:55099, state=recovering, node_name=127.0.0.1:55099_, collection=collection1, shard=shard2, core=collection1}
[junit4:junit4] 2> 531018 T1749 C1897 P55099 oasc.RecoveryStrategy.doRecovery Attempting to PeerSync from http://127.0.0.1:37412/collection1/ core=collection1 - recoveringAfterStartup=true
[junit4:junit4] 2> 531019 T1749 C1897 P55099 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:55099 START replicas=[http://127.0.0.1:37412/collection1/] nUpdates=100
[junit4:junit4] 2> 531019 T1749 C1897 P55099 oasu.PeerSync.sync WARN no frame of reference to tell of we've missed updates
[junit4:junit4] 2> 531020 T1749 C1897 P55099 oasc.RecoveryStrategy.doRecovery PeerSync Recovery was not successful - trying replication. core=collection1
[junit4:junit4] 2> 531020 T1749 C1897 P55099 oasc.RecoveryStrategy.doRecovery Starting Replication Recovery. core=collection1
[junit4:junit4] 2> 531021 T1749 C1897 P55099 oasc.RecoveryStrategy.doRecovery Begin buffering updates. core=collection1
[junit4:junit4] 2> 531021 T1749 C1897 P55099 oasu.UpdateLog.bufferUpdates Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null}
[junit4:junit4] 2> 531022 T1749 C1897 P55099 oasc.RecoveryStrategy.replicate Attempting to replicate from http://127.0.0.1:37412/collection1/. core=collection1
[junit4:junit4] 2> 531022 T1749 C1897 P55099 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> ASYNC NEW_CORE C1898 name=collection1 org.apache.solr.core.SolrCore@15041504 url=http://127.0.0.1:37412/collection1 node=127.0.0.1:37412_ C1898_STATE=coll:collection1 core:collection1 props:{base_url=http://127.0.0.1:37412, state=active, node_name=127.0.0.1:37412_, collection=collection1, shard=shard2, core=collection1, leader=true}
[junit4:junit4] 2> 531026 T1708 C1898 P37412 oasc.SolrCore.execute [collection1] webapp= path=/get params={getVersions=100&qt=/get&distrib=false&version=2&wt=javabin} status=0 QTime=0
[junit4:junit4] 2> 531029 T1706 C1898 P37412 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 531032 T1706 C1898 P37412 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@54765476 lockFactory=org.apache.lucene.store.NativeFSLockFactory@30d230d2),segFN=segments_1,generation=1}
[junit4:junit4] 2> 531034 T1706 C1898 P37412 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 531035 T1706 C1898 P37412 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@54765476 lockFactory=org.apache.lucene.store.NativeFSLockFactory@30d230d2),segFN=segments_1,generation=1}
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@54765476 lockFactory=org.apache.lucene.store.NativeFSLockFactory@30d230d2),segFN=segments_2,generation=2}
[junit4:junit4] 2> 531037 T1706 C1898 P37412 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 2
[junit4:junit4] 2> 531038 T1706 C1898 P37412 oass.SolrIndexSearcher.<init> Opening Searcher@26e226e2 realtime
[junit4:junit4] 2> 531040 T1706 C1898 P37412 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 531041 T1706 C1898 P37412 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={softCommit=false&commit_end_point=true&commit=true&version=2&waitSearcher=true&wt=javabin&openSearcher=false} {commit=} 0 12
[junit4:junit4] 2> 531042 T1749 C1897 P55099 oash.ReplicationHandler.inform Commits will be reserved for 10000
[junit4:junit4] 2> 531043 T1749 C1897 P55099 oash.SnapPuller.<init> No value set for 'pollInterval'. Timer Task not started.
[junit4:junit4] 2> 531045 T1709 C1898 P37412 oash.ReplicationHandler.inform Commits will be reserved for 10000
[junit4:junit4] 2> 531046 T1709 C1898 P37412 oasc.SolrCore.execute [collection1] webapp= path=/replication params={qt=/replication&version=2&command=indexversion&wt=javabin} status=0 QTime=1
[junit4:junit4] 2> 531047 T1749 C1897 P55099 oash.SnapPuller.fetchLatestIndex Master's generation: 2
[junit4:junit4] 2> 531047 T1749 C1897 P55099 oash.SnapPuller.fetchLatestIndex Slave's generation: 1
[junit4:junit4] 2> 531048 T1749 C1897 P55099 oash.SnapPuller.fetchLatestIndex Starting replication process
[junit4:junit4] 2> 531050 T1708 C1898 P37412 oasc.SolrCore.execute [collection1] webapp= path=/replication params={qt=/replication&version=2&generation=2&command=filelist&wt=javabin} status=0 QTime=0
[junit4:junit4] 2> 531051 T1749 C1897 P55099 oash.SnapPuller.fetchLatestIndex Number of files in latest index in master: 1
[junit4:junit4] 2> 531052 T1749 C1897 P55099 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.BasicDistributedZk2Test-1370341553455/jetty4/index.20130604052616427
[junit4:junit4] 2> 531052 T1749 C1897 P55099 oash.SnapPuller.fetchLatestIndex Starting download to MockDirWrapper(org.apache.lucene.store.RAMDirectory@1e261e26 lockFactory=org.apache.lucene.store.NativeFSLockFactory@54145414) fullCopy=false
[junit4:junit4] 2> 531054 T1706 C1898 P37412 oasc.SolrCore.execute [collection1] webapp= path=/replication params={checksum=true&qt=/replication&generation=2&command=filecontent&file=segments_2&wt=filestream} status=0 QTime=0
[junit4:junit4] 2> 531056 T1749 C1897 P55099 oash.SnapPuller.fetchLatestIndex Total time taken for download : 0 secs
[junit4:junit4] 2> 531057 T1749 C1897 P55099 oasu.DefaultSolrCoreState.newIndexWriter Creating new IndexWriter...
[junit4:junit4] 2> 531058 T1749 C1897 P55099 oasu.DefaultSolrCoreState.newIndexWriter Waiting until IndexWriter is unused... core=collection1
[junit4:junit4] 2> 531059 T1749 C1897 P55099 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@2d042d04 lockFactory=org.apache.lucene.store.NativeFSLockFactory@5d165d16),segFN=segments_1,generation=1}
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@2d042d04 lockFactory=org.apache.lucene.store.NativeFSLockFactory@5d165d16),segFN=segments_2,generation=2}
[junit4:junit4] 2> 531060 T1749 C1897 P55099 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 2
[junit4:junit4] 2> 531061 T1749 C1897 P55099 oasu.DefaultSolrCoreState.newIndexWriter New IndexWriter is ready to be used.
[junit4:junit4] 2> 531062 T1749 C1897 P55099 oass.SolrIndexSearcher.<init> Opening Searcher@22562256 main
[junit4:junit4] 2> 531063 T1748 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@22562256 main{StandardDirectoryReader(segments_2:1:nrt)}
[junit4:junit4] 2> 531064 T1749 C1897 P55099 oasc.CachingDirectoryFactory.closeCacheValue looking to close ./org.apache.solr.cloud.BasicDistributedZk2Test-1370341553455/jetty4/index.20130604052616427 [CachedDir<<refCount=0;path=./org.apache.solr.cloud.BasicDistributedZk2Test-1370341553455/jetty4/index.20130604052616427;done=true>>]
[junit4:junit4] 2> 531065 T1749 C1897 P55099 oasc.CachingDirectoryFactory.close Closing directory: ./org.apache.solr.cloud.BasicDistributedZk2Test-1370341553455/jetty4/index.20130604052616427
[junit4:junit4] 2> 531066 T1749 C1897 P55099 oasc.CachingDirectoryFactory.closeCacheValue Removing directory before core close: ./org.apache.solr.cloud.BasicDistributedZk2Test-1370341553455/jetty4/index.20130604052616427
[junit4:junit4] 2> 531067 T1749 C1897 P55099 oasc.RecoveryStrategy.replay No replay needed. core=collection1
[junit4:junit4] 2> 531067 T1749 C1897 P55099 oasc.RecoveryStrategy.doRecovery Replication Recovery was successful - registering as Active. core=collection1
[junit4:junit4] 2> 531068 T1749 C1897 P55099 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4] 2> 531068 T1749 C1897 P55099 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 531071 T1749 C1897 P55099 oasc.RecoveryStrategy.doRecovery Finished recovery process. core=collection1
[junit4:junit4] 2> 531410 T1655 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 531904 T1678 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 531905 T1678 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:35035",
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"127.0.0.1:35035__onenodecollectioncore",
[junit4:junit4] 2> "node_name":"127.0.0.1:35035_",
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "roles":"none",
[junit4:junit4] 2> "collection":"onenodecollection",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "core":"onenodecollectioncore"}
[junit4:junit4] 2> 531920 T1678 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:55099",
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"5",
[junit4:junit4] 2> "node_name":"127.0.0.1:55099_",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard":"shard2",
[junit4:junit4] 2> "core":"collection1"}
[junit4:junit4] 2> 531929 T1684 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 531929 T1746 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 531929 T1697 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 531930 T1729 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 531929 T1677 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 531930 T1713 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 532413 T1655 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 532414 T1655 oasc.AbstractDistribZkTestBase.waitForRecoveriesToFinish Recoveries finished - collection: onenodecollection
[junit4:junit4] 2> 532415 T1655 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 532423 T1655 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 532425 T1757 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@34953495 name:ZooKeeperConnection Watcher:127.0.0.1:56712 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 532425 T1655 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 532532 T1655 oasc.ChaosMonkey.monkeyLog monkey: stop shard! 39584
[junit4:junit4] 2> 532533 T1655 oasc.CoreContainer.shutdown Shutting down CoreContainer instance=22282580
[junit4:junit4] 2> 533445 T1678 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 533447 T1678 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=null message={
[junit4:junit4] 2> "base_url":"http://127.0.0.1:39584",
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"1",
[junit4:junit4] 2> "node_name":"127.0.0.1:39584_",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "collection":"control_collection",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "core":"collection1"}
[junit4:junit4] 2> 533467 T1729 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 533468 T1746 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 533468 T1677 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 533467 T1713 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 533467 T1697 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 534535 T1655 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 534536 T1655 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 534539 T1655 oasc.SolrCore.close [collection1] CLOSING SolrCore org.apache.solr.core.SolrCore@2c342c34
[junit4:junit4] 2> 534545 T1655 oasu.DirectUpdateHandler2.close closing DirectUpdateHandler2{commits=0,autocommits=0,soft autocommits=0,optimizes=0,rollbacks=0,expungeDeletes=0,docsPending=0,adds=0,deletesById=0,deletesByQuery=0,errors=0,cumulative_adds=0,cumulative_deletesById=0,cumulative_deletesByQuery=0,cumulative_errors=0}
[junit4:junit4] 2> 534547 T1655 oasu.SolrCoreState.decrefSolrCoreState Closing SolrCoreState
[junit4:junit4] 2> 534547 T1655 oasu.DefaultSolrCoreState.closeIndexWriter SolrCoreState ref count has reached 0 - closing IndexWriter
[junit4:junit4] 2> 534548 T1655 oasu.DefaultSolrCoreState.closeIndexWriter closing IndexWriter with IndexWriterCloser
[junit4:junit4] 2> 534549 T1655 oasc.SolrCore.closeSearcher [collection1] Closing main searcher on request.
[junit4:junit4] 2> 534550 T1655 oasc.CachingDirectoryFactory.close Closing MockDirectoryFactory - 2 directories currently being tracked
[junit4:junit4] 2> 534550 T1655 oasc.CachingDirectoryFactory.closeCacheValue looking to close ./org.apache.solr.cloud.BasicDistributedZk2Test-1370341553455/control/data [CachedDir<<refCount=0;path=./org.apache.solr.cloud.BasicDistributedZk2Test-1370341553455/control/data;done=false>>]
[junit4:junit4] 2> 534551 T1655 oasc.CachingDirectoryFactory.close Closing directory: ./org.apache.solr.cloud.BasicDistributedZk2Test-1370341553455/control/data
[junit4:junit4] 2> 534552 T1655 oasc.CachingDirectoryFactory.closeCacheValue looking to close ./org.apache.solr.cloud.BasicDistributedZk2Test-1370341553455/control/data/index [CachedDir<<refCount=0;path=./org.apache.solr.cloud.BasicDistributedZk2Test-1370341553455/control/data/index;done=false>>]
[junit4:junit4] 2> 534552 T1655 oasc.CachingDirectoryFactory.close Closing directory: ./org.apache.solr.cloud.BasicDistributedZk2Test-1370341553455/control/data/index
[junit4:junit4] 2> 534553 T1678 oasc.Overseer$ClusterStateUpdater.amILeader According to ZK I (id=89806704047423491-127.0.0.1:39584_-n_0000000000) am no longer a leader.
[junit4:junit4] 2> 534554 T1729 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 534555 T1746 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 534555 T1713 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 534557 T1697 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
[junit4:junit4] 2> 534559 T1729 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 534560 T1746 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 534560 T1713 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 534562 T1697 oasc.Overseer.start Overseer (id=89806704047423494-127.0.0.1:35035_-n_0000000001) starting
[junit4:junit4] 2> 534571 T1759 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
[junit4:junit4] 2> 534572 T1697 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 534573 T1697 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 534574 T1758 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
[junit4:junit4] 2> 534576 T1655 oejsh.ContextHandler.doStop stopped o.e.j.s.ServletContextHandler{/,null}
[junit4:junit4] 2> 534581 T1758 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 534583 T1758 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:39584",
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"1",
[junit4:junit4] 2> "node_name":"127.0.0.1:39584_",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "collection":"control_collection",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "core":"collection1"}
[junit4:junit4] 2> 534591 T1729 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 534591 T1746 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 534591 T1697 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 534591 T1713 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 534628 T1655 oasc.ChaosMonkey.monkeyLog monkey: stop shard! 35035
[junit4:junit4] 2> 534628 T1655 oasc.CoreContainer.shutdown Shutting down CoreContainer instance=1114391148
[junit4:junit4] 2> 535855 T1677 oascc.ZkStateReader$3.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4:junit4] 2> 535856 T1677 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 535857 T1677 oascc.ZkStateReader$2.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4:junit4] 2> 536104 T1758 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 536105 T1758 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=null message={
[junit4:junit4] 2> "base_url":"http://127.0.0.1:35035",
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"127.0.0.1:35035__onenodecollectioncore",
[junit4:junit4] 2> "node_name":"127.0.0.1:35035_",
[junit4:junit4] 2> "roles":"none",
[junit4:junit4] 2> "collection":"onenodecollection",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "core":"onenodecollectioncore"}
[junit4:junit4] 2> 536111 T1758 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=null message={
[junit4:junit4] 2> "base_url":"http://127.0.0.1:35035",
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"2",
[junit4:junit4] 2> "node_name":"127.0.0.1:35035_",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "core":"collection1"}
[junit4:junit4] 2> 536120 T1713 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 536120 T1729 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 536120 T1746 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 536120 T1697 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 537632 T1655 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 537633 T1655 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 537635 T1655 oasc.ZkController.publish publishing core=onenodecollectioncore state=down
[junit4:junit4] 2> 537638 T1655 oasc.SolrCore.close [collection1] CLOSING SolrCore org.apache.solr.core.SolrCore@76677667
[junit4:junit4] 2> 537654 T1758 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 537655 T1655 oasu.DirectUpdateHandler2.close closing DirectUpdateHandler2{commits=1,autocommits=0,soft autocommits=0,optimizes=0,rollbacks=0,expungeDeletes=0,docsPending=0,adds=0,deletesById=0,deletesByQuery=0,errors=0,cumulative_adds=0,cumulative_deletesById=0,cumulative_deletesByQuery=0,cumulative_errors=0}
[junit4:junit4] 2> 537656 T1758 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:35035",
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"2",
[junit4:junit4] 2> "node_name":"127.0.0.1:35035_",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "core":"collection1"}
[junit4:junit4] 2> 537656 T1655 oasu.SolrCoreState.decrefSolrCoreState Closing SolrCoreState
[junit4:junit4] 2> 537657 T1655 oasu.DefaultSolrCoreState.closeIndexWriter SolrCoreState ref count has reached 0 - closing IndexWriter
[junit4:junit4] 2> 537658 T1655 oasu.DefaultSolrCoreState.closeIndexWriter closing IndexWriter with IndexWriterCloser
[junit4:junit4] 2> 537659 T1655 oasc.SolrCore.closeSearcher [collection1] Closing main searcher on request.
[junit4:junit4] 2> 537660 T1655 oasc.CachingDirectoryFactory.close Closing MockDirectoryFactory - 2 directories currently being tracked
[junit4:junit4] 2> 537661 T1655 oasc.CachingDirectoryFactory.closeCacheValue looking to close ./org.apache.solr.cloud.BasicDistributedZk2Test-1370341553455/jetty1 [CachedDir<<refCount=0;path=./org.apache.solr.cloud.BasicDistributedZk2Test-1370341553455/jetty1;done=false>>]
[junit4:junit4] 2> 537661 T1655 oasc.CachingDirectoryFactory.close Closing directory: ./org.apache.solr.cloud.BasicDistributedZk2Test-1370341553455/jetty1
[junit4:junit4] 2> 537662 T1655 oasc.CachingDirectoryFactory.closeCacheValue looking to close ./org.apache.solr.cloud.BasicDistributedZk2Test-1370341553455/jetty1/index [CachedDir<<refCount=0;path=./org.apache.solr.cloud.BasicDistributedZk2Test-1370341553455/jetty1/index;done=false>>]
[junit4:junit4] 2> 537663 T1655 oasc.CachingDirectoryFactory.close Closing directory: ./org.apache.solr.cloud.BasicDistributedZk2Test-1370341553455/jetty1/index
[junit4:junit4] 2> 537663 T1655 oasc.SolrCore.close [onenodecollectioncore] CLOSING SolrCore org.apache.solr.core.SolrCore@29b729b7
[junit4:junit4] 2> 537664 T1758 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:35035",
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"127.0.0.1:35035__onenodecollectioncore",
[junit4:junit4] 2> "node_name":"127.0.0.1:35035_",
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "roles":"none",
[junit4:junit4] 2> "collection":"onenodecollection",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "core":"onenodecollectioncore"}
[junit4:junit4] 2> 537672 T1655 oasu.DirectUpdateHandler2.close closing DirectUpdateHandler2{commits=0,autocommits=0,soft autocommits=0,optimizes=0,rollbacks=0,expungeDeletes=0,docsPending=0,adds=0,deletesById=0,deletesByQuery=0,errors=0,cumulative_adds=0,cumulative_deletesById=0,cumulative_deletesByQuery=0,cumulative_errors=0}
[junit4:junit4] 2> 537673 T1655 oasu.SolrCoreState.decrefSolrCoreState Closing SolrCoreState
[junit4:junit4] 2> 537674 T1655 oasu.DefaultSolrCoreState.closeIndexWriter SolrCoreState ref count has reached 0 - closing IndexWriter
[junit4:junit4] 2> 537675 T1655 oasu.DefaultSolrCoreState.closeIndexWriter closing IndexWriter with IndexWriterCloser
[junit4:junit4] 2> 537676 T1655 oasc.SolrCore.closeSearcher [onenodecollectioncore] Closing main searcher on request.
[junit4:junit4] 2> 537677 T1655 oasc.CachingDirectoryFactory.close Closing MockDirectoryFactory - 2 directories currently being tracked
[junit4:junit4] 2> 537678 T1655 oasc.CachingDirectoryFactory.closeCacheValue looking to close /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./solrtest-BasicDistributedZk2Test-1370341553455/onenodecollection/index [CachedDir<<refCount=0;path=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./solrtest-BasicDistributedZk2Test-1370341553455/onenodecollection/index;done=false>>]
[junit4:junit4] 2> 537679 T1655 oasc.CachingDirectoryFactory.close Closing directory: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./solrtest-BasicDistributedZk2Test-1370341553455/onenodecollection/index
[junit4:junit4] 2> 537680 T1655 oasc.CachingDirectoryFactory.closeCacheValue looking to close /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./solrtest-BasicDistributedZk2Test-1370341553455/onenodecollection [CachedDir<<refCount=0;path=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./solrtest-BasicDistributedZk2Test-1370341553455/onenodecollection;done=false>>]
[junit4:junit4] 2> 537681 T1655 oasc.CachingDirectoryFactory.close Closing directory: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./solrtest-BasicDistributedZk2Test-1370341553455/onenodecollection
[junit4:junit4] 2> 537688 T1729 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 537688 T1746 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 537689 T1697 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 537691 T1697 oascc.ZkStateReader$2.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4:junit4] 2> 537692 T1697 oascc.ZkStateReader$3.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4:junit4] 2> 537692 T1746 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 537692 T1697 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 537688 T1713 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 537694 T1697 oascc.ZkStateReader$2.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4:junit4] 2> 537696 T1729 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
[junit4:junit4] 2> 537697 T1746 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4] 2> 537699 T1713 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
[junit4:junit4] 2> 537700 T1729 oasc.ShardLeaderElectionContext.shouldIBeLeader Checking if I should try and be the leader.
[junit4:junit4] 2> 537700 T1729 oasc.ShardLeaderElectionContext.shouldIBeLeader My last published State was Active, it's okay to be the leader.
[junit4:junit4] 2> 537701 T1729 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4] 2> 537701 T1729 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:45714/collection1/
[junit4:junit4] 2> 537702 T1729 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:45714 START replicas=[http://127.0.0.1:35035/collection1/] nUpdates=100
[junit4:junit4] 2> 537704 T1729 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:45714 DONE. We have no versions. sync failed.
[junit4:junit4] 2> 537704 T1713 oasc.Overseer.start Overseer (id=89806704047423496-127.0.0.1:37412_-n_0000000002) starting
[junit4:junit4] 2> 537704 T1729 oasc.SyncStrategy.syncReplicas Leader's attempt to sync with shard failed, moving to the next canidate
[junit4:junit4] 2> 537705 T1729 oasc.ShardLeaderElectionContext.runLeaderProcess We failed sync, but we have no versions - we can't sync in that case - we were active before, so become leader anyway
[junit4:junit4] 2> 537706 T1729 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:45714/collection1/
[junit4:junit4] 2> 537707 T1729 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
[junit4:junit4] 2> 537711 T1655 oejsh.ContextHandler.doStop stopped o.e.j.s.ServletContextHandler{/,null}
[junit4:junit4] 2> 537711 T1762 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
[junit4:junit4] 2> 537711 T1713 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 537715 T1729 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 537715 T1761 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 537716 T1761 oasc.Overseer$ClusterStateUpdater.run Replaying operations from work queue.
[junit4:junit4] 2> 537717 T1729 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4] 2> 537717 T1761 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:35035",
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"2",
[junit4:junit4] 2> "node_name":"127.0.0.1:35035_",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "core":"collection1"}
[junit4:junit4] 2> 537719 T1729 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 537720 T1746 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 537725 T1761 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:35035",
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"127.0.0.1:35035__onenodecollectioncore",
[junit4:junit4] 2> "node_name":"127.0.0.1:35035_",
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "roles":"none",
[junit4:junit4] 2> "collection":"onenodecollection",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "core":"onenodecollectioncore"}
[junit4:junit4] 2> 537726 T1729 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 537726 T1746 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 537729 T1761 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
[junit4:junit4] 2> 537734 T1761 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 537745 T1729 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 537745 T1746 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 537753 T1713 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4] 2> 537753 T1713 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 537755 T1713 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 537763 T1655 oasc.ChaosMonkey.monkeyLog monkey: stop shard! 37412
[junit4:junit4] 2> 537763 T1655 oasc.CoreContainer.shutdown Shutting down CoreContainer instance=1475500018
[junit4:junit4] 2> 538765 T1655 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 538766 T1655 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 538768 T1655 oasc.SolrCore.close [collection1] CLOSING SolrCore org.apache.solr.core.SolrCore@15041504
[junit4:junit4] 2> 538772 T1655 oasu.DirectUpdateHandler2.close closing DirectUpdateHandler2{commits=1,autocommits=0,soft autocommits=0,optimizes=0,rollbacks=0,expungeDeletes=0,docsPending=0,adds=0,deletesById=0,deletesByQuery=0,errors=0,cumulative_adds=0,cumulative_deletesById=0,cumulative_deletesByQuery=0,cumulative_errors=0}
[junit4:junit4] 2> 538773 T1655 oasu.SolrCoreState.decrefSolrCoreState Closing SolrCoreState
[junit4:junit4] 2> 538774 T1655 oasu.DefaultSolrCoreState.closeIndexWriter SolrCoreState ref count has reached 0 - closing IndexWriter
[junit4:junit4] 2> 538774 T1655 oasu.DefaultSolrCoreState.closeIndexWriter closing IndexWriter with IndexWriterCloser
[junit4:junit4] 2> 538776 T1655 oasc.SolrCore.closeSearcher [collection1] Closing main searcher on request.
[junit4:junit4] 2> 538777 T1655 oasc.CachingDirectoryFactory.close Closing MockDirectoryFactory - 2 directories currently being tracked
[junit4:junit4] 2> 538777 T1655 oasc.CachingDirectoryFactory.closeCacheValue looking to close ./org.apache.solr.cloud.BasicDistributedZk2Test-1370341553455/jetty2 [CachedDir<<refCount=0;path=./org.apache.solr.cloud.BasicDistributedZk2Test-1370341553455/jetty2;done=false>>]
[junit4:junit4] 2> 538778 T1655 oasc.CachingDirectoryFactory.close Closing directory: ./org.apache.solr.cloud.BasicDistributedZk2Test-1370341553455/jetty2
[junit4:junit4] 2> 538778 T1655 oasc.CachingDirectoryFactory.closeCacheValue looking to close ./org.apache.solr.cloud.BasicDistributedZk2Test-1370341553455/jetty2/index [CachedDir<<refCount=0;path=./org.apache.solr.cloud.BasicDistributedZk2Test-1370341553455/jetty2/index;done=false>>]
[junit4:junit4] 2> 538779 T1655 oasc.CachingDirectoryFactory.close Closing directory: ./org.apache.solr.cloud.BasicDistributedZk2Test-1370341553455/jetty2/index
[junit4:junit4] 2> 538780 T1761 oasc.Overseer$ClusterStateUpdater.amILeader According to ZK I (id=89806704047423496-127.0.0.1:37412_-n_0000000002) am no longer a leader.
[junit4:junit4] 2> 538781 T1713 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 538808 T1713 oascc.ZkStateReader$2.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4:junit4] 2> 538808 T1713 oascc.ZkStateReader$3.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4:junit4] 2> 538810 T1746 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
[junit4:junit4] 2> 538810 T1729 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
[junit4:junit4] 2> 538814 T1746 oasc.ShardLeaderElectionContext.shouldIBeLeader Checking if I should try and be the leader.
[junit4:junit4] 2> 538815 T1746 oasc.ShardLeaderElectionContext.shouldIBeLeader My last published State was Active, it's okay to be the leader.
[junit4:junit4] 2> 538815 T1746 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4] 2> 538816 T1729 oasc.Overseer.start Overseer (id=89806704047423498-127.0.0.1:45714_-n_0000000003) starting
[junit4:junit4] 2> 538816 T1746 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:55099/collection1/
[junit4:junit4] 2> 538817 T1746 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:55099 START replicas=[http://127.0.0.1:37412/collection1/] nUpdates=100
[junit4:junit4] 2> 538817 T1746 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:55099 DONE. We have no versions. sync failed.
[junit4:junit4] 2> 538818 T1746 oasc.SyncStrategy.syncReplicas Leader's attempt to sync with shard failed, moving to the next canidate
[junit4:junit4] 2> 538819 T1746 oasc.ShardLeaderElectionContext.runLeaderProcess We failed sync, but we have no versions - we can't sync in that case - we were active before, so become leader anyway
[junit4:junit4] 2> 538819 T1746 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:55099/collection1/
[junit4:junit4] 2> 538820 T1746 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard2
[junit4:junit4] 2> 538823 T1765 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
[junit4:junit4] 2> 538823 T1729 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 538825 T1764 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
[junit4:junit4] 2> 538828 T1655 oejsh.ContextHandler.doStop stopped o.e.j.s.ServletContextHandler{/,null}
[junit4:junit4] 2> 538829 T1746 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 538832 T1764 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 538832 T1746 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4] 2> 538834 T1764 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=null message={
[junit4:junit4] 2> "base_url":"http://127.0.0.1:37412",
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"3",
[junit4:junit4] 2> "node_name":"127.0.0.1:37412_",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard":"shard2",
[junit4:junit4] 2> "core":"collection1"}
[junit4:junit4] 2> 538843 T1764 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:37412",
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"3",
[junit4:junit4] 2> "node_name":"127.0.0.1:37412_",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard":"shard2",
[junit4:junit4] 2> "core":"collection1"}
[junit4:junit4] 2> 538865 T1746 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 538881 T1729 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4] 2> 538881 T1655 oasc.ChaosMonkey.monkeyLog monkey: stop shard! 45714
[junit4:junit4] 2> 538881 T1729 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 538882 T1655 oasc.CoreContainer.shutdown Shutting down CoreContainer instance=420026633
[junit4:junit4] 2> 539884 T1655 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 539885 T1655 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 539887 T1655 oasc.SolrCore.close [collection1] CLOSING SolrCore org.apache.solr.core.SolrCore@7d317d31
[junit4:junit4] 2> 539891 T1655 oasu.DirectUpdateHandler2.close closing DirectUpdateHandler2{commits=0,autocommits=0,soft autocommits=0,optimizes=0,rollbacks=0,expungeDeletes=0,docsPending=0,adds=0,deletesById=0,deletesByQuery=0,errors=0,cumulative_adds=0,cumulative_deletesById=0,cumulative_deletesByQuery=0,cumulative_errors=0}
[junit4:junit4] 2> 539892 T1655 oasu.SolrCoreState.decrefSolrCoreState Closing SolrCoreState
[junit4:junit4] 2> 539892 T1655 oasu.DefaultSolrCoreState.closeIndexWriter SolrCoreState ref count has reached 0 - closing IndexWriter
[junit4:junit4] 2> 539893 T1655 oasu.DefaultSolrCoreState.closeIndexWriter closing IndexWriter with IndexWriterCloser
[junit4:junit4] 2> 539894 T1655 oasc.SolrCore.closeSearcher [collection1] Closing main searcher on request.
[junit4:junit4] 2> 539895 T1655 oasc.CachingDirectoryFactory.close Closing MockDirectoryFactory - 2 directories currently being tracked
[junit4:junit4] 2> 539896 T1655 oasc.CachingDirectoryFactory.closeCacheValue looking to close ./org.apache.solr.cloud.BasicDistributedZk2Test-1370341553455/jetty3/index [CachedDir<<refCount=0;path=./org.apache.solr.cloud.BasicDistributedZk2Test-1370341553455/jetty3/index;done=false>>]
[junit4:junit4] 2> 539896 T1655 oasc.CachingDirectoryFactory.close Closing directory: ./org.apache.solr.cloud.BasicDistributedZk2Test-1370341553455/jetty3/index
[junit4:junit4] 2> 539897 T1655 oasc.CachingDirectoryFactory.closeCacheValue looking to close ./org.apache.solr.cloud.BasicDistributedZk2Test-1370341553455/jetty3 [CachedDir<<refCount=0;path=./org.apache.solr.cloud.BasicDistributedZk2Test-1370341553455/jetty3;done=false>>]
[junit4:junit4] 2> 539898 T1655 oasc.CachingDirectoryFactory.close Closing directory: ./org.apache.solr.cloud.BasicDistributedZk2Test-1370341553455/jetty3
[junit4:junit4] 2> 539899 T1764 oasc.Overseer$ClusterStateUpdater.amILeader According to ZK I (id=89806704047423498-127.0.0.1:45714_-n_0000000003) am no longer a leader.
[junit4:junit4] 2> 539899 T1746 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 539900 T1729 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 539901 T1729 oascc.ZkStateReader$2.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4:junit4] 2> 539901 T1729 oascc.ZkStateReader$3.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4:junit4] 2> 539903 T1746 oascc.ZkStateReader$3.process Updating live nodes... (1)
[junit4:junit4] 2> 539905 T1746 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
[junit4:junit4] 2> 539909 T1746 oasc.Overseer.start Overseer (id=89806704047423500-127.0.0.1:55099_-n_0000000004) starting
[junit4:junit4] 2> 539915 T1767 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
[junit4:junit4] 2> 539917 T1766 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
[junit4:junit4] 2> 539921 T1766 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 539921 T1655 oejsh.ContextHandler.doStop stopped o.e.j.s.ServletContextHandler{/,null}
[junit4:junit4] 2> 539922 T1766 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=null message={
[junit4:junit4] 2> "base_url":"http://127.0.0.1:45714",
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"4",
[junit4:junit4] 2> "node_name":"127.0.0.1:45714_",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "core":"collection1"}
[junit4:junit4] 2> 539928 T1766 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:45714",
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"4",
[junit4:junit4] 2> "node_name":"127.0.0.1:45714_",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "core":"collection1"}
[junit4:junit4] 2> 539935 T1746 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 539975 T1655 oasc.ChaosMonkey.monkeyLog monkey: stop shard! 55099
[junit4:junit4] 2> 539976 T1655 oasc.CoreContainer.shutdown Shutting down CoreContainer instance=1868787555
[junit4:junit4] 2> 540979 T1655 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 540980 T1655 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 540982 T1655 oasc.SolrCore.close [collection1] CLOSING SolrCore org.apache.solr.core.SolrCore@466a466a
[junit4:junit4] 2> 540990 T1655 oasu.DirectUpdateHandler2.close closing DirectUpdateHandler2{commits=0,autocommits=0,soft autocommits=0,optimizes=0,rollbacks=0,expungeDeletes=0,docsPending=0,adds=0,deletesById=0,deletesByQuery=0,errors=0,cumulative_adds=0,cumulative_deletesById=0,cumulative_deletesByQuery=0,cumulative_errors=0}
[junit4:junit4] 2> 540991 T1655 oasu.SolrCoreState.decrefSolrCoreState Closing SolrCoreState
[junit4:junit4] 2> 540992 T1655 oasu.DefaultSolrCoreState.closeIndexWriter SolrCoreState ref count has reached 0 - closing IndexWriter
[junit4:junit4] 2> 540993 T1655 oasu.DefaultSolrCoreState.closeIndexWriter closing IndexWriter with IndexWriterCloser
[junit4:junit4] 2> 541005 T1655 oasc.SolrCore.closeSearcher [collection1] Closing main searcher on request.
[junit4:junit4] 2> 541006 T1655 oasc.CachingDirectoryFactory.close Closing MockDirectoryFactory - 2 directories currently being tracked
[junit4:junit4] 2> 541007 T1655 oasc.CachingDirectoryFactory.closeCacheValue looking to close ./org.apache.solr.cloud.BasicDistributedZk2Test-1370341553455/jetty4/index [CachedDir<<refCount=0;path=./org.apache.solr.cloud.BasicDistributedZk2Test-1370341553455/jetty4/index;done=false>>]
[junit4:junit4] 2> 541008 T1655 oasc.CachingDirectoryFactory.close Closing directory: ./org.apache.solr.cloud.BasicDistributedZk2Test-1370341553455/jetty4/index
[junit4:junit4] 2> 541009 T1655 oasc.CachingDirectoryFactory.closeCacheValue looking to close ./org.apache.solr.cloud.BasicDistributedZk2Test-1370341553455/jetty4 [CachedDir<<refCount=0;path=./org.apache.solr.cloud.BasicDistributedZk2Test-1370341553455/jetty4;done=false>>]
[junit4:junit4] 2> 541010 T1655 oasc.CachingDirectoryFactory.close Closing directory: ./org.apache.solr.cloud.BasicDistributedZk2Test-1370341553455/jetty4
[junit4:junit4] 2> 541012 T1766 oasc.Overseer$ClusterStateUpdater.amILeader According to ZK I (id=89806704047423500-127.0.0.1:55099_-n_0000000004) am no longer a leader.
[junit4:junit4] 2> 541013 T1746 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 541014 T1746 oascc.ZkStateReader$2.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4:junit4] 2> 541015 T1746 oascc.ZkStateReader$3.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4:junit4] 2> 541035 T1655 oejsh.ContextHandler.doStop stopped o.e.j.s.ServletContextHandler{/,null}
[junit4:junit4] 2> 541091 T1655 oas.SolrTestCaseJ4.tearDown ###Ending testDistribSearch
[junit4:junit4] 2> 541093 T1655 oasc.ZkTestServer.send4LetterWord connecting to 127.0.0.1:56712 56712
[junit4:junit4] 1> client port:0.0.0.0/0.0.0.0:0
[junit4:junit4] 1> / (2)
[junit4:junit4] 1> DATA:
[junit4:junit4] 1>
[junit4:junit4] 1> /solr (7)
[junit4:junit4] 1> /solr/configs (1)
[junit4:junit4] 1> /solr/configs/conf1 (9)
[junit4:junit4] 1> /solr/configs/conf1/old_synonyms.txt (0)
[junit4:junit4] 1> DATA: ...supressed...
[junit4:junit4] 1> /solr/configs/conf1/protwords.txt (0)
[junit4:junit4] 1> DATA: ...supressed...
[junit4:junit4] 1> /solr/configs/conf1/stopwords.txt (0)
[junit4:junit4] 1> DATA: ...supressed...
[junit4:junit4] 1> /solr/configs/conf1/synonyms.txt (0)
[junit4:junit4] 1> DATA: ...supressed...
[junit4:junit4] 1> /solr/configs/conf1/schema.xml (0)
[junit4:junit4] 1> DATA: ...supressed...
[junit4:junit4] 1> /solr/configs/conf1/currency.xml (0)
[junit4:junit4] 1> DATA: ...supressed...
[junit4:junit4] 1> /solr/configs/conf1/open-exchange-rates.json (0)
[junit4:junit4] 1> DATA:
[junit4:junit4] 1> {
[junit4:junit4] 1> "disclaimer": "This data is not real, it was synthetically created to match currency.xml. It is modeled after the data format available from openexchangerates.org. See https://openexchangerates.org/documentation for details",
[junit4:junit4] 1> "license": "http://www.apache.org/licenses/LICENSE-2.0",
[junit4:junit4] 1> "timestamp": 1332070464,
[junit4:junit4] 1>
[junit4:junit4] 1>
[junit4:junit4] 1> "IMPORTANT NOTE": "In order for tests to work, this data must be kept in sync with ./currency.xml",
[junit4:junit4] 1>
[junit4:junit4] 1>
[junit4:junit4] 1> "base": "USD",
[junit4:junit4] 1> "rates": {
[junit4:junit4] 1> "USD": 1,
[junit4:junit4] 1> "JPY": 81.29,
[junit4:junit4] 1> "EUR": 2.5,
[junit4:junit4] 1> "GBP": 0.5,
[junit4:junit4] 1> "MXN": 2.0
[junit4:junit4] 1> }
[junit4:junit4] 1> }
[junit4:junit4] 1>
[junit4:junit4] 1> /solr/configs/conf1/solrconfig.xml (0)
[junit4:junit4] 1> DATA: ...supressed...
[junit4:junit4] 1> /solr/configs/conf1/mapping-ISOLatin1Accent.txt (0)
[junit4:junit4] 1> DATA: ...supressed...
[junit4:junit4] 1> /solr/overseer (3)
[junit4:junit4] 1> DATA:
[junit4:junit4] 1>
[junit4:junit4] 1> /solr/overseer/queue (0)
[junit4:junit4] 1> /solr/overseer/collection-queue-work (0)
[junit4:junit4] 1> /solr/overseer/queue-work (0)
[junit4:junit4] 1> /solr/overseer_elect (2)
[junit4:junit4] 1> /solr/overseer_elect/leader (0)
[junit4:junit4] 1> DATA:
[junit4:junit4] 1> {"id":"89806704047423491-127.0.0.1:39584_-n_0000000000"}
[junit4:junit4] 1> /solr/overseer_elect/election (5)
[junit4:junit4] 1> /solr/overseer_elect/election/89806704047423496-127.0.0.1:37412_-n_0000000002 (0)
[junit4:junit4] 1> /solr/overseer_elect/election/89806704047423498-127.0.0.1:45714_-n_0000000003 (0)
[junit4:junit4] 1> /solr/overseer_elect/election/89806704047423491-127.0.0.1:39584_-n_0000000000 (0)
[junit4:junit4] 1> /solr/overseer_elect/election/89806704047423494-127.0.0.1:35035_-n_0000000001 (0)
[junit4:junit4] 1> /solr/overseer_elect/election/89806704047423500-127.0.0.1:55099_-n_0000000004 (0)
[junit4:junit4] 1> /solr/collections (3)
[junit4:junit4] 1> /solr/collections/control_collection (3)
[junit4:junit4] 1> DATA:
[junit4:junit4] 1> {"configName":"conf1"}
[junit4:junit4] 1> /solr/collections/control_collection/leader_elect (1)
[junit4:junit4] 1> /solr/collections/control_collection/leader_elect/shard1 (1)
[junit4:junit4] 1> /solr/collections/control_collection/leader_elect/shard1/election (1)
[junit4:junit4] 1> /solr/collections/control_collection/leader_elect/shard1/election/89806704047423491-1-n_0000000000 (0)
[junit4:junit4] 1> /solr/collections/control_collection/leaders (1)
[junit4:junit4] 1> /solr/collections/control_collection/leaders/shard1 (0)
[junit4:junit4] 1> DATA:
[junit4:junit4] 1> {
[junit4:junit4] 1> "base_url":"http://127.0.0.1:39584",
[junit4:junit4] 1> "node_name":"127.0.0.1:39584_",
[junit4:junit4] 1> "core":"collection1"}
[junit4:junit4] 1> /solr/collections/control_collection/shards (0)
[junit4:junit4] 1> /solr/collections/collection1 (3)
[junit4:junit4] 1> DATA:
[junit4:junit4] 1> {"configName":"conf1"}
[junit4:junit4] 1> /solr/collections/collection1/leader_elect (2)
[junit4:junit4] 1> /solr/collections/collection1/leader_elec
[...truncated too long message...]
"node_name":"127.0.0.1:35035_",
[junit4:junit4] 1> "core":"onenodecollectioncore"}
[junit4:junit4] 1> /solr/aliases.json (0)
[junit4:junit4] 1> /solr/live_nodes (5)
[junit4:junit4] 1> /solr/live_nodes/127.0.0.1:39584_ (0)
[junit4:junit4] 1> /solr/live_nodes/127.0.0.1:45714_ (0)
[junit4:junit4] 1> /solr/live_nodes/127.0.0.1:37412_ (0)
[junit4:junit4] 1> /solr/live_nodes/127.0.0.1:55099_ (0)
[junit4:junit4] 1> /solr/live_nodes/127.0.0.1:35035_ (0)
[junit4:junit4] 1> /solr/clusterstate.json (0)
[junit4:junit4] 1> DATA:
[junit4:junit4] 1> {
[junit4:junit4] 1> "control_collection":{
[junit4:junit4] 1> "shards":{"shard1":{
[junit4:junit4] 1> "range":"80000000-7fffffff",
[junit4:junit4] 1> "state":"active",
[junit4:junit4] 1> "replicas":{"1":{
[junit4:junit4] 1> "base_url":"http://127.0.0.1:39584",
[junit4:junit4] 1> "state":"active",
[junit4:junit4] 1> "node_name":"127.0.0.1:39584_",
[junit4:junit4] 1> "collection":"control_collection",
[junit4:junit4] 1> "shard":"shard1",
[junit4:junit4] 1> "core":"collection1",
[junit4:junit4] 1> "leader":"true"}}}},
[junit4:junit4] 1> "router":"compositeId"},
[junit4:junit4] 1> "onenodecollection":{
[junit4:junit4] 1> "shards":{"shard1":{
[junit4:junit4] 1> "range":"80000000-7fffffff",
[junit4:junit4] 1> "state":"active",
[junit4:junit4] 1> "replicas":{"127.0.0.1:35035__onenodecollectioncore":{
[junit4:junit4] 1> "base_url":"http://127.0.0.1:35035",
[junit4:junit4] 1> "state":"active",
[junit4:junit4] 1> "node_name":"127.0.0.1:35035_",
[junit4:junit4] 1> "roles":"none",
[junit4:junit4] 1> "collection":"onenodecollection",
[junit4:junit4] 1> "shard":"shard1",
[junit4:junit4] 1> "core":"onenodecollectioncore",
[junit4:junit4] 1> "leader":"true"}}}},
[junit4:junit4] 1> "router":"compositeId"},
[junit4:junit4] 1> "collection1":{
[junit4:junit4] 1> "shards":{
[junit4:junit4] 1> "shard1":{
[junit4:junit4] 1> "range":"80000000-ffffffff",
[junit4:junit4] 1> "state":"active",
[junit4:junit4] 1> "replicas":{
[junit4:junit4] 1> "2":{
[junit4:junit4] 1> "base_url":"http://127.0.0.1:35035",
[junit4:junit4] 1> "state":"active",
[junit4:junit4] 1> "node_name":"127.0.0.1:35035_",
[junit4:junit4] 1> "collection":"collection1",
[junit4:junit4] 1> "shard":"shard1",
[junit4:junit4] 1> "core":"collection1",
[junit4:junit4] 1> "leader":"true"},
[junit4:junit4] 1> "4":{
[junit4:junit4] 1> "base_url":"http://127.0.0.1:45714",
[junit4:junit4] 1> "state":"active",
[junit4:junit4] 1> "node_name":"127.0.0.1:45714_",
[junit4:junit4] 1> "collection":"collection1",
[junit4:junit4] 1> "shard":"shard1",
[junit4:junit4] 1> "core":"collection1"}}},
[junit4:junit4] 1> "shard2":{
[junit4:junit4] 1> "range":"0-7fffffff",
[junit4:junit4] 1> "state":"active",
[junit4:junit4] 1> "replicas":{
[junit4:junit4] 1> "3":{
[junit4:junit4] 1> "base_url":"http://127.0.0.1:37412",
[junit4:junit4] 1> "state":"active",
[junit4:junit4] 1> "node_name":"127.0.0.1:37412_",
[junit4:junit4] 1> "collection":"collection1",
[junit4:junit4] 1> "shard":"shard2",
[junit4:junit4] 1> "core":"collection1",
[junit4:junit4] 1> "leader":"true"},
[junit4:junit4] 1> "5":{
[junit4:junit4] 1> "base_url":"http://127.0.0.1:55099",
[junit4:junit4] 1> "state":"active",
[junit4:junit4] 1> "node_name":"127.0.0.1:55099_",
[junit4:junit4] 1> "collection":"collection1",
[junit4:junit4] 1> "shard":"shard2",
[junit4:junit4] 1> "core":"collection1"}}}},
[junit4:junit4] 1> "router":"compositeId"}}
[junit4:junit4] 1> /zookeeper (1)
[junit4:junit4] 1> DATA:
[junit4:junit4] 1>
[junit4:junit4] 1>
[junit4:junit4] 2> NOTE: reproduce with: ant test -Dtestcase=BasicDistributedZk2Test -Dtests.method=testDistribSearch -Dtests.seed=7B13816205D9C92F -Dtests.multiplier=3 -Dtests.slow=true -Dtests.locale=pl_PL -Dtests.timezone=America/Rankin_Inlet -Dtests.file.encoding=US-ASCII
[junit4:junit4] ERROR 33.0s J0 | BasicDistributedZk2Test.testDistribSearch <<<
[junit4:junit4] > Throwable #1: org.apache.solr.client.solrj.impl.HttpSolrServer$RemoteSolrException: Server at http://127.0.0.1:37412/onenodecollectioncore returned non ok status:404, message:Can not find: /onenodecollectioncore/update
[junit4:junit4] > at __randomizedtesting.SeedInfo.seed([7B13816205D9C92F:FAF50F7A7286A913]:0)
[junit4:junit4] > at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:385)
[junit4:junit4] > at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:180)
[junit4:junit4] > at org.apache.solr.client.solrj.request.AbstractUpdateRequest.process(AbstractUpdateRequest.java:117)
[junit4:junit4] > at org.apache.solr.client.solrj.SolrServer.add(SolrServer.java:116)
[junit4:junit4] > at org.apache.solr.client.solrj.SolrServer.add(SolrServer.java:102)
[junit4:junit4] > at org.apache.solr.cloud.BasicDistributedZk2Test.testNodeWithoutCollectionForwarding(BasicDistributedZk2Test.java:198)
[junit4:junit4] > at org.apache.solr.cloud.BasicDistributedZk2Test.doTest(BasicDistributedZk2Test.java:90)
[junit4:junit4] > at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:815)
[junit4:junit4] > at java.lang.Thread.run(Thread.java:738)
[junit4:junit4] 2> 541126 T1655 oas.SolrTestCaseJ4.deleteCore ###deleteCore
[junit4:junit4] 2> 33062 T1654 ccr.ThreadLeakControl.checkThreadLeaks WARNING Will linger awaiting termination of 1 leaked thread(s).
[junit4:junit4] 2> NOTE: test params are: codec=Lucene40, sim=DefaultSimilarity, locale=pl_PL, timezone=America/Rankin_Inlet
[junit4:junit4] 2> NOTE: Linux 3.2.0-45-generic x86/IBM Corporation 1.6.0 (32-bit)/cpus=8,threads=3,free=38886360,total=82150400
[junit4:junit4] 2> NOTE: All tests run in this JVM: [TestLMDirichletSimilarityFactory, PeerSyncTest, TestStressRecovery, ReturnFieldsTest, CircularListTest, CSVRequestHandlerTest, TestGroupingSearch, TestSerializedLuceneMatchVersion, TestValueSourceCache, TestDefaultSearchFieldResource, SolrInfoMBeanTest, TestCollationKeyRangeQueries, TestFaceting, TestPluginEnable, RegexBoostProcessorTest, MoreLikeThisHandlerTest, JsonLoaderTest, ShowFileRequestHandlerTest, DirectSolrConnectionTest, CoreAdminHandlerTest, TestFastLRUCache, SOLR749Test, TestConfig, TestCharFilters, TestSweetSpotSimilarityFactory, BadComponentTest, TestReversedWildcardFilterFactory, TestCoreContainer, SuggesterFSTTest, TestExtendedDismaxParser, RAMDirectoryFactoryTest, ResponseLogComponentTest, TestPerFieldSimilarity, IndexSchemaTest, DistributedSpellCheckComponentTest, TestQuerySenderListener, FileBasedSpellCheckerTest, TestSort, SyncSliceTest, TestIBSimilarityFactory, DistributedTermsComponentTest, FieldMutatingUpdateProcessorTest, TestCloudManagedSchema, ZkCLITest, SpellingQueryConverterTest, TestFunctionQuery, SchemaVersionSpecificBehaviorTest, CacheHeaderTest, LoggingHandlerTest, DirectUpdateHandlerOptimizeTest, TestFastWriter, TestSolrQueryParserResource, BadCopyFieldTest, PolyFieldTest, SpatialFilterTest, LeaderElectionTest, UpdateRequestProcessorFactoryTest, QueryElevationComponentTest, TestTrie, DocValuesTest, PreAnalyzedUpdateProcessorTest, LukeRequestHandlerTest, PluginInfoTest, TestPseudoReturnFields, SoftAutoCommitTest, DateFieldTest, TermVectorComponentTest, MinimalSchemaTest, TestDynamicFieldResource, TestLRUCache, ExternalFileFieldSortTest, QueryParsingTest, BasicDistributedZkTest, BasicDistributedZk2Test]
[junit4:junit4] Completed on J0 in 33.58s, 1 test, 1 error <<< FAILURES!
[...truncated 466 lines...]
BUILD FAILED
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/build.xml:386: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/build.xml:379: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/build.xml:39: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build.xml:181: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/common-build.xml:437: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/lucene/common-build.xml:1243: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/lucene/common-build.xml:887: There were test failures: 297 suites, 1238 tests, 1 error, 14 ignored (8 assumptions)
Total time: 47 minutes 52 seconds
Build step 'Invoke Ant' marked build as failure
Description set: Java: 32bit/ibm-j9-jdk6 -Xjit:exclude={org/apache/lucene/util/fst/FST.pack(IIF)Lorg/apache/lucene/util/fst/FST;}
Archiving artifacts
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure